This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

[PATCH v2] __builtin_expect cleanup for iconv{,data}/*.c (was: Re: [PATCH] __builtin_expect cleanup for iconvdata/*.c)


This is the second iteration of these patches. The rewriting script has grown considerably and seems useful for subsequent clean-ups. The first patch installs it as scripts/builtin_expect.py.

The second patch is the automated part of the conversion. The changes look reasonable to me, but double-checking is certainly welcome.

The third patch contains the manual part. It is now much smaller because the rewriting script is more sophisticated.

Again, I used a preprocessor #define to disable __builtin_expect and objdump -d --reloc to compare the results. Because of a line merge and the resulting line number difference in an assert statement, there was one assembly difference.

Testing on Fedora 20 x86_64 also revealed no regressions.

--
Florian Weimer / Red Hat Product Security
>From 0f3b23168cfa991c32870f00550804bb9d9b856b Mon Sep 17 00:00:00 2001
From: Florian Weimer <fweimer@redhat.com>
Date: Tue, 9 Sep 2014 20:18:58 +0200
Subject: [PATCH 1/3] scripts/builtin_expect.py: New script to rewrite
 __builtin_expect

to __glibc_likely and __glibc_unlikely, as appropriate.

2014-09-09  Florian Weimer  <fweimer@redhat.com>

	* scripts/builtin_expect.py: New file.

diff --git a/scripts/builtin_expect.py b/scripts/builtin_expect.py
new file mode 100644
index 0000000..3f4a43f
--- /dev/null
+++ b/scripts/builtin_expect.py
@@ -0,0 +1,425 @@
+#!/usr/bin/python
+# Convert occurrences of __builtin_expect to __glibc_{,un}likely
+# Copyright (C) 2014 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, see <http://www.gnu.org/licenses/>.
+
+"""
+This script attempts to rewrite expressions involving __builtin_expect
+to use __glibc_likely or __glibc_unlikely, as appropriate.  The script
+rewrites in-place the input files whose names are passed as arguments
+on the command line.
+
+The following patterns are recognized:
+
+(1)   __builtin_expect (EXPR1, EXPR2) OP TERM
+
+is rewritten to:
+
+(1a)  __glibc_likely (EXPR1 OP TERM)
+
+or
+
+(1b)  __glibc_unlikely (EXPR1 OP TERM)
+
+based on the evaluation of EXPR2 OP TERM.
+
+(2)   __builtin_expect (EXPR, 1)
+
+is rewritten to:
+
+(2a)  __glibc_likely (EXPR)
+
+And finally,
+
+(3)  __builtin_expect (EXPR, 0)
+
+turns into
+
+(3a)  __glibc_unlikely (EXPR)
+
+The transformations to (2a) and (3a) are always safe because the
+macros expand back to the original expression, however the
+transformation of (1) usually gives results which better reflect the
+original intent.  The latter transformation alters the result of the
+branch hints, so it can result in different machine code.  Therefore,
+to verify the correctness of the changes, you need to compile with a
+definition like
+
+#define __builtin_expect(a, b) (a)
+
+in place, presumably in the file misc/sys/cdefs.h, right next to the
+definitions of the __glibc_likely and __glibc_unlikely macros.
+
+The direction of the branch hint is determined with heuristics.  In
+particular, it is assumed that if the expected value EXPR2 is 0, and
+TERM is not a number, then TERM does not evaluate to 0.  (Additional
+values for TERM can be defined by extending the NORMALIZE dictionary
+below.)
+
+The expression parser is just a crude approximation to the C language
+rules.  It may have to be extended to deal with additional
+constructs.  Heuristics for evaluating EXPR2 and TERM are necessary
+because the changes are applied to macro definitions as well.
+
+An attempt is made at merging a line with the next line, and the merge
+is kept if it results in a different (presumably better) match.  If
+after such a merge, the line length exceeds the 80 character limit, a
+warning is printed to standard error.  Three lines are not folded,
+such expressions have to be converted manually.
+
+The column of a line continuation character "\" is preserved, if
+possible.
+
+On success, a ChangeLog fragment is printed to standard output.
+"""
+
+import os
+import subprocess
+import sys
+import textwrap
+import time
+
+def expand(s):
+    "Expand tabs to spaces."
+    if "\t" not in s:
+        return s
+    result = ""
+    for ch in s:
+        if ch == '\t':
+            result += ' '
+            while (len(result) % 8) != 0:
+                result += ' '
+        else:
+            result += ch
+    return result
+
+def unexpand(s):
+    "Contract spaces to tabs, including non-leading spaces."
+    pos = 0
+    result = ""
+    while pos < len(s):
+        oldpos = pos
+        while pos < len(s) and s[pos] == ' ':
+            pos += 1
+        if pos - oldpos <= 1:
+            # Not enough spaces to insert a tab.
+            result += s[oldpos]
+            pos = oldpos + 1
+        else:
+            while oldpos < pos:
+                newpos = (oldpos + 8) & ~7
+                if newpos <= pos:
+                    result += '\t'
+                    oldpos = newpos
+                else:
+                    result += ' '
+                    oldpos += 1
+    return result
+
+assert expand("\t  foo\t   X") == "          foo      X"
+assert unexpand("          foo      X") == "\t  foo\t   X"
+assert unexpand("          foo      ") == "\t  foo\t   "
+assert unexpand("          foo   ") == "\t  foo\t"
+assert unexpand(r"    if (HAS_HOLES && & *inptr != '\0')     " + "\\") \
+    == "    if (HAS_HOLES && & *inptr != '\\0')\t   \\"
+
+def eol(line, pos):
+    "True if pos is at the end of line or at at a backslash."
+    return pos == len(line) or line[pos] in "\n\\"
+
+def skip_ws(line, pos):
+    "Skips over tabs and spaces starting at pos, resturns the new pos."
+    while pos < len(line) and line[pos] in " \t":
+        pos += 1
+    return pos
+
+def skip_expr(line, pos, partial = False):
+    """Skips over (most) C expressions, starting at pos.
+    
+    Returns None if the line does not contain a complete subexpression.
+    Otherwise, returns the position after the expression.
+    If partial is set, stop on the first """
+    nesting = 0
+    instr = False
+    while True:
+        if eol(line, pos) and not instr:
+            if partial:
+                break
+            return None
+        ch = line[pos]
+        if ch in '([':
+            nesting += 1
+        elif ch in ')]':
+            if nesting == 0:
+                break
+            nesting -= 1
+        elif ch == ',' and nesting == 0:
+            break
+        elif partial and ch in " \t\n\\" and nesting == 0 and not instr:
+            break
+        elif ch in "\"\'":
+            instr = not instr
+        pos += 1
+    return pos
+
+FUNC = "__builtin_expect"
+
+def match_builtin_expect(line):
+    """Finds __builtin_expect on the current line and extracts arguments.
+
+    Returns None on match failure, or (start-position, end-position,
+    first-argument, second-argument) on success."""
+    start = line.find(FUNC)
+    if start < 0:
+        return None
+    pos = skip_ws(line, start + len(FUNC))
+    if eol(line, pos):
+        return None
+    assert line[pos] == '(', repr(line[pos:])
+
+    pos += 1
+    expr_start = pos
+    pos = skip_expr(line, pos)
+    if pos is None:
+        return None
+    expr_end = pos
+    assert line[pos] == ',', repr(line[pos:])
+
+    pos += 1
+    pos = skip_ws(line, pos)
+    expect_start = pos
+    pos = skip_expr(line, pos)
+    if pos is None:
+        return None
+    expect_end = pos
+    assert line[pos] == ')', repr(line[pos:])
+
+    pos += 1
+    return start, pos, line[expr_start:expr_end], line[expect_start:expect_end]
+
+def match_operator(line, pos):
+    """Tries to match a comparison operator on the line at pos.
+
+    Returns None for no match, or
+    (new-position, operator-string, matched-operand) on success."""
+    pos = skip_ws(line, pos)
+    operator = line[pos:pos + 2]
+    if operator not in ("==", "!=", "<=", ">="):
+        operator = line[pos:pos + 1]
+        if operator not in "<>":
+            return None
+    pos = skip_ws(line, pos + len(operator))
+    start_expr = pos
+    pos = skip_expr(line, pos, True)
+    if pos is None:
+        return None
+    return pos, operator, line[start_expr:pos]
+
+def replace(line, start, end, new):
+    assert start is not None
+    assert end is not None
+    return line[:start] + new + line[end:]
+
+# Helper table for value normalization.
+NORMALIZE = {
+    "L'\\0'" : "0",
+    "'\0'" : "0",
+    "L'\\1'" : "1",
+    "'\1'" : "1",
+    "__UNKNOWN_10646_CHAR" : "0xfffd",
+}
+
+def normalize(value):
+    "Convert the value to a normalized string and an int, if possible."
+    value = value.strip()
+    value = NORMALIZE.get(value, value)
+    try:
+        num = int(value, 0)
+    except ValueError:
+        return value
+    return num
+
+def compare(left, op, right):
+    """Heuristics to determine the direction of the branch hint.
+
+    Returns None if the direction cannot be determined.
+    """
+    left = normalize(left)
+    right = normalize(right)
+    if type(left) != type(right):
+        if left == 0 or right == 0:
+            # Special case: 0 as non-matching wildcard
+            left = str(left)
+            right = str(right)
+        else:
+            return None
+    if op == '==':
+        return left == right
+    if op == '!=':
+        return left != right
+    if type(left) != type(0):
+        return None
+    if op == '<':
+        return left < right
+    if op == '<=':
+        return left <= right
+    if op == '>':
+        return left > right
+    if op == '>=':
+        return left >= right
+
+def rewrite_line(line):
+    "Rewrite __builtin_expect calls."
+    m = match_builtin_expect(line)
+    if m is None:
+        return line
+    start, end, expr, expect = m
+    m = match_operator(line, end)
+    if m is None:
+        if expect in "01":
+            if expect == "0":
+                tilt = "un"
+            else:
+                tilt = ""
+            new = "__glibc_" + tilt + "likely (" + expr + ")"
+            return replace(line, start, end, new)
+        # Else funny RHS, do not replace.
+        return line
+
+    # Comparison operator follows.
+    end, operator, value = m
+    tilt = compare(expect, operator, value)
+    if tilt is None:
+        return line
+    if tilt:
+        tilt = ""
+    else:
+        tilt = "un"
+    new = "__glibc_" + tilt + "likely (" + expr + " " + operator + " " + value + ")"
+    return replace(line, start, end, new)
+
+def adjust_continuation(old, new):
+    "Adjusts the location of the \\ continuation character."
+    if old != new and old.endswith("\\\n"):
+        assert new.endswith("\\\n"), repr((old, new))
+        oldlen = len(expand(old)) - 2
+        new = expand(new[:-2])
+        while len(new) < oldlen:
+            new += ' '
+        new = unexpand(new + "\\\n")
+    return new
+
+def starts_with_comparison(line):
+    "Check if line starts with a comparison operator."
+    line = line.strip()
+    for operator in ("==", "!=", "<", ">"):
+        if line.startswith(operator):
+            return True
+    return False
+
+def test():
+    line = "XX  __builtin_expect (ch < sizeof (array), 0) XX"
+    d = match_builtin_expect(line)
+    assert d == (4, len(line) - 3, "ch < sizeof (array)", "0"), repr(d)
+    line = "XX  __builtin_expect (strcmp (a, b), 0) == 0) X"
+    d = match_builtin_expect(line)
+    assert d == (4, len(line) - 8, "strcmp (a, b)", "0"), repr(d)
+    d = match_operator(line, len(line) - 8)
+    assert d == (len(line) - 3, "==", "0"), repr(d)
+    line =  "    else if (__builtin_expect (ch, 0) == 0xa0 \\"
+    d = rewrite_line(line)
+    assert d == "    else if (__glibc_unlikely (ch == 0xa0) \\", repr(d)
+    line = r"   __builtin_expect (cp[0], L'\1') == L'\0' && ch != '0'))    \\"
+    d = rewrite_line(line)
+    assert d == r"   __glibc_unlikely (cp[0] == L'\0') && ch != '0'))    \\", repr(d)
+    line = "if (__builtin_expect (__fxstat64 (_STAT_VER, fd, &st), 0) < 0"
+    d = rewrite_line(line)
+    assert d == "if (__glibc_unlikely (__fxstat64 (_STAT_VER, fd, &st) < 0)", repr(d)
+test()
+del test
+
+def strip_continuation_nl(line):
+    if line.endswith("\\\n"):
+        return line[:-2].rstrip()
+    return line.rstrip()
+
+def changelog_excerpt(files):
+    email = subprocess.check_output("git config --get user.email".split())
+    name = subprocess.check_output("git config --get user.name".split())
+    tm = time.localtime(time.time())
+    result = ""
+    for path in sorted(files):
+        line = "* " + path + ": "
+        if result:
+            line += "Likewise."
+        else:
+            line += "Run scripts/builtin_expect.py to rewrite __builtin_expect expressions."
+        lines = "\n\t".join(textwrap.wrap(line))
+        result += "\t" + lines + "\n"
+    result = "{}  {}  <{}>\n\n{}".format(
+        time.strftime("%Y-%m-%d", tm), name.strip(), email.strip(), result)
+    return result
+
+def rewrite_file(path):
+    outpath = path + ".tmp"
+    with file(path, "r") as inf:
+        inflines = list(inf.readlines())
+
+    pos = 0
+    outlineno = 1
+    changed = False
+    with file(outpath, "w") as outf:
+        while pos < len(inflines):
+            line = inflines[pos]
+            if pos + 1 < len(inflines):
+                nextline = inflines[pos + 1]
+            else:
+                nextline = ''
+
+            new = rewrite_line(line)
+            if (new == line and line.find(FUNC)) \
+               or (new != line and starts_with_comparison(nextline)):
+                # Try to rewrite both lines together.
+                combined = \
+                    strip_continuation_nl(line) + ' ' + nextline.lstrip()
+                new_combined = rewrite_line(combined)
+                if not new_combined.startswith(strip_continuation_nl(new)):
+                    # Rewriting both lines together resulted in a
+                    # material change, so use this combination.
+                    new = new_combined
+                    pos += 1
+            # Adjust the position of the \ character, if any.
+            new = adjust_continuation(line, new)
+            if len(expand(new)) >= 80 and len(expand(line)) < 80:
+                sys.stderr.write("{}:{}: long line in result\n".format(
+                    path, outlineno))
+            outf.write(new)
+            changed = changed or new != line
+            pos += 1
+            outlineno += 1
+    os.rename(outpath, path)
+    return changed
+
+if len(sys.argv) < 2:
+    sys.stderr.write("usage: {} PATH...\n".format(sys.argv[0]))
+    sys.exit(1)
+
+changed = []
+for path in sys.argv[1:]:
+    if rewrite_file(path):
+        changed.append(path)
+if changed:
+    sys.stdout.write(changelog_excerpt(changed))
+
-- 
1.9.3

>From 9cbe57c7461e924217d76e336d38db2b8a7b3ca9 Mon Sep 17 00:00:00 2001
From: Florian Weimer <fweimer@redhat.com>
Date: Tue, 9 Sep 2014 20:28:12 +0200
Subject: [PATCH 2/3] Convert iconv/ and iconvdata/ from __builtin_expect to
 __glibc_{,un}likely

This the automated part of the conversion, generated by
scripts/builtin_expect.py.

2014-09-09  Florian Weimer  <fweimer@redhat.com>

	* iconv/gconv_cache.c: Run scripts/builtin_expect.py to rewrite
	__builtin_expect expressions.
	* iconv/gconv_db.c: Likewise.
	* iconv/gconv_dl.c: Likewise.
	* iconv/gconv_open.c: Likewise.
	* iconv/gconv_simple.c: Likewise.
	* iconv/iconv_open.c: Likewise.
	* iconv/loop.c: Likewise.
	* iconv/skeleton.c: Likewise.
	* iconvdata/8bit-gap.c: Likewise.
	* iconvdata/8bit-generic.c: Likewise.
	* iconvdata/ansi_x3.110.c: Likewise.
	* iconvdata/big5.c: Likewise.
	* iconvdata/big5hkscs.c: Likewise.
	* iconvdata/cp1255.c: Likewise.
	* iconvdata/cp1258.c: Likewise.
	* iconvdata/cp932.c: Likewise.
	* iconvdata/euc-cn.c: Likewise.
	* iconvdata/euc-jp-ms.c: Likewise.
	* iconvdata/euc-jp.c: Likewise.
	* iconvdata/euc-kr.c: Likewise.
	* iconvdata/euc-tw.c: Likewise.
	* iconvdata/gb18030.c: Likewise.
	* iconvdata/gbbig5.c: Likewise.
	* iconvdata/gbgbk.c: Likewise.
	* iconvdata/gbk.c: Likewise.
	* iconvdata/ibm1364.c: Likewise.
	* iconvdata/ibm930.c: Likewise.
	* iconvdata/ibm932.c: Likewise.
	* iconvdata/ibm933.c: Likewise.
	* iconvdata/ibm935.c: Likewise.
	* iconvdata/ibm937.c: Likewise.
	* iconvdata/ibm939.c: Likewise.
	* iconvdata/ibm943.c: Likewise.
	* iconvdata/iso-2022-cn.c: Likewise.
	* iconvdata/iso-2022-jp-3.c: Likewise.
	* iconvdata/iso-2022-jp.c: Likewise.
	* iconvdata/iso-2022-kr.c: Likewise.
	* iconvdata/iso646.c: Likewise.
	* iconvdata/iso_6937-2.c: Likewise.
	* iconvdata/iso_6937.c: Likewise.
	* iconvdata/johab.c: Likewise.
	* iconvdata/sjis.c: Likewise.
	* iconvdata/t.61.c: Likewise.
	* iconvdata/tcvn5712-1.c: Likewise.
	* iconvdata/tscii.c: Likewise.
	* iconvdata/uhc.c: Likewise.
	* iconvdata/utf-16.c: Likewise.
	* iconvdata/utf-32.c: Likewise.
	* iconvdata/utf-7.c: Likewise.

diff --git a/iconv/gconv_cache.c b/iconv/gconv_cache.c
index f3a4337..cb48eca 100644
--- a/iconv/gconv_cache.c
+++ b/iconv/gconv_cache.c
@@ -60,12 +60,12 @@ __gconv_load_cache (void)
 
   /* See whether the cache file exists.  */
   fd = open_not_cancel (GCONV_MODULES_CACHE, O_RDONLY, 0);
-  if (__builtin_expect (fd, 0) == -1)
+  if (__glibc_unlikely (fd == -1))
     /* Not available.  */
     return -1;
 
   /* Get information about the file.  */
-  if (__builtin_expect (__fxstat64 (_STAT_VER, fd, &st), 0) < 0
+  if (__glibc_unlikely (__fxstat64 (_STAT_VER, fd, &st) < 0)
       /* We do not have to start looking at the file if it cannot contain
 	 at least the cache header.  */
       || (size_t) st.st_size < sizeof (struct gconvcache_header))
@@ -93,7 +93,7 @@ __gconv_load_cache (void)
 	{
 	  ssize_t n = __read (fd, (char *) gconv_cache + already_read,
 			      cache_size - already_read);
-	  if (__builtin_expect (n, 0) == -1)
+	  if (__glibc_unlikely (n == -1))
 	    {
 	      free (gconv_cache);
 	      gconv_cache = NULL;
@@ -112,15 +112,15 @@ __gconv_load_cache (void)
 
   /* Check the consistency.  */
   header = (struct gconvcache_header *) gconv_cache;
-  if (__builtin_expect (header->magic, GCONVCACHE_MAGIC) != GCONVCACHE_MAGIC
-      || __builtin_expect (header->string_offset >= cache_size, 0)
-      || __builtin_expect (header->hash_offset >= cache_size, 0)
-      || __builtin_expect (header->hash_size == 0, 0)
+  if (__glibc_unlikely (header->magic != GCONVCACHE_MAGIC)
+      || __glibc_unlikely (header->string_offset >= cache_size)
+      || __glibc_unlikely (header->hash_offset >= cache_size)
+      || __glibc_unlikely (header->hash_size == 0)
       || __builtin_expect ((header->hash_offset
 			    + header->hash_size * sizeof (struct hash_entry))
 			   > cache_size, 0)
-      || __builtin_expect (header->module_offset >= cache_size, 0)
-      || __builtin_expect (header->otherconv_offset > cache_size, 0))
+      || __glibc_unlikely (header->module_offset >= cache_size)
+      || __glibc_unlikely (header->otherconv_offset > cache_size))
     {
       if (cache_malloced)
 	{
@@ -283,12 +283,12 @@ __gconv_lookup_cache (const char *toset, const char *fromset,
   to_module = &modtab[toidx];
 
   /* Avoid copy-only transformations if the user requests.   */
-  if (__builtin_expect (flags & GCONV_AVOID_NOCONV, 0) && fromidx == toidx)
+  if (__glibc_unlikely (flags & GCONV_AVOID_NOCONV) && fromidx == toidx)
     return __GCONV_NULCONV;
 
   /* If there are special conversions available examine them first.  */
   if (fromidx != 0 && toidx != 0
-      && __builtin_expect (from_module->extra_offset, 0) != 0)
+      && __glibc_unlikely (from_module->extra_offset != 0))
     {
       /* Search through the list to see whether there is a module
 	 matching the destination character set.  */
@@ -339,7 +339,7 @@ __gconv_lookup_cache (const char *toset, const char *fromset,
 		  res = find_module (strtab + extra->module[idx].dir_offset,
 				     strtab + extra->module[idx].name_offset,
 				     &result[idx]);
-		  if (__builtin_expect (res, __GCONV_OK) != __GCONV_OK)
+		  if (__glibc_unlikely (res != __GCONV_OK))
 		    {
 		      /* Something went wrong.  */
 		      free (result);
@@ -362,8 +362,8 @@ __gconv_lookup_cache (const char *toset, const char *fromset,
 
  try_internal:
   /* See whether we can convert via the INTERNAL charset.  */
-  if ((fromidx != 0 && __builtin_expect (from_module->fromname_offset, 1) == 0)
-      || (toidx != 0 && __builtin_expect (to_module->toname_offset, 1) == 0)
+  if ((fromidx != 0 && __glibc_unlikely (from_module->fromname_offset == 0))
+      || (toidx != 0 && __glibc_unlikely (to_module->toname_offset == 0))
       || (fromidx == 0 && toidx == 0))
     /* Not possible.  Nothing we can do.  */
     return __GCONV_NOCONV;
@@ -392,7 +392,7 @@ __gconv_lookup_cache (const char *toset, const char *fromset,
 	  int res = find_module (strtab + from_module->todir_offset,
 				 strtab + from_module->toname_offset,
 				 &result[0]);
-	  if (__builtin_expect (res, __GCONV_OK) != __GCONV_OK)
+	  if (__glibc_unlikely (res != __GCONV_OK))
 	    {
 	      /* Something went wrong.  */
 	      free (result);
@@ -426,7 +426,7 @@ __gconv_lookup_cache (const char *toset, const char *fromset,
 	  int res = find_module (strtab + to_module->fromdir_offset,
 				 strtab + to_module->fromname_offset,
 				 &result[idx]);
-	  if (__builtin_expect (res, __GCONV_OK) != __GCONV_OK)
+	  if (__glibc_unlikely (res != __GCONV_OK))
 	    {
 	      /* Something went wrong.  */
 	      if (idx != 0)
diff --git a/iconv/gconv_db.c b/iconv/gconv_db.c
index 7d752bc..5487d9d 100644
--- a/iconv/gconv_db.c
+++ b/iconv/gconv_db.c
@@ -302,7 +302,7 @@ gen_steps (struct derivation_step *best, const char *toset,
 # endif
 		  status = DL_CALL_FCT (init_fct, (&result[step_cnt]));
 
-		  if (__builtin_expect (status, __GCONV_OK) != __GCONV_OK)
+		  if (__glibc_unlikely (status != __GCONV_OK))
 		    {
 		      failed = 1;
 		      /* Make sure we unload this modules.  */
@@ -326,7 +326,7 @@ gen_steps (struct derivation_step *best, const char *toset,
 	  current = current->last;
 	}
 
-      if (__builtin_expect (failed, 0) != 0)
+      if (__glibc_unlikely (failed != 0))
 	{
 	  /* Something went wrong while initializing the modules.  */
 	  while (++step_cnt < *nsteps)
@@ -746,7 +746,7 @@ __gconv_find_transform (const char *toset, const char *fromset,
   fromset_expand = do_lookup_alias (fromset);
   toset_expand = do_lookup_alias (toset);
 
-  if (__builtin_expect (flags & GCONV_AVOID_NOCONV, 0)
+  if (__glibc_unlikely (flags & GCONV_AVOID_NOCONV)
       /* We are not supposed to create a pseudo transformation (means
 	 copying) when the input and output character set are the same.  */
       && (strcmp (toset, fromset) == 0
diff --git a/iconv/gconv_dl.c b/iconv/gconv_dl.c
index 9aeaf0e..840793d 100644
--- a/iconv/gconv_dl.c
+++ b/iconv/gconv_dl.c
@@ -93,8 +93,7 @@ __gconv_find_shlib (const char *name)
 	  found->counter = -TRIES_BEFORE_UNLOAD - 1;
 	  found->handle = NULL;
 
-	  if (__builtin_expect (__tsearch (found, &loaded, known_compare)
-				== NULL, 0))
+	  if (__glibc_unlikely (__tsearch (found, &loaded, known_compare) == NULL))
 	    {
 	      /* Something went wrong while inserting the entry.  */
 	      free (found);
diff --git a/iconv/gconv_open.c b/iconv/gconv_open.c
index bfbe22b..7d64153 100644
--- a/iconv/gconv_open.c
+++ b/iconv/gconv_open.c
@@ -179,7 +179,7 @@ __gconv_open (const char *toset, const char *fromset, __gconv_t *handle,
       for (runp = trans; runp != NULL; runp = runp->next)
 	{
 	  if (runp->name == NULL
-	      || __builtin_expect (__gconv_translit_find (runp), 0) == 0)
+	      || __glibc_likely (__gconv_translit_find (runp) == 0))
 	    lastp = runp;
 	  else
 	    {
diff --git a/iconv/gconv_simple.c b/iconv/gconv_simple.c
index 4ed4505..6ab90c4 100644
--- a/iconv/gconv_simple.c
+++ b/iconv/gconv_simple.c
@@ -387,8 +387,7 @@ ucs4_internal_loop_single (struct __gconv_step *step,
       return __GCONV_INCOMPLETE_INPUT;
     }
 
-  if (__builtin_expect (((unsigned char *) state->__value.__wchb)[0] > 0x80,
-			0))
+  if (__glibc_unlikely (((unsigned char *) state->__value.__wchb)[0] > 0x80))
     {
       /* The value is too large.  We don't try transliteration here since
 	 this is not an error because of the lack of possibilities to
@@ -762,8 +761,7 @@ ucs4le_internal_loop_single (struct __gconv_step *step,
       return __GCONV_INCOMPLETE_INPUT;
     }
 
-  if (__builtin_expect (((unsigned char *) state->__value.__wchb)[3] > 0x80,
-			0))
+  if (__glibc_unlikely (((unsigned char *) state->__value.__wchb)[3] > 0x80))
     {
       /* The value is too large.  We don't try transliteration here since
 	 this is not an error because of the lack of possibilities to
diff --git a/iconv/iconv_open.c b/iconv/iconv_open.c
index 7bf19f5..5d2e926 100644
--- a/iconv/iconv_open.c
+++ b/iconv/iconv_open.c
@@ -75,7 +75,7 @@ iconv_open (const char *tocode, const char *fromcode)
   if (! tocode_usealloca)
     free (tocode_conv);
 
-  if (__builtin_expect (res, __GCONV_OK) != __GCONV_OK)
+  if (__glibc_unlikely (res != __GCONV_OK))
     {
       /* We must set the error number according to the specs.  */
       if (res == __GCONV_NOCONV || res == __GCONV_NODB)
diff --git a/iconv/loop.c b/iconv/loop.c
index a480c0c..3d831c9 100644
--- a/iconv/loop.c
+++ b/iconv/loop.c
@@ -307,7 +307,7 @@ FCTNAME (LOOPFCT) (struct __gconv_step *step,
 	 compiler generating better code.  They will be optimized away
 	 since MIN_NEEDED_OUTPUT is always a constant.  */
       if (MIN_NEEDED_INPUT > 1
-	  && __builtin_expect (inptr + MIN_NEEDED_INPUT > inend, 0))
+	  && __glibc_unlikely (inptr + MIN_NEEDED_INPUT > inend))
 	{
 	  /* We don't have enough input for another complete input
 	     character.  */
@@ -315,9 +315,9 @@ FCTNAME (LOOPFCT) (struct __gconv_step *step,
 	  break;
 	}
       if ((MIN_NEEDED_OUTPUT != 1
-	   && __builtin_expect (outptr + MIN_NEEDED_OUTPUT > outend, 0))
+	   && __glibc_unlikely (outptr + MIN_NEEDED_OUTPUT > outend))
 	  || (MIN_NEEDED_OUTPUT == 1
-	      && __builtin_expect (outptr >= outend, 0)))
+	      && __glibc_unlikely (outptr >= outend)))
 	{
 	  /* Overflow in the output buffer.  */
 	  result = __GCONV_FULL_OUTPUT;
@@ -394,7 +394,7 @@ SINGLE(LOOPFCT) (struct __gconv_step *step,
 
   /* Are there enough bytes in the input buffer?  */
   if (MIN_NEEDED_INPUT > 1
-      && __builtin_expect (inptr + (MIN_NEEDED_INPUT - inlen) > inend, 0))
+      && __glibc_unlikely (inptr + (MIN_NEEDED_INPUT - inlen) > inend))
     {
       *inptrp = inend;
 #  ifdef STORE_REST
diff --git a/iconv/skeleton.c b/iconv/skeleton.c
index 73dc186..b1c5f01 100644
--- a/iconv/skeleton.c
+++ b/iconv/skeleton.c
@@ -333,7 +333,7 @@ gconv_init (struct __gconv_step *step)
       step->__btowc_fct = FROM_ONEBYTE;
 #endif
     }
-  else if (__builtin_expect (strcmp (step->__to_name, CHARSET_NAME), 0) == 0)
+  else if (__glibc_likely (strcmp (step->__to_name, CHARSET_NAME) == 0))
     {
       step->__data = TO_DIRECTION_VAL;
 
@@ -503,7 +503,7 @@ FUNCTION_NAME (struct __gconv_step *step, struct __gconv_step_data *data,
     {
       /* We preserve the initial values of the pointer variables.  */
       const unsigned char *inptr = *inptrp;
-      unsigned char *outbuf = (__builtin_expect (outbufstart == NULL, 1)
+      unsigned char *outbuf = (__glibc_likely (outbufstart == NULL)
 			       ? data->__outbuf : *outbufstart);
       unsigned char *outend = data->__outbufend;
       unsigned char *outstart;
@@ -573,7 +573,7 @@ FUNCTION_NAME (struct __gconv_step *step, struct __gconv_step_data *data,
 #  endif
 # endif
 
-	  if (__builtin_expect (status, __GCONV_OK) != __GCONV_OK)
+	  if (__glibc_unlikely (status != __GCONV_OK))
 	    return status;
 	}
 #endif
@@ -772,7 +772,7 @@ FUNCTION_NAME (struct __gconv_step *step, struct __gconv_step_data *data,
       if (((FROM_LOOP_MAX_NEEDED_FROM > 1 && TO_LOOP_MAX_NEEDED_FROM > 1)
 	   || (FROM_LOOP_MAX_NEEDED_FROM > 1 && FROM_DIRECTION)
 	   || (TO_LOOP_MAX_NEEDED_FROM > 1 && !FROM_DIRECTION))
-	  && __builtin_expect (consume_incomplete, 0)
+	  && __glibc_unlikely (consume_incomplete)
 	  && status == __GCONV_INCOMPLETE_INPUT)
 	{
 # ifdef STORE_REST
diff --git a/iconvdata/8bit-gap.c b/iconvdata/8bit-gap.c
index 3bd7149..e08b15a 100644
--- a/iconvdata/8bit-gap.c
+++ b/iconvdata/8bit-gap.c
@@ -53,7 +53,7 @@ struct gap
   {									      \
     uint32_t ch = to_ucs4[*inptr];					      \
 									      \
-    if (HAS_HOLES && __builtin_expect (ch == L'\0', 0) && NONNUL (*inptr))    \
+    if (HAS_HOLES && __glibc_unlikely (ch == L'\0') && NONNUL (*inptr))	      \
       {									      \
 	/* This is an illegal character.  */				      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -71,7 +71,7 @@ struct gap
   {									      \
     uint32_t ch = to_ucs4[c];						      \
 									      \
-    if (HAS_HOLES && __builtin_expect (ch == L'\0', 0) && NONNUL (c))	      \
+    if (HAS_HOLES && __glibc_unlikely (ch == L'\0') && NONNUL (c))	      \
       return WEOF;							      \
     else								      \
       return ch;							      \
@@ -97,10 +97,10 @@ struct gap
     else								      \
       while (ch > rp->end)						      \
 	++rp;								      \
-    if (__builtin_expect (rp == NULL, 0)				      \
-	|| __builtin_expect (ch < rp->start, 0)				      \
+    if (__glibc_unlikely (rp == NULL)					      \
+	|| __glibc_unlikely (ch < rp->start)				      \
 	|| (res = from_ucs4[ch + rp->idx],				      \
-	    __builtin_expect (res, '\1') == '\0' && ch != 0))		      \
+	    __glibc_unlikely (res == '\0') && ch != 0))			      \
       {									      \
 	/* This is an illegal character.  */				      \
 	STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/8bit-generic.c b/iconvdata/8bit-generic.c
index efc0fd5..3d6a7c5 100644
--- a/iconvdata/8bit-generic.c
+++ b/iconvdata/8bit-generic.c
@@ -37,7 +37,7 @@
   {									      \
     uint32_t ch = to_ucs4[*inptr];					      \
 									      \
-    if (HAS_HOLES && __builtin_expect (ch == L'\0', 0) && *inptr != '\0')     \
+    if (HAS_HOLES && __glibc_unlikely (ch == L'\0') && *inptr != '\0')	      \
       {									      \
 	/* This is an illegal character.  */				      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -52,7 +52,7 @@
   {									      \
     uint32_t ch = to_ucs4[c];						      \
 									      \
-    if (HAS_HOLES && __builtin_expect (ch == L'\0', 0) && c != '\0')	      \
+    if (HAS_HOLES && __glibc_unlikely (ch == L'\0') && c != '\0')	      \
       return WEOF;							      \
     else								      \
       return ch;							      \
@@ -68,8 +68,8 @@
   {									      \
     uint32_t ch = get32 (inptr);					      \
 									      \
-    if (__builtin_expect (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0]), 0)\
-	|| (__builtin_expect (from_ucs4[ch], '\1') == '\0' && ch != 0))	      \
+    if (__glibc_unlikely (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0]))   \
+	|| (__glibc_unlikely (from_ucs4[ch] == '\0') && ch != 0))	      \
       {									      \
 	UNICODE_TAG_HANDLER (ch, 4);					      \
 									      \
diff --git a/iconvdata/ansi_x3.110.c b/iconvdata/ansi_x3.110.c
index d602dfa..e645b5f 100644
--- a/iconvdata/ansi_x3.110.c
+++ b/iconvdata/ansi_x3.110.c
@@ -402,7 +402,7 @@ static const char from_ucs4[][2] =
     uint32_t ch = *inptr;						      \
     int incr;								      \
 									      \
-    if (__builtin_expect (ch >= 0xc1, 0) && ch <= 0xcf)			      \
+    if (__glibc_unlikely (ch >= 0xc1) && ch <= 0xcf)			      \
       {									      \
 	/* Composed character.  First test whether the next byte	      \
 	   is also available.  */					      \
@@ -417,8 +417,8 @@ static const char from_ucs4[][2] =
 									      \
 	ch2 = inptr[1];							      \
 									      \
-	if (__builtin_expect (ch2 < 0x20, 0)				      \
-	    || __builtin_expect (ch2 >= 0x80, 0))			      \
+	if (__glibc_unlikely (ch2 < 0x20)				      \
+	    || __glibc_unlikely (ch2 >= 0x80))				      \
 	  {								      \
 	    /* This is illegal.  */					      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -435,7 +435,7 @@ static const char from_ucs4[][2] =
 	incr = 1;							      \
       }									      \
 									      \
-    if (__builtin_expect (ch == 0, 0) && *inptr != '\0')		      \
+    if (__glibc_unlikely (ch == 0) && *inptr != '\0')			      \
       {									      \
 	/* This is an illegal character.  */				      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (incr);				      \
@@ -453,7 +453,7 @@ static const char from_ucs4[][2] =
   {									      \
     uint32_t ch = to_ucs4[c];						      \
 									      \
-    if (__builtin_expect (ch == 0, 0) && c != '\0')			      \
+    if (__glibc_unlikely (ch == 0) && c != '\0')			      \
       return WEOF;							      \
     else								      \
       return ch;							      \
@@ -472,8 +472,7 @@ static const char from_ucs4[][2] =
     uint32_t ch = get32 (inptr);					      \
     const char *cp;							      \
 									      \
-    if (__builtin_expect (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0]),   \
-	0))								      \
+    if (__glibc_unlikely (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0])))								      \
       {									      \
 	if (ch == 0x2c7)						      \
 	  cp = "\xcf\x20";						      \
@@ -543,7 +542,7 @@ static const char from_ucs4[][2] =
 	    tmp[1] = '\0';						      \
 	    cp = tmp;							      \
 	  }								      \
-	else if (__builtin_expect (ch, 0x266a) == 0x266a)		      \
+	else if (__glibc_likely (ch == 0x266a))				      \
 	  cp = "\xd5";							      \
 	else								      \
 	  {								      \
@@ -557,7 +556,7 @@ static const char from_ucs4[][2] =
       {									      \
 	cp = from_ucs4[ch];						      \
 									      \
-	if (__builtin_expect (cp[0], '\1') == '\0' && ch != 0)		      \
+	if (__glibc_unlikely (cp[0] == '\0') && ch != 0)		      \
 	  {								      \
 	    /* Illegal characters.  */					      \
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/big5.c b/iconvdata/big5.c
index ee3dede..69258d9 100644
--- a/iconvdata/big5.c
+++ b/iconvdata/big5.c
@@ -8414,8 +8414,8 @@ static const char from_ucs4_tab15[][2] =
 	/* See whether the second byte is in the correct range.  */	      \
 	if (ch2 >= 0x40 && ch2 <= 0x7e)					      \
 	  idx += ch2 - 0x40;						      \
-	else if (__builtin_expect (ch2 >= 0xa1, 1)			      \
-		 && __builtin_expect (ch2 <= 0xfe, 1))			      \
+	else if (__glibc_likely (ch2 >= 0xa1)				      \
+		 && __glibc_likely (ch2 <= 0xfe))			      \
 	  idx += 0x3f + (ch2 - 0xa1);					      \
 	else								      \
 	  {								      \
@@ -8467,8 +8467,7 @@ static const char from_ucs4_tab15[][2] =
     uint32_t ch = get32 (inptr);					      \
     const char *cp;							      \
 									      \
-    if (__builtin_expect (ch >= (sizeof (from_ucs4_tab1)		      \
-				 / sizeof (from_ucs4_tab1[0])), 0))	      \
+    if (__glibc_unlikely (ch >= (sizeof (from_ucs4_tab1) / sizeof (from_ucs4_tab1[0]))))	      \
       switch (ch)							      \
 	{								      \
         case 0x2c7 ... 0x2d9:						      \
@@ -8555,7 +8554,7 @@ static const char from_ucs4_tab15[][2] =
     else								      \
       cp = from_ucs4_tab1[ch];						      \
 									      \
-    if (__builtin_expect (cp[0], '\1') == '\0' && ch != 0)		      \
+    if (__glibc_unlikely (cp[0] == '\0') && ch != 0)			      \
       {									      \
 	/* Illegal character.  */					      \
 	STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
@@ -8563,8 +8562,8 @@ static const char from_ucs4_tab15[][2] =
     else								      \
       {									      \
 	/* See whether there is enough room for the second byte we write.  */ \
-	if (__builtin_expect (cp[1], '\1') != '\0'			      \
-	    && __builtin_expect (outptr + 1 >= outend, 0))		      \
+	if (__glibc_likely (cp[1] != '\0')				      \
+	    && __glibc_unlikely (outptr + 1 >= outend))			      \
 	  {								      \
 	    /* We have not enough room.  */				      \
 	    result = __GCONV_FULL_OUTPUT;				      \
diff --git a/iconvdata/big5hkscs.c b/iconvdata/big5hkscs.c
index 5dce195..64cc161 100644
--- a/iconvdata/big5hkscs.c
+++ b/iconvdata/big5hkscs.c
@@ -18001,8 +18001,8 @@ static struct
 	      }								      \
 									      \
 	    /* See whether there is enough room to write the second byte.  */ \
-	    if (__builtin_expect (cp[1] != '\0', 1)			      \
-		&& __builtin_expect (outptr + 1 >= outend, 0))		      \
+	    if (__glibc_likely (cp[1] != '\0')				      \
+		&& __glibc_unlikely (outptr + 1 >= outend))		      \
 	      {								      \
 		/* We have not enough room.  */				      \
 		result = __GCONV_FULL_OUTPUT;				      \
diff --git a/iconvdata/cp1255.c b/iconvdata/cp1255.c
index 1fab525..c4a3bec 100644
--- a/iconvdata/cp1255.c
+++ b/iconvdata/cp1255.c
@@ -307,7 +307,7 @@ static const struct { unsigned int idx; unsigned int len; } comp_table[8] = {
 									      \
 	/* If we don't have enough room to output ch as well, then deal	      \
 	   with it in another round.  */				      \
-	if (!must_buffer_ch && __builtin_expect (outptr + 4 > outend, 0))     \
+	if (!must_buffer_ch && __glibc_unlikely (outptr + 4 > outend))	      \
 	  continue;							      \
       }									      \
 									      \
diff --git a/iconvdata/cp1258.c b/iconvdata/cp1258.c
index 5153eea..c93886f 100644
--- a/iconvdata/cp1258.c
+++ b/iconvdata/cp1258.c
@@ -465,7 +465,7 @@ static const struct
 									      \
 	/* If we don't have enough room to output ch as well, then deal	      \
 	   with it in another round.  */				      \
-	if (!must_buffer_ch && __builtin_expect (outptr + 4 > outend, 0))     \
+	if (!must_buffer_ch && __glibc_unlikely (outptr + 4 > outend))	      \
 	  continue;							      \
       }									      \
 									      \
diff --git a/iconvdata/cp932.c b/iconvdata/cp932.c
index a358eda..7c094b2 100644
--- a/iconvdata/cp932.c
+++ b/iconvdata/cp932.c
@@ -4552,9 +4552,9 @@ static const char from_ucs4_extra[229][2] =
 	ch += 0xfec0;                                                         \
 	++inptr;                                                              \
       }									      \
-    else if (__builtin_expect (ch, 0) == 0xa0				      \
-	     || __builtin_expect (ch <= 0x80, 0)			      \
-	     || __builtin_expect (ch > 0xfc, 0))			      \
+    else if (__glibc_unlikely (ch == 0xa0)				      \
+	     || __glibc_unlikely (ch <= 0x80)				      \
+	     || __glibc_unlikely (ch > 0xfc))				      \
       {									      \
 	/* These are illegal.  */					      \
 	if (! ignore_errors_p ())					      \
@@ -4585,16 +4585,16 @@ static const char from_ucs4_extra[229][2] =
 									      \
 	ch2 = inptr[1];							      \
 	idx = ch * 256 + ch2;						      \
-	if (__builtin_expect (ch2 < 0x40, 0)				      \
-	    || __builtin_expect (ch2 > 0xfc, 0)				      \
-	    || __builtin_expect (ch2 == 0x7f, 0)			      \
-	    || (__builtin_expect (idx > 0x84be, 0) && idx < 0x8740)	      \
-	    || (__builtin_expect (idx > 0x879c, 0) && idx < 0x889f)	      \
-	    || (__builtin_expect (idx > 0x88fc, 0) && idx < 0x8940)	      \
-	    || (__builtin_expect (idx > 0x9ffc, 0) && idx < 0xe040)	      \
-	    || (__builtin_expect (idx > 0xeaa4, 0) && idx < 0xed40)	      \
-	    || (__builtin_expect (idx > 0xeefc, 0) && idx < 0xf040)	      \
-	    || __builtin_expect (idx > 0xfc4b, 0))			      \
+	if (__glibc_unlikely (ch2 < 0x40)				      \
+	    || __glibc_unlikely (ch2 > 0xfc)				      \
+	    || __glibc_unlikely (ch2 == 0x7f)				      \
+	    || (__glibc_unlikely (idx > 0x84be) && idx < 0x8740)	      \
+	    || (__glibc_unlikely (idx > 0x879c) && idx < 0x889f)	      \
+	    || (__glibc_unlikely (idx > 0x88fc) && idx < 0x8940)	      \
+	    || (__glibc_unlikely (idx > 0x9ffc) && idx < 0xe040)	      \
+	    || (__glibc_unlikely (idx > 0xeaa4) && idx < 0xed40)	      \
+	    || (__glibc_unlikely (idx > 0xeefc) && idx < 0xf040)	      \
+	    || __glibc_unlikely (idx > 0xfc4b))				      \
 	  {								      \
 	    /* This is illegal.  */					      \
 	    if (! ignore_errors_p ())					      \
@@ -4629,7 +4629,7 @@ static const char from_ucs4_extra[229][2] =
 	else								      \
 	  ch = cjk_block7[(ch - 0xfa) * 192 + ch2 - 0x40];		      \
 									      \
-	if (__builtin_expect (ch, 1) == 0)				      \
+	if (__glibc_unlikely (ch == 0))					      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    if (! ignore_errors_p ())					      \
@@ -4681,8 +4681,8 @@ static const char from_ucs4_extra[229][2] =
 	  }								      \
 	else if (ch >= 0xf929 && ch <= 0xfa2d)				      \
 	  cp = from_ucs4_cjkcpt[ch - 0xf929];				      \
-	else if (__builtin_expect (ch >= 0xff01, 1)			      \
-		 && __builtin_expect (ch <= 0xffe5, 1))			      \
+	else if (__glibc_likely (ch >= 0xff01)				      \
+		 && __glibc_likely (ch <= 0xffe5))			      \
 	  cp = from_ucs4_extra[ch - 0xff01];				      \
 	else								      \
 	  {								      \
@@ -4694,7 +4694,7 @@ static const char from_ucs4_extra[229][2] =
     else								      \
       cp = from_ucs4_lat1[ch];						      \
 									      \
-    if (__builtin_expect (cp[0], '\1') == '\0' && ch != 0)		      \
+    if (__glibc_unlikely (cp[0] == '\0') && ch != 0)			      \
       {									      \
 	/* Illegal character.  */					      \
 	STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/euc-cn.c b/iconvdata/euc-cn.c
index 0d625b4..1b0b307 100644
--- a/iconvdata/euc-cn.c
+++ b/iconvdata/euc-cn.c
@@ -45,8 +45,8 @@
     if (ch <= 0x7f)							      \
       ++inptr;								      \
     else								      \
-      if ((__builtin_expect (ch <= 0xa0, 0) && ch != 0x8e && ch != 0x8f)      \
-	  || __builtin_expect (ch > 0xfe, 0))				      \
+      if ((__glibc_unlikely (ch <= 0xa0) && ch != 0x8e && ch != 0x8f)	      \
+	  || __glibc_unlikely (ch > 0xfe))				      \
 	{								      \
 	  /* This is illegal.  */					      \
 	  STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -115,9 +115,9 @@
 	size_t found;							      \
 									      \
 	found = ucs4_to_gb2312 (ch, outptr, outend - outptr);		      \
-	if (__builtin_expect (found, 1) != 0)				      \
+	if (__glibc_likely (found != 0))				      \
 	  {								      \
-	    if (__builtin_expect (found, 0) == __UNKNOWN_10646_CHAR)	      \
+	    if (__glibc_unlikely (found == __UNKNOWN_10646_CHAR))	      \
 	      {								      \
 		UNICODE_TAG_HANDLER (ch, 4);				      \
 									      \
diff --git a/iconvdata/euc-jp-ms.c b/iconvdata/euc-jp-ms.c
index 8aed520..bb18a9d 100644
--- a/iconvdata/euc-jp-ms.c
+++ b/iconvdata/euc-jp-ms.c
@@ -4699,11 +4699,11 @@ static const unsigned char from_ucs4_extra[229][2] =
 	    continue;							      \
 	  }								      \
 									      \
-	if (__builtin_expect(ch == 0x8e, 0))				      \
+	if (__glibc_unlikely (ch == 0x8e))				      \
 	  {								      \
 	    /* This is code set 2: half-width katakana.  */		      \
 	    ch = jisx0201_to_ucs4 (ch2);				      \
-	    /*if (__builtin_expect (ch, 0) == __UNKNOWN_10646_CHAR)*/	      \
+	    /*if (__glibc_unlikely (ch == __UNKNOWN_10646_CHAR))*/	      \
 	    if (__glibc_unlikely (ch == __UNKNOWN_10646_CHAR))		      \
 	      {								      \
 		/* Illegal character.  */				      \
@@ -4722,7 +4722,7 @@ static const unsigned char from_ucs4_extra[229][2] =
 	    const unsigned char *endp = inptr;				      \
 	    int mblen = 1;						      \
 									      \
-	    if (__builtin_expect(ch == 0x8f, 0))			      \
+	    if (__glibc_unlikely (ch == 0x8f))				      \
 	      {								      \
 		if (inend - inptr < 3)					      \
 		  ch = 0;						      \
@@ -4731,8 +4731,8 @@ static const unsigned char from_ucs4_extra[229][2] =
 		    unsigned char ch3 = (unsigned char)inptr[2];	      \
 		    mblen = 3;						      \
 									      \
-		    if (__builtin_expect(ch3 == 0xff, 0)		      \
-		        || __builtin_expect(ch3 < 0xa1, 0))		      \
+		    if (__glibc_unlikely (ch3 == 0xff)			      \
+			|| __glibc_unlikely (ch3 < 0xa1))		      \
 		      ch = __UNKNOWN_10646_CHAR;			      \
 		    else if (ch2 <= 0xf2)				      \
 		      {							      \
@@ -4769,13 +4769,13 @@ static const unsigned char from_ucs4_extra[229][2] =
 		      ch = __UNKNOWN_10646_CHAR;			      \
 		  }							      \
 	      }								      \
-	    else if (__builtin_expect(0xa1 <= ch, 1))			      \
+	    else if (__glibc_likely (0xa1 <= ch))			      \
 	      {								      \
 		mblen = 2;						      \
 									      \
 		if (inend - inptr < 2)					      \
 		  ch = 0;						      \
-		else if (__builtin_expect(ch2 == 0xff, 0))		      \
+		else if (__glibc_unlikely (ch2 == 0xff))		      \
 		  ch = __UNKNOWN_10646_CHAR;				      \
 		else if (ch <= 0xa8)					      \
 		  {							      \
@@ -4809,13 +4809,13 @@ static const unsigned char from_ucs4_extra[229][2] =
 	    else							      \
 	      ch = __UNKNOWN_10646_CHAR;				      \
 									      \
-	    if (__builtin_expect (ch, 1) == 0)				      \
+	    if (__glibc_unlikely (ch == 0))				      \
 	      {								      \
 		/* Not enough input available.  */			      \
 		result = __GCONV_INCOMPLETE_INPUT;			      \
 		break;							      \
 	      }								      \
-	    if (__builtin_expect (ch, 0) == __UNKNOWN_10646_CHAR)	      \
+	    if (__glibc_unlikely (ch == __UNKNOWN_10646_CHAR))		      \
 	      {								      \
 		/* Illegal character.  */				      \
 		if (! ignore_errors_p ())				      \
@@ -4873,8 +4873,8 @@ static const unsigned char from_ucs4_extra[229][2] =
 	  }								      \
 	else if (ch >= 0xf929 && ch <= 0xfa2d)				      \
 	  cp = from_ucs4_cjkcpt[ch - 0xf929];				      \
-	else if (__builtin_expect (ch >= 0xff01, 1)			      \
-		 && __builtin_expect (ch <= 0xffe5, 1))			      \
+	else if (__glibc_likely (ch >= 0xff01)				      \
+		 && __glibc_likely (ch <= 0xffe5))			      \
 	  cp = from_ucs4_extra[ch - 0xff01];				      \
 	else								      \
 	  {								      \
@@ -4886,7 +4886,7 @@ static const unsigned char from_ucs4_extra[229][2] =
     else								      \
       cp = from_ucs4_lat1[ch];						      \
 									      \
-    if (__builtin_expect (cp[0], '\1') == '\0' && ch != 0)		      \
+    if (__glibc_unlikely (cp[0] == '\0') && ch != 0)			      \
       {									      \
 	/* Illegal character.  */					      \
 	STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/euc-jp.c b/iconvdata/euc-jp.c
index 7689250..616045e 100644
--- a/iconvdata/euc-jp.c
+++ b/iconvdata/euc-jp.c
@@ -76,7 +76,7 @@
 	  {								      \
 	    /* This is code set 2: half-width katakana.  */		      \
 	    ch = jisx0201_to_ucs4 (ch2);				      \
-	    if (__builtin_expect (ch, 0) == __UNKNOWN_10646_CHAR)	      \
+	    if (__glibc_unlikely (ch == __UNKNOWN_10646_CHAR))		      \
 	      STANDARD_FROM_LOOP_ERR_HANDLER (1);			      \
 									      \
 	    inptr += 2;							      \
@@ -100,7 +100,7 @@
 		ch = jisx0208_to_ucs4 (&endp, inend - inptr, 0x80);	      \
 	      }								      \
 									      \
-	    if (__builtin_expect (ch, 1) == 0)				      \
+	    if (__glibc_unlikely (ch == 0))				      \
 	      {								      \
 		/* Not enough input available.  */			      \
 		result = __GCONV_INCOMPLETE_INPUT;			      \
@@ -182,13 +182,13 @@
 		found = ucs4_to_jisx0212 (ch, outptr + 1,		      \
 					  outend - outptr - 1);		      \
 		  							      \
-		if (__builtin_expect (found, 1) == 0)			      \
+		if (__glibc_unlikely (found == 0))			      \
 		  {							      \
 		    /* We ran out of space.  */				      \
 		    result = __GCONV_FULL_OUTPUT;			      \
 		    break;						      \
 		  }							      \
-		else if (__builtin_expect (found, 0) != __UNKNOWN_10646_CHAR) \
+		else if (__glibc_likely (found != __UNKNOWN_10646_CHAR))      \
 		  {							      \
 		    /* It's a JIS 0212 character, adjust it for EUC-JP.  */   \
 		    *outptr++ = 0x8f;					      \
diff --git a/iconvdata/euc-kr.c b/iconvdata/euc-kr.c
index 3b13c2e..50d1a8c 100644
--- a/iconvdata/euc-kr.c
+++ b/iconvdata/euc-kr.c
@@ -29,7 +29,7 @@ euckr_from_ucs4 (uint32_t ch, unsigned char *cp)
 {
   if (ch > 0x9f)
     {
-      if (__builtin_expect (ch, 0) == 0x20a9)
+      if (__glibc_unlikely (ch == 0x20a9))
 	{
 	  /* Half-width Korean Currency WON sign.  There is no
              equivalent in EUC-KR.  Some mappings use \x5c because
@@ -38,8 +38,7 @@ euckr_from_ucs4 (uint32_t ch, unsigned char *cp)
 	  cp[0] = '\xa3';
 	  cp[1] = '\xdc';
 	}
-      else if (__builtin_expect (ucs4_to_ksc5601 (ch, cp, 2), 0)
-	  != __UNKNOWN_10646_CHAR)
+      else if (__glibc_likely (ucs4_to_ksc5601 (ch, cp, 2) != __UNKNOWN_10646_CHAR))
 	{
 	  cp[0] |= 0x80;
 	  cp[1] |= 0x80;
@@ -82,9 +81,9 @@ euckr_from_ucs4 (uint32_t ch, unsigned char *cp)
       ++inptr;								      \
     /* 0xfe(->0x7e : row 94) and 0xc9(->0x59 : row 41) are		      \
        user-defined areas.  */						      \
-    else if (__builtin_expect (ch == 0xa0, 0)				      \
-	     || __builtin_expect (ch > 0xfe, 0)				      \
-	     || __builtin_expect (ch == 0xc9, 0))			      \
+    else if (__glibc_unlikely (ch == 0xa0)				      \
+	     || __glibc_unlikely (ch > 0xfe)				      \
+	     || __glibc_unlikely (ch == 0xc9))				      \
       {									      \
 	/* This is illegal.  */						      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -133,7 +132,7 @@ euckr_from_ucs4 (uint32_t ch, unsigned char *cp)
        Jamos should be considered either here or in euckr_from_ucs4() */      \
     euckr_from_ucs4 (ch, cp);						      \
 									      \
-    if (__builtin_expect (cp[0], '\1') == '\0' && ch != 0)		      \
+    if (__glibc_unlikely (cp[0] == '\0') && ch != 0)			      \
       {									      \
 	UNICODE_TAG_HANDLER (ch, 4);					      \
 									      \
diff --git a/iconvdata/euc-tw.c b/iconvdata/euc-tw.c
index 153e3a0..9218091 100644
--- a/iconvdata/euc-tw.c
+++ b/iconvdata/euc-tw.c
@@ -140,13 +140,13 @@
 	size_t found;							      \
 									      \
 	found = ucs4_to_cns11643l1 (ch, outptr, outend - outptr);	      \
-	if (__builtin_expect (found, 1) == 0)				      \
+	if (__glibc_unlikely (found == 0))				      \
 	  {								      \
 	    /* We ran out of space.  */					      \
 	    result = __GCONV_FULL_OUTPUT;				      \
 	    break;							      \
 	  }								      \
-	if (__builtin_expect (found, 1) != __UNKNOWN_10646_CHAR)	      \
+	if (__glibc_likely (found != __UNKNOWN_10646_CHAR))		      \
 	  {								      \
 	    /* It's a CNS 11643, plane 1 character, adjust it for EUC-TW.  */ \
 	    *outptr++ += 0x80;						      \
@@ -157,13 +157,13 @@
 	    /* No CNS 11643, plane 1 character.  */			      \
 									      \
 	    found = ucs4_to_cns11643 (ch, outptr + 1, outend - outptr - 1);   \
-	    if (__builtin_expect (found, 1) == 0)			      \
+	    if (__glibc_unlikely (found == 0))				      \
 	      {								      \
 		/* We ran out of space.  */				      \
 		result = __GCONV_FULL_OUTPUT;				      \
 		break;							      \
 	      }								      \
-	    if (__builtin_expect (found, 0) == __UNKNOWN_10646_CHAR)	      \
+	    if (__glibc_unlikely (found == __UNKNOWN_10646_CHAR))	      \
 	      {								      \
 		UNICODE_TAG_HANDLER (ch, 4);				      \
 									      \
diff --git a/iconvdata/gb18030.c b/iconvdata/gb18030.c
index f12c451..cb466f3 100644
--- a/iconvdata/gb18030.c
+++ b/iconvdata/gb18030.c
@@ -24350,8 +24350,8 @@ static const unsigned char __ucs_to_gb18030_tab2[][2] =
 	else								      \
 	  len = 0;							      \
 									      \
-	if (__builtin_expect (len, 2) == 0				      \
-	    || (len == 2 && __builtin_expect (cp[0], '\1') == '\0'))          \
+	if (__glibc_unlikely (len == 0)					      \
+	    || (len == 2 && __glibc_unlikely (cp[0] == '\0')))		      \
 	  {								      \
 	    /* Illegal character.  */					      \
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
@@ -24361,7 +24361,7 @@ static const unsigned char __ucs_to_gb18030_tab2[][2] =
 	  {								      \
 	    /* See whether there is enough room for the second byte we	      \
 	       write.  */						      \
-	    if (cp[1] != '\0' && __builtin_expect (outptr + 1 >= outend, 0))  \
+	    if (cp[1] != '\0' && __glibc_unlikely (outptr + 1 >= outend))     \
 	      {								      \
 		/* We have not enough room.  */				      \
 		result = __GCONV_FULL_OUTPUT;				      \
diff --git a/iconvdata/gbbig5.c b/iconvdata/gbbig5.c
index ddaa468..cf4bc4b 100644
--- a/iconvdata/gbbig5.c
+++ b/iconvdata/gbbig5.c
@@ -4852,8 +4852,8 @@ const char __from_big5_to_gb2312 [13973][2] =
 	else								      \
 	  {								      \
 	    /* See whether there is enough room to write the second byte. */  \
-	    if (__builtin_expect (cp[1], '\1') != '\0'			      \
-	        && __builtin_expect (outptr + 1 >= outend, 0))		      \
+	    if (__glibc_likely (cp[1] != '\0')				      \
+		&& __glibc_unlikely (outptr + 1 >= outend))		      \
 	      {								      \
 	        /* We do not have enough room.  */			      \
 	        result = __GCONV_FULL_OUTPUT;				      \
@@ -4908,8 +4908,8 @@ const char __from_big5_to_gb2312 [13973][2] =
 	/* See if the second byte is in the correct range. */		      \
 	if (ch >= 0x40 && ch <= 0x7e)					      \
 	  idx += ch - 0x40;						      \
-	else if (__builtin_expect (ch >= 0xa1, 1)		  	      \
-		 && __builtin_expect (ch <= 0xfe, 1))			      \
+	else if (__glibc_likely (ch >= 0xa1)				      \
+		 && __glibc_likely (ch <= 0xfe))			      \
 	  idx += 0x3f + (ch - 0xa1);					      \
 	else								      \
 	  {								      \
@@ -4943,8 +4943,8 @@ const char __from_big5_to_gb2312 [13973][2] =
 	else								      \
 	  {								      \
 	    /* see if there is enough room to write the second byte. */	      \
-	    if (__builtin_expect (cp[1], '\1') != '\0'			      \
-	        && __builtin_expect (outptr + 1 >= outend, 0))		      \
+	    if (__glibc_likely (cp[1] != '\0')				      \
+		&& __glibc_unlikely (outptr + 1 >= outend))		      \
 	      {								      \
 	        /* We do not have enough room.  */			      \
 	        result = __GCONV_FULL_OUTPUT;				      \
diff --git a/iconvdata/gbgbk.c b/iconvdata/gbgbk.c
index 620c39d..eaefecb 100644
--- a/iconvdata/gbgbk.c
+++ b/iconvdata/gbgbk.c
@@ -96,16 +96,16 @@
 	  ch = 0xa1aa;							      \
 									      \
 	/* Now determine whether the character is valid.  */		      \
-	if (__builtin_expect (ch < 0xa1a1, 0)				      \
-	    || __builtin_expect (ch > 0xf7fe, 0)			      \
-	    || __builtin_expect (inptr[1] < 0xa1, 0)			      \
+	if (__glibc_unlikely (ch < 0xa1a1)				      \
+	    || __glibc_unlikely (ch > 0xf7fe)				      \
+	    || __glibc_unlikely (inptr[1] < 0xa1)			      \
 	    /* Now test the exceptions.  */				      \
-	    || (__builtin_expect (ch >= 0xa2a1, 0)			      \
-		&& __builtin_expect (ch <= 0xa2aa, 0))			      \
-	    || (__builtin_expect (ch >= 0xa6e0, 0)			      \
-		&& __builtin_expect (ch <= 0xa6f5, 0))			      \
-	    || (__builtin_expect (ch >= 0xa8bb, 0)			      \
-		&& __builtin_expect (ch <= 0xa8c0, 0)))			      \
+	    || (__glibc_unlikely (ch >= 0xa2a1)				      \
+		&& __glibc_unlikely (ch <= 0xa2aa))			      \
+	    || (__glibc_unlikely (ch >= 0xa6e0)				      \
+		&& __glibc_unlikely (ch <= 0xa6f5))			      \
+	    || (__glibc_unlikely (ch >= 0xa8bb)				      \
+		&& __glibc_unlikely (ch <= 0xa8c0)))			      \
 	  {								      \
 	    /* One of the characters we cannot map.  */			      \
 	    STANDARD_TO_LOOP_ERR_HANDLER (2);				      \
diff --git a/iconvdata/gbk.c b/iconvdata/gbk.c
index b1a7719..f3ca343 100644
--- a/iconvdata/gbk.c
+++ b/iconvdata/gbk.c
@@ -13145,8 +13145,8 @@ static const char __gbk_from_ucs4_tab12[][2] =
     if (ch <= 0x7f)							      \
       ++inptr;								      \
     else								      \
-      if (__builtin_expect (ch <= 0x80, 0)				      \
-	  || __builtin_expect (ch > 0xfe, 0))				      \
+      if (__glibc_unlikely (ch <= 0x80)					      \
+	  || __glibc_unlikely (ch > 0xfe))				      \
 	{								      \
 	  /* This is illegal.  */					      \
 	  STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -13170,8 +13170,8 @@ static const char __gbk_from_ucs4_tab12[][2] =
 									      \
 	  /* All second bytes of a multibyte character must be >= 0x40, and   \
 	     the __gbk_to_ucs table only covers the range up to 0xfe 0xa0. */ \
-	  if (__builtin_expect (ch2 < 0x40, 0)				      \
-	      || (__builtin_expect (ch, 0x81) == 0xfe && ch2 > 0xa0))	      \
+	  if (__glibc_unlikely (ch2 < 0x40)				      \
+	      || (__glibc_unlikely (ch == 0xfe) && ch2 > 0xa0))		      \
 	    {								      \
 	      /* This is an illegal character.  */			      \
 	      STANDARD_FROM_LOOP_ERR_HANDLER (1);			      \
@@ -13182,7 +13182,7 @@ static const char __gbk_from_ucs4_tab12[][2] =
 									      \
 	  ch = __gbk_to_ucs[idx];					      \
 									      \
-	  if (__builtin_expect (ch, 1) == 0 && *inptr != '\0')		      \
+	  if (__glibc_unlikely (ch == 0) && *inptr != '\0')		      \
 	    {								      \
 	      /* This is an illegal character.  */			      \
 	      STANDARD_FROM_LOOP_ERR_HANDLER (2);			      \
@@ -13456,13 +13456,13 @@ static const char __gbk_from_ucs4_tab12[][2] =
 	  cp = "";							      \
 	  break; 							      \
 	}								      \
-      if (__builtin_expect (cp[0], '\1') == '\0' && ch != 0)		      \
+      if (__glibc_unlikely (cp[0] == '\0') && ch != 0)			      \
 	{								      \
 	  /* Illegal character.  */					      \
 	  STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
 	}								      \
       /* See whether there is enough room for the second byte we write.  */   \
-      else if (cp[1] != '\0' && __builtin_expect (outptr + 1 >= outend, 0))   \
+      else if (cp[1] != '\0' && __glibc_unlikely (outptr + 1 >= outend))      \
 	{								      \
 	  /* We have not enough room.  */				      \
 	  result = __GCONV_FULL_OUTPUT;					      \
diff --git a/iconvdata/ibm1364.c b/iconvdata/ibm1364.c
index cf80993..f6dcf50 100644
--- a/iconvdata/ibm1364.c
+++ b/iconvdata/ibm1364.c
@@ -155,7 +155,7 @@ enum
   {									      \
     uint32_t ch = *inptr;						      \
 									      \
-    if (__builtin_expect (ch, 0) == SO)					      \
+    if (__glibc_unlikely (ch == SO))					      \
       {									      \
 	/* Shift OUT, change to DBCS converter.  */			      \
 	if (curcs == db)						      \
@@ -167,7 +167,7 @@ enum
 	++inptr;							      \
 	continue;							      \
       }									      \
-    if (__builtin_expect (ch, 0) == SI)					      \
+    if (__glibc_unlikely (ch == SI))					      \
       {									      \
 	/* Shift IN, change to SBCS converter.  */			      \
 	if (curcs == sb)						      \
@@ -184,7 +184,7 @@ enum
       {									      \
 	/* Use the IBM13XX table for single byte.  */			      \
 	uint32_t res = SB_TO_UCS4[ch];				      \
-	if (__builtin_expect (res, L'\1') == L'\0' && ch != '\0')	      \
+	if (__glibc_unlikely (res == L'\0') && ch != '\0')		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    if (! ignore_errors_p ())					      \
@@ -221,10 +221,10 @@ enum
 	  ++rp2;							      \
 									      \
 	uint32_t res;							      \
-	if (__builtin_expect (rp2->start == 0xffff, 0)			      \
-	    || __builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (rp2->start == 0xffff)			      \
+	    || __glibc_unlikely (ch < rp2->start)			      \
 	    || (res = DB_TO_UCS4[ch + rp2->idx],			      \
-		__builtin_expect (res, L'\1') == L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (res == L'\0') && ch != '\0'))		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    if (! ignore_errors_p ())					      \
@@ -325,18 +325,18 @@ enum
 									      \
     /* Use the UCS4 table for single byte.  */				      \
     const char *cp;							      \
-    if (__builtin_expect (ch < rp1->start, 0)				      \
+    if (__glibc_unlikely (ch < rp1->start)				      \
 	|| (cp = UCS4_TO_SB[ch + rp1->idx],				      \
-	    __builtin_expect (cp[0], L'\1') == L'\0' && ch != '\0'))	      \
+	    __glibc_unlikely (cp[0] == L'\0') && ch != '\0'))		      \
       {									      \
 	/* Use the UCS4 table for double byte.  */			      \
 	const struct gap *rp2 = UCS4_TO_DB_IDX;				      \
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (ch < rp2->start)				      \
 	    || (cp = UCS4_TO_DB[ch + rp2->idx],				      \
-		__builtin_expect (cp[0], L'\1') == L'\0' && ch != '\0'))      \
+		__glibc_unlikely (cp[0] == L'\0') && ch != '\0'))	      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    if (! ignore_errors_p ())					      \
diff --git a/iconvdata/ibm930.c b/iconvdata/ibm930.c
index 768a444..c1fa824 100644
--- a/iconvdata/ibm930.c
+++ b/iconvdata/ibm930.c
@@ -103,7 +103,7 @@ enum
     uint32_t ch = *inptr;						      \
     uint32_t res;							      \
 									      \
-    if (__builtin_expect (ch, 0) == SO)					      \
+    if (__glibc_unlikely (ch == SO))					      \
       {									      \
 	/* Shift OUT, change to DBCS converter.  */			      \
 	if (curcs == db)						      \
@@ -115,7 +115,7 @@ enum
 	++inptr;							      \
 	continue;							      \
       }									      \
-    else if (__builtin_expect (ch, 0) == SI)				      \
+    else if (__glibc_unlikely (ch == SI))				      \
       {									      \
 	/* Shift IN, change to SBCS converter */			      \
 	if (curcs == sb)						      \
@@ -132,7 +132,7 @@ enum
       {									      \
 	/* Use the IBM930 table for single byte.  */			      \
 	res = __ibm930sb_to_ucs4[ch];					      \
-	if (__builtin_expect (res, L'\1') == L'\0' && ch != '\0')	      \
+	if (__glibc_unlikely (res == L'\0') && ch != '\0')		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -163,10 +163,10 @@ enum
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (rp2->start == 0xffff, 0)			      \
-	    || __builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (rp2->start == 0xffff)			      \
+	    || __glibc_unlikely (ch < rp2->start)			      \
 	    || (res = __ibm930db_to_ucs4[ch + rp2->idx],		      \
-		__builtin_expect (res, L'\1') == L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (res == L'\0') && ch != '\0'))		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (2);				      \
@@ -209,17 +209,17 @@ enum
       ++rp1;								      \
 									      \
     /* Use the UCS4 table for single byte.  */				      \
-    if (__builtin_expect (ch < rp1->start, 0)				      \
+    if (__glibc_unlikely (ch < rp1->start)				      \
 	|| (cp = __ucs4_to_ibm930sb[ch + rp1->idx],			      \
-	    __builtin_expect (cp[0], L'\1') == L'\0' && ch != '\0'))	      \
+	    __glibc_unlikely (cp[0] == L'\0') && ch != '\0'))		      \
       {									      \
 	/* Use the UCS4 table for double byte. */			      \
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (ch < rp2->start)				      \
 	    || (cp = __ucs4_to_ibm930db[ch + rp2->idx],			      \
-		__builtin_expect (cp[0], L'\1')== L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (cp[0] == L'\0') && ch != '\0'))	      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/ibm932.c b/iconvdata/ibm932.c
index aa69d65..9a1288f 100644
--- a/iconvdata/ibm932.c
+++ b/iconvdata/ibm932.c
@@ -50,11 +50,11 @@
     uint32_t ch = *inptr;						      \
     uint32_t res;							      \
 									      \
-    if (__builtin_expect (ch == 0x80, 0)				      \
-	|| __builtin_expect (ch == 0xa0, 0)				      \
-	|| __builtin_expect (ch == 0xfd, 0)				      \
-	|| __builtin_expect (ch == 0xfe, 0)				      \
-	|| __builtin_expect (ch == 0xff, 0))				      \
+    if (__glibc_unlikely (ch == 0x80)					      \
+	|| __glibc_unlikely (ch == 0xa0)				      \
+	|| __glibc_unlikely (ch == 0xfd)				      \
+	|| __glibc_unlikely (ch == 0xfe)				      \
+	|| __glibc_unlikely (ch == 0xff))				      \
       {									      \
 	/* This is an illegal character.  */				      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -62,7 +62,7 @@
 									      \
     /* Use the IBM932 table for single byte.  */			      \
     res = __ibm932sb_to_ucs4[ch];					      \
-    if (__builtin_expect (res == 0, 0) && ch != 0)			      \
+    if (__glibc_unlikely (res == 0) && ch != 0)				      \
       {									      \
 	/* Use the IBM932 table for double byte.  */			      \
 	if (__glibc_unlikely (inptr + 1 >= inend))			      \
@@ -79,9 +79,9 @@
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (ch < rp2->start)				      \
 	    || (res = __ibm932db_to_ucs4[ch + rp2->idx],		      \
-	    __builtin_expect (res, '\1') == 0 && ch !=0))		      \
+	    __glibc_unlikely (res == 0) && ch !=0))			      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (2);				      \
@@ -146,10 +146,10 @@
 	++rp;								      \
 									      \
     /* Use the UCS4 table for single byte.  */				      \
-    if (__builtin_expect (rp == NULL, 0)				      \
-	|| __builtin_expect (ch < rp->start, 0)				      \
+    if (__glibc_unlikely (rp == NULL)					      \
+	|| __glibc_unlikely (ch < rp->start)				      \
 	|| (sc = __ucs4_to_ibm932sb[ch + rp->idx],			      \
-	__builtin_expect (sc, '\1') == '\0' && ch != L'\0'))		      \
+	__glibc_unlikely (sc == '\0') && ch != L'\0'))			      \
       {									      \
 									      \
 	/* Use the UCS4 table for double byte.  */			      \
diff --git a/iconvdata/ibm933.c b/iconvdata/ibm933.c
index 461fb5e..acfafad 100644
--- a/iconvdata/ibm933.c
+++ b/iconvdata/ibm933.c
@@ -102,7 +102,7 @@ enum
     uint32_t ch = *inptr;						      \
     uint32_t res;							      \
 									      \
-    if (__builtin_expect (ch, 0) == SO)					      \
+    if (__glibc_unlikely (ch == SO))					      \
       {									      \
 	/* Shift OUT, change to DBCS converter.  */			      \
 	if (curcs == db)						      \
@@ -114,7 +114,7 @@ enum
 	++inptr;							      \
 	continue;							      \
       }									      \
-    else if (__builtin_expect (ch, 0) == SI)				      \
+    else if (__glibc_unlikely (ch == SI))				      \
       {									      \
 	/* Shift IN, change to SBCS converter.  */			      \
 	if (curcs == sb)						      \
@@ -131,7 +131,7 @@ enum
       {									      \
 	/* Use the IBM933 table for single byte.  */			      \
 	res = __ibm933sb_to_ucs4[ch];					      \
-	if (__builtin_expect (res, L'\1') == L'\0' && ch != '\0')	      \
+	if (__glibc_unlikely (res == L'\0') && ch != '\0')		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -162,10 +162,10 @@ enum
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (rp2->start == 0xffff, 0)			      \
-	    || __builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (rp2->start == 0xffff)			      \
+	    || __glibc_unlikely (ch < rp2->start)			      \
 	    || (res = __ibm933db_to_ucs4[ch + rp2->idx],		      \
-		__builtin_expect (res, L'\1') == L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (res == L'\0') && ch != '\0'))		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (2);				      \
@@ -208,17 +208,17 @@ enum
       ++rp1;								      \
 									      \
     /* Use the UCS4 table for single byte.  */				      \
-    if (__builtin_expect (ch < rp1->start, 0)				      \
+    if (__glibc_unlikely (ch < rp1->start)				      \
 	|| (cp = __ucs4_to_ibm933sb[ch + rp1->idx],			      \
-	    __builtin_expect (cp[0], L'\1') == L'\0' && ch != '\0'))	      \
+	    __glibc_unlikely (cp[0] == L'\0') && ch != '\0'))		      \
       {									      \
 	/* Use the UCS4 table for double byte.  */			      \
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (ch < rp2->start)				      \
 	    || (cp = __ucs4_to_ibm933db[ch + rp2->idx],			      \
-		__builtin_expect (cp[0], L'\1')==L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (cp[0] == L'\0') && ch != '\0'))	      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/ibm935.c b/iconvdata/ibm935.c
index 132d816..4c3dddf 100644
--- a/iconvdata/ibm935.c
+++ b/iconvdata/ibm935.c
@@ -102,7 +102,7 @@ enum
     uint32_t ch = *inptr;						      \
     uint32_t res;							      \
 									      \
-    if (__builtin_expect(ch, 0) == SO)					      \
+    if (__glibc_unlikely (ch == SO))					      \
       {									      \
 	/* Shift OUT, change to DBCS converter.  */			      \
 	if (curcs == db)						      \
@@ -114,7 +114,7 @@ enum
 	++inptr;							      \
 	continue;							      \
       }									      \
-    else if (__builtin_expect (ch, 0) == SI)				      \
+    else if (__glibc_unlikely (ch == SI))				      \
       {									      \
 	/* Shift IN, change to SBCS converter.  */			      \
 	if (curcs == sb)						      \
@@ -131,7 +131,7 @@ enum
       {									      \
 	/* Use the IBM935 table for single byte.  */			      \
 	res = __ibm935sb_to_ucs4[ch];					      \
-	if (__builtin_expect (res, L'\1') == L'\0' && ch != '\0')	      \
+	if (__glibc_unlikely (res == L'\0') && ch != '\0')		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -162,10 +162,10 @@ enum
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (rp2->start == 0xffff, 0)			      \
-	    || __builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (rp2->start == 0xffff)			      \
+	    || __glibc_unlikely (ch < rp2->start)			      \
 	    || (res = __ibm935db_to_ucs4[ch + rp2->idx],		      \
-		__builtin_expect (res, L'\1') == L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (res == L'\0') && ch != '\0'))		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (2);				      \
@@ -208,17 +208,17 @@ enum
       ++rp1;								      \
 									      \
     /* Use the UCS4 table for single byte.  */				      \
-    if (__builtin_expect (ch < rp1->start, 0)				      \
+    if (__glibc_unlikely (ch < rp1->start)				      \
 	|| (cp = __ucs4_to_ibm935sb[ch + rp1->idx],			      \
-	    __builtin_expect (cp[0], L'\1') == L'\0' && ch != '\0'))	      \
+	    __glibc_unlikely (cp[0] == L'\0') && ch != '\0'))		      \
       {									      \
 	/* Use the UCS4 table for double byte. */			      \
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (ch < rp2->start)				      \
 	    || (cp = __ucs4_to_ibm935db[ch + rp2->idx],			      \
-		__builtin_expect (cp[0], L'\1')==L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (cp[0] == L'\0') && ch != '\0'))	      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/ibm937.c b/iconvdata/ibm937.c
index 69b154d..3f69b43 100644
--- a/iconvdata/ibm937.c
+++ b/iconvdata/ibm937.c
@@ -102,7 +102,7 @@ enum
     uint32_t ch = *inptr;						      \
     uint32_t res;							      \
 									      \
-    if (__builtin_expect (ch, 0) == SO)					      \
+    if (__glibc_unlikely (ch == SO))					      \
       {									      \
 	/* Shift OUT, change to DBCS converter.  */			      \
 	if (curcs == db)						      \
@@ -114,7 +114,7 @@ enum
 	++inptr;							      \
 	continue;							      \
       }									      \
-    else if (__builtin_expect (ch, 0) == SI)				      \
+    else if (__glibc_unlikely (ch == SI))				      \
       {									      \
 	/* Shift IN, change to SBCS converter.  */			      \
 	if (curcs == sb)						      \
@@ -131,7 +131,7 @@ enum
       {									      \
 	/* Use the IBM937 table for single byte.  */			      \
 	res = __ibm937sb_to_ucs4[ch];					      \
-	if (__builtin_expect (res, L'\1') == L'\0' && ch != '\0')	      \
+	if (__glibc_unlikely (res == L'\0') && ch != '\0')		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -162,10 +162,10 @@ enum
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (rp2->start == 0xffff, 0)			      \
-	    || __builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (rp2->start == 0xffff)			      \
+	    || __glibc_unlikely (ch < rp2->start)			      \
 	    || (res = __ibm937db_to_ucs4[ch + rp2->idx],		      \
-		__builtin_expect (res, L'\1') == L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (res == L'\0') && ch != '\0'))		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (2);				      \
@@ -208,17 +208,17 @@ enum
       ++rp1;								      \
 									      \
     /* Use the UCS4 table for single byte.  */				      \
-    if (__builtin_expect (ch < rp1->start, 0)				      \
+    if (__glibc_unlikely (ch < rp1->start)				      \
 	|| (cp = __ucs4_to_ibm937sb[ch + rp1->idx],			      \
-	    __builtin_expect (cp[0], L'\1') == L'\0' && ch != '\0'))	      \
+	    __glibc_unlikely (cp[0] == L'\0') && ch != '\0'))		      \
       {									      \
 	/* Use the UCS4 table for double byte. */			      \
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (ch < rp2->start)				      \
 	    || (cp = __ucs4_to_ibm937db[ch + rp2->idx],			      \
-		__builtin_expect (cp[0], L'\1')==L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (cp[0] == L'\0') && ch != '\0'))	      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/ibm939.c b/iconvdata/ibm939.c
index 9936e2c..f2184fc 100644
--- a/iconvdata/ibm939.c
+++ b/iconvdata/ibm939.c
@@ -102,7 +102,7 @@ enum
     uint32_t ch = *inptr;						      \
     uint32_t res;							      \
 									      \
-    if (__builtin_expect (ch, 0) == SO)					      \
+    if (__glibc_unlikely (ch == SO))					      \
       {									      \
 	/* Shift OUT, change to DBCS converter.  */			      \
 	if (curcs == db)						      \
@@ -114,7 +114,7 @@ enum
 	++inptr;							      \
 	continue;							      \
       }									      \
-    else if (__builtin_expect (ch, 0) == SI)				      \
+    else if (__glibc_unlikely (ch == SI))				      \
       {									      \
 	/* Shift IN, change to SBCS converter.  */			      \
 	if (curcs == sb)						      \
@@ -131,7 +131,7 @@ enum
       {									      \
 	/* Use the IBM939 table for single byte.  */			      \
 	res = __ibm939sb_to_ucs4[ch];					      \
-	if (__builtin_expect (res == L'\0', 0) && ch != '\0')		      \
+	if (__glibc_unlikely (res == L'\0') && ch != '\0')		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -162,10 +162,10 @@ enum
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (rp2->start == 0xffff, 0)			      \
-	    || __builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (rp2->start == 0xffff)			      \
+	    || __glibc_unlikely (ch < rp2->start)			      \
 	    || (res = __ibm939db_to_ucs4[ch + rp2->idx],		      \
-		__builtin_expect (res, L'\1') == L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (res == L'\0') && ch != '\0'))		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (2);				      \
@@ -207,17 +207,17 @@ enum
       ++rp1;								      \
 									      \
     /* Use the UCS4 table for single byte.  */				      \
-    if (__builtin_expect (ch < rp1->start, 0)				      \
+    if (__glibc_unlikely (ch < rp1->start)				      \
 	|| (cp = __ucs4_to_ibm939sb[ch + rp1->idx],			      \
-	    __builtin_expect (cp[0], L'\1') == L'\0' && ch != '\0'))	      \
+	    __glibc_unlikely (cp[0] == L'\0') && ch != '\0'))		      \
       {									      \
 	/* Use the UCS4 table for double byte.  */			      \
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (ch < rp2->start)				      \
 	    || (cp = __ucs4_to_ibm939db[ch + rp2->idx],			      \
-		__builtin_expect (cp[0], L'\1')==L'\0' && ch != '\0'))	      \
+		__glibc_unlikely (cp[0] == L'\0') && ch != '\0'))	      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	  ibm939_invalid_char:						      \
diff --git a/iconvdata/ibm943.c b/iconvdata/ibm943.c
index c5d5742..fc12db7 100644
--- a/iconvdata/ibm943.c
+++ b/iconvdata/ibm943.c
@@ -50,20 +50,20 @@
     uint32_t ch = *inptr;						      \
     uint32_t res;							      \
 									      \
-    if (__builtin_expect (ch == 0x80, 0)				      \
-	|| __builtin_expect (ch == 0xa0, 0)				      \
-	|| __builtin_expect (ch == 0xfd, 0)				      \
-	|| __builtin_expect (ch == 0xfe, 0)				      \
-	|| __builtin_expect (ch == 0xff, 0))				      \
+    if (__glibc_unlikely (ch == 0x80)					      \
+	|| __glibc_unlikely (ch == 0xa0)				      \
+	|| __glibc_unlikely (ch == 0xfd)				      \
+	|| __glibc_unlikely (ch == 0xfe)				      \
+	|| __glibc_unlikely (ch == 0xff))				      \
       {									      \
 	/* This is an illegal character.  */				      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
       }									      \
 									      \
     /* Use the IBM943 table for single byte.  */			      \
-    if (__builtin_expect (ch > 0xdf, 0)					      \
+    if (__glibc_unlikely (ch > 0xdf)					      \
 	|| (res = __ibm943sb_to_ucs4[ch],				      \
-	    __builtin_expect (res == 0, 0) && ch != 0))			      \
+	    __glibc_unlikely (res == 0) && ch != 0))			      \
       {									      \
 	/* Use the IBM943 table for double byte.  */			      \
 	if (__glibc_unlikely (inptr + 1 >= inend))			      \
@@ -80,9 +80,9 @@
 	while (ch > rp2->end)						      \
 	  ++rp2;							      \
 									      \
-	if (__builtin_expect (ch < rp2->start, 0)			      \
+	if (__glibc_unlikely (ch < rp2->start)				      \
 	    || (res = __ibm943db_to_ucs4[ch + rp2->idx],		      \
-	    __builtin_expect (res, '\1') == 0 && ch !=0))		      \
+	    __glibc_unlikely (res == 0) && ch !=0))			      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (2);				      \
@@ -147,10 +147,10 @@
 	++rp;								      \
 									      \
     /* Use the UCS4 table for single byte.  */				      \
-    if (__builtin_expect (rp == NULL, 0)				      \
-	|| __builtin_expect (ch < rp->start, 0)				      \
+    if (__glibc_unlikely (rp == NULL)					      \
+	|| __glibc_unlikely (ch < rp->start)				      \
 	|| (sc = __ucs4_to_ibm943sb[ch + rp->idx],			      \
-	__builtin_expect (sc, '\1') == '\0' && ch != L'\0'))		      \
+	__glibc_unlikely (sc == '\0') && ch != L'\0'))			      \
       {									      \
 									      \
 	/* Use the UCS4 table for double byte.  */			      \
diff --git a/iconvdata/iso-2022-cn.c b/iconvdata/iso-2022-cn.c
index 5434ef4..e65e90a 100644
--- a/iconvdata/iso-2022-cn.c
+++ b/iconvdata/iso-2022-cn.c
@@ -126,22 +126,22 @@ enum
       STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
 									      \
     /* Recognize escape sequences.  */					      \
-    if (__builtin_expect (ch, 0) == ESC)				      \
+    if (__glibc_unlikely (ch == ESC))					      \
       {									      \
 	/* There are two kinds of escape sequences we have to handle:	      \
 	   - those announcing the use of GB and CNS characters on the	      \
 	     line; we can simply ignore them				      \
 	   - the initial byte of the SS2 sequence.			      \
 	*/								      \
-	if (__builtin_expect (inptr + 2 > inend, 0)			      \
+	if (__glibc_unlikely (inptr + 2 > inend)			      \
 	    || (inptr[1] == '$'						      \
-		&& (__builtin_expect (inptr + 3 > inend, 0)		      \
+		&& (__glibc_unlikely (inptr + 3 > inend)		      \
 		    || (inptr[2] == ')'					      \
-			&& __builtin_expect (inptr + 4 > inend, 0))	      \
+			&& __glibc_unlikely (inptr + 4 > inend))	      \
 		    || (inptr[2] == '*'					      \
-			&& __builtin_expect (inptr + 4 > inend, 0))))	      \
+			&& __glibc_unlikely (inptr + 4 > inend))))	      \
 	    || (inptr[1] == SS2_1					      \
-		&& __builtin_expect (inptr + 4 > inend, 0)))		      \
+		&& __glibc_unlikely (inptr + 4 > inend)))		      \
 	  {								      \
 	    result = __GCONV_INCOMPLETE_INPUT;				      \
 	    break;							      \
@@ -159,7 +159,7 @@ enum
 	    continue;							      \
 	  }								      \
       }									      \
-    else if (__builtin_expect (ch, 0) == SO)				      \
+    else if (__glibc_unlikely (ch == SO))				      \
       {									      \
 	/* Switch to use GB2312 or CNS 11643 plane 1, depending on which      \
 	   S0 designation came last.  The only problem is what to do with     \
@@ -170,7 +170,7 @@ enum
 	set = ann == CNS11643_1_ann ? CNS11643_1_set : GB2312_set;	      \
 	continue;							      \
       }									      \
-    else if (__builtin_expect (ch, 0) == SI)				      \
+    else if (__glibc_unlikely (ch == SI))				      \
       {									      \
 	/* Switch to use ASCII.  */					      \
 	++inptr;							      \
@@ -178,14 +178,14 @@ enum
 	continue;							      \
       }									      \
 									      \
-    if (__builtin_expect (ch, 0) == ESC && inptr[1] == SS2_1)		      \
+    if (__glibc_unlikely (ch == ESC) && inptr[1] == SS2_1)		      \
       {									      \
 	/* This is a character from CNS 11643 plane 2.			      \
 	   XXX We could test here whether the use of this character	      \
 	   set was announced.  */					      \
 	inptr += 2;							      \
 	ch = cns11643l2_to_ucs4 (&inptr, 2, 0);				      \
-	if (__builtin_expect (ch, 0) == __UNKNOWN_10646_CHAR)		      \
+	if (__glibc_unlikely (ch == __UNKNOWN_10646_CHAR))		      \
 	  {								      \
 	    inptr -= 2;							      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (2);				      \
@@ -207,12 +207,12 @@ enum
 	    ch = cns11643l1_to_ucs4 (&inptr, inend - inptr, 0);		      \
 	  }								      \
 									      \
-	if (__builtin_expect (ch, 1) == 0)				      \
+	if (__glibc_unlikely (ch == 0))					      \
 	  {								      \
 	    result = __GCONV_INCOMPLETE_INPUT;				      \
 	    break;							      \
 	  }								      \
-	else if (__builtin_expect (ch, 1) == __UNKNOWN_10646_CHAR)	      \
+	else if (__glibc_unlikely (ch == __UNKNOWN_10646_CHAR))		      \
 	  {								      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
 	  }								      \
@@ -297,7 +297,7 @@ enum
 		else							      \
 		  written = ucs4_to_gb2312 (ch, buf, 2);		      \
 									      \
-		if (__builtin_expect (written, 0) != __UNKNOWN_10646_CHAR)    \
+		if (__glibc_likely (written != __UNKNOWN_10646_CHAR))	      \
 		  /* Oh well, then switch SO.  */			      \
 		  used = GB2312_set + CNS11643_1_set - used;		      \
 		else							      \
diff --git a/iconvdata/iso-2022-jp-3.c b/iconvdata/iso-2022-jp-3.c
index 4b4ea01..b06330a 100644
--- a/iconvdata/iso-2022-jp-3.c
+++ b/iconvdata/iso-2022-jp-3.c
@@ -161,9 +161,9 @@ enum
 	   ends we terminate with an error since we must not risk missing     \
 	   an escape sequence just because it is not entirely in the	      \
 	   current input buffer.  */					      \
-	if (__builtin_expect (inptr + 2 >= inend, 0)			      \
+	if (__glibc_unlikely (inptr + 2 >= inend)			      \
 	    || (inptr[1] == '$' && inptr[2] == '('			      \
-		&& __builtin_expect (inptr + 3 >= inend, 0)))		      \
+		&& __glibc_unlikely (inptr + 3 >= inend)))		      \
 	  {								      \
 	    /* Not enough input available.  */				      \
 	    result = __GCONV_INCOMPLETE_INPUT;				      \
@@ -722,7 +722,7 @@ static const struct
 		      {							      \
 			if (set != JISX0201_Kana_set)			      \
 			  {						      \
-			    if (__builtin_expect (outptr + 3 > outend, 0))    \
+			    if (__glibc_unlikely (outptr + 3 > outend))	      \
 			      {						      \
 				result = __GCONV_FULL_OUTPUT;		      \
 				break;					      \
diff --git a/iconvdata/iso-2022-jp.c b/iconvdata/iso-2022-jp.c
index 8381c30..e89bb4f 100644
--- a/iconvdata/iso-2022-jp.c
+++ b/iconvdata/iso-2022-jp.c
@@ -164,7 +164,7 @@ gconv_init (struct __gconv_step *step)
     }
 
   result = __GCONV_NOCONV;
-  if (__builtin_expect (dir, from_iso2022jp) != illegal_dir)
+  if (__glibc_likely (dir != illegal_dir))
     {
       new_data
 	= (struct iso2022jp_data *) malloc (sizeof (struct iso2022jp_data));
@@ -269,16 +269,16 @@ gconv_end (struct __gconv_step *data)
     uint32_t ch = *inptr;						      \
 									      \
     /* Recognize escape sequences.  */					      \
-    if (__builtin_expect (ch, 0) == ESC)				      \
+    if (__glibc_unlikely (ch == ESC))					      \
       {									      \
 	/* We now must be prepared to read two to three more		      \
 	   characters.  If we have a match in the first character but	      \
 	   then the input buffer ends we terminate with an error since	      \
 	   we must not risk missing an escape sequence just because it	      \
 	   is not entirely in the current input buffer.  */		      \
-	if (__builtin_expect (inptr + 2 >= inend, 0)			      \
+	if (__glibc_unlikely (inptr + 2 >= inend)			      \
 	    || (var == iso2022jp2 && inptr[1] == '$' && inptr[2] == '('	      \
-		&& __builtin_expect (inptr + 3 >= inend, 0)))		      \
+		&& __glibc_unlikely (inptr + 3 >= inend)))		      \
 	  {								      \
 	    /* Not enough input available.  */				      \
 	    result = __GCONV_INCOMPLETE_INPUT;				      \
@@ -379,7 +379,7 @@ gconv_end (struct __gconv_step *data)
 	    ch = inptr[2] | 0x80;					      \
 	    inptr += 3;							      \
 	  }								      \
-	else if (__builtin_expect (set2, ISO88597_set) == ISO88597_set)	      \
+	else if (__glibc_likely (set2 == ISO88597_set))			      \
 	  {								      \
 	    /* We use the table from the ISO 8859-7 module.  */		      \
 	    if (inptr[2] < 0x20 || inptr[2] >= 0x80)			      \
@@ -560,7 +560,7 @@ static const cvlist_t conversion_lists[4] =
        have to be known whether the last line ended using ASCII or	      \
        JIS X 0201.  */							      \
     else if (set == JISX0201_Roman_set					      \
-	     && (__builtin_expect (tag == TAG_none, 1)			      \
+	     && (__glibc_likely (tag == TAG_none)			      \
 		 || tag == TAG_language_ja))				      \
       {									      \
 	unsigned char buf[1];						      \
@@ -577,7 +577,7 @@ static const cvlist_t conversion_lists[4] =
 	  }								      \
       }									      \
     else if (set == JISX0201_Kana_set					      \
-	     && (__builtin_expect (tag == TAG_none, 1)			      \
+	     && (__glibc_likely (tag == TAG_none)			      \
 		 || tag == TAG_language_ja))				      \
       {									      \
 	unsigned char buf[1];						      \
@@ -596,19 +596,19 @@ static const cvlist_t conversion_lists[4] =
     else								      \
       {									      \
 	if ((set == JISX0208_1978_set || set == JISX0208_1983_set)	      \
-	    && (__builtin_expect (tag == TAG_none, 1)			      \
+	    && (__glibc_likely (tag == TAG_none)			      \
 		|| tag == TAG_language_ja))				      \
 	  written = ucs4_to_jisx0208 (ch, outptr, outend - outptr);	      \
 	else if (set == JISX0212_set					      \
-		 && (__builtin_expect (tag == TAG_none, 1)		      \
+		 && (__glibc_likely (tag == TAG_none)			      \
 		     || tag == TAG_language_ja))			      \
 	  written = ucs4_to_jisx0212 (ch, outptr, outend - outptr);	      \
 	else if (set == GB2312_set					      \
-		 && (__builtin_expect (tag == TAG_none, 1)		      \
+		 && (__glibc_likely (tag == TAG_none)			      \
 		     || tag == TAG_language_zh))			      \
 	  written = ucs4_to_gb2312 (ch, outptr, outend - outptr);	      \
 	else if (set == KSC5601_set					      \
-		 && (__builtin_expect (tag == TAG_none, 1)		      \
+		 && (__glibc_likely (tag == TAG_none)			      \
 		     || tag == TAG_language_ko))			      \
 	  written = ucs4_to_ksc5601 (ch, outptr, outend - outptr);	      \
 	else								      \
@@ -624,7 +624,7 @@ static const cvlist_t conversion_lists[4] =
       }									      \
 									      \
     if (written == __UNKNOWN_10646_CHAR					      \
-	&& __builtin_expect (tag == TAG_none, 1))			      \
+	&& __glibc_likely (tag == TAG_none))				      \
       {									      \
 	if (set2 == ISO88591_set)					      \
 	  {								      \
@@ -732,7 +732,7 @@ static const cvlist_t conversion_lists[4] =
 		    {							      \
 		      if (set2 != ISO88591_set)				      \
 			{						      \
-			  if (__builtin_expect (outptr + 3 > outend, 0))      \
+			  if (__glibc_unlikely (outptr + 3 > outend))	      \
 			    {						      \
 			      res = __GCONV_FULL_OUTPUT;		      \
 			      break;					      \
@@ -770,8 +770,7 @@ static const cvlist_t conversion_lists[4] =
 			    {						      \
 			      if (set2 != ISO88597_set)			      \
 				{					      \
-				  if (__builtin_expect (outptr + 3 > outend,  \
-							0))		      \
+				  if (__glibc_unlikely (outptr + 3 > outend))		      \
 				    {					      \
 				      res = __GCONV_FULL_OUTPUT;	      \
 				      break;				      \
@@ -782,7 +781,7 @@ static const cvlist_t conversion_lists[4] =
 				  set2 = ISO88597_set;			      \
 				}					      \
 									      \
-			      if (__builtin_expect (outptr + 3 > outend, 0))  \
+			      if (__glibc_unlikely (outptr + 3 > outend))     \
 				{					      \
 				  res = __GCONV_FULL_OUTPUT;		      \
 				  break;				      \
@@ -807,7 +806,7 @@ static const cvlist_t conversion_lists[4] =
 		    {							      \
 		      if (set != JISX0201_Roman_set)			      \
 			{						      \
-			  if (__builtin_expect (outptr + 3 > outend, 0))      \
+			  if (__glibc_unlikely (outptr + 3 > outend))	      \
 			    {						      \
 			      res = __GCONV_FULL_OUTPUT;		      \
 			      break;					      \
@@ -834,7 +833,7 @@ static const cvlist_t conversion_lists[4] =
 		    {							      \
 		      if (set != JISX0208_1983_set)			      \
 			{						      \
-			  if (__builtin_expect (outptr + 3 > outend, 0))      \
+			  if (__glibc_unlikely (outptr + 3 > outend))	      \
 			    {						      \
 			      res = __GCONV_FULL_OUTPUT;		      \
 			      break;					      \
@@ -866,7 +865,7 @@ static const cvlist_t conversion_lists[4] =
 		    {							      \
 		      if (set != JISX0212_set)				      \
 			{						      \
-			  if (__builtin_expect (outptr + 4 > outend, 0))      \
+			  if (__glibc_unlikely (outptr + 4 > outend))	      \
 			    {						      \
 			      res = __GCONV_FULL_OUTPUT;		      \
 			      break;					      \
@@ -900,7 +899,7 @@ static const cvlist_t conversion_lists[4] =
 		    {							      \
 		      if (set != GB2312_set)				      \
 			{						      \
-			  if (__builtin_expect (outptr + 3 > outend, 0))      \
+			  if (__glibc_unlikely (outptr + 3 > outend))	      \
 			    {						      \
 			      res = __GCONV_FULL_OUTPUT;		      \
 			      break;					      \
@@ -933,7 +932,7 @@ static const cvlist_t conversion_lists[4] =
 		    {							      \
 		      if (set != KSC5601_set)				      \
 			{						      \
-			  if (__builtin_expect (outptr + 4 > outend, 0))      \
+			  if (__glibc_unlikely (outptr + 4 > outend))	      \
 			    {						      \
 			      res = __GCONV_FULL_OUTPUT;		      \
 			      break;					      \
@@ -969,7 +968,7 @@ static const cvlist_t conversion_lists[4] =
 		    {							      \
 		      if (set != JISX0201_Kana_set)			      \
 			{						      \
-			  if (__builtin_expect (outptr + 3 > outend, 0))      \
+			  if (__glibc_unlikely (outptr + 3 > outend))	      \
 			    {						      \
 			      res = __GCONV_FULL_OUTPUT;		      \
 			      break;					      \
diff --git a/iconvdata/iso-2022-kr.c b/iconvdata/iso-2022-kr.c
index c1babc8..16a0abe 100644
--- a/iconvdata/iso-2022-kr.c
+++ b/iconvdata/iso-2022-kr.c
@@ -123,17 +123,17 @@ enum
       STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
 									      \
     /* Recognize escape sequences.  */					      \
-    if (__builtin_expect (ch, 0) == ESC)				      \
+    if (__glibc_unlikely (ch == ESC))					      \
       {									      \
 	/* We don't really have to handle escape sequences since all the      \
 	   switching is done using the SI and SO bytes.  But we have to	      \
 	   recognize `Esc $ ) C' since this is a kind of flag for this	      \
 	   encoding.  We simply ignore it.  */				      \
-	if (__builtin_expect (inptr + 2 > inend, 0)			      \
+	if (__glibc_unlikely (inptr + 2 > inend)			      \
 	    || (inptr[1] == '$'						      \
-		&& (__builtin_expect (inptr + 3 > inend, 0)		      \
+		&& (__glibc_unlikely (inptr + 3 > inend)		      \
 		    || (inptr[2] == ')'					      \
-			&& __builtin_expect (inptr + 4 > inend, 0)))))	      \
+			&& __glibc_unlikely (inptr + 4 > inend)))))	      \
 	  {								      \
 	    result = __GCONV_INCOMPLETE_INPUT;				      \
 	    break;							      \
@@ -145,14 +145,14 @@ enum
 	    continue;							      \
 	  }								      \
       }									      \
-    else if (__builtin_expect (ch, 0) == SO)				      \
+    else if (__glibc_unlikely (ch == SO))				      \
       {									      \
 	/* Switch to use KSC.  */					      \
 	++inptr;							      \
 	set = KSC5601_set;						      \
 	continue;							      \
       }									      \
-    else if (__builtin_expect (ch, 0) == SI)				      \
+    else if (__glibc_unlikely (ch == SI))				      \
       {									      \
 	/* Switch to use ASCII.  */					      \
 	++inptr;							      \
@@ -226,7 +226,7 @@ enum
 	asm ("" : "=m" (buf));						      \
 									      \
 	size_t written = ucs4_to_ksc5601 (ch, buf, 2);			      \
-	if (__builtin_expect (written, 0) == __UNKNOWN_10646_CHAR)	      \
+	if (__glibc_unlikely (written == __UNKNOWN_10646_CHAR))		      \
 	  {								      \
 	    UNICODE_TAG_HANDLER (ch, 4);				      \
 									      \
diff --git a/iconvdata/iso646.c b/iconvdata/iso646.c
index 5a0402e..b540e7f 100644
--- a/iconvdata/iso646.c
+++ b/iconvdata/iso646.c
@@ -146,7 +146,7 @@ gconv_init (struct __gconv_step *step)
       }
 
   result = __GCONV_NOCONV;
-  if (__builtin_expect (dir, from_iso646) != illegal_dir)
+  if (__glibc_likely (dir != illegal_dir))
     {
       new_data = (struct iso646_data *) malloc (sizeof (struct iso646_data));
 
@@ -406,7 +406,7 @@ gconv_end (struct __gconv_step *data)
 									      \
     /* Hopefully gcc can recognize that the following `if' is only true	      \
        when we reach the default case in the `switch' statement.  */	      \
-    if (__builtin_expect (failure, __GCONV_OK) == __GCONV_ILLEGAL_INPUT)      \
+    if (__glibc_unlikely (failure == __GCONV_ILLEGAL_INPUT))		      \
       {									      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
       }									      \
@@ -880,7 +880,7 @@ gconv_end (struct __gconv_step *data)
 	break;								      \
       }									      \
 									      \
-    if (__builtin_expect (failure, __GCONV_OK) == __GCONV_ILLEGAL_INPUT)      \
+    if (__glibc_unlikely (failure == __GCONV_ILLEGAL_INPUT))		      \
       {									      \
 	STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
       }									      \
diff --git a/iconvdata/iso_6937-2.c b/iconvdata/iso_6937-2.c
index 8aba77d..7c708c7 100644
--- a/iconvdata/iso_6937-2.c
+++ b/iconvdata/iso_6937-2.c
@@ -392,7 +392,7 @@ static const char from_ucs4[][2] =
   {									      \
     uint32_t ch = *inptr;						      \
 									      \
-    if (__builtin_expect (ch >= 0xc1, 0) && ch <= 0xcf)			      \
+    if (__glibc_unlikely (ch >= 0xc1) && ch <= 0xcf)			      \
       {									      \
 	/* Composed character.  First test whether the next byte	      \
 	   is also available.  */					      \
@@ -408,8 +408,8 @@ static const char from_ucs4[][2] =
 									      \
 	ch2 = inptr[1];							      \
 									      \
-	if (__builtin_expect (ch2 < 0x20, 0)				      \
-	    || __builtin_expect (ch2 >= 0x80, 0))			      \
+	if (__glibc_unlikely (ch2 < 0x20)				      \
+	    || __glibc_unlikely (ch2 >= 0x80))				      \
 	  {								      \
 	    /* This is illegal.  */					      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -429,7 +429,7 @@ static const char from_ucs4[][2] =
       {									      \
 	ch = to_ucs4[ch];						      \
 									      \
-	if (__builtin_expect (ch == 0, 0) && *inptr != '\0')		      \
+	if (__glibc_unlikely (ch == 0) && *inptr != '\0')		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -462,8 +462,7 @@ static const char from_ucs4[][2] =
     uint32_t ch = get32 (inptr);					      \
     const char *cp;							      \
 									      \
-    if (__builtin_expect (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0]),   \
-			  0))						      \
+    if (__glibc_unlikely (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0])))						      \
       {									      \
 	switch (ch)							      \
 	  {								      \
@@ -525,7 +524,7 @@ static const char from_ucs4[][2] =
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
 	  }								      \
       }									      \
-    else if (__builtin_expect (from_ucs4[ch][0] == '\0', 0) && ch != 0)	      \
+    else if (__glibc_unlikely (from_ucs4[ch][0] == '\0') && ch != 0)	      \
       {									      \
 	/* Illegal characters.  */					      \
 	STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/iso_6937.c b/iconvdata/iso_6937.c
index f7bbcf0..3331998 100644
--- a/iconvdata/iso_6937.c
+++ b/iconvdata/iso_6937.c
@@ -392,7 +392,7 @@ static const char from_ucs4[][2] =
   {									      \
     uint32_t ch = *inptr;						      \
 									      \
-    if (__builtin_expect (ch >= 0xc1, 0) && ch <= 0xcf)			      \
+    if (__glibc_unlikely (ch >= 0xc1) && ch <= 0xcf)			      \
       {									      \
 	/* Composed character.  First test whether the next byte	      \
 	   is also available.  */					      \
@@ -408,8 +408,8 @@ static const char from_ucs4[][2] =
 									      \
 	ch2 = inptr[1];							      \
 									      \
-	if (__builtin_expect (ch2 < 0x20, 0)				      \
-	    || __builtin_expect (ch2 >= 0x80, 0))			      \
+	if (__glibc_unlikely (ch2 < 0x20)				      \
+	    || __glibc_unlikely (ch2 >= 0x80))				      \
 	  {								      \
 	    /* This is illegal.  */					      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -429,7 +429,7 @@ static const char from_ucs4[][2] =
       {									      \
 	ch = to_ucs4[ch];						      \
 									      \
-	if (__builtin_expect (ch == 0, 0) && *inptr != '\0')		      \
+	if (__glibc_unlikely (ch == 0) && *inptr != '\0')		      \
 	  {								      \
 	    /* This is an illegal character.  */			      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -462,8 +462,7 @@ static const char from_ucs4[][2] =
     uint32_t ch = get32 (inptr);					      \
     const char *cp;							      \
 									      \
-    if (__builtin_expect (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0]),   \
-			  0))						      \
+    if (__glibc_unlikely (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0])))						      \
       {									      \
 	int fail = 0;							      \
 	switch (ch)							      \
@@ -527,7 +526,7 @@ static const char from_ucs4[][2] =
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
 	  }								      \
       }									      \
-    else if (__builtin_expect (from_ucs4[ch][0] == '\0', 0) && ch != 0)	      \
+    else if (__glibc_unlikely (from_ucs4[ch][0] == '\0') && ch != 0)	      \
       {									      \
 	/* Illegal characters.  */					      \
 	STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/johab.c b/iconvdata/johab.c
index 5b5f042..cdce1c4 100644
--- a/iconvdata/johab.c
+++ b/iconvdata/johab.c
@@ -178,10 +178,10 @@ johab_sym_hanja_to_ucs (uint_fast32_t idx, uint_fast32_t c1, uint_fast32_t c2)
        0xd831-0xd87e and 0xd891-0xd8fe are user-defined area */		      \
     else								      \
       {									      \
-	if (__builtin_expect (ch > 0xf9, 0)				      \
-	    || __builtin_expect (ch == 0xdf, 0)				      \
-	    || (__builtin_expect (ch > 0x7e, 0) && ch < 0x84)		      \
-	    || (__builtin_expect (ch > 0xd3, 0) && ch < 0xd9))		      \
+	if (__glibc_unlikely (ch > 0xf9)				      \
+	    || __glibc_unlikely (ch == 0xdf)				      \
+	    || (__glibc_unlikely (ch > 0x7e) && ch < 0x84)		      \
+	    || (__glibc_unlikely (ch > 0xd3) && ch < 0xd9))		      \
 	  {								      \
 	    /* These are illegal.  */					      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -212,9 +212,9 @@ johab_sym_hanja_to_ucs (uint_fast32_t idx, uint_fast32_t c1, uint_fast32_t c2)
 		m = mid[(idx & 0x03e0) >> 5];				      \
 		f = final[idx & 0x001f];				      \
 									      \
-		if (__builtin_expect (i == -1, 0)			      \
-		    || __builtin_expect (m == -1, 0)			      \
-		    || __builtin_expect (f == -1, 0))			      \
+		if (__glibc_unlikely (i == -1)				      \
+		    || __glibc_unlikely (m == -1)			      \
+		    || __glibc_unlikely (f == -1))			      \
 		  {							      \
 		    /* This is illegal.  */				      \
 		    STANDARD_FROM_LOOP_ERR_HANDLER (1);			      \
@@ -225,8 +225,8 @@ johab_sym_hanja_to_ucs (uint_fast32_t idx, uint_fast32_t c1, uint_fast32_t c2)
 		  ch = init_to_ucs[i - 1];				      \
 		else if (i == 0 && m > 0 && f == 0)			      \
 		  ch = 0x314e + m;	/* 0x314f + m - 1 */		      \
-		else if (__builtin_expect ((i | m) == 0, 1)		      \
-			 && __builtin_expect (f > 0, 1))		      \
+		else if (__glibc_likely ((i | m) == 0)			      \
+			 && __glibc_likely (f > 0))			      \
 		  ch = final_to_ucs[f - 1];	/* round trip?? */	      \
 		else							      \
 		  {							      \
@@ -236,13 +236,13 @@ johab_sym_hanja_to_ucs (uint_fast32_t idx, uint_fast32_t c1, uint_fast32_t c2)
 	      }								      \
 	    else							      \
 	      {								      \
-		if (__builtin_expect (ch2 < 0x31, 0)			      \
-		    || (__builtin_expect (ch2 > 0x7e, 0) && ch2 < 0x91)	      \
-		    || __builtin_expect (ch2, 0) == 0xff		      \
-		    || (__builtin_expect (ch, 0) == 0xd9 && ch2 > 0xe8)	      \
-		    || (__builtin_expect (ch, 0) == 0xda		      \
+		if (__glibc_unlikely (ch2 < 0x31)			      \
+		    || (__glibc_unlikely (ch2 > 0x7e) && ch2 < 0x91)	      \
+		    || __glibc_unlikely (ch2 == 0xff)			      \
+		    || (__glibc_unlikely (ch == 0xd9) && ch2 > 0xe8)	      \
+		    || (__glibc_unlikely (ch == 0xda)			      \
 			&& ch2 > 0xa0 && ch2 < 0xd4)			      \
-		    || (__builtin_expect (ch, 0) == 0xde && ch2 > 0xf1))      \
+		    || (__glibc_unlikely (ch == 0xde) && ch2 > 0xf1))	      \
 		  {							      \
 		    /* This is illegal.  */				      \
 		    STANDARD_FROM_LOOP_ERR_HANDLER (1);			      \
@@ -351,7 +351,7 @@ johab_sym_hanja_to_ucs (uint_fast32_t idx, uint_fast32_t c1, uint_fast32_t c2)
 	    uint32_t temp;						      \
 									      \
 	    written = ucs4_to_ksc5601_hanja (ch, outptr, outend - outptr);    \
-	    if (__builtin_expect (written, 1) == 0)			      \
+	    if (__glibc_unlikely (written == 0))			      \
 	      {								      \
 		result = __GCONV_FULL_OUTPUT;				      \
 		break;							      \
@@ -380,12 +380,12 @@ johab_sym_hanja_to_ucs (uint_fast32_t idx, uint_fast32_t c1, uint_fast32_t c2)
 	    uint32_t temp;						      \
 									      \
 	    written = ucs4_to_ksc5601_sym (ch, outptr, outend - outptr);      \
-	    if (__builtin_expect (written, 1) == 0)			      \
+	    if (__glibc_unlikely (written == 0))			      \
 	      {								      \
 		result = __GCONV_FULL_OUTPUT;				      \
 		break;							      \
 	      }								      \
-	    if (__builtin_expect (written == __UNKNOWN_10646_CHAR, 0)	      \
+	    if (__glibc_unlikely (written == __UNKNOWN_10646_CHAR)	      \
 		|| (outptr[0] == 0x22 && outptr[1] > 0x68))		      \
 	      {								      \
 		UNICODE_TAG_HANDLER (ch, 4);				      \
diff --git a/iconvdata/sjis.c b/iconvdata/sjis.c
index 2590b33..48fce89 100644
--- a/iconvdata/sjis.c
+++ b/iconvdata/sjis.c
@@ -4331,12 +4331,12 @@ static const char from_ucs4_extra[0x100][2] =
   {									      \
     uint32_t ch = *inptr;						      \
 									      \
-    if (__builtin_expect (ch, 0) == 0x5c)				      \
+    if (__glibc_unlikely (ch == 0x5c))					      \
       {									      \
 	ch = 0xa5;							      \
 	++inptr;							      \
       }									      \
-    else if (__builtin_expect (ch, 0) == 0x7e)				      \
+    else if (__glibc_unlikely (ch == 0x7e))				      \
       {									      \
 	ch = 0x203e;							      \
 	++inptr;							      \
@@ -4348,9 +4348,9 @@ static const char from_ucs4_extra[0x100][2] =
 	ch += 0xfec0;							      \
 	++inptr;							      \
       }									      \
-    else if (__builtin_expect (ch > 0xea, 0)				      \
-	     || __builtin_expect (ch, 0) == 0xa0			      \
-	     || __builtin_expect (ch <= 0x80, 0))			      \
+    else if (__glibc_unlikely (ch > 0xea)				      \
+	     || __glibc_unlikely (ch == 0xa0)				      \
+	     || __glibc_unlikely (ch <= 0x80))				      \
       {									      \
 	/* These are illegal.  */					      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -4377,10 +4377,10 @@ static const char from_ucs4_extra[0x100][2] =
 	    /* This is illegal.  */					      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
 	  }								      \
-	else if ((__builtin_expect (idx > 0x84be && idx < 0x889f, 0))	      \
-		 || (__builtin_expect (idx > 0x88fc && idx < 0x8940, 0))      \
-		 || (__builtin_expect (idx > 0x9ffc && idx < 0xe040, 0))      \
-		 || __builtin_expect (idx > 0xeaa4, 0))			      \
+	else if ((__glibc_unlikely (idx > 0x84be && idx < 0x889f))	      \
+		 || (__glibc_unlikely (idx > 0x88fc && idx < 0x8940))	      \
+		 || (__glibc_unlikely (idx > 0x9ffc && idx < 0xe040))	      \
+		 || __glibc_unlikely (idx > 0xeaa4))			      \
 	  {								      \
 	    /* This is illegal.  */					      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (2);				      \
@@ -4446,8 +4446,8 @@ static const char from_ucs4_extra[0x100][2] =
 	  cp = from_ucs4_greek[ch - 0x391];				      \
 	else if (ch >= 0x2010 && ch <= 0x9fa0)				      \
 	  cp = from_ucs4_cjk[ch - 0x02010];				      \
-	else if (__builtin_expect (ch >= 0xff01, 1)			      \
-		 && __builtin_expect (ch <= 0xffef, 1))			      \
+	else if (__glibc_likely (ch >= 0xff01)				      \
+		 && __glibc_likely (ch <= 0xffef))			      \
 	  cp = from_ucs4_extra[ch - 0xff00];				      \
 	else								      \
 	  {								      \
@@ -4459,7 +4459,7 @@ static const char from_ucs4_extra[0x100][2] =
     else								      \
       cp = from_ucs4_lat1[ch];						      \
 									      \
-    if (__builtin_expect (cp[0] == '\0', 0) && ch != 0)			      \
+    if (__glibc_unlikely (cp[0] == '\0') && ch != 0)			      \
       {									      \
 	/* Illegal character.  */					      \
 	STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/t.61.c b/iconvdata/t.61.c
index e082b50..2d83df6 100644
--- a/iconvdata/t.61.c
+++ b/iconvdata/t.61.c
@@ -385,7 +385,7 @@ static const char from_ucs4[][2] =
     uint32_t ch = *inptr;						      \
     int increment = 1;							      \
 									      \
-    if (__builtin_expect (ch >= 0xc1, 0) && ch <= 0xcf)			      \
+    if (__glibc_unlikely (ch >= 0xc1) && ch <= 0xcf)			      \
       {									      \
 	/* Composed character.  First test whether the next byte	      \
 	   is also available.  */					      \
@@ -400,8 +400,8 @@ static const char from_ucs4[][2] =
 									      \
 	ch2 = inptr[1];							      \
 									      \
-	if (__builtin_expect (ch2 < 0x20, 0)				      \
-	    || __builtin_expect (ch2 >= 0x80, 0))			      \
+	if (__glibc_unlikely (ch2 < 0x20)				      \
+	    || __glibc_unlikely (ch2 >= 0x80))				      \
 	  {								      \
 	    /* This is illegal.  */					      \
 	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -414,7 +414,7 @@ static const char from_ucs4[][2] =
     else								      \
       ch = to_ucs4[ch];							      \
 									      \
-    if (__builtin_expect (ch == 0, 0) && *inptr != '\0')		      \
+    if (__glibc_unlikely (ch == 0) && *inptr != '\0')			      \
       {									      \
 	/* This is an illegal character.  */				      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (increment);			      \
@@ -448,16 +448,15 @@ static const char from_ucs4[][2] =
     uint32_t ch = get32 (inptr);					      \
     const char *cp;							      \
 									      \
-    if (__builtin_expect (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0]),   \
-			  0))						      \
+    if (__glibc_unlikely (ch >= sizeof (from_ucs4) / sizeof (from_ucs4[0])))						      \
       {									      \
-	if (__builtin_expect (ch, 0) == 0x2126)				      \
+	if (__glibc_unlikely (ch == 0x2126))				      \
 	  cp = "\xe0";							      \
-	else if (__builtin_expect (ch, 0) == 0x2c7)			      \
+	else if (__glibc_unlikely (ch == 0x2c7))			      \
 	  cp = "\xcf\x20";						      \
-	else if (__builtin_expect (ch < 0x2d8, 0)			      \
-		 || __builtin_expect (ch > 0x2dd, 0)			      \
-		 || __builtin_expect (ch == 0x2dc, 0))			      \
+	else if (__glibc_unlikely (ch < 0x2d8)				      \
+		 || __glibc_unlikely (ch > 0x2dd)			      \
+		 || __glibc_unlikely (ch == 0x2dc))			      \
 	  {								      \
 	    UNICODE_TAG_HANDLER (ch, 4);				      \
 									      \
@@ -477,7 +476,7 @@ static const char from_ucs4[][2] =
       {									      \
 	cp = from_ucs4[ch];						      \
 									      \
-	if (__builtin_expect (cp[0] == '\0', 0) && ch != 0)		      \
+	if (__glibc_unlikely (cp[0] == '\0') && ch != 0)		      \
 	  {								      \
 	    /* Illegal.  */						      \
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/tcvn5712-1.c b/iconvdata/tcvn5712-1.c
index 5edc653..9add2e1 100644
--- a/iconvdata/tcvn5712-1.c
+++ b/iconvdata/tcvn5712-1.c
@@ -460,7 +460,7 @@ static const struct
 									      \
 	/* If we don't have enough room to output ch as well, then deal	      \
 	   with it in another round.  */				      \
-	if (!must_buffer_ch && __builtin_expect (outptr + 4 > outend, 0))     \
+	if (!must_buffer_ch && __glibc_unlikely (outptr + 4 > outend))	      \
 	  continue;							      \
       }									      \
 									      \
diff --git a/iconvdata/tscii.c b/iconvdata/tscii.c
index 644cb96..2bd566c 100644
--- a/iconvdata/tscii.c
+++ b/iconvdata/tscii.c
@@ -347,7 +347,7 @@ static const uint32_t tscii_next_state[6] =
 	    /* Retrieve the successor state.  */			      \
 	    *statep = tscii_next_state[(*statep >> 4) & 0x0f];		      \
 	  }								      \
-	while (*statep != 0 && __builtin_expect (outptr + 4 <= outend, 1));   \
+	while (*statep != 0 && __glibc_likely (outptr + 4 <= outend));	      \
 									      \
 	if (*statep != 0)						      \
 	  {								      \
diff --git a/iconvdata/uhc.c b/iconvdata/uhc.c
index 629f2d6..3494e1f 100644
--- a/iconvdata/uhc.c
+++ b/iconvdata/uhc.c
@@ -3064,9 +3064,9 @@ static const char uhc_hangul_from_ucs[11172][2] =
 */									      \
     if (ch <= 0x7f)							      \
       ++inptr;								      \
-    else if (__builtin_expect (ch <= 0x80, 0)				      \
-	     || __builtin_expect (ch >= 0xfe, 0)			      \
-	     || __builtin_expect (ch == 0xc9, 0))			      \
+    else if (__glibc_unlikely (ch <= 0x80)				      \
+	     || __glibc_unlikely (ch >= 0xfe)				      \
+	     || __glibc_unlikely (ch == 0xc9))				      \
       {									      \
 	/* This is illegal.  */						      \
 	STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
@@ -3107,12 +3107,12 @@ static const char uhc_hangul_from_ucs[11172][2] =
 									      \
 	if (ch < 0xa1 || ch2 < 0xa1)					      \
 	  {								      \
-	    if (__builtin_expect (ch > 0xc6, 0)				      \
-		|| __builtin_expect (ch2 < 0x41, 0)			      \
-		|| __builtin_expect (ch2 > 0xfe, 0)			      \
-		|| (__builtin_expect (ch2 > 0x5a, 0) && ch2 < 0x61)	      \
-		|| (__builtin_expect (ch2 > 0x7a, 0) && ch2 < 0x81)	      \
-		|| (__builtin_expect (ch == 0xc6, 0) && ch2 > 0x52))	      \
+	    if (__glibc_unlikely (ch > 0xc6)				      \
+		|| __glibc_unlikely (ch2 < 0x41)			      \
+		|| __glibc_unlikely (ch2 > 0xfe)			      \
+		|| (__glibc_unlikely (ch2 > 0x5a) && ch2 < 0x61)	      \
+		|| (__glibc_unlikely (ch2 > 0x7a) && ch2 < 0x81)	      \
+		|| (__glibc_unlikely (ch == 0xc6) && ch2 > 0x52))	      \
 	      {								      \
 		/* This is not legal.  */				      \
 		STANDARD_FROM_LOOP_ERR_HANDLER (1);			      \
@@ -3135,8 +3135,8 @@ static const char uhc_hangul_from_ucs[11172][2] =
 	else								      \
 	  {								      \
 	    ch = ksc5601_to_ucs4 (&inptr, 2, 0x80);			      \
-	    if (__builtin_expect (ch == __UNKNOWN_10646_CHAR, 0)	      \
-		|| __builtin_expect (ch == 0x327e, 0))			      \
+	    if (__glibc_unlikely (ch == __UNKNOWN_10646_CHAR)		      \
+		|| __glibc_unlikely (ch == 0x327e))			      \
 	      {								      \
 		/* Illegal.  */						      \
 		STANDARD_FROM_LOOP_ERR_HANDLER (2);			      \
@@ -3208,8 +3208,8 @@ static const char uhc_hangul_from_ucs[11172][2] =
       {									      \
 	size_t written = ucs4_to_ksc5601_sym (ch, outptr, outend - outptr);   \
 									      \
-	if (__builtin_expect (ch == 0x327e, 0)				      \
-	    || __builtin_expect (written == __UNKNOWN_10646_CHAR, 0))	      \
+	if (__glibc_unlikely (ch == 0x327e)				      \
+	    || __glibc_unlikely (written == __UNKNOWN_10646_CHAR))	      \
 	  {								      \
 	    UNICODE_TAG_HANDLER (ch, 4);				      \
 	    STANDARD_TO_LOOP_ERR_HANDLER (4);				      \
diff --git a/iconvdata/utf-16.c b/iconvdata/utf-16.c
index 31c2d6e..c4b2c48 100644
--- a/iconvdata/utf-16.c
+++ b/iconvdata/utf-16.c
@@ -148,7 +148,7 @@ gconv_init (struct __gconv_step *step)
     }
 
   result = __GCONV_NOCONV;
-  if (__builtin_expect (dir, to_utf16) != illegal_dir)
+  if (__glibc_likely (dir != illegal_dir))
     {
       new_data = (struct utf16_data *) malloc (sizeof (struct utf16_data));
 
@@ -285,7 +285,7 @@ gconv_end (struct __gconv_step *data)
       {									      \
 	u1 = bswap_16 (u1);						      \
 									      \
-	if (__builtin_expect (u1 < 0xd800, 1) || u1 > 0xdfff)		      \
+	if (__glibc_likely (u1 < 0xd800) || u1 > 0xdfff)		      \
 	  {								      \
 	    /* No surrogate.  */					      \
 	    put32 (outptr, u1);						      \
@@ -307,8 +307,8 @@ gconv_end (struct __gconv_step *data)
 									      \
 	    inptr += 2;							      \
 	    u2 = bswap_16 (get16 (inptr));				      \
-	    if (__builtin_expect (u2 < 0xdc00, 0)			      \
-		|| __builtin_expect (u2 > 0xdfff, 0))			      \
+	    if (__glibc_unlikely (u2 < 0xdc00)				      \
+		|| __glibc_unlikely (u2 > 0xdfff))			      \
 	      {								      \
 		/* This is no valid second word for a surrogate.  */	      \
 		inptr -= 2;						      \
@@ -321,7 +321,7 @@ gconv_end (struct __gconv_step *data)
       }									      \
     else								      \
       {									      \
-	if (__builtin_expect (u1 < 0xd800, 1) || u1 > 0xdfff)		      \
+	if (__glibc_likely (u1 < 0xd800) || u1 > 0xdfff)		      \
 	  {								      \
 	    /* No surrogate.  */					      \
 	    put32 (outptr, u1);						      \
@@ -341,8 +341,8 @@ gconv_end (struct __gconv_step *data)
 									      \
 	    inptr += 2;							      \
 	    uint16_t u2 = get16 (inptr);				      \
-	    if (__builtin_expect (u2 < 0xdc00, 0)			      \
-		|| __builtin_expect (u2 > 0xdfff, 0))			      \
+	    if (__glibc_unlikely (u2 < 0xdc00)				      \
+		|| __glibc_unlikely (u2 > 0xdfff))			      \
 	      {								      \
 		/* This is no valid second word for a surrogate.  */	      \
 		inptr -= 2;						      \
diff --git a/iconvdata/utf-32.c b/iconvdata/utf-32.c
index e3e4243..fe779f9 100644
--- a/iconvdata/utf-32.c
+++ b/iconvdata/utf-32.c
@@ -72,7 +72,7 @@
       put32u (outbuf, BOM);						      \
       outbuf += 4;							      \
     }									      \
-  else if (__builtin_expect (data->__invocation_counter == 0, 0)	      \
+  else if (__glibc_unlikely (data->__invocation_counter == 0)		      \
 	   && ((var == UTF_32LE && BYTE_ORDER == BIG_ENDIAN)		      \
 	       || (var == UTF_32BE && BYTE_ORDER == LITTLE_ENDIAN)))	      \
     data->__flags |= __GCONV_SWAP;					      \
@@ -145,7 +145,7 @@ gconv_init (struct __gconv_step *step)
     }
 
   result = __GCONV_NOCONV;
-  if (__builtin_expect (dir, to_utf32) != illegal_dir)
+  if (__glibc_likely (dir != illegal_dir))
     {
       new_data = (struct utf32_data *) malloc (sizeof (struct utf32_data));
 
diff --git a/iconvdata/utf-7.c b/iconvdata/utf-7.c
index 637d85c..4ca91e5 100644
--- a/iconvdata/utf-7.c
+++ b/iconvdata/utf-7.c
@@ -218,8 +218,8 @@ base64 (unsigned int i)
 									      \
 	    /* If accumulated data is nonzero, the input is invalid.  */      \
 	    /* Also, partial UTF-16 characters are invalid.  */		      \
-	    if (__builtin_expect (statep->__value.__wch != 0, 0)	      \
-		|| __builtin_expect ((statep->__count >> 3) <= 26, 0))	      \
+	    if (__glibc_unlikely (statep->__value.__wch != 0)		      \
+		|| __glibc_unlikely ((statep->__count >> 3) <= 26))	      \
 	      {								      \
 		STANDARD_FROM_LOOP_ERR_HANDLER ((statep->__count = 0, 1));    \
 	      }								      \
@@ -261,7 +261,7 @@ base64 (unsigned int i)
 		   indeed form a Low Surrogate.  */			      \
 		uint32_t wc2 = wch & 0xffff;				      \
 									      \
-		if (! __builtin_expect (wc2 >= 0xdc00 && wc2 < 0xe000, 1))    \
+		if (! __glibc_likely (wc2 >= 0xdc00 && wc2 < 0xe000))	      \
 		  {							      \
 		    STANDARD_FROM_LOOP_ERR_HANDLER ((statep->__count = 0, 1));\
 		  }							      \
-- 
1.9.3

>From 25282e6d2d0bb5d9314ec02845747aa9c86c5606 Mon Sep 17 00:00:00 2001
From: Florian Weimer <fweimer@redhat.com>
Date: Tue, 9 Sep 2014 20:37:28 +0200
Subject: [PATCH 3/3] Manual cleanups after __builtin_expect conversion of
 iconv/ and iconvdata/

This commit removes the one remaining __builtin_expect expressions which
can be converted, and fixes long lines introduces by the automated
rewriting.

2014-09-09  Florian Weimer  <fweimer@redhat.com>
 
	* iconv/gconv_dl.c (__gconv_find_shlib): Wrap long line resulting
	from the automated __builtin_expect conversion.
	* iconvdata/euc-kr.c (euckr_from_ucs4): Likewise.
	* iconv/gconv_cache.c (__gconv_load_cache): Replace multi-line
	__builtin_expect with __glibc_unlikely.

diff --git a/iconv/gconv_cache.c b/iconv/gconv_cache.c
index cb48eca..dab5ac6 100644
--- a/iconv/gconv_cache.c
+++ b/iconv/gconv_cache.c
@@ -116,9 +116,9 @@ __gconv_load_cache (void)
       || __glibc_unlikely (header->string_offset >= cache_size)
       || __glibc_unlikely (header->hash_offset >= cache_size)
       || __glibc_unlikely (header->hash_size == 0)
-      || __builtin_expect ((header->hash_offset
+      || __glibc_unlikely ((header->hash_offset
 			    + header->hash_size * sizeof (struct hash_entry))
-			   > cache_size, 0)
+			   > cache_size)
       || __glibc_unlikely (header->module_offset >= cache_size)
       || __glibc_unlikely (header->otherconv_offset > cache_size))
     {
diff --git a/iconv/gconv_dl.c b/iconv/gconv_dl.c
index 840793d..05448b3 100644
--- a/iconv/gconv_dl.c
+++ b/iconv/gconv_dl.c
@@ -93,7 +93,8 @@ __gconv_find_shlib (const char *name)
 	  found->counter = -TRIES_BEFORE_UNLOAD - 1;
 	  found->handle = NULL;
 
-	  if (__glibc_unlikely (__tsearch (found, &loaded, known_compare) == NULL))
+	  if (__glibc_unlikely (__tsearch (found, &loaded, known_compare)
+				== NULL))
 	    {
 	      /* Something went wrong while inserting the entry.  */
 	      free (found);
diff --git a/iconvdata/euc-kr.c b/iconvdata/euc-kr.c
index 50d1a8c..2f10453 100644
--- a/iconvdata/euc-kr.c
+++ b/iconvdata/euc-kr.c
@@ -38,7 +38,8 @@ euckr_from_ucs4 (uint32_t ch, unsigned char *cp)
 	  cp[0] = '\xa3';
 	  cp[1] = '\xdc';
 	}
-      else if (__glibc_likely (ucs4_to_ksc5601 (ch, cp, 2) != __UNKNOWN_10646_CHAR))
+      else if (__glibc_likely (ucs4_to_ksc5601 (ch, cp, 2)
+			       != __UNKNOWN_10646_CHAR))
 	{
 	  cp[0] |= 0x80;
 	  cp[1] |= 0x80;
-- 
1.9.3


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]