This is the mail archive of the
glibc-cvs@sourceware.org
mailing list for the glibc project.
GNU C Library master sources branch master updated. glibc-2.22-170-gb022830
- From: krebbel at sourceware dot org
- To: glibc-cvs at sourceware dot org
- Date: 26 Aug 2015 08:27:24 -0000
- Subject: GNU C Library master sources branch master updated. glibc-2.22-170-gb022830
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU C Library master sources".
The branch, master has been updated
via b022830bebc23456fc40825c6b5507bc0f52381f (commit)
via 808d70228891ab4d4795ab3dd1e015bf63ba18d6 (commit)
via 1efad39b227c0d3cd0641cae70c7e95c8ca290a6 (commit)
via 798f5b4b5d24e5687dd55acc535954ed1d59c10c (commit)
via f21216015b7395c535abc01b9585a1ae3ceaa132 (commit)
via 2e9e166761fed5dfaa53480412bef0656f1b5306 (commit)
via 9b593dc3055d44a4179c03050be58a437ae385a1 (commit)
via 88eefd344b3cf4a41284a1dfdaca61667e3a1b4b (commit)
via b4c21601b147efc3c2b0e679e4ffc554b3987f0b (commit)
via f0ba659847446eec3b2477d60c97c77ef4680e81 (commit)
via f1ffad98be7ec4111fbd1cd1f58f3e3343257519 (commit)
via f40132d4bda984479bac89dfcd6968e9ff56e088 (commit)
via d23d4ef19f1f5463745f2af205acb8c4975fd680 (commit)
via cf150d45a97c6bb8410d6d3acf6e1560e8fe96cc (commit)
via cee82e70ccb7b2f054cd781b0a603ae244523e72 (commit)
via 63724a6db60f98e91da474d11d83a19aa10fc54e (commit)
via e1fe91180e4e29549f35f8ecd705b0bb4f208606 (commit)
via d626a24f235dbd4c446b241211a9a264a1eedb9e (commit)
via b3a0c176d1185621c4dd2bb3a51ec961bdb29123 (commit)
via d183b96ee6dc694e95f212c9272a178163351b19 (commit)
via 8ade3db78db17e0112648d302f98eda115949cd5 (commit)
via 680df122ab8a07806cb38d044292896334f76c01 (commit)
via fcf40ebe2682fd65d64f94d69a3df798960cf1b7 (commit)
via 9472f35a0a6dbbda82ce103aaf0f5013f5d46e34 (commit)
via fd484e057dd4d2813182df08584a4c48d6f1dd7a (commit)
via 4f0a1cea34c05fb2acc16f1a2d291f53230eb4fb (commit)
via a1b0488fc9df3d895a2e5eefbcd348d3f7fe0e52 (commit)
via 4e28fa80886c71e6aaf85016b82ce981c0f12e6d (commit)
via 31556246c3ac168a2dfec8f9036d913765bbb73d (commit)
via 5d96fe8c0dc3450bafe6c2aae2dabc76819df3e0 (commit)
from f97194930886838796546646e26a49bb5899075b (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=b022830bebc23456fc40825c6b5507bc0f52381f
commit b022830bebc23456fc40825c6b5507bc0f52381f
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:26 2015 +0200
Adjust _Unwind_Word in unwind.h to version in libgcc.
Building glibc on s390-32 with gcc option -mzarch produces the error due to
sysdeps/s390/jmpbuf-unwind.h:37:10: (void *) (_Unwind_GetCFA (_context):
cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
Building on s390-32 in esa-mode or s390-64 is fine.
_Unwind_GetCFA returns an _Unwind_Word which is an unsigned
with a size of 4 bytes on s390-32 (esa-mode) and 8 bytes on s390-64.
On s390-32 (zarch-mode), _Unwind_Word has a size of 8 bytes, too.
_Unwind_Word is defined in sysdeps/generic/unwind.h as
typedef unsigned _Unwind_Word __attribute__((__mode__(__word__)));
In libgcc unwind header (<gcc-src>/libgcc/unwind-generic.h) this typedef has
changed to "typedef unsigned _Unwind_Word __attribute__((__mode__(__unwind_word__)));"
in June 2008.
With this mode, _Unwind_Word has a size of 4 bytes on s390-32 (zarch-mode).
The same change applies to _Unwind_Sword.
Thus this patch updates the unwind header according to these changes.
Afterwards, the int-to-pointer-cast-warning is gone away on s390-32 (zarch-mode)
and the testsuite runs with the same test-failures as s390-32 (esa-mode)
plus FAIL: c++-types-check. Here register_t is expected to has a size of 4 bytes,
but it has a size of 8 bytes due to:
posix/sys/types.h:205:typedef int register_t __attribute__ ((__mode__ (__word__)));
The libgcc-patch for gcc 4.4 can be found here:
"[PATCH, spu, unwind] Remove attribute ((mode (word))) from unwind.h"
https://gcc.gnu.org/ml/gcc-patches/2008-06/msg00969.html
ChangeLog:
* sysdeps/generic/unwind.h
(_Unwind_Word): Use __mode__(__unwind_word__)
instead of __mode__(__word__).
(_Unwind_Sword): Likewise.
diff --git a/ChangeLog b/ChangeLog
index ea5367c..7225551 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,12 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/generic/unwind.h
+ (_Unwind_Word): Use __mode__(__unwind_word__)
+ instead of __mode__(__word__).
+ (_Unwind_Sword): Likewise.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/s390-64/utf8-utf16-z9.c
(MAX_NEEDED_INPUT): New define.
(MAX_NEEDED_OUTPUT): New define.
diff --git a/sysdeps/generic/unwind.h b/sysdeps/generic/unwind.h
index 41b6aec..75d3084 100644
--- a/sysdeps/generic/unwind.h
+++ b/sysdeps/generic/unwind.h
@@ -31,8 +31,8 @@ extern "C" {
/* @@@ The IA-64 ABI uses uint64 throughout. Most places this is
inefficient for 32-bit and smaller machines. */
-typedef unsigned _Unwind_Word __attribute__((__mode__(__word__)));
-typedef signed _Unwind_Sword __attribute__((__mode__(__word__)));
+typedef unsigned _Unwind_Word __attribute__((__mode__(__unwind_word__)));
+typedef signed _Unwind_Sword __attribute__((__mode__(__unwind_word__)));
#if defined(__ia64__) && defined(__hpux__)
typedef unsigned _Unwind_Ptr __attribute__((__mode__(__word__)));
#else
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=808d70228891ab4d4795ab3dd1e015bf63ba18d6
commit 808d70228891ab4d4795ab3dd1e015bf63ba18d6
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:26 2015 +0200
S390: Fix build error with gcc6 in utf8_utf16-z9.c.
This patch fixes the build error with gcc6:
array subscript is above array bounds [-Werror=array-bounds]
While including loop.c to construct the SINGLE(LOOPFCT) method
for converting from UTF-16 to UTF-8, the bytebuf array with length
MAX_NEEDED_INPUT is used as inptr. MAX_NEEDED_INPUT defaults to
MIN_NEEDED_INPUT if not defined before including loop.c.
Thus bytebuf has a length of 2.
This patch defines MAX_NEEDED_INPUT to MAX_NEEDED_TO, which is 4.
ChangeLog:
* sysdeps/s390/s390-64/utf8-utf16-z9.c
(MAX_NEEDED_INPUT): New define.
(MAX_NEEDED_OUTPUT): New define.
diff --git a/ChangeLog b/ChangeLog
index 8738516..ea5367c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,11 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/s390-64/utf8-utf16-z9.c
+ (MAX_NEEDED_INPUT): New define.
+ (MAX_NEEDED_OUTPUT): New define.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* NEWS: New item for IBM z13 string optimizations.
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
diff --git a/sysdeps/s390/s390-64/utf8-utf16-z9.c b/sysdeps/s390/s390-64/utf8-utf16-z9.c
index 1425cb1..6712c1c 100644
--- a/sysdeps/s390/s390-64/utf8-utf16-z9.c
+++ b/sysdeps/s390/s390-64/utf8-utf16-z9.c
@@ -183,6 +183,7 @@ gconv_end (struct __gconv_step *data)
#define MIN_NEEDED_INPUT MIN_NEEDED_FROM
#define MAX_NEEDED_INPUT MAX_NEEDED_FROM
#define MIN_NEEDED_OUTPUT MIN_NEEDED_TO
+#define MAX_NEEDED_OUTPUT MAX_NEEDED_TO
#define LOOPFCT FROM_LOOP
/* The software implementation is based on the code in gconv_simple.c. */
#define BODY \
@@ -340,6 +341,7 @@ gconv_end (struct __gconv_step *data)
/* Conversion from UTF-16 to UTF-8. */
#define MIN_NEEDED_INPUT MIN_NEEDED_TO
+#define MAX_NEEDED_INPUT MAX_NEEDED_TO
#define MIN_NEEDED_OUTPUT MIN_NEEDED_FROM
#define MAX_NEEDED_OUTPUT MAX_NEEDED_FROM
#define LOOPFCT TO_LOOP
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=1efad39b227c0d3cd0641cae70c7e95c8ca290a6
commit 1efad39b227c0d3cd0641cae70c7e95c8ca290a6
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:26 2015 +0200
S390: Optimize string, wcsmbs and memory functions.
This patch set introduces optimized string, wcsmbs and memory functions for
S390/S390x. The functions are accelerated by the usage of the new z13 vector
instructions.
The Principles of Operations manual for IBM z13 is publically available:
http://publibfi.boulder.ibm.com/epubs/pdf/dz9zr010.pdf
The support for these instructions in assembler was introduced by commits:
-"[Committed] S/390: Add support for IBM z13."
(https://sourceware.org/ml/binutils/2015-01/msg00197.html)
-"[Committed] S/390: Add more IBM z13 instructions"
(https://sourceware.org/ml/binutils/2015-03/msg00088.html)
The first patches do preparation for the latter optimization patches.
The floating point exception handling - fetestexcept(), ... - is fixed and
the platform and hwcap strings are extended.
The current ifunc routines memset, memcpy and memcmp are refactored and the
ifunc test-framework is now enabled.
A S390 specific configure-check tests if the used binutils supports the new
vector instructions. The optimized functions are provided via ifunc if the
binutils supports the vector instructions. Otherwise a message is dumped to
configure output and only the currently used common code functions are
available.
The optimized functions are implemented in common for s390-32 and s390-64
and the few differences are handled via #ifdef.
The ifunc-resolvers are defined in files sysdeps/s390/multiarch/<func>.c,
which choose either the current implementation __<func>_c() or the vector
implementation __<func>_vx() depending on the HWCAP_S390_VX flag bit in
AT_HWCAP field. If the bit is set, the hardware and the kernel are supporting
vector registers and instructions. If the used binutils lacks vector-support,
then the default implementation in string or wcsmbs directory is included
here instead.
The file sysdeps/s390/multiarch/<func>-c.c includes the current implementation
and defines the function name __<func>_c.
The assembler files sysdeps/s390/multiarch/<func>-vx.S with the vector
instructions are using the directive '.machine "z13"' to allow building glibc
without option '-march=z13'. Additionally the directive '.machinemode
"zarch_nohighgprs"' is needed for the 31bit glibc. This mode does not set the
highgprs flag in ELF header, which would lead to an unloadable libc on a 31bit
kernel.
The most optimized string functions are structured in the same way:
The first 16 bytes of the string is loaded unaligned via vlbb - vector load
to block boundary (e.g. 4k). This instruction loads 16 bytes if possible.
In case of a page cross, it only loads the last bytes of the current page
without a segmentation fault.
Afterwards these first part of string is processed. If e.g. for strlen the end
of string is reached within this first part, the function returns. Otherwise
the pointer is aligned to 16 byte, so i can load a full vector register with vl
without checking for a page cross. Afterwards the first part of string is
processed. If e.g. for strlen the end of string is reached within this first
part, the function returns. Otherwise the pointer is aligned to 16 byte, so
a full vector register can be loaded with vl - vector load - without checking
for a page cross. The remaining string is processed in a four times unrolled
loop, because benchmark results measured improvements compared to a non
unrolled loop.
The optimized wide string functions can only handle 4byte aligned string
pointers. Although a wchar_t pointer should always be 4byte aligned, the most
current common code wide string functions can handle non aligned strings.
Thus the optimized functions will fall back to the common code functions in
case of a non aligned wide string to behave the same as before this patch.
Some string tests can test the string and the wide string version of a function.
The remaining ones are extended and new wide string tests are added.
This is the same in case of the benchtests.
ChangeLog:
* NEWS: New item for IBM z13 string optimizations.
diff --git a/ChangeLog b/ChangeLog
index 3080969..8738516 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,9 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * NEWS: New item for IBM z13 string optimizations.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/memrchr-c.c: New File.
* sysdeps/s390/multiarch/memrchr-vx.S: Likewise.
* sysdeps/s390/multiarch/memrchr.c: Likewise.
diff --git a/NEWS b/NEWS
index ee0a9be..80bddc8 100644
--- a/NEWS
+++ b/NEWS
@@ -10,12 +10,15 @@ Version 2.23
* The following bugs are resolved with this release:
14341, 16517, 16519, 16520, 16734, 16973, 17787, 17905, 18084, 18086,
- 18240, 18265, 18370, 18421, 18480, 18525, 18618, 18647, 18661, 18674,
- 18681, 18778, 18781, 18787, 18789, 18790, 18795, 18796, 18820, 18823,
- 18824.
+ 18240, 18265, 18370, 18421, 18480, 18525, 18610, 18618, 18647, 18661,
+ 18674,18681, 18778, 18781, 18787, 18789, 18790, 18795, 18796, 18820,
+ 18823, 18824.
* The obsolete header <regexp.h> has been removed. Programs that require
this header must be updated to use <regex.h> instead.
+
+* Optimized string, wcsmbs and memory functions for IBM z13.
+ Implemented by Stefan Liebler.
Version 2.22
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=798f5b4b5d24e5687dd55acc535954ed1d59c10c
commit 798f5b4b5d24e5687dd55acc535954ed1d59c10c
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:26 2015 +0200
S390: Optimize memrchr.
This patch provides optimized version of memrchr with the z13 vector
instructions.
ChangeLog:
* sysdeps/s390/multiarch/memrchr-c.c: New File.
* sysdeps/s390/multiarch/memrchr-vx.S: Likewise.
* sysdeps/s390/multiarch/memrchr.c: Likewise.
* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Add memrchr functions.
* sysdeps/s390/multiarch/ifunc-impl-list-common.c
(__libc_ifunc_impl_list_common): Add ifunc test for memrchr.
diff --git a/ChangeLog b/ChangeLog
index c1cefbc..3080969 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,15 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/memrchr-c.c: New File.
+ * sysdeps/s390/multiarch/memrchr-vx.S: Likewise.
+ * sysdeps/s390/multiarch/memrchr.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile
+ (sysdep_routines): Add memrchr functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list-common.c
+ (__libc_ifunc_impl_list_common): Add ifunc test for memrchr.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/wmemcmp-c.c: New File.
* sysdeps/s390/multiarch/wmemcmp-vx.S: Likewise.
* sysdeps/s390/multiarch/wmemcmp.c: Likewise.
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 929a545..0805b07 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -17,7 +17,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strcspn strcspn-vx strcspn-c \
memchr memchr-vx \
rawmemchr rawmemchr-vx rawmemchr-c \
- memccpy memccpy-vx memccpy-c
+ memccpy memccpy-vx memccpy-c \
+ memrchr memrchr-vx memrchr-c
endif
ifeq ($(subdir),wcsmbs)
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 5ea258b..c235bdc 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -137,6 +137,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (wmemcmp);
+ IFUNC_VX_IMPL (memrchr);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/memrchr-c.c b/sysdeps/s390/multiarch/memrchr-c.c
new file mode 100644
index 0000000..ac481fd
--- /dev/null
+++ b/sysdeps/s390/multiarch/memrchr-c.c
@@ -0,0 +1,25 @@
+/* Default memrchr implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define MEMRCHR __memrchr_c
+
+# include <string.h>
+extern __typeof (__memrchr) __memrchr_c;
+# include <string/memrchr.c>
+#endif
diff --git a/sysdeps/s390/multiarch/memrchr-vx.S b/sysdeps/s390/multiarch/memrchr-vx.S
new file mode 100644
index 0000000..a86ca60
--- /dev/null
+++ b/sysdeps/s390/multiarch/memrchr-vx.S
@@ -0,0 +1,160 @@
+/* Vector optimized 32/64 bit S/390 version of memrchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* void *memrchr (const void *s, int c, size_t n)
+ Scans memory for character c backwards
+ and returns pointer to first c.
+
+ Register usage:
+ -r0=tmp
+ -r1=tmp
+ -r2=s
+ -r3=c
+ -r4=n
+ -r5=s in loop
+
+ -v16=part of s
+ -v17=index of found c
+ -v18=c replicated
+ -v20=permute pattern
+*/
+ENTRY(__memrchr_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+ clgije %r4,0,.Lnot_found
+
+ vlvgb %v18,%r3,0 /* Generate vector which elements are all c.
+ If c > 255, c will be truncated. */
+ vrepb %v18,%v18,0
+
+ llcr %r3,%r3 /* char c_char = (char) c. */
+
+ /* check byte n - 1. */
+ llc %r0,-1(%r4,%r2)
+ slgfi %r4,1
+ clrje %r0,%r3,.Lfound_end
+ jh .Lnot_found /* Return NULL if n is now 0. */
+
+ larl %r1,.Lpermute_mask /* Load permute mask. */
+ vl %v20,0(%r1)
+
+ /* check byte n - 2. */
+ llc %r0,-1(%r4,%r2)
+ slgfi %r4,1
+ clrje %r0,%r3,.Lfound_end
+ jh .Lnot_found /* Return NULL if n is now 0. */
+
+ clgijhe %r4,64,.Lloop64 /* If n >= 64 -> loop64. */
+
+.Llt64:
+ /* Process n < 64 bytes. */
+ clgijl %r4,16,.Llt16 /* Jump away if n < 16. */
+ aghi %r4,-16
+ vl %v16,0(%r4,%r2)
+ vfeebs %v17,%v16,%v18
+ jno .Lfound0
+ clgijl %r4,16,.Llt16
+ aghi %r4,-16
+ vl %v16,0(%r4,%r2)
+ vfeebs %v17,%v16,%v18
+ jno .Lfound0
+ clgijl %r4,16,.Llt16
+ aghi %r4,-16
+ vl %v16,0(%r4,%r2)
+ vfeebs %v17,%v16,%v18
+ jno .Lfound0
+.Llt16:
+ clgfi %r4,0 /* if remaining bytes == 0, return NULL. */
+ locghie %r2,0
+ ber %r14
+
+ aghi %r4,-1 /* vll needs highest index. */
+ vll %v16,%r4,0(%r2) /* Load remaining bytes. */
+
+ /* Right-shift of v16 to mask bytes after highest index. */
+ lhi %r0,15
+ slr %r0,%r4 /* Compute byte count for vector shift right. */
+ sll %r0,3 /* Convert to bit count. */
+ vlvgb %v17,%r0,7
+ vsrlb %v16,%v16,%v17 /* Vector shift right by byte by number of bytes
+ specified in bits 1-4 of byte 7 in v17. */
+ j .Lfound_permute
+
+.Lfound48:
+ aghi %r4,16
+.Lfound32:
+ aghi %r4,16
+.Lfound16:
+ aghi %r4,16
+.Lfound0:
+ la %r2,0(%r4,%r2) /* Set pointer to start of v16. */
+ lghi %r4,15 /* Set highest index in v16 to last index. */
+.Lfound_permute:
+ /* Search for a c in v16 in reversed byte order. v16 contains %r4 + 1
+ bytes. If v16 was not fully loaded, the bytes are already
+ right shifted, so that the bytes in v16 can simply be reversed. */
+ vperm %v16,%v16,%v16,%v20 /* Permute v16 to reversed order. */
+ vfeeb %v16,%v16,%v18 /* Find c in reversed v16. */
+ vlgvb %r1,%v16,7 /* Index of c or 16 if not found. */
+
+ /* Return NULL if there is no c in loaded bytes. */
+ clrjh %r1,%r4,.Lnot_found
+
+ slgr %r4,%r1
+.Lfound_end:
+ la %r2,0(%r4,%r2) /* Return pointer to c. */
+ br %r14
+
+.Lnot_found:
+ lghi %r2,0
+ br %r14
+
+.Lpermute_mask:
+ .byte 0x0F,0x0E,0x0D,0x0C,0x0B,0x0A,0x09,0x08
+ .byte 0x07,0x06,0x05,0x04,0x03,0x02,0x01,0x00
+
+.Lloop64:
+ aghi %r4,-64
+ vl %v16,48(%r4,%r2) /* Load 16bytes of memory area. */
+ vfeebs %v17,%v16,%v18 /* Find c. */
+ jno .Lfound48 /* Jump away if c was found. */
+ vl %v16,32(%r4,%r2)
+ vfeebs %v17,%v16,%v18
+ jno .Lfound32
+ vl %v16,16(%r4,%r2)
+ vfeebs %v17,%v16,%v18
+ jno .Lfound16
+ vl %v16,0(%r4,%r2)
+ vfeebs %v17,%v16,%v18
+ jno .Lfound0
+
+ clgijhe %r4,64,.Lloop64 /* If n >= 64 -> loop64. */
+ j .Llt64
+END(__memrchr_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/memrchr.c b/sysdeps/s390/multiarch/memrchr.c
new file mode 100644
index 0000000..8ac2f52
--- /dev/null
+++ b/sysdeps/s390/multiarch/memrchr.c
@@ -0,0 +1,28 @@
+/* Multiple versions of memrchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc (__memrchr)
+weak_alias (__memrchr, memrchr)
+
+#else
+# include <string/memrchr.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=f21216015b7395c535abc01b9585a1ae3ceaa132
commit f21216015b7395c535abc01b9585a1ae3ceaa132
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:25 2015 +0200
S390: Optimize wmemcmp.
This patch provides optimized version of wmemcmp with the z13 vector
instructions.
ChangeLog:
* sysdeps/s390/multiarch/wmemcmp-c.c: New File.
* sysdeps/s390/multiarch/wmemcmp-vx.S: Likewise.
* sysdeps/s390/multiarch/wmemcmp.c: Likewise.
* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Add wmemcmp functions.
* sysdeps/s390/multiarch/ifunc-impl-list-common.c
(__libc_ifunc_impl_list_common): Add ifunc test for wmemcmp.
* benchtests/bench-wmemcmp.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wmemcmp.
diff --git a/ChangeLog b/ChangeLog
index 76fc7c2..c1cefbc 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,17 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/wmemcmp-c.c: New File.
+ * sysdeps/s390/multiarch/wmemcmp-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wmemcmp.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile
+ (sysdep_routines): Add wmemcmp functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list-common.c
+ (__libc_ifunc_impl_list_common): Add ifunc test for wmemcmp.
+ * benchtests/bench-wmemcmp.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wmemcmp.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/wmemset-c.c: New File.
* sysdeps/s390/multiarch/wmemset-vx.S: Likewise.
* sysdeps/s390/multiarch/wmemset.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 28a5170..dc2a5df 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -38,7 +38,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
wcscmp wcsncmp wcschr wcschrnul wcsrchr wcsspn wcspbrk wcscspn \
- wmemchr wmemset
+ wmemchr wmemset wmemcmp
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-wmemcmp.c b/benchtests/bench-wmemcmp.c
new file mode 100644
index 0000000..8b33f89
--- /dev/null
+++ b/benchtests/bench-wmemcmp.c
@@ -0,0 +1,20 @@
+/* Measure wmemcmp functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-memcmp.c"
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index eac88e0..929a545 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -38,5 +38,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcspbrk wcspbrk-vx wcspbrk-c \
wcscspn wcscspn-vx wcscspn-c \
wmemchr wmemchr-vx wmemchr-c \
- wmemset wmemset-vx wmemset-c
+ wmemset wmemset-vx wmemset-c \
+ wmemcmp wmemcmp-vx wmemcmp-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 93789ac..5ea258b 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -134,6 +134,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (memccpy);
IFUNC_VX_IMPL (wmemset);
+
+ IFUNC_VX_IMPL (wmemcmp);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/wmemcmp-c.c b/sysdeps/s390/multiarch/wmemcmp-c.c
new file mode 100644
index 0000000..7ca9f46
--- /dev/null
+++ b/sysdeps/s390/multiarch/wmemcmp-c.c
@@ -0,0 +1,26 @@
+/* Default wmemcmp implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WMEMCMP __wmemcmp_c
+
+# include <wchar.h>
+extern __typeof (wmemcmp) __wmemcmp_c;
+
+# include <wcsmbs/wmemcmp.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wmemcmp-vx.S b/sysdeps/s390/multiarch/wmemcmp-vx.S
new file mode 100644
index 0000000..eabfd2b
--- /dev/null
+++ b/sysdeps/s390/multiarch/wmemcmp-vx.S
@@ -0,0 +1,149 @@
+/* Vector Optimized 32/64 bit S/390 version of wmemcmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* int wmemcmp (const wchar_t *s1, const wchar_t *s2, size_t n)
+ Compare at most n characters of two wchar_t-arrays.
+
+ Register usage:
+ -r0=tmp
+ -r1=number of blocks
+ -r2=s1
+ -r3=s2
+ -r4=n
+ -r5=current_len
+ -v16=part of s1
+ -v17=part of s2
+ -v18=index of unequal
+*/
+ENTRY(__wmemcmp_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+ clgije %r4,0,.Lend_equal /* Nothing to do if n == 0. */
+
+ /* Check range of maxlen and convert to byte-count. */
+# ifdef __s390x__
+ tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ lghi %r1,-4 /* Max byte-count is 18446744073709551612. */
+# else
+ tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ llilf %r1,4294967292 /* Max byte-count is 4294967292. */
+# endif /* !__s390x__ */
+ sllg %r4,%r4,2 /* Convert character-count to byte-count. */
+ locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ clgijh %r4,16,.Lgt16
+
+.Lremaining:
+ aghi %r4,-1 /* vstl needs highest index. */
+ vll %v16,%r4,0(%r2)
+ vll %v17,%r4,0(%r3)
+ vfenef %v18,%v16,%v17 /* Compare not equal. */
+ vlgvb %r1,%v18,7 /* Load unequal index or 16 if not found. */
+ clrj %r1,%r4,12,.Lfound2 /* r1 <= r4 -> unequal within loaded
+ bytes. */
+
+.Lend_equal:
+ lghi %r2,0
+ br %r14
+
+.Lfound:
+ /* vfenezf found an unequal element or zero.
+ This instruction compares unsigned words, but wchar_t is signed.
+ Thus we have to compare the found element again. */
+ vlgvb %r1,%v18,7 /* Extract not equal byte-index. */
+.Lfound2:
+ srl %r1,2 /* And convert it to character-index. */
+ vlgvf %r0,%v16,0(%r1) /* Load character-values. */
+ vlgvf %r1,%v17,0(%r1)
+ cr %r0,%r1
+ je .Lend_equal
+ lghi %r2,1
+ lghi %r1,-1
+ locgrl %r2,%r1
+ br %r14
+
+.Lgt16:
+ clgijh %r4,64,.Lpreloop64
+
+.Lpreloop16:
+ srlg %r1,%r4,4 /* Split into 16byte blocks */
+.Lloop16:
+ vl %v16,0(%r5,%r2)
+ vl %v17,0(%r5,%r3)
+ aghi %r5,16
+ vfenefs %v18,%v16,%v17 /* Compare not equal. */
+ jno .Lfound
+ brctg %r1,.Lloop16 /* Loop until all blocks are processed. */
+
+ llgfr %r4,%r4
+ nilf %r4,15 /* Get remaining bytes */
+ locgre %r2,%r4
+ ber %r14
+ la %r2,0(%r5,%r2)
+ la %r3,0(%r5,%r3)
+ j .Lremaining
+
+.Lpreloop64:
+ srlg %r1,%r4,6 /* Split into 64byte blocks */
+.Lloop64:
+ vl %v16,0(%r5,%r2)
+ vl %v17,0(%r5,%r3)
+ vfenefs %v18,%v16,%v17 /* Compare not equal. */
+ jno .Lfound
+
+ vl %v16,16(%r5,%r2)
+ vl %v17,16(%r5,%r3)
+ vfenefs %v18,%v16,%v17
+ jno .Lfound
+
+ vl %v16,32(%r5,%r2)
+ vl %v17,32(%r5,%r3)
+ vfenefs %v18,%v16,%v17
+ jno .Lfound
+
+ vl %v16,48(%r5,%r2)
+ vl %v17,48(%r5,%r3)
+ aghi %r5,64
+ vfenefs %v18,%v16,%v17
+ jno .Lfound
+
+ brctg %r1,.Lloop64 /* Loop until all blocks are processed. */
+
+ llgfr %r4,%r4
+ nilf %r4,63 /* Get remaining bytes */
+ locgre %r2,%r4
+ ber %r14
+ clgijh %r4,16,.Lpreloop16
+ la %r2,0(%r5,%r2)
+ la %r3,0(%r5,%r3)
+ j .Lremaining
+END(__wmemcmp_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wmemcmp.c b/sysdeps/s390/multiarch/wmemcmp.c
new file mode 100644
index 0000000..aed0ede
--- /dev/null
+++ b/sysdeps/s390/multiarch/wmemcmp.c
@@ -0,0 +1,27 @@
+/* Multiple versions of wmemcmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__wmemcmp, wmemcmp)
+
+#else
+# include <wcsmbs/wmemcmp.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=2e9e166761fed5dfaa53480412bef0656f1b5306
commit 2e9e166761fed5dfaa53480412bef0656f1b5306
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:25 2015 +0200
S390: Optimize wmemset.
This patch provides optimized version of wmemset with the z13 vector
instructions.
ChangeLog:
* sysdeps/s390/multiarch/wmemset-c.c: New File.
* sysdeps/s390/multiarch/wmemset-vx.S: Likewise.
* sysdeps/s390/multiarch/wmemset.c: Likewise.
* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Add wmemset functions.
* sysdeps/s390/multiarch/ifunc-impl-list-common.c
(__libc_ifunc_impl_list_common): Add ifunc test for wmemset.
* wcsmbs/wmemset.c: Use WMEMSET if defined.
* string/test-memset.c: Add wmemset support.
* wcsmbs/test-wmemset.c: New File.
* wcsmbs/Makefile (strop-tests): Add wmemset.
* benchtests/bench-memset.c: Add wmemset support.
* benchtests/bench-wmemset.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wmemset.
diff --git a/ChangeLog b/ChangeLog
index 3354f1d..76fc7c2 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,22 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/wmemset-c.c: New File.
+ * sysdeps/s390/multiarch/wmemset-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wmemset.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile
+ (sysdep_routines): Add wmemset functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list-common.c
+ (__libc_ifunc_impl_list_common): Add ifunc test for wmemset.
+ * wcsmbs/wmemset.c: Use WMEMSET if defined.
+ * string/test-memset.c: Add wmemset support.
+ * wcsmbs/test-wmemset.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wmemset.
+ * benchtests/bench-memset.c: Add wmemset support.
+ * benchtests/bench-wmemset.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wmemset.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/memccpy-c.c: New File.
* sysdeps/s390/multiarch/memccpy-vx.S: Likewise.
* sysdeps/s390/multiarch/memccpy.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 95712a3..28a5170 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -38,7 +38,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
wcscmp wcsncmp wcschr wcschrnul wcsrchr wcsspn wcspbrk wcscspn \
- wmemchr
+ wmemchr wmemset
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-memset.c b/benchtests/bench-memset.c
index 9786ce7..66c3c45 100644
--- a/benchtests/bench-memset.c
+++ b/benchtests/bench-memset.c
@@ -20,12 +20,29 @@
#ifdef TEST_BZERO
# define TEST_NAME "bzero"
#else
-# define TEST_NAME "memset"
-#endif
+# ifndef WIDE
+# define TEST_NAME "memset"
+# else
+# define TEST_NAME "wmemset"
+# endif /* WIDE */
+#endif /* !TEST_BZERO */
#define MIN_PAGE_SIZE 131072
#include "bench-string.h"
-char *simple_memset (char *, int, size_t);
+#ifndef WIDE
+# define MEMSET memset
+# define CHAR char
+# define SIMPLE_MEMSET simple_memset
+# define MEMCMP memcmp
+#else
+# include <wchar.h>
+# define MEMSET wmemset
+# define CHAR wchar_t
+# define SIMPLE_MEMSET simple_wmemset
+# define MEMCMP wmemcmp
+#endif /* WIDE */
+
+CHAR *SIMPLE_MEMSET (CHAR *, int, size_t);
#ifdef TEST_BZERO
typedef void (*proto_t) (char *, size_t);
@@ -39,7 +56,7 @@ IMPL (bzero, 1)
void
simple_bzero (char *s, size_t n)
{
- simple_memset (s, 0, n);
+ SIMPLE_MEMSET (s, 0, n);
}
void
@@ -48,46 +65,50 @@ builtin_bzero (char *s, size_t n)
__builtin_bzero (s, n);
}
#else
-typedef char *(*proto_t) (char *, int, size_t);
-char *builtin_memset (char *, int, size_t);
+typedef CHAR *(*proto_t) (CHAR *, int, size_t);
-IMPL (simple_memset, 0)
+IMPL (SIMPLE_MEMSET, 0)
+# ifndef WIDE
+char *builtin_memset (char *, int, size_t);
IMPL (builtin_memset, 0)
-IMPL (memset, 1)
+# endif /* !WIDE */
+IMPL (MEMSET, 1)
+# ifndef WIDE
char *
builtin_memset (char *s, int c, size_t n)
{
return __builtin_memset (s, c, n);
}
-#endif
+# endif /* !WIDE */
+#endif /* !TEST_BZERO */
-char *
+CHAR *
inhibit_loop_to_libcall
-simple_memset (char *s, int c, size_t n)
+SIMPLE_MEMSET (CHAR *s, int c, size_t n)
{
- char *r = s, *end = s + n;
+ CHAR *r = s, *end = s + n;
while (r < end)
*r++ = c;
return s;
}
static void
-do_one_test (impl_t *impl, char *s, int c __attribute ((unused)), size_t n)
+do_one_test (impl_t *impl, CHAR *s, int c __attribute ((unused)), size_t n)
{
size_t i, iters = INNER_LOOP_ITERS;
timing_t start, stop, cur;
- char tstbuf[n];
+ CHAR tstbuf[n];
#ifdef TEST_BZERO
simple_bzero (tstbuf, n);
CALL (impl, s, n);
if (memcmp (s, tstbuf, n) != 0)
#else
- char *res = CALL (impl, s, c, n);
+ CHAR *res = CALL (impl, s, c, n);
if (res != s
- || simple_memset (tstbuf, c, n) != tstbuf
- || memcmp (s, tstbuf, n) != 0)
-#endif
+ || SIMPLE_MEMSET (tstbuf, c, n) != tstbuf
+ || MEMCMP (s, tstbuf, n) != 0)
+#endif /* !TEST_BZERO */
{
error (0, 0, "Wrong result in function %s", impl->name);
ret = 1;
@@ -101,7 +122,7 @@ do_one_test (impl_t *impl, char *s, int c __attribute ((unused)), size_t n)
CALL (impl, s, n);
#else
CALL (impl, s, c, n);
-#endif
+#endif /* !TEST_BZERO */
}
TIMING_NOW (stop);
@@ -114,13 +135,13 @@ static void
do_test (size_t align, int c, size_t len)
{
align &= 7;
- if (align + len > page_size)
+ if ((align + len) * sizeof (CHAR) > page_size)
return;
printf ("Length %4zd, alignment %2zd, c %2d:", len, align, c);
FOR_EACH_IMPL (impl, 0)
- do_one_test (impl, (char *) buf1 + align, c, len);
+ do_one_test (impl, (CHAR *) (buf1) + align, c, len);
putchar ('\n');
}
diff --git a/benchtests/bench-wmemset.c b/benchtests/bench-wmemset.c
new file mode 100644
index 0000000..540829c
--- /dev/null
+++ b/benchtests/bench-wmemset.c
@@ -0,0 +1,20 @@
+/* Measure wmemset functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-memset.c"
diff --git a/string/test-memset.c b/string/test-memset.c
index 9f3af46..79b2983 100644
--- a/string/test-memset.c
+++ b/string/test-memset.c
@@ -1,4 +1,4 @@
-/* Test and measure memset functions.
+/* Test memset functions.
Copyright (C) 1999-2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
Written by Jakub Jelinek <jakub@redhat.com>, 1999.
@@ -21,12 +21,33 @@
#ifdef TEST_BZERO
# define TEST_NAME "bzero"
#else
-# define TEST_NAME "memset"
-#endif
+# ifndef WIDE
+# define TEST_NAME "memset"
+# else
+# define TEST_NAME "wmemset"
+# endif /* WIDE */
+#endif /* !TEST_BZERO */
#define MIN_PAGE_SIZE 131072
#include "test-string.h"
-char *simple_memset (char *, int, size_t);
+#ifndef WIDE
+# define MEMSET memset
+# define CHAR char
+# define UCHAR unsigned char
+# define SIMPLE_MEMSET simple_memset
+# define MEMCMP memcmp
+# define BIG_CHAR CHAR_MAX
+#else
+# include <wchar.h>
+# define MEMSET wmemset
+# define CHAR wchar_t
+# define UCHAR wchar_t
+# define SIMPLE_MEMSET simple_wmemset
+# define MEMCMP wmemcmp
+# define BIG_CHAR WCHAR_MAX
+#endif /* WIDE */
+
+CHAR *SIMPLE_MEMSET (CHAR *, int, size_t);
#ifdef TEST_BZERO
typedef void (*proto_t) (char *, size_t);
@@ -40,7 +61,7 @@ IMPL (bzero, 1)
void
simple_bzero (char *s, size_t n)
{
- simple_memset (s, 0, n);
+ SIMPLE_MEMSET (s, 0, n);
}
void
@@ -49,44 +70,48 @@ builtin_bzero (char *s, size_t n)
__builtin_bzero (s, n);
}
#else
-typedef char *(*proto_t) (char *, int, size_t);
-char *builtin_memset (char *, int, size_t);
+typedef CHAR *(*proto_t) (CHAR *, int, size_t);
-IMPL (simple_memset, 0)
+IMPL (SIMPLE_MEMSET, 0)
+# ifndef WIDE
+char *builtin_memset (char *, int, size_t);
IMPL (builtin_memset, 0)
-IMPL (memset, 1)
+# endif /* !WIDE */
+IMPL (MEMSET, 1)
+# ifndef WIDE
char *
builtin_memset (char *s, int c, size_t n)
{
return __builtin_memset (s, c, n);
}
-#endif
+# endif /* !WIDE */
+#endif /* !TEST_BZERO */
-char *
+CHAR *
inhibit_loop_to_libcall
-simple_memset (char *s, int c, size_t n)
+SIMPLE_MEMSET (CHAR *s, int c, size_t n)
{
- char *r = s, *end = s + n;
+ CHAR *r = s, *end = s + n;
while (r < end)
*r++ = c;
return s;
}
static void
-do_one_test (impl_t *impl, char *s, int c __attribute ((unused)), size_t n)
+do_one_test (impl_t *impl, CHAR *s, int c __attribute ((unused)), size_t n)
{
- char tstbuf[n];
+ CHAR tstbuf[n];
#ifdef TEST_BZERO
simple_bzero (tstbuf, n);
CALL (impl, s, n);
if (memcmp (s, tstbuf, n) != 0)
#else
- char *res = CALL (impl, s, c, n);
+ CHAR *res = CALL (impl, s, c, n);
if (res != s
- || simple_memset (tstbuf, c, n) != tstbuf
- || memcmp (s, tstbuf, n) != 0)
-#endif
+ || SIMPLE_MEMSET (tstbuf, c, n) != tstbuf
+ || MEMCMP (s, tstbuf, n) != 0)
+#endif /* !TEST_BZERO */
{
error (0, 0, "Wrong result in function %s", impl->name);
ret = 1;
@@ -98,11 +123,11 @@ static void
do_test (size_t align, int c, size_t len)
{
align &= 7;
- if (align + len > page_size)
+ if ((align + len) * sizeof (CHAR) > page_size)
return;
FOR_EACH_IMPL (impl, 0)
- do_one_test (impl, (char *) buf1 + align, c, len);
+ do_one_test (impl, (CHAR *) (buf1) + align, c, len);
}
#ifndef TEST_BZERO
@@ -111,18 +136,19 @@ do_random_tests (void)
{
size_t i, j, k, n, align, len, size;
int c, o;
- unsigned char *p, *res;
+ UCHAR *p, *res;
+ UCHAR *p2 = (UCHAR *) buf2;
- for (i = 0; i < 65536; ++i)
- buf2[i] = random () & 255;
+ for (i = 0; i < 65536 / sizeof (CHAR); ++i)
+ p2[i] = random () & BIG_CHAR;
for (n = 0; n < ITERATIONS; n++)
{
if ((random () & 31) == 0)
- size = 65536;
+ size = 65536 / sizeof (CHAR);
else
size = 512;
- p = buf1 + page_size - size;
+ p = (UCHAR *) (buf1 + page_size) - size;
len = random () & (size - 1);
align = size - len - (random () & 31);
if (align > size)
@@ -132,10 +158,10 @@ do_random_tests (void)
if ((random () & 7) == 0)
c = 0;
else
- c = random () & 255;
- o = random () & 255;
+ c = random () & BIG_CHAR;
+ o = random () & BIG_CHAR;
if (o == c)
- o = (c + 1) & 255;
+ o = (c + 1) & BIG_CHAR;
j = len + align + 128;
if (j > size)
j = size;
@@ -152,11 +178,11 @@ do_random_tests (void)
{
for (i = 0; i < len; ++i)
{
- p[i + align] = buf2[i];
+ p[i + align] = p2[i];
if (p[i + align] == c)
p[i + align] = o;
}
- res = (unsigned char *) CALL (impl, (char *) p + align, c, len);
+ res = (UCHAR *) CALL (impl, (CHAR *) p + align, c, len);
if (res != p + align)
{
error (0, 0, "Iteration %zd - wrong result in function %s (%zd, %d, %zd) %p != %p",
@@ -190,7 +216,7 @@ do_random_tests (void)
}
}
}
-#endif
+#endif /* !TEST_BZERO */
int
test_main (void)
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 87dff0f..eac88e0 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -37,5 +37,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsspn wcsspn-vx wcsspn-c \
wcspbrk wcspbrk-vx wcspbrk-c \
wcscspn wcscspn-vx wcscspn-c \
- wmemchr wmemchr-vx wmemchr-c
+ wmemchr wmemchr-vx wmemchr-c \
+ wmemset wmemset-vx wmemset-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index c90fb6b..93789ac 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -133,6 +133,7 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (memccpy);
+ IFUNC_VX_IMPL (wmemset);
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/wcsmbs/wmemset.c b/sysdeps/s390/multiarch/wmemset-c.c
similarity index 50%
copy from wcsmbs/wmemset.c
copy to sysdeps/s390/multiarch/wmemset-c.c
index 88fc015..cec2339 100644
--- a/wcsmbs/wmemset.c
+++ b/sysdeps/s390/multiarch/wmemset-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1996-2015 Free Software Foundation, Inc.
+/* Default wmemset implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.org>, 1996.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,42 +16,22 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
-
-
-wchar_t *
-__wmemset (s, c, n)
- wchar_t *s;
- wchar_t c;
- size_t n;
-{
- wchar_t *wp = s;
-
- while (n >= 4)
- {
- wp[0] = c;
- wp[1] = c;
- wp[2] = c;
- wp[3] = c;
- wp += 4;
- n -= 4;
- }
-
- if (n > 0)
- {
- wp[0] = c;
-
- if (n > 1)
- {
- wp[1] = c;
-
- if (n > 2)
- wp[2] = c;
- }
- }
-
- return s;
-}
-libc_hidden_def (__wmemset)
-weak_alias (__wmemset, wmemset)
-libc_hidden_weak (wmemset)
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WMEMSET __wmemset_c
+
+# include <wchar.h>
+extern __typeof (__wmemset) __wmemset_c;
+# undef weak_alias
+# define weak_alias(name, alias)
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__wmemset_c, __GI___wmemset, __wmemset_c);
+# undef libc_hidden_weak
+# define libc_hidden_weak(name) \
+ strong_alias (__wmemset_c, __wmemset_c_1); \
+ __hidden_ver1 (__wmemset_c_1, __GI_wmemset, __wmemset_c_1);
+# endif /* SHARED */
+
+# include <wcsmbs/wmemset.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wmemset-vx.S b/sysdeps/s390/multiarch/wmemset-vx.S
new file mode 100644
index 0000000..22bddcd
--- /dev/null
+++ b/sysdeps/s390/multiarch/wmemset-vx.S
@@ -0,0 +1,142 @@
+/* Vector Optimized 32/64 bit S/390 version of wmemset.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t *wmemset(wchar_t *dest, wchar_t wc, size_t n)
+ Fill an array of wide-characters with a constant wide character
+ and returns dest.
+
+ Register usage:
+ -r0=tmp
+ -r1=tmp
+ -r2=dest or current-pointer
+ -r3=wc
+ -r4=n
+ -r5=tmp
+ -v16=replicated wc
+ -v17,v18,v19=copy of v16 for vstm
+ -v31=saved dest for return
+*/
+ENTRY(__wmemset_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ vlvgg %v31,%r2,0 /* Save destination pointer for return. */
+ clgije %r4,0,.Lend
+
+ vlvgf %v16,%r3,0 /* Generate vector with wchar_t wc. */
+ vrepf %v16,%v16,0
+
+ /* Check range of maxlen and convert to byte-count. */
+# ifdef __s390x__
+ tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ lghi %r5,-4 /* Max byte-count is 18446744073709551612. */
+# else
+ tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ llilf %r5,4294967292 /* Max byte-count is 4294967292. */
+# endif /* !__s390x__ */
+ sllg %r4,%r4,2 /* Convert character-count to byte-count. */
+ locgrne %r4,%r5 /* Use max byte-count, if bit 0/1 was one. */
+
+ /* Align dest to 16 byte. */
+ risbg %r0,%r2,60,128+63,0 /* Test if s is aligned and
+ %r3 = bits 60-63 'and' 15. */
+ je .Lpreloop /* If s is aligned, loop aligned. */
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+ lghi %r1,16
+ slr %r1,%r0 /* Compute byte count to load (16-x). */
+ clgr %r1,%r4
+ locgrh %r1,%r4 /* min (byte count, n) */
+ aghik %r5,%r1,-1 /* vstl needs highest index. */
+ vstl %v16,%r5,0(%r2) /* Store remaining bytes. */
+ clgrje %r1,%r4,.Lend /* Return if n bytes where set. */
+ slgr %r4,%r1 /* Compute remaining byte count. */
+ la %r2,0(%r1,%r2)
+
+.Lpreloop:
+ /* Now we are 16-byte aligned. */
+ clgijl %r4,17,.Lremaining
+ srlg %r1,%r4,8 /* Split into 256byte blocks */
+ clgije %r1,0,.Lpreloop64
+ vlr %v17,%v16
+ vlr %v18,%v16
+ vlr %v19,%v16
+
+.Lloop256:
+ vstm %v16,%v19,0(%r2)
+ vstm %v16,%v19,64(%r2)
+ vstm %v16,%v19,128(%r2)
+ vstm %v16,%v19,192(%r2)
+ la %r2,256(%r2)
+ brctg %r1,.Lloop256 /* Loop until all blocks are processed. */
+
+ llgfr %r4,%r4
+ nilf %r4,255 /* Get remaining bytes */
+ je .Lend /* Skip store remaining bytes if zero. */
+
+.Lpreloop64:
+ clgijl %r4,17,.Lremaining
+ clgijl %r4,33,.Lpreloop16
+ srlg %r1,%r4,5 /* Split into 32byte blocks */
+
+.Lloop32:
+ vst %v16,0(%r2)
+ vst %v16,16(%r2)
+ la %r2,32(%r2)
+ brctg %r1,.Lloop32 /* Loop until all blocks are processed. */
+
+ llgfr %r4,%r4
+ nilf %r4,31 /* Get remaining bytes */
+ je .Lend /* Skip store remaining bytes if zero. */
+
+.Lpreloop16:
+ clgijl %r4,17,.Lremaining
+ srlg %r1,%r4,4 /* Split into 16byte blocks */
+
+.Lloop16:
+ vst %v16,0(%r2)
+ la %r2,16(%r2)
+ brctg %r1,.Lloop16 /* Loop until all blocks are processed. */
+
+ llgfr %r4,%r4
+ nilf %r4,15 /* Get remaining bytes */
+ je .Lend /* Skip store remaining bytes if zero. */
+
+.Lremaining:
+ aghi %r4,-1 /* vstl needs highest index. */
+ vstl %v16,%r4,0(%r2)
+
+.Lend:
+ vlgvg %r2,%v31,0 /* Load saved dest for return value. */
+ br %r14
+.Lfallback:
+ srlg %r4,%r4,2 /* Convert byte-count to character-count. */
+ jg __wmemset_c
+END(__wmemset_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wmemset.c b/sysdeps/s390/multiarch/wmemset.c
similarity index 60%
copy from wcsmbs/wmemset.c
copy to sysdeps/s390/multiarch/wmemset.c
index 88fc015..884fea9 100644
--- a/wcsmbs/wmemset.c
+++ b/sysdeps/s390/multiarch/wmemset.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1996-2015 Free Software Foundation, Inc.
+/* Multiple versions of wmemset.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.org>, 1996.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,42 +16,14 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
-
-wchar_t *
-__wmemset (s, c, n)
- wchar_t *s;
- wchar_t c;
- size_t n;
-{
- wchar_t *wp = s;
-
- while (n >= 4)
- {
- wp[0] = c;
- wp[1] = c;
- wp[2] = c;
- wp[3] = c;
- wp += 4;
- n -= 4;
- }
-
- if (n > 0)
- {
- wp[0] = c;
-
- if (n > 1)
- {
- wp[1] = c;
-
- if (n > 2)
- wp[2] = c;
- }
- }
-
- return s;
-}
-libc_hidden_def (__wmemset)
+s390_vx_libc_ifunc (__wmemset)
weak_alias (__wmemset, wmemset)
libc_hidden_weak (wmemset)
+
+#else
+# include <wcsmbs/wmemset.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index e8ce579..dc72ba4 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -44,7 +44,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul wcsspn wcspbrk \
- wcscspn wmemchr
+ wcscspn wmemchr wmemset
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/test-wmemset.c b/wcsmbs/test-wmemset.c
new file mode 100644
index 0000000..c03c759
--- /dev/null
+++ b/wcsmbs/test-wmemset.c
@@ -0,0 +1,20 @@
+/* Test wmemset functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "../string/test-memset.c"
diff --git a/wcsmbs/wmemset.c b/wcsmbs/wmemset.c
index 88fc015..1eb6b2b 100644
--- a/wcsmbs/wmemset.c
+++ b/wcsmbs/wmemset.c
@@ -18,6 +18,9 @@
#include <wchar.h>
+#ifdef WMEMSET
+# define __wmemset WMEMSET
+#endif
wchar_t *
__wmemset (s, c, n)
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=9b593dc3055d44a4179c03050be58a437ae385a1
commit 9b593dc3055d44a4179c03050be58a437ae385a1
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:25 2015 +0200
S390: Optimize memccpy.
This patch provides optimized versions of memccpy with the z13 vector
instructions.
ChangeLog:
* sysdeps/s390/multiarch/memccpy-c.c: New File.
* sysdeps/s390/multiarch/memccpy-vx.S: Likewise.
* sysdeps/s390/multiarch/memccpy.c: Likewise.
* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Add memccpy functions.
* sysdeps/s390/multiarch/ifunc-impl-list-common.c
(__libc_ifunc_impl_list_common): Add ifunc test for memccpy.
* string/memccpy.c: Use MEMCCPY if defined.
diff --git a/ChangeLog b/ChangeLog
index c6df132..3354f1d 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,16 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/memccpy-c.c: New File.
+ * sysdeps/s390/multiarch/memccpy-vx.S: Likewise.
+ * sysdeps/s390/multiarch/memccpy.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile
+ (sysdep_routines): Add memccpy functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list-common.c
+ (__libc_ifunc_impl_list_common): Add ifunc test for memccpy.
+ * string/memccpy.c: Use MEMCCPY if defined.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/memchr-vx.S: New File.
* sysdeps/s390/multiarch/memchr.c: Likewise.
* sysdeps/s390/multiarch/rawmemchr-c.c: Likewise.
diff --git a/string/memccpy.c b/string/memccpy.c
index 0987c84..f10dafd 100644
--- a/string/memccpy.c
+++ b/string/memccpy.c
@@ -20,6 +20,10 @@
#undef __memccpy
#undef memccpy
+#ifdef MEMCCPY
+# define __memccpy MEMCCPY
+#endif
+
/* Copy no more than N bytes of SRC to DEST, stopping when C is found.
Return the position in DEST one byte past where C was copied, or
NULL if C was not found in the first N bytes of SRC. */
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 4a04c34..87dff0f 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -16,7 +16,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strpbrk strpbrk-vx strpbrk-c \
strcspn strcspn-vx strcspn-c \
memchr memchr-vx \
- rawmemchr rawmemchr-vx rawmemchr-c
+ rawmemchr rawmemchr-vx rawmemchr-c \
+ memccpy memccpy-vx memccpy-c
endif
ifeq ($(subdir),wcsmbs)
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index d4c7c0d..c90fb6b 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -131,6 +131,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (wmemchr);
IFUNC_VX_IMPL (rawmemchr);
+ IFUNC_VX_IMPL (memccpy);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/string/memccpy.c b/sysdeps/s390/multiarch/memccpy-c.c
similarity index 57%
copy from string/memccpy.c
copy to sysdeps/s390/multiarch/memccpy-c.c
index 0987c84..d034d23 100644
--- a/string/memccpy.c
+++ b/sysdeps/s390/multiarch/memccpy-c.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Default memccpy implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,24 +16,10 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define MEMCCPY __memccpy_c
-#undef __memccpy
-#undef memccpy
-
-/* Copy no more than N bytes of SRC to DEST, stopping when C is found.
- Return the position in DEST one byte past where C was copied, or
- NULL if C was not found in the first N bytes of SRC. */
-void *
-__memccpy (void *dest, const void *src, int c, size_t n)
-{
- void *p = memchr (src, c, n);
-
- if (p != NULL)
- return __mempcpy (dest, src, p - src + 1);
-
- memcpy (dest, src, n);
- return NULL;
-}
-
-weak_alias (__memccpy, memccpy)
+# include <string.h>
+extern __typeof (__memccpy) __memccpy_c;
+# include <string/memccpy.c>
+#endif
diff --git a/sysdeps/s390/multiarch/memccpy-vx.S b/sysdeps/s390/multiarch/memccpy-vx.S
new file mode 100644
index 0000000..5c662d3
--- /dev/null
+++ b/sysdeps/s390/multiarch/memccpy-vx.S
@@ -0,0 +1,156 @@
+/* Vector optimized 32/64 bit S/390 version of memccpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* void *memccpy (void * dest, const void *src, int c, size_t n)
+ Copies no more than n bytes from src to dest,
+ stopping when the character c is found
+ and returns pointer next to c in dest or null if c not found.
+
+ Register usage:
+ -r0=tmp
+ -r1=tmp
+ -r2=dest
+ -r3=src
+ -r4=c
+ -r5=n
+ -r6=current_len
+ -v16=part of s
+ -v17=index of found c
+ -v18=c replicated
+ -v19=part #2 of s
+ -v31=save area for r6
+*/
+ENTRY(__memccpy_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r5,%r5
+# endif /* !defined __s390x__ */
+
+ vlvgp %v31,%r6,%r7 /* Save registers. */
+ clgije %r5,0,.Lnf_end /* If len == 0 then exit. */
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r0,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r0,%r0 /* Convert 32bit to 64bit. */
+
+ vlvgb %v18,%r4,0 /* Generate vector which elements are all c.
+ if c > 255, c will be truncated. */
+ vrepb %v18,%v18,0
+ lghi %r6,0 /* current_len = 0. */
+
+ clgrjle %r5,%r0,.Lremaining_v16 /* If maxlen <= loaded-bytes
+ -> Process remaining. */
+
+ vfeebs %v17,%v16,%v18 /* Find c. */
+ vlgvb %r1,%v17,7 /* Load byte index of c. */
+ clgrjl %r1,%r0,.Lfound_v16 /* Found c is within loaded bytes. */
+
+ /* Align s to 16 byte. */
+ risbgn %r1,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r6,15 /* current_len = 15. */
+ slr %r6,%r1 /* Compute highest index to 16byte boundary. */
+
+ vstl %v16,%r6,0(%r2) /* Store prcessed bytes */
+ ahi %r6,1
+
+.Lpreloop1:
+ /* Now we are 16byte aligned, so we can load
+ a full vreg without page fault. */
+ vl %v16,0(%r6,%r3) /* Load s. */
+ clgijl %r5,17,.Lremaining_v16 /* If n <= 16,
+ process remaining bytes. */
+ lgr %r7,%r5
+ slgfi %r7,16 /* border_len = n - 16. */
+ j .Lloop1
+
+.Lloop2:
+ vl %v16,16(%r6,%r3)
+ vst %v19,0(%r6,%r2)
+ aghi %r6,16
+
+.Lloop1:
+ clgrjhe %r6,%r7,.Lremaining_v16 /* If current_len >= border
+ then process remaining bytes. */
+ vfeebs %v17,%v16,%v18 /* Find c. */
+ jl .Lfound_v16 /* Jump away if c was found. */
+ vl %v19,16(%r6,%r3) /* Load next s part. */
+ vst %v16,0(%r6,%r2) /* Store previous part without c. */
+ aghi %r6,16
+
+ clgrjhe %r6,%r7,.Lremaining_v19
+ vfeebs %v17,%v19,%v18
+ jl .Lfound_v19
+ vl %v16,16(%r6,%r3)
+ vst %v19,0(%r6,%r2)
+ aghi %r6,16
+
+ clgrjhe %r6,%r7,.Lremaining_v16
+ vfeebs %v17,%v16,%v18
+ jl .Lfound_v16
+ vl %v19,16(%r6,%r3)
+ vst %v16,0(%r6,%r2)
+ aghi %r6,16
+
+ clgrjhe %r6,%r7,.Lremaining_v19
+ vfeebs %v17,%v19,%v18
+ jo .Lloop2
+
+.Lfound_v19:
+ vlr %v16,%v19
+.Lfound_v16:
+ /* v16 contains c. Store remaining bytes to c. currlen hasn´t
+ reached border, thus checking for maxlen is not needed! */
+ vlgvb %r1,%v17,7 /* Load byte index of c. */
+ la %r2,0(%r6,%r2) /* vstl has no support for index-register. */
+.Lfound_v16_store:
+ vstl %v16,%r1,0(%r2) /* Copy bytes including c. */
+ la %r2,1(%r1,%r2) /* Return pointer next to c in dest. */
+ vlgvg %r6,%v31,0
+ vlgvg %r7,%v31,1
+ br %r14
+
+.Lremaining_v19:
+ vlr %v16,%v19
+.Lremaining_v16:
+ /* v16 contains the remaining bytes [1...16].
+ Check and store remaining bytes. */
+ vfeebs %v17,%v16,%v18
+ slgrk %r7,%r5,%r6 /* Remaining bytes = maxlen - current_len. */
+ aghi %r7,-1 /* vstl needs highest index. */
+ la %r2,0(%r6,%r2) /* vstl has no index register. */
+ vlgvb %r1,%v17,7 /* Load index of c or 16 if not found. */
+ /* c in remaining bytes? -> Jump away (c-index <= max-index) */
+ clrjle %r1,%r7,.Lfound_v16_store
+ vstl %v16,%r7,0(%r2) /* Store remaining bytes. */
+
+.Lnf_end:
+ vlgvg %r6,%v31,0
+ vlgvg %r7,%v31,1
+ lghi %r2,0 /* Return null. */
+ br %r14
+END(__memccpy_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/string/memccpy.c b/sysdeps/s390/multiarch/memccpy.c
similarity index 59%
copy from string/memccpy.c
copy to sysdeps/s390/multiarch/memccpy.c
index 0987c84..c5a9628 100644
--- a/string/memccpy.c
+++ b/sysdeps/s390/multiarch/memccpy.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Multiple versions of memccpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,24 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
-
-#undef __memccpy
-#undef memccpy
-
-/* Copy no more than N bytes of SRC to DEST, stopping when C is found.
- Return the position in DEST one byte past where C was copied, or
- NULL if C was not found in the first N bytes of SRC. */
-void *
-__memccpy (void *dest, const void *src, int c, size_t n)
-{
- void *p = memchr (src, c, n);
-
- if (p != NULL)
- return __mempcpy (dest, src, p - src + 1);
-
- memcpy (dest, src, n);
- return NULL;
-}
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+s390_vx_libc_ifunc (__memccpy)
weak_alias (__memccpy, memccpy)
+
+#else
+# include <string/memccpy.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=88eefd344b3cf4a41284a1dfdaca61667e3a1b4b
commit 88eefd344b3cf4a41284a1dfdaca61667e3a1b4b
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:24 2015 +0200
S390: Optimize memchr, rawmemchr and wmemchr.
This patch provides optimized versions of memchr, rawmemchr and wmemchr with the
z13 vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/memchr-vx.S: New File.
* sysdeps/s390/multiarch/memchr.c: Likewise.
* sysdeps/s390/multiarch/rawmemchr-c.c: Likewise.
* sysdeps/s390/multiarch/rawmemchr-vx.S: Likewise.
* sysdeps/s390/multiarch/rawmemchr.c: Likewise.
* sysdeps/s390/multiarch/wmemchr-c.c: Likewise.
* sysdeps/s390/multiarch/wmemchr-vx.S: Likewise.
* sysdeps/s390/multiarch/wmemchr.c: Likewise.
* sysdeps/s390/s390-32/multiarch/memchr.c: Likewise.
* sysdeps/s390/s390-64/multiarch/memchr.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add memchr, wmemchr
and rawmemchr functions.
* sysdeps/s390/multiarch/ifunc-impl-list-common.c
(__libc_ifunc_impl_list_common): Add ifunc test for memchr, rawmemchr
and wmemchr.
* wcsmbs/wmemchr.c: Use WMEMCHR if defined.
* string/test-memchr.c: Add wmemchr support.
* wcsmbs/test-wmemchr.c: New File.
* wcsmbs/Makefile (strop-tests): Add wmemchr.
* benchtests/bench-memchr.c: Add wmemchr support.
* benchtests/bench-wmemchr.c: New File.
* benchtests/Makefile (wcsmbs-bench): wmemchr.
diff --git a/ChangeLog b/ChangeLog
index 4502084..c6df132 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,30 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/memchr-vx.S: New File.
+ * sysdeps/s390/multiarch/memchr.c: Likewise.
+ * sysdeps/s390/multiarch/rawmemchr-c.c: Likewise.
+ * sysdeps/s390/multiarch/rawmemchr-vx.S: Likewise.
+ * sysdeps/s390/multiarch/rawmemchr.c: Likewise.
+ * sysdeps/s390/multiarch/wmemchr-c.c: Likewise.
+ * sysdeps/s390/multiarch/wmemchr-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wmemchr.c: Likewise.
+ * sysdeps/s390/s390-32/multiarch/memchr.c: Likewise.
+ * sysdeps/s390/s390-64/multiarch/memchr.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add memchr, wmemchr
+ and rawmemchr functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list-common.c
+ (__libc_ifunc_impl_list_common): Add ifunc test for memchr, rawmemchr
+ and wmemchr.
+ * wcsmbs/wmemchr.c: Use WMEMCHR if defined.
+ * string/test-memchr.c: Add wmemchr support.
+ * wcsmbs/test-wmemchr.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wmemchr.
+ * benchtests/bench-memchr.c: Add wmemchr support.
+ * benchtests/bench-wmemchr.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): wmemchr.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strcspn-c.c: New File.
* sysdeps/s390/multiarch/strcspn-vx.S: Likewise.
* sysdeps/s390/multiarch/strcspn.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 46cb34f..95712a3 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -37,7 +37,8 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
- wcscmp wcsncmp wcschr wcschrnul wcsrchr wcsspn wcspbrk wcscspn
+ wcscmp wcsncmp wcschr wcschrnul wcsrchr wcsspn wcspbrk wcscspn \
+ wmemchr
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-memchr.c b/benchtests/bench-memchr.c
index 9879ec8..3ead3cd 100644
--- a/benchtests/bench-memchr.c
+++ b/benchtests/bench-memchr.c
@@ -16,31 +16,52 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
+#ifndef WIDE
+# define CHAR char
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define CHAR wchar_t
+# define SMALL_CHAR 1273
+#endif /* WIDE */
+
#ifndef USE_AS_MEMRCHR
# define TEST_MAIN
-# define TEST_NAME "memchr"
+# ifndef WIDE
+# define TEST_NAME "memchr"
+# else
+# define TEST_NAME "wmemchr"
+# endif /* WIDE */
# include "bench-string.h"
-typedef char *(*proto_t) (const char *, int, size_t);
-char *simple_memchr (const char *, int, size_t);
+# ifndef WIDE
+# define MEMCHR memchr
+# define SIMPLE_MEMCHR simple_memchr
+# else
+# define MEMCHR wmemchr
+# define SIMPLE_MEMCHR simple_wmemchr
+# endif /* WIDE */
+
+typedef CHAR *(*proto_t) (const CHAR *, int, size_t);
+CHAR *SIMPLE_MEMCHR (const CHAR *, int, size_t);
-IMPL (simple_memchr, 0)
-IMPL (memchr, 1)
+IMPL (SIMPLE_MEMCHR, 0)
+IMPL (MEMCHR, 1)
-char *
-simple_memchr (const char *s, int c, size_t n)
+CHAR *
+SIMPLE_MEMCHR (const CHAR *s, int c, size_t n)
{
while (n--)
- if (*s++ == (char) c)
- return (char *) s - 1;
+ if (*s++ == (CHAR) c)
+ return (CHAR *) s - 1;
return NULL;
}
-#endif
+#endif /* !USE_AS_MEMRCHR */
static void
-do_one_test (impl_t *impl, const char *s, int c, size_t n, char *exp_res)
+do_one_test (impl_t *impl, const CHAR *s, int c, size_t n, CHAR *exp_res)
{
- char *res = CALL (impl, s, c, n);
+ CHAR *res = CALL (impl, s, c, n);
size_t i, iters = INNER_LOOP_ITERS;
timing_t start, stop, cur;
@@ -68,36 +89,38 @@ static void
do_test (size_t align, size_t pos, size_t len, int seek_char)
{
size_t i;
- char *result;
+ CHAR *result;
align &= 7;
- if (align + len >= page_size)
+ if ((align + len) * sizeof (CHAR) >= page_size)
return;
+ CHAR *buf = (CHAR *) (buf1);
+
for (i = 0; i < len; ++i)
{
- buf1[align + i] = 1 + 23 * i % 127;
- if (buf1[align + i] == seek_char)
- buf1[align + i] = seek_char + 1;
+ buf[align + i] = 1 + 23 * i % SMALL_CHAR;
+ if (buf[align + i] == seek_char)
+ buf[align + i] = seek_char + 1;
}
- buf1[align + len] = 0;
+ buf[align + len] = 0;
if (pos < len)
{
- buf1[align + pos] = seek_char;
- buf1[align + len] = -seek_char;
- result = (char *) (buf1 + align + pos);
+ buf[align + pos] = seek_char;
+ buf[align + len] = -seek_char;
+ result = (CHAR *) (buf + align + pos);
}
else
{
result = NULL;
- buf1[align + len] = seek_char;
+ buf[align + len] = seek_char;
}
printf ("Length %4zd, alignment %2zd:", pos, align);
FOR_EACH_IMPL (impl, 0)
- do_one_test (impl, (char *) (buf1 + align), seek_char, len, result);
+ do_one_test (impl, (CHAR *) (buf + align), seek_char, len, result);
putchar ('\n');
}
diff --git a/benchtests/bench-wmemchr.c b/benchtests/bench-wmemchr.c
new file mode 100644
index 0000000..d796a69
--- /dev/null
+++ b/benchtests/bench-wmemchr.c
@@ -0,0 +1,20 @@
+/* Measure wmemchr functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-memchr.c"
diff --git a/string/test-memchr.c b/string/test-memchr.c
index 19c8978..fe7d82f 100644
--- a/string/test-memchr.c
+++ b/string/test-memchr.c
@@ -1,4 +1,4 @@
-/* Test and measure memchr functions.
+/* Test memchr functions.
Copyright (C) 1999-2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
Written by Jakub Jelinek <jakub@redhat.com>, 1999.
@@ -18,28 +18,49 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "memchr"
+#ifndef WIDE
+# define TEST_NAME "memchr"
+#else
+# define TEST_NAME "wmemchr"
+#endif /* WIDE */
#include "test-string.h"
-typedef char *(*proto_t) (const char *, int, size_t);
-char *simple_memchr (const char *, int, size_t);
-
-IMPL (simple_memchr, 0)
-IMPL (memchr, 1)
-
-char *
-simple_memchr (const char *s, int c, size_t n)
+#ifndef WIDE
+# define MEMCHR memchr
+# define CHAR char
+# define UCHAR unsigned char
+# define SIMPLE_MEMCHR simple_memchr
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define MEMCHR wmemchr
+# define CHAR wchar_t
+# define UCHAR wchar_t
+# define SIMPLE_MEMCHR simple_wmemchr
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+#endif /* WIDE */
+
+typedef CHAR *(*proto_t) (const CHAR *, int, size_t);
+CHAR *SIMPLE_MEMCHR (const CHAR *, int, size_t);
+
+IMPL (SIMPLE_MEMCHR, 0)
+IMPL (MEMCHR, 1)
+
+CHAR *
+SIMPLE_MEMCHR (const CHAR *s, int c, size_t n)
{
while (n--)
- if (*s++ == (char) c)
- return (char *) s - 1;
+ if (*s++ == (CHAR) c)
+ return (CHAR *) s - 1;
return NULL;
}
static void
-do_one_test (impl_t *impl, const char *s, int c, size_t n, char *exp_res)
+do_one_test (impl_t *impl, const CHAR *s, int c, size_t n, CHAR *exp_res)
{
- char *res = CALL (impl, s, c, n);
+ CHAR *res = CALL (impl, s, c, n);
if (res != exp_res)
{
error (0, 0, "Wrong result in function %s %p %p", impl->name,
@@ -53,34 +74,36 @@ static void
do_test (size_t align, size_t pos, size_t len, int seek_char)
{
size_t i;
- char *result;
+ CHAR *result;
align &= 7;
- if (align + len >= page_size)
+ if ((align + len) * sizeof (CHAR) >= page_size)
return;
+ CHAR *buf = (CHAR *) (buf1);
+
for (i = 0; i < len; ++i)
{
- buf1[align + i] = 1 + 23 * i % 127;
- if (buf1[align + i] == seek_char)
- buf1[align + i] = seek_char + 1;
+ buf[align + i] = 1 + 23 * i % SMALL_CHAR;
+ if (buf[align + i] == seek_char)
+ buf[align + i] = seek_char + 1;
}
- buf1[align + len] = 0;
+ buf[align + len] = 0;
if (pos < len)
{
- buf1[align + pos] = seek_char;
- buf1[align + len] = -seek_char;
- result = (char *) (buf1 + align + pos);
+ buf[align + pos] = seek_char;
+ buf[align + len] = -seek_char;
+ result = (CHAR *) (buf + align + pos);
}
else
{
result = NULL;
- buf1[align + len] = seek_char;
+ buf[align + len] = seek_char;
}
FOR_EACH_IMPL (impl, 0)
- do_one_test (impl, (char *) (buf1 + align), seek_char, len, result);
+ do_one_test (impl, (CHAR *) (buf + align), seek_char, len, result);
}
static void
@@ -88,8 +111,8 @@ do_random_tests (void)
{
size_t i, j, n, align, pos, len;
int seek_char;
- char *result;
- unsigned char *p = buf1 + page_size - 512;
+ CHAR *result;
+ UCHAR *p = (UCHAR *) (buf1 + page_size) - 512;
for (n = 0; n < ITERATIONS; n++)
{
@@ -101,11 +124,11 @@ do_random_tests (void)
if (pos >= len)
len = pos + (random () & 7);
if (len + align >= 512)
- len = 512 - align - (random () & 7);
- seek_char = random () & 255;
+ len = 512 - align - (random () & 7);
+ seek_char = random () & BIG_CHAR;
j = len + align + 64;
if (j > 512)
- j = 512;
+ j = 512;
for (i = 0; i < j; i++)
{
@@ -113,7 +136,7 @@ do_random_tests (void)
p[i] = seek_char;
else
{
- p[i] = random () & 255;
+ p[i] = random () & BIG_CHAR;
if (i < pos + align && p[i] == seek_char)
p[i] = seek_char + 13;
}
@@ -124,17 +147,17 @@ do_random_tests (void)
size_t r = random ();
if ((r & 31) == 0)
len = ~(uintptr_t) (p + align) - ((r >> 5) & 31);
- result = (char *) (p + pos + align);
+ result = (CHAR *) (p + pos + align);
}
else
result = NULL;
FOR_EACH_IMPL (impl, 1)
- if (CALL (impl, (char *) (p + align), seek_char, len) != result)
+ if (CALL (impl, (CHAR *) (p + align), seek_char, len) != result)
{
error (0, 0, "Iteration %zd - wrong result in function %s (%zd, %d, %zd, %zd) %p != %p, p %p",
n, impl->name, align, seek_char, len, pos,
- CALL (impl, (char *) (p + align), seek_char, len),
+ CALL (impl, (CHAR *) (p + align), seek_char, len),
result, p);
ret = 1;
}
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 2c1fce0..4a04c34 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -14,7 +14,9 @@ sysdep_routines += strlen strlen-vx strlen-c \
strrchr strrchr-vx strrchr-c \
strspn strspn-vx strspn-c \
strpbrk strpbrk-vx strpbrk-c \
- strcspn strcspn-vx strcspn-c
+ strcspn strcspn-vx strcspn-c \
+ memchr memchr-vx \
+ rawmemchr rawmemchr-vx rawmemchr-c
endif
ifeq ($(subdir),wcsmbs)
@@ -33,5 +35,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsrchr wcsrchr-vx wcsrchr-c \
wcsspn wcsspn-vx wcsspn-c \
wcspbrk wcspbrk-vx wcspbrk-c \
- wcscspn wcscspn-vx wcscspn-c
+ wcscspn wcscspn-vx wcscspn-c \
+ wmemchr wmemchr-vx wmemchr-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 7f62e49..d4c7c0d 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -127,6 +127,10 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strcspn);
IFUNC_VX_IMPL (wcscspn);
+ IFUNC_VX_IMPL (memchr);
+ IFUNC_VX_IMPL (wmemchr);
+ IFUNC_VX_IMPL (rawmemchr);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/memchr-vx.S b/sysdeps/s390/multiarch/memchr-vx.S
new file mode 100644
index 0000000..e0aa9e6
--- /dev/null
+++ b/sysdeps/s390/multiarch/memchr-vx.S
@@ -0,0 +1,159 @@
+/* Vector optimized 32/64 bit S/390 version of memchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* void *memchr (const void *s, int c, size_t n)
+ Scans memory for character c
+ and returns pointer to first c.
+
+ Register usage:
+ -r0=tmp
+ -r1=tmp
+ -r2=s
+ -r3=c
+ -r4=n
+ -r5=current_len
+ -v16=part of s
+ -v17=index of found c
+ -v18=c replicated
+*/
+ENTRY(__memchr_vx)
+
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgije %r4,0,.Lnf_end /* If len == 0 then exit. */
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r0,%r0 /* Convert 32bit to 64bit. */
+
+ vlvgb %v18,%r3,0 /* Generate vector which elements are all c.
+ if c > 255, c will be truncated. */
+ vrepb %v18,%v18,0
+ lghi %r5,16 /* current_len = 16. */
+
+ clgrjhe %r0,%r4,.Llastcmp /* If (bytes to boundary) >= n,
+ jump to lastcmp. */
+
+ vfeebs %v17,%v16,%v18 /* Find c. */
+ vlgvb %r1,%v17,7 /* Load byte index of c. */
+ clgrjl %r1,%r0,.Lfound2 /* Found c is within loaded bytes. */
+
+ /* Align s to 16 byte. */
+ risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ slr %r5,%r1 /* Compute bytes to 16bytes boundary. */
+
+ lgr %r0,%r5 /* If %r5 + 64 < n? -> loop64. */
+ aghi %r0,64
+ clgrjl %r0,%r4,.Lloop64
+.Llt64:
+ vl %v16,0(%r5,%r2)
+ aghi %r5,16
+ clgrjhe %r5,%r4,.Llastcmp /* Do last compare if curr-len >= n. */
+ vfeebs %v17,%v16,%v18 /* Find c. */
+ jl .Lfound /* Jump away if c was found. */
+
+ vl %v16,0(%r5,%r2)
+ aghi %r5,16
+ clgrjhe %r5,%r4,.Llastcmp
+ vfeebs %v17,%v16,%v18
+ jl .Lfound
+
+ vl %v16,0(%r5,%r2)
+ aghi %r5,16
+ clgrjhe %r5,%r4,.Llastcmp
+ vfeebs %v17,%v16,%v18
+ jl .Lfound
+
+ vl %v16,0(%r5,%r2)
+ aghi %r5,16
+
+.Llastcmp:
+ /* Use comparision result only if located within first n characters.
+ %r5: current_len;
+ %r4: n;
+ (current_len - n): [0...16[
+ first ignored match index: vr-width - (current_len - n) ]0...16]
+ */
+ vfeebs %v17,%v16,%v18 /* Find c. */
+ slgrk %r4,%r5,%r4 /* %r5 = current_len - n. */
+ lghi %r0,16 /* Register width = 16. */
+ vlgvb %r1,%v17,7 /* Extract found index or 16 if all equal. */
+ slr %r0,%r4 /* %r0 = first ignored match index. */
+ clrjl %r1,%r0,.Lfound2 /* Go away if miscompare is below n bytes. */
+ /* c not found within n-bytes. */
+.Lnf_end:
+ lghi %r2,0 /* Return null. */
+ br %r14
+
+.Lfound48:
+ aghi %r5,16
+.Lfound32:
+ aghi %r5,16
+.Lfound16:
+ aghi %r5,16
+.Lfound0:
+ aghi %r5,16
+.Lfound:
+ vlgvb %r1,%v17,7 /* Load byte index of c. */
+.Lfound2:
+ slgfi %r5,16 /* current_len -=16 */
+ algr %r5,%r1 /* Zero byte index is added to current len. */
+ la %r2,0(%r5,%r2) /* Return pointer to c. */
+ br %r14
+
+
+.Lloop64:
+ vl %v16,0(%r5,%r2)
+ vfeebs %v17,%v16,%v18 /* Find c. */
+ jl .Lfound0 /* Jump away if c was found. */
+ vl %v16,16(%r5,%r2)
+ vfeebs %v17,%v16,%v18
+ jl .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfeebs %v17,%v16,%v18
+ jl .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfeebs %v17,%v16,%v18
+ jl .Lfound48
+
+ aghi %r5,64
+ lgr %r0,%r5 /* If %r5 + 64 < n? -> loop64. */
+ aghi %r0,64
+ clgrjl %r0,%r4,.Lloop64
+
+ j .Llt64
+END(__memchr_vx)
+
+# define memchr __memchr_c
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) strong_alias(__memchr_c, __GI_memchr)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+
+#include <memchr.S>
diff --git a/sysdeps/s390/multiarch/memchr.c b/sysdeps/s390/multiarch/memchr.c
new file mode 100644
index 0000000..b653c1d
--- /dev/null
+++ b/sysdeps/s390/multiarch/memchr.c
@@ -0,0 +1,24 @@
+/* Multiple versions of memchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__memchr, memchr)
+#endif
diff --git a/sysdeps/s390/multiarch/rawmemchr-c.c b/sysdeps/s390/multiarch/rawmemchr-c.c
new file mode 100644
index 0000000..a85e83f
--- /dev/null
+++ b/sysdeps/s390/multiarch/rawmemchr-c.c
@@ -0,0 +1,34 @@
+/* Default rawmemchr implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+
+# define RAWMEMCHR __rawmemchr_c
+# undef weak_alias
+# define weak_alias(a, b)
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__rawmemchr_c, __GI___rawmemchr, __rawmemchr_c);
+# endif /* SHARED */
+
+extern __typeof (rawmemchr) __rawmemchr_c attribute_hidden;
+
+# include <string/rawmemchr.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/multiarch/rawmemchr-vx.S b/sysdeps/s390/multiarch/rawmemchr-vx.S
new file mode 100644
index 0000000..0648296
--- /dev/null
+++ b/sysdeps/s390/multiarch/rawmemchr-vx.S
@@ -0,0 +1,92 @@
+/* Vector optimized 32/64 bit S/390 version of rawmemchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* void *rawmemchr (const void *s, int c)
+ Scans memory for character c
+ and returns pointer to first c.
+
+ Register usage:
+ -r1=tmp
+ -r2=s
+ -r3=c
+ -r4=tmp
+ -r5=current_len
+ -v16=part of s
+ -v17=index of unequal
+ -v18=c replicated
+*/
+ENTRY(__rawmemchr_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vlvgb %v18,%r3,0 /* Generate vector which elements are all c.
+ If c > 255, c will be truncated. */
+ vrepb %v18,%v18,0
+
+ vfeeb %v17,%v16,%v18 /* Vector find element equal. */
+ vlgvb %r5,%v17,7 /* Load byte index of character or zero. */
+ clrjl %r5,%r1,.Lend_found /* If found c is in loaded bytes, end. */
+
+ /* Align s to 16 byte. */
+ risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16
+ slr %r5,%r1 /* Compute bytes to 16bytes boundary. */
+
+ /* Find c in a 16byte aligned loop. */
+.Lloop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfeebs %v17,%v16,%v18 /* Vector find element equal. */
+ jno .Lcharacter /* Jump away if element found. */
+ vl %v16,16(%r5,%r2)
+ vfeebs %v17,%v16,%v18
+ jno .Lcharacter16
+ vl %v16,32(%r5,%r2)
+ vfeebs %v17,%v16,%v18
+ jno .Lcharacter32
+ vl %v16,48(%r5,%r2)
+ vfeebs %v17,%v16,%v18
+ jno .Lcharacter48
+
+ aghi %r5,64
+ j .Lloop /* No character found -> loop. */
+
+ /* Found character. */
+.Lcharacter48:
+ aghi %r5,16
+.Lcharacter32:
+ aghi %r5,16
+.Lcharacter16:
+ aghi %r5,16
+.Lcharacter:
+ vlgvb %r1,%v17,7 /* Load byte index of character. */
+ algr %r5,%r1
+.Lend_found:
+ la %r2,0(%r5,%r2) /* Return pointer to character. */
+ br %r14
+END(__rawmemchr_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/rawmemchr.c b/sysdeps/s390/multiarch/rawmemchr.c
new file mode 100644
index 0000000..f93064c
--- /dev/null
+++ b/sysdeps/s390/multiarch/rawmemchr.c
@@ -0,0 +1,28 @@
+/* Multiple versions of rawmemchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc (__rawmemchr)
+weak_alias (__rawmemchr, rawmemchr)
+
+#else
+# include <string/rawmemchr.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/multiarch/wmemchr-c.c b/sysdeps/s390/multiarch/wmemchr-c.c
new file mode 100644
index 0000000..c3b12ea
--- /dev/null
+++ b/sysdeps/s390/multiarch/wmemchr-c.c
@@ -0,0 +1,37 @@
+/* Default wmemchr implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WMEMCHR __wmemchr_c
+
+# include <wchar.h>
+extern __typeof (wmemchr) __wmemchr_c;
+# undef weak_alias
+# define weak_alias(name, alias)
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__wmemchr_c, __GI___wmemchr, __wmemchr_c);
+# undef libc_hidden_weak
+# define libc_hidden_weak(name) \
+ strong_alias (__wmemchr_c, __wmemchr_c_1); \
+ __hidden_ver1 (__wmemchr_c_1, __GI_wmemchr, __wmemchr_c_1);
+# endif /* SHARED */
+
+# include <wcsmbs/wmemchr.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wmemchr-vx.S b/sysdeps/s390/multiarch/wmemchr-vx.S
new file mode 100644
index 0000000..a32b2f2
--- /dev/null
+++ b/sysdeps/s390/multiarch/wmemchr-vx.S
@@ -0,0 +1,166 @@
+/* Vector optimized 32/64 bit S/390 version of wmemchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t *wmemchr (const wchar_t *s, wchar_t c, size_t n)
+ Scans memory for character c
+ and returns pointer to first c.
+
+ Register usage:
+ -r0=tmp
+ -r1=tmp
+ -r2=s
+ -r3=c
+ -r4=n
+ -r5=current_len
+ -v16=part of s
+ -v17=index of found c
+ -v18=c replicated
+*/
+ENTRY(__wmemchr_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgije %r4,0,.Lnf_end /* If len == 0 then exit. */
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r0,%r0 /* Convert 32bit to 64bit. */
+
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ vlvgf %v18,%r3,0 /* Generate vector which elements are all c. */
+ vrepf %v18,%v18,0
+ lghi %r5,16 /* current_len = 16. */
+
+ /* Check range of maxlen and convert to byte-count. */
+# ifdef __s390x__
+ tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ lghi %r1,-4 /* Max byte-count is 18446744073709551612. */
+# else
+ tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ llilf %r1,4294967292 /* Max byte-count is 4294967292. */
+# endif /* !__s390x__ */
+ sllg %r4,%r4,2 /* Convert character-count to byte-count. */
+ locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */
+
+ clgrjhe %r0,%r4,.Llastcmp /* If (bytes to boundary) >= n,
+ jump to lastcmp. */
+
+ vfeefs %v17,%v16,%v18 /* Find c. */
+ vlgvb %r1,%v17,7 /* Load byte index of c. */
+ clgrjl %r1,%r0,.Lfound2 /* Found c is within loaded bytes. */
+
+ /* Align s to 16 byte. */
+ risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ slr %r5,%r1 /* Compute bytes to 16bytes boundary. */
+
+ lgr %r0,%r5 /* If %r5 + 64 < n? -> loop64. */
+ aghi %r0,64
+ clgrjl %r0,%r4,.Lloop64
+.Llt64:
+ vl %v16,0(%r5,%r2)
+ aghi %r5,16
+ clgrjhe %r5,%r4,.Llastcmp /* Do last compare if curr-len >= n. */
+ vfeefs %v17,%v16,%v18 /* Find c. */
+ jl .Lfound /* Jump away if c was found. */
+
+ vl %v16,0(%r5,%r2)
+ aghi %r5,16
+ clgrjhe %r5,%r4,.Llastcmp
+ vfeefs %v17,%v16,%v18
+ jl .Lfound
+
+ vl %v16,0(%r5,%r2)
+ aghi %r5,16
+ clgrjhe %r5,%r4,.Llastcmp
+ vfeefs %v17,%v16,%v18
+ jl .Lfound
+
+ vl %v16,0(%r5,%r2)
+ aghi %r5,16
+
+.Llastcmp:
+ /* Use comparision result only if located within first n characters.
+ %r5: current_len;
+ %r4: n;
+ (current_len - n): [0...16[
+ first ignored match index = vr-width - (current_len - n) ]0...16]
+ */
+ vfeefs %v17,%v16,%v18 /* Find c. */
+ slgrk %r4,%r5,%r4 /* %r5 = current_len - n. */
+ lghi %r0,16 /* Register width = 16. */
+ vlgvb %r1,%v17,7 /* Extract found index or 16 if all equal. */
+ slr %r0,%r4 /* %r0 = first ignored match index. */
+ clrjl %r1,%r0,.Lfound2 /* Go away if miscompare is below n bytes. */
+ /* c not found within n-bytes. */
+.Lnf_end:
+ lghi %r2,0 /* Return null. */
+ br %r14
+
+.Lfound48:
+ aghi %r5,16
+.Lfound32:
+ aghi %r5,16
+.Lfound16:
+ aghi %r5,16
+.Lfound0:
+ aghi %r5,16
+.Lfound:
+ vlgvb %r1,%v17,7 /* Load byte index of c. */
+.Lfound2:
+ slgfi %r5,16 /* current_len -=16 */
+ algr %r5,%r1 /* Zero byte index is added to current len. */
+ la %r2,0(%r5,%r2) /* Return pointer to c. */
+ br %r14
+
+.Lloop64:
+ vl %v16,0(%r5,%r2)
+ vfeefs %v17,%v16,%v18 /* Find c. */
+ jl .Lfound0 /* Jump away if c was found. */
+ vl %v16,16(%r5,%r2)
+ vfeefs %v17,%v16,%v18
+ jl .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfeefs %v17,%v16,%v18
+ jl .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfeefs %v17,%v16,%v18
+ jl .Lfound48
+
+ aghi %r5,64
+ lgr %r0,%r5 /* If %r5 + 64 < n? -> loop64. */
+ aghi %r0,64
+ clgrjl %r0,%r4,.Lloop64
+
+ j .Llt64
+.Lfallback:
+ jg __wmemchr_c
+END(__wmemchr_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wmemchr.c b/sysdeps/s390/multiarch/wmemchr.c
new file mode 100644
index 0000000..a44e6f7
--- /dev/null
+++ b/sysdeps/s390/multiarch/wmemchr.c
@@ -0,0 +1,29 @@
+/* Multiple versions of wmemchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc (__wmemchr)
+weak_alias (__wmemchr, wmemchr)
+libc_hidden_weak (wmemchr)
+
+#else
+# include <wcsmbs/wmemchr.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/s390-32/multiarch/memchr.c b/sysdeps/s390/s390-32/multiarch/memchr.c
new file mode 100644
index 0000000..00958cd
--- /dev/null
+++ b/sysdeps/s390/s390-32/multiarch/memchr.c
@@ -0,0 +1,21 @@
+/* Multiple versions of memchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* This wrapper-file is needed, because otherwise file
+ sysdeps/s390/s390-[32|64]/memchr.S will be used. */
+#include <sysdeps/s390/multiarch/memchr.c>
diff --git a/sysdeps/s390/s390-64/multiarch/memchr.c b/sysdeps/s390/s390-64/multiarch/memchr.c
new file mode 100644
index 0000000..00958cd
--- /dev/null
+++ b/sysdeps/s390/s390-64/multiarch/memchr.c
@@ -0,0 +1,21 @@
+/* Multiple versions of memchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* This wrapper-file is needed, because otherwise file
+ sysdeps/s390/s390-[32|64]/memchr.S will be used. */
+#include <sysdeps/s390/multiarch/memchr.c>
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index 9c848ca..e8ce579 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -44,7 +44,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul wcsspn wcspbrk \
- wcscspn
+ wcscspn wmemchr
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/test-wmemchr.c b/wcsmbs/test-wmemchr.c
new file mode 100644
index 0000000..90c7f68
--- /dev/null
+++ b/wcsmbs/test-wmemchr.c
@@ -0,0 +1,20 @@
+/* Test wmemchr functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "../string/test-memchr.c"
diff --git a/wcsmbs/wmemchr.c b/wcsmbs/wmemchr.c
index 266950c..9e8d57b 100644
--- a/wcsmbs/wmemchr.c
+++ b/wcsmbs/wmemchr.c
@@ -18,6 +18,10 @@
#include <wchar.h>
+#ifdef WMEMCHR
+# define __wmemchr WMEMCHR
+#endif
+
wchar_t *
__wmemchr (s, c, n)
const wchar_t *s;
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=b4c21601b147efc3c2b0e679e4ffc554b3987f0b
commit b4c21601b147efc3c2b0e679e4ffc554b3987f0b
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:24 2015 +0200
S390: Optimize strcspn and wcscspn.
This patch provides optimized versions of strcspn and wcscspn with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strcspn-c.c: New File.
* sysdeps/s390/multiarch/strcspn-vx.S: Likewise.
* sysdeps/s390/multiarch/strcspn.c: Likewise.
* sysdeps/s390/multiarch/wcscspn-c.c: Likewise.
* sysdeps/s390/multiarch/wcscspn-vx.S: Likewise.
* sysdeps/s390/multiarch/wcscspn.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strcspn and
wcscspn functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strcspn, wcscspn.
* wcsmbs/wcscspn.c: Use WCSCSPN if defined.
* string/test-strcspn.c: Add wcscspn support.
* wcsmbs/test-wcscspn.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcscspn.
* benchtests/bench-strcspn.c: Add wcscspn support.
* benchtests/bench-wcscspn.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcscspn.
diff --git a/ChangeLog b/ChangeLog
index 49d6704..4502084 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strcspn-c.c: New File.
+ * sysdeps/s390/multiarch/strcspn-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strcspn.c: Likewise.
+ * sysdeps/s390/multiarch/wcscspn-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcscspn-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcscspn.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strcspn and
+ wcscspn functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strcspn, wcscspn.
+ * wcsmbs/wcscspn.c: Use WCSCSPN if defined.
+ * string/test-strcspn.c: Add wcscspn support.
+ * wcsmbs/test-wcscspn.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcscspn.
+ * benchtests/bench-strcspn.c: Add wcscspn support.
+ * benchtests/bench-wcscspn.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcscspn.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strpbrk-c.c: New File.
* sysdeps/s390/multiarch/strpbrk-vx.S: Likewise.
* sysdeps/s390/multiarch/strpbrk.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index f882f98..46cb34f 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -37,7 +37,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
- wcscmp wcsncmp wcschr wcschrnul wcsrchr wcsspn wcspbrk
+ wcscmp wcsncmp wcschr wcschrnul wcsrchr wcsspn wcspbrk wcscspn
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-strcspn.c b/benchtests/bench-strcspn.c
index 2fe7efc..7ddb3cb 100644
--- a/benchtests/bench-strcspn.c
+++ b/benchtests/bench-strcspn.c
@@ -19,22 +19,41 @@
#define STRPBRK_RESULT(s, pos) (pos)
#define RES_TYPE size_t
#define TEST_MAIN
-#define TEST_NAME "strcspn"
+#ifndef WIDE
+# define TEST_NAME "strcspn"
+#else
+# define TEST_NAME "wcscspn"
+#endif /* WIDE */
#include "bench-string.h"
-typedef size_t (*proto_t) (const char *, const char *);
-size_t simple_strcspn (const char *, const char *);
-size_t stupid_strcspn (const char *, const char *);
+#ifndef WIDE
+# define STRCSPN strcspn
+# define CHAR char
+# define SIMPLE_STRCSPN simple_strcspn
+# define STUPID_STRCSPN stupid_strcspn
+# define STRLEN strlen
+#else
+# include <wchar.h>
+# define STRCSPN wcscspn
+# define CHAR wchar_t
+# define SIMPLE_STRCSPN simple_wcscspn
+# define STUPID_STRCSPN stupid_wcscspn
+# define STRLEN wcslen
+#endif /* WIDE */
-IMPL (stupid_strcspn, 0)
-IMPL (simple_strcspn, 0)
-IMPL (strcspn, 1)
+typedef size_t (*proto_t) (const CHAR *, const CHAR *);
+size_t SIMPLE_STRCSPN (const CHAR *, const CHAR *);
+size_t STUPID_STRCSPN (const CHAR *, const CHAR *);
+
+IMPL (STUPID_STRCSPN, 0)
+IMPL (SIMPLE_STRCSPN, 0)
+IMPL (STRCSPN, 1)
size_t
-simple_strcspn (const char *s, const char *rej)
+SIMPLE_STRCSPN (const CHAR *s, const CHAR *rej)
{
- const char *r, *str = s;
- char c;
+ const CHAR *r, *str = s;
+ CHAR c;
while ((c = *s++) != '\0')
for (r = rej; *r != '\0'; ++r)
@@ -44,9 +63,9 @@ simple_strcspn (const char *s, const char *rej)
}
size_t
-stupid_strcspn (const char *s, const char *rej)
+STUPID_STRCSPN (const CHAR *s, const CHAR *rej)
{
- size_t ns = strlen (s), nrej = strlen (rej);
+ size_t ns = STRLEN (s), nrej = STRLEN (rej);
size_t i, j;
for (i = 0; i < ns; ++i)
@@ -56,4 +75,6 @@ stupid_strcspn (const char *s, const char *rej)
return i;
}
+#undef CHAR
+#undef STRLEN
#include "bench-strpbrk.c"
diff --git a/wcsmbs/wcscspn.c b/benchtests/bench-wcscspn.c
similarity index 59%
copy from wcsmbs/wcscspn.c
copy to benchtests/bench-wcscspn.c
index d8cc24a..3991951 100644
--- a/wcsmbs/wcscspn.c
+++ b/benchtests/bench-wcscspn.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Measure wcscspn functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,23 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
-
-
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters not in REJECT. */
-size_t
-wcscspn (wcs, reject)
- const wchar_t *wcs;
- const wchar_t *reject;
-{
- size_t count = 0;
-
- while (*wcs != L'\0')
- if (wcschr (reject, *wcs++) == NULL)
- ++count;
- else
- return count;
-
- return count;
-}
+#define WIDE 1
+#include "bench-strcspn.c"
diff --git a/string/test-strcspn.c b/string/test-strcspn.c
index b60a048..8508a5a 100644
--- a/string/test-strcspn.c
+++ b/string/test-strcspn.c
@@ -1,4 +1,4 @@
-/* Test and measure strcspn functions.
+/* Test strcspn functions.
Copyright (C) 1999-2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
Written by Jakub Jelinek <jakub@redhat.com>, 1999.
@@ -20,22 +20,41 @@
#define STRPBRK_RESULT(s, pos) (pos)
#define RES_TYPE size_t
#define TEST_MAIN
-#define TEST_NAME "strcspn"
+#ifndef WIDE
+# define TEST_NAME "strcspn"
+#else
+# define TEST_NAME "wcscspn"
+#endif /* WIDE */
#include "test-string.h"
-typedef size_t (*proto_t) (const char *, const char *);
-size_t simple_strcspn (const char *, const char *);
-size_t stupid_strcspn (const char *, const char *);
+#ifndef WIDE
+# define STRCSPN strcspn
+# define CHAR char
+# define SIMPLE_STRCSPN simple_strcspn
+# define STUPID_STRCSPN stupid_strcspn
+# define STRLEN strlen
+#else
+# include <wchar.h>
+# define STRCSPN wcscspn
+# define CHAR wchar_t
+# define SIMPLE_STRCSPN simple_wcscspn
+# define STUPID_STRCSPN stupid_wcscspn
+# define STRLEN wcslen
+#endif /* WIDE */
-IMPL (stupid_strcspn, 0)
-IMPL (simple_strcspn, 0)
-IMPL (strcspn, 1)
+typedef size_t (*proto_t) (const CHAR *, const CHAR *);
+size_t SIMPLE_STRCSPN (const CHAR *, const CHAR *);
+size_t STUPID_STRCSPN (const CHAR *, const CHAR *);
+
+IMPL (STUPID_STRCSPN, 0)
+IMPL (SIMPLE_STRCSPN, 0)
+IMPL (STRCSPN, 1)
size_t
-simple_strcspn (const char *s, const char *rej)
+SIMPLE_STRCSPN (const CHAR *s, const CHAR *rej)
{
- const char *r, *str = s;
- char c;
+ const CHAR *r, *str = s;
+ CHAR c;
while ((c = *s++) != '\0')
for (r = rej; *r != '\0'; ++r)
@@ -45,9 +64,9 @@ simple_strcspn (const char *s, const char *rej)
}
size_t
-stupid_strcspn (const char *s, const char *rej)
+STUPID_STRCSPN (const CHAR *s, const CHAR *rej)
{
- size_t ns = strlen (s), nrej = strlen (rej);
+ size_t ns = STRLEN (s), nrej = STRLEN (rej);
size_t i, j;
for (i = 0; i < ns; ++i)
@@ -57,4 +76,6 @@ stupid_strcspn (const char *s, const char *rej)
return i;
}
+#undef CHAR
+#undef STRLEN
#include "test-strpbrk.c"
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 5765a8c..2c1fce0 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -13,7 +13,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strchrnul strchrnul-vx strchrnul-c \
strrchr strrchr-vx strrchr-c \
strspn strspn-vx strspn-c \
- strpbrk strpbrk-vx strpbrk-c
+ strpbrk strpbrk-vx strpbrk-c \
+ strcspn strcspn-vx strcspn-c
endif
ifeq ($(subdir),wcsmbs)
@@ -31,5 +32,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcschrnul wcschrnul-vx wcschrnul-c \
wcsrchr wcsrchr-vx wcsrchr-c \
wcsspn wcsspn-vx wcsspn-c \
- wcspbrk wcspbrk-vx wcspbrk-c
+ wcspbrk wcspbrk-vx wcspbrk-c \
+ wcscspn wcscspn-vx wcscspn-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index b39a5c5..7f62e49 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -124,6 +124,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strpbrk);
IFUNC_VX_IMPL (wcspbrk);
+ IFUNC_VX_IMPL (strcspn);
+ IFUNC_VX_IMPL (wcscspn);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/wcsmbs/wcscspn.c b/sysdeps/s390/multiarch/strcspn-c.c
similarity index 59%
copy from wcsmbs/wcscspn.c
copy to sysdeps/s390/multiarch/strcspn-c.c
index d8cc24a..e718672 100644
--- a/wcsmbs/wcscspn.c
+++ b/sysdeps/s390/multiarch/strcspn-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Default strcspn implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,23 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRCSPN __strcspn_c
+# ifdef SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strcspn_c, __GI_strcspn, __strcspn_c);
+# endif /* SHARED */
-
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters not in REJECT. */
-size_t
-wcscspn (wcs, reject)
- const wchar_t *wcs;
- const wchar_t *reject;
-{
- size_t count = 0;
-
- while (*wcs != L'\0')
- if (wcschr (reject, *wcs++) == NULL)
- ++count;
- else
- return count;
-
- return count;
-}
+# include <string/strcspn.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strcspn-vx.S b/sysdeps/s390/multiarch/strcspn-vx.S
new file mode 100644
index 0000000..0a50fc2
--- /dev/null
+++ b/sysdeps/s390/multiarch/strcspn-vx.S
@@ -0,0 +1,281 @@
+/* Vector optimized 32/64 bit S/390 version of strcspn.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* size_t strcspn (const char *s, const char * reject)
+ The strcspn() function calculates the length of the initial segment
+ of s which consists entirely of characters not in reject.
+
+ This method checks the length of reject string. If it fits entirely
+ in one vector register, a fast algorithm is used, which does not need
+ to check multiple parts of accept-string. Otherwise a slower full
+ check of accept-string is used.
+
+ register overview:
+ r3: pointer to start of reject-string
+ r2: pointer to start of search-string
+ r0: loaded byte count of vlbb search-string
+ r4: found byte index
+ r1: current return len
+ v16: search-string
+ v17: reject-string
+ v18: temp-vreg
+
+ ONLY FOR SLOW:
+ v19: first reject-string
+ v20: zero for preparing acc-vector
+ v21: global mask; 1 indicates a match between
+ search-string-vreg and any reject-character
+ v22: current mask; 1 indicates a match between
+ search-string-vreg and any reject-character in current acc-vreg
+ v24: one for result-checking of former string-part
+ v30, v31: for re-/storing registers r6, r8, r9
+ r5: current len of reject-string
+ r6: zero-index in search-string or 16 if no zero
+ or min(zero-index, loaded byte count)
+ r8: >0, if former reject-string-part contains a zero,
+ otherwise =0;
+ r9: loaded byte count of vlbb reject-string
+*/
+ENTRY(__strcspn_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ /*
+ Check if reject-string fits in one vreg:
+ ----------------------------------------
+ */
+ vlbb %v17,0(%r3),6 /* Load reject. */
+ lghi %r1,0 /* Zero out current len. */
+ lcbb %r0,0(%r3),6
+ jo .Lcheck_onbb /* Special case if reject
+ lays on block-boundary. */
+.Lcheck_notonbb:
+ vistrbs %v17,%v17 /* Fill with zeros after first zero. */
+ je .Lfast /* Zero found -> reject fits in one vreg. */
+ j .Lslow /* No zero -> reject exceeds one vreg. */
+
+
+.Lcheck_onbb:
+ /* Reject lays on block-boundary. */
+ vfenezb %v18,%v17,%v17 /* Search zero in loaded reject bytes. */
+ vlgvb %r4,%v18,7 /* Get index of zero or 16 if not found. */
+ clrjl %r4,%r0,.Lcheck_notonbb /* Zero index < loaded bytes count ->
+ Reject fits in one vreg;
+ Fill with zeros and proceed
+ with FAST. */
+ vl %v17,0(%r3) /* Load reject, which exceeds loaded bytes. */
+ j .Lcheck_notonbb /* Check if reject fits in one vreg. */
+
+
+ /*
+ Search s for reject in one vreg
+ -------------------------------
+ */
+.Lfast:
+ /* Complete reject-string in v17 and remaining bytes are zero. */
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfaezbs %v18,%v16,%v17,0 /* Find first element in v16
+ unequal to any in v17
+ or first zero element. */
+
+ vlgvb %r4,%v18,7 /* Load byte index of found element. */
+ clrjl %r4,%r0,.Lfast_loop_found2 /* If found index is within loaded
+ bytes, return with found element
+ index (=equal count). */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r1,16 /* current_len = 16. */
+ slr %r1,%r4 /* Compute bytes to 16bytes boundary. */
+
+ /* Process s in 16byte aligned loop. */
+.Lfast_loop:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ vfaezbs %v18,%v16,%v17,0 /* Find first element in v16 equal to any
+ in v17 or first zero element. */
+ jno .Lfast_loop_found
+
+ vl %v16,16(%r1,%r2)
+ vfaezbs %v18,%v16,%v17,0
+ jno .Lfast_loop_found16
+
+ vl %v16,32(%r1,%r2)
+ vfaezbs %v18,%v16,%v17,0
+ jno .Lfast_loop_found32
+
+ vl %v16,48(%r1,%r2)
+ vfaezbs %v18,%v16,%v17,0
+ jno .Lfast_loop_found48
+
+ aghi %r1,64
+ j .Lfast_loop /* Loop if no element was unequal to reject
+ and not zero. */
+
+ /* Found equal or zero element. */
+.Lfast_loop_found48:
+ aghi %r1,16
+.Lfast_loop_found32:
+ aghi %r1,16
+.Lfast_loop_found16:
+ aghi %r1,16
+.Lfast_loop_found:
+ vlgvb %r4,%v18,7 /* Load byte index of found element or zero. */
+.Lfast_loop_found2:
+ algrk %r2,%r1,%r4 /* Add found index to current len. */
+ br %r14
+
+
+
+ /*
+ Search s for reject in multiple vregs
+ -------------------------------------
+ */
+.Lslow:
+ /* Save registers. */
+ vlvgg %v30,%r6,0
+ vlvgp %v31,%r8,%r9
+
+ /* Reject in v17 without zero. */
+ vlr %v19,%v17 /* Save first acc-part for a fast reload. */
+ vzero %v20 /* Zero for preparing acc-vector. */
+ vone %v24 /* One for checking result of former
+ string-part. */
+
+ /* Align s to 16 byte. */
+ risbg %r4,%r2,60,128+63,0 /* Test if s is aligned and
+ %r4 = bits 60-63 'and' 15. */
+ je .Lslow_loop_str /* If s is aligned, loop aligned. */
+ lghi %r0,15
+ slr %r0,%r4 /* Compute highest index to load (15-x). */
+ vll %v16,%r0,0(%r2) /* Load up to 16 byte boundary (vll needs
+ highest index, remaining bytes are 0). */
+ ahi %r0,1 /* Work with loaded byte count. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of reject-string to zero. */
+ vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first reject-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+ clije %r6,0,.Lslow_end /* If first element is zero -> return 0. */
+ clr %r0,%r6 /* cc==1 if loaded byte count < zero-index. */
+ locrl %r6,%r0 /* Load on cc==1; zero-index = lbc. */
+ j .Lslow_loop_acc
+
+
+ /* Process s in 16byte aligned loop. */
+.Lslow_next_str:
+ /* Check results of former processed str-part. */
+ vfeeb %v18,%v21,%v24 /* Find first equal match in global mask
+ (ones in element). */
+ vlgvb %r4,%v18,7 /* Get index of first one (=equal) or 16. */
+ /* Equal-index < min(zero-index, loaded byte count)
+ -> Return pointer to equal element. */
+ clrjl %r4,%r6,.Lslow_index_found
+ /* Zero-index < loaded byte count
+ -> Former str-part was last str-part
+ -> Return null */
+ clrjl %r6,%r0,.Lslow_end_not_found
+
+ /* All elements are zero (=no match) -> Proceed with next str-part. */
+ vlr %v17,%v19 /* Load first part of reject (no zero). */
+ algfr %r1,%r0 /* Add loaded byte count to current len. */
+
+.Lslow_loop_str:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ lghi %r0,16 /* Loaded byte count is 16. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of reject to zero. */
+ vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first reject-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+ clije %r6,0,.Lslow_end /* If first element is zero (end of string)
+ -> Return current length. */
+
+.Lslow_loop_acc:
+ vfaeb %v22,%v16,%v17,4 /* Create matching-mask (1 in mask ->
+ Character matches any rejected character in
+ this reject-string-part) IN=0, RT=1. */
+ vlgvb %r4,%v22,0 /* Get result of first element. */
+ /* First element is equal to any rejected characters?
+ (all other parts of reject cannot lead to a match before this one)
+ -> Return current len, which is pointing to this element. */
+ clijh %r4,0,.Lslow_end
+ vo %v21,%v21,%v22 /* Global-mask = global-|matching-mask. */
+ /* Proceed with next acc until end of acc is reached. */
+
+
+.Lslow_next_acc:
+ clijh %r8,0,.Lslow_next_str /* There was a zero in last reject-part
+ -> Add found index to current len
+ and end. */
+ vlbb %v17,16(%r5,%r3),6 /* Load next reject part. */
+ aghi %r5,16 /* Increment current len of reject-string. */
+ lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of reject-string. */
+ jo .Lslow_next_acc_onbb /* Jump away if reject-string is
+ on block-boundary. */
+.Lslow_next_acc_notonbb:
+ vistrbs %v17,%v17 /* Fill with zeros after first zero. */
+ jo .Lslow_loop_acc /* No zero found -> no preparation needed. */
+
+.Lslow_next_acc_prepare_zero:
+ /* Zero in reject-part: fill zeros with first-reject-character. */
+ vlgvb %r8,%v17,0 /* Load first element of reject-part. */
+ clije %r8,0,.Lslow_next_str /* Process next str-part if first
+ character in this part of reject
+ is a zero. */
+ /* r8>0 -> zero found in this acc-part. */
+ vrepb %v18,%v17,0 /* Replicate first char accross all chars. */
+ vceqb %v22,%v20,%v17 /* Create a mask (v22) of null chars
+ by comparing with 0 (v20). */
+ vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */
+ j .Lslow_loop_acc /* Reject-string part is prepared. */
+
+.Lslow_next_acc_onbb:
+ vfenezb %v18,%v17,%v17 /* Find zero in loaded bytes of reject part. */
+ vlgvb %r8,%v18,7 /* Load byte index of zero. */
+ clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes
+ -> Prepare vreg. */
+ vl %v17,0(%r5,%r3) /* Load over boundary ... */
+ lghi %r8,0 /* r8=0 -> no zero in this part of acc,
+ check for zero is in jump-target. */
+ j .Lslow_next_acc_notonbb /* ... and search for zero in
+ fully loaded vreg again. */
+
+.Lslow_end_not_found:
+ algfr %r1,%r6 /* Add zero-index to current len. */
+ j .Lslow_end
+.Lslow_index_found:
+ algfr %r1,%r4 /* Add found index of char to current len. */
+.Lslow_end:
+ lgr %r2,%r1
+ /* Restore registers. */
+ vlgvg %r6,%v30,0
+ vlgvg %r8,%v31,0
+ vlgvg %r9,%v31,1
+ br %r14
+END(__strcspn_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcscspn.c b/sysdeps/s390/multiarch/strcspn.c
similarity index 59%
copy from wcsmbs/wcscspn.c
copy to sysdeps/s390/multiarch/strcspn.c
index d8cc24a..7da142e 100644
--- a/wcsmbs/wcscspn.c
+++ b/sysdeps/s390/multiarch/strcspn.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Multiple versions of strcspn.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,23 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+s390_vx_libc_ifunc2 (__strcspn, strcspn)
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters not in REJECT. */
-size_t
-wcscspn (wcs, reject)
- const wchar_t *wcs;
- const wchar_t *reject;
-{
- size_t count = 0;
-
- while (*wcs != L'\0')
- if (wcschr (reject, *wcs++) == NULL)
- ++count;
- else
- return count;
-
- return count;
-}
+#else
+# include <string/strcspn.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/wcscspn.c b/sysdeps/s390/multiarch/wcscspn-c.c
similarity index 59%
copy from wcsmbs/wcscspn.c
copy to sysdeps/s390/multiarch/wcscspn-c.c
index d8cc24a..24134c0 100644
--- a/wcsmbs/wcscspn.c
+++ b/sysdeps/s390/multiarch/wcscspn-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Default wcscscpn implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,23 +16,11 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSCSPN __wcscspn_c
+# include <wchar.h>
+extern __typeof (wcscspn) __wcscspn_c;
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters not in REJECT. */
-size_t
-wcscspn (wcs, reject)
- const wchar_t *wcs;
- const wchar_t *reject;
-{
- size_t count = 0;
-
- while (*wcs != L'\0')
- if (wcschr (reject, *wcs++) == NULL)
- ++count;
- else
- return count;
-
- return count;
-}
+# include <wcsmbs/wcscspn.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcscspn-vx.S b/sysdeps/s390/multiarch/wcscspn-vx.S
new file mode 100644
index 0000000..7c17b3e
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcscspn-vx.S
@@ -0,0 +1,293 @@
+/* Vector optimized 32/64 bit S/390 version of wcscspn.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* size_t wcscspn (const wchar_t *s, const wchar_t * reject)
+ The wcscspn() function calculates the length of the initial segment
+ of s which consists entirely of characters not in reject.
+
+ This method checks the length of reject string. If it fits entirely
+ in one vector register, a fast algorithm is used, which does not need
+ to check multiple parts of accept-string. Otherwise a slower full
+ check of accept-string is used.
+
+ register overview:
+ r3: pointer to start of reject-string
+ r2: pointer to start of search-string
+ r0: loaded byte count of vlbb search-string
+ r4: found byte index
+ r1: current return len
+ v16: search-string
+ v17: reject-string
+ v18: temp-vreg
+
+ ONLY FOR SLOW:
+ v19: first reject-string
+ v20: zero for preparing acc-vector
+ v21: global mask; 1 indicates a match between
+ search-string-vreg and any reject-character
+ v22: current mask; 1 indicates a match between
+ search-string-vreg and any reject-character in current acc-vreg
+ v30, v31: for re-/storing registers r6, r8, r9
+ r5: current len of reject-string
+ r6: zero-index in search-string or 16 if no zero
+ or min(zero-index, loaded byte count)
+ r8: >0, if former reject-string-part contains a zero,
+ otherwise =0;
+ r9: loaded byte count of vlbb reject-string
+*/
+ENTRY(__wcscspn_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ /*
+ Check if reject-string fits in one vreg:
+ ----------------------------------------
+ */
+ vlbb %v17,0(%r3),0 /* Load reject. */
+ lcbb %r0,0(%r3),0
+ jo .Lcheck_onbb /* Special case if reject
+ lays on block-boundary. */
+
+.Lcheck_notonbb:
+ lghi %r1,0 /* Zero out current len. */
+ vistrfs %v17,%v17 /* Fill with zeros after first zero. */
+ je .Lfast /* Zero found -> reject fits in one vreg. */
+ j .Lslow /* No zero -> reject exceeds one vreg. */
+
+
+.Lcheck_onbb:
+ /* Reject lays on block-boundary. */
+ nill %r0,65532 /* Recognize only fully loaded characters. */
+ je .Lcheck_onbb2 /* Reload vr, if we loaded no full wchar_t. */
+ vfenezf %v18,%v17,%v17 /* Search zero in loaded reject bytes. */
+ vlgvb %r4,%v18,7 /* Get index of zero or 16 if not found. */
+ clrjl %r4,%r0,.Lcheck_notonbb /* Zero index < loaded bytes count ->
+ Reject fits in one vreg;
+ Fill with zeros and proceed
+ with FAST. */
+.Lcheck_onbb2:
+ vl %v17,0(%r3) /* Load reject, which exceeds loaded bytes. */
+ j .Lcheck_notonbb /* Check if reject fits in one vreg. */
+
+
+ /*
+ Search s for reject in one vreg
+ -------------------------------
+ */
+.Lfast:
+ /* Complete reject-string in v17 and remaining bytes are zero. */
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfaezfs %v18,%v16,%v17,0 /* Find first element in v16
+ unequal to any in v17
+ or first zero element. */
+ vlgvb %r4,%v18,7 /* Load byte index of found element. */
+ clrjl %r4,%r0,.Lfast_loop_found2 /* If found index is within loaded
+ bytes, return with found element
+ index (=equal count). */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r1,16 /* current_len = 16. */
+ slr %r1,%r4 /* Compute bytes to 16bytes boundary. */
+
+ /* Process s in 16byte aligned loop. */
+.Lfast_loop:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ vfaezfs %v18,%v16,%v17,0 /* Find first element in v16 equal to any
+ in v17 or first zero element. */
+ jno .Lfast_loop_found
+
+ vl %v16,16(%r1,%r2)
+ vfaezfs %v18,%v16,%v17,0
+ jno .Lfast_loop_found16
+
+ vl %v16,32(%r1,%r2)
+ vfaezfs %v18,%v16,%v17,0
+ jno .Lfast_loop_found32
+
+ vl %v16,48(%r1,%r2)
+ vfaezfs %v18,%v16,%v17,0
+ jno .Lfast_loop_found48
+
+ aghi %r1,64
+ j .Lfast_loop /* Loop if no element was unequal to reject
+ and not zero. */
+
+ /* Found equal or zero element. */
+.Lfast_loop_found48:
+ aghi %r1,16
+.Lfast_loop_found32:
+ aghi %r1,16
+.Lfast_loop_found16:
+ aghi %r1,16
+.Lfast_loop_found:
+ vlgvb %r4,%v18,7 /* Load byte index of found element or zero. */
+.Lfast_loop_found2:
+ algrk %r2,%r1,%r4 /* Add found index to current len. */
+ srlg %r2,%r2,2 /* Convert byte-count to character-count. */
+ br %r14
+
+
+
+ /*
+ Search s for reject in multiple vregs
+ -------------------------------------
+ */
+.Lslow:
+ /* Save registers. */
+ vlvgg %v30,%r6,0
+ vlvgp %v31,%r8,%r9
+
+ /* Reject in v17 without zero. */
+ vlr %v19,%v17 /* Save first acc-part for a fast reload. */
+ vzero %v20 /* Zero for preparing acc-vector. */
+ vone %v24 /* One for checking result of former
+ string-part. */
+
+ /* Align s to 16 byte. */
+ risbg %r4,%r2,60,128+63,0 /* Test if s is aligned and
+ %r4 = bits 60-63 'and' 15. */
+ je .Lslow_loop_str /* If s is aligned, loop aligned. */
+ lghi %r0,15
+ slr %r0,%r4 /* Compute highest index to load (15-x). */
+ vll %v16,%r0,0(%r2) /* Load up to 16byte boundary (vll needs
+ highest index, remaining bytes are 0). */
+ ahi %r0,1 /* Work with loaded byte count. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of reject-string to zero. */
+ vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first reject-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+ clije %r6,0,.Lslow_end /* If first element is zero -> return 0. */
+ clr %r0,%r6 /* cc==1 if loaded byte count < zero-index. */
+ locrl %r6,%r0 /* Load on cc==1; zero-index = lbc. */
+ j .Lslow_loop_acc
+
+
+ /* Process s in 16byte aligned loop. */
+.Lslow_next_str:
+ /* Check results of former processed str-part. */
+ vfeef %v18,%v21,%v24 /* Find first equal match in global mask
+ (ones in element). */
+ vlgvb %r4,%v18,7 /* Get index of first one (=equal) or 16. */
+ /* Equal-index < min(zero-index, loaded byte count)
+ -> Return pointer to equal element. */
+ clrjl %r4,%r6,.Lslow_index_found
+ /* Zero-index < loaded byte count
+ -> Former str-part was last str-part
+ -> Return null */
+ clrjl %r6,%r0,.Lslow_end_not_found
+
+ /* All elements are zero (=no match) -> proceed with next str-part. */
+ vlr %v17,%v19 /* Load first part of reject (no zero). */
+ algfr %r1,%r0 /* Add loaded byte count to current len. */
+
+.Lslow_loop_str:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ lghi %r0,16 /* Loaded byte count is 16. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of reject to zero. */
+ vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first reject-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+ clije %r6,0,.Lslow_end /* If first element is zero (end of string)
+ -> Return current length. */
+
+.Lslow_loop_acc:
+ vfaef %v22,%v16,%v17,4 /* Create matching-mask (1 in mask ->
+ Character matches any rejected character in
+ this reject-string-part) IN=0, RT=1. */
+ vlgvf %r4,%v22,0 /* Get result of first element. */
+ /* First element is equal to any rejected characters?
+ (All other parts of reject cannot lead to a match before this one)
+ -> Return current len, which is pointing to this element. */
+ clijh %r4,0,.Lslow_end
+ vo %v21,%v21,%v22 /* Global-mask = global-|matching-mask. */
+ /* Proceed with next acc until end of acc is reached. */
+
+
+.Lslow_next_acc:
+ clijh %r8,0,.Lslow_next_str /* There was a zero in last reject-part
+ -> Add found index to current len
+ and end. */
+ vlbb %v17,16(%r5,%r3),6 /* Load next reject part. */
+ aghi %r5,16 /* Increment current len of reject-string. */
+ lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of reject-string. */
+ jo .Lslow_next_acc_onbb /* Jump away if reject-string is
+ on block-boundary. */
+.Lslow_next_acc_notonbb:
+ vistrfs %v17,%v17 /* Fill with zeros after first zero. */
+ jo .Lslow_loop_acc /* No zero found -> no preparation needed. */
+
+.Lslow_next_acc_prepare_zero:
+ /* Zero in reject-part: fill zeros with first-reject-character. */
+ vlgvf %r8,%v17,0 /* Load first element of reject-part. */
+ clije %r8,0,.Lslow_next_str /* Process next str-part if first
+ character in this part of reject
+ is a zero. */
+ /* r8>0 -> zero found in this acc-part. */
+ vrepf %v18,%v17,0 /* Replicate first char accross all chars. */
+ vceqf %v22,%v20,%v17 /* Create a mask (v22) of null chars
+ by comparing with 0 (v20). */
+ vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */
+ j .Lslow_loop_acc /* Reject-string part is prepared. */
+
+.Lslow_next_acc_onbb:
+ nill %r9,65532 /* Recognize only fully loaded characters. */
+ je .Lslow_next_acc_onbb2 /* Reload vr, if no full wchar_t
+ loaded. */
+ vfenezf %v18,%v17,%v17 /* Find zero in loaded bytes of reject part. */
+ vlgvb %r8,%v18,7 /* Load byte index of zero. */
+ clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes
+ -> Prepare vreg. */
+.Lslow_next_acc_onbb2:
+ vl %v17,0(%r5,%r3) /* Load over boundary ... */
+ lghi %r8,0 /* r8=0 -> no zero in this part of acc,
+ check for zero is in jump-target. */
+ j .Lslow_next_acc_notonbb /* ... and search for zero in
+ fully loaded vreg again. */
+
+.Lslow_end_not_found:
+ algfr %r1,%r6 /* Add zero-index to current len. */
+ j .Lslow_end
+.Lslow_index_found:
+ algfr %r1,%r4 /* Add found index of char to current len. */
+.Lslow_end:
+ srlg %r2,%r1,2 /* Convert byte-count to character-count. */
+ /* Restore registers. */
+ vlgvg %r6,%v30,0
+ vlgvg %r8,%v31,0
+ vlgvg %r9,%v31,1
+ br %r14
+.Lfallback:
+ jg __wcscspn_c
+END(__wcscspn_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcscspn.c b/sysdeps/s390/multiarch/wcscspn.c
similarity index 59%
copy from wcsmbs/wcscspn.c
copy to sysdeps/s390/multiarch/wcscspn.c
index d8cc24a..4c92653 100644
--- a/wcsmbs/wcscspn.c
+++ b/sysdeps/s390/multiarch/wcscspn.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Multiple versions of wcscspn.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,23 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+s390_vx_libc_ifunc2 (__wcscspn, wcscspn)
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters not in REJECT. */
-size_t
-wcscspn (wcs, reject)
- const wchar_t *wcs;
- const wchar_t *reject;
-{
- size_t count = 0;
-
- while (*wcs != L'\0')
- if (wcschr (reject, *wcs++) == NULL)
- ++count;
- else
- return count;
-
- return count;
-}
+#else
+# include <wcsmbs/wcscspn.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index 03edaec..9c848ca 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -43,7 +43,8 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
mbrtoc16 c16rtomb
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
- wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul wcsspn wcspbrk
+ wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul wcsspn wcspbrk \
+ wcscspn
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/wcscspn.c b/wcsmbs/test-wcscspn.c
similarity index 59%
copy from wcsmbs/wcscspn.c
copy to wcsmbs/test-wcscspn.c
index d8cc24a..d09cd3d 100644
--- a/wcsmbs/wcscspn.c
+++ b/wcsmbs/test-wcscspn.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Test wcscspn functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,23 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
-
-
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters not in REJECT. */
-size_t
-wcscspn (wcs, reject)
- const wchar_t *wcs;
- const wchar_t *reject;
-{
- size_t count = 0;
-
- while (*wcs != L'\0')
- if (wcschr (reject, *wcs++) == NULL)
- ++count;
- else
- return count;
-
- return count;
-}
+#define WIDE 1
+#include "../string/test-strcspn.c"
diff --git a/wcsmbs/wcscspn.c b/wcsmbs/wcscspn.c
index d8cc24a..d735f8c 100644
--- a/wcsmbs/wcscspn.c
+++ b/wcsmbs/wcscspn.c
@@ -18,6 +18,9 @@
#include <wchar.h>
+#ifdef WCSCSPN
+# define wcscspn WCSCSPN
+#endif
/* Return the length of the maximum initial segment
of WCS which contains only wide-characters not in REJECT. */
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=f0ba659847446eec3b2477d60c97c77ef4680e81
commit f0ba659847446eec3b2477d60c97c77ef4680e81
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:24 2015 +0200
S390: Optimize strpbrk and wcspbrk.
This patch provides optimized versions of strpbrk and wcspbrk with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strpbrk-c.c: New File.
* sysdeps/s390/multiarch/strpbrk-vx.S: Likewise.
* sysdeps/s390/multiarch/strpbrk.c: Likewise.
* sysdeps/s390/multiarch/wcspbrk-c.c: Likewise.
* sysdeps/s390/multiarch/wcspbrk-vx.S: Likewise.
* sysdeps/s390/multiarch/wcspbrk.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strpbrk and
wcspbrk functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strpbrk, wcspbrk.
* wcsmbs/wcspbrk.c: Use WCSPBRK if defined.
* string/test-strpbrk.c: Add wcspbrk support.
* wcsmbs/test-wcspbrk.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcspbrk.
* benchtests/bench-strpbrk.c: Add wcspbrk support.
* benchtests/bench-wcspbrk.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcspbrk.
diff --git a/ChangeLog b/ChangeLog
index e16f354..49d6704 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strpbrk-c.c: New File.
+ * sysdeps/s390/multiarch/strpbrk-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strpbrk.c: Likewise.
+ * sysdeps/s390/multiarch/wcspbrk-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcspbrk-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcspbrk.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strpbrk and
+ wcspbrk functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strpbrk, wcspbrk.
+ * wcsmbs/wcspbrk.c: Use WCSPBRK if defined.
+ * string/test-strpbrk.c: Add wcspbrk support.
+ * wcsmbs/test-wcspbrk.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcspbrk.
+ * benchtests/bench-strpbrk.c: Add wcspbrk support.
+ * benchtests/bench-wcspbrk.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcspbrk.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strspn-c.c: New File.
* sysdeps/s390/multiarch/strspn-vx.S: Likewise.
* sysdeps/s390/multiarch/strspn.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index f00196b..f882f98 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -37,7 +37,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
- wcscmp wcsncmp wcschr wcschrnul wcsrchr wcsspn
+ wcscmp wcsncmp wcschr wcschrnul wcsrchr wcsspn wcspbrk
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-strpbrk.c b/benchtests/bench-strpbrk.c
index a3bc8d6..eb09a13 100644
--- a/benchtests/bench-strpbrk.c
+++ b/benchtests/bench-strpbrk.c
@@ -16,50 +16,80 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
+#ifndef WIDE
+# define CHAR char
+# define STRLEN strlen
+# define STRCHR strchr
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define CHAR wchar_t
+# define STRLEN wcslen
+# define STRCHR wcschr
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+#endif /* WIDE */
+
#ifndef STRPBRK_RESULT
# define STRPBRK_RESULT(s, pos) ((s)[(pos)] ? (s) + (pos) : NULL)
-# define RES_TYPE char *
+# define RES_TYPE CHAR *
# define TEST_MAIN
-# define TEST_NAME "strpbrk"
+# ifndef WIDE
+# define TEST_NAME "strpbrk"
+# else
+# define TEST_NAME "wcspbrk"
+# endif /* WIDE */
# include "bench-string.h"
-typedef char *(*proto_t) (const char *, const char *);
-char *simple_strpbrk (const char *, const char *);
-char *stupid_strpbrk (const char *, const char *);
-
-IMPL (stupid_strpbrk, 0)
-IMPL (simple_strpbrk, 0)
-IMPL (strpbrk, 1)
-
-char *
-simple_strpbrk (const char *s, const char *rej)
+# ifndef WIDE
+# define STRPBRK strpbrk
+# define SIMPLE_STRPBRK simple_strpbrk
+# define STUPID_STRPBRK stupid_strpbrk
+# else
+# include <wchar.h>
+# define STRPBRK wcspbrk
+# define SIMPLE_STRPBRK simple_wcspbrk
+# define STUPID_STRPBRK stupid_wcspbrk
+# endif /* WIDE */
+
+typedef CHAR *(*proto_t) (const CHAR *, const CHAR *);
+CHAR *SIMPLE_STRPBRK (const CHAR *, const CHAR *);
+CHAR *STUPID_STRPBRK (const CHAR *, const CHAR *);
+
+IMPL (STUPID_STRPBRK, 0)
+IMPL (SIMPLE_STRPBRK, 0)
+IMPL (STRPBRK, 1)
+
+CHAR *
+SIMPLE_STRPBRK (const CHAR *s, const CHAR *rej)
{
- const char *r;
- char c;
+ const CHAR *r;
+ CHAR c;
while ((c = *s++) != '\0')
for (r = rej; *r != '\0'; ++r)
if (*r == c)
- return (char *) s - 1;
+ return (CHAR *) s - 1;
return NULL;
}
-char *
-stupid_strpbrk (const char *s, const char *rej)
+CHAR *
+STUPID_STRPBRK (const CHAR *s, const CHAR *rej)
{
- size_t ns = strlen (s), nrej = strlen (rej);
+ size_t ns = STRLEN (s), nrej = STRLEN (rej);
size_t i, j;
for (i = 0; i < ns; ++i)
for (j = 0; j < nrej; ++j)
if (s[i] == rej[j])
- return (char *) s + i;
+ return (CHAR *) s + i;
return NULL;
}
-#endif
+#endif /* !STRPBRK_RESULT */
static void
-do_one_test (impl_t *impl, const char *s, const char *rej, RES_TYPE exp_res)
+do_one_test (impl_t *impl, const CHAR *s, const CHAR *rej, RES_TYPE exp_res)
{
RES_TYPE res = CALL (impl, s, rej);
size_t i, iters = INNER_LOOP_ITERS;
@@ -91,35 +121,35 @@ do_test (size_t align, size_t pos, size_t len)
size_t i;
int c;
RES_TYPE result;
- char *rej, *s;
+ CHAR *rej, *s;
align &= 7;
- if (align + pos + 10 >= page_size || len > 240)
+ if ((align + pos + 10) * sizeof (CHAR) >= page_size || len > 240)
return;
- rej = (char *) (buf2 + (random () & 255));
- s = (char *) (buf1 + align);
+ rej = (CHAR *) (buf2) + (random () & 255);
+ s = (CHAR *) (buf1) + align;
for (i = 0; i < len; ++i)
{
- rej[i] = random () & 255;
+ rej[i] = random () & BIG_CHAR;
if (!rej[i])
- rej[i] = random () & 255;
+ rej[i] = random () & BIG_CHAR;
if (!rej[i])
- rej[i] = 1 + (random () & 127);
+ rej[i] = 1 + (random () & SMALL_CHAR);
}
rej[len] = '\0';
- for (c = 1; c <= 255; ++c)
- if (strchr (rej, c) == NULL)
+ for (c = 1; c <= BIG_CHAR; ++c)
+ if (STRCHR (rej, c) == NULL)
break;
for (i = 0; i < pos; ++i)
{
- s[i] = random () & 255;
- if (strchr (rej, s[i]))
+ s[i] = random () & BIG_CHAR;
+ if (STRCHR (rej, s[i]))
{
- s[i] = random () & 255;
- if (strchr (rej, s[i]))
+ s[i] = random () & BIG_CHAR;
+ if (STRCHR (rej, s[i]))
s[i] = c;
}
}
@@ -127,7 +157,7 @@ do_test (size_t align, size_t pos, size_t len)
if (s[pos])
{
for (i = pos + 1; i < pos + 10; ++i)
- s[i] = random () & 255;
+ s[i] = random () & BIG_CHAR;
s[i] = '\0';
}
result = STRPBRK_RESULT (s, pos);
diff --git a/wcsmbs/wcspbrk.c b/benchtests/bench-wcspbrk.c
similarity index 61%
copy from wcsmbs/wcspbrk.c
copy to benchtests/bench-wcspbrk.c
index 17e821c..3d9f00f 100644
--- a/wcsmbs/wcspbrk.c
+++ b/benchtests/bench-wcspbrk.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Measure wcspbrk functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper, <drepper@gnu.ai.mit.edu>
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,21 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
-
-
-/* Find the first occurrence in WCS of any wide-character in ACCEPT. */
-wchar_t *
-wcspbrk (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- while (*wcs != L'\0')
- if (wcschr (accept, *wcs) == NULL)
- ++wcs;
- else
- return (wchar_t *) wcs;
-
- return NULL;
-}
-libc_hidden_def (wcspbrk)
+#define WIDE 1
+#include "bench-strpbrk.c"
diff --git a/string/test-strpbrk.c b/string/test-strpbrk.c
index b4ac389..a4dffe7 100644
--- a/string/test-strpbrk.c
+++ b/string/test-strpbrk.c
@@ -17,50 +17,82 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
+#ifndef WIDE
+# define CHAR char
+# define UCHAR unsigned char
+# define STRLEN strlen
+# define STRCHR strchr
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define CHAR wchar_t
+# define UCHAR wchar_t
+# define STRLEN wcslen
+# define STRCHR wcschr
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+#endif /* WIDE */
+
#ifndef STRPBRK_RESULT
# define STRPBRK_RESULT(s, pos) ((s)[(pos)] ? (s) + (pos) : NULL)
-# define RES_TYPE char *
+# define RES_TYPE CHAR *
# define TEST_MAIN
-# define TEST_NAME "strpbrk"
+# ifndef WIDE
+# define TEST_NAME "strpbrk"
+# else
+# define TEST_NAME "wcspbrk"
+# endif /* WIDE */
# include "test-string.h"
-typedef char *(*proto_t) (const char *, const char *);
-char *simple_strpbrk (const char *, const char *);
-char *stupid_strpbrk (const char *, const char *);
+# ifndef WIDE
+# define STRPBRK strpbrk
+# define SIMPLE_STRPBRK simple_strpbrk
+# define STUPID_STRPBRK stupid_strpbrk
+# else
+# include <wchar.h>
+# define STRPBRK wcspbrk
+# define SIMPLE_STRPBRK simple_wcspbrk
+# define STUPID_STRPBRK stupid_wcspbrk
+# endif /* WIDE */
+
+typedef CHAR *(*proto_t) (const CHAR *, const CHAR *);
+CHAR *SIMPLE_STRPBRK (const CHAR *, const CHAR *);
+CHAR *STUPID_STRPBRK (const CHAR *, const CHAR *);
-IMPL (stupid_strpbrk, 0)
-IMPL (simple_strpbrk, 0)
-IMPL (strpbrk, 1)
+IMPL (STUPID_STRPBRK, 0)
+IMPL (SIMPLE_STRPBRK, 0)
+IMPL (STRPBRK, 1)
-char *
-simple_strpbrk (const char *s, const char *rej)
+CHAR *
+SIMPLE_STRPBRK (const CHAR *s, const CHAR *rej)
{
- const char *r;
- char c;
+ const CHAR *r;
+ CHAR c;
while ((c = *s++) != '\0')
for (r = rej; *r != '\0'; ++r)
if (*r == c)
- return (char *) s - 1;
+ return (CHAR *) s - 1;
return NULL;
}
-char *
-stupid_strpbrk (const char *s, const char *rej)
+CHAR *
+STUPID_STRPBRK (const CHAR *s, const CHAR *rej)
{
- size_t ns = strlen (s), nrej = strlen (rej);
+ size_t ns = STRLEN (s), nrej = STRLEN (rej);
size_t i, j;
for (i = 0; i < ns; ++i)
for (j = 0; j < nrej; ++j)
if (s[i] == rej[j])
- return (char *) s + i;
+ return (CHAR *) s + i;
return NULL;
}
-#endif
+#endif /* !STRPBRK_RESULT */
static void
-do_one_test (impl_t *impl, const char *s, const char *rej, RES_TYPE exp_res)
+do_one_test (impl_t *impl, const CHAR *s, const CHAR *rej, RES_TYPE exp_res)
{
RES_TYPE res = CALL (impl, s, rej);
if (res != exp_res)
@@ -78,35 +110,35 @@ do_test (size_t align, size_t pos, size_t len)
size_t i;
int c;
RES_TYPE result;
- char *rej, *s;
+ CHAR *rej, *s;
align &= 7;
- if (align + pos + 10 >= page_size || len > 240)
+ if ((align + pos + 10) * sizeof (CHAR) >= page_size || len > 240)
return;
- rej = (char *) (buf2 + (random () & 255));
- s = (char *) (buf1 + align);
+ rej = (CHAR *) (buf2) + (random () & 255);
+ s = (CHAR *) (buf1) + align;
for (i = 0; i < len; ++i)
{
- rej[i] = random () & 255;
+ rej[i] = random () & BIG_CHAR;
if (!rej[i])
- rej[i] = random () & 255;
+ rej[i] = random () & BIG_CHAR;
if (!rej[i])
- rej[i] = 1 + (random () & 127);
+ rej[i] = 1 + (random () & SMALL_CHAR);
}
rej[len] = '\0';
- for (c = 1; c <= 255; ++c)
- if (strchr (rej, c) == NULL)
+ for (c = 1; c <= BIG_CHAR; ++c)
+ if (STRCHR (rej, c) == NULL)
break;
for (i = 0; i < pos; ++i)
{
- s[i] = random () & 255;
- if (strchr (rej, s[i]))
+ s[i] = random () & BIG_CHAR;
+ if (STRCHR (rej, s[i]))
{
- s[i] = random () & 255;
- if (strchr (rej, s[i]))
+ s[i] = random () & BIG_CHAR;
+ if (STRCHR (rej, s[i]))
s[i] = c;
}
}
@@ -114,7 +146,7 @@ do_test (size_t align, size_t pos, size_t len)
if (s[pos])
{
for (i = pos + 1; i < pos + 10; ++i)
- s[i] = random () & 255;
+ s[i] = random () & BIG_CHAR;
s[i] = '\0';
}
result = STRPBRK_RESULT (s, pos);
@@ -129,8 +161,8 @@ do_random_tests (void)
size_t i, j, n, align, pos, len, rlen;
RES_TYPE result;
int c;
- unsigned char *p = buf1 + page_size - 512;
- unsigned char *rej;
+ UCHAR *p = (UCHAR *) (buf1 + page_size) - 512;
+ UCHAR *rej;
for (n = 0; n < ITERATIONS; n++)
{
@@ -147,18 +179,18 @@ do_random_tests (void)
rlen = random () & 63;
else
rlen = random () & 15;
- rej = buf2 + page_size - rlen - 1 - (random () & 7);
+ rej = (UCHAR *) (buf2 + page_size) - rlen - 1 - (random () & 7);
for (i = 0; i < rlen; ++i)
{
- rej[i] = random () & 255;
+ rej[i] = random () & BIG_CHAR;
if (!rej[i])
- rej[i] = random () & 255;
+ rej[i] = random () & BIG_CHAR;
if (!rej[i])
- rej[i] = 1 + (random () & 127);
+ rej[i] = 1 + (random () & SMALL_CHAR);
}
rej[i] = '\0';
- for (c = 1; c <= 255; ++c)
- if (strchr ((char *) rej, c) == NULL)
+ for (c = 1; c <= BIG_CHAR; ++c)
+ if (STRCHR ((CHAR *) rej, c) == NULL)
break;
j = (pos > len ? pos : len) + align + 64;
if (j > 512)
@@ -171,27 +203,27 @@ do_random_tests (void)
else if (i == pos + align)
p[i] = rej[random () % (rlen + 1)];
else if (i < align || i > pos + align)
- p[i] = random () & 255;
+ p[i] = random () & BIG_CHAR;
else
{
- p[i] = random () & 255;
- if (strchr ((char *) rej, p[i]))
+ p[i] = random () & BIG_CHAR;
+ if (STRCHR ((CHAR *) rej, p[i]))
{
- p[i] = random () & 255;
- if (strchr ((char *) rej, p[i]))
+ p[i] = random () & BIG_CHAR;
+ if (STRCHR ((CHAR *) rej, p[i]))
p[i] = c;
}
}
}
- result = STRPBRK_RESULT ((char *) (p + align), pos < len ? pos : len);
+ result = STRPBRK_RESULT ((CHAR *) (p + align), pos < len ? pos : len);
FOR_EACH_IMPL (impl, 1)
- if (CALL (impl, (char *) (p + align), (char *) rej) != result)
+ if (CALL (impl, (CHAR *) (p + align), (CHAR *) rej) != result)
{
error (0, 0, "Iteration %zd - wrong result in function %s (%zd, %p, %zd, %zd, %zd) %p != %p",
n, impl->name, align, rej, rlen, pos, len,
- (void *) CALL (impl, (char *) (p + align), (char *) rej),
+ (void *) CALL (impl, (CHAR *) (p + align), (CHAR *) rej),
(void *) result);
ret = 1;
}
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 9403169..5765a8c 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -12,7 +12,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strchr strchr-vx strchr-c \
strchrnul strchrnul-vx strchrnul-c \
strrchr strrchr-vx strrchr-c \
- strspn strspn-vx strspn-c
+ strspn strspn-vx strspn-c \
+ strpbrk strpbrk-vx strpbrk-c
endif
ifeq ($(subdir),wcsmbs)
@@ -29,5 +30,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcschr wcschr-vx wcschr-c \
wcschrnul wcschrnul-vx wcschrnul-c \
wcsrchr wcsrchr-vx wcsrchr-c \
- wcsspn wcsspn-vx wcsspn-c
+ wcsspn wcsspn-vx wcsspn-c \
+ wcspbrk wcspbrk-vx wcspbrk-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index cbedf64..b39a5c5 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -121,6 +121,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strspn);
IFUNC_VX_IMPL (wcsspn);
+ IFUNC_VX_IMPL (strpbrk);
+ IFUNC_VX_IMPL (wcspbrk);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/wcsmbs/wcspbrk.c b/sysdeps/s390/multiarch/strpbrk-c.c
similarity index 61%
copy from wcsmbs/wcspbrk.c
copy to sysdeps/s390/multiarch/strpbrk-c.c
index 17e821c..3923ae8 100644
--- a/wcsmbs/wcspbrk.c
+++ b/sysdeps/s390/multiarch/strpbrk-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Default strpbrk implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper, <drepper@gnu.ai.mit.edu>
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,21 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRPBRK __strpbrk_c
+# ifdef SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strpbrk_c, __GI_strpbrk, __strpbrk_c);
+# endif /* SHARED */
-
-/* Find the first occurrence in WCS of any wide-character in ACCEPT. */
-wchar_t *
-wcspbrk (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- while (*wcs != L'\0')
- if (wcschr (accept, *wcs) == NULL)
- ++wcs;
- else
- return (wchar_t *) wcs;
-
- return NULL;
-}
-libc_hidden_def (wcspbrk)
+# include <string/strpbrk.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strpbrk-vx.S b/sysdeps/s390/multiarch/strpbrk-vx.S
new file mode 100644
index 0000000..907143b
--- /dev/null
+++ b/sysdeps/s390/multiarch/strpbrk-vx.S
@@ -0,0 +1,302 @@
+/* Vector optimized 32/64 bit S/390 version of strpbrk.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char *strpbrk (const char *s, const char * accept)
+ The strpbrk() function locates the first occurrence in the string s
+ of any of the characters in the string accept and returns a pointer
+ to that character or NULL if not found.
+
+ This method checks the length of accept string. If it fits entirely
+ in one vector register, a fast algorithm is used, which does not need
+ to check multiple parts of accept-string. Otherwise a slower full
+ check of accept-string is used.
+
+ register overview:
+ r3: pointer to start of accept-string
+ r2: pointer to start of search-string
+ r0: loaded byte count of vlbb search-string (32bit unsigned)
+ r4: found byte index (32bit unsigned)
+ r1: current return len (64bit unsigned)
+ v16: search-string
+ v17: accept-string
+ v18: temp-vreg
+
+ ONLY FOR SLOW:
+ v19: first accept-string
+ v20: zero for preparing acc-vector
+ v21: global mask; 1 indicates a match between
+ search-string-vreg and any accept-character
+ v22: current mask; 1 indicates a match between
+ search-string-vreg and any accept-character in current acc-vreg
+ v24: one for result-checking of former string-part
+ v30, v31: for re-/storing registers r6, r8, r9
+ r5: current len of accept-string
+ r6: zero-index in search-string or 16 if no zero
+ or min(zero-index, loaded byte count)
+ r8: >0, if former accept-string-part contains a zero,
+ otherwise =0;
+ r9: loaded byte count of vlbb accept-string
+*/
+ENTRY(__strpbrk_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ /*
+ Check if accept-string fits in one vreg:
+ ----------------------------------------
+ */
+ vlbb %v17,0(%r3),6 /* Load accept. */
+ lghi %r1,0 /* Zero out current len. */
+ vlgvb %r0,%v17,0 /* Get first element. */
+ clije %r0,0,.Lfast_end_null /* Return null if accept is empty. */
+ lcbb %r0,0(%r3),6
+ jo .Lcheck_onbb /* Special case if accept lays
+ on block-boundary. */
+.Lcheck_notonbb:
+ vistrbs %v17,%v17 /* Fill with zeros after first zero. */
+ je .Lfast /* Zero found -> accept fits in one vreg. */
+ j .Lslow /* No zero -> accept exceeds one vreg */
+
+
+.Lcheck_onbb:
+ /* Accept lays on block-boundary. */
+ vfenezb %v18,%v17,%v17 /* Search zero in loaded accept bytes. */
+ vlgvb %r4,%v18,7 /* Get index of zero or 16 if not found. */
+ clrjl %r4,%r0,.Lcheck_notonbb /* Zero index < loaded bytes count ->
+ Accept fits in one vreg;
+ Fill with zeros and proceed
+ with FAST. */
+ vl %v17,0(%r3) /* Load accept, which exceeds loaded bytes. */
+ j .Lcheck_notonbb /* Check if accept fits in one vreg. */
+
+
+ /*
+ Search s for accept in one vreg
+ -------------------------------
+ */
+.Lfast:
+ /* Complete accept-string in v17 and remaining bytes are zero. */
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfaezbs %v18,%v16,%v17,0 /* Find first element in v16 unequal to any
+ in v17 or first zero element. */
+
+ vlgvb %r4,%v18,7 /* Load byte index of found element. */
+ /* If found index is within loaded bytes, return with found
+ element index (=equal count). */
+ clrjl %r4,%r0,.Lfast_loop_found2
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r1,16 /* current_len = 16. */
+ slr %r1,%r4 /* Compute bytes to 16bytes boundary. */
+
+ /* Process s in 16byte aligned loop. */
+.Lfast_loop:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ vfaezbs %v18,%v16,%v17,0 /* Find first element in v16 equal to any
+ in v17 or first zero element. */
+ jno .Lfast_loop_found
+
+ vl %v16,16(%r1,%r2)
+ vfaezbs %v18,%v16,%v17,0
+ jno .Lfast_loop_found16
+
+ vl %v16,32(%r1,%r2)
+ vfaezbs %v18,%v16,%v17,0
+ jno .Lfast_loop_found32
+
+ vl %v16,48(%r1,%r2)
+ vfaezbs %v18,%v16,%v17,0
+ jno .Lfast_loop_found48
+
+ aghi %r1,64
+ j .Lfast_loop /* Loop if no element was unequal to accept
+ and not zero. */
+
+ /* Found equal or zero element. */
+.Lfast_loop_found48:
+ aghi %r1,16
+.Lfast_loop_found32:
+ aghi %r1,16
+.Lfast_loop_found16:
+ aghi %r1,16
+.Lfast_loop_found:
+ vlgvb %r4,%v18,7 /* Load byte index of found element. */
+.Lfast_loop_found2:
+ vlgvb %r0,%v16,0(%r4) /* Get found element. */
+ clije %r0,0,.Lfast_end_null /* Return null if no accept-char found */
+ algfr %r1,%r4 /* Add found index of char to current len. */
+ la %r2,0(%r1,%r2) /* And return pointer to first equal char. */
+ br %r14
+
+.Lfast_end_null:
+ lghi %r2,0 /* Return null if no character is equal. */
+ br %r14
+
+
+
+
+ /*
+ Search s for accept in multiple vregs
+ -------------------------------------
+ */
+.Lslow:
+ /* Save registers. */
+ vlvgg %v30,%r6,0
+ vlvgp %v31,%r8,%r9
+
+ /* accept in v17 without zero. */
+ vlr %v19,%v17 /* Save first acc-part for a fast reload. */
+ vzero %v20 /* Zero for preparing acc-vector. */
+ vone %v24 /* One for checking result of former string. */
+
+ /* Align s to 16 byte. */
+ risbg %r4,%r2,60,128+63,0 /* Test if s is aligned and
+ %r4 = bits 60-63 'and' 15. */
+ je .Lslow_loop_str /* If s is aligned, loop aligned. */
+ lghi %r0,15
+ slr %r0,%r4 /* Compute highest index to load (15-x). */
+ vll %v16,%r0,0(%r2) /* Load up to 16 byte boundary (vll needs
+ highest index, remaining bytes are 0). */
+ ahi %r0,1 /* Work with loaded byte count. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of accept-string to zero. */
+ vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first accept-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+ clije %r6,0,.Lslow_end_null /* If first element is zero
+ (end of string) -> return null */
+ clr %r0,%r6 /* cc==1 if loaded byte count < zero-index. */
+ locrl %r6,%r0 /* Load on cc==1; zero-index = lbc. */
+ j .Lslow_loop_acc
+
+
+ /* Process s in 16byte aligned loop. */
+.Lslow_next_str:
+ /* Check results of former processed str-part. */
+ vfeeb %v18,%v21,%v24 /* Find first equal match in global mask
+ (ones in element). */
+ vlgvb %r4,%v18,7 /* Get index of first one (=equal)
+ or 16 if no match. */
+ /* Equal-index < min(zero-index, loaded byte count)
+ -> return pointer to equal element. */
+ clrjl %r4,%r6,.Lslow_index_found
+ /* Zero-index < loaded byte count
+ -> former str-part was last str-part
+ -> return null */
+ clrjl %r6,%r0,.Lslow_end_null
+ /* All elements are zero (=no match) -> proceed with next str-part. */
+
+ vlr %v17,%v19 /* Load first part of accept (no zero). */
+ algfr %r1,%r0 /* Add loaded byte count to current len. */
+
+.Lslow_loop_str:
+ vl %v16,0(%r1,%r2) /* Load search-string */
+ lghi %r0,16 /* Loaded byte count is 16. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of accept to zero. */
+ vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first accept-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+ clije %r6,0,.Lslow_end_null /* If first element is zero
+ (end of string) -> return null. */
+
+.Lslow_loop_acc:
+ vfaeb %v22,%v16,%v17,4 /* Create matching-mask (1 in mask ->
+ Character matches any accepted character in
+ this accept-string-part) IN=0, RT=1. */
+ vlgvb %r4,%v22,0 /* Get result of first element. */
+ /* First element is equal to any accepted characters
+ (all other parts of accept cannot lead to a match before this one)
+ -> current len is pointing to first element
+ -> return found */
+ clijh %r4,0,.Lslow_end_found
+ vo %v21,%v21,%v22 /* Global-mask = global-|matching-mask. */
+ /* Proceed with next acc until end of acc is reached. */
+
+
+.Lslow_next_acc:
+ clijh %r8,0,.Lslow_next_str /* There was a zero in the last acc-part
+ -> add index to current_len and
+ end. */
+ vlbb %v17,16(%r5,%r3),6 /* Load next accept part. */
+ aghi %r5,16 /* Increment current len of accept-string. */
+ lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of accept-string. */
+ jo .Lslow_next_acc_onbb /* Jump away ifaccept-string is
+ on block-boundary. */
+.Lslow_next_acc_notonbb:
+ vistrbs %v17,%v17 /* Fill with zeros after first zero. */
+ jo .Lslow_loop_acc /* No zero found -> no preparation needed. */
+
+.Lslow_next_acc_prepare_zero:
+ /* Zero in accept-part: fill zeros with first-accept-character. */
+ vlgvb %r8,%v17,0 /* Load first element of acc-part. */
+ clije %r8,0,.Lslow_next_str /* Proceed with next string-part,
+ if first char in this part of accept
+ is a zero. */
+ /* r8>0 -> zero found in this acc-part. */
+ vrepb %v18,%v17,0 /* Replicate first char accross all chars. */
+ vceqb %v22,%v20,%v17 /* Create a mask (v22) of null chars
+ by comparing with 0 (v20). */
+ vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */
+ j .Lslow_loop_acc /* Accept part is prepared -> process. */
+
+.Lslow_next_acc_onbb:
+ vfenezb %v18,%v17,%v17 /* Find zero in loaded bytes of accept part. */
+ vlgvb %r8,%v18,7 /* Load byte index of zero. */
+ clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes
+ -> Prepare vreg. */
+ vl %v17,0(%r5,%r3) /* Load over boundary ... */
+ lghi %r8,0 /* r8=0 -> no zero in this part of acc,
+ check for zero is in jump-target. */
+ j .Lslow_next_acc_notonbb /* ... and search for zero in
+ fully loaded vreg again. */
+
+.Lslow_end_null:
+ lghi %r1,0 /* Return null if no character is equal. */
+ j .Lslow_end
+
+.Lslow_loop_found:
+ vlgvb %r4,%v18,7 /* Load byte index of found element. */
+ vlgvb %r0,%v16,0(%r4) /* Get found element. */
+ clije %r0,0,.Lslow_end_null /* Return null if no acc-char found. */
+
+.Lslow_index_found:
+ algfr %r1,%r4 /* Add found index of char to current len. */
+.Lslow_end_found:
+ la %r1,0(%r1,%r2) /* And return pointer to first equal char. */
+
+.Lslow_end:
+ /* Restore registers. */
+ vlgvg %r6,%v30,0
+ vlgvg %r8,%v31,0
+ vlgvg %r9,%v31,1
+ lgr %r2,%r1
+ br %r14
+END(__strpbrk_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcspbrk.c b/sysdeps/s390/multiarch/strpbrk.c
similarity index 61%
copy from wcsmbs/wcspbrk.c
copy to sysdeps/s390/multiarch/strpbrk.c
index 17e821c..96dad5d 100644
--- a/wcsmbs/wcspbrk.c
+++ b/sysdeps/s390/multiarch/strpbrk.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Multiple versions of strpbrk.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper, <drepper@gnu.ai.mit.edu>
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,21 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+s390_vx_libc_ifunc2 (__strpbrk, strpbrk)
-/* Find the first occurrence in WCS of any wide-character in ACCEPT. */
-wchar_t *
-wcspbrk (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- while (*wcs != L'\0')
- if (wcschr (accept, *wcs) == NULL)
- ++wcs;
- else
- return (wchar_t *) wcs;
-
- return NULL;
-}
-libc_hidden_def (wcspbrk)
+#else
+# include <string/strpbrk.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/wcspbrk.c b/sysdeps/s390/multiarch/wcspbrk-c.c
similarity index 60%
copy from wcsmbs/wcspbrk.c
copy to sysdeps/s390/multiarch/wcspbrk-c.c
index 17e821c..634497b 100644
--- a/wcsmbs/wcspbrk.c
+++ b/sysdeps/s390/multiarch/wcspbrk-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Default wcspbrk implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper, <drepper@gnu.ai.mit.edu>
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,21 +16,16 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSPBRK __wcspbrk_c
+# include <wchar.h>
+extern __typeof (wcspbrk) __wcspbrk_c;
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__wcspbrk_c, __GI_wcspbrk, __wcspbrk_c);
+# endif /* SHARED */
-/* Find the first occurrence in WCS of any wide-character in ACCEPT. */
-wchar_t *
-wcspbrk (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- while (*wcs != L'\0')
- if (wcschr (accept, *wcs) == NULL)
- ++wcs;
- else
- return (wchar_t *) wcs;
-
- return NULL;
-}
-libc_hidden_def (wcspbrk)
+# include <wcsmbs/wcspbrk.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcspbrk-vx.S b/sysdeps/s390/multiarch/wcspbrk-vx.S
new file mode 100644
index 0000000..9cf542a
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcspbrk-vx.S
@@ -0,0 +1,315 @@
+/* Vector optimized 32/64 bit S/390 version of wcspbrk.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t *wcspbrk (const wchar_t *s, const wchar_t * accept)
+ The wcspbrk() function locates the first occurrence in the string s
+ of any of the characters in the string accept and returns a pointer
+ to that character or NULL if not found.
+
+ This method checks the length of accept string. If it fits entirely
+ in one vector register, a fast algorithm is used, which does not need
+ to check multiple parts of accept-string. Otherwise a slower full
+ check of accept-string is used.
+
+ register overview:
+ r3: pointer to start of accept-string
+ r2: pointer to start of search-string
+ r0: loaded byte count of vlbb search-string (32bit unsigned)
+ r4: found byte index (32bit unsigned)
+ r1: current return len (64bit unsigned)
+ v16: search-string
+ v17: accept-string
+ v18: temp-vreg
+
+ ONLY FOR SLOW:
+ v19: first accept-string
+ v20: zero for preparing acc-vector
+ v21: global mask; 1 indicates a match between
+ search-string-vreg and any accept-character
+ v22: current mask; 1 indicates a match between
+ search-string-vreg and any accept-character in current acc-vreg
+ v24: one for result-checking of former string-part
+ v30, v31: for re-/storing registers r6, r8, r9
+ r5: current len of accept-string
+ r6: zero-index in search-string or 16 if no zero
+ or min(zero-index, loaded byte count)
+ r8: >0, if former accept-string-part contains a zero,
+ otherwise =0;
+ r9: loaded byte count of vlbb accept-string
+*/
+ENTRY(__wcspbrk_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ /*
+ Check if accept-string fits in one vreg:
+ ----------------------------------------
+ */
+ vlbb %v17,0(%r3),6 /* Load accept. */
+ lcbb %r0,0(%r3),6
+ jo .Lcheck_onbb /* Special case if accept lays
+ on block-boundary. */
+
+.Lcheck_notonbb:
+ lghi %r1,0 /* Zero out current len. */
+ vlgvf %r0,%v17,0 /* Get first element. */
+ clije %r0,0,.Lfast_end_null /* Return null if accept is empty. */
+
+ vistrfs %v17,%v17 /* Fill with zeros after first zero. */
+ je .Lfast /* Zero found -> accept fits in one vreg. */
+ j .Lslow /* No zero -> accept exceeds one vreg */
+
+
+.Lcheck_onbb:
+ /* Accept lays on block-boundary. */
+ nill %r0,65532 /* Recognize only fully loaded characters. */
+ je .Lcheck_onbb2 /* Reload vr, if we loaded no full wchar_t. */
+ vfenezf %v18,%v17,%v17 /* Search zero in loaded accept bytes. */
+ vlgvb %r4,%v18,7 /* Get index of zero or 16 if not found. */
+ clrjl %r4,%r0,.Lcheck_notonbb /* Zero index < loaded bytes count ->
+ accept fits in one vreg;
+ Fill with zeros and proceed
+ with FAST. */
+.Lcheck_onbb2:
+ vl %v17,0(%r3) /* Load accept, which exceeds loaded bytes. */
+ j .Lcheck_notonbb /* Check if accept fits in one vreg. */
+
+
+ /*
+ Search s for accept in one vreg
+ -------------------------------
+ */
+.Lfast:
+ /* Complete accept-string in v17 and remaining bytes are zero. */
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfaezfs %v18,%v16,%v17,0 /* Find first element in v16 unequal to any
+ in v17 or first zero element. */
+ vlgvb %r4,%v18,7 /* Load byte index of found element. */
+ /* If found index is within loaded bytes, return with found
+ element index (=equal count). */
+ clrjl %r4,%r0,.Lfast_loop_found2
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r1,16 /* current_len = 16. */
+ slr %r1,%r4 /* Compute bytes to 16bytes boundary. */
+
+.Lfast_loop:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ vfaezfs %v18,%v16,%v17,0 /* Find first element in v16 equal to any
+ in v17 or first zero element. */
+ jno .Lfast_loop_found
+
+ vl %v16,16(%r1,%r2)
+ vfaezfs %v18,%v16,%v17,0
+ jno .Lfast_loop_found16
+
+ vl %v16,32(%r1,%r2)
+ vfaezfs %v18,%v16,%v17,0
+ jno .Lfast_loop_found32
+
+ vl %v16,48(%r1,%r2)
+ vfaezfs %v18,%v16,%v17,0
+ jno .Lfast_loop_found48
+
+ aghi %r1,64
+ j .Lfast_loop /* Loop if no element was unequal to accept
+ and not zero. */
+
+ /* Found equal or zero element. */
+.Lfast_loop_found48:
+ aghi %r1,16
+.Lfast_loop_found32:
+ aghi %r1,16
+.Lfast_loop_found16:
+ aghi %r1,16
+.Lfast_loop_found:
+ vlgvb %r4,%v18,7 /* Load byte index of found element. */
+.Lfast_loop_found2:
+ srlg %r5,%r4,2 /* Convert byte-index to character-index. */
+ vlgvf %r0,%v16,0(%r5) /* Get found element. */
+ clije %r0,0,.Lfast_end_null /* Return null if no accept-char found */
+ algfr %r1,%r4 /* Add found index of char to current len. */
+ la %r2,0(%r1,%r2) /* And return pointer to first equal char. */
+ br %r14
+
+.Lfast_end_null:
+ lghi %r2,0 /* Return null if no character is equal. */
+ br %r14
+
+
+
+
+ /*
+ Search s for accept in multiple vregs
+ -------------------------------------
+ */
+.Lslow:
+ /* Save registers. */
+ vlvgg %v30,%r6,0
+ vlvgp %v31,%r8,%r9
+
+ /* Accept in v17 without zero */
+ vlr %v19,%v17 /* Save first acc-part for a fast reload. */
+ vzero %v20 /* Zero for preparing acc-vector. */
+ vone %v24 /* One for checking result of former string. */
+
+ /* Align s to 16 byte. */
+ risbg %r4,%r2,60,128+63,0 /* Test if s is aligned and
+ %r4 = bits 60-63 'and' 15. */
+ je .Lslow_loop_str /* If s is aligned, loop aligned. */
+ lghi %r0,15
+ slr %r0,%r4 /* Compute highest index to load (15-x). */
+ vll %v16,%r0,0(%r2) /* Load up to 16byte boundary;
+ needs highest index, left bytes are 0. */
+ ahi %r0,1 /* Work with loaded byte count. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of accept-string to zero. */
+ vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first accept-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+ clije %r6,0,.Lslow_end_null /* If first element is zero
+ (end of string) -> return null */
+ clr %r0,%r6 /* cc==1 if loaded byte count < zero-index. */
+ locrl %r6,%r0 /* Load on cc==1; zero-index = lbc. */
+ j .Lslow_loop_acc
+
+
+ /* Process s in 16byte aligned loop. */
+.Lslow_next_str:
+ /* Check results of former processed str-part. */
+ vfeef %v18,%v21,%v24 /* Find first equal match in global mask
+ (ones in element). */
+ vlgvb %r4,%v18,7 /* Get index of first one (=equal)
+ or 16 if no match. */
+ /* Equal-index < min(zero-index, loaded byte count)
+ -> return pointer to equal element. */
+ clrjl %r4,%r6,.Lslow_index_found
+ /* Zero-index < loaded byte count
+ -> former str-part was last str-part
+ -> return null */
+ clrjl %r6,%r0,.Lslow_end_null
+ /* All elements are zero (=no match) -> proceed with next str-part. */
+
+ vlr %v17,%v19 /* Load first part of accept (no zero). */
+ algfr %r1,%r0 /* Add loaded byte count to current len. */
+
+.Lslow_loop_str:
+ vl %v16,0(%r1,%r2) /* Load search-string */
+ lghi %r0,16 /* Loaded byte count is 16. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of accept to zero. */
+ vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first accept-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+ clije %r6,0,.Lslow_end_null /* If first element is zero
+ (end of string) -> return null. */
+
+.Lslow_loop_acc:
+ vfaef %v22,%v16,%v17,4 /* Create matching-mask (1 in mask ->
+ Character matches any accepted character in
+ this accept-string-part) IN=0, RT=1. */
+ vlgvf %r4,%v22,0 /* Get result of first element. */
+ /* First element is equal to any accepted characters
+ (all other parts of accept cannot lead to a match before this one)
+ -> current len is pointing to first element
+ -> return found */
+ clijh %r4,0,.Lslow_end_found
+ vo %v21,%v21,%v22 /* Global-mask = global-|matching-mask. */
+ /* Proceed with next acc until end of acc is reached. */
+
+
+.Lslow_next_acc:
+ clijh %r8,0,.Lslow_next_str /* There was a zero in the last acc-part
+ -> add index to current len and
+ end. */
+ vlbb %v17,16(%r5,%r3),6 /* Load next accept part. */
+ aghi %r5,16 /* Increment current len of accept-string. */
+ lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of accept-string. */
+ jo .Lslow_next_acc_onbb /* Jump away ifaccept-string is
+ on block-boundary. */
+.Lslow_next_acc_notonbb:
+ vistrfs %v17,%v17 /* Fill with zeros after first zero. */
+ jo .Lslow_loop_acc /* No zero found -> no preparation needed. */
+
+.Lslow_next_acc_prepare_zero:
+ /* Zero in accept-part: fill zeros with first-accept-character. */
+ vlgvf %r8,%v17,0 /* Load first element of acc-part. */
+ clije %r8,0,.Lslow_next_str /* Proceed with next string-part,
+ If first char in this part of accept
+ is a zero. */
+ /* r8>0 -> zero found in this acc-part. */
+ vrepf %v18,%v17,0 /* Replicate first char accross all chars. */
+ vceqf %v22,%v20,%v17 /* Create a mask (v22) of null chars
+ by comparing with 0 (v20). */
+ vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */
+ j .Lslow_loop_acc /* Accept part is prepared -> process. */
+
+.Lslow_next_acc_onbb:
+ nill %r9,65532 /* Recognize only fully loaded characters. */
+ je .Lslow_next_acc_onbb2 /* Reload vr, if no full wchar_t. */
+ vfenezf %v18,%v17,%v17 /* Find zero in loaded bytes of accept part. */
+ vlgvb %r8,%v18,7 /* Load byte index of zero. */
+ clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes
+ -> Prepare vreg. */
+.Lslow_next_acc_onbb2:
+ vl %v17,0(%r5,%r3) /* Load over boundary ... */
+ lghi %r8,0 /* r8=0 -> no zero in this part of acc,
+ check for zero is in jump-target. */
+ j .Lslow_next_acc_notonbb /* ... and search for zero in
+ fully loaded vreg again. */
+
+.Lslow_end_null:
+ lghi %r1,0 /* Return null if no character is equal. */
+ j .Lslow_end
+
+.Lslow_loop_found:
+ vlgvb %r4,%v18,7 /* Load byte index of found element. */
+ srlg %r5,%r4,2 /* Convert byte-index to character-index. */
+ vlgvf %r0,%v16,0(%r5) /* Get found element. */
+ clije %r0,0,.Lslow_end_null /* Return null if no acc-char found. */
+
+.Lslow_index_found:
+ algfr %r1,%r4 /* Add found index of char to current len. */
+.Lslow_end_found:
+ la %r1,0(%r1,%r2) /* And return pointer to first equal char. */
+
+.Lslow_end:
+ /* Restore registers. */
+ vlgvg %r6,%v30,0
+ vlgvg %r8,%v31,0
+ vlgvg %r9,%v31,1
+ lgr %r2,%r1
+ br %r14
+.Lfallback:
+ jg __wcspbrk_c
+END(__wcspbrk_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcspbrk.c b/sysdeps/s390/multiarch/wcspbrk.c
similarity index 61%
copy from wcsmbs/wcspbrk.c
copy to sysdeps/s390/multiarch/wcspbrk.c
index 17e821c..442532f 100644
--- a/wcsmbs/wcspbrk.c
+++ b/sysdeps/s390/multiarch/wcspbrk.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Multiple versions of wcspbrk.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper, <drepper@gnu.ai.mit.edu>
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,21 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+s390_vx_libc_ifunc2 (__wcspbrk, wcspbrk)
-/* Find the first occurrence in WCS of any wide-character in ACCEPT. */
-wchar_t *
-wcspbrk (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- while (*wcs != L'\0')
- if (wcschr (accept, *wcs) == NULL)
- ++wcs;
- else
- return (wchar_t *) wcs;
-
- return NULL;
-}
-libc_hidden_def (wcspbrk)
+#else
+# include <wcsmbs/wcspbrk.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index b51e7b6..03edaec 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -43,7 +43,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
mbrtoc16 c16rtomb
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
- wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul wcsspn
+ wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul wcsspn wcspbrk
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/wcspbrk.c b/wcsmbs/test-wcspbrk.c
similarity index 61%
copy from wcsmbs/wcspbrk.c
copy to wcsmbs/test-wcspbrk.c
index 17e821c..98e44e5 100644
--- a/wcsmbs/wcspbrk.c
+++ b/wcsmbs/test-wcspbrk.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Test wcspbrk functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper, <drepper@gnu.ai.mit.edu>
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,21 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
-
-
-/* Find the first occurrence in WCS of any wide-character in ACCEPT. */
-wchar_t *
-wcspbrk (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- while (*wcs != L'\0')
- if (wcschr (accept, *wcs) == NULL)
- ++wcs;
- else
- return (wchar_t *) wcs;
-
- return NULL;
-}
-libc_hidden_def (wcspbrk)
+#define WIDE 1
+#include "../string/test-strpbrk.c"
diff --git a/wcsmbs/wcspbrk.c b/wcsmbs/wcspbrk.c
index 17e821c..9bb43d6 100644
--- a/wcsmbs/wcspbrk.c
+++ b/wcsmbs/wcspbrk.c
@@ -18,6 +18,9 @@
#include <wchar.h>
+#ifdef WCSPBRK
+# define wcspbrk WCSPBRK
+#endif
/* Find the first occurrence in WCS of any wide-character in ACCEPT. */
wchar_t *
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=f1ffad98be7ec4111fbd1cd1f58f3e3343257519
commit f1ffad98be7ec4111fbd1cd1f58f3e3343257519
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:24 2015 +0200
S390: Optimize strspn and wcsspn.
This patch provides optimized versions of strspn and wcsspn with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strspn-c.c: New File.
* sysdeps/s390/multiarch/strspn-vx.S: Likewise.
* sysdeps/s390/multiarch/strspn.c: Likewise.
* sysdeps/s390/multiarch/wcsspn-c.c: Likewise.
* sysdeps/s390/multiarch/wcsspn-vx.S: Likewise.
* sysdeps/s390/multiarch/wcsspn.c: Likewise.
* wcsmbs/wcsspn.c: Use WCSSPN if defined.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strspn and
wcsspn functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strspn, wcsspn.
* string/test-strspn.c: Add wcsspn support.
* wcsmbs/test-wcsspn.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcsspn.
* benchtests/bench-strspn.c: Add wcsspn support.
* benchtests/bench-wcsspn.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcsspn.
diff --git a/ChangeLog b/ChangeLog
index 4ece1f4..e16f354 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strspn-c.c: New File.
+ * sysdeps/s390/multiarch/strspn-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strspn.c: Likewise.
+ * sysdeps/s390/multiarch/wcsspn-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcsspn-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcsspn.c: Likewise.
+ * wcsmbs/wcsspn.c: Use WCSSPN if defined.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strspn and
+ wcsspn functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strspn, wcsspn.
+ * string/test-strspn.c: Add wcsspn support.
+ * wcsmbs/test-wcsspn.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcsspn.
+ * benchtests/bench-strspn.c: Add wcsspn support.
+ * benchtests/bench-wcsspn.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcsspn.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strrchr-c.c: New File.
* sysdeps/s390/multiarch/strrchr-vx.S: Likewise.
* sysdeps/s390/multiarch/strrchr.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 6a0052a..f00196b 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -37,7 +37,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
- wcscmp wcsncmp wcschr wcschrnul wcsrchr
+ wcscmp wcsncmp wcschr wcschrnul wcsrchr wcsspn
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-strspn.c b/benchtests/bench-strspn.c
index 6061f2b..88192dd 100644
--- a/benchtests/bench-strspn.c
+++ b/benchtests/bench-strspn.c
@@ -17,22 +17,47 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "strspn"
+#ifndef WIDE
+# define TEST_NAME "strspn"
+#else
+# define TEST_NAME "wcsspn"
+#endif /* WIDE */
#include "bench-string.h"
-typedef size_t (*proto_t) (const char *, const char *);
-size_t simple_strspn (const char *, const char *);
-size_t stupid_strspn (const char *, const char *);
-
-IMPL (stupid_strspn, 0)
-IMPL (simple_strspn, 0)
-IMPL (strspn, 1)
+#ifndef WIDE
+# define STRSPN strspn
+# define CHAR char
+# define SIMPLE_STRSPN simple_strspn
+# define STUPID_STRSPN stupid_strspn
+# define STRLEN strlen
+# define STRCHR strchr
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define STRSPN wcsspn
+# define CHAR wchar_t
+# define SIMPLE_STRSPN simple_wcsspn
+# define STUPID_STRSPN stupid_wcsspn
+# define STRLEN wcslen
+# define STRCHR wcschr
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+#endif /* WIDE */
+
+typedef size_t (*proto_t) (const CHAR *, const CHAR *);
+size_t SIMPLE_STRSPN (const CHAR *, const CHAR *);
+size_t STUPID_STRSPN (const CHAR *, const CHAR *);
+
+IMPL (STUPID_STRSPN, 0)
+IMPL (SIMPLE_STRSPN, 0)
+IMPL (STRSPN, 1)
size_t
-simple_strspn (const char *s, const char *acc)
+SIMPLE_STRSPN (const CHAR *s, const CHAR *acc)
{
- const char *r, *str = s;
- char c;
+ const CHAR *r, *str = s;
+ CHAR c;
while ((c = *s++) != '\0')
{
@@ -46,9 +71,9 @@ simple_strspn (const char *s, const char *acc)
}
size_t
-stupid_strspn (const char *s, const char *acc)
+STUPID_STRSPN (const CHAR *s, const CHAR *acc)
{
- size_t ns = strlen (s), nacc = strlen (acc);
+ size_t ns = STRLEN (s), nacc = STRLEN (acc);
size_t i, j;
for (i = 0; i < ns; ++i)
@@ -63,7 +88,7 @@ stupid_strspn (const char *s, const char *acc)
}
static void
-do_one_test (impl_t *impl, const char *s, const char *acc, size_t exp_res)
+do_one_test (impl_t *impl, const CHAR *s, const CHAR *acc, size_t exp_res)
{
size_t res = CALL (impl, s, acc), i, iters = INNER_LOOP_ITERS;
timing_t start, stop, cur;
@@ -92,34 +117,34 @@ static void
do_test (size_t align, size_t pos, size_t len)
{
size_t i;
- char *acc, *s;
+ CHAR *acc, *s;
align &= 7;
- if (align + pos + 10 >= page_size || len > 240 || ! len)
+ if ((align + pos + 10) * sizeof (CHAR) >= page_size || len > 240 || ! len)
return;
- acc = (char *) (buf2 + (random () & 255));
- s = (char *) (buf1 + align);
+ acc = (CHAR *) (buf2) + (random () & 255);
+ s = (CHAR *) (buf1) + align;
for (i = 0; i < len; ++i)
{
- acc[i] = random () & 255;
+ acc[i] = random () & BIG_CHAR;
if (!acc[i])
- acc[i] = random () & 255;
+ acc[i] = random () & BIG_CHAR;
if (!acc[i])
- acc[i] = 1 + (random () & 127);
+ acc[i] = 1 + (random () & SMALL_CHAR);
}
acc[len] = '\0';
for (i = 0; i < pos; ++i)
s[i] = acc[random () % len];
- s[pos] = random () & 255;
- if (strchr (acc, s[pos]))
+ s[pos] = random () & BIG_CHAR;
+ if (STRCHR (acc, s[pos]))
s[pos] = '\0';
else
{
for (i = pos + 1; i < pos + 10; ++i)
- s[i] = random () & 255;
+ s[i] = random () & BIG_CHAR;
s[i] = '\0';
}
diff --git a/benchtests/bench-wcsspn.c b/benchtests/bench-wcsspn.c
new file mode 100644
index 0000000..7bdef50
--- /dev/null
+++ b/benchtests/bench-wcsspn.c
@@ -0,0 +1,20 @@
+/* Measure wcsspn functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strspn.c"
diff --git a/string/test-strspn.c b/string/test-strspn.c
index 89fef87..7d76d3a 100644
--- a/string/test-strspn.c
+++ b/string/test-strspn.c
@@ -18,22 +18,49 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "strspn"
+#ifndef WIDE
+# define TEST_NAME "strspn"
+#else
+# define TEST_NAME "wcsspn"
+#endif /* WIDE */
#include "test-string.h"
-typedef size_t (*proto_t) (const char *, const char *);
-size_t simple_strspn (const char *, const char *);
-size_t stupid_strspn (const char *, const char *);
-
-IMPL (stupid_strspn, 0)
-IMPL (simple_strspn, 0)
-IMPL (strspn, 1)
+#ifndef WIDE
+# define STRSPN strspn
+# define CHAR char
+# define UCHAR unsigned char
+# define SIMPLE_STRSPN simple_strspn
+# define STUPID_STRSPN stupid_strspn
+# define STRLEN strlen
+# define STRCHR strchr
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define STRSPN wcsspn
+# define CHAR wchar_t
+# define UCHAR wchar_t
+# define SIMPLE_STRSPN simple_wcsspn
+# define STUPID_STRSPN stupid_wcsspn
+# define STRLEN wcslen
+# define STRCHR wcschr
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+#endif /* WIDE */
+
+typedef size_t (*proto_t) (const CHAR *, const CHAR *);
+size_t SIMPLE_STRSPN (const CHAR *, const CHAR *);
+size_t STUPID_STRSPN (const CHAR *, const CHAR *);
+
+IMPL (STUPID_STRSPN, 0)
+IMPL (SIMPLE_STRSPN, 0)
+IMPL (STRSPN, 1)
size_t
-simple_strspn (const char *s, const char *acc)
+SIMPLE_STRSPN (const CHAR *s, const CHAR *acc)
{
- const char *r, *str = s;
- char c;
+ const CHAR *r, *str = s;
+ CHAR c;
while ((c = *s++) != '\0')
{
@@ -47,9 +74,9 @@ simple_strspn (const char *s, const char *acc)
}
size_t
-stupid_strspn (const char *s, const char *acc)
+STUPID_STRSPN (const CHAR *s, const CHAR *acc)
{
- size_t ns = strlen (s), nacc = strlen (acc);
+ size_t ns = STRLEN (s), nacc = STRLEN (acc);
size_t i, j;
for (i = 0; i < ns; ++i)
@@ -64,7 +91,7 @@ stupid_strspn (const char *s, const char *acc)
}
static void
-do_one_test (impl_t *impl, const char *s, const char *acc, size_t exp_res)
+do_one_test (impl_t *impl, const CHAR *s, const CHAR *acc, size_t exp_res)
{
size_t res = CALL (impl, s, acc);
if (res != exp_res)
@@ -80,34 +107,34 @@ static void
do_test (size_t align, size_t pos, size_t len)
{
size_t i;
- char *acc, *s;
+ CHAR *acc, *s;
align &= 7;
- if (align + pos + 10 >= page_size || len > 240 || ! len)
+ if ((align + pos + 10) * sizeof (CHAR) >= page_size || len > 240 || ! len)
return;
- acc = (char *) (buf2 + (random () & 255));
- s = (char *) (buf1 + align);
+ acc = (CHAR *) (buf2) + (random () & 255);
+ s = (CHAR *) (buf1) + align;
for (i = 0; i < len; ++i)
{
- acc[i] = random () & 255;
+ acc[i] = random () & BIG_CHAR;
if (!acc[i])
- acc[i] = random () & 255;
+ acc[i] = random () & BIG_CHAR;
if (!acc[i])
- acc[i] = 1 + (random () & 127);
+ acc[i] = 1 + (random () & SMALL_CHAR);
}
acc[len] = '\0';
for (i = 0; i < pos; ++i)
s[i] = acc[random () % len];
- s[pos] = random () & 255;
- if (strchr (acc, s[pos]))
+ s[pos] = random () & BIG_CHAR;
+ if (STRCHR (acc, s[pos]))
s[pos] = '\0';
else
{
for (i = pos + 1; i < pos + 10; ++i)
- s[i] = random () & 255;
+ s[i] = random () & BIG_CHAR;
s[i] = '\0';
}
@@ -119,8 +146,8 @@ static void
do_random_tests (void)
{
size_t i, j, n, align, pos, alen, len;
- unsigned char *p = buf1 + page_size - 512;
- unsigned char *acc;
+ UCHAR *p = (UCHAR *) (buf1 + page_size) - 512;
+ UCHAR *acc;
for (n = 0; n < ITERATIONS; n++)
{
@@ -138,14 +165,14 @@ do_random_tests (void)
len = random () & 511;
if (len + align >= 512)
len = 511 - align - (random () & 7);
- acc = buf2 + page_size - alen - 1 - (random () & 7);
+ acc = (UCHAR *) (buf2 + page_size) - alen - 1 - (random () & 7);
for (i = 0; i < alen; ++i)
{
- acc[i] = random () & 255;
+ acc[i] = random () & BIG_CHAR;
if (!acc[i])
- acc[i] = random () & 255;
+ acc[i] = random () & BIG_CHAR;
if (!acc[i])
- acc[i] = 1 + (random () & 127);
+ acc[i] = 1 + (random () & SMALL_CHAR);
}
acc[i] = '\0';
j = (pos > len ? pos : len) + align + 64;
@@ -158,23 +185,23 @@ do_random_tests (void)
p[i] = '\0';
else if (i == pos + align)
{
- p[i] = random () & 255;
- if (strchr ((char *) acc, p[i]))
+ p[i] = random () & BIG_CHAR;
+ if (STRCHR ((CHAR *) acc, p[i]))
p[i] = '\0';
}
else if (i < align || i > pos + align)
- p[i] = random () & 255;
+ p[i] = random () & BIG_CHAR;
else
p[i] = acc [random () % alen];
}
FOR_EACH_IMPL (impl, 1)
- if (CALL (impl, (char *) (p + align),
- (char *) acc) != (pos < len ? pos : len))
+ if (CALL (impl, (CHAR *) (p + align),
+ (CHAR *) acc) != (pos < len ? pos : len))
{
error (0, 0, "Iteration %zd - wrong result in function %s (%zd, %p, %zd, %zd, %zd) %zd != %zd",
n, impl->name, align, acc, alen, pos, len,
- CALL (impl, (char *) (p + align), (char *) acc),
+ CALL (impl, (CHAR *) (p + align), (CHAR *) acc),
(pos < len ? pos : len));
ret = 1;
}
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index b8b141e..9403169 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -11,7 +11,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strncmp strncmp-vx strncmp-c \
strchr strchr-vx strchr-c \
strchrnul strchrnul-vx strchrnul-c \
- strrchr strrchr-vx strrchr-c
+ strrchr strrchr-vx strrchr-c \
+ strspn strspn-vx strspn-c
endif
ifeq ($(subdir),wcsmbs)
@@ -27,5 +28,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsncmp wcsncmp-vx wcsncmp-c \
wcschr wcschr-vx wcschr-c \
wcschrnul wcschrnul-vx wcschrnul-c \
- wcsrchr wcsrchr-vx wcsrchr-c
+ wcsrchr wcsrchr-vx wcsrchr-c \
+ wcsspn wcsspn-vx wcsspn-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index ee84d80..cbedf64 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -118,6 +118,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strrchr);
IFUNC_VX_IMPL (wcsrchr);
+ IFUNC_VX_IMPL (strspn);
+ IFUNC_VX_IMPL (wcsspn);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/wcsmbs/wcsspn.c b/sysdeps/s390/multiarch/strspn-c.c
similarity index 54%
copy from wcsmbs/wcsspn.c
copy to sysdeps/s390/multiarch/strspn-c.c
index db51acc..ca330d5 100644
--- a/wcsmbs/wcsspn.c
+++ b/sysdeps/s390/multiarch/strspn-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Default strspn implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,31 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRSPN __strspn_c
+# ifdef SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strspn_c, __GI_strspn, __strspn_c);
+# endif /* SHARED */
-
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters in ACCEPT. */
-size_t
-wcsspn (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- const wchar_t *p;
- const wchar_t *a;
- size_t count = 0;
-
- for (p = wcs; *p != L'\0'; ++p)
- {
- for (a = accept; *a != L'\0'; ++a)
- if (*p == *a)
- break;
- if (*a == L'\0')
- return count;
- else
- ++count;
- }
-
- return count;
-}
-libc_hidden_def (wcsspn)
+# include <string/strspn.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strspn-vx.S b/sysdeps/s390/multiarch/strspn-vx.S
new file mode 100644
index 0000000..f224e26
--- /dev/null
+++ b/sysdeps/s390/multiarch/strspn-vx.S
@@ -0,0 +1,256 @@
+/* Vector optimized 32/64 bit S/390 version of strspn.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* size_t strspn (const char *s, const char * accept)
+ The strspn() function calculates the length of the initial segment
+ of s which consists entirely of characters in accept.
+
+ This method checks the length of accept string. If it fits entirely
+ in one vector register, a fast algorithm is used, which does not need
+ to check multiple parts of accept-string. Otherwise a slower full
+ check of accept-string is used.
+
+ register overview:
+ r3: pointer to start of accept-string
+ r2: pointer to start of search-string
+ r4: loaded byte count of vl search-string
+ r0: found byte index
+ r1: current return len of s
+ v16: search-string
+ v17: accept-string
+ v18: temp-vreg
+
+ ONLY FOR SLOW:
+ v19: first accept-string
+ v20: zero for preparing acc-vector
+ v21: global mask; 1 indicates a match between
+ search-string-vreg and any accept-character
+ v22: current mask; 1 indicates a match between
+ search-string-vreg and any accept-character in current acc-vreg
+ v30, v31: for re-/storing registers r6, r8, r9
+ r5: current len of accept-string
+ r6: zero-index in search-string or 16 if no zero
+ or min(zero-index, loaded byte count)
+ r8: >0, if former accept-string-part contains a zero,
+ otherwise =0;
+ r9: loaded byte count of vlbb accept-string
+*/
+ENTRY(__strspn_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ /*
+ Check if accept-string fits in one vreg:
+ ----------------------------------------
+ */
+ vlbb %v17,0(%r3),6 /* Load accept. */
+ lcbb %r4,0(%r3),6
+ jo .Lcheck_onbb /* Special case if accept lays
+ on block-boundary. */
+.Lcheck_notonbb:
+ vistrbs %v17,%v17 /* Fill with zeros after first zero. */
+ je .Lfast /* Zero found -> accept fits in one vreg. */
+ j .Lslow /* No zero -> accept exceeds one vreg. */
+
+.Lcheck_onbb:
+ /* Accept lays on block-boundary. */
+ vfenezb %v18,%v17,%v17 /* Search zero in loaded accept bytes. */
+ vlgvb %r0,%v18,7 /* Get index of zero or 16 if not found. */
+ clrjl %r0,%r4,.Lcheck_notonbb /* Zero index < loaded bytes count ->
+ Accept fits in one vreg;
+ Fill with zeros and proceed
+ with FAST. */
+ vl %v17,0(%r3) /* Load accept, which exceeds loaded bytes. */
+ j .Lcheck_notonbb /* Check if accept fits in one vreg. */
+
+
+ /*
+ Search s for accept in one vreg
+ -------------------------------
+ */
+.Lfast:
+ /* Complete accept-string is in v17 and remaining bytes are zero. */
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfaezbs %v16,%v16,%v17,8 /* Find first element in v16
+ unequal to any in v17
+ or first zero element. */
+ vlgvb %r0,%v16,7 /* Load byte index of found element. */
+ /* If found index is within loaded bytes (%r0 < %r1),
+ return with found element index (=equal count). */
+ clr %r0,%r1
+ locgrl %r2,%r0
+ blr %r14
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r1,16 /* current_len = 16. */
+ slr %r1,%r4 /* Compute bytes to 16bytes boundary. */
+
+.Lfast_loop:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ vfaezbs %v16,%v16,%v17,8 /* Find first element in v16
+ unequal to any in v17
+ or first zero element. */
+ jno .Lfast_loop_found
+ vl %v16,16(%r1,%r2)
+ vfaezbs %v16,%v16,%v17,8
+ jno .Lfast_loop_found16
+ vl %v16,32(%r1,%r2)
+ vfaezbs %v16,%v16,%v17,8
+ jno .Lfast_loop_found32
+ vl %v16,48(%r1,%r2)
+ vfaezbs %v16,%v16,%v17,8
+ jno .Lfast_loop_found48
+
+ aghi %r1,64
+ j .Lfast_loop /* Loop if no element was unequal to accept
+ and not zero. */
+
+ /* Found unequal or zero element. */
+.Lfast_loop_found48:
+ aghi %r1,16
+.Lfast_loop_found32:
+ aghi %r1,16
+.Lfast_loop_found16:
+ aghi %r1,16
+.Lfast_loop_found:
+ vlgvb %r0,%v16,7 /* Load byte index of found element. */
+ algrk %r2,%r1,%r0 /* And add it to current len. */
+ br %r14
+
+
+ /*
+ Search s for accept in multiple vregs
+ -------------------------------------
+ */
+.Lslow:
+ /* Save registers. */
+ vlvgg %v30,%r6,0
+ vlvgp %v31,%r8,%r9
+ lghi %r1,0 /* current_len = 0. */
+
+ /* Accept in v17 without zero. */
+ vlr %v19,%v17 /* Save first acc-part for a fast reload. */
+ vzero %v20 /* Zero for preparing acc-vector. */
+
+ /* Align s to 16 byte. */
+ risbg %r0,%r2,60,128+63,0 /* Test if s is aligned and
+ %r0 = bits 60-63 'and' 15 */
+ je .Lslow_loop_str /* If s is aligned, loop aligned */
+ lghi %r4,15
+ slr %r4,%r0 /* Compute highest index to load (15-x). */
+ vll %v16,%r4,0(%r2) /* Load up to 16byte boundary (vll needs
+ highest index, left bytes are 0). */
+ ahi %r4,1 /* Work with loaded byte count. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of accept-string to zero. */
+ vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first accept-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16
+ if there is no zero. */
+ clr %r4,%r6 /* cc==1 if loaded byte count < zero-index. */
+ locrl %r6,%r4 /* Load on cc==1. */
+ j .Lslow_loop_acc
+
+ /* Process s in 16byte aligned loop. */
+.Lslow_next_str:
+ vlr %v17,%v19 /* Load first part of accept (no zero). */
+ algfr %r1,%r4 /* Add loaded byte count to current len. */
+.Lslow_loop_str:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ lghi %r4,16 /* Loaded byte count is 16. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of accept-string to zero. */
+ vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first accept-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+
+.Lslow_loop_acc:
+ vfaeb %v22,%v16,%v17,4 /* Create matching-mask (1 in mask ->
+ character matches any accepted character in
+ this accept-string-part) IN=0, RT=1. */
+ vo %v21,%v21,%v22 /* global-mask = global- | matching-mask. */
+ vfenezb %v18,%v21,%v21 /* Find first zero in global-mask. */
+ vlgvb %r0,%v18,7 /* Get first found zero-index
+ (= first mismatch). */
+ clrjl %r0,%r6,.Lslow_next_acc /* Mismatch-index < min(lbc,zero-index)
+ -> Process this string-part
+ with next acc-part. */
+ clrjhe %r0,%r4,.Lslow_next_str /* Found-index >= loaded byte count
+ -> All loaded bytes are matching
+ any accept-character
+ and are not zero. */
+ /* All bytes are matching any characters in accept-string
+ and search-string is fully processed (found-index == zero-index) */
+.Lslow_add_lbc_end:
+ algrk %r2,%r1,%r0 /* Add matching characters to current_len. */
+ /* Restore registers. */
+ vlgvg %r6,%v30,0
+ vlgvg %r8,%v31,0
+ vlgvg %r9,%v31,1
+ br %r14
+
+
+
+.Lslow_next_acc:
+ clijh %r8,0,.Lslow_add_lbc_end /* There was a zero in last acc-part
+ -> Add found index to current len
+ and end. */
+ vlbb %v17,16(%r5,%r3),6 /* Load next accept part. */
+ aghi %r5,16 /* Add current_len of accept-string. */
+ lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of accept-string. */
+ jo .Lslow_next_acc_onbb /* Jump away if accept-string is
+ on block-boundary. */
+.Lslow_next_acc_notonbb:
+ vistrbs %v17,%v17 /* Fill with zeros after first zero. */
+ jo .Lslow_loop_acc /* No zero found -> no preparation needed. */
+
+.Lslow_next_acc_prepare_zero:
+ /* Zero in accept-part: fill zeros with first-accept-character. */
+ vlgvb %r8,%v17,0 /* Load first element of acc-part. */
+ clije %r8,0,.Lslow_add_lbc_end /* End if zero is first character
+ in this part of accept-string. */
+ /* r8>0 -> zero found in this acc-part. */
+ vrepb %v18,%v17,0 /* Replicate first char accross all chars. */
+ vceqb %v22,%v20,%v17 /* Create a mask (v22) of null chars
+ by comparing with 0 (v20). */
+ vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */
+ j .Lslow_loop_acc /* Accept part is prepared -> process. */
+
+.Lslow_next_acc_onbb:
+ vfenezb %v18,%v17,%v17 /* Find zero in loaded bytes of accept part. */
+ vlgvb %r8,%v18,7 /* Load byte index of zero. */
+ clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes
+ -> Prepare vr. */
+ vl %v17,0(%r5,%r3) /* Load over boundary ... */
+ lghi %r8,0 /* r8=0 -> no zero in this part of acc,
+ Check for zero is in jump-target. */
+ j .Lslow_next_acc_notonbb /* ... and search for zero in
+ fully loaded vreg again. */
+END(__strspn_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcsspn.c b/sysdeps/s390/multiarch/strspn.c
similarity index 54%
copy from wcsmbs/wcsspn.c
copy to sysdeps/s390/multiarch/strspn.c
index db51acc..204da69 100644
--- a/wcsmbs/wcsspn.c
+++ b/sysdeps/s390/multiarch/strspn.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Multiple versions of strspn.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,31 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+s390_vx_libc_ifunc2 (__strspn, strspn)
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters in ACCEPT. */
-size_t
-wcsspn (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- const wchar_t *p;
- const wchar_t *a;
- size_t count = 0;
-
- for (p = wcs; *p != L'\0'; ++p)
- {
- for (a = accept; *a != L'\0'; ++a)
- if (*p == *a)
- break;
- if (*a == L'\0')
- return count;
- else
- ++count;
- }
-
- return count;
-}
-libc_hidden_def (wcsspn)
+#else
+# include <string/strspn.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/wcsspn.c b/sysdeps/s390/multiarch/wcsspn-c.c
similarity index 54%
copy from wcsmbs/wcsspn.c
copy to sysdeps/s390/multiarch/wcsspn-c.c
index db51acc..5c54209 100644
--- a/wcsmbs/wcsspn.c
+++ b/sysdeps/s390/multiarch/wcsspn-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Default wcsspn implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,31 +16,16 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSSPN __wcsspn_c
+# include <wchar.h>
+extern __typeof (wcsspn) __wcsspn_c;
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__wcsspn_c, __GI_wcsspn, __wcsspn_c);
+# endif /* SHARED */
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters in ACCEPT. */
-size_t
-wcsspn (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- const wchar_t *p;
- const wchar_t *a;
- size_t count = 0;
-
- for (p = wcs; *p != L'\0'; ++p)
- {
- for (a = accept; *a != L'\0'; ++a)
- if (*p == *a)
- break;
- if (*a == L'\0')
- return count;
- else
- ++count;
- }
-
- return count;
-}
-libc_hidden_def (wcsspn)
+# include <wcsmbs/wcsspn.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcsspn-vx.S b/sysdeps/s390/multiarch/wcsspn-vx.S
new file mode 100644
index 0000000..1a87d38
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsspn-vx.S
@@ -0,0 +1,270 @@
+/* Vector optimized 32/64 bit S/390 version of wcsspn.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* size_t wcsspn (const wchar_t *s, const wchar_t * accept)
+ The wcsspn() function calculates the length of the initial segment
+ of s which consists entirely of characters in accept.
+
+ This method checks the length of accept string. If it fits entirely
+ in one vector register, a fast algorithm is used, which does not need
+ to check multiple parts of accept-string. Otherwise a slower full
+ check of accept-string is used.
+
+ register overview:
+ r3: pointer to start of accept-string
+ r2: pointer to start of search-string
+ r4: loaded byte count of vl search-string
+ r0: found byte index
+ r1: current return len of s
+ v16: search-string
+ v17: accept-string
+ v18: temp-vreg
+
+ ONLY FOR SLOW:
+ v19: first accept-string
+ v20: zero for preparing acc-vector
+ v21: global mask; 1 indicates a match between
+ search-string-vreg and any accept-character
+ v22: current mask; 1 indicates a match between
+ search-string-vreg and any accept-character in current acc-vreg
+ v30, v31: for re-/storing registers r6, r8, r9
+ r5: current len of accept-string
+ r6: zero-index in search-string or 16 if no zero
+ or min(zero-index, loaded byte count)
+ r8: >0, if former accept-string-part contains a zero,
+ otherwise =0;
+ r9: loaded byte count of vlbb accept-string
+*/
+ENTRY(__wcsspn_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ /*
+ Check if accept-string fits in one vreg:
+ ----------------------------------------
+ */
+ vlbb %v17,0(%r3),6 /* Load accept. */
+ lcbb %r4,0(%r3),6
+ jo .Lcheck_onbb /* Special case if accept lays
+ on block-boundary. */
+.Lcheck_notonbb:
+ vistrfs %v17,%v17 /* Fill with zeros after first zero. */
+ je .Lfast /* Zero found -> accept fits in one vreg. */
+ j .Lslow /* No zero -> accept exceeds one vreg. */
+
+.Lcheck_onbb:
+ /* Accept lays on block-boundary. */
+ nill %r4,65532 /* Recognize only fully loaded characters. */
+ je .Lcheck_onbb2 /* Reload vr if no full wchar_t. */
+ vfenezf %v18,%v17,%v17 /* Search zero in loaded accept bytes. */
+ vlgvb %r0,%v18,7 /* Get index of zero or 16 if not found. */
+ clrjl %r0,%r4,.Lcheck_notonbb /* Zero index < loaded bytes count ->
+ Accept fits in one vreg;
+ Fill with zeros and proceed
+ with FAST. */
+.Lcheck_onbb2:
+ vl %v17,0(%r3) /* Load accept, which exceeds loaded bytes. */
+ j .Lcheck_notonbb /* Check if accept fits in one vreg. */
+
+
+ /*
+ Search s for accept in one vreg
+ -------------------------------
+ */
+.Lfast:
+ /* Complete accept-string in v17 and remaining bytes are zero. */
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfaezfs %v16,%v16,%v17,8 /* Find first element in v16
+ unequal to any in v17
+ or first zero element. */
+
+ vlgvb %r0,%v16,7 /* Load byte index of found element. */
+ /* If found index is within loaded bytes (%r0 < %r1),
+ return with found element index (=equal count). */
+ clr %r0,%r1
+ srlg %r0,%r0,2 /* Convert byte-count to character-count. */
+ locgrl %r2,%r0
+ blr %r14
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r1,16 /* current_len = 16. */
+ slr %r1,%r4 /* Compute bytes to 16bytes boundary. */
+
+.Lfast_loop:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ vfaezfs %v16,%v16,%v17,8 /* Find first element in v16
+ unequal to any in v17
+ or first zero element. */
+ jno .Lfast_loop_found
+ vl %v16,16(%r1,%r2)
+ vfaezfs %v16,%v16,%v17,8
+ jno .Lfast_loop_found16
+ vl %v16,32(%r1,%r2)
+ vfaezfs %v16,%v16,%v17,8
+ jno .Lfast_loop_found32
+ vl %v16,48(%r1,%r2)
+ vfaezfs %v16,%v16,%v17,8
+ jno .Lfast_loop_found48
+
+ aghi %r1,64
+ j .Lfast_loop /* Loop if no element was unequal to accept
+ and not zero. */
+
+ /* Found unequal or zero element. */
+.Lfast_loop_found48:
+ aghi %r1,16
+.Lfast_loop_found32:
+ aghi %r1,16
+.Lfast_loop_found16:
+ aghi %r1,16
+.Lfast_loop_found:
+ vlgvb %r0,%v16,7 /* Load byte index of found element. */
+ algrk %r2,%r1,%r0 /* And add it to current len. */
+ srlg %r2,%r2,2 /* Convert byte-count to character-count. */
+ br %r14
+
+
+ /*
+ Search s for accept in multiple vregs
+ -------------------------------------
+ */
+.Lslow:
+ /* Save registers. */
+ vlvgg %v30,%r6,0
+ vlvgp %v31,%r8,%r9
+ lghi %r1,0 /* Zero out current len. */
+
+ /* accept in v17 without zero. */
+ vlr %v19,%v17 /* Save first acc-part for a fast reload. */
+ vzero %v20 /* Zero for preparing acc-vector. */
+
+ /* Align s to 16 byte. */
+ risbg %r0,%r2,60,128+63,0 /* Test if s is aligned and
+ %r0 = bits 60-63 'and' 15. */
+ je .Lslow_loop_str /* If s is aligned, loop aligned */
+ lghi %r4,15
+ slr %r4,%r0 /* Compute highest index to load (15-x). */
+ vll %v16,%r4,0(%r2) /* Load up to 16byte boundary (vll needs
+ highest index, remaining bytes are 0). */
+ aghi %r4,1 /* Work with loaded byte count. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of accept-string to zero. */
+ vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first accept-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16
+ if there is no zero. */
+ clr %r4,%r6 /* cc==1 if loaded byte count < zero-index. */
+ locrl %r6,%r4 /* Load on cc==1. */
+ j .Lslow_loop_acc
+
+ /* Process s in 16byte aligned loop. */
+.Lslow_next_str:
+ vlr %v17,%v19 /* Load first part of accept (no zero). */
+ algfr %r1,%r4 /* Add loaded byte count to current len. */
+.Lslow_loop_str:
+ vl %v16,0(%r1,%r2) /* Load search-string. */
+ lghi %r4,16 /* Loaded byte count is 16. */
+ vzero %v21 /* Zero out global mask. */
+ lghi %r5,0 /* Set current len of accept-string to zero. */
+ vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */
+ lghi %r8,0 /* There is no zero in first accept-part. */
+ vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */
+
+.Lslow_loop_acc:
+ vfaef %v22,%v16,%v17,4 /* Create matching-mask (1 in mask ->
+ character matches any accepted character in
+ this accept-string-part) IN=0, RT=1. */
+ vo %v21,%v21,%v22 /* global-mask = global- | matching-mask. */
+ vfenezf %v18,%v21,%v21 /* Find first zero in global-mask. */
+ vlgvb %r0,%v18,7 /* Get first found zero-index
+ (= first mismatch). */
+ clrjl %r0,%r6,.Lslow_next_acc /* Mismatch-index < min(lbc,zero-index)
+ -> Process this string-part
+ with next acc-part. */
+ clrjhe %r0,%r4,.Lslow_next_str /* Found-index >= loaded byte count
+ -> All loaded bytes are matching
+ any accept-character
+ and are not zero. */
+ /* All bytes are matching any characters in accept-string
+ and search-string is fully processed (found-index == zero-index). */
+.Lslow_add_lbc_end:
+ algrk %r2,%r1,%r0 /* Add matching characters to current len. */
+ srlg %r2,%r2,2 /* Convert byte-count to character-count. */
+ /* Restore registers. */
+ vlgvg %r6,%v30,0
+ vlgvg %r8,%v31,0
+ vlgvg %r9,%v31,1
+ br %r14
+
+.Lslow_next_acc:
+ clijh %r8,0,.Lslow_add_lbc_end /* There was a zero in last acc-part
+ -> Add found index to current len
+ and end. */
+ vlbb %v17,16(%r5,%r3),6 /* Load next accept part. */
+ aghi %r5,16 /* Increment current len of accept-string. */
+ lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of accept-string. */
+ jo .Lslow_next_acc_onbb /* Jump away if accept-string is
+ on block-boundary. */
+.Lslow_next_acc_notonbb:
+ vistrfs %v17,%v17 /* Fill with zeros after first zero. */
+ jo .Lslow_loop_acc /* No zero found -> no preparation needed. */
+
+.Lslow_next_acc_prepare_zero:
+ /* Zero in accept-part: fill zeros with first-accept-character. */
+ vlgvf %r8,%v17,0 /* Load first element of acc-part. */
+ clije %r8,0,.Lslow_add_lbc_end /* End if zero is first character
+ in this part of accept-string. */
+ /* r8>0 -> zero found in this acc-part. */
+ vrepf %v18,%v17,0 /* Replicate first char accross all chars. */
+ vceqf %v22,%v20,%v17 /* Create a mask (v22) of null chars
+ by comparing with 0 (v20). */
+ vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */
+ j .Lslow_loop_acc /* Accept part is prepared -> process. */
+
+.Lslow_next_acc_onbb:
+ nill %r9,65532 /* Recognize only fully loaded characters. */
+ je .Lslow_next_acc_onbb2 /* Reload vr, if we loaded no full
+ wchar_t. */
+ vfenezf %v18,%v17,%v17 /* Find zero in loaded bytes of accept part. */
+ vlgvb %r8,%v18,7 /* Load byte index of zero. */
+ clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes
+ -> Prepare vreg. */
+.Lslow_next_acc_onbb2:
+ vl %v17,0(%r5,%r3) /* Load over boundary ... */
+ lghi %r8,0 /* r8=0 -> no zero in this part of acc,
+ check for zero is in jump-target. */
+ j .Lslow_next_acc_notonbb /* ... and search for zero in
+ fully loaded vreg again. */
+.Lfallback:
+ jg __wcsspn_c
+END(__wcsspn_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcsspn.c b/sysdeps/s390/multiarch/wcsspn.c
similarity index 54%
copy from wcsmbs/wcsspn.c
copy to sysdeps/s390/multiarch/wcsspn.c
index db51acc..8dbd202 100644
--- a/wcsmbs/wcsspn.c
+++ b/sysdeps/s390/multiarch/wcsspn.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Multiple versions of wcsspn.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1995.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,31 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+s390_vx_libc_ifunc2 (__wcsspn, wcsspn)
-/* Return the length of the maximum initial segment
- of WCS which contains only wide-characters in ACCEPT. */
-size_t
-wcsspn (wcs, accept)
- const wchar_t *wcs;
- const wchar_t *accept;
-{
- const wchar_t *p;
- const wchar_t *a;
- size_t count = 0;
-
- for (p = wcs; *p != L'\0'; ++p)
- {
- for (a = accept; *a != L'\0'; ++a)
- if (*p == *a)
- break;
- if (*a == L'\0')
- return count;
- else
- ++count;
- }
-
- return count;
-}
-libc_hidden_def (wcsspn)
+#else
+# include <wcsmbs/wcsspn.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index a4c7077..b51e7b6 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -43,7 +43,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
mbrtoc16 c16rtomb
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
- wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul
+ wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul wcsspn
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/test-wcsspn.c b/wcsmbs/test-wcsspn.c
new file mode 100644
index 0000000..171043c
--- /dev/null
+++ b/wcsmbs/test-wcsspn.c
@@ -0,0 +1,20 @@
+/* Test wcsspn functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "../string/test-strspn.c"
diff --git a/wcsmbs/wcsspn.c b/wcsmbs/wcsspn.c
index db51acc..7a3cdf7 100644
--- a/wcsmbs/wcsspn.c
+++ b/wcsmbs/wcsspn.c
@@ -18,6 +18,9 @@
#include <wchar.h>
+#ifdef WCSSPN
+# define wcsspn WCSSPN
+#endif
/* Return the length of the maximum initial segment
of WCS which contains only wide-characters in ACCEPT. */
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=f40132d4bda984479bac89dfcd6968e9ff56e088
commit f40132d4bda984479bac89dfcd6968e9ff56e088
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:23 2015 +0200
S390: Optimize strrchr and wcsrchr.
This patch provides optimized versions of strrchr and wcsrchr with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strrchr-c.c: New File.
* sysdeps/s390/multiarch/strrchr-vx.S: Likewise.
* sysdeps/s390/multiarch/strrchr.c: Likewise.
* sysdeps/s390/multiarch/wcsrchr-c.c: Likewise.
* sysdeps/s390/multiarch/wcsrchr-vx.S: Likewise.
* sysdeps/s390/multiarch/wcsrchr.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strrchr and
wcsrchr functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strrchr, wcsrchr.
* benchtests/bench-wcsrchr.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcsrchr.
diff --git a/ChangeLog b/ChangeLog
index 3903d1f..4ece1f4 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,20 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strrchr-c.c: New File.
+ * sysdeps/s390/multiarch/strrchr-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strrchr.c: Likewise.
+ * sysdeps/s390/multiarch/wcsrchr-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcsrchr-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcsrchr.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strrchr and
+ wcsrchr functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strrchr, wcsrchr.
+ * benchtests/bench-wcsrchr.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcsrchr.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strchrnul-c.c: New File.
* sysdeps/s390/multiarch/strchrnul-vx.S: Likewise.
* sysdeps/s390/multiarch/strchrnul.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index b3298d4..6a0052a 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -37,7 +37,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
- wcscmp wcsncmp wcschr wcschrnul
+ wcscmp wcsncmp wcschr wcschrnul wcsrchr
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-wcsrchr.c b/benchtests/bench-wcsrchr.c
new file mode 100644
index 0000000..0d3923f
--- /dev/null
+++ b/benchtests/bench-wcsrchr.c
@@ -0,0 +1,20 @@
+/* Measure wcsrchr functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strrchr.c"
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index b6b64c9..b8b141e 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -10,7 +10,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strcmp strcmp-vx \
strncmp strncmp-vx strncmp-c \
strchr strchr-vx strchr-c \
- strchrnul strchrnul-vx strchrnul-c
+ strchrnul strchrnul-vx strchrnul-c \
+ strrchr strrchr-vx strrchr-c
endif
ifeq ($(subdir),wcsmbs)
@@ -25,5 +26,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcscmp wcscmp-vx wcscmp-c \
wcsncmp wcsncmp-vx wcsncmp-c \
wcschr wcschr-vx wcschr-c \
- wcschrnul wcschrnul-vx wcschrnul-c
+ wcschrnul wcschrnul-vx wcschrnul-c \
+ wcsrchr wcsrchr-vx wcsrchr-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index ea474a8..ee84d80 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -115,6 +115,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strchrnul);
IFUNC_VX_IMPL (wcschrnul);
+ IFUNC_VX_IMPL (strrchr);
+ IFUNC_VX_IMPL (wcsrchr);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/strrchr-c.c b/sysdeps/s390/multiarch/strrchr-c.c
new file mode 100644
index 0000000..345930a
--- /dev/null
+++ b/sysdeps/s390/multiarch/strrchr-c.c
@@ -0,0 +1,29 @@
+/* Default strrchr implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRRCHR __strrchr_c
+# undef weak_alias
+# ifdef SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strrchr_c, __GI_strrchr, __strrchr_c);
+# endif /* SHARED */
+
+# include <string/strrchr.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strrchr-vx.S b/sysdeps/s390/multiarch/strrchr-vx.S
new file mode 100644
index 0000000..eda48c0
--- /dev/null
+++ b/sysdeps/s390/multiarch/strrchr-vx.S
@@ -0,0 +1,180 @@
+/* Vector optimized 32/64 bit S/390 version of strrchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char *strrchr (const char *s, int c)
+ Locate the last character c in string.
+
+ Register usage:
+ -r0=loaded bytes in first part of s.
+ -r1=pointer to last occurence of c or NULL if not found.
+ -r2=s
+ -r3=c
+ -r4=tmp
+ -r5=current_len
+ -v16=part of s
+ -v17=index of found element
+ -v18=replicated c
+ -v19=part of s with last occurence of c.
+ -v20=permute pattern
+*/
+ENTRY(__strrchr_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vlvgb %v18,%r3,0 /* Generate vector which elements are all c.
+ if c > 255, c will be truncated. */
+ vrepb %v18,%v18,0
+
+ lghi %r1,-1 /* Currently no c found. */
+ lghi %r5,0 /* current_len = 0. */
+
+ vfeezbs %v17,%v16,%v18 /* Find element equal or zero. */
+ vlgvb %r4,%v17,7 /* Load byte index of c/zero or 16. */
+ clrjl %r4,%r0,.Lfound_first_part /* Found c/zero in loaded bytes. */
+.Lalign:
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r4 /* Compute bytes to 16bytes boundary. */
+
+.Lloop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfeezbs %v17,%v16,%v18 /* Find element equal with zero search. */
+ jno .Lfound /* Found c/zero (cc=0|1|2). */
+ vl %v16,16(%r5,%r2)
+ vfeezbs %v17,%v16,%v18
+ jno .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfeezbs %v17,%v16,%v18
+ jno .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfeezbs %v17,%v16,%v18
+ jno .Lfound48
+
+ aghi %r5,64
+ j .Lloop /* No character and no zero -> loop. */
+
+.Lfound48:
+ la %r5,16(%r5) /* Use la since aghi would clobber cc. */
+.Lfound32:
+ la %r5,16(%r5)
+.Lfound16:
+ la %r5,16(%r5)
+.Lfound:
+ je .Lzero /* Found zero, but no c before that zero. */
+ /* Save this part of s to check for further matches after reaching
+ the end of the complete string. */
+ vlr %v19,%v16
+ lgr %r1,%r5
+
+ jh .Lzero /* Found a zero after the found c. */
+ aghi %r5,16 /* Start search of next part of s. */
+ j .Lloop
+
+.Lfound_first_part:
+ /* This code is only executed if the found c/zero is whithin loaded
+ bytes. If no c/zero was found (cc==3) the found index = 16, thus
+ this code is not called.
+ Resulting condition code of vector find element equal:
+ cc==0: no c, found zero
+ cc==1: c found, no zero
+ cc==2: c found, found zero after c
+ cc==3: no c, no zero (this case can be ignored). */
+ je .Lzero /* Found zero, but no c before that zero. */
+
+ locgrne %r1,%r5 /* Mark c as found in first part of s. */
+ vlr %v19,%v16
+
+ jl .Lalign /* No zero (e.g. if vr was fully loaded)
+ -> Align and loop afterwards. */
+
+ /* Found a zero in vr. If vr was not fully loaded due to block
+ boundary, the remaining bytes are filled with zero and we can't
+ rely on zero indication of condition code here! */
+
+ vfenezb %v17,%v16,%v16 /* Find zero. */
+ vlgvb %r4,%v17,7 /* Load byte index of zero or 16. */
+ clrjl %r4,%r0,.Lzero /* Zero within loaded bytes -> end. */
+ j .Lalign /* Align and loop afterwards. */
+
+.Lend_searched_zero:
+ vlgvb %r4,%v17,7 /* Load byte index of zero. */
+ algr %r5,%r4
+ la %r2,0(%r5,%r2) /* Return pointer to zero. */
+ br %r14
+
+.Lzero:
+ /* Reached end of string. Check if one c was found before. */
+ clije %r3,0,.Lend_searched_zero /* Found zero and c is zero. */
+
+ cgfi %r1,-1 /* No c found -> return NULL. */
+ locghie %r2,0
+ ber %r14
+
+ larl %r3,.Lpermute_mask /* Load permute mask. */
+ vl %v20,0(%r3)
+
+ /* c was found and is part of v19. */
+ vfenezb %v17,%v19,%v19 /* Find zero. */
+ vlgvb %r4,%v17,7 /* Load byte index of zero or 16. */
+
+ clgfi %r5,0 /* Loaded byte count in v19 is 16, ... */
+ lochine %r0,16 /* ... if v19 is not the first part of s. */
+ ahi %r0,-1 /* Convert byte count to highest index. */
+
+ clr %r0,%r4
+ locrl %r4,%r0 /* r4 = min (zero-index, highest-index). */
+
+ /* Right-shift of v19 to mask bytes after zero. */
+ clije %r4,15,.Lzero_permute /* No shift is needed if highest index
+ in vr is 15. */
+ lhi %r0,15
+ slr %r0,%r4 /* Compute byte count for vector shift right. */
+ sll %r0,3 /* Convert to bit count. */
+ vlvgb %v17,%r0,7
+ vsrlb %v19,%v19,%v17 /* Vector shift right by byte by number of bytes
+ specified in bits 1-4 of byte 7 in v17. */
+
+ /* Reverse bytes in v19. */
+.Lzero_permute:
+ vperm %v19,%v19,%v19,%v20 /* Permute v19 to reversed order. */
+
+ /* Find c in reversed v19. */
+ vfeeb %v19,%v19,%v18 /* Find c. */
+ la %r2,0(%r1,%r2)
+ vlgvb %r3,%v19,7 /* Load byte index of c. */
+
+ /* Compute index in real s and return. */
+ slgr %r4,%r3
+ la %r2,0(%r4,%r2) /* Return pointer to zero. */
+ br %r14
+.Lpermute_mask:
+ .byte 0x0F,0x0E,0x0D,0x0C,0x0B,0x0A,0x09,0x08
+ .byte 0x07,0x06,0x05,0x04,0x03,0x02,0x01,0x00
+END(__strrchr_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strrchr.c b/sysdeps/s390/multiarch/strrchr.c
new file mode 100644
index 0000000..5de0381
--- /dev/null
+++ b/sysdeps/s390/multiarch/strrchr.c
@@ -0,0 +1,28 @@
+/* Multiple versions of strrchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__strrchr, strrchr)
+weak_alias (strrchr, rindex)
+
+#else
+# include <string/strrchr.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/multiarch/wcsrchr-c.c b/sysdeps/s390/multiarch/wcsrchr-c.c
new file mode 100644
index 0000000..b2a9900
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsrchr-c.c
@@ -0,0 +1,25 @@
+/* Default wcsrchr implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSRCHR __wcsrchr_c
+
+# include <wchar.h>
+extern __typeof (wcsrchr) __wcsrchr_c;
+# include <wcsmbs/wcsrchr.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcsrchr-vx.S b/sysdeps/s390/multiarch/wcsrchr-vx.S
new file mode 100644
index 0000000..984f1d3
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsrchr-vx.S
@@ -0,0 +1,190 @@
+/* Vector optimized 32/64 bit S/390 version of wcsrchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t *wcsrchr (const wchar_t *s, wchar_t c)
+ Locate the last character c in string.
+
+ Register usage:
+ -r0=loaded bytes in first part of s.
+ -r1=pointer to last occurence of c or NULL if not found.
+ -r2=s
+ -r3=c
+ -r4=tmp
+ -r5=current_len
+ -v16=part of s
+ -v17=index of found element
+ -v18=replicated c
+ -v19=part of s with last occurence of c.
+ -v20=permute pattern
+*/
+ENTRY(__wcsrchr_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ vlvgf %v18,%r3,0 /* Generate vector which elements are all c. */
+ vrepf %v18,%v18,0
+
+ lghi %r1,-1 /* Currently no c found. */
+ lghi %r5,0 /* current_len = 0. */
+
+ vfeezfs %v17,%v16,%v18 /* Find element equal or zero. */
+ vlgvb %r4,%v17,7 /* Load byte index of c/zero or 16. */
+ clrjl %r4,%r0,.Lfound_first_part /* Found c/zero in loaded bytes. */
+.Lalign:
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r4 /* Compute bytes to 16bytes boundary. */
+
+.Lloop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfeezfs %v17,%v16,%v18 /* Find element equal with zero search. */
+ jno .Lfound /* Found c/zero (cc=0|1|2). */
+ vl %v16,16(%r5,%r2)
+ vfeezfs %v17,%v16,%v18
+ jno .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfeezfs %v17,%v16,%v18
+ jno .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfeezfs %v17,%v16,%v18
+ jno .Lfound48
+
+ aghi %r5,64
+ j .Lloop /* No character and no zero -> loop. */
+
+.Lfound48:
+ la %r5,16(%r5) /* Use la since aghi would clobber cc. */
+.Lfound32:
+ la %r5,16(%r5)
+.Lfound16:
+ la %r5,16(%r5)
+.Lfound:
+ je .Lzero /* Found zero, but no c before that zero. */
+ /* Save this part of s to check for further matches after reaching
+ the end of the complete string. */
+ vlr %v19,%v16
+ lgr %r1,%r5
+
+ jh .Lzero /* Found a zero after the found c. */
+ aghi %r5,16 /* Start search of next part of s. */
+ j .Lloop
+
+.Lfound_first_part:
+ /* This code is only executed if the found c/zero is whithin loaded
+ bytes. If no c/zero was found (cc==3) the found index = 16, thus
+ this code is not called.
+ Resulting condition code of vector find element equal:
+ cc==0: no c, found zero
+ cc==1: c found, no zero
+ cc==2: c found, found zero after c
+ cc==3: no c, no zero (this case can be ignored). */
+ je .Lzero /* Found zero, but no c before that zero. */
+
+ locgrne %r1,%r5 /* Mark c as found in first part of s. */
+ vlr %v19,%v16
+
+ jl .Lalign /* No zero (e.g. if vr was fully loaded)
+ -> Align and loop afterwards. */
+
+ /* Found a zero in vr. If vr was not fully loaded due to block
+ boundary, the remaining bytes are filled with zero and we can't
+ rely on zero indication of condition code here! */
+
+ vfenezf %v17,%v16,%v16
+ vlgvb %r4,%v17,7 /* Load byte index of zero or 16. */
+ clrjl %r4,%r0,.Lzero /* Zero within loaded bytes -> end. */
+ j .Lalign /* Align and loop afterwards. */
+
+.Lend_searched_zero:
+ vlgvb %r4,%v17,7 /* Load byte index of zero. */
+ algr %r5,%r4
+ la %r2,0(%r5,%r2) /* Return pointer to zero. */
+ br %r14
+
+.Lzero:
+ /* Reached end of string. Check if one c was found before. */
+ clije %r3,0,.Lend_searched_zero /* Found zero and c is zero. */
+
+ cgfi %r1,-1 /* No c found -> return NULL. */
+ locghie %r2,0
+ ber %r14
+
+ larl %r3,.Lpermute_mask /* Load permute mask. */
+ vl %v20,0(%r3)
+
+ /* c was found and is part of v19. */
+ vfenezf %v17,%v19,%v19 /* Find zero. */
+ vlgvb %r4,%v17,7 /* Load byte index of zero or 16. */
+ ahi %r4,3 /* Found zero index is first byte,
+ thus highest byte index is last byte of
+ wchar_t zero. */
+
+ clgfi %r5,0 /* Loaded byte count in v19 is 16, ... */
+ lochine %r0,16 /* ... if v19 is not the first part of s. */
+ ahi %r0,-1 /* Convert byte count to highest index. */
+
+ clr %r0,%r4
+ locrl %r4,%r0 /* r4 = min (zero-index, highest-index). */
+
+ /* Right-shift of v19 to mask bytes after zero. */
+ clije %r4,15,.Lzero_permute /* No shift is needed if highest index
+ in vr is 15. */
+ lhi %r0,15
+ slr %r0,%r4 /* Compute byte count for vector shift left. */
+ sll %r0,3 /* Convert to bit count. */
+ vlvgb %v17,%r0,7
+ vsrlb %v19,%v19,%v17 /* Vector shift right by byte by number of bytes
+ specified in bits 1-4 of byte 7 in v17. */
+
+ /* Reverse bytes in v19. */
+.Lzero_permute:
+ vperm %v19,%v19,%v19,%v20 /* Permute v19 to reversed order. */
+
+ /* Find c in reversed v19. */
+ vfeef %v19,%v19,%v18 /* Find c. */
+ la %r2,0(%r1,%r2)
+ vlgvb %r3,%v19,7 /* Load byte index of c. */
+
+ /* Compute index in real s and return. */
+ slgr %r4,%r3
+ lay %r2,-3(%r4,%r2) /* Return pointer to zero. -3 is needed,
+ because the found byte index is reversed in
+ vector-register. Thus point to first byte of
+ wchar_t. */
+ br %r14
+.Lpermute_mask:
+ .byte 0x0C,0x0D,0x0E,0x0F,0x08,0x09,0x0A,0x0B
+ .byte 0x04,0x05,0x06,0x07,0x00,0x01,0x02,0x03
+.Lfallback:
+ jg __wcsrchr_c
+END(__wcsrchr_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcsrchr.c b/sysdeps/s390/multiarch/wcsrchr.c
new file mode 100644
index 0000000..c840289
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsrchr.c
@@ -0,0 +1,27 @@
+/* Multiple versions of wcsrchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__wcsrchr, wcsrchr)
+
+#else
+# include <wcsmbs/wcsrchr.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=d23d4ef19f1f5463745f2af205acb8c4975fd680
commit d23d4ef19f1f5463745f2af205acb8c4975fd680
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:23 2015 +0200
S390: Optimize strchrnul and wcschrnul.
This patch provides optimized versions of strchrnul and wcschrnul with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strchrnul-c.c: New File.
* sysdeps/s390/multiarch/strchrnul-vx.S: Likewise.
* sysdeps/s390/multiarch/strchrnul.c: Likewise.
* sysdeps/s390/multiarch/wcschrnul-c.c: Likewise.
* sysdeps/s390/multiarch/wcschrnul-vx.S: Likewise.
* sysdeps/s390/multiarch/wcschrnul.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strchrnul and
wcschrnul functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strchrnul, wcschrnul.
* wcsmbs/wcschrnul.c: Use WCSCHRNUL if defined.
* string/test-strchr.c: Add wcschrnul support.
* wcsmbs/test-wcschrnul.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcschrnul.
* benchtests/bench-strchr.c: Add wcschrnul support.
* benchtests/bench-wcschrnul.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcschrnul.
diff --git a/ChangeLog b/ChangeLog
index 04e8552..3903d1f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strchrnul-c.c: New File.
+ * sysdeps/s390/multiarch/strchrnul-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strchrnul.c: Likewise.
+ * sysdeps/s390/multiarch/wcschrnul-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcschrnul-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcschrnul.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strchrnul and
+ wcschrnul functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strchrnul, wcschrnul.
+ * wcsmbs/wcschrnul.c: Use WCSCHRNUL if defined.
+ * string/test-strchr.c: Add wcschrnul support.
+ * wcsmbs/test-wcschrnul.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcschrnul.
+ * benchtests/bench-strchr.c: Add wcschrnul support.
+ * benchtests/bench-wcschrnul.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcschrnul.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strchr-c.c: New File.
* sysdeps/s390/multiarch/strchr-vx.S: Likewise.
* sysdeps/s390/multiarch/strchr.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index b931f60..b3298d4 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -37,7 +37,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
- wcscmp wcsncmp wcschr
+ wcscmp wcsncmp wcschr wcschrnul
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-strchr.c b/benchtests/bench-strchr.c
index 89e177a..e4e2775 100644
--- a/benchtests/bench-strchr.c
+++ b/benchtests/bench-strchr.c
@@ -22,10 +22,14 @@
# define TEST_NAME "strchrnul"
# else
# define TEST_NAME "strchr"
-# endif
+# endif /* !USE_FOR_STRCHRNUL */
#else
-# define TEST_NAME "wcschr"
-#endif
+# ifdef USE_FOR_STRCHRNUL
+# define TEST_NAME "wcschrnul"
+# else
+# define TEST_NAME "wcschr"
+# endif /* !USE_FOR_STRCHRNUL */
+#endif /* WIDE */
#include "bench-string.h"
#ifndef WIDE
@@ -35,7 +39,7 @@
# define simple_STRCHR simple_STRCHRNUL
# else
# define STRCHR strchr
-# endif
+# endif /* !USE_FOR_STRCHRNUL */
# define STRLEN strlen
# define CHAR char
# define BIG_CHAR CHAR_MAX
@@ -44,20 +48,26 @@
# define UCHAR unsigned char
#else
# include <wchar.h>
-# define STRCHR wcschr
+# ifdef USE_FOR_STRCHRNUL
+# define STRCHR wcschrnul
+# define stupid_STRCHR stupid_WCSCHRNUL
+# define simple_STRCHR simple_WCSCHRNUL
+# else
+# define STRCHR wcschr
+# endif /* !USE_FOR_STRCHRNUL */
# define STRLEN wcslen
# define CHAR wchar_t
# define BIG_CHAR WCHAR_MAX
# define MIDDLE_CHAR 1121
# define SMALL_CHAR 851
# define UCHAR wchar_t
-#endif
+#endif /* WIDE */
#ifdef USE_FOR_STRCHRNUL
# define NULLRET(endptr) endptr
#else
# define NULLRET(endptr) NULL
-#endif
+#endif /* !USE_FOR_STRCHRNUL */
typedef CHAR *(*proto_t) (const CHAR *, int);
diff --git a/wcsmbs/wcschrnul.c b/benchtests/bench-wcschrnul.c
similarity index 67%
copy from wcsmbs/wcschrnul.c
copy to benchtests/bench-wcschrnul.c
index 4bc42d1..124b602 100644
--- a/wcsmbs/wcschrnul.c
+++ b/benchtests/bench-wcschrnul.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Measure wcschrnul functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,21 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
-
-
-/* Find the first occurrence of WC in WCS. */
-wchar_t *
-__wcschrnul (wcs, wc)
- const wchar_t *wcs;
- const wchar_t wc;
-{
- while (*wcs != L'\0')
- if (*wcs == wc)
- break;
- else
- ++wcs;
-
- return (wchar_t *) wcs;
-}
-weak_alias (__wcschrnul, wcschrnul)
+#define WIDE 1
+#include "bench-strchrnul.c"
diff --git a/string/test-strchr.c b/string/test-strchr.c
index ea06c43..b3a7ba4 100644
--- a/string/test-strchr.c
+++ b/string/test-strchr.c
@@ -1,4 +1,4 @@
-/* Test and measure STRCHR functions.
+/* Test STRCHR functions.
Copyright (C) 1999-2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
Written by Jakub Jelinek <jakub@redhat.com>, 1999.
@@ -24,10 +24,14 @@
# define TEST_NAME "strchrnul"
# else
# define TEST_NAME "strchr"
-# endif
+# endif /* !USE_FOR_STRCHRNUL */
#else
-# define TEST_NAME "wcschr"
-#endif
+# ifdef USE_FOR_STRCHRNUL
+# define TEST_NAME "wcschrnul"
+# else
+# define TEST_NAME "wcschr"
+# endif /* !USE_FOR_STRCHRNUL */
+#endif /* WIDE */
#include "test-string.h"
#ifndef WIDE
@@ -37,7 +41,7 @@
# define simple_STRCHR simple_STRCHRNUL
# else
# define STRCHR strchr
-# endif
+# endif /* !USE_FOR_STRCHRNUL */
# define STRLEN strlen
# define CHAR char
# define BIG_CHAR CHAR_MAX
@@ -47,7 +51,13 @@
# define L(s) s
#else
# include <wchar.h>
-# define STRCHR wcschr
+# ifdef USE_FOR_STRCHRNUL
+# define STRCHR wcschrnul
+# define stupid_STRCHR stupid_WCSCHRNUL
+# define simple_STRCHR simple_WCSCHRNUL
+# else
+# define STRCHR wcschr
+# endif /* !USE_FOR_STRCHRNUL */
# define STRLEN wcslen
# define CHAR wchar_t
# define BIG_CHAR WCHAR_MAX
@@ -55,13 +65,13 @@
# define SMALL_CHAR 851
# define UCHAR wchar_t
# define L(s) L ## s
-#endif
+#endif /* WIDE */
#ifdef USE_FOR_STRCHRNUL
# define NULLRET(endptr) endptr
#else
# define NULLRET(endptr) NULL
-#endif
+#endif /* !USE_FOR_STRCHRNUL */
typedef CHAR *(*proto_t) (const CHAR *, int);
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 1a3673b..b6b64c9 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -9,7 +9,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strncat strncat-vx strncat-c \
strcmp strcmp-vx \
strncmp strncmp-vx strncmp-c \
- strchr strchr-vx strchr-c
+ strchr strchr-vx strchr-c \
+ strchrnul strchrnul-vx strchrnul-c
endif
ifeq ($(subdir),wcsmbs)
@@ -23,5 +24,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsncat wcsncat-vx wcsncat-c \
wcscmp wcscmp-vx wcscmp-c \
wcsncmp wcsncmp-vx wcsncmp-c \
- wcschr wcschr-vx wcschr-c
+ wcschr wcschr-vx wcschr-c \
+ wcschrnul wcschrnul-vx wcschrnul-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index b0782ff..ea474a8 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -112,6 +112,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strchr);
IFUNC_VX_IMPL (wcschr);
+ IFUNC_VX_IMPL (strchrnul);
+ IFUNC_VX_IMPL (wcschrnul);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/wcsmbs/wcschrnul.c b/sysdeps/s390/multiarch/strchrnul-c.c
similarity index 67%
copy from wcsmbs/wcschrnul.c
copy to sysdeps/s390/multiarch/strchrnul-c.c
index 4bc42d1..1f288e4 100644
--- a/wcsmbs/wcschrnul.c
+++ b/sysdeps/s390/multiarch/strchrnul-c.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Default strchrnul implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,21 +16,11 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRCHRNUL __strchrnul_c
+# define __strchrnul STRCHRNUL
+# undef weak_alias
+# define weak_alias(name, alias)
-
-/* Find the first occurrence of WC in WCS. */
-wchar_t *
-__wcschrnul (wcs, wc)
- const wchar_t *wcs;
- const wchar_t wc;
-{
- while (*wcs != L'\0')
- if (*wcs == wc)
- break;
- else
- ++wcs;
-
- return (wchar_t *) wcs;
-}
-weak_alias (__wcschrnul, wcschrnul)
+# include <string/strchrnul.c>
+#endif
diff --git a/sysdeps/s390/multiarch/strchrnul-vx.S b/sysdeps/s390/multiarch/strchrnul-vx.S
new file mode 100644
index 0000000..5b43a93
--- /dev/null
+++ b/sysdeps/s390/multiarch/strchrnul-vx.S
@@ -0,0 +1,93 @@
+/* Vector optimized 32/64 bit S/390 version of strchrnul.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char *strchrnul (const char *s, int c)
+ Returns pointer to first c or to \0 if c not found.
+
+ Register usage:
+ -r1=tmp
+ -r2=s and return pointer
+ -r3=c
+ -r4=tmp
+ -r5=current_len
+ -v16=part of s
+ -v18=vector with c replicated in every byte
+*/
+ENTRY(__strchrnul_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ vlvgb %v18,%r3,0 /* Generate vector which elements are all c.
+ If c > 255, c will be truncated. */
+ vrepb %v18,%v18,0
+
+ vfeezbs %v16,%v16,%v18 /* Find element equal with zero search. */
+ vlgvb %r4,%v16,7 /* Load byte index of character or zero. */
+ clrjl %r4,%r1,.Lfound /* Return if c/zero is in loaded bytes. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r4 /* Compute bytes to 16bytes boundary. */
+
+ /* Find c/zero in 16byte aligned loop */
+.Lloop:
+ vl %v16,0(%r5,%r2) /* Load s */
+ vfeezbs %v16,%v16,%v18 /* Find element equal with zero search. */
+ jno .Lfound /* Found c/zero (cc=0|1|2). */
+ vl %v16,16(%r5,%r2)
+ vfeezbs %v16,%v16,%v18
+ jno .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfeezbs %v16,%v16,%v18
+ jno .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfeezbs %v16,%v16,%v18
+ jno .Lfound48
+
+ aghi %r5,64
+ j .Lloop /* No character and no zero -> loop. */
+
+ /* Found character or zero */
+.Lfound48:
+ aghi %r5,16
+.Lfound32:
+ aghi %r5,16
+.Lfound16:
+ aghi %r5,16
+.Lfound:
+ vlgvb %r1,%v16,7 /* Load byte index of character. */
+ algr %r5,%r1
+ la %r2,0(%r5,%r2) /* Return pointer to character. */
+
+.Lend:
+ br %r14
+END(__strchrnul_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcschrnul.c b/sysdeps/s390/multiarch/strchrnul.c
similarity index 67%
copy from wcsmbs/wcschrnul.c
copy to sysdeps/s390/multiarch/strchrnul.c
index 4bc42d1..652d517 100644
--- a/wcsmbs/wcschrnul.c
+++ b/sysdeps/s390/multiarch/strchrnul.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Multiple versions of strchrnul.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,21 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+s390_vx_libc_ifunc (__strchrnul)
+weak_alias (__strchrnul, strchrnul)
-/* Find the first occurrence of WC in WCS. */
-wchar_t *
-__wcschrnul (wcs, wc)
- const wchar_t *wcs;
- const wchar_t wc;
-{
- while (*wcs != L'\0')
- if (*wcs == wc)
- break;
- else
- ++wcs;
-
- return (wchar_t *) wcs;
-}
-weak_alias (__wcschrnul, wcschrnul)
+#else
+# include <string/strchrnul.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/wcschrnul.c b/sysdeps/s390/multiarch/wcschrnul-c.c
similarity index 67%
copy from wcsmbs/wcschrnul.c
copy to sysdeps/s390/multiarch/wcschrnul-c.c
index 4bc42d1..10c7ab6 100644
--- a/wcsmbs/wcschrnul.c
+++ b/sysdeps/s390/multiarch/wcschrnul-c.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Default wcschrnul implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,21 +16,10 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSCHRNUL __wcschrnul_c
-
-/* Find the first occurrence of WC in WCS. */
-wchar_t *
-__wcschrnul (wcs, wc)
- const wchar_t *wcs;
- const wchar_t wc;
-{
- while (*wcs != L'\0')
- if (*wcs == wc)
- break;
- else
- ++wcs;
-
- return (wchar_t *) wcs;
-}
-weak_alias (__wcschrnul, wcschrnul)
+# include <wchar.h>
+extern __typeof (__wcschrnul) __wcschrnul_c;
+# include <wcsmbs/wcschrnul.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcschrnul-vx.S b/sysdeps/s390/multiarch/wcschrnul-vx.S
new file mode 100644
index 0000000..f69064f
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcschrnul-vx.S
@@ -0,0 +1,97 @@
+/* Vector optimized 32/64 bit S/390 version of wcschrnul.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t* wcschrnul (const wchar_t *s, wchar_t c)
+ Returns pointer to first c or to \0 if c not found.
+
+ Register usage:
+ -r1=tmp
+ -r2=s and return pointer
+ -r3=c
+ -r4=tmp
+ -r5=current_len
+ -v16=part of s
+ -v18=vector with c replicated in every byte
+*/
+ENTRY(__wcschrnul_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ vlvgf %v18,%r3,0 /* Generate vector which elements are all c. */
+ vrepf %v18,%v18,0
+
+ vfeezfs %v16,%v16,%v18 /* Find element equal with zero search. */
+ vlgvb %r4,%v16,7 /* Load byte index of character or zero. */
+ clrjl %r4,%r1,.Lfound /* Return if c/zero is in loaded bytes. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r4 /* Compute bytes to 16bytes boundary. */
+
+ /* Find c/zero in 16byte aligned loop */
+.Lloop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfeezfs %v16,%v16,%v18 /* Find element equal with zero search. */
+ jno .Lfound /* Found c/zero (cc=0|1|2). */
+ vl %v16,16(%r5,%r2)
+ vfeezfs %v16,%v16,%v18
+ jno .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfeezfs %v16,%v16,%v18
+ jno .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfeezfs %v16,%v16,%v18
+ jno .Lfound48
+
+ aghi %r5,64
+ j .Lloop /* No character and no zero -> loop. */
+
+ /* Found character or zero */
+.Lfound48:
+ aghi %r5,16
+.Lfound32:
+ aghi %r5,16
+.Lfound16:
+ aghi %r5,16
+.Lfound:
+ vlgvb %r1,%v16,7 /* Load byte index of character. */
+ algr %r5,%r1
+ la %r2,0(%r5,%r2) /* Return pointer to character. */
+
+.Lend:
+ br %r14
+.Lfallback:
+ jg __wcschrnul_c
+END(__wcschrnul_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcschrnul.c b/sysdeps/s390/multiarch/wcschrnul.c
similarity index 70%
copy from wcsmbs/wcschrnul.c
copy to sysdeps/s390/multiarch/wcschrnul.c
index 4bc42d1..0f91e26 100644
--- a/wcsmbs/wcschrnul.c
+++ b/sysdeps/s390/multiarch/wcschrnul.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Multiple versions of wcschrnul.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,21 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
-
-/* Find the first occurrence of WC in WCS. */
-wchar_t *
-__wcschrnul (wcs, wc)
- const wchar_t *wcs;
- const wchar_t wc;
-{
- while (*wcs != L'\0')
- if (*wcs == wc)
- break;
- else
- ++wcs;
-
- return (wchar_t *) wcs;
-}
+s390_vx_libc_ifunc (__wcschrnul)
weak_alias (__wcschrnul, wcschrnul)
+
+#else
+# include <wcsmbs/wcschrnul.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index 3b6c472..a4c7077 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -43,7 +43,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
mbrtoc16 c16rtomb
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
- wcpcpy wcsncpy wcpncpy wcscat wcsncat
+ wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/wcschrnul.c b/wcsmbs/test-wcschrnul.c
similarity index 67%
copy from wcsmbs/wcschrnul.c
copy to wcsmbs/test-wcschrnul.c
index 4bc42d1..baea763 100644
--- a/wcsmbs/wcschrnul.c
+++ b/wcsmbs/test-wcschrnul.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1995-2015 Free Software Foundation, Inc.
+/* Test wcschrnul functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,21 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
-
-
-/* Find the first occurrence of WC in WCS. */
-wchar_t *
-__wcschrnul (wcs, wc)
- const wchar_t *wcs;
- const wchar_t wc;
-{
- while (*wcs != L'\0')
- if (*wcs == wc)
- break;
- else
- ++wcs;
-
- return (wchar_t *) wcs;
-}
-weak_alias (__wcschrnul, wcschrnul)
+#define WIDE 1
+#include "../string/test-strchrnul.c"
diff --git a/wcsmbs/wcschrnul.c b/wcsmbs/wcschrnul.c
index 4bc42d1..97ef3b9 100644
--- a/wcsmbs/wcschrnul.c
+++ b/wcsmbs/wcschrnul.c
@@ -17,6 +17,9 @@
#include <wchar.h>
+#ifdef WCSCHRNUL
+# define __wcschrnul WCSCHRNUL
+#endif
/* Find the first occurrence of WC in WCS. */
wchar_t *
@@ -32,4 +35,6 @@ __wcschrnul (wcs, wc)
return (wchar_t *) wcs;
}
+#ifndef WCSCHRNUL
weak_alias (__wcschrnul, wcschrnul)
+#endif
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=cf150d45a97c6bb8410d6d3acf6e1560e8fe96cc
commit cf150d45a97c6bb8410d6d3acf6e1560e8fe96cc
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:23 2015 +0200
S390: Optimize strchr and wcschr.
This patch provides optimized versions of strchr and wcschr with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strchr-c.c: New File.
* sysdeps/s390/multiarch/strchr-vx.S: Likewise.
* sysdeps/s390/multiarch/strchr.c: Likewise.
* sysdeps/s390/multiarch/wcschr-c.c: Likewise.
* sysdeps/s390/multiarch/wcschr-vx.S: Likewise.
* sysdeps/s390/multiarch/wcschr.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strchr and
wcschr functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strchr, wcschr.
* string/strchr.c (STRCHR): Define and use macro.
* benchtests/bench-wcschr.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcschr.
diff --git a/ChangeLog b/ChangeLog
index a9cdda3..04e8552 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,21 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strchr-c.c: New File.
+ * sysdeps/s390/multiarch/strchr-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strchr.c: Likewise.
+ * sysdeps/s390/multiarch/wcschr-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcschr-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcschr.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strchr and
+ wcschr functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strchr, wcschr.
+ * string/strchr.c (STRCHR): Define and use macro.
+ * benchtests/bench-wcschr.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcschr.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strncmp-c.c: New File.
* sysdeps/s390/multiarch/strncmp-vx.S: Likewise.
* sysdeps/s390/multiarch/strncmp.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 0a85797..b931f60 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -37,7 +37,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
- wcscmp wcsncmp
+ wcscmp wcsncmp wcschr
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-wcschr.c b/benchtests/bench-wcschr.c
new file mode 100644
index 0000000..4287724
--- /dev/null
+++ b/benchtests/bench-wcschr.c
@@ -0,0 +1,20 @@
+/* Measure wcschr functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strchr.c"
diff --git a/string/strchr.c b/string/strchr.c
index f13b2b3..f73891d 100644
--- a/string/strchr.c
+++ b/string/strchr.c
@@ -25,9 +25,13 @@
#undef strchr
+#ifndef STRCHR
+# define STRCHR strchr
+#endif
+
/* Find the first occurrence of C in S. */
char *
-strchr (const char *s, int c_in)
+STRCHR (const char *s, int c_in)
{
const unsigned char *char_ptr;
const unsigned long int *longword_ptr;
@@ -175,7 +179,7 @@ strchr (const char *s, int c_in)
}
#ifdef weak_alias
-#undef index
+# undef index
weak_alias (strchr, index)
#endif
libc_hidden_builtin_def (strchr)
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index d77bee5..1a3673b 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -8,7 +8,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strcat strcat-vx strcat-c \
strncat strncat-vx strncat-c \
strcmp strcmp-vx \
- strncmp strncmp-vx strncmp-c
+ strncmp strncmp-vx strncmp-c \
+ strchr strchr-vx strchr-c
endif
ifeq ($(subdir),wcsmbs)
@@ -21,5 +22,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcscat wcscat-vx wcscat-c \
wcsncat wcsncat-vx wcsncat-c \
wcscmp wcscmp-vx wcscmp-c \
- wcsncmp wcsncmp-vx wcsncmp-c
+ wcsncmp wcsncmp-vx wcsncmp-c \
+ wcschr wcschr-vx wcschr-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 5bfc493..b0782ff 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -109,6 +109,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strncmp);
IFUNC_VX_IMPL (wcsncmp);
+ IFUNC_VX_IMPL (strchr);
+ IFUNC_VX_IMPL (wcschr);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/strchr-c.c b/sysdeps/s390/multiarch/strchr-c.c
new file mode 100644
index 0000000..a71f878
--- /dev/null
+++ b/sysdeps/s390/multiarch/strchr-c.c
@@ -0,0 +1,29 @@
+/* Default strchr implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRCHR __strchr_c
+# undef weak_alias
+# ifdef SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strchr_c, __GI_strchr, __strchr_c);
+# endif /* SHARED */
+
+# include <string/strchr.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strchr-vx.S b/sysdeps/s390/multiarch/strchr-vx.S
new file mode 100644
index 0000000..db740ab
--- /dev/null
+++ b/sysdeps/s390/multiarch/strchr-vx.S
@@ -0,0 +1,100 @@
+/* Vector optimized 32/64 bit S/390 version of strchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char *strchr (const char *s, int c)
+ Locate character in string.
+
+ Register usage:
+ -r1=tmp
+ -r2=s
+ -r3=c
+ -r4=tmp
+ -r5=current_len
+ -v16=part of s
+ -v17=index of unequal
+ -v18=replicated c
+*/
+ENTRY(__strchr_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ vlvgb %v18,%r3,0 /* Generate vector which elements are all c.
+ If c > 255, c will be truncated. */
+ vrepb %v18,%v18,0
+
+ vfeezbs %v16,%v16,%v18 /* Find element equal with zero search. */
+ vlgvb %r4,%v16,7 /* Load byte index of character or zero. */
+ clrjl %r4,%r1,.Lfound /* Return if c/zero is in loaded bytes. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r4 /* Compute bytes to 16bytes boundary. */
+
+ /* Find c/zero in 16 byte aligned loop */
+.Lloop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfeezbs %v16,%v16,%v18 /* Find element equal with zero search. */
+ jno .Lfound /* Found c/zero (cc=0|1|2). */
+ vl %v16,16(%r5,%r2)
+ vfeezbs %v16,%v16,%v18
+ jno .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfeezbs %v16,%v16,%v18
+ jno .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfeezbs %v16,%v16,%v18
+ jno .Lfound48
+
+ aghi %r5,64
+ j .Lloop /* No character and no zero -> loop. */
+
+.Lfound48:
+ la %r5,16(%r5) /* Use la since aghi would clobber cc. */
+.Lfound32:
+ la %r5,16(%r5)
+.Lfound16:
+ la %r5,16(%r5)
+.Lfound:
+ je .Lzero /* Found zero, but no c before that zero. */
+
+.Lcharacter:
+ vlgvb %r4,%v16,7 /* Load byte index of character. */
+ algr %r5,%r4
+ la %r2,0(%r5,%r2) /* Return pointer to character. */
+ br %r14
+
+.Lzero:
+ llgcr %r3,%r3 /* char c_char = (char) c. */
+ clije %r3,0,.Lcharacter /* Found zero and c is zero. */
+ lghi %r2,0 /* Return null if character not found. */
+ br %r14
+END(__strchr_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strchr.c b/sysdeps/s390/multiarch/strchr.c
new file mode 100644
index 0000000..40419b9
--- /dev/null
+++ b/sysdeps/s390/multiarch/strchr.c
@@ -0,0 +1,28 @@
+/* Multiple versions of strchr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__strchr, strchr)
+weak_alias (strchr, index)
+
+#else
+# include <string/strchr.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/multiarch/wcschr-c.c b/sysdeps/s390/multiarch/wcschr-c.c
new file mode 100644
index 0000000..c6ffb2f
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcschr-c.c
@@ -0,0 +1,37 @@
+/* Default wcschr implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSCHR __wcschr_c
+
+# include <wchar.h>
+extern __typeof (__wcschr) __wcschr_c;
+# undef weak_alias
+# define weak_alias(name, alias)
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__wcschr_c, __GI_wcschr, __wcschr_c); \
+ strong_alias (__wcschr_c, __wcschr_c_1); \
+ __hidden_ver1 (__wcschr_c_1, __GI___wcschr, __wcschr_c_1);
+# undef libc_hidden_weak
+# define libc_hidden_weak(name)
+# endif /* SHARED */
+
+# include <wcsmbs/wcschr.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcschr-vx.S b/sysdeps/s390/multiarch/wcschr-vx.S
new file mode 100644
index 0000000..d29c559
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcschr-vx.S
@@ -0,0 +1,103 @@
+/* Vector optimized 32/64 bit S/390 version of wcschr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t *wcschr (const wchar_t *s, wchar_t c)
+ Locate character in string.
+
+ Register usage:
+ -r1=tmp
+ -r2=s
+ -r3=c
+ -r4=tmp
+ -r5=current_len
+ -v16=part of s
+ -v17=index of unequal
+ -v18=replicated c
+*/
+ENTRY(__wcschr_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ vlvgf %v18,%r3,0 /* Generate vector which elements are all c. */
+ vrepf %v18,%v18,0
+
+ vfeezfs %v16,%v16,%v18 /* Find element equal with zero search. */
+ vlgvb %r4,%v16,7 /* Load byte index of character or zero. */
+ clrjl %r4,%r1,.Lfound /* Return if c/zero is in loaded bytes. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r4 /* Compute bytes to 16bytes boundary. */
+
+ /* Find c/zero in 16byte aligned loop */
+.Lloop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfeezfs %v16,%v16,%v18 /* Find element equal with zero search. */
+ jno .Lfound /* Found c/zero (cc=0|1|2). */
+ vl %v16,16(%r5,%r2)
+ vfeezfs %v16,%v16,%v18
+ jno .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfeezfs %v16,%v16,%v18
+ jno .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfeezfs %v16,%v16,%v18
+ jno .Lfound48
+
+ aghi %r5,64
+ j .Lloop /* No character and no zero -> loop. */
+
+.Lfound48:
+ la %r5,16(%r5) /* Use la since aghi would clobber cc. */
+.Lfound32:
+ la %r5,16(%r5)
+.Lfound16:
+ la %r5,16(%r5)
+.Lfound:
+ je .Lzero /* Found zero, but no c before that zero. */
+
+.Lcharacter:
+ vlgvb %r4,%v16,7 /* Load byte index of character. */
+ algr %r5,%r4
+ la %r2,0(%r5,%r2) /* Return pointer to character. */
+ br %r14
+
+.Lzero:
+ clije %r3,0,.Lcharacter /* Found zero and c is zero. */
+ lghi %r2,0 /* Return null if character not found. */
+ br %r14
+.Lfallback:
+ jg __wcschr_c
+END(__wcschr_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcschr.c b/sysdeps/s390/multiarch/wcschr.c
new file mode 100644
index 0000000..40281eb
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcschr.c
@@ -0,0 +1,29 @@
+/* Multiple versions of wcschr.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc (__wcschr)
+weak_alias (__wcschr, wcschr)
+libc_hidden_weak (wcschr)
+
+#else
+# include <wcsmbs/wcschr.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=cee82e70ccb7b2f054cd781b0a603ae244523e72
commit cee82e70ccb7b2f054cd781b0a603ae244523e72
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:22 2015 +0200
S390: Optimize strncmp and wcsncmp.
This patch provides optimized versions of strncmp and wcsncmp with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strncmp-c.c: New File.
* sysdeps/s390/multiarch/strncmp-vx.S: Likewise.
* sysdeps/s390/multiarch/strncmp.c: Likewise.
* sysdeps/s390/multiarch/wcsncmp-c.c: Likewise.
* sysdeps/s390/multiarch/wcsncmp-vx.S: Likewise.
* sysdeps/s390/multiarch/wcsncmp.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strncmp and
wcsncmp functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strncmp, wcsncmp.
* wcsmbs/wcsncmp.c (WCSNCMP): Define and use macro.
* benchtests/bench-strncmp.c: Add wcsncmp support.
* benchtests/bench-wcsncmp.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcsncmp.
diff --git a/ChangeLog b/ChangeLog
index 8dfab01..a9cdda3 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,22 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strncmp-c.c: New File.
+ * sysdeps/s390/multiarch/strncmp-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strncmp.c: Likewise.
+ * sysdeps/s390/multiarch/wcsncmp-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcsncmp-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcsncmp.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strncmp and
+ wcsncmp functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strncmp, wcsncmp.
+ * wcsmbs/wcsncmp.c (WCSNCMP): Define and use macro.
+ * benchtests/bench-strncmp.c: Add wcsncmp support.
+ * benchtests/bench-wcsncmp.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcsncmp.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strcmp-vx.S: New File.
* sysdeps/s390/multiarch/strcmp.c: Likewise.
* sysdeps/s390/multiarch/wcscmp-c.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index b47d5c1..0a85797 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -37,7 +37,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
- wcscmp
+ wcscmp wcsncmp
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-strncmp.c b/benchtests/bench-strncmp.c
index 1d06e5b..3b26414 100644
--- a/benchtests/bench-strncmp.c
+++ b/benchtests/bench-strncmp.c
@@ -17,17 +17,68 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "strncmp"
+#ifdef WIDE
+# define TEST_NAME "wcsncmp"
+#else
+# define TEST_NAME "strncmp"
+#endif /* !WIDE */
#include "bench-string.h"
-typedef int (*proto_t) (const char *, const char *, size_t);
-int simple_strncmp (const char *, const char *, size_t);
-int stupid_strncmp (const char *, const char *, size_t);
+#ifdef WIDE
+# include <wchar.h>
-IMPL (stupid_strncmp, 0)
-IMPL (simple_strncmp, 0)
-IMPL (strncmp, 1)
+# define L(str) L##str
+# define STRNCMP wcsncmp
+# define SIMPLE_STRNCMP simple_wcsncmp
+# define STUPID_STRNCMP stupid_wcsncmp
+# define CHAR wchar_t
+# define CHARBYTES 4
+/* Wcsncmp uses signed semantics for comparison, not unsigned.
+ Avoid using substraction since possible overflow. */
+int
+simple_wcsncmp (const CHAR *s1, const CHAR *s2, size_t n)
+{
+ wchar_t c1, c2;
+
+ while (n--)
+ {
+ c1 = *s1++;
+ c2 = *s2++;
+ if (c1 == L ('\0') || c1 != c2)
+ return c1 > c2 ? 1 : (c1 < c2 ? -1 : 0);
+ }
+ return 0;
+}
+
+int
+stupid_wcsncmp (const CHAR *s1, const CHAR *s2, size_t n)
+{
+ wchar_t c1, c2;
+ size_t ns1 = wcsnlen (s1, n) + 1, ns2 = wcsnlen (s2, n) + 1;
+
+ n = ns1 < n ? ns1 : n;
+ n = ns2 < n ? ns2 : n;
+
+ while (n--)
+ {
+ c1 = *s1++;
+ c2 = *s2++;
+ if (c1 != c2)
+ return c1 > c2 ? 1 : -1;
+ }
+ return 0;
+}
+
+#else
+# define L(str) str
+# define STRNCMP strncmp
+# define SIMPLE_STRNCMP simple_strncmp
+# define STUPID_STRNCMP stupid_strncmp
+# define CHAR char
+# define CHARBYTES 1
+
+/* Strncmp uses unsigned semantics for comparison. */
int
simple_strncmp (const char *s1, const char *s2, size_t n)
{
@@ -46,12 +97,21 @@ stupid_strncmp (const char *s1, const char *s2, size_t n)
n = ns1 < n ? ns1 : n;
n = ns2 < n ? ns2 : n;
- while (n-- && (ret = *(unsigned char *) s1++ - * (unsigned char *) s2++) == 0);
+ while (n-- && (ret = *(unsigned char *) s1++ - *(unsigned char *) s2++) == 0);
return ret;
}
+#endif /* !WIDE */
+
+typedef int (*proto_t) (const CHAR *, const CHAR *, size_t);
+
+IMPL (STUPID_STRNCMP, 0)
+IMPL (SIMPLE_STRNCMP, 0)
+IMPL (STRNCMP, 1)
+
+
static void
-do_one_test (impl_t *impl, const char *s1, const char *s2, size_t n,
+do_one_test (impl_t *impl, const CHAR *s1, const CHAR *s2, size_t n,
int exp_result)
{
size_t i, iters = INNER_LOOP_ITERS;
@@ -74,12 +134,12 @@ do_test_limit (size_t align1, size_t align2, size_t len, size_t n, int max_char,
int exp_result)
{
size_t i, align_n;
- char *s1, *s2;
+ CHAR *s1, *s2;
if (n == 0)
{
- s1 = (char*)(buf1 + page_size);
- s2 = (char*)(buf2 + page_size);
+ s1 = (CHAR *) (buf1 + page_size);
+ s2 = (CHAR *) (buf2 + page_size);
printf ("Length %4zd/%4zd:", len, n);
FOR_EACH_IMPL (impl, 0)
@@ -92,16 +152,16 @@ do_test_limit (size_t align1, size_t align2, size_t len, size_t n, int max_char,
align1 &= 15;
align2 &= 15;
- align_n = (page_size - n) & 15;
+ align_n = (page_size - n * CHARBYTES) & 15;
- s1 = (char*)(buf1 + page_size - n);
- s2 = (char*)(buf2 + page_size - n);
+ s1 = (CHAR *) (buf1 + page_size - n * CHARBYTES);
+ s2 = (CHAR *) (buf2 + page_size - n * CHARBYTES);
if (align1 < align_n)
- s1 -= (align_n - align1);
+ s1 = (CHAR *) ((char *) s1 - (align_n - align1));
if (align2 < align_n)
- s2 -= (align_n - align2);
+ s2 = (CHAR *) ((char *) s2 - (align_n - align2));
for (i = 0; i < n; i++)
s1[i] = s2[i] = 1 + 23 * i % max_char;
@@ -129,24 +189,24 @@ do_test (size_t align1, size_t align2, size_t len, size_t n, int max_char,
int exp_result)
{
size_t i;
- char *s1, *s2;
+ CHAR *s1, *s2;
if (n == 0)
return;
- align1 &= 7;
- if (align1 + n + 1 >= page_size)
+ align1 &= 63;
+ if (align1 + (n + 1) * CHARBYTES >= page_size)
return;
align2 &= 7;
- if (align2 + n + 1 >= page_size)
+ if (align2 + (n + 1) * CHARBYTES >= page_size)
return;
- s1 = (char*)(buf1 + align1);
- s2 = (char*)(buf2 + align2);
+ s1 = (CHAR *) (buf1 + align1);
+ s2 = (CHAR *) (buf2 + align2);
for (i = 0; i < n; i++)
- s1[i] = s2[i] = 1 + 23 * i % max_char;
+ s1[i] = s2[i] = 1 + (23 << ((CHARBYTES - 1) * 8)) * i % max_char;
s1[n] = 24 + exp_result;
s2[n] = 23;
@@ -162,7 +222,7 @@ do_test (size_t align1, size_t align2, size_t len, size_t n, int max_char,
printf ("Length %4zd/%4zd, alignment %2zd/%2zd:", len, n, align1, align2);
FOR_EACH_IMPL (impl, 0)
- do_one_test (impl, (char*)s1, (char*)s2, n, exp_result);
+ do_one_test (impl, s1, s2, n, exp_result);
putchar ('\n');
}
diff --git a/benchtests/bench-wcsncmp.c b/benchtests/bench-wcsncmp.c
new file mode 100644
index 0000000..8720060
--- /dev/null
+++ b/benchtests/bench-wcsncmp.c
@@ -0,0 +1,20 @@
+/* Measure wcsncmp functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strncmp.c"
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index d8fbd55..d77bee5 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -7,7 +7,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
stpncpy stpncpy-vx stpncpy-c \
strcat strcat-vx strcat-c \
strncat strncat-vx strncat-c \
- strcmp strcmp-vx
+ strcmp strcmp-vx \
+ strncmp strncmp-vx strncmp-c
endif
ifeq ($(subdir),wcsmbs)
@@ -19,5 +20,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcpncpy wcpncpy-vx wcpncpy-c \
wcscat wcscat-vx wcscat-c \
wcsncat wcsncat-vx wcsncat-c \
- wcscmp wcscmp-vx wcscmp-c
+ wcscmp wcscmp-vx wcscmp-c \
+ wcsncmp wcsncmp-vx wcsncmp-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 196d3ec..5bfc493 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -106,6 +106,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strcmp);
IFUNC_VX_IMPL (wcscmp);
+ IFUNC_VX_IMPL (strncmp);
+ IFUNC_VX_IMPL (wcsncmp);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/strncmp-c.c b/sysdeps/s390/multiarch/strncmp-c.c
new file mode 100644
index 0000000..b32a994
--- /dev/null
+++ b/sysdeps/s390/multiarch/strncmp-c.c
@@ -0,0 +1,28 @@
+/* Default strncmp implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRNCMP __strncmp_c
+# ifdef SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strncmp_c, __GI_strncmp, __strncmp_c);
+# endif /* SHARED */
+
+# include <string/strncmp.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strncmp-vx.S b/sysdeps/s390/multiarch/strncmp-vx.S
new file mode 100644
index 0000000..ad01079
--- /dev/null
+++ b/sysdeps/s390/multiarch/strncmp-vx.S
@@ -0,0 +1,137 @@
+/* Vector optimized 32/64 bit S/390 version of strncmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* int strncmp (const char *s1, const char *s2, size_t n)
+ Compare at most n characters of two strings.
+
+ Register usage:
+ -r0=tmp
+ -r1=tmp
+ -r2=s1
+ -r3=s2
+ -r4=n
+ -r5=current_len
+ -v16=part of s1
+ -v17=part of s2
+ -v18=index of unequal
+*/
+ENTRY(__strncmp_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgije %r4,0,.Lend_equal /* Nothing to do if n == 0, */
+ lghi %r5,0 /* current_len = 0. */
+
+.Lloop:
+ vlbb %v16,0(%r5,%r2),6 /* Load s1 to block boundary. */
+ vlbb %v17,0(%r5,%r3),6 /* Load s2 to block boundary. */
+ lcbb %r0,0(%r5,%r2),6 /* Get loaded byte count of s1. */
+ jo .Llt16_1 /* Jump away if vr is not fully loaded. */
+ lcbb %r1,0(%r5,%r3),6 /* Get loaded byte count of s2. */
+ jo .Llt16_2 /* Jump away if vr is not fully loaded. */
+ aghi %r5,16 /* Both vrs are fully loaded. */
+ clgrjhe %r5,%r4,.Llastcmp /* If current_len >= n ->last compare. */
+ vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */
+ jno .Lfound
+
+ vlbb %v16,0(%r5,%r2),6
+ vlbb %v17,0(%r5,%r3),6
+ lcbb %r0,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r1,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ clgrjhe %r5,%r4,.Llastcmp
+ vfenezbs %v18,%v16,%v17
+ jno .Lfound
+
+ vlbb %v16,0(%r5,%r2),6
+ vlbb %v17,0(%r5,%r3),6
+ lcbb %r0,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r1,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ clgrjhe %r5,%r4,.Llastcmp
+ vfenezbs %v18,%v16,%v17
+ jno .Lfound
+
+ vlbb %v16,0(%r5,%r2),6
+ vlbb %v17,0(%r5,%r3),6
+ lcbb %r0,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r1,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ clgrjhe %r5,%r4,.Llastcmp
+ vfenezbs %v18,%v16,%v17
+ jno .Lfound
+ j .Lloop
+
+.Llt16_1:
+ lcbb %r1,0(%r5,%r3),6 /* Get loaded byte count ofs2. */
+.Llt16_2:
+ clr %r0,%r1 /* Compare logical. */
+ locrh %r0,%r1 /* Compute minimum of bytes loaded. */
+ algfr %r5,%r0 /* Add smallest loaded bytes to current_len. */
+ clgrj %r5,%r4,10,.Llastcmp /* If current_len >= n ->last compare. */
+ vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */
+ vlgvb %r1,%v18,7 /* Get not equal index or 16 if all equal. */
+ clrjl %r1,%r0,.Lfound /* Jump away if miscompare is within
+ loaded bytes (index < loaded-bytes) */
+ j .Lloop
+
+.Llastcmp:
+ /* Use comparision result only if located within first n characters.
+ %r0: loaded byte count in vreg;
+ %r5: current_len;
+ %r4: n;
+ (current_len - n): [0...16[
+ First ignored match index: loaded bytes - (current_len-n): ]0...16]
+ */
+ slgr %r5,%r4 /* %r5 = current_len - n. */
+ slr %r0,%r5 /* %r0 = first ignored match index. */
+ vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */
+ vlgvb %r1,%v18,7 /* Get not equal index or 16 if all equal. */
+ clrjl %r1,%r0,.Lfound /* Jump away if miscompare is within
+ loaded bytes and below n bytes. */
+ j .Lend_equal /* Miscompare after n-bytes -> end equal. */
+
+.Lfound:
+ /* Difference or end of string. */
+ je .Lend_equal
+ lghi %r2,1
+ lghi %r1,-1
+ locgrl %r2,%r1
+ br %r14
+.Lend_equal:
+ lghi %r2,0
+ br %r14
+END(__strncmp_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strncmp.c b/sysdeps/s390/multiarch/strncmp.c
new file mode 100644
index 0000000..9accafc
--- /dev/null
+++ b/sysdeps/s390/multiarch/strncmp.c
@@ -0,0 +1,30 @@
+/* Multiple versions of strncmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+
+# undef strcmp
+extern __typeof (strncmp) __strncmp;
+s390_vx_libc_ifunc2 (__strncmp, strncmp)
+
+#else
+# include <string/strncmp.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/multiarch/wcsncmp-c.c b/sysdeps/s390/multiarch/wcsncmp-c.c
new file mode 100644
index 0000000..3d796ca
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsncmp-c.c
@@ -0,0 +1,25 @@
+/* Default wcsncmp implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSNCMP __wcsncmp_c
+
+# include <wchar.h>
+extern __typeof (wcsncmp) __wcsncmp_c;
+# include <wcsmbs/wcsncmp.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcsncmp-vx.S b/sysdeps/s390/multiarch/wcsncmp-vx.S
new file mode 100644
index 0000000..e13727b
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsncmp-vx.S
@@ -0,0 +1,177 @@
+/* Vector optimized 32/64 bit S/390 version of wcsncmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* int wcsncmp (const wchar_t *s1, const wchar_t *s2, size_t n)
+ Compare at most n characters of two strings.
+
+ Register usage:
+ -r0=tmp
+ -r1=tmp
+ -r2=s1
+ -r3=s2
+ -r4=n
+ -r5=current_len
+ -v16=part of s1
+ -v17=part of s2
+ -v18=index of unequal
+*/
+ENTRY(__wcsncmp_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgije %r4,0,.Lend_equal /* Nothing to do if n == 0. */
+
+ /* Check range of n and convert to byte-count. */
+# ifdef __s390x__
+ tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ lghi %r1,-4 /* Max byte-count is 18446744073709551612. */
+# else
+ tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ llilf %r1,4294967292 /* Max byte-count is 4294967292. */
+# endif /* !__s390x__ */
+ sllg %r4,%r4,2 /* Convert character-count to byte-count. */
+ locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */
+
+ /* Check first character without vector load. */
+ lghi %r5,4 /* current_len = 4 bytes. */
+ /* Check s1/2[0]. */
+ lt %r0,0(%r2)
+ l %r1,0(%r3)
+ je .Lend_cmp_one_char
+ crjne %r0,%r1,.Lend_cmp_one_char
+
+.Lloop:
+ vlbb %v17,0(%r5,%r3),6 /* Load s2 to block boundary. */
+ vlbb %v16,0(%r5,%r2),6 /* Load s1 to block boundary. */
+ lcbb %r0,0(%r5,%r2),6 /* Get loaded byte count of s1. */
+ jo .Llt16_1 /* Jump away if vector not fully loaded. */
+ lcbb %r1,0(%r5,%r3),6 /* Get loaded byte count of s2. */
+ jo .Llt16_2 /* Jump away if vector not fully loaded. */
+ aghi %r5,16 /* Both vectors are fully loaded. */
+ vfenezfs %v18,%v16,%v17 /* Compare not equal with zero search. */
+ clgrjhe %r5,%r4,.Llastcmp /* If current_len >= n ->last compare. */
+ jno .Lfound
+
+ vlbb %v17,0(%r5,%r3),6
+ vlbb %v16,0(%r5,%r2),6
+ lcbb %r0,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r1,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ vfenezfs %v18,%v16,%v17
+ clgrjhe %r5,%r4,.Llastcmp
+ jno .Lfound
+
+ vlbb %v17,0(%r5,%r3),6
+ vlbb %v16,0(%r5,%r2),6
+ lcbb %r0,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r1,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ vfenezfs %v18,%v16,%v17
+ clgrjhe %r5,%r4,.Llastcmp
+ jno .Lfound
+
+ vlbb %v17,0(%r5,%r3),6
+ vlbb %v16,0(%r5,%r2),6
+ lcbb %r0,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r1,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ vfenezfs %v18,%v16,%v17
+ clgrjhe %r5,%r4,.Llastcmp
+ jno .Lfound
+
+ j .Lloop
+
+.Llt16_1:
+ lcbb %r1,0(%r5,%r3),6 /* Get loaded byte count of s2. */
+.Llt16_2:
+ clr %r0,%r1 /* Compare logical. */
+ locrh %r0,%r1 /* Compute minimum of bytes loaded. */
+ nill %r0,65532 /* Align bytes loaded to full characters. */
+ jz .Lcmp_one_char /* Jump away if no full char is available. */
+.Llt_cmp:
+ algfr %r5,%r0 /* Add smallest loaded bytes to current_len. */
+ vfenezfs %v18,%v16,%v17 /* Compare not equal with zero search. */
+ clgrj %r5,%r4,10,.Llastcmp /* If current_len >= n -> last compare */
+ vlgvb %r1,%v18,7 /* Get not equal index or 16 if all equal. */
+ clrjl %r1,%r0,.Lfound /* Jump away if miscompare is within
+ loaded bytes; (index < loaded-bytes) */
+ j .Lloop
+
+.Lcmp_one_char:
+ /* At least one of both strings is not 4-byte aligned
+ and there is no full character before next block-boundary.
+ Compare one character to get over the boundary and
+ proceed with normal loop! */
+ vlef %v16,0(%r5,%r2),0 /* Load one character. */
+ lghi %r0,4 /* Loaded byte count is 4. */
+ vlef %v17,0(%r5,%r3),0
+ j .Llt_cmp /* Proceed with comparision. */
+
+.Llastcmp:
+ /* Use comparision result only if located within first n characters.
+ %r0: loaded byte count in vreg;
+ %r5: current_len;
+ %r4: n;
+ (current_len - n): [0...16[
+ First ignored match index: loaded bytes - (current_len-n): ]0...16]
+ */
+ slgr %r5,%r4 /* %r5 = current_len - n. */
+ slr %r0,%r5 /* %r0 = first ignored match index. */
+ vlgvb %r4,%v18,7 /* Get not equal index or 16 if all equal. */
+ clrjl %r4,%r0,.Lfound2 /* Jump away if miscompare is within
+ loaded bytes and below n bytes. */
+.Lend_equal:
+ lghi %r2,0
+ br %r14
+
+.Lfound:
+ /* Difference or end of string. */
+ /* vfenezf found an unequal element or zero.
+ This instruction compares unsigned words, but wchar_t is signed.
+ Thus we have to compare the found element again. */
+ vlgvb %r4,%v18,7 /* Extract not equal byte-index. */
+.Lfound2:
+ srl %r4,2 /* And convert it to character-index. */
+ vlgvf %r0,%v16,0(%r4) /* Load character-values. */
+ vlgvf %r1,%v17,0(%r4)
+.Lend_cmp_one_char:
+ cr %r0,%r1
+ je .Lend_equal
+ lghi %r2,1
+ lghi %r1,-1
+ locgrl %r2,%r1
+ br %r14
+END(__wcsncmp_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcsncmp.c b/sysdeps/s390/multiarch/wcsncmp.c
new file mode 100644
index 0000000..165c002
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsncmp.c
@@ -0,0 +1,27 @@
+/* Multiple versions of wcsncmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__wcsncmp, wcsncmp)
+
+#else
+# include <wcsmbs/wcsncmp.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/wcsncmp.c b/wcsmbs/wcsncmp.c
index e083ad8..4de2ca8 100644
--- a/wcsmbs/wcsncmp.c
+++ b/wcsmbs/wcsncmp.c
@@ -18,13 +18,16 @@
#include <wchar.h>
+#ifndef WCSNCMP
+# define WCSNCMP wcsncmp
+#endif
/* Compare no more than N characters of S1 and S2,
returning less than, equal to or greater than zero
if S1 is lexicographically less than, equal to or
greater than S2. */
int
-wcsncmp (s1, s2, n)
+WCSNCMP (s1, s2, n)
const wchar_t *s1;
const wchar_t *s2;
size_t n;
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=63724a6db60f98e91da474d11d83a19aa10fc54e
commit 63724a6db60f98e91da474d11d83a19aa10fc54e
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:22 2015 +0200
S390: Optimize strcmp and wcscmp.
This patch provides optimized versions of strcmp and wcscmp with the z13
vector instructions.
The architecture specific string.h had a typo, which leads to ommiting the
inline version in this file if __USE_STRING_INLINES is defined.
Tested this inline version by tweaking test-strcmp.c.
ChangeLog:
* sysdeps/s390/multiarch/strcmp-vx.S: New File.
* sysdeps/s390/multiarch/strcmp.c: Likewise.
* sysdeps/s390/multiarch/wcscmp-c.c: Likewise.
* sysdeps/s390/multiarch/wcscmp-vx.S: Likewise.
* sysdeps/s390/multiarch/wcscmp.c: Likewise.
* sysdeps/s390/s390-32/multiarch/strcmp.c: Likewise.
* sysdeps/s390/s390-64/multiarch/strcmp.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strcmp and
wcscmp functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strcmp, wcscmp.
* string/strcmp.c (STRCMP): Define and use macro.
* benchtests/bench-wcscmp.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcscmp.
* sysdeps/s390/bits/string.h: Fix typo: _HAVE_STRING_ARCH_strcmp
instead of _HAVE_STRING_ARCH_memchr.
diff --git a/ChangeLog b/ChangeLog
index 9d8948a..8dfab01 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,24 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strcmp-vx.S: New File.
+ * sysdeps/s390/multiarch/strcmp.c: Likewise.
+ * sysdeps/s390/multiarch/wcscmp-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcscmp-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcscmp.c: Likewise.
+ * sysdeps/s390/s390-32/multiarch/strcmp.c: Likewise.
+ * sysdeps/s390/s390-64/multiarch/strcmp.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strcmp and
+ wcscmp functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strcmp, wcscmp.
+ * string/strcmp.c (STRCMP): Define and use macro.
+ * benchtests/bench-wcscmp.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcscmp.
+ * sysdeps/s390/bits/string.h: Fix typo: _HAVE_STRING_ARCH_strcmp
+ instead of _HAVE_STRING_ARCH_memchr.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strncat-c.c: New File.
* sysdeps/s390/multiarch/strncat-vx.S: Likewise.
* sysdeps/s390/multiarch/strncat.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 44a0471..b47d5c1 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -36,7 +36,8 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
-wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat
+wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat \
+ wcscmp
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-wcscmp.c b/benchtests/bench-wcscmp.c
new file mode 100644
index 0000000..bd483a2
--- /dev/null
+++ b/benchtests/bench-wcscmp.c
@@ -0,0 +1,20 @@
+/* Measure wcscmp functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strcmp.c"
diff --git a/string/strcmp.c b/string/strcmp.c
index 4d4c044..bf5cf5b 100644
--- a/string/strcmp.c
+++ b/string/strcmp.c
@@ -19,11 +19,15 @@
#undef strcmp
+#ifndef STRCMP
+# define STRCMP strcmp
+#endif
+
/* Compare S1 and S2, returning less than, equal to or
greater than zero if S1 is lexicographically less than,
equal to or greater than S2. */
int
-strcmp (const char *p1, const char *p2)
+STRCMP (const char *p1, const char *p2)
{
const unsigned char *s1 = (const unsigned char *) p1;
const unsigned char *s2 = (const unsigned char *) p2;
diff --git a/sysdeps/s390/bits/string.h b/sysdeps/s390/bits/string.h
index e69dc3a..b48d511 100644
--- a/sysdeps/s390/bits/string.h
+++ b/sysdeps/s390/bits/string.h
@@ -226,8 +226,8 @@ memchr (const void *__str, int __c, size_t __n)
}
#endif
-/* Search N bytes of S for C. */
-#define _HAVE_STRING_ARCH_memchr 1
+/* Compare S1 and S2. */
+#define _HAVE_STRING_ARCH_strcmp 1
#ifndef _FORCE_INLINES
__STRING_INLINE int
strcmp (const char *__s1, const char *__s2)
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 33c1398..d8fbd55 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -6,7 +6,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strncpy strncpy-vx \
stpncpy stpncpy-vx stpncpy-c \
strcat strcat-vx strcat-c \
- strncat strncat-vx strncat-c
+ strncat strncat-vx strncat-c \
+ strcmp strcmp-vx
endif
ifeq ($(subdir),wcsmbs)
@@ -17,5 +18,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsncpy wcsncpy-vx wcsncpy-c \
wcpncpy wcpncpy-vx wcpncpy-c \
wcscat wcscat-vx wcscat-c \
- wcsncat wcsncat-vx wcsncat-c
+ wcsncat wcsncat-vx wcsncat-c \
+ wcscmp wcscmp-vx wcscmp-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 1e57c0e..196d3ec 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -103,6 +103,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strncat);
IFUNC_VX_IMPL (wcsncat);
+ IFUNC_VX_IMPL (strcmp);
+ IFUNC_VX_IMPL (wcscmp);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/strcmp-vx.S b/sysdeps/s390/multiarch/strcmp-vx.S
new file mode 100644
index 0000000..316c69a
--- /dev/null
+++ b/sysdeps/s390/multiarch/strcmp-vx.S
@@ -0,0 +1,116 @@
+/* Vector optimized 32/64 bit S/390 version of strcmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* int strcmp (const char *s1, const char *s2)
+ Compare two strings
+
+ Register usage:
+ -r1=loaded byte count s1
+ -r2=s1
+ -r3=s2
+ -r4=loaded byte coutn s2, tmp
+ -r5=current_len
+ -v16=part of s1
+ -v17=part of s2
+ -v18=index of unequal
+*/
+ENTRY(__strcmp_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ lghi %r5,0 /* current_len = 0. */
+
+.Lloop:
+ vlbb %v16,0(%r5,%r2),6 /* Load s1 to block boundary. */
+ vlbb %v17,0(%r5,%r3),6 /* Load s2 to block boundary. */
+ lcbb %r1,0(%r5,%r2),6 /* Get loaded byte count of s1. */
+ jo .Llt16_1 /* Jump away if vr is not fully loaded. */
+ lcbb %r4,0(%r5,%r3),6
+ jo .Llt16_2 /* Jump away if vr is not fully loaded. */
+ /* Both vrs are fully loaded. */
+ aghi %r5,16
+ vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */
+ jno .Lfound
+
+ vlbb %v16,0(%r5,%r2),6
+ vlbb %v17,0(%r5,%r3),6
+ lcbb %r1,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r4,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ vfenezbs %v18,%v16,%v17
+ jno .Lfound
+
+ vlbb %v16,0(%r5,%r2),6
+ vlbb %v17,0(%r5,%r3),6
+ lcbb %r1,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r4,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ vfenezbs %v18,%v16,%v17
+ jno .Lfound
+
+ vlbb %v16,0(%r5,%r2),6
+ vlbb %v17,0(%r5,%r3),6
+ lcbb %r1,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r4,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ vfenezbs %v18,%v16,%v17
+ jno .Lfound
+ j .Lloop
+
+.Llt16_1:
+ lcbb %r4,0(%r5,%r3),6 /* Get loaded byte count of s2. */
+.Llt16_2:
+ clr %r1,%r4
+ locrh %r1,%r4 /* Get minimum of bytes loaded in s1/2. */
+ algfr %r5,%r1 /* Add smallest loaded bytes to current_len. */
+ vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */
+ vlgvb %r4,%v18,7 /* Get not equal index or 16 if all equal. */
+ clrjl %r4,%r1,.Lfound /* Jump away if miscompare is within loaded
+ bytes. */
+ j .Lloop
+
+.Lfound:
+ je .Lend_equal
+ lghi %r2,1
+ lghi %r1,-1
+ locgrl %r2,%r1
+ br %r14
+.Lend_equal:
+ lghi %r2,0
+ br %r14
+END(__strcmp_vx)
+
+# define strcmp __strcmp_c
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) strong_alias(__strcmp_c, __GI_strcmp)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+
+#include <strcmp.S>
diff --git a/string/strcmp.c b/sysdeps/s390/multiarch/strcmp.c
similarity index 54%
copy from string/strcmp.c
copy to sysdeps/s390/multiarch/strcmp.c
index 4d4c044..3e1ddcd 100644
--- a/string/strcmp.c
+++ b/sysdeps/s390/multiarch/strcmp.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Multiple versions of strcmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,29 +16,11 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
-#undef strcmp
-/* Compare S1 and S2, returning less than, equal to or
- greater than zero if S1 is lexicographically less than,
- equal to or greater than S2. */
-int
-strcmp (const char *p1, const char *p2)
-{
- const unsigned char *s1 = (const unsigned char *) p1;
- const unsigned char *s2 = (const unsigned char *) p2;
- unsigned char c1, c2;
-
- do
- {
- c1 = (unsigned char) *s1++;
- c2 = (unsigned char) *s2++;
- if (c1 == '\0')
- return c1 - c2;
- }
- while (c1 == c2);
-
- return c1 - c2;
-}
-libc_hidden_builtin_def (strcmp)
+# undef strcmp
+s390_vx_libc_ifunc2 (__strcmp, strcmp)
+#endif
diff --git a/string/strcmp.c b/sysdeps/s390/multiarch/wcscmp-c.c
similarity index 53%
copy from string/strcmp.c
copy to sysdeps/s390/multiarch/wcscmp-c.c
index 4d4c044..0d8ba7f 100644
--- a/string/strcmp.c
+++ b/sysdeps/s390/multiarch/wcscmp-c.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Default wcscmp implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,29 +16,17 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
-
-#undef strcmp
-
-/* Compare S1 and S2, returning less than, equal to or
- greater than zero if S1 is lexicographically less than,
- equal to or greater than S2. */
-int
-strcmp (const char *p1, const char *p2)
-{
- const unsigned char *s1 = (const unsigned char *) p1;
- const unsigned char *s2 = (const unsigned char *) p2;
- unsigned char c1, c2;
-
- do
- {
- c1 = (unsigned char) *s1++;
- c2 = (unsigned char) *s2++;
- if (c1 == '\0')
- return c1 - c2;
- }
- while (c1 == c2);
-
- return c1 - c2;
-}
-libc_hidden_builtin_def (strcmp)
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSCMP __wcscmp_c
+
+# include <wchar.h>
+extern __typeof (wcscmp) __wcscmp_c;
+# undef weak_alias
+# define weak_alias(name, alias)
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__wcscmp_c, __GI___wcscmp, __wcscmp_c);
+# endif /* SHARED */
+# include <wcsmbs/wcscmp.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcscmp-vx.S b/sysdeps/s390/multiarch/wcscmp-vx.S
new file mode 100644
index 0000000..f47ba1e
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcscmp-vx.S
@@ -0,0 +1,131 @@
+/* Vector optimized 32/64 bit S/390 version of wcscmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* int wcscmp (const wchar_t *s1, const wchar_t *s2)
+ Compare two strings
+
+ Register usage:
+ -r1=loaded byte count s1
+ -r2=s1
+ -r3=s2
+ -r4=loaded byte coutn s2, tmp
+ -r5=current_len
+ -v16=part of s1
+ -v17=part of s2
+ -v18=index of unequal
+*/
+ENTRY(__wcscmp_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ lghi %r5,0 /* current_len = 0. */
+
+.Lloop:
+ vlbb %v16,0(%r5,%r2),6 /* Load s1 to block boundary. */
+ vlbb %v17,0(%r5,%r3),6 /* Load s2 to block boundary. */
+ lcbb %r1,0(%r5,%r2),6 /* Get loaded byte count of s1. */
+ jo .Llt16_1 /* Jump away if vr is not fully loaded. */
+ lcbb %r4,0(%r5,%r3),6
+ jo .Llt16_2 /* Jump away if vr is not fully loaded. */
+ /* Both vrs are fully loaded. */
+ aghi %r5,16
+ vfenezfs %v18,%v16,%v17 /* Compare not equal with zero search. */
+ jno .Lfound
+
+ vlbb %v16,0(%r5,%r2),6
+ vlbb %v17,0(%r5,%r3),6
+ lcbb %r1,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r4,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ vfenezfs %v18,%v16,%v17
+ jno .Lfound
+
+ vlbb %v16,0(%r5,%r2),6
+ vlbb %v17,0(%r5,%r3),6
+ lcbb %r1,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r4,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ vfenezfs %v18,%v16,%v17
+ jno .Lfound
+
+ vlbb %v16,0(%r5,%r2),6
+ vlbb %v17,0(%r5,%r3),6
+ lcbb %r1,0(%r5,%r2),6
+ jo .Llt16_1
+ lcbb %r4,0(%r5,%r3),6
+ jo .Llt16_2
+ aghi %r5,16
+ vfenezfs %v18,%v16,%v17
+ jno .Lfound
+ j .Lloop
+
+.Lcmp_one_char:
+ /* At least one of both strings is not 4-byte aligned
+ and there is no full character before next block-boundary.
+ Compare one character to get over the boundary and
+ proceed with normal loop! */
+ vlef %v16,0(%r5,%r2),0 /* Load one character. */
+ vlef %v17,0(%r5,%r3),0
+ lghi %r1,4 /* Loaded byte count is 4. */
+ j .Llt_cmp /* Proceed with comparision. */
+
+.Llt16_1:
+ lcbb %r4,0(%r5,%r3),6 /* Get loaded byte count of s2. */
+.Llt16_2:
+ clr %r1,%r4
+ locrh %r1,%r4 /* Get minimum of bytes loaded in s1/2. */
+ nill %r1,65532 /* Align bytes loaded to full characters. */
+ jz .Lcmp_one_char /* Jump away if no full char is available. */
+.Llt_cmp:
+ algfr %r5,%r1 /* Add smallest loaded bytes to current_len. */
+ vfenezfs %v18,%v16,%v17 /* Compare not equal with zero search. */
+ vlgvb %r4,%v18,7 /* Get not equal index or 16 if all equal. */
+ clrjl %r4,%r1,.Lfound /* Jump away if miscompare is within loaded
+ bytes. */
+ j .Lloop
+
+.Lfound:
+ /* vfenezf found an unequal element or zero.
+ This instruction compares unsigned words, but wchar_t is signed.
+ Thus we have to compare the found element again. */
+ vlgvb %r4,%v18,7 /* Extract not equal byte-index, */
+ srl %r4,2 /* Convert it to character-index. */
+ vlgvf %r3,%v16,0(%r4) /* Load character-values. */
+ vlgvf %r4,%v17,0(%r4)
+ cr %r3,%r4
+ je .Lend_equal
+ lghi %r2,1
+ lghi %r1,-1
+ locgrl %r2,%r1
+ br %r14
+.Lend_equal:
+ lghi %r2,0
+ br %r14
+END(__wcscmp_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/string/strcmp.c b/sysdeps/s390/multiarch/wcscmp.c
similarity index 54%
copy from string/strcmp.c
copy to sysdeps/s390/multiarch/wcscmp.c
index 4d4c044..9216691 100644
--- a/string/strcmp.c
+++ b/sysdeps/s390/multiarch/wcscmp.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Multiple versions of wcscmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,29 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
-#undef strcmp
+s390_vx_libc_ifunc (__wcscmp)
+weak_alias (__wcscmp, wcscmp)
-/* Compare S1 and S2, returning less than, equal to or
- greater than zero if S1 is lexicographically less than,
- equal to or greater than S2. */
-int
-strcmp (const char *p1, const char *p2)
-{
- const unsigned char *s1 = (const unsigned char *) p1;
- const unsigned char *s2 = (const unsigned char *) p2;
- unsigned char c1, c2;
-
- do
- {
- c1 = (unsigned char) *s1++;
- c2 = (unsigned char) *s2++;
- if (c1 == '\0')
- return c1 - c2;
- }
- while (c1 == c2);
-
- return c1 - c2;
-}
-libc_hidden_builtin_def (strcmp)
+#else
+# include <wcsmbs/wcscmp.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/string/strcmp.c b/sysdeps/s390/s390-32/multiarch/strcmp.c
similarity index 53%
copy from string/strcmp.c
copy to sysdeps/s390/s390-32/multiarch/strcmp.c
index 4d4c044..1598bbc 100644
--- a/string/strcmp.c
+++ b/sysdeps/s390/s390-32/multiarch/strcmp.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Multiple versions of strcmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,29 +16,6 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
-
-#undef strcmp
-
-/* Compare S1 and S2, returning less than, equal to or
- greater than zero if S1 is lexicographically less than,
- equal to or greater than S2. */
-int
-strcmp (const char *p1, const char *p2)
-{
- const unsigned char *s1 = (const unsigned char *) p1;
- const unsigned char *s2 = (const unsigned char *) p2;
- unsigned char c1, c2;
-
- do
- {
- c1 = (unsigned char) *s1++;
- c2 = (unsigned char) *s2++;
- if (c1 == '\0')
- return c1 - c2;
- }
- while (c1 == c2);
-
- return c1 - c2;
-}
-libc_hidden_builtin_def (strcmp)
+/* This wrapper-file is needed, because otherwise file
+ sysdeps/s390/s390-[32|64]/strcmp.S will be used. */
+#include <sysdeps/s390/multiarch/strcmp.c>
diff --git a/string/strcmp.c b/sysdeps/s390/s390-64/multiarch/strcmp.c
similarity index 53%
copy from string/strcmp.c
copy to sysdeps/s390/s390-64/multiarch/strcmp.c
index 4d4c044..1598bbc 100644
--- a/string/strcmp.c
+++ b/sysdeps/s390/s390-64/multiarch/strcmp.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Multiple versions of strcmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,29 +16,6 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
-
-#undef strcmp
-
-/* Compare S1 and S2, returning less than, equal to or
- greater than zero if S1 is lexicographically less than,
- equal to or greater than S2. */
-int
-strcmp (const char *p1, const char *p2)
-{
- const unsigned char *s1 = (const unsigned char *) p1;
- const unsigned char *s2 = (const unsigned char *) p2;
- unsigned char c1, c2;
-
- do
- {
- c1 = (unsigned char) *s1++;
- c2 = (unsigned char) *s2++;
- if (c1 == '\0')
- return c1 - c2;
- }
- while (c1 == c2);
-
- return c1 - c2;
-}
-libc_hidden_builtin_def (strcmp)
+/* This wrapper-file is needed, because otherwise file
+ sysdeps/s390/s390-[32|64]/strcmp.S will be used. */
+#include <sysdeps/s390/multiarch/strcmp.c>
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=e1fe91180e4e29549f35f8ecd705b0bb4f208606
commit e1fe91180e4e29549f35f8ecd705b0bb4f208606
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:22 2015 +0200
S390: Optimize strncat wcsncat.
This patch provides optimized versions of strncat and wcsncat with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strncat-c.c: New File.
* sysdeps/s390/multiarch/strncat-vx.S: Likewise.
* sysdeps/s390/multiarch/strncat.c: Likewise.
* sysdeps/s390/multiarch/wcsncat-c.c: Likewise.
* sysdeps/s390/multiarch/wcsncat-vx.S: Likewise.
* sysdeps/s390/multiarch/wcsncat.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strncat and
wcsncat functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strncat, wcsncat.
* wcsmbs/wcsncat.c (WCSNCAT): Define and use macro.
* string/test-strncat.c: Add wcsncat support.
* wcsmbs/test-wcsncat.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcsncat.
* benchtests/bench-strncat.c: Add wcsncat support.
* benchtests/bench-wcsncat.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcsncat.
diff --git a/ChangeLog b/ChangeLog
index d87525f..9d8948a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strncat-c.c: New File.
+ * sysdeps/s390/multiarch/strncat-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strncat.c: Likewise.
+ * sysdeps/s390/multiarch/wcsncat-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcsncat-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcsncat.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strncat and
+ wcsncat functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strncat, wcsncat.
+ * wcsmbs/wcsncat.c (WCSNCAT): Define and use macro.
+ * string/test-strncat.c: Add wcsncat support.
+ * wcsmbs/test-wcsncat.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcsncat.
+ * benchtests/bench-strncat.c: Add wcsncat support.
+ * benchtests/bench-wcsncat.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcsncat.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strcat-c.c: New File.
* sysdeps/s390/multiarch/strcat-vx.S: Likewise.
* sysdeps/s390/multiarch/strcat.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 7c724fb..44a0471 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -36,7 +36,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
-wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat
+wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat wcsncat
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-strncat.c b/benchtests/bench-strncat.c
index 85a3135..8f3339d 100644
--- a/benchtests/bench-strncat.c
+++ b/benchtests/bench-strncat.c
@@ -17,33 +17,58 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "strncat"
+#ifndef WIDE
+# define TEST_NAME "strncat"
+#else
+# define TEST_NAME "wcsncat"
+#endif /* WIDE */
#include "bench-string.h"
-typedef char *(*proto_t) (char *, const char *, size_t);
-char *stupid_strncat (char *, const char *, size_t);
-char *simple_strncat (char *, const char *, size_t);
-
-IMPL (stupid_strncat, 0)
-IMPL (strncat, 2)
-
-char *
-stupid_strncat (char *dst, const char *src, size_t n)
+#ifndef WIDE
+# define STRNCAT strncat
+# define CHAR char
+# define SIMPLE_STRNCAT simple_strncat
+# define STUPID_STRNCAT stupid_strncat
+# define STRLEN strlen
+# define MEMCMP memcmp
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define STRNCAT wcsncat
+# define CHAR wchar_t
+# define SIMPLE_STRNCAT simple_wcsncat
+# define STUPID_STRNCAT stupid_wcsncat
+# define STRLEN wcslen
+# define MEMCMP wmemcmp
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+#endif /* WIDE */
+
+typedef CHAR *(*proto_t) (CHAR *, const CHAR *, size_t);
+CHAR *STUPID_STRNCAT (CHAR *, const CHAR *, size_t);
+CHAR *SIMPLE_STRNCAT (CHAR *, const CHAR *, size_t);
+
+IMPL (STUPID_STRNCAT, 0)
+IMPL (STRNCAT, 2)
+
+CHAR *
+STUPID_STRNCAT (CHAR *dst, const CHAR *src, size_t n)
{
- char *ret = dst;
+ CHAR *ret = dst;
while (*dst++ != '\0');
--dst;
while (n--)
- if ( (*dst++ = *src++) == '\0')
+ if ((*dst++ = *src++) == '\0')
return ret;
*dst = '\0';
return ret;
}
static void
-do_one_test (impl_t *impl, char *dst, const char *src, size_t n)
+do_one_test (impl_t *impl, CHAR *dst, const CHAR *src, size_t n)
{
- size_t k = strlen (dst), i, iters = INNER_LOOP_ITERS;
+ size_t k = STRLEN (dst), i, iters = INNER_LOOP_ITERS;
timing_t start, stop, cur;
if (CALL (impl, dst, src, n) != dst)
@@ -54,10 +79,10 @@ do_one_test (impl_t *impl, char *dst, const char *src, size_t n)
return;
}
- size_t len = strlen (src);
- if (memcmp (dst + k, src, len + 1 > n ? n : len + 1) != 0)
+ size_t len = STRLEN (src);
+ if (MEMCMP (dst + k, src, len + 1 > n ? n : len + 1) != 0)
{
- error (0, 0, "Incorrect cancatination in function %s",
+ error (0, 0, "Incorrect concatenation in function %s",
impl->name);
ret = 1;
return;
@@ -88,20 +113,20 @@ do_test (size_t align1, size_t align2, size_t len1, size_t len2,
size_t n, int max_char)
{
size_t i;
- char *s1, *s2;
+ CHAR *s1, *s2;
align1 &= 7;
- if (align1 + len1 >= page_size)
+ if ((align1 + len1) * sizeof (CHAR) >= page_size)
return;
- if (align1 + n > page_size)
+ if ((align1 + n) * sizeof (CHAR) > page_size)
return;
align2 &= 7;
- if (align2 + len1 + len2 >= page_size)
+ if ((align2 + len1 + len2) * sizeof (CHAR) >= page_size)
return;
- if (align2 + len1 + n > page_size)
+ if ((align2 + len1 + n) * sizeof (CHAR) > page_size)
return;
- s1 = (char *) (buf1 + align1);
- s2 = (char *) (buf2 + align2);
+ s1 = (CHAR *) (buf1) + align1;
+ s2 = (CHAR *) (buf2) + align2;
for (i = 0; i < len1; ++i)
s1[i] = 32 + 23 * i % (max_char - 32);
@@ -136,25 +161,25 @@ main (void)
for (n = 2; n <= 2048; n*=4)
{
- do_test (0, 2, 2, 2, n, 127);
- do_test (0, 0, 4, 4, n, 127);
- do_test (4, 0, 4, 4, n, 255);
- do_test (0, 0, 8, 8, n, 127);
- do_test (0, 8, 8, 8, n, 127);
+ do_test (0, 2, 2, 2, n, SMALL_CHAR);
+ do_test (0, 0, 4, 4, n, SMALL_CHAR);
+ do_test (4, 0, 4, 4, n, BIG_CHAR);
+ do_test (0, 0, 8, 8, n, SMALL_CHAR);
+ do_test (0, 8, 8, 8, n, SMALL_CHAR);
for (i = 1; i < 8; ++i)
{
- do_test (0, 0, 8 << i, 8 << i, n, 127);
- do_test (8 - i, 2 * i, 8 << i, 8 << i, n, 127);
- do_test (0, 0, 8 << i, 2 << i, n, 127);
- do_test (8 - i, 2 * i, 8 << i, 2 << i, n, 127);
+ do_test (0, 0, 8 << i, 8 << i, n, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 8 << i, 8 << i, n, SMALL_CHAR);
+ do_test (0, 0, 8 << i, 2 << i, n, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 8 << i, 2 << i, n, SMALL_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (i, 2 * i, 8 << i, 1, n, 127);
- do_test (2 * i, i, 8 << i, 1, n, 255);
- do_test (i, i, 8 << i, 10, n, 127);
+ do_test (i, 2 * i, 8 << i, 1, n, SMALL_CHAR);
+ do_test (2 * i, i, 8 << i, 1, n, BIG_CHAR);
+ do_test (i, i, 8 << i, 10, n, SMALL_CHAR);
}
}
diff --git a/benchtests/bench-wcsncat.c b/benchtests/bench-wcsncat.c
new file mode 100644
index 0000000..b9d7c3f
--- /dev/null
+++ b/benchtests/bench-wcsncat.c
@@ -0,0 +1,20 @@
+/* Measure wcsncat functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strncat.c"
diff --git a/string/test-strncat.c b/string/test-strncat.c
index 366e6fc..3571568 100644
--- a/string/test-strncat.c
+++ b/string/test-strncat.c
@@ -1,4 +1,4 @@
-/* Test and measure strncat functions.
+/* Test strncat functions.
Copyright (C) 2011-2015 Free Software Foundation, Inc.
Contributed by Intel Corporation.
@@ -17,33 +17,64 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "strncat"
+#ifndef WIDE
+# define TEST_NAME "strncat"
+#else
+# define TEST_NAME "wcsncat"
+#endif /* WIDE */
#include "test-string.h"
-typedef char *(*proto_t) (char *, const char *, size_t);
-char *stupid_strncat (char *, const char *, size_t);
-char *simple_strncat (char *, const char *, size_t);
+#ifndef WIDE
+# define STRNCAT strncat
+# define CHAR char
+# define UCHAR unsigned char
+# define SIMPLE_STRNCAT simple_strncat
+# define STUPID_STRNCAT stupid_strncat
+# define STRLEN strlen
+# define MEMSET memset
+# define MEMCPY memcpy
+# define MEMCMP memcmp
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define STRNCAT wcsncat
+# define CHAR wchar_t
+# define UCHAR wchar_t
+# define SIMPLE_STRNCAT simple_wcsncat
+# define STUPID_STRNCAT stupid_wcsncat
+# define STRLEN wcslen
+# define MEMSET wmemset
+# define MEMCPY wmemcpy
+# define MEMCMP wmemcmp
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+#endif /* WIDE */
-IMPL (stupid_strncat, 0)
-IMPL (strncat, 2)
+typedef CHAR *(*proto_t) (CHAR *, const CHAR *, size_t);
+CHAR *STUPID_STRNCAT (CHAR *, const CHAR *, size_t);
+CHAR *SIMPLE_STRNCAT (CHAR *, const CHAR *, size_t);
-char *
-stupid_strncat (char *dst, const char *src, size_t n)
+IMPL (STUPID_STRNCAT, 0)
+IMPL (STRNCAT, 2)
+
+CHAR *
+STUPID_STRNCAT (CHAR *dst, const CHAR *src, size_t n)
{
- char *ret = dst;
+ CHAR *ret = dst;
while (*dst++ != '\0');
--dst;
while (n--)
- if ( (*dst++ = *src++) == '\0')
+ if ((*dst++ = *src++) == '\0')
return ret;
*dst = '\0';
return ret;
}
static void
-do_one_test (impl_t *impl, char *dst, const char *src, size_t n)
+do_one_test (impl_t *impl, CHAR *dst, const CHAR *src, size_t n)
{
- size_t k = strlen (dst);
+ size_t k = STRLEN (dst);
if (CALL (impl, dst, src, n) != dst)
{
error (0, 0, "Wrong result in function %s %p != %p", impl->name,
@@ -52,10 +83,10 @@ do_one_test (impl_t *impl, char *dst, const char *src, size_t n)
return;
}
- size_t len = strlen (src);
- if (memcmp (dst + k, src, len + 1 > n ? n : len + 1) != 0)
+ size_t len = STRLEN (src);
+ if (MEMCMP (dst + k, src, len + 1 > n ? n : len + 1) != 0)
{
- error (0, 0, "Incorrect cancatination in function %s",
+ error (0, 0, "Incorrect concatenation in function %s",
impl->name);
ret = 1;
return;
@@ -74,20 +105,20 @@ do_test (size_t align1, size_t align2, size_t len1, size_t len2,
size_t n, int max_char)
{
size_t i;
- char *s1, *s2;
+ CHAR *s1, *s2;
align1 &= 7;
- if (align1 + len1 >= page_size)
+ if ((align1 + len1) * sizeof (CHAR) >= page_size)
return;
- if (align1 + n > page_size)
+ if ((align1 + n) * sizeof (CHAR) > page_size)
return;
align2 &= 7;
- if (align2 + len1 + len2 >= page_size)
+ if ((align2 + len1 + len2) * sizeof (CHAR) >= page_size)
return;
- if (align2 + len1 + n > page_size)
+ if ((align2 + len1 + n) * sizeof (CHAR) > page_size)
return;
- s1 = (char *) (buf1 + align1);
- s2 = (char *) (buf2 + align2);
+ s1 = (CHAR *) (buf1) + align1;
+ s2 = (CHAR *) (buf2) + align2;
for (i = 0; i < len1; ++i)
s1[i] = 32 + 23 * i % (max_char - 32);
@@ -107,9 +138,10 @@ static void
do_random_tests (void)
{
size_t i, j, n, align1, align2, len1, len2, N;
- unsigned char *p1 = buf1 + page_size - 512;
- unsigned char *p2 = buf2 + page_size - 512;
- unsigned char *res;
+ UCHAR *p1 = (UCHAR *) (buf1 + page_size) - 512;
+ UCHAR *p2 = (UCHAR *) (buf2 + page_size) - 512;
+ UCHAR *p3 = (UCHAR *) buf1;
+ UCHAR *res;
fprintf (stdout, "Number of iterations in random test = %zd\n",
ITERATIONS);
for (n = 0; n < ITERATIONS; n++)
@@ -148,26 +180,26 @@ do_random_tests (void)
p1[i] = 0;
else
{
- p1[i] = random () & 255;
+ p1[i] = random () & BIG_CHAR;
if (i >= align1 && i < len1 + align1 && !p1[i])
- p1[i] = (random () & 127) + 3;
+ p1[i] = (random () & SMALL_CHAR) + 3;
}
}
for (i = 0; i < len2; i++)
{
- buf1[i] = random () & 255;
- if (!buf1[i])
- buf1[i] = (random () & 127) + 3;
+ p3[i] = random () & BIG_CHAR;
+ if (!p3[i])
+ p3[i] = (random () & SMALL_CHAR) + 3;
}
- buf1[len2] = 0;
+ p3[len2] = 0;
FOR_EACH_IMPL (impl, 1)
{
- memset (p2 - 64, '\1', align2 + 64);
- memset (p2 + align2 + len2 + 1, '\1', 512 - align2 - len2 - 1);
- memcpy (p2 + align2, buf1, len2 + 1);
- res = (unsigned char *) CALL (impl, (char *) (p2 + align2),
- (char *) (p1 + align1), N);
+ MEMSET (p2 - 64, '\1', align2 + 64);
+ MEMSET (p2 + align2 + len2 + 1, '\1', 512 - align2 - len2 - 1);
+ MEMCPY (p2 + align2, p3, len2 + 1);
+ res = (UCHAR *) CALL (impl, (CHAR *) (p2 + align2),
+ (CHAR *) (p1 + align1), N);
if (res != p2 + align2)
{
error (0, 0, "Iteration %zd - wrong result in function %s "
@@ -187,7 +219,7 @@ do_random_tests (void)
break;
}
}
- if (memcmp (p2 + align2, buf1, len2))
+ if (MEMCMP (p2 + align2, p3, len2))
{
error (0, 0, "Iteration %zd - garbage in string before, %s "
"(%zd, %zd, %zd, %zd, %zd)",
@@ -220,7 +252,7 @@ do_random_tests (void)
ret = 1;
}
}
- if (memcmp (p1 + align1, p2 + align2 + len2,
+ if (MEMCMP (p1 + align1, p2 + align2 + len2,
(len1 + 1) > N ? N : len1 + 1))
{
error (0, 0, "Iteration %zd - different strings, %s "
@@ -233,7 +265,7 @@ do_random_tests (void)
}
int
-main (void)
+test_main (void)
{
size_t i, n;
@@ -246,28 +278,30 @@ main (void)
for (n = 2; n <= 2048; n*=4)
{
- do_test (0, 2, 2, 2, n, 127);
- do_test (0, 0, 4, 4, n, 127);
- do_test (4, 0, 4, 4, n, 255);
- do_test (0, 0, 8, 8, n, 127);
- do_test (0, 8, 8, 8, n, 127);
+ do_test (0, 2, 2, 2, n, SMALL_CHAR);
+ do_test (0, 0, 4, 4, n, SMALL_CHAR);
+ do_test (4, 0, 4, 4, n, BIG_CHAR);
+ do_test (0, 0, 8, 8, n, SMALL_CHAR);
+ do_test (0, 8, 8, 8, n, SMALL_CHAR);
for (i = 1; i < 8; ++i)
{
- do_test (0, 0, 8 << i, 8 << i, n, 127);
- do_test (8 - i, 2 * i, 8 << i, 8 << i, n, 127);
- do_test (0, 0, 8 << i, 2 << i, n, 127);
- do_test (8 - i, 2 * i, 8 << i, 2 << i, n, 127);
+ do_test (0, 0, 8 << i, 8 << i, n, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 8 << i, 8 << i, n, SMALL_CHAR);
+ do_test (0, 0, 8 << i, 2 << i, n, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 8 << i, 2 << i, n, SMALL_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (i, 2 * i, 8 << i, 1, n, 127);
- do_test (2 * i, i, 8 << i, 1, n, 255);
- do_test (i, i, 8 << i, 10, n, 127);
+ do_test (i, 2 * i, 8 << i, 1, n, SMALL_CHAR);
+ do_test (2 * i, i, 8 << i, 1, n, BIG_CHAR);
+ do_test (i, i, 8 << i, 10, n, SMALL_CHAR);
}
}
do_random_tests ();
return ret;
}
+
+#include "../test-skeleton.c"
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 6283999..33c1398 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -5,7 +5,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
stpcpy stpcpy-vx stpcpy-c \
strncpy strncpy-vx \
stpncpy stpncpy-vx stpncpy-c \
- strcat strcat-vx strcat-c
+ strcat strcat-vx strcat-c \
+ strncat strncat-vx strncat-c
endif
ifeq ($(subdir),wcsmbs)
@@ -15,5 +16,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcpcpy wcpcpy-vx wcpcpy-c \
wcsncpy wcsncpy-vx wcsncpy-c \
wcpncpy wcpncpy-vx wcpncpy-c \
- wcscat wcscat-vx wcscat-c
+ wcscat wcscat-vx wcscat-c \
+ wcsncat wcsncat-vx wcsncat-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index ccf4dea..1e57c0e 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -100,6 +100,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strcat);
IFUNC_VX_IMPL (wcscat);
+ IFUNC_VX_IMPL (strncat);
+ IFUNC_VX_IMPL (wcsncat);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/strncat-c.c b/sysdeps/s390/multiarch/strncat-c.c
new file mode 100644
index 0000000..ac3a057
--- /dev/null
+++ b/sysdeps/s390/multiarch/strncat-c.c
@@ -0,0 +1,23 @@
+/* Default strncat implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRNCAT __strncat_c
+
+# include <string/strncat.c>
+#endif
diff --git a/sysdeps/s390/multiarch/strncat-vx.S b/sysdeps/s390/multiarch/strncat-vx.S
new file mode 100644
index 0000000..4435d9f
--- /dev/null
+++ b/sysdeps/s390/multiarch/strncat-vx.S
@@ -0,0 +1,239 @@
+/* Vector optimized 32/64 bit S/390 version of strncat.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char * strncat (const char *dest, const char *src, size_t n)
+ Concatenate two strings - at most n characters of src.
+
+ Register usage:
+ -r0=saved dest pointer for return
+ -r1=tmp
+ -r2=dest
+ -r3=src
+ -r4=n
+ -r5=current_len
+ -r6=tmp
+ -r7=tmp
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+ -v31=register save area for r6, r7
+*/
+ENTRY(__strncat_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgfi %r4,0
+ ber %r14 /* Nothing to do, if n == 0. */
+ lgr %r0,%r2 /* Save destination pointer for return. */
+ vlvgp %v31,%r6,%r7 /* Save registers. */
+
+ /* STRLEN
+ %r1 = loaded bytes (tmp)
+ %r6 = zero byte index (tmp)
+ %r2 = dst
+ */
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfenezb %v16,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Llen_end /* Found zero within loaded bytes, end. */
+
+ /* Align s to 16 byte. */
+ risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r1 /* Compute bytes to 16bytes boundary. */
+
+ /* Find zero in 16byte aligned loop. */
+.Llen_loop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Llen_found /* Jump away if zero was found. */
+ vl %v16,16(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Llen_found16
+ vl %v16,32(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Llen_found32
+ vl %v16,48(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Llen_found48
+
+ aghi %r5,64
+ j .Llen_loop /* No zero -> loop. */
+
+.Llen_found48:
+ aghi %r5,16
+.Llen_found32:
+ aghi %r5,16
+.Llen_found16:
+ aghi %r5,16
+.Llen_found:
+ vlgvb %r1,%v16,7 /* Load byte index of zero. */
+ algr %r5,%r1
+
+.Llen_end:
+ /* STRCPY
+ %r1 = zero byte index (tmp)
+ %r6 = loaded bytes (tmp)
+ %r3 = curr src pointer
+ %r2 = curr dst pointer
+ %r7 = border, tmp
+ */
+ la %r2,0(%r5,%r2) /* strcpy at end of dst-string. */
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r6,%r6 /* Convert 32bit to 64bit. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ clgrjle %r4,%r6,.Lcpy_remaining_v16 /* If n <= loaded-bytes
+ -> process remaining. */
+
+ /* n > loaded-byte-count. */
+ vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r1,%r6,.Lcpy_found_v16_store /* Found zero within loaded
+ bytes, copy and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r7 /* Compute highest index to 16byte boundary. */
+
+ /* Zero not found and n > loaded-byte-count. */
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /*
+ Now we are 16byte aligned, so we can load a full vreg
+ without page fault.
+ */
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lcpy_loop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ clgijl %r4,17,.Lcpy_remaining_v16 /* If n <=16,
+ process remaining bytes. */
+.Lcpy_lt64:
+ lgr %r7,%r4
+ slgfi %r7,16 /* border_len = n - 16. */
+
+ /* If current_len >= border then process remaining bytes. */
+ clgrjhe %r5,%r7,.Lcpy_remaining_v16
+ vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lcpy_found_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lcpy_remaining_v18
+ vfenezbs %v17,%v18,%v18
+ je .Lcpy_found_v18
+ vl %v16,16(%r5,%r3)
+ vst %v18,0(%r5,%r2)
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lcpy_remaining_v16
+ vfenezbs %v17,%v16,%v16
+ je .Lcpy_found_v16
+ vl %v18,16(%r5,%r3)
+ vst %v16,0(%r5,%r2)
+ aghi %r5,16
+
+.Lcpy_remaining_v18:
+ vlr %v16,%v18
+.Lcpy_remaining_v16:
+ /* v16 contains the remaining bytes [1...16].
+ Store remaining bytes and append string-termination. */
+ vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */
+ slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len. */
+ aghi %r7,-1 /* vstl needs highest index. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ la %r2,0(%r5,%r2) /* vstl has no index register. */
+ /* Zero-index within remaining-bytes, store up to zero and end. */
+ clgrjle %r1,%r7,.Lcpy_found_v16_store
+ vstl %v16,%r7,0(%r2) /* Store remaining bytes. */
+ lghi %r1,0
+ stc %r1,1(%r7,%r2) /* Store string-null-termination beyond n. */
+.Lcpy_end:
+ /* Restore saved registers. */
+ vlgvg %r6,%v31,0
+ vlgvg %r7,%v31,1
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+.Lcpy_found_v16_32:
+ aghi %r5,32
+ j .Lcpy_found_v16
+.Lcpy_found_v18_48:
+ aghi %r5,32
+.Lcpy_found_v18_16:
+ aghi %r5,16
+.Lcpy_found_v18:
+ vlr %v16,%v18
+.Lcpy_found_v16:
+ /* v16 contains a zero. Store remaining bytes to zero. current_len
+ has not reached border, thus checking for n is not needed! */
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ la %r2,0(%r5,%r2)
+.Lcpy_found_v16_store:
+ vstl %v16,%r1,0(%r2) /* Copy characters including zero. */
+ j .Lcpy_end
+
+ /* Find zero in 16byte aligned loop. */
+.Lcpy_loop64:
+ vl %v16,0(%r5,%r3) /* Load s. */
+ vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lcpy_found_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ vfenezbs %v17,%v18,%v18
+ je .Lcpy_found_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezbs %v17,%v16,%v16
+ je .Lcpy_found_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezbs %v17,%v18,%v18
+ je .Lcpy_found_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lcpy_loop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ j .Lcpy_lt64
+END(__strncat_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strncat.c b/sysdeps/s390/multiarch/strncat.c
new file mode 100644
index 0000000..856dc14
--- /dev/null
+++ b/sysdeps/s390/multiarch/strncat.c
@@ -0,0 +1,27 @@
+/* Multiple versions of strncat.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__strncat, strncat)
+
+#else
+# include <string/strncat.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/multiarch/wcsncat-c.c b/sysdeps/s390/multiarch/wcsncat-c.c
new file mode 100644
index 0000000..c079f43
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsncat-c.c
@@ -0,0 +1,25 @@
+/* Default wcsncat implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSNCAT __wcsncat_c
+
+# include <wchar.h>
+extern __typeof (wcsncat) __wcsncat_c;
+# include <wcsmbs/wcsncat.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcsncat-vx.S b/sysdeps/s390/multiarch/wcsncat-vx.S
new file mode 100644
index 0000000..985414e
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsncat-vx.S
@@ -0,0 +1,265 @@
+/* Vector optimized 32/64 bit S/390 version of wcsncat.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t * wcsncat (wchar_t *dest, const wchar_t *src, size_t n)
+ Concatenate two strings - at most n characters of src.
+
+ Register usage:
+ -r0=saved dest pointer for return
+ -r1=tmp
+ -r2=dest
+ -r3=src
+ -r4=n
+ -r5=current_len
+ -r6=tmp
+ -r7=tmp
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+ -v31=register save area for r6, r7
+*/
+ENTRY(__wcsncat_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgfi %r4,0
+ ber %r14 /* Nothing to do, if n == 0. */
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ /* If either src or dest is not 4byte aligned, use __wcsncat_c. */
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+ tmll %r3,3 /* Test if src is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ lgr %r0,%r2 /* Save destination pointer for return. */
+ vlvgp %v31,%r6,%r7 /* Save registers. */
+
+ /* WCSLEN
+ %r1 = loaded bytes (tmp)
+ %r6 = zero byte index (tmp)
+ %r2 = dst
+ */
+ vfenezf %v16,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Llen_end /* Found zero within loaded bytes, end. */
+
+ /* Align s to 16 byte. */
+ risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r1 /* Compute bytes to 16bytes boundary. */
+
+ /* Find zero in 16byte aligned loop. */
+.Llen_loop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Llen_found /* Jump away if zero was found. */
+ vl %v16,16(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Llen_found16
+ vl %v16,32(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Llen_found32
+ vl %v16,48(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Llen_found48
+
+ aghi %r5,64
+ j .Llen_loop /* No zero -> loop. */
+
+.Llen_found48:
+ aghi %r5,16
+.Llen_found32:
+ aghi %r5,16
+.Llen_found16:
+ aghi %r5,16
+.Llen_found:
+ vlgvb %r1,%v16,7 /* Load byte index of zero. */
+ algr %r5,%r1
+
+.Llen_end:
+ /* WCSNCPY
+ %r1 = zero byte index (tmp)
+ %r6 = loaded bytes (tmp)
+ %r3 = curr src pointer
+ %r2 = curr dst pointer
+ %r7 = border, tmp
+ */
+ la %r2,0(%r5,%r2) /* strcpy at end of dst-string. */
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r6,%r6 /* Convert 32bit to 64bit. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ /* Check range of maxlen and convert to byte-count. */
+# ifdef __s390x__
+ tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ lghi %r1,-4 /* Max byte-count is 18446744073709551612. */
+# else
+ tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ llilf %r1,4294967292 /* Max byte-count is 4294967292. */
+# endif /* !__s390x__ */
+ sllg %r4,%r4,2 /* Convert character-count to byte-count. */
+ locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */
+
+ clgrjle %r4,%r6,.Lcpy_remaining_v16 /* If n <= loaded-bytes
+ -> process remaining. */
+
+ /* n > loaded-byte-count. */
+ vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r1,%r6,.Lcpy_found_v16_store /* Found zero within loaded bytes,
+ copy and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r1,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r1 /* Compute highest index to 16byte boundary. *
+
+ /* Zero not found and maxlen > loaded-byte-count. */
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /*
+ Now we are 16byte aligned, so we can load a full vreg
+ without page fault.
+ */
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lcpy_loop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ clgijl %r4,17,.Lcpy_remaining_v16 /* If n <=16,
+ process remaining bytes. */
+.Lcpy_lt64:
+ lgr %r7,%r4
+ slgfi %r7,16 /* border_len = n - 16. */
+
+ clgrjhe %r5,%r7,.Lcpy_remaining_v16
+ vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lcpy_found_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lcpy_remaining_v18
+ vfenezfs %v17,%v18,%v18
+ je .Lcpy_found_v18
+ vl %v16,16(%r5,%r3)
+ vst %v18,0(%r5,%r2)
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lcpy_remaining_v16
+ vfenezfs %v17,%v16,%v16
+ je .Lcpy_found_v16
+ vl %v18,16(%r5,%r3)
+ vst %v16,0(%r5,%r2)
+ aghi %r5,16
+
+.Lcpy_remaining_v18:
+ vlr %v16,%v18
+.Lcpy_remaining_v16:
+ /* v16 contains the remaining bytes [1...16].
+ Store remaining bytes and append string-termination. */
+ vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */
+ slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len. */
+ aghi %r7,-1 /* vstl needs highest index. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ la %r2,0(%r5,%r2) /* vstl has no index register. */
+ /* Zero-index within remaining-bytes, store up to zero and end. */
+ clgrjle %r1,%r7,.Lcpy_found_v16_store
+ vstl %v16,%r7,0(%r2) /* Store remaining bytes. */
+ lghi %r1,0
+ st %r1,1(%r7,%r2) /* Store string-null-termination beyond n. */
+.Lcpy_end:
+ /* Restore saved registers. */
+ vlgvg %r6,%v31,0
+ vlgvg %r7,%v31,1
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+.Lcpy_found_v16_32:
+ aghi %r5,32
+ j .Lcpy_found_v16
+.Lcpy_found_v18_48:
+ aghi %r5,32
+.Lcpy_found_v18_16:
+ aghi %r5,16
+.Lcpy_found_v18:
+ vlr %v16,%v18
+.Lcpy_found_v16:
+ /* v16 contains a zero. Store remaining bytes to zero. current_len
+ has not reached border, thus checking for n is not needed! */
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ la %r2,0(%r5,%r2)
+.Lcpy_found_v16_store:
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ vstl %v16,%r1,0(%r2) /* Copy characters including zero. */
+ j .Lcpy_end
+
+ /* Find zero in 16byte aligned loop. */
+.Lcpy_loop2:
+ vl %v16,16(%r5,%r3)
+ vst %v18,0(%r5,%r2)
+ aghi %r5,16
+
+.Lcpy_loop64:
+ vl %v16,0(%r5,%r3)
+ vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lcpy_found_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */
+ vfenezfs %v17,%v18,%v18
+ je .Lcpy_found_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezfs %v17,%v16,%v16
+ je .Lcpy_found_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezfs %v17,%v18,%v18
+ je .Lcpy_found_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lcpy_loop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ j .Lcpy_lt64
+
+.Lfallback:
+ jg __wcsncat_c
+END(__wcsncat_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcsncat.c b/sysdeps/s390/multiarch/wcsncat.c
new file mode 100644
index 0000000..2c21b8a
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsncat.c
@@ -0,0 +1,27 @@
+/* Multiple versions of wcsncat.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__wcsncat, wcsncat)
+
+#else
+# include <wcsmbs/wcsncat.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index 5eb959c..3b6c472 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -43,7 +43,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
mbrtoc16 c16rtomb
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
- wcpcpy wcsncpy wcpncpy wcscat
+ wcpcpy wcsncpy wcpncpy wcscat wcsncat
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/test-wcsncat.c b/wcsmbs/test-wcsncat.c
new file mode 100644
index 0000000..8b91a18
--- /dev/null
+++ b/wcsmbs/test-wcsncat.c
@@ -0,0 +1,20 @@
+/* Test wcsncat functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "../string/test-strncat.c"
diff --git a/wcsmbs/wcsncat.c b/wcsmbs/wcsncat.c
index 63eb126..78fe765 100644
--- a/wcsmbs/wcsncat.c
+++ b/wcsmbs/wcsncat.c
@@ -18,10 +18,13 @@
#include <wchar.h>
+#ifndef WCSNCAT
+# define WCSNCAT wcsncat
+#endif
/* Append no more than N wide-character of SRC onto DEST. */
wchar_t *
-wcsncat (dest, src, n)
+WCSNCAT (dest, src, n)
wchar_t *dest;
const wchar_t *src;
size_t n;
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=d626a24f235dbd4c446b241211a9a264a1eedb9e
commit d626a24f235dbd4c446b241211a9a264a1eedb9e
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:21 2015 +0200
S390: Optimize strcat and wcscat.
This patch provides optimized versions of strcat and wcscat with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strcat-c.c: New File.
* sysdeps/s390/multiarch/strcat-vx.S: Likewise.
* sysdeps/s390/multiarch/strcat.c: Likewise.
* sysdeps/s390/multiarch/wcscat-c.c: Likewise.
* sysdeps/s390/multiarch/wcscat-vx.S: Likewise.
* sysdeps/s390/multiarch/wcscat.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strcat and
wcscat functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strcat, wcscat.
* string/strcat.c (STRCAT): Define and use macro.
* wcsmbs/wcscat.c: Use WCSCAT if defined.
* string/test-strcat.c: Add wcscat support.
* wcsmbs/test-wcscat.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcscat.
* benchtests/bench-strcat.c: Add wcscat support.
* benchtests/bench-wcscat.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcscat.
diff --git a/ChangeLog b/ChangeLog
index ecd9887..d87525f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,26 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strcat-c.c: New File.
+ * sysdeps/s390/multiarch/strcat-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strcat.c: Likewise.
+ * sysdeps/s390/multiarch/wcscat-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcscat-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcscat.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strcat and
+ wcscat functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strcat, wcscat.
+ * string/strcat.c (STRCAT): Define and use macro.
+ * wcsmbs/wcscat.c: Use WCSCAT if defined.
+ * string/test-strcat.c: Add wcscat support.
+ * wcsmbs/test-wcscat.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcscat.
+ * benchtests/bench-strcat.c: Add wcscat support.
+ * benchtests/bench-wcscat.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcscat.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/stpncpy-c.c: New File.
* sysdeps/s390/multiarch/stpncpy-vx.S: Likewise.
* sysdeps/s390/multiarch/stpncpy.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index c3531a9..7c724fb 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -36,7 +36,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
-wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy
+wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy wcscat
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-strcat.c b/benchtests/bench-strcat.c
index 7dd79fb..1abf6d3 100644
--- a/benchtests/bench-strcat.c
+++ b/benchtests/bench-strcat.c
@@ -17,19 +17,45 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "strcat"
+#ifndef WIDE
+# define TEST_NAME "strcat"
+#else
+# define TEST_NAME "wcscat"
+#endif /* WIDE */
#include "bench-string.h"
-typedef char *(*proto_t) (char *, const char *);
-char *simple_strcat (char *, const char *);
-
-IMPL (simple_strcat, 0)
-IMPL (strcat, 1)
-
-char *
-simple_strcat (char *dst, const char *src)
+#ifndef WIDE
+# define STRCAT strcat
+# define CHAR char
+# define sfmt "s"
+# define SIMPLE_STRCAT simple_strcat
+# define STRLEN strlen
+# define STRCMP strcmp
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define STRCAT wcscat
+# define CHAR wchar_t
+# define sfmt "ls"
+# define SIMPLE_STRCAT simple_wcscat
+# define STRLEN wcslen
+# define STRCMP wcscmp
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+#endif /* WIDE */
+
+
+typedef CHAR *(*proto_t) (CHAR *, const CHAR *);
+CHAR *SIMPLE_STRCAT (CHAR *, const CHAR *);
+
+IMPL (SIMPLE_STRCAT, 0)
+IMPL (STRCAT, 1)
+
+CHAR *
+SIMPLE_STRCAT (CHAR *dst, const CHAR *src)
{
- char *ret = dst;
+ CHAR *ret = dst;
while (*dst++ != '\0');
--dst;
while ((*dst++ = *src++) != '\0');
@@ -37,9 +63,9 @@ simple_strcat (char *dst, const char *src)
}
static void
-do_one_test (impl_t *impl, char *dst, const char *src)
+do_one_test (impl_t *impl, CHAR *dst, const CHAR *src)
{
- size_t k = strlen (dst), i, iters = INNER_LOOP_ITERS;
+ size_t k = STRLEN (dst), i, iters = INNER_LOOP_ITERS;
timing_t start, stop, cur;
if (CALL (impl, dst, src) != dst)
@@ -50,9 +76,9 @@ do_one_test (impl_t *impl, char *dst, const char *src)
return;
}
- if (strcmp (dst + k, src) != 0)
+ if (STRCMP (dst + k, src) != 0)
{
- error (0, 0, "Wrong result in function %s dst \"%s\" src \"%s\"",
+ error (0, 0, "Wrong result in function %s dst \"%" sfmt "\" src \"%" sfmt "\"",
impl->name, dst, src);
ret = 1;
return;
@@ -75,18 +101,18 @@ static void
do_test (size_t align1, size_t align2, size_t len1, size_t len2, int max_char)
{
size_t i;
- char *s1, *s2;
+ CHAR *s1, *s2;
align1 &= 7;
- if (align1 + len1 >= page_size)
+ if ((align1 + len1) * sizeof (CHAR) >= page_size)
return;
align2 &= 7;
- if (align2 + len1 + len2 >= page_size)
+ if ((align2 + len1 + len2) * sizeof (CHAR) >= page_size)
return;
- s1 = (char *) (buf1 + align1);
- s2 = (char *) (buf2 + align2);
+ s1 = (CHAR *) (buf1) + align1;
+ s2 = (CHAR *) (buf2) + align2;
for (i = 0; i < len1; ++i)
s1[i] = 32 + 23 * i % (max_char - 32);
@@ -120,26 +146,26 @@ test_main (void)
for (i = 0; i < 16; ++i)
{
- do_test (0, 0, i, i, 127);
- do_test (0, 0, i, i, 255);
- do_test (0, i, i, i, 127);
- do_test (i, 0, i, i, 255);
+ do_test (0, 0, i, i, SMALL_CHAR);
+ do_test (0, 0, i, i, BIG_CHAR);
+ do_test (0, i, i, i, SMALL_CHAR);
+ do_test (i, 0, i, i, BIG_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (0, 0, 8 << i, 8 << i, 127);
- do_test (8 - i, 2 * i, 8 << i, 8 << i, 127);
- do_test (0, 0, 8 << i, 2 << i, 127);
- do_test (8 - i, 2 * i, 8 << i, 2 << i, 127);
+ do_test (0, 0, 8 << i, 8 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 8 << i, 8 << i, SMALL_CHAR);
+ do_test (0, 0, 8 << i, 2 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 8 << i, 2 << i, SMALL_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (i, 2 * i, 8 << i, 1, 127);
- do_test (2 * i, i, 8 << i, 1, 255);
- do_test (i, i, 8 << i, 10, 127);
- do_test (i, i, 8 << i, 10, 255);
+ do_test (i, 2 * i, 8 << i, 1, SMALL_CHAR);
+ do_test (2 * i, i, 8 << i, 1, BIG_CHAR);
+ do_test (i, i, 8 << i, 10, SMALL_CHAR);
+ do_test (i, i, 8 << i, 10, BIG_CHAR);
}
return ret;
diff --git a/string/strcat.c b/benchtests/bench-wcscat.c
similarity index 73%
copy from string/strcat.c
copy to benchtests/bench-wcscat.c
index 6e1f1ab..cd0dbc0 100644
--- a/string/strcat.c
+++ b/benchtests/bench-wcscat.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Measure wcscat functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,15 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
-
-#undef strcat
-
-/* Append SRC on the end of DEST. */
-char *
-strcat (char *dest, const char *src)
-{
- strcpy (dest + strlen (dest), src);
- return dest;
-}
-libc_hidden_builtin_def (strcat)
+#define WIDE 1
+#include "bench-strcat.c"
diff --git a/string/strcat.c b/string/strcat.c
index 6e1f1ab..9d2e699 100644
--- a/string/strcat.c
+++ b/string/strcat.c
@@ -19,9 +19,13 @@
#undef strcat
+#ifndef STRCAT
+# define STRCAT strcat
+#endif
+
/* Append SRC on the end of DEST. */
char *
-strcat (char *dest, const char *src)
+STRCAT (char *dest, const char *src)
{
strcpy (dest + strlen (dest), src);
return dest;
diff --git a/string/test-strcat.c b/string/test-strcat.c
index 0a97e25..22e67ee 100644
--- a/string/test-strcat.c
+++ b/string/test-strcat.c
@@ -1,4 +1,4 @@
-/* Test and measure strcat functions.
+/* Test strcat functions.
Copyright (C) 1999-2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
Written by Jakub Jelinek <jakub@redhat.com>, 1999.
@@ -18,19 +18,52 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "strcat"
+#ifndef WIDE
+# define TEST_NAME "strcat"
+#else
+# define TEST_NAME "wcscat"
+#endif /* WIDE */
#include "test-string.h"
-typedef char *(*proto_t) (char *, const char *);
-char *simple_strcat (char *, const char *);
+#ifndef WIDE
+# define STRCAT strcat
+# define CHAR char
+# define UCHAR unsigned char
+# define sfmt "s"
+# define SIMPLE_STRCAT simple_strcat
+# define STRLEN strlen
+# define STRCMP strcmp
+# define MEMSET memset
+# define MEMCPY memcpy
+# define MEMCMP memcmp
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+#else
+# include <wchar.h>
+# define STRCAT wcscat
+# define CHAR wchar_t
+# define UCHAR wchar_t
+# define sfmt "ls"
+# define SIMPLE_STRCAT simple_wcscat
+# define STRLEN wcslen
+# define STRCMP wcscmp
+# define MEMSET wmemset
+# define MEMCPY wmemcpy
+# define MEMCMP wmemcmp
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+#endif /* WIDE */
-IMPL (simple_strcat, 0)
-IMPL (strcat, 1)
+typedef CHAR *(*proto_t) (CHAR *, const CHAR *);
+CHAR *SIMPLE_STRCAT (CHAR *, const CHAR *);
-char *
-simple_strcat (char *dst, const char *src)
+IMPL (SIMPLE_STRCAT, 0)
+IMPL (STRCAT, 1)
+
+CHAR *
+SIMPLE_STRCAT (CHAR *dst, const CHAR *src)
{
- char *ret = dst;
+ CHAR *ret = dst;
while (*dst++ != '\0');
--dst;
while ((*dst++ = *src++) != '\0');
@@ -38,9 +71,9 @@ simple_strcat (char *dst, const char *src)
}
static void
-do_one_test (impl_t *impl, char *dst, const char *src)
+do_one_test (impl_t *impl, CHAR *dst, const CHAR *src)
{
- size_t k = strlen (dst);
+ size_t k = STRLEN (dst);
if (CALL (impl, dst, src) != dst)
{
error (0, 0, "Wrong result in function %s %p %p", impl->name,
@@ -49,9 +82,9 @@ do_one_test (impl_t *impl, char *dst, const char *src)
return;
}
- if (strcmp (dst + k, src) != 0)
+ if (STRCMP (dst + k, src) != 0)
{
- error (0, 0, "Wrong result in function %s dst \"%s\" src \"%s\"",
+ error (0, 0, "Wrong result in function %s dst \"%" sfmt "\" src \"%" sfmt "\"",
impl->name, dst, src);
ret = 1;
return;
@@ -62,18 +95,18 @@ static void
do_test (size_t align1, size_t align2, size_t len1, size_t len2, int max_char)
{
size_t i;
- char *s1, *s2;
+ CHAR *s1, *s2;
align1 &= 7;
- if (align1 + len1 >= page_size)
+ if ((align1 + len1) * sizeof (CHAR) >= page_size)
return;
align2 &= 7;
- if (align2 + len1 + len2 >= page_size)
+ if ((align2 + len1 + len2) * sizeof (CHAR) >= page_size)
return;
- s1 = (char *) (buf1 + align1);
- s2 = (char *) (buf2 + align2);
+ s1 = (CHAR *) (buf1) + align1;
+ s2 = (CHAR *) (buf2) + align2;
for (i = 0; i < len1; ++i)
s1[i] = 32 + 23 * i % (max_char - 32);
@@ -93,9 +126,10 @@ static void
do_random_tests (void)
{
size_t i, j, n, align1, align2, len1, len2;
- unsigned char *p1 = buf1 + page_size - 512;
- unsigned char *p2 = buf2 + page_size - 512;
- unsigned char *res;
+ UCHAR *p1 = (UCHAR *) (buf1 + page_size) - 512;
+ UCHAR *p2 = (UCHAR *) (buf2 + page_size) - 512;
+ UCHAR *p3 = (UCHAR *) buf1;
+ UCHAR *res;
for (n = 0; n < ITERATIONS; n++)
{
@@ -132,26 +166,26 @@ do_random_tests (void)
p1[i] = 0;
else
{
- p1[i] = random () & 255;
+ p1[i] = random () & BIG_CHAR;
if (i >= align1 && i < len1 + align1 && !p1[i])
- p1[i] = (random () & 127) + 3;
+ p1[i] = (random () & SMALL_CHAR) + 3;
}
}
for (i = 0; i < len2; i++)
{
- buf1[i] = random () & 255;
- if (!buf1[i])
- buf1[i] = (random () & 127) + 3;
+ p3[i] = random () & BIG_CHAR;
+ if (!p3[i])
+ p3[i] = (random () & SMALL_CHAR) + 3;
}
- buf1[len2] = 0;
+ p3[len2] = 0;
FOR_EACH_IMPL (impl, 1)
{
- memset (p2 - 64, '\1', align2 + 64);
- memset (p2 + align2 + len2 + 1, '\1', 512 - align2 - len2 - 1);
- memcpy (p2 + align2, buf1, len2 + 1);
- res = (unsigned char *) CALL (impl, (char *) (p2 + align2),
- (char *) (p1 + align1));
+ MEMSET (p2 - 64, '\1', align2 + 64);
+ MEMSET (p2 + align2 + len2 + 1, '\1', 512 - align2 - len2 - 1);
+ MEMCPY (p2 + align2, p3, len2 + 1);
+ res = (UCHAR *) CALL (impl, (CHAR *) (p2 + align2),
+ (CHAR *) (p1 + align1));
if (res != p2 + align2)
{
error (0, 0, "Iteration %zd - wrong result in function %s (%zd, %zd, %zd %zd) %p != %p",
@@ -169,7 +203,7 @@ do_random_tests (void)
break;
}
}
- if (memcmp (p2 + align2, buf1, len2))
+ if (MEMCMP (p2 + align2, p3, len2))
{
error (0, 0, "Iteration %zd - garbage in string before, %s (%zd, %zd, %zd, %zd)",
n, impl->name, align1, align2, len1, len2);
@@ -185,7 +219,7 @@ do_random_tests (void)
break;
}
}
- if (memcmp (p1 + align1, p2 + align2 + len2, len1 + 1))
+ if (MEMCMP (p1 + align1, p2 + align2 + len2, len1 + 1))
{
error (0, 0, "Iteration %zd - different strings, %s (%zd, %zd, %zd, %zd)",
n, impl->name, align1, align2, len1, len2);
@@ -209,26 +243,26 @@ test_main (void)
for (i = 0; i < 16; ++i)
{
- do_test (0, 0, i, i, 127);
- do_test (0, 0, i, i, 255);
- do_test (0, i, i, i, 127);
- do_test (i, 0, i, i, 255);
+ do_test (0, 0, i, i, SMALL_CHAR);
+ do_test (0, 0, i, i, BIG_CHAR);
+ do_test (0, i, i, i, SMALL_CHAR);
+ do_test (i, 0, i, i, BIG_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (0, 0, 8 << i, 8 << i, 127);
- do_test (8 - i, 2 * i, 8 << i, 8 << i, 127);
- do_test (0, 0, 8 << i, 2 << i, 127);
- do_test (8 - i, 2 * i, 8 << i, 2 << i, 127);
+ do_test (0, 0, 8 << i, 8 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 8 << i, 8 << i, SMALL_CHAR);
+ do_test (0, 0, 8 << i, 2 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 8 << i, 2 << i, SMALL_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (i, 2 * i, 8 << i, 1, 127);
- do_test (2 * i, i, 8 << i, 1, 255);
- do_test (i, i, 8 << i, 10, 127);
- do_test (i, i, 8 << i, 10, 255);
+ do_test (i, 2 * i, 8 << i, 1, SMALL_CHAR);
+ do_test (2 * i, i, 8 << i, 1, BIG_CHAR);
+ do_test (i, i, 8 << i, 10, SMALL_CHAR);
+ do_test (i, i, 8 << i, 10, BIG_CHAR);
}
do_random_tests ();
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 98b588f..6283999 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -4,7 +4,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strcpy strcpy-vx \
stpcpy stpcpy-vx stpcpy-c \
strncpy strncpy-vx \
- stpncpy stpncpy-vx stpncpy-c
+ stpncpy stpncpy-vx stpncpy-c \
+ strcat strcat-vx strcat-c
endif
ifeq ($(subdir),wcsmbs)
@@ -13,5 +14,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcscpy wcscpy-vx wcscpy-c \
wcpcpy wcpcpy-vx wcpcpy-c \
wcsncpy wcsncpy-vx wcsncpy-c \
- wcpncpy wcpncpy-vx wcpncpy-c
+ wcpncpy wcpncpy-vx wcpncpy-c \
+ wcscat wcscat-vx wcscat-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index ca69983..ccf4dea 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -97,6 +97,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (stpncpy);
IFUNC_VX_IMPL (wcpncpy);
+ IFUNC_VX_IMPL (strcat);
+ IFUNC_VX_IMPL (wcscat);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/string/strcat.c b/sysdeps/s390/multiarch/strcat-c.c
similarity index 62%
copy from string/strcat.c
copy to sysdeps/s390/multiarch/strcat-c.c
index 6e1f1ab..51c12e3 100644
--- a/string/strcat.c
+++ b/sysdeps/s390/multiarch/strcat-c.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Default strcat implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,15 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRCAT __strcat_c
+# ifdef SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strcat_c, __GI_strcat, __strcat_c);
+# endif /* SHARED */
-#undef strcat
-
-/* Append SRC on the end of DEST. */
-char *
-strcat (char *dest, const char *src)
-{
- strcpy (dest + strlen (dest), src);
- return dest;
-}
-libc_hidden_builtin_def (strcat)
+# include <string/strcat.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strcat-vx.S b/sysdeps/s390/multiarch/strcat-vx.S
new file mode 100644
index 0000000..ba553be
--- /dev/null
+++ b/sysdeps/s390/multiarch/strcat-vx.S
@@ -0,0 +1,161 @@
+/* Vector optimized 32/64 bit S/390 version of strcat.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char * strcat (const char *dest, const char *src)
+ Concatenate two strings.
+
+ Register usage:
+ -r0=saved dest pointer for return
+ -r1=tmp
+ -r2=dest
+ -r3=src
+ -r4=tmp
+ -r5=current_len
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+*/
+ENTRY(__strcat_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ lgr %r0,%r2 /* Save destination pointer for return. */
+
+ /* STRLEN
+ r1 = loaded bytes (tmp)
+ r4 = zero byte index (tmp)
+ r2 = dst
+ */
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfenezb %v16,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Llen_end /* Found zero within loaded bytes, end. */
+
+ /* Align s to 16 byte. */
+ risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r1 /* Compute bytes to 16bytes boundary. */
+
+ /* Find zero in 16byte aligned loop. */
+.Llen_loop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Llen_found /* Jump away if zero was found. */
+ vl %v16,16(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Llen_found16
+ vl %v16,32(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Llen_found32
+ vl %v16,48(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Llen_found48
+
+ aghi %r5,64
+ j .Llen_loop /* No zero -> loop. */
+
+.Llen_found48:
+ aghi %r5,16
+.Llen_found32:
+ aghi %r5,16
+.Llen_found16:
+ aghi %r5,16
+.Llen_found:
+ vlgvb %r4,%v16,7 /* Load byte index of zero. */
+ algr %r5,%r4
+
+.Llen_end:
+ /* STRCPY
+ %r1 = loaded bytes (tmp)
+ %r4 = zero byte index (tmp)
+ %r3 = curr src pointer
+ %r2 = curr dst pointer
+ */
+ la %r2,0(%r5,%r2) /* strcpy at end of dst-string. */
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Lcpy_found_align /* If found zero within loaded bytes,
+ copy bytes before and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r4 /* Compute highest index to 16byte boundary. */
+
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Find zero in 16byte aligned loop. */
+.Lcpy_loop:
+ vl %v16,0(%r5,%r3) /* Load s. */
+ vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lcpy_found_v16_0 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3)/* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ vfenezbs %v17,%v18,%v18
+ je .Lcpy_found_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezbs %v17,%v16,%v16
+ je .Lcpy_found_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezbs %v17,%v18,%v18
+ je .Lcpy_found_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ j .Lcpy_loop /* No zero -> loop. */
+
+.Lcpy_found_v16_32:
+ aghi %r5,32
+.Lcpy_found_v16_0:
+ la %r4,0(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ vstl %v16,%r1,0(%r4) /* Copy characters including zero. */
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+.Lcpy_found_v18_48:
+ aghi %r5,32
+.Lcpy_found_v18_16:
+ la %r4,16(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ vstl %v18,%r1,0(%r4) /* Copy characters including zero. */
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+.Lcpy_found_align:
+ vstl %v16,%r5,0(%r2) /* Copy characters including zero. */
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+END(__strcat_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/string/strcat.c b/sysdeps/s390/multiarch/strcat.c
similarity index 69%
copy from string/strcat.c
copy to sysdeps/s390/multiarch/strcat.c
index 6e1f1ab..e518eb3 100644
--- a/string/strcat.c
+++ b/sysdeps/s390/multiarch/strcat.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Multiple versions of strcat.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,15 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
-#undef strcat
+s390_vx_libc_ifunc2 (__strcat, strcat)
-/* Append SRC on the end of DEST. */
-char *
-strcat (char *dest, const char *src)
-{
- strcpy (dest + strlen (dest), src);
- return dest;
-}
-libc_hidden_builtin_def (strcat)
+#else
+# include <string/strcat.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/string/strcat.c b/sysdeps/s390/multiarch/wcscat-c.c
similarity index 72%
copy from string/strcat.c
copy to sysdeps/s390/multiarch/wcscat-c.c
index 6e1f1ab..d86b8a3 100644
--- a/string/strcat.c
+++ b/sysdeps/s390/multiarch/wcscat-c.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Default wcscat implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,15 +16,10 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSCAT __wcscat_c
-#undef strcat
-
-/* Append SRC on the end of DEST. */
-char *
-strcat (char *dest, const char *src)
-{
- strcpy (dest + strlen (dest), src);
- return dest;
-}
-libc_hidden_builtin_def (strcat)
+# include <wchar.h>
+extern __typeof (__wcscat) __wcscat_c;
+# include <wcsmbs/wcscat.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcscat-vx.S b/sysdeps/s390/multiarch/wcscat-vx.S
new file mode 100644
index 0000000..71f7b1a
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcscat-vx.S
@@ -0,0 +1,175 @@
+/* Vector optimized 32/64 bit S/390 version of wcscat.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t * wcscat (wchar_t *dest, const wchar_t *src)
+ Concatenate two strings.
+
+ Register usage:
+ -r0=saved dest pointer for return
+ -r1=tmp
+ -r2=dest
+ -r3=src
+ -r4=tmp
+ -r5=current_len
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+*/
+ENTRY(__wcscat_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ /* __wcslen_c can handle non 4byte aligned pointers,
+ but __wcscpy_c not. Thus if either src or dest is
+ not 4byte aligned, use __wcscat_c. */
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+ tmll %r3,3 /* Test if src is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ lgr %r0,%r2 /* Save destination pointer for return. */
+
+ /* WCSLEN
+ r1 = loaded bytes (tmp)
+ r4 = zero byte index (tmp)
+ r2 = dst
+ */
+
+ vfenezf %v16,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Llen_end /* Found zero within loaded bytes, end. */
+
+ /* Align s to 16 byte. */
+ risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r1 /* Compute bytes to 16bytes boundary. */
+
+ /* Find zero in 16byte aligned loop. */
+.Llen_loop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Llen_found /* Jump away if zero was found. */
+ vl %v16,16(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Llen_found16
+ vl %v16,32(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Llen_found32
+ vl %v16,48(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Llen_found48
+
+ aghi %r5,64
+ j .Llen_loop /* No zero -> loop. */
+
+.Llen_found48:
+ aghi %r5,16
+.Llen_found32:
+ aghi %r5,16
+.Llen_found16:
+ aghi %r5,16
+.Llen_found:
+ vlgvb %r4,%v16,7 /* Load byte index of zero. */
+ algr %r5,%r4
+
+.Llen_end:
+ /* WCSCPY
+ %r1 = loaded bytes (tmp)
+ %r4 = zero byte index (tmp)
+ %r3 = curr src pointer
+ %r2 = curr dst pointer
+ */
+ la %r2,0(%r5,%r2) /* strcpy at end of dst-string. */
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Lcpy_found_align /* If found zero within loaded bytes,
+ copy bytes before and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r4 /* Compute highest index to 16byte boundary. */
+
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Find zero in 16byte aligned loop. */
+.Lcpy_loop:
+ vl %v16,0(%r5,%r3) /* Load s. */
+ vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lcpy_found_v16_0 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */
+ vfenezfs %v17,%v18,%v18
+ je .Lcpy_found_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezfs %v17,%v16,%v16
+ je .Lcpy_found_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezfs %v17,%v18,%v18
+ je .Lcpy_found_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ j .Lcpy_loop /* No zero -> loop. */
+
+.Lcpy_found_v16_32:
+ aghi %r5,32
+.Lcpy_found_v16_0:
+ la %r4,0(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ vstl %v16,%r1,0(%r4) /* Copy characters including zero. */
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+.Lcpy_found_v18_48:
+ aghi %r5,32
+.Lcpy_found_v18_16:
+ la %r4,16(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ vstl %v18,%r1,0(%r4) /* Copy characters including zero. */
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+.Lcpy_found_align:
+ aghi %r5,3 /* Also copy remaining bytes of found zero. */
+ vstl %v16,%r5,0(%r2) /* Copy characters including zero. */
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+.Lfallback:
+ jg __wcscat_c
+END(__wcscat_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/string/strcat.c b/sysdeps/s390/multiarch/wcscat.c
similarity index 67%
copy from string/strcat.c
copy to sysdeps/s390/multiarch/wcscat.c
index 6e1f1ab..a2a000b 100644
--- a/string/strcat.c
+++ b/sysdeps/s390/multiarch/wcscat.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Multiple versions of wcscat.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,15 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
-#undef strcat
+s390_vx_libc_ifunc (__wcscat)
+weak_alias (__wcscat, wcscat)
-/* Append SRC on the end of DEST. */
-char *
-strcat (char *dest, const char *src)
-{
- strcpy (dest + strlen (dest), src);
- return dest;
-}
-libc_hidden_builtin_def (strcat)
+#else
+# include <wcsmbs/wcscat.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index 1d07c51..5eb959c 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -43,7 +43,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
mbrtoc16 c16rtomb
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
- wcpcpy wcsncpy wcpncpy
+ wcpcpy wcsncpy wcpncpy wcscat
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/string/strcat.c b/wcsmbs/test-wcscat.c
similarity index 73%
copy from string/strcat.c
copy to wcsmbs/test-wcscat.c
index 6e1f1ab..9bab33b 100644
--- a/string/strcat.c
+++ b/wcsmbs/test-wcscat.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2015 Free Software Foundation, Inc.
+/* Test wcscat functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,15 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
-
-#undef strcat
-
-/* Append SRC on the end of DEST. */
-char *
-strcat (char *dest, const char *src)
-{
- strcpy (dest + strlen (dest), src);
- return dest;
-}
-libc_hidden_builtin_def (strcat)
+#define WIDE 1
+#include "../string/test-strcat.c"
diff --git a/wcsmbs/wcscat.c b/wcsmbs/wcscat.c
index b993613..b0f0873 100644
--- a/wcsmbs/wcscat.c
+++ b/wcsmbs/wcscat.c
@@ -18,6 +18,9 @@
#include <wchar.h>
+#ifdef WCSCAT
+# define __wcscat WCSCAT
+#endif
/* Append SRC on the end of DEST. */
wchar_t *
@@ -47,4 +50,6 @@ __wcscat (dest, src)
return dest;
}
+#ifndef WCSCAT
weak_alias (__wcscat, wcscat)
+#endif
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=b3a0c176d1185621c4dd2bb3a51ec961bdb29123
commit b3a0c176d1185621c4dd2bb3a51ec961bdb29123
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:21 2015 +0200
S390: Optimize stpncpy and wcpncpy.
This patch provides optimized versions of stpncpy and wcpncpy with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/stpncpy-c.c: New File.
* sysdeps/s390/multiarch/stpncpy-vx.S: Likewise.
* sysdeps/s390/multiarch/stpncpy.c: Likewise.
* sysdeps/s390/multiarch/wcpncpy-c.c: Likewise.
* sysdeps/s390/multiarch/wcpncpy-vx.S: Likewise.
* sysdeps/s390/multiarch/wcpncpy.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add stpncpy and
wcpncpy functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for stpncpy, wcpncpy.
* wcsmbs/wcpncpy.c: Use WCPNCPY if defined.
* string/test-stpncpy.c: Add wcpncpy support.
* wcsmbs/test-wcpncpy.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcpncpy.
* benchtests/bench-stpncpy.c: Add wcpncpy support.
* benchtests/bench-wcpncpy.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcpncpy.
diff --git a/ChangeLog b/ChangeLog
index 5cf5cf7..ecd9887 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/stpncpy-c.c: New File.
+ * sysdeps/s390/multiarch/stpncpy-vx.S: Likewise.
+ * sysdeps/s390/multiarch/stpncpy.c: Likewise.
+ * sysdeps/s390/multiarch/wcpncpy-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcpncpy-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcpncpy.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add stpncpy and
+ wcpncpy functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for stpncpy, wcpncpy.
+ * wcsmbs/wcpncpy.c: Use WCPNCPY if defined.
+ * string/test-stpncpy.c: Add wcpncpy support.
+ * wcsmbs/test-wcpncpy.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcpncpy.
+ * benchtests/bench-stpncpy.c: Add wcpncpy support.
+ * benchtests/bench-wcpncpy.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcpncpy.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strncpy-vx.S: New File.
* sysdeps/s390/multiarch/strncpy.c: Likewise.
* sysdeps/s390/multiarch/wcsncpy-c.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 3785328..c3531a9 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -36,7 +36,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
-wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy
+wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-stpncpy.c b/benchtests/bench-stpncpy.c
index 53539c7..822db1f 100644
--- a/benchtests/bench-stpncpy.c
+++ b/benchtests/bench-stpncpy.c
@@ -18,18 +18,36 @@
#define STRNCPY_RESULT(dst, len, n) ((dst) + ((len) > (n) ? (n) : (len)))
#define TEST_MAIN
-#define TEST_NAME "stpncpy"
+#ifndef WIDE
+# define TEST_NAME "stpncpy"
+#else
+# define TEST_NAME "wcpncpy"
+#endif /* WIDE */
#include "bench-string.h"
+#ifndef WIDE
+# define CHAR char
+# define SIMPLE_STPNCPY simple_stpncpy
+# define STUPID_STPNCPY stupid_stpncpy
+# define STPNCPY stpncpy
+# define STRNLEN strnlen
+#else
+# include <wchar.h>
+# define CHAR wchar_t
+# define SIMPLE_STPNCPY simple_wcpncpy
+# define STUPID_STPNCPY stupid_wcpncpy
+# define STPNCPY wcpncpy
+# define STRNLEN wcsnlen
+#endif /* WIDE */
-char *simple_stpncpy (char *, const char *, size_t);
-char *stupid_stpncpy (char *, const char *, size_t);
+CHAR *SIMPLE_STPNCPY (CHAR *, const CHAR *, size_t);
+CHAR *STUPID_STPNCPY (CHAR *, const CHAR *, size_t);
-IMPL (stupid_stpncpy, 0)
-IMPL (simple_stpncpy, 0)
-IMPL (stpncpy, 1)
+IMPL (STUPID_STPNCPY, 0)
+IMPL (SIMPLE_STPNCPY, 0)
+IMPL (STPNCPY, 1)
-char *
-simple_stpncpy (char *dst, const char *src, size_t n)
+CHAR *
+SIMPLE_STPNCPY (CHAR *dst, const CHAR *src, size_t n)
{
while (n--)
if ((*dst++ = *src++) == '\0')
@@ -43,10 +61,10 @@ simple_stpncpy (char *dst, const char *src, size_t n)
return dst;
}
-char *
-stupid_stpncpy (char *dst, const char *src, size_t n)
+CHAR *
+STUPID_STPNCPY (CHAR *dst, const CHAR *src, size_t n)
{
- size_t nc = strnlen (src, n);
+ size_t nc = STRNLEN (src, n);
size_t i;
for (i = 0; i < nc; ++i)
@@ -56,4 +74,5 @@ stupid_stpncpy (char *dst, const char *src, size_t n)
return dst + nc;
}
+#undef CHAR
#include "bench-strncpy.c"
diff --git a/benchtests/bench-wcpncpy.c b/benchtests/bench-wcpncpy.c
new file mode 100644
index 0000000..8aa529e
--- /dev/null
+++ b/benchtests/bench-wcpncpy.c
@@ -0,0 +1,20 @@
+/* Measure wcpncpy functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-stpncpy.c"
diff --git a/string/test-stpncpy.c b/string/test-stpncpy.c
index 1adab34..8624bba 100644
--- a/string/test-stpncpy.c
+++ b/string/test-stpncpy.c
@@ -19,18 +19,36 @@
#define STRNCPY_RESULT(dst, len, n) ((dst) + ((len) > (n) ? (n) : (len)))
#define TEST_MAIN
-#define TEST_NAME "stpncpy"
+#ifndef WIDE
+# define TEST_NAME "stpncpy"
+#else
+# define TEST_NAME "wcpncpy"
+#endif /* WIDE */
#include "test-string.h"
+#ifndef WIDE
+# define CHAR char
+# define SIMPLE_STPNCPY simple_stpncpy
+# define STUPID_STPNCPY stupid_stpncpy
+# define STPNCPY stpncpy
+# define STRNLEN strnlen
+#else
+# include <wchar.h>
+# define CHAR wchar_t
+# define SIMPLE_STPNCPY simple_wcpncpy
+# define STUPID_STPNCPY stupid_wcpncpy
+# define STPNCPY wcpncpy
+# define STRNLEN wcsnlen
+#endif /* WIDE */
-char *simple_stpncpy (char *, const char *, size_t);
-char *stupid_stpncpy (char *, const char *, size_t);
+CHAR *SIMPLE_STPNCPY (CHAR *, const CHAR *, size_t);
+CHAR *STUPID_STPNCPY (CHAR *, const CHAR *, size_t);
-IMPL (stupid_stpncpy, 0)
-IMPL (simple_stpncpy, 0)
-IMPL (stpncpy, 1)
+IMPL (STUPID_STPNCPY, 0)
+IMPL (SIMPLE_STPNCPY, 0)
+IMPL (STPNCPY, 1)
-char *
-simple_stpncpy (char *dst, const char *src, size_t n)
+CHAR *
+SIMPLE_STPNCPY (CHAR *dst, const CHAR *src, size_t n)
{
while (n--)
if ((*dst++ = *src++) == '\0')
@@ -44,10 +62,10 @@ simple_stpncpy (char *dst, const char *src, size_t n)
return dst;
}
-char *
-stupid_stpncpy (char *dst, const char *src, size_t n)
+CHAR *
+STUPID_STPNCPY (CHAR *dst, const CHAR *src, size_t n)
{
- size_t nc = strnlen (src, n);
+ size_t nc = STRNLEN (src, n);
size_t i;
for (i = 0; i < nc; ++i)
@@ -57,4 +75,5 @@ stupid_stpncpy (char *dst, const char *src, size_t n)
return dst + nc;
}
+#undef CHAR
#include "test-strncpy.c"
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 0dff2dc..98b588f 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -3,7 +3,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
strnlen strnlen-vx strnlen-c \
strcpy strcpy-vx \
stpcpy stpcpy-vx stpcpy-c \
- strncpy strncpy-vx
+ strncpy strncpy-vx \
+ stpncpy stpncpy-vx stpncpy-c
endif
ifeq ($(subdir),wcsmbs)
@@ -11,5 +12,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsnlen wcsnlen-vx wcsnlen-c \
wcscpy wcscpy-vx wcscpy-c \
wcpcpy wcpcpy-vx wcpcpy-c \
- wcsncpy wcsncpy-vx wcsncpy-c
+ wcsncpy wcsncpy-vx wcsncpy-c \
+ wcpncpy wcpncpy-vx wcpncpy-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 940421d..ca69983 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -94,6 +94,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strncpy);
IFUNC_VX_IMPL (wcsncpy);
+ IFUNC_VX_IMPL (stpncpy);
+ IFUNC_VX_IMPL (wcpncpy);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/stpncpy-c.c b/sysdeps/s390/multiarch/stpncpy-c.c
new file mode 100644
index 0000000..bea15fa
--- /dev/null
+++ b/sysdeps/s390/multiarch/stpncpy-c.c
@@ -0,0 +1,28 @@
+/* Default stpncpy implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STPNCPY __stpncpy_c
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__stpncpy_c, __GI___stpncpy, __stpncpy_c);
+# endif /* SHARED */
+
+# include <string/stpncpy.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/stpncpy-vx.S b/sysdeps/s390/multiarch/stpncpy-vx.S
new file mode 100644
index 0000000..6208260
--- /dev/null
+++ b/sysdeps/s390/multiarch/stpncpy-vx.S
@@ -0,0 +1,200 @@
+/* Vector optimized 32/64 bit S/390 version of stpncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char * stpncpy (char *dest, const char *src, size_t n)
+ Copies at most n characters of string src to dest
+ returning a pointer to its end or dest+n
+ if src is smaller than n.
+
+ Register usage:
+ -%r0 = return value
+ -%r1 = zero byte index
+ -%r2 = curr dst pointer
+ -%r3 = curr src pointer
+ -%r4 = n
+ -%r5 = current_len
+ -%r6 = loaded bytes
+ -%r7 = border, tmp
+*/
+ENTRY(__stpncpy_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgfi %r4,0
+ ber %r14 /* Nothing to do, if n == 0. */
+
+ la %r0,0(%r4,%r2) /* Save destination pointer + n for return. */
+ vlvgp %v31,%r6,%r7 /* Save registers. */
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r6,%r6 /* Convert 32bit to 64bit. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes
+ -> process remaining. */
+
+ /* n > loaded-byte-count */
+ vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes,
+ copy and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r7 /* Compute highest index to 16byte boundary. */
+
+ /* Zero not found and n > loaded-byte-count. */
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Now we are 16byte aligned, so we can load a full vreg
+ without page fault. */
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lloop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ clgijl %r4,17,.Lremaining_v16 /* If n <= 16, process remaining
+ bytes. */
+.Llt64:
+ lgr %r7,%r4
+ slgfi %r7,16 /* border_len = n - 16. */
+
+ clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border
+ then process remaining bytes. */
+ vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lremaining_v18
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18
+ vl %v16,16(%r5,%r3)
+ vst %v18,0(%r5,%r2)
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lremaining_v16
+ vfenezbs %v17,%v16,%v16
+ je .Lfound_v16
+ vl %v18,16(%r5,%r3)
+ vst %v16,0(%r5,%r2)
+ aghi %r5,16
+
+.Lremaining_v18:
+ vlr %v16,%v18
+.Lremaining_v16:
+ /* v16 contains the remaining bytes [1...16].
+ Store remaining bytes and append string-termination. */
+ vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */
+ slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len */
+ aghi %r7,-1 /* vstl needs highest index. */
+ la %r2,0(%r5,%r2) /* vstl has no index register. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ /* Zero in remaining bytes? -> jump away (zero-index <= max-index). */
+ clrjle %r1,%r7,.Lfound_v16_store
+ vstl %v16,%r7,0(%r2) /* Store remaining bytes without null
+ termination! */
+.Lend:
+ /* Restore saved registers. */
+ vlgvg %r6,%v31,0
+ vlgvg %r7,%v31,1
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+.Lfound_v16_32:
+ aghi %r5,32
+ j .Lfound_v16
+.Lfound_v18_48:
+ aghi %r5,32
+.Lfound_v18_16:
+ aghi %r5,16
+.Lfound_v18:
+ vlr %v16,%v18
+.Lfound_v16:
+ /* v16 contains a zero. Store remaining bytes to zero. current_len
+ has not reached border, thus checking for n is not needed! */
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ la %r2,0(%r5,%r2) /* vstl has no support for index-register. */
+.Lfound_v16_store:
+ vstl %v16,%r1,0(%r2) /* Copy characters including zero. */
+ /* Fill remaining bytes with zero - remaining count always > 0. */
+ algr %r5,%r1 /* Remaining bytes (=%r4) = ... */
+ slgr %r4,%r5 /* = maxlen - (currlen + zero_index + 1) */
+ la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */
+ lgr %r0,%r2 /* Save return-pointer to found zero. */
+ clgije %r4,1,.Lend /* Skip zero-filling, if found zero is last
+ possible character.
+ (1 is substracted from r4 below!). */
+ aghi %r4,-2 /* mvc with exrl needs count - 1.
+ (additional -1, see remaining bytes above) */
+ srlg %r6,%r4,8 /* Split into 256 byte blocks. */
+ ltgr %r6,%r6
+ je .Lzero_lt256
+.Lzero_loop256:
+ mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */
+ la %r2,256(%r2)
+ brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */
+.Lzero_lt256:
+ exrl %r4,.Lmvc_lt256
+ j .Lend
+.Lmvc_lt256:
+ mvc 1(1,%r2),0(%r2)
+
+.Lloop64:
+ vl %v16,0(%r5,%r3)
+ vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezbs %v17,%v16,%v16
+ je .Lfound_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lloop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ j .Llt64
+END(__stpncpy_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/stpncpy.c b/sysdeps/s390/multiarch/stpncpy.c
new file mode 100644
index 0000000..cf8e56c
--- /dev/null
+++ b/sysdeps/s390/multiarch/stpncpy.c
@@ -0,0 +1,28 @@
+/* Multiple versions of stpncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc (__stpncpy)
+weak_alias (__stpncpy, stpncpy)
+
+#else
+# include <string/stpncpy.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/multiarch/wcpncpy-c.c b/sysdeps/s390/multiarch/wcpncpy-c.c
new file mode 100644
index 0000000..9810e52
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcpncpy-c.c
@@ -0,0 +1,25 @@
+/* Default wcsncpy implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCPNCPY __wcpncpy_c
+
+# include <wchar.h>
+extern __typeof (__wcpncpy) __wcpncpy_c;
+# include <wcsmbs/wcpncpy.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcpncpy-vx.S b/sysdeps/s390/multiarch/wcpncpy-vx.S
new file mode 100644
index 0000000..142c19a
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcpncpy-vx.S
@@ -0,0 +1,222 @@
+/* Vector optimized 32/64 bit S/390 version of wcpncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t * wcpncpy (wchar_t *dest, const wchar_t *src, size_t n)
+ Copies at most n characters of string src to dest
+ returning a pointer to its end or dest+n
+ if src is smaller than n.
+
+ Register usage:
+ -%r0 = return value
+ -%r1 = zero byte index
+ -%r2 = curr dst pointer
+ -%r3 = curr src pointer
+ -%r4 = n
+ -%r5 = current_len
+ -%r6 = loaded bytes
+ -%r7 = border, tmp
+*/
+ENTRY(__wcpncpy_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgfi %r4,0
+ ber %r14 /* Nothing to do, if n == 0. */
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+
+ tmll %r3,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ vlvgp %v31,%r6,%r7 /* Save registers. */
+ lghi %r5,0 /* current_len = 0. */
+
+ lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r6,%r6 /* Convert 32bit to 64bit. */
+
+ /* Check range of maxlen and convert to byte-count. */
+# ifdef __s390x__
+ tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ lghi %r1,-4 /* Max byte-count is 18446744073709551612. */
+# else
+ tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */
+ llilf %r1,4294967292 /* Max byte-count is 4294967292. */
+# endif /* !__s390x__ */
+ sllg %r4,%r4,2 /* Convert character-count to byte-count. */
+ locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */
+
+ la %r0,0(%r4,%r2) /* Save destination pointer + n for return. */
+
+ clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes
+ -> process remaining. */
+
+ /* n > loaded-byte-count */
+ vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes,
+ copy and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r7 /* Compute highest index to 16byte boundary. */
+
+ /* Zero not found and n > loaded-byte-count. */
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Now we are 16byte aligned, so we can load a full vreg
+ without page fault. */
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lloop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ clgijl %r4,17,.Lremaining_v16 /* If n <=16,
+ process remaining bytes. */
+.Llt64:
+ lgr %r7,%r4
+ slgfi %r7,16 /* border_len = n - 16. */
+
+ clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border
+ then process remaining bytes. */
+ vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lremaining_v18
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18
+ vl %v16,16(%r5,%r3)
+ vst %v18,0(%r5,%r2)
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lremaining_v16
+ vfenezfs %v17,%v16,%v16
+ je .Lfound_v16
+ vl %v18,16(%r5,%r3)
+ vst %v16,0(%r5,%r2)
+ aghi %r5,16
+
+.Lremaining_v18:
+ vlr %v16,%v18
+.Lremaining_v16:
+ /* v16 contains the remaining bytes [1...16].
+ Store remaining bytes and append string-termination. */
+ vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */
+ slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len */
+ aghi %r7,-1 /* vstl needs highest index. */
+ la %r2,0(%r5,%r2) /* vstl has no index register. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ /* Zero in remaining bytes? -> jump away (zero-index <= max-index). */
+ clrjle %r1,%r7,.Lfound_v16_store
+ vstl %v16,%r7,0(%r2) /* Store remaining bytes without null
+ termination! */
+.Lend:
+ /* Restore saved registers. */
+ vlgvg %r6,%v31,0
+ vlgvg %r7,%v31,1
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+.Lfound_v16_32:
+ aghi %r5,32
+ j .Lfound_v16
+.Lfound_v18_48:
+ aghi %r5,32
+.Lfound_v18_16:
+ aghi %r5,16
+.Lfound_v18:
+ vlr %v16,%v18
+.Lfound_v16:
+ /* v16 contains a zero. Store remaining bytes to zero. current_len
+ has not reached border, thus checking for n is not needed! */
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ la %r2,0(%r5,%r2) /* vstl has no support for index-register. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+.Lfound_v16_store:
+ vstl %v16,%r1,0(%r2) /* Copy characters including zero. */
+ /* Fill remaining bytes with zero - remaining byte count always > 0. */
+ algr %r5,%r1 /* Remaining bytes (=%r4) = ... */
+ slgr %r4,%r5 /* = n - (currlen + zero_index + 1) */
+ la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */
+ lay %r0,-3(%r2) /* Save return-pointer to found zero. */
+ clgije %r4,1,.Lend /* Skip zero-filling, if found-zero is last
+ possible character.
+ (1 is substracted from r4 below!). */
+ aghi %r4,-2 /* mvc with exrl needs count - 1.
+ (additional -1, see remaining bytes above) */
+ srlg %r6,%r4,8 /* Split into 256 byte blocks. */
+ ltgr %r6,%r6
+ je .Lzero_lt256
+.Lzero_loop256:
+ mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */
+ la %r2,256(%r2)
+ brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */
+.Lzero_lt256:
+ exrl %r4,.Lmvc_lt256
+ j .Lend
+.Lmvc_lt256:
+ mvc 1(1,%r2),0(%r2)
+
+ /* Find zero in 16byte aligned loop. */
+.Lloop64:
+ vl %v16,0(%r5,%r3)
+ vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezfs %v17,%v16,%v16
+ je .Lfound_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lloop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ j .Llt64
+
+.Lfallback:
+ jg __wcpncpy_c
+END(__wcpncpy_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcpncpy.c b/sysdeps/s390/multiarch/wcpncpy.c
new file mode 100644
index 0000000..e15acb7
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcpncpy.c
@@ -0,0 +1,28 @@
+/* Multiple versions of wcpncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc (__wcpncpy)
+weak_alias (__wcpncpy, wcpncpy)
+
+#else
+# include <wcsmbs/wcpncpy.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index 426eab0..1d07c51 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -43,7 +43,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
mbrtoc16 c16rtomb
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
- wcpcpy wcsncpy
+ wcpcpy wcsncpy wcpncpy
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/test-wcpncpy.c b/wcsmbs/test-wcpncpy.c
new file mode 100644
index 0000000..de302b4
--- /dev/null
+++ b/wcsmbs/test-wcpncpy.c
@@ -0,0 +1,20 @@
+/* Test wcpncpy functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "../string/test-stpncpy.c"
diff --git a/wcsmbs/wcpncpy.c b/wcsmbs/wcpncpy.c
index 98c77a5..e19e44c 100644
--- a/wcsmbs/wcpncpy.c
+++ b/wcsmbs/wcpncpy.c
@@ -18,6 +18,9 @@
#include <wchar.h>
+#ifdef WCPNCPY
+# define __wcpncpy WCPNCPY
+#endif
/* Copy no more than N wide-characters of SRC to DEST, returning the
address of the last character written into DEST. */
@@ -82,4 +85,6 @@ __wcpncpy (dest, src, n)
return dest - 1;
}
+#ifndef WCPNCPY
weak_alias (__wcpncpy, wcpncpy)
+#endif
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=d183b96ee6dc694e95f212c9272a178163351b19
commit d183b96ee6dc694e95f212c9272a178163351b19
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:21 2015 +0200
S390: Optimize strncpy and wcsncpy.
This patch provides optimized versions of strncpy and wcsncpy with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strncpy-vx.S: New File.
* sysdeps/s390/multiarch/strncpy.c: Likewise.
* sysdeps/s390/multiarch/wcsncpy-c.c: Likewise.
* sysdeps/s390/multiarch/wcsncpy-vx.S: Likewise.
* sysdeps/s390/multiarch/wcsncpy.c: Likewise.
* sysdeps/s390/s390-32/multiarch/strncpy.c: Likewise.
* sysdeps/s390/s390-64/multiarch/strncpy.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strncpy and
wcsncpy functions.
* wcsmbs/wcsncpy.c: Use WCSNCPY if defined.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strncpy, wcsncpy.
* string/test-strncpy.c: Add wcsncpy support.
* wcsmbs/test-wcsncpy.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcsncpy.
* benchtests/bench-strncpy.c: Add wcsncpy support.
* benchtests/bench-wcsncpy.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcsncpy
diff --git a/ChangeLog b/ChangeLog
index 90cd456..5cf5cf7 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,26 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strncpy-vx.S: New File.
+ * sysdeps/s390/multiarch/strncpy.c: Likewise.
+ * sysdeps/s390/multiarch/wcsncpy-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcsncpy-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcsncpy.c: Likewise.
+ * sysdeps/s390/s390-32/multiarch/strncpy.c: Likewise.
+ * sysdeps/s390/s390-64/multiarch/strncpy.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strncpy and
+ wcsncpy functions.
+ * wcsmbs/wcsncpy.c: Use WCSNCPY if defined.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strncpy, wcsncpy.
+ * string/test-strncpy.c: Add wcsncpy support.
+ * wcsmbs/test-wcsncpy.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcsncpy.
+ * benchtests/bench-strncpy.c: Add wcsncpy support.
+ * benchtests/bench-wcsncpy.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcsncpy
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/stpcpy-c.c: New File.
* sysdeps/s390/multiarch/stpcpy-vx.S: Likewise.
* sysdeps/s390/multiarch/stpcpy.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index bf1f6dc..3785328 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -36,7 +36,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
-wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy
+wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-strncpy.c b/benchtests/bench-strncpy.c
index 517daa0..de1e2af 100644
--- a/benchtests/bench-strncpy.c
+++ b/benchtests/bench-strncpy.c
@@ -16,23 +16,56 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
+#ifdef WIDE
+# include <wchar.h>
+# define CHAR wchar_t
+# define UCHAR wchar_t
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+# define MEMCMP wmemcmp
+# define MEMSET wmemset
+# define STRNLEN wcsnlen
+#else
+# define CHAR char
+# define UCHAR unsigned char
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+# define MEMCMP memcmp
+# define MEMSET memset
+# define STRNLEN strnlen
+#endif /* !WIDE */
+
+
#ifndef STRNCPY_RESULT
# define STRNCPY_RESULT(dst, len, n) dst
# define TEST_MAIN
-# define TEST_NAME "strncpy"
+# ifndef WIDE
+# define TEST_NAME "strncpy"
+# else
+# define TEST_NAME "wcsncpy"
+# endif /* WIDE */
# include "bench-string.h"
-
-char *simple_strncpy (char *, const char *, size_t);
-char *stupid_strncpy (char *, const char *, size_t);
-
-IMPL (stupid_strncpy, 0)
-IMPL (simple_strncpy, 0)
-IMPL (strncpy, 1)
-
-char *
-simple_strncpy (char *dst, const char *src, size_t n)
+# ifndef WIDE
+# define SIMPLE_STRNCPY simple_strncpy
+# define STUPID_STRNCPY stupid_strncpy
+# define STRNCPY strncpy
+# else
+# define SIMPLE_STRNCPY simple_wcsncpy
+# define STUPID_STRNCPY stupid_wcsncpy
+# define STRNCPY wcsncpy
+# endif /* WIDE */
+
+CHAR *SIMPLE_STRNCPY (CHAR *, const CHAR *, size_t);
+CHAR *STUPID_STRNCPY (CHAR *, const CHAR *, size_t);
+
+IMPL (STUPID_STRNCPY, 0)
+IMPL (SIMPLE_STRNCPY, 0)
+IMPL (STRNCPY, 1)
+
+CHAR *
+SIMPLE_STRNCPY (CHAR *dst, const CHAR *src, size_t n)
{
- char *ret = dst;
+ CHAR *ret = dst;
while (n--)
if ((*dst++ = *src++) == '\0')
{
@@ -43,10 +76,10 @@ simple_strncpy (char *dst, const char *src, size_t n)
return ret;
}
-char *
-stupid_strncpy (char *dst, const char *src, size_t n)
+CHAR *
+STUPID_STRNCPY (CHAR *dst, const CHAR *src, size_t n)
{
- size_t nc = strnlen (src, n);
+ size_t nc = STRNLEN (src, n);
size_t i;
for (i = 0; i < nc; ++i)
@@ -55,12 +88,12 @@ stupid_strncpy (char *dst, const char *src, size_t n)
dst[i] = '\0';
return dst;
}
-#endif
+#endif /* !STRNCPY_RESULT */
-typedef char *(*proto_t) (char *, const char *, size_t);
+typedef CHAR *(*proto_t) (CHAR *, const CHAR *, size_t);
static void
-do_one_test (impl_t *impl, char *dst, const char *src, size_t len, size_t n)
+do_one_test (impl_t *impl, CHAR *dst, const CHAR *src, size_t len, size_t n)
{
size_t i, iters = INNER_LOOP_ITERS;
timing_t start, stop, cur;
@@ -73,7 +106,7 @@ do_one_test (impl_t *impl, char *dst, const char *src, size_t len, size_t n)
return;
}
- if (memcmp (dst, src, len > n ? n : len) != 0)
+ if (memcmp (dst, src, (len > n ? n : len) * sizeof (CHAR)) != 0)
{
error (0, 0, "Wrong result in function %s", impl->name);
ret = 1;
@@ -109,23 +142,26 @@ static void
do_test (size_t align1, size_t align2, size_t len, size_t n, int max_char)
{
size_t i;
- char *s1, *s2;
+ CHAR *s1, *s2;
+/* For wcsncpy: align1 and align2 here mean alignment not in bytes,
+ but in wchar_ts, in bytes it will equal to align * (sizeof (wchar_t)). */
align1 &= 7;
- if (align1 + len >= page_size)
+ if ((align1 + len) * sizeof (CHAR) >= page_size)
return;
align2 &= 7;
- if (align2 + len >= page_size)
+ if ((align2 + len) * sizeof (CHAR) >= page_size)
return;
- s1 = (char *) (buf1 + align1);
- s2 = (char *) (buf2 + align2);
+ s1 = (CHAR *) (buf1) + align1;
+ s2 = (CHAR *) (buf2) + align2;
for (i = 0; i < len; ++i)
s1[i] = 32 + 23 * i % (max_char - 32);
s1[len] = 0;
- for (i = len + 1; i + align1 < page_size && i < len + 64; ++i)
+ for (i = len + 1; (i + align1) * sizeof (CHAR) < page_size && i < len + 64;
+ ++i)
s1[i] = 32 + 32 * i % (max_char - 32);
printf ("Length %4zd, n %4zd, alignment %2zd/%2zd:", len, n, align1, align2);
@@ -150,22 +186,22 @@ test_main (void)
for (i = 1; i < 8; ++i)
{
- do_test (i, i, 16, 16, 127);
- do_test (i, i, 16, 16, 255);
- do_test (i, 2 * i, 16, 16, 127);
- do_test (2 * i, i, 16, 16, 255);
- do_test (8 - i, 2 * i, 1 << i, 2 << i, 127);
- do_test (2 * i, 8 - i, 2 << i, 1 << i, 127);
- do_test (8 - i, 2 * i, 1 << i, 2 << i, 255);
- do_test (2 * i, 8 - i, 2 << i, 1 << i, 255);
+ do_test (i, i, 16, 16, SMALL_CHAR);
+ do_test (i, i, 16, 16, BIG_CHAR);
+ do_test (i, 2 * i, 16, 16, SMALL_CHAR);
+ do_test (2 * i, i, 16, 16, BIG_CHAR);
+ do_test (8 - i, 2 * i, 1 << i, 2 << i, SMALL_CHAR);
+ do_test (2 * i, 8 - i, 2 << i, 1 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 1 << i, 2 << i, BIG_CHAR);
+ do_test (2 * i, 8 - i, 2 << i, 1 << i, BIG_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (0, 0, 4 << i, 8 << i, 127);
- do_test (0, 0, 16 << i, 8 << i, 127);
- do_test (8 - i, 2 * i, 4 << i, 8 << i, 127);
- do_test (8 - i, 2 * i, 16 << i, 8 << i, 127);
+ do_test (0, 0, 4 << i, 8 << i, SMALL_CHAR);
+ do_test (0, 0, 16 << i, 8 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 4 << i, 8 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 16 << i, 8 << i, SMALL_CHAR);
}
return ret;
diff --git a/benchtests/bench-wcsncpy.c b/benchtests/bench-wcsncpy.c
new file mode 100644
index 0000000..d6f63c9
--- /dev/null
+++ b/benchtests/bench-wcsncpy.c
@@ -0,0 +1,20 @@
+/* Measure wcsncpy functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strncpy.c"
diff --git a/string/test-strncpy.c b/string/test-strncpy.c
index d1c312a..f10a3ab 100644
--- a/string/test-strncpy.c
+++ b/string/test-strncpy.c
@@ -1,4 +1,4 @@
-/* Test and measure strncpy functions.
+/* Test strncpy functions.
Copyright (C) 1999-2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
Written by Jakub Jelinek <jakub@redhat.com>, 1999.
@@ -17,23 +17,56 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
+#ifdef WIDE
+# include <wchar.h>
+# define CHAR wchar_t
+# define UCHAR wchar_t
+# define BIG_CHAR WCHAR_MAX
+# define SMALL_CHAR 1273
+# define MEMCMP wmemcmp
+# define MEMSET wmemset
+# define STRNLEN wcsnlen
+#else
+# define CHAR char
+# define UCHAR unsigned char
+# define BIG_CHAR CHAR_MAX
+# define SMALL_CHAR 127
+# define MEMCMP memcmp
+# define MEMSET memset
+# define STRNLEN strnlen
+#endif /* !WIDE */
+
+
#ifndef STRNCPY_RESULT
# define STRNCPY_RESULT(dst, len, n) dst
# define TEST_MAIN
-# define TEST_NAME "strncpy"
+# ifndef WIDE
+# define TEST_NAME "strncpy"
+# else
+# define TEST_NAME "wcsncpy"
+# endif /* WIDE */
# include "test-string.h"
+# ifndef WIDE
+# define SIMPLE_STRNCPY simple_strncpy
+# define STUPID_STRNCPY stupid_strncpy
+# define STRNCPY strncpy
+# else
+# define SIMPLE_STRNCPY simple_wcsncpy
+# define STUPID_STRNCPY stupid_wcsncpy
+# define STRNCPY wcsncpy
+# endif /* WIDE */
-char *simple_strncpy (char *, const char *, size_t);
-char *stupid_strncpy (char *, const char *, size_t);
+CHAR *SIMPLE_STRNCPY (CHAR *, const CHAR *, size_t);
+CHAR *STUPID_STRNCPY (CHAR *, const CHAR *, size_t);
-IMPL (stupid_strncpy, 0)
-IMPL (simple_strncpy, 0)
-IMPL (strncpy, 1)
+IMPL (STUPID_STRNCPY, 0)
+IMPL (SIMPLE_STRNCPY, 0)
+IMPL (STRNCPY, 1)
-char *
-simple_strncpy (char *dst, const char *src, size_t n)
+CHAR *
+SIMPLE_STRNCPY (CHAR *dst, const CHAR *src, size_t n)
{
- char *ret = dst;
+ CHAR *ret = dst;
while (n--)
if ((*dst++ = *src++) == '\0')
{
@@ -44,10 +77,10 @@ simple_strncpy (char *dst, const char *src, size_t n)
return ret;
}
-char *
-stupid_strncpy (char *dst, const char *src, size_t n)
+CHAR *
+STUPID_STRNCPY (CHAR *dst, const CHAR *src, size_t n)
{
- size_t nc = strnlen (src, n);
+ size_t nc = STRNLEN (src, n);
size_t i;
for (i = 0; i < nc; ++i)
@@ -56,12 +89,12 @@ stupid_strncpy (char *dst, const char *src, size_t n)
dst[i] = '\0';
return dst;
}
-#endif
+#endif /* !STRNCPY_RESULT */
-typedef char *(*proto_t) (char *, const char *, size_t);
+typedef CHAR *(*proto_t) (CHAR *, const CHAR *, size_t);
static void
-do_one_test (impl_t *impl, char *dst, const char *src, size_t len, size_t n)
+do_one_test (impl_t *impl, CHAR *dst, const CHAR *src, size_t len, size_t n)
{
if (CALL (impl, dst, src, n) != STRNCPY_RESULT (dst, len, n))
{
@@ -71,7 +104,7 @@ do_one_test (impl_t *impl, char *dst, const char *src, size_t len, size_t n)
return;
}
- if (memcmp (dst, src, len > n ? n : len) != 0)
+ if (memcmp (dst, src, (len > n ? n : len) * sizeof (CHAR)) != 0)
{
error (0, 0, "Wrong result in function %s", impl->name);
ret = 1;
@@ -96,23 +129,26 @@ static void
do_test (size_t align1, size_t align2, size_t len, size_t n, int max_char)
{
size_t i;
- char *s1, *s2;
+ CHAR *s1, *s2;
+/* For wcsncpy: align1 and align2 here mean alignment not in bytes,
+ but in wchar_ts, in bytes it will equal to align * (sizeof (wchar_t)). */
align1 &= 7;
- if (align1 + len >= page_size)
+ if ((align1 + len) * sizeof (CHAR) >= page_size)
return;
align2 &= 7;
- if (align2 + len >= page_size)
+ if ((align2 + len) * sizeof (CHAR) >= page_size)
return;
- s1 = (char *) (buf1 + align1);
- s2 = (char *) (buf2 + align2);
+ s1 = (CHAR *) (buf1) + align1;
+ s2 = (CHAR *) (buf2) + align2;
for (i = 0; i < len; ++i)
s1[i] = 32 + 23 * i % (max_char - 32);
s1[len] = 0;
- for (i = len + 1; i + align1 < page_size && i < len + 64; ++i)
+ for (i = len + 1; (i + align1) * sizeof (CHAR) < page_size && i < len + 64;
+ ++i)
s1[i] = 32 + 32 * i % (max_char - 32);
FOR_EACH_IMPL (impl, 0)
@@ -123,12 +159,16 @@ static void
do_random_tests (void)
{
size_t i, j, n, align1, align2, len, size, mode;
- unsigned char *p1 = buf1 + page_size - 512;
- unsigned char *p2 = buf2 + page_size - 512;
- unsigned char *res;
+ UCHAR *p1 = (UCHAR *) (buf1 + page_size) - 512;
+ UCHAR *p2 = (UCHAR *) (buf2 + page_size) - 512;
+ UCHAR *res;
for (n = 0; n < ITERATIONS; n++)
{
+ /* For wcsncpy: align1 and align2 here mean align not in bytes,
+ but in wchar_ts, in bytes it will equal to align * (sizeof
+ (wchar_t)). */
+
mode = random ();
if (mode & 1)
{
@@ -166,7 +206,7 @@ do_random_tests (void)
{
size = random () & 511;
if (size + j > 512)
- size = 512 - j - (random() & 31);
+ size = 512 - j - (random () & 31);
}
else
size = 512 - j;
@@ -182,18 +222,17 @@ do_random_tests (void)
p1[i] = 0;
else
{
- p1[i] = random () & 255;
+ p1[i] = random () & BIG_CHAR;
if (i >= align1 && i < len + align1 && !p1[i])
- p1[i] = (random () & 127) + 3;
+ p1[i] = (random () & SMALL_CHAR) + 3;
}
}
FOR_EACH_IMPL (impl, 1)
{
- memset (p2 - 64, '\1', 512 + 64);
- res = (unsigned char *) CALL (impl,
- (char *) (p2 + align2),
- (char *) (p1 + align1), size);
+ MEMSET (p2 - 64, '\1', 512 + 64);
+ res = (UCHAR *) CALL (impl, (CHAR *) (p2 + align2),
+ (CHAR *) (p1 + align1), size);
if (res != STRNCPY_RESULT (p2 + align2, len, size))
{
error (0, 0, "Iteration %zd - wrong result in function %s (%zd, %zd, %zd) %p != %p",
@@ -235,7 +274,7 @@ do_random_tests (void)
j = len + 1;
if (size < j)
j = size;
- if (memcmp (p1 + align1, p2 + align2, j))
+ if (MEMCMP (p1 + align1, p2 + align2, j))
{
error (0, 0, "Iteration %zd - different strings, %s (%zd, %zd, %zd)",
n, impl->name, align1, align2, len);
@@ -259,22 +298,22 @@ test_main (void)
for (i = 1; i < 8; ++i)
{
- do_test (i, i, 16, 16, 127);
- do_test (i, i, 16, 16, 255);
- do_test (i, 2 * i, 16, 16, 127);
- do_test (2 * i, i, 16, 16, 255);
- do_test (8 - i, 2 * i, 1 << i, 2 << i, 127);
- do_test (2 * i, 8 - i, 2 << i, 1 << i, 127);
- do_test (8 - i, 2 * i, 1 << i, 2 << i, 255);
- do_test (2 * i, 8 - i, 2 << i, 1 << i, 255);
+ do_test (i, i, 16, 16, SMALL_CHAR);
+ do_test (i, i, 16, 16, BIG_CHAR);
+ do_test (i, 2 * i, 16, 16, SMALL_CHAR);
+ do_test (2 * i, i, 16, 16, BIG_CHAR);
+ do_test (8 - i, 2 * i, 1 << i, 2 << i, SMALL_CHAR);
+ do_test (2 * i, 8 - i, 2 << i, 1 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 1 << i, 2 << i, BIG_CHAR);
+ do_test (2 * i, 8 - i, 2 << i, 1 << i, BIG_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (0, 0, 4 << i, 8 << i, 127);
- do_test (0, 0, 16 << i, 8 << i, 127);
- do_test (8 - i, 2 * i, 4 << i, 8 << i, 127);
- do_test (8 - i, 2 * i, 16 << i, 8 << i, 127);
+ do_test (0, 0, 4 << i, 8 << i, SMALL_CHAR);
+ do_test (0, 0, 16 << i, 8 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 4 << i, 8 << i, SMALL_CHAR);
+ do_test (8 - i, 2 * i, 16 << i, 8 << i, SMALL_CHAR);
}
do_random_tests ();
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 5b57342..0dff2dc 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -2,12 +2,14 @@ ifeq ($(subdir),string)
sysdep_routines += strlen strlen-vx strlen-c \
strnlen strnlen-vx strnlen-c \
strcpy strcpy-vx \
- stpcpy stpcpy-vx stpcpy-c
+ stpcpy stpcpy-vx stpcpy-c \
+ strncpy strncpy-vx
endif
ifeq ($(subdir),wcsmbs)
sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsnlen wcsnlen-vx wcsnlen-c \
wcscpy wcscpy-vx wcscpy-c \
- wcpcpy wcpcpy-vx wcpcpy-c
+ wcpcpy wcpcpy-vx wcpcpy-c \
+ wcsncpy wcsncpy-vx wcsncpy-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index a402301..940421d 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -91,6 +91,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (stpcpy);
IFUNC_VX_IMPL (wcpcpy);
+ IFUNC_VX_IMPL (strncpy);
+ IFUNC_VX_IMPL (wcsncpy);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/strncpy-vx.S b/sysdeps/s390/multiarch/strncpy-vx.S
new file mode 100644
index 0000000..0938280
--- /dev/null
+++ b/sysdeps/s390/multiarch/strncpy-vx.S
@@ -0,0 +1,207 @@
+/* Vector optimized 32/64 bit S/390 version of strncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char * strncpy (const char *dest, const char *src, size_t n)
+ Copy at most n characters of string src to dest.
+
+ Register usage:
+ -r0=dest pointer for return
+ -r1=tmp, zero byte index
+ -r2=dest
+ -r3=src
+ -r4=n
+ -r5=current_len
+ -r6=tmp, loaded bytes
+ -r7=tmp, border
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+ -v31=register save area for r6, r7
+*/
+ENTRY(__strncpy_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgfi %r4,0
+ ber %r14 /* Nothing to do, if n == 0. */
+ lgr %r0,%r2 /* Save destination pointer for return. */
+ vlvgp %v31,%r6,%r7 /* Save registers. */
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r6,%r6 /* Convert 32bit to 64bit. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes
+ -> process remaining. */
+
+ /* n > loaded-byte-count. */
+ vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes,
+ copy and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r7 /* Compute highest index to 16byte boundary. */
+
+ /* Zero not found and n > loaded-byte-count. */
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Now we are 16byte aligned, so we can load
+ a full vreg without page fault. */
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lloop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ clgijl %r4,17,.Lremaining_v16 /* If n <= 16, process remaining
+ bytes. */
+.Llt64:
+ lgr %r7,%r4
+ slgfi %r7,16 /* border_len = n - 16. */
+
+ clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border
+ then process remaining bytes. */
+ vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lremaining_v18
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18
+ vl %v16,16(%r5,%r3)
+ vst %v18,0(%r5,%r2)
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lremaining_v16
+ vfenezbs %v17,%v16,%v16
+ je .Lfound_v16
+ vl %v18,16(%r5,%r3)
+ vst %v16,0(%r5,%r2)
+ aghi %r5,16
+
+.Lremaining_v18:
+ vlr %v16,%v18
+.Lremaining_v16:
+ /* v16 contains the remaining bytes [1...16].
+ Store remaining bytes and append string-termination. */
+ vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */
+ slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len. */
+ aghi %r7,-1 /* vstl needs highest index. */
+ la %r2,0(%r5,%r2) /* vstl has no index register. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ /* Zero in remaining bytes? -> jump away (zero-index < max-index)
+ Do not jump away if zero-index == max-index,
+ but simply copy zero with vstl below. */
+ clrjl %r1,%r7,.Lfound_v16_store
+ vstl %v16,%r7,0(%r2) /* Store remaining bytes without null
+ termination!. */
+.Lend:
+ /* Restore saved registers. */
+ vlgvg %r6,%v31,0
+ vlgvg %r7,%v31,1
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+
+.Lfound_v16_32:
+ aghi %r5,32
+ j .Lfound_v16
+.Lfound_v18_48:
+ aghi %r5,32
+.Lfound_v18_16:
+ aghi %r5,16
+.Lfound_v18:
+ vlr %v16,%v18
+.Lfound_v16:
+ /* v16 contains a zero. Store remaining bytes to zero. current_len
+ has not reached border, thus checking for n is not needed! */
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ la %r2,0(%r5,%r2) /* vstl has no support for index-register. */
+.Lfound_v16_store:
+ vstl %v16,%r1,0(%r2) /* Copy characters including zero. */
+ /* Fill remaining bytes with zero - remaining count always > 0. */
+ algr %r5,%r1 /* Remaining bytes (=%r4) = ... */
+ slgr %r4,%r5 /* = n - (current_len + zero_index + 1). */
+ la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */
+ aghi %r4,-2 /* mvc with exrl needs count - 1.
+ (additional -1, see remaining bytes above) */
+ srlg %r6,%r4,8 /* Split into 256 byte blocks. */
+ ltgr %r6,%r6
+ je .Lzero_lt256
+.Lzero_loop256:
+ mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */
+ la %r2,256(%r2)
+ brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */
+.Lzero_lt256:
+ exrl %r4,.Lmvc_lt256
+ j .Lend
+.Lmvc_lt256:
+ mvc 1(1,%r2),0(%r2)
+
+.Lloop64:
+ vl %v16,0(%r5,%r3) /* Load s. */
+ vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezbs %v17,%v16,%v16
+ je .Lfound_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lloop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ j .Llt64
+END(__strncpy_vx)
+
+# define strncpy __strncpy_c
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) strong_alias(__strncpy_c, __GI_strncpy)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+
+/* Include strncpy-implementation in s390-32/s390-64 subdirectory. */
+#include <strncpy.S>
diff --git a/sysdeps/s390/multiarch/strncpy.c b/sysdeps/s390/multiarch/strncpy.c
new file mode 100644
index 0000000..c1b73d4
--- /dev/null
+++ b/sysdeps/s390/multiarch/strncpy.c
@@ -0,0 +1,24 @@
+/* Multiple versions of strncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__strncpy, strncpy)
+#endif
diff --git a/sysdeps/s390/multiarch/wcsncpy-c.c b/sysdeps/s390/multiarch/wcsncpy-c.c
new file mode 100644
index 0000000..b9ed28f
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsncpy-c.c
@@ -0,0 +1,25 @@
+/* Default wcsncpy implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSNCPY __wcsncpy_c
+
+# include <wchar.h>
+extern __typeof (__wcsncpy) __wcsncpy_c;
+# include <wcsmbs/wcsncpy.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcsncpy-vx.S b/sysdeps/s390/multiarch/wcsncpy-vx.S
new file mode 100644
index 0000000..8d692c0
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsncpy-vx.S
@@ -0,0 +1,223 @@
+/* Vector optimized 32/64 bit S/390 version of wcsncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t *wcsncpy (const wchar_t *dest, const wchar_t *src, size_t n)
+ Copy at most n characters of string src to dest.
+
+ Register usage:
+ -r0=dest pointer for return
+ -r1=tmp, zero byte index
+ -r2=dest
+ -r3=src
+ -r4=n
+ -r5=current_len
+ -r6=tmp, loaded bytes
+ -r7=tmp, border
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+ -v31=register save area for r6, r7
+*/
+ENTRY(__wcsncpy_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+
+ clgfi %r4,0
+ ber %r14 /* Nothing to do, if n == 0. */
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+
+ tmll %r3,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ vlvgp %v31,%r6,%r7 /* Save registers. */
+ lgr %r0,%r2 /* Save destination pointer for return. */
+
+ lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r6,%r6 /* Convert 32bit to 64bit. */
+
+ lghi %r5,0 /* current_len = 0. */
+
+ /* Check range of maxlen and convert to byte-count. */
+# ifdef __s390x__
+ tmhh %r4,49152 /* Test bit 0 or 1 of n. */
+ lghi %r1,-4 /* Max byte-count is 18446744073709551612. */
+# else
+ tmlh %r4,49152 /* Test bit 0 or 1 of n. */
+ llilf %r1,4294967292 /* Max byte-count is 4294967292. */
+# endif /* !__s390x__ */
+ sllg %r4,%r4,2 /* Convert character-count to byte-count. */
+ locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */
+
+ clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes
+ -> process remaining. */
+
+ /* n > loaded-byte-count. */
+ vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes,
+ copy and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r7 /* Compute highest index to 16byte boundary. */
+
+ /* Zero not found and n > loaded-byte-count. */
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Now we are 16byte aligned, so we can load
+ a full vreg without page fault. */
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lloop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ clgijl %r4,17,.Lremaining_v16 /* If n <=16, process remaining
+ bytes. */
+.Llt64:
+ lgr %r7,%r4
+ slgfi %r7,16 /* border_len = maxlen - 16. */
+
+ clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border
+ then process remaining bytes. */
+ vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lremaining_v18
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18
+ vl %v16,16(%r5,%r3)
+ vst %v18,0(%r5,%r2)
+ aghi %r5,16
+
+ clgrjhe %r5,%r7,.Lremaining_v16
+ vfenezfs %v17,%v16,%v16
+ je .Lfound_v16
+ vl %v18,16(%r5,%r3)
+ vst %v16,0(%r5,%r2)
+ aghi %r5,16
+
+.Lremaining_v18:
+ vlr %v16,%v18
+.Lremaining_v16:
+ /* v16 contains the remaining bytes [1...16].
+ Store remaining bytes and append string-termination. */
+ vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */
+ slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len. */
+ aghi %r7,-1 /* vstl needs highest index. */
+ la %r2,0(%r5,%r2) /* vstl has no index register. */
+ vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ /* Zero in remaining bytes? -> jump away (zero-index < max-index)
+ Do not jump away if zero-index == max-index,
+ but simply copy zero with vstl below. */
+ clrjl %r1,%r7,.Lfound_v16_store
+ vstl %v16,%r7,0(%r2) /* Store remaining bytes without null
+ termination!. */
+.Lend:
+ /* Restore saved registers. */
+ vlgvg %r6,%v31,0
+ vlgvg %r7,%v31,1
+ lgr %r2,%r0 /* Load saved dest-ptr. */
+ br %r14
+
+.Lfound_v16_32:
+ aghi %r5,32
+ j .Lfound_v16
+.Lfound_v18_48:
+ aghi %r5,32
+.Lfound_v18_16:
+ aghi %r5,16
+.Lfound_v18:
+ vlr %v16,%v18
+.Lfound_v16:
+ /* v16 contains a zero. Store remaining bytes to zero. current_len
+ has not reached border, thus checking for n is not needed! */
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ la %r2,0(%r5,%r2) /* vstl has no support for index-register. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+.Lfound_v16_store:
+ vstl %v16,%r1,0(%r2) /* Copy characters including zero. */
+ /* Fill remaining bytes with zero - remaining count always > 0. */
+ algr %r5,%r1 /* Remaining bytes (=%r4) = ... */
+ slgr %r4,%r5 /* = maxlen - (currlen + zero_index + 1). */
+ la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */
+ aghi %r4,-2 /* mvc with exrl needs count - 1.
+ (additional -1, see remaining bytes above) */
+ srlg %r6,%r4,8 /* Split into 256 byte blocks. */
+ ltgr %r6,%r6
+ je .Lzero_lt256
+.Lzero_loop256:
+ mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */
+ la %r2,256(%r2)
+ brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */
+.Lzero_lt256:
+ exrl %r4,.Lmvc_lt256
+ j .Lend
+.Lmvc_lt256:
+ mvc 1(1,%r2),0(%r2)
+
+ /* Find zero in 16byte aligned loop. */
+.Lloop64:
+ vl %v16,0(%r5,%r3) /* Load s. */
+ vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezfs %v17,%v16,%v16
+ je .Lfound_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r4,.Lloop64
+
+ vl %v16,0(%r5,%r3) /* Load s. */
+ j .Llt64
+
+.Lfallback:
+ jg __wcsncpy_c
+END(__wcsncpy_vx)
+
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcsncpy.c b/sysdeps/s390/multiarch/wcsncpy.c
new file mode 100644
index 0000000..3ab3476
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsncpy.c
@@ -0,0 +1,28 @@
+/* Multiple versions of wcsncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc (__wcsncpy)
+weak_alias (__wcsncpy, wcsncpy)
+
+#else
+# include <wcsmbs/wcsncpy.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/s390-32/multiarch/strncpy.c b/sysdeps/s390/s390-32/multiarch/strncpy.c
new file mode 100644
index 0000000..bc9c336
--- /dev/null
+++ b/sysdeps/s390/s390-32/multiarch/strncpy.c
@@ -0,0 +1,21 @@
+/* Multiple versions of strncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* This wrapper-file is needed, because otherwise file
+ sysdeps/s390/s390-[32|64]/strncpy.S will be used. */
+#include <sysdeps/s390/multiarch/strncpy.c>
diff --git a/sysdeps/s390/s390-64/multiarch/strncpy.c b/sysdeps/s390/s390-64/multiarch/strncpy.c
new file mode 100644
index 0000000..bc9c336
--- /dev/null
+++ b/sysdeps/s390/s390-64/multiarch/strncpy.c
@@ -0,0 +1,21 @@
+/* Multiple versions of strncpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* This wrapper-file is needed, because otherwise file
+ sysdeps/s390/s390-[32|64]/strncpy.S will be used. */
+#include <sysdeps/s390/multiarch/strncpy.c>
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index 1112e05..426eab0 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -43,7 +43,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
mbrtoc16 c16rtomb
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
- wcpcpy
+ wcpcpy wcsncpy
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/test-wcsncpy.c b/wcsmbs/test-wcsncpy.c
new file mode 100644
index 0000000..27de6f8
--- /dev/null
+++ b/wcsmbs/test-wcsncpy.c
@@ -0,0 +1,20 @@
+/* Test wcsncpy functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "../string/test-strncpy.c"
diff --git a/wcsmbs/wcsncpy.c b/wcsmbs/wcsncpy.c
index 7016f41..5ee5ee6 100644
--- a/wcsmbs/wcsncpy.c
+++ b/wcsmbs/wcsncpy.c
@@ -18,6 +18,9 @@
#include <wchar.h>
+#ifdef WCSNCPY
+# define __wcsncpy WCSNCPY
+#endif
/* Copy no more than N wide-characters of SRC to DEST. */
wchar_t *
@@ -83,4 +86,6 @@ __wcsncpy (dest, src, n)
return s;
}
+#ifndef WCSNCPY
weak_alias (__wcsncpy, wcsncpy)
+#endif
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=8ade3db78db17e0112648d302f98eda115949cd5
commit 8ade3db78db17e0112648d302f98eda115949cd5
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:21 2015 +0200
S390: Optimize stpcpy and wcpcpy.
This patch provides optimized versions of stpcpy and wcpcpy with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/stpcpy-c.c: New File.
* sysdeps/s390/multiarch/stpcpy-vx.S: Likewise.
* sysdeps/s390/multiarch/stpcpy.c: Likewise.
* sysdeps/s390/multiarch/wcpcpy-c.c: Likewise.
* sysdeps/s390/multiarch/wcpcpy-vx.S: Likewise.
* sysdeps/s390/multiarch/wcpcpy.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add stpcpy and
wcpcpy functions.
* string/stpcpy.c: Use STPCPY if defined.
* wcsmbs/wcpcpy.c: Use WCPCPY if defined.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for stpcpy, wcpcpy.
* string/test-stpcpy.c: Add wcpcpy support.
* wcsmbs/test-wcpcpy.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcpcpy.
* benchtests/bench-stpcpy.c: Add wcpcpy support.
* benchtests/bench-wcpcpy.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcpcpy.
diff --git a/ChangeLog b/ChangeLog
index 2b631dc..90cd456 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,26 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/stpcpy-c.c: New File.
+ * sysdeps/s390/multiarch/stpcpy-vx.S: Likewise.
+ * sysdeps/s390/multiarch/stpcpy.c: Likewise.
+ * sysdeps/s390/multiarch/wcpcpy-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcpcpy-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcpcpy.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add stpcpy and
+ wcpcpy functions.
+ * string/stpcpy.c: Use STPCPY if defined.
+ * wcsmbs/wcpcpy.c: Use WCPCPY if defined.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for stpcpy, wcpcpy.
+ * string/test-stpcpy.c: Add wcpcpy support.
+ * wcsmbs/test-wcpcpy.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcpcpy.
+ * benchtests/bench-stpcpy.c: Add wcpcpy support.
+ * benchtests/bench-wcpcpy.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcpcpy.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strcpy-vx.S: New File.
* sysdeps/s390/multiarch/strcpy.c: Likewise.
* sysdeps/s390/multiarch/wcscpy-c.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 28892c8..bf1f6dc 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -36,7 +36,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
-wcsmbs-bench := wcslen wcsnlen wcscpy
+wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-stpcpy.c b/benchtests/bench-stpcpy.c
index 3d79432..843e6a9 100644
--- a/benchtests/bench-stpcpy.c
+++ b/benchtests/bench-stpcpy.c
@@ -18,19 +18,34 @@
#define STRCPY_RESULT(dst, len) ((dst) + (len))
#define TEST_MAIN
-#define TEST_NAME "stpcpy"
+#ifndef WIDE
+# define TEST_NAME "stpcpy"
+#else
+# define TEST_NAME "wcpcpy"
+#endif /* WIDE */
#include "bench-string.h"
-
-char *simple_stpcpy (char *, const char *);
-
-IMPL (simple_stpcpy, 0)
-IMPL (stpcpy, 1)
-
-char *
-simple_stpcpy (char *dst, const char *src)
+#ifndef WIDE
+# define CHAR char
+# define SIMPLE_STPCPY simple_stpcpy
+# define STPCPY stpcpy
+#else
+# include <wchar.h>
+# define CHAR wchar_t
+# define SIMPLE_STPCPY simple_wcpcpy
+# define STPCPY wcpcpy
+#endif /* WIDE */
+
+CHAR *SIMPLE_STPCPY (CHAR *, const CHAR *);
+
+IMPL (SIMPLE_STPCPY, 0)
+IMPL (STPCPY, 1)
+
+CHAR *
+SIMPLE_STPCPY (CHAR *dst, const CHAR *src)
{
while ((*dst++ = *src++) != '\0');
return dst - 1;
}
+#undef CHAR
#include "bench-strcpy.c"
diff --git a/benchtests/bench-stpcpy.c b/benchtests/bench-wcpcpy.c
similarity index 62%
copy from benchtests/bench-stpcpy.c
copy to benchtests/bench-wcpcpy.c
index 3d79432..cfbfccf 100644
--- a/benchtests/bench-stpcpy.c
+++ b/benchtests/bench-wcpcpy.c
@@ -1,5 +1,5 @@
-/* Measure stpcpy functions.
- Copyright (C) 2013-2015 Free Software Foundation, Inc.
+/* Measure wcpcpy functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -16,21 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#define STRCPY_RESULT(dst, len) ((dst) + (len))
-#define TEST_MAIN
-#define TEST_NAME "stpcpy"
-#include "bench-string.h"
-
-char *simple_stpcpy (char *, const char *);
-
-IMPL (simple_stpcpy, 0)
-IMPL (stpcpy, 1)
-
-char *
-simple_stpcpy (char *dst, const char *src)
-{
- while ((*dst++ = *src++) != '\0');
- return dst - 1;
-}
-
-#include "bench-strcpy.c"
+#define WIDE 1
+#include "bench-stpcpy.c"
diff --git a/string/test-stpcpy.c b/string/test-stpcpy.c
index 3d138d9..7336834 100644
--- a/string/test-stpcpy.c
+++ b/string/test-stpcpy.c
@@ -1,4 +1,4 @@
-/* Test and measure stpcpy functions.
+/* Test stpcpy functions.
Copyright (C) 1999-2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
Written by Jakub Jelinek <jakub@redhat.com>, 1999.
@@ -19,19 +19,34 @@
#define STRCPY_RESULT(dst, len) ((dst) + (len))
#define TEST_MAIN
-#define TEST_NAME "stpcpy"
+#ifndef WIDE
+# define TEST_NAME "stpcpy"
+#else
+# define TEST_NAME "wcpcpy"
+#endif /* !WIDE */
#include "test-string.h"
-
-char *simple_stpcpy (char *, const char *);
-
-IMPL (simple_stpcpy, 0)
-IMPL (stpcpy, 1)
-
-char *
-simple_stpcpy (char *dst, const char *src)
+#ifndef WIDE
+# define CHAR char
+# define SIMPLE_STPCPY simple_stpcpy
+# define STPCPY stpcpy
+#else
+# include <wchar.h>
+# define CHAR wchar_t
+# define SIMPLE_STPCPY simple_wcpcpy
+# define STPCPY wcpcpy
+#endif /* !WIDE */
+
+CHAR *SIMPLE_STPCPY (CHAR *, const CHAR *);
+
+IMPL (SIMPLE_STPCPY, 0)
+IMPL (STPCPY, 1)
+
+CHAR *
+SIMPLE_STPCPY (CHAR *dst, const CHAR *src)
{
while ((*dst++ = *src++) != '\0');
return dst - 1;
}
+#undef CHAR
#include "test-strcpy.c"
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index e2202b7..5b57342 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,11 +1,13 @@
ifeq ($(subdir),string)
sysdep_routines += strlen strlen-vx strlen-c \
strnlen strnlen-vx strnlen-c \
- strcpy strcpy-vx
+ strcpy strcpy-vx \
+ stpcpy stpcpy-vx stpcpy-c
endif
ifeq ($(subdir),wcsmbs)
sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsnlen wcsnlen-vx wcsnlen-c \
- wcscpy wcscpy-vx wcscpy-c
+ wcscpy wcscpy-vx wcscpy-c \
+ wcpcpy wcpcpy-vx wcpcpy-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index c9228d6..a402301 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -88,6 +88,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strcpy);
IFUNC_VX_IMPL (wcscpy);
+ IFUNC_VX_IMPL (stpcpy);
+ IFUNC_VX_IMPL (wcpcpy);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/wcsmbs/wcpcpy.c b/sysdeps/s390/multiarch/stpcpy-c.c
similarity index 53%
copy from wcsmbs/wcpcpy.c
copy to sysdeps/s390/multiarch/stpcpy-c.c
index 1c05bca..1f59cad 100644
--- a/wcsmbs/wcpcpy.c
+++ b/sysdeps/s390/multiarch/stpcpy-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1996-2015 Free Software Foundation, Inc.
+/* Default stpcpy implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@gnu.ai.mit.edu>, 1996.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,31 +16,20 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
-
-#define __need_ptrdiff_t
-#include <stddef.h>
-
-
-/* Copy SRC to DEST, returning the address of the terminating L'\0' in
- DEST. */
-wchar_t *
-__wcpcpy (dest, src)
- wchar_t *dest;
- const wchar_t *src;
-{
- wchar_t *wcp = (wchar_t *) dest - 1;
- wint_t c;
- const ptrdiff_t off = src - dest + 1;
-
- do
- {
- c = wcp[off];
- *++wcp = c;
- }
- while (c != L'\0');
-
- return wcp;
-}
-
-weak_alias (__wcpcpy, wcpcpy)
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STPCPY __stpcpy_c
+# undef weak_alias
+# define weak_alias(a, b)
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__stpcpy_c, __GI___stpcpy, __stpcpy_c);
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ strong_alias (__stpcpy_c, __stpcpy_c_1); \
+ __hidden_ver1 (__stpcpy_c_1, __GI_stpcpy, __stpcpy_c_1);
+# endif /* SHARED */
+
+
+# include <string/stpcpy.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/stpcpy-vx.S b/sysdeps/s390/multiarch/stpcpy-vx.S
new file mode 100644
index 0000000..28606e4
--- /dev/null
+++ b/sysdeps/s390/multiarch/stpcpy-vx.S
@@ -0,0 +1,104 @@
+/* Vector optimized 32/64 bit S/390 version of stpcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char * stpcpy (const char *dest, const char *src)
+ Copy string src to dest returning a pointer to its end.
+
+ Register usage:
+ -r1=tmp
+ -r2=dest and return value
+ -r3=src
+ -r4=tmp
+ -r5=current_len
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+*/
+ENTRY(__stpcpy_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Lfound_align /* If found zero within loaded bytes,
+ copy bytes before and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r4 /* Compute highest index to 16byte boundary. */
+
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Find zero in 16byte aligned loop. */
+.Lloop:
+ vl %v16,0(%r5,%r3) /* Load s. */
+ vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16_0 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezbs %v17,%v16,%v16
+ je .Lfound_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ j .Lloop /* No zero found -> loop. */
+
+.Lfound_v16_32:
+ aghi %r5,32
+.Lfound_v16_0:
+ la %r3,0(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ vstl %v16,%r1,0(%r3) /* Copy characters including zero. */
+ la %r2,0(%r1,%r3) /* Return pointer to zero. */
+ br %r14
+
+.Lfound_v18_48:
+ aghi %r5,32
+.Lfound_v18_16:
+ la %r3,16(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ vstl %v18,%r1,0(%r3) /* Copy characters including zero. */
+ la %r2,0(%r1,%r3) /* Return pointer to zero. */
+ br %r14
+
+.Lfound_align:
+ vstl %v16,%r5,0(%r2) /* Copy characters including zero. */
+ la %r2,0(%r5,%r2) /* Return pointer to zero. */
+ br %r14
+END(__stpcpy_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/benchtests/bench-stpcpy.c b/sysdeps/s390/multiarch/stpcpy.c
similarity index 63%
copy from benchtests/bench-stpcpy.c
copy to sysdeps/s390/multiarch/stpcpy.c
index 3d79432..7c65c02 100644
--- a/benchtests/bench-stpcpy.c
+++ b/sysdeps/s390/multiarch/stpcpy.c
@@ -1,5 +1,5 @@
-/* Measure stpcpy functions.
- Copyright (C) 2013-2015 Free Software Foundation, Inc.
+/* Multiple versions of stpcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -16,21 +16,15 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#define STRCPY_RESULT(dst, len) ((dst) + (len))
-#define TEST_MAIN
-#define TEST_NAME "stpcpy"
-#include "bench-string.h"
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define NO_MEMPCPY_STPCPY_REDIRECT
+# include <string.h>
+# include <ifunc-resolve.h>
-char *simple_stpcpy (char *, const char *);
+s390_vx_libc_ifunc (__stpcpy)
+weak_alias (__stpcpy, stpcpy)
+libc_hidden_builtin_def (stpcpy)
-IMPL (simple_stpcpy, 0)
-IMPL (stpcpy, 1)
-
-char *
-simple_stpcpy (char *dst, const char *src)
-{
- while ((*dst++ = *src++) != '\0');
- return dst - 1;
-}
-
-#include "bench-strcpy.c"
+#else
+# include <string/stpcpy.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/benchtests/bench-stpcpy.c b/sysdeps/s390/multiarch/wcpcpy-c.c
similarity index 63%
copy from benchtests/bench-stpcpy.c
copy to sysdeps/s390/multiarch/wcpcpy-c.c
index 3d79432..e5a58e4 100644
--- a/benchtests/bench-stpcpy.c
+++ b/sysdeps/s390/multiarch/wcpcpy-c.c
@@ -1,5 +1,5 @@
-/* Measure stpcpy functions.
- Copyright (C) 2013-2015 Free Software Foundation, Inc.
+/* Default wcslen implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -16,21 +16,10 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#define STRCPY_RESULT(dst, len) ((dst) + (len))
-#define TEST_MAIN
-#define TEST_NAME "stpcpy"
-#include "bench-string.h"
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCPCPY __wcpcpy_c
-char *simple_stpcpy (char *, const char *);
-
-IMPL (simple_stpcpy, 0)
-IMPL (stpcpy, 1)
-
-char *
-simple_stpcpy (char *dst, const char *src)
-{
- while ((*dst++ = *src++) != '\0');
- return dst - 1;
-}
-
-#include "bench-strcpy.c"
+# include <wchar.h>
+extern __typeof (__wcpcpy) __wcpcpy_c;
+# include <wcsmbs/wcpcpy.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcpcpy-vx.S b/sysdeps/s390/multiarch/wcpcpy-vx.S
new file mode 100644
index 0000000..30b1b47
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcpcpy-vx.S
@@ -0,0 +1,114 @@
+/* Vector optimized 32/64 bit S/390 version of wcpcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* wchar_t * wcpcpy (const wchar_t *dest, const wchar_t *src)
+ Copy string src to dest returning a pointer to its end.
+
+ Register usage:
+ -r0=border-len for switching to vector-instructions
+ -r1=tmp
+ -r2=dest and return value
+ -r3=src
+ -r4=tmp
+ -r5=current_len
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+*/
+ENTRY(__wcpcpy_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ tmll %r3,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Lfound_align /* If found zero within loaded bytes,
+ copy bytes before and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r4 /* Compute highest index to 16byte boundary. */
+
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Find zero in 16byte aligned loop. */
+.Lloop:
+ vl %v16,0(%r5,%r3) /* Load s. */
+ vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16_0 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezfs %v17,%v16,%v16
+ je .Lfound_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ j .Lloop /* No zero found -> loop. */
+
+.Lfound_v16_32:
+ aghi %r5,32
+.Lfound_v16_0:
+ la %r3,0(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ vstl %v16,%r1,0(%r3) /* Copy characters including zero. */
+ lay %r2,-3(%r1,%r3) /* Return pointer to zero. */
+ br %r14
+
+.Lfound_v18_48:
+ aghi %r5,32
+.Lfound_v18_16:
+ la %r3,16(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ vstl %v18,%r1,0(%r3) /* Copy characters including zero. */
+ lay %r2,-3(%r1,%r3) /* Return pointer to zero. */
+ br %r14
+
+.Lfound_align:
+ aghi %r5,3 /* Also copy remaining bytes of zero. */
+ vstl %v16,%r5,0(%r2) /* Copy characters including zero. */
+ lay %r2,-3(%r5,%r2) /* Return pointer to zero. */
+ br %r14
+
+.Lfallback:
+ jg __wcpcpy_c
+END(__wcpcpy_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/benchtests/bench-stpcpy.c b/sysdeps/s390/multiarch/wcpcpy.c
similarity index 63%
copy from benchtests/bench-stpcpy.c
copy to sysdeps/s390/multiarch/wcpcpy.c
index 3d79432..53c4444 100644
--- a/benchtests/bench-stpcpy.c
+++ b/sysdeps/s390/multiarch/wcpcpy.c
@@ -1,5 +1,5 @@
-/* Measure stpcpy functions.
- Copyright (C) 2013-2015 Free Software Foundation, Inc.
+/* Multiple versions of wcpcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -16,21 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#define STRCPY_RESULT(dst, len) ((dst) + (len))
-#define TEST_MAIN
-#define TEST_NAME "stpcpy"
-#include "bench-string.h"
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
-char *simple_stpcpy (char *, const char *);
+s390_vx_libc_ifunc (__wcpcpy)
+weak_alias (__wcpcpy, wcpcpy)
-IMPL (simple_stpcpy, 0)
-IMPL (stpcpy, 1)
-
-char *
-simple_stpcpy (char *dst, const char *src)
-{
- while ((*dst++ = *src++) != '\0');
- return dst - 1;
-}
-
-#include "bench-strcpy.c"
+#else
+# include <wcsmbs/wcpcpy.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index 3858d74..1112e05 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -42,7 +42,8 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
isoc99_swscanf isoc99_vswscanf \
mbrtoc16 c16rtomb
-strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen
+strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
+ wcpcpy
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/benchtests/bench-stpcpy.c b/wcsmbs/test-wcpcpy.c
similarity index 62%
copy from benchtests/bench-stpcpy.c
copy to wcsmbs/test-wcpcpy.c
index 3d79432..2cf91a1 100644
--- a/benchtests/bench-stpcpy.c
+++ b/wcsmbs/test-wcpcpy.c
@@ -1,5 +1,5 @@
-/* Measure stpcpy functions.
- Copyright (C) 2013-2015 Free Software Foundation, Inc.
+/* Test wcspcpy functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -16,21 +16,5 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#define STRCPY_RESULT(dst, len) ((dst) + (len))
-#define TEST_MAIN
-#define TEST_NAME "stpcpy"
-#include "bench-string.h"
-
-char *simple_stpcpy (char *, const char *);
-
-IMPL (simple_stpcpy, 0)
-IMPL (stpcpy, 1)
-
-char *
-simple_stpcpy (char *dst, const char *src)
-{
- while ((*dst++ = *src++) != '\0');
- return dst - 1;
-}
-
-#include "bench-strcpy.c"
+#define WIDE 1
+#include "../string/test-stpcpy.c"
diff --git a/wcsmbs/wcpcpy.c b/wcsmbs/wcpcpy.c
index 1c05bca..0c8ded3 100644
--- a/wcsmbs/wcpcpy.c
+++ b/wcsmbs/wcpcpy.c
@@ -21,7 +21,9 @@
#define __need_ptrdiff_t
#include <stddef.h>
-
+#ifdef WCPCPY
+# define __wcpcpy WCPCPY
+#endif
/* Copy SRC to DEST, returning the address of the terminating L'\0' in
DEST. */
wchar_t *
@@ -43,4 +45,6 @@ __wcpcpy (dest, src)
return wcp;
}
+#ifndef WCPCPY
weak_alias (__wcpcpy, wcpcpy)
+#endif
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=680df122ab8a07806cb38d044292896334f76c01
commit 680df122ab8a07806cb38d044292896334f76c01
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:20 2015 +0200
S390: Optimize strcpy and wcscpy.
This patch provides optimized versions of strcpy and wcscpy with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strcpy-vx.S: New File.
* sysdeps/s390/multiarch/strcpy.c: Likewise.
* sysdeps/s390/multiarch/wcscpy-c.c: Likewise.
* sysdeps/s390/multiarch/wcscpy-vx.S: Likewise.
* sysdeps/s390/multiarch/wcscpy.c: Likewise.
* sysdeps/s390/s390-32/multiarch/strcpy.c: Likewise.
* sysdeps/s390/s390-64/multiarch/strcpy.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strcpy and
wcscpy functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strcpy, wcscpy.
* benchtests/bench-wcscpy.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcscpy.
diff --git a/ChangeLog b/ChangeLog
index e43a29d..2b631dc 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,21 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strcpy-vx.S: New File.
+ * sysdeps/s390/multiarch/strcpy.c: Likewise.
+ * sysdeps/s390/multiarch/wcscpy-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcscpy-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcscpy.c: Likewise.
+ * sysdeps/s390/s390-32/multiarch/strcpy.c: Likewise.
+ * sysdeps/s390/s390-64/multiarch/strcpy.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strcpy and
+ wcscpy functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strcpy, wcscpy.
+ * benchtests/bench-wcscpy.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcscpy.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/strnlen-c.c: New File.
* sysdeps/s390/multiarch/strnlen-vx.S: Likewise.
* sysdeps/s390/multiarch/strnlen.c: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 295738e..28892c8 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -36,7 +36,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
-wcsmbs-bench := wcslen wcsnlen
+wcsmbs-bench := wcslen wcsnlen wcscpy
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-wcscpy.c b/benchtests/bench-wcscpy.c
new file mode 100644
index 0000000..62c5825
--- /dev/null
+++ b/benchtests/bench-wcscpy.c
@@ -0,0 +1,20 @@
+/* Measure wcscpy functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strcpy.c"
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 3397f24..e2202b7 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,9 +1,11 @@
ifeq ($(subdir),string)
sysdep_routines += strlen strlen-vx strlen-c \
- strnlen strnlen-vx strnlen-c
+ strnlen strnlen-vx strnlen-c \
+ strcpy strcpy-vx
endif
ifeq ($(subdir),wcsmbs)
sysdep_routines += wcslen wcslen-vx wcslen-c \
- wcsnlen wcsnlen-vx wcsnlen-c
+ wcsnlen wcsnlen-vx wcsnlen-c \
+ wcscpy wcscpy-vx wcscpy-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index bc17c59..c9228d6 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -85,6 +85,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strnlen);
IFUNC_VX_IMPL (wcsnlen);
+ IFUNC_VX_IMPL (strcpy);
+ IFUNC_VX_IMPL (wcscpy);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/sysdeps/s390/multiarch/strcpy-vx.S b/sysdeps/s390/multiarch/strcpy-vx.S
new file mode 100644
index 0000000..9f6838e
--- /dev/null
+++ b/sysdeps/s390/multiarch/strcpy-vx.S
@@ -0,0 +1,109 @@
+/* Vector optimized 32/64 bit S/390 version of strcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char * strcpy (const char *dest, const char *src)
+ Copy string src to dest.
+
+ Register usage:
+ -r1=tmp
+ -r2=dest and return_value
+ -r3=src
+ -r4=tmp
+ -r5=current_len
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+*/
+ENTRY(__strcpy_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Lfound_align /* If found zero within loaded bytes,
+ copy bytes before and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r4 /* Compute highest index to 16byte boundary. */
+
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Find zero in 16byte aligned loop. */
+.Lloop:
+ vl %v16,0(%r5,%r3) /* Load s. */
+ vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16_0 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3)/* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezbs %v17,%v16,%v16
+ je .Lfound_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezbs %v17,%v18,%v18
+ je .Lfound_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ j .Lloop /* No zero found -> loop. */
+
+.Lfound_v16_32:
+ aghi %r5,32
+.Lfound_v16_0:
+ la %r3,0(%r5,%r2)
+ vlgvb %r4,%v17,7 /* Load byte index of zero. */
+ vstl %v16,%r4,0(%r3) /* Store characters including zero. */
+ br %r14
+
+.Lfound_v18_48:
+ aghi %r5,32
+.Lfound_v18_16:
+ la %r3,16(%r5,%r2)
+ vlgvb %r4,%v17,7 /* Load byte index of zero. */
+ vstl %v18,%r4,0(%r3) /* Store characters including zero. */
+ br %r14
+
+.Lfound_align:
+ vstl %v16,%r5,0(%r2) /* Copy characters including zero. */
+ br %r14
+END(__strcpy_vx)
+
+/* Use mvst-strcpy-implementation as default implementation. */
+# define strcpy __strcpy_c
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) strong_alias(__strcpy_c, __GI_strcpy)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+
+/* Include mvst-strcpy-implementation in s390-32/s390-64 subdirectory. */
+#include <strcpy.S>
diff --git a/sysdeps/s390/multiarch/strcpy.c b/sysdeps/s390/multiarch/strcpy.c
new file mode 100644
index 0000000..d57b5fa
--- /dev/null
+++ b/sysdeps/s390/multiarch/strcpy.c
@@ -0,0 +1,24 @@
+/* Multiple versions of strcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__strcpy, strcpy)
+#endif
diff --git a/sysdeps/s390/multiarch/wcscpy-c.c b/sysdeps/s390/multiarch/wcscpy-c.c
new file mode 100644
index 0000000..cff6208
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcscpy-c.c
@@ -0,0 +1,25 @@
+/* Default wcscpy implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSCPY __wcscpy_c
+
+# include <wchar.h>
+extern __typeof (wcscpy) __wcscpy_c;
+# include <wcsmbs/wcscpy.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcscpy-vx.S b/sysdeps/s390/multiarch/wcscpy-vx.S
new file mode 100644
index 0000000..d4a8099
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcscpy-vx.S
@@ -0,0 +1,111 @@
+/* Vector optimized 32/64 bit S/390 version of wcscpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* char * wcscpy (const wchar_t *dest, const wchar_t *src)
+ Copy string src to dest.
+
+ Register usage:
+ -r0=border-len for switching to vector-instructions
+ -r1=tmp
+ -r2=dest and return value
+ -r3=src
+ -r4=tmp
+ -r5=current_len
+ -v16=part of src
+ -v17=index of zero
+ -v18=part of src
+*/
+ENTRY(__wcscpy_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ tmll %r3,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Lfound_align /* If found zero within loaded bytes,
+ copy bytes before and return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,15 /* current_len = 15. */
+ slr %r5,%r4 /* Compute highest index to 16byte boundary. */
+
+ vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */
+ ahi %r5,1 /* Start loop at next character. */
+
+ /* Find zero in 16byte aligned loop. */
+.Lloop:
+ vl %v16,0(%r5,%r3) /* Load s. */
+ vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound_v16_0 /* Jump away if zero was found. */
+ vl %v18,16(%r5,%r3) /* Load next part of s. */
+ vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18_16
+ vl %v16,32(%r5,%r3)
+ vst %v18,16(%r5,%r2)
+ vfenezfs %v17,%v16,%v16
+ je .Lfound_v16_32
+ vl %v18,48(%r5,%r3)
+ vst %v16,32(%r5,%r2)
+ vfenezfs %v17,%v18,%v18
+ je .Lfound_v18_48
+ vst %v18,48(%r5,%r2)
+
+ aghi %r5,64
+ j .Lloop /* No zero found -> loop. */
+
+.Lfound_v16_32:
+ aghi %r5,32
+.Lfound_v16_0:
+ la %r3,0(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ vstl %v16,%r1,0(%r3) /* Copy characters including zero. */
+ br %r14
+
+.Lfound_v18_48:
+ aghi %r5,32
+.Lfound_v18_16:
+ la %r3,16(%r5,%r2)
+ vlgvb %r1,%v17,7 /* Load byte index of zero. */
+ aghi %r1,3 /* Also copy remaining bytes of zero. */
+ vstl %v18,%r1,0(%r3) /* Copy characters including zero. */
+ br %r14
+
+.Lfound_align:
+ aghi %r5,3 /* Also copy remaining bytes of zero. */
+ vstl %v16,%r5,0(%r2) /* Copy characters including zero. */
+ br %r14
+
+.Lfallback:
+ jg __wcscpy_c
+END(__wcscpy_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcscpy.c b/sysdeps/s390/multiarch/wcscpy.c
new file mode 100644
index 0000000..0df1779
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcscpy.c
@@ -0,0 +1,27 @@
+/* Multiple versions of wcscpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__wcscpy, wcscpy)
+
+#else
+# include <wcsmbs/wcscpy.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/s390-32/multiarch/strcpy.c b/sysdeps/s390/s390-32/multiarch/strcpy.c
new file mode 100644
index 0000000..b02c392
--- /dev/null
+++ b/sysdeps/s390/s390-32/multiarch/strcpy.c
@@ -0,0 +1,21 @@
+/* Multiple versions of strcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* This wrapper-file is needed, because otherwise file
+ sysdeps/s390/s390-[32|64]/strcpy.S will be used. */
+#include <sysdeps/s390/multiarch/strcpy.c>
diff --git a/sysdeps/s390/s390-64/multiarch/strcpy.c b/sysdeps/s390/s390-64/multiarch/strcpy.c
new file mode 100644
index 0000000..b02c392
--- /dev/null
+++ b/sysdeps/s390/s390-64/multiarch/strcpy.c
@@ -0,0 +1,21 @@
+/* Multiple versions of strcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* This wrapper-file is needed, because otherwise file
+ sysdeps/s390/s390-[32|64]/strcpy.S will be used. */
+#include <sysdeps/s390/multiarch/strcpy.c>
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=fcf40ebe2682fd65d64f94d69a3df798960cf1b7
commit fcf40ebe2682fd65d64f94d69a3df798960cf1b7
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:20 2015 +0200
S390: Optimize strnlen and wcsnlen.
This patch provides optimized versions of strnlen and wcsnlen with the z13
vector instructions.
ChangeLog:
* sysdeps/s390/multiarch/strnlen-c.c: New File.
* sysdeps/s390/multiarch/strnlen-vx.S: Likewise.
* sysdeps/s390/multiarch/strnlen.c: Likewise.
* sysdeps/s390/multiarch/wcsnlen-c.c: Likewise.
* sysdeps/s390/multiarch/wcsnlen-vx.S: Likewise.
* sysdeps/s390/multiarch/wcsnlen.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strnlen and
wcsnlen functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strnlen, wcsnlen.
* wcsmbs/wcsnlen.c: Use WCSNLEN if defined.
* string/test-strnlen.c: Add wcsnlen support.
* wcsmbs/test-wcsnlen.c: New File.
* wcsmbs/Makefile (strop-tests): Add wcsnlen.
* benchtests/bench-strnlen.c: Add wcsnlen support.
* benchtests/bench-wcsnlen.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcsnlen.
diff --git a/ChangeLog b/ChangeLog
index 2d69da1..e43a29d 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/strnlen-c.c: New File.
+ * sysdeps/s390/multiarch/strnlen-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strnlen.c: Likewise.
+ * sysdeps/s390/multiarch/wcsnlen-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcsnlen-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcsnlen.c: Likewise.
+ * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strnlen and
+ wcsnlen functions.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add ifunc test for strnlen, wcsnlen.
+ * wcsmbs/wcsnlen.c: Use WCSNLEN if defined.
+ * string/test-strnlen.c: Add wcsnlen support.
+ * wcsmbs/test-wcsnlen.c: New File.
+ * wcsmbs/Makefile (strop-tests): Add wcsnlen.
+ * benchtests/bench-strnlen.c: Add wcsnlen support.
+ * benchtests/bench-wcsnlen.c: New File.
+ * benchtests/Makefile (wcsmbs-bench): Add wcsnlen.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/Makefile: New File.
* sysdeps/s390/multiarch/strlen-c.c: Likewise.
* sysdeps/s390/multiarch/strlen-vx.S: Likewise.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 5d4afab..295738e 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -36,7 +36,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
-wcsmbs-bench := wcslen
+wcsmbs-bench := wcslen wcsnlen
string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
diff --git a/benchtests/bench-strnlen.c b/benchtests/bench-strnlen.c
index 35266e5..cbdce75 100644
--- a/benchtests/bench-strnlen.c
+++ b/benchtests/bench-strnlen.c
@@ -17,17 +17,36 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "strnlen"
+#ifndef WIDE
+# define TEST_NAME "strnlen"
+#else
+# define TEST_NAME "wcsnlen"
+#endif /* WIDE */
#include "bench-string.h"
-typedef size_t (*proto_t) (const char *, size_t);
-size_t simple_strnlen (const char *, size_t);
-
-IMPL (simple_strnlen, 0)
-IMPL (strnlen, 1)
+#ifndef WIDE
+# define STRNLEN strnlen
+# define CHAR char
+# define BIG_CHAR CHAR_MAX
+# define MIDDLE_CHAR 127
+# define SIMPLE_STRNLEN simple_strnlen
+#else
+# include <wchar.h>
+# define STRNLEN wcsnlen
+# define CHAR wchar_t
+# define BIG_CHAR WCHAR_MAX
+# define MIDDLE_CHAR 1121
+# define SIMPLE_STRNLEN simple_wcsnlen
+#endif /* WIDE */
+
+typedef size_t (*proto_t) (const CHAR *, size_t);
+size_t SIMPLE_STRNLEN (const CHAR *, size_t);
+
+IMPL (SIMPLE_STRNLEN, 0)
+IMPL (STRNLEN, 1)
size_t
-simple_strnlen (const char *s, size_t maxlen)
+SIMPLE_STRNLEN (const CHAR *s, size_t maxlen)
{
size_t i;
@@ -36,7 +55,7 @@ simple_strnlen (const char *s, size_t maxlen)
}
static void
-do_one_test (impl_t *impl, const char *s, size_t maxlen, size_t exp_len)
+do_one_test (impl_t *impl, const CHAR *s, size_t maxlen, size_t exp_len)
{
size_t len = CALL (impl, s, maxlen), i, iters = INNER_LOOP_ITERS;
timing_t start, stop, cur;
@@ -66,18 +85,20 @@ do_test (size_t align, size_t len, size_t maxlen, int max_char)
{
size_t i;
- align &= 7;
- if (align + len >= page_size)
+ align &= 63;
+ if ((align + len) * sizeof (CHAR) >= page_size)
return;
+ CHAR *buf = (CHAR *) (buf1);
+
for (i = 0; i < len; ++i)
- buf1[align + i] = 1 + 7 * i % max_char;
- buf1[align + len] = 0;
+ buf[align + i] = 1 + 7 * i % max_char;
+ buf[align + len] = 0;
printf ("Length %4zd, alignment %2zd:", len, align);
FOR_EACH_IMPL (impl, 0)
- do_one_test (impl, (char *) (buf1 + align), maxlen, MIN (len, maxlen));
+ do_one_test (impl, (CHAR *) (buf + align), maxlen, MIN (len, maxlen));
putchar ('\n');
}
@@ -96,34 +117,34 @@ test_main (void)
for (i = 1; i < 8; ++i)
{
- do_test (0, i, i - 1, 127);
- do_test (0, i, i, 127);
- do_test (0, i, i + 1, 127);
+ do_test (0, i, i - 1, MIDDLE_CHAR);
+ do_test (0, i, i, MIDDLE_CHAR);
+ do_test (0, i, i + 1, MIDDLE_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (i, i, i - 1, 127);
- do_test (i, i, i, 127);
- do_test (i, i, i + 1, 127);
+ do_test (i, i, i - 1, MIDDLE_CHAR);
+ do_test (i, i, i, MIDDLE_CHAR);
+ do_test (i, i, i + 1, MIDDLE_CHAR);
}
for (i = 2; i <= 10; ++i)
{
- do_test (0, 1 << i, 5000, 127);
- do_test (1, 1 << i, 5000, 127);
+ do_test (0, 1 << i, 5000, MIDDLE_CHAR);
+ do_test (1, 1 << i, 5000, MIDDLE_CHAR);
}
for (i = 1; i < 8; ++i)
- do_test (0, i, 5000, 255);
+ do_test (0, i, 5000, BIG_CHAR);
for (i = 1; i < 8; ++i)
- do_test (i, i, 5000, 255);
+ do_test (i, i, 5000, BIG_CHAR);
for (i = 2; i <= 10; ++i)
{
- do_test (0, 1 << i, 5000, 255);
- do_test (1, 1 << i, 5000, 255);
+ do_test (0, 1 << i, 5000, BIG_CHAR);
+ do_test (1, 1 << i, 5000, BIG_CHAR);
}
return ret;
diff --git a/benchtests/bench-wcsnlen.c b/benchtests/bench-wcsnlen.c
new file mode 100644
index 0000000..2b5a51c
--- /dev/null
+++ b/benchtests/bench-wcsnlen.c
@@ -0,0 +1,20 @@
+/* Measure wcsnlen functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strnlen.c"
diff --git a/string/test-strnlen.c b/string/test-strnlen.c
index 96797e1..8d35cac 100644
--- a/string/test-strnlen.c
+++ b/string/test-strnlen.c
@@ -1,4 +1,4 @@
-/* Test and measure strlen functions.
+/* Test strlen functions.
Copyright (C) 1999-2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
Written by Jakub Jelinek <jakub@redhat.com>, 1999.
@@ -18,17 +18,36 @@
<http://www.gnu.org/licenses/>. */
#define TEST_MAIN
-#define TEST_NAME "strnlen"
+#ifndef WIDE
+# define TEST_NAME "strnlen"
+#else
+# define TEST_NAME "wcsnlen"
+#endif /* !WIDE */
#include "test-string.h"
-typedef size_t (*proto_t) (const char *, size_t);
-size_t simple_strnlen (const char *, size_t);
-
-IMPL (simple_strnlen, 0)
-IMPL (strnlen, 1)
+#ifndef WIDE
+# define STRNLEN strnlen
+# define CHAR char
+# define BIG_CHAR CHAR_MAX
+# define MIDDLE_CHAR 127
+# define SIMPLE_STRNLEN simple_strnlen
+#else
+# include <wchar.h>
+# define STRNLEN wcsnlen
+# define CHAR wchar_t
+# define BIG_CHAR WCHAR_MAX
+# define MIDDLE_CHAR 1121
+# define SIMPLE_STRNLEN simple_wcsnlen
+#endif /* !WIDE */
+
+typedef size_t (*proto_t) (const CHAR *, size_t);
+size_t SIMPLE_STRNLEN (const CHAR *, size_t);
+
+IMPL (SIMPLE_STRNLEN, 0)
+IMPL (STRNLEN, 1)
size_t
-simple_strnlen (const char *s, size_t maxlen)
+SIMPLE_STRNLEN (const CHAR *s, size_t maxlen)
{
size_t i;
@@ -37,7 +56,7 @@ simple_strnlen (const char *s, size_t maxlen)
}
static void
-do_one_test (impl_t *impl, const char *s, size_t maxlen, size_t exp_len)
+do_one_test (impl_t *impl, const CHAR *s, size_t maxlen, size_t exp_len)
{
size_t len = CALL (impl, s, maxlen);
if (len != exp_len)
@@ -54,23 +73,25 @@ do_test (size_t align, size_t len, size_t maxlen, int max_char)
{
size_t i;
- align &= 7;
- if (align + len >= page_size)
+ align &= 63;
+ if ((align + len) * sizeof (CHAR) >= page_size)
return;
+ CHAR *buf = (CHAR *) (buf1);
+
for (i = 0; i < len; ++i)
- buf1[align + i] = 1 + 7 * i % max_char;
- buf1[align + len] = 0;
+ buf[align + i] = 1 + 11111 * i % max_char;
+ buf[align + len] = 0;
FOR_EACH_IMPL (impl, 0)
- do_one_test (impl, (char *) (buf1 + align), maxlen, MIN (len, maxlen));
+ do_one_test (impl, (CHAR *) (buf + align), maxlen, MIN (len, maxlen));
}
static void
do_random_tests (void)
{
size_t i, j, n, align, len;
- unsigned char *p = buf1 + page_size - 512;
+ CHAR *p = (CHAR *) (buf1 + page_size - 512 * sizeof (CHAR));
for (n = 0; n < ITERATIONS; n++)
{
@@ -97,25 +118,25 @@ do_random_tests (void)
FOR_EACH_IMPL (impl, 1)
{
if (len > 0
- && CALL (impl, (char *) (p + align), len - 1) != len - 1)
+ && CALL (impl, (CHAR *) (p + align), len - 1) != len - 1)
{
error (0, 0, "Iteration %zd (limited) - wrong result in function %s (%zd) %zd != %zd, p %p",
n, impl->name, align,
- CALL (impl, (char *) (p + align), len - 1), len - 1, p);
+ CALL (impl, (CHAR *) (p + align), len - 1), len - 1, p);
ret = 1;
}
- if (CALL (impl, (char *) (p + align), len) != len)
+ if (CALL (impl, (CHAR *) (p + align), len) != len)
{
error (0, 0, "Iteration %zd (exact) - wrong result in function %s (%zd) %zd != %zd, p %p",
n, impl->name, align,
- CALL (impl, (char *) (p + align), len), len, p);
+ CALL (impl, (CHAR *) (p + align), len), len, p);
ret = 1;
}
- if (CALL (impl, (char *) (p + align), len + 1) != len)
+ if (CALL (impl, (CHAR *) (p + align), len + 1) != len)
{
error (0, 0, "Iteration %zd (long) - wrong result in function %s (%zd) %zd != %zd, p %p",
n, impl->name, align,
- CALL (impl, (char *) (p + align), len + 1), len, p);
+ CALL (impl, (CHAR *) (p + align), len + 1), len, p);
ret = 1;
}
}
@@ -136,34 +157,34 @@ test_main (void)
for (i = 1; i < 8; ++i)
{
- do_test (0, i, i - 1, 127);
- do_test (0, i, i, 127);
- do_test (0, i, i + 1, 127);
+ do_test (0, i, i - 1, MIDDLE_CHAR);
+ do_test (0, i, i, MIDDLE_CHAR);
+ do_test (0, i, i + 1, MIDDLE_CHAR);
}
for (i = 1; i < 8; ++i)
{
- do_test (i, i, i - 1, 127);
- do_test (i, i, i, 127);
- do_test (i, i, i + 1, 127);
+ do_test (i, i, i - 1, MIDDLE_CHAR);
+ do_test (i, i, i, MIDDLE_CHAR);
+ do_test (i, i, i + 1, MIDDLE_CHAR);
}
for (i = 2; i <= 10; ++i)
{
- do_test (0, 1 << i, 5000, 127);
- do_test (1, 1 << i, 5000, 127);
+ do_test (0, 1 << i, 5000, MIDDLE_CHAR);
+ do_test (1, 1 << i, 5000, MIDDLE_CHAR);
}
for (i = 1; i < 8; ++i)
- do_test (0, i, 5000, 255);
+ do_test (0, i, 5000, BIG_CHAR);
for (i = 1; i < 8; ++i)
- do_test (i, i, 5000, 255);
+ do_test (i, i, 5000, BIG_CHAR);
for (i = 2; i <= 10; ++i)
{
- do_test (0, 1 << i, 5000, 255);
- do_test (1, 1 << i, 5000, 255);
+ do_test (0, 1 << i, 5000, BIG_CHAR);
+ do_test (1, 1 << i, 5000, BIG_CHAR);
}
do_random_tests ();
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 3a98098..3397f24 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,7 +1,9 @@
ifeq ($(subdir),string)
-sysdep_routines += strlen strlen-vx strlen-c
+sysdep_routines += strlen strlen-vx strlen-c \
+ strnlen strnlen-vx strnlen-c
endif
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcslen wcslen-vx wcslen-c
+sysdep_routines += wcslen wcslen-vx wcslen-c \
+ wcsnlen wcsnlen-vx wcsnlen-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index e9639ef..bc17c59 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -82,6 +82,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (strlen);
IFUNC_VX_IMPL (wcslen);
+ IFUNC_VX_IMPL (strnlen);
+ IFUNC_VX_IMPL (wcsnlen);
+
#endif /* HAVE_S390_VX_ASM_SUPPORT */
return i;
diff --git a/wcsmbs/wcsnlen.c b/sysdeps/s390/multiarch/strnlen-c.c
similarity index 55%
copy from wcsmbs/wcsnlen.c
copy to sysdeps/s390/multiarch/strnlen-c.c
index 2bee705..72e82ae 100644
--- a/wcsmbs/wcsnlen.c
+++ b/sysdeps/s390/multiarch/strnlen-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1998-2015 Free Software Foundation, Inc.
+/* Default strnlen implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@cygnus.com>, 1998.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,32 +16,15 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
-
-
-/* Copy SRC to DEST. */
-size_t
-__wcsnlen (s, maxlen)
- const wchar_t *s;
- size_t maxlen;
-{
- size_t len = 0;
-
- while (maxlen > 0 && s[len] != L'\0')
- {
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- --maxlen;
- }
-
- return len;
-}
-weak_alias (__wcsnlen, wcsnlen)
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRNLEN __strnlen_c
+# ifdef SHARED
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__strnlen_c, __GI_strnlen, __strnlen_c); \
+ strong_alias (__strnlen_c, __strnlen_c_1); \
+ __hidden_ver1 (__strnlen_c_1, __GI___strnlen, __strnlen_c_1);
+# endif /* SHARED */
+
+# include <string/strnlen.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strnlen-vx.S b/sysdeps/s390/multiarch/strnlen-vx.S
new file mode 100644
index 0000000..12c49de
--- /dev/null
+++ b/sysdeps/s390/multiarch/strnlen-vx.S
@@ -0,0 +1,134 @@
+/* Vector optimized 32/64 bit S/390 version of strnlen.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* size_t strnlen (const char *s, size_t maxlen)
+ Returns the number of characters in s or at most maxlen.
+
+ Register usage:
+ -r1=tmp
+ -r2=address of string
+ -r3=maxlen (number of characters to be read)
+ -r4=tmp
+ -r5=current_len and return_value
+ -v16=part of s
+*/
+ENTRY(__strnlen_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r3,%r3
+# endif /* !defined __s390x__ */
+
+ clgfi %r3,0 /* if maxlen == 0, return 0. */
+ locgre %r2,%r3
+ ber %r14
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r1,%r1 /* Convert 32bit to 64bit. */
+
+ vfenezb %v16,%v16,%v16 /* Find element not equal with zero search. */
+ clgr %r1,%r3
+ locgrh %r1,%r3 /* loaded_byte_count
+ = min (loaded_byte_count, maxlen) */
+
+ vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */
+ clr %r5,%r1 /* If found zero within loaded bytes? */
+ locgrl %r2,%r5 /* Then copy return value. */
+ blr %r14 /* And return. */
+
+ clgr %r1,%r3 /* If loaded_byte_count == maxlen? */
+ locgre %r2,%r3 /* Then copy return value. */
+ ber %r14 /* And return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r4 /* Compute bytes to 16bytes boundary. */
+
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r3,.Lloop64
+
+ /* Find zero in max 64byte with aligned s. */
+.Llt64:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound /* Jump away if zero was found. */
+ aghi %r5,16
+ clgrjhe %r5,%r3,.Lfound /* current_len >= maxlen -> end. */
+ vl %v16,0(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Lfound
+ aghi %r5,16
+ clgrjhe %r5,%r3,.Lfound
+ vl %v16,0(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Lfound
+ aghi %r5,16
+ clgrjhe %r5,%r3,.Lfound
+ vl %v16,0(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ j .Lfound
+
+.Lfound48:
+ aghi %r5,16
+.Lfound32:
+ aghi %r5,16
+.Lfound16:
+ aghi %r5,16
+.Lfound:
+ vlgvb %r4,%v16,7 /* Load byte index of zero or 16 if no zero. */
+ algr %r5,%r4
+
+ clgr %r5,%r3
+ locgrh %r5,%r3 /* Return min (current_len, maxlen). */
+ lgr %r2,%r5
+ br %r14
+
+ /* Find zero in 16 byte aligned loop. */
+.Lloop64:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound /* Jump away if zero was found. */
+ vl %v16,16(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Lfound48
+
+ aghi %r5,64
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r3,.Lloop64
+
+ j .Llt64
+END(__strnlen_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcsnlen.c b/sysdeps/s390/multiarch/strnlen.c
similarity index 55%
copy from wcsmbs/wcsnlen.c
copy to sysdeps/s390/multiarch/strnlen.c
index 2bee705..51659de 100644
--- a/wcsmbs/wcsnlen.c
+++ b/sysdeps/s390/multiarch/strnlen.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1998-2015 Free Software Foundation, Inc.
+/* Multiple versions of strnlen.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@cygnus.com>, 1998.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,32 +16,14 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+s390_vx_libc_ifunc (__strnlen)
+weak_alias (__strnlen, strnlen)
+libc_hidden_def (strnlen)
-/* Copy SRC to DEST. */
-size_t
-__wcsnlen (s, maxlen)
- const wchar_t *s;
- size_t maxlen;
-{
- size_t len = 0;
-
- while (maxlen > 0 && s[len] != L'\0')
- {
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- --maxlen;
- }
-
- return len;
-}
-weak_alias (__wcsnlen, wcsnlen)
+#else
+# include <string/strnlen.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/wcsnlen.c b/sysdeps/s390/multiarch/wcsnlen-c.c
similarity index 55%
copy from wcsmbs/wcsnlen.c
copy to sysdeps/s390/multiarch/wcsnlen-c.c
index 2bee705..1ead4b1 100644
--- a/wcsmbs/wcsnlen.c
+++ b/sysdeps/s390/multiarch/wcsnlen-c.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1998-2015 Free Software Foundation, Inc.
+/* Default wcsnlen implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@cygnus.com>, 1998.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,32 +16,10 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSNLEN __wcsnlen_c
-
-/* Copy SRC to DEST. */
-size_t
-__wcsnlen (s, maxlen)
- const wchar_t *s;
- size_t maxlen;
-{
- size_t len = 0;
-
- while (maxlen > 0 && s[len] != L'\0')
- {
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- --maxlen;
- }
-
- return len;
-}
-weak_alias (__wcsnlen, wcsnlen)
+# include <wchar.h>
+extern __typeof (__wcsnlen) __wcsnlen_c;
+# include <wcsmbs/wcsnlen.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcsnlen-vx.S b/sysdeps/s390/multiarch/wcsnlen-vx.S
new file mode 100644
index 0000000..7932e1b
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcsnlen-vx.S
@@ -0,0 +1,151 @@
+/* Vector optimized 32/64 bit S/390 version of wcsnlen.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* size_t wcsnlen (const wchar_t *s, size_t maxlen)
+ Returns the number of characters in s or at most maxlen.
+
+ Register usage:
+ -r1=tmp
+ -r2=address of string
+ -r3=maxlen (number of characters to be read)
+ -r4=tmp
+ -r5=current_len and return_value
+ -v16=part of s
+*/
+ENTRY(__wcsnlen_vx)
+
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+# if !defined __s390x__
+ llgfr %r3,%r3
+# endif /* !defined __s390x__ */
+
+ clgfi %r3,0 /* if maxlen == 0, return 0. */
+ locgre %r2,%r3
+ ber %r14
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+ llgfr %r1,%r1 /* Convert 32bit to 64bit. */
+
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ /* Check range of maxlen and convert to byte-count. */
+# ifdef __s390x__
+ tmhh %r3,49152 /* Test bit 0 or 1 of maxlen. */
+ lghi %r4,-4 /* Max byte-count is 18446744073709551612. */
+# else
+ tmlh %r3,49152 /* Test bit 0 or 1 of maxlen. */
+ llilf %r4,4294967292 /* Max byte-count is 4294967292. */
+# endif /* !__s390x__ */
+ sllg %r3,%r3,2 /* Convert character-count to byte-count. */
+ locgrne %r3,%r4 /* Use max byte-count, if bit 0/1 was one. */
+
+ vfenezf %v16,%v16,%v16 /* Find element not equal with zero search. */
+ clgr %r1,%r3
+ locgrh %r1,%r3 /* loaded_byte_count
+ = min (loaded_byte_count, maxlen) */
+
+ vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */
+ clrjl %r5,%r1,.Lend /* Found zero within loaded bytes -> return. */
+
+ clgr %r1,%r3 /* If loaded_byte_count == maxlen -> end. */
+ locgre %r5,%r3
+ je .Lend
+
+ /* Align s to 16 byte. */
+ risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r4 /* Compute bytes to 16bytes boundary. */
+
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r3,.Lloop64
+
+ /* Find zero in max 64byte with aligned s. */
+.Llt64:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound /* Jump away if zero was found. */
+ aghi %r5,16
+ clgrjhe %r5,%r3,.Lfound /* If current_len >= maxlen -> end. */
+ vl %v16,0(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Lfound
+ aghi %r5,16
+ clgrjhe %r5,%r3,.Lfound
+ vl %v16,0(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Lfound
+ aghi %r5,16
+ clgrjhe %r5,%r3,.Lfound
+ vl %v16,0(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ j .Lfound
+
+.Lfound48:
+ aghi %r5,16
+.Lfound32:
+ aghi %r5,16
+.Lfound16:
+ aghi %r5,16
+.Lfound:
+ vlgvb %r4,%v16,7 /* Load byte index of zero or 16 if no zero. */
+ algr %r5,%r4
+
+ clgr %r5,%r3
+ locgrh %r5,%r3 /* Return min (current_len, maxlen). */
+.Lend:
+ srlg %r2,%r5,2 /* Convert byte-count to character-count. */
+ br %r14
+
+ /* Find zero in 16byte aligned loop. */
+.Lloop64:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound /* Jump away if zero was found. */
+ vl %v16,16(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Lfound48
+
+ aghi %r5,64
+ lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */
+ aghi %r1,64
+ clgrjl %r1,%r3,.Lloop64
+
+ j .Llt64
+
+.Lfallback:
+ jg __wcsnlen_c
+END(__wcsnlen_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/wcsmbs/wcsnlen.c b/sysdeps/s390/multiarch/wcsnlen.c
similarity index 57%
copy from wcsmbs/wcsnlen.c
copy to sysdeps/s390/multiarch/wcsnlen.c
index 2bee705..bf2fd63 100644
--- a/wcsmbs/wcsnlen.c
+++ b/sysdeps/s390/multiarch/wcsnlen.c
@@ -1,6 +1,6 @@
-/* Copyright (C) 1998-2015 Free Software Foundation, Inc.
+/* Multiple versions of wcsnlen.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Contributed by Ulrich Drepper <drepper@cygnus.com>, 1998.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -16,32 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <wchar.h>
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
-
-/* Copy SRC to DEST. */
-size_t
-__wcsnlen (s, maxlen)
- const wchar_t *s;
- size_t maxlen;
-{
- size_t len = 0;
-
- while (maxlen > 0 && s[len] != L'\0')
- {
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- if (--maxlen == 0 || s[len] == L'\0')
- return len;
- ++len;
- --maxlen;
- }
-
- return len;
-}
+s390_vx_libc_ifunc (__wcsnlen)
weak_alias (__wcsnlen, wcsnlen)
+
+#else
+# include <wcsmbs/wcsnlen.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
index 44a4494..3858d74 100644
--- a/wcsmbs/Makefile
+++ b/wcsmbs/Makefile
@@ -42,7 +42,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
isoc99_swscanf isoc99_vswscanf \
mbrtoc16 c16rtomb
-strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy
+strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen
tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \
tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \
tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests))
diff --git a/wcsmbs/test-wcsnlen.c b/wcsmbs/test-wcsnlen.c
new file mode 100644
index 0000000..262ab30
--- /dev/null
+++ b/wcsmbs/test-wcsnlen.c
@@ -0,0 +1,20 @@
+/* Test wcsnlen function.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "../string/test-strnlen.c"
diff --git a/wcsmbs/wcsnlen.c b/wcsmbs/wcsnlen.c
index 2bee705..e928ab6 100644
--- a/wcsmbs/wcsnlen.c
+++ b/wcsmbs/wcsnlen.c
@@ -18,8 +18,11 @@
#include <wchar.h>
+#ifdef WCSNLEN
+# define __wcsnlen WCSNLEN
+#endif
-/* Copy SRC to DEST. */
+/* Return length of string S at most maxlen. */
size_t
__wcsnlen (s, maxlen)
const wchar_t *s;
@@ -44,4 +47,6 @@ __wcsnlen (s, maxlen)
return len;
}
+#ifndef WCSNLEN
weak_alias (__wcsnlen, wcsnlen)
+#endif
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=9472f35a0a6dbbda82ce103aaf0f5013f5d46e34
commit 9472f35a0a6dbbda82ce103aaf0f5013f5d46e34
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:20 2015 +0200
S390: Optimize strlen and wcslen.
This patch provides optimized versions of strlen and wcslen with the z13 vector
instructions.
The helper macro IFUNC_VX_IMPL is introduced and is used to register all
__<func>_c() and __<func>_vx() functions within __libc_ifunc_impl_list()
to the ifunc test framework.
ChangeLog:
* sysdeps/s390/multiarch/Makefile: New File.
* sysdeps/s390/multiarch/strlen-c.c: Likewise.
* sysdeps/s390/multiarch/strlen-vx.S: Likewise.
* sysdeps/s390/multiarch/strlen.c: Likewise.
* sysdeps/s390/multiarch/wcslen-c.c: Likewise.
* sysdeps/s390/multiarch/wcslen-vx.S: Likewise.
* sysdeps/s390/multiarch/wcslen.c: Likewise.
* string/strlen.c (STRLEN): Define and use macro.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(IFUNC_VX_IMPL): New macro function.
(__libc_ifunc_impl_list): Add ifunc test for strlen, wcslen.
* benchtests/Makefile (wcsmbs-bench): New variable.
(string-bench-all): Added wcsmbs-bench.
* benchtests/bench-wcslen.c: New File.
diff --git a/ChangeLog b/ChangeLog
index 9304378..2d69da1 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,22 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/Makefile: New File.
+ * sysdeps/s390/multiarch/strlen-c.c: Likewise.
+ * sysdeps/s390/multiarch/strlen-vx.S: Likewise.
+ * sysdeps/s390/multiarch/strlen.c: Likewise.
+ * sysdeps/s390/multiarch/wcslen-c.c: Likewise.
+ * sysdeps/s390/multiarch/wcslen-vx.S: Likewise.
+ * sysdeps/s390/multiarch/wcslen.c: Likewise.
+ * string/strlen.c (STRLEN): Define and use macro.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c
+ (IFUNC_VX_IMPL): New macro function.
+ (__libc_ifunc_impl_list): Add ifunc test for strlen, wcslen.
+ * benchtests/Makefile (wcsmbs-bench): New variable.
+ (string-bench-all): Added wcsmbs-bench.
+ * benchtests/bench-wcslen.c: New File.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/multiarch/ifunc-resolve.h (s390_vx_libc_ifunc,
s390_vx_libc_ifunc2): New macro function.
diff --git a/benchtests/Makefile b/benchtests/Makefile
index 8e615e5..5d4afab 100644
--- a/benchtests/Makefile
+++ b/benchtests/Makefile
@@ -36,7 +36,8 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \
strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \
strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \
strcoll
-string-bench-all := $(string-bench)
+wcsmbs-bench := wcslen
+string-bench-all := $(string-bench) ${wcsmbs-bench}
# We have to generate locales
LOCALES := en_US.UTF-8 tr_TR.UTF-8 cs_CZ.UTF-8 fa_IR.UTF-8 fr_FR.UTF-8 \
diff --git a/benchtests/bench-wcslen.c b/benchtests/bench-wcslen.c
new file mode 100644
index 0000000..4e9d085
--- /dev/null
+++ b/benchtests/bench-wcslen.c
@@ -0,0 +1,20 @@
+/* Measure wcslen functions.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#define WIDE 1
+#include "bench-strlen.c"
diff --git a/string/strlen.c b/string/strlen.c
index d066bde..b7fd645 100644
--- a/string/strlen.c
+++ b/string/strlen.c
@@ -23,10 +23,14 @@
#undef strlen
+#ifndef STRLEN
+# define STRLEN strlen
+#endif
+
/* Return the length of the null-terminated string STR. Scan for
the null terminator quickly by testing four bytes at a time. */
size_t
-strlen (const char *str)
+STRLEN (const char *str)
{
const char *char_ptr;
const unsigned long int *longword_ptr;
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
new file mode 100644
index 0000000..3a98098
--- /dev/null
+++ b/sysdeps/s390/multiarch/Makefile
@@ -0,0 +1,7 @@
+ifeq ($(subdir),string)
+sysdep_routines += strlen strlen-vx strlen-c
+endif
+
+ifeq ($(subdir),wcsmbs)
+sysdep_routines += wcslen wcslen-vx wcslen-c
+endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index c330904..e9639ef 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -18,6 +18,7 @@
#include <assert.h>
#include <string.h>
+#include <wchar.h>
#include <ifunc-impl-list.h>
#include <ifunc-resolve.h>
@@ -70,5 +71,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
#endif /* SHARED */
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+
+# define IFUNC_VX_IMPL(FUNC) \
+ IFUNC_IMPL (i, name, FUNC, \
+ IFUNC_IMPL_ADD (array, i, FUNC, dl_hwcap & HWCAP_S390_VX, \
+ __##FUNC##_vx) \
+ IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))
+
+ IFUNC_VX_IMPL (strlen);
+ IFUNC_VX_IMPL (wcslen);
+
+#endif /* HAVE_S390_VX_ASM_SUPPORT */
+
return i;
}
diff --git a/sysdeps/s390/multiarch/strlen-c.c b/sysdeps/s390/multiarch/strlen-c.c
new file mode 100644
index 0000000..96635ce
--- /dev/null
+++ b/sysdeps/s390/multiarch/strlen-c.c
@@ -0,0 +1,28 @@
+/* Default strlen implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define STRLEN __strlen_c
+# ifdef SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strlen_c, __GI_strlen, __strlen_c);
+# endif /* SHARED */
+
+# include <string/strlen.c>
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strlen-vx.S b/sysdeps/s390/multiarch/strlen-vx.S
new file mode 100644
index 0000000..83f360f
--- /dev/null
+++ b/sysdeps/s390/multiarch/strlen-vx.S
@@ -0,0 +1,84 @@
+/* Vector optimized 32/64 bit S/390 version of strlen.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* size_t strlen (const char *s)
+ Returns length of string s.
+
+ Register usage:
+ -r1=bytes to 4k-byte boundary
+ -r2=s
+ -r3=tmp
+ -r4=tmp
+ -r5=current_len and return_value
+ -v16=part of s
+*/
+ENTRY(__strlen_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ vfenezb %v16,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r4,%v16,7 /* Load zero index or 16 if not found. */
+ clr %r4,%r1 /* If found zero within loaded bytes? */
+ locgrl %r2,%r4 /* Then copy return value. */
+ blr %r14 /* And return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r3,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r3 /* Compute bytes to 16bytes boundary. */
+
+ /* Find zero in 16 byte aligned loop. */
+.Lloop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound /* Jump away if zero was found. */
+ vl %v16,16(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfenezbs %v16,%v16,%v16
+ je .Lfound48
+
+ aghi %r5,64
+ j .Lloop /* No zero found -> loop. */
+
+.Lfound48:
+ aghi %r5,16
+.Lfound32:
+ aghi %r5,16
+.Lfound16:
+ aghi %r5,16
+.Lfound:
+ vlgvb %r2,%v16,7 /* Load byte index of zero. */
+ algr %r2,%r5
+ br %r14
+END(__strlen_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/strlen.c b/sysdeps/s390/multiarch/strlen.c
new file mode 100644
index 0000000..d5342ff
--- /dev/null
+++ b/sysdeps/s390/multiarch/strlen.c
@@ -0,0 +1,27 @@
+/* Multiple versions of strlen.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <string.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc2 (__strlen, strlen)
+
+#else
+# include <string/strlen.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
diff --git a/sysdeps/s390/multiarch/wcslen-c.c b/sysdeps/s390/multiarch/wcslen-c.c
new file mode 100644
index 0000000..b41a956
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcslen-c.c
@@ -0,0 +1,25 @@
+/* Default wcslen implementation for S/390.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# define WCSLEN __wcslen_c
+
+# include <wchar.h>
+extern __typeof (__wcslen) __wcslen_c;
+# include <wcsmbs/wcslen.c>
+#endif
diff --git a/sysdeps/s390/multiarch/wcslen-vx.S b/sysdeps/s390/multiarch/wcslen-vx.S
new file mode 100644
index 0000000..f32195f
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcslen-vx.S
@@ -0,0 +1,91 @@
+/* Vector optimized 32/64 bit S/390 version of wcslen.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+
+# include "sysdep.h"
+# include "asm-syntax.h"
+
+ .text
+
+/* size_t wcslen (const wchar_t *s)
+ Returns length of string s.
+
+ Register usage:
+ -r1=bytes to 4k-byte boundary
+ -r2=s
+ -r3=tmp
+ -r4=tmp
+ -r5=current_len and return_value
+ -v16=part of s
+*/
+ENTRY(__wcslen_vx)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+
+ vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */
+ lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */
+
+ tmll %r2,3 /* Test if s is 4-byte aligned? */
+ jne .Lfallback /* And use common-code variant if not. */
+
+ vfenezf %v16,%v16,%v16 /* Find element not equal with zero search. */
+ vlgvb %r4,%v16,7 /* Load zero index or 16 if not found. */
+ clr %r4,%r1 /* If found zero within loaded bytes? */
+ locgrl %r2,%r4 /* Then copy return value. */
+ jl .Lend /* And return. */
+
+ /* Align s to 16 byte. */
+ risbgn %r3,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */
+ lghi %r5,16 /* current_len = 16. */
+ slr %r5,%r3 /* Compute bytes to 16bytes boundary. */
+
+ /* Find zero in 16byte aligned loop. */
+.Lloop:
+ vl %v16,0(%r5,%r2) /* Load s. */
+ vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */
+ je .Lfound /* Jump away if zero was found. */
+ vl %v16,16(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Lfound16
+ vl %v16,32(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Lfound32
+ vl %v16,48(%r5,%r2)
+ vfenezfs %v16,%v16,%v16
+ je .Lfound48
+
+ aghi %r5,64
+ j .Lloop /* No zero found -> loop. */
+
+.Lfound48:
+ aghi %r5,16
+.Lfound32:
+ aghi %r5,16
+.Lfound16:
+ aghi %r5,16
+.Lfound:
+ vlgvb %r2,%v16,7 /* Load byte index of zero. */
+ algr %r2,%r5
+.Lend:
+ srlg %r2,%r2,2 /* Convert byte-count to character-count. */
+ br %r14
+.Lfallback:
+ jg __wcslen_c
+END(__wcslen_vx)
+#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
diff --git a/sysdeps/s390/multiarch/wcslen.c b/sysdeps/s390/multiarch/wcslen.c
new file mode 100644
index 0000000..750bd6b
--- /dev/null
+++ b/sysdeps/s390/multiarch/wcslen.c
@@ -0,0 +1,28 @@
+/* Multiple versions of wcslen.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+# include <wchar.h>
+# include <ifunc-resolve.h>
+
+s390_vx_libc_ifunc (__wcslen)
+weak_alias (__wcslen, wcslen)
+
+#else
+# include <wcsmbs/wcslen.c>
+#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=fd484e057dd4d2813182df08584a4c48d6f1dd7a
commit fd484e057dd4d2813182df08584a4c48d6f1dd7a
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:19 2015 +0200
S390: Ifunc resolver macro for vector instructions.
This patch introduces a s390 specific ifunc resolver macro for 32/64bit,
which chooses <func>_vx with vector instructions if HWCAP_S390_VX flag
in hwcaps is set or <func>_c if not.
ChangeLog:
* sysdeps/s390/multiarch/ifunc-resolve.h (s390_vx_libc_ifunc,
s390_vx_libc_ifunc2): New macro function.
diff --git a/ChangeLog b/ChangeLog
index 19d089a..9304378 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,10 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/multiarch/ifunc-resolve.h (s390_vx_libc_ifunc,
+ s390_vx_libc_ifunc2): New macro function.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* config.h.in (HAVE_S390_VX_ASM_SUPPORT): New macro undefine.
* sysdeps/s390/configure.ac: Add test for S390 vector instruction
assembler support.
diff --git a/sysdeps/s390/multiarch/ifunc-resolve.h b/sysdeps/s390/multiarch/ifunc-resolve.h
index 491df68..e9fd90e 100644
--- a/sysdeps/s390/multiarch/ifunc-resolve.h
+++ b/sysdeps/s390/multiarch/ifunc-resolve.h
@@ -73,3 +73,22 @@
else \
return &__##FUNC##_default; \
}
+
+#define s390_vx_libc_ifunc(FUNC) \
+ s390_vx_libc_ifunc2(FUNC, FUNC)
+
+#define s390_vx_libc_ifunc2(RESOLVERFUNC, FUNC) \
+ /* Make the declarations of the optimized functions hidden in order
+ to prevent GOT slots being generated for them. */ \
+ extern __typeof (FUNC) RESOLVERFUNC##_vx attribute_hidden; \
+ extern __typeof (FUNC) RESOLVERFUNC##_c attribute_hidden; \
+ extern void *__resolve_##RESOLVERFUNC (unsigned long int) __asm__ (#FUNC); \
+ \
+ void *__resolve_##RESOLVERFUNC (unsigned long int dl_hwcap) \
+ { \
+ if (dl_hwcap & HWCAP_S390_VX) \
+ return &RESOLVERFUNC##_vx; \
+ else \
+ return &RESOLVERFUNC##_c; \
+ } \
+ __asm__ (".type " #FUNC ", %gnu_indirect_function");
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=4f0a1cea34c05fb2acc16f1a2d291f53230eb4fb
commit 4f0a1cea34c05fb2acc16f1a2d291f53230eb4fb
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:19 2015 +0200
S390: configure check for vector instruction support in assembler.
The S390 specific test checks if the assembler has support for the new z13
vector instructions by compiling a vector instruction. The .machine and
.machinemode directives are needed to compile the vector instruction without
-march=z13 option on 31/64 bit.
On success the macro HAVE_S390_VX_ASM_SUPPORT is defined. This macro is used
to determine if the optimized functions can be build without compile errors.
If the used assembler lacks vector support, then a warning is dumped while
configuring and only the common code functions are build.
The z13 instruction support was introduced in
"[Committed] S/390: Add support for IBM z13."
(https://sourceware.org/ml/binutils/2015-01/msg00197.html)
ChangeLog:
* config.h.in (HAVE_S390_VX_ASM_SUPPORT): New macro undefine.
* sysdeps/s390/configure.ac: Add test for S390 vector instruction
assembler support.
* sysdeps/s390/configure: Regenerated.
diff --git a/ChangeLog b/ChangeLog
index c648db6..19d089a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,12 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * config.h.in (HAVE_S390_VX_ASM_SUPPORT): New macro undefine.
+ * sysdeps/s390/configure.ac: Add test for S390 vector instruction
+ assembler support.
+ * sysdeps/s390/configure: Regenerated.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/dl-procinfo.c (_dl_s390_cap_flags): Add z13.
* sysdeps/s390/dl-procinfo.h (_DL_PLATFORMS_COUNT): Increased.
diff --git a/config.h.in b/config.h.in
index 2b9aa1a..8d44b8a 100644
--- a/config.h.in
+++ b/config.h.in
@@ -91,6 +91,9 @@
/* Define if assembler supports AVX512. */
#undef HAVE_AVX512_ASM_SUPPORT
+/* Define if assembler supports vector instructions on S390. */
+#undef HAVE_S390_VX_ASM_SUPPORT
+
/* Define if assembler supports Intel MPX. */
#undef HAVE_MPX_SUPPORT
diff --git a/sysdeps/s390/configure b/sysdeps/s390/configure
index be6397e..0fa54c3 100644
--- a/sysdeps/s390/configure
+++ b/sysdeps/s390/configure
@@ -104,5 +104,46 @@ if test "$enable_lock_elision" = yes && test "$libc_cv_gcc_builtin_tbegin" = no
critic_missing="$critic_missing The used GCC has no support for __builtin_tbegin, which is needed for lock-elision on target S390."
fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for S390 vector instruction support" >&5
+$as_echo_n "checking for S390 vector instruction support... " >&6; }
+if ${libc_cv_asm_s390_vx+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ cat > conftest.c <<\EOF
+void testvecinsn ()
+{
+ __asm__ (".machine \"z13\" \n\t"
+ ".machinemode \"zarch_nohighgprs\" \n\t"
+ "vistrbs %%v16,%%v17 \n\t"
+ "locghie %%r1,0" : :);
+}
+EOF
+if { ac_try='${CC-cc} --shared conftest.c -o conftest.o &> /dev/null'
+ { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5
+ (eval $ac_try) 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; } ;
+then
+ libc_cv_asm_s390_vx=yes
+else
+ libc_cv_asm_s390_vx=no
+fi
+rm -f conftest*
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_asm_s390_vx" >&5
+$as_echo "$libc_cv_asm_s390_vx" >&6; }
+
+if test "$libc_cv_asm_s390_vx" = yes ;
+then
+ $as_echo "#define HAVE_S390_VX_ASM_SUPPORT 1" >>confdefs.h
+
+else
+ { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Use binutils with vector-support in order to use optimized implementations." >&5
+$as_echo "$as_me: WARNING: Use binutils with vector-support in order to use optimized implementations." >&2;}
+fi
+
+
test -n "$critic_missing" && as_fn_error $? "
*** $critic_missing" "$LINENO" 5
diff --git a/sysdeps/s390/configure.ac b/sysdeps/s390/configure.ac
index 493e9a4..4da134e 100644
--- a/sysdeps/s390/configure.ac
+++ b/sysdeps/s390/configure.ac
@@ -36,5 +36,34 @@ if test "$enable_lock_elision" = yes && test "$libc_cv_gcc_builtin_tbegin" = no
critic_missing="$critic_missing The used GCC has no support for __builtin_tbegin, which is needed for lock-elision on target S390."
fi
+
+AC_CACHE_CHECK(for S390 vector instruction support, libc_cv_asm_s390_vx, [dnl
+cat > conftest.c <<\EOF
+void testvecinsn ()
+{
+ __asm__ (".machine \"z13\" \n\t"
+ ".machinemode \"zarch_nohighgprs\" \n\t"
+ "vistrbs %%v16,%%v17 \n\t"
+ "locghie %%r1,0" : :);
+}
+EOF
+dnl
+dnl test, if assembler supports S390 vector instructions
+if AC_TRY_COMMAND([${CC-cc} --shared conftest.c -o conftest.o &> /dev/null]) ;
+then
+ libc_cv_asm_s390_vx=yes
+else
+ libc_cv_asm_s390_vx=no
+fi
+rm -f conftest* ])
+
+if test "$libc_cv_asm_s390_vx" = yes ;
+then
+ AC_DEFINE(HAVE_S390_VX_ASM_SUPPORT)
+else
+ AC_MSG_WARN([Use binutils with vector-support in order to use optimized implementations.])
+fi
+
+
test -n "$critic_missing" && AC_MSG_ERROR([
*** $critic_missing])
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=a1b0488fc9df3d895a2e5eefbcd348d3f7fe0e52
commit a1b0488fc9df3d895a2e5eefbcd348d3f7fe0e52
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:19 2015 +0200
S390: Add new s390 platform.
The new IBM z13 is added to platform string array.
The macro _DL_PLATFORMS_COUNT is incremented to 8,
because it was not incremented by commit
"S/390: Sync AUXV capabilities and archs with kernel".
ChangeLog:
* sysdeps/s390/dl-procinfo.c (_dl_s390_cap_flags): Add z13.
* sysdeps/s390/dl-procinfo.h (_DL_PLATFORMS_COUNT): Increased.
diff --git a/ChangeLog b/ChangeLog
index 08c2fa7..c648db6 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,10 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/dl-procinfo.c (_dl_s390_cap_flags): Add z13.
+ * sysdeps/s390/dl-procinfo.h (_DL_PLATFORMS_COUNT): Increased.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/dl-procinfo.c (_dl_s390_platforms): Add vector flag.
* sysdeps/s390/dl-procinfo.h: Add vector capability.
* sysdeps/unix/sysv/linux/s390/bits/hwcap.h (HWCAP_S390_VX): Define.
diff --git a/sysdeps/s390/dl-procinfo.c b/sysdeps/s390/dl-procinfo.c
index 0f57f03..2bb1f21 100644
--- a/sysdeps/s390/dl-procinfo.c
+++ b/sysdeps/s390/dl-procinfo.c
@@ -62,11 +62,11 @@ PROCINFO_CLASS const char _dl_s390_cap_flags[12][9]
#if !defined PROCINFO_DECL && defined SHARED
._dl_s390_platforms
#else
-PROCINFO_CLASS const char _dl_s390_platforms[7][7]
+PROCINFO_CLASS const char _dl_s390_platforms[8][7]
#endif
#ifndef PROCINFO_DECL
= {
- "g5", "z900", "z990", "z9-109", "z10", "z196", "zEC12"
+ "g5", "z900", "z990", "z9-109", "z10", "z196", "zEC12", "z13"
}
#endif
#if !defined SHARED || defined PROCINFO_DECL
diff --git a/sysdeps/s390/dl-procinfo.h b/sysdeps/s390/dl-procinfo.h
index 151b891..78da65f 100644
--- a/sysdeps/s390/dl-procinfo.h
+++ b/sysdeps/s390/dl-procinfo.h
@@ -23,7 +23,7 @@
#define _DL_HWCAP_COUNT 12
-#define _DL_PLATFORMS_COUNT 5
+#define _DL_PLATFORMS_COUNT 8
/* The kernel provides up to 32 capability bits with elf_hwcap. */
#define _DL_FIRST_PLATFORM 32
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=4e28fa80886c71e6aaf85016b82ce981c0f12e6d
commit 4e28fa80886c71e6aaf85016b82ce981c0f12e6d
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:19 2015 +0200
S390: Add hwcaps value for vector facility.
The HWCAP_S390_VX flag in hwcap field of auxiliary vector indicates
if the vector facility is available and the kernel is aware of it.
This can be tested with LD_SHOW_AUXV=1 <prog>.
Currently it does not show te, because it was not incremented
by commit "S/390: Add hwcap value for transactional execution.".
Thus _DL_HWCAP_COUNT is incremented by two.
ChangeLog:
* sysdeps/s390/dl-procinfo.c (_dl_s390_platforms): Add vector flag.
* sysdeps/s390/dl-procinfo.h: Add vector capability.
* sysdeps/unix/sysv/linux/s390/bits/hwcap.h (HWCAP_S390_VX): Define.
diff --git a/ChangeLog b/ChangeLog
index 12d9595..08c2fa7 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,11 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/dl-procinfo.c (_dl_s390_platforms): Add vector flag.
+ * sysdeps/s390/dl-procinfo.h: Add vector capability.
+ * sysdeps/unix/sysv/linux/s390/bits/hwcap.h (HWCAP_S390_VX): Define.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
* sysdeps/s390/s390-32/multiarch/Makefile (sysdep_routines):
Remove ifunc-resolve, add memset-s390, memcpy-s390, memcmp-s390.
* sysdeps/s390/s390-32/multiarch/ifunc-resolve.c: Delete File.
diff --git a/sysdeps/s390/dl-procinfo.c b/sysdeps/s390/dl-procinfo.c
index 96106f1..0f57f03 100644
--- a/sysdeps/s390/dl-procinfo.c
+++ b/sysdeps/s390/dl-procinfo.c
@@ -46,11 +46,11 @@
#if !defined PROCINFO_DECL && defined SHARED
._dl_s390_cap_flags
#else
-PROCINFO_CLASS const char _dl_s390_cap_flags[11][9]
+PROCINFO_CLASS const char _dl_s390_cap_flags[12][9]
#endif
#ifndef PROCINFO_DECL
= {
- "esan3", "zarch", "stfle", "msa", "ldisp", "eimm", "dfp", "edat", "etf3eh", "highgprs", "te"
+ "esan3", "zarch", "stfle", "msa", "ldisp", "eimm", "dfp", "edat", "etf3eh", "highgprs", "te", "vx"
}
#endif
#if !defined SHARED || defined PROCINFO_DECL
diff --git a/sysdeps/s390/dl-procinfo.h b/sysdeps/s390/dl-procinfo.h
index 0f728ab..151b891 100644
--- a/sysdeps/s390/dl-procinfo.h
+++ b/sysdeps/s390/dl-procinfo.h
@@ -21,7 +21,7 @@
#define _DL_PROCINFO_H 1
#include <ldsodefs.h>
-#define _DL_HWCAP_COUNT 10
+#define _DL_HWCAP_COUNT 12
#define _DL_PLATFORMS_COUNT 5
@@ -50,6 +50,7 @@ enum
HWCAP_S390_ETF3EH = 1 << 8,
HWCAP_S390_HIGH_GPRS = 1 << 9,
HWCAP_S390_TE = 1 << 10,
+ HWCAP_S390_VX = 1 << 11,
};
#define HWCAP_IMPORTANT (HWCAP_S390_ZARCH | HWCAP_S390_LDISP \
diff --git a/sysdeps/unix/sysv/linux/s390/bits/hwcap.h b/sysdeps/unix/sysv/linux/s390/bits/hwcap.h
index 35d47be..986d4a7 100644
--- a/sysdeps/unix/sysv/linux/s390/bits/hwcap.h
+++ b/sysdeps/unix/sysv/linux/s390/bits/hwcap.h
@@ -35,3 +35,4 @@
#define HWCAP_S390_ETF3EH 256
#define HWCAP_S390_HIGH_GPRS 512
#define HWCAP_S390_TE 1024
+#define HWCAP_S390_VX 2048
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=31556246c3ac168a2dfec8f9036d913765bbb73d
commit 31556246c3ac168a2dfec8f9036d913765bbb73d
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:18 2015 +0200
S390: Refactor ifunc implementations and enable ifunc-test-framework.
On s390 all ifunc resolvers were implemented in multiarch/ifunc-resolve.c.
The resulting single object files has undefined references to all ifunc-functions.
This patch introduces one multiarch/<func>.c file for each of memcpy, memcmp
and memset with the function specific ifunc resolver. The different function
implementations are now implemented in multiarch/<func>-s390x.S
(moved from multiarch/<func>.S).
The new multiarch/ifunc-resolve.h file contains the ifunc-resolver macro
and other helper-macros. They are merged and are now used in common for
32/64bit. Therefore the __<func>_g5/__<func>_z900 functions were renamed to
__<func>_default.
This patch also enables testing the ifunc implementations by implementing
the function __libc_ifunc_impl_list. It uses the helper-macros of ifunc-resolve.h.
ChangeLog:
* sysdeps/s390/s390-32/multiarch/Makefile (sysdep_routines):
Remove ifunc-resolve, add memset-s390, memcpy-s390, memcmp-s390.
* sysdeps/s390/s390-32/multiarch/ifunc-resolve.c: Delete File.
* sysdeps/s390/s390-32/multiarch/memcmp.S: Move to ...
* sysdeps/s390/s390-32/multiarch/memcmp-s390.S: ... here.
(memcmp, bcmp): Use __memcmp_default as alias source.
* sysdeps/s390/s390-32/multiarch/memcmp.c: New File.
* sysdeps/s390/s390-32/memcmp.S (__memcmp_g5):
Rename to __memcmp_default.
* sysdeps/s390/s390-32/multiarch/memcpy.S: Move to ...
* sysdeps/s390/s390-32/multiarch/memcpy-s390.S: ... here.
(memcpy): Use __memcpy_default as alias source.
* sysdeps/s390/s390-32/multiarch/memcpy.c: New File.
* sysdeps/s390/s390-32/memcpy.S (__memcpy_g5):
Rename to __memcpy_default.
* sysdeps/s390/s390-32/multiarch/memset.S: Move to ...
* sysdeps/s390/s390-32/multiarch/memset-s390.S: ... here.
(memset): Use __memset_default as alias source.
* sysdeps/s390/s390-32/multiarch/memset.c: New File.
* sysdeps/s390/s390-32/memset.S (__memset_g5):
Rename to __memset_default.
* sysdeps/s390/s390-64/multiarch/Makefile (sysdep_routines):
Remove ifunc-resolve, add memset-s390x, memcpy-s390x, memcmp-s390x.
* sysdeps/s390/s390-64/multiarch/ifunc-resolve.c: Delete File.
* sysdeps/s390/s390-64/multiarch/memcmp.S: Move to ...
* sysdeps/s390/s390-64/multiarch/memcmp-s390x.S: ... here.
(memcmp, bcmp): Use __memcmp_default as alias source.
* sysdeps/s390/s390-64/multiarch/memcmp.c: New File.
* sysdeps/s390/s390-64/memcmp.S (__memcmp_z900):
Rename to __memcmp_default.
* sysdeps/s390/s390-64/multiarch/memcpy.S: Move to ...
* sysdeps/s390/s390-64/multiarch/memcpy-s390x.S: ... here.
(memcpy): Use __memcpy_default as alias source.
* sysdeps/s390/s390-64/multiarch/memcpy.c: New File.
* sysdeps/s390/s390-64/memcpy.S (__memcpy_z900):
Rename to __memcpy_default.
* sysdeps/s390/s390-64/multiarch/memset.S: Move to ...
* sysdeps/s390/s390-64/multiarch/memset-s390x.S: ... here.
(memset): Use __memset_default as alias source.
* sysdeps/s390/s390-64/multiarch/memset.c: New File.
* sysdeps/s390/s390-64/memset.S (__memset_z900):
Rename to __memset_default.
* sysdeps/s390/multiarch/ifunc-resolve.h: New File.
* sysdeps/s390/multiarch/ifunc-impl-list.c: New File.
diff --git a/ChangeLog b/ChangeLog
index ba251a1..12d9595 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,52 @@
2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+ * sysdeps/s390/s390-32/multiarch/Makefile (sysdep_routines):
+ Remove ifunc-resolve, add memset-s390, memcpy-s390, memcmp-s390.
+ * sysdeps/s390/s390-32/multiarch/ifunc-resolve.c: Delete File.
+ * sysdeps/s390/s390-32/multiarch/memcmp.S: Move to ...
+ * sysdeps/s390/s390-32/multiarch/memcmp-s390.S: ... here.
+ (memcmp, bcmp): Use __memcmp_default as alias source.
+ * sysdeps/s390/s390-32/multiarch/memcmp.c: New File.
+ * sysdeps/s390/s390-32/memcmp.S (__memcmp_g5):
+ Rename to __memcmp_default.
+ * sysdeps/s390/s390-32/multiarch/memcpy.S: Move to ...
+ * sysdeps/s390/s390-32/multiarch/memcpy-s390.S: ... here.
+ (memcpy): Use __memcpy_default as alias source.
+ * sysdeps/s390/s390-32/multiarch/memcpy.c: New File.
+ * sysdeps/s390/s390-32/memcpy.S (__memcpy_g5):
+ Rename to __memcpy_default.
+ * sysdeps/s390/s390-32/multiarch/memset.S: Move to ...
+ * sysdeps/s390/s390-32/multiarch/memset-s390.S: ... here.
+ (memset): Use __memset_default as alias source.
+ * sysdeps/s390/s390-32/multiarch/memset.c: New File.
+ * sysdeps/s390/s390-32/memset.S (__memset_g5):
+ Rename to __memset_default.
+ * sysdeps/s390/s390-64/multiarch/Makefile (sysdep_routines):
+ Remove ifunc-resolve, add memset-s390x, memcpy-s390x, memcmp-s390x.
+ * sysdeps/s390/s390-64/multiarch/ifunc-resolve.c: Delete File.
+ * sysdeps/s390/s390-64/multiarch/memcmp.S: Move to ...
+ * sysdeps/s390/s390-64/multiarch/memcmp-s390x.S: ... here.
+ (memcmp, bcmp): Use __memcmp_default as alias source.
+ * sysdeps/s390/s390-64/multiarch/memcmp.c: New File.
+ * sysdeps/s390/s390-64/memcmp.S (__memcmp_z900):
+ Rename to __memcmp_default.
+ * sysdeps/s390/s390-64/multiarch/memcpy.S: Move to ...
+ * sysdeps/s390/s390-64/multiarch/memcpy-s390x.S: ... here.
+ (memcpy): Use __memcpy_default as alias source.
+ * sysdeps/s390/s390-64/multiarch/memcpy.c: New File.
+ * sysdeps/s390/s390-64/memcpy.S (__memcpy_z900):
+ Rename to __memcpy_default.
+ * sysdeps/s390/s390-64/multiarch/memset.S: Move to ...
+ * sysdeps/s390/s390-64/multiarch/memset-s390x.S: ... here.
+ (memset): Use __memset_default as alias source.
+ * sysdeps/s390/s390-64/multiarch/memset.c: New File.
+ * sysdeps/s390/s390-64/memset.S (__memset_z900):
+ Rename to __memset_default.
+ * sysdeps/s390/multiarch/ifunc-resolve.h: New File.
+ * sysdeps/s390/multiarch/ifunc-impl-list.c: New File.
+
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
[BZ #18610]
* sysdeps/s390/fpu/bits/fenv.h (fenv_t): Rename
__ieee_instruction_pointer to __unused.
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
new file mode 100644
index 0000000..c330904
--- /dev/null
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -0,0 +1,74 @@
+/* Enumerate available IFUNC implementations of a function. s390/s390x version.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <assert.h>
+#include <string.h>
+#include <ifunc-impl-list.h>
+#include <ifunc-resolve.h>
+
+/* Maximum number of IFUNC implementations. */
+#define MAX_IFUNC 3
+
+/* Fill ARRAY of MAX elements with IFUNC implementations for function
+ NAME supported on target machine and return the number of valid
+ entries. */
+size_t
+__libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
+ size_t max)
+{
+ assert (max >= MAX_IFUNC);
+
+ size_t i = 0;
+
+ /* Get hardware information. */
+ unsigned long int dl_hwcap = GLRO (dl_hwcap);
+ unsigned long long stfle_bits = 0ULL;
+ if ((dl_hwcap & HWCAP_S390_STFLE)
+ && (dl_hwcap & HWCAP_S390_ZARCH)
+ && (dl_hwcap & HWCAP_S390_HIGH_GPRS))
+ {
+ S390_STORE_STFLE (stfle_bits);
+ }
+
+ IFUNC_IMPL (i, name, memset,
+ IFUNC_IMPL_ADD (array, i, memset,
+ S390_IS_Z196 (stfle_bits), __memset_z196)
+ IFUNC_IMPL_ADD (array, i, memset,
+ S390_IS_Z10 (stfle_bits), __memset_z10)
+ IFUNC_IMPL_ADD (array, i, memset, 1, __memset_default))
+
+ IFUNC_IMPL (i, name, memcmp,
+ IFUNC_IMPL_ADD (array, i, memcmp,
+ S390_IS_Z196 (stfle_bits), __memcmp_z196)
+ IFUNC_IMPL_ADD (array, i, memcmp,
+ S390_IS_Z10 (stfle_bits), __memcmp_z10)
+ IFUNC_IMPL_ADD (array, i, memcmp, 1, __memcmp_default))
+
+#ifdef SHARED
+
+ IFUNC_IMPL (i, name, memcpy,
+ IFUNC_IMPL_ADD (array, i, memcpy,
+ S390_IS_Z196 (stfle_bits), __memcpy_z196)
+ IFUNC_IMPL_ADD (array, i, memcpy,
+ S390_IS_Z10 (stfle_bits), __memcpy_z10)
+ IFUNC_IMPL_ADD (array, i, memcpy, 1, __memcpy_default))
+
+#endif /* SHARED */
+
+ return i;
+}
diff --git a/sysdeps/s390/s390-32/multiarch/ifunc-resolve.c b/sysdeps/s390/multiarch/ifunc-resolve.h
similarity index 59%
rename from sysdeps/s390/s390-32/multiarch/ifunc-resolve.c
rename to sysdeps/s390/multiarch/ifunc-resolve.h
index 8e0cdd5..491df68 100644
--- a/sysdeps/s390/s390-32/multiarch/ifunc-resolve.c
+++ b/sysdeps/s390/multiarch/ifunc-resolve.h
@@ -1,6 +1,6 @@
/* IFUNC resolver function for CPU specific functions.
- 32 bit S/390 version.
- Copyright (C) 2012-2015 Free Software Foundation, Inc.
+ 32/64 bit S/390 version.
+ Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -20,12 +20,28 @@
#include <unistd.h>
#include <dl-procinfo.h>
-#define STFLE_BITS_Z10 34 /* General instructions extension */
-#define STFLE_BITS_Z196 45 /* Distinct operands, pop ... */
+#define S390_STFLE_BITS_Z10 34 /* General instructions extension */
+#define S390_STFLE_BITS_Z196 45 /* Distinct operands, pop ... */
-#if IS_IN (libc)
+#define S390_IS_Z196(STFLE_BITS) \
+ ((STFLE_BITS & (1ULL << (63 - S390_STFLE_BITS_Z196))) != 0)
-#define IFUNC_RESOLVE(FUNC) \
+#define S390_IS_Z10(STFLE_BITS) \
+ ((STFLE_BITS & (1ULL << (63 - S390_STFLE_BITS_Z10))) != 0)
+
+#define S390_STORE_STFLE(STFLE_BITS) \
+ /* We want just 1 double word to be returned. */ \
+ register unsigned long reg0 asm("0") = 0; \
+ \
+ asm volatile(".machine push" "\n\t" \
+ ".machine \"z9-109\"" "\n\t" \
+ ".machinemode \"zarch_nohighgprs\"\n\t" \
+ "stfle %0" "\n\t" \
+ ".machine pop" "\n" \
+ : "=QS" (STFLE_BITS), "+d" (reg0) \
+ : : "cc");
+
+#define s390_libc_ifunc(FUNC) \
asm (".globl " #FUNC "\n\t" \
".type " #FUNC ",@gnu_indirect_function\n\t" \
".set " #FUNC ",__resolve_" #FUNC "\n\t" \
@@ -36,7 +52,7 @@
to prevent GOT slots being generated for them. */ \
extern void *__##FUNC##_z196 attribute_hidden; \
extern void *__##FUNC##_z10 attribute_hidden; \
- extern void *__##FUNC##_g5 attribute_hidden; \
+ extern void *__##FUNC##_default attribute_hidden; \
\
void *__resolve_##FUNC (unsigned long int dl_hwcap) \
{ \
@@ -44,29 +60,16 @@
&& (dl_hwcap & HWCAP_S390_ZARCH) \
&& (dl_hwcap & HWCAP_S390_HIGH_GPRS)) \
{ \
- /* We want just 1 double word to be returned. */ \
- register unsigned long reg0 asm("0") = 0; \
unsigned long long stfle_bits; \
+ S390_STORE_STFLE (stfle_bits); \
\
- asm volatile(".insn s,0xb2b00000,%0" "\n\t" /* stfle */ \
- : "=QS" (stfle_bits), "+d" (reg0) \
- : : "cc"); \
- \
- if ((stfle_bits & (1ULL << (63 - STFLE_BITS_Z196))) != 0) \
+ if (S390_IS_Z196 (stfle_bits)) \
return &__##FUNC##_z196; \
- else if ((stfle_bits & (1ULL << (63 - STFLE_BITS_Z10))) != 0) \
+ else if (S390_IS_Z10 (stfle_bits)) \
return &__##FUNC##_z10; \
+ else \
+ return &__##FUNC##_default; \
} \
- return &__##FUNC##_g5; \
+ else \
+ return &__##FUNC##_default; \
}
-
-IFUNC_RESOLVE(memset)
-IFUNC_RESOLVE(memcmp)
-asm(".weak bcmp ; bcmp = memcmp");
-
-/* In the static lib memcpy is needed before the reloc is resolved. */
-#ifdef SHARED
-IFUNC_RESOLVE(memcpy)
-#endif
-
-#endif
diff --git a/sysdeps/s390/s390-32/memcmp.S b/sysdeps/s390/s390-32/memcmp.S
index 93f06d5..33c2e8c 100644
--- a/sysdeps/s390/s390-32/memcmp.S
+++ b/sysdeps/s390/s390-32/memcmp.S
@@ -27,38 +27,38 @@
.text
#ifdef USE_MULTIARCH
-ENTRY(__memcmp_g5)
+ENTRY(__memcmp_default)
#else
ENTRY(memcmp)
#endif
.machine "g5"
- basr %r5,0
+ basr %r5,0
.L_G5_16:
- ltr %r4,%r4
- je .L_G5_4
- ahi %r4,-1
- lr %r1,%r4
- srl %r1,8
- ltr %r1,%r1
- jne .L_G5_12
- ex %r4,.L_G5_17-.L_G5_16(%r5)
+ ltr %r4,%r4
+ je .L_G5_4
+ ahi %r4,-1
+ lr %r1,%r4
+ srl %r1,8
+ ltr %r1,%r1
+ jne .L_G5_12
+ ex %r4,.L_G5_17-.L_G5_16(%r5)
.L_G5_4:
- ipm %r2
- sll %r2,2
- sra %r2,30
- br %r14
+ ipm %r2
+ sll %r2,2
+ sra %r2,30
+ br %r14
.L_G5_12:
- clc 0(256,%r3),0(%r2)
- jne .L_G5_4
- la %r3,256(%r3)
- la %r2,256(%r2)
- brct %r1,.L_G5_12
- ex %r4,.L_G5_17-.L_G5_16(%r5)
- j .L_G5_4
+ clc 0(256,%r3),0(%r2)
+ jne .L_G5_4
+ la %r3,256(%r3)
+ la %r2,256(%r2)
+ brct %r1,.L_G5_12
+ ex %r4,.L_G5_17-.L_G5_16(%r5)
+ j .L_G5_4
.L_G5_17:
- clc 0(1,%r3),0(%r2)
+ clc 0(1,%r3),0(%r2)
#ifdef USE_MULTIARCH
-END(__memcmp_g5)
+END(__memcmp_default)
#else
END(memcmp)
libc_hidden_builtin_def (memcmp)
diff --git a/sysdeps/s390/s390-32/memcpy.S b/sysdeps/s390/s390-32/memcpy.S
index f26fd00..e2a1d98 100644
--- a/sysdeps/s390/s390-32/memcpy.S
+++ b/sysdeps/s390/s390-32/memcpy.S
@@ -26,41 +26,41 @@
%r4 = number of bytes to copy. */
#ifdef USE_MULTIARCH
-ENTRY(__memcpy_g5)
+ENTRY(__memcpy_default)
#else
ENTRY(memcpy)
#endif
.machine "g5"
- st %r13,52(%r15)
- .cfi_offset 13, -44
- basr %r13,0
+ st %r13,52(%r15)
+ .cfi_offset 13, -44
+ basr %r13,0
.L_G5_16:
- ltr %r4,%r4
- je .L_G5_4
- ahi %r4,-1
- lr %r5,%r4
- srl %r5,8
- ltr %r5,%r5
- lr %r1,%r2
- jne .L_G5_12
- ex %r4,.L_G5_17-.L_G5_16(%r13)
+ ltr %r4,%r4
+ je .L_G5_4
+ ahi %r4,-1
+ lr %r5,%r4
+ srl %r5,8
+ ltr %r5,%r5
+ lr %r1,%r2
+ jne .L_G5_12
+ ex %r4,.L_G5_17-.L_G5_16(%r13)
.L_G5_4:
- l %r13,52(%r15)
- br %r14
+ l %r13,52(%r15)
+ br %r14
.L_G5_13:
- chi %r5,4096 # Switch to mvcle for copies >1MB
- jh __memcpy_mvcle
+ chi %r5,4096 # Switch to mvcle for copies >1MB
+ jh __memcpy_mvcle
.L_G5_12:
- mvc 0(256,%r1),0(%r3)
- la %r1,256(%r1)
- la %r3,256(%r3)
- brct %r5,.L_G5_12
- ex %r4,.L_G5_17-.L_G5_16(%r13)
- j .L_G5_4
+ mvc 0(256,%r1),0(%r3)
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ brct %r5,.L_G5_12
+ ex %r4,.L_G5_17-.L_G5_16(%r13)
+ j .L_G5_4
.L_G5_17:
- mvc 0(1,%r1),0(%r3)
+ mvc 0(1,%r1),0(%r3)
#ifdef USE_MULTIARCH
-END(__memcpy_g5)
+END(__memcpy_default)
#else
END(memcpy)
libc_hidden_builtin_def (memcpy)
diff --git a/sysdeps/s390/s390-32/memset.S b/sysdeps/s390/s390-32/memset.S
index 0abba61..adc1233 100644
--- a/sysdeps/s390/s390-32/memset.S
+++ b/sysdeps/s390/s390-32/memset.S
@@ -28,37 +28,37 @@
.text
#ifdef USE_MULTIARCH
-ENTRY(__memset_g5)
+ENTRY(__memset_default)
#else
ENTRY(memset)
#endif
.machine "g5"
- basr %r5,0
+ basr %r5,0
.L_G5_19:
- ltr %r4,%r4
- je .L_G5_4
- stc %r3,0(%r2)
- chi %r4,1
- lr %r1,%r2
- je .L_G5_4
- ahi %r4,-2
- lr %r3,%r4
- srl %r3,8
- ltr %r3,%r3
- jne .L_G5_14
- ex %r4,.L_G5_20-.L_G5_19(%r5)
+ ltr %r4,%r4
+ je .L_G5_4
+ stc %r3,0(%r2)
+ chi %r4,1
+ lr %r1,%r2
+ je .L_G5_4
+ ahi %r4,-2
+ lr %r3,%r4
+ srl %r3,8
+ ltr %r3,%r3
+ jne .L_G5_14
+ ex %r4,.L_G5_20-.L_G5_19(%r5)
.L_G5_4:
- br %r14
+ br %r14
.L_G5_14:
- mvc 1(256,%r1),0(%r1)
- la %r1,256(%r1)
- brct %r3,.L_G5_14
- ex %r4,.L_G5_20-.L_G5_19(%r5)
- j .L_G5_4
+ mvc 1(256,%r1),0(%r1)
+ la %r1,256(%r1)
+ brct %r3,.L_G5_14
+ ex %r4,.L_G5_20-.L_G5_19(%r5)
+ j .L_G5_4
.L_G5_20:
- mvc 1(1,%r1),0(%r1)
+ mvc 1(1,%r1),0(%r1)
#ifdef USE_MULTIARCH
-END(__memset_g5)
+END(__memset_default)
#else
END(memset)
libc_hidden_builtin_def (memset)
diff --git a/sysdeps/s390/s390-32/multiarch/Makefile b/sysdeps/s390/s390-32/multiarch/Makefile
index 9baeecd..f8aee14 100644
--- a/sysdeps/s390/s390-32/multiarch/Makefile
+++ b/sysdeps/s390/s390-32/multiarch/Makefile
@@ -1,3 +1,4 @@
ifeq ($(subdir),string)
-sysdep_routines += ifunc-resolve memset memcpy memcmp
+sysdep_routines += memset memset-s390 memcpy memcpy-s390 \
+ memcmp memcmp-s390
endif
diff --git a/sysdeps/s390/s390-32/multiarch/memcmp.S b/sysdeps/s390/s390-32/multiarch/memcmp-s390.S
similarity index 54%
rename from sysdeps/s390/s390-32/multiarch/memcmp.S
rename to sysdeps/s390/s390-32/multiarch/memcmp-s390.S
index 584dc99..8540f67 100644
--- a/sysdeps/s390/s390-32/multiarch/memcmp.S
+++ b/sysdeps/s390/s390-32/multiarch/memcmp-s390.S
@@ -32,73 +32,73 @@
ENTRY(__memcmp_z196)
.machine "z196"
.machinemode "zarch_nohighgprs"
- ltr %r4,%r4
- je .L_Z196_4
- ahi %r4,-1
- srlk %r1,%r4,8
- ltr %r1,%r1
- jne .L_Z196_2
+ ltr %r4,%r4
+ je .L_Z196_4
+ ahi %r4,-1
+ srlk %r1,%r4,8
+ ltr %r1,%r1
+ jne .L_Z196_2
.L_Z196_3:
- exrl %r4,.L_Z196_14
+ exrl %r4,.L_Z196_14
.L_Z196_4:
- ipm %r2
- sll %r2,2
- sra %r2,30
- br %r14
+ ipm %r2
+ sll %r2,2
+ sra %r2,30
+ br %r14
.L_Z196_17:
- la %r3,256(%r3)
- la %r2,256(%r2)
- ahi %r1,-1
- je .L_Z196_3
+ la %r3,256(%r3)
+ la %r2,256(%r2)
+ ahi %r1,-1
+ je .L_Z196_3
.L_Z196_2:
- pfd 1,512(%r3)
- pfd 1,512(%r2)
- clc 0(256,%r3),0(%r2)
- je .L_Z196_17
- ipm %r2
- sll %r2,2
- sra %r2,30
- br %r14
+ pfd 1,512(%r3)
+ pfd 1,512(%r2)
+ clc 0(256,%r3),0(%r2)
+ je .L_Z196_17
+ ipm %r2
+ sll %r2,2
+ sra %r2,30
+ br %r14
.L_Z196_14:
- clc 0(1,%r3),0(%r2)
+ clc 0(1,%r3),0(%r2)
END(__memcmp_z196)
ENTRY(__memcmp_z10)
.machine "z10"
.machinemode "zarch_nohighgprs"
- ltr %r4,%r4
- je .L_Z10_4
- ahi %r4,-1
- lr %r1,%r4
- srl %r1,8
- cijlh %r1,0,.L_Z10_12
+ ltr %r4,%r4
+ je .L_Z10_4
+ ahi %r4,-1
+ lr %r1,%r4
+ srl %r1,8
+ cijlh %r1,0,.L_Z10_12
.L_Z10_3:
- exrl %r4,.L_Z10_15
+ exrl %r4,.L_Z10_15
.L_Z10_4:
- ipm %r2
- sll %r2,2
- sra %r2,30
- br %r14
+ ipm %r2
+ sll %r2,2
+ sra %r2,30
+ br %r14
.L_Z10_12:
- pfd 1,512(%r3)
- pfd 1,512(%r2)
- clc 0(256,%r3),0(%r2)
- jne .L_Z10_4
- la %r3,256(%r3)
- la %r2,256(%r2)
- brct %r1,.L_Z10_12
- j .L_Z10_3
+ pfd 1,512(%r3)
+ pfd 1,512(%r2)
+ clc 0(256,%r3),0(%r2)
+ jne .L_Z10_4
+ la %r3,256(%r3)
+ la %r2,256(%r2)
+ brct %r1,.L_Z10_12
+ j .L_Z10_3
.L_Z10_15:
- clc 0(1,%r3),0(%r2)
+ clc 0(1,%r3),0(%r2)
END(__memcmp_z10)
-#endif
+#endif /* IS_IN (libc) */
#include "../memcmp.S"
#if !IS_IN (libc)
.globl memcmp
-.set memcmp,__memcmp_g5
+.set memcmp,__memcmp_default
.weak bcmp
-.set bcmp,__memcmp_g5
+.set bcmp,__memcmp_default
#endif
diff --git a/sysdeps/s390/s390-32/multiarch/memcmp.c b/sysdeps/s390/s390-32/multiarch/memcmp.c
new file mode 100644
index 0000000..a3607e4
--- /dev/null
+++ b/sysdeps/s390/s390-32/multiarch/memcmp.c
@@ -0,0 +1,24 @@
+/* Multiple versions of memcmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if IS_IN (libc)
+# include <ifunc-resolve.h>
+
+s390_libc_ifunc (memcmp)
+asm(".weak bcmp ; bcmp = memcmp");
+#endif
diff --git a/sysdeps/s390/s390-32/multiarch/memcpy.S b/sysdeps/s390/s390-32/multiarch/memcpy-s390.S
similarity index 55%
rename from sysdeps/s390/s390-32/multiarch/memcpy.S
rename to sysdeps/s390/s390-32/multiarch/memcpy-s390.S
index 51f4fcf..6be319f 100644
--- a/sysdeps/s390/s390-32/multiarch/memcpy.S
+++ b/sysdeps/s390/s390-32/multiarch/memcpy-s390.S
@@ -32,67 +32,67 @@
ENTRY(__memcpy_z196)
.machine "z196"
.machinemode "zarch_nohighgprs"
- llgfr %r4,%r4
- ltgr %r4,%r4
- je .L_Z196_4
- aghi %r4,-1
- lr %r1,%r2
- srlg %r5,%r4,8
- ltgr %r5,%r5
- jne .L_Z196_5
+ llgfr %r4,%r4
+ ltgr %r4,%r4
+ je .L_Z196_4
+ aghi %r4,-1
+ lr %r1,%r2
+ srlg %r5,%r4,8
+ ltgr %r5,%r5
+ jne .L_Z196_5
.L_Z196_3:
- exrl %r4,.L_Z196_14
+ exrl %r4,.L_Z196_14
.L_Z196_4:
- br %r14
+ br %r14
.L_Z196_5:
- cgfi %r5,262144 # Switch to mvcle for copies >64MB
- jh __memcpy_mvcle
+ cgfi %r5,262144 # Switch to mvcle for copies >64MB
+ jh __memcpy_mvcle
.L_Z196_2:
- pfd 1,768(%r3)
- pfd 2,768(%r1)
- mvc 0(256,%r1),0(%r3)
- aghi %r5,-1
- la %r1,256(%r1)
- la %r3,256(%r3)
- jne .L_Z196_2
- j .L_Z196_3
+ pfd 1,768(%r3)
+ pfd 2,768(%r1)
+ mvc 0(256,%r1),0(%r3)
+ aghi %r5,-1
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ jne .L_Z196_2
+ j .L_Z196_3
.L_Z196_14:
- mvc 0(1,%r1),0(%r3)
+ mvc 0(1,%r1),0(%r3)
END(__memcpy_z196)
ENTRY(__memcpy_z10)
.machine "z10"
.machinemode "zarch_nohighgprs"
- llgfr %r4,%r4
- cgije %r4,0,.L_Z10_4
- aghi %r4,-1
- lr %r1,%r2
- srlg %r5,%r4,8
- cgijlh %r5,0,.L_Z10_13
+ llgfr %r4,%r4
+ cgije %r4,0,.L_Z10_4
+ aghi %r4,-1
+ lr %r1,%r2
+ srlg %r5,%r4,8
+ cgijlh %r5,0,.L_Z10_13
.L_Z10_3:
- exrl %r4,.L_Z10_15
+ exrl %r4,.L_Z10_15
.L_Z10_4:
- br %r14
+ br %r14
.L_Z10_13:
- cgfi %r5,65535 # Switch to mvcle for copies >16MB
- jh __memcpy_mvcle
+ cgfi %r5,65535 # Switch to mvcle for copies >16MB
+ jh __memcpy_mvcle
.L_Z10_12:
- pfd 1,768(%r3)
- pfd 2,768(%r1)
- mvc 0(256,%r1),0(%r3)
- la %r1,256(%r1)
- la %r3,256(%r3)
- brctg %r5,.L_Z10_12
- j .L_Z10_3
+ pfd 1,768(%r3)
+ pfd 2,768(%r1)
+ mvc 0(256,%r1),0(%r3)
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ brctg %r5,.L_Z10_12
+ j .L_Z10_3
.L_Z10_15:
- mvc 0(1,%r1),0(%r3)
+ mvc 0(1,%r1),0(%r3)
END(__memcpy_z10)
-#endif
+#endif /* SHARED && IS_IN (libc) */
#include "../memcpy.S"
#if !defined SHARED || !IS_IN (libc)
.globl memcpy
-.set memcpy,__memcpy_g5
+.set memcpy,__memcpy_default
#endif
diff --git a/sysdeps/s390/s390-32/multiarch/memcpy.c b/sysdeps/s390/s390-32/multiarch/memcpy.c
new file mode 100644
index 0000000..cf2068c
--- /dev/null
+++ b/sysdeps/s390/s390-32/multiarch/memcpy.c
@@ -0,0 +1,24 @@
+/* Multiple versions of memcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* In the static lib memcpy is needed before the reloc is resolved. */
+#if defined SHARED && IS_IN (libc)
+# include <ifunc-resolve.h>
+
+s390_libc_ifunc (memcpy)
+#endif
diff --git a/sysdeps/s390/s390-32/multiarch/memset.S b/sysdeps/s390/s390-32/multiarch/memset-s390.S
similarity index 67%
rename from sysdeps/s390/s390-32/multiarch/memset.S
rename to sysdeps/s390/s390-32/multiarch/memset-s390.S
index 1a7b45f..36e77ee 100644
--- a/sysdeps/s390/s390-32/multiarch/memset.S
+++ b/sysdeps/s390/s390-32/multiarch/memset-s390.S
@@ -32,61 +32,61 @@
ENTRY(__memset_z196)
.machine "z196"
.machinemode "zarch_nohighgprs"
- llgfr %r4,%r4
- ltgr %r4,%r4
- je .L_Z196_4
- stc %r3,0(%r2)
- lr %r1,%r2
- cghi %r4,1
- je .L_Z196_4
- aghi %r4,-2
- srlg %r5,%r4,8
- ltgr %r5,%r5
- jne .L_Z196_1
+ llgfr %r4,%r4
+ ltgr %r4,%r4
+ je .L_Z196_4
+ stc %r3,0(%r2)
+ lr %r1,%r2
+ cghi %r4,1
+ je .L_Z196_4
+ aghi %r4,-2
+ srlg %r5,%r4,8
+ ltgr %r5,%r5
+ jne .L_Z196_1
.L_Z196_3:
- exrl %r4,.L_Z196_17
+ exrl %r4,.L_Z196_17
.L_Z196_4:
- br %r14
+ br %r14
.L_Z196_1:
cgfi %r5,1048576
jh __memset_mvcle # Switch to mvcle for >256MB
.L_Z196_2:
- pfd 2,1024(%r1)
- mvc 1(256,%r1),0(%r1)
- aghi %r5,-1
- la %r1,256(%r1)
- jne .L_Z196_2
- j .L_Z196_3
+ pfd 2,1024(%r1)
+ mvc 1(256,%r1),0(%r1)
+ aghi %r5,-1
+ la %r1,256(%r1)
+ jne .L_Z196_2
+ j .L_Z196_3
.L_Z196_17:
- mvc 1(1,%r1),0(%r1)
+ mvc 1(1,%r1),0(%r1)
END(__memset_z196)
ENTRY(__memset_z10)
.machine "z10"
.machinemode "zarch_nohighgprs"
- llgfr %r4,%r4
- cgije %r4,0,.L_Z10_4
- stc %r3,0(%r2)
- lr %r1,%r2
- cgije %r4,1,.L_Z10_4
- aghi %r4,-2
- srlg %r5,%r4,8
- cgijlh %r5,0,.L_Z10_15
+ llgfr %r4,%r4
+ cgije %r4,0,.L_Z10_4
+ stc %r3,0(%r2)
+ lr %r1,%r2
+ cgije %r4,1,.L_Z10_4
+ aghi %r4,-2
+ srlg %r5,%r4,8
+ cgijlh %r5,0,.L_Z10_15
.L_Z10_3:
- exrl %r4,.L_Z10_18
+ exrl %r4,.L_Z10_18
.L_Z10_4:
- br %r14
+ br %r14
.L_Z10_15:
cgfi %r5,163840 # Switch to mvcle for >40MB
jh __memset_mvcle
.L_Z10_14:
- pfd 2,1024(%r1)
- mvc 1(256,%r1),0(%r1)
- la %r1,256(%r1)
- brctg %r5,.L_Z10_14
- j .L_Z10_3
+ pfd 2,1024(%r1)
+ mvc 1(256,%r1),0(%r1)
+ la %r1,256(%r1)
+ brctg %r5,.L_Z10_14
+ j .L_Z10_3
.L_Z10_18:
- mvc 1(1,%r1),0(%r1)
+ mvc 1(1,%r1),0(%r1)
END(__memset_z10)
ENTRY(__memset_mvcle)
@@ -103,11 +103,11 @@ ENTRY(__memset_mvcle)
br %r14
END(__memset_mvcle)
-#endif
+#endif /* IS_IN (libc) */
#include "../memset.S"
#if !IS_IN (libc)
.globl memset
-.set memset,__memset_g5
+.set memset,__memset_default
#endif
diff --git a/sysdeps/s390/s390-32/multiarch/memset.c b/sysdeps/s390/s390-32/multiarch/memset.c
new file mode 100644
index 0000000..ae0568b
--- /dev/null
+++ b/sysdeps/s390/s390-32/multiarch/memset.c
@@ -0,0 +1,23 @@
+/* Multiple versions of memset.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if IS_IN (libc)
+# include <ifunc-resolve.h>
+
+s390_libc_ifunc (memset)
+#endif
diff --git a/sysdeps/s390/s390-64/memcmp.S b/sysdeps/s390/s390-64/memcmp.S
index 6767438..1ab3fb1 100644
--- a/sysdeps/s390/s390-64/memcmp.S
+++ b/sysdeps/s390/s390-64/memcmp.S
@@ -27,36 +27,36 @@
.text
#ifdef USE_MULTIARCH
-ENTRY(__memcmp_z900)
+ENTRY(__memcmp_default)
#else
ENTRY(memcmp)
#endif
.machine "z900"
- ltgr %r4,%r4
- je .L_Z900_4
- aghi %r4,-1
- srlg %r1,%r4,8
- ltgr %r1,%r1
- jne .L_Z900_12
+ ltgr %r4,%r4
+ je .L_Z900_4
+ aghi %r4,-1
+ srlg %r1,%r4,8
+ ltgr %r1,%r1
+ jne .L_Z900_12
.L_Z900_3:
- larl %r1,.L_Z900_15
- ex %r4,0(%r1)
+ larl %r1,.L_Z900_15
+ ex %r4,0(%r1)
.L_Z900_4:
- ipm %r2
- sllg %r2,%r2,34
- srag %r2,%r2,62
- br %r14
+ ipm %r2
+ sllg %r2,%r2,34
+ srag %r2,%r2,62
+ br %r14
.L_Z900_12:
- clc 0(256,%r3),0(%r2)
- jne .L_Z900_4
- la %r3,256(%r3)
- la %r2,256(%r2)
- brctg %r1,.L_Z900_12
- j .L_Z900_3
+ clc 0(256,%r3),0(%r2)
+ jne .L_Z900_4
+ la %r3,256(%r3)
+ la %r2,256(%r2)
+ brctg %r1,.L_Z900_12
+ j .L_Z900_3
.L_Z900_15:
- clc 0(1,%r3),0(%r2)
+ clc 0(1,%r3),0(%r2)
#ifdef USE_MULTIARCH
-END(__memcmp_z900)
+END(__memcmp_default)
#else
END(memcmp)
libc_hidden_builtin_def (memcmp)
diff --git a/sysdeps/s390/s390-64/memcpy.S b/sysdeps/s390/s390-64/memcpy.S
index 3f122dc..98a1b11 100644
--- a/sysdeps/s390/s390-64/memcpy.S
+++ b/sysdeps/s390/s390-64/memcpy.S
@@ -29,37 +29,37 @@
.text
#ifdef USE_MULTIARCH
-ENTRY(__memcpy_z900)
+ENTRY(__memcpy_default)
#else
ENTRY(memcpy)
#endif
.machine "z900"
- ltgr %r4,%r4
- je .L_Z900_4
- aghi %r4,-1
- srlg %r5,%r4,8
- ltgr %r5,%r5
- lgr %r1,%r2
- jne .L_Z900_13
+ ltgr %r4,%r4
+ je .L_Z900_4
+ aghi %r4,-1
+ srlg %r5,%r4,8
+ ltgr %r5,%r5
+ lgr %r1,%r2
+ jne .L_Z900_13
.L_Z900_3:
- larl %r5,.L_Z900_15
- ex %r4,0(%r5)
+ larl %r5,.L_Z900_15
+ ex %r4,0(%r5)
.L_Z900_4:
- br %r14
+ br %r14
.L_Z900_13:
- chi %r5,4096 # Switch to mvcle for copies >1MB
- jh __memcpy_mvcle
+ chi %r5,4096 # Switch to mvcle for copies >1MB
+ jh __memcpy_mvcle
.L_Z900_12:
- mvc 0(256,%r1),0(%r3)
- la %r1,256(%r1)
- la %r3,256(%r3)
- brctg %r5,.L_Z900_12
- j .L_Z900_3
+ mvc 0(256,%r1),0(%r3)
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ brctg %r5,.L_Z900_12
+ j .L_Z900_3
.L_Z900_15:
- mvc 0(1,%r1),0(%r3)
+ mvc 0(1,%r1),0(%r3)
#ifdef USE_MULTIARCH
-END(__memcpy_z900)
+END(__memcpy_default)
#else
END(memcpy)
libc_hidden_builtin_def (memcpy)
diff --git a/sysdeps/s390/s390-64/memset.S b/sysdeps/s390/s390-64/memset.S
index 1e307d7..e16e44a 100644
--- a/sysdeps/s390/s390-64/memset.S
+++ b/sysdeps/s390/s390-64/memset.S
@@ -29,35 +29,35 @@
.text
#ifdef USE_MULTIARCH
-ENTRY(__memset_z900)
+ENTRY(__memset_default)
#else
ENTRY(memset)
#endif
.machine "z900"
- ltgr %r4,%r4
- je .L_Z900_4
- stc %r3,0(%r2)
- cghi %r4,1
- lgr %r1,%r2
- je .L_Z900_4
- aghi %r4,-2
- srlg %r3,%r4,8
- ltgr %r3,%r3
- jne .L_Z900_14
+ ltgr %r4,%r4
+ je .L_Z900_4
+ stc %r3,0(%r2)
+ cghi %r4,1
+ lgr %r1,%r2
+ je .L_Z900_4
+ aghi %r4,-2
+ srlg %r3,%r4,8
+ ltgr %r3,%r3
+ jne .L_Z900_14
.L_Z900_3:
- larl %r3,.L_Z900_18
- ex %r4,0(%r3)
+ larl %r3,.L_Z900_18
+ ex %r4,0(%r3)
.L_Z900_4:
- br %r14
+ br %r14
.L_Z900_14:
- mvc 1(256,%r1),0(%r1)
- la %r1,256(%r1)
- brctg %r3,.L_Z900_14
- j .L_Z900_3
+ mvc 1(256,%r1),0(%r1)
+ la %r1,256(%r1)
+ brctg %r3,.L_Z900_14
+ j .L_Z900_3
.L_Z900_18:
- mvc 1(1,%r1),0(%r1)
+ mvc 1(1,%r1),0(%r1)
#ifdef USE_MULTIARCH
-END(__memset_z900)
+END(__memset_default)
#else
END(memset)
libc_hidden_builtin_def (memset)
diff --git a/sysdeps/s390/s390-64/multiarch/Makefile b/sysdeps/s390/s390-64/multiarch/Makefile
index 9baeecd..91053b5 100644
--- a/sysdeps/s390/s390-64/multiarch/Makefile
+++ b/sysdeps/s390/s390-64/multiarch/Makefile
@@ -1,3 +1,4 @@
ifeq ($(subdir),string)
-sysdep_routines += ifunc-resolve memset memcpy memcmp
+sysdep_routines += memset memset-s390x memcpy memcpy-s390x \
+ memcmp memcmp-s390x
endif
diff --git a/sysdeps/s390/s390-64/multiarch/ifunc-resolve.c b/sysdeps/s390/s390-64/multiarch/ifunc-resolve.c
deleted file mode 100644
index b303304..0000000
--- a/sysdeps/s390/s390-64/multiarch/ifunc-resolve.c
+++ /dev/null
@@ -1,76 +0,0 @@
-/* IFUNC resolver function for CPU specific functions.
- 64 bit S/390 version.
- Copyright (C) 2012-2015 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-#include <unistd.h>
-#include <dl-procinfo.h>
-
-#define STFLE_BITS_Z10 34 /* General instructions extension */
-#define STFLE_BITS_Z196 45 /* Distinct operands, pop ... */
-
-#if IS_IN (libc)
-
-#define IFUNC_RESOLVE(FUNC) \
- asm (".globl " #FUNC "\n\t" \
- ".type " #FUNC ",@gnu_indirect_function\n\t" \
- ".set " #FUNC ",__resolve_" #FUNC "\n\t" \
- ".globl __GI_" #FUNC "\n\t" \
- ".set __GI_" #FUNC "," #FUNC "\n"); \
- \
- /* Make the declarations of the optimized functions hidden in order
- to prevent GOT slots being generated for them. */ \
- extern void *__##FUNC##_z196 attribute_hidden; \
- extern void *__##FUNC##_z10 attribute_hidden; \
- extern void *__##FUNC##_z900 attribute_hidden; \
- \
- void *__resolve_##FUNC (unsigned long int dl_hwcap) \
- { \
- if (dl_hwcap & HWCAP_S390_STFLE) \
- { \
- /* We want just 1 double word to be returned. */ \
- register unsigned long reg0 asm("0") = 0; \
- unsigned long stfle_bits; \
- \
- asm volatile(".machine push" "\n\t" \
- ".machine \"z9-109\"" "\n\t" \
- "stfle %0" "\n\t" \
- ".machine pop" "\n" \
- : "=QS" (stfle_bits), "+d" (reg0) \
- : : "cc"); \
- \
- if ((stfle_bits & (1UL << (63 - STFLE_BITS_Z196))) != 0) \
- return &__##FUNC##_z196; \
- else if ((stfle_bits & (1UL << (63 - STFLE_BITS_Z10))) != 0) \
- return &__##FUNC##_z10; \
- else \
- return &__##FUNC##_z900; \
- } \
- else \
- return &__##FUNC##_z900; \
- }
-
-IFUNC_RESOLVE(memset)
-IFUNC_RESOLVE(memcmp)
-asm(".weak bcmp ; bcmp = memcmp");
-
-/* In the static lib memcpy is needed before the reloc is resolved. */
-#ifdef SHARED
-IFUNC_RESOLVE(memcpy)
-#endif
-
-#endif
diff --git a/sysdeps/s390/s390-64/multiarch/memcmp.S b/sysdeps/s390/s390-64/multiarch/memcmp-s390x.S
similarity index 53%
rename from sysdeps/s390/s390-64/multiarch/memcmp.S
rename to sysdeps/s390/s390-64/multiarch/memcmp-s390x.S
index 049847d..b475ed4 100644
--- a/sysdeps/s390/s390-64/multiarch/memcmp.S
+++ b/sysdeps/s390/s390-64/multiarch/memcmp-s390x.S
@@ -31,71 +31,71 @@
ENTRY(__memcmp_z196)
.machine "z196"
- ltgr %r4,%r4
- je .L_Z196_4
- aghi %r4,-1
- srlg %r1,%r4,8
- ltgr %r1,%r1
- jne .L_Z196_2
+ ltgr %r4,%r4
+ je .L_Z196_4
+ aghi %r4,-1
+ srlg %r1,%r4,8
+ ltgr %r1,%r1
+ jne .L_Z196_2
.L_Z196_3:
- exrl %r4,.L_Z196_14
+ exrl %r4,.L_Z196_14
.L_Z196_4:
- ipm %r2
- sllg %r2,%r2,34
- srag %r2,%r2,62
- br %r14
+ ipm %r2
+ sllg %r2,%r2,34
+ srag %r2,%r2,62
+ br %r14
.L_Z196_17:
- la %r3,256(%r3)
- la %r2,256(%r2)
- aghi %r1,-1
- je .L_Z196_3
+ la %r3,256(%r3)
+ la %r2,256(%r2)
+ aghi %r1,-1
+ je .L_Z196_3
.L_Z196_2:
- pfd 1,512(%r3)
- pfd 1,512(%r2)
- clc 0(256,%r3),0(%r2)
- je .L_Z196_17
- ipm %r2
- sllg %r2,%r2,34
- srag %r2,%r2,62
- br %r14
+ pfd 1,512(%r3)
+ pfd 1,512(%r2)
+ clc 0(256,%r3),0(%r2)
+ je .L_Z196_17
+ ipm %r2
+ sllg %r2,%r2,34
+ srag %r2,%r2,62
+ br %r14
.L_Z196_14:
- clc 0(1,%r3),0(%r2)
+ clc 0(1,%r3),0(%r2)
END(__memcmp_z196)
ENTRY(__memcmp_z10)
.machine "z10"
- ltgr %r4,%r4
- je .L_Z10_4
- aghi %r4,-1
- srlg %r1,%r4,8
- cgijlh %r1,0,.L_Z10_12
+ ltgr %r4,%r4
+ je .L_Z10_4
+ aghi %r4,-1
+ srlg %r1,%r4,8
+ cgijlh %r1,0,.L_Z10_12
.L_Z10_3:
- exrl %r4,.L_Z10_15
+ exrl %r4,.L_Z10_15
.L_Z10_4:
- ipm %r2
- sllg %r2,%r2,34
- srag %r2,%r2,62
- br %r14
+ ipm %r2
+ sllg %r2,%r2,34
+ srag %r2,%r2,62
+ br %r14
.L_Z10_12:
- pfd 1,512(%r3)
- pfd 1,512(%r2)
- clc 0(256,%r3),0(%r2)
- jne .L_Z10_4
- la %r3,256(%r3)
- la %r2,256(%r2)
- brctg %r1,.L_Z10_12
- j .L_Z10_3
+ pfd 1,512(%r3)
+ pfd 1,512(%r2)
+ clc 0(256,%r3),0(%r2)
+ jne .L_Z10_4
+ la %r3,256(%r3)
+ la %r2,256(%r2)
+ brctg %r1,.L_Z10_12
+ j .L_Z10_3
.L_Z10_15:
- clc 0(1,%r3),0(%r2)
+ clc 0(1,%r3),0(%r2)
END(__memcmp_z10)
-#endif
+#endif /* IS_IN (libc) */
#include "../memcmp.S"
#if !IS_IN (libc)
.globl memcmp
-.set memcmp,__memcmp_z900
+.set memcmp,__memcmp_default
.weak bcmp
-.set bcmp,__memcmp_z900
+.set bcmp,__memcmp_default
#endif
diff --git a/sysdeps/s390/s390-64/multiarch/memcmp.c b/sysdeps/s390/s390-64/multiarch/memcmp.c
new file mode 100644
index 0000000..a3607e4
--- /dev/null
+++ b/sysdeps/s390/s390-64/multiarch/memcmp.c
@@ -0,0 +1,24 @@
+/* Multiple versions of memcmp.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if IS_IN (libc)
+# include <ifunc-resolve.h>
+
+s390_libc_ifunc (memcmp)
+asm(".weak bcmp ; bcmp = memcmp");
+#endif
diff --git a/sysdeps/s390/s390-64/multiarch/memcpy.S b/sysdeps/s390/s390-64/multiarch/memcpy-s390x.S
similarity index 54%
rename from sysdeps/s390/s390-64/multiarch/memcpy.S
rename to sysdeps/s390/s390-64/multiarch/memcpy-s390x.S
index fc670c7..4b0e6cb 100644
--- a/sysdeps/s390/s390-64/multiarch/memcpy.S
+++ b/sysdeps/s390/s390-64/multiarch/memcpy-s390x.S
@@ -30,65 +30,65 @@
#if defined SHARED && IS_IN (libc)
ENTRY(__memcpy_z196)
- .machine "z196"
- ltgr %r4,%r4
- je .L_Z196_4
- aghi %r4,-1
- lgr %r1,%r2
- srlg %r5,%r4,8
- ltgr %r5,%r5
- jne .L_Z196_5
+ .machine "z196"
+ ltgr %r4,%r4
+ je .L_Z196_4
+ aghi %r4,-1
+ lgr %r1,%r2
+ srlg %r5,%r4,8
+ ltgr %r5,%r5
+ jne .L_Z196_5
.L_Z196_3:
- exrl %r4,.L_Z196_14
+ exrl %r4,.L_Z196_14
.L_Z196_4:
- br %r14
+ br %r14
.L_Z196_5:
- cgfi %r5,262144 # Switch to mvcle for copies >64MB
- jh __memcpy_mvcle
+ cgfi %r5,262144 # Switch to mvcle for copies >64MB
+ jh __memcpy_mvcle
.L_Z196_2:
- pfd 1,768(%r3)
- pfd 2,768(%r1)
- mvc 0(256,%r1),0(%r3)
- aghi %r5,-1
- la %r1,256(%r1)
- la %r3,256(%r3)
- jne .L_Z196_2
- j .L_Z196_3
+ pfd 1,768(%r3)
+ pfd 2,768(%r1)
+ mvc 0(256,%r1),0(%r3)
+ aghi %r5,-1
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ jne .L_Z196_2
+ j .L_Z196_3
.L_Z196_14:
- mvc 0(1,%r1),0(%r3)
+ mvc 0(1,%r1),0(%r3)
END(__memcpy_z196)
ENTRY(__memcpy_z10)
.machine "z10"
- cgije %r4,0,.L_Z10_4
- aghi %r4,-1
- lgr %r1,%r2
- srlg %r5,%r4,8
- cgijlh %r5,0,.L_Z10_13
+ cgije %r4,0,.L_Z10_4
+ aghi %r4,-1
+ lgr %r1,%r2
+ srlg %r5,%r4,8
+ cgijlh %r5,0,.L_Z10_13
.L_Z10_3:
- exrl %r4,.L_Z10_15
+ exrl %r4,.L_Z10_15
.L_Z10_4:
- br %r14
+ br %r14
.L_Z10_13:
- cgfi %r5,65535 # Switch to mvcle for copies >16MB
- jh __memcpy_mvcle
+ cgfi %r5,65535 # Switch to mvcle for copies >16MB
+ jh __memcpy_mvcle
.L_Z10_12:
- pfd 1,768(%r3)
- pfd 2,768(%r1)
- mvc 0(256,%r1),0(%r3)
- la %r1,256(%r1)
- la %r3,256(%r3)
- brctg %r5,.L_Z10_12
- j .L_Z10_3
+ pfd 1,768(%r3)
+ pfd 2,768(%r1)
+ mvc 0(256,%r1),0(%r3)
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ brctg %r5,.L_Z10_12
+ j .L_Z10_3
.L_Z10_15:
- mvc 0(1,%r1),0(%r3)
+ mvc 0(1,%r1),0(%r3)
END(__memcpy_z10)
-#endif
+#endif /* SHARED && IS_IN (libc) */
#include "../memcpy.S"
#if !defined SHARED || !IS_IN (libc)
.globl memcpy
-.set memcpy,__memcpy_z900
+.set memcpy,__memcpy_default
#endif
diff --git a/sysdeps/s390/s390-64/multiarch/memcpy.c b/sysdeps/s390/s390-64/multiarch/memcpy.c
new file mode 100644
index 0000000..cf2068c
--- /dev/null
+++ b/sysdeps/s390/s390-64/multiarch/memcpy.c
@@ -0,0 +1,24 @@
+/* Multiple versions of memcpy.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* In the static lib memcpy is needed before the reloc is resolved. */
+#if defined SHARED && IS_IN (libc)
+# include <ifunc-resolve.h>
+
+s390_libc_ifunc (memcpy)
+#endif
diff --git a/sysdeps/s390/s390-64/multiarch/memset.S b/sysdeps/s390/s390-64/multiarch/memset-s390x.S
similarity index 66%
rename from sysdeps/s390/s390-64/multiarch/memset.S
rename to sysdeps/s390/s390-64/multiarch/memset-s390x.S
index 3ac110a..241c128 100644
--- a/sysdeps/s390/s390-64/multiarch/memset.S
+++ b/sysdeps/s390/s390-64/multiarch/memset-s390x.S
@@ -31,58 +31,58 @@
ENTRY(__memset_z196)
.machine "z196"
- ltgr %r4,%r4
- je .L_Z196_4
- stc %r3,0(%r2)
- lgr %r1,%r2
- cghi %r4,1
- je .L_Z196_4
- aghi %r4,-2
- srlg %r5,%r4,8
- ltgr %r5,%r5
- jne .L_Z196_1
+ ltgr %r4,%r4
+ je .L_Z196_4
+ stc %r3,0(%r2)
+ lgr %r1,%r2
+ cghi %r4,1
+ je .L_Z196_4
+ aghi %r4,-2
+ srlg %r5,%r4,8
+ ltgr %r5,%r5
+ jne .L_Z196_1
.L_Z196_3:
- exrl %r4,.L_Z196_17
+ exrl %r4,.L_Z196_17
.L_Z196_4:
- br %r14
+ br %r14
.L_Z196_1:
cgfi %r5,1048576
jh __memset_mvcle # Switch to mvcle for >256MB
.L_Z196_2:
- pfd 2,1024(%r1)
- mvc 1(256,%r1),0(%r1)
- aghi %r5,-1
- la %r1,256(%r1)
- jne .L_Z196_2
- j .L_Z196_3
+ pfd 2,1024(%r1)
+ mvc 1(256,%r1),0(%r1)
+ aghi %r5,-1
+ la %r1,256(%r1)
+ jne .L_Z196_2
+ j .L_Z196_3
.L_Z196_17:
- mvc 1(1,%r1),0(%r1)
+ mvc 1(1,%r1),0(%r1)
END(__memset_z196)
ENTRY(__memset_z10)
.machine "z10"
- cgije %r4,0,.L_Z10_4
- stc %r3,0(%r2)
- lgr %r1,%r2
- cgije %r4,1,.L_Z10_4
- aghi %r4,-2
- srlg %r5,%r4,8
- cgijlh %r5,0,.L_Z10_15
+ cgije %r4,0,.L_Z10_4
+ stc %r3,0(%r2)
+ lgr %r1,%r2
+ cgije %r4,1,.L_Z10_4
+ aghi %r4,-2
+ srlg %r5,%r4,8
+ cgijlh %r5,0,.L_Z10_15
.L_Z10_3:
- exrl %r4,.L_Z10_18
+ exrl %r4,.L_Z10_18
.L_Z10_4:
- br %r14
+ br %r14
.L_Z10_15:
cgfi %r5,163840 # Switch to mvcle for >40MB
jh __memset_mvcle
.L_Z10_14:
- pfd 2,1024(%r1)
- mvc 1(256,%r1),0(%r1)
- la %r1,256(%r1)
- brctg %r5,.L_Z10_14
- j .L_Z10_3
+ pfd 2,1024(%r1)
+ mvc 1(256,%r1),0(%r1)
+ la %r1,256(%r1)
+ brctg %r5,.L_Z10_14
+ j .L_Z10_3
.L_Z10_18:
- mvc 1(1,%r1),0(%r1)
+ mvc 1(1,%r1),0(%r1)
END(__memset_z10)
ENTRY(__memset_mvcle)
@@ -99,11 +99,11 @@ ENTRY(__memset_mvcle)
br %r14
END(__memset_mvcle)
-#endif
+#endif /* IS_IN (libc) */
#include "../memset.S"
#if !IS_IN (libc)
.globl memset
-.set memset,__memset_z900
+.set memset,__memset_default
#endif
diff --git a/sysdeps/s390/s390-64/multiarch/memset.c b/sysdeps/s390/s390-64/multiarch/memset.c
new file mode 100644
index 0000000..ae0568b
--- /dev/null
+++ b/sysdeps/s390/s390-64/multiarch/memset.c
@@ -0,0 +1,23 @@
+/* Multiple versions of memset.
+ Copyright (C) 2015 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if IS_IN (libc)
+# include <ifunc-resolve.h>
+
+s390_libc_ifunc (memset)
+#endif
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=5d96fe8c0dc3450bafe6c2aae2dabc76819df3e0
commit 5d96fe8c0dc3450bafe6c2aae2dabc76819df3e0
Author: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Wed Aug 26 10:26:18 2015 +0200
S390: Fix handling of DXC-byte in FPC-register.
On s390, the DXC(data-exception-code)-byte in FPC(floating-point-control)-
register contains a code of the last occured exception.
If bits 6 and 7 of DXC-byte are zero, the bits 0-5 correspond to the
ieee-exception flag bits.
The current implementation always uses these bits as ieee-exception flag bits.
fetestexcept() reports any exception after the first usage of a
vector-instruction in a process, because it raises an "vector instruction
exception" with DXC-code 0xFE.
This patch fixes the handling of the DXC-byte. The DXC-Byte is only handled
if bits 6 and 7 are zero.
The #define _FPU_RESERVED is extended by the DXC-Byte.
Otherwise the tests math/test-fpucw-static and math/test-fpucw-ieee-static
fails, because DXC-Byte contains the vector instruction exception when reaching
main(). This exception was triggered by strrchr() call in __init_misc().
__init_misc() is called after __setfpucw () in __libc_init_first().
The field __ieee_instruction_pointer in struct fenv_t is renamed to __unused
because it is a relict from commit "Remove PTRACE_PEEKUSER"
(87b9b50f0d4b92248905e95a06a13c513dc45e59) and isn´t used anymore.
ChangeLog:
[BZ #18610]
* sysdeps/s390/fpu/bits/fenv.h (fenv_t): Rename
__ieee_instruction_pointer to __unused.
* sysdeps/s390/fpu/fesetenv.c (__fesetenv): Remove usage of
__ieee_instruction_pointer.
* sysdeps/s390/fpu/fclrexcpt.c (feclearexcept): Fix dxc-field handling.
* sysdeps/s390/fpu/fgetexcptflg.c (fegetexceptflag): Likewise.
* sysdeps/s390/fpu/fsetexcptflg.c (fesetexceptflag): Likewise.
* sysdeps/s390/fpu/ftestexcept.c (fetestexcept): Likewise.
* sysdeps/s390/fpu/fpu_control.h (_FPU_RESERVED):
Mark dxc-field as reserved.
diff --git a/ChangeLog b/ChangeLog
index d642fb0..ba251a1 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,17 @@
+2015-08-26 Stefan Liebler <stli@linux.vnet.ibm.com>
+
+ [BZ #18610]
+ * sysdeps/s390/fpu/bits/fenv.h (fenv_t): Rename
+ __ieee_instruction_pointer to __unused.
+ * sysdeps/s390/fpu/fesetenv.c (__fesetenv): Remove usage of
+ __ieee_instruction_pointer.
+ * sysdeps/s390/fpu/fclrexcpt.c (feclearexcept): Fix dxc-field handling.
+ * sysdeps/s390/fpu/fgetexcptflg.c (fegetexceptflag): Likewise.
+ * sysdeps/s390/fpu/fsetexcptflg.c (fesetexceptflag): Likewise.
+ * sysdeps/s390/fpu/ftestexcept.c (fetestexcept): Likewise.
+ * sysdeps/s390/fpu/fpu_control.h (_FPU_RESERVED):
+ Mark dxc-field as reserved.
+
2015-08-25 Roland McGrath <roland@hack.frob.com>
* sysdeps/nacl/start.c (_start): Call __nacl_main instead of main
diff --git a/sysdeps/s390/fpu/bits/fenv.h b/sysdeps/s390/fpu/bits/fenv.h
index 052b2b6..e5b0117 100644
--- a/sysdeps/s390/fpu/bits/fenv.h
+++ b/sysdeps/s390/fpu/bits/fenv.h
@@ -77,8 +77,10 @@ typedef unsigned int fexcept_t; /* size of fpc */
typedef struct
{
fexcept_t __fpc;
- void *__ieee_instruction_pointer;
- /* failing instruction for ieee exceptions */
+ void *__unused;
+ /* The field __unused (formerly __ieee_instruction_pointer) is a relict from
+ commit "Remove PTRACE_PEEKUSER" (87b9b50f0d4b92248905e95a06a13c513dc45e59)
+ and isn´t used anymore. */
} fenv_t;
/* If the default argument is used we use this value. */
diff --git a/sysdeps/s390/fpu/fclrexcpt.c b/sysdeps/s390/fpu/fclrexcpt.c
index dd13808..7f42084 100644
--- a/sysdeps/s390/fpu/fclrexcpt.c
+++ b/sysdeps/s390/fpu/fclrexcpt.c
@@ -29,7 +29,12 @@ feclearexcept (int excepts)
_FPU_GETCW (temp);
/* Clear the relevant bits. */
- temp &= ~((excepts << FPC_DXC_SHIFT)|(excepts << FPC_FLAGS_SHIFT));
+ temp &= ~(excepts << FPC_FLAGS_SHIFT);
+ if ((temp & FPC_NOT_FPU_EXCEPTION) == 0)
+ /* Bits 6, 7 of dxc-byte are zero,
+ thus bits 0-5 of dxc-byte correspond to the flag-bits.
+ Clear the relevant bits in flags and dxc-field. */
+ temp &= ~(excepts << FPC_DXC_SHIFT);
/* Put the new data in effect. */
_FPU_SETCW (temp);
diff --git a/sysdeps/s390/fpu/fesetenv.c b/sysdeps/s390/fpu/fesetenv.c
index 467716a..bcfba5c 100644
--- a/sysdeps/s390/fpu/fesetenv.c
+++ b/sysdeps/s390/fpu/fesetenv.c
@@ -32,12 +32,10 @@ __fesetenv (const fenv_t *envp)
if (envp == FE_DFL_ENV)
{
env.__fpc = _FPU_DEFAULT;
- env.__ieee_instruction_pointer = 0;
}
else if (envp == FE_NOMASK_ENV)
{
env.__fpc = FPC_EXCEPTION_MASK;
- env.__ieee_instruction_pointer = 0;
}
else
env = (*envp);
diff --git a/sysdeps/s390/fpu/fgetexcptflg.c b/sysdeps/s390/fpu/fgetexcptflg.c
index c14dc15..0a5a081 100644
--- a/sysdeps/s390/fpu/fgetexcptflg.c
+++ b/sysdeps/s390/fpu/fgetexcptflg.c
@@ -27,7 +27,13 @@ fegetexceptflag (fexcept_t *flagp, int excepts)
/* Get the current exceptions. */
_FPU_GETCW (temp);
- newexcepts = (excepts << FPC_DXC_SHIFT) | (excepts << FPC_FLAGS_SHIFT);
+ newexcepts = excepts << FPC_FLAGS_SHIFT;
+ if ((temp & FPC_NOT_FPU_EXCEPTION) == 0)
+ /* Bits 6, 7 of dxc-byte are zero,
+ thus bits 0-5 of dxc-byte correspond to the flag-bits.
+ Evaluate flags and last dxc-exception-code. */
+ newexcepts |= excepts << FPC_DXC_SHIFT;
+
*flagp = temp & newexcepts;
/* Success. */
diff --git a/sysdeps/s390/fpu/fpu_control.h b/sysdeps/s390/fpu/fpu_control.h
index 098f0d0..1f663b3 100644
--- a/sysdeps/s390/fpu/fpu_control.h
+++ b/sysdeps/s390/fpu/fpu_control.h
@@ -19,12 +19,12 @@
<http://www.gnu.org/licenses/>. */
#ifndef _FPU_CONTROL_H
-# define _FPU_CONTROL_H
+#define _FPU_CONTROL_H
-# include <features.h>
+#include <features.h>
/* These bits are reserved are not changed. */
-# define _FPU_RESERVED 0x070700FC
+#define _FPU_RESERVED 0x0707FFFC
/* The fdlibm code requires no interrupts for exceptions. Don't
change the rounding mode, it would break long double I/O! */
diff --git a/sysdeps/s390/fpu/fsetexcptflg.c b/sysdeps/s390/fpu/fsetexcptflg.c
index cbe9f34..4ccc249 100644
--- a/sysdeps/s390/fpu/fsetexcptflg.c
+++ b/sysdeps/s390/fpu/fsetexcptflg.c
@@ -24,16 +24,25 @@
int
fesetexceptflag (const fexcept_t *flagp, int excepts)
{
- fexcept_t temp,newexcepts;
+ fexcept_t temp, newexcepts;
/* Get the current environment. We have to do this since we cannot
separately set the status word. */
_FPU_GETCW (temp);
/* Install the new exception bits in the Accrued Exception Byte. */
excepts = excepts & FE_ALL_EXCEPT;
- newexcepts = (excepts << FPC_DXC_SHIFT) | (excepts << FPC_FLAGS_SHIFT);
+ newexcepts = excepts << FPC_FLAGS_SHIFT;
temp &= ~newexcepts;
- temp |= *flagp & newexcepts;
+ if ((temp & FPC_NOT_FPU_EXCEPTION) == 0)
+ /* Bits 6, 7 of dxc-byte are zero,
+ thus bits 0-5 of dxc-byte correspond to the flag-bits.
+ Clear given exceptions in dxc-field. */
+ temp &= ~(excepts << FPC_DXC_SHIFT);
+
+ /* Integrate dxc-byte of flagp into flags. The dxc-byte of flagp contains
+ either an ieee-exception or 0 (see fegetexceptflag). */
+ temp |= (*flagp | ((*flagp >> FPC_DXC_SHIFT) << FPC_FLAGS_SHIFT))
+ & newexcepts;
/* Store the new status word (along with the rest of the environment.
Possibly new exceptions are set but they won't get executed unless
diff --git a/sysdeps/s390/fpu/ftestexcept.c b/sysdeps/s390/fpu/ftestexcept.c
index 6889632..55b7c4a 100644
--- a/sysdeps/s390/fpu/ftestexcept.c
+++ b/sysdeps/s390/fpu/ftestexcept.c
@@ -23,11 +23,17 @@
int
fetestexcept (int excepts)
{
- fexcept_t temp;
+ fexcept_t temp, res;
/* Get current exceptions. */
_FPU_GETCW (temp);
- temp = (temp >> FPC_DXC_SHIFT) | (temp >> FPC_FLAGS_SHIFT);
- return temp & excepts & FE_ALL_EXCEPT;
+ res = temp >> FPC_FLAGS_SHIFT;
+ if ((temp & FPC_NOT_FPU_EXCEPTION) == 0)
+ /* Bits 6, 7 of dxc-byte are zero,
+ thus bits 0-5 of dxc-byte correspond to the flag-bits.
+ Evaluate flags and last dxc-exception-code. */
+ res |= temp >> FPC_DXC_SHIFT;
+
+ return res & excepts & FE_ALL_EXCEPT;
}
libm_hidden_def (fetestexcept)
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 479 ++++++++++++++++++++++++
NEWS | 9 +-
benchtests/Makefile | 5 +-
benchtests/bench-memchr.c | 69 +++--
benchtests/bench-memset.c | 63 ++-
benchtests/bench-stpcpy.c | 33 ++-
benchtests/bench-stpncpy.c | 41 ++-
benchtests/bench-strcat.c | 88 +++--
benchtests/bench-strchr.c | 24 +-
benchtests/bench-strcspn.c | 45 ++-
benchtests/bench-strncat.c | 97 +++--
benchtests/bench-strncmp.c | 110 +++++--
benchtests/bench-strncpy.c | 110 ++++--
benchtests/bench-strnlen.c | 71 +++--
benchtests/bench-strpbrk.c | 100 ++++--
benchtests/bench-strspn.c | 73 +++--
benchtests/bench-wcpcpy.c | 20 +
benchtests/bench-wcpncpy.c | 20 +
benchtests/bench-wcscat.c | 20 +
benchtests/bench-wcschr.c | 20 +
benchtests/bench-wcschrnul.c | 20 +
benchtests/bench-wcscmp.c | 20 +
benchtests/bench-wcscpy.c | 20 +
benchtests/bench-wcscspn.c | 20 +
benchtests/bench-wcslen.c | 20 +
benchtests/bench-wcsncat.c | 20 +
benchtests/bench-wcsncmp.c | 20 +
benchtests/bench-wcsncpy.c | 20 +
benchtests/bench-wcsnlen.c | 20 +
benchtests/bench-wcspbrk.c | 20 +
benchtests/bench-wcsrchr.c | 20 +
benchtests/bench-wcsspn.c | 20 +
benchtests/bench-wmemchr.c | 20 +
benchtests/bench-wmemcmp.c | 20 +
benchtests/bench-wmemset.c | 20 +
config.h.in | 3 +
string/memccpy.c | 4 +
string/strcat.c | 6 +-
string/strchr.c | 8 +-
string/strcmp.c | 6 +-
string/strlen.c | 6 +-
string/test-memchr.c | 91 +++--
string/test-memset.c | 90 +++--
string/test-stpcpy.c | 35 ++-
string/test-stpncpy.c | 41 ++-
string/test-strcat.c | 126 ++++---
string/test-strchr.c | 26 +-
string/test-strcspn.c | 47 ++-
string/test-strncat.c | 138 +++++---
string/test-strncpy.c | 131 +++++---
string/test-strnlen.c | 87 +++--
string/test-strpbrk.c | 130 ++++---
string/test-strspn.c | 99 +++--
sysdeps/generic/unwind.h | 4 +-
sysdeps/s390/bits/string.h | 4 +-
sysdeps/s390/configure | 41 ++
sysdeps/s390/configure.ac | 29 ++
sysdeps/s390/dl-procinfo.c | 8 +-
sysdeps/s390/dl-procinfo.h | 5 +-
sysdeps/s390/fpu/bits/fenv.h | 6 +-
sysdeps/s390/fpu/fclrexcpt.c | 7 +-
sysdeps/s390/fpu/fesetenv.c | 2 -
sysdeps/s390/fpu/fgetexcptflg.c | 8 +-
sysdeps/s390/fpu/fpu_control.h | 6 +-
sysdeps/s390/fpu/fsetexcptflg.c | 15 +-
sysdeps/s390/fpu/ftestexcept.c | 12 +-
sysdeps/s390/multiarch/Makefile | 44 +++
sysdeps/s390/multiarch/ifunc-impl-list.c | 145 +++++++
sysdeps/s390/multiarch/ifunc-resolve.h | 94 +++++
sysdeps/s390/multiarch/memccpy-c.c | 25 ++
sysdeps/s390/multiarch/memccpy-vx.S | 156 ++++++++
sysdeps/s390/multiarch/memccpy.c | 28 ++
sysdeps/s390/multiarch/memchr-vx.S | 159 ++++++++
sysdeps/s390/multiarch/memchr.c | 24 ++
sysdeps/s390/multiarch/memrchr-c.c | 25 ++
sysdeps/s390/multiarch/memrchr-vx.S | 160 ++++++++
sysdeps/s390/multiarch/memrchr.c | 28 ++
sysdeps/s390/multiarch/rawmemchr-c.c | 34 ++
sysdeps/s390/multiarch/rawmemchr-vx.S | 92 +++++
sysdeps/s390/multiarch/rawmemchr.c | 28 ++
sysdeps/s390/multiarch/stpcpy-c.c | 35 ++
sysdeps/s390/multiarch/stpcpy-vx.S | 104 +++++
sysdeps/s390/multiarch/stpcpy.c | 30 ++
sysdeps/s390/multiarch/stpncpy-c.c | 28 ++
sysdeps/s390/multiarch/stpncpy-vx.S | 200 ++++++++++
sysdeps/s390/multiarch/stpncpy.c | 28 ++
sysdeps/s390/multiarch/strcat-c.c | 28 ++
sysdeps/s390/multiarch/strcat-vx.S | 161 ++++++++
sysdeps/s390/multiarch/strcat.c | 27 ++
sysdeps/s390/multiarch/strchr-c.c | 29 ++
sysdeps/s390/multiarch/strchr-vx.S | 100 +++++
sysdeps/s390/multiarch/strchr.c | 28 ++
sysdeps/s390/multiarch/strchrnul-c.c | 26 ++
sysdeps/s390/multiarch/strchrnul-vx.S | 93 +++++
sysdeps/s390/multiarch/strchrnul.c | 28 ++
sysdeps/s390/multiarch/strcmp-vx.S | 116 ++++++
sysdeps/s390/multiarch/strcmp.c | 26 ++
sysdeps/s390/multiarch/strcpy-vx.S | 109 ++++++
sysdeps/s390/multiarch/strcpy.c | 24 ++
sysdeps/s390/multiarch/strcspn-c.c | 28 ++
sysdeps/s390/multiarch/strcspn-vx.S | 281 ++++++++++++++
sysdeps/s390/multiarch/strcspn.c | 27 ++
sysdeps/s390/multiarch/strlen-c.c | 28 ++
sysdeps/s390/multiarch/strlen-vx.S | 84 ++++
sysdeps/s390/multiarch/strlen.c | 27 ++
sysdeps/s390/multiarch/strncat-c.c | 23 ++
sysdeps/s390/multiarch/strncat-vx.S | 239 ++++++++++++
sysdeps/s390/multiarch/strncat.c | 27 ++
sysdeps/s390/multiarch/strncmp-c.c | 28 ++
sysdeps/s390/multiarch/strncmp-vx.S | 137 +++++++
sysdeps/s390/multiarch/strncmp.c | 30 ++
sysdeps/s390/multiarch/strncpy-vx.S | 207 ++++++++++
sysdeps/s390/multiarch/strncpy.c | 24 ++
sysdeps/s390/multiarch/strnlen-c.c | 30 ++
sysdeps/s390/multiarch/strnlen-vx.S | 134 +++++++
sysdeps/s390/multiarch/strnlen.c | 29 ++
sysdeps/s390/multiarch/strpbrk-c.c | 28 ++
sysdeps/s390/multiarch/strpbrk-vx.S | 302 +++++++++++++++
sysdeps/s390/multiarch/strpbrk.c | 27 ++
sysdeps/s390/multiarch/strrchr-c.c | 29 ++
sysdeps/s390/multiarch/strrchr-vx.S | 180 +++++++++
sysdeps/s390/multiarch/strrchr.c | 28 ++
sysdeps/s390/multiarch/strspn-c.c | 28 ++
sysdeps/s390/multiarch/strspn-vx.S | 256 +++++++++++++
sysdeps/s390/multiarch/strspn.c | 27 ++
sysdeps/s390/multiarch/wcpcpy-c.c | 25 ++
sysdeps/s390/multiarch/wcpcpy-vx.S | 114 ++++++
sysdeps/s390/multiarch/wcpcpy.c | 28 ++
sysdeps/s390/multiarch/wcpncpy-c.c | 25 ++
sysdeps/s390/multiarch/wcpncpy-vx.S | 222 +++++++++++
sysdeps/s390/multiarch/wcpncpy.c | 28 ++
sysdeps/s390/multiarch/wcscat-c.c | 25 ++
sysdeps/s390/multiarch/wcscat-vx.S | 175 +++++++++
sysdeps/s390/multiarch/wcscat.c | 28 ++
sysdeps/s390/multiarch/wcschr-c.c | 37 ++
sysdeps/s390/multiarch/wcschr-vx.S | 103 +++++
sysdeps/s390/multiarch/wcschr.c | 29 ++
sysdeps/s390/multiarch/wcschrnul-c.c | 25 ++
sysdeps/s390/multiarch/wcschrnul-vx.S | 97 +++++
sysdeps/s390/multiarch/wcschrnul.c | 28 ++
sysdeps/s390/multiarch/wcscmp-c.c | 32 ++
sysdeps/s390/multiarch/wcscmp-vx.S | 131 +++++++
sysdeps/s390/multiarch/wcscmp.c | 28 ++
sysdeps/s390/multiarch/wcscpy-c.c | 25 ++
sysdeps/s390/multiarch/wcscpy-vx.S | 111 ++++++
sysdeps/s390/multiarch/wcscpy.c | 27 ++
sysdeps/s390/multiarch/wcscspn-c.c | 26 ++
sysdeps/s390/multiarch/wcscspn-vx.S | 293 +++++++++++++++
sysdeps/s390/multiarch/wcscspn.c | 27 ++
sysdeps/s390/multiarch/wcslen-c.c | 25 ++
sysdeps/s390/multiarch/wcslen-vx.S | 91 +++++
sysdeps/s390/multiarch/wcslen.c | 28 ++
sysdeps/s390/multiarch/wcsncat-c.c | 25 ++
sysdeps/s390/multiarch/wcsncat-vx.S | 265 +++++++++++++
sysdeps/s390/multiarch/wcsncat.c | 27 ++
sysdeps/s390/multiarch/wcsncmp-c.c | 25 ++
sysdeps/s390/multiarch/wcsncmp-vx.S | 177 +++++++++
sysdeps/s390/multiarch/wcsncmp.c | 27 ++
sysdeps/s390/multiarch/wcsncpy-c.c | 25 ++
sysdeps/s390/multiarch/wcsncpy-vx.S | 223 +++++++++++
sysdeps/s390/multiarch/wcsncpy.c | 28 ++
sysdeps/s390/multiarch/wcsnlen-c.c | 25 ++
sysdeps/s390/multiarch/wcsnlen-vx.S | 151 ++++++++
sysdeps/s390/multiarch/wcsnlen.c | 28 ++
sysdeps/s390/multiarch/wcspbrk-c.c | 31 ++
sysdeps/s390/multiarch/wcspbrk-vx.S | 315 ++++++++++++++++
sysdeps/s390/multiarch/wcspbrk.c | 27 ++
sysdeps/s390/multiarch/wcsrchr-c.c | 25 ++
sysdeps/s390/multiarch/wcsrchr-vx.S | 190 ++++++++++
sysdeps/s390/multiarch/wcsrchr.c | 27 ++
sysdeps/s390/multiarch/wcsspn-c.c | 31 ++
sysdeps/s390/multiarch/wcsspn-vx.S | 270 +++++++++++++
sysdeps/s390/multiarch/wcsspn.c | 27 ++
sysdeps/s390/multiarch/wmemchr-c.c | 37 ++
sysdeps/s390/multiarch/wmemchr-vx.S | 166 ++++++++
sysdeps/s390/multiarch/wmemchr.c | 29 ++
sysdeps/s390/multiarch/wmemcmp-c.c | 26 ++
sysdeps/s390/multiarch/wmemcmp-vx.S | 149 ++++++++
sysdeps/s390/multiarch/wmemcmp.c | 27 ++
sysdeps/s390/multiarch/wmemset-c.c | 37 ++
sysdeps/s390/multiarch/wmemset-vx.S | 142 +++++++
sysdeps/s390/multiarch/wmemset.c | 29 ++
sysdeps/s390/s390-32/memcmp.S | 46 ++--
sysdeps/s390/s390-32/memcpy.S | 50 ++--
sysdeps/s390/s390-32/memset.S | 44 ++--
sysdeps/s390/s390-32/multiarch/Makefile | 3 +-
sysdeps/s390/s390-32/multiarch/ifunc-resolve.c | 72 ----
sysdeps/s390/s390-32/multiarch/memchr.c | 21 +
sysdeps/s390/s390-32/multiarch/memcmp-s390.S | 104 +++++
sysdeps/s390/s390-32/multiarch/memcmp.S | 104 -----
sysdeps/s390/s390-32/multiarch/memcmp.c | 24 ++
sysdeps/s390/s390-32/multiarch/memcpy-s390.S | 98 +++++
sysdeps/s390/s390-32/multiarch/memcpy.S | 98 -----
sysdeps/s390/s390-32/multiarch/memcpy.c | 24 ++
sysdeps/s390/s390-32/multiarch/memset-s390.S | 113 ++++++
sysdeps/s390/s390-32/multiarch/memset.S | 113 ------
sysdeps/s390/s390-32/multiarch/memset.c | 23 ++
sysdeps/s390/s390-32/multiarch/strcmp.c | 21 +
sysdeps/s390/s390-32/multiarch/strcpy.c | 21 +
sysdeps/s390/s390-32/multiarch/strncpy.c | 21 +
sysdeps/s390/s390-64/memcmp.S | 42 +-
sysdeps/s390/s390-64/memcpy.S | 40 +-
sysdeps/s390/s390-64/memset.S | 40 +-
sysdeps/s390/s390-64/multiarch/Makefile | 3 +-
sysdeps/s390/s390-64/multiarch/ifunc-resolve.c | 76 ----
sysdeps/s390/s390-64/multiarch/memchr.c | 21 +
sysdeps/s390/s390-64/multiarch/memcmp-s390x.S | 101 +++++
sysdeps/s390/s390-64/multiarch/memcmp.S | 101 -----
sysdeps/s390/s390-64/multiarch/memcmp.c | 24 ++
sysdeps/s390/s390-64/multiarch/memcpy-s390x.S | 94 +++++
sysdeps/s390/s390-64/multiarch/memcpy.S | 94 -----
sysdeps/s390/s390-64/multiarch/memcpy.c | 24 ++
sysdeps/s390/s390-64/multiarch/memset-s390x.S | 109 ++++++
sysdeps/s390/s390-64/multiarch/memset.S | 109 ------
sysdeps/s390/s390-64/multiarch/memset.c | 23 ++
sysdeps/s390/s390-64/multiarch/strcmp.c | 21 +
sysdeps/s390/s390-64/multiarch/strcpy.c | 21 +
sysdeps/s390/s390-64/multiarch/strncpy.c | 21 +
sysdeps/s390/s390-64/utf8-utf16-z9.c | 2 +
sysdeps/unix/sysv/linux/s390/bits/hwcap.h | 1 +
wcsmbs/Makefile | 4 +-
wcsmbs/test-wcpcpy.c | 20 +
wcsmbs/test-wcpncpy.c | 20 +
wcsmbs/test-wcscat.c | 20 +
wcsmbs/test-wcschrnul.c | 20 +
wcsmbs/test-wcscspn.c | 20 +
wcsmbs/test-wcsncat.c | 20 +
wcsmbs/test-wcsncpy.c | 20 +
wcsmbs/test-wcsnlen.c | 20 +
wcsmbs/test-wcspbrk.c | 20 +
wcsmbs/test-wcsspn.c | 20 +
wcsmbs/test-wmemchr.c | 20 +
wcsmbs/test-wmemset.c | 20 +
wcsmbs/wcpcpy.c | 6 +-
wcsmbs/wcpncpy.c | 5 +
wcsmbs/wcscat.c | 5 +
wcsmbs/wcschrnul.c | 5 +
wcsmbs/wcscspn.c | 3 +
wcsmbs/wcsncat.c | 5 +-
wcsmbs/wcsncmp.c | 5 +-
wcsmbs/wcsncpy.c | 5 +
wcsmbs/wcsnlen.c | 7 +-
wcsmbs/wcspbrk.c | 3 +
wcsmbs/wcsspn.c | 3 +
wcsmbs/wmemchr.c | 4 +
wcsmbs/wmemset.c | 3 +
246 files changed, 12677 insertions(+), 1605 deletions(-)
create mode 100644 benchtests/bench-wcpcpy.c
create mode 100644 benchtests/bench-wcpncpy.c
create mode 100644 benchtests/bench-wcscat.c
create mode 100644 benchtests/bench-wcschr.c
create mode 100644 benchtests/bench-wcschrnul.c
create mode 100644 benchtests/bench-wcscmp.c
create mode 100644 benchtests/bench-wcscpy.c
create mode 100644 benchtests/bench-wcscspn.c
create mode 100644 benchtests/bench-wcslen.c
create mode 100644 benchtests/bench-wcsncat.c
create mode 100644 benchtests/bench-wcsncmp.c
create mode 100644 benchtests/bench-wcsncpy.c
create mode 100644 benchtests/bench-wcsnlen.c
create mode 100644 benchtests/bench-wcspbrk.c
create mode 100644 benchtests/bench-wcsrchr.c
create mode 100644 benchtests/bench-wcsspn.c
create mode 100644 benchtests/bench-wmemchr.c
create mode 100644 benchtests/bench-wmemcmp.c
create mode 100644 benchtests/bench-wmemset.c
create mode 100644 sysdeps/s390/multiarch/Makefile
create mode 100644 sysdeps/s390/multiarch/ifunc-impl-list.c
create mode 100644 sysdeps/s390/multiarch/ifunc-resolve.h
create mode 100644 sysdeps/s390/multiarch/memccpy-c.c
create mode 100644 sysdeps/s390/multiarch/memccpy-vx.S
create mode 100644 sysdeps/s390/multiarch/memccpy.c
create mode 100644 sysdeps/s390/multiarch/memchr-vx.S
create mode 100644 sysdeps/s390/multiarch/memchr.c
create mode 100644 sysdeps/s390/multiarch/memrchr-c.c
create mode 100644 sysdeps/s390/multiarch/memrchr-vx.S
create mode 100644 sysdeps/s390/multiarch/memrchr.c
create mode 100644 sysdeps/s390/multiarch/rawmemchr-c.c
create mode 100644 sysdeps/s390/multiarch/rawmemchr-vx.S
create mode 100644 sysdeps/s390/multiarch/rawmemchr.c
create mode 100644 sysdeps/s390/multiarch/stpcpy-c.c
create mode 100644 sysdeps/s390/multiarch/stpcpy-vx.S
create mode 100644 sysdeps/s390/multiarch/stpcpy.c
create mode 100644 sysdeps/s390/multiarch/stpncpy-c.c
create mode 100644 sysdeps/s390/multiarch/stpncpy-vx.S
create mode 100644 sysdeps/s390/multiarch/stpncpy.c
create mode 100644 sysdeps/s390/multiarch/strcat-c.c
create mode 100644 sysdeps/s390/multiarch/strcat-vx.S
create mode 100644 sysdeps/s390/multiarch/strcat.c
create mode 100644 sysdeps/s390/multiarch/strchr-c.c
create mode 100644 sysdeps/s390/multiarch/strchr-vx.S
create mode 100644 sysdeps/s390/multiarch/strchr.c
create mode 100644 sysdeps/s390/multiarch/strchrnul-c.c
create mode 100644 sysdeps/s390/multiarch/strchrnul-vx.S
create mode 100644 sysdeps/s390/multiarch/strchrnul.c
create mode 100644 sysdeps/s390/multiarch/strcmp-vx.S
create mode 100644 sysdeps/s390/multiarch/strcmp.c
create mode 100644 sysdeps/s390/multiarch/strcpy-vx.S
create mode 100644 sysdeps/s390/multiarch/strcpy.c
create mode 100644 sysdeps/s390/multiarch/strcspn-c.c
create mode 100644 sysdeps/s390/multiarch/strcspn-vx.S
create mode 100644 sysdeps/s390/multiarch/strcspn.c
create mode 100644 sysdeps/s390/multiarch/strlen-c.c
create mode 100644 sysdeps/s390/multiarch/strlen-vx.S
create mode 100644 sysdeps/s390/multiarch/strlen.c
create mode 100644 sysdeps/s390/multiarch/strncat-c.c
create mode 100644 sysdeps/s390/multiarch/strncat-vx.S
create mode 100644 sysdeps/s390/multiarch/strncat.c
create mode 100644 sysdeps/s390/multiarch/strncmp-c.c
create mode 100644 sysdeps/s390/multiarch/strncmp-vx.S
create mode 100644 sysdeps/s390/multiarch/strncmp.c
create mode 100644 sysdeps/s390/multiarch/strncpy-vx.S
create mode 100644 sysdeps/s390/multiarch/strncpy.c
create mode 100644 sysdeps/s390/multiarch/strnlen-c.c
create mode 100644 sysdeps/s390/multiarch/strnlen-vx.S
create mode 100644 sysdeps/s390/multiarch/strnlen.c
create mode 100644 sysdeps/s390/multiarch/strpbrk-c.c
create mode 100644 sysdeps/s390/multiarch/strpbrk-vx.S
create mode 100644 sysdeps/s390/multiarch/strpbrk.c
create mode 100644 sysdeps/s390/multiarch/strrchr-c.c
create mode 100644 sysdeps/s390/multiarch/strrchr-vx.S
create mode 100644 sysdeps/s390/multiarch/strrchr.c
create mode 100644 sysdeps/s390/multiarch/strspn-c.c
create mode 100644 sysdeps/s390/multiarch/strspn-vx.S
create mode 100644 sysdeps/s390/multiarch/strspn.c
create mode 100644 sysdeps/s390/multiarch/wcpcpy-c.c
create mode 100644 sysdeps/s390/multiarch/wcpcpy-vx.S
create mode 100644 sysdeps/s390/multiarch/wcpcpy.c
create mode 100644 sysdeps/s390/multiarch/wcpncpy-c.c
create mode 100644 sysdeps/s390/multiarch/wcpncpy-vx.S
create mode 100644 sysdeps/s390/multiarch/wcpncpy.c
create mode 100644 sysdeps/s390/multiarch/wcscat-c.c
create mode 100644 sysdeps/s390/multiarch/wcscat-vx.S
create mode 100644 sysdeps/s390/multiarch/wcscat.c
create mode 100644 sysdeps/s390/multiarch/wcschr-c.c
create mode 100644 sysdeps/s390/multiarch/wcschr-vx.S
create mode 100644 sysdeps/s390/multiarch/wcschr.c
create mode 100644 sysdeps/s390/multiarch/wcschrnul-c.c
create mode 100644 sysdeps/s390/multiarch/wcschrnul-vx.S
create mode 100644 sysdeps/s390/multiarch/wcschrnul.c
create mode 100644 sysdeps/s390/multiarch/wcscmp-c.c
create mode 100644 sysdeps/s390/multiarch/wcscmp-vx.S
create mode 100644 sysdeps/s390/multiarch/wcscmp.c
create mode 100644 sysdeps/s390/multiarch/wcscpy-c.c
create mode 100644 sysdeps/s390/multiarch/wcscpy-vx.S
create mode 100644 sysdeps/s390/multiarch/wcscpy.c
create mode 100644 sysdeps/s390/multiarch/wcscspn-c.c
create mode 100644 sysdeps/s390/multiarch/wcscspn-vx.S
create mode 100644 sysdeps/s390/multiarch/wcscspn.c
create mode 100644 sysdeps/s390/multiarch/wcslen-c.c
create mode 100644 sysdeps/s390/multiarch/wcslen-vx.S
create mode 100644 sysdeps/s390/multiarch/wcslen.c
create mode 100644 sysdeps/s390/multiarch/wcsncat-c.c
create mode 100644 sysdeps/s390/multiarch/wcsncat-vx.S
create mode 100644 sysdeps/s390/multiarch/wcsncat.c
create mode 100644 sysdeps/s390/multiarch/wcsncmp-c.c
create mode 100644 sysdeps/s390/multiarch/wcsncmp-vx.S
create mode 100644 sysdeps/s390/multiarch/wcsncmp.c
create mode 100644 sysdeps/s390/multiarch/wcsncpy-c.c
create mode 100644 sysdeps/s390/multiarch/wcsncpy-vx.S
create mode 100644 sysdeps/s390/multiarch/wcsncpy.c
create mode 100644 sysdeps/s390/multiarch/wcsnlen-c.c
create mode 100644 sysdeps/s390/multiarch/wcsnlen-vx.S
create mode 100644 sysdeps/s390/multiarch/wcsnlen.c
create mode 100644 sysdeps/s390/multiarch/wcspbrk-c.c
create mode 100644 sysdeps/s390/multiarch/wcspbrk-vx.S
create mode 100644 sysdeps/s390/multiarch/wcspbrk.c
create mode 100644 sysdeps/s390/multiarch/wcsrchr-c.c
create mode 100644 sysdeps/s390/multiarch/wcsrchr-vx.S
create mode 100644 sysdeps/s390/multiarch/wcsrchr.c
create mode 100644 sysdeps/s390/multiarch/wcsspn-c.c
create mode 100644 sysdeps/s390/multiarch/wcsspn-vx.S
create mode 100644 sysdeps/s390/multiarch/wcsspn.c
create mode 100644 sysdeps/s390/multiarch/wmemchr-c.c
create mode 100644 sysdeps/s390/multiarch/wmemchr-vx.S
create mode 100644 sysdeps/s390/multiarch/wmemchr.c
create mode 100644 sysdeps/s390/multiarch/wmemcmp-c.c
create mode 100644 sysdeps/s390/multiarch/wmemcmp-vx.S
create mode 100644 sysdeps/s390/multiarch/wmemcmp.c
create mode 100644 sysdeps/s390/multiarch/wmemset-c.c
create mode 100644 sysdeps/s390/multiarch/wmemset-vx.S
create mode 100644 sysdeps/s390/multiarch/wmemset.c
delete mode 100644 sysdeps/s390/s390-32/multiarch/ifunc-resolve.c
create mode 100644 sysdeps/s390/s390-32/multiarch/memchr.c
create mode 100644 sysdeps/s390/s390-32/multiarch/memcmp-s390.S
delete mode 100644 sysdeps/s390/s390-32/multiarch/memcmp.S
create mode 100644 sysdeps/s390/s390-32/multiarch/memcmp.c
create mode 100644 sysdeps/s390/s390-32/multiarch/memcpy-s390.S
delete mode 100644 sysdeps/s390/s390-32/multiarch/memcpy.S
create mode 100644 sysdeps/s390/s390-32/multiarch/memcpy.c
create mode 100644 sysdeps/s390/s390-32/multiarch/memset-s390.S
delete mode 100644 sysdeps/s390/s390-32/multiarch/memset.S
create mode 100644 sysdeps/s390/s390-32/multiarch/memset.c
create mode 100644 sysdeps/s390/s390-32/multiarch/strcmp.c
create mode 100644 sysdeps/s390/s390-32/multiarch/strcpy.c
create mode 100644 sysdeps/s390/s390-32/multiarch/strncpy.c
delete mode 100644 sysdeps/s390/s390-64/multiarch/ifunc-resolve.c
create mode 100644 sysdeps/s390/s390-64/multiarch/memchr.c
create mode 100644 sysdeps/s390/s390-64/multiarch/memcmp-s390x.S
delete mode 100644 sysdeps/s390/s390-64/multiarch/memcmp.S
create mode 100644 sysdeps/s390/s390-64/multiarch/memcmp.c
create mode 100644 sysdeps/s390/s390-64/multiarch/memcpy-s390x.S
delete mode 100644 sysdeps/s390/s390-64/multiarch/memcpy.S
create mode 100644 sysdeps/s390/s390-64/multiarch/memcpy.c
create mode 100644 sysdeps/s390/s390-64/multiarch/memset-s390x.S
delete mode 100644 sysdeps/s390/s390-64/multiarch/memset.S
create mode 100644 sysdeps/s390/s390-64/multiarch/memset.c
create mode 100644 sysdeps/s390/s390-64/multiarch/strcmp.c
create mode 100644 sysdeps/s390/s390-64/multiarch/strcpy.c
create mode 100644 sysdeps/s390/s390-64/multiarch/strncpy.c
create mode 100644 wcsmbs/test-wcpcpy.c
create mode 100644 wcsmbs/test-wcpncpy.c
create mode 100644 wcsmbs/test-wcscat.c
create mode 100644 wcsmbs/test-wcschrnul.c
create mode 100644 wcsmbs/test-wcscspn.c
create mode 100644 wcsmbs/test-wcsncat.c
create mode 100644 wcsmbs/test-wcsncpy.c
create mode 100644 wcsmbs/test-wcsnlen.c
create mode 100644 wcsmbs/test-wcspbrk.c
create mode 100644 wcsmbs/test-wcsspn.c
create mode 100644 wcsmbs/test-wmemchr.c
create mode 100644 wcsmbs/test-wmemset.c
hooks/post-receive
--
GNU C Library master sources