This is the mail archive of the
glibc-cvs@sourceware.org
mailing list for the glibc project.
GNU C Library master sources branch ibm/2.19/master updated. glibc-2.19-47-gfa7cc06
- From: azanella at sourceware dot org
- To: glibc-cvs at sourceware dot org
- Date: 28 Aug 2014 16:17:11 -0000
- Subject: GNU C Library master sources branch ibm/2.19/master updated. glibc-2.19-47-gfa7cc06
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU C Library master sources".
The branch, ibm/2.19/master has been updated
via fa7cc069f4eb29c00ec3a833d73ec4a473b11c8a (commit)
via 3c640c4acb9bc2c2cc7fa77d5ce1254953761dc1 (commit)
via 957afa3407c426969eaaa348981b9648d5191ae2 (commit)
via 8d9513a103bdd202ffa4884bdedc2c3c0dbab210 (commit)
via b00ec143897f076ecbcedc7369b4b74e0c7f6d14 (commit)
via 565e3d6c8230affd7089bf5ebfcebbf72f32a27c (commit)
via 6fae3527af330c32399e3a4cdfac3958fc440eb8 (commit)
via 5d55f9b05ecb85b7a543f641829479cfb081f380 (commit)
via dde00e9914370ddd90c9bbc4f3f0e455efae4b47 (commit)
via 9841a0850ed3be4310ec6b49c3349e39a6f0f481 (commit)
via ded8852b37f673b8e66163b44f70504dc5af0985 (commit)
via 7e986751f5c05f3363c01c717972f87a681da0d0 (commit)
from 2289a56644fc05786e2d5637c76d47afea7d38b9 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=fa7cc069f4eb29c00ec3a833d73ec4a473b11c8a
commit fa7cc069f4eb29c00ec3a833d73ec4a473b11c8a
Author: Adhemerval Zanella <azanella@linux.vnet.ibm.com>
Date: Tue Jul 29 13:56:44 2014 -0500
PowerPC: Fix gprof entry point for LE
This patch fixes the ELFv2 gprof entry point since the ABI
does not define function descriptors. It fixes BZ#17213.
This is a backport of a53fbd8e6cd2f69bdfa3431d616a5f332aea6664.
diff --git a/ChangeLog b/ChangeLog
index 63e2be7..914fca9 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2013-07-29 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+
+ [BZ #17213]
+ * sysdeps/powerpc/powerpc64/entry.h: Fix TEXT_START definition for
+ powerpc64le.
+
2014-07-14 Alan Modra <amodra@gmail.com>
[BZ #17153]
diff --git a/NEWS b/NEWS
index d830583..a007829 100644
--- a/NEWS
+++ b/NEWS
@@ -10,7 +10,7 @@ Version 2.19.1
* The following bugs are resolved with this release:
16545, 16683, 16689, 16701, 16706, 16707, 16739, 16815, 16619, 16740,
- 17031, 17153.
+ 17031, 17153, 17213.
Version 2.19
diff --git a/sysdeps/powerpc/powerpc64/entry.h b/sysdeps/powerpc/powerpc64/entry.h
index 76ead1d..30553c1 100644
--- a/sysdeps/powerpc/powerpc64/entry.h
+++ b/sysdeps/powerpc/powerpc64/entry.h
@@ -23,6 +23,7 @@ extern void _start (void);
#define ENTRY_POINT _start
+#if _CALL_ELF != 2
/* We have to provide a special declaration. */
#define ENTRY_POINT_DECL(class) class void _start (void);
@@ -33,3 +34,4 @@ extern void _start (void);
#define TEXT_START \
({ extern unsigned long int _start_as_data[] asm ("_start"); \
_start_as_data[0]; })
+#endif
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=3c640c4acb9bc2c2cc7fa77d5ce1254953761dc1
commit 3c640c4acb9bc2c2cc7fa77d5ce1254953761dc1
Author: Alan Modra <amodra@gmail.com>
Date: Mon Jul 14 21:14:50 2014 +0930
Correct DT_PPC64_NUM
[BZ #17153]
* elf/elf.h (DT_PPC64_NUM): Correct value.
* NEWS: Add to fixed bug list.
This is a backport of f6c44d475104e931bab2b4ffa499961088de673c.
diff --git a/ChangeLog b/ChangeLog
index b23aace..63e2be7 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2014-07-14 Alan Modra <amodra@gmail.com>
+
+ [BZ #17153]
+ * elf/elf.h (DT_PPC64_NUM): Correct value.
+ * NEWS: Add to fixed bug list.
+
2014-07-08 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
* sysdeps/powerpc/memmove.c: Remove file.
diff --git a/NEWS b/NEWS
index 96b7621..d830583 100644
--- a/NEWS
+++ b/NEWS
@@ -10,7 +10,7 @@ Version 2.19.1
* The following bugs are resolved with this release:
16545, 16683, 16689, 16701, 16706, 16707, 16739, 16815, 16619, 16740,
- 17031.
+ 17031, 17153.
Version 2.19
diff --git a/elf/elf.h b/elf/elf.h
index 40e87b2..78815e8 100644
--- a/elf/elf.h
+++ b/elf/elf.h
@@ -2283,7 +2283,7 @@ typedef Elf32_Addr Elf32_Conflict;
#define DT_PPC64_OPD (DT_LOPROC + 1)
#define DT_PPC64_OPDSZ (DT_LOPROC + 2)
#define DT_PPC64_OPT (DT_LOPROC + 3)
-#define DT_PPC64_NUM 3
+#define DT_PPC64_NUM 4
/* PowerPC64 specific values for the DT_PPC64_OPT Dyn entry. */
#define PPC64_OPT_TLS 1
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=957afa3407c426969eaaa348981b9648d5191ae2
commit 957afa3407c426969eaaa348981b9648d5191ae2
Author: Adhemerval Zanella <azanella@linux.vnet.ibm.com>
Date: Tue Jul 8 08:54:09 2014 -0500
PowerPC: Cleanup powerpc memmove
Now that MEMCPY_OK_FOR_FWD_MEMMOVE should be define on memcopy.h there
is no need to specialized powerpc memmove implementation. This patch
moves the define set to powerpc memcopy and cleanup its definition on
powerpc code.
diff --git a/ChangeLog b/ChangeLog
index 46028e5..b23aace 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,15 @@
2014-07-08 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+ * sysdeps/powerpc/memmove.c: Remove file.
+ * sysdeps/powerpc/powerpc32/power4/memcopy.h
+ [MEMCPY_OK_FOR_FWD_MEMMOVE]: Define it to 1.
+ * sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c
+ [MEMCPY_OK_FOR_FWD_MEMMOVE]: Remove define.
+ * sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c:
+ [MEMCPY_OK_FOR_FWD_MEMMOVE]: Likewise.
+ * sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c: Include default
+ string memmove instead of removed powerpc one.
+
* sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp_l-power7.S:
[weak_alias]: Fix compiler warning due trailing data.
* sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp-power7.S:
diff --git a/sysdeps/powerpc/memmove.c b/sysdeps/powerpc/memmove.c
deleted file mode 100644
index 9c62ecb..0000000
--- a/sysdeps/powerpc/memmove.c
+++ /dev/null
@@ -1,22 +0,0 @@
-/* Copy memory to memory until the specified number of bytes
- has been copied. Overlap is handled correctly.
- Copyright (C) 1991-2014 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
- Contributed by Torbjorn Granlund (tege@sics.se).
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; see the file COPYING.LIB. If
- not, see <http://www.gnu.org/licenses/>. */
-
-#define MEMCPY_OK_FOR_FWD_MEMMOVE 1
-#include <string/memmove.c>
diff --git a/sysdeps/powerpc/powerpc32/power4/memcopy.h b/sysdeps/powerpc/powerpc32/power4/memcopy.h
index d3752dc..3431084 100644
--- a/sysdeps/powerpc/powerpc32/power4/memcopy.h
+++ b/sysdeps/powerpc/powerpc32/power4/memcopy.h
@@ -110,3 +110,7 @@
((byte *) dst_ep)[0] = __x; \
} \
} while (0)
+
+/* The powerpc memcpy implementation is safe to use for memmove. */
+#undef MEMCPY_OK_FOR_FWD_MEMMOVE
+#define MEMCPY_OK_FOR_FWD_MEMMOVE 1
diff --git a/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c
index 2861071..b14c3b1 100644
--- a/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c
+++ b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c
@@ -27,5 +27,4 @@ extern __typeof (memmove) __memmove_power7;
#undef libc_hidden_builtin_def
#define libc_hidden_builtin_def(name)
-#define MEMCPY_OK_FOR_FWD_MEMMOVE 1
#include <string/memmove.c>
diff --git a/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c
index e323a4d..d56b77a 100644
--- a/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c
+++ b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c
@@ -30,5 +30,4 @@ extern __typeof (memmove) __memmove_ppc;
__hidden_ver1 (__memmove_ppc, __GI_memmove, __memmove_ppc);
#endif
-#define MEMCPY_OK_FOR_FWD_MEMMOVE 1
#include <string/memmove.c>
diff --git a/sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c b/sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c
index b39348f..ff78fe6 100644
--- a/sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c
@@ -25,4 +25,4 @@
extern __typeof (memmove) __memmove_ppc attribute_hidden;
-#include <sysdeps/powerpc/memmove.c>
+#include <string/memmove.c>
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=8d9513a103bdd202ffa4884bdedc2c3c0dbab210
commit 8d9513a103bdd202ffa4884bdedc2c3c0dbab210
Author: Adhemerval Zanella <azanella@linux.vnet.ibm.com>
Date: Tue Jul 8 08:49:54 2014 -0500
PowerPC: Fix compiler warnings
This patch fixes some compiler due trailing data in #undef directives
and due missing prototypes.
diff --git a/ChangeLog b/ChangeLog
index 547dd56..46028e5 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,12 @@
2014-07-08 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+ * sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp_l-power7.S:
+ [weak_alias]: Fix compiler warning due trailing data.
+ * sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp-power7.S:
+ [weak_alias]: Likewise.
+ * sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c: Fix compile
+ warnigs due missing definition of __strcpy_power7 and __strlen_power7.
+
* sysdeps/powerpc/powerpc32/power4/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add memmove functions.
diff --git a/sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp-power7.S b/sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp-power7.S
index 930564c..0b00011 100644
--- a/sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp-power7.S
+++ b/sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp-power7.S
@@ -30,7 +30,7 @@
cfi_endproc; \
ASM_SIZE_DIRECTIVE(__strcasecmp_power7)
-#undef weak_alias(name, alias)
+#undef weak_alias
#define weak_alias(name, alias)
#undef libc_hidden_builtin_def
diff --git a/sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp_l-power7.S b/sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp_l-power7.S
index 46733f5..4b3ce51 100644
--- a/sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp_l-power7.S
+++ b/sysdeps/powerpc/powerpc32/power4/multiarch/strcasecmp_l-power7.S
@@ -30,7 +30,7 @@
cfi_endproc; \
ASM_SIZE_DIRECTIVE(__strcasecmp_l_power7)
-#undef weak_alias(name, alias)
+#undef weak_alias
#define weak_alias(name, alias)
#undef libc_hidden_builtin_def
diff --git a/sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c b/sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c
index ba9a460..628f8bd 100644
--- a/sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c
@@ -22,7 +22,9 @@
#undef libc_hidden_def
#define libc_hidden_def(name)
+extern typeof (strcpy) __strcpy_power7;
+extern typeof (strlen) __strlen_power7;
+
#define strcpy __strcpy_power7
#define strlen __strlen_power7
-
#include <sysdeps/powerpc/strcat.c>
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=b00ec143897f076ecbcedc7369b4b74e0c7f6d14
commit b00ec143897f076ecbcedc7369b4b74e0c7f6d14
Author: Adhemerval Zanella <azanella@linux.vnet.ibm.com>
Date: Tue Jul 8 08:35:44 2014 -0500
PowerPC: Add ifunc tests for memmove
This patch add the missing ifunc tests definition for memmove ppc32
optimization patch (commit 07aedd7).
This is a backport of 91f4b564bd7bedcd93e7047cad570ce292d6330b.
diff --git a/ChangeLog b/ChangeLog
index 3c12f8d..547dd56 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2014-07-08 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+
+ * sysdeps/powerpc/powerpc32/power4/multiarch/ifunc-impl-list.c
+ (__libc_ifunc_impl_list): Add memmove functions.
+
2014-07-07 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
* sysdeps/powerpc/powerpc64/power7/memcpy.S: Align VSX copies to 16B
diff --git a/sysdeps/powerpc/powerpc32/power4/multiarch/ifunc-impl-list.c b/sysdeps/powerpc/powerpc32/power4/multiarch/ifunc-impl-list.c
index 8ba6a80..38bf28e 100644
--- a/sysdeps/powerpc/powerpc32/power4/multiarch/ifunc-impl-list.c
+++ b/sysdeps/powerpc/powerpc32/power4/multiarch/ifunc-impl-list.c
@@ -59,6 +59,12 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__memcpy_cell)
IFUNC_IMPL_ADD (array, i, memcpy, 1, __memcpy_ppc))
+ /* Support sysdeps/powerpc/powerpc32/power4/multiarch/memmove.c. */
+ IFUNC_IMPL (i, name, memmove,
+ IFUNC_IMPL_ADD (array, i, memmove, hwcap & PPC_FEATURE_HAS_VSX,
+ __memmove_power7)
+ IFUNC_IMPL_ADD (array, i, memmove, 1, __memmove_ppc))
+
/* Support sysdeps/powerpc/powerpc32/power4/multiarch/memset.c. */
IFUNC_IMPL (i, name, memset,
IFUNC_IMPL_ADD (array, i, memset, hwcap & PPC_FEATURE_HAS_VSX,
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=565e3d6c8230affd7089bf5ebfcebbf72f32a27c
commit 565e3d6c8230affd7089bf5ebfcebbf72f32a27c
Author: Adhemerval Zanella <azanella@linux.vnet.ibm.com>
Date: Wed Jun 25 11:54:31 2014 -0500
PowerPC: Align power7 memcpy using VSX to quadword
This patch changes power7 memcpy to use VSX instructions only when
memory is aligned to quardword. It is to avoid unaligned kernel traps
on non-cacheable memory (for instance, memory-mapped I/O).
diff --git a/ChangeLog b/ChangeLog
index c3ee635..3c12f8d 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,9 @@
2014-07-07 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+ * sysdeps/powerpc/powerpc64/power7/memcpy.S: Align VSX copies to 16B
+ to avoid alignment traps in non-cacheable memory.
+ * sysdeps/powerpc/powerpc32/power7/memcpy.S: Likewise.
+
* sysdeps/powerpc/powerpc32/power4/multiarch/Makefile: Add memmove
multiarch objects.
* sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c: New
diff --git a/sysdeps/powerpc/powerpc32/power7/memcpy.S b/sysdeps/powerpc/powerpc32/power7/memcpy.S
index 52c2a6b..e540fea 100644
--- a/sysdeps/powerpc/powerpc32/power7/memcpy.S
+++ b/sysdeps/powerpc/powerpc32/power7/memcpy.S
@@ -38,8 +38,8 @@ EALIGN (memcpy, 5, 0)
ble cr1, L(copy_LT_32) /* If move < 32 bytes use short move
code. */
- andi. 11,3,7 /* Check alignment of DST. */
- clrlwi 10,4,29 /* Check alignment of SRC. */
+ andi. 11,3,15 /* Check alignment of DST. */
+ clrlwi 10,4,28 /* Check alignment of SRC. */
cmplw cr6,10,11 /* SRC and DST alignments match? */
mr 12,4
mr 31,5
diff --git a/sysdeps/powerpc/powerpc64/power7/memcpy.S b/sysdeps/powerpc/powerpc64/power7/memcpy.S
index bbfd381..58d9b12 100644
--- a/sysdeps/powerpc/powerpc64/power7/memcpy.S
+++ b/sysdeps/powerpc/powerpc64/power7/memcpy.S
@@ -36,16 +36,11 @@ EALIGN (memcpy, 5, 0)
ble cr1, L(copy_LT_32) /* If move < 32 bytes use short move
code. */
-#ifdef __LITTLE_ENDIAN__
-/* In little-endian mode, power7 takes an alignment trap on any lxvd2x
- or stxvd2x crossing a 32-byte boundary, so ensure the aligned_copy
- loop is only used for quadword aligned copies. */
+/* Align copies using VSX instructions to quadword. It is to avoid alignment
+ traps when memcpy is used on non-cacheable memory (for instance, memory
+ mapped I/O). */
andi. 10,3,15
clrldi 11,4,60
-#else
- andi. 10,3,7 /* Check alignment of DST. */
- clrldi 11,4,61 /* Check alignment of SRC. */
-#endif
cmpld cr6,10,11 /* SRC and DST alignments match? */
mr dst,3
@@ -53,13 +48,9 @@ EALIGN (memcpy, 5, 0)
beq L(aligned_copy)
mtocrf 0x01,0
-#ifdef __LITTLE_ENDIAN__
clrldi 0,0,60
-#else
- clrldi 0,0,61
-#endif
-/* Get the DST and SRC aligned to 8 bytes (16 for little-endian). */
+/* Get the DST and SRC aligned to 16 bytes. */
1:
bf 31,2f
lbz 6,0(src)
@@ -79,14 +70,12 @@ EALIGN (memcpy, 5, 0)
stw 6,0(dst)
addi dst,dst,4
8:
-#ifdef __LITTLE_ENDIAN__
bf 28,16f
ld 6,0(src)
addi src,src,8
std 6,0(dst)
addi dst,dst,8
16:
-#endif
subf cnt,0,cnt
/* Main aligned copy loop. Copies 128 bytes at a time. */
@@ -298,9 +287,6 @@ L(copy_LE_8):
.align 4
L(copy_GE_32_unaligned):
clrldi 0,0,60 /* Number of bytes until the 1st dst quadword. */
-#ifndef __LITTLE_ENDIAN__
- andi. 10,3,15 /* Check alignment of DST (against quadwords). */
-#endif
srdi 9,cnt,4 /* Number of full quadwords remaining. */
beq L(copy_GE_32_unaligned_cont)
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=6fae3527af330c32399e3a4cdfac3958fc440eb8
commit 6fae3527af330c32399e3a4cdfac3958fc440eb8
Author: Adhemerval Zanella <azanella@linux.vnet.ibm.com>
Date: Tue Jun 24 08:47:52 2014 -0500
PowerPC: optimized memmove for POWER7/PPC32
This patch adds a optimized memmove for power7 by using the optimized
power7 memcpy for forward copying.
diff --git a/ChangeLog b/ChangeLog
index 1cd2280..c3ee635 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,14 @@
2014-07-07 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+ * sysdeps/powerpc/powerpc32/power4/multiarch/Makefile: Add memmove
+ multiarch objects.
+ * sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c: New
+ file: multiarch power7 memmove.
+ * sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c: New file:
+ multiarch default memmove.
+ * sysdeps/powerpc/powerpc32/power4/multiarch/memmove.c: New file:
+ multiarch memove for powerpc32/power4.
+
* string/bcopy.c: Use full path to include memmove.c.
* sysdeps/powerpc/powerpc64/multiarch/Makefile: Add memmove and bcopy
multiarch objects.
diff --git a/sysdeps/powerpc/powerpc32/power4/multiarch/Makefile b/sysdeps/powerpc/powerpc32/power4/multiarch/Makefile
index a465685..a7d33b9 100644
--- a/sysdeps/powerpc/powerpc32/power4/multiarch/Makefile
+++ b/sysdeps/powerpc/powerpc32/power4/multiarch/Makefile
@@ -11,7 +11,7 @@ sysdep_routines += memcpy-power7 memcpy-a2 memcpy-power6 memcpy-cell \
strchr-power7 strchr-ppc32 wcschr-power7 wcschr-power6 \
wcschr-ppc32 wcsrchr-power7 wcsrchr-power6 wcsrchr-ppc32 \
wcscpy-power7 wcscpy-power6 wcscpy-ppc32 wordcopy-power7 \
- wordcopy-power6 wordcopy-ppc32
+ wordcopy-power6 wordcopy-ppc32 memmove-power7 memmove-ppc
CFLAGS-strncase-power7.c += -mcpu=power7 -funroll-loops
CFLAGS-strncase_l-power7.c += -mcpu=power7 -funroll-loops
diff --git a/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c
new file mode 100644
index 0000000..2861071
--- /dev/null
+++ b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c
@@ -0,0 +1,31 @@
+/* Power7 multiarch memmove.
+ Copyright (C) 2014 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; see the file COPYING.LIB. If
+ not, see <http://www.gnu.org/licenses/>. */
+
+#include <string.h>
+
+extern __typeof (memcpy) __memcpy_power7;
+#define memcpy __memcpy_power7
+
+extern __typeof (memmove) __memmove_power7;
+#define MEMMOVE __memmove_power7
+
+#undef libc_hidden_builtin_def
+#define libc_hidden_builtin_def(name)
+
+#define MEMCPY_OK_FOR_FWD_MEMMOVE 1
+#include <string/memmove.c>
diff --git a/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c
new file mode 100644
index 0000000..e323a4d
--- /dev/null
+++ b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c
@@ -0,0 +1,34 @@
+/* Power7 multiarch memmove.
+ Copyright (C) 2014 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; see the file COPYING.LIB. If
+ not, see <http://www.gnu.org/licenses/>. */
+
+#include <string.h>
+
+extern __typeof (memcpy) __memcpy_ppc;
+#define memcpy __memcpy_ppc
+
+extern __typeof (memmove) __memmove_ppc;
+#define MEMMOVE __memmove_ppc
+
+#if defined SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__memmove_ppc, __GI_memmove, __memmove_ppc);
+#endif
+
+#define MEMCPY_OK_FOR_FWD_MEMMOVE 1
+#include <string/memmove.c>
diff --git a/sysdeps/powerpc/powerpc32/power4/multiarch/memmove.c b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove.c
new file mode 100644
index 0000000..1070148
--- /dev/null
+++ b/sysdeps/powerpc/powerpc32/power4/multiarch/memmove.c
@@ -0,0 +1,34 @@
+/* Multiple versions of memmove. PowerPC32 version.
+ Copyright (C) 2014 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined SHARED && !defined NOT_IN_libc
+/* Redefine memmove so that the compiler won't complain about the type
+ mismatch with the IFUNC selector in strong_alias, below. */
+# include <string.h>
+# include "init-arch.h"
+
+extern __typeof (memmove) __memmove_ppc attribute_hidden;
+extern __typeof (memmove) __memmove_power7 attribute_hidden;
+
+libc_ifunc (memmove,
+ (hwcap & PPC_FEATURE_HAS_VSX)
+ ? __memmove_power7
+ : __memmove_ppc);
+#else
+# include <string/memmove.c>
+#endif
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=5d55f9b05ecb85b7a543f641829479cfb081f380
commit 5d55f9b05ecb85b7a543f641829479cfb081f380
Author: Adhemerval Zanella <azanella@linux.vnet.ibm.com>
Date: Fri Jun 20 12:55:16 2014 -0500
PowerPC: optimized memmove for POWER7/PPC64
This patch adds an optimized memmove optimization for POWER7/powerpc64.
Basically the idea is to use the memcpy for POWER7 on non-overlapped
memory regions and a optimized backward memcpy for memory regions
that overlap (similar to the idea of string/memmove.c).
The backward memcpy algorithm used is similar the one use for memcpy for
POWER7, with adjustments done for alignment. The difference is memory
is always aligned to 16 bytes before using VSX/altivec instructions.
diff --git a/ChangeLog b/ChangeLog
index 3edba9f..1cd2280 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2014-07-07 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+ * string/bcopy.c: Use full path to include memmove.c.
+ * sysdeps/powerpc/powerpc64/multiarch/Makefile: Add memmove and bcopy
+ multiarch objects.
+ * sysdeps/powerpc/powerpc64/multiarch/bcopy-ppc64.c: New file: default
+ bcopy for powerpc64.
+ * sysdeps/powerpc/powerpc64/multiarch/bcopy.c: New file: multiarch
+ bcopy for powerpc64.
+ * sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c: Add bcopy
+ and memmove implementations.
+ * sysdeps/powerpc/powerpc64/multiarch/memmove-power7.S: New file:
+ optimized multiarch memmove for POWER7/powerpc64.
+ * sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c: New file:
+ default multiarch memmove for powerpc64.
+ * sysdeps/powerpc/powerpc64/multiarch/memmove.c: New file: memmove
+ multiarch for powerpc64.
+ * sysdeps/powerpc/powerpc64/power7/bcopy.c: New file: optimized bcopy
+ for POWER7/powerpc64.
+ * sysdeps/powerpc/powerpc64/power7/memmove.S: New file: optimized
+ memmove for POWER7/powerpc64.
+
* sysdeps/powerpc/memmove.c (memmove): Cleanup impplementation to use
glibc default one.
diff --git a/string/bcopy.c b/string/bcopy.c
index 7c1225c..f497b5d 100644
--- a/string/bcopy.c
+++ b/string/bcopy.c
@@ -25,4 +25,4 @@
#define a2 dest
#define a2const
-#include <memmove.c>
+#include <string/memmove.c>
diff --git a/sysdeps/powerpc/powerpc64/multiarch/Makefile b/sysdeps/powerpc/powerpc64/multiarch/Makefile
index 524055f..82722fb 100644
--- a/sysdeps/powerpc/powerpc64/multiarch/Makefile
+++ b/sysdeps/powerpc/powerpc64/multiarch/Makefile
@@ -18,7 +18,8 @@ sysdep_routines += memcpy-power7 memcpy-a2 memcpy-power6 memcpy-cell \
strspn-power7 strspn-ppc64 strcspn-power7 strcspn-ppc64 \
strpbrk-power7 strpbrk-ppc64 strncpy-power7 strncpy-ppc64 \
stpncpy-power7 stpncpy-ppc64 strcmp-power7 strcmp-ppc64 \
- strcat-power7 strcat-ppc64
+ strcat-power7 strcat-ppc64 memmove-power7 memmove-ppc64 \
+ bcopy-ppc64
CFLAGS-strncase-power7.c += -mcpu=power7 -funroll-loops
CFLAGS-strncase_l-power7.c += -mcpu=power7 -funroll-loops
diff --git a/string/bcopy.c b/sysdeps/powerpc/powerpc64/multiarch/bcopy-ppc64.c
similarity index 77%
copy from string/bcopy.c
copy to sysdeps/powerpc/powerpc64/multiarch/bcopy-ppc64.c
index 7c1225c..f8d5aa5 100644
--- a/string/bcopy.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/bcopy-ppc64.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2014 Free Software Foundation, Inc.
+/* PowerPC64 default bcopy.
+ Copyright (C) 2014 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -17,12 +18,8 @@
#include <string.h>
-#define memmove bcopy
-#define rettype void
-#define RETURN(s) return
-#define a1 src
-#define a1const const
-#define a2 dest
-#define a2const
+extern __typeof (bcopy) __bcopy_ppc attribute_hidden;
-#include <memmove.c>
+#define bcopy __bcopy_ppc
+
+#include <string/bcopy.c>
diff --git a/string/bcopy.c b/sysdeps/powerpc/powerpc64/multiarch/bcopy.c
similarity index 65%
copy from string/bcopy.c
copy to sysdeps/powerpc/powerpc64/multiarch/bcopy.c
index 7c1225c..0688907 100644
--- a/string/bcopy.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/bcopy.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2014 Free Software Foundation, Inc.
+/* PowerPC64 multiarch bcopy.
+ Copyright (C) 2014 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -16,13 +17,13 @@
<http://www.gnu.org/licenses/>. */
#include <string.h>
+#include "init-arch.h"
-#define memmove bcopy
-#define rettype void
-#define RETURN(s) return
-#define a1 src
-#define a1const const
-#define a2 dest
-#define a2const
+extern __typeof (bcopy) __bcopy_ppc attribute_hidden;
+/* __bcopy_power7 symbol is implemented at memmove-power7.S */
+extern __typeof (bcopy) __bcopy_power7 attribute_hidden;
-#include <memmove.c>
+libc_ifunc (bcopy,
+ (hwcap & PPC_FEATURE_HAS_VSX)
+ ? __bcopy_power7
+ : __bcopy_ppc);
diff --git a/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c b/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c
index ec2e027..a574487 100644
--- a/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c
@@ -61,6 +61,12 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__memcpy_power4)
IFUNC_IMPL_ADD (array, i, memcpy, 1, __memcpy_ppc))
+ /* Support sysdeps/powerpc/powerpc64/multiarch/memmove.c. */
+ IFUNC_IMPL (i, name, memmove,
+ IFUNC_IMPL_ADD (array, i, memmove, hwcap & PPC_FEATURE_HAS_VSX,
+ __memmove_power7)
+ IFUNC_IMPL_ADD (array, i, memmove, 1, __memmove_ppc))
+
/* Support sysdeps/powerpc/powerpc64/multiarch/memset.c. */
IFUNC_IMPL (i, name, memset,
IFUNC_IMPL_ADD (array, i, memset, hwcap & PPC_FEATURE_HAS_VSX,
@@ -136,6 +142,12 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__bzero_power4)
IFUNC_IMPL_ADD (array, i, bzero, 1, __bzero_ppc))
+ /* Support sysdeps/powerpc/powerpc64/multiarch/bcopy.c. */
+ IFUNC_IMPL (i, name, bcopy,
+ IFUNC_IMPL_ADD (array, i, bcopy, hwcap & PPC_FEATURE_HAS_VSX,
+ __bcopy_power7)
+ IFUNC_IMPL_ADD (array, i, bcopy, 1, __bcopy_ppc))
+
/* Support sysdeps/powerpc/powerpc64/multiarch/mempcpy.c. */
IFUNC_IMPL (i, name, mempcpy,
IFUNC_IMPL_ADD (array, i, mempcpy,
diff --git a/string/bcopy.c b/sysdeps/powerpc/powerpc64/multiarch/memmove-power7.S
similarity index 50%
copy from string/bcopy.c
copy to sysdeps/powerpc/powerpc64/multiarch/memmove-power7.S
index 7c1225c..667e7bc 100644
--- a/string/bcopy.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/memmove-power7.S
@@ -1,4 +1,5 @@
-/* Copyright (C) 1991-2014 Free Software Foundation, Inc.
+/* Optimized memmove implementation for PowerPC64/POWER7.
+ Copyright (C) 2014 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -15,14 +16,28 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
+#include <sysdep.h>
-#define memmove bcopy
-#define rettype void
-#define RETURN(s) return
-#define a1 src
-#define a1const const
-#define a2 dest
-#define a2const
+#undef EALIGN
+#define EALIGN(name, alignt, words) \
+ .section ".text"; \
+ ENTRY_2(__memmove_power7) \
+ .align ALIGNARG(alignt); \
+ EALIGN_W_##words; \
+ BODY_LABEL(__memmove_power7): \
+ cfi_startproc; \
+ LOCALENTRY(__memmove_power7)
-#include <memmove.c>
+#undef END_GEN_TB
+#define END_GEN_TB(name, mask) \
+ cfi_endproc; \
+ TRACEBACK_MASK(__memmove_power7,mask) \
+ END_2(__memmove_power7)
+
+#undef libc_hidden_builtin_def
+#define libc_hidden_builtin_def(name)
+
+#undef bcopy
+#define bcopy __bcopy_power7
+
+#include <sysdeps/powerpc/powerpc64/power7/memmove.S>
diff --git a/string/bcopy.c b/sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c
similarity index 71%
copy from string/bcopy.c
copy to sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c
index 7c1225c..b39348f 100644
--- a/string/bcopy.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c
@@ -1,4 +1,4 @@
-/* Copyright (C) 1991-2014 Free Software Foundation, Inc.
+/* Copyright (C) 2014 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -17,12 +17,12 @@
#include <string.h>
-#define memmove bcopy
-#define rettype void
-#define RETURN(s) return
-#define a1 src
-#define a1const const
-#define a2 dest
-#define a2const
+#define MEMMOVE __memmove_ppc
+#if !defined(NOT_IN_libc) && defined(SHARED)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name)
+#endif
-#include <memmove.c>
+extern __typeof (memmove) __memmove_ppc attribute_hidden;
+
+#include <sysdeps/powerpc/memmove.c>
diff --git a/sysdeps/powerpc/powerpc64/multiarch/memmove.c b/sysdeps/powerpc/powerpc64/multiarch/memmove.c
new file mode 100644
index 0000000..9a1ce8f
--- /dev/null
+++ b/sysdeps/powerpc/powerpc64/multiarch/memmove.c
@@ -0,0 +1,45 @@
+/* Multiple versions of memmove. PowerPC64 version.
+ Copyright (C) 2014 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* Define multiple versions only for the definition in lib and for
+ DSO. In static binaries we need memmove before the initialization
+ happened. */
+#if defined SHARED && !defined NOT_IN_libc
+/* Redefine memmove so that the compiler won't complain about the type
+ mismatch with the IFUNC selector in strong_alias, below. */
+# undef memmove
+# define memmove __redirect_memmove
+# include <string.h>
+# include "init-arch.h"
+
+extern __typeof (__redirect_memmove) __libc_memmove;
+
+extern __typeof (__redirect_memmove) __memmove_ppc attribute_hidden;
+extern __typeof (__redirect_memmove) __memmove_power7 attribute_hidden;
+
+libc_ifunc (__libc_memmove,
+ (hwcap & PPC_FEATURE_HAS_VSX)
+ ? __memmove_power7
+ : __memmove_ppc);
+
+#undef memmove
+strong_alias (__libc_memmove, memmove);
+libc_hidden_ver (__libc_memmove, memmove);
+#else
+# include <string/memmove.c>
+#endif
diff --git a/sysdeps/powerpc/powerpc64/power7/bcopy.c b/sysdeps/powerpc/powerpc64/power7/bcopy.c
new file mode 100644
index 0000000..4a6a400
--- /dev/null
+++ b/sysdeps/powerpc/powerpc64/power7/bcopy.c
@@ -0,0 +1 @@
+/* Implemented at memmove.S */
diff --git a/sysdeps/powerpc/powerpc64/power7/memmove.S b/sysdeps/powerpc/powerpc64/power7/memmove.S
new file mode 100644
index 0000000..8663dfb
--- /dev/null
+++ b/sysdeps/powerpc/powerpc64/power7/memmove.S
@@ -0,0 +1,831 @@
+/* Optimized memmove implementation for PowerPC64/POWER7.
+ Copyright (C) 2014 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <sysdep.h>
+
+
+/* void* [r3] memmove (void *dest [r3], const void *src [r4], size_t len [r5])
+
+ This optimization check if memory 'dest' overlaps with 'src'. If it does
+ not then it calls an optimized memcpy call (similar to memcpy for POWER7,
+ embedded here to gain some cycles).
+ If source and destiny overlaps, a optimized backwards memcpy is used
+ instead. */
+
+ .machine power7
+EALIGN (memmove, 5, 0)
+ CALL_MCOUNT 3
+
+L(_memmove):
+ subf r9,r4,r3
+ cmpld cr7,r9,r5
+ blt cr7,L(memmove_bwd)
+
+ cmpldi cr1,r5,31
+ neg 0,3
+ ble cr1, L(copy_LT_32) /* If move < 32 bytes use short move
+ code. */
+
+ andi. 10,3,15
+ clrldi 11,4,60
+ cmpld cr6,10,11 /* SRC and DST alignments match? */
+
+ mr r11,3
+ bne cr6,L(copy_GE_32_unaligned)
+ beq L(aligned_copy)
+
+ mtocrf 0x01,0
+ clrldi 0,0,60
+
+/* Get the DST and SRC aligned to 8 bytes (16 for little-endian). */
+1:
+ bf 31,2f
+ lbz 6,0(r4)
+ addi r4,r4,1
+ stb 6,0(r11)
+ addi r11,r11,1
+2:
+ bf 30,4f
+ lhz 6,0(r4)
+ addi r4,r4,2
+ sth 6,0(r11)
+ addi r11,r11,2
+4:
+ bf 29,8f
+ lwz 6,0(r4)
+ addi r4,r4,4
+ stw 6,0(r11)
+ addi r11,r11,4
+8:
+ bf 28,16f
+ ld 6,0(r4)
+ addi r4,r4,8
+ std 6,0(r11)
+ addi r11,r11,8
+16:
+ subf r5,0,r5
+
+/* Main aligned copy loop. Copies 128 bytes at a time. */
+L(aligned_copy):
+ li 6,16
+ li 7,32
+ li 8,48
+ mtocrf 0x02,r5
+ srdi 12,r5,7
+ cmpdi 12,0
+ beq L(aligned_tail)
+ lxvd2x 6,0,r4
+ lxvd2x 7,r4,6
+ mtctr 12
+ b L(aligned_128loop)
+
+ .align 4
+L(aligned_128head):
+ /* for the 2nd + iteration of this loop. */
+ lxvd2x 6,0,r4
+ lxvd2x 7,r4,6
+L(aligned_128loop):
+ lxvd2x 8,r4,7
+ lxvd2x 9,r4,8
+ stxvd2x 6,0,r11
+ addi r4,r4,64
+ stxvd2x 7,r11,6
+ stxvd2x 8,r11,7
+ stxvd2x 9,r11,8
+ lxvd2x 6,0,r4
+ lxvd2x 7,r4,6
+ addi r11,r11,64
+ lxvd2x 8,r4,7
+ lxvd2x 9,r4,8
+ addi r4,r4,64
+ stxvd2x 6,0,r11
+ stxvd2x 7,r11,6
+ stxvd2x 8,r11,7
+ stxvd2x 9,r11,8
+ addi r11,r11,64
+ bdnz L(aligned_128head)
+
+L(aligned_tail):
+ mtocrf 0x01,r5
+ bf 25,32f
+ lxvd2x 6,0,r4
+ lxvd2x 7,r4,6
+ lxvd2x 8,r4,7
+ lxvd2x 9,r4,8
+ addi r4,r4,64
+ stxvd2x 6,0,r11
+ stxvd2x 7,r11,6
+ stxvd2x 8,r11,7
+ stxvd2x 9,r11,8
+ addi r11,r11,64
+32:
+ bf 26,16f
+ lxvd2x 6,0,r4
+ lxvd2x 7,r4,6
+ addi r4,r4,32
+ stxvd2x 6,0,r11
+ stxvd2x 7,r11,6
+ addi r11,r11,32
+16:
+ bf 27,8f
+ lxvd2x 6,0,r4
+ addi r4,r4,16
+ stxvd2x 6,0,r11
+ addi r11,r11,16
+8:
+ bf 28,4f
+ ld 6,0(r4)
+ addi r4,r4,8
+ std 6,0(r11)
+ addi r11,r11,8
+4: /* Copies 4~7 bytes. */
+ bf 29,L(tail2)
+ lwz 6,0(r4)
+ stw 6,0(r11)
+ bf 30,L(tail5)
+ lhz 7,4(r4)
+ sth 7,4(r11)
+ bflr 31
+ lbz 8,6(r4)
+ stb 8,6(r11)
+ /* Return original DST pointer. */
+ blr
+
+/* Handle copies of 0~31 bytes. */
+ .align 4
+L(copy_LT_32):
+ mr r11,3
+ cmpldi cr6,r5,8
+ mtocrf 0x01,r5
+ ble cr6,L(copy_LE_8)
+
+ /* At least 9 bytes to go. */
+ neg 8,4
+ andi. 0,8,3
+ cmpldi cr1,r5,16
+ beq L(copy_LT_32_aligned)
+
+ /* Force 4-byte alignment for SRC. */
+ mtocrf 0x01,0
+ subf r5,0,r5
+2:
+ bf 30,1f
+ lhz 6,0(r4)
+ addi r4,r4,2
+ sth 6,0(r11)
+ addi r11,r11,2
+1:
+ bf 31,L(end_4bytes_alignment)
+ lbz 6,0(r4)
+ addi r4,r4,1
+ stb 6,0(r11)
+ addi r11,r11,1
+
+ .align 4
+L(end_4bytes_alignment):
+ cmpldi cr1,r5,16
+ mtocrf 0x01,r5
+
+L(copy_LT_32_aligned):
+ /* At least 6 bytes to go, and SRC is word-aligned. */
+ blt cr1,8f
+
+ /* Copy 16 bytes. */
+ lwz 6,0(r4)
+ lwz 7,4(r4)
+ stw 6,0(r11)
+ lwz 8,8(r4)
+ stw 7,4(r11)
+ lwz 6,12(r4)
+ addi r4,r4,16
+ stw 8,8(r11)
+ stw 6,12(r11)
+ addi r11,r11,16
+8: /* Copy 8 bytes. */
+ bf 28,L(tail4)
+ lwz 6,0(r4)
+ lwz 7,4(r4)
+ addi r4,r4,8
+ stw 6,0(r11)
+ stw 7,4(r11)
+ addi r11,r11,8
+
+ .align 4
+/* Copies 4~7 bytes. */
+L(tail4):
+ bf 29,L(tail2)
+ lwz 6,0(r4)
+ stw 6,0(r11)
+ bf 30,L(tail5)
+ lhz 7,4(r4)
+ sth 7,4(r11)
+ bflr 31
+ lbz 8,6(r4)
+ stb 8,6(r11)
+ /* Return original DST pointer. */
+ blr
+
+ .align 4
+/* Copies 2~3 bytes. */
+L(tail2):
+ bf 30,1f
+ lhz 6,0(r4)
+ sth 6,0(r11)
+ bflr 31
+ lbz 7,2(r4)
+ stb 7,2(r11)
+ blr
+
+ .align 4
+L(tail5):
+ bflr 31
+ lbz 6,4(r4)
+ stb 6,4(r11)
+ blr
+
+ .align 4
+1:
+ bflr 31
+ lbz 6,0(r4)
+ stb 6,0(r11)
+ /* Return original DST pointer. */
+ blr
+
+/* Handles copies of 0~8 bytes. */
+ .align 4
+L(copy_LE_8):
+ bne cr6,L(tail4)
+
+ /* Though we could've used ld/std here, they are still
+ slow for unaligned cases. */
+
+ lwz 6,0(r4)
+ lwz 7,4(r4)
+ stw 6,0(r11)
+ stw 7,4(r11)
+ blr
+
+
+/* Handle copies of 32+ bytes where DST is aligned (to quadword) but
+ SRC is not. Use aligned quadword loads from SRC, shifted to realign
+ the data, allowing for aligned DST stores. */
+ .align 4
+L(copy_GE_32_unaligned):
+ clrldi 0,0,60 /* Number of bytes until the 1st r11 quadword. */
+ srdi 9,r5,4 /* Number of full quadwords remaining. */
+
+ beq L(copy_GE_32_unaligned_cont)
+
+ /* DST is not quadword aligned, get it aligned. */
+
+ mtocrf 0x01,0
+ subf r5,0,r5
+
+ /* Vector instructions work best when proper alignment (16-bytes)
+ is present. Move 0~15 bytes as needed to get DST quadword-aligned. */
+1:
+ bf 31,2f
+ lbz 6,0(r4)
+ addi r4,r4,1
+ stb 6,0(r11)
+ addi r11,r11,1
+2:
+ bf 30,4f
+ lhz 6,0(r4)
+ addi r4,r4,2
+ sth 6,0(r11)
+ addi r11,r11,2
+4:
+ bf 29,8f
+ lwz 6,0(r4)
+ addi r4,r4,4
+ stw 6,0(r11)
+ addi r11,r11,4
+8:
+ bf 28,0f
+ ld 6,0(r4)
+ addi r4,r4,8
+ std 6,0(r11)
+ addi r11,r11,8
+0:
+ srdi 9,r5,4 /* Number of full quadwords remaining. */
+
+ /* The proper alignment is present, it is OK to copy the bytes now. */
+L(copy_GE_32_unaligned_cont):
+
+ /* Setup two indexes to speed up the indexed vector operations. */
+ clrldi 10,r5,60
+ li 6,16 /* Index for 16-bytes offsets. */
+ li 7,32 /* Index for 32-bytes offsets. */
+ cmpldi cr1,10,0
+ srdi 8,r5,5 /* Setup the loop counter. */
+ mtocrf 0x01,9
+ cmpldi cr6,9,1
+#ifdef __LITTLE_ENDIAN__
+ lvsr 5,0,r4
+#else
+ lvsl 5,0,r4
+#endif
+ lvx 3,0,r4
+ li 0,0
+ bf 31,L(setup_unaligned_loop)
+
+ /* Copy another 16 bytes to align to 32-bytes due to the loop. */
+ lvx 4,r4,6
+#ifdef __LITTLE_ENDIAN__
+ vperm 6,4,3,5
+#else
+ vperm 6,3,4,5
+#endif
+ addi r4,r4,16
+ stvx 6,0,r11
+ addi r11,r11,16
+ vor 3,4,4
+ clrrdi 0,r4,60
+
+L(setup_unaligned_loop):
+ mtctr 8
+ ble cr6,L(end_unaligned_loop)
+
+ /* Copy 32 bytes at a time using vector instructions. */
+ .align 4
+L(unaligned_loop):
+
+ /* Note: vr6/vr10 may contain data that was already copied,
+ but in order to get proper alignment, we may have to copy
+ some portions again. This is faster than having unaligned
+ vector instructions though. */
+
+ lvx 4,r4,6
+#ifdef __LITTLE_ENDIAN__
+ vperm 6,4,3,5
+#else
+ vperm 6,3,4,5
+#endif
+ lvx 3,r4,7
+#ifdef __LITTLE_ENDIAN__
+ vperm 10,3,4,5
+#else
+ vperm 10,4,3,5
+#endif
+ addi r4,r4,32
+ stvx 6,0,r11
+ stvx 10,r11,6
+ addi r11,r11,32
+ bdnz L(unaligned_loop)
+
+ clrrdi 0,r4,60
+
+ .align 4
+L(end_unaligned_loop):
+
+ /* Check for tail bytes. */
+ mtocrf 0x01,r5
+ beqlr cr1
+
+ add r4,r4,0
+
+ /* We have 1~15 tail bytes to copy, and DST is quadword aligned. */
+ /* Copy 8 bytes. */
+ bf 28,4f
+ lwz 6,0(r4)
+ lwz 7,4(r4)
+ addi r4,r4,8
+ stw 6,0(r11)
+ stw 7,4(r11)
+ addi r11,r11,8
+4: /* Copy 4~7 bytes. */
+ bf 29,L(tail2)
+ lwz 6,0(r4)
+ stw 6,0(r11)
+ bf 30,L(tail5)
+ lhz 7,4(r4)
+ sth 7,4(r11)
+ bflr 31
+ lbz 8,6(r4)
+ stb 8,6(r11)
+ /* Return original DST pointer. */
+ blr
+
+ /* Start to memcpy backward implementation: the algorith first check if
+ src and dest have the same alignment and if it does align both to 16
+ bytes and copy using VSX instructions.
+ If does not, align dest to 16 bytes and use VMX (altivec) instruction
+ to read two 16 bytes at time, shift/permute the bytes read and write
+ aligned to dest. */
+L(memmove_bwd):
+ cmpldi cr1,r5,31
+ /* Copy is done backwards: update the pointers and check alignment. */
+ add r11,r3,r5
+ add r4,r4,r5
+ mr r0,r11
+ ble cr1, L(copy_LT_32_bwd) /* If move < 32 bytes use short move
+ code. */
+
+ andi. r10,r11,15 /* Check if r11 is aligned to 16 bytes */
+ clrldi r9,r4,60 /* Check if r4 is aligned to 16 bytes */
+ cmpld cr6,r10,r9 /* SRC and DST alignments match? */
+
+ bne cr6,L(copy_GE_32_unaligned_bwd)
+ beq L(aligned_copy_bwd)
+
+ mtocrf 0x01,r0
+ clrldi r0,r0,60
+
+/* Get the DST and SRC aligned to 16 bytes. */
+1:
+ bf 31,2f
+ lbz r6,-1(r4)
+ subi r4,r4,1
+ stb r6,-1(r11)
+ subi r11,r11,1
+2:
+ bf 30,4f
+ lhz r6,-2(r4)
+ subi r4,r4,2
+ sth r6,-2(r11)
+ subi r11,r11,2
+4:
+ bf 29,8f
+ lwz r6,-4(r4)
+ subi r4,r4,4
+ stw r6,-4(r11)
+ subi r11,r11,4
+8:
+ bf 28,16f
+ ld r6,-8(r4)
+ subi r4,r4,8
+ std r6,-8(r11)
+ subi r11,r11,8
+16:
+ subf r5,0,r5
+
+/* Main aligned copy loop. Copies 128 bytes at a time. */
+L(aligned_copy_bwd):
+ li r6,-16
+ li r7,-32
+ li r8,-48
+ li r9,-64
+ mtocrf 0x02,r5
+ srdi r12,r5,7
+ cmpdi r12,0
+ beq L(aligned_tail_bwd)
+ lxvd2x v6,r4,r6
+ lxvd2x v7,r4,r7
+ mtctr 12
+ b L(aligned_128loop_bwd)
+
+ .align 4
+L(aligned_128head_bwd):
+ /* for the 2nd + iteration of this loop. */
+ lxvd2x v6,r4,r6
+ lxvd2x v7,r4,r7
+L(aligned_128loop_bwd):
+ lxvd2x v8,r4,r8
+ lxvd2x v9,r4,r9
+ stxvd2x v6,r11,r6
+ subi r4,r4,64
+ stxvd2x v7,r11,r7
+ stxvd2x v8,r11,r8
+ stxvd2x v9,r11,r9
+ lxvd2x v6,r4,r6
+ lxvd2x v7,r4,7
+ subi r11,r11,64
+ lxvd2x v8,r4,r8
+ lxvd2x v9,r4,r9
+ subi r4,r4,64
+ stxvd2x v6,r11,r6
+ stxvd2x v7,r11,r7
+ stxvd2x v8,r11,r8
+ stxvd2x v9,r11,r9
+ subi r11,r11,64
+ bdnz L(aligned_128head_bwd)
+
+L(aligned_tail_bwd):
+ mtocrf 0x01,r5
+ bf 25,32f
+ lxvd2x v6,r4,r6
+ lxvd2x v7,r4,r7
+ lxvd2x v8,r4,r8
+ lxvd2x v9,r4,r9
+ subi r4,r4,64
+ stxvd2x v6,r11,r6
+ stxvd2x v7,r11,r7
+ stxvd2x v8,r11,r8
+ stxvd2x v9,r11,r9
+ subi r11,r11,64
+32:
+ bf 26,16f
+ lxvd2x v6,r4,r6
+ lxvd2x v7,r4,r7
+ subi r4,r4,32
+ stxvd2x v6,r11,r6
+ stxvd2x v7,r11,r7
+ subi r11,r11,32
+16:
+ bf 27,8f
+ lxvd2x v6,r4,r6
+ subi r4,r4,16
+ stxvd2x v6,r11,r6
+ subi r11,r11,16
+8:
+ bf 28,4f
+ ld r6,-8(r4)
+ subi r4,r4,8
+ std r6,-8(r11)
+ subi r11,r11,8
+4: /* Copies 4~7 bytes. */
+ bf 29,L(tail2_bwd)
+ lwz r6,-4(r4)
+ stw r6,-4(r11)
+ bf 30,L(tail5_bwd)
+ lhz r7,-6(r4)
+ sth r7,-6(r11)
+ bflr 31
+ lbz r8,-7(r4)
+ stb r8,-7(r11)
+ /* Return original DST pointer. */
+ blr
+
+/* Handle copies of 0~31 bytes. */
+ .align 4
+L(copy_LT_32_bwd):
+ cmpldi cr6,r5,8
+ mtocrf 0x01,r5
+ ble cr6,L(copy_LE_8_bwd)
+
+ /* At least 9 bytes to go. */
+ neg r8,r4
+ andi. r0,r8,3
+ cmpldi cr1,r5,16
+ beq L(copy_LT_32_aligned_bwd)
+
+ /* Force 4-byte alignment for SRC. */
+ mtocrf 0x01,0
+ subf r5,0,r5
+2:
+ bf 30,1f
+ lhz r6,-2(r4)
+ subi r4,r4,2
+ sth r6,-2(r11)
+ subi r11,r11,2
+1:
+ bf 31,L(end_4bytes_alignment_bwd)
+ lbz 6,-1(r4)
+ subi r4,r4,1
+ stb 6,-1(r11)
+ subi r11,r11,1
+
+ .align 4
+L(end_4bytes_alignment_bwd):
+ cmpldi cr1,r5,16
+ mtocrf 0x01,r5
+
+L(copy_LT_32_aligned_bwd):
+ /* At least 6 bytes to go, and SRC is word-aligned. */
+ blt cr1,8f
+
+ /* Copy 16 bytes. */
+ lwz r6,-4(r4)
+ lwz r7,-8(r4)
+ stw r6,-4(r11)
+ lwz r8,-12(r4)
+ stw r7,-8(r11)
+ lwz r6,-16(r4)
+ subi r4,r4,16
+ stw r8,-12(r11)
+ stw r6,-16(r11)
+ subi r11,r11,16
+8: /* Copy 8 bytes. */
+ bf 28,L(tail4_bwd)
+ lwz r6,-4(r4)
+ lwz r7,-8(r4)
+ subi r4,r4,8
+ stw r6,-4(r11)
+ stw r7,-8(r11)
+ subi r11,r11,8
+
+ .align 4
+/* Copies 4~7 bytes. */
+L(tail4_bwd):
+ bf 29,L(tail2_bwd)
+ lwz 6,-4(r4)
+ stw 6,-4(r11)
+ bf 30,L(tail5_bwd)
+ lhz 7,-6(r4)
+ sth 7,-6(r11)
+ bflr 31
+ lbz 8,-7(r4)
+ stb 8,-7(r11)
+ /* Return original DST pointer. */
+ blr
+
+ .align 4
+/* Copies 2~3 bytes. */
+L(tail2_bwd):
+ bf 30,1f
+ lhz 6,-2(r4)
+ sth 6,-2(r11)
+ bflr 31
+ lbz 7,-3(r4)
+ stb 7,-3(r11)
+ blr
+
+ .align 4
+L(tail5_bwd):
+ bflr 31
+ lbz 6,-5(r4)
+ stb 6,-5(r11)
+ blr
+
+ .align 4
+1:
+ bflr 31
+ lbz 6,-1(r4)
+ stb 6,-1(r11)
+ /* Return original DST pointer. */
+ blr
+
+
+/* Handles copies of 0~8 bytes. */
+ .align 4
+L(copy_LE_8_bwd):
+ bne cr6,L(tail4_bwd)
+
+ /* Though we could've used ld/std here, they are still
+ slow for unaligned cases. */
+ lwz 6,-8(r4)
+ lwz 7,-4(r4)
+ stw 6,-8(r11)
+ stw 7,-4(r11)
+ blr
+
+
+/* Handle copies of 32+ bytes where DST is aligned (to quadword) but
+ SRC is not. Use aligned quadword loads from SRC, shifted to realign
+ the data, allowing for aligned DST stores. */
+ .align 4
+L(copy_GE_32_unaligned_bwd):
+ andi. r10,r11,15 /* Check alignment of DST against 16 bytes.. */
+ srdi r9,r5,4 /* Number of full quadwords remaining. */
+
+ beq L(copy_GE_32_unaligned_cont_bwd)
+
+ /* DST is not quadword aligned and r10 holds the address masked to
+ compare alignments. */
+ mtocrf 0x01,r10
+ subf r5,r10,r5
+
+ /* Vector instructions work best when proper alignment (16-bytes)
+ is present. Move 0~15 bytes as needed to get DST quadword-aligned. */
+1:
+ bf 31,2f
+ lbz r6,-1(r4)
+ subi r4,r4,1
+ stb r6,-1(r11)
+ subi r11,r11,1
+2:
+ bf 30,4f
+ lhz r6,-2(r4)
+ subi r4,r4,2
+ sth r6,-2(r11)
+ subi r11,r11,2
+4:
+ bf 29,8f
+ lwz r6,-4(r4)
+ subi r4,r4,4
+ stw r6,-4(r11)
+ subi r11,r11,4
+8:
+ bf 28,0f
+ ld r6,-8(r4)
+ subi r4,r4,8
+ std r6,-8(r11)
+ subi r11,r11,8
+0:
+ srdi r9,r5,4 /* Number of full quadwords remaining. */
+
+ /* The proper alignment is present, it is OK to copy the bytes now. */
+L(copy_GE_32_unaligned_cont_bwd):
+
+ /* Setup two indexes to speed up the indexed vector operations. */
+ clrldi r10,r5,60
+ li r6,-16 /* Index for 16-bytes offsets. */
+ li r7,-32 /* Index for 32-bytes offsets. */
+ cmpldi cr1,10,0
+ srdi r8,r5,5 /* Setup the loop counter. */
+ mtocrf 0x01,9
+ cmpldi cr6,r9,1
+#ifdef __LITTLE_ENDIAN__
+ lvsr v5,r0,r4
+#else
+ lvsl v5,r0,r4
+#endif
+ lvx v3,0,r4
+ li r0,0
+ bf 31,L(setup_unaligned_loop_bwd)
+
+ /* Copy another 16 bytes to align to 32-bytes due to the loop. */
+ lvx v4,r4,r6
+#ifdef __LITTLE_ENDIAN__
+ vperm v6,v3,v4,v5
+#else
+ vperm v6,v4,v3,v5
+#endif
+ subi r4,r4,16
+ stvx v6,r11,r6
+ subi r11,r11,16
+ vor v3,v4,v4
+ clrrdi r0,r4,60
+
+L(setup_unaligned_loop_bwd):
+ mtctr r8
+ ble cr6,L(end_unaligned_loop_bwd)
+
+ /* Copy 32 bytes at a time using vector instructions. */
+ .align 4
+L(unaligned_loop_bwd):
+
+ /* Note: vr6/vr10 may contain data that was already copied,
+ but in order to get proper alignment, we may have to copy
+ some portions again. This is faster than having unaligned
+ vector instructions though. */
+
+ lvx v4,r4,r6
+#ifdef __LITTLE_ENDIAN__
+ vperm v6,v3,v4,v5
+#else
+ vperm v6,v4,v3,v5
+#endif
+ lvx v3,r4,r7
+#ifdef __LITTLE_ENDIAN__
+ vperm v10,v4,v3,v5
+#else
+ vperm v10,v3,v4,v5
+#endif
+ subi r4,r4,32
+ stvx v6,r11,r6
+ stvx v10,r11,r7
+ subi r11,r11,32
+ bdnz L(unaligned_loop_bwd)
+
+ clrrdi r0,r4,60
+
+ .align 4
+L(end_unaligned_loop_bwd):
+
+ /* Check for tail bytes. */
+ mtocrf 0x01,r5
+ beqlr cr1
+
+ add r4,r4,0
+
+ /* We have 1~15 tail bytes to copy, and DST is quadword aligned. */
+ /* Copy 8 bytes. */
+ bf 28,4f
+ lwz r6,-4(r4)
+ lwz r7,-8(r4)
+ subi r4,r4,8
+ stw r6,-4(r11)
+ stw r7,-8(r11)
+ subi r11,r11,8
+4: /* Copy 4~7 bytes. */
+ bf 29,L(tail2_bwd)
+ lwz r6,-4(r4)
+ stw r6,-4(r11)
+ bf 30,L(tail5_bwd)
+ lhz r7,-6(r4)
+ sth r7,-6(r11)
+ bflr 31
+ lbz r8,-7(r4)
+ stb r8,-7(r11)
+ /* Return original DST pointer. */
+ blr
+END_GEN_TB (memmove, TB_TOCLESS)
+libc_hidden_builtin_def (memmove)
+
+
+/* void bcopy(const void *src [r3], void *dest [r4], size_t n [r5])
+ Implemented in this file to avoid linker create a stub function call
+ in the branch to '_memmove'. */
+ENTRY (bcopy)
+ mr r6,r3
+ mr r3,r4
+ mr r4,r6
+ b L(_memmove)
+END (bcopy)
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=dde00e9914370ddd90c9bbc4f3f0e455efae4b47
commit dde00e9914370ddd90c9bbc4f3f0e455efae4b47
Author: Adhemerval Zanella <azanella@linux.vnet.ibm.com>
Date: Tue Jun 24 06:42:31 2014 -0500
PowerPC: memmove default implementation cleanup
This patch removes the powerpc specific logic in memmove and instead
include default implementation with MEMCPY_OK_FOR_FWD_MEMMOVE defined.
This lead in a increase performance, since the constraints to use
memcpy in powerpc code are too restrictive and memcpy can be used for
any forward memmove.
This is a backport of d6f68bbef4427850c2901728a1d13efc0e687297.
diff --git a/ChangeLog b/ChangeLog
index 54db831..3edba9f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2014-07-07 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+
+ * sysdeps/powerpc/memmove.c (memmove): Cleanup impplementation to use
+ glibc default one.
+
2014-07-02 Vidya Ranganathan <vidya@linux.vnet.ibm.com>
Adhemerval Zanella <azanella@linux.vnet.ibm.com>
diff --git a/sysdeps/powerpc/memmove.c b/sysdeps/powerpc/memmove.c
index 3859da8..9c62ecb 100644
--- a/sysdeps/powerpc/memmove.c
+++ b/sysdeps/powerpc/memmove.c
@@ -18,101 +18,5 @@
License along with the GNU C Library; see the file COPYING.LIB. If
not, see <http://www.gnu.org/licenses/>. */
-#include <string.h>
-#include <memcopy.h>
-#include <pagecopy.h>
-
-/* All this is so that bcopy.c can #include
- this file after defining some things. */
-#ifndef a1
-#define a1 dest /* First arg is DEST. */
-#define a1const
-#define a2 src /* Second arg is SRC. */
-#define a2const const
-#undef memmove
-#endif
-#if !defined(RETURN) || !defined(rettype)
-#define RETURN(s) return (s) /* Return DEST. */
-#define rettype void *
-#endif
-
-#ifndef MEMMOVE
-#define MEMMOVE memmove
-#endif
-
-rettype
-MEMMOVE (a1, a2, len)
- a1const void *a1;
- a2const void *a2;
- size_t len;
-{
- unsigned long int dstp = (long int) dest;
- unsigned long int srcp = (long int) src;
-
- /* If there is no overlap between ranges, call the builtin memcpy. */
- if (dstp >= srcp + len || srcp > dstp + len)
- __builtin_memcpy (dest, src, len);
-
- /* This test makes the forward copying code be used whenever possible.
- Reduces the working set. */
- else if (dstp - srcp >= len) /* *Unsigned* compare! */
- {
- /* Copy from the beginning to the end. */
-
- /* If there not too few bytes to copy, use word copy. */
- if (len >= OP_T_THRES)
- {
- /* Copy just a few bytes to make DSTP aligned. */
- len -= (-dstp) % OPSIZ;
- BYTE_COPY_FWD (dstp, srcp, (-dstp) % OPSIZ);
-
- /* Copy whole pages from SRCP to DSTP by virtual address
- manipulation, as much as possible. */
-
- PAGE_COPY_FWD_MAYBE (dstp, srcp, len, len);
-
- /* Copy from SRCP to DSTP taking advantage of the known
- alignment of DSTP. Number of bytes remaining is put
- in the third argument, i.e. in LEN. This number may
- vary from machine to machine. */
-
- WORD_COPY_FWD (dstp, srcp, len, len);
-
- /* Fall out and copy the tail. */
- }
-
- /* There are just a few bytes to copy. Use byte memory operations. */
- BYTE_COPY_FWD (dstp, srcp, len);
- }
- else
- {
- /* Copy from the end to the beginning. */
- srcp += len;
- dstp += len;
-
- /* If there not too few bytes to copy, use word copy. */
- if (len >= OP_T_THRES)
- {
- /* Copy just a few bytes to make DSTP aligned. */
- len -= dstp % OPSIZ;
- BYTE_COPY_BWD (dstp, srcp, dstp % OPSIZ);
-
- /* Copy from SRCP to DSTP taking advantage of the known
- alignment of DSTP. Number of bytes remaining is put
- in the third argument, i.e. in LEN. This number may
- vary from machine to machine. */
-
- WORD_COPY_BWD (dstp, srcp, len, len);
-
- /* Fall out and copy the tail. */
- }
-
- /* There are just a few bytes to copy. Use byte memory operations. */
- BYTE_COPY_BWD (dstp, srcp, len);
- }
-
- RETURN (dest);
-}
-#ifndef memmove
-libc_hidden_builtin_def (memmove)
-#endif
+#define MEMCPY_OK_FOR_FWD_MEMMOVE 1
+#include <string/memmove.c>
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=9841a0850ed3be4310ec6b49c3349e39a6f0f481
commit 9841a0850ed3be4310ec6b49c3349e39a6f0f481
Author: Vidya Ranganathan <vidya@linux.vnet.ibm.com>
Date: Wed Jun 11 22:21:20 2014 -0500
PowerPC: strcat optimization for PPC64/POWER7
This patch adds an ifunc power7 strcat symbol that uses the logic on
sysdeps/powerpc/strcat.c but call power7 strlen/strcpy symbols instead
of default ones.
This is a backport of bc8ea38590070604006399e42469087e943fc8ec.
diff --git a/ChangeLog b/ChangeLog
index 39d09d7..54db831 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2014-07-02 Vidya Ranganathan <vidya@linux.vnet.ibm.com>
+ Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+
+ * sysdeps/powerpc/strcat.c: Using macro to redefine symbol name.
+ * sysdeps/powerpc/powerpc64/multiarch/Makefile: Add strcat multiarch
+ optimizations.
+ * sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c:
+ (__libc_ifunc_impl_list): Likewise.
+ * sysdeps/powerpc/powerpc64/multiarch/strcat.c: New file:
+ multiarch strcat for PPC64.
+ * sysdeps/powerpc/powerpc64/multiarch/strcat-ppc64.c: New file/
+ * sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c: New file.
+
2014-06-23 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
* sysdeps/powerpc/bits/hwcap.h [PPC_FEATURE2_HAS_VEC_CRYPTO]: New
diff --git a/sysdeps/powerpc/powerpc64/multiarch/Makefile b/sysdeps/powerpc/powerpc64/multiarch/Makefile
index 05744e9..524055f 100644
--- a/sysdeps/powerpc/powerpc64/multiarch/Makefile
+++ b/sysdeps/powerpc/powerpc64/multiarch/Makefile
@@ -17,7 +17,8 @@ sysdep_routines += memcpy-power7 memcpy-a2 memcpy-power6 memcpy-cell \
strrchr-power7 strrchr-ppc64 strncat-power7 strncat-ppc64 \
strspn-power7 strspn-ppc64 strcspn-power7 strcspn-ppc64 \
strpbrk-power7 strpbrk-ppc64 strncpy-power7 strncpy-ppc64 \
- stpncpy-power7 stpncpy-ppc64 strcmp-power7 strcmp-ppc64
+ stpncpy-power7 stpncpy-ppc64 strcmp-power7 strcmp-ppc64 \
+ strcat-power7 strcat-ppc64
CFLAGS-strncase-power7.c += -mcpu=power7 -funroll-loops
CFLAGS-strncase_l-power7.c += -mcpu=power7 -funroll-loops
diff --git a/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c b/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c
index b3933a5..ec2e027 100644
--- a/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c
@@ -301,5 +301,14 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__strcmp_power7)
IFUNC_IMPL_ADD (array, i, strcmp, 1,
__strcmp_ppc))
+
+ /* Support sysdeps/powerpc/powerpc64/multiarch/strcat.c. */
+ IFUNC_IMPL (i, name, strcat,
+ IFUNC_IMPL_ADD (array, i, strcat,
+ hwcap & PPC_FEATURE_HAS_VSX,
+ __strcat_power7)
+ IFUNC_IMPL_ADD (array, i, strcat, 1,
+ __strcat_ppc))
+
return i;
}
diff --git a/sysdeps/powerpc/strcat.c b/sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c
similarity index 61%
copy from sysdeps/powerpc/strcat.c
copy to sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c
index 06ceca7..ba9a460 100644
--- a/sysdeps/powerpc/strcat.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c
@@ -1,5 +1,4 @@
-/* strcat version that uses fast strcpy/strlen.
- Copyright (C) 1997-2014 Free Software Foundation, Inc.
+/* Copyright (C) 2014 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -9,22 +8,21 @@
The GNU C Library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
+ <http://www.gnu.org/licenses/ >. */
#include <string.h>
-#undef strcat
+#define STRCAT __strcat_power7
-/* Append SRC on the end of DEST. */
-char *
-strcat (char *dest, const char *src)
-{
- strcpy (dest + strlen (dest), src);
- return dest;
-}
-libc_hidden_builtin_def (strcat)
+#undef libc_hidden_def
+#define libc_hidden_def(name)
+
+#define strcpy __strcpy_power7
+#define strlen __strlen_power7
+
+#include <sysdeps/powerpc/strcat.c>
diff --git a/sysdeps/powerpc/strcat.c b/sysdeps/powerpc/powerpc64/multiarch/strcat-ppc64.c
similarity index 59%
copy from sysdeps/powerpc/strcat.c
copy to sysdeps/powerpc/powerpc64/multiarch/strcat-ppc64.c
index 06ceca7..1245764 100644
--- a/sysdeps/powerpc/strcat.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/strcat-ppc64.c
@@ -1,5 +1,4 @@
-/* strcat version that uses fast strcpy/strlen.
- Copyright (C) 1997-2014 Free Software Foundation, Inc.
+/* Copyright (C) 2014 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -9,22 +8,22 @@
The GNU C Library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
+ <http://www.gnu.org/licenses/ >. */
#include <string.h>
-#undef strcat
+#define STRCAT __strcat_ppc
+#ifdef SHARED
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strcat_ppc, __GI_strcat, __strcat_ppc);
+#endif
-/* Append SRC on the end of DEST. */
-char *
-strcat (char *dest, const char *src)
-{
- strcpy (dest + strlen (dest), src);
- return dest;
-}
-libc_hidden_builtin_def (strcat)
+extern __typeof (strcat) __strcat_ppc attribute_hidden;
+
+#include <sysdeps/powerpc/strcat.c>
diff --git a/sysdeps/powerpc/strcat.c b/sysdeps/powerpc/powerpc64/multiarch/strcat.c
similarity index 62%
copy from sysdeps/powerpc/strcat.c
copy to sysdeps/powerpc/powerpc64/multiarch/strcat.c
index 06ceca7..847a62d 100644
--- a/sysdeps/powerpc/strcat.c
+++ b/sysdeps/powerpc/powerpc64/multiarch/strcat.c
@@ -1,5 +1,5 @@
-/* strcat version that uses fast strcpy/strlen.
- Copyright (C) 1997-2014 Free Software Foundation, Inc.
+/* Multiple versions of strcat. PowerPC64 version.
+ Copyright (C) 2014 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
@@ -16,15 +16,16 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
-#include <string.h>
+#ifndef NOT_IN_libc
+# include <string.h>
+# include <shlib-compat.h>
+# include "init-arch.h"
-#undef strcat
+extern __typeof (strcat) __strcat_ppc attribute_hidden;
+extern __typeof (strcat) __strcat_power7 attribute_hidden;
-/* Append SRC on the end of DEST. */
-char *
-strcat (char *dest, const char *src)
-{
- strcpy (dest + strlen (dest), src);
- return dest;
-}
-libc_hidden_builtin_def (strcat)
+libc_ifunc (strcat,
+ (hwcap & PPC_FEATURE_HAS_VSX)
+ ? __strcat_power7
+ : __strcat_ppc);
+#endif
diff --git a/sysdeps/powerpc/strcat.c b/sysdeps/powerpc/strcat.c
index 06ceca7..4ff37ea 100644
--- a/sysdeps/powerpc/strcat.c
+++ b/sysdeps/powerpc/strcat.c
@@ -18,13 +18,16 @@
#include <string.h>
-#undef strcat
+#ifndef STRCAT
+# undef strcat
+# define STRCAT strcat
+#endif
/* Append SRC on the end of DEST. */
char *
-strcat (char *dest, const char *src)
+STRCAT(char *dest, const char *src)
{
strcpy (dest + strlen (dest), src);
return dest;
}
-libc_hidden_builtin_def (strcat)
+libc_hidden_builtin_def (STRCAT)
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=ded8852b37f673b8e66163b44f70504dc5af0985
commit ded8852b37f673b8e66163b44f70504dc5af0985
Author: Adhemerval Zanella <azanella@linux.vnet.ibm.com>
Date: Mon Jun 23 09:38:47 2014 -0500
PowerPC: sync hwcap.h capabilities
Linux commit dd58a092c4202f2bd490adab7285b3ff77f8e467 added the
PPC_FEATURE2_VEC_CRYPTO auvx capability to indicate whether to
hardware supports vector crypto hardware instructions. This patch
adds its definition to powerpc hwcap bits.
This is a backport of db22400947e1c82153e5270d23fed53fc1e3a659.
diff --git a/ChangeLog b/ChangeLog
index 95dcce8..39d09d7 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2014-06-23 Adhemerval Zanella <azanella@linux.vnet.ibm.com>
+
+ * sysdeps/powerpc/bits/hwcap.h [PPC_FEATURE2_HAS_VEC_CRYPTO]: New
+ macro: hardware supports Vector Crypto instructions.
+
2014-16-17 Rajalakshmi Srinivasaraghavan <raji@linux.vnet.ibm.com>
[BZ #17031]
diff --git a/sysdeps/powerpc/bits/hwcap.h b/sysdeps/powerpc/bits/hwcap.h
index 1af8c82..dab1a58 100644
--- a/sysdeps/powerpc/bits/hwcap.h
+++ b/sysdeps/powerpc/bits/hwcap.h
@@ -62,3 +62,5 @@
#define PPC_FEATURE2_HAS_EBB 0x10000000 /* Event Base Branching */
#define PPC_FEATURE2_HAS_ISEL 0x08000000 /* Integer Select */
#define PPC_FEATURE2_HAS_TAR 0x04000000 /* Target Address Register */
+#define PPC_FEATURE2_HAS_VEC_CRYPTO 0x02000000 /* Target supports vector
+ instruction. */
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=7e986751f5c05f3363c01c717972f87a681da0d0
commit 7e986751f5c05f3363c01c717972f87a681da0d0
Author: Rajalakshmi Srinivasaraghavan <raji@linux.vnet.ibm.com>
Date: Tue Jun 17 08:46:25 2014 -0500
PowerPC: Fix nearbyintl failure for few inputs
This patch fixes few failures in nearbyintl() where the fraction part is
close to 0.5.i The new tests added report few extra failures in
nearbyint_downward and nearbyint_towardzero which is a known issue.
Fixes #17031.
This is a backport of 754c5a08aacb44895d1ab97c553ce424eb43f761.
diff --git a/ChangeLog b/ChangeLog
index a8a24b5..95dcce8 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2014-16-17 Rajalakshmi Srinivasaraghavan <raji@linux.vnet.ibm.com>
+
+ [BZ #17031]
+ * sysdeps/ieee754/ldbl-128ibm/s_nearbyintl.c: Consider the low
+ double, adjusted for any remainder from the high double.
+ * math/libm-test.inc (nearbyint): Add tests.
+ (rint): Likewise.
+
2014-06-11 Vidya Ranganathan <vidya@linux.vnet.ibm.com>
* sysdeps/powerpc/powerpc64/power7/strcmp.S: New file: Optimization.
diff --git a/NEWS b/NEWS
index 99bda3c..96b7621 100644
--- a/NEWS
+++ b/NEWS
@@ -9,7 +9,8 @@ Version 2.19.1
* The following bugs are resolved with this release:
- 16545, 16683, 16689, 16701, 16706, 16707, 16739, 16815, 16619, 16740.
+ 16545, 16683, 16689, 16701, 16706, 16707, 16739, 16815, 16619, 16740,
+ 17031.
Version 2.19
diff --git a/math/libm-test.inc b/math/libm-test.inc
index 1c5c022..848b89e 100644
--- a/math/libm-test.inc
+++ b/math/libm-test.inc
@@ -10519,6 +10519,10 @@ static const struct test_f_f_data nearbyint_test_data[] =
TEST_f_f (nearbyint, 34503599627370498.515625L, 34503599627370499.0L),
TEST_f_f (nearbyint, -34503599627370498.515625L, -34503599627370499.0L),
# if LDBL_MANT_DIG >= 106
+ TEST_f_f (nearbyint, 1024.5000000000001L, 1025.0L, NO_INEXACT_EXCEPTION),
+ TEST_f_f (nearbyint, 1025.5000000000001L, 1026.0L, NO_INEXACT_EXCEPTION),
+ TEST_f_f (nearbyint, -1024.5000000000001L, -1025.0L, NO_INEXACT_EXCEPTION),
+ TEST_f_f (nearbyint, -1025.5000000000001L, -1026.0L, NO_INEXACT_EXCEPTION),
TEST_f_f (nearbyint, 1192568192774434123539907640624.484375L, 1192568192774434123539907640624.0L),
TEST_f_f (nearbyint, -1192568192774434123539907640624.484375L, -1192568192774434123539907640624.0L),
# endif
@@ -11335,6 +11339,10 @@ static const struct test_f_f_data rint_test_data[] =
TEST_f_f (rint, 4503599627370497.5L, 4503599627370498.0L, INEXACT_EXCEPTION),
# if LDBL_MANT_DIG > 100
+ TEST_f_f (rint, 1024.5000000000001L, 1025.0L, INEXACT_EXCEPTION),
+ TEST_f_f (rint, 1025.5000000000001L, 1026.0L, INEXACT_EXCEPTION),
+ TEST_f_f (rint, -1024.5000000000001L, -1025.0L, INEXACT_EXCEPTION),
+ TEST_f_f (rint, -1025.5000000000001L, -1026.0L, INEXACT_EXCEPTION),
TEST_f_f (rint, 4503599627370494.5000000000001L, 4503599627370495.0L, INEXACT_EXCEPTION),
TEST_f_f (rint, 4503599627370495.5000000000001L, 4503599627370496.0L, INEXACT_EXCEPTION),
TEST_f_f (rint, 4503599627370496.5000000000001L, 4503599627370497.0L, INEXACT_EXCEPTION),
diff --git a/sysdeps/ieee754/ldbl-128ibm/s_nearbyintl.c b/sysdeps/ieee754/ldbl-128ibm/s_nearbyintl.c
index 4e997a6..8f34604 100644
--- a/sysdeps/ieee754/ldbl-128ibm/s_nearbyintl.c
+++ b/sysdeps/ieee754/ldbl-128ibm/s_nearbyintl.c
@@ -38,6 +38,7 @@ __nearbyintl (long double x)
if (fabs (u.d[0].d) < TWO52)
{
+ double xh = u.d[0].d;
double high = u.d[0].d;
feholdexcept (&env);
if (high > 0.0)
@@ -52,6 +53,10 @@ __nearbyintl (long double x)
high += TWO52;
if (high == 0.0) high = -0.0;
}
+ if (u.d[1].d > 0.0 && (xh - high == 0.5))
+ high += 1.0;
+ else if (u.d[1].d < 0.0 && (-(xh - high) == 0.5))
+ high -= 1.0;
u.d[0].d = high;
u.d[1].d = 0.0;
math_force_eval (u.d[0]);
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 98 +++
NEWS | 3 +-
elf/elf.h | 2 +-
math/libm-test.inc | 8 +
string/bcopy.c | 2 +-
sysdeps/ieee754/ldbl-128ibm/s_nearbyintl.c | 5 +
sysdeps/powerpc/bits/hwcap.h | 2 +
sysdeps/powerpc/memmove.c | 118 ---
sysdeps/powerpc/powerpc32/power4/memcopy.h | 4 +
.../powerpc/powerpc32/power4/multiarch/Makefile | 2 +-
.../powerpc32/power4/multiarch/ifunc-impl-list.c | 6 +
.../powerpc32/power4/multiarch/memmove-power7.c | 30 +
.../powerpc32/power4/multiarch/memmove-ppc.c | 33 +
.../powerpc/powerpc32/power4/multiarch/memmove.c | 34 +
.../powerpc32/power4/multiarch/strcasecmp-power7.S | 2 +-
.../power4/multiarch/strcasecmp_l-power7.S | 2 +-
sysdeps/powerpc/powerpc32/power7/memcpy.S | 4 +-
sysdeps/powerpc/powerpc64/entry.h | 2 +
sysdeps/powerpc/powerpc64/multiarch/Makefile | 4 +-
sysdeps/powerpc/powerpc64/multiarch/bcopy-ppc64.c | 25 +
sysdeps/powerpc/powerpc64/multiarch/bcopy.c | 29 +
.../powerpc/powerpc64/multiarch/ifunc-impl-list.c | 21 +
.../powerpc/powerpc64/multiarch/memmove-power7.S | 43 +
.../powerpc/powerpc64/multiarch/memmove-ppc64.c | 28 +
sysdeps/powerpc/powerpc64/multiarch/memmove.c | 45 ++
.../powerpc/powerpc64/multiarch/strcat-power7.c | 30 +
sysdeps/powerpc/powerpc64/multiarch/strcat-ppc64.c | 29 +
sysdeps/powerpc/powerpc64/multiarch/strcat.c | 31 +
sysdeps/powerpc/powerpc64/power7/bcopy.c | 1 +
sysdeps/powerpc/powerpc64/power7/memcpy.S | 22 +-
sysdeps/powerpc/powerpc64/power7/memmove.S | 831 ++++++++++++++++++++
sysdeps/powerpc/strcat.c | 9 +-
32 files changed, 1357 insertions(+), 148 deletions(-)
delete mode 100644 sysdeps/powerpc/memmove.c
create mode 100644 sysdeps/powerpc/powerpc32/power4/multiarch/memmove-power7.c
create mode 100644 sysdeps/powerpc/powerpc32/power4/multiarch/memmove-ppc.c
create mode 100644 sysdeps/powerpc/powerpc32/power4/multiarch/memmove.c
create mode 100644 sysdeps/powerpc/powerpc64/multiarch/bcopy-ppc64.c
create mode 100644 sysdeps/powerpc/powerpc64/multiarch/bcopy.c
create mode 100644 sysdeps/powerpc/powerpc64/multiarch/memmove-power7.S
create mode 100644 sysdeps/powerpc/powerpc64/multiarch/memmove-ppc64.c
create mode 100644 sysdeps/powerpc/powerpc64/multiarch/memmove.c
create mode 100644 sysdeps/powerpc/powerpc64/multiarch/strcat-power7.c
create mode 100644 sysdeps/powerpc/powerpc64/multiarch/strcat-ppc64.c
create mode 100644 sysdeps/powerpc/powerpc64/multiarch/strcat.c
create mode 100644 sysdeps/powerpc/powerpc64/power7/bcopy.c
create mode 100644 sysdeps/powerpc/powerpc64/power7/memmove.S
hooks/post-receive
--
GNU C Library master sources