This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
[PATCH] Remove statisticaly unsound benchmarks
- From: OndÅej BÃlka <neleai at seznam dot cz>
- To: libc-alpha at sourceware dot org
- Date: Fri, 19 Apr 2013 14:46:02 +0200
- Subject: [PATCH] Remove statisticaly unsound benchmarks
Hi,
I found that testcase measures time. However its measurement is flawed
in severeal ways described below.
Problem number one. Minimum performance does not correspond to average
performance.
You can have implementations A and B where A has minimum 3 and average
100 while B has minimum 10 and average 20(even in ideal conditions).
Picking variant with smaller minimum is obviously flawed due of this
bias.
If we take aside this problem then second problem is lack of robustnes.
See http://en.wikipedia.org/wiki/Sample_maximum_and_minimum
In our conditions minimum will pick more random noise than signal.
An mean http://en.wikipedia.org/wiki/Sample_mean_and_sample_covariance
should be used instead.
However you may need to first filter outliers (context switch, minor page faults...)
Second problem is that measurements are done in tight loop. Even if
measurements were done correctly repeately calling function with same
arguments is also problematic.
It estimates performance when users repeately calls function with same
arguments and little else.
In experiments you must replicate real world conditions all much as
possible. By neglecting factors you introduce more and more errors into
estimates.
Here doing randomized tests capture more factors and will almost always
be closer to real world performance. If you need to choose if to trust
randomized or idealized benchmark an randomized is better bet.
Why should we keep, read this data when we know it is useless?
Another argument againist tight loops is following:
Assume you picked a set (say 5) of benchmarks. When you pick benchmarks
that cover cases encountered in real world and are as close as posible
maximizing performance will likely improve performance in real world.
However if you pick benchmarks that are distant and really do not cover
entire use case space then maximizing results on those benchmarks will
result in random change for real world.
Given these flaws any results that use them are unscientific and we
should remove them.
* debug/test-strcpy_chk.c (do_one_test, do_test): Remove benchmark.
* string/test-memccpy.c (do_one_test, do_test): Likewise.
* string/test-memccpy.c (do_one_test, do_test): Likewise.
* string/test-memchr.c (do_one_test, do_test): Likewise.
* string/test-memcmp.c (do_one_test, do_test): Likewise.
* string/test-memcpy.c (do_one_test, do_test): Likewise.
* string/test-memmem.c (do_one_test, do_test): Likewise.
* string/test-memmove.c (do_one_test, do_test): Likewise.
* string/test-memset.c (do_one_test, do_test): Likewise.
* string/test-rawmemchr.c (do_one_test, do_test): Likewise.
* string/test-strcasecmp.c (do_one_test, do_test): Likewise.
* string/test-strcasestr.c (do_one_test, do_test): Likewise.
* string/test-strcat.c (do_one_test, do_test): Likewise.
* string/test-strchr.c (do_one_test, do_test): Likewise.
* string/test-strcmp.c (do_one_test, do_test): Likewise.
* string/test-strcpy.c (do_one_test, do_test): Likewise.
* string/test-strlen.c (do_one_test, do_test): Likewise.
* string/test-strncasecmp.c (do_one_test, do_test): Likewise.
* string/test-strncat.c (do_one_test, do_test): Likewise.
* string/test-strncmp.c (do_one_test, do_test): Likewise.
* string/test-strncpy.c (do_one_test, do_test): Likewise.
* string/test-strnlen.c (do_one_test, do_test): Likewise.
* string/test-strpbrk.c (do_one_test, do_test): Likewise.
* string/test-strrchr.c (do_one_test, do_test): Likewise.
* string/test-strspn.c (do_one_test, do_test): Likewise.
* string/test-strstr.c (do_one_test, do_test): Likewise.
* string/test-string.h (do_one_test, do_test): Remove HP_TIMING_BEST.
---
debug/test-strcpy_chk.c | 24 ------------------------
string/test-memccpy.c | 21 ---------------------
string/test-memchr.c | 22 ----------------------
string/test-memcmp.c | 22 ----------------------
string/test-memcpy.c | 24 ------------------------
string/test-memmem.c | 30 ------------------------------
string/test-memmove.c | 28 ----------------------------
string/test-memset.c | 29 -----------------------------
string/test-rawmemchr.c | 24 ------------------------
string/test-strcasecmp.c | 24 ------------------------
string/test-strcasestr.c | 25 -------------------------
string/test-strcat.c | 25 -------------------------
string/test-strchr.c | 25 -------------------------
string/test-strcmp.c | 24 ------------------------
string/test-strcpy.c | 24 ------------------------
string/test-string.h | 10 ----------
string/test-strlen.c | 24 ------------------------
string/test-strncasecmp.c | 24 ------------------------
string/test-strncat.c | 24 ------------------------
string/test-strncmp.c | 35 -----------------------------------
string/test-strncpy.c | 24 ------------------------
string/test-strnlen.c | 24 ------------------------
string/test-strpbrk.c | 24 ------------------------
string/test-strrchr.c | 24 ------------------------
string/test-strspn.c | 24 ------------------------
string/test-strstr.c | 25 -------------------------
26 files changed, 0 insertions(+), 633 deletions(-)
diff --git a/debug/test-strcpy_chk.c b/debug/test-strcpy_chk.c
index 1c64c60..736ef3c 100644
--- a/debug/test-strcpy_chk.c
+++ b/debug/test-strcpy_chk.c
@@ -110,24 +110,6 @@ do_one_test (impl_t *impl, char *dst, const char *src,
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));;
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, dst, src, dlen);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -151,14 +133,8 @@ do_test (size_t align1, size_t align2, size_t len, size_t dlen, int max_char)
s1[i] = 32 + 23 * i % (max_char - 32);
s1[len] = 0;
- if (HP_TIMING_AVAIL && dlen > len)
- printf ("Length %4zd, alignment %2zd/%2zd:", len, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s2, s1, len, dlen);
-
- if (HP_TIMING_AVAIL && dlen > len)
- putchar ('\n');
}
static void
diff --git a/string/test-memccpy.c b/string/test-memccpy.c
index 577a82e..c193aba 100644
--- a/string/test-memccpy.c
+++ b/string/test-memccpy.c
@@ -75,23 +75,6 @@ do_one_test (impl_t *impl, void *dst, const void *src, int c, size_t len,
return;
}
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute__ ((unused));
- hp_timing_t stop __attribute__ ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, dst, src, c, n);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -122,14 +105,10 @@ do_test (size_t align1, size_t align2, int c, size_t len, size_t n,
for (i = len; i + align1 < page_size && i < len + 64; ++i)
s1[i] = 32 + 32 * i % (max_char - 32);
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, n %4zd, char %d, alignment %2zd/%2zd:", len, n, c, align1, align2);
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s2, s1, c, len, n);
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-memchr.c b/string/test-memchr.c
index dffc858..105b9ea 100644
--- a/string/test-memchr.c
+++ b/string/test-memchr.c
@@ -47,24 +47,6 @@ do_one_test (impl_t *impl, const char *s, int c, size_t n, char *exp_res)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s, c, n);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -97,14 +79,10 @@ do_test (size_t align, size_t pos, size_t len, int seek_char)
buf1[align + len] = seek_char;
}
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd:", pos, align);
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, (char *) (buf1 + align), seek_char, len, result);
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-memcmp.c b/string/test-memcmp.c
index 654dfa2..ae9be2d 100644
--- a/string/test-memcmp.c
+++ b/string/test-memcmp.c
@@ -100,24 +100,6 @@ do_one_test (impl_t *impl, const CHAR *s1, const CHAR *s2, size_t len,
{
if (check_result (impl, s1, s2, len, exp_result) < 0)
return;
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s1, s2, len);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -147,14 +129,10 @@ do_test (size_t align1, size_t align2, size_t len, int exp_result)
s2[len] = align2;
s2[len - 1] -= exp_result;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd/%2zd:", len, align1, align2);
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s1, s2, len, exp_result);
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-memcpy.c b/string/test-memcpy.c
index eb9c308..d121ef0 100644
--- a/string/test-memcpy.c
+++ b/string/test-memcpy.c
@@ -68,24 +68,6 @@ do_one_test (impl_t *impl, char *dst, const char *src,
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, dst, src, len);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -108,14 +90,8 @@ do_test (size_t align1, size_t align2, size_t len)
for (i = 0, j = 1; i < len; i++, j += 23)
s1[i] = j;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd/%2zd:", len, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s2, s1, len);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-memmem.c b/string/test-memmem.c
index fec8b55..d98ede7 100644
--- a/string/test-memmem.c
+++ b/string/test-memmem.c
@@ -82,24 +82,6 @@ do_one_test (impl_t *impl, const void *haystack, size_t haystack_len,
if (check_result (impl, haystack, haystack_len, needle, needle_len,
expected) < 0)
return;
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, haystack, haystack_len, needle, needle_len);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -110,16 +92,10 @@ do_test (const char *str, size_t len, size_t idx)
memcpy (tmpbuf, buf1 + idx, len);
memcpy (buf1 + idx, str, len);
- if (HP_TIMING_AVAIL)
- printf ("String %s, offset %zd:", str, idx);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, buf1, BUF1PAGES * page_size, str, len, buf1 + idx);
memcpy (buf1 + idx, tmpbuf, len);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
@@ -143,16 +119,10 @@ do_random_tests (void)
buf1[idx + off] = ch;
}
- if (HP_TIMING_AVAIL)
- printf ("String %.*s, offset %zd:", (int) len, buf1 + idx, idx);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, buf1, BUF1PAGES * page_size, buf1 + idx, len,
buf1 + idx);
- if (HP_TIMING_AVAIL)
- putchar ('\n');
-
memcpy (buf1 + idx, tmpbuf, len);
}
}
diff --git a/string/test-memmove.c b/string/test-memmove.c
index e47446c..4ec55b2 100644
--- a/string/test-memmove.c
+++ b/string/test-memmove.c
@@ -90,28 +90,6 @@ do_one_test (impl_t *impl, char *dst, char *src, const char *orig_src,
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
-#ifdef TEST_BCOPY
- CALL (impl, src, dst, len);
-#else
- CALL (impl, dst, src, len);
-#endif
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -134,14 +112,8 @@ do_test (size_t align1, size_t align2, size_t len)
for (i = 0, j = 1; i < len; i++, j += 23)
s1[i] = j;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd/%2zd:", len, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s2, (char *) (buf2 + align1), s1, len);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-memset.c b/string/test-memset.c
index 969772d..9981fce 100644
--- a/string/test-memset.c
+++ b/string/test-memset.c
@@ -91,29 +91,6 @@ do_one_test (impl_t *impl, char *s, int c __attribute ((unused)), size_t n)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
-#ifdef TEST_BZERO
- CALL (impl, s, n);
-#else
- CALL (impl, s, c, n);
-#endif
-
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -123,14 +100,8 @@ do_test (size_t align, int c, size_t len)
if (align + len > page_size)
return;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd, c %2d:", len, align, c);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, (char *) buf1 + align, c, len);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
#ifndef TEST_BZERO
diff --git a/string/test-rawmemchr.c b/string/test-rawmemchr.c
index ebfd678..9e4952f 100644
--- a/string/test-rawmemchr.c
+++ b/string/test-rawmemchr.c
@@ -49,24 +49,6 @@ do_one_test (impl_t *impl, const char *s, int c, char *exp_res)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s, c);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -93,14 +75,8 @@ do_test (size_t align, size_t pos, size_t len, int seek_char)
buf1[align + len] = -seek_char;
result = (char *) (buf1 + align + pos);
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd:", pos, align);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, (char *) (buf1 + align), seek_char, result);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strcasecmp.c b/string/test-strcasecmp.c
index ed49733..f321501 100644
--- a/string/test-strcasecmp.c
+++ b/string/test-strcasecmp.c
@@ -73,24 +73,6 @@ do_one_test (impl_t *impl, const char *s1, const char *s2, int exp_result)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s1, s2);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -129,14 +111,8 @@ do_test (size_t align1, size_t align2, size_t len, int max_char,
else
s2[len - 1] -= exp_result;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd/%2zd:", len, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s1, s2, exp_result);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strcasestr.c b/string/test-strcasestr.c
index 85e65fa..5b1d827 100644
--- a/string/test-strcasestr.c
+++ b/string/test-strcasestr.c
@@ -78,24 +78,6 @@ do_one_test (impl_t *impl, const char *s1, const char *s2, char *exp_result)
{
if (check_result (impl, s1, s2, exp_result) < 0)
return;
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~(hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s1, s2);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
@@ -135,15 +117,8 @@ do_test (size_t align1, size_t align2, size_t len1, size_t len2,
}
s1[len1] = '\0';
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd/%zd, alignment %2zd/%2zd, %s:",
- len1, len2, align1, align2, fail ? "fail" : "found");
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s1, s2, fail ? NULL : s1 + len1 - len2);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strcat.c b/string/test-strcat.c
index 7a47a4b..89fc90d 100644
--- a/string/test-strcat.c
+++ b/string/test-strcat.c
@@ -56,25 +56,6 @@ do_one_test (impl_t *impl, char *dst, const char *src)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- dst[k] = '\0';
- HP_TIMING_NOW (start);
- CALL (impl, dst, src);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -101,17 +82,11 @@ do_test (size_t align1, size_t align2, size_t len1, size_t len2, int max_char)
for (i = 0; i < len2; i++)
s2[i] = 32 + 23 * i % (max_char - 32);
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd/%4zd, alignment %2zd/%2zd:", len1, len2, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
{
s2[len2] = '\0';
do_one_test (impl, s2, s1);
}
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strchr.c b/string/test-strchr.c
index deca516..cbcf53e 100644
--- a/string/test-strchr.c
+++ b/string/test-strchr.c
@@ -107,24 +107,6 @@ do_one_test (impl_t *impl, const CHAR *s, int c, const CHAR *exp_res)
{
if (check_result (impl, s, c, exp_res) < 0)
return;
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s, c);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -160,15 +142,8 @@ do_test (size_t align, size_t pos, size_t len, int seek_char, int max_char)
else
result = NULLRET (buf + align + len);
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment in bytes %2zd:",
- pos, align * sizeof (CHAR));
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, buf + align, seek_char, result);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strcmp.c b/string/test-strcmp.c
index d9b69fa..f666993 100644
--- a/string/test-strcmp.c
+++ b/string/test-strcmp.c
@@ -161,24 +161,6 @@ do_one_test (impl_t *impl,
{
if (check_result (impl, s1, s2, exp_result) < 0)
return;
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s1, s2);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -214,14 +196,8 @@ do_test (size_t align1, size_t align2, size_t len, int max_char,
s2[len + 1] = 24 + exp_result;
s2[len - 1] -= exp_result;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd/%2zd:", len, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s1, s2, exp_result);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strcpy.c b/string/test-strcpy.c
index a48ccbc..d973267 100644
--- a/string/test-strcpy.c
+++ b/string/test-strcpy.c
@@ -92,24 +92,6 @@ do_one_test (impl_t *impl, CHAR *dst, const CHAR *src,
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));;
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, dst, src);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -135,14 +117,8 @@ do_test (size_t align1, size_t align2, size_t len, int max_char)
s1[i] = 32 + 23 * i % (max_char - 32);
s1[len] = 0;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignments in bytes %2zd/%2zd:", len, align1 * sizeof(CHAR), align2 * sizeof(CHAR));
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s2, s1, len);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-string.h b/string/test-string.h
index 47659d0..5b8f045 100644
--- a/string/test-string.h
+++ b/string/test-string.h
@@ -159,16 +159,6 @@ static impl_t *impl_array;
if (!notall || impl->test)
#endif
-#define HP_TIMING_BEST(best_time, start, end) \
- do \
- { \
- hp_timing_t tmptime; \
- HP_TIMING_DIFF (tmptime, start + _dl_hp_timing_overhead, end); \
- if (best_time > tmptime) \
- best_time = tmptime; \
- } \
- while (0)
-
#ifndef BUF1PAGES
# define BUF1PAGES 1
#endif
diff --git a/string/test-strlen.c b/string/test-strlen.c
index be2a7f9..f8b791d 100644
--- a/string/test-strlen.c
+++ b/string/test-strlen.c
@@ -72,24 +72,6 @@ do_one_test (impl_t *impl, const CHAR *s, size_t exp_len)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -107,14 +89,8 @@ do_test (size_t align, size_t len)
buf[align + i] = 1 + 11111 * i % MAX_CHAR;
buf[align + len] = 0;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd:", len, align);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, (CHAR *) (buf + align), len);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strncasecmp.c b/string/test-strncasecmp.c
index 3b39085..4eedd3d 100644
--- a/string/test-strncasecmp.c
+++ b/string/test-strncasecmp.c
@@ -94,24 +94,6 @@ do_one_test (impl_t *impl, const char *s1, const char *s2, size_t n,
{
if (check_result (impl, s1, s2, n, exp_result) < 0)
return;
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s1, s2, n);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -150,14 +132,8 @@ do_test (size_t align1, size_t align2, size_t n, size_t len, int max_char,
else
s2[len - 1] -= exp_result;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd/%2zd:", len, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s1, s2, n, exp_result);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strncat.c b/string/test-strncat.c
index 41d409b..a54c0b6 100644
--- a/string/test-strncat.c
+++ b/string/test-strncat.c
@@ -67,24 +67,7 @@ do_one_test (impl_t *impl, char *dst, const char *src, size_t n)
ret = 1;
return;
}
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- dst[k] = '\0';
- HP_TIMING_NOW (start);
- CALL (impl, dst, src, n);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -114,18 +97,11 @@ do_test (size_t align1, size_t align2, size_t len1, size_t len2,
for (i = 0; i < len2; i++)
s2[i] = 32 + 23 * i % (max_char - 32);
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd/%4zd, alignment %2zd/%2zd, N %4zd:",
- len1, len2, align1, align2, n);
-
FOR_EACH_IMPL (impl, 0)
{
s2[len2] = '\0';
do_one_test (impl, s2, s1, n);
}
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strncmp.c b/string/test-strncmp.c
index 74b0dd4..47c33e3 100644
--- a/string/test-strncmp.c
+++ b/string/test-strncmp.c
@@ -75,24 +75,6 @@ do_one_test (impl_t *impl, const char *s1, const char *s2, size_t n,
{
if (check_result (impl, s1, s2, n, exp_result) < 0)
return;
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s1, s2, n);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -106,15 +88,10 @@ do_test_limit (size_t align1, size_t align2, size_t len, size_t n, int max_char,
{
s1 = (char*)(buf1 + page_size);
s2 = (char*)(buf2 + page_size);
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd/%4zd:", len, n);
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s1, s2, n, 0);
- if (HP_TIMING_AVAIL)
- putchar ('\n');
-
return;
}
@@ -144,14 +121,8 @@ do_test_limit (size_t align1, size_t align2, size_t len, size_t n, int max_char,
s1[len] = 64;
}
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd/%4zd, alignment %2zd/%2zd:", len, n, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s1, s2, n, exp_result);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
@@ -189,14 +160,8 @@ do_test (size_t align1, size_t align2, size_t len, size_t n, int max_char,
if (len >= n)
s2[n - 1] -= exp_result;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd/%4zd, alignment %2zd/%2zd:", len, n, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, (char*)s1, (char*)s2, n, exp_result);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strncpy.c b/string/test-strncpy.c
index 107daad..ac0e000 100644
--- a/string/test-strncpy.c
+++ b/string/test-strncpy.c
@@ -90,24 +90,6 @@ do_one_test (impl_t *impl, char *dst, const char *src, size_t len, size_t n)
return;
}
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute__ ((unused));
- hp_timing_t stop __attribute__ ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, dst, src, n);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -133,14 +115,8 @@ do_test (size_t align1, size_t align2, size_t len, size_t n, int max_char)
for (i = len + 1; i + align1 < page_size && i < len + 64; ++i)
s1[i] = 32 + 32 * i % (max_char - 32);
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, n %4zd, alignment %2zd/%2zd:", len, n, align1, align2);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s2, s1, len, n);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strnlen.c b/string/test-strnlen.c
index 27411fb..bc7459b 100644
--- a/string/test-strnlen.c
+++ b/string/test-strnlen.c
@@ -47,24 +47,6 @@ do_one_test (impl_t *impl, const char *s, size_t maxlen, size_t exp_len)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s, maxlen);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -80,14 +62,8 @@ do_test (size_t align, size_t len, size_t maxlen, int max_char)
buf1[align + i] = 1 + 7 * i % max_char;
buf1[align + len] = 0;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd:", len, align);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, (char *) (buf1 + align), maxlen, MIN (len, maxlen));
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strpbrk.c b/string/test-strpbrk.c
index d580bac..9f1b4a7 100644
--- a/string/test-strpbrk.c
+++ b/string/test-strpbrk.c
@@ -70,24 +70,6 @@ do_one_test (impl_t *impl, const char *s, const char *rej, RES_TYPE exp_res)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s, rej);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -137,14 +119,8 @@ do_test (size_t align, size_t pos, size_t len)
}
result = STRPBRK_RESULT (s, pos);
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd, rej len %2zd:", pos, align, len);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s, rej, result);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strrchr.c b/string/test-strrchr.c
index 984561d..f5afe75 100644
--- a/string/test-strrchr.c
+++ b/string/test-strrchr.c
@@ -73,24 +73,6 @@ do_one_test (impl_t *impl, const CHAR *s, int c, CHAR *exp_res)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s, c);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -129,14 +111,8 @@ do_test (size_t align, size_t pos, size_t len, int seek_char, int max_char)
else
result = NULL;
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment in bytes %2zd:", pos, align * sizeof(CHAR));
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, (CHAR *) (buf + align), seek_char, result);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strspn.c b/string/test-strspn.c
index 92d0191..4943faa 100644
--- a/string/test-strspn.c
+++ b/string/test-strspn.c
@@ -74,24 +74,6 @@ do_one_test (impl_t *impl, const char *s, const char *acc, size_t exp_res)
ret = 1;
return;
}
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~ (hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s, acc);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
static void
@@ -129,14 +111,8 @@ do_test (size_t align, size_t pos, size_t len)
s[i] = '\0';
}
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd, alignment %2zd, acc len %2zd:", pos, align, len);
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s, acc, pos);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
diff --git a/string/test-strstr.c b/string/test-strstr.c
index e788be9..57c4621 100644
--- a/string/test-strstr.c
+++ b/string/test-strstr.c
@@ -77,24 +77,6 @@ do_one_test (impl_t *impl, const char *s1, const char *s2, char *exp_result)
{
if (check_result (impl, s1, s2, exp_result) < 0)
return;
-
- if (HP_TIMING_AVAIL)
- {
- hp_timing_t start __attribute ((unused));
- hp_timing_t stop __attribute ((unused));
- hp_timing_t best_time = ~(hp_timing_t) 0;
- size_t i;
-
- for (i = 0; i < 32; ++i)
- {
- HP_TIMING_NOW (start);
- CALL (impl, s1, s2);
- HP_TIMING_NOW (stop);
- HP_TIMING_BEST (best_time, start, stop);
- }
-
- printf ("\t%zd", (size_t) best_time);
- }
}
@@ -133,15 +115,8 @@ do_test (size_t align1, size_t align2, size_t len1, size_t len2,
}
s1[len1] = '\0';
- if (HP_TIMING_AVAIL)
- printf ("Length %4zd/%zd, alignment %2zd/%2zd, %s:",
- len1, len2, align1, align2, fail ? "fail" : "found");
-
FOR_EACH_IMPL (impl, 0)
do_one_test (impl, s1, s2, fail ? NULL : s1 + len1 - len2);
-
- if (HP_TIMING_AVAIL)
- putchar ('\n');
}
static void
--
1.7.4.4