Re: [PATCH] benchtests: Add strtok benchmark

[Will Newton <will dot newton at linaro dot org>]

On 11 November 2013 13:33, Rajalakshmi Srinivasaraghavan
<raji@linux.vnet.ibm.com> wrote:
> On 11/11/2013 05:16 PM, Will Newton wrote:
>>
>> On 11 November 2013 11:36, Rajalakshmi Srinivasaraghavan
>> <raji@linux.vnet.ibm.com> wrote:
>>>
>>> Hi All,
>>>
>>> This patch adds a benchtest for 'strtok'.
>>>
>>>          * benchtests/Makefile: Add bench-strtok.
>>>          * benchtests/bench-strtok.c: New file: strtok benchtest.
>>>          * benchtests/bench-strtok-ifunc.c: New file: strtok benchtest.
>>>
>>> Signed-off-by: Rajalakshmi Srinivasaraghavan <raji@linux.vnet.ibm.com>
>>> ---
>>>   benchtests/Makefile             |   2 +-
>>>   benchtests/bench-strtok-ifunc.c |  20 ++++
>>>   benchtests/bench-strtok.c       | 205
>>> ++++++++++++++++++++++++++++++++++++++++
>>>   3 files changed, 226 insertions(+), 1 deletion(-)
>>>   create mode 100644 benchtests/bench-strtok-ifunc.c
>>>   create mode 100644 benchtests/bench-strtok.c
>>
>> Thanks for doing this work! I have one question below...
>>
>>> diff --git a/benchtests/Makefile b/benchtests/Makefile
>>> index aec395a..dabad81 100644
>>> --- a/benchtests/Makefile
>>> +++ b/benchtests/Makefile
>>> @@ -28,7 +28,7 @@ string-bench := bcopy bzero memccpy memchr memcmp
>>> memcpy memmem memmove \
>>>                  mempcpy memset rawmemchr stpcpy stpncpy strcasecmp
>>> strcasestr \
>>>                  strcat strchr strchrnul strcmp strcpy strcspn strlen \
>>>                  strncasecmp strncat strncmp strncpy strnlen strpbrk
>>> strrchr \
>>> -               strspn strstr strcpy_chk stpcpy_chk memrchr
>>> +               strspn strstr strcpy_chk stpcpy_chk memrchr strtok
>>>   string-bench-ifunc := $(addsuffix -ifunc, $(string-bench))
>>>   string-bench-all := $(string-bench) $(string-bench-ifunc)
>>>
>>> diff --git a/benchtests/bench-strtok-ifunc.c
>>> b/benchtests/bench-strtok-ifunc.c
>>> new file mode 100644
>>> index 0000000..edcca9b
>>> --- /dev/null
>>> +++ b/benchtests/bench-strtok-ifunc.c
>>> @@ -0,0 +1,20 @@
>>> +/* Measure IFUNC implementations of strtok function.
>>> +   Copyright (C) 2013 Free Software Foundation, Inc.
>>> +   This file is part of the GNU C Library.
>>> +
>>> +   The GNU C Library is free software; you can redistribute it and/or
>>> +   modify it under the terms of the GNU Lesser General Public
>>> +   License as published by the Free Software Foundation; either
>>> +   version 2.1 of the License, or (at your option) any later version.
>>> +
>>> +   The GNU C Library is distributed in the hope that it will be useful,
>>> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>>> +   Lesser General Public License for more details.
>>> +
>>> +   You should have received a copy of the GNU Lesser General Public
>>> +   License along with the GNU C Library; if not, see
>>> +   <http://www.gnu.org/licenses/>.  */
>>> +
>>> +#define TEST_IFUNC 1
>>> +#include "bench-strtok.c"
>>> diff --git a/benchtests/bench-strtok.c b/benchtests/bench-strtok.c
>>> new file mode 100644
>>> index 0000000..bd937e5
>>> --- /dev/null
>>> +++ b/benchtests/bench-strtok.c
>>> @@ -0,0 +1,205 @@
>>> +/* Measure strtok functions.
>>> +   Copyright (C) 2013 Free Software Foundation, Inc.
>>> +   This file is part of the GNU C Library.
>>> +
>>> +   The GNU C Library is free software; you can redistribute it and/or
>>> +   modify it under the terms of the GNU Lesser General Public
>>> +   License as published by the Free Software Foundation; either
>>> +   version 2.1 of the License, or (at your option) any later version.
>>> +
>>> +   The GNU C Library is distributed in the hope that it will be useful,
>>> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>>> +   Lesser General Public License for more details.
>>> +
>>> +   You should have received a copy of the GNU Lesser General Public
>>> +   License along with the GNU C Library; if not, see
>>> +   <http://www.gnu.org/licenses/>.  */
>>> +
>>> +#define TEST_MAIN
>>> +#define TEST_NAME "strtok"
>>> +#include "bench-string.h"
>>> +
>>> +char *
>>> +simple_strtok (char *s1, char *s2)
>>> +{
>>> +  static char *saveptr;
>>> +  char *token;
>>> +  ssize_t i = 0, j = 0;
>>> +  int found = 0;
>>> +  size_t s2len = strlen (s2);
>>> +
>>> +  if (s1 == NULL)
>>> +    s1 = saveptr;
>>> +  if (s1 == NULL || *s1 == '\0')
>>> +    return NULL;
>>> +
>>> +  while (!found)
>>> +    {
>>> +      if (s1[i] == '\0')
>>> +       {
>>> +         saveptr = NULL;
>>> +         return NULL;
>>> +       }
>>> +      for (j = 0; j < s2len; j++)
>>> +       {
>>> +         if (s1[i] == s2[j])
>>> +           {
>>> +             i++;
>>> +             found = 0;
>>> +             break;
>>> +           }
>>> +         found = 1;
>>> +       }
>>> +    }
>>> +  token = s1 + i;
>>> +  i++;
>>> +  found = 0;
>>> +  while (!found)
>>> +    {
>>> +      if (s1[i] == '\0')
>>> +       {
>>> +         saveptr = NULL;
>>> +         return token;
>>> +       }
>>> +      for (j = 0; j < s2len; j++)
>>> +       {
>>> +         if (s1[i] == s2[j])
>>> +           {
>>> +             found = 1;
>>> +             break;
>>> +           }
>>> +       }
>>> +      i++;
>>> +    }
>>> +  s1[i - 1] = '\0';
>>> +  saveptr = s1 + i;
>>> +  return token;
>>> +}
>>> +
>>> +typedef char *(*proto_t) (const char *, const char *);
>>> +
>>> +IMPL (simple_strtok, 0)
>>> +IMPL (strtok, 1)
>>> +
>>> +static void
>>> +do_one_test (impl_t * impl, const char *s1, const char *s2)
>>> +{
>>> +  size_t i, iters = INNER_LOOP_ITERS;
>>> +  timing_t start, stop, cur;
>>> +  TIMING_NOW (start);
>>> +  for (i = 0; i < iters; ++i)
>>> +    {
>>> +      CALL (impl, s1, s2);
>>> +      CALL (impl, NULL, s2);
>>> +      CALL (impl, NULL, s2);
>>> +    }
>>> +  TIMING_NOW (stop);
>>> +
>>> +  TIMING_DIFF (cur, start, stop);
>>> +
>>> +  TIMING_PRINT_MEAN ((double) cur, (double) iters);
>>> +
>>> +}
>>> +
>>> +
>>> +static void
>>> +do_test (size_t align1, size_t align2, size_t len1, size_t len2, int
>>> fail)
>>> +{
>>> +  char *s2 = (char *) (buf2 + align2);
>>> +  static const char d[] = "1234567890abcdef";
>>> +#define dl (sizeof (d) - 1)
>>> +  char *ss2 = s2;
>>> +  for (size_t l = len2; l > 0; l = l > dl ? l - dl : 0)
>>> +    {
>>> +      size_t t = l > dl ? dl : l;
>>> +      ss2 = mempcpy (ss2, d, t);
>>> +    }
>>> +  s2[len2] = '\0';
>>> +
>>> +  printf ("Length %4zd/%zd, alignment %2zd/%2zd, %s:",
>>> +         len1, len2, align1, align2, fail ? "fail" : "found");
>>> +
>>> +  FOR_EACH_IMPL (impl, 0)
>>> +  {
>>> +    char *s1 = (char *) (buf1 + align1);
>>> +    if (fail)
>>> +      {
>>> +       char *ss1 = s1;
>>> +       for (size_t l = len1; l > 0; l = l > dl ? l - dl : 0)
>>> +         {
>>> +           size_t t = l > dl ? dl : l;
>>> +           memcpy (ss1, d, t);
>>> +           ++ss1[len2 > 7 ? 7 : len2 - 1];
>>> +           ss1 += t;
>>> +         }
>>> +      }
>>> +    else
>>> +      {
>>> +       memset (s1, '0', len1);
>>> +       memcpy (s1 + (len1 - len2) - 2, s2, len2);
>>> +       if ((len1 / len2) > 4)
>>> +         memcpy (s1 + (len1 - len2) - (3 * len2), s2, len2);
>>> +      }
>>> +    s1[len1] = '\0';
>>> +    do_one_test (impl, s1, s2);
>>> +  }
>>> +  putchar ('\n');
>>> +}
>>> +
>>> +static int
>>> +test_main (void)
>>> +{
>>> +  test_init ();
>>> +
>>> +  printf ("%23s", "");
>>> +  FOR_EACH_IMPL (impl, 0)
>>> +    printf ("\t%s", impl->name);
>>> +  putchar ('\n');
>>> +
>>> +  for (size_t klen = 2; klen < 32; ++klen)
>>> +    for (size_t hlen = 2 * klen; hlen < 16 * klen; hlen += klen)
>>> +      {
>>> +       do_test (0, 0, hlen, klen, 0);
>>> +       do_test (0, 0, hlen, klen, 1);
>>> +       do_test (0, 3, hlen, klen, 0);
>>> +       do_test (0, 3, hlen, klen, 1);
>>> +       do_test (0, 9, hlen, klen, 0);
>>> +       do_test (0, 9, hlen, klen, 1);
>>> +       do_test (0, 15, hlen, klen, 0);
>>> +       do_test (0, 15, hlen, klen, 1);
>>> +
>>> +       do_test (3, 0, hlen, klen, 0);
>>> +       do_test (3, 0, hlen, klen, 1);
>>> +       do_test (3, 3, hlen, klen, 0);
>>> +       do_test (3, 3, hlen, klen, 1);
>>> +       do_test (3, 9, hlen, klen, 0);
>>> +       do_test (3, 9, hlen, klen, 1);
>>> +       do_test (3, 15, hlen, klen, 0);
>>> +       do_test (3, 15, hlen, klen, 1);
>>> +
>>> +       do_test (9, 0, hlen, klen, 0);
>>> +       do_test (9, 0, hlen, klen, 1);
>>> +       do_test (9, 3, hlen, klen, 0);
>>> +       do_test (9, 3, hlen, klen, 1);
>>> +       do_test (9, 9, hlen, klen, 0);
>>> +       do_test (9, 9, hlen, klen, 1);
>>> +       do_test (9, 15, hlen, klen, 0);
>>> +       do_test (9, 15, hlen, klen, 1);
>>> +
>>> +       do_test (15, 0, hlen, klen, 0);
>>> +       do_test (15, 0, hlen, klen, 1);
>>> +       do_test (15, 3, hlen, klen, 0);
>>> +       do_test (15, 3, hlen, klen, 1);
>>> +       do_test (15, 9, hlen, klen, 0);
>>> +       do_test (15, 9, hlen, klen, 1);
>>> +       do_test (15, 15, hlen, klen, 0);
>>> +       do_test (15, 15, hlen, klen, 1);
>>> +      }
>>
>> How were these alignment values chosen? It seems a strange selection
>> (I know other string benchmarks seem to be afflicted by this as well).
>>
> I selected them from strstr benchmark.
> If you have a better selection, I can change this.

Unfortunately I don't have a ready made set of alignments that would
be good and it really requires some thought to come up with a good
representative set of alignments.

This benchmark is obviously much better than having no benchmark at
all but I would like to see the existing alignments used either
documented or improved if possible.

-- 
Will Newton
Toolchain Working Group, Linaro