This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [PATCH] randomize benchtests
- From: Torvald Riegel <triegel at redhat dot com>
- To: OndÅej BÃlka <neleai at seznam dot cz>
- Cc: Siddhesh Poyarekar <siddhesh dot poyarekar at gmail dot com>, c at domone dot kolej dot mff dot cuni dot cz, GNU C Library <libc-alpha at sourceware dot org>
- Date: Fri, 17 May 2013 12:24:30 +0200
- Subject: Re: [PATCH] randomize benchtests
- References: <20130422120018 dot GA30323 at domone dot kolej dot mff dot cuni dot cz> <CAAHN_R1pHJLoS3iP7KrQMmA4gPLawvuRWoK8Xy13VPMBbyPk+Q at mail dot gmail dot com> <20130422125625 dot GA30639 at domone dot kolej dot mff dot cuni dot cz>
On Mon, 2013-04-22 at 14:56 +0200, OndÅej BÃlka wrote:
> On Mon, Apr 22, 2013 at 05:44:14PM +0530, Siddhesh Poyarekar wrote:
> > On 22 April 2013 17:30, OndÅej BÃlka <neleai@seznam.cz> wrote:
> > > + clock_gettime (CLOCK_PROCESS_CPUTIME_ID, &start);
> > > + for (k = 0; k < iters; k++)
> > > + {
> > > + i = rand_r (&seed)%NUM_SAMPLES;
> > > + BENCH_FUNC(i);
> > > + }
> > > + clock_gettime (CLOCK_PROCESS_CPUTIME_ID, &end);
> >
> > This is wrong. The interval also has the time taken to call rand_r.
> >
> This is not wrong. You are interested only on differences between
> implementations and adding same time from rand_r calls does not change
> that.
But if we should be changing the rand_r implementation in the future
(e.g., if we might be getting HW support for it on a certain
architecture), then this will lead to a difference in all our
performance numbers between the prior code revisions and the newer ones.
Remember that we eventually also want to find performance regressions.
I think that we should make the inner loops as reproducible as possible,
so this should either be using a custom pseudo RNG, or calibrate it
against a loop with just a rand_r call, or don't get the random numbers
in the loop. The latter might not really be an option too, because we'd
then need to read those from a precomputed random set in memory, which
might be even more of a distortion.