This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [PATCH] Simple malloc benchtest.


On 21 December 2013 21:47, Frank Ch. Eigler <fche@redhat.com> wrote:
> Will Newton <will.newton@linaro.org> writes:
>
>> [...]  It looks like you are using uniformly distributed random
>> numbers for allocation sizes. This doesn't necessarily bear any
>> relation to what actual allocation sizes are used in a real
>> application [...]  This means we run through doing a long stream of
>> malloc and then a long stream of free. That is again not close to
>> application behaviour so I would recommend we interleave malloc and
>> free calls in order to introduce some stress on the allocator.
>
> Instead of making up ad-hoc microbenchmarks, how about tracing the
> malloc/free traffic of a real application or a dozen, and using an
> amalgam of such large traces to drive the measurements?  (Insert
> cache-invalidation between operations as indicated by e.g. cachegrind
> or timestamps.)

That is certainly something we could do. However the problem with that
approach is that you need to ensure that you have up to date traces
for modern applications (e.g. many benchmarks used in papers even to
day use quite obsolete software), that the traces are sufficiently
different (i.e. do not show any bias to particular types of
applications or specific applications - "you benchmark against MySQL
but not Postgres, no fair") and even then you are discounting locality
and some cache effects because you only run the allocator not the
application so its not actually a complete panacea (you may have made
the allocation run fast but it may actually slow down the
application). On top of that you have the problem that some complex
benchmarks will make it difficult to see the effect of a change e.g.
if you accidentally slow down large allocations, the overall benchmark
will get slower but it's not obvious why.

So in summary there is a lot of work and bikeshedding still to be had
in that discussion and we badly need some kind of reasonably simple
benchmarks today that people can use.

-- 
Will Newton
Toolchain Working Group, Linaro


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]