Re: [PATCH][malloc] Avoid atomics in have_fastchunks

[Carlos O'Donell <carlos at redhat dot com>]

On 09/19/2017 04:39 PM, Wilco Dijkstra wrote:
> I agree we need some benchmarks and workloads. But it's not only about
> getting a few percent speedup. The current implementation is without a
> doubt ancient and extremely hard to understand. There are about 100
> if statements in the fast paths, 90% of which are completely redundant.
> It even manages to check for the exact same special case several times in
> a row (eg. mmap).
>
> The sort of questions my simple patch raises is another example of how
> complex and nontrivial it has become. So there are very solid reasons to
> look beyond the current implementation. I would find it hard to believe
> there aren't multiple more advanced implementations available that not
> only improve performance but have a significantly simpler and smaller
> codebase as well... 

You are looking at this from the wrong way around.

Changing allocators in your application is easy. Go grab any allocator
from the internet and LD_PRELOAD it. We go to great lengths to allow this.

What matters to *us* as glibc developers is not "What modern allocator
should we change to?" but "Why should we change to a different allocator?"
Since there will always be a new allocator to pick from. In fact we want
to be able to tell users which allocator may work better for their workload!

Eventually all allocators get crufty, and glibc overall has suffered from
a lack of investment and repair of technical debt. We have turned that
trend around in many subsystems of glibc. In 2013 [1] I argued we must own 
the malloc we have, whatever that means, and turn it into what we need, and 
that can include using newer allocators. Since then we have *really* cleaned
up malloc. You should have seen it in 2013!

The key turning point is providing objective engineering proof that
another allocator is better on a given workload for some measure of better
that includes performance, VmRSS, VmSize, and security.

I will continue to push DJ's trace/simulation to get it ready for upstream
integration as another way to further our engineering rigor in this area
of system benchmarking. The trace/simulation is allocator neutral, it's just
an API trace, and we've used it to compare glibc, jemalloc, and tcmalloc
across several workloads.

In summary:

1) We need more engineering rigor and objective data driven decisions before
   we can decide which allocator will really be the best system default
   allocator.

2) In the mean time we should own the existing glibc malloc and make any 
   changes we need to improve the situation incrementally.

3) We will continue to work diligently to ensure that users can interpose
   a new application-specific allocator without issue.

Your patch today contributes to (2). Thank you! :-)

-- 
Cheers,
Carlos.

[1] https://sourceware.org/ml/libc-alpha/2013-12/msg00343.html