Re: [PATCH v2] Add malloc micro benchmark

[Carlos O'Donell <carlos at redhat dot com>]

On 02/28/2018 09:01 AM, Wilco Dijkstra wrote:
> Ondřej Bílka wrote:
>   
>>> I think a heap-style allocator which does not segregate allocations
>>> of different sizes still has its place, and why not provide one in
>>> glibc?
>>>
>> That isn't case for any allocator and it is asking for trouble. You want
>> to avoid sitation where two big chunks couldn't be merged because of
>> tiny chunk between them.
> 
> Agreed, you always want to special case small blocks. I don't believe there is
> any advantage in using a single big heap.
> 
>> For larger size this representation is still problematic and you could
>> improve performance with another representation that also avoids
>> alignment problem by placing metadata elsewhere(for mine only 4 bytes are needed).
> 
> Larger sizes would be helped a lot once small blocks are dealt with separately.
> So I don't think we need complicated balanced binary trees when dealing with a
> small number of large blocks. You won't need an unsorted list either, large blocks
> can be merged in O(1) time.
> 
> There may be an advantage to place meta data elsewhere, for example it could make
> adding/removing/walking free lists much faster (spatial locality) or to make heap
> overflow attacks almost impossible,

I agree with many of the things you and Ondrej are proposing.

The outcome of our current discussions should be an incremental plan to go from
where we are today, to where we want to be tomorrow.

However, I do *not* believe it is a good plan to simply throw away the present
allocator and claim it should be replaced from scratch. We do not have that luxury
as a core project, we must remain answerable to our users.

A high-level concrete problem today with glibc's malloc, and the only problem being
reported by our users is that it consumes too much RSS. Solving that problem in the
abstract is what we should be looking at.

If we think that having multiple heaps for different sized objects is the way to
do this, then we should think about how to go down that path with an experiment.

Any cleanup we do before that is a win.

-- 
Cheers,
Carlos.