This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [patch] Make the mmap/brk threshold in malloc dynamic to improve performance


Hello,

> Mmaping to allocate memory is a loss if we are going to free it and
> reuse it, because we have to do this clearing unnecessarily.

This can be true or false -- if you are going to free a large chunk
and then re-allocate it hours later the clearing will not matter much
compared to the amount of memory saved (to be multiplied by the amount
of time during which the chunk was available to other threads or
applications).

> The real question Arjan and I have is what are the conditions under
> which allocating using mmap is a win?  Here's what I can think of:
> 
> * When we're running out of brk() space - when does this happen?

It will happen some time, and then it was "nice" to be able to fall
back to mmap().

> * When we don't ever touch most of the allocated memory

Well that would be almost the same with brk()..

> * When it will reduce fragmentation in the brk()'d memory

This one was and is the real issue, and was the original reason for
introducing mmap().  Note that fragmentation is always leading to at
least linear effects: the larger the amount of memory allocated, the
more memory is wasted by fragmentation.  So, the fact that we are now
having much more RAM and driving much larger working sets than before
does _not_ invalidate striving for less fragmentation.  In fact, the
opposite should be the case.

> It seems like having larger address spaces is obviating most reasons
> for using mmap.

I disagree, obviating address space limits affecting brk() was just a
very minor reason for using mmap().  The main reason was conserving
memory -- not a fixed amount but a proportion of the working set.

Making the mmap threshold dynamic sure seems like an interesting idea
to me.  However, the original patch posted by Arjan would IMHO only
quickly raise the threshold up to the maximum, and never decrease it
again.  Since mmap() only comes into play for large allocations, it
could well be worthwhile to track allocation times for these chunks
and adjust the threshold based on the tendency to _quickly_ free() and
re-allocate them.

I think the better quick fix would just be to raise the default mmap
threshold like it was done for the trim threshold some years ago.  The
default mmap and trim thresholds could also be set as a fraction of
available RAM (between sane limits of course).  Do we have
sysconf(_SC_PHYS_PAGES) available already at time of the initial
malloc() call?

Regards,
Wolfram.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]