This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [PATCH] [RFC] malloc: Reduce worst-case behaviour with madvise and refault overhead


On 02/09/2015 05:49 PM, Mel Gorman wrote:
> I would also welcome suggestions on how madvise could be throttled without
> the use of counters. The counters are heap-local where I do not expect
> there will be cache conflicts and the allocation-side counter is only
> updated after a recent heap shrink to minimise updates.
> 
> Initially I worked around this in the kernel but any solution there
> breaks the existing semantics of MADV_DONTNEED and was rejected. See
> last paragraph of https://lkml.org/lkml/2015/2/2/696 .

The truth is that glibc doesn't want to use MADV_DONTNEED in malloc,
but it's the only interface we have right now that has similar
semantics to what we need.

Similarly Kostya from google told me that ASAN also has this problem since 
it has tons of statistics pages that it will touch soon, but doesn't care
if they come back as zero or the original data.

Two years ago I spoke with Motohiro-san and we talked about
MADV_"Want but don't need", but no mainline solution was present at the
time.

The immediate work that comes to mind is the `vrange` work by Minchan Kim.
http://lwn.net/Articles/542504/

I agree with Rik's comment in the above link that we really want MADV_FREE
in these cases, and it gives a 200% speedup over MADV_DONTNEED (as reported
by testers using jemalloc patches).

Thus, instead of adding more complex heuristics to glibc's malloc I think
we should be testing the addition of MADV_FREE in glibc's malloc and then
supporting or adjusting the proposal for MADV_FREE or vrange dependent on
the outcome.

In the meantime we can talk about mitigating the problems in glibc's
allocator for systems with old kernels, but it should not be the primary
solution. In glibc we would conditionalize the changes against the first
kernel version that included MADV_FREE, and when the minimum supported
kernel version is higher than that we would remove the code in question.

My suggested next steps are:

(a) Test using kernel+MADV_FREE with a hacked glibc malloc that uses
    MADV_FREE, see how that performs, and inform upstream kernel.

(b) Continue discussion over rate-limiting MADV_DONTNEED as a temporary
    measure. Before we go any further here, please increase M_TRIM_THRESHOLD
    in ebizzy and see if that makes a difference? It should make a difference
    by increasing the threshold at which we trim back to the OS, both sbrk,
    and mmaps, and thus reduce the MADV_DONTNEED calls at the cost of increased
    memory pressure. Changing the default though is not a trivial thing, since
    it could lead to immediate OOM for existing applications that run close to
    the limit of RAM. Discussion and analysis will be required.

Comments?

Cheers,
Carlos.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]