Optimal kernel semantics for memory allocators and other users?

["Carlos O'Donell" <carlos at redhat dot com>]

Motohiro-san,

At the Linux Foundataion Collaboration Summit 2013 you
asked Roland and I several questions, but one of them
had to do with your recent VM related work.

The question itself was "What kind of kernel features
would allow an optimal implementation of malloc?" I'm
paraphrasing because I don't remember the exact question.
More specifically: "Do the semantics of vrange allow more 
optimal memory management in malloc?"

I've included Konstantin, since he jumped into the
conversation at lunch and had another useful case for
this kind of memory management. In his case asan needs
lots of memory for measuring usage information but
would rather the memory go away under memory pressure
instead of crashing the application with OOM.

I located the following "vrange" patch:
https://lkml.org/lkml/2013/3/12/105

The answer to your question is "Yes," and "No."

Yes, because vrange gives us "Want but don't need"
semantics that asan and malloc use. That is to say
that they would like the memory but don't need it
and the kernel can feel free to take it, and later
give back a zero page or the original page.

No, because the most optimal implementation would
have a return value for vrange, which would tell
the application if vrange was returning zero-filled
pages or the existing pages for the range.

This way if malloc pins memory again and gets back
a "zero-filled pages" result then it can avoid
zeroing memory in the case of a calloc() call.

The rest of the implementation looks good to me
and if it's faster than MADV_DONTNEED then it would
make sense to prototype a change to support vrange
and then benchmark the results with ebizzy before
and after.

Does this make sense?

Cheers,
Carlos.