Re: malloc: performance improvements and bugfixes

[Florian Weimer <fweimer at redhat dot com>]

On 01/26/2016 09:50 PM, Steven Munroe wrote:

> So why not fix emacs to stop doing this (purely evil behavior).
> 
> If they want to persist their internal state from session to session
> there are better ways. For example: https://sphde.github.io/

It's complicated, but not just for the technical challenges involved.
My warning to the Emacs developers that we are going to clean this up on
the glibc side was not universally well-received.

Some of the Emacs developers think they have a stop-gap solution for
future Emacs binaries: They are going to interpose their own malloc.  My
testing agrees; it should happen automatically once we remove
long-deprecated symbols from <malloc.h> because that will cause their
autoconf check to fail, triggering a switch to their built-in malloc.
(This will happen even before with deprecate and eventually remove
__malloc_get_state and __malloc_set_state from the API.)

For existing binaries with newer glibc, I think I found a way to quickly
rewrite the dumped heap in such a way that we do not have to add any
special cases to the hot paths in free/realloc.  (We don't even have to
use Siddhesh's corrupt arena code for that, although that could be an
option as well.)  The main challenge is whether we can obtain the
address of the first *allocated* chunk on the main arena.  Once we have
that, we can do the heap traversal.  I expect maybe a few dozen lines of
code for this in total, and the constraints on future malloc evolution
are minimal.  It's much better than maintaining two mallocs inside glibc.

I also know one reason why valgrind does not work with dumped Emacs
binaries, and it's quite straightforward to fix (it could easily be part
of the traversal to rewrite the heap, but we have to implement it twice,
once in glibc and once for the Emacs malloc implementation).  Let's hope
this is the only fix needed to get valgrind going.

In short, things aren't as bad as we thought they are, and we can
finally fix bug 6527 after we have implemented the traversal of the
dumped heap I mentioned above.

Florian