[Bug malloc/21731] Bad malloc heuristic result in 30-40x memory consumption

["adhemerval.zanella at linaro dot org" <sourceware-bugzilla at sourceware dot org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=21731

Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |adhemerval.zanella at linaro dot o
                   |                            |rg

--- Comment #1 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
The 'system bytes' field from malloc_stats reports the main arena size which is
size requested with 'sbrk'.  And since you are explicit 'leaking' memory, most
likely it requires to continually grow the main arena virtual memory due
fragmentation.  Keep in mind that although it does increases the maximum
virtual memory of the process, it does not actually increase the total RSS
usage.

Using the same example, if you intercept malloc_stat to print total RSS usage
using this naive approach:

---
$ cat intercept.c 
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>

void malloc_stats (void)
{
  //system ("cat /proc/self/status | grep -E '^Vm(RSS|Data)'");
  system ("cat /proc/self/status | grep -E '^VmRSS'");
}
$ gcc -Wall intercept.c -shared -o libintercept.so -fpic
$ LD_PRELOAD=./libintercept.so ./malloc
---

You will see that total RSS usage does not really change significantly.  For
instance, a run I am seeing:

---
$ LD_PRELOAD=./libintercept.so ./malloc
VmRSS:       708 kB
VmRSS:       744 kB
[...]
VmRSS:       860 kB
VmRSS:       696 kB
[...]
VmRSS:       708 kB
VmRSS:       796 kB
---

Also you can change the sbrk behavior and force mmap allocations for large
sizes on main arena using MALLOC_TOP_PAD_=0:

---
$ MALLOC_TOP_PAD_=0 ./malloc
Arena 0:
system bytes     =   16076800
in use bytes     =   15697952
Total (incl. mmap):
system bytes     =   57049088
in use bytes     =   56670240
max mmap regions =         10
max mmap bytes   =   41500672
[...]
Arena 0:
system bytes     =   16076800
in use bytes     =   15879792
Total (incl. mmap):
system bytes     =   83451904
in use bytes     =   83254896
max mmap regions =         12
max mmap bytes   =   67903488
---

Now, what do you mean by 'bad heuristic' and by 'memory consumption' in this
specific case?

-- 
You are receiving this mail because:
You are on the CC list for the bug.