Malloc Problem in glibc 2.2

[Andreas Jaeger <aj at suse dot de>]

Hi Wolfram,

looking at a problem with getrlimit/setrlimit, I noticed that malloc
in glibc 2.2 doesn't work as expected.  The appended program gives
with a glibc 2.1 system:

gromit:/tmp/pr:[0]$ ./rlimit 
rlim_cur 1240, rlim_max: 2147483647
malloc returned: (nil)

But the current glibc 2.2 version does:
$ LD_LIBRARY_PATH=. elf/ld-linux.so.2 /tmp/pr/rlimit
rlim_cur 1240, rlim_max: 2147483647
malloc returned: 0x40117008

Looking at the syscalls I notice the following IMO significant
difference:
glibc 2.1 uses:
write(1, "rlim_cur 1240, rlim_max: 2147483"..., 36rlim_cur 1240, rlim_max: 2147483647
) = 36
brk(0)                                  = 0x8049820
brk(0x804bf48)                          = 0x8049820
brk(0x804bf48)                          = 0x8049820
write(1, "malloc returned: (nil)\n", 23malloc returned: (nil)
) = 23
munmap(0x40014000, 4096)                = 0
_exit(23)                               = ?

and glibc 2.2 has:
write(1, "rlim_cur 1240, rlim_max: 2147483"..., 36rlim_cur 1240, rlim_max: 2147483647
) = 36
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4012e000
write(1, "malloc returned: 0x4012e000\n", 28malloc returned: 0x4012e000
) = 28
munmap(0x40018000, 4096)                = 0
_exit(28)                               = ?

Why is glibc 2.2 using mmap and glibc 2.1 not?

Trying to disable mmap with glibc 2.2 didn't work:
With glibc 2.1 I can do:
$ MALLOC_MMAP_THRESHOLD_=1000 ./rlimit 
rlim_cur 1240, rlim_max: 2147483647
malloc returned: 0x40015008

But 2.2 ignores the value completly:
gromit:/usr/src/test/glibc-2.2:[127]$ MALLOC_MMAP_THRESHOLD_=1000 ./rlimit
rlim_cur 1240, rlim_max: 2147483647
malloc returned: 0x4012e000
gromit:/usr/src/test/glibc-2.2:[28]$ MALLOC_MMAP_THRESHOLD_=10000 ./rlimit
rlim_cur 1240, rlim_max: 2147483647
malloc returned: 0x4012e000
gromit:/usr/src/test/glibc-2.2:[28]$ MALLOC_MMAP_THRESHOLD_=100000 ./rlimit
rlim_cur 1240, rlim_max: 2147483647
malloc returned: 0x4012e000

MALLOC_MMAP_MAX_=0 didn't work either:
with 2.1:
$ MALLOC_MMAP_THRESHOLD_=1000 MALLOC_MMAP_MAX_=0 ./rlimit 
rlim_cur 1240, rlim_max: 2147483647
malloc returned: (nil)

and with 2.2:
$ MALLOC_MMAP_THRESHOLD_=1000 MALLOC_MMAP_MAX_=0 ./rlimit
rlim_cur 1240, rlim_max: 2147483647
malloc returned: 0x4012e000

Do you have any idea what's broken here?  This is with Linux
2.4.0-test4, a current gcc (one or two weeks old) and the current
glibc sources.

Andreas

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#include <sys/resource.h>
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>

int
main (void)
{
  struct rlimit rlim;
  void *p;
  
  assert(-1 != getrlimit(RLIMIT_DATA, &rlim));
  rlim.rlim_cur = 1240;
  assert(-1 != setrlimit(RLIMIT_DATA, &rlim));
  /* Get it again: */
  assert(-1 != getrlimit(RLIMIT_DATA, &rlim));

  printf ("rlim_cur %ld, rlim_max: %ld\n", rlim.rlim_cur, rlim.rlim_max);
  
  p = malloc (1000);

  printf ("malloc returned: %p\n", p);
  
}

-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.inka.de