This is the mail archive of the libc-help@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Possible malloc/mmap bug?


On Fri, Nov 18, 2011 at 3:33 PM, Charles Hines <hines@cert.org> wrote:
>> Your application is a pathological case of VM fragmentation.
> But in a 64 bit operating system w/ the amount of virtual address space in the Exabyte range, and the fact that while we are processing large amounts of data (pre-reduction) and us preparing batches to feed to the program that won't exceed (pre-reduction) the system RAM (48GB), it's not an issue here.

It's the same problem, but just takes longer to reach the limit of the
kernel VM fragmentation.

~~~
...
-----
>>>>>
>>>>> ERROR: malloc returned NULL at call number 65569 <<<<<
>>>>>
-----
total allocated: 32.27 GB, total allocated after reallocs: 0.25 GB
[elapsed time: 32 sec]
malloc_stats() data:
Arena 0:
system bytes     =     663552
in use bytes     =     135696
Total (incl. mmap):
system bytes     =  549068800
in use bytes     =  548540944
max mmap regions =      65536
max mmap bytes   =  548405248
-----
~~~
1321649768.153047 mmap(NULL, 532480, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f113ffe3000
1321649768.153132 mremap(0x7f113ffe3000, 532480, 8192, MREMAP_MAYMOVE)
= -1 ENOMEM (Cannot allocate memory)
...
1321649768.153170 brk(0)                = 0x19f0000
1321649768.153197 brk(0x1a92000)        = 0x1a92000
1321649768.153415 brk(0x1a12000)        = 0x1a92000
...
1321649768.155301 mmap(NULL, 1048576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
1321649768.155323 mmap(NULL, 134217728, PROT_NONE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot
allocate memory)
1321649768.155343 mmap(NULL, 67108864, PROT_NONE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot
allocate memory)
1321649768.155362 mmap(NULL, 134217728, PROT_NONE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot
allocate memory)
1321649768.155382 mmap(NULL, 67108864, PROT_NONE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot
allocate memory)
1321649768.155410 write(1, ">>>>>\n", 6) = 6
1321649768.155439 write(1, ">>>>> ERROR: malloc returned NUL"..., 61) = 61
1321649768.155461 write(1, ">>>>>\n", 6) = 6
1321649768.155480 write(1, "-----\n", 6) = 6
...
~~~
* mremap fails to find a new mapping for the pages.
* glibc starts trying to use kernel brk call to get new memory but
that fails too.
* final set of mmap allocations fails and glibc gives up.

Fix your application or fix the kernel.

Cheers,
Carlos.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]