[PATCH] malloc change for large-memory/small-address-space

[Wolfram Gloger <wmglo at dent dot med dot uni-muenchen dot de>]

Hello,

More and more people are apparently building ix86-systems with very
large (>2GB) amounts of memory (the latest Linux kernels have good
support for this).  With the malloc changes from last November
(already in glibc-2.1), more memory is available to applications when
sbrk() has already hit the address space limit imposed by the kernel.
However, in some cases even this isn't enough, as the additional
`heaps' allocated with mmap cannot grow arbitrarily, either.  The
remaining space between those heaps can only be claimed with
individual mmap()ed chunks, however this has so far not always been
tried in malloc: there is a fixed maximum number of chunks allocated
with mmap(), and extending the `linear' heap is preferred.

The patch below (meant for glibc-2.2) changes this: if extension of
the `linear' heap fails, as a last resort mmap() is tried despite the
fact that n_mmaps_max has been reached.

People have reported successfully allocating close to the maximum of
virtual memory with this patch.

Regards,
Wolfram.

Index: libc/malloc/malloc.c
===================================================================
RCS file: /cvs/glibc/libc/malloc/malloc.c,v
retrieving revision 1.62
diff -u -r1.62 malloc.c
--- malloc.c	2000/05/25 05:01:35	1.62
+++ malloc.c	2000/06/28 11:19:05
@@ -1872,8 +1872,6 @@
   size_t page_mask = malloc_getpagesize - 1;
   mchunkptr p;
 
-  if(n_mmaps >= n_mmaps_max) return 0; /* too many regions */
-
   /* For mmapped chunks, the overhead is one SIZE_SZ unit larger, because
    * there is no following chunk whose prev_size field could be used.
    */
@@ -2930,8 +2928,10 @@
   {
 
 #if HAVE_MMAP
-    /* If big and would otherwise need to extend, try to use mmap instead */
+    /* If the request is big and there are not yet too many regions,
+       and we would otherwise need to extend, try to use mmap instead.  */
     if ((unsigned long)nb >= (unsigned long)mmap_threshold &&
+        n_mmaps < n_mmaps_max &&
         (victim = mmap_chunk(nb)) != 0)
       return victim;
 #endif
@@ -2939,7 +2939,15 @@
     /* Try to extend */
     malloc_extend_top(ar_ptr, nb);
     if ((remainder_size = chunksize(top(ar_ptr)) - nb) < (long)MINSIZE)
+    {
+#if HAVE_MMAP
+      /* A last attempt:  when we are out of address space in the arena,
+         try mmap anyway, disregarding n_mmaps_max.  */
+      if((victim = mmap_chunk(nb)) != 0)
+        return victim;
+#endif
       return 0; /* propagate failure */
+    }
   }
 
   victim = top(ar_ptr);