This is the mail archive of the glibc-bugs@sources.redhat.com mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

[Bug libc/1128] New: Malloc not trimming filled-up sbrk space causes unnecessary system call overhead


Overview:

After filling up the sbrk-allocated space, each call to free() incurs a system call to brk(), thus consuming 
unnecessary CPU time.



Further Details:

This thread from libc-hacker:

   "Malloc not trimming sbrk space once it fills up"
   http://sourceware.org/ml/libc-hacker/2004-01/msg00012.html

describes how malloc doesn't trim sbrk space once it has started using an mmap-ed region.

In a program I'm developing that does hundreds of thousands of malloc and free per second, I noticed a 
sudden increase in CPU usage and system calls once this threshold had been reached. It turns out that 
every free() operation ends up invoking the brk() system call to find out what the current limit is. When 
this is compared to the current end-of-memory as known by the malloc library at line 3208 of malloc.c, 
in function sYSTRIm(), no trimming is performed so the brk() system call (via morecore) is called again 
at next free().

Relevant lines from malloc.c:

    /*
      Only proceed if end of memory is where we last set it.
      This avoids problems if there were foreign sbrk calls.
    */
    current_brk = (char*)(MORECORE(0));
    if (current_brk == (char*)(av->top) + top_size) {   // malloc.c:3208

I modified Jakub Jelinek's tst-malloc-trim.c from his original email linked above to demonstrate the 
problem. The output from the program contains "count %d" showing the number of times morecore was 
called when freeing blocks. This should ideally be small. As it is, it tends to be close to the number of 
blocks freed. For example:

Actual Result: "free did not trim sbrk area at all: midsum 540672 sum 540672 count 450"
Expected Result: "free did not trim sbrk area at all: midsum 540672 sum 540672 count 0"

I'm currently doing 400,000 frees per second, which adds a non-trivial overhead from the 400,000 brk
() system calls.



Workaround:

I can workaround this by disabling trimming (since it doesn't work anyway) through mallopt
(M_TRIM_THRESHOLD,-1).



Systems Tested:

Confirmed with glibc-2.3.2 from debian 3.0:
GNU C Library stable release version 2.3.2, by Roland McGrath et al.
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 3.3.5 (Debian 1:3.3.5-12).
Compiled on a Linux 2.6.0-test7 system on 2005-05-10.
Available extensions:
        GNU libio by Per Bothner
        crypt add-on version 2.1 by Michael Glad and others
        linuxthreads-0.10 by Xavier Leroy
        BIND-8.2.3-T5B
        libthread_db work sponsored by Alpha Processor Inc
        NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
Thread-local storage support included.

Also with self-compiled glibc-2.3.5 on debian 3.0:
GNU C Library stable release version 2.3.5, by Roland McGrath et al.
Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 3.3.5 (Debian 1:3.3.5-13).
Compiled on a Linux 2.6.0-test7 system on 2005-07-27.
Available extensions:
        GNU libio by Per Bothner
        crypt add-on version 2.1 by Michael Glad and others
        Native POSIX Threads Library by Ulrich Drepper et al
        BIND-8.2.3-T5B
        NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
Thread-local storage support included.




test-malloc-trim2.c
==SNIP==
/* Copyright (C) 2004 Free Software Foundation, Inc.
   This file is part of the GNU C Library.
   Contributed by Jakub Jelinek <jakub@redhat.com>, 2004.

   The GNU C Library is free software; you can redistribute it and/or
   modify it under the terms of the GNU Lesser General Public
   License as published by the Free Software Foundation; either
   version 2.1 of the License, or (at your option) any later version.

   The GNU C Library is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
   Lesser General Public License for more details.

   You should have received a copy of the GNU Lesser General Public
   License along with the GNU C Library; if not, write to the Free
   Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
   02111-1307 USA.  */

#include <malloc.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/mman.h>
#include <unistd.h>

ptrdiff_t sum;
int count;

__malloc_ptr_t
counting_morecore (ptrdiff_t size)
{
  sum += size;
  count++;
  return __default_morecore (size);
}

int
main (void)
{
  size_t pagesize = sysconf (_SC_PAGESIZE);

  char *p = sbrk (0);
  p += (128 * pagesize) & ~(pagesize - 1);
  if (mmap (p, pagesize, PROT_NONE, MAP_PRIVATE|MAP_ANON, -1, 0) != p)
    {
      puts ("Couldn't test");
      return 0;
    }

  if (mallopt (M_TRIM_THRESHOLD, 32 * pagesize) == 0)
    {
      puts ("Couldn't test");
      return 0;
    }

  // uncomment to enable work-around
  //mallopt(M_TRIM_THRESHOLD, (unsigned long) -1);

  __morecore = counting_morecore;

  char *array[512];
  int i;
  for (i = 0; i < 512; ++i)
    array[i] = malloc (pagesize / 4);

  ptrdiff_t midsum = sum;
 
  count = 0;
  for (i = 511; i >= 0; --i)
    free (array[i]);

  __morecore = __default_morecore;

  if (midsum == sum && midsum > 32 * pagesize)
    {
      printf ("free did not trim sbrk area at all: midsum %td sum %td count %td\n",
	      midsum, sum, count);
      return 1;
    }
  return 0;
}
==SNIP==

-- 
           Summary: Malloc not trimming filled-up sbrk space causes
                    unnecessary system call overhead
           Product: glibc
           Version: 2.3.5
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: gotom at debian dot or dot jp
        ReportedBy: jp-www at dcs dot gla dot ac dot uk
                CC: glibc-bugs at sources dot redhat dot com
 GCC build triplet: i686-pc-linux-gnu
  GCC host triplet: i686-pc-linux-gnu
GCC target triplet: i686-pc-linux-gnu


http://sources.redhat.com/bugzilla/show_bug.cgi?id=1128

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]