GNU C Library master sources branch master updated. glibc-2.24-297-g68fc2cc

[siddhesh at sourceware dot org]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU C Library master sources".

The branch, master has been updated
       via  68fc2ccc1aebc15b92e596b2bdc5605da1e25f3c (commit)
       via  aceb22c1f59231909777f7d0a6b955adbf7096a2 (commit)
       via  c1234e60f975da09764683cddff4ef7e2a21ce78 (commit)
       via  2bce30357c8285415eb02019555440db8119ffad (commit)
      from  abff18c0c6055ca5d1cd46923fd1205c057139a5 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=68fc2ccc1aebc15b92e596b2bdc5605da1e25f3c

commit 68fc2ccc1aebc15b92e596b2bdc5605da1e25f3c
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Wed Oct 26 15:07:33 2016 +0530

    Remove redundant definitions of M_ARENA_* macros
    
    The M_ARENA_MAX and M_ARENA_TEST macros are defined in malloc.c as
    well as malloc.h, and the former is unnecessary.  This patch removes
    the duplicate.  Tested on x86_64 to verify that the generated code
    remains unchanged barring changed line numbers to __malloc_assert.
    
    	* malloc/malloc.c (M_ARENA_TEST, M_ARENA_MAX): Remove.

diff --git a/ChangeLog b/ChangeLog
index d9dc849..4db6d2c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,7 @@
 2016-10-26  Siddhesh Poyarekar  <siddhesh@sourceware.org>
 
+	* malloc/malloc.c (M_ARENA_TEST, M_ARENA_MAX): Remove.
+
 	* manual/memory.texi (M_TOP_PAD): Remove reference to sbrk.
 	(M_TRIM_THRESHOLD): Likewise.
 
diff --git a/malloc/malloc.c b/malloc/malloc.c
index a849901..e336e38 100644
--- a/malloc/malloc.c
+++ b/malloc/malloc.c
@@ -1717,11 +1717,6 @@ static struct malloc_par mp_ =
   .arena_test = NARENAS_FROM_NCORES (1)
 };
 
-
-#define M_ARENA_TEST -7
-#define M_ARENA_MAX  -8
-
-
 /* Maximum size of memory handled in fastbins.  */
 static INTERNAL_SIZE_T global_max_fast;
 

http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=aceb22c1f59231909777f7d0a6b955adbf7096a2

commit aceb22c1f59231909777f7d0a6b955adbf7096a2
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Wed Oct 26 15:07:07 2016 +0530

    Remove references to sbrk to grow/shrink arenas
    
    The manual incorrectly references sbrk as the method used to grow and
    shrink heaps and the fact that M_TRIM_THRESHOLD and M_TOP_PAD control
    that behavior.  In reality, a heap may be grown or shrunk through
    multiple methods depending on whether it is the main arena (in which
    case sbrk is correct) or not (in which case, there are a number of
    strategies including allocating an additional heap to grow an arena
    and/or 'mprotect' a region to make it available for allocation).
    
    Remove references to sbrk so that it covers the behavior more
    accurately.
    
    	* manual/memory.texi (M_TOP_PAD): Remove reference to sbrk.
    	(M_TRIM_THRESHOLD): Likewise.

diff --git a/ChangeLog b/ChangeLog
index 1b21469..d9dc849 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,8 @@
 2016-10-26  Siddhesh Poyarekar  <siddhesh@sourceware.org>
 
+	* manual/memory.texi (M_TOP_PAD): Remove reference to sbrk.
+	(M_TRIM_THRESHOLD): Likewise.
+
 	* manual/memory.texi (M_ARENA_TEST): Add documentation.
 	(M_ARENA_MAX): Likewise.
 	* malloc/malloc.c: Remove obsolete comment.
diff --git a/manual/memory.texi b/manual/memory.texi
index 4ad2c96..b66de60 100644
--- a/manual/memory.texi
+++ b/manual/memory.texi
@@ -1132,11 +1132,10 @@ This parameter can also be set for the process at startup by setting the
 environment variable @env{MALLOC_MMAP_PERTURB_} to the desired value.
 
 @item M_TOP_PAD
-This parameter determines the amount of extra memory to obtain from the
-system when a call to @code{sbrk} is required.  It also specifies the
-number of bytes to retain when shrinking the heap by calling @code{sbrk}
-with a negative argument.  This provides the necessary hysteresis in
-heap size such that excessive amounts of system calls can be avoided.
+This parameter determines the amount of extra memory to obtain from the system
+when an arena needs to be extended.  It also specifies the number of bytes to
+retain when shrinking an arena.  This provides the necessary hysteresis in heap
+size such that excessive amounts of system calls can be avoided.
 
 The default value of this parameter is @code{0}.
 
@@ -1145,8 +1144,7 @@ environment variable @env{MALLOC_TOP_PAD_} to the desired value.
 
 @item M_TRIM_THRESHOLD
 This is the minimum size (in bytes) of the top-most, releasable chunk
-that will cause @code{sbrk} to be called with a negative argument in
-order to return memory to the system.
+that will trigger a system call in order to return memory to the system.
 
 If this parameter is not set, the default value is set as 128 KiB and the
 threshold is adjusted dynamically to suit the allocation patterns of the

http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=c1234e60f975da09764683cddff4ef7e2a21ce78

commit c1234e60f975da09764683cddff4ef7e2a21ce78
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Wed Oct 26 15:06:21 2016 +0530

    Document the M_ARENA_* mallopt parameters
    
    The M_ARENA_* mallopt parameters are in wide use in production to
    control the number of arenas that a long lived process creates and
    hence there is no point in stating that this interface is non-public.
    Document this interface and remove the obsolete comment.
    
    	* manual/memory.texi (M_ARENA_TEST): Add documentation.
    	(M_ARENA_MAX): Likewise.
    	* malloc/malloc.c: Remove obsolete comment.

diff --git a/ChangeLog b/ChangeLog
index ab8c4da..1b21469 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,9 @@
 2016-10-26  Siddhesh Poyarekar  <siddhesh@sourceware.org>
 
+	* manual/memory.texi (M_ARENA_TEST): Add documentation.
+	(M_ARENA_MAX): Likewise.
+	* malloc/malloc.c: Remove obsolete comment.
+
 	* manual/memory.texi: Add environment variable alternatives to
 	setting mallopt parameters.
 
diff --git a/malloc/malloc.c b/malloc/malloc.c
index ef04360..a849901 100644
--- a/malloc/malloc.c
+++ b/malloc/malloc.c
@@ -1718,7 +1718,6 @@ static struct malloc_par mp_ =
 };
 
 
-/*  Non public mallopt parameters.  */
 #define M_ARENA_TEST -7
 #define M_ARENA_MAX  -8
 
diff --git a/manual/memory.texi b/manual/memory.texi
index c6263d2..4ad2c96 100644
--- a/manual/memory.texi
+++ b/manual/memory.texi
@@ -162,6 +162,8 @@ special to @theglibc{} and GNU Compiler.
 
 @menu
 * Memory Allocation and C::     How to get different kinds of allocation in C.
+* The GNU Allocator::		An overview of the GNU @code{malloc}
+				implementation.
 * Unconstrained Allocation::    The @code{malloc} facility allows fully general
 		 		 dynamic allocation.
 * Allocation Debugging::        Finding memory leaks and not freed memory.
@@ -258,6 +260,45 @@ address of the space.  Then you can use the operators @samp{*} and
 @}
 @end smallexample
 
+@node The GNU Allocator
+@subsection The GNU Allocator
+@cindex gnu allocator
+
+The @code{malloc} implementation in @theglibc{} is derived from ptmalloc
+(pthreads malloc), which in turn is derived from dlmalloc (Doug Lea malloc).
+This malloc may allocate memory in two different ways depending on their size
+and certain parameters that may be controlled by users. The most common way is
+to allocate portions of memory (called chunks) from a large contiguous area of
+memory and manage these areas to optimize their use and reduce wastage in the
+form of unusable chunks. Traditionally the system heap was set up to be the one
+large memory area but the @glibcadj{} @code{malloc} implementation maintains
+multiple such areas to optimize their use in multi-threaded applications.  Each
+such area is internally referred to as an @dfn{arena}.
+
+As opposed to other versions, the @code{malloc} in @theglibc{} does not round
+up chunk sizes to powers of two, neither for large nor for small sizes.
+Neighboring chunks can be coalesced on a @code{free} no matter what their size
+is.  This makes the implementation suitable for all kinds of allocation
+patterns without generally incurring high memory waste through fragmentation.
+The presence of multiple arenas allows multiple threads to allocate
+memory simultaneously in separate arenas, thus improving performance.
+
+The other way of memory allocation is for very large blocks, i.e. much larger
+than a page. These requests are allocated with @code{mmap} (anonymous or via
+@file{/dev/zero}; @pxref{Memory-mapped I/O})). This has the great advantage
+that these chunks are returned to the system immediately when they are freed.
+Therefore, it cannot happen that a large chunk becomes ``locked'' in between
+smaller ones and even after calling @code{free} wastes memory.  The size
+threshold for @code{mmap} to be used is dynamic and gets adjusted according to
+allocation patterns of the program.  @code{mallopt} can be used to statically
+adjust the threshold using @code{M_MMAP_THRESHOLD} and the use of @code{mmap}
+can be disabled completely with @code{M_MMAP_MAX};
+@pxref{Malloc Tunable Parameters}.
+
+A more detailed technical description of the GNU Allocator is maintained in
+the @glibcadj{} wiki. See
+@uref{https://sourceware.org/glibc/wiki/MallocInternals}.
+
 @node Unconstrained Allocation
 @subsection Unconstrained Allocation
 @cindex unconstrained memory allocation
@@ -278,8 +319,6 @@ any time (or never).
 				 bigger or smaller.
 * Allocating Cleared Space::    Use @code{calloc} to allocate a
 				 block and clear it.
-* Efficiency and Malloc::       Efficiency considerations in use of
-				 these functions.
 * Aligned Memory Blocks::       Allocating specially aligned memory.
 * Malloc Tunable Parameters::   Use @code{mallopt} to adjust allocation
                                  parameters.
@@ -867,59 +906,6 @@ But in general, it is not guaranteed that @code{calloc} calls
 @code{malloc}/@code{realloc}/@code{free} outside the C library, it
 should always define @code{calloc}, too.
 
-@node Efficiency and Malloc
-@subsubsection Efficiency Considerations for @code{malloc}
-@cindex efficiency and @code{malloc}
-
-
-
-
-@ignore
-
-@c No longer true, see below instead.
-To make the best use of @code{malloc}, it helps to know that the GNU
-version of @code{malloc} always dispenses small amounts of memory in
-blocks whose sizes are powers of two.  It keeps separate pools for each
-power of two.  This holds for sizes up to a page size.  Therefore, if
-you are free to choose the size of a small block in order to make
-@code{malloc} more efficient, make it a power of two.
-@c !!! xref getpagesize
-
-Once a page is split up for a particular block size, it can't be reused
-for another size unless all the blocks in it are freed.  In many
-programs, this is unlikely to happen.  Thus, you can sometimes make a
-program use memory more efficiently by using blocks of the same size for
-many different purposes.
-
-When you ask for memory blocks of a page or larger, @code{malloc} uses a
-different strategy; it rounds the size up to a multiple of a page, and
-it can coalesce and split blocks as needed.
-
-The reason for the two strategies is that it is important to allocate
-and free small blocks as fast as possible, but speed is less important
-for a large block since the program normally spends a fair amount of
-time using it.  Also, large blocks are normally fewer in number.
-Therefore, for large blocks, it makes sense to use a method which takes
-more time to minimize the wasted space.
-
-@end ignore
-
-As opposed to other versions, the @code{malloc} in @theglibc{}
-does not round up block sizes to powers of two, neither for large nor
-for small sizes.  Neighboring chunks can be coalesced on a @code{free}
-no matter what their size is.  This makes the implementation suitable
-for all kinds of allocation patterns without generally incurring high
-memory waste through fragmentation.
-
-Very large blocks (much larger than a page) are allocated with
-@code{mmap} (anonymous or via @code{/dev/zero}) by this implementation.
-This has the great advantage that these chunks are returned to the
-system immediately when they are freed.  Therefore, it cannot happen
-that a large chunk becomes ``locked'' in between smaller ones and even
-after calling @code{free} wastes memory.  The size threshold for
-@code{mmap} to be used can be adjusted with @code{mallopt}.  The use of
-@code{mmap} can also be disabled completely.
-
 @node Aligned Memory Blocks
 @subsubsection Allocating Aligned Memory Blocks
 
@@ -1105,10 +1091,6 @@ parameter to be set, and @var{value} the new value to be set.  Possible
 choices for @var{param}, as defined in @file{malloc.h}, are:
 
 @table @code
-@comment TODO: @item M_ARENA_MAX
-@comment       - Document ARENA_MAX env var.
-@comment TODO: @item M_ARENA_TEST
-@comment       - Document ARENA_TEST env var.
 @comment TODO: @item M_CHECK_ACTION
 @item M_MMAP_MAX
 The maximum number of chunks to allocate with @code{mmap}.  Setting this
@@ -1174,6 +1156,29 @@ value is set statically to the provided input.
 This parameter can also be set for the process at startup by setting the
 environment variable @env{MALLOC_TRIM_THRESHOLD_} to the desired value.
 
+@item M_ARENA_TEST
+This parameter specifies the number of arenas that can be created before the
+test on the limit to the number of arenas is conducted. The value is ignored if
+@code{M_ARENA_MAX} is set.
+
+The default value of this parameter is 2 on 32-bit systems and 8 on 64-bit
+systems.
+
+This parameter can also be set for the process at startup by setting the
+environment variable @env{MALLOC_ARENA_TEST} to the desired value.
+
+@item M_ARENA_MAX
+This parameter sets the number of arenas to use regardless of the number of
+cores in the system.
+
+The default value of this tunable is @code{0}, meaning that the limit on the
+number of arenas is determined by the number of CPU cores online. For 32-bit
+systems the limit is twice the number of cores online and on 64-bit systems, it
+is eight times the number of cores online.  Note that the default value is not
+derived from the default value of M_ARENA_TEST and is computed independently.
+
+This parameter can also be set for the process at startup by setting the
+environment variable @env{MALLOC_ARENA_MAX} to the desired value.
 @end table
 
 @end deftypefun
@@ -1515,8 +1520,8 @@ This is the total size of memory allocated with @code{sbrk} by
 @item int ordblks
 This is the number of chunks not in use.  (The memory allocator
 internally gets chunks of memory from the operating system, and then
-carves them up to satisfy individual @code{malloc} requests; see
-@ref{Efficiency and Malloc}.)
+carves them up to satisfy individual @code{malloc} requests;
+@pxref{The GNU Allocator}.)
 
 @item int smblks
 This field is unused.

http://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=2bce30357c8285415eb02019555440db8119ffad

commit 2bce30357c8285415eb02019555440db8119ffad
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Wed Oct 26 15:05:37 2016 +0530

    Add note on MALLOC_MMAP_* environment variables
    
    The mallopt parameters manual does not mention the environment
    variables that can be used to set these parameters at program startup.
    Mention those environment variables for completeness.
    
    	* manual/memory.texi: Add environment variable alternatives to
    	setting mallopt parameters.

diff --git a/ChangeLog b/ChangeLog
index 9c4d06f..ab8c4da 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2016-10-26  Siddhesh Poyarekar  <siddhesh@sourceware.org>
+
+	* manual/memory.texi: Add environment variable alternatives to
+	setting mallopt parameters.
+
 2016-10-26  Carlos O'Donell  <carlos@redhat.com>
 
 	* include/atomic.h
diff --git a/manual/memory.texi b/manual/memory.texi
index 5383105..c6263d2 100644
--- a/manual/memory.texi
+++ b/manual/memory.texi
@@ -1113,13 +1113,28 @@ choices for @var{param}, as defined in @file{malloc.h}, are:
 @item M_MMAP_MAX
 The maximum number of chunks to allocate with @code{mmap}.  Setting this
 to zero disables all use of @code{mmap}.
+
+The default value of this parameter is @code{65536}.
+
+This parameter can also be set for the process at startup by setting the
+environment variable @env{MALLOC_MMAP_MAX_} to the desired value.
+
 @item M_MMAP_THRESHOLD
 All chunks larger than this value are allocated outside the normal
 heap, using the @code{mmap} system call.  This way it is guaranteed
 that the memory for these chunks can be returned to the system on
 @code{free}.  Note that requests smaller than this threshold might still
 be allocated via @code{mmap}.
+
+If this parameter is not set, the default value is set as 128 KiB and the
+threshold is adjusted dynamically to suit the allocation patterns of the
+program. If the parameter is set, the dynamic adjustment is disabled and the
+value is set statically to the input value.
+
+This parameter can also be set for the process at startup by setting the
+environment variable @env{MALLOC_MMAP_THRESHOLD_} to the desired value.
 @comment TODO: @item M_MXFAST
+
 @item M_PERTURB
 If non-zero, memory blocks are filled with values depending on some
 low order bits of this parameter when they are allocated (except when
@@ -1128,16 +1143,37 @@ use of uninitialized or freed heap memory.  Note that this option does not
 guarantee that the freed block will have any specific values.  It only
 guarantees that the content the block had before it was freed will be
 overwritten.
+
+The default value of this parameter is @code{0}.
+
+This parameter can also be set for the process at startup by setting the
+environment variable @env{MALLOC_MMAP_PERTURB_} to the desired value.
+
 @item M_TOP_PAD
 This parameter determines the amount of extra memory to obtain from the
 system when a call to @code{sbrk} is required.  It also specifies the
 number of bytes to retain when shrinking the heap by calling @code{sbrk}
 with a negative argument.  This provides the necessary hysteresis in
 heap size such that excessive amounts of system calls can be avoided.
+
+The default value of this parameter is @code{0}.
+
+This parameter can also be set for the process at startup by setting the
+environment variable @env{MALLOC_TOP_PAD_} to the desired value.
+
 @item M_TRIM_THRESHOLD
 This is the minimum size (in bytes) of the top-most, releasable chunk
 that will cause @code{sbrk} to be called with a negative argument in
 order to return memory to the system.
+
+If this parameter is not set, the default value is set as 128 KiB and the
+threshold is adjusted dynamically to suit the allocation patterns of the
+program. If the parameter is set, the dynamic adjustment is disabled and the
+value is set statically to the provided input.
+
+This parameter can also be set for the process at startup by setting the
+environment variable @env{MALLOC_TRIM_THRESHOLD_} to the desired value.
+
 @end table
 
 @end deftypefun

-----------------------------------------------------------------------

Summary of changes:
 ChangeLog          |   14 ++++
 malloc/malloc.c    |    6 --
 manual/memory.texi |  175 ++++++++++++++++++++++++++++++++--------------------
 3 files changed, 121 insertions(+), 74 deletions(-)


hooks/post-receive
-- 
GNU C Library master sources