Re: [PATCH 2/2] Document the M_ARENA_* mallopt parameters

[Michael Kerrisk <mtk dot manpages at gmail dot com>]

On Mon, Oct 17, 2016 at 6:08 PM, Carlos O'Donell <carlos@redhat.com> wrote:
> On 10/10/2016 01:26 PM, Siddhesh Poyarekar wrote:
>> The M_ARENA_* mallopt parameters are in wide use in production to
>> control the number of arenas that a long lived process creates and
>> hence there is no point in stating that this interface is non-public.
>> Document this interface and remove the obsolete comment.
>>
>>       * manual/memory.texi (M_ARENA_TEST): Add documentation.
>>       (M_ARENA_MAX): Likewise.
>>       * malloc/malloc.c: Remove obsolete comment.
>> ---
>>  malloc/malloc.c    |  1 -
>>  manual/memory.texi | 21 +++++++++++++++++++++
>>  2 files changed, 21 insertions(+), 1 deletion(-)
>>
>> diff --git a/malloc/malloc.c b/malloc/malloc.c
>> index 09e004b..0011a6d 100644
>> --- a/malloc/malloc.c
>> +++ b/malloc/malloc.c
>> @@ -1718,7 +1718,6 @@ static struct malloc_par mp_ =
>>  };
>>
>>
>> -/*  Non public mallopt parameters.  */
>>  #define M_ARENA_TEST -7
>>  #define M_ARENA_MAX  -8
>>
>> diff --git a/manual/memory.texi b/manual/memory.texi
>> index 222f126..b98dcf2 100644
>> --- a/manual/memory.texi
>> +++ b/manual/memory.texi
>> @@ -1153,6 +1153,27 @@ order to return memory to the system.
>>
>>  This parameter can also be set for the process at startup by setting the
>>  environment variable @code{MALLOC_TRIM_THRESHOLD_} to the desired value.
>> +
>> +@item M_ARENA_TEST
>
> This description of an arena doesn't belong here, it belongs higher up in the
> manual, and in fact I would suggest the following:
>
> - Move 3.2.2.6 "Efficiency considerations for malloc" up to just under
>   "Dynamic Memory allocation" and put it under a new subsection "The GNU allocator".
>
> - Under "The GNU allocator" give a brief 1 paragraph description of how
>   the allocator works e.g. https://sourceware.org/glibc/wiki/MallocInternals
>   and talk about arenas.
>
>> +An arena is a memory pool that is allocated to act as a heap alongside the
>> +system heap that the kernel creates for the process.  This is to provide
>> +multiple heap structures for multiple threads of a process to reduce contention
>> +between them.  The limit to the number of such arenas per process is determined
>> +by the number of cores of the system and whether it is a 32-bit or a 64-bit
>> +processor.
>
> You don't say what M_ARENA_TEST is for?
>
>> +
>> +The value is ignored if @code{M_ARENA_MAX} is set either via @code{mallopt} or
>> +via the @code{MALLOC_ARENA_MAX} environment variable.
>> +
>> +This parameter can also be set for the process at startup by setting the
>> +environment variable @code{MALLOC_ARENA_TEST} to the desired value.
>
> What is the default?

So my reading from malloc/malloc.c:

#define NARENAS_FROM_NCORES(n) ((n) * (sizeof (long) == 4 ? 2 : 8))
  .arena_test = NARENAS_FROM_NCORES (1)

So, the default value for this parameter is 2 on systems where
sizeof(long) is 4; otherwise the default value is 8.

>> +
>> +@item M_ARENA_MAX
>> +This parameter sets the number of arenas to use regardless of the number of
>> +cores in the system.
>> +
>> +This parameter can also be set for the process at startup by setting the
>> +environment variable @code{MALLOC_ARENA_MAX} to the desired value.
>
> What is the default?

So, IIUC, the default value of this parameter is 0, meaning that there
is no limit on the number of arenas that can be created. Do you
confirm, Siddhesh?

Siddhesh, in case you want to use any of my wordings (assuming they
are correct), I place them in the public domain. (The text will also
land in the mallopt(3) manual page.)

Cheers,

Michael