This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
[ping3][PATCH v2 1/2] Add framework for tunables
- From: Siddhesh Poyarekar <sid at reserved-bit dot com>
- To: libc-alpha at sourceware dot org
- Cc: roland at hack dot frob dot com, carlos at redhat dot com, Andi Kleen <andi at firstfloor dot org>, tuliom at linux dot vnet dot ibm dot com, "Paul E. Murphy" <murphyp at linux dot vnet dot ibm dot com>, munroesj at linux dot vnet dot ibm dot com
- Date: Mon, 8 Feb 2016 10:39:55 +0530
- Subject: [ping3][PATCH v2 1/2] Add framework for tunables
- Authentication-results: sourceware.org; auth=none
- References: <20160116185545 dot GA17772 at devel dot intra dot reserved-bit dot com> <20160125032505 dot GA22776 at devel dot intra dot reserved-bit dot com> <20160201094030 dot GF17552 at devel dot intra dot reserved-bit dot com>
Ping!
On Mon, Feb 01, 2016 at 03:10:32PM +0530, Siddhesh Poyarekar wrote:
> Ping!
>
> On Mon, Jan 25, 2016 at 08:55:06AM +0530, Siddhesh Poyarekar wrote:
> > Ping!
> >
> > On Sun, Jan 17, 2016 at 12:25:50AM +0530, Siddhesh Poyarekar wrote:
> > > The tunables framework allows us to uniformly manage and expose global
> > > variables inside glibc as switches to users. tunables/README has
> > > instructions for glibc developers to add new tunables.
> > >
> > > Tunables support can be enabled by passing the --enable-tunables
> > > configure flag to the configure script. This patch only adds a
> > > framework and does not pose any limitations on how tunable values are
> > > read from the user. It also adds environment variables used in malloc
> > > behaviour tweaking to the tunables framework as a PoC of the
> > > compatibility interface.
> > >
> > > * manual/install.texi: Add --enable-tunables option.
> > > * INSTALL: Regenerate.
> > > * Makeconfig (CPPFLAGS): Define TOP_NAMESPACE.
> > > (all-subdirs): Add tunables.
> > > * config.h.in: Add BUILD_TUNABLES.
> > > * config.make.in: Add build-tunables.
> > > * configure.ac: Add --enable-tunables option.
> > > * configure: Regenerate.
> > > * csu/init-first.c: Include tunables.h.
> > > (_init): Initialize tunables.
> > > * nptl/nptl-init.c [BUILD_TUNABLES]: Include tunables.h.
> > > (__pthread_initialize_minimal_internal): Accept argc, argv and
> > > envp. Initialize tunables.
> > > * csu/libc-start.c (LIBC_START_MAIN): Adjust.
> > > * malloc/arena.c [BUILD_TUNABLES]: Include tunables.h. Define
> > > TUNABLE_NAMESPACE.
> > > (tunable_set_mallopt_top_pad, tunable_set_mallopt_perturb,
> > > tunable_set_mallopt_mmap_max, tunable_set_mallopt_arena_max,
> > > tunable_set_mallopt_arena_test,
> > > tunable_set_mallopt_trim_threshold,
> > > tunable_set_mallopt_mmap_threshold,
> > > tunable_set_mallopt_check): New functions.
> > > (ptmalloc_init): Use them and register tunables.
> > > * scripts/gen-tunables.awk: New file.
> > > * tunables/Makefile: New file.
> > > * tunables/README: New file.
> > > * tunables/Versions: New file.
> > > * tunables/tunable-list.h: New auto-generated file.
> > > * tunables/tunables.c: New file.
> > > * tunables/tunables.h: New file.
> > > * tunables/tunables.list: New file.
> > > ---
> > > INSTALL | 6 ++
> > > Makeconfig | 9 +++
> > > config.h.in | 3 +
> > > config.make.in | 1 +
> > > configure | 16 +++++
> > > configure.ac | 10 +++
> > > csu/init-first.c | 7 ++
> > > csu/libc-start.c | 4 +-
> > > malloc/arena.c | 86 +++++++++++++++++++++++++
> > > manual/install.texi | 5 ++
> > > nptl/nptl-init.c | 9 ++-
> > > scripts/gen-tunables.awk | 84 ++++++++++++++++++++++++
> > > tunables/Makefile | 33 ++++++++++
> > > tunables/README | 93 +++++++++++++++++++++++++++
> > > tunables/Versions | 8 +++
> > > tunables/tunable-list.h | 32 ++++++++++
> > > tunables/tunables.c | 163 +++++++++++++++++++++++++++++++++++++++++++++++
> > > tunables/tunables.h | 110 ++++++++++++++++++++++++++++++++
> > > tunables/tunables.list | 12 ++++
> > > 19 files changed, 688 insertions(+), 3 deletions(-)
> > > create mode 100644 scripts/gen-tunables.awk
> > > create mode 100644 tunables/Makefile
> > > create mode 100644 tunables/README
> > > create mode 100644 tunables/Versions
> > > create mode 100644 tunables/tunable-list.h
> > > create mode 100644 tunables/tunables.c
> > > create mode 100644 tunables/tunables.h
> > > create mode 100644 tunables/tunables.list
> > >
> > > diff --git a/INSTALL b/INSTALL
> > > index c70ea9f..dd06e2e 100644
> > > --- a/INSTALL
> > > +++ b/INSTALL
> > > @@ -164,6 +164,12 @@ will be used, and CFLAGS sets optimization options for the compiler.
> > > By default for x86_64, the GNU C Library is built with vector math
> > > library. Use this option to disable vector math library.
> > >
> > > +'--enable-tunables'
> > > + Tunables support allows additional library parameters to be
> > > + customized at runtime for each application. This is an
> > > + experimental feature and affects startup time and is thus disabled
> > > + by default.
> > > +
> > > '--build=BUILD-SYSTEM'
> > > '--host=HOST-SYSTEM'
> > > These options are for cross-compiling. If you specify both options
> > > diff --git a/Makeconfig b/Makeconfig
> > > index 87a22e8..8ac199b 100644
> > > --- a/Makeconfig
> > > +++ b/Makeconfig
> > > @@ -883,6 +883,11 @@ CPPFLAGS = $(config-extra-cppflags) $(CPPUNDEFS) $(CPPFLAGS-config) \
> > > $(foreach lib,$(libof-$(basename $(@F))) \
> > > $(libof-$(<F)) $(libof-$(@F)),$(CPPFLAGS-$(lib))) \
> > > $(CPPFLAGS-$(<F)) $(CPPFLAGS-$(@F)) $(CPPFLAGS-$(basename $(@F)))
> > > +
> > > +ifeq (yes,$(build-tunables))
> > > +CPPFLAGS += -DTOP_NAMESPACE=glibc
> > > +endif
> > > +
> > > override CFLAGS = -std=gnu11 -fgnu89-inline $(config-extra-cflags) \
> > > $(filter-out %frame-pointer,$(+cflags)) $(+gccwarn-c) \
> > > $(sysdep-CFLAGS) $(CFLAGS-$(suffix $@)) $(CFLAGS-$(<F)) \
> > > @@ -1098,6 +1103,10 @@ all-subdirs = csu assert ctype locale intl catgets math setjmp signal \
> > > crypt localedata timezone rt conform debug mathvec \
> > > $(add-on-subdirs) dlfcn elf
> > >
> > > +ifeq (yes,$(build-tunables))
> > > +all-subdirs += tunables
> > > +endif
> > > +
> > > ifndef avoid-generated
> > > # sysd-sorted itself will contain rules making the sysd-sorted target
> > > # depend on Depend files. But if you just added a Depend file to an
> > > diff --git a/config.h.in b/config.h.in
> > > index ec9c8bc..9b07580 100644
> > > --- a/config.h.in
> > > +++ b/config.h.in
> > > @@ -231,4 +231,7 @@
> > > /* PowerPC32 uses fctidz for floating point to long long conversions. */
> > > #define HAVE_PPC_FCTIDZ 0
> > >
> > > +/* Build glibc with tunables support. */
> > > +#define BUILD_TUNABLES 0
> > > +
> > > #endif
> > > diff --git a/config.make.in b/config.make.in
> > > index 05ed6ec..b158f51 100644
> > > --- a/config.make.in
> > > +++ b/config.make.in
> > > @@ -90,6 +90,7 @@ use-nscd = @use_nscd@
> > > build-hardcoded-path-in-tests= @hardcoded_path_in_tests@
> > > build-pt-chown = @build_pt_chown@
> > > enable-lock-elision = @enable_lock_elision@
> > > +build-tunables = @build_tunables@
> > >
> > > # Build tools.
> > > CC = @CC@
> > > diff --git a/configure b/configure
> > > index aa05d49..6fef02c 100755
> > > --- a/configure
> > > +++ b/configure
> > > @@ -659,6 +659,7 @@ multi_arch
> > > base_machine
> > > add_on_subdirs
> > > add_ons
> > > +build_tunables
> > > build_pt_chown
> > > build_nscd
> > > link_obsolete_rpc
> > > @@ -773,6 +774,7 @@ enable_systemtap
> > > enable_build_nscd
> > > enable_nscd
> > > enable_pt_chown
> > > +enable_tunables
> > > enable_mathvec
> > > with_cpu
> > > '
> > > @@ -1440,6 +1442,7 @@ Optional Features:
> > > --disable-build-nscd disable building and installing the nscd daemon
> > > --disable-nscd library functions will not contact the nscd daemon
> > > --enable-pt_chown Enable building and installing pt_chown
> > > + --enable-tunables Enable tunables support
> > > --enable-mathvec Enable building and installing mathvec [default
> > > depends on architecture]
> > >
> > > @@ -3642,6 +3645,19 @@ if test "$build_pt_chown" = yes; then
> > >
> > > fi
> > >
> > > +# Check whether --enable-tunables was given.
> > > +if test "${enable_tunables+set}" = set; then :
> > > + enableval=$enable_tunables; build_tunables=$enableval
> > > +else
> > > + build_tunables=no
> > > +fi
> > > +
> > > +
> > > +if test "$build_tunables" = yes; then
> > > + $as_echo "#define BUILD_TUNABLES 1" >>confdefs.h
> > > +
> > > +fi
> > > +
> > > # The abi-tags file uses a fairly simplistic model for name recognition that
> > > # can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a
> > > # $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell.
> > > diff --git a/configure.ac b/configure.ac
> > > index ee7a3f1..41507ac 100644
> > > --- a/configure.ac
> > > +++ b/configure.ac
> > > @@ -392,6 +392,16 @@ if test "$build_pt_chown" = yes; then
> > > AC_DEFINE(HAVE_PT_CHOWN)
> > > fi
> > >
> > > +AC_ARG_ENABLE([tunables],
> > > + [AS_HELP_STRING([--enable-tunables],
> > > + [Enable tunables support])],
> > > + [build_tunables=$enableval],
> > > + [build_tunables=no])
> > > +AC_SUBST(build_tunables)
> > > +if test "$build_tunables" = yes; then
> > > + AC_DEFINE(BUILD_TUNABLES)
> > > +fi
> > > +
> > > # The abi-tags file uses a fairly simplistic model for name recognition that
> > > # can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a
> > > # $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell.
> > > diff --git a/csu/init-first.c b/csu/init-first.c
> > > index 77c6e1c..a4e0c54 100644
> > > --- a/csu/init-first.c
> > > +++ b/csu/init-first.c
> > > @@ -26,6 +26,9 @@
> > > #include <sys/param.h>
> > > #include <sys/types.h>
> > > #include <libc-internal.h>
> > > +#if BUILD_TUNABLES
> > > +# include <tunables/tunables.h>
> > > +#endif
> > >
> > > #include <ldsodefs.h>
> > >
> > > @@ -96,6 +99,10 @@ _init (int argc, char **argv, char **envp)
> > > #if defined SHARED && !defined NO_CTORS_DTORS_SECTIONS
> > > __libc_global_ctors ();
> > > #endif
> > > +
> > > +#if BUILD_TUNABLES
> > > + __tunables_init (__environ);
> > > +#endif
> > > }
> > >
> > > /* This function is defined here so that if this file ever gets into
> > > diff --git a/csu/libc-start.c b/csu/libc-start.c
> > > index f4aa01a..7629fec 100644
> > > --- a/csu/libc-start.c
> > > +++ b/csu/libc-start.c
> > > @@ -32,7 +32,7 @@ extern int __libc_multiple_libcs;
> > > #include <tls.h>
> > > #ifndef SHARED
> > > # include <dl-osinfo.h>
> > > -extern void __pthread_initialize_minimal (void);
> > > +extern void __pthread_initialize_minimal (int, char **, char **);
> > > # ifndef THREAD_SET_STACK_GUARD
> > > /* Only exported for architectures that don't store the stack guard canary
> > > in thread local area. */
> > > @@ -193,7 +193,7 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL),
> > > /* Initialize the thread library at least a bit since the libgcc
> > > functions are using thread functions if these are available and
> > > we need to setup errno. */
> > > - __pthread_initialize_minimal ();
> > > + __pthread_initialize_minimal (argc, argv, __environ);
> > >
> > > /* Set up the stack checker's canary. */
> > > uintptr_t stack_chk_guard = _dl_setup_stack_chk_guard (_dl_random);
> > > diff --git a/malloc/arena.c b/malloc/arena.c
> > > index 1edb4d4..f773b10 100644
> > > --- a/malloc/arena.c
> > > +++ b/malloc/arena.c
> > > @@ -19,6 +19,11 @@
> > >
> > > #include <stdbool.h>
> > >
> > > +#if BUILD_TUNABLES
> > > +# define TUNABLE_NAMESPACE malloc
> > > +# include <tunables/tunables.h>
> > > +#endif
> > > +
> > > /* Compile-time constants. */
> > >
> > > #define HEAP_MIN_SIZE (32 * 1024)
> > > @@ -332,6 +337,61 @@ ptmalloc_unlock_all2 (void)
> > > # endif
> > > #endif /* !NO_THREADS */
> > >
> > > +#if BUILD_TUNABLES
> > > +static void
> > > +tunable_set_mallopt_top_pad (const char *val)
> > > +{
> > > + __libc_mallopt (M_TOP_PAD, atoi (val));
> > > +}
> > > +
> > > +static void
> > > +tunable_set_mallopt_perturb (const char *val)
> > > +{
> > > + __libc_mallopt (M_PERTURB, atoi (val));
> > > +}
> > > +
> > > +static void
> > > +tunable_set_mallopt_mmap_max (const char *val)
> > > +{
> > > + __libc_mallopt (M_MMAP_MAX, atoi (val));
> > > +}
> > > +
> > > +static void
> > > +tunable_set_mallopt_arena_max (const char *val)
> > > +{
> > > + __libc_mallopt (M_ARENA_MAX, atoi (val));
> > > +}
> > > +
> > > +static void
> > > +tunable_set_mallopt_arena_test (const char *val)
> > > +{
> > > + __libc_mallopt (M_ARENA_TEST, atoi (val));
> > > +}
> > > +
> > > +static void
> > > +tunable_set_mallopt_trim_threshold (const char *val)
> > > +{
> > > + __libc_mallopt (M_TRIM_THRESHOLD, atoi (val));
> > > +}
> > > +
> > > +static void
> > > +tunable_set_mallopt_mmap_threshold (const char *val)
> > > +{
> > > + __libc_mallopt (M_MMAP_THRESHOLD, atoi (val));
> > > +}
> > > +
> > > +static void
> > > +tunable_set_mallopt_check (const char *val)
> > > +{
> > > + if (val[0])
> > > + {
> > > + __libc_mallopt (M_CHECK_ACTION, (int) (val[0] - '0'));
> > > + if (check_action != 0)
> > > + __malloc_check_init ();
> > > + }
> > > +}
> > > +
> > > +#else
> > > /* Initialization routine. */
> > > #include <string.h>
> > > extern char **_environ;
> > > @@ -366,6 +426,7 @@ next_env_entry (char ***position)
> > >
> > > return result;
> > > }
> > > +#endif
> > >
> > >
> > > #ifdef SHARED
> > > @@ -401,6 +462,29 @@ ptmalloc_init (void)
> > >
> > > thread_arena = &main_arena;
> > > thread_atfork (ptmalloc_lock_all, ptmalloc_unlock_all, ptmalloc_unlock_all2);
> > > +
> > > +#if BUILD_TUNABLES
> > > + COMPAT_TUNABLES_NAMESPACE_BEGIN (8);
> > > +
> > > + COMPAT_TUNABLE_REGISTER_SECURE (check, "MALLOC_CHECK_",
> > > + tunable_set_mallopt_check);
> > > + COMPAT_TUNABLE_REGISTER (top_pad, "MALLOC_TOP_PAD_",
> > > + tunable_set_mallopt_top_pad);
> > > + COMPAT_TUNABLE_REGISTER (perturb, "MALLOC_PERTURB_",
> > > + tunable_set_mallopt_perturb);
> > > + COMPAT_TUNABLE_REGISTER (mmap_threshold, "MALLOC_MMAP_THRESHOLD_",
> > > + tunable_set_mallopt_mmap_threshold);
> > > + COMPAT_TUNABLE_REGISTER (trim_threshold, "MALLOC_TRIM_THRESHOLD_",
> > > + tunable_set_mallopt_trim_threshold);
> > > + COMPAT_TUNABLE_REGISTER (mmap_max, "MALLOC_MMAP_MAX_",
> > > + tunable_set_mallopt_mmap_max);
> > > + COMPAT_TUNABLE_REGISTER (arena_max, "MALLOC_ARENA_MAX",
> > > + tunable_set_mallopt_arena_max);
> > > + COMPAT_TUNABLE_REGISTER (arena_test, "MALLOC_ARENA_TEST",
> > > + tunable_set_mallopt_arena_test);
> > > +
> > > + COMPAT_TUNABLES_NAMESPACE_INIT ();
> > > +#else
> > > const char *s = NULL;
> > > if (__glibc_likely (_environ != NULL))
> > > {
> > > @@ -469,6 +553,8 @@ ptmalloc_init (void)
> > > if (check_action != 0)
> > > __malloc_check_init ();
> > > }
> > > +#endif
> > > +
> > > void (*hook) (void) = atomic_forced_read (__malloc_initialize_hook);
> > > if (hook != NULL)
> > > (*hook)();
> > > diff --git a/manual/install.texi b/manual/install.texi
> > > index de9d270..95392ea 100644
> > > --- a/manual/install.texi
> > > +++ b/manual/install.texi
> > > @@ -195,6 +195,11 @@ configure with @option{--disable-werror}.
> > > By default for x86_64, @theglibc{} is built with vector math library.
> > > Use this option to disable vector math library.
> > >
> > > +@item --enable-tunables
> > > +Tunables support allows additional library parameters to be customized at
> > > +runtime for each application. This is an experimental feature and affects
> > > +startup time and is thus disabled by default.
> > > +
> > > @item --build=@var{build-system}
> > > @itemx --host=@var{host-system}
> > > These options are for cross-compiling. If you specify both options and
> > > diff --git a/nptl/nptl-init.c b/nptl/nptl-init.c
> > > index bdbdfed..6667801 100644
> > > --- a/nptl/nptl-init.c
> > > +++ b/nptl/nptl-init.c
> > > @@ -38,6 +38,9 @@
> > > #include <kernel-features.h>
> > > #include <libc-internal.h>
> > > #include <pthread-pids.h>
> > > +#if BUILD_TUNABLES
> > > +# include <tunables/tunables.h>
> > > +#endif
> > >
> > > #ifndef TLS_MULTIPLE_THREADS_IN_TCB
> > > /* Pointer to the corresponding variable in libc. */
> > > @@ -297,7 +300,7 @@ extern void **__libc_dl_error_tsd (void) __attribute__ ((const));
> > > static bool __nptl_initial_report_events __attribute_used__;
> > >
> > > void
> > > -__pthread_initialize_minimal_internal (void)
> > > +__pthread_initialize_minimal_internal (int argc, char **argv, char **envp)
> > > {
> > > #ifndef SHARED
> > > /* Unlike in the dynamically linked case the dynamic linker has not
> > > @@ -311,6 +314,10 @@ __pthread_initialize_minimal_internal (void)
> > > __asm __volatile ("");
> > > #endif
> > >
> > > +#if BUILD_TUNABLES
> > > + __tunables_init (envp);
> > > +#endif
> > > +
> > > /* Minimal initialization of the thread descriptor. */
> > > struct pthread *pd = THREAD_SELF;
> > > __pthread_initialize_pids (pd);
> > > diff --git a/scripts/gen-tunables.awk b/scripts/gen-tunables.awk
> > > new file mode 100644
> > > index 0000000..ccbf789
> > > --- /dev/null
> > > +++ b/scripts/gen-tunables.awk
> > > @@ -0,0 +1,84 @@
> > > +# Generate tunable-list.h from tunables.list
> > > +
> > > +BEGIN {
> > > + ns=""
> > > + top_ns=""
> > > +}
> > > +
> > > +$2 == "{" {
> > > + if (top_ns == "") {
> > > + top_ns = $1
> > > + }
> > > + else if (ns == "") {
> > > + ns = $1
> > > + count = 0
> > > + }
> > > + else {
> > > + printf ("Unexpected occurrence of '{' inside a namespace: %s:%d\n",
> > > + FILENAME, FNR)
> > > + exit 1
> > > + }
> > > +
> > > + next
> > > +}
> > > +
> > > +$1 == "}" {
> > > + if (ns != "") {
> > > + ns = ""
> > > + }
> > > + else if (top_ns != "") {
> > > + top_ns = ""
> > > + }
> > > + else {
> > > + printf ("syntax error: extra }: %s:%d\n", FILENAME, FNR)
> > > + exit 1
> > > + }
> > > + next
> > > +}
> > > +
> > > +{
> > > + if (ns == "") {
> > > + print "Invalid tunable outside a namespace"
> > > + exit 1
> > > + }
> > > + val[top_ns][ns][count] = $1
> > > + count = count + 1
> > > +}
> > > +
> > > +END {
> > > + if (ns != "") {
> > > + print "Unterminated namespace. Is a closing brace missing?"
> > > + exit 1
> > > + }
> > > +
> > > + print "/* Print a full tunable enum name. */"
> > > + print "#include <stddef.h>"
> > > + print "#define TUNABLE_ENUM_NAME(top,ns,id) TUNABLE_ENUM_NAME1 (top,ns,id)"
> > > + print "#define TUNABLE_ENUM_NAME1(top,ns,id) top ## _ ## ns ## _ ## id\n"
> > > +
> > > + print "/* Full name for a tunable is top_ns.tunable_ns.id. */"
> > > + print "#define TUNABLE_NAME_S(top,ns,id) #top \".\" #ns \".\" #id\n"
> > > +
> > > + print "typedef enum"
> > > + print "{"
> > > + for (t in val) {
> > > + for (n in val[t]) {
> > > + for (c in val[t][n]) {
> > > + printf (" TUNABLE_ENUM_NAME(%s, %s, %s),\n", t, n, val[t][n][c]);
> > > + }
> > > + }
> > > + }
> > > + print "} tunable_id_t;\n"
> > > + print "#ifdef TUNABLES_INTERNAL"
> > > + print "static tunable_t tunable_list[] = {"
> > > + for (t in val) {
> > > + for (n in val[t]) {
> > > + for (c in val[t][n]) {
> > > + printf (" {TUNABLE_NAME_S(%s, %s, %s), NULL, NULL, false},\n",
> > > + t, n, val[t][n][c], t, n, val[t][n][c]);
> > > + }
> > > + }
> > > + }
> > > + print "};"
> > > + print "#endif"
> > > +}
> > > diff --git a/tunables/Makefile b/tunables/Makefile
> > > new file mode 100644
> > > index 0000000..44439f6
> > > --- /dev/null
> > > +++ b/tunables/Makefile
> > > @@ -0,0 +1,33 @@
> > > +# Copyright (C) 2016 Free Software Foundation, Inc.
> > > +# This file is part of the GNU C Library.
> > > +
> > > +# The GNU C Library is free software; you can redistribute it and/or
> > > +# modify it under the terms of the GNU Lesser General Public
> > > +# License as published by the Free Software Foundation; either
> > > +# version 2.1 of the License, or (at your option) any later version.
> > > +
> > > +# The GNU C Library is distributed in the hope that it will be useful,
> > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > > +# Lesser General Public License for more details.
> > > +
> > > +# You should have received a copy of the GNU Lesser General Public
> > > +# License along with the GNU C Library; if not, see
> > > +# <http://www.gnu.org/licenses/>.
> > > +
> > > +#
> > > +# Makefile for tunables.
> > > +#
> > > +subdir := tunables
> > > +
> > > +include ../Makeconfig
> > > +
> > > +routines = tunables
> > > +
> > > +$(objpfx)tunables.os: tunable-list.h
> > > +
> > > +tunable-list.h: $(..)scripts/gen-tunables.awk tunables.list
> > > + $(AWK) -f $^ > $@.tmp
> > > + mv $@{.tmp,}
> > > +
> > > +include ../Rules
> > > diff --git a/tunables/README b/tunables/README
> > > new file mode 100644
> > > index 0000000..02a3f7a
> > > --- /dev/null
> > > +++ b/tunables/README
> > > @@ -0,0 +1,93 @@
> > > + TUNABLE FRAMEWORK
> > > + =================
> > > +
> > > +The tunable framework allows modules within glibc to register variables that
> > > +may be tweaked through an environment variable or an API call. It aims to
> > > +enforce a strict namespace rule to bring consistency to naming of these tunable
> > > +environment variables across the project.
> > > +
> > > +ADDING A NEW TUNABLE
> > > +--------------------
> > > +
> > > +The TOP_NAMESPACE is defined by default as 'glibc' and it may be overridden in
> > > +distributions for specific tunables if they want to add their own tunables.
> > > +Downstream implementations are discouraged from using the 'glibc' namespace for
> > > +tunables they don't already have consensus to push upstream.
> > > +
> > > +There are two steps to adding a tunable:
> > > +
> > > +1. Add a tunable ID:
> > > +
> > > +Modules that wish to use the tunables interface must define the
> > > +TUNABLE_NAMESPACE macro. Following this, for each tunable you want to
> > > +register, call the TUNABLE_REGISTER macro.
> > > +
> > > +The TUNABLE_REGISTER macro takes the following arguments:
> > > +
> > > +- id: The short name of the tunable. It will be concatenated with
> > > + the TOP_NAMESPACE and TUNABLE_NAMESPACE to build the full ID of
> > > + the tunable.
> > > +
> > > +- setter: A function that accepts a string input and initializes the
> > > + tunable variable. The prototype of the function should be as
> > > + follows:
> > > +
> > > + void setter_func (const char *)
> > > +
> > > +2. Add to tunables.list:
> > > +
> > > +This file nests tunable ids inside their tunable namespace within curly braces
> > > +and multiple such namespaces are in turn nested inside their top namespaces in
> > > +a similar manner. The malloc and resolv snippets for example could look like
> > > +this:
> > > +
> > > +glibc {
> > > + malloc {
> > > + check
> > > + top_pad
> > > + }
> > > + resolv {
> > > + secure_dns
> > > + }
> > > +}
> > > +
> > > +Add your tunable in this hierarchy using the proper nesting for your namespace
> > > +and top namespace. If the top namespace is not glibc, create a new snippet
> > > +with its own hierarchy, separate from the glibc hierarchy.
> > > +
> > > +ADDING A LEGACY TUNABLE
> > > +-----------------------
> > > +
> > > +One may add tunables for variables that are currently controlled by users with
> > > +environment variables. Multiple such variables should be defined together for
> > > +better performance by using the COMPAT_TUNABLES_NAMESPACE_BEGIN macro, which
> > > +accepts a numeric parameter with the number of tunables you intend to register
> > > +in the block.
> > > +
> > > +Individual tunables are then registered using the COMPAT_TUNABLE_REGISTER and
> > > +COMPAT_TUNABLE_REGISTER_SECURE macros. They take the following parameters:
> > > +
> > > +- id: The tunable ID, similar to the normal way of adding a tunable
> > > +
> > > +- env: The name of the legacy environment variable so that one may
> > > + read its value and use it.
> > > +
> > > +- setter: The setter function, similar to the normal way of adding a
> > > + tunable.
> > > +
> > > +COMPAT_TUNABLE_REGISTER reads the environment variable only for non-setuid
> > > +binaries, whereas COMPAT_TUNABLE_REGISTER_SECURE always attempts to read the
> > > +environment variable. Use COMPAT_TUNABLE_REGISTER_SECURE with care.
> > > +
> > > +You will also have to add the tunable id to tunables/tunables.list.
> > > +
> > > +DO NOT ADD NEW TUNABLES USING THIS MECHANISM!
> > > +
> > > +FUTURE WORK
> > > +-----------
> > > +
> > > +The framework currently only allows a one-time initialization of variables
> > > +through environment variables and in some cases, modification of variables via
> > > +an API call. A future goal for this project is to allow tweaking of some
> > > +values in a running process, possibly through some kind of shared memory
> > > +mechanism.
> > > diff --git a/tunables/Versions b/tunables/Versions
> > > new file mode 100644
> > > index 0000000..4c68b0e
> > > --- /dev/null
> > > +++ b/tunables/Versions
> > > @@ -0,0 +1,8 @@
> > > +# Exports from tunables should only be in the GLIBC_PRIVATE namespace.
> > > +libc {
> > > + GLIBC_PRIVATE {
> > > + __compat_tunables_init_envvars;
> > > + __tunables_init;
> > > + __tunable_register;
> > > + }
> > > +}
> > > diff --git a/tunables/tunable-list.h b/tunables/tunable-list.h
> > > new file mode 100644
> > > index 0000000..2b1caa4
> > > --- /dev/null
> > > +++ b/tunables/tunable-list.h
> > > @@ -0,0 +1,32 @@
> > > +/* Print a full tunable enum name. */
> > > +#include <stddef.h>
> > > +#define TUNABLE_ENUM_NAME(top,ns,id) TUNABLE_ENUM_NAME1 (top,ns,id)
> > > +#define TUNABLE_ENUM_NAME1(top,ns,id) top ## _ ## ns ## _ ## id
> > > +
> > > +/* Full name for a tunable is top_ns.tunable_ns.id. */
> > > +#define TUNABLE_NAME_S(top,ns,id) #top "." #ns "." #id
> > > +
> > > +typedef enum
> > > +{
> > > + TUNABLE_ENUM_NAME(glibc, malloc, check),
> > > + TUNABLE_ENUM_NAME(glibc, malloc, top_pad),
> > > + TUNABLE_ENUM_NAME(glibc, malloc, perturb),
> > > + TUNABLE_ENUM_NAME(glibc, malloc, mmap_threshold),
> > > + TUNABLE_ENUM_NAME(glibc, malloc, trim_threshold),
> > > + TUNABLE_ENUM_NAME(glibc, malloc, mmap_max),
> > > + TUNABLE_ENUM_NAME(glibc, malloc, arena_max),
> > > + TUNABLE_ENUM_NAME(glibc, malloc, arena_test),
> > > +} tunable_id_t;
> > > +
> > > +#ifdef TUNABLES_INTERNAL
> > > +static tunable_t tunable_list[] = {
> > > + {TUNABLE_NAME_S(glibc, malloc, check), NULL, NULL, false},
> > > + {TUNABLE_NAME_S(glibc, malloc, top_pad), NULL, NULL, false},
> > > + {TUNABLE_NAME_S(glibc, malloc, perturb), NULL, NULL, false},
> > > + {TUNABLE_NAME_S(glibc, malloc, mmap_threshold), NULL, NULL, false},
> > > + {TUNABLE_NAME_S(glibc, malloc, trim_threshold), NULL, NULL, false},
> > > + {TUNABLE_NAME_S(glibc, malloc, mmap_max), NULL, NULL, false},
> > > + {TUNABLE_NAME_S(glibc, malloc, arena_max), NULL, NULL, false},
> > > + {TUNABLE_NAME_S(glibc, malloc, arena_test), NULL, NULL, false},
> > > +};
> > > +#endif
> > > diff --git a/tunables/tunables.c b/tunables/tunables.c
> > > new file mode 100644
> > > index 0000000..8834aff
> > > --- /dev/null
> > > +++ b/tunables/tunables.c
> > > @@ -0,0 +1,163 @@
> > > +/* The tunable framework. See the README to know how to use the tunable in
> > > + a glibc module.
> > > +
> > > + Copyright (C) 2016 Free Software Foundation, Inc.
> > > + This file is part of the GNU C Library.
> > > +
> > > + The GNU C Library is free software; you can redistribute it and/or
> > > + modify it under the terms of the GNU Lesser General Public
> > > + License as published by the Free Software Foundation; either
> > > + version 2.1 of the License, or (at your option) any later version.
> > > +
> > > + The GNU C Library is distributed in the hope that it will be useful,
> > > + but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > > + Lesser General Public License for more details.
> > > +
> > > + You should have received a copy of the GNU Lesser General Public
> > > + License along with the GNU C Library; if not, see
> > > + <http://www.gnu.org/licenses/>. */
> > > +
> > > +#include <stddef.h>
> > > +#include <stdlib.h>
> > > +#include <string.h>
> > > +#include <stdbool.h>
> > > +#include <unistd.h>
> > > +#include <sys/param.h>
> > > +
> > > +extern char **__environ;
> > > +
> > > +#define TUNABLES_INTERNAL 1
> > > +#include "tunables.h"
> > > +
> > > +/* We avoid calling into the C library as much as we can, especially functions
> > > + that we know could use tunables in future for some reason or the other. For
> > > + example, we call mmap directly instead of malloc because malloc uses
> > > + tunables. Similarly, the string functions may use tunables in future to
> > > + select optimal routines and we want to make sure we stay out of the way and
> > > + not invoke their initializers before the tunable list has been
> > > + initialized. */
> > > +
> > > +static void
> > > +t_memcpy (char *dest, const char *src, size_t len)
> > > +{
> > > + for (size_t i = 0; i < len; i++)
> > > + dest[i] = src[i];
> > > +}
> > > +
> > > +static int
> > > +t_strcmp (const char *a, const char *b)
> > > +{
> > > + size_t i = 0;
> > > +
> > > + for (i = 0; *a != '\0' && *b != '\0'; i++, a++, b++)
> > > + if (*a != *b)
> > > + return *a - *b;
> > > +
> > > + return *a - *b;
> > > +}
> > > +
> > > +static bool
> > > +get_next_env (char ***envp, char **name, size_t *namelen, char **val)
> > > +{
> > > + char **ev = *envp;
> > > +
> > > + while (ev != NULL && *ev != '\0')
> > > + {
> > > + char *envline = *ev;
> > > + int len = 0;
> > > +
> > > + while (envline[len] != '\0' && envline[len] != '=')
> > > + len++;
> > > +
> > > + /* Just the name and no value, go to the next one. */
> > > + if (envline[len] == '\0')
> > > + continue;
> > > +
> > > + *name = envline;
> > > + *namelen = len;
> > > + *val = &envline[len + 1];
> > > + *envp = ++ev;
> > > +
> > > + return true;
> > > + }
> > > +
> > > + return false;
> > > +}
> > > +
> > > +/* Initialize tunables from the GLIBC_TUNABLES environment variable. The
> > > + variable is set as colon separated name=value pairs. This routine may be
> > > + called from the constructor of either libpthread.so or lic.so. Constructor
> > > + calls for libraries are protected by the dl_load_lock, so this routine won't
> > > + be called concurrently from diferent threads. */
> > > +void
> > > +__tunables_init (char **envp)
> > > +{
> > > + /* Empty for now. */
> > > +}
> > > +libc_hidden_def (__tunables_init)
> > > +
> > > +/* Initialize all tunables using its legacy environment variable values whose
> > > + names are passed in ENVVARS. */
> > > +void
> > > +__compat_tunables_init_envvars (struct compat_tunable_env *envvars, int count)
> > > +{
> > > + /* Traverse through the environment to find environment variables we may need
> > > + to set. */
> > > + char **envp = __environ;
> > > + char *envname = NULL;
> > > + char *envval = NULL;
> > > + size_t len = 0;
> > > +
> > > + while (get_next_env (&envp, &envname, &len, &envval))
> > > + {
> > > + int init_count = 0;
> > > + for (int i = 0; i < count; i++)
> > > + {
> > > + tunable_id_t t = envvars[i].id;
> > > + tunable_t *cur = &tunable_list[t];
> > > +
> > > + /* Skip over tunables that have already been initialized. */
> > > + if (cur->initialized)
> > > + {
> > > + init_count++;
> > > + continue;
> > > + }
> > > +
> > > + const char *name = envvars[i].env;
> > > + char *ename = alloca (len + 1);
> > > +
> > > + t_memcpy (ename, envname, len);
> > > + ename[len] = '\0';
> > > +
> > > + /* We have a match. Initialize and move on to the next line. */
> > > + if (t_strcmp (ename, name) == 0)
> > > + {
> > > + cur->val = envval;
> > > + cur->set (cur->val);
> > > + cur->initialized = true;
> > > + break;
> > > + }
> > > + }
> > > +
> > > + /* All of the tunable envvars have been initialized. */
> > > + if (count == init_count)
> > > + break;
> > > + }
> > > +}
> > > +libc_hidden_def (__compat_tunables_init_envvars)
> > > +
> > > +/* Initialize a tunable and set its value. */
> > > +void
> > > +__tunable_register (tunable_id_t id, tunable_setter_t set_func)
> > > +{
> > > + tunable_t *cur = &tunable_list[id];
> > > +
> > > + cur->set = set_func;
> > > + if (cur->val != NULL)
> > > + {
> > > + set_func (cur->val);
> > > + cur->initialized = true;
> > > + }
> > > +}
> > > +libc_hidden_def (__tunable_register)
> > > diff --git a/tunables/tunables.h b/tunables/tunables.h
> > > new file mode 100644
> > > index 0000000..6aef30e
> > > --- /dev/null
> > > +++ b/tunables/tunables.h
> > > @@ -0,0 +1,110 @@
> > > +/* The tunable framework. See the README to know how to use the tunable in
> > > + a glibc module.
> > > +
> > > + Copyright (C) 2016 Free Software Foundation, Inc.
> > > + This file is part of the GNU C Library.
> > > +
> > > + The GNU C Library is free software; you can redistribute it and/or
> > > + modify it under the terms of the GNU Lesser General Public
> > > + License as published by the Free Software Foundation; either
> > > + version 2.1 of the License, or (at your option) any later version.
> > > +
> > > + The GNU C Library is distributed in the hope that it will be useful,
> > > + but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > > + Lesser General Public License for more details.
> > > +
> > > + You should have received a copy of the GNU Lesser General Public
> > > + License along with the GNU C Library; if not, see
> > > + <http://www.gnu.org/licenses/>. */
> > > +
> > > +#include <stdbool.h>
> > > +
> > > +typedef void (*tunable_setter_t) (const char *);
> > > +
> > > +/* A tunable. */
> > > +struct _tunable
> > > +{
> > > + const char *name;
> > > + const char *val;
> > > + tunable_setter_t set;
> > > + bool initialized;
> > > +};
> > > +
> > > +typedef struct _tunable tunable_t;
> > > +
> > > +#include "tunable-list.h"
> > > +
> > > +struct compat_tunable_env
> > > +{
> > > + tunable_id_t id;
> > > + const char *env;
> > > +};
> > > +
> > > +
> > > +extern void __compat_tunables_init_envvars (struct compat_tunable_env *, int);
> > > +extern void __tunable_register (tunable_id_t, tunable_setter_t);
> > > +extern void __tunables_init (char **);
> > > +
> > > +/* Avoid a PLT reference when called from within libc.so. */
> > > +libc_hidden_proto (__compat_tunables_init_envvars)
> > > +libc_hidden_proto (__tunable_register)
> > > +libc_hidden_proto (__tunables_init)
> > > +
> > > +/* Register a tunable ID in namespace NS inside the glibc top namespace with
> > > + SET as the setter function. */
> > > +#define TUNABLE_REGISTER(ns,id,set) \
> > > + TUNABLE_REGISTER_FULL (TOP_NAMESPACE, ns, id, set)
> > > +
> > > +/* Register a tunable ID in namespace NS inside the TNS top namespace with
> > > + SET as the setter function. */
> > > +#define TUNABLE_REGISTER_FULL(tns,ns,id,set) \
> > > + __tunable_register (TUNABLE_ENUM_NAME (tns, ns, id), (set))
> > > +
> > > +
> > > +
> > > +/* COMPATIBILITY INTERFACE. These macros should be used to define tunables for
> > > + variables that are currently being controlled using environment variables.
> > > + New tunables must not use this interface. */
> > > +
> > > +#define _ADD_COMPAT_TUNABLE_ENV(__id,__env) \
> > > +({ \
> > > + envvars[envvars_cnt].id = TUNABLE_ENUM_NAME (TOP_NAMESPACE, \
> > > + TUNABLE_NAMESPACE, __id); \
> > > + envvars[envvars_cnt++].env = (__env); \
> > > +})
> > > +
> > > +/* Start registering tunables in the current namespace. */
> > > +#define COMPAT_TUNABLES_NAMESPACE_BEGIN(size) \
> > > + { \
> > > + struct compat_tunable_env envvars[size]; \
> > > + int envvars_cnt = 0;
> > > +
> > > +/* Register a tunable. This macro validates that the call is OK and then calls
> > > + tunable_init to do the real work of adding the tunable and setting its value
> > > + based on its environment variable(s). */
> > > +#define COMPAT_TUNABLE_REGISTER(id,env,set) \
> > > +({ \
> > > + assert (envvars_cnt < (sizeof (envvars) \
> > > + / sizeof (struct compat_tunable_env))); \
> > > + if (!__libc_enable_secure) \
> > > + _ADD_COMPAT_TUNABLE_ENV (id, env); \
> > > + TUNABLE_REGISTER (TUNABLE_NAMESPACE, id, set); \
> > > +})
> > > +
> > > +/* Does exactly the same thing as TUNABLE_REGISTER, except that it allows the
> > > + tunable to look for environment variable values even for setuid binaries.
> > > + This is a separate macro and not just another parameter in TUNABLE_REGISTER
> > > + to avoid accidentally setting a secure flag where it is not required. */
> > > +#define COMPAT_TUNABLE_REGISTER_SECURE(id,env,set) \
> > > +({ \
> > > + assert (envvars_cnt < (sizeof (envvars) \
> > > + / sizeof (struct compat_tunable_env))); \
> > > + _ADD_COMPAT_TUNABLE_ENV(id, env); \
> > > + TUNABLE_REGISTER (TUNABLE_NAMESPACE, id, set); \
> > > +})
> > > +
> > > +/* Initialize tunables in the namespace. */
> > > +#define COMPAT_TUNABLES_NAMESPACE_INIT() \
> > > + __compat_tunables_init_envvars (envvars, envvars_cnt); \
> > > + }
> > > diff --git a/tunables/tunables.list b/tunables/tunables.list
> > > new file mode 100644
> > > index 0000000..f1335cc
> > > --- /dev/null
> > > +++ b/tunables/tunables.list
> > > @@ -0,0 +1,12 @@
> > > +glibc {
> > > + malloc {
> > > + check
> > > + top_pad
> > > + perturb
> > > + mmap_threshold
> > > + trim_threshold
> > > + mmap_max
> > > + arena_max
> > > + arena_test
> > > + }
> > > +}
> > > --
> > > 2.5.0
> > >
>