This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [PATCH v2 1/2] Add framework for tunables
- From: Siddhesh Poyarekar <sid at reserved-bit dot com>
- To: libc-alpha at sourceware dot org
- Cc: roland at hack dot frob dot com, carlos at redhat dot com, Andi Kleen <andi at firstfloor dot org>, tuliom at linux dot vnet dot ibm dot com, "Paul E. Murphy" <murphyp at linux dot vnet dot ibm dot com>, munroesj at linux dot vnet dot ibm dot com
- Date: Mon, 25 Jan 2016 08:55:06 +0530
- Subject: Re: [PATCH v2 1/2] Add framework for tunables
- Authentication-results: sourceware.org; auth=none
- References: <20160116185545 dot GA17772 at devel dot intra dot reserved-bit dot com>
Ping!
On Sun, Jan 17, 2016 at 12:25:50AM +0530, Siddhesh Poyarekar wrote:
> The tunables framework allows us to uniformly manage and expose global
> variables inside glibc as switches to users. tunables/README has
> instructions for glibc developers to add new tunables.
>
> Tunables support can be enabled by passing the --enable-tunables
> configure flag to the configure script. This patch only adds a
> framework and does not pose any limitations on how tunable values are
> read from the user. It also adds environment variables used in malloc
> behaviour tweaking to the tunables framework as a PoC of the
> compatibility interface.
>
> * manual/install.texi: Add --enable-tunables option.
> * INSTALL: Regenerate.
> * Makeconfig (CPPFLAGS): Define TOP_NAMESPACE.
> (all-subdirs): Add tunables.
> * config.h.in: Add BUILD_TUNABLES.
> * config.make.in: Add build-tunables.
> * configure.ac: Add --enable-tunables option.
> * configure: Regenerate.
> * csu/init-first.c: Include tunables.h.
> (_init): Initialize tunables.
> * nptl/nptl-init.c [BUILD_TUNABLES]: Include tunables.h.
> (__pthread_initialize_minimal_internal): Accept argc, argv and
> envp. Initialize tunables.
> * csu/libc-start.c (LIBC_START_MAIN): Adjust.
> * malloc/arena.c [BUILD_TUNABLES]: Include tunables.h. Define
> TUNABLE_NAMESPACE.
> (tunable_set_mallopt_top_pad, tunable_set_mallopt_perturb,
> tunable_set_mallopt_mmap_max, tunable_set_mallopt_arena_max,
> tunable_set_mallopt_arena_test,
> tunable_set_mallopt_trim_threshold,
> tunable_set_mallopt_mmap_threshold,
> tunable_set_mallopt_check): New functions.
> (ptmalloc_init): Use them and register tunables.
> * scripts/gen-tunables.awk: New file.
> * tunables/Makefile: New file.
> * tunables/README: New file.
> * tunables/Versions: New file.
> * tunables/tunable-list.h: New auto-generated file.
> * tunables/tunables.c: New file.
> * tunables/tunables.h: New file.
> * tunables/tunables.list: New file.
> ---
> INSTALL | 6 ++
> Makeconfig | 9 +++
> config.h.in | 3 +
> config.make.in | 1 +
> configure | 16 +++++
> configure.ac | 10 +++
> csu/init-first.c | 7 ++
> csu/libc-start.c | 4 +-
> malloc/arena.c | 86 +++++++++++++++++++++++++
> manual/install.texi | 5 ++
> nptl/nptl-init.c | 9 ++-
> scripts/gen-tunables.awk | 84 ++++++++++++++++++++++++
> tunables/Makefile | 33 ++++++++++
> tunables/README | 93 +++++++++++++++++++++++++++
> tunables/Versions | 8 +++
> tunables/tunable-list.h | 32 ++++++++++
> tunables/tunables.c | 163 +++++++++++++++++++++++++++++++++++++++++++++++
> tunables/tunables.h | 110 ++++++++++++++++++++++++++++++++
> tunables/tunables.list | 12 ++++
> 19 files changed, 688 insertions(+), 3 deletions(-)
> create mode 100644 scripts/gen-tunables.awk
> create mode 100644 tunables/Makefile
> create mode 100644 tunables/README
> create mode 100644 tunables/Versions
> create mode 100644 tunables/tunable-list.h
> create mode 100644 tunables/tunables.c
> create mode 100644 tunables/tunables.h
> create mode 100644 tunables/tunables.list
>
> diff --git a/INSTALL b/INSTALL
> index c70ea9f..dd06e2e 100644
> --- a/INSTALL
> +++ b/INSTALL
> @@ -164,6 +164,12 @@ will be used, and CFLAGS sets optimization options for the compiler.
> By default for x86_64, the GNU C Library is built with vector math
> library. Use this option to disable vector math library.
>
> +'--enable-tunables'
> + Tunables support allows additional library parameters to be
> + customized at runtime for each application. This is an
> + experimental feature and affects startup time and is thus disabled
> + by default.
> +
> '--build=BUILD-SYSTEM'
> '--host=HOST-SYSTEM'
> These options are for cross-compiling. If you specify both options
> diff --git a/Makeconfig b/Makeconfig
> index 87a22e8..8ac199b 100644
> --- a/Makeconfig
> +++ b/Makeconfig
> @@ -883,6 +883,11 @@ CPPFLAGS = $(config-extra-cppflags) $(CPPUNDEFS) $(CPPFLAGS-config) \
> $(foreach lib,$(libof-$(basename $(@F))) \
> $(libof-$(<F)) $(libof-$(@F)),$(CPPFLAGS-$(lib))) \
> $(CPPFLAGS-$(<F)) $(CPPFLAGS-$(@F)) $(CPPFLAGS-$(basename $(@F)))
> +
> +ifeq (yes,$(build-tunables))
> +CPPFLAGS += -DTOP_NAMESPACE=glibc
> +endif
> +
> override CFLAGS = -std=gnu11 -fgnu89-inline $(config-extra-cflags) \
> $(filter-out %frame-pointer,$(+cflags)) $(+gccwarn-c) \
> $(sysdep-CFLAGS) $(CFLAGS-$(suffix $@)) $(CFLAGS-$(<F)) \
> @@ -1098,6 +1103,10 @@ all-subdirs = csu assert ctype locale intl catgets math setjmp signal \
> crypt localedata timezone rt conform debug mathvec \
> $(add-on-subdirs) dlfcn elf
>
> +ifeq (yes,$(build-tunables))
> +all-subdirs += tunables
> +endif
> +
> ifndef avoid-generated
> # sysd-sorted itself will contain rules making the sysd-sorted target
> # depend on Depend files. But if you just added a Depend file to an
> diff --git a/config.h.in b/config.h.in
> index ec9c8bc..9b07580 100644
> --- a/config.h.in
> +++ b/config.h.in
> @@ -231,4 +231,7 @@
> /* PowerPC32 uses fctidz for floating point to long long conversions. */
> #define HAVE_PPC_FCTIDZ 0
>
> +/* Build glibc with tunables support. */
> +#define BUILD_TUNABLES 0
> +
> #endif
> diff --git a/config.make.in b/config.make.in
> index 05ed6ec..b158f51 100644
> --- a/config.make.in
> +++ b/config.make.in
> @@ -90,6 +90,7 @@ use-nscd = @use_nscd@
> build-hardcoded-path-in-tests= @hardcoded_path_in_tests@
> build-pt-chown = @build_pt_chown@
> enable-lock-elision = @enable_lock_elision@
> +build-tunables = @build_tunables@
>
> # Build tools.
> CC = @CC@
> diff --git a/configure b/configure
> index aa05d49..6fef02c 100755
> --- a/configure
> +++ b/configure
> @@ -659,6 +659,7 @@ multi_arch
> base_machine
> add_on_subdirs
> add_ons
> +build_tunables
> build_pt_chown
> build_nscd
> link_obsolete_rpc
> @@ -773,6 +774,7 @@ enable_systemtap
> enable_build_nscd
> enable_nscd
> enable_pt_chown
> +enable_tunables
> enable_mathvec
> with_cpu
> '
> @@ -1440,6 +1442,7 @@ Optional Features:
> --disable-build-nscd disable building and installing the nscd daemon
> --disable-nscd library functions will not contact the nscd daemon
> --enable-pt_chown Enable building and installing pt_chown
> + --enable-tunables Enable tunables support
> --enable-mathvec Enable building and installing mathvec [default
> depends on architecture]
>
> @@ -3642,6 +3645,19 @@ if test "$build_pt_chown" = yes; then
>
> fi
>
> +# Check whether --enable-tunables was given.
> +if test "${enable_tunables+set}" = set; then :
> + enableval=$enable_tunables; build_tunables=$enableval
> +else
> + build_tunables=no
> +fi
> +
> +
> +if test "$build_tunables" = yes; then
> + $as_echo "#define BUILD_TUNABLES 1" >>confdefs.h
> +
> +fi
> +
> # The abi-tags file uses a fairly simplistic model for name recognition that
> # can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a
> # $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell.
> diff --git a/configure.ac b/configure.ac
> index ee7a3f1..41507ac 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -392,6 +392,16 @@ if test "$build_pt_chown" = yes; then
> AC_DEFINE(HAVE_PT_CHOWN)
> fi
>
> +AC_ARG_ENABLE([tunables],
> + [AS_HELP_STRING([--enable-tunables],
> + [Enable tunables support])],
> + [build_tunables=$enableval],
> + [build_tunables=no])
> +AC_SUBST(build_tunables)
> +if test "$build_tunables" = yes; then
> + AC_DEFINE(BUILD_TUNABLES)
> +fi
> +
> # The abi-tags file uses a fairly simplistic model for name recognition that
> # can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a
> # $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell.
> diff --git a/csu/init-first.c b/csu/init-first.c
> index 77c6e1c..a4e0c54 100644
> --- a/csu/init-first.c
> +++ b/csu/init-first.c
> @@ -26,6 +26,9 @@
> #include <sys/param.h>
> #include <sys/types.h>
> #include <libc-internal.h>
> +#if BUILD_TUNABLES
> +# include <tunables/tunables.h>
> +#endif
>
> #include <ldsodefs.h>
>
> @@ -96,6 +99,10 @@ _init (int argc, char **argv, char **envp)
> #if defined SHARED && !defined NO_CTORS_DTORS_SECTIONS
> __libc_global_ctors ();
> #endif
> +
> +#if BUILD_TUNABLES
> + __tunables_init (__environ);
> +#endif
> }
>
> /* This function is defined here so that if this file ever gets into
> diff --git a/csu/libc-start.c b/csu/libc-start.c
> index f4aa01a..7629fec 100644
> --- a/csu/libc-start.c
> +++ b/csu/libc-start.c
> @@ -32,7 +32,7 @@ extern int __libc_multiple_libcs;
> #include <tls.h>
> #ifndef SHARED
> # include <dl-osinfo.h>
> -extern void __pthread_initialize_minimal (void);
> +extern void __pthread_initialize_minimal (int, char **, char **);
> # ifndef THREAD_SET_STACK_GUARD
> /* Only exported for architectures that don't store the stack guard canary
> in thread local area. */
> @@ -193,7 +193,7 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL),
> /* Initialize the thread library at least a bit since the libgcc
> functions are using thread functions if these are available and
> we need to setup errno. */
> - __pthread_initialize_minimal ();
> + __pthread_initialize_minimal (argc, argv, __environ);
>
> /* Set up the stack checker's canary. */
> uintptr_t stack_chk_guard = _dl_setup_stack_chk_guard (_dl_random);
> diff --git a/malloc/arena.c b/malloc/arena.c
> index 1edb4d4..f773b10 100644
> --- a/malloc/arena.c
> +++ b/malloc/arena.c
> @@ -19,6 +19,11 @@
>
> #include <stdbool.h>
>
> +#if BUILD_TUNABLES
> +# define TUNABLE_NAMESPACE malloc
> +# include <tunables/tunables.h>
> +#endif
> +
> /* Compile-time constants. */
>
> #define HEAP_MIN_SIZE (32 * 1024)
> @@ -332,6 +337,61 @@ ptmalloc_unlock_all2 (void)
> # endif
> #endif /* !NO_THREADS */
>
> +#if BUILD_TUNABLES
> +static void
> +tunable_set_mallopt_top_pad (const char *val)
> +{
> + __libc_mallopt (M_TOP_PAD, atoi (val));
> +}
> +
> +static void
> +tunable_set_mallopt_perturb (const char *val)
> +{
> + __libc_mallopt (M_PERTURB, atoi (val));
> +}
> +
> +static void
> +tunable_set_mallopt_mmap_max (const char *val)
> +{
> + __libc_mallopt (M_MMAP_MAX, atoi (val));
> +}
> +
> +static void
> +tunable_set_mallopt_arena_max (const char *val)
> +{
> + __libc_mallopt (M_ARENA_MAX, atoi (val));
> +}
> +
> +static void
> +tunable_set_mallopt_arena_test (const char *val)
> +{
> + __libc_mallopt (M_ARENA_TEST, atoi (val));
> +}
> +
> +static void
> +tunable_set_mallopt_trim_threshold (const char *val)
> +{
> + __libc_mallopt (M_TRIM_THRESHOLD, atoi (val));
> +}
> +
> +static void
> +tunable_set_mallopt_mmap_threshold (const char *val)
> +{
> + __libc_mallopt (M_MMAP_THRESHOLD, atoi (val));
> +}
> +
> +static void
> +tunable_set_mallopt_check (const char *val)
> +{
> + if (val[0])
> + {
> + __libc_mallopt (M_CHECK_ACTION, (int) (val[0] - '0'));
> + if (check_action != 0)
> + __malloc_check_init ();
> + }
> +}
> +
> +#else
> /* Initialization routine. */
> #include <string.h>
> extern char **_environ;
> @@ -366,6 +426,7 @@ next_env_entry (char ***position)
>
> return result;
> }
> +#endif
>
>
> #ifdef SHARED
> @@ -401,6 +462,29 @@ ptmalloc_init (void)
>
> thread_arena = &main_arena;
> thread_atfork (ptmalloc_lock_all, ptmalloc_unlock_all, ptmalloc_unlock_all2);
> +
> +#if BUILD_TUNABLES
> + COMPAT_TUNABLES_NAMESPACE_BEGIN (8);
> +
> + COMPAT_TUNABLE_REGISTER_SECURE (check, "MALLOC_CHECK_",
> + tunable_set_mallopt_check);
> + COMPAT_TUNABLE_REGISTER (top_pad, "MALLOC_TOP_PAD_",
> + tunable_set_mallopt_top_pad);
> + COMPAT_TUNABLE_REGISTER (perturb, "MALLOC_PERTURB_",
> + tunable_set_mallopt_perturb);
> + COMPAT_TUNABLE_REGISTER (mmap_threshold, "MALLOC_MMAP_THRESHOLD_",
> + tunable_set_mallopt_mmap_threshold);
> + COMPAT_TUNABLE_REGISTER (trim_threshold, "MALLOC_TRIM_THRESHOLD_",
> + tunable_set_mallopt_trim_threshold);
> + COMPAT_TUNABLE_REGISTER (mmap_max, "MALLOC_MMAP_MAX_",
> + tunable_set_mallopt_mmap_max);
> + COMPAT_TUNABLE_REGISTER (arena_max, "MALLOC_ARENA_MAX",
> + tunable_set_mallopt_arena_max);
> + COMPAT_TUNABLE_REGISTER (arena_test, "MALLOC_ARENA_TEST",
> + tunable_set_mallopt_arena_test);
> +
> + COMPAT_TUNABLES_NAMESPACE_INIT ();
> +#else
> const char *s = NULL;
> if (__glibc_likely (_environ != NULL))
> {
> @@ -469,6 +553,8 @@ ptmalloc_init (void)
> if (check_action != 0)
> __malloc_check_init ();
> }
> +#endif
> +
> void (*hook) (void) = atomic_forced_read (__malloc_initialize_hook);
> if (hook != NULL)
> (*hook)();
> diff --git a/manual/install.texi b/manual/install.texi
> index de9d270..95392ea 100644
> --- a/manual/install.texi
> +++ b/manual/install.texi
> @@ -195,6 +195,11 @@ configure with @option{--disable-werror}.
> By default for x86_64, @theglibc{} is built with vector math library.
> Use this option to disable vector math library.
>
> +@item --enable-tunables
> +Tunables support allows additional library parameters to be customized at
> +runtime for each application. This is an experimental feature and affects
> +startup time and is thus disabled by default.
> +
> @item --build=@var{build-system}
> @itemx --host=@var{host-system}
> These options are for cross-compiling. If you specify both options and
> diff --git a/nptl/nptl-init.c b/nptl/nptl-init.c
> index bdbdfed..6667801 100644
> --- a/nptl/nptl-init.c
> +++ b/nptl/nptl-init.c
> @@ -38,6 +38,9 @@
> #include <kernel-features.h>
> #include <libc-internal.h>
> #include <pthread-pids.h>
> +#if BUILD_TUNABLES
> +# include <tunables/tunables.h>
> +#endif
>
> #ifndef TLS_MULTIPLE_THREADS_IN_TCB
> /* Pointer to the corresponding variable in libc. */
> @@ -297,7 +300,7 @@ extern void **__libc_dl_error_tsd (void) __attribute__ ((const));
> static bool __nptl_initial_report_events __attribute_used__;
>
> void
> -__pthread_initialize_minimal_internal (void)
> +__pthread_initialize_minimal_internal (int argc, char **argv, char **envp)
> {
> #ifndef SHARED
> /* Unlike in the dynamically linked case the dynamic linker has not
> @@ -311,6 +314,10 @@ __pthread_initialize_minimal_internal (void)
> __asm __volatile ("");
> #endif
>
> +#if BUILD_TUNABLES
> + __tunables_init (envp);
> +#endif
> +
> /* Minimal initialization of the thread descriptor. */
> struct pthread *pd = THREAD_SELF;
> __pthread_initialize_pids (pd);
> diff --git a/scripts/gen-tunables.awk b/scripts/gen-tunables.awk
> new file mode 100644
> index 0000000..ccbf789
> --- /dev/null
> +++ b/scripts/gen-tunables.awk
> @@ -0,0 +1,84 @@
> +# Generate tunable-list.h from tunables.list
> +
> +BEGIN {
> + ns=""
> + top_ns=""
> +}
> +
> +$2 == "{" {
> + if (top_ns == "") {
> + top_ns = $1
> + }
> + else if (ns == "") {
> + ns = $1
> + count = 0
> + }
> + else {
> + printf ("Unexpected occurrence of '{' inside a namespace: %s:%d\n",
> + FILENAME, FNR)
> + exit 1
> + }
> +
> + next
> +}
> +
> +$1 == "}" {
> + if (ns != "") {
> + ns = ""
> + }
> + else if (top_ns != "") {
> + top_ns = ""
> + }
> + else {
> + printf ("syntax error: extra }: %s:%d\n", FILENAME, FNR)
> + exit 1
> + }
> + next
> +}
> +
> +{
> + if (ns == "") {
> + print "Invalid tunable outside a namespace"
> + exit 1
> + }
> + val[top_ns][ns][count] = $1
> + count = count + 1
> +}
> +
> +END {
> + if (ns != "") {
> + print "Unterminated namespace. Is a closing brace missing?"
> + exit 1
> + }
> +
> + print "/* Print a full tunable enum name. */"
> + print "#include <stddef.h>"
> + print "#define TUNABLE_ENUM_NAME(top,ns,id) TUNABLE_ENUM_NAME1 (top,ns,id)"
> + print "#define TUNABLE_ENUM_NAME1(top,ns,id) top ## _ ## ns ## _ ## id\n"
> +
> + print "/* Full name for a tunable is top_ns.tunable_ns.id. */"
> + print "#define TUNABLE_NAME_S(top,ns,id) #top \".\" #ns \".\" #id\n"
> +
> + print "typedef enum"
> + print "{"
> + for (t in val) {
> + for (n in val[t]) {
> + for (c in val[t][n]) {
> + printf (" TUNABLE_ENUM_NAME(%s, %s, %s),\n", t, n, val[t][n][c]);
> + }
> + }
> + }
> + print "} tunable_id_t;\n"
> + print "#ifdef TUNABLES_INTERNAL"
> + print "static tunable_t tunable_list[] = {"
> + for (t in val) {
> + for (n in val[t]) {
> + for (c in val[t][n]) {
> + printf (" {TUNABLE_NAME_S(%s, %s, %s), NULL, NULL, false},\n",
> + t, n, val[t][n][c], t, n, val[t][n][c]);
> + }
> + }
> + }
> + print "};"
> + print "#endif"
> +}
> diff --git a/tunables/Makefile b/tunables/Makefile
> new file mode 100644
> index 0000000..44439f6
> --- /dev/null
> +++ b/tunables/Makefile
> @@ -0,0 +1,33 @@
> +# Copyright (C) 2016 Free Software Foundation, Inc.
> +# This file is part of the GNU C Library.
> +
> +# The GNU C Library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +
> +# The GNU C Library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> +# Lesser General Public License for more details.
> +
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with the GNU C Library; if not, see
> +# <http://www.gnu.org/licenses/>.
> +
> +#
> +# Makefile for tunables.
> +#
> +subdir := tunables
> +
> +include ../Makeconfig
> +
> +routines = tunables
> +
> +$(objpfx)tunables.os: tunable-list.h
> +
> +tunable-list.h: $(..)scripts/gen-tunables.awk tunables.list
> + $(AWK) -f $^ > $@.tmp
> + mv $@{.tmp,}
> +
> +include ../Rules
> diff --git a/tunables/README b/tunables/README
> new file mode 100644
> index 0000000..02a3f7a
> --- /dev/null
> +++ b/tunables/README
> @@ -0,0 +1,93 @@
> + TUNABLE FRAMEWORK
> + =================
> +
> +The tunable framework allows modules within glibc to register variables that
> +may be tweaked through an environment variable or an API call. It aims to
> +enforce a strict namespace rule to bring consistency to naming of these tunable
> +environment variables across the project.
> +
> +ADDING A NEW TUNABLE
> +--------------------
> +
> +The TOP_NAMESPACE is defined by default as 'glibc' and it may be overridden in
> +distributions for specific tunables if they want to add their own tunables.
> +Downstream implementations are discouraged from using the 'glibc' namespace for
> +tunables they don't already have consensus to push upstream.
> +
> +There are two steps to adding a tunable:
> +
> +1. Add a tunable ID:
> +
> +Modules that wish to use the tunables interface must define the
> +TUNABLE_NAMESPACE macro. Following this, for each tunable you want to
> +register, call the TUNABLE_REGISTER macro.
> +
> +The TUNABLE_REGISTER macro takes the following arguments:
> +
> +- id: The short name of the tunable. It will be concatenated with
> + the TOP_NAMESPACE and TUNABLE_NAMESPACE to build the full ID of
> + the tunable.
> +
> +- setter: A function that accepts a string input and initializes the
> + tunable variable. The prototype of the function should be as
> + follows:
> +
> + void setter_func (const char *)
> +
> +2. Add to tunables.list:
> +
> +This file nests tunable ids inside their tunable namespace within curly braces
> +and multiple such namespaces are in turn nested inside their top namespaces in
> +a similar manner. The malloc and resolv snippets for example could look like
> +this:
> +
> +glibc {
> + malloc {
> + check
> + top_pad
> + }
> + resolv {
> + secure_dns
> + }
> +}
> +
> +Add your tunable in this hierarchy using the proper nesting for your namespace
> +and top namespace. If the top namespace is not glibc, create a new snippet
> +with its own hierarchy, separate from the glibc hierarchy.
> +
> +ADDING A LEGACY TUNABLE
> +-----------------------
> +
> +One may add tunables for variables that are currently controlled by users with
> +environment variables. Multiple such variables should be defined together for
> +better performance by using the COMPAT_TUNABLES_NAMESPACE_BEGIN macro, which
> +accepts a numeric parameter with the number of tunables you intend to register
> +in the block.
> +
> +Individual tunables are then registered using the COMPAT_TUNABLE_REGISTER and
> +COMPAT_TUNABLE_REGISTER_SECURE macros. They take the following parameters:
> +
> +- id: The tunable ID, similar to the normal way of adding a tunable
> +
> +- env: The name of the legacy environment variable so that one may
> + read its value and use it.
> +
> +- setter: The setter function, similar to the normal way of adding a
> + tunable.
> +
> +COMPAT_TUNABLE_REGISTER reads the environment variable only for non-setuid
> +binaries, whereas COMPAT_TUNABLE_REGISTER_SECURE always attempts to read the
> +environment variable. Use COMPAT_TUNABLE_REGISTER_SECURE with care.
> +
> +You will also have to add the tunable id to tunables/tunables.list.
> +
> +DO NOT ADD NEW TUNABLES USING THIS MECHANISM!
> +
> +FUTURE WORK
> +-----------
> +
> +The framework currently only allows a one-time initialization of variables
> +through environment variables and in some cases, modification of variables via
> +an API call. A future goal for this project is to allow tweaking of some
> +values in a running process, possibly through some kind of shared memory
> +mechanism.
> diff --git a/tunables/Versions b/tunables/Versions
> new file mode 100644
> index 0000000..4c68b0e
> --- /dev/null
> +++ b/tunables/Versions
> @@ -0,0 +1,8 @@
> +# Exports from tunables should only be in the GLIBC_PRIVATE namespace.
> +libc {
> + GLIBC_PRIVATE {
> + __compat_tunables_init_envvars;
> + __tunables_init;
> + __tunable_register;
> + }
> +}
> diff --git a/tunables/tunable-list.h b/tunables/tunable-list.h
> new file mode 100644
> index 0000000..2b1caa4
> --- /dev/null
> +++ b/tunables/tunable-list.h
> @@ -0,0 +1,32 @@
> +/* Print a full tunable enum name. */
> +#include <stddef.h>
> +#define TUNABLE_ENUM_NAME(top,ns,id) TUNABLE_ENUM_NAME1 (top,ns,id)
> +#define TUNABLE_ENUM_NAME1(top,ns,id) top ## _ ## ns ## _ ## id
> +
> +/* Full name for a tunable is top_ns.tunable_ns.id. */
> +#define TUNABLE_NAME_S(top,ns,id) #top "." #ns "." #id
> +
> +typedef enum
> +{
> + TUNABLE_ENUM_NAME(glibc, malloc, check),
> + TUNABLE_ENUM_NAME(glibc, malloc, top_pad),
> + TUNABLE_ENUM_NAME(glibc, malloc, perturb),
> + TUNABLE_ENUM_NAME(glibc, malloc, mmap_threshold),
> + TUNABLE_ENUM_NAME(glibc, malloc, trim_threshold),
> + TUNABLE_ENUM_NAME(glibc, malloc, mmap_max),
> + TUNABLE_ENUM_NAME(glibc, malloc, arena_max),
> + TUNABLE_ENUM_NAME(glibc, malloc, arena_test),
> +} tunable_id_t;
> +
> +#ifdef TUNABLES_INTERNAL
> +static tunable_t tunable_list[] = {
> + {TUNABLE_NAME_S(glibc, malloc, check), NULL, NULL, false},
> + {TUNABLE_NAME_S(glibc, malloc, top_pad), NULL, NULL, false},
> + {TUNABLE_NAME_S(glibc, malloc, perturb), NULL, NULL, false},
> + {TUNABLE_NAME_S(glibc, malloc, mmap_threshold), NULL, NULL, false},
> + {TUNABLE_NAME_S(glibc, malloc, trim_threshold), NULL, NULL, false},
> + {TUNABLE_NAME_S(glibc, malloc, mmap_max), NULL, NULL, false},
> + {TUNABLE_NAME_S(glibc, malloc, arena_max), NULL, NULL, false},
> + {TUNABLE_NAME_S(glibc, malloc, arena_test), NULL, NULL, false},
> +};
> +#endif
> diff --git a/tunables/tunables.c b/tunables/tunables.c
> new file mode 100644
> index 0000000..8834aff
> --- /dev/null
> +++ b/tunables/tunables.c
> @@ -0,0 +1,163 @@
> +/* The tunable framework. See the README to know how to use the tunable in
> + a glibc module.
> +
> + Copyright (C) 2016 Free Software Foundation, Inc.
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library; if not, see
> + <http://www.gnu.org/licenses/>. */
> +
> +#include <stddef.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <stdbool.h>
> +#include <unistd.h>
> +#include <sys/param.h>
> +
> +extern char **__environ;
> +
> +#define TUNABLES_INTERNAL 1
> +#include "tunables.h"
> +
> +/* We avoid calling into the C library as much as we can, especially functions
> + that we know could use tunables in future for some reason or the other. For
> + example, we call mmap directly instead of malloc because malloc uses
> + tunables. Similarly, the string functions may use tunables in future to
> + select optimal routines and we want to make sure we stay out of the way and
> + not invoke their initializers before the tunable list has been
> + initialized. */
> +
> +static void
> +t_memcpy (char *dest, const char *src, size_t len)
> +{
> + for (size_t i = 0; i < len; i++)
> + dest[i] = src[i];
> +}
> +
> +static int
> +t_strcmp (const char *a, const char *b)
> +{
> + size_t i = 0;
> +
> + for (i = 0; *a != '\0' && *b != '\0'; i++, a++, b++)
> + if (*a != *b)
> + return *a - *b;
> +
> + return *a - *b;
> +}
> +
> +static bool
> +get_next_env (char ***envp, char **name, size_t *namelen, char **val)
> +{
> + char **ev = *envp;
> +
> + while (ev != NULL && *ev != '\0')
> + {
> + char *envline = *ev;
> + int len = 0;
> +
> + while (envline[len] != '\0' && envline[len] != '=')
> + len++;
> +
> + /* Just the name and no value, go to the next one. */
> + if (envline[len] == '\0')
> + continue;
> +
> + *name = envline;
> + *namelen = len;
> + *val = &envline[len + 1];
> + *envp = ++ev;
> +
> + return true;
> + }
> +
> + return false;
> +}
> +
> +/* Initialize tunables from the GLIBC_TUNABLES environment variable. The
> + variable is set as colon separated name=value pairs. This routine may be
> + called from the constructor of either libpthread.so or lic.so. Constructor
> + calls for libraries are protected by the dl_load_lock, so this routine won't
> + be called concurrently from diferent threads. */
> +void
> +__tunables_init (char **envp)
> +{
> + /* Empty for now. */
> +}
> +libc_hidden_def (__tunables_init)
> +
> +/* Initialize all tunables using its legacy environment variable values whose
> + names are passed in ENVVARS. */
> +void
> +__compat_tunables_init_envvars (struct compat_tunable_env *envvars, int count)
> +{
> + /* Traverse through the environment to find environment variables we may need
> + to set. */
> + char **envp = __environ;
> + char *envname = NULL;
> + char *envval = NULL;
> + size_t len = 0;
> +
> + while (get_next_env (&envp, &envname, &len, &envval))
> + {
> + int init_count = 0;
> + for (int i = 0; i < count; i++)
> + {
> + tunable_id_t t = envvars[i].id;
> + tunable_t *cur = &tunable_list[t];
> +
> + /* Skip over tunables that have already been initialized. */
> + if (cur->initialized)
> + {
> + init_count++;
> + continue;
> + }
> +
> + const char *name = envvars[i].env;
> + char *ename = alloca (len + 1);
> +
> + t_memcpy (ename, envname, len);
> + ename[len] = '\0';
> +
> + /* We have a match. Initialize and move on to the next line. */
> + if (t_strcmp (ename, name) == 0)
> + {
> + cur->val = envval;
> + cur->set (cur->val);
> + cur->initialized = true;
> + break;
> + }
> + }
> +
> + /* All of the tunable envvars have been initialized. */
> + if (count == init_count)
> + break;
> + }
> +}
> +libc_hidden_def (__compat_tunables_init_envvars)
> +
> +/* Initialize a tunable and set its value. */
> +void
> +__tunable_register (tunable_id_t id, tunable_setter_t set_func)
> +{
> + tunable_t *cur = &tunable_list[id];
> +
> + cur->set = set_func;
> + if (cur->val != NULL)
> + {
> + set_func (cur->val);
> + cur->initialized = true;
> + }
> +}
> +libc_hidden_def (__tunable_register)
> diff --git a/tunables/tunables.h b/tunables/tunables.h
> new file mode 100644
> index 0000000..6aef30e
> --- /dev/null
> +++ b/tunables/tunables.h
> @@ -0,0 +1,110 @@
> +/* The tunable framework. See the README to know how to use the tunable in
> + a glibc module.
> +
> + Copyright (C) 2016 Free Software Foundation, Inc.
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library; if not, see
> + <http://www.gnu.org/licenses/>. */
> +
> +#include <stdbool.h>
> +
> +typedef void (*tunable_setter_t) (const char *);
> +
> +/* A tunable. */
> +struct _tunable
> +{
> + const char *name;
> + const char *val;
> + tunable_setter_t set;
> + bool initialized;
> +};
> +
> +typedef struct _tunable tunable_t;
> +
> +#include "tunable-list.h"
> +
> +struct compat_tunable_env
> +{
> + tunable_id_t id;
> + const char *env;
> +};
> +
> +
> +extern void __compat_tunables_init_envvars (struct compat_tunable_env *, int);
> +extern void __tunable_register (tunable_id_t, tunable_setter_t);
> +extern void __tunables_init (char **);
> +
> +/* Avoid a PLT reference when called from within libc.so. */
> +libc_hidden_proto (__compat_tunables_init_envvars)
> +libc_hidden_proto (__tunable_register)
> +libc_hidden_proto (__tunables_init)
> +
> +/* Register a tunable ID in namespace NS inside the glibc top namespace with
> + SET as the setter function. */
> +#define TUNABLE_REGISTER(ns,id,set) \
> + TUNABLE_REGISTER_FULL (TOP_NAMESPACE, ns, id, set)
> +
> +/* Register a tunable ID in namespace NS inside the TNS top namespace with
> + SET as the setter function. */
> +#define TUNABLE_REGISTER_FULL(tns,ns,id,set) \
> + __tunable_register (TUNABLE_ENUM_NAME (tns, ns, id), (set))
> +
> +
> +
> +/* COMPATIBILITY INTERFACE. These macros should be used to define tunables for
> + variables that are currently being controlled using environment variables.
> + New tunables must not use this interface. */
> +
> +#define _ADD_COMPAT_TUNABLE_ENV(__id,__env) \
> +({ \
> + envvars[envvars_cnt].id = TUNABLE_ENUM_NAME (TOP_NAMESPACE, \
> + TUNABLE_NAMESPACE, __id); \
> + envvars[envvars_cnt++].env = (__env); \
> +})
> +
> +/* Start registering tunables in the current namespace. */
> +#define COMPAT_TUNABLES_NAMESPACE_BEGIN(size) \
> + { \
> + struct compat_tunable_env envvars[size]; \
> + int envvars_cnt = 0;
> +
> +/* Register a tunable. This macro validates that the call is OK and then calls
> + tunable_init to do the real work of adding the tunable and setting its value
> + based on its environment variable(s). */
> +#define COMPAT_TUNABLE_REGISTER(id,env,set) \
> +({ \
> + assert (envvars_cnt < (sizeof (envvars) \
> + / sizeof (struct compat_tunable_env))); \
> + if (!__libc_enable_secure) \
> + _ADD_COMPAT_TUNABLE_ENV (id, env); \
> + TUNABLE_REGISTER (TUNABLE_NAMESPACE, id, set); \
> +})
> +
> +/* Does exactly the same thing as TUNABLE_REGISTER, except that it allows the
> + tunable to look for environment variable values even for setuid binaries.
> + This is a separate macro and not just another parameter in TUNABLE_REGISTER
> + to avoid accidentally setting a secure flag where it is not required. */
> +#define COMPAT_TUNABLE_REGISTER_SECURE(id,env,set) \
> +({ \
> + assert (envvars_cnt < (sizeof (envvars) \
> + / sizeof (struct compat_tunable_env))); \
> + _ADD_COMPAT_TUNABLE_ENV(id, env); \
> + TUNABLE_REGISTER (TUNABLE_NAMESPACE, id, set); \
> +})
> +
> +/* Initialize tunables in the namespace. */
> +#define COMPAT_TUNABLES_NAMESPACE_INIT() \
> + __compat_tunables_init_envvars (envvars, envvars_cnt); \
> + }
> diff --git a/tunables/tunables.list b/tunables/tunables.list
> new file mode 100644
> index 0000000..f1335cc
> --- /dev/null
> +++ b/tunables/tunables.list
> @@ -0,0 +1,12 @@
> +glibc {
> + malloc {
> + check
> + top_pad
> + perturb
> + mmap_threshold
> + trim_threshold
> + mmap_max
> + arena_max
> + arena_test
> + }
> +}
> --
> 2.5.0
>