Consensus: Tuning runtime behaviour with environment variables.

["Carlos O'Donell" <carlos at redhat dot com>]

Community,

I'd like to drive some consensus around the idea of using
environment variables to tune runtime behaviour in glibc.

e.g.

LD_BIND_NOW=1 ./application

or

GLIBC_PTHREAD_MUTEX=elision ./application

I would really appreciate any feedback from the senior members
of the project. This is an issue I'd like to see come to some
kind of consensus.

We now have two sets of patches that have aspects of runtime behaviour
tuning with environment variables:

* Andi Kleen's TSX lock elision patch.
  - Uses env vars to enable, force enable, or tune elision algorithms.
  - Documentation added for these in the glibc manual.

* Siddhesh Poyarekar's default thread stack size patch.
  - Use env var to set default thread stack size in the event the
    program can't be recompiled.
  - Will be documented in the glibc manual.

We have established prior-art for setting runtime behaviour
based on environment variables. It's useful.

We have come to the point where need to make a decision.

We need to tell Andi that his patches which use env vars 
to control elision behaviour are or are not acceptable to the 
project.

I say they are acceptable, but this requires consensus.

---

Roland, 

I will admit that I have a self-serving bias towards the idea of
using environment variables to tune certain aspects of the runtime
behaviour. This makes it difficult for me to act as arbiter,
but I will try my hardest to be an impartial counter of consensus.

I'm trying to understand and enumerate your objections
to environment variables that impact runtime behaviour.

Your most recent post has perhaps the clearest statement:
~~~
Those concerns remain about all new environment variables and I
certainly don't think we should be introducing any new magic
environment variables at this point in the cycle.  Perhaps in the long
run environment variables will in some fashion be part of the
interface for tunables.  But we need to have a thorough contemplation
of the whole area of tunables before we add anything to any part of
the libc interface.  Special new magic interpretation of user-fungible
bits like the environment is among the most dangerous approaches and
we must be extremely circumspect.
~~~
In summary
- Dislike of magic interpretation of user-fungible bits.

  * You mention "magic", as if to imply the new environment variables
    are not sufficiently documented, but don't explicitly mention that.
    In the event that you do mean to say they are not sufficiently
    documented, could you point out which part of Andi's documentation
    patch is insufficient?

    http://sourceware.org/ml/libc-alpha/2013-05/msg00694.html

- User changeable environment variables that impact library runtime
  behaviour is dangerous. 

  * Could you specifically point out what you find dangerous? 
    I see no more danger than we already face when adding a new 
    API or reviewing code changes. How is this different
    than all of the other work we do?

- Thorough contemplation must be had before doing anything.

  * This is perhaps the most difficult claim to refute since it's true
    again for all of the work we undertake.

Reaching consensus:

(a) Guarantee no stable set of env vars.

If it's a matter of clearly documenting the guarantees we give users
about the environment variables and their accepted values, then
I would be happy to draft such a statement and make sure our env
vars follow it.

I'm happy to meet in the middle and tell users they have no
guarantees about the new env vars we will export, and further we
will put double underscores on them to indicate, similar to symbols,
that they are implementation symbols that are subject to change
e.g. __GLIBC_PTHREAD_MUTEX.

(b) Actively work to document what real runtime tuning would look like.

There is a need to tune the GNU C Library at runtime. Not all applications
are the same and not all applications behave the same way.

I've started documenting this here:
http://sourceware.org/glibc/wiki/TuningLibraryRuntimeBehavior

I've been working on env var guidelines here:
http://sourceware.org/glibc/wiki/EnvVarGuide

I'm happy to push forward the definitions we would need to meet
your criteria of a "thorough contemplation" of the problem.

If you tell me the whole idea stinks and that you will never agree
to such an abomination, then we (you and I) would be at an impasse.
At that point I expect the community to break the tie.

Cheers,
Carlos.