This is the mail archive of the
libc-help@sourceware.org
mailing list for the glibc project.
Dynamic Linker feature set query
- From: "Darryl L. Miles" <darryl-mailinglists at netbauds dot net>
- To: libc-help at sourceware dot org
- Date: Tue, 29 Jun 2010 09:52:05 +0100
- Subject: Dynamic Linker feature set query
Hi,
I am looking for information about what assistance the dynamic linker,
binutils, ABI, ELF file formats provide to DSO vendors to solve a
particular problem.
The OpenSSL library supports a very wide variety of platforms that it
can be built to run on, from tiny embedded devices to huge non-uniform
systems, it also support multithreading. Due to the wide variety of
platforms (which is a major project goal) it doesn't tie itself to any
particular platform or threading model. This means that every
multithreaded user must initialize themselves the threading model
callbacks before the rest of the library can be consider thread-safe (in
respect of the libraries own set of multithreaded rules).
I use the term "superior executable" to mean to executable that owns the
process and OpenSSL is more often known as a pair of libraries
libcrypto.so and libssl.so.
This presents a major problem with GNU based systems in that it is
possible a "superior executable" doesn't link or use OpenSSL directly
itself, but two or more modules which are loaded via dynamic linking
do. When those two modules are built they have a dependency on OpenSSL,
I shall refer to these modules as "another.so" and "yetanother.so".
Therefore when the "superior executable" loads "another.so" this
implicitly drags in "libcrypto.so" now because the process is
multithreaded the initialization code of "another.so" ultimately
configures up the threading model callbacks before it proceeds to use
"libcrypto.so".
The "superior executable" due to its modular design then calls to load
"yetanother.so" this also implicitly drags in "libcrypto.so" now because
the process is multithreaded the initialization code of "yetanother.so"
ultimately configures up the threading model callbacks before it
proceeds to use "libcrypto.so".
This presents a chicken and egg problem for OpenSSL, because by default
when first loaded it is a single-threaded DSO, only after configuration
does it become multithreaded, the OpenSSL DSO vendor has done all the
correct things in cleanly abstracting the threading support in such as
way that a single DSO can be distributed on a platform. It is also
important that a subsequent user loading OpenSSL doesn't reinitialize
the threading model, this is because another thread has already started
using the previously setup model. This means that between OpenSSL and
the Dynamic Linker there needs to be some arbitration over these affairs.
This problem does not exist when OpenSSL is linked statically. This
pushes the problem to be solely caused by support for dynamic linking,
which to me means the solution should also come from the supporting
operations of the dynamic linker.
So my question first is one of "What assistance do the other building
blocks of a modern GNU based system provide to address this issue ?" if
I better understand what is already there to make use of maybe a
solution can be found.
My initial brainstorm throws up (only 2 of these building block items
would be needed to resolve OpenSSLs predicament, none of the items
should have any change in performance where they are not being used),
this brainstorm is really over "what services can/should the dynamic
linker provide to the runtime executable that would help solve a set of
similar problems" :
* The dynamic linker should allow the "superior executable" to
register itself to receive callbacks in relation to the dynamic linking
operations.
* Support for enumerating all existing loaded libraries and lookup by
name.
* Support for a callback to allow the "superior executable" to
intercept all block load/unload operations (before unload, after load).
Block meaning a single dlopen() might cause multiple DSOs to be mapped.
So this callback would occur just inside the dlopen() from the caller's
perspective.
* Support for a callback to allow the "superior executable" to
intercept all individual load/unload operations (before unload, after
load, possible redirection of file-path before load). Individual
meaning each single DSO file that is loaded, so this would surround the
internal loading operations.
* Support for thread attach/creation and detach/destroy that would be
advisory. (DL would maintain callback chain list internally, handle
thread safey of handlers, provide function API to fire chain, but
application must actively participate by calling the DL function to fire
chain as required, this is a bit like_DllMain on windows except the
application must actively participate to use it).
* Attaching userptr to dynamic linker DSO handles (opaque data to DSO
handles, pref allowing 2+ users to attach independant userptr data so it
would need to be indexable and magic-numbered with a common format).
The DL handles memory allocation and cleanup following the DSO lifecycle.
* Having the DL expose a simple non-recursive low-performance spinlock
API, or in the case of a single threaded system advise that it is a
single threaded environment (in query data) but still provide noop
operations. The DL must know and already be configured for threading
support internally before any DSOs are loaded, but in the case of
OpenSSL because it has abstracted threading support cleanly, the only
thing it that needs to be co-ordinated with the DSO platform is "the
setup of the threading model".
* Having a callback into the DSO itself during load/unload operations,
which also provide a link back to the Dynamic Linker. This means that
.init/.fini is not good enough unless they already provide function
invocation arguments that allow the DSO to call DL API from them. It
would also be necessary for .init to bailout on error, thus doing a
rollback on the dlopen() operation. This optional mechanic could be
provided by exposing a symbol of a specific name presumably with some
kind of API version attached to it with future proofing and progression
accounted for (for example reserving a particular symbol prefix
"__gnu__foobar_dl_dso_controlblock_v1").
Darryl