This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: Evolution of ELF symbol management
On 11/19/2016 06:25 PM, Zack Weinberg wrote:
On 11/18/2016 10:48 AM, Florian Weimer wrote:
On 11/16/2016 04:55 PM, Zack Weinberg wrote:
First, I don't have a problem with adding __libc_* aliases as
non-default but public alternative names, so that third-party libraries
can avoid namespace pollution in static linkage and/or choose to refer
to symbols that are more likely to get resolved to the C library's
export than to a colliding symbol in the main executable. I'd reserve
that for cases where there is already a demonstrated need, though, such
as libstdc++. I also don't have a problem with a hypothetical policy
that all new symbols that aren't in the current revisions of C+POSIX
should be given an impl-namespace name as their primary definition, with
the user-namespace name being a weak alias (but still the only name used
in the public headers).
If we don't declare the library-safe names in headers, how can libraries
call them?
We could _declare_ the library-safe names in the headers, just not as
the primaries. Like how string.h currently declares both bzero and __bzero.
I have no idea at all why __bzero was introduced. It doesn't look like
a particularly compelling example (similar to __secure_getenv). It may
have been a SunOS thing.
Incidentally, it occurrs to me that the user-namespace name must exist,
for the sake of people using dlsym(RTLD_DEFAULT, "whatever") to access
symbols that they anticipate existing in future revisions of libc
(relative to the one they used at link time).
There also configure scripts which do
extern char whatever();
main () { whatever(); }
for link testing, without including the appropriate header (perhaps
deliberately). That could be considered broken. I'm not sure.
I also do not want to encourage application or library code to reference
the implementation namespace at the source code level. It's ugly, and I
suspect it encourages implementation namespace pollution once
programmers are used to it.
I don't like it either, but how else could a library's headers opt into
these special names on a per-symbol, per-use basis?
In general, headers should avoid using libc types, particularly off_t,
time_t, struct timeval, struct stat, and so on. But there might be
exceptions.
For C, it's not clear at all to me whether we need any kind of opt-in
besides compiling with a particular _*_SOURCE variant (which introduces
the definition). We accept that switching to a new glibc version
requires adjusting source code across the system:
<https://sourceware.org/ml/libc-alpha/2016-10/msg00307.html>
We can't provide complete source level compatibility on updates, and for
things compiled with _GNU_SOURCE, it seems to be an explicit non-goal.
At least that's how I read the previous discussion.
For C++, we might use something based on namespaces to get a clear
separation. However, the problem there is that type names and struct
tags end up in C++ mangled identifiers and thus impact application ABI.
I have no good idea what to do there.
But to be honest, I was mainly concentrating on function symbols. I
understand that other identifiers also present problems. But with
those, you can work around issues by reducing the size of translation
units, at least for C.
Come to think of it, to actually avoid polluting the user namespace, any
library that wants to use these will need a secondary set of libc
headers that declare _only_ the private names. (This is especially
relevant for C++ with so much code in headers.) If we don't do that,
the user-namespace libc prototype (which still exists under your plan)
might conflict with an unrelated application definition.
Yes, that's what using a namespace for C++ would achieve.
I'm less convinced we should do this for C. There is precedent,
Microsoft did exactly this for the POSIX-inspired interfaces in their
libc (functions like _open, _close and so on).
I don't like _that_, because now we have to maintain multiple copies of
the same set of prototypes with different names *in different headers*.
See Josephs' idea about auto-generating public headers. It's definitely
a requirement for this.
But again, I don't see an alternative in C.
Reducing the size of translation units can be used to sidestep the
issue, and you can write bridge functions to glue together parts which
are not directly compatible, providing that there are no symbol clashes
(which is why I'm particularly interested in symbol collisions).
I _do_ have problems with causing these symbols to be used by default.
My main concern is that I think it's tackling the problem in the wrong
place. If we want to back away from the original principle that the
executable always wins, isn't an expanded version of -Bsymbolic mode
what's _really_ wanted?
We could cover a lot of ground if we had a new flag on versioned symbol
definitions which tells the static linker to set a flag on the versioned
symbol reference, and the dynamic linker would then use this flag to
ignore unversioned symbols for binding symbols.
I was imagining a new annotation on _all_ undefined symbols in a shared
object, giving the soname of the object that they were satisfied by at
link time. At load time, 'getrandom!libc.so.6' resolves to the
'getrandom' definition in libc.so.6, ignoring all other definitions of
the same name. If there are symbol versions involved, only the versions
exported by libc.so.6 are considered. For instance,
'getrandom!libc.so.6@GLIBC_2.25' cannot be satisfied by
'getrandom@GLIBC_2.25' exported by libmissing-syscalls.so.1.
We still need to support LD_PRELOAD and interposition of arbitrary
symbols, and not just malloc-related ones, for the benefit of Address
Sanitizer, fakeroot, cwrap, memstomp and other tools.
This is why hard-coding the DSO name does not seem advisable.
(The static linker currently does not add the version of the interposed
symbol when interposition happens at static link time.)
I don't understand this statement, and that makes me worry that you are
trying to solve a different problem from the one I thought you were
talking about -- a problem that I might not even know exists. Can you
elaborate, please? How can a shared object be interposed upon at static
link time? Its own static link has already happened!
When an application is linked against a shared object, if it interposes
any symbols in it, the symbols becomes exported, so that interposition
works at run time (otherwise, it could not happen). You can see an
example here:
$ nm -Dg malloc/tst-interpose-nothread | grep ' T '
The application is *not* compiled with -Bdynamic or something like that,
it happens automatically.
But the symbol version from libc.so.6 is not attached to this symbol
(“nm” would not show it, but you can check with eu-readelf, for example).
You keep talking about symbol versioning but, again, I don't understand
how symbol versioning is relevant,
It's a namespace mechanism for ELF, so it's tempting to use it to
address this problem as well.
and that makes me worry that you're
trying to solve a different problem that I don't even know about. I
thought the issue here was controlling *which library* provides a
symbol, independent of whether the symbol has versions.
I'm not convinced this desirable because of the exceptions I listed
above. Both manual name mangling (which I currently prefer) and symbol
versioning with the no-interpose flag (the one I sketched earlier) would
support them. But only name mangling addresses collisions before the
first static link.
As for what appears in the headers -- again, see above: to solve the
libstdc++/_GNU_SOURCE problem, which I believe I *do* understand, there
needs to be a set of alternative names that libstdc++ headers can
*explicitly* refer to; otherwise we have still polluted the user
namespace. REDIRECTed declarations do not solve that problem. In fact,
with my proposed which-library annotations, we could perfectly well
redirect the mangled names to the normal names:
extern ssize_t __libc_getrandom (void *, size_t, unsigned int)
__asm ("getrandom") __attribute__ ((__bind_to_library ("libc.so.6")));
#ifdef __USE_GNU
extern typeof (__libc_getrandom) getrandom;
#endif
Yes, I this would work (subject to the existing shortcomings).
All this aside, this discussion is still very brainstormy and that makes
me think that we should *not* yet be supplying mangled names for public
use. Once we start doing that we are stuck with it forever, after all.
Contrariwise, we *can* always retrofit __libc_* aliases or whatever once
we know what we ought to be doing.
I think getrandom is special because the name is rather generic, just
like some of the new libm names (where we still might have to introduce
mangling based on feedback from distribution rebuilds; I just don't know
yet).
Thanks,
Florian