This is the mail archive of the
mailing list for the binutils project.
Re: Fwd: Preventing preemption of 'protected' symbols in GNU ld 2.26
- From: Rich Felker <dalias at libc dot org>
- To: Joe Groff <jgroff at apple dot com>
- Cc: "H.J. Lu" <hjl dot tools at gmail dot com>, Alan Modra <amodra at gmail dot com>, Cary Coutant <ccoutant at gmail dot com>, Binutils <binutils at sourceware dot org>
- Date: Tue, 19 Apr 2016 15:47:23 -0400
- Subject: Re: Fwd: Preventing preemption of 'protected' symbols in GNU ld 2.26
- Authentication-results: sourceware.org; auth=none
- References: <AB592ABD-D6D7-4D2F-A0D6-45738F168DC4 at apple dot com> <BEDD88C6-7F80-45DA-9021-10587244AAE5 at apple dot com>
On Tue, Mar 29, 2016 at 12:31:38PM -0700, Joe Groff wrote:
> On Mar 29, 2016, at 8:44 AM, H.J. Lu <firstname.lastname@example.org> wrote:
> > On Mon, Mar 28, 2016 at 4:21 PM, Alan Modra <email@example.com> wrote:
> >> On Mon, Mar 28, 2016 at 03:38:01PM -0700, Cary Coutant wrote:
> >>>>>> Did you look at what the costs were in startup time and dirty pages by using
> >>>>>> copy relocations? What do you do if the size of the definition changes in a
> >>>>>> new version of the library?
> >>>>> There wouldn't be a measurable cost in dirty pages; the copied objects
> >>>>> are simply allocated in bss in the executable.
> >>>> Wouldn't references to the symbol from within the .so need to be relocated to reference the now-canonical copy in the executable?
> >>> No, references from within the .so would have always used the GOT.
> >>> Non-protected global symbols in a shared library are still
> >>> pre-emptible, so they are always indirect, and there's always a
> >>> dynamic relocation for the GOT entry. Whether the prevailing
> >>> definition winds up in the executable or the shared library, the
> >>> dynamic loader still has to bind the symbol and apply the relocation.
> >> HJ's changes to protected visibility meant compiler changes so that
> >> protected visibility in shared libraries is no longer seen as local.
> >> So yes, protected visibility symbols in shared libraries now go
> >> through the GOT. Prior to his changes, they were optimized to a
> >> pc-relative access. Joe is correct in pointing out that shared
> >> libraries needed a change. Bad luck if you're using an older
> >> compiler. Also bad luck if you want to use protected visibility to
> >> optimize your shared library.
> >> HJ also made glibc ld.so changes to ensure the semantics of protected
> >> visibility symbols remain unchanged when multiple shared libraries
> >> define the same protected visibility symbol.
> >> Apparently most people in the gcc and glibc communities saw these
> >> toolchain modifications as fiendishly clever.
> > As I said before, copy relocation and protected symbol are fundamentally
> > incompatible. Since copy relocation is the part of x86 psABIs, I updated
> > GCC, glibc and ld to make protected symbol to work with copy relocation.
> > That is protected symbol may be external, but won't be preempted. The
> > price I paid is that protected symbol won't be accessed via PC-relative
> > relocation within the shared object. To access protected symbol via
> > PC-relative relocation within the shared object, we need to disable copy
> > relocation in executable, which is a psABI change. That is why I proposed
> > to mark the object as such so that we won't get surprise at run-time.
> I think what Cary's arguing (and I honestly would expect) is that
> copying the protected symbol *is* for all intents and purposes a
> preemption. I'd expect copy relocations against protected symbols to
> be linker errors. I guess what's missing for gcc's intended
> optimization is an indication to the compiler that a symbol is
> protected in its home library, to suppress emitting PC-relative
> references to a copy relocation.
As one of the strong advocates for the fix that was made to make
protected visibility work correctly with data symbols, I'd like to
explain why it was the right decision and why it matters. This whole
process is really frustrating to me -- having invested a lot of effort
into getting something important fixed, only to have people come
trying to break it again -- but I'm going to try to be calm and not to
snap at anybody. I was only vaguely aware of this thread until a few
days ago (because I've been trying to make new progress on other
things rather than revisit issues I thought were closed), so I'm just
replying to the beginning of this thread. I'll also try to comment on
particularly important points elsewhere in the thread as I read it,
but I don't want to pile on a bunch of scattered replies that focus on
small details I think others have gotten wrong and ignore the big
>From a programming standpoint, the semantics of protected visibility
need to be free of arch-specific implementation details. Otherwise
programmers can't use it without hard-coding arch-specific details,
which for practical purposes, means good software can't use it at all.
My original motivation for wanting protected visibility to "just work"
was to be able to use:
#pragma GCC visibility push(protected)
around the inclusion of a library's public headers when including them
from the implementation files, This is far from being the only usage
case, and I'll expand on more important usage cases below, but it is
an important one because it allows you to eliminate all GOT/PLT cost
of intra-library function calls without any fine-grained maintenance
of which declarations to apply visibility too (and without any
GNUC-specific clutter in the header files themselves).
I understand that some people want protected visibility to avoid the
GOT for data symbols too for the sake of performance, but for my usage
case, the fact that the semantics were wrong for data symbols meant
that my configure check for "does protected visibility work" would
return "no", and the whole optimization would get turned off.
Anyway, let's move past optimization, because it's a distraction.
After all, with the old (broken) behavior of protected data, one
_could_ work around the above problem and still get the performance
benefits for functions without breaking data by explicitly declaring
all data with default visibility. In fact, this is how I solve the
problem in musl libc, where there are only a small number of data
symbols that should be externally accessible, and maintaining a list
of them is managable:
This is done for the sake of compatibility with a wide range of
toolchains including ones with the old/broken behavior for protected
The actual documented purpose of protected visibility is to prevent
other definitions of a symbol from taking precedence over the one in
the library itself. For example, suppose you have the following
situation: mainapp depends on libA which defines symbol foo with
normal visibility, and libA depends on libB, which also defines foo,
but intentionally with protected visibility so that libB always uses
its own definition, not the one from libA. There is no reasonable way
to obtain the desired semantics here without the current/correct
behavior for protected data. Any other approaches I'm aware of would
either allow libB to bind to the wrong definition of foo, or would
prevent another main app which links libB (but not libA) from being
able to use the symbol foo from libB.
On the other hand, there are plenty of other ways to get the
old/broken behavior if desired. The easiest is to simply use hidden
visibility when you don't want the symbol to be accessible outside the
library. If you _do_ want it to be visible to and usable by other
shared libraries, just not main-programs, this can be achieved using a
hidden alias for a default-visibility symbol:
extern int fast_foo __attribute__((__alias__("foo"),
I expressed that here explicitly for the sake of clarity but of course
in practice people use things like the glibc macro-maze to do this
kind of binding to hidden aliases.