This is the mail archive of the libc-help@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Dynamic library loading inner workings


On Tue, 7 Dec 2010 08:26:18 -0500
"Carlos O'Donell" <carlos@systemhalted.org> wrote:

> On Sun, Dec 5, 2010 at 1:43 PM, Andreas Fink
> <andreas.fink85@googlemail.com> wrote:
> > I've been reading a bit the last days about this topic (really interesting, as I must admit).
> >
> > My problem looks like this:
> > Abstractly seen I have two types of libraries. Libraries of type A are called libA, libB, libC. Libraries of type B are called libX, libY, libZ. Libraries of type A are quite stable and good to ship. Libraries of type B are very unstable and in development. However libraries of type B link against libraries of type A, and libraries of type B are implemented as plugins (so they are loaded via dlopen in my program).
> > libX links against libA, libB
> > libY links against libB, libC
> > libZ links against libA, libC
> >
> > Loading all three of them (in the order libX, libY and then libZ) makes it impossible to only unload libX (since libA and libB have a dependency on libX because of relocation). So I would need to unload all libX, libY, and libZ to be able to reload a recompiled libX.
> 
> The shared library X can only be unloaded if there are no relocations
> against it, but those types of relocations are very rare. Show me
> proof that libA and libB actually have a relocation against libX? What
> relocation is it?

A relocation in the case of libQtGui and libQtCore is for example:
     30478:     symbol=_ZTI31QAbstractEventDispatcherPrivate;  lookup in file=/usr/lib/qt4/libQtGui.so [0]
     30478:
     30478:     file=/usr/lib/qt4/libQtGui.so [0];  needed by /usr/lib64/qt4/libQtCore.so.4 [0] (relocation dependency)

Unmangled this would be: typeinfo for QAbstractEventDispatcherPrivate
This kind of relocation happens quite frequently for classes with virtual functions. Both libQtGui and libQtCore would have the typeinfo to QAbstractEventDispatcherPrivate, but libQtGui wins since it was dlopened and libQtCore is only loaded as a dependency to libQtGui.
However I could not yet reduce it to a simple example where this kind of relocation happens. Another relocation for libQtGui/libQtCore:
_ZTV27QEventDispatcherGlibPrivate (or unmangled: vtable for QEventDispatcherGlibPrivate).

> 
> The problem you are describing exists, but it would be a fundamental
> design flaw in your plugin system to have libA and libB reference
> something in libX. After all a relocation is a references from one
> location to another. The only way that would exist is if libA or libB
> touched global data in libX or called libX functions (and not via a
> callback function pointer).
You may call it a fundamental design flaw, I call it using virtual functions ;)

> 
> The question you should be asking yourself is "Why does libA and libB
> reference something in libX and how do I fix that?"
> 
> It is possibly a flaw in your plugin system to have this dependency.
> 
> > Anyway it does not seem that my problem can be solved with the current linker. I have another question to the order libraries are loaded.
> > If I dlopen(libX) and look at the order in which symbols are tried to be resolved I see that symbols in libX are first looked at before libA and libB. Thus when relocation of libA happens a symbol is found in libX (which would also exist in libA, but the one in libX is used and hence the relocation dependency of libA to libX occurs).
> 
> OK.
> 
> > So my new question:
> > Why is the linker not putting dependent libraries before the loaded library, i.e. the current situation looks like this:
> > dlopen(libX) --> symbol resolution in this order:
> > (... some libraries loaded at program startup)
> > libX
> > libA
> > libB
> 
> That's because you loaded libX first, and given ELF symbol resolution
> rules it is allowed to override libA functions.
> 
> Change the libX function names so they don't override libA functions.
> 
> > my preferred solution:
> > (... some libraries loaded at startup)
> > libA
> > libB
> > libX
> >
> > Then the relocation dependency of libA on libX would not occur since all symbols in libA can be resolved with symbols in libraries prior to libA or libA itself.
> 
> Then dlopen libA first, *or* build libA such that those symbols are
> always resolved within libA.
How can I build libA, such that all symbols are resolved within libA? Is there some compile switch?

> 
> > To summarize my whole point:
> > I understand now, why libQtGui cannot be unloaded, but I don't understand why symbol resolution happens in a (for me) quite unnatural order... My natural order for symbol resolution on dlopen(libX) would be: Libraries loaded at program startup --> dependent libraries of libX (libA, libB) --> libX itself
> 
> That's not a natural order for ELF symbol resolution rules. Those
> rules require that code loaded first can override weak symbols in code
> that is loaded later. This allows user code to override some symbols
> in shared libraries to implement custom functions that do
> user-specific things e.g. override malloc in  libc.
> 
> Cheers,
> Carlos.

I think the problem here is the c++ language with its RTTI?

Regards
Andreas


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]