Re: dlopen and C++ libraries (vtables)

[Matthew Chapman <enofish at gmail dot com>]

Hi Carlos,

Thanks for your time and brain space.

On Thu, Jul 21, 2011 at 2:20 AM, Carlos O'Donell
<carlos@systemhalted.org> wrote:
> On 7/20/2011 9:39 AM, Matthew Chapman wrote:
>> During relocation processing, the dynamic linker decides:
>> ? ? ? 3199: ? ? file=./libmine.so [0]; ?needed by
>> /usr/lib64/libxerces-c.so [0] (relocation dependency)
>> (_ZTVN11xercesc_2_713XSerializableE)
>
> Why do you call this a false circular dependency?
>
> You can have two DSO, each with a weak symbol, and the dynamic linker
> gets to choose whichever one it wants.

I agree that technically it can choose whichever one it wants - but
now the link maps have both "libmine.so requires libxerces-c.so
[explicit dependency]" and "libxerces-c.so requires libmine.so
[relocation dependency]" which is a problem when it comes to sorting
the fini calls (see below for test case).  I call this a false
dependency because libxerces-c is a system library which does not
really depend on libmine.so that I wrote later.  The dependency was
introduced by the linker via its choice of weak symbols.

>> The problem then is that - due to the false dependency - the libraries
>> then sometimes get unloaded in the wrong order, causing symbol
>> resolution errors (or worse) in destructors. ?I can come up with a
>> test case if needed.
>
> You need to prove they are unloaded in the wrong order.
> You need to provide a reduced test case that shows the behaviour.

I'm attaching a test case.  Sorry it's a bit larger than I'd like, but
I've cut it down as much as I could.

Dependency tree:
  main or dlmain
   \- libplugin [*]
       |- libmine
       |   \- libcplusplus
       \- libresolv [*]

[*] libplugin and libresolv are just dummy placeholders because
_dl_sort_fini is quite sensitive to the initial order of the link map,
and this helps reproduce the problem.

libmine and libcplusplus are two C++ libraries that both contain the
same weak vtables (to trigger the problem) and globals that do
reference counting (to demonstrate the wrong fini order).

$ make

$ ./main
libcplusplus_refcnt
libcplusplus_refcnt: refcnt++
libcplusplus_refcnt: refcnt--
~libcplusplus_refcnt

$ ./dlmain
libcplusplus_refcnt
libcplusplus_refcnt: refcnt++
~libcplusplus_refcnt
oops, destroying libcplusplus_refcnt with refcnt > 0

$ LD_DEBUG=files ./dlmain
...
     25507:     file=./libmine.so [0];  needed by ./libcplusplus.so
[0] (relocation dependency)
...
     25507:     calling fini: ./libcplusplus.so [0]
     25507:
~libcplusplus_refcnt
oops, destroying libcplusplus_refcnt with refcnt > 0
     25507:
     25507:     calling fini: ./libmine.so [0]
     25507:
     25507:     ./libmine.so: error: symbol lookup error: undefined
symbol: _ZN19libcplusplus_refcnt6decrefEv (fatal)


Thanks,
Matt

Attachment: libdependtest-1.0.tar.gz
Description: GNU Zip compressed data