Re: A possible libc/dlmopen/pthreads bug

[Carlos O'Donell <carlos at redhat dot com>]

On 01/24/2018 11:34 AM, Szabolcs Nagy wrote:
> On 24/01/18 17:25, Adhemerval Zanella wrote:
>> In fact I think Vivek examples of running process with two different libcs
>> are working mainly because the different glibc define and access the shared
>> resources in same manner.  I bet it will break the moment we change some
>> state internal layout or semantic (for instance the __stack_user struct size).
> 
> even if the two libc has same code and same layout for
> internal data structures it is completely broken:
> there are two copies of the libc global data section
> with inconsistent content.
> 
> if printf called twice via different libcs they get
> buffered independently, but write to the same fd, so
> the observable effect on stdout can be out of order,
> fflush on namespace boundary does not help since the
> locks of one libc don't work on the other, the writes
> can happen concurrently.
> 
> setxid has to coordinate with all threads via signals,
> even with the ones created by the other libc.
> 
> same for atfork handlers: the handlers of the other
> libc has to run too.
> 
> same of tls/tsd dtors: at thread exit the other libc
> tls/tsd data should be freed too.
> 
> the dynamic loader + libc + libpthread needs to have
> one unique instance or some apis won't work across
> all instances.

Correct.

All of these things need to be shared to fix dlmopen
in the "simple" use case e.g. a shared glibc, but everything
else isolated.

The "isolation" use case in which a potentially new
dynamic loader / libc is loaded into the dlmopen namespace will
require a stable API between the loaders to coordinate
such issues, and I'm not entirely sure we want that. So I
would not start by implementing this higher level of isolation.
For example such isolation may mean you must check API compat
level and may be rejected such that all you can do is run
dlmopen, but not call into the namespace for compat reasons
(letting the namespace run on it's own with constructors that
create threads that only live in the namespace).

For now I'm happy with the "simple" use case of a matching
C runtime implementation in both the outer and inner
namespaces.

However, keep in mind that the problems you face with the
C runtime will appear again with higher level libraries.

This concept of "shim" libraries will only work for some
limited set of libraries that are relatively stateless
(and as you see glibc is not).

-- 
Cheers,
Carlos.