Re: librt and libpthread in alternate prefix

[Poor Yorick <org dot sourceware dot libchelp at pooryorick dot com>]

Mike Frysinger wrote:
> On Tuesday 05 May 2009 04:26:01 Poor Yorick wrote:
>> Mike Frysinger wrote:
>>  > On Monday 04 May 2009 13:59:39 Poor Yorick wrote:
>>  >> Built and installed glibc-2.9 in an alternate location.
>>  >> <prefix>/bin/ldconfig shows:
>>  >>
>>  >>         libpthread.so.0 (libc6, hwcap: 0x8000000000000000, OS ABI:
>>  >> Linux 2.4.20) => /lib/tls/libpthread.so.0
>>  >>         libpthread.so.0 (libc6, hwcap: 0x0008000000000000, OS ABI:
>>  >> Linux 2.4.1) => /lib/i686/libpthread.so.0
>>  >>         libpthread.so.0 (libc6, OS ABI: Linux 2.2.5) =>
>>  >> /lib/libpthread.so.0 libpthread.so.0 (libc6, OS ABI: Linux 2.0.0) =>
>>  >> <prefix>/glibc-2.9/lib/libpthread.so.0
>>  >>
>>  >>
>>  >> Programs that link to /alternate/glibc-2.9/lib/librt.so.1 end up linked
>>  >> to /lib/tls/libpthread.so.0 instead of to
>>  >> /alternate/libc-2.9/lib/libpthread.so.0, presumably because the OS ABI
>>  >> is greater, but this combination causes executables to segfault.
>>  >
>>  > you must either use all of the new glibc or none of it.  mixing glibc
>>  > libs is not supported in any way and will most likely crash.
>>  >
>>  >> I thought the workaround might be to embed an RPATH/RUNPATH in librt.so
>>  >> so that it always finds its sibling libpthread.  I've tried getting
>>  >> paths embedded into libc shared objects before, but the builds never
>>  >> completed succesfully.  Any hints on the right way to do this?
>>  >
>>  > no, using runpaths isnt going to matter because you need to execute the
>>  > new ldso as well.  please review the glibc wiki as there are pages
>>  > covering how to do this.
>>
>> But I am using the new ldso, and using all of the new glibc is precisely
>> what I'm trying to do.  It's just that the new ldso insists on linking
>> against another libpthread on the system.  In the output above, you'll see
>> that the new pthread is in the new ld.so.cache, but there are also other
>> libpthreads listed there which have a higher OS ABI.  I need the new libc
>> to use its own libpthread, but that isn't happening, and I'm not sure how
>> to make it happen, given that I do need the new ld.so.cache to also index a
>> few other paths (/lib, /usr/lib).  A cursory search through the wiki didn't
>> turn up a page that deals with this situation.  A pointer would be much
>> appreciated.
>
> more likely the ldso is doing exactly what it's told.  you probably arent
> setting the --library-path option manually which means it's going to search
> the standard paths and find the old glibc.  setting RPATH on a library or two
> wont matter as they are not the first ELFs processed by glibc -- the
> application being executed is.  please read the wiki i referred to earlier for
> how to properly invoke the ldso.  the values in the ld.so cache are irrelevant
> in terms of the search algorithm as it is exactly as it claims to be and
> nothing more -- a *cache*.
> -mike

Because my goal is to build a software collection in an alternate library path
which uses its own glibc, I'm trying to get the new loader to work from the
alternate location without setting --library-path.  I have embedded the new
ld.so into my test program (findutils), and created a new ld.so.cache which
includes both the /path/to/new/glibc/lib, and /usr/lib.  What doesn't add up is
why the new ld.so finds the correct /path/to/new/glibc/librt.so, but then goes
off and finds /lib/tls/libpthread.so.0 instead /path/to/new/glibc/libpthread.so

I have read many of the pages on the wiki, but am not sure which page you are
referring to.  I don't see much information on how to set up glibc as the
default library and loader for a secondary software collection.  Is this
feasible?

I did find a way to embed RPATH/RUNPATH into the libraries, and though it
solved the problem of finding libpthread, my test programs are failing in
unexpected ways:

    --- Makerules   2008-11-13 16:41:41.000000000 -0500
    +++ Makerules.new       2009-05-05 03:47:23.000000000 -0400
    @@ -448,7 +448,8 @@
    +         -L\$(subst :, -L,\$(rpath-link)) -Wl,-rpath-link=\$(rpath-link) \\
    +         -Wl,-enable-new-dtags -Wl,-rpath,/path/to/alternate/lib


--
Yorick