dlopen with RTLD_NOLOAD returns handle for library that failed to load previously

[David Aldrich <David dot Aldrich at EMEA dot NEC dot COM>]

Hi

We have observed what seems to be a bug in dlopen. The problem first appeared when we moved from glibc 2.5 to glibc 2.12. We also observe it in glibc 2.19. We call dlopen to open a .so file. This call fails (returns null) because of an unresolved symbol in the library. However a subsequent call to dlopen using the RTLD_NOLOAD flag returns a non-zero handle. Our understanding is that a call to dlopen with RTLD_NOLOAD can be used to test if a library is resident or not, so we expect it to return null if the load failed (and this is indeed the behaviour in glibc 2.5). The example code at the bottom of this email shows the problem.

We are wondering if this could be due to the presence of STB_GNU_UNIQUE symbols, as we have seen reports of this causing similar problems. If so is there any workaround for it?

Some background about our application: Our application needs to load a list of .so files that may depend on each other in arbitrary ways. The dependencies are not known to the application. To do this we attempt to load each .so file in turn. If a particular .so file fails (due to missing dependencies), we put it at the back of the queue and try it again after trying all of the others. This continues until we have successfully loaded all of them. We can rewrite this to avoid using the RTLD_NOLOAD option, but we still need a robust way to check if a load attempt has succeeded or not, and we need to be sure that if a load attempt fails then the system is still in a safe state to make another attempt at loading the same file later.

Best regards

David

Test case
========

In this test case, we build shared library lib.so from source file lib.cpp.  We also build application main.out, which attempts to load lib.so and fails, but receives a non-zero handle from dlopen on the second attempt, which is unexpected. Finally, I show the build script we used to build the library and application.

lib.cpp
---------

extern int missing_function();

// Singleton class
class lib_class
{
public:
    static lib_class * instance() { static lib_class inst; return &inst; }
    int dummy() { return 1; }
private:
    lib_class() {}
    ~lib_class() {}
};

int lib_function()
{
    missing_function();          // Missing dependency will cause dlopen to fail
    lib_class::instance()->dummy();
    return 0;
}
//  EOF

main.cpp
-------------

#include <iostream>
#include <string>
#include <dlfcn.h>

using namespace std;

int main()
{
    string libName("./lib.so");

    void* handle = dlopen(libName.c_str(),RTLD_NOW | RTLD_GLOBAL);
    if ( handle==NULL )
    {
        cout << "FAILED TO LOAD " << libName;
        handle = dlopen(libName.c_str(),RTLD_NOW | RTLD_GLOBAL | RTLD_NOLOAD);   
        if ( handle!=NULL ) 
        { 
            cout << " ... BUT HANDLE EXISTS!";  
        }
    }
    else
    {
        cout << "LOADED " << libName;
    }
    cout << endl;
    return(0);
}
//  EOF

Build script
---------------

#!/bin/bash
g++ -c -Wall -m64 -O3 main.cpp -o main.o
g++ -o main.out main.o -Wl,--no-whole-archive -ldl
g++ -c -Wall -m64 -fpic -O3 lib.cpp -o lib.o
g++ -shared -o lib.so lib.o