dlopen()/__thread behaviour

[robert dot rex at exasol dot com]

Hi all,

I was recently analyzing some memory leak for a project and tracked it
down to the following (maybe uncommon) test case attached to this email.

In short, the leak occurs in the following situation:
(1) A program opens a shared library object with dlopen().
(2) A new thread is created (pthread_create()).
(3) The thread calls a function of the shared library which references a
thread-local storage variable.
(4) The function returns and the thread exits.

(my test case repeats steps 2-4 10000 times for a better demonstration)

-> Result: The thread-local storage variable has been created via
___tls_get_addr() for each thread, but nothing is deallocated.
This only happens in case the dynamic library has been loaded by
dlopen() and not "directly" at load time.
I reproduced this on several platforms (CentOS 5.5, Ubuntu 10.10
[EGLIBC] and some Debian system).

Valgrind seems to confirm this assumption:

# valgrind --leak-check=full ./test
[...]
==18274== HEAP SUMMARY:
==18274==     in use at exit: 41,114 bytes in 10,005 blocks
==18274==   total heap usage: 10,012 allocs, 7 frees, 42,196 bytes allocated
==18274==
==18274== 39,996 bytes in 9,999 blocks are definitely lost in loss
record 7 of 7
==18274==    at 0x4024DB9: memalign (vg_replace_malloc.c:581)
==18274==    by 0x401119B: tls_get_addr_tail (dl-tls.c:529)
==18274==    by 0x4011689: ___tls_get_addr (dl-tls.c:767)
==18274==    by 0x402B5B6: ???
==18274==    by 0x45B7CC8: start_thread (pthread_create.c:304)
==18274==    by 0x411269D: clone (clone.S:130)
==18274==
==18274== LEAK SUMMARY:
[...]

Maybe I am missing something and this is a problem with my understanding
of how thread-local storage works in this case.
But currently, I ran out of explanations. There was a similar issue on
this mailing list
(http://sourceware.org/ml/libc-help/2010-02/msg00016.html), but no
answer at all.

Thanks for any advice,
Robert

/* Compile with "gcc -o test test.c -ldl -lpthread" */
#include <pthread.h>
#include <stdlib.h>
#include <stdio.h>
#include <dlfcn.h>

int main(void)
{
	int i;
	void *handle;
	pthread_t pt;
	void *(*do_something)(void *);

	handle = dlopen("./test_lib.so", RTLD_LAZY);
	if (!handle) {
		perror("dlopen");
		return EXIT_FAILURE;
	}

	do_something = dlsym(handle, "do_something");
	if (!do_something) {
		perror("dlsym");
		return EXIT_FAILURE;
	}

	for (i = 0; i < 10000; i++) {
		if (pthread_create(&pt, NULL, do_something, NULL)) {
			perror("pthread_create");
			return EXIT_FAILURE;
		}

		if (pthread_join(pt, NULL))  {
			perror("pthread_join");
			return EXIT_FAILURE;
		}
	}

	if (dlclose(handle)) {
		perror("dlclose");
		return EXIT_FAILURE;
	}

	return EXIT_SUCCESS;
}

/* Compile with "gcc -o test_lib.so test_lib.c -fPIC -shared" */
#include <stdlib.h>

static __thread int a;

void *do_something(void *arg)
{
	a = 1;
	return NULL;
}