Re: Conservative GC and glibc data structures

[Roland McGrath <roland at redhat dot com>]

We are certainly open to making things work better and more cleanly with
things like your collector.  I can only speak for libc issues here.  For
issues in libstdc++ or other things provided by GCC, you need to talk to
the GCC folks.  For glibc, it is almost a good time to be asking.  We
are getting close to making a 2.4 release, and we have supposedly frozen
the ABI.  If we can agree very soon on worthwhile additions that we're
sure are adequate and going to remain satisfactory, then we can still
add them now for the 2.4 ABI.  If it's not done now, then it will wait
for the next glibc ABI release cycle some months away, and it will
likely be many more months before you can expect to see it available on
a lot of users' systems.  If there are any other issues besides TLS data
structures where glibc changes might be worthwhile to improve things for
your collector, please bring those up now.

For your purposes, you don't really need to know about the structure of
TLS data or to enumerate it per se.  You just need to find all of it
from some roots, right?  The TLS data structures are allocated space
containing pointers.  Some of that space is allocated normally with
malloc, and some is part of the thread data structures.  For threads
created by pthread_create, the thread data structures sit at the base of
the stack, which is either user-provided or is allocated with mmap.  For
the initial thread, the thread data structures are allocated by the
dynamic linker early in startup and reside in startup dynamic linker
heap space, which means either the partial page after the dynamic
linker's own .bss, or pages allocated with mmap.  I don't know if you
already examine the process's mappings to consider heap roots set up
before your initialization.  If so, that should be able to find the
initial thread's data structures.  If you can find these thread data
structures and take them as roots, then that should cover (indirectly)
all the TLS data.  (Depending on the kinds of TLS access used, some
modules' TLS data will be in separately malloc'd space and some will be
directly in the thread data structures at the base of thread stacks.)

These same thread structures point to all the thread-specific data that
the pthread_getspecific et al functions access.  Since there isn't a
POSIX interface to enumerate pthread_key_t's created so as to take each
pointer from pthread_getspecific as a root for that thread, I don't know
if you already try to track these.

You didn't propose any specific interface additions, but you mentioned
enumerating memory containing TLS values.  I am glad to consider any
specific proposal on its particular merits.  My instinct is to focus on
the fundamental requirements of what you need in your context, and see
if we can avoid too much detail about the problem case you've cited.
TLS data is C data structures living in the heap, or in part of the
thread stack.  These are kinds of things you already track as roots,
though I don't know the details of how you enumerate heap roots.  If you
take each thread's whole stack from base page boundary to the current
SP, you will come across TLS segments or pointers to malloc'd TLS
segments for that thread.

The initial thread's TLS data is perhaps a special beast in that it's
allocated in the dynamic linker startup heap, and perhaps there aren't
any other kinds of roots you care about in there (just dynamic linker
data structures) and so you don't track that.  I'd like to understand
better how you do go about tracking heap data and how you decide to
enumerate it, before suggesting specific solutions.

If you do already know about the memory regions that are potentially
heap (from looking at the memory mappings) and want a pointer into the
right region, that is available already just given the ELF TLS ABI for
each machine.  On most machines, a particular register is either zero
(no TLS support) or is a pointer with an ABI-specified bias (zero for
some machines) to the thread data structures.  On x86, if %gs matches
%ds then no TLS support is in use, and if not then loading %gs:0 in the
thread itself gets the pointer.  On x86-64, I guess there isn't any
generic way to check whether %fs is set up or not; loading %fs:0 gets
you the pointer, but will fault on an older system that is not using %fs
for thread data.


Thanks,
Roland