Re: Inter-CU DWARF size optimizations and gcc -flto

[Tom Tromey <tromey at redhat dot com>]

Jan> (b) .gdb_index will have limited scope, only to select which
Jan> objfiles to expand, no longer to select which CUs to expand.

I suspect we are going to need a better approach here anyway.
I sometimes hear about programs with more than 800 shared libraries.
If you assume separate debuginfo this means 1600 objfiles.
I think this will just crush most of the existing algorithms in gdb.

Jan> (c) Partial CU expansion Tom Tromey talks about is a must in such case.

I realized I never wrote up how this could work.  The below is sort of a
sketch that devolves into random thoughts.

I have been thinking about it since we discussed it and I think it has a
potentially severe problem.


The basic idea is simple: right now we have two DWARF readers in
dwarf2read.c, the psymtab reader and the full symbol reader.

Right now when we find a psymbol, we expand the whole CU to full
symbols.  This normally isn't too bad -- but there are some CUs out
there in practice that are quite large, and the delay reading them is
noticeable.

So, what if we unified the two readers -- eliminating one source of bugs
-- and also changed CU expansion to be DIE-based.  That is, in symtab.c,
before returning a symbol from a symtab, we would call some back-end
function to expand the symbol.  The DWARF reader would then just read
the DIEs needed to instantiate that one particular symbol plus whatever
dependencies (types usually) it has.

Ok, that sounds good, but there is a problem: struct symbol is really
big, much bigger than a psymbol.  We could just read psymbol-like
structs on our first pass, but we need somewhere to store the DIE offset
for efficient expansion.

We can solve that by updating and applying an old patch that shrinks
psymbol.  Then we can use the saved space to store the DIE -- so this
change can be space-neutral.

However, this neglects the bcache.  In fact, the bcache sinks the whole
project, since DIE offsets will vary by definition.


Well, the DIE offset sinks this particular approach.  Maybe there is
another approach, not space-neutral but also not too bad, that can be
used.  For example, keeping the bcache but having the symtabs contain
both {psymbol+DIE} pairs and fully-expanded symbols (depending on what
has been expanded).


If we went a bit deeper and had hierarchical symbol tables, we could
skip whole DIE subtrees even in the partial reader.

A related idea here that I was idly wondering about is whether we could
make the psymtab reader hierarchical without touching full symbols.


The deeper rewrite seems eventually necessary.  The symbol table code is
pretty horrible, in multiple ways.  However, at least for me it hasn't
yet reached the pain point where we can justify spending months and
months on it, which I think is what it would take.

Your thoughts welcome.

Tom