Re: Dwarf expertise needed

[Roland McGrath <roland at redhat dot com>]

> Okay so I have been looking at dwarf_getscopes.c trying to find a fix 
> for  https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=230235.

I have not had my head in that particular code in quite some time, and I
have not yet looked at all what's going on in Stan's test case.  Before any
bugs were reported, I already knew there were some issues with the function
from the time I wrote it, which relate to the distinction you are asking about.

> It looks like getscopes is trying to return the scopes from narrowest up 
> to the scope before the concrete inlined instance then the abstract 
> definition of the concrete instance, and then the scopes that contain 
> that. Is this a correct understanding of the code ?
> 
> What I am expecting it to return is the the scopes from narrowest to and 
> including the scope corresponding to the concrete inlined instance, then 
> the function within which it has been inlined and the scopes containing 
> that; as per the dwarf spec. Is this a correct understanding of what the 
> function /should/ do ? And should I fix the function, write a parallel 
> one that does what i want, or is there another code pathway to get it ?

I don't know why you call that "as per the dwarf spec".  (The spec says
what the format means, not what any function does.)

The chief purpose of dwarf_getscopes is to get the lexical scope DIEs
containing the PC, from innermost to outermost.  This means the literal
lexical scope, i.e. for C-like syntax { } groups in the text.  In the
case of a PC inside a concrete inlined instance, it's still the literal
lexical scope of that function.  The lexical scope as expressed by the
generic DWARF structure corresponds exactly to the identifier scope
rules of a simple language like C.  So this is what you need to pass to
dwarf_getscopevar to resolve a C identifier at the scope of this PC.
Take the example:

	static int cu_var = 1;
	inline int a_fn (int a_arg)
	{
	  static int a_var = 1;
 PC1-->	  return ++a_var + a_arg + ++cu_var;
	}

	int b_fn ()
	{
	  static int b_var = 2;
	  int c_fn ()
	  {
	    static int c_var = 3;
	    c_var += a_fn (b_var++);
 PC2-->	    return c_var;
	  }
	  return c_fn ();
	}

The DWARF structure here will look like (much omitted):

<CU>	compile_unit "foo.c"
		variable "cu_var"
<A>		subprogram "a_fn"	<-- abstract inlined instance
			formal_parameter "a_arg"    ^  <--
 				type	 	    |     `
			variable "a_var"    	    |     |
				type		    |	  |
				location 	    |     |
<B>		subprogram "b_fn"		    |     |
			variable "b_var"	    |     |
<C>			subprogram "c_fn"	    |     |
				frame_base	    |	  |
				variable "c_var"    |     |
<C_A>				inlined_subroutine -'     |  <-- concrete
					low_pc PC1        |  	 inlined
					formal_parameter -'	 instance
						location

(My pretty lines indicate DIEs with an abstract_origin ref to another DIE.)

For PC2, dwarf_getscopes should yield [<C>, <B>, <CU>].
For PC1, dwarf_getscopes should yield [<C_A>, <A>, <CU>].

<C_A> is physically inside <C>, but it is lexically where <A> is.  In the
scope context of PC1 (<C_A>), "a_var" is visible and "cu_var" is visible,
but "b_var" and "c_var" are not.  

<C_A> has its own DIE for "a_arg" because its DW_AT_location details are
specific to the concrete inlined instance.  (That one a partial DIE, it has
location but refers to its abstract counterpart for invariant attributes
like type.)  But <C_A> elides the "a_var" DIE entirely because it is the
same for all possible concrete instances of <A>.  This is why <A> appears
as a scope outside <C_A>, or rather, it's why <C_A> appears as a scope
inside <A>.  <C_A> is not really a lexical scope at all, just a physical
instance.  For naive uses like dwarf_getscopevar, it works fine to treat
<C_A> as a scope inside <A>.  But any sophisticated caller needs to check
each "scope" DIE in the array for actually being an inlined_subroutine,
meaning that its containing scope is not really a different scope, but the
two together describe one lexical scope.  e.g., something visualizing
scopes and showing all the identifiers defined in each scope would need to
grok this.

Now imagine "a_var" is called "b_var" and "cu_var" is called "c_var", so it
really matters which scope you mean when you ask for "c_var" by name.  You
can see why it's vital that for "identifiers visible at the PC1 context",
you not be looking at a list that includes <B> or <C>.

So that covers what "return scopes containing" should mean, and what
dwarf_getscopes is intended to do in the interface it has now.  Note that
this is only the beginning of the story for mapping an identifier in a
scope to a DIE in general, though it's the end of the story for C.  In a
language like C++, the literal lexical scope list from dwarf_getscopes
feeds into a language-specific identifier scope engine that knows that
the DIE for a method definition is a scope that can see identifiers from
its class as well as from its lexically containing scope (e.g. the
CU)--which is different from its class DIE being its lexical container,
which it may or may not be.

Now, the trouble with this highly sensical picture comes when you get to
the nitty-gritty of making use of the DIEs you found in those scopes.
It turns out that sometimes you need to know something about the
physical structure of the code, not just the semantic lexical scope.
Say you're at ye olde PC1 and you want to see "a_arg".  Great, it's in
<C_A>.  Now this bugger's location attribute says {DW_OP_fbreg(+24)}.
Well, whose frame_base did he mean?  He meant <C>'s.  But I just
convinced you that <C> better be nowhere in your handy list of DIEs to
consult!

If you take all this and contemplate all the permutations of inlining and
nested functions there could be, it's rather mindbending.  My mind was
bent in this way while writing the function, and I still kept thinking it
could be done with the simple interface yielding one ordered list of
DIEs.  In that frame of mind, I wrote and tweaked and rewrote to whatever
it actually does now, and goodness knows what exactly that came out to
be.  After a libdw with this interface got released, I realized it just
can't do it all for all cases with that interface.  I had other fish to
fry and let it lie, but knew I would have to revisit it and get it all
clear at some point.

Contemplating it now, I don't think it is really so quite confusing what's
required, at least for frame_base.  I probably need to think more about the
nested function cases, and might confuse myself again doing it.  I also
haven't thought of anything other than frame_base that a dwarf_getscopes
caller might need from a physically relevant but lexically disjoint DIE.
Something else I'm overlooking might have different requirements.  To get
the right frame_base, I think a straightforward change of the getscopes
interface can cover it.  As well as the array of lexical scope DIEs, it
would yield a parallel array of Dwarf_Die pointers.  Each scope[i]
corresponds to base_scope[i].  Locations from scope[i] are resolved in the
physical context of base_scope[i].  For simple cases, scope[i]==base_scope[i].
When scope[i] is an inlined_subroutine, base_scope[i] is the containing
subprogram that has the actual code.  So, in all cases, base_scope[i] is
where you need to look for a frame_base attribute when scope[i] leads you to
a location that uses frame_base.

But I'd like to try to think of other things than frame_base one might
need, and think through the nested function cases, before deciding what
exact change the interface should get.

Right now, if your test cases do not require the frame_base attribute,
then we can just work on fixing any bugs in the implementation of the
current definition of the dwarf_getscopes interface as I described it above.  

dwarf_getscopes has always been for the source-level semantic view of
lexical identifier scope.  For other purposes, you might instead be
looking for the physical structural view of the compiled code.  For
example, if you are displaying the running code in an inlined instance
and want to visualize "this code inlined into here", or are synthesizing
fictional call frames for a semantic backtrace including inlined calls.
(If you are only displaying the source location of the caller, you have
the call_{file,line,column} attributes at hand and don't need the
caller's DIE at all.)  dwarf_getscopes_die does exactly this, though its
name and comments would lead you to think it acts like dwarf_getscopes
when given a DIE that is part of a concrete inlined instance tree.
(That's what it probably should do, and there should be a different call
for the simple structural visitor.)  So with 0.128 code, you can use
dwarf_getscopes on a PC, then take a scope[i] of interest such as an
inlined_subroutine DIE, and pass it to dwarf_getscopes_die to see its
caller's scope context.


Thanks,
Roland