GDB and typedefs

[Jim Blandy <jimb at red-bean dot com>]

In the process of providing pretty-printers and other tools for
debugging SpiderMonkey, the Mozilla JavaScript interpreter, I've come
across a subtlety of GDB's behavior which is making the work
difficult.

JavaScript is dynamically typed.  The SpiderMonkey 'jsval' type uses
its lowest bits as a type tag (integer vs. string vs. boolean vs.
object), and then uses the upper bits for either the value (integer)
or a pointer to the real value (strings; objects).  Obviously, one
would like to define a pretty-printer for jsval types that uses the
type tag bits to print the value appropriately.

Here's the odd part.  If one prints a variable or the value of a
function declared to have type 'jsval', this works fine.  But if one
casts a value to 'jsval' in GDB, say:

  (gdb) print (jsval) 16

then the pretty-printer doesn't trigger.  In fact, GDB believes that
expression to have type 'jsword' (jsval is defined by 'typedef jsword
jsval'), and thus doesn't find the pretty-printer.  jsword is used in
a number of other contexts in SpiderMonkey, so it wouldn't be
appropriate to define a pretty printer for jsword values.

When the DWARF 2 reader processes a DW_TAG_typedef die, it creates a
symbol whose name is the typedef name (jsval), but whose type is the
definition type (jsword).  It will later create a typedef type named
jsval, but that's not what gets associated with the jsval symbol.  I
haven't written a patch for this, because I'm afraid it will change
the behavior of commands like 'whatis' ('whatis jsval' does exactly
one level of typedef expansion, which is kind of useful, and I bet the
test suite knows this) and I don't have the time right now to fix that
up as well.

There are also some problems with the pretty-printing mechanism
aggressively dereferencing typedefs before looking up a
pretty-printer; obviously, for something like jsval, that's not what
one wants.  I do have a patch for that, which makes archer look for a
pretty-printer for each type in the typedef chain.

Just thought folks might be interested.