[binutils-gdb] Fix type casts losing typedefs and reimplement "whatis" typedef stripping

[sergiodj+buildbot at sergiodj dot net]

*** TEST RESULTS FOR COMMIT c973d0aa4a2c737ab527ae44a617f1c357e07364 ***

Author: Pedro Alves <palves@redhat.com>
Branch: master
Commit: c973d0aa4a2c737ab527ae44a617f1c357e07364

Fix type casts losing typedefs and reimplement "whatis" typedef stripping

(Ref: https://sourceware.org/ml/gdb/2017-06/msg00020.html)

Assuming int_t is a typedef to int:

 typedef int int_t;

gdb currently loses this expression's typedef:

 (gdb) p (int_t) 0
 $1 = 0
 (gdb) whatis $1
 type = int

or:

 (gdb) whatis (int_t) 0
 type = int

or, to get "whatis" out of the way:

 (gdb) maint print type (int_t) 0
 ...
 name 'int'
 code 0x8 (TYPE_CODE_INT)
 ...

This prevents a type printer for "int_t" kicking in, with e.g.:

 (gdb) p (int_t) 0

>From the manual, we can see that that "whatis (int_t) 0" command
invocation should have printed "type = int_t":

 If @var{arg} is a variable or an expression, @code{whatis} prints its
 literal type as it is used in the source code.  If the type was
 defined using a @code{typedef}, @code{whatis} will @emph{not} print
 the data type underlying the @code{typedef}.
 (...)
 If @var{arg} is a type name that was defined using @code{typedef},
 @code{whatis} @dfn{unrolls} only one level of that @code{typedef}.

That one-level stripping is currently done here, in
gdb/eval.c:evaluate_subexp_standard, handling OP_TYPE:

...
     else if (noside == EVAL_AVOID_SIDE_EFFECTS)
	{
	  struct type *type = exp->elts[pc + 1].type;

	  /* If this is a typedef, then find its immediate target.  We
	     use check_typedef to resolve stubs, but we ignore its
	     result because we do not want to dig past all
	     typedefs.  */
	  check_typedef (type);
	  if (TYPE_CODE (type) == TYPE_CODE_TYPEDEF)
	    type = TYPE_TARGET_TYPE (type);
	  return allocate_value (type);
	}

However, this stripping is reachable in both:

 #1 - (gdb) whatis (int_t)0     # ARG is an expression with a cast to
                                # typedef type.
 #2 - (gdb) whatis int_t        # ARG is a type name.

while only case #2 should strip the typedef.  Removing that code from
evaluate_subexp_standard is part of the fix.  Instead, we make the
"whatis" command implementation itself strip one level of typedefs
when the command argument is a type name.

We then run into another problem, also fixed by this commit:
value_cast always drops any typedefs of the destination type.

With all that fixed, "whatis (int_t) 0" now works as expected:

 (gdb) whatis int_t
 type = int
 (gdb) whatis (int_t)0
 type = int_t

value_cast has many different exit/convertion paths, for handling many
different kinds of casts/conversions, and most of them had to be
tweaked to construct the value of the right "to" type.  The new tests
try to exercise most of it, by trying castin of many different
combinations of types.  With:

 $ make check TESTS="*/whatis-ptype*.exp */gnu_vector.exp */dfp-test.exp"

... due to combinatorial explosion, the testsuite results for the
tests above alone grow like:

 - # of expected passes            246
 + # of expected passes            3811

You'll note that the tests exposed one GCC buglet, filed here:

  Missing DW_AT_type in DW_TAG_typedef of "typedef of typedef of void"
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81267

gdb/ChangeLog:
2017-08-21  Pedro Alves  <palves@redhat.com>

	* eval.c (evaluate_subexp_standard) <OP_TYPE>: Don't dig past
	typedefs.
	* typeprint.c (whatis_exp): If handling "whatis", and expression
	is OP_TYPE, strip one typedef level.  Otherwise don't strip
	typedefs here.
	* valops.c (value_cast): Save "to" type before resolving
	stubs/typedefs.  Use that type as resulting value's type.

gdb/testsuite/ChangeLog:
2017-08-21  Pedro Alves  <palves@redhat.com>

	* gdb.base/dfp-test.c
	(d32_t, d64_t, d128_t, d32_t2, d64_t2, d128_t2, v_d32_t, v_d64_t)
	(v_d128_t, v_d32_t2, v_d64_t2, v_d128_t2): New.
	* gdb.base/dfp-test.exp: Add whatis/ptype/cast tests.
	* gdb.base/gnu_vector.exp: Add whatis/ptype/cast tests.
	* gdb.base/whatis-ptype-typedefs.c: New.
	* gdb.base/whatis-ptype-typedefs.exp: New.
	* gdb.python/py-prettyprint.c (int_type, int_type2): New typedefs.
	(an_int, an_int_type, an_int_type2): New globals.
	* gdb.python/py-prettyprint.exp (run_lang_tests): Add tests
	involving typedefs and cast expressions.
	* gdb.python/py-prettyprint.py (class pp_int_typedef): New.
	(lookup_typedefs_function): New.
	(typedefs_pretty_printers_dict): New.
	(top level): Register lookup_typedefs_function in
	gdb.pretty_printers.