src/gdb ChangeLog ppc-sysv-tdep.c

[brobecke at sourceware dot org]

CVSROOT:	/cvs/src
Module name:	src
Changes by:	brobecke@sourceware.org	2011-10-24 16:51:37

Modified files:
	gdb            : ChangeLog ppc-sysv-tdep.c 

Log message:
	[powerpc] crash trying to allocate memory in inferior
	
	Our testsuite noticed a crash when trying to call a function which
	requires GDB to allocate memory in the inferior. Typically, this
	happens when one of the parameters is a string.  For instance, our
	testcase tries:
	
	(gdb) call debug.trace (me, "You")
	[1]    32737 segmentation fault /path/to/gdb
	
	What happens is that GDB sees the string, and thus tries to allocate
	memory for it in the inferior:
	
	> /* Allocate NBYTES of space in the inferior using the inferior's
	>    malloc and return a value that is a pointer to the allocated
	>    space.  */
	>
	> struct value *
	> value_allocate_space_in_inferior (int len)
	> {
	>   struct objfile *objf;
	>   struct value *val = find_function_in_inferior ("malloc", &objf);
	
	And find_function_in_inferior first searches the symtab in case
	we have debug info.  But, in our case (bareboard powerpc), we don't,
	so it gets "malloc"'s address from the minimal symbols, and builds
	a value whose type is a TYPE_CODE_PTR, not a TYPE_CODE_FUNC.
	
	As a result, when we later try to make the call to malloc, we end up
	inside the powerpc tdep code that has:
	
	> do_ppc_sysv_return_value (struct gdbarch *gdbarch, struct type *func_type,
	[...]
	>   if (func_type
	>       && TYPE_CALLING_CONVENTION (func_type) == DW_CC_GDB_IBM_OpenCL)
	
	The problem is that func_type is not a TYPE_CODE_FUNC, and thus
	the type-specific kind is not TYPE_SPECIFIC_FUNC, and so we do
	TYPE_CALLING_CONVENTION is an invalid access.
	
	Interestingly, the other call to TYPE_CALLING_CONVENTION is correctly
	preceded by a check of the type's TYPE_CODE (making sure that it is
	TYPE_CODE_FUNC).
	
	gdb/ChangeLog:
	
	* ppc-sysv-tdep.c (do_ppc_sysv_return_value): Do not check
	FUNC_TYPE's calling convention if FUNC_TYPE is not a function.

Patches:
http://sourceware.org/cgi-bin/cvsweb.cgi/src/gdb/ChangeLog.diff?cvsroot=src&r1=1.13455&r2=1.13456
http://sourceware.org/cgi-bin/cvsweb.cgi/src/gdb/ppc-sysv-tdep.c.diff?cvsroot=src&r1=1.63&r2=1.64