Problems with i386_register_convert_to_virtual

[Robert Hoehne <robert dot hoehne at gmx dot net>]

in the file i386-tdep.c is (introduced in gdb 5.0) the following function:

--- snip ---
/* Convert data from raw format for register REGNUM in buffer FROM to
   virtual format with type TYPE in buffer TO.  In principle both
   formats are identical except that the virtual format has two extra
   bytes appended that aren't used.  We set these to zero.  */

void
i386_register_convert_to_virtual (int regnum, struct type *type,
				  char *from, char *to)
{
  /* Copy straight over, but take care of the padding.  */
  memcpy (to, from, FPU_REG_RAW_SIZE);
  memset (to + FPU_REG_RAW_SIZE, 0, TYPE_LENGTH (type) - FPU_REG_RAW_SIZE);
}

-- snip --

When I see correct, this function assumes only to be called
for long double types. (see at the comment, where is said, that
the virtual format has two bytes more and in the function the
FPU_REG_RAW_SIZE macro is used) But since this
function is called using the REGISTER_CONVERT_TO_VIRTUAL
macro, it can get be called for any (register) type. But in the case,
that TYPE_LENGTH(type) is for instance 4, then there is at first
a larger block copied than (maybe) allowed and second, the
memset ist called with a negative count (or better a very large
count, since the count argument of memset is size_t which is
very often unsigned).

I can't imagine, that this has never broken in some situations, but
at least for me, when used directly the macro like

  char d[sizeof(long double)];
  char buf[MAX_REGISTER_RAW_SIZE];
  read_register_gen(4, buf);
  REGISTER_CONVERT_TO_VIRTUAL(4, REGISTER_VIRTUAL_TYPE(4), buf, d);

it bombed me out (on DJGPP). (Note here the register number 4, for
the registers 0..3 it probably worked in some way, since I got the
SEGV in a loop starting from 0)

Maybe I have here some errors in my mind, so please correct me, if
I'm wrong.

Robert

******************************************************
* email:   Robert Hoehne <robert.hoehne@gmx.net>     *
* Post:    Am Berg 3, D-09573 Dittmannsdorf, Germany *
* WWW:     http://www.tu-chemnitz.de/~sho/rho        *
******************************************************