Re: vfprintf typing problem

[Paul Eggert <eggert at cs dot ucla dot edu>]

Thanks for hacking on this.  Some further comments:

On 03/30/2012 07:59 PM, David Miller wrote:
> +    if (__builtin_expect ((unsigned int) INT_MAX - (unsigned int) done	      \
> +			  < (Len), 0))					      \

The two casts to 'unsigned int' aren't needed and should be removed.

> -	prec = read_int (&f);
> +	{
> +	  prec = read_int (&f);
> +
> +	  /* The precision was specified in this case as an extremely
> +	     large positive value.  */
> +	  if (prec < 0)

If read_int returns a value greater than INT_MAX, the behavior of the
assignment to 'prec' is undefined, which means that anything can
happen; in particular GCC might simply assume 'prec < 0' yields 0.

Also, because read_int does not check for overflow, a large width or
precision might be treated incorrectly, because read_int might return
(say) 1 even if the integer has overflowed.

So I suggest the following fixes in this area:

  * Change read_int to return 'int', not 'unsigned'.

  * Have read_int check for overflow and return -1 if it occurs.

  * Change all read_int callers to do the right thing when read_int
    returns -1.