[Bug libc/4943] Inconsistent rounding behaviour for sprintf and IEEE doubles

["madcoder at debian dot org" <sourceware-bugzilla at sourceware dot org>]

------- Additional Comments From madcoder at debian dot org  2007-09-22 10:39 -------
Subject: Re:  Inconsistent rounding behaviour for sprintf and IEEE doubles

On Sat, Sep 22, 2007 at 02:09:47PM +0000, paul at inet dot co dot za wrote:
> 
> ------- Additional Comments From paul at inet dot co dot za  2007-09-22 10:09 -------
> (In reply to comment #17)
> > The number 2597.525 does not exist.  There are no numbers between
> > 2597.5249999999996362 and 2597.5250000000000909 in IEEE double.
> > 
> 
> You are wrong.  All numbers between 
>   2597.5249999999996362 and 
>   2597.5250000000000909 are 
>   2597.52500000000 stored to 15 decimals of precision.
> 
> You are incorrect to assume that there exists a floating point number such as 
>   2597.5250000000000909
> since this is simply
>   2597.52500000000 stored with an error of 0.0000000000000909

  No it's not. It's what you _think_ it is. But not what the FPU knows
it is.

  Just think about this one second. What if you had stored
2597.52500000000009 and not 2597.5250000 ? It would have made no
difference, because for your FPU, 2597.5250000, 2597.52500000000009 and
2597.5250000000000909 have the same representation.

  So when you ask for the rounded value the libc has to round
2597.5250000000000909, not 2597.52500000000009, not 2597.525 but
2597.5250000000000909 because it's what is stored in your double value.

  So it rounds to XXX.53, because default rounding scheme is Nearest
even.

  0.125 (or 0.625 or XXXX.625) is another story, because 0.125 indeed is
an existing value in the IEEE world, but it falls right in between 0.12
and 0.13. And here, IEEE says that "Nearest even" rounding will round
half of the time up, half of the time down. For 0.125 it rounds down,
try for 0.0125 it rounds up. And this is done on purpose to spread the
errors evenly accross different computations.

  STOP ARGUING that 2597.5250000000000909 is 2597.52500000000 stored to
15 decimals of precision. You're uterly wrong. 2597.5250* does not
exists in IEEE so once stored, it becomes 2597.5250000000000909. The FPU
has no way to know you meant 2597.525, it sees 2597.5250000000000909,
and 2597.5250000000000909 only. Not {2597.5250000000000909 but only the
15 first digits are meaningful}.



-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=4943

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.