RE: Precision of doubles and stdio

["Dave Korn" <dave dot korn at artimi dot com>]

On 06 March 2006 13:33, Dave Korn wrote:
> On 03 March 2006 21:44, Roberto Bagnara wrote:
>> does this on Linux/i686
>> 
>> $ gcc -W -Wall in.c
>> $ a.out
>> 70.9
>> 70.900000000000005684341886080801486968994140625
>> 
>> and does the following under Cygwin on the same machine:
>> 
>> roberto@quark /tmp
>> $ gcc -W -Wall in.c
>> 
>> roberto@quark /tmp
>> $ ./a.exe
>> 70.9
>> 70.90000000000000568434188608080148696899414

> On 03 March 2006 22:21, Tim Prince wrote:
> 
>> If you haven't gone out of your way to install similar printf() support
>> libraries on cygwin and linux, they will definitely not be the same.  My
>> past reading of various relevant documents convinced me that digits
>> beyond the 17th in formatting of doubles are not required by any
>> standard to be consistent between implementations.  They have no useful
>> function, as 17 digits are sufficient to determine uniquely the
>> corresponding binary value in IEEE 754 format.

  I don't see why we still shouldn't print out well-defined numbers precisely.

  Sure, if you /assume/ that the number in question was originally converted
from an ASCII representation, there's no point converting it back with any
more precision in a way that would falsely suggest it was possible to
discriminate between numbers that have the same representation once scanned,
but if we /don't/ assume the fp number in question was obtained by converting
ascii -> float, then there's no reason to arbitrarily truncate the displayed
value at some point  just because it would convert back to the same fp value
if we re-scanf'ed it.  We should assume the fp number could just as easily be
a precise representation of the result of a calculation.

  Finally, I would suggest that in any case, if the user has explicitly
requested such a long field length, there's even less reason to truncate the
output early.


>> Notice also that any IEEE 754 number can be (exactly!) printed
>> with a finite number of decimal digits.  Finally, notice that
>> writing an algorithm to print them correctly is not rocket science.

>   I agree with you that the number is of course precisely representable. 

> before and probably will do again; we can look into this.  I notice, for
> instance, that newlib doesn't define DECIMAL_DIG or the related symbols
> AFAICT.

  Well, it may not have DECIMAL_DIG, but it has an internal symbol called NDEC
defined in .../src/newlib/libc/stdlib/ldtoa.c that seems to serve more or less
the same purpose:

-----------------------------------snip-----------------------------------
 /* Number of 16 bit words in external x type format */
 #define NE 10

 /* Number of 16 bit words in internal format */
 #define NI (NE+3)

 /* Array offset to exponent */
 #define E 1

 /* Array offset to high guard word */
 #define M 2

 /* Number of bits of precision */
 #define NBITS ((NI-4)*16)

 /* Maximum number of decimal digits in ASCII conversion
  * = NBITS*log10(2)
  */
 #define NDEC (NBITS*8/27)

 /* The exponent of 1.0 */
 #define EXONE (0x3fff)

 /* Maximum exponent digits - base 10 */
 #define MAX_EXP_DIGITS 5
-----------------------------------snip-----------------------------------




  Roberto, just as an experiment I turned NDEC up to 64, and I now get the
exact same output on Cygwin that you get from your linux version of the code.
I can't see any problems that should arise from this, but I don't understand
the reasoning behind the NBITS*log10(2) formula, or to be precise, I don't
understand why that is or isn't enough.  I suspect it has something to do with
the fact that the number of ascii digits needed to represent the conversion of
NBITS bits from binary to decimal isn't the same thing as the number of digits
required to represent the reciprocal of (an NBITS-sized number), but that's
not worded very precisely.... what I'm getting at is that b100.0 can be
represented in 1 decimal sig.fig., but b0.001 takes 3 decimal sig.figs; the
calculation is valid for NBITS binary places above the decimal point, but not
NBITS binary places below it.


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....