This is the mail archive of the newlib@sourceware.org mailing list for the newlib project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

question on wrong result for small numbers from strtod(), strtof()


I'm compiling for Renesas RX with gcc 4.5 (Newlib 1.18.0)
As RX floating point unit is 32bits the compiler defines double=float
(32bits)
and DBL_DIG=FLT_DIG=6

( the following is exactly the same for strtod() )
When I do a strof() for the string "1.17549434E-37"
I end with the float 1.175494e-035, where the mantissa is ok but the
exponent is wrong.

following the code of strtod.c at line 442

	k = nd < DBL_DIG + 1 ? nd : DBL_DIG + 1;

k is 7 (nd=9 and DBL_DIG=6)
then goes to line 523

	e1 += nd - k;


where e1 (-45) is adjusted to -43 
(the mantissa was converted correctly, rv.d=1.1754943e+8)

then the approximation will end with 1.175494e-035 (instead of
1.175494e-037)

I don't understand all the details of the precision check and the
approximation that follows so I looking for help to solve my problem.

When I compile for double=64 bits and float=32bits so DBL_DIG=15 the results
are correct.

Daniel Calcoen

Attachment: smime.p7s
Description: S/MIME cryptographic signature


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]