This is the mail archive of the newlib@sources.redhat.com mailing list for the newlib project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Serious __fpclassify bug


There is an apparent typo with serious consequences in newlib/libm/common/s_fpclassify.c. I've attached a patch for newlib 1.11.0. (cvsweb seems to indicate there haven't been any changes to that file recently.)

I first noticed the problem when fmax(-12.0, 0.0) returned -12.0. I looked through the code, and since the values in question in __fpclassify{f,d} are unsigned ints, some of the comparisons are always false (e.g., (w >= a && w <= b) where a > b). As a result, negative numbers were being classified as FP_NAN and thence being returned without comparison by fmax(). I tried the patched version, and now fmax(-12.0, 0.0) returns 0.0 like it should.

I was only concerned about fmax() and fmin(); I'm not sure what else is affected by this, though I would guess it's a fair bit.

Joe Vornehm
The MITRE Corporation


--- newlib/libm/common/s_fpclassify.c.orig	2002-06-07 17:59:56.000000000 -0400
+++ newlib/libm/common/s_fpclassify.c	2003-06-27 14:30:23.000000000 -0400
@@ -16,10 +16,10 @@
   if (w == 0x00000000 || w == 0x80000000)
     return FP_ZERO;
   else if ((w >= 0x00800000 && w <= 0x7f7fffff) ||
-           (w >= 0xff7fffff && w <= 0x80800000))
+           (w >= 0x80800000 && w <= 0xff7fffff))
     return FP_NORMAL;
   else if ((w >= 0x00000001 && w <= 0x007fffff) ||
-           (w >= 0x807fffff && w <= 0x80000001))
+           (w >= 0x80000001 && w <= 0x807fffff))
     return FP_SUBNORMAL;
   else if (w == 0x7f800000 || w == 0xff800000)
     return FP_INFINITE;
@@ -38,10 +38,10 @@
       (msw == 0x80000000 && lsw == 0x00000000))
     return FP_ZERO;
   else if ((msw >= 0x00100000 && msw <= 0x7fefffff) ||
-           (msw >= 0xffefffff && msw <= 0x80100000))
+           (msw >= 0x80100000 && msw <= 0xffefffff))
     return FP_NORMAL;
   else if ((msw >= 0x00000000 && msw <= 0x000fffff) ||
-           (msw >= 0x800fffff && msw <= 0x80000000))
+           (msw >= 0x80000000 && msw <= 0x800fffff))
     /* zero is already handled above */
     return FP_SUBNORMAL;
   else if ((msw == 0x7ff00000 && lsw == 0x00000000) ||

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]