[PATCH] Fix i386 fast math tan x87 fpu stack underflow bug.

[Dave Korn <dave dot korn dot cygwin at googlemail dot com>]

[ Ref: http://cygwin.com/ml/cygwin/2009-11/threads.html#00548 ]

    Hi Jeff,

  We had a bug report on the cygwin list (ref above) about the tan function
when using -ffast-math.  The STC:

#include <stdio.h>
#include <math.h>

int main(void)
{
    double d1 = 0.0;
    double d2 = 0.0;
    d1 = tan(d1);
    d2 = tan(d2);
    (void) printf("d1 = %lg, expecting 0 (or -0)\n", d1);
    (void) printf("d2 = %lg, expecting 0 (or -0)\n", d2);
    return 0;
}

when compiled with "gcc -ffast-math testprog.c -o testprog", produces this output:

> d1 = -0, expecting 0 (or -0)
> d2 = nan, expecting 0 (or -0)

  I analysed it, and what happens is that the x87 fptan instruction leaves two
values on the fpu register stack, a constant +1.0 as well as the actual tan
result.  The _f_tan and _f_tanf routines advance the fp stack pointer to skip
over the useless (to us in this context) constant, but that doesn't work the
same as when you advance the sp of an ordinary stack, because the x87 fpu
tracks which registers are occupied by valid values, and next time the stack
pointer reaches down to this same register, it sees the value is valid (still
contains +1.0) and assumes the stack pointer has wrapped round, leaving QNaNs
in the result and signaling stack underflow as a result.  This breaks both the
printf (that first one actually is a +0 when it's passed to printf, but it
comes out as -0 in the end) and tan itself (the second call to _f_tan returns
NaN).

  The intel manual warns that using fincstp to advance the stack pointer isn't
the same thing as actually popping the stack, and the simple solution is to
add an ffree instruction to mark the register containing the constant unused
before we skip over it.  The attached patch adds ffree instructions before the
only two fincstp instructions I could find by grepping libm/, and it fixes the
testcase.

newlib/ChangeLog:

	* libm/machine/i386/f_tan.S (_f_tan): Free fp stack register
	containing useless constant before advancing fp sp over it.
	(_f_tanf): Likewise.


  OK?

    cheers,
      DaveK


? newlib/libm/autom4te.cache
? newlib/libm/libm.info
? newlib/libm/machine/autom4te.cache
? newlib/libm/machine/i386/autom4te.cache
? newlib/libm/machine/spu/autom4te.cache
Index: newlib/libm/machine/i386/f_tan.S
===================================================================
RCS file: /cvs/src/src/newlib/libm/machine/i386/f_tan.S,v
retrieving revision 1.3
diff -p -u -r1.3 f_tan.S
--- newlib/libm/machine/i386/f_tan.S	20 Dec 2002 21:31:20 -0000	1.3
+++ newlib/libm/machine/i386/f_tan.S	18 Nov 2009 10:03:27 -0000
@@ -29,6 +29,7 @@ SYM (_f_tan):
 	movl esp,ebp
 	fldl 8(ebp)
 	fptan
+	ffree %st(0)
 	fincstp
 
 	leave
Index: newlib/libm/machine/i386/f_tanf.S
===================================================================
RCS file: /cvs/src/src/newlib/libm/machine/i386/f_tanf.S,v
retrieving revision 1.3
diff -p -u -r1.3 f_tanf.S
--- newlib/libm/machine/i386/f_tanf.S	20 Dec 2002 21:31:20 -0000	1.3
+++ newlib/libm/machine/i386/f_tanf.S	18 Nov 2009 10:03:27 -0000
@@ -29,6 +29,7 @@ SYM (_f_tanf):
 	movl esp,ebp
 	flds 8(ebp)
 	fptan
+	ffree %st(0)
 	fincstp
 
 	leave