Re: Ideas for rewrite of libm-test

[Andreas Jaeger <aj at suse dot de>]

On Wed, 22 Sep 1999, Geoff Keating wrote:

> > Date: Tue, 21 Sep 1999 22:41:46 +0100
> > From: Philip Blundell <Philip.Blundell@pobox.com>
> > 
> > >It just seems that basing the test cases on what the math functions
> > >produce (as is done now in practise, and will be done formally with
> > >your scheme) is doing it the wrong way around.
> > 
> > You can make arguments both ways.  If the required epsilons are greater than 
> > the calculated error margin then clearly something is wrong (in either libc, 
> > the compiler or the floatng point system). But it seems that compiler and 
> > FP bugs are quite common, and we don't really want "make check" to fail on 
> > platforms where the floating point support is substandard.
> 
> I do want 'make check' to fail, at least on ppc.  ppc should always
> do at least as well as the calculated error margin, or something is
> very wrong.  I think sparc and ARM are similar, they're both normal
> IEEE machines.  For Alpha, it should also be similar enough that the
> calculated margins will do---certainly, you want to know if not.

That's the reason why I suggested platform specific error margins that
will be merged with a general file.  The general file would contain the
theoretical limits - and the platform specific files would contain
additional, platform specific, bounds if neccessary.  If we agree that the
theoretical bounds are all that's needed for e.g. PPC, then there will be
no platform specific file.

> 
> Of course, sometimes the platform is sufficiently different to IEEE
> arithmetic that the maximum errors should be recalculated, in which
> case you can allow some flexibility---I'm thinking of x86 here.
> 
> > Perhaps it would be worth including two sets; the theoretical values
> > and those that are actually achievable in practice on each platform.
> > Or alternatively with a more sophisticated test suite that allowed
> > for expected failures I suppose this would come out in the wash.
> > What happened to Zack's dejagnu patches?
> 
> Hmmm.  This makes sense, but not quite the way you phrased it.  I'd
> like to have two _test cases_, one the theoretical values, and one the
> (better) values that were achieved, either of which could fail.  This
> way I'd know about regressions, too; and new porters would know when
> their port's FP support wasn't working.

This should be possible (but I did not consider it before).  In the
framework I suggested we could generate a file with only the theoretical
limits and use it.

Andreas
-- 
 Andreas Jaeger   aj@suse.de    aj@arthur.rhein-neckar.de
  for pgp-key finger ajaeger@aixd1.rhrk.uni-kl.de