Re: new release candidate (gsl-1.5.90)

[Brian Gough <bjg at network-theory dot co dot uk>]

Peter Teuben writes:
 >   didn't have any problems on RH9 and FC3, but without looking at
 > the way how you did this, I've often had problems comparing numeric
 > values from linux and solaris, specifically in -g and -O type
 > compilations, because the order of evaluations is different,
 > rounding on Intel's is sometimes done in 80bits reals and passed on
 > to 64/32, where on solaris it's closer to IEEE perhaps...
 > 
 > So when i see lines like
 > 
 > PASS: pidigits autocorrelation (-0.00355099 observed vs -0.00355099
 > expected)
 > 
 > i was surprised to see that it all works so well, unless of course
 > you use linux for the tests also and since i didn't bother with
 > solaris.... well, anyways, you get my drift perhaps.  Is there an
 > importsant consideration in the design of these tests?

Hello,

Here is a description of the testing philosophy:

1. All the tests have an individually chosen tolerance value. This is
generally large enough to accommodate any differences in the results
on different platforms.  The tolerance value is theoretically
motivated, e.g. O(N*eps) or O(sqrt(N)*eps)) depending on the
algorithm, although they are not "strict" bounds as we just choose a
suitable values O(1-10) of any constant factors.

2. Also, the test programs respect the GSL_IEEE_MODE environment
variable so they can be run in both extended precision and double
precision on x86 -- and this gives a good indication of whether things
will work on SPARC (I always do this as part of the release process).
I don't have any other machines so I only test on x86 with GCC.

3. In addition, on the special functions specfun/ directory only the
relative error tolerance is bumped up by a factor of 100 on released
versions just to avoid spurious errors.  Simply due to the large
number of tests and different platforms and compilers there were
always one or two statistical deviations getting reported as bugs.  Of
course, I test without the additional margin of 100 in the relative error.

The error estimates returned by the special functions are always
required to be stringent (no more than 1.5 sigma deviation), so the
extra margin does not apply to those. i.e. the computed value may
deviate on different platforms provided the error estimate increases
correspondingly.  

All of these effects are generally at the level of O(10^-13) to
O(10^-16) so they are not really a big deal most of the time.

 > PS gsl passed, but i'm also going to build it with NEMO to see if
 > that still works.

Thanks.  Compiling against an existing application is one of the best
tests!  It would be quite possible to release something which passed
"make check" but still had problems.  Generally I just compile a few
of the example programs in doc/examples by hand and see if they work
but I don't have anything systematic in that area.

-- 
Brian Gough