Don't include individual test ulps in libm-test-ulps

["Joseph S. Myers" <joseph at codesourcery dot com>]

As recently discussed
<https://sourceware.org/ml/libc-alpha/2014-02/msg00670.html>, it
doesn't seem particularly useful for libm-test-ulps files to contain
huge amounts of data on ulps for individual tests; just the global
maximum observed ulps for each function, together with the
verification of exceptions, errno and special results such as
infinities and NaNs for each test, suffices to verify that a
function's behavior on the given test inputs is within the expected
accuracy.  Removing this data reduces source tree churn caused by
updates to these files when libm tests are added, and reduces the
frequency with which testsuite additions actually need libm-test-ulps
changes at all.

Accordingly, this patch removes that data, so that individual tests
get checked against the global bounds for the given function and only
generate an error if those are exceeded.  Tested x86_64 (including
verifying that if an ulps value is artificially reduced, the tests do
indeed fail as they should and "make regen-ulps" generates the
expected changes).  It's still expected for architecture maintainers
to regenerate these files from scratch during each release freeze, but
the diffs from such regeneration will be a lot smaller now (and so a
lot easier to review manually for ulps that shouldn't be there,
although I also intend to make libm-test.inc bound the ulps it outputs
for regeneration so no such manual review is needed at all).

(5MB of libm-test-ulps diffs omitted below.)

2014-03-04  Joseph Myers  <joseph@codesourcery.com>

	* math/libm-test.inc (struct ulp_data): Don't refer to ulps for
	individual tests in comment.
	(libm-test-ulps.h): Don't refer to test_ulps in #include comment.
	(prev_max_error): New variable.
	(prev_real_max_error): Likewise.
	(prev_imag_max_error): Likewise.
	(compare_ulp_data): Don't refer to test names in comment.
	(find_test_ulps): Remove function.
	(find_function_ulps): Likewise.
	(find_complex_function_ulps): Likewise.
	(init_max_error): Take function name as argument.  Look up ulps
	for that function.
	(print_ulps): Remove function.
	(print_max_error): Use prev_max_error instead of calling
	find_function_ulps.
	(print_complex_max_error): Use prev_real_max_error and
	prev_imag_max_error instead of calling find_complex_function_ulps.
	(check_float_internal): Take max_ulp parameter instead of calling
	find_test_ulps.  Don't call print_ulps.
	(check_float): Update call to check_float_internal.
	(check_complex): Update calls to check_float_internal.
	(START): Pass argument to init_max_error.
	* math/gen-libm-test.pl (%results): Don't include "kind"
	information.
	(parse_ulps): Don't handle ulps of individual tests.
	(print_ulps_file): Likewise.
	(output_ulps): Likewise.
	* manual/libm-err-tab.pl (parse_ulps): Don't handle ulps of
	individual tests.
	* sysdeps/aarch64/libm-test-ulps: Remove individual test ulps.
	* sysdeps/alpha/fpu/libm-test-ulps: Likewise.
	* sysdeps/arm/libm-test-ulps: Likewise.
	* sysdeps/i386/fpu/libm-test-ulps: Likewise.
	* sysdeps/ia64/fpu/libm-test-ulps: Likewise.
	* sysdeps/m68k/coldfire/fpu/libm-test-ulps: Likewise.
	* sysdeps/m68k/m680x0/fpu/libm-test-ulps: Likewise.
	* sysdeps/microblaze/libm-test-ulps: Likewise.
	* sysdeps/mips/mips32/libm-test-ulps: Likewise.
	* sysdeps/mips/mips64/libm-test-ulps: Likewise.
	* sysdeps/powerpc/fpu/libm-test-ulps: Likewise.
	* sysdeps/powerpc/nofpu/libm-test-ulps: Likewise.
	* sysdeps/s390/fpu/libm-test-ulps: Likewise.
	* sysdeps/sh/libm-test-ulps: Likewise.
	* sysdeps/sparc/fpu/libm-test-ulps: Likewise.
	* sysdeps/tile/libm-test-ulps: Likewise.
	* sysdeps/x86_64/fpu/libm-test-ulps: Likewise.

ports/ChangeLog.hppa:
2014-03-04  Joseph Myers  <joseph@codesourcery.com>

	* sysdeps/hppa/fpu/libm-test-ulps: Remove individual test ulps.

diff --git a/manual/libm-err-tab.pl b/manual/libm-err-tab.pl
index 7ac9af2..5cd0d1c 100755
--- a/manual/libm-err-tab.pl
+++ b/manual/libm-err-tab.pl
@@ -104,7 +104,7 @@ sub find_files {
 # Parse ulps file
 sub parse_ulps {
   my ($file, $platform) = @_;
-  my ($test, $type, $float, $eps, $kind);
+  my ($test, $type, $float, $eps);
 
   # $type has the following values:
   # "normal": No complex variable
@@ -116,10 +116,6 @@ sub parse_ulps {
     # ignore comments and empty lines
     next if /^#/;
     next if /^\s*$/;
-    if (/^Test/) {
-      $kind = 'test';
-      next;
-    }
     if (/^Function: /) {
       if (/Real part of/) {
 	s/Real part of //;
@@ -131,11 +127,8 @@ sub parse_ulps {
 	$type = 'normal';
       }
       ($test) = ($_ =~ /^Function:\s*\"([a-zA-Z0-9_]+)\"/);
-      $kind = 'fct';
       next;
     }
-    # Only handle maximal errors of functions
-    next if ($kind eq 'test');
     if (/^i?(float|double|ldouble):/) {
       ($float, $eps) = split /\s*:\s*/,$_,2;
       if ($eps eq 'fail') {
diff --git a/math/gen-libm-test.pl b/math/gen-libm-test.pl
index a1c528d..919f0f2 100755
--- a/math/gen-libm-test.pl
+++ b/math/gen-libm-test.pl
@@ -21,8 +21,6 @@
 # Note that functions and tests share the same namespace.
 
 # Information about tests are stored in: %results
-# $results{$test}{"kind"} is either "fct" or "test" and flags whether this
-# is a maximal error of a function or a single test.
 # $results{$test}{"type"} is the result type, e.g. normal or complex.
 # $results{$test}{"has_ulps"} is set if deltas exist.
 # In the following description $type and $float are:
@@ -484,7 +482,7 @@ sub generate_testfile {
 # Parse ulps file
 sub parse_ulps {
   my ($file) = @_;
-  my ($test, $type, $float, $eps, $kind);
+  my ($test, $type, $float, $eps);
 
   # $type has the following values:
   # "normal": No complex variable
@@ -496,21 +494,6 @@ sub parse_ulps {
     # ignore comments and empty lines
     next if /^#/;
     next if /^\s*$/;
-    if (/^Test/) {
-      if (/Real part of:/) {
-	s/Real part of: //;
-	$type = 'real';
-      } elsif (/Imaginary part of:/) {
-	s/Imaginary part of: //;
-	$type = 'imag';
-      } else {
-	$type = 'normal';
-      }
-      s/^.+\"(.*)\".*$/$1/;
-      $test = $_;
-      $kind = 'test';
-      next;
-    }
     if (/^Function: /) {
       if (/Real part of/) {
 	s/Real part of //;
@@ -522,7 +505,6 @@ sub parse_ulps {
 	$type = 'normal';
       }
       ($test) = ($_ =~ /^Function:\s*\"([a-zA-Z0-9_]+)\"/);
-      $kind = 'fct';
       next;
     }
     if (/^i?(float|double|ldouble):/) {
@@ -540,7 +522,6 @@ sub parse_ulps {
       } elsif ($type eq 'normal') {
 	$results{$test}{'type'} = 'normal';
       }
-      $results{$test}{'kind'} = $kind;
       next;
     }
     print "Skipping unknown entry: `$_'\n";
@@ -569,39 +550,9 @@ sub print_ulps_file {
   $last_fct = '';
   open NEWULP, ">$file" or die ("Can't open $file: $!");
   print NEWULP "# Begin of automatic generation\n";
-  # first the function calls
-  foreach $test (sort keys %results) {
-    next if ($results{$test}{'kind'} ne 'test');
-    foreach $type ('real', 'imag', 'normal') {
-      if (exists $results{$test}{$type}) {
-	if (defined $results{$test}) {
-	  ($fct) = ($test =~ /^(\w+)\s/);
-	  if ($fct ne $last_fct) {
-	    $last_fct = $fct;
-	    print NEWULP "\n# $fct\n";
-	  }
-	}
-	if ($type eq 'normal') {
-	  print NEWULP "Test \"$test\":\n";
-	} elsif ($type eq 'real') {
-	  print NEWULP "Test \"Real part of: $test\":\n";
-	} elsif ($type eq 'imag') {
-	  print NEWULP "Test \"Imaginary part of: $test\":\n";
-	}
-	foreach $float (@all_floats) {
-	  if (exists $results{$test}{$type}{'ulp'}{$float}) {
-	    print NEWULP "$float: ",
-	    &clean_up_number ($results{$test}{$type}{'ulp'}{$float}),
-	    "\n";
-	  }
-	}
-      }
-    }
-  }
   print NEWULP "\n# Maximal error of functions:\n";
 
   foreach $fct (sort keys %results) {
-    next if ($results{$fct}{'kind'} ne 'fct');
     foreach $type ('real', 'imag', 'normal') {
       if (exists $results{$fct}{$type}) {
 	if ($type eq 'normal') {
@@ -656,7 +607,7 @@ sub get_all_ulps_for_test {
 sub output_ulps {
   my ($file, $ulps_filename) = @_;
   my ($i, $fct, $type, $ulp, $ulp_real, $ulp_imag);
-  my (%test_ulps, %func_ulps, %func_real_ulps, %func_imag_ulps);
+  my (%func_ulps, %func_real_ulps, %func_imag_ulps);
 
   open ULP, ">$file" or die ("Can't open $file: $!");
 
@@ -674,22 +625,11 @@ sub output_ulps {
     } else {
       die "unknown results ($fct) type $type\n";
     }
-    if ($results{$fct}{'kind'} eq 'fct') {
-      if ($type eq 'normal') {
-	$func_ulps{$fct} = $ulp;
-      } else {
-	$func_real_ulps{$fct} = $ulp_real;
-	$func_imag_ulps{$fct} = $ulp_imag;
-      }
-    } elsif ($results{$fct}{'kind'} eq 'test') {
-      if ($type eq 'normal') {
-	$test_ulps{$fct} = $ulp;
-      } else {
-	$test_ulps{"Real part of: $fct"} = $ulp_real;
-	$test_ulps{"Imaginary part of: $fct"} = $ulp_imag;
-      }
+    if ($type eq 'normal') {
+      $func_ulps{$fct} = $ulp;
     } else {
-      die "unknown results ($fct) kind $results{$fct}{'kind'}\n";
+      $func_real_ulps{$fct} = $ulp_real;
+      $func_imag_ulps{$fct} = $ulp_imag;
     }
   }
   print ULP "\n/* Maximal error of functions.  */\n";
@@ -708,13 +648,6 @@ sub output_ulps {
     print ULP "    { \"$fct\", $func_imag_ulps{$fct} },\n";
   }
   print ULP "  };\n";
-
-  print ULP "\n/* Error of single function calls.  */\n";
-  print ULP "static const struct ulp_data test_ulps[] =\n  {\n";
-  foreach $fct (sort keys %test_ulps) {
-    print ULP "    { \"$fct\", $test_ulps{$fct} },\n";
-  }
-  print ULP "  };\n";
   close ULP;
 }
 
diff --git a/math/libm-test.inc b/math/libm-test.inc
index 4cb239f..f8cd947 100644
--- a/math/libm-test.inc
+++ b/math/libm-test.inc
@@ -127,16 +127,16 @@
 #include <tininess.h>
 #include <math-tests.h>
 
-/* Structure for ulp data for a test, a function, or the real or
-   imaginary part of a function.  */
+/* Structure for ulp data for a function, or the real or imaginary
+   part of a function.  */
 struct ulp_data
 {
   const char *name;
   FLOAT max_ulp;
 };
 
-/* This header defines test_ulps, func_ulps, func_real_ulps and
-   func_imag_ulps arrays.  */
+/* This header defines func_ulps, func_real_ulps and func_imag_ulps
+   arrays.  */
 #include "libm-test-ulps.h"
 
 /* Allow platforms without all rounding modes to test properly,
@@ -324,6 +324,7 @@ static int ignore_max_ulp;	/* Should we ignore max_ulp?  */
 
 static FLOAT max_error, real_max_error, imag_max_error;
 
+static FLOAT prev_max_error, prev_real_max_error, prev_imag_max_error;
 
 #define BUILD_COMPLEX(real, imag) \
   ({ __complex__ FLOAT __retval;					      \
@@ -336,9 +337,9 @@ static FLOAT max_error, real_max_error, imag_max_error;
 #define MIN_EXP CHOOSE ((LDBL_MIN_EXP-1), (DBL_MIN_EXP-1), (FLT_MIN_EXP-1),	\
 			(LDBL_MIN_EXP-1), (DBL_MIN_EXP-1), (FLT_MIN_EXP-1))
 
-/* Compare KEY (a string, with the name of a test or a function) with
-   ULP (a pointer to a struct ulp_data structure), returning a value
-   less than, equal to or greater than zero for use in bsearch.  */
+/* Compare KEY (a string, with the name of a function) with ULP (a
+   pointer to a struct ulp_data structure), returning a value less
+   than, equal to or greater than zero for use in bsearch.  */
 
 static int
 compare_ulp_data (const void *key, const void *ulp)
@@ -362,44 +363,20 @@ find_ulps (const char *name, const struct ulp_data *data, size_t nmemb)
     return entry->max_ulp;
 }
 
-/* Return the ulps for test NAME.  */
-
-static FLOAT
-find_test_ulps (const char *name)
-{
-  return find_ulps (name, test_ulps,
-		    sizeof (test_ulps) / sizeof (test_ulps[0]));
-}
-
-/* Return the ulps for real function NAME.  */
-
-static FLOAT
-find_function_ulps (const char *name)
-{
-  return find_ulps (name, func_ulps,
-		    sizeof (func_ulps) / sizeof (func_ulps[0]));
-}
-
-/* Return the ulps for complex function NAME.  */
-
-static __complex__ FLOAT
-find_complex_function_ulps (const char *name)
-{
-  FLOAT ulp_real = find_ulps (name, func_real_ulps,
-			      (sizeof (func_real_ulps)
-			       / sizeof (func_real_ulps[0])));
-  FLOAT ulp_imag = find_ulps (name, func_imag_ulps,
-			      (sizeof (func_imag_ulps)
-			       / sizeof (func_imag_ulps[0])));
-  return BUILD_COMPLEX (ulp_real, ulp_imag);
-}
-
 static void
-init_max_error (void)
+init_max_error (const char *name)
 {
   max_error = 0;
   real_max_error = 0;
   imag_max_error = 0;
+  prev_max_error = find_ulps (name, func_ulps,
+			      sizeof (func_ulps) / sizeof (func_ulps[0]));
+  prev_real_max_error = find_ulps (name, func_real_ulps,
+				   (sizeof (func_real_ulps)
+				    / sizeof (func_real_ulps[0])));
+  prev_imag_max_error = find_ulps (name, func_imag_ulps,
+				   (sizeof (func_imag_ulps)
+				    / sizeof (func_imag_ulps[0])));
   feclearexcept (FE_ALL_EXCEPT);
   errno = 0;
 }
@@ -460,19 +437,6 @@ update_stats (int ok)
 }
 
 static void
-print_ulps (const char *test_name, FLOAT ulp)
-{
-  if (output_ulps)
-    {
-      fprintf (ulps_file, "Test \"%s\":\n", test_name);
-      fprintf (ulps_file, "%s: %.0" PRINTF_NEXPR "\n",
-	       CHOOSE("ldouble", "double", "float",
-		      "ildouble", "idouble", "ifloat"),
-	       FUNC(ceil) (ulp));
-    }
-}
-
-static void
 print_function_ulps (const char *function_name, FLOAT ulp)
 {
   if (output_ulps)
@@ -541,10 +505,9 @@ fpstack_test (const char *test_name)
 static void
 print_max_error (const char *func_name)
 {
-  FLOAT allowed = find_function_ulps (func_name);
   int ok = 0;
 
-  if (max_error == 0.0 || (max_error <= allowed && !ignore_max_ulp))
+  if (max_error == 0.0 || (max_error <= prev_max_error && !ignore_max_ulp))
     {
       ok = 1;
     }
@@ -557,7 +520,8 @@ print_max_error (const char *func_name)
     {
       printf ("Maximal error of `%s'\n", func_name);
       printf (" is      : %.0" PRINTF_NEXPR " ulp\n", FUNC(ceil) (max_error));
-      printf (" accepted: %.0" PRINTF_NEXPR " ulp\n", FUNC(ceil) (allowed));
+      printf (" accepted: %.0" PRINTF_NEXPR " ulp\n",
+	      FUNC(ceil) (prev_max_error));
     }
 
   update_stats (ok);
@@ -567,17 +531,16 @@ print_max_error (const char *func_name)
 static void
 print_complex_max_error (const char *func_name)
 {
-  __complex__ FLOAT allowed = find_complex_function_ulps (func_name);
   int real_ok = 0, imag_ok = 0, ok;
 
   if (real_max_error == 0
-      || (real_max_error <= __real__ allowed && !ignore_max_ulp))
+      || (real_max_error <= prev_real_max_error && !ignore_max_ulp))
     {
       real_ok = 1;
     }
 
   if (imag_max_error == 0
-      || (imag_max_error <= __imag__ allowed && !ignore_max_ulp))
+      || (imag_max_error <= prev_imag_max_error && !ignore_max_ulp))
     {
       imag_ok = 1;
     }
@@ -595,12 +558,12 @@ print_complex_max_error (const char *func_name)
       printf (" is      : %.0" PRINTF_NEXPR " ulp\n",
 	      FUNC(ceil) (real_max_error));
       printf (" accepted: %.0" PRINTF_NEXPR " ulp\n",
-	      FUNC(ceil) (__real__ allowed));
+	      FUNC(ceil) (prev_real_max_error));
       printf ("Maximal error of imaginary part of: %s\n", func_name);
       printf (" is      : %.0" PRINTF_NEXPR " ulp\n",
 	      FUNC(ceil) (imag_max_error));
       printf (" accepted: %.0" PRINTF_NEXPR " ulp\n",
-	      FUNC(ceil) (__imag__ allowed));
+	      FUNC(ceil) (prev_imag_max_error));
     }
 
   update_stats (ok);
@@ -773,7 +736,7 @@ ulp (FLOAT value)
 static void
 check_float_internal (const char *test_name, FLOAT computed, FLOAT expected,
 		      int exceptions,
-		      FLOAT *curr_max_error)
+		      FLOAT *curr_max_error, FLOAT max_ulp)
 {
   int ok = 0;
   int print_diff = 0;
@@ -785,7 +748,6 @@ check_float_internal (const char *test_name, FLOAT computed, FLOAT expected,
   test_errno (test_name, errno_value, exceptions);
   if (exceptions & IGNORE_RESULT)
     goto out;
-  FLOAT max_ulp = find_test_ulps (test_name);
   if (issignaling (computed) && issignaling (expected))
     {
       if ((exceptions & TEST_NAN_SIGN) != 0
@@ -839,11 +801,7 @@ check_float_internal (const char *test_name, FLOAT computed, FLOAT expected,
       else if (ulps <= 0.5 || (ulps <= max_ulp && !ignore_max_ulp))
 	ok = 1;
       else
-	{
-	  ok = 0;
-	  print_ulps (test_name, ulps);
-	}
-
+	ok = 0;
     }
   if (print_screen (ok))
     {
@@ -876,7 +834,7 @@ check_float (const char *test_name, FLOAT computed, FLOAT expected,
 	     int exceptions)
 {
   check_float_internal (test_name, computed, expected,
-			exceptions, &max_error);
+			exceptions, &max_error, prev_max_error);
 }
 
 
@@ -895,7 +853,7 @@ check_complex (const char *test_name, __complex__ FLOAT computed,
   part_exp = __real__ expected;
 
   check_float_internal (str, part_comp, part_exp,
-			exception, &real_max_error);
+			exception, &real_max_error, prev_real_max_error);
   free (str);
 
   if (asprintf (&str, "Imaginary part of: %s", test_name) == -1)
@@ -910,7 +868,7 @@ check_complex (const char *test_name, __complex__ FLOAT computed,
 			exception & (IGNORE_ZERO_INF_SIGN
 				     | TEST_NAN_SIGN
 				     | IGNORE_RESULT),
-			&imag_max_error);
+			&imag_max_error, prev_imag_max_error);
   free (str);
 }
 
@@ -1629,7 +1587,7 @@ struct test_fFF_11_data
 /* Start and end the tests for a given function.  */
 #define START(FUNC)				\
   const char *this_func = #FUNC;		\
-  init_max_error ()
+  init_max_error (this_func)
 #define END					\
   print_max_error (this_func)
 #define END_COMPLEX				\

-- 
Joseph S. Myers
joseph@codesourcery.com