Re: [PATCH][AArch64] update libm-test-ulps

[Szabolcs Nagy <szabolcs dot nagy at arm dot com>]

On 08/04/15 17:41, Joseph Myers wrote:
> On Wed, 8 Apr 2015, Szabolcs Nagy wrote:
>> is more than 9 ulp error considered a bug?
>> (ie. shall i send bug reports about it?)
> 
> Yes, such bugs should be reported to Bugzilla, if they don't duplicate 
> existing bugs (we have existing bug reports for Bessel functions, cpow, 
> lgamma for negative arguments and atan/atan2 in non-default rounding 
> modes, for example, and if the essential implementation issue is the same 
> it's not really useful to have extra bug reports just because there are 
> multiple implementations with the same bug).  Similarly, non-duplicate 
> bugs about any inaccuracy in functions with fully-determined results are 
> useful.
> 

i don't see the bessel function bug report on bugzilla (but y0 is hard
to search for) i assume the large ulp errors near zeros are known then.

i don't see lgammal bug report either, my test case is lgammal(-2.5)
which gives -10.465 ulp error on x86_64.

i don't have ldbl-128 tests yet, i plan to do that, but it is significant
amount of work (mpfr could be more helpful by providing functions to
easily get IEEE semantics).

i ran libc-test against glibc now (we use this for musl libc)
and found the listed issues
http://repo.or.cz/w/libc-test.git

api:
BZ #16747 aio.h missing required restrict in lio_listio
BZ #18228 fcntl.h missing O_ constants
BZ #18229 limits.h missing _XOPEN constants
BZ #18230 netinet/in.h IN6_IS_ADDR_* macros are broken
BZ #18231 sys/ipc.h wrong ipc_perm mode member type
BZ #18232 sys/sem.h wrong semid_ds sem_nsems member type
BZ #18233 sys/socket.h wrong msghdr and cmsghdr types
BZ #18234 sys/stat.h missing st_atim, etc in linux generic stat
BZ #18235 utmpx.h utmpx struct ut_tv member type
BZ #18236 wchar.h missing va_list type

functions:
BZ #18237 dn_expand(empty, emptyend, output, 1) fails
BZ #18238 QoI fmin/fmax on signed zeros
BZ #18239 fnmatch "[[:a]" does not fail
BZ #18240 hcreate((size_t)-1) should fail
BZ #16637 inet_pton accepts invalid ipv6 address
BZ #18241 fseek does not work on memstreams
BZ #18242 powf(0,-INFINITY) does not raise divbyzero
BZ #18243 cancellation point bugs: sem_wait, sem_timedwait, shm_open
BZ #18244 remquol(0, INFINITY, &q) returns nan
BZ #18245 sqrtl(-1.1L) raises the inexact flag
BZ #18246 sscanf("0x12", "%2d", &x) should fail
BZ #18247 strtof is not correctly rounded
BZ #18248 omitted underflow in math functions

some unclear cases others might be interested in:

ERE /(a)\1/ matches "aa" not "a1" in glibc, but ERE should not
support backrefs. (however \1 is ub in the spec so anything goes)

i think regcomp should fail on illegal sequence in a pattern
including /\xff/ in UTF-8 locale

%a printf format behaviour on glibc is ugly, but i don't see exact
spec for it (subnormals and . alignment are not what i expect)

there are some corner cases for fnmatch where i'm not yet sure
what's the right behaviour: "[\\]" and "\\" patterns

shm_open does not work with static linking because __shm_directory
depends on dynamic linking (fails at link time), sem_open likewise
(except it segfaults at runtime)

rounding functions (ceil,trunc,floor,round) don't have specified
inexact behaviour, but i think the next C floating-point standard
will be more strict, on x86_64 i see inconsistent raise of inexact.

BZ #13154 was closed WONTFIX but i think strtol("0",&endptr,99)
should set endptr for quality of implementation: the failure of
strtol is usually checked by comparing endptr to the input, which
is UB if it's not set.

> I encourage people to look for bugs with mpcheck on non-x86_64 
> architectures (as far as I know the tests described at 
> <http://www.loria.fr/~zimmerma/glibc/> are for x86_64 only), though I 
> don't think results for powerpc would be that useful (because too many 
> issues would arise with bugs in the underlying libgcc arithmetic for 
> ldbl-128ibm).  Especially, tests for ldbl-128 would be useful, and it's 
> quite likely there are issues in function implementations for 32-bit x86.  
> Testing with other tools for libm testing is also useful - mpcheck only 
> covers a limited subset of libm functions, and even within those it covers 
> I've found issues it missed such as bug 18196.

i think i have full coverage in libc-test, but the code needs to be
reorganised a bit to make it easy to compare mpfr results with libc
results on random inputs.