This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: conformtest: split up running of tests from makefile
- From: "Carlos O'Donell" <carlos at redhat dot com>
- To: "Joseph S. Myers" <joseph at codesourcery dot com>, libc-alpha at sourceware dot org
- Date: Thu, 06 Mar 2014 10:55:44 -0500
- Subject: Re: conformtest: split up running of tests from makefile
- Authentication-results: sourceware.org; auth=none
- References: <Pine dot LNX dot 4 dot 64 dot 1403052324170 dot 28007 at digraph dot polyomino dot org dot uk>
On 03/05/2014 06:26 PM, Joseph S. Myers wrote:
> Tested x86_64 (GCC 4.7 branch). It's quite possible some failures
> vary from platform to platform, in which case either additional XFAILs
> can be added here, or platform-specific header bugs (if applicable)
> the failures show up can be fixed. I made XFAILs unconditional for
> tests that should only fail for GCC 4.6 and earlier; we could make the
> GCC version available in the makefiles and condition these, but simply
> moving the XFAILing to a finer granularity seems a clear improvement
> on the previous state of the whole of conformtest being XFAILed.
>
> 2014-03-05 Joseph Myers <joseph@codesourcery.com>
>
> * Makeconfig (test-xfail-name): New variable.
> (evaluate-test): Use $(test-xfail-name) instead of $(@F:.out=) to
> compute variable name for expected failures.
> * conform/Makefile (conformtest-headers-data): New variable.
> (conformtest-standards): Likewise.
> (conformtest-headers-ISO): Likewise.
> (conformtest-headers-ISO99): Likewise.
> (conformtest-headers-ISO11): Likewise.
> (conformtest-headers-POSIX): Likewise.
> (conformtest-headers-XPG3): Likewise.
> (conformtest-headers-XPG4): Likewise.
> (conformtest-headers-UNIX98): Likewise.
> (conformtest-headers-XOPEN2K): Likewise.
> (conformtest-headers-POSIX2008): Likewise.
> (conformtest-headers-XOPEN2K8): Likewise.
> (conformtest-header-list-base): Likewise.
> (conformtest-header-list-tests): Likewise.
> (conformtest-header-base): Likewise.
> (conformtest-header-tests): Likewise.
> (tests-special): Add $(conformtest-header-list-tests). If
> [$(fast-check) && !$(cross-compiling)], add
> $(conformtest-header-tests) instead of
> $(objpfx)run-conformtest.out.
> (generated): Add $(conformtest-header-list-base). If
> [$(fast-check) && !$(cross-compiling)], add
> $(conformtest-header-base). Remove previous setting.
> ($(conformtest-header-list-tests)): New target.
> (test-xfail-run-conformtest): Remove variable.
> ($(objpfx)run-conformtest.out): Remove target.
> (test-xfail-ISO11/complex.h/conform): New variable.
> (test-xfail-ISO11/stdalign.h/conform): Likewise.
> (test-xfail-ISO11/stdnoreturn.h/conform): Likewise.
> (test-xfail-XPG3/varargs.h/conform): Likewise.
> (test-xfail-XPG4/varargs.h/conform): Likewise.
> (test-xfail-UNIX98/varargs.h/conform): Likewise.
> (test-xfail-XPG4/ndbm.h/conform): Likewise.
> (test-xfail-UNIX98/ndbm.h/conform): Likewise.
> (test-xfail-XOPEN2K/ndbm.h/conform): Likewise.
> (test-xfail-XOPEN2K8/ndbm.h/conform): Likewise.
> (test-xfail-XPG3/fcntl.h/conform): Likewise.
> (test-xfail-XPG3/ftw.h/conform): Likewise.
> (test-xfail-XPG3/grp.h/conform): Likewise.
> (test-xfail-XPG3/langinfo.h/conform): Likewise.
> (test-xfail-XPG3/limits.h/conform): Likewise.
> (test-xfail-XPG3/pwd.h/conform): Likewise.
> (test-xfail-XPG3/search.h/conform): Likewise.
> (test-xfail-XPG3/signal.h/conform): Likewise.
> (test-xfail-XPG3/stdio.h/conform): Likewise.
> (test-xfail-XPG3/stdlib.h/conform): Likewise.
> (test-xfail-XPG3/string.h/conform): Likewise.
> (test-xfail-XPG3/sys/ipc.h/conform): Likewise.
> (test-xfail-XPG3/sys/msg.h/conform): Likewise.
> (test-xfail-XPG3/sys/sem.h/conform): Likewise.
> (test-xfail-XPG3/sys/shm.h/conform): Likewise.
> (test-xfail-XPG3/sys/stat.h/conform): Likewise.
> (test-xfail-XPG3/sys/types.h/conform): Likewise.
> (test-xfail-XPG3/sys/wait.h/conform): Likewise.
> (test-xfail-XPG3/termios.h/conform): Likewise.
> (test-xfail-XPG3/time.h/conform): Likewise.
> (test-xfail-XPG3/unistd.h/conform): Likewise.
> (test-xfail-XPG4/arpa/inet.h/conform): Likewise.
> (test-xfail-XPG4/fcntl.h/conform): Likewise.
> (test-xfail-XPG4/langinfo.h/conform): Likewise.
> (test-xfail-XPG4/netdb.h/conform): Likewise.
> (test-xfail-XPG4/netinet/in.h/conform): Likewise.
> (test-xfail-XPG4/signal.h/conform): Likewise.
> (test-xfail-XPG4/stdio.h/conform): Likewise.
> (test-xfail-XPG4/stdlib.h/conform): Likewise.
> (test-xfail-XPG4/stropts.h/conform): Likewise.
> (test-xfail-XPG4/sys/ipc.h/conform): Likewise.
> (test-xfail-XPG4/sys/msg.h/conform): Likewise.
> (test-xfail-XPG4/sys/sem.h/conform): Likewise.
> (test-xfail-XPG4/sys/shm.h/conform): Likewise.
> (test-xfail-XPG4/sys/socket.h/conform): Likewise.
> (test-xfail-XPG4/sys/stat.h/conform): Likewise.
> (test-xfail-XPG4/sys/time.h/conform): Likewise.
> (test-xfail-XPG4/sys/types.h/conform): Likewise.
> (test-xfail-XPG4/sys/wait.h/conform): Likewise.
> (test-xfail-XPG4/termios.h/conform): Likewise.
> (test-xfail-XPG4/ucontext.h/conform): Likewise.
> (test-xfail-XPG4/unistd.h/conform): Likewise.
> (test-xfail-XPG4/utmpx.h/conform): Likewise.
> (test-xfail-POSIX/sched.h/conform): Likewise.
> (test-xfail-POSIX/signal.h/conform): Likewise.
> (test-xfail-POSIX/sys/wait.h/conform): Likewise.
> (test-xfail-POSIX/tar.h/conform): Likewise.
> (test-xfail-UNIX98/arpa/inet.h/conform): Likewise.
> (test-xfail-UNIX98/fcntl.h/conform): Likewise.
> (test-xfail-UNIX98/langinfo.h/conform): Likewise.
> (test-xfail-UNIX98/netdb.h/conform): Likewise.
> (test-xfail-UNIX98/netinet/in.h/conform): Likewise.
> (test-xfail-UNIX98/sched.h/conform): Likewise.
> (test-xfail-UNIX98/signal.h/conform): Likewise.
> (test-xfail-UNIX98/stdio.h/conform): Likewise.
> (test-xfail-UNIX98/stdlib.h/conform): Likewise.
> (test-xfail-UNIX98/stropts.h/conform): Likewise.
> (test-xfail-UNIX98/sys/ipc.h/conform): Likewise.
> (test-xfail-UNIX98/sys/msg.h/conform): Likewise.
> (test-xfail-UNIX98/sys/sem.h/conform): Likewise.
> (test-xfail-UNIX98/sys/shm.h/conform): Likewise.
> (test-xfail-UNIX98/sys/socket.h/conform): Likewise.
> (test-xfail-UNIX98/sys/time.h/conform): Likewise.
> (test-xfail-UNIX98/sys/wait.h/conform): Likewise.
> (test-xfail-UNIX98/ucontext.h/conform): Likewise.
> (test-xfail-UNIX98/unistd.h/conform): Likewise.
> (test-xfail-UNIX98/utmpx.h/conform): Likewise.
> (test-xfail-XOPEN2K/aio.h/conform): Likewise.
> (test-xfail-XOPEN2K/arpa/inet.h/conform): Likewise.
> (test-xfail-XOPEN2K/fcntl.h/conform): Likewise.
> (test-xfail-XOPEN2K/langinfo.h/conform): Likewise.
> (test-xfail-XOPEN2K/math.h/conform): Likewise.
> (test-xfail-XOPEN2K/mqueue.h/conform): Likewise.
> (test-xfail-XOPEN2K/netdb.h/conform): Likewise.
> (test-xfail-XOPEN2K/netinet/in.h/conform): Likewise.
> (test-xfail-XOPEN2K/semaphore.h/conform): Likewise.
> (test-xfail-XOPEN2K/signal.h/conform): Likewise.
> (test-xfail-XOPEN2K/stdarg.h/conform): Likewise.
> (test-xfail-XOPEN2K/stdio.h/conform): Likewise.
> (test-xfail-XOPEN2K/stropts.h/conform): Likewise.
> (test-xfail-XOPEN2K/sys/ipc.h/conform): Likewise.
> (test-xfail-XOPEN2K/sys/msg.h/conform): Likewise.
> (test-xfail-XOPEN2K/sys/sem.h/conform): Likewise.
> (test-xfail-XOPEN2K/sys/shm.h/conform): Likewise.
> (test-xfail-XOPEN2K/sys/socket.h/conform): Likewise.
> (test-xfail-XOPEN2K/sys/wait.h/conform): Likewise.
> (test-xfail-XOPEN2K/termios.h/conform): Likewise.
> (test-xfail-XOPEN2K/tgmath.h/conform): Likewise.
> (test-xfail-XOPEN2K/ucontext.h/conform): Likewise.
> (test-xfail-XOPEN2K/utmpx.h/conform): Likewise.
> (test-xfail-POSIX2008/arpa/inet.h/conform): Likewise.
> (test-xfail-POSIX2008/fcntl.h/conform): Likewise.
> (test-xfail-POSIX2008/netdb.h/conform): Likewise.
> (test-xfail-POSIX2008/netinet/in.h/conform): Likewise.
> (test-xfail-POSIX2008/signal.h/conform): Likewise.
> (test-xfail-POSIX2008/stropts.h/conform): Likewise.
> (test-xfail-POSIX2008/sys/socket.h/conform): Likewise.
> (test-xfail-POSIX2008/sys/wait.h/conform): Likewise.
> (test-xfail-XOPEN2K8/arpa/inet.h/conform): Likewise.
> (test-xfail-XOPEN2K8/fcntl.h/conform): Likewise.
> (test-xfail-XOPEN2K8/limits.h/conform): Likewise.
> (test-xfail-XOPEN2K8/math.h/conform): Likewise.
> (test-xfail-XOPEN2K8/netdb.h/conform): Likewise.
> (test-xfail-XOPEN2K8/netinet/in.h/conform): Likewise.
> (test-xfail-XOPEN2K8/signal.h/conform): Likewise.
> (test-xfail-XOPEN2K8/stdio.h/conform): Likewise.
> (test-xfail-XOPEN2K8/stropts.h/conform): Likewise.
> (test-xfail-XOPEN2K8/sys/ipc.h/conform): Likewise.
> (test-xfail-XOPEN2K8/sys/msg.h/conform): Likewise.
> (test-xfail-XOPEN2K8/sys/select.h/conform): Likewise.
> (test-xfail-XOPEN2K8/sys/sem.h/conform): Likewise.
> (test-xfail-XOPEN2K8/sys/shm.h/conform): Likewise.
> (test-xfail-XOPEN2K8/sys/socket.h/conform): Likewise.
> (test-xfail-XOPEN2K8/sys/time.h/conform): Likewise.
> (test-xfail-XOPEN2K8/sys/wait.h/conform): Likewise.
> (test-xfail-XOPEN2K8/termios.h/conform): Likewise.
> (test-xfail-XOPEN2K8/tgmath.h/conform): Likewise.
> (test-xfail-XOPEN2K8/utmpx.h/conform): Likewise.
> (conformtest-cc-flags): Likewise.
> ($(conformtest-header-tests): New target.
> * conform/check-header-lists.sh: New file.
> * conform/run-conformtest.sh: Remove.
OK.
Cheers,
Carlos.