This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: conformtest: split up running of tests from makefile


On 03/05/2014 06:26 PM, Joseph S. Myers wrote:
> Tested x86_64 (GCC 4.7 branch).  It's quite possible some failures
> vary from platform to platform, in which case either additional XFAILs
> can be added here, or platform-specific header bugs (if applicable)
> the failures show up can be fixed.  I made XFAILs unconditional for
> tests that should only fail for GCC 4.6 and earlier; we could make the
> GCC version available in the makefiles and condition these, but simply
> moving the XFAILing to a finer granularity seems a clear improvement
> on the previous state of the whole of conformtest being XFAILed.
> 
> 2014-03-05  Joseph Myers  <joseph@codesourcery.com>
> 
> 	* Makeconfig (test-xfail-name): New variable.
> 	(evaluate-test): Use $(test-xfail-name) instead of $(@F:.out=) to
> 	compute variable name for expected failures.
> 	* conform/Makefile (conformtest-headers-data): New variable.
> 	(conformtest-standards): Likewise.
> 	(conformtest-headers-ISO): Likewise.
> 	(conformtest-headers-ISO99): Likewise.
> 	(conformtest-headers-ISO11): Likewise.
> 	(conformtest-headers-POSIX): Likewise.
> 	(conformtest-headers-XPG3): Likewise.
> 	(conformtest-headers-XPG4): Likewise.
> 	(conformtest-headers-UNIX98): Likewise.
> 	(conformtest-headers-XOPEN2K): Likewise.
> 	(conformtest-headers-POSIX2008): Likewise.
> 	(conformtest-headers-XOPEN2K8): Likewise.
> 	(conformtest-header-list-base): Likewise.
> 	(conformtest-header-list-tests): Likewise.
> 	(conformtest-header-base): Likewise.
> 	(conformtest-header-tests): Likewise.
> 	(tests-special): Add $(conformtest-header-list-tests).  If
> 	[$(fast-check) && !$(cross-compiling)], add
> 	$(conformtest-header-tests) instead of
> 	$(objpfx)run-conformtest.out.
> 	(generated): Add $(conformtest-header-list-base).  If
> 	[$(fast-check) && !$(cross-compiling)], add
> 	$(conformtest-header-base).  Remove previous setting.
> 	($(conformtest-header-list-tests)): New target.
> 	(test-xfail-run-conformtest): Remove variable.
> 	($(objpfx)run-conformtest.out): Remove target.
> 	(test-xfail-ISO11/complex.h/conform): New variable.
> 	(test-xfail-ISO11/stdalign.h/conform): Likewise.
> 	(test-xfail-ISO11/stdnoreturn.h/conform): Likewise.
> 	(test-xfail-XPG3/varargs.h/conform): Likewise.
> 	(test-xfail-XPG4/varargs.h/conform): Likewise.
> 	(test-xfail-UNIX98/varargs.h/conform): Likewise.
> 	(test-xfail-XPG4/ndbm.h/conform): Likewise.
> 	(test-xfail-UNIX98/ndbm.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/ndbm.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/ndbm.h/conform): Likewise.
> 	(test-xfail-XPG3/fcntl.h/conform): Likewise.
> 	(test-xfail-XPG3/ftw.h/conform): Likewise.
> 	(test-xfail-XPG3/grp.h/conform): Likewise.
> 	(test-xfail-XPG3/langinfo.h/conform): Likewise.
> 	(test-xfail-XPG3/limits.h/conform): Likewise.
> 	(test-xfail-XPG3/pwd.h/conform): Likewise.
> 	(test-xfail-XPG3/search.h/conform): Likewise.
> 	(test-xfail-XPG3/signal.h/conform): Likewise.
> 	(test-xfail-XPG3/stdio.h/conform): Likewise.
> 	(test-xfail-XPG3/stdlib.h/conform): Likewise.
> 	(test-xfail-XPG3/string.h/conform): Likewise.
> 	(test-xfail-XPG3/sys/ipc.h/conform): Likewise.
> 	(test-xfail-XPG3/sys/msg.h/conform): Likewise.
> 	(test-xfail-XPG3/sys/sem.h/conform): Likewise.
> 	(test-xfail-XPG3/sys/shm.h/conform): Likewise.
> 	(test-xfail-XPG3/sys/stat.h/conform): Likewise.
> 	(test-xfail-XPG3/sys/types.h/conform): Likewise.
> 	(test-xfail-XPG3/sys/wait.h/conform): Likewise.
> 	(test-xfail-XPG3/termios.h/conform): Likewise.
> 	(test-xfail-XPG3/time.h/conform): Likewise.
> 	(test-xfail-XPG3/unistd.h/conform): Likewise.
> 	(test-xfail-XPG4/arpa/inet.h/conform): Likewise.
> 	(test-xfail-XPG4/fcntl.h/conform): Likewise.
> 	(test-xfail-XPG4/langinfo.h/conform): Likewise.
> 	(test-xfail-XPG4/netdb.h/conform): Likewise.
> 	(test-xfail-XPG4/netinet/in.h/conform): Likewise.
> 	(test-xfail-XPG4/signal.h/conform): Likewise.
> 	(test-xfail-XPG4/stdio.h/conform): Likewise.
> 	(test-xfail-XPG4/stdlib.h/conform): Likewise.
> 	(test-xfail-XPG4/stropts.h/conform): Likewise.
> 	(test-xfail-XPG4/sys/ipc.h/conform): Likewise.
> 	(test-xfail-XPG4/sys/msg.h/conform): Likewise.
> 	(test-xfail-XPG4/sys/sem.h/conform): Likewise.
> 	(test-xfail-XPG4/sys/shm.h/conform): Likewise.
> 	(test-xfail-XPG4/sys/socket.h/conform): Likewise.
> 	(test-xfail-XPG4/sys/stat.h/conform): Likewise.
> 	(test-xfail-XPG4/sys/time.h/conform): Likewise.
> 	(test-xfail-XPG4/sys/types.h/conform): Likewise.
> 	(test-xfail-XPG4/sys/wait.h/conform): Likewise.
> 	(test-xfail-XPG4/termios.h/conform): Likewise.
> 	(test-xfail-XPG4/ucontext.h/conform): Likewise.
> 	(test-xfail-XPG4/unistd.h/conform): Likewise.
> 	(test-xfail-XPG4/utmpx.h/conform): Likewise.
> 	(test-xfail-POSIX/sched.h/conform): Likewise.
> 	(test-xfail-POSIX/signal.h/conform): Likewise.
> 	(test-xfail-POSIX/sys/wait.h/conform): Likewise.
> 	(test-xfail-POSIX/tar.h/conform): Likewise.
> 	(test-xfail-UNIX98/arpa/inet.h/conform): Likewise.
> 	(test-xfail-UNIX98/fcntl.h/conform): Likewise.
> 	(test-xfail-UNIX98/langinfo.h/conform): Likewise.
> 	(test-xfail-UNIX98/netdb.h/conform): Likewise.
> 	(test-xfail-UNIX98/netinet/in.h/conform): Likewise.
> 	(test-xfail-UNIX98/sched.h/conform): Likewise.
> 	(test-xfail-UNIX98/signal.h/conform): Likewise.
> 	(test-xfail-UNIX98/stdio.h/conform): Likewise.
> 	(test-xfail-UNIX98/stdlib.h/conform): Likewise.
> 	(test-xfail-UNIX98/stropts.h/conform): Likewise.
> 	(test-xfail-UNIX98/sys/ipc.h/conform): Likewise.
> 	(test-xfail-UNIX98/sys/msg.h/conform): Likewise.
> 	(test-xfail-UNIX98/sys/sem.h/conform): Likewise.
> 	(test-xfail-UNIX98/sys/shm.h/conform): Likewise.
> 	(test-xfail-UNIX98/sys/socket.h/conform): Likewise.
> 	(test-xfail-UNIX98/sys/time.h/conform): Likewise.
> 	(test-xfail-UNIX98/sys/wait.h/conform): Likewise.
> 	(test-xfail-UNIX98/ucontext.h/conform): Likewise.
> 	(test-xfail-UNIX98/unistd.h/conform): Likewise.
> 	(test-xfail-UNIX98/utmpx.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/aio.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/arpa/inet.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/fcntl.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/langinfo.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/math.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/mqueue.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/netdb.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/netinet/in.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/semaphore.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/signal.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/stdarg.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/stdio.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/stropts.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/sys/ipc.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/sys/msg.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/sys/sem.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/sys/shm.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/sys/socket.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/sys/wait.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/termios.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/tgmath.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/ucontext.h/conform): Likewise.
> 	(test-xfail-XOPEN2K/utmpx.h/conform): Likewise.
> 	(test-xfail-POSIX2008/arpa/inet.h/conform): Likewise.
> 	(test-xfail-POSIX2008/fcntl.h/conform): Likewise.
> 	(test-xfail-POSIX2008/netdb.h/conform): Likewise.
> 	(test-xfail-POSIX2008/netinet/in.h/conform): Likewise.
> 	(test-xfail-POSIX2008/signal.h/conform): Likewise.
> 	(test-xfail-POSIX2008/stropts.h/conform): Likewise.
> 	(test-xfail-POSIX2008/sys/socket.h/conform): Likewise.
> 	(test-xfail-POSIX2008/sys/wait.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/arpa/inet.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/fcntl.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/limits.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/math.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/netdb.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/netinet/in.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/signal.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/stdio.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/stropts.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/sys/ipc.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/sys/msg.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/sys/select.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/sys/sem.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/sys/shm.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/sys/socket.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/sys/time.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/sys/wait.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/termios.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/tgmath.h/conform): Likewise.
> 	(test-xfail-XOPEN2K8/utmpx.h/conform): Likewise.
> 	(conformtest-cc-flags): Likewise.
> 	($(conformtest-header-tests): New target.
> 	* conform/check-header-lists.sh: New file.
> 	* conform/run-conformtest.sh: Remove.

OK.

Cheers,
Carlos.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]