This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [PATCH 0/N] test-suite improvement - PASS/FAIL: initial patch
On Mon, 17 Sep 2012, Siddhesh Poyarekar wrote:
> If a file per test is the only way out, I would be more comfortable
> patching test-skeleton.c to write PASS/FAIL/etc. as the last line of the
> testcase.out file. I know that there are a lot of tests that don't use
> the skeleton and the right thing to do would be to fix those in the
> process.
I don't think it's ideal to rely on test-skeleton finishing cleanly in all
cases. Despite the forking, when there are kernel bugs affecting e.g.
threading testcases it can be necessary to kill tests from outside the
testsuite run.
Also, some tests are shell scripts and may not readily be convertible to
use test-skeleton.c.
Also, UNSUPPORTED tests may never build or run anything with
test-skeleton.c at all and so that status needs to be generated from the
makefiles in such cases.
Thus, while I think C tests should be converted to use test-skeleton.c, I
think we should define conventions for return values for particular
statuses, so that any test, no matter how it terminates, has its result
interpreted in some way (if killed by a signal from outside, that would be
FAIL, but it wouldn't rely on a clean termination that actually wrote FAIL
to some file). Of course we could have a test-skeleton.h file defining
names for the standard status values for tests.
(I do think every test that is run should create a .out file with the
stdout/stderr of the test - and that special cases not doing so should be
fixed to so so.)
--
Joseph S. Myers
joseph@codesourcery.com