Re: [PATCH] support: Introduce new subdirectory for test infrastructure

[Florian Weimer <fweimer at redhat dot com>]

On 12/12/2016 02:11 PM, Stefan Liebler wrote:
> Hi Florian,
>
> I've observed a new test-failure string/tst-strcoll-overflow after this
> patch has been commited.
>
> The test uses:
> #define TIMEOUT 300
> #define EXPECTED_SIGNAL SIGALRM
> #define EXPECTED_STATUS 0
>
> With the old test-skeleton, it either timed out or strcoll returned in
> time and do_test returns 0. The test passes as WEXITSTATUS (status) ==
> EXPECTED_STATUS.
>
> With the new test-driver, the out-file contains:
> 0
> Expected signal 'Alarm clock' from child, got none

My bad, I did not run the xtests. :(

> If do_test would return e.g. 2 and EXPECTED_STATUS is defined to 2, the
> test would pass. But if the testcase is then called with --direct, it
> would return 2 instead of 0!

--direct is not compatible with EXPECTED_SIGNAL or EXPECTED_STATUS; the 
caller has to know if any of those expectations apply.  As far as I 
know, that has always been the case.

> Shall the new test-driver be able to handle this case?

I'm not sure what the intended behavior is.  The test says that 
collating such a huge string could take days.  But that's clearly not 
what's happening.  For me, it finishes within 15 seconds.

Can we assume that the test will run to completion with a 300-second 
timeout (plus TIMEOUTFACTOR scaling)?  Then I'll update the test 
accordingly.

Thanks,
Florian