Re: PLEASE FIX NOW! make check-abi exits zero on failure

[Brooks Moses <bmoses at google dot com>]

On 03/17/2014 04:13 PM, Carlos O'Donell wrote:
> On 03/17/2014 06:50 PM, Roland McGrath wrote:
> > Whatever changes whoever made need to be fixed or reverted right away.
> > 'make check-abi' exits with status zero when there are failures.
> > This is unacceptable.
> >
> > If the people who broke this don't fix this it quickly, I'll start randomly
> > reverting changes that look related.
>
> Not for me?
>
> [carlos@koi glibc-wundef]$ make -k check
> ...

What happens if you run "make check-abi"?  Note that Roland isn't running 
"make check", so in particular he's not going to be running the code that 
produces the summary result and does "if grep "FAIL"; then exit 1" on it.

My guess is that this zero exit happens for any make target that runs a 
specific test.  The discussion on the "Do not terminate default test runs on 
test failure" thread [1] specifically referred to "make check", and did not 
discuss behavior for other make targets.  It is unfortunate that this 
additional requirement was not realized in the patch review.

As a potential solution strategy, I would suggest using a target-specific 
variable value to only set the "stop on test failure" flag to false when we 
are executing the literal "make check" target.

- Brooks


[1] E.g., https://sourceware.org/ml/libc-alpha/2014-03/msg00251.html