Re: exit status of 'make check'

[George France <france at handhelds dot org>]

On Sunday 25 November 2001 13:54, Daniel Jacobowitz wrote:
> On Sun, Nov 25, 2001 at 01:40:09PM -0500, George France wrote:
> > We are not discussing, the results of 'runtest'. We are discussing the
> > results of 'make check'.  Here is another simple example, On 2001-11-22,
> > I built the gcc_3.0.2 HEAD branch.  'make check' returned a zero for an
> > exit code, but many of the testsuites FAILed.
>
> GCC does this by completely ignoring the exit code of runtest; in other
> words, make check will always succeed.  I don't think that's an
> acceptable step.  Do you?
>

I looked at the Makefiles (gdb & gcc).  You are correct.  It is equally 
unacceptable to always succeed.  <sigh>

> We are discussing the result of runtest, because the DejaGNU
> maintainers are the appropriate people to set policy for this sort of
> thing.

I am disappointed that there is not a more consistent GNU policy in regards 
to the meaning of exit code for 'make check'.  For the auto-build system that 
I use, it makes much more sense to be able to separate errors from 'make 
check' into two categories:

A) errors in the check system - Example, 'runtest' not being available.
B) testsuite results either PASS / FAIL - Which are the result of a 
successful run of the check system.

I can see that this is not possible under the current policy.  I will have to 
research this matter.  Any further suggestions would be greatly appreciated.

Thank you for your help.

Best Regards,


--George