Re: to XFAIL or not to XFAIL

[Ian Lance Taylor <ian at zembu dot com>]

   From: cgd@sibyte.com (Chris G. Demetriou)
   Date: 08 Oct 2000 22:24:02 -0700

   Should XFAIL be used to indicate testcases which:

   (a) are known to fail but actually indicate bugs, or

   (b) are known to fail but do not actually indicate bugs.

The main use of the testsuite is to see whether the binutils are
behaving as expected.  That means that `make check' should normally
succeed, which means it should not generate any FAIL messages.
Otherwise, you have to remember the expected number of FAIL messages,
which is error-prone and tedious.

However, we should not lose track of whether some binutils behaviour,
though expected, actually indicates a bug.  Hence, I believe XFAIL
should be used for your case (a) (and your patch should not be
applied).

If anything, I think a test which does not indicate a bug, your case
(b), should result in a PASS.  Perhaps DejaGNU should provide some
easy way to say ``reverse the sense of this test for this target.''

Ian