Re: to XFAIL or not to XFAIL

[DJ Delorie <dj at delorie dot com>]

> However, we should not lose track of whether some binutils behaviour,
> though expected, actually indicates a bug.  Hence, I believe XFAIL
> should be used for your case (a) (and your patch should not be
> applied).

As much as I agree that keeping track of persistently failing cases is
annoying, I disagree.  A test should fail if it indicates a bug in
binutils, regardless of how annoying it is to see the failure every
single time you run the testsuite.

> If anything, I think a test which does not indicate a bug, your case
> (b), should result in a PASS.  Perhaps DejaGNU should provide some
> easy way to say ``reverse the sense of this test for this target.''

It's always been my understanding that XFAIL was to be used for
specific targets under which a test would fail through no fault of the
program being tested.  In other words, "we expect it to fail on this
target, but there's nothing we can do about it."  In other words,
there should be *no* cases which XFAIL for *all* hosts/targets.

Using XFAIL to make it easier to ignore persistent problems is evil.