This is the mail archive of the mailing list for the GDB project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [patch/rfc] Remove all setup_xfail's from testsuite/gdb.mi/

Andrew Cagney wrote:

What is the resolution of this thread?
(you'll probably want to read all of it :-)
(I'm now going back and emptying my mail box :-()


Hi Andrew,

I am just starting to think about this, so bear with me while I think aloud a little bit :-)  There are lots of good points made on that discussion and I don't think this is a clear cut yet or no, lets vote, or anything like that.  I am trying to see if I can collect all good suggestions and propose a course of action.

It seems that a widespread concern is not to through the baby with the water. On the other hand there is the feeling that we do not act now nothing will ever happen.  And we all agree that many xfails were added incorrectly.

Independently of the "big plan", one thing is clear.  If you know that a xfail is incorrect (I mean, know for good), then it should be ripped off. But read on...

Maybe we may not be in full agreement here, but I think it would be nice to enter a bug report even if does not include a full analysis -- that should come eventually.  Something like "something is wrong with the '--xxx-yyy' command output" may eventually cause someone to look into it.  And you can make the xfail a kfail in that case, which is much better.  But I know it is much more work than just removing the xfail and letting the test fail -- you would have to enter 28 bug reports just for the MI tests.  Still sounds like a better approach to me, but there is no consensus on it.
Some things I think worth noting.

In this process, I don't think it is reasonable make it a requirement that people fully kfail/analize old badly specified xfails. Rather I think it is only reasonable to ask people to ensure that new tests, or changes to existing tests, meet the new requirement (there must be a bug report).

An initial bar is established, and then, over time, that bar is raised.

The difference is important. It avoids the problem of developement stalling because of the impossible requirement fixing all the old code before the new code can be approved.

As a specific example, the i386 has an apparently low failure rate. That rate is badly misleading and the real number of failures is much higher :-( It's just that those failures have been [intentionally] camoflaged using xfail. It would be unfortunate if people, for the i386, tried to use that false result (almost zero fails) when initally setting the bar.

This is also why I think the xfail's should simply be yanked. It acts as a one time reset of gdb's test results, restoring them to their true values. While this may cause the bar to start out lower than some would like, I think that is far better and far more realistic than trying to start with a bar falsely set too high.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]