This is the mail archive of the
mailing list for the GDB project.
Re: Print KFAIL's in dejagnu summary?
- From: David Carlton <carlton at kealia dot com>
- To: Andrew Cagney <ac131313 at redhat dot com>
- Cc: dejagnu at gnu dot org, gdb at sources dot redhat dot com,Fernando Nasser <fnasser at redhat dot com>
- Date: Thu, 25 Sep 2003 15:13:53 -0700
- Subject: Re: Print KFAIL's in dejagnu summary?
- References: <3F7361BB.email@example.com>
On Thu, 25 Sep 2003 17:44:27 -0400, Andrew Cagney <firstname.lastname@example.org> said:
> At present KFAILs are supressed from the summary output (the stuff
> on the terminal from "make check"). I'd like to change this so that
> KFAILs, just like FAILs, are included in the summary. A KFAIL, just
> like a FAIL, indicates a bug in the system under test, and hence
> should be included in the summary.
I have a mild preference for the current behavior. Mostly I use the
summary output to get a feel for whether or not a change of mine has
obviously gone wrong; the noisier the summary output is, the harder it
is to use it this way. Of course, I always search the entire gdb.sum
for regressions, just to make sure, so it won't make a big practical
difference to me one way or another.
I also think that, right now, a bigger issue is to get the testsuite
and GNATS to agree about what the bugs are in GDB. So I'm more
interesting in removing the undiagnosed XFAILs and in making some
effort to diagnose existing FAILs than I am in making the KFAILs more
prominent. (I'd also like to see some effort towards making sure that
open bugs in GNATS are reflected via KFAILs in the test suite.) It
would also be nice if we had a feel for what bugs are low-hanging
The part of the test suite that I pay the most attention to is, of
course, gdb.cp; I haven't noticed that the addition of KFAILs there
has had much of an effect on the rate of bug fixing. (There are a lot
of old bugs that are still present, but they'd been present for a
while.) Of course, a lot of the KFAIL additions are from tests that
used to be XFAILs, which didn't change the visibility level of the
Some more random musings about the possible roles of the testsuite:
* Its main role is to catch regressions. Here, I think that making
FAILs more prominent is good: a FAIL is more likely to be a
regression than a KFAIL (at least if people delete the appropriate
setup_kfails from the test suite when they fix bugs).
* It's also useful for people trying to fix bugs: if you've decided to
fix bug X and if the testsuite contains a KFAIL associated to X,
then that makes it a lot easier to check if you've fixed the bug
(and to try to figure out what's causing the bug, for that matter).
Here, too, there's no reason for KFAILs to be prominent.
* Another role is to remind people of the existence of bugs, to exhort
people to fix them; that's important to you, but a little less
important to me than the other two roles.