Re: RFC: Known Failures [Was: RFD: Testsuite cases for inferior function calls]

[Fernando Nasser <fnasser at cygnus dot com>]

"Peter.Schauer" wrote:
> 
> > > No, I meant the standard output of a testsuite run. I can quickly see any
> > > FAILs there without grepping etc., and I think it only fair to expose them
> > > to any installer of GDB who bothers to run the testsuite.
> >
> > That may be a question of taste or opinion.  I definitively don't want to see
> > anything more than FAILs on the stdout.  Maybe we can vote on this, or make it
> > an option switch.
> 
> So why do we see XPASSes then ?
> 

Because I was not here to protest when they were included :-)  Just kidding, they
should actually be there.

Seriously, I believe the reason is that nothing should normally XPASS.  If it is
XPASSing is because it is marked as XFAIL and maybe the reason for which it was
marked so has been eliminated.  So it requires immediate attention (someone must
investigate if the xfail should now be removed).

That is the distinction: immediate attention goes into the stdout while the
steady state (even if it contains some known bugs) does not.  It should, however,
be documented in our bug database (which will hopefully be available soon).


> > > There used to be a time when testcases were required for every submitted
> > > change.
> > > Gathering from recent postings it seems like you might not be allowed to add
> > > new testcases if they fail on any conceivable platform :-).
> >
> > That is not what I said.  We are talking about tests that a known to generally
> > fail and to pass only on a few targets (3, I believe) that have been fixed.
> >
> > I did not even said about not checking it in, but to xfail them to the targets
> > that have not yet been fixed (and for which we have no precise plan about when
> > we will be able to do it).
> 
> There seems to be a thread mixup here, sorry for the confusion.
> 
> I was not referring to my callfuncs proposal (otherwise I would have stated
> it explicitly), but to some other discussions in the past.
> 

Sorry, I misunderstood you.


> As to the callfuncs tests, they are not known to generally fail, they
> are just not yet tested on other targets (Orjan's target did fail, but that
> should be fixed now, thanks to the testcase).
> I am just very reluctant to xfail them for every non-tested target, as you had
> proposed:
> 

OK, we can try them.  But if they add FAILures everywhere and it is not possible for
us to fix those in a reasonable amount of time we xfail them, agreed?



-- 
Fernando Nasser
Red Hat - Toronto                       E-Mail:  fnasser@cygnus.com
2323 Yonge Street, Suite #300           Tel:  416-482-2661 ext. 311
Toronto, Ontario   M4P 2C9              Fax:  416-482-6299