Re: [RFA] testsuite/gdb.base/callfuncs.*: Add inferior call testcases

[Fernando Nasser <fnasser at cygnus dot com>]

"Peter.Schauer" wrote:
> 
> > OK, suppose we xfail it for all targets except the ones you've already verified?
> > We still get the other targets tested and it will either show as a XFAIL or a
> > XPASS (if we are that lucky).  No FAILs will be added and, with the *known problem*
> > comment added we can check it in as a start point for some improvements on this area.
> >
> > Sounds reasonable?
> 
> I beg to differ.
> XFAILs do not show up in the testsuite output and have a tendency to be
> `forgotten' (e.g. many of the XFAILs in virtfunc.exp are real regressions from
> gdb-4.17, making C++ debugging somewhat impossible on larger C++ projects, and
> have not been fixed for nearly two years).
> 
> The new testcases uncovered a serious bug on Linux x86 (calling an inferior
> function which used the FPU clobbered the FPU state of the current process),
> and a problem with the target Orjan Friberg is working on (which is now solved,
> due to the new testcases).
> 
> Any problems that show up on other targets might be serious, and should be
> clearly visible. If the problem can't be fixed immediately (or if the target
> maintainer is lazy), we still have the option to xfail it for specific
> targets.
>

The XFAILs are forgotten because there are two classes of things mixed up.
That is why I wanted to create the KFAILs.  KFAILs would be used to generate
a list of features not yet supported and known bugs.
It was decided, at that time, that we should use XFAILs and add a comment,
much to my dismay.

There is a current debate regarding if we keep the previous (current police) of only
checking test cases after the bug fixes (i.e., regression tests) or if we add cases
that test the specifications, even if the implementation is currently wrong or not yet
existent, thus raising the number of FAILs.  Have a non-zero reference makes it
difficult for developers to test for regressions.  On the other hand, adding tests
means adding knowledge, and we would like to add tests for anything that is not being
tested yet.  (Again, KFAILs would do the trick...)

We must wait for Andrew to come back to have this issue decided.  This is a strategic
decision that affects how we keep quality, do development etc.  So, IMO, it goes beyond
the testsuite maintainership.

P.S.: I also have tests that FAIL waiting in my sandbox.
 
> > P.S.: I believe there is still one maintainer that have not answered regarding to one
> > of the bug fixes in your patch.  You should wait to check at least these fixes n with
> > it.  Maybe ping the maintainer directly in case he missed the posting thinking t was
> > only a testsuite patch and not related to the specific target.
> 
> Even if you accept the callfuncs.* changes below, I will not check them in,
> until the bug fixes (I have sent in a proper RFA for them by now) are accepted
> or properly reworked.
> 

That was what I meant.  Thanks.


-- 
Fernando Nasser
Red Hat Canada Ltd.                     E-Mail:  fnasser@cygnus.com
2323 Yonge Street, Suite #300           Tel:  416-482-2661 ext. 311
Toronto, Ontario   M4P 2C9              Fax:  416-482-6299