This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: About FAILs in the internal testsuite
- From: David Smith <dsmith at redhat dot com>
- To: peter wang <shangwaters at gmail dot com>
- Cc: systemtap at sourceware dot org
- Date: Mon, 18 Apr 2016 13:44:47 -0500
- Subject: Re: About FAILs in the internal testsuite
- Authentication-results: sourceware.org; auth=none
- References: <CAP+nm+wEi=ebN5qaL0WFbb_KCK8nradFPduxstKYyrRPQ346Rw at mail dot gmail dot com> <570FDE49 dot 30400 at redhat dot com> <CAP+nm+xHFQRsYH0gjFgj7OmCmJ6ccEy5+DqK=FFGrQGvGee3Zw at mail dot gmail dot com> <5710F7F0 dot 3050208 at redhat dot com> <CAP+nm+yj5UFC7Q41mkjL=x8FaXV6af4oBObc-74oY5R21t5POw at mail dot gmail dot com>
On 04/15/2016 10:52 PM, peter wang wrote:
> On Fri, Apr 15, 2016 at 10:17 PM, David Smith <dsmith@redhat.com> wrote:
>> On 04/15/2016 06:03 AM, peter wang wrote:
>>> Hello David,
>>>
>>> Thanks again for your reply.
>>> There is one more thing which confused me.
>>>
>>> I'm not sure if I have some misunderstandings.
>>> Some failures are resulted by either case's bug or systemtap's bug and
>>> others should be set to KFAIL.
>>>
>>> In other words, all FAIL should be fixed whatever arch or system is.
>>> Is that right?
>>
>> All FAILs should certainly be investigated to see out what is going on.
>> There have been times when I've done that and left the FAIL in place.
>> Here's why. In the past, I've found a few KFAIL tests that were no
>> longer failing because of the orignal reason (most likely bad
>> debuginfo), but failing because the underlying probe point has changed.
>>
>
> Does that mean most of the FAILs have been investigated actually?
> Since investigated FAILs are still left, it is hard for others to find
> the real bug, I think.
> Is there any docs recording or describing the investigated FAILs?
The remaining FAILs could *certainly* use more investigation. Earlier,
when I was talking about leaving KFAILs I was really talking about
testsuite/systemtap.pass1-4/buildok.exp. That why you'll see the
following comment in that test:
# (Note that if a test fails because of debuginfo-quality
# problems, don't kfail a tapset's entire test. Extract
# the failing probe/function's test into a new file and
# kfail that.)
I'll certainly be happy for help in figuring out the remaining FAILs.
--
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)