This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: About FAILs in the internal testsuite
- From: peter wang <shangwaters at gmail dot com>
- To: David Smith <dsmith at redhat dot com>
- Cc: systemtap at sourceware dot org
- Date: Fri, 15 Apr 2016 19:03:29 +0800
- Subject: Re: About FAILs in the internal testsuite
- Authentication-results: sourceware.org; auth=none
- References: <CAP+nm+wEi=ebN5qaL0WFbb_KCK8nradFPduxstKYyrRPQ346Rw at mail dot gmail dot com> <570FDE49 dot 30400 at redhat dot com>
Hello David,
Thanks again for your reply.
There is one more thing which confused me.
I'm not sure if I have some misunderstandings.
Some failures are resulted by either case's bug or systemtap's bug and
others should be set to KFAIL.
In other words, all FAIL should be fixed whatever arch or system is.
Is that right?
--
Thanks
Wang
On Fri, Apr 15, 2016 at 2:15 AM, David Smith <dsmith@redhat.com> wrote:
> On 04/13/2016 07:17 PM, peter wang wrote:
>> Hello,
>>
>> I run the internal testsuite in CentOS 6.7 and got some FAILs in the result.
>> Then I got serveral questions:
>> 1. Is the testsuite designed for all the kernel version?
>
> Yes, it is supposed to be. If a feature is needed we're supposed to test
> that it exists before running the test (or marking the test as a 'KFAIL'
> - a known failure).
>
> However, that doesn't always happen.
>
>> 2. Is it bug which need to be fixed that leads to some FAILs in some
>> environment?
>
> The answer here is "it varies". Some of the failures you see are test
> case failures, some are actual failures.
>
> We'd be *happy* to get some help in fixing failures.
>
>> 3. In which system can I get no FAIL with the internal testsuite?
>
> There isn't any os/arch combination that will receive 0 failures. On
> x86_64 (probably our most tested architecture) and a semi-recent kernels
> (such as RHEL7's 3.10 or a more current Fedora release), you'll get in
> the neighborhood of approximately 100 unexpected failures. I've done a
> fair amount of work in the past several months getting the failures down
> to that number.
>
> Why isn't that number 0? Several reasons, one of the biggest being gcc
> debuginfo quality. Sometimes because of poor debuginfo, we can't find a
> function's parameters (for instance).
>
> Note that the number of unexpected failures can vary wildly per
> architecture.
>
>> Can anyone help me?
>
> I hope I've answered your questions. As I said earlier, we'd be happy to
> have help in debugging and fixing up the remaining testsuite failures.
>
> --
> David Smith
> dsmith@redhat.com
> Red Hat
> http://www.redhat.com
> 256.217.0141 (direct)
> 256.837.0057 (fax)