Re: About FAILs in the internal testsuite

[David Smith <dsmith at redhat dot com>]

On 04/13/2016 07:17 PM, peter wang wrote:
> Hello,
> 
> I run the internal testsuite in CentOS 6.7 and got some FAILs in the result.
> Then I got serveral questions:
> 1. Is the testsuite designed for all the kernel version?

Yes, it is supposed to be. If a feature is needed we're supposed to test
that it exists before running the test (or marking the test as a 'KFAIL'
- a known failure).

However, that doesn't always happen.

> 2. Is it bug which need to be fixed that leads to some FAILs in some
> environment?

The answer here is "it varies". Some of the failures you see are test
case failures, some are actual failures.

We'd be *happy* to get some help in fixing failures.

> 3. In which system can I get no FAIL with the internal testsuite?

There isn't any os/arch combination that will receive 0 failures. On
x86_64 (probably our most tested architecture) and a semi-recent kernels
(such as RHEL7's 3.10 or a more current Fedora release), you'll get in
the neighborhood of approximately 100 unexpected failures. I've done a
fair amount of work in the past several months getting the failures down
to that number.

Why isn't that number 0? Several reasons, one of the biggest being gcc
debuginfo quality. Sometimes because of poor debuginfo, we can't find a
function's parameters (for instance).

Note that the number of unexpected failures can vary wildly per
architecture.

> Can anyone help me?

I hope I've answered your questions. As I said earlier, we'd be happy to
have help in debugging and fixing up the remaining testsuite failures.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)