Re: [PATCH] Improve analysis of racy testcases

[Sergio Durigan Junior <sergiodj at redhat dot com>]

On Thursday, February 25 2016, Antoine Tremblay wrote:

> Sergio Durigan Junior writes:
>
>> This patch is a proposal to introduce some mechanisms to identify racy
>> testcases present in our testsuite.  As can be seen in previous
>> discussions, racy tests are really bothersome and cause our BuildBot to
>> pollute the gdb-testers mailing list with hundreds of false-positives
>> messages every month.  Hopefully, by identifying these racy tests in
>> advance (and automatically) will contribute to the reduction of noise
>> traffic to gdb-testers, maybe to the point where we will be able to send
>> the failure messages directly to the authors of the commits.
>> [...]
> Thanks for this ! This was quite a problem for me while testing on arm.
> I'm testing it now...

Thanks!  Please let me know if you find anything wrong with the script.

> One note maybe it would be nice output the list of unracy tests too to
> be able to auto-build a list of tests to run out of this since I'm not
> sure you can set an exclusion list ?

Hm, it would be possible to output the non-racy tests, but only to a
different file (instead of outputting to stdout, as I'm doing with the
racy tests).  Perhaps this could be a separate option to the script?
I'm not sure if the users would always want this information...

As for the exclusion list you mentioned, DejaGNU's runtest allows one to
specify a --ignore flag with the names of the tests you don't want to
run.  It should be possible to pass this via RUNTESTFLAGS, but I haven't
tried.  I'll give it a go tomorrow.

Cheers,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/