Re: Automated build and test system (overview)

[Andrew Cagney <cagney at redhat dot com>]

Hi Kris,

To what extent is this system expected to test the less used build / 
test / release paths?

For instance, a deployed frysk release includes its testsuite (funit, 
fsystest, ftail).  We've found lets us quickly ascertain the state of a 
system and identify existing problems - especially by running fsystest.  
Unfortunately developers rarely run the full "make distcheck" or test 
with no build tree, so that is missing some love.

Similarly, it is possible to run additional stress tests - funit 
--stress - these tests are non-deterministic by design and are expected 
to expose new edge cases.  Any problems they identify are reduced to 
more-deterministic unit tests and added to the default test pool.  
Unfortunately, they are only run occasionally.

There's also the option of running << funit --unresolved >> to confirm 
status of the unresolved bugs; although I suspect more useful is to 
check to see if the bug report is still open :-)

Build.samba.org looks like the old tinerbox system for mozilla; being 
able to drill down to the logs using a web interface, and not be swamped 
by daily e-mail, is certainly appreciated.

Andrew