Re: Automated build and test system (overview)

[Mark Wielaard <mark at klomp dot org>]

Hi Kris,

On Fri, 2007-03-09 at 09:15 -0800, Kris Van Hees wrote:
> The main objectives for the build farm system are:
> 
> 	- Any system that is connected to the Internet, and that has the
> 	  required software package dependencies installed needed to
> 	  build frysk should be eligible to become part of the build
> 	  farm.
> 	- The build farm should not limit itself to just supporting
> 	  frysk builds.  Virtually any project can be added to it.
> 	- Systems in the build farm do not all have to build every
> 	  single project.  This can be controlled per system.
> 	- The build farm scripts must support unattended operation as
> 	  much as possible, including automatic updating of the scripts
> 	  as new ones are released.

Ambitious goals! :)

> The build_and_test function sets up project-specific files (for logging,
> checksum tracking, and per-project locking), acquires a lock for the
> project, and executed the following steps:
> 
> 	1. Fetch the source code (using cvs, wget, svn, or rsync)
> 	2. Compute a checksum of the freshly retrieved tree
> 	3. If nothing changed in the source code (or the build farm
> 	   scripts), there is no need to build and test, and the
> 	   function returns
> 	4. Configure the build of the project
> 	5. Build the project
> 	6. Execute the tests for the project
> 
> Each step calls a project-specific pre-action function before doing the
> actual action, and a post-action function after doing the action, to
> allow for customization.  The actions themselves can also be provided as
> a project-specific override.
> 
> At the completion of the build_and_test function, the logfile will be
> "publish" to a defined location (generic or project-specific).  This can
> be uploading it using scp, rsync, or moving it to a specific location on
> the local system.
> 
> The logfile will be used to report on the success (or failure) of the
> build, and is analyzed by scripts (not part of the build farm scripts)
> to extract the information needed for the reporting.

Do you have any plans on having more "intelligent" parsing of the result
files? Just having the result files published for various architectures
over time will be of great help. Having something that can pinpoint when
which test result flipped from PASS/FAIL would be great.

Also, have you looked at The Gnome BuildBrigade
http://live.gnome.org/BuildBrigade It would be nice to somehow integrate
with that to make us feel more part of the Gnome world. There goals seem
pretty broad though, you might want to concentrate on Frysk first of
course.

Cheers,

Mark