This is the mail archive of the
frysk@sources.redhat.com
mailing list for the frysk project.
Re: Automated build and test system (overview)
On Mon, Mar 12, 2007 at 01:23:28PM +0100, Mark Wielaard wrote:
> Hi Kris,
>
> On Fri, 2007-03-09 at 09:15 -0800, Kris Van Hees wrote:
> > The main objectives for the build farm system are:
> >
> > - Any system that is connected to the Internet, and that has the
> > required software package dependencies installed needed to
> > build frysk should be eligible to become part of the build
> > farm.
> > - The build farm should not limit itself to just supporting
> > frysk builds. Virtually any project can be added to it.
> > - Systems in the build farm do not all have to build every
> > single project. This can be controlled per system.
> > - The build farm scripts must support unattended operation as
> > much as possible, including automatic updating of the scripts
> > as new ones are released.
>
> Ambitious goals! :)
Fortunately, this has been done (a couple of times over in fact) before,
so in the end it isn't too hard to do. Note that the objectives do not
mention reporting, which is a whole different thing. Right now, the
scripts support all goals listed above, although the 'unattended
operation' is a bit of a moving target, especially with kernel bugs
getting triggered during frysk builds :)
The inspiration for the build farm system as I wrote it came from the
Samba build farm (http://build.samba.org/) for anyone interested in the
roots of this design.
<<snip>>
> > The logfile will be used to report on the success (or failure) of the
> > build, and is analyzed by scripts (not part of the build farm scripts)
> > to extract the information needed for the reporting.
>
> Do you have any plans on having more "intelligent" parsing of the result
> files? Just having the result files published for various architectures
> over time will be of great help. Having something that can pinpoint when
> which test result flipped from PASS/FAIL would be great.
I definitely hope to be able to make the reporting more than just
pass/fail kind of information for the various stages. Given that CVS
does not provide for a project-wide revision id for commits, and instead
handles it on a per-file basis, it becomes a bit more complicated to
report on all changes that happened between two build runs, but it
certainly can be done. At a minimum, I am working on storing the
current revision of every file in the source tree along with the logs,
so upon a switch from PASS to FAIL, I should be able to list the files
that got changed from the last build until now, along with their
revision id then and now. Reporting on changelog entries is possible
also, of course, but since I'd need the revision ids for that anyway, I
figure sticking to the ids being reported for now may be less ambitious.
> Also, have you looked at The Gnome BuildBrigade
> http://live.gnome.org/BuildBrigade It would be nice to somehow integrate
> with that to make us feel more part of the Gnome world. There goals seem
> pretty broad though, you might want to concentrate on Frysk first of
> course.
I'll definitely have a look, but for now I think keeping focus on frysk
is crucial. The reporting piece is still quite a bit of work.
Cheers,
Kris