This is the mail archive of the frysk@sources.redhat.com mailing list for the frysk project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: ANNOUNCEMENT: Release of the automated build-and-test system


Checking in from vacation (hence the late response)...

On Mon, Apr 02, 2007 at 02:43:39PM +0200, Mark Wielaard wrote:
> Hoi Kris,
> 
> On Wed, 2007-03-28 at 22:01 -0700, Kris Van Hees wrote:
> > I am happy to announce that the automated build-and-test system is
> > available for general participation.  This is a major milestone in the
> > development of this system, because it allows direct (and safe)
> > contributions to the overall coverage that automated tests can provide
> > by anyone interested in donating some cycles on their system(s).
> 
> Thanks! Doing a test build now.

Great!  If you are able to commit a few cycles every night or so, please
let me know (and provide me with the uname -a output for the system you
can run it on) and I will send you a password to enable you to upload
results.

> > You can find more information (and instructions for download and
> > installation) at:
> > 
> > 	http://build.alchar.org/~aedil/
> > 
> > Please note that this system is currently in *beta* mode, so things are
> > still changing in various places, and the central reporting is not yet
> > operational (although upload of logs is).  I look forward to comments.
> 
> It is pretty generic, but also very centralistic. Could the build
> scripts, or at least the frysk_*.pkg files be stored in frysk CVS?
> Likewise for mailing/uploading the results. That way a project could
> handle its own tests and result processing.

The system is not meant to be Frysk specific and truly intended to be
outside of Frysk CVS for all of its functions.  If I were to put the
frysk_*.pkg files in Frysk CVS, then people would not be able to use
those files without having the build_farm scripts, and the other way
around.  It is meant to be an independent unit that pulls the source
tree of Frysk (and any other project we define in it), and performs a
build-and-test run on it.  The project (Frysk in this case) is still
responsible for defining tests in its testsuite.  And (of course) there
is also a project specific piece on the reporting server that parses the
project specific data from the logs.  But I do think that there is more
value in keeping that outside of Frysk rather than putting it in Frysk
CVS.  The reporting piece has more dependencies on the build_farm stuff
than it has on any of the supported projects.

Another reason for being centralistic is that reporting is meant to be
done on a central system, where support for a variety of projects can be
coordinated.  The goal is to have an automated build-and-test system
that is very lightweight, easy to deploy, and with minimal dependencies.

> Asking contributors to request a password is an extra step that I would
> not add as long as you can identify different build hosts by email
> address or ip-address. Lowering barriers to contribute is good. See for
> example how gcc does it, you can do an anonymous checkout and then run
> the test scripts that will email the results to a public list so
> everybody can easily contribute and get at the raw results:
> (See the various build scripts and the test_summary script)
> http://gcc.gnu.org/viewcvs/trunk/contrib/
> http://gcc.gnu.org/ml/gcc-testresults/current/

The main reason for putting up a small barrier is that we obviously are
facing potential malicious uploads.  While a password-protection is only
a limited level of security, it does pose enough of a barrier to avoid
the majority of pests.  Given that the central server will provide access
to the results, both in parsed and raw format, everybody should be able
to get to that data at will.  The advantage of using an upload area and
a central reporting server is that we have a place where (rather large)
logs are stored.  If a bug is found in the parsing scripts, it is easy
enough to re-run reporting on older logs.  Doing this on a central
server also allows us to automatically run comparisons between various
build-and-test runs.  In all, it offers (limited) data mining
possibilities that otherwise would only be available if we were to have
nodes mail the full logs, and that would be rather heavy.

> You say that you keep version information as a list of CVS revision ids.
> Wouldn't a date/time suffice? Or just a revision tag in the tree?

Well, perhaps.  However, note that CVS keeps revisions on a per-file
basis, and that commits are not exactly atomic (when multiple files are
commited at the same time).  Given the granularity of date/time stamps,
there is a potential for differences between the tree used for the build
and one to re-create the build.  What we do is actually a combination.
The revision list contains a date/time stamp as well (in UTC), which is
used as the first indicator of a tree revision.  After doing a checkout
using the date/time stamp found in the rev file, the per-file revision
ids are compared between the checked out tree and the rev file, and any
discrepancies are resolved using the revision id from the rev file.
that guarantees that the tree will be identicial to the one used to
generate the rev file.

Using a revision tag in the tree would work if we were having nodes do
coordinated builds, i.e. builds of specific revision tags, like one per
night.  The trouble is that we'd be cluttering up the CVS space with
tons and tons of tags that have only a limited use in time, and we also
end up having nodes perform build-and-test runs on revisions that may
already be known to be bad.  By having every node set their own schedule
for when they run tests, and have them test against the current CVS
HEAD, we cover a lot more ground.

> BTW. There are no distribution terms on these sources, could you put
> them under the GPL?

Thanks for catching this.  The scripts have now been released under the
GPLv2.

	Cheers,
	Kris


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]