Re: buildbot for glibc

[Sergey Matveev <earthdok at google dot com>]

Resending.

On Thu, Feb 12, 2015 at 6:56 AM, Sergey Matveev <earthdok@google.com> wrote:
>
>
> On Thu, Feb 12, 2015 at 5:33 AM, Carlos O'Donell <carlos@redhat.com> wrote:
>>
>> On 02/11/2015 08:06 PM, Roland McGrath wrote:
>> > Google is now hosting a new buildbot setup for glibc as a service to the
>> > community.  Things are extremely primitive at the moment, but we hope to
>> > improve and expand the setup in the future (with your help!).
>>
>> Thank you! This is really awesome!
>>
>> > The hosts are virtual machines provided by Google Compute Engine.  The
>> > machine resources and setup help are being donated by Google's Dynamic
>> > Tools group (maintainers of AddressSanitizer et al).  Special thanks to
>> > Sergey Matveev for doing the initial installation and for ongoing help
>> > with
>> > the setup and administration.
>> >
>> > You can see the build status at:
>> >       http://glibc-build.hack.frob.com/waterfall
>>
>> Cool.
>>
>> > That name (on my personal server) is a simple redirect to the actual
>> > page,
>> > which does not yet have a pretty URL.  We'll need some help figuring out
>> > exactly how best to set things up to yield a more pleasant browsing
>> > experience.
>>
>> This looks good to me. We should get your server out of the loop to avoid
>> loading your system.
>>
>> I've added a redirect from the main site for you here:
>> http://www.gnu.org/software/libc/waterfall
>>
>> > You can currently get the buildbot configuration and build scripts with:
>> >       svn co
>> > https://address-sanitizer.googlecode.com/svn/trunk/glibc_buildbot
>> > We want eventually to find a more appropriate place for that stuff to
>> > live,
>> > where the glibc community can collaborate directly on maintaining the
>> > setup.
>>
>> Agreed.
>>
>> > For now, there are two "bots", both actually running on the same one
>> > virtual machine (called a "slave" in Buildbot lingo).  One does the
>> > basic
>> > x86_64-linux-gnu build, and one the basic i686-linux-gnu build.
>>
>> Awesome.
>>
>> > Fresh builds are kicked off every time there is a new commit to master.
>> > It polls every few minutes to notice.
>>
>> Perfect.
>>
>> > The slave OS installation is based on Debian wheezy, which has only GCC
>> > 4.7.  The i686 build always fails in 'make check' because I haven't
>> > figured
>> > out what packages to install so that 'g++ -m32' will find -lstdc++.
>> > (Actually, I just installed another one and it might work next cycle.)
>>
>> Saw that in the build logs :-)
>>
>> > The x86_64 build has a few test failures (but all the tests do build),
>> > which I have not looked into yet.  The build logs do not get much
>> > information about how they fail, just the summary emitted at the end.  I
>> > can log into the slave and poke around, but I don't think I'll be able
>> > to
>> > offer that access to people outside Google.  Perhaps someone should whip
>> > up a script that examines tests.sum and prints out the .test-result and
>> > .out files for each FAIL--though that is by no means necessarily enough
>> > information to understand the failure mode.
>
> You can also make that output appear as a separate snippet on the build
> status page by printing some magic strings along with it. Example:
> http://goo.gl/tw20wA (click on stdio to see how it's done). This is pretty
> useful as you don't have to scroll through the full log. This is done by an
> annotator script which I picked up from Chromium (hopefully this doesn't
> cause any licensing issues).
>
>>
>>
>> That's what we do for Fedora. Our build logs are a combination of the
>> glibc logs, plus this snippet:
>>
>> for i in `sed -n 's|^.*\*\*\* \[\([^]]*\.out\)\].*$|\1|p'
>> build-*-linux*/check.log`; do
>>   echo =====$i=====
>>   cat $i || :
>>   echo ============
>> done
>>
>> > As with all infrastructure for the project, there is no big supply of
>> > paid
>> > hacker time to work on this.  It's intended to grow to be of great use
>> > to
>> > us developers and maintainers, but the only way that will happen is if
>> > more
>> > folks in the community pitch in to improve and maintain it.
>>
>> How can I add more slaves? Can we add more slaves from outside of the
>> google
>> network?
>
> Yes, it's pretty simple:
> - add your slave in the master config, restart the master;
> - on your slave machine, do a slave setup as described here:
> http://docs.buildbot.net/latest/manual/installation.html For master ip
> address/port use 130.211.48.148:9991. Slave name must match the name in the
> master config. Roland can give you the password - it's the same for every
> slave.
> - start the slave. The rest happens automatically. (Don't forget to schedule
> the slave to auto-start on reboot.)
>
>
>>
>>
>> At a minimum I have access to and the ability to add ppc64, ppc64le,
>> s390x,
>> s390, and hppa slaves.
>>
>> Cheers,
>> Carlos.
>>
>