Re: Large parallel glibc builds.

[Florian Weimer <fweimer at redhat dot com>]

On 12/23/2015 03:15 PM, Carlos O'Donell wrote:

>> I still see the linknamespace failures
>>
>> FAIL: conform/ISO/assert.h/linknamespace
>> FAIL: conform/ISO/ctype.h/linknamespace
>> FAIL: conform/ISO/errno.h/linknamespace
>> FAIL: conform/ISO/locale.h/linknamespace
>> â
>>
>> reported here first:
>>
>>   <https://sourceware.org/ml/libc-alpha/2015-05/msg00490.html>
> 
> Please file a bug about this so we can track it (like the locale
> issue H.J. reported).

Done:

  <https://sourceware.org/bugzilla/show_bug.cgi?id=19398>

I still don't know the exact cause.  Maybe it's a Makefile dependency
issue (e.g., not all inputs are actually generated at the time the
command is run).  It does not seem to be a file overwrite in
liknamespace.pl itself, the strace output looks clean.

>> Some tests have problems because they run into scalability issues
>> (thread creation is particularly slow on large machines, likely due to
>> mmap being extremely expensive).
> 
> If we could find out which tests have problems that would be useful.

I just posted a workaround for tst-malloc-thread-exit.

>> Consequently, this machine is not significantly faster for building
>> glibc than an average laptop, and testing is in fact quite a bit slower.
> 
> What do you think is the bottleneck here?

On top of the scaling issue of some of the threading tests, it looks as
if lack of parallelism between subdirectories is the major culprit.
This means that single-thread performance determines build and test
times to a large extent.

math/gen-libm-test.pl is major seralization point during builds (not
tests).  During tests, it's compilation test-tgmath2.c (related to poor
performance of debugging information generation in GCC, as discussed
before).

Florian