This is the mail archive of the
libc-locales@sourceware.org
mailing list for the GNU libc locales project.
Re: localedef during tests suddenly needs a lot of memory
- From: Florian Weimer <fweimer at redhat dot com>
- To: Joseph Myers <joseph at codesourcery dot com>
- Cc: GNU C Library <libc-alpha at sourceware dot org>, libc-locales at sourceware dot org
- Date: Thu, 17 Aug 2017 18:43:20 +0200
- Subject: Re: localedef during tests suddenly needs a lot of memory
- Authentication-results: sourceware.org; auth=none
- Authentication-results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com
- Authentication-results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=fweimer at redhat dot com
- Dmarc-filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 193CEC04B927
- References: <8d69bb17-b684-dfa4-2248-36a7fd65457d@redhat.com> <alpine.DEB.2.20.1708171626530.25361@digraph.polyomino.org.uk>
On 08/17/2017 06:29 PM, Joseph Myers wrote:
> On Thu, 17 Aug 2017, Florian Weimer wrote:
>
>> With upstream master as of commit
>> bb6274ee1293a6bc76d9d7c889783303de181295, I see that make check in the
>> localedata subdirectory needs lots and lots of memory (several gigabytes).
>
> localedef has never been particularly parsimonious in its use of memory,
> but historically that was of the form "if you're testing on a board with
> e.g. only 32 MB of memory, you might need to cross-build the locales". It
> shouldn't need several GB; I'd say if it's even using 256 MB it should be
> made smarter.
I see ~4.8 GiB for fr_FR.UTF-8 on a 64-bit system:
0.68user 1.21system 0:01.89elapsed 99%CPU (0avgtext+0avgdata
4828636maxresident)k
0inputs+3160outputs (0major+1235980minor)pagefaults 0swaps
This is with the localdef from glibc 2.24, so it's definitely the data
files which cause this.
valgrind says:
==9080== HEAP SUMMARY:
==9080== in use at exit: 4,942,949,686 bytes in 238,207 blocks
==9080== total heap usage: 282,795 allocs, 44,588 frees, 5,021,038,614
bytes allocated
On a 32-bit system, its:
0.78user 0.48system 0:01.26elapsed 99%CPU (0avgtext+0avgdata
2442808maxresident)k
0inputs+3160outputs (0major+626570minor)pagefaults 0swaps
==24== in use at exit: 2,494,524,387 bytes in 234,694 blocks
==24== total heap usage: 279,269 allocs, 44,575 frees, 2,550,480,781
bytes allocated
That's probably beyond what's available to user space on a 32-bit
kernel. 8-(
Thanks,
Florian