This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [PATCH] Count number of logical processors sharing L2 cache
- From: Carlos O'Donell <carlos at redhat dot com>
- To: "H.J. Lu" <hjl dot tools at gmail dot com>
- Cc: Florian Weimer <fweimer at redhat dot com>, GNU C Library <libc-alpha at sourceware dot org>
- Date: Tue, 31 May 2016 11:26:33 -0400
- Subject: Re: [PATCH] Count number of logical processors sharing L2 cache
- Authentication-results: sourceware.org; auth=none
- References: <CAMe9rOoy2YaQTdyqZpQ3=ytDc5dywNshzHAN2ymN60=L5KwbiA at mail dot gmail dot com> <CAMe9rOoq8MNkX0GvoePQ-C51mfUr2ikrRJgqCZE0CoGoJEmOOw at mail dot gmail dot com> <d4cf36ee-f402-41fe-5108-e072b47f2399 at redhat dot com> <CAMe9rOpUuYboLH9WgyHH4HiBaSBXJ+uB=MPUft2S26N+wYJ9-A at mail dot gmail dot com> <76801b5c-7770-23a9-9b7c-4e44722247e1 at redhat dot com> <CAMe9rOqAuxZZ=gpd1zXvbRrsqjhT8G6C9WBbpwaqa65s=-ZTnQ at mail dot gmail dot com> <57449424 dot 1000009 at redhat dot com> <CAMe9rOpzKgS_uNz1ZFg-sUxCbuY4tFOy19eLjyFaO7UbNYUr1g at mail dot gmail dot com> <574D994F dot 5050207 at redhat dot com> <CAMe9rOq1UkT_uCqtt33KOxt_+gfpg9MHd6pV0W76_CMBgafm9g at mail dot gmail dot com>
On 05/31/2016 10:57 AM, H.J. Lu wrote:
> On Tue, May 31, 2016 at 7:01 AM, Carlos O'Donell <carlos@redhat.com> wrote:
>> On 05/24/2016 05:35 PM, H.J. Lu wrote:
>>> CAT dedicates part of L3 cache to certain processor or process so
>>> that L3 cache is always available to them. Glibc tries not to take all
>>> L3 cache in memcpy/memset so thar L3 cache is available for other
>>> operations within the same process as well as to other processor/process.
>>> CAT and glibc work at different angels. There is no direct conflict between
>>> CAT and glibc. At the moment, I am not sure if CAT-aware glibc will
>>> improve performance.
>>
>> What do you mean by "no direct conflict?"
>>
>> If glibc tuns its own algorithms to use 1/4 of L3, but CAT has only
>> allocated 1/5 of L3 to that process, then glibc's algoirthms, whose
>> intent was to use a small amount of L3 are now using *more* L3 than
>> the entire process has and that could impact performance?
>
> Cache sizes are only used for instructions selections in
> string/memory functions, which don't use cache directly.
> Glibc has NO control whatsoever on how much cache
> string/memory functions use.
Even if glibc doesn't use cache directly, it's making algorithm choices
based on those sizes and tunings.
For example:
sysdeps/x86/cacheinfo.c
765 /* The large memcpy micro benchmark in glibc shows that 6 times of
766 shared cache size is the approximate value above which non-temporal
767 store becomes faster. */
768 __x86_shared_non_temporal_threshold = __x86_shared_cache_size * 6;
If, for example, CAT changed the shared L3 cache size to be smaller, would
it invalidate the above static tuning?
If it does, then we should change the comments in several places to mention
CAT.
>> Did I miss something?
>>
>> There could be opportunities for incorrect tunings if glibc is not
>> CAT-aware?
>>
>
> Since CAT can change at any time during the life of a process,
> checking CAT inside glibc isn't very practical.
Most reasonable administrators would use CAT to limit one particular
workload and leave the CAT setting until the workload changed.
Therefore it is entirely reasonable IMO to check CAT settings at
process startup and adjust accordingly e.g. CAT-aware glibc.
Checking at every use is too expensive and does not mirror what
an administrator would be doing with CAT settings?
--
Cheers,
Carlos.