Re: [PATCH] glibc: Remove CPU set size checking from affinity functions [BZ #19143]

["Michael Kerrisk (man-pages)" <mtk dot manpages at gmail dot com>]

Hello Florian,

On 03/08/2016 12:17 PM, Florian Weimer wrote:
> On 03/02/2016 03:12 PM, Michael Kerrisk wrote:
> 
>> With this change, I wonder if some pieces in the sched_setaffinity(2)
>> man page (http://man7.org/linux/man-pages/man2/sched_setaffinity.2.html)
>> may be in order. Below, I've quoted some relevant pieces from the man
>> page. Anything there that needs tweaking?
>>
>>    ERRORS
>>        EINVAL The affinity bit mask mask contains no processors that  are
>>               currently  physically  on  the  system and permitted to the
>>               thread according to any restrictions that may be imposed by
>>               the "cpuset" mechanism described in cpuset(7).
>>
>>        EINVAL (sched_getaffinity()   and,   in   kernels   before  2.6.9,
>>               sched_setaffinity()) cpusetsize is smaller than the size of
>>               the affinity mask used by the kernel.
>>    [...]
>>    NOTES
>>    [...]
>>      Handling systems with large CPU affinity masks
>>        The  underlying  system  calls  (which  represent CPU masks as bit
>>        masks of type unsigned long *) impose no restriction on  the  size
>>        of  the  CPU mask.  However, the cpu_set_t data type used by glibc
>>        has a fixed size of 128 bytes, meaning that the maximum CPU number
>>        that  can be represented is 1023.  If the kernel CPU affinity mask
>>        is larger than 1024, then calls of the form:
>>
>>            sched_getaffinity(pid, sizeof(cpu_set_t), &mask);
>>
>>        will fail with the error EINVAL, the error produced by the  underâ
>>        lying  system  call  for the case where the mask size specified in
>>        cpusetsize is smaller than the size of the affinity mask  used  by
>>        the  kernel.   (Depending  on  the system CPU topology, the kernel
>>        affinity mask can be  substantially  larger  than  the  number  of
>>        active CPUs in the system.)
> 
> That's still true.
> 
>>        When  working on systems with large kernel CPU affinity masks, one
>>        must dynamically allocate the mask argument.  Currently, the  only
>>        way  to  do  this  is by probing for the size of the required mask
>>        using sched_getaffinity() calls with increasing mask sizes  (until
>>        the call does not fail with the error EINVAL).
> 
> I think this needs to reference the CPU_ALLOC manual page.
> 
> One caveat is that sched_getaffinity can set bits beyond the requested
> allocation size (in bits) because the kernel gets a padded CPU vector
> and sees a few additional bits.  

I'm not quite clear on this point. Does it get a padded CPU vector
because CPU_ALLOC() might allocate a vector of size larger than the
user requested?

Cheers,

Michael

> The fix for that is to iterate over the
> bits, counting those which are set, and stop if you reach the value of
> CPU_COUNT, rather than iterating over the bits you allocated.
> 
> Florian
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/