[issue33166] os.cpu_count() returns wrong number of processors on specific systems

[Eryk Sun]

Eryk Sun <eryksun at gmail.com> added the comment:

> if not available fallback on 
> GetActiveProcessorCount(ALL_PROCESSOR_GROUPS)

The fallback for older versions of Windows is dwNumberOfProcessors from GetSystemInfo. This can be removed from 3.7 and 3.8, which no longer support Windows versions prior to Windows 7.

> GetLogicalProcessorInformationEx() is what should really be used. 

GetActiveProcessorCount and GetMaximumProcessorCount are implemented via GetLogicalProcessorInformationEx (i.e. NtQuerySystemInformation, SystemLogicalProcessorAndGroupInformation). They query the RelationGroup information. For ALL_PROCESSOR_GROUPS, they respectively sum the ActiveProcessorCount and MaximumProcessorCount over all groups.

These functions were added in Windows 7 to support the implementation of logical processor groups, which allows up to 64 logical processors per group. Each process is created in a single group, which is assigned round-robin. A thread can call SetThreadGroupAffinity to manually switch to another group.

Apparently someone at Microsoft advised calling GetMaximumProcessorCount (see issue 30581), but I don't follow this decision. Why should os.cpu_count() include CPUs that may or may not come online? Also, on POSIX it reports sysconf(_SC_NPROCESSORS_ONLN), not sysconf(_SC_NPROCESSORS_CONF), so for Windows it should instead call GetActiveProcessorCount. I assume on BSD that HW_NCPU is similar, though I'm not sure for MacOS. Also on MacOS it appears to be deprecated in favor of HW_LOGICALCPU, HW_LOGICALCPU_MAX, HW_PHYSICALCPU, and HW_PHYSICALCPU_MAX.


> which may still report the wrong number of CPUs on 32 bit processes.

32-bit Windows and WOW64 emulation are limited to 32 CPUs. Applications that need more logical processors should be 64-bit

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33166>
_______________________________________