[issue43634] Extensions build does not respect --jobs setting

[JustAnotherArchivist]

New submission from JustAnotherArchivist <justanotherarchivist at riseup.net>:

The extension building does not respect the --jobs option passed to make. Specifically, in that step, `python setup.py build` always spawns as many gcc processes as there are CPU cores available regardless of that option. This caused problems for me because I have a VM that sees all host machine CPU cores but only has a limited amount of RAM. Despite running `make -j 4`, many more gcc processes are spawned, and this immediately causes memory starvation and a system freeze after a few seconds.

The reason for this is that setup.py blindly enables parallelism in the extension compilation if '-j' appears in the MAKEFLAGS at 3.9.2/setup.py:355. Later on, distutils uses os.cpu_count to set the worker count, i.e. the '-j' *value* is ignored. This behaviour was first introduced with #5309 as far as I can see, though I haven't tested anything other than version 3.9.2.

Hacky workaround: patching the above setup.py line to `self.parallel = 4`.

Cf. https://github.com/pyenv/pyenv/issues/1857

----------
components: Build
messages: 389558
nosy: JustAnotherArchivist
priority: normal
severity: normal
status: open
title: Extensions build does not respect --jobs setting
type: resource usage
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43634>
_______________________________________