[New-bugs-announce] [issue42411] respect cgroups limits when trying to allocate memory

[Carlos Alexandro Becker]

New submission from Carlos Alexandro Becker <caarlos0 at gmail.com>:

A common use case is running python inside containers, for instance, for training models and things like that.

The python process sees the host memory/cpu, and ignores its limits, which often leads to OOMKills, for instance:

docker run -m 1G --cpus 1 python:rc-alpine python -c 'x = bytearray(80 * 1024 * 1024 * 1000)'


Linux will kill the process once it reaches 1GB of RAM used.

Ideally, we should have an option to make Python try to allocate only the ram its limited to, maybe something similar to Java's +X:UseContainerSupport.

----------
components: IO
messages: 381442
nosy: caarlos0
priority: normal
severity: normal
status: open
title: respect cgroups limits when trying to allocate memory
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42411>
_______________________________________