[issue35238] Alleviate memory reservation of fork_exec in subprocess.Popen via forkserver

Antoine Pitrou report at bugs.python.org
Mon Dec 17 15:57:15 EST 2018


Antoine Pitrou <pitrou at free.fr> added the comment:

At any rate, given the constraints you're working with (thousands of child processes, memory conservation issues), I suggest you abandon the idea of using multiprocessing and write your own subprocess-server instead.

I would suggest doing so using asyncio, which should allow you to control as many subprocesses as you want without spawning countless threads or intermediate processes:

https://docs.python.org/3/library/asyncio-subprocess.html

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35238>
_______________________________________


More information about the Python-bugs-list mailing list