[issue46367] multiprocessing's "spawn" doesn't actually use spawn

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

> It appears the `multiprocessing`'s "spawn" mode doesn't actually use POSIX spawn, but instead uses fork+exec[1].

The documentation doesn't pretend to use posix_spawn(). It only says: "starts a fresh python interpreter process".
https://docs.python.org/dev/library/multiprocessing.html#contexts-and-start-methods

I suggest to close the issue as "not a bug". I don't see anything wrong in the current documentation.

--

posix_spawn() is a function of the C library. It is implemented as fork+exec on most operating systems. I'm only aware of macOS which has a dedicated syscall. Well, posix_spawn() implementation is usually faster thanks to some optimizations.

Python has os.posix_spawn() since Python 3.8.

The subprocess can use os.posix_spawn() on Linux under some conditions:
https://docs.python.org/dev/whatsnew/3.8.html#optimizations

Sadly, it's not used by default, since close_fds=True remains subprocess.Popen() default.

I'm open to use it on more platforms. os.posix_spawn() can only be used if it reports properly errors to the parent process, and some other things and bugs. It's a complex function!

--

Oh, about multiprocessing. Well, someone has to propose a patch! I don't know why multiprocessing uses directly _posixsubprocess.fork_exec() rather than the subprocess module. It's also a complex module with many specific constraints.

posix_spawn() looks nice, but it cannot be used in many cases :-(

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46367>
_______________________________________