[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor <vstinner at redhat.com> added the comment:

Options that should be easy to support later:

* relative executable: use shutil.which() or expose C posix_spawnp() function as os.posix_spawnp()

* env: just pass it as the 3rd argument of os.posix_spawn() (trivial patch)

* restore_signals: use setsigdef (Joannah has a patch)


I don't see how to support following options:

* preexec_fn: I really hate this feature...

* cwd

* start_new_session

* close_fds: there is posix_spawn_file_actions_addclose(), but I'm not sure that we can create a list of file descriptor in a reliable way, since a different thread can open a FD in the meanwhile... Currently, _posixsubprocess iterates on /proc/self/fd/ *after* the fork, when there is a single thread.

* pass_fds: there is not API to mark a fd as inheritable (clear O_CLOEXEC flag). We cannot just change the O_CLOEXEC temporarily since a different thread can spawn a child process "at the same time". The GIL doesn't protect C threads which don't use the GIL.


For pipes like stdout=PIPE or stdout=fd, I didn't look yet how to implement that properly.


Maybe one way to work around the pass_fds limitation in an applicaton is to mark the FDs to pass as inheritable (os.set_inheritable(fd, True)) and use close_fds=False:

fd = ...
subprocess.Popen(..., pass_fds={fd})
os.close(fd)

would become:

fd = ...
os.set_inheritable(fd, True)
subprocess.Popen(..., close_fds=False)
os.close(fd)

But this pattern is not thread if the application has other threads. So that should be the responsibility of the developer, not of Python, to write "unsafe" code" which requires the knownledge of the application behavior.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35537>
_______________________________________