[New-bugs-announce] [issue46720] Add support of path-like objects to multiprocessing.set_executable for Windows to match Unix-like systems

Géry report at bugs.python.org
Fri Feb 11 09:57:09 EST 2022


New submission from Géry <gery.ogam at gmail.com>:

Any [path-like object](https://docs.python.org/3/glossary.html) can be passed to `multiprocessing.set_executable`, i.e. objects with `str`, `bytes`, or `os.PathLike` type.

For instance these work (tested on MacOS with all start methods: ‘spawn’, ‘fork’, and ‘forkserver’):

- `multiprocessing.set_executable(sys.executable)` (`str`);
- `multiprocessing.set_executable(sys.executable.encode())` (`bytes`);
- `multiprocessing.set_executable(pathlib.Path(sys.executable))` (`os.PathLike`).

This is because the ‘fork’ start method does not exec any program in the subprocess, the ‘spawn’ start method converts its path argument to `bytes` with `os.fsencode` before passing to [`_posixsubprocess.fork_exec`](https://github.com/python/cpython/blob/v3.10.2/Lib/multiprocessing/util.py#L452-L455), and the ‘forkserver’ start method spawns a server process (like with the ‘spawn’ start method) which then forks itself at each request (like the ‘fork’ start method):

```
        return _posixsubprocess.fork_exec(
            args, [os.fsencode(path)], True, passfds, None, None,
            -1, -1, -1, -1, -1, -1, errpipe_read, errpipe_write,
            False, False, None, None, None, -1, None)
```

Linux (and other Unix-like systems) uses the same code than MacOS for the three start methods so it should work for it too.

However I have not tested this on Windows which uses the function [`_winapi.CreateProcess`](https://github.com/python/cpython/blob/v3.10.2/Lib/multiprocessing/popen_spawn_win32.py#L73-L75) for the ‘spawn’ start method (the only start method available on this OS) but I noticed that no conversion to `str` (not to `bytes` this time, since [the function expects `str`](https://github.com/python/cpython/blob/v3.10.2/Modules/_winapi.c#L1049)) of the path argument with `os.fsdecode` (not `os.fsencode` this time) is performed before passing it to the function:

```
                hp, ht, pid, tid = _winapi.CreateProcess(
                    python_exe, cmd,
                    None, None, False, 0, env, None, None)
```

So on Windows only `str` path can be passed to `multiprocessing.set_executable`. This PR fixes this to be on a par with Unix-like systems which accept any path-like objects.

----------
components: Library (Lib)
messages: 413073
nosy: maggyero
priority: normal
severity: normal
status: open
title: Add support of path-like objects to multiprocessing.set_executable for Windows to match Unix-like systems
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46720>
_______________________________________


More information about the New-bugs-announce mailing list