[issue35797] concurrent.futures.ProcessPoolExecutor does not work in venv on Windows

[Steve Dower]

Steve Dower <steve.dower at python.org> added the comment:

> I would prefer a -X command-line option, such as "VIRTUAL_ENV". multiprocessing could propagate this option to worker processes.

Right, so would I, but it needs to be propagated basically everywhere (unless "everywhere" all defers through multiprocessing, which would be great). I chose to go with the same solution as used on macOS for the same problem, though it had to be implemented in a different part of the code. Plenty of other environment variables will cause problems if they propagate to a totally different python.exe, so perhaps we should document __PYVENV_LAUNCHER__ as one to clear in this case (along with PYTHONHOME and PYTHONPATH)?

For future enhancement, there's also the stalled PEP 582 that would "fix" venv's reliance on shell/environment variables by having a different way to enable a separate package directory (FWIW, it stalled because the PEP doesn't fix *all* packaging problems, but only some). 

But as it stands, people rely on there being something they can launch in their venv directory, and that thing has to know (at least for a given terminal session) how to propagate the correct search paths to child processes. The current solution is the simplest one that ensures the most compatibility for the least amount of risk, even though that was not zero risk, as we've seen. And it also has to remain for 3.7 now, since venvs are no longer automatically broken when updating to 3.7.3.

Changing sys.executable to the final executable and trusting/relying/hoping that everyone who invokes it also allows environment variables to propagate is probably the best solution (including whatever site.py fixes are required, if any).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35797>
_______________________________________