[issue35797] concurrent.futures.ProcessPoolExecutor does not work in venv on Windows

[Eryk Sun]

Eryk Sun <eryksun at gmail.com> added the comment:

> The current solution is the simplest one that ensures the most 
> compatibility for the least amount of risk, even though that 
> was not zero risk, as we've seen. 

How about making the py[w].exe launcher unset __PYVENV_LAUNCHER__ whenever it runs a registered version explicitly?

We could also modify the embedded-script (entry-point) launchers, which would be simpler if we had them in core instead of distlib. They could be updated to look for pyvenv.cfg. If found, it would override the shebang and set the __PYVENV_LAUNCHER__ environment variable. This would eliminate the interjected process in a 3.7.2 virtual environment, e.g. pip.exe -> python.exe (venv launcher) -> python.exe (installed) would become pip.exe -> python.exe (installed). More importantly, it would unset __PYVENV_LAUNCHER__ in case it's not a virtual environment script. 

This way running pip.exe for an installed Python won't end up installing into a virtual environment, as can happen now in 3.7.2. For example:

    >>> print(os.environ['__PYVENV_LAUNCHER__'])
    C:\Temp\env37\Scripts\python.exe

    >>> import requests
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ModuleNotFoundError: No module named 'requests'

    >>> pip = 'Scripts\\pip.exe'
    >>> out = subprocess.check_output('{} install requests'.format(pip))

    >>> import requests
    >>> requests.__version__
    '2.21.0'

Note that "Scripts\\pip.exe" in a CreateProcess command line gets resolved first relative to the application directory, before trying the current directory, system directories, and PATH directories.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35797>
_______________________________________