[issue42041] venv subprocess call to python resolves to wrong interpreter

[Eryk Sun]

Eryk Sun <eryksun at gmail.com> added the comment:

> I'd read it because I'm interested, but that is probably too much detail
> for someone who is trying to get something done quickly.

I did say that I don't know how much should be documented. I tend to dump a lot of information in issues so that people can make informed decisions. Personally I would keep it limited. I do want more help, and generic help, in the warning, but I don't want to spam the already exhausting and MEGO-prone (i.e. my eyes glaze over) subprocess docs.

With regard to the use of a platform search, note that subprocess actually implements its own search in POSIX. (subprocess supports posix_spawn -- but not posix_spawnp -- on macOS and on Linux with glibc 2.24+, but it's limited to a narrow subset of Popen arguments, excluding `cwd`, and it requires a qualified `executable` path.) A name containing a slash is resolved against `cwd` or the process current working directory. An unqualified name is searched for in the os.get_exec_path(env) search path, based on PATH from `env` or os.environ, else os.defpath. This allows `env` to override the POSIX search path, just as `cwd` in POSIX overrides the current working directory when resolving a qualified args[0] path and `executable`. This is sort of documented by saying that POSIX Popen() works "os.execvp()-like", but to be more correct it should say "os.execvpe()-like". The final "e" in "execvpe" is fundamentally important since PATH can be sourced from `env`.

In Windows, the search path and the working directory used for resolving paths can't be overridden without passing shell=True. subprocess incorrectly documents this by claiming in general that "the function looks for `executable` (or for the first item in `args`) relative to `cwd` if the executable path is a relative path". That wording isn't even correct for POSIX, for which the working directory only applies to args[0] if it contains a slash.

In light of this, how about inserting the following warning:

      Resolving the path of *executable* or the first item of *args* is
      platform dependent even with ``shell=False``. For POSIX, see
      :meth:`os.execvpe`, and note that when resolving or searching for the
      executable path, *cwd* overrides the current working directory and *env*
      can override the ``PATH`` environment variable. For Windows, see the
      documentation of the ``lpApplicationName`` and ``lpCommandLine``
      parameters of WinAPI ``CreateProcess``, and note that when resolving or
      searching for the executable path with ``shell=False``, *cwd* does not
      override the current working directory and *env* cannot override the
      ``PATH`` environment variable. Cross-platform usage can improve
      reliability by using a fully-qualified path for the executable, such as
      from :meth:`shutil.which`.

      On all platforms, using :data:`sys.executable` as the path of the
      executable is the recommended and reliable way to run the current Python
      interpreter.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42041>
_______________________________________