[Linux-SIG] Revisit of PEP 394 -- The "python" Command on Unix-Like Systems

[Nick Coghlan]

On 2 August 2017 at 08:54, Barry Warsaw <barry at python.org> wrote:
> On Jul 31, 2017, at 19:40, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> - we declare "/usr/bin/env python" and other environment aware
>> invocations as already being effectively addressed, thanks to venv,
>> environment modules, conda, and other existing PATH based solutions
>> that change the interactive meaning of "python"
>> - by some suitable mechanism, we offer 3 different configurations for
>> the absolute "/usr/bin/python" path: symlink to /usr/bin/python2,
>> symlink to /usr/bin/python3, and "Not configured yet, here's how to
>> select your default interpreter" (as a matter of UX, I'm beginning to
>> think we'll need to make the last one an actual installable script in
>> its own right, as the default interpreter not found error from the
>> shebang handler isn't particularly informative)
>
> Another option is to make /usr/bin/python not actually *the* Python interpreter, but instead a wrapper that does $magic to get the right version.  Several options have been bandied about over the years, and IIRC, Windows does something like this.  Maybe it’s switchable based on a config file (both system and user overridable), or maybe there’s some other way.  It won’t help start-up time <wink> but it could provide a better user experience.

The main case I'm interested in handling is systems that only have a
Python 3 stack and either don't have a Python 2 stack available at
all, or else the user doesn't actually want two parallel Python
installs (e.g. in a container image).

This already works OK-ish for "/usr/bin/env python" scripts - you can
just use venv or something like Software Collections to make "which
python" resolve to a Python 3 runtime, and you're done.

However, what I'm now considering is that instead of having the
default state be "/usr/bin/python is missing", I may be able to make
it "/usr/bin/python is present, but will be a Python script that tries
to compile the given script with the default Python runtime and
reports the results, rather than actually running it". That way:

- for scripts attempting to use Python 2 syntax on Python 3 only
systems, the script will still report the syntax error, but it will
*also* explain that Python 2.7 is not installed (and how to install
it)
- for scripts syntactically compatible with Python 3 on Python 3 only
systems, the script will be able to say "Either run this explicitly
with python3, or else reconfigure the system to do that implicitly"
(since there are still potential runtime compatibility problems that
mere compilation won't find, so just because it compiled, doesn't mean
it will work as intended)
- if both Python 2.7 and 3.x are installed, than the details of the
feedback would depend on how the respective compilations went

There would then be a distro supported way to switch that custom
script out for a plain symlink to the desired Python stack (in Fedora,
it's looking like we're going to have to use the new modularity
tooling in order to handle immutable Atomic Host images correctly, and
then work out how we want to handle the traditional monolithic repo
model based on that).

Another variant of that idea would be to have the default handler be
something like Geoffrey Thomas's pythonmux project
(https://github.com/geofft/pythonmux), and then patch
sys.excepthandler to report the Python version executed when
"sys.executable" isn't qualified as either Python 2 or Python 3 for
easier debugging of otherwise potentially cryptic error messages.

If we did do that, then I'd skip the "pyversions" comment idea, and
instead have the logic just be:

- if only /usr/bin/python2 is present, use it
- if only /usr/bin/python3 is present, use it
- if both are present, use /usr/bin/python2

As far I'm aware, that would match the logic currently used in the
Windows launcher, although PEP 397 is a bit ambiguous on that point.

Bringing up WIndows is also interesting in general, as for its shebang
line processing, Windows kinda follows PEP 394, where it supports all
of the following:

    python
    /usr/bin/python
    /usr/local/bin/python
    /usr/bin/env python

Plus their "python2" and "python3" equivalents and the specific
"pythonX.Y" versions.

In 3.6, the launcher changed such that just typing "py" will launch
the newest Python version it knows about, rather than defaulting to
Python 2 when both are present. However, the *shebang line* emulation
still defaults to Python 2 if it has the option, with a series of
environment variables and config files used to manage their
interpretation (that's mainly due to the challenges of using symlinks
on Windows, rather than for any other reason).

I'm thinking that if we update the guidance in PEP 394, it's also
likely to make sense to change the Windows launcher's shebang line
processing in 3.7+ to be such that it defaults to Python 3 even when
both 2 & 3 are present.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia