From cstratak at redhat.com  Tue Jul 25 08:02:04 2017
From: cstratak at redhat.com (Charalampos Stratakis)
Date: Tue, 25 Jul 2017 08:02:04 -0400 (EDT)
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on Unix-Like
 Systems
In-Reply-To: <718257630.37973746.1500984074624.JavaMail.zimbra@redhat.com>
Message-ID: <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>

Hello everyone,

tl;dr Let's make sure that PEP 394 says "/usr/bin/python should be Python 3 by early 2020"

As Python 2's end of life is approaching, various Linux distributions are taking measures to tackle the issue, raising awareness, porting packages
and tools to Python 3, having only Python 3 preinstalled on their base images, and some have even made the ultimate change: making /usr/bin/python
point to the Python 3 interpreter. Some references [0][1][2][3][4].

Various upstream Python projects have also started down to the path to completely dropping Python 2 support in order to start taking full advantage
of Python 3 only features: http://www.python3statement.org/

In Fedora, we'd like to change /usr/bin/python to point to Python 3 by 2020 (when PEP 404 kicks in). However, we'd rather not do it without upstream's
support in PEP 394. Hence we would like to drive the change of PEP 394 to either say that /usr/bin/python should point to Python 3, or to say it is up
to the distribution maintainers. (And we would prefer the first option, so that the situation may eventually become consistent.)

The message we want to send is: "everything that can be ported to Python 3 should be ported, and everything that can run on Python 3 should run on it."
However, having /usr/bin/python point to Python 2 is continuing the concrete mindset for Linux users that "the default" version of Python is still Python 2.
Those mixed signals are especially problematic with maintainers that aren't familiar with the Python ecosystem (e.g. they only use a Python-based buildsystem).

PEP 394 says that it "is anticipated that there will eventually come a time where the third party ecosystem surrounding Python 3 is sufficiently
mature for this recommendation to be updated to suggest that the python symlink refer to python3 rather than python2," and that its recommendation
will be periodically reviewed.

We believe the ecosystem is now mature enough, and so we should set a specific target timeline for distributions to make that switch (aligning it
with the end of community support for the Python 2.7 branch).

[0] https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3 
[1] https://lists.opensuse.org/opensuse-factory/2017-06/msg00740.html
[2] https://wiki.ubuntu.com/Python/3
[3] https://wiki.openmandriva.org/en/3.02/Release_Notes#Python


Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat


From ncoghlan at gmail.com  Wed Jul 26 00:29:53 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 26 Jul 2017 14:29:53 +1000
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on
 Unix-Like Systems
In-Reply-To: <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
References: <718257630.37973746.1500984074624.JavaMail.zimbra@redhat.com>
 <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
Message-ID: <CADiSq7eDytm=YFKC2kAWq9_7i6kmhbTPMPzgUOGKdc1LvwDepg@mail.gmail.com>

On 25 July 2017 at 22:02, Charalampos Stratakis <cstratak at redhat.com> wrote:
> In Fedora, we'd like to change /usr/bin/python to point to Python 3 by 2020 (when PEP 404 kicks in). However, we'd rather not do it without upstream's
> support in PEP 394. Hence we would like to drive the change of PEP 394 to either say that /usr/bin/python should point to Python 3, or to say it is up
> to the distribution maintainers. (And we would prefer the first option, so that the situation may eventually become consistent.)

Some additional context to this idea for non-Fedora-using folks: while
Python 2 and hence "/usr/bin/python" have been missing from our
default installations since the release of Fedora 23 in 2015, it's
still pretty easy to inadvertently get them back:

- an unqualified "dnf install python-<whatever>" will usually do it
- any package with "/usr/bin/python" in a shebang line implicitly
gains a dependency on "python2"

At the distribution level, our plan is to prevent the latter by way of
Fedora's packaging policy and associated automated tests (i.e. all
Python scripts in packages built for Fedora itself will be required to
qualify the required Python version), while the former will be handled
through RPM macros and virtual provides (essentially, there'll be a
cutover release, probably Fedora 32 in early 2020, where "dnf install
python-foo" starts to give you the Python 3 version of things instead
of the Python 2 version).

At the PEP 394 level, what I'd personally like to see us do is to
split the "Recommendations" section into two separate sections:
"Python User Recommendations" and "Platform Publisher Recommendations"

"Python User Recommendations" would cover three main points:

1. Recommend the use of virtual environments and Python level
packaging tools to decouple Python development activities from distro
choices of default Python runtimes
2. Recommend writing hybrid Python 2/3 code when specifically
targeting "/usr/bin/python" (including favouring either
'print(fmt.(*args)) and 'print(fmt % args)' or 'from __future__ import
print_function' over multi-argument print statements)
3. Recommend targeting "/usr/bin/python2" instead of "/usr/bin/python"
when publishing specifically Python 2 code

"Platform Publisher Recommendations" would be updated to say:

1. "/usr/bin/python" should point to either Python 2.7 or Python 3.5+
(to ensure that bytes objects support mod-formatting)
2. If it's updated to point to Python 3.x, then system administrators
switching it back to Python 2.7 *should* be a supported operation
(i.e. it shouldn't break the OS itself if they do that)
3. When a Python 2.7 stack is provided, make sure to also provide
"virtualenv" or some other mechanism for user level environment
management
4. Regardless of whether "/usr/bin/python" points to Python 2 or
Python 3, that default stack should provide the "six" and "future"
cross-compatibility libraries to make life easier for folks writing
unpackaged hybrid 2/3 code that can't readily indicate any third party
library dependencies

As far as the specifics of actually managing a cutover in
"/usr/bin/python" itself go, we think that's mainly a question to be
resolved between distro developers and their particular user
community.

For Fedora for example, we think that dropping "/usr/bin/python" from
the repositories entirely would be too large of a compatibility break,
whereas we're comfortable with the idea that by 2020, it will be
reasonable to require that anything still running in "/usr/bin/python"
be hybrid Python 2/3 code, such that finally changing the default
won't cause too much immediate breakage (and for things that do still
break, the recommended fix will be code modernisation to get into the
hybrid subset prior to upgrading the OS).

There's unfortunately no completely non-disruptive way to manage this
shift (hence why it's taking so long), but we think this is a
reasonable approach that allows each distro to devise a migration plan
that makes sense for them and their userbase while still allow Python
end users to write readable cross-distro code that doesn't particular
care whether it's run under Python 2 or Python 3, and for open source
Python project maintainers to provide developer guidelines that are
entirely independent of particular distro's choices about default
Python runtimes.

Before taking this proposal to python-dev, I'd turn the general
concept into an actual PR with specific proposed wording changes, but
I figured it made sense for us to seek some initial feedback here
before doing that.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rosuav at gmail.com  Wed Jul 26 01:53:16 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 26 Jul 2017 15:53:16 +1000
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on
 Unix-Like Systems
In-Reply-To: <CADiSq7eDytm=YFKC2kAWq9_7i6kmhbTPMPzgUOGKdc1LvwDepg@mail.gmail.com>
References: <718257630.37973746.1500984074624.JavaMail.zimbra@redhat.com>
 <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
 <CADiSq7eDytm=YFKC2kAWq9_7i6kmhbTPMPzgUOGKdc1LvwDepg@mail.gmail.com>
Message-ID: <CAPTjJmrvsrnXar_hcksMU4ZKBN3UE1zwJ2yXKV-iMn7vNScssQ@mail.gmail.com>

On Wed, Jul 26, 2017 at 2:29 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> There's unfortunately no completely non-disruptive way to manage this
> shift (hence why it's taking so long), but we think this is a
> reasonable approach that allows each distro to devise a migration plan
> that makes sense for them and their userbase while still allow Python
> end users to write readable cross-distro code that doesn't particular
> care whether it's run under Python 2 or Python 3, and for open source
> Python project maintainers to provide developer guidelines that are
> entirely independent of particular distro's choices about default
> Python runtimes.
>
> Before taking this proposal to python-dev, I'd turn the general
> concept into an actual PR with specific proposed wording changes, but
> I figured it made sense for us to seek some initial feedback here
> before doing that.

Sounds good to me. (I'm not subbed to linux-sig, though perhaps I
should be.) Did I get copied in on this for assistance with the PEP
writing? In any case, I'm wholly in favour of the transition.

ChrisA

From ncoghlan at gmail.com  Mon Jul 31 06:44:27 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 31 Jul 2017 20:44:27 +1000
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on
 Unix-Like Systems
In-Reply-To: <CAPTjJmrvsrnXar_hcksMU4ZKBN3UE1zwJ2yXKV-iMn7vNScssQ@mail.gmail.com>
References: <718257630.37973746.1500984074624.JavaMail.zimbra@redhat.com>
 <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
 <CADiSq7eDytm=YFKC2kAWq9_7i6kmhbTPMPzgUOGKdc1LvwDepg@mail.gmail.com>
 <CAPTjJmrvsrnXar_hcksMU4ZKBN3UE1zwJ2yXKV-iMn7vNScssQ@mail.gmail.com>
Message-ID: <CADiSq7fcCcvT5qKidiZLCix8iUAzG2ztHkbtDeiZyNg+1+Lj-A@mail.gmail.com>

On 26 July 2017 at 15:53, Chris Angelico <rosuav at gmail.com> wrote:
> On Wed, Jul 26, 2017 at 2:29 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> There's unfortunately no completely non-disruptive way to manage this
>> shift (hence why it's taking so long), but we think this is a
>> reasonable approach that allows each distro to devise a migration plan
>> that makes sense for them and their userbase while still allow Python
>> end users to write readable cross-distro code that doesn't particular
>> care whether it's run under Python 2 or Python 3, and for open source
>> Python project maintainers to provide developer guidelines that are
>> entirely independent of particular distro's choices about default
>> Python runtimes.
>>
>> Before taking this proposal to python-dev, I'd turn the general
>> concept into an actual PR with specific proposed wording changes, but
>> I figured it made sense for us to seek some initial feedback here
>> before doing that.

Draft updated PEP is posted at https://github.com/python/peps/pull/315

I ended up proposing a replacement PEP to supersede 394, rather than
proposing changes to 394 itself.

It's essentially a PEP 394 superset that allows for a few more
"endorsed" configurations for redistributors (essentially saying
"Arch's move was ~5 years ahead of its time"), with more of an
emphasis on "What might a post-2020 sans-Python-2 platform release
look like?"

Structurally, it clearly separates the recommendations into 3 distinct
sets (ad hoc scripting, app development, and platform publication)

> Sounds good to me. (I'm not subbed to linux-sig, though perhaps I
> should be.) Did I get copied in on this for assistance with the PEP
> writing? In any case, I'm wholly in favour of the transition.

I'm not sure how you ended up receiving it, unless there's something
odd going on with the configuration of linux-sig-owner in the mail
server.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From barry at python.org  Mon Jul 31 10:25:11 2017
From: barry at python.org (Barry Warsaw)
Date: Mon, 31 Jul 2017 10:25:11 -0400
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on
 Unix-Like Systems
In-Reply-To: <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
References: <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
Message-ID: <FE99FCCE-B100-4597-894F-5652058863CC@python.org>

Thanks for getting this discussion going again.  The more I talk to people about the upcoming EOL for Python 2.7, the more I realize that while 2.5 years *seems* like a long way away, it?ll be here sooner that many are planning for.  ?Winter is coming? is a phrase that pops into my mind (although maybe it should be Summer, as I think Python 3 invokes a lot more optimism than the White Walkers. :)

A couple of top-line points first:

* As I mentioned to Nick off-list, I think we should amend PEP 394 rather than supersede it.  It?s widely cited as *the* upstream policy on the matter, so I think we?ll get more bang for the buck, and less overall confusion, by updating it rather than replacing it.

* My personal preferences have been evolving on this subject too, so I won?t claim to represent the Debian ecosystem here.  That?s a discussion to be had over in that forum.  For now, take all of the following as my personal opinion.

> On Jul 25, 2017, at 08:02, Charalampos Stratakis <cstratak at redhat.com> wrote:
> 
> tl;dr Let's make sure that PEP 394 says "/usr/bin/python should be Python 3 by early 2020?

I think that?s just a bit too early.  Consider the timeline.  Guido has proclaimed[*] that Python 2.7 will EOL at Pycon 2020, which will likely be in the May-June timeframe.  I think any changes we make should either be timed to coincide with that event, or happen in the latter part of 2020.  It?s also very likely that Python 2.7 support won?t abruptly end on that date, but instead will continue with some level of security-only, source-only releases for a while after.  For the moment though, let?s not quibble on the timing.

[*] At the Language Summit 2017; sorry no reference handy atm.

> Various upstream Python projects have also started down to the path to completely dropping Python 2 support in order to start taking full advantage
> of Python 3 only features: http://www.python3statement.org/

Oh nice!  I wasn?t aware of this site, and will likely sign up to it soon.  I?ve already mostly dropped Python 2 support for most of my personal projects, and of course GNU Mailman 3 Core has been Python 3-only for a while. (Some of the other components are not quite there yet, but we?re committed to it.  Mailman 2 will likely never get ported.  I suspect that fate will be shared by many projects, alas.)

> In Fedora, we'd like to change /usr/bin/python to point to Python 3 by 2020 (when PEP 404 kicks in). However, we'd rather not do it without upstream's
> support in PEP 394. Hence we would like to drive the change of PEP 394 to either say that /usr/bin/python should point to Python 3, or to say it is up
> to the distribution maintainers. (And we would prefer the first option, so that the situation may eventually become consistent.)

Consistency across the Python ecosystem should be a top concern.  The Python 3 transition has been a tumultuous one, and we?ve had successes and challenges.  Overall, I think we?re in a good place right now, so let?s make sure we handle the end of the transition as well as possible!  People may argue about whether such-and-such a decision is right or wrong, but uncertainty is much worse.

> The message we want to send is: "everything that can be ported to Python 3 should be ported, and everything that can run on Python 3 should run on it.?

+1

> However, having /usr/bin/python point to Python 2 is continuing the concrete mindset for Linux users that "the default" version of Python is still Python 2.
> Those mixed signals are especially problematic with maintainers that aren't familiar with the Python ecosystem (e.g. they only use a Python-based buildsystem).

IMHO, the reason why this is so is because `python` (at the prompt, a.k.a. typically /usr/bin/python) is the CLI API for *users*.  It?s what they?re taught to type at the command line to run their scripts or to start the interpreter to play with things.

PEP 394 already recommends `python2` and `python3` in the shebang line for distro-provided executables.  I think we turn some SHOULDs into MUSTs so that any PEP-394 compliant *nix distro will be required to follow the same rules.  Further, I?d add that PEP 394-compliant distros MUST NOT put /usr/bin/python in shebangs.

(PEP 394 does not talk about `/usr/bin/env python` which is often used by developers, although highly discouraged for distro-provided scripts, at least in Debian.  If we want to add a note about this, I think we make it clear that this usage relies solely on the user?s own $PATH environment, and therefore lies outside the scope of the PEP.)

I?d like to discourage the use of naming schemes like `idle2`, `pydoc2`, etc.  That?s a problematic approach, for which -m invocation is much more predictable, albeit less convenient.  We still don?t have a good solution for that, although many have been discussed.  That?s probably outside the scope of PEP 394 too.

Lastly, we come to what /usr/bin/python points to, and that?s always been the sticking point.  There are two use cases.

* When people type that at the command line, which version of Python do they get?
* When people use that in their own third-party (i.e. not distro-provided) scripts, what version of Python do they get?

So, it?s a UI/UX problem now :)

I would personally like to say /usr/bin/python will always point to python3 after such-and-such a date, although as I?ve said above, my timing would put that to later in 2020.  However, I think we need to allow distributions to make their own decisions on that timing.  Some may choose to never make it, others may be proactive.

There are two ways to go here.  Either we can make it a MUST to point /usr/bin/python to python3, and just label any distro that decides otherwise as ?non-PEP-394-compliant?, or we make the choice more lax (possibly SHOULD, possibly MAY) and allow leeway for otherwise in-compliance distros.  I suspect there will be some in Debian that will argue for the latter, and they will make good points.  OTOH, I think we?re well past the tipping point, and will face the cliff pretty darn soon.  I haven?t completely made my own mind up on this point.  In the meantime, I think we can make a lot of progress on PEP 394 as I?ve outlined above.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/linux-sig/attachments/20170731/e8c3f19f/attachment.sig>

From guido at python.org  Mon Jul 31 10:19:45 2017
From: guido at python.org (Guido van Rossum)
Date: Mon, 31 Jul 2017 07:19:45 -0700
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on
 Unix-Like Systems
In-Reply-To: <CADiSq7fcCcvT5qKidiZLCix8iUAzG2ztHkbtDeiZyNg+1+Lj-A@mail.gmail.com>
References: <718257630.37973746.1500984074624.JavaMail.zimbra@redhat.com>
 <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
 <CADiSq7eDytm=YFKC2kAWq9_7i6kmhbTPMPzgUOGKdc1LvwDepg@mail.gmail.com>
 <CAPTjJmrvsrnXar_hcksMU4ZKBN3UE1zwJ2yXKV-iMn7vNScssQ@mail.gmail.com>
 <CADiSq7fcCcvT5qKidiZLCix8iUAzG2ztHkbtDeiZyNg+1+Lj-A@mail.gmail.com>
Message-ID: <CAP7+vJKbJn+T6xE8X6cdoAjwsBgf5qX4Z8ZjMjs4+7hpfhv7ZA@mail.gmail.com>

I hope I don't have to read that (I am behind on so many things to review
already). It's just an informational PEP, so you can just write whatever
you want without seeking approval. Right?

On Mon, Jul 31, 2017 at 3:44 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 26 July 2017 at 15:53, Chris Angelico <rosuav at gmail.com> wrote:
> > On Wed, Jul 26, 2017 at 2:29 PM, Nick Coghlan <ncoghlan at gmail.com>
> wrote:
> >> There's unfortunately no completely non-disruptive way to manage this
> >> shift (hence why it's taking so long), but we think this is a
> >> reasonable approach that allows each distro to devise a migration plan
> >> that makes sense for them and their userbase while still allow Python
> >> end users to write readable cross-distro code that doesn't particular
> >> care whether it's run under Python 2 or Python 3, and for open source
> >> Python project maintainers to provide developer guidelines that are
> >> entirely independent of particular distro's choices about default
> >> Python runtimes.
> >>
> >> Before taking this proposal to python-dev, I'd turn the general
> >> concept into an actual PR with specific proposed wording changes, but
> >> I figured it made sense for us to seek some initial feedback here
> >> before doing that.
>
> Draft updated PEP is posted at https://github.com/python/peps/pull/315
>
> I ended up proposing a replacement PEP to supersede 394, rather than
> proposing changes to 394 itself.
>
> It's essentially a PEP 394 superset that allows for a few more
> "endorsed" configurations for redistributors (essentially saying
> "Arch's move was ~5 years ahead of its time"), with more of an
> emphasis on "What might a post-2020 sans-Python-2 platform release
> look like?"
>
> Structurally, it clearly separates the recommendations into 3 distinct
> sets (ad hoc scripting, app development, and platform publication)
>
> > Sounds good to me. (I'm not subbed to linux-sig, though perhaps I
> > should be.) Did I get copied in on this for assistance with the PEP
> > writing? In any case, I'm wholly in favour of the transition.
>
> I'm not sure how you ended up receiving it, unless there's something
> odd going on with the configuration of linux-sig-owner in the mail
> server.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Linux-sig mailing list
> Linux-sig at python.org
> https://mail.python.org/mailman/listinfo/linux-sig
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/linux-sig/attachments/20170731/c51b5c5b/attachment.html>

From encukou at gmail.com  Mon Jul 31 12:34:57 2017
From: encukou at gmail.com (Petr Viktorin)
Date: Mon, 31 Jul 2017 18:34:57 +0200
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on
 Unix-Like Systems
In-Reply-To: <FE99FCCE-B100-4597-894F-5652058863CC@python.org>
References: <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
 <FE99FCCE-B100-4597-894F-5652058863CC@python.org>
Message-ID: <c82d7e83-9c6d-a1b4-3480-c495f72e833b@gmail.com>

On 07/31/2017 04:25 PM, Barry Warsaw wrote:
[...]
>> The message we want to send is: "everything that can be ported to Python 3 should be ported, and everything that can run on Python 3 should run on it.?
> 
> +1
> 
>> However, having /usr/bin/python point to Python 2 is continuing the concrete mindset for Linux users that "the default" version of Python is still Python 2.
>> Those mixed signals are especially problematic with maintainers that aren't familiar with the Python ecosystem (e.g. they only use a Python-based buildsystem).
> 
> IMHO, the reason why this is so is because `python` (at the prompt, a.k.a. typically /usr/bin/python) is the CLI API for *users*.  It?s what they?re taught to type at the command line to run their scripts or to start the interpreter to play with things.
> 
> PEP 394 already recommends `python2` and `python3` in the shebang line for distro-provided executables.  I think we turn some SHOULDs into MUSTs so that any PEP-394 compliant *nix distro will be required to follow the same rules.  Further, I?d add that PEP 394-compliant distros MUST NOT put /usr/bin/python in shebangs.

+1
Ah, yes, this is missing from the "PEP 548" pull request.

> (PEP 394 does not talk about `/usr/bin/env python` which is often used by developers, although highly discouraged for distro-provided scripts, at least in Debian.  If we want to add a note about this, I think we make it clear that this usage relies solely on the user?s own $PATH environment, and therefore lies outside the scope of the PEP.)

+1

> I?d like to discourage the use of naming schemes like `idle2`, `pydoc2`, etc.  That?s a problematic approach, for which -m invocation is much more predictable, albeit less convenient.  We still don?t have a good solution for that, although many have been discussed.  That?s probably outside the scope of PEP 394 too.

I think that ship has sailed. The unversioned names should switch along 
with /usr/bin/python, and the versioned ones should be allowed to die 
along with /usr/bin/python[23]. Improving the -m situation is out of scope.

> Lastly, we come to what /usr/bin/python points to, and that?s always been the sticking point.  There are two use cases.
> 
> * When people type that at the command line, which version of Python do they get?
> * When people use that in their own third-party (i.e. not distro-provided) scripts, what version of Python do they get?
> 
> So, it?s a UI/UX problem now :)
> > I would personally like to say /usr/bin/python will always point to 
python3 after such-and-such a date, although as I?ve said above, my 
timing would put that to later in 2020.  However, I think we need to 
allow distributions to make their own decisions on that timing.  Some 
may choose to never make it, others may be proactive.
> 
> There are two ways to go here.  Either we can make it a MUST to point /usr/bin/python to python3, and just label any distro that decides otherwise as ?non-PEP-394-compliant?, or we make the choice more lax (possibly SHOULD, possibly MAY) and allow leeway for otherwise in-compliance distros.  I suspect there will be some in Debian that will argue for the latter, and they will make good points.  OTOH, I think we?re well past the tipping point, and will face the cliff pretty darn soon.  I haven?t completely made my own mind up on this point.  In the meantime, I think we can make a lot of progress on PEP 394 as I?ve outlined above.

The other recommendations proposed here are designed so they can be 
implemented right now, with no bad side effects, but switching 
/usr/bin/python will break things. I think here it's good to leave the 
timing up to distros, and to suggest that the distros allow users to 
override their decisions.


From barry at python.org  Mon Jul 31 13:47:28 2017
From: barry at python.org (Barry Warsaw)
Date: Mon, 31 Jul 2017 13:47:28 -0400
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on
 Unix-Like Systems
In-Reply-To: <CADiSq7eDytm=YFKC2kAWq9_7i6kmhbTPMPzgUOGKdc1LvwDepg@mail.gmail.com>
References: <718257630.37973746.1500984074624.JavaMail.zimbra@redhat.com>
 <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
 <CADiSq7eDytm=YFKC2kAWq9_7i6kmhbTPMPzgUOGKdc1LvwDepg@mail.gmail.com>
Message-ID: <B2304DB9-6645-4D5F-829B-FF8BD03F12B4@python.org>

On Jul 26, 2017, at 00:29, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> At the PEP 394 level, what I'd personally like to see us do is to
> split the "Recommendations" section into two separate sections:
> "Python User Recommendations" and "Platform Publisher Recommendations?

I?d like to keep PEP 394 as short and proscriptive as possible.  I?ve always read PEP 394 as being aimed at ?platform publisher recommendations?, i.e. distributions who provide.  I think a user guide would make a good companion (i.e. separate) PEP.

> For Fedora for example, we think that dropping "/usr/bin/python" from
> the repositories entirely would be too large of a compatibility break,
> whereas we're comfortable with the idea that by 2020, it will be
> reasonable to require that anything still running in "/usr/bin/python"
> be hybrid Python 2/3 code, such that finally changing the default
> won't cause too much immediate breakage (and for things that do still
> break, the recommended fix will be code modernisation to get into the
> hybrid subset prior to upgrading the OS).

I like recommending /usr/bin/python be bilingual, but I?m wondering how you?ll enforce this, and what the implication is for running that at the command line.  Again, that gets to the UI/UX issue.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/linux-sig/attachments/20170731/fe80069f/attachment-0001.sig>

From barry at python.org  Mon Jul 31 13:54:42 2017
From: barry at python.org (Barry Warsaw)
Date: Mon, 31 Jul 2017 13:54:42 -0400
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on
 Unix-Like Systems
In-Reply-To: <c82d7e83-9c6d-a1b4-3480-c495f72e833b@gmail.com>
References: <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
 <FE99FCCE-B100-4597-894F-5652058863CC@python.org>
 <c82d7e83-9c6d-a1b4-3480-c495f72e833b@gmail.com>
Message-ID: <9728B708-D0A1-4E33-B196-BB1B80173C82@python.org>

On Jul 31, 2017, at 12:34, Petr Viktorin <encukou at gmail.com> wrote:
> 
>> I?d like to discourage the use of naming schemes like `idle2`, `pydoc2`, etc.  That?s a problematic approach, for which -m invocation is much more predictable, albeit less convenient.  We still don?t have a good solution for that, although many have been discussed.  That?s probably outside the scope of PEP 394 too.
> 
> I think that ship has sailed. The unversioned names should switch along with /usr/bin/python, and the versioned ones should be allowed to die along with /usr/bin/python[23]. Improving the -m situation is out of scope.

Definitely, improving -m is out of scope, but I?m not so sure about the location of that ship.  Maybe that should be a distro-specific decision as well.  E.g. some of Debian?s unversioned tools are already shebanged for Python 3, but that also causes problems sometimes.  OTOH, it should only be a relatively small number of scripts that need to make the distinction in their /usr/bin name.

> The other recommendations proposed here are designed so they can be implemented right now, with no bad side effects, but switching /usr/bin/python will break things. I think here it's good to leave the timing up to distros, and to suggest that the distros allow users to override their decisions.

Again, I?d like the recommendations to use MUST, SHOULD, MAY language so readers and implementers of the PEP will be clear in what?s expected of them and their distros.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/linux-sig/attachments/20170731/9c9188eb/attachment.sig>

From ncoghlan at gmail.com  Mon Jul 31 19:40:19 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 1 Aug 2017 09:40:19 +1000
Subject: [Linux-SIG] Revisit of PEP 394 -- The "python" Command on
 Unix-Like Systems
In-Reply-To: <B2304DB9-6645-4D5F-829B-FF8BD03F12B4@python.org>
References: <718257630.37973746.1500984074624.JavaMail.zimbra@redhat.com>
 <1481944193.37973848.1500984124214.JavaMail.zimbra@redhat.com>
 <CADiSq7eDytm=YFKC2kAWq9_7i6kmhbTPMPzgUOGKdc1LvwDepg@mail.gmail.com>
 <B2304DB9-6645-4D5F-829B-FF8BD03F12B4@python.org>
Message-ID: <CADiSq7fc6YJWKrhMS5zD9gf4_WbxGw0vRNDS2WjPWYn7W0=kfQ@mail.gmail.com>

On 1 August 2017 at 03:47, Barry Warsaw <barry at python.org> wrote:
> On Jul 26, 2017, at 00:29, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> At the PEP 394 level, what I'd personally like to see us do is to
>> split the "Recommendations" section into two separate sections:
>> "Python User Recommendations" and "Platform Publisher Recommendations?
>
> I?d like to keep PEP 394 as short and proscriptive as possible.  I?ve always read PEP 394 as being aimed at ?platform publisher recommendations?, i.e. distributions who provide.  I think a user guide would make a good companion (i.e. separate) PEP.

Until yesterday I would have said the same thing regarding the current
contents of PEP 394, but a surprisingly large number of the points in
the new user recommendations section actually came from the current
text of PEP 394 - they're down at the end of the recommendations
section, and when they were all in one list, it was easy to miss that
we'd crossed over from "Advice to platform publishers" into "Advice
for platform users".

>> For Fedora for example, we think that dropping "/usr/bin/python" from
>> the repositories entirely would be too large of a compatibility break,
>> whereas we're comfortable with the idea that by 2020, it will be
>> reasonable to require that anything still running in "/usr/bin/python"
>> be hybrid Python 2/3 code, such that finally changing the default
>> won't cause too much immediate breakage (and for things that do still
>> break, the recommended fix will be code modernisation to get into the
>> hybrid subset prior to upgrading the OS).
>
> I like recommending /usr/bin/python be bilingual, but I?m wondering how you?ll enforce this, and what the implication is for running that at the command line.  Again, that gets to the UI/UX issue.

We're still working on the details (see
https://fedoraproject.org/wiki/User:Ncoghlan/Default_python_module for
my current notes), but what I'm currently leaning towards advocating
at the Fedora level is:

- we declare "/usr/bin/env python" and other environment aware
invocations as already being effectively addressed, thanks to venv,
environment modules, conda, and other existing PATH based solutions
that change the interactive meaning of "python"
- by some suitable mechanism, we offer 3 different configurations for
the absolute "/usr/bin/python" path: symlink to /usr/bin/python2,
symlink to /usr/bin/python3, and "Not configured yet, here's how to
select your default interpreter" (as a matter of UX, I'm beginning to
think we'll need to make the last one an actual installable script in
its own right, as the default interpreter not found error from the
shebang handler isn't particularly informative)

In that light, it may make sense to more explicitly focus the PEP 394
recommendations on *absolute* invocations that *don't* rely on PATH.
Right now it still talks solely about the environment dependent
"python", "python2" and "python3" commands, when the actual unsolved
problem is specifically the handling of the absolute "/usr/bin/python"
path:

- on non-Linux Unix-like systems, that path may not be present at all
(see https://rubenerd.com/python-path-for-ansible/ )
- even on Linux systems, using the absolute path bypasses (by design)
any PATH changes in the current environment

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia