[Numpy-discussion] Long term plans for dropping Python 2.7

[Ralf Gommers]

On Sat, Apr 15, 2017 at 7:02 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Fri, Apr 14, 2017 at 10:47 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> > On Sat, Apr 15, 2017 at 5:19 PM, Nathaniel Smith <njs at pobox.com> wrote:
> [...]
> >> From numpy's perspective, I feel like the most important reason to
> >> continue supporting 2.7 is our ability to convince people to keep
> >> upgrading. (Not the only reason, but the most important.) What I mean
> >> is: if we dropped 2.7 support tomorrow then it wouldn't actually make
> >> numpy unavailable on python 2.7; it would just mean that lots of users
> >> stayed at 1.12 indefinitely. Which is awkward, but it wouldn't be the
> >> end of the world – numpy is mature software and 1.12 works pretty
> >> well. The big problem IMO would be if this then meant that lots of
> >> downstream projects felt that they had to continue supporting 1.12
> >> going forward, which makes it very difficult for us to effectively
> >> ship new features or even bug fixes – I mean, we can ship them, but
> >> no-one will use them. And if a downstream project finds a bug in numpy
> >> and can't upgrade numpy, then the tendency is to work around it
> >> instead of reporting it upstream. I think this is the main thing we
> >> want to avoid.
> >
> >
> > +1
> >
> >>
> >>
> >> This kind of means that we're at the mercy of downstream projects,
> >> though – if scipy/pandas/etc. decide they want to support 2.7 until
> >> 2022, it might be in our best interest to do the same. But there's a
> >> collective action problem here: we want to keep supporting 2.7 so long
> >> as they do, but at the same time they may feel they need to keep
> >> supporting 2.7 as long as we do. And all of us would prefer to drop
> >> 2.7 support sooner rather than later, but we might all get stuck
> >>
> >> because we're waiting for someone else to move first.
> >
> >
> > I don't quite agree about being stuck. These kind of upgrades should and
> > usually do go top of stack to bottom. Something like Jupyter which is
> mostly
> > an end user tool goes first (they announced 2020 quite a while ago),
> domain
> > specific packages go at a similar time, then scipy & co, and only after
> that
> > numpy. Cython will be even later I'm sure - it still supports Python 2.6.
>
> To make sure we're on the same page about what "2020" means here: the
> latest release of IPython is 5.0, which came out in July last year.
> This is the last release that supports py2; they dropped support for
> py2 in master months ago, and 6.0 (whose schedule has been slipping,
> but I think should be out Any Time Now?) won't support py2. Their plan
> is to keep backporting bug fixes to 5.x until the end of 2017; after
> that the core team won't support py2 at all. And they've also
> announced that if volunteers want to step up to maintain 5.x after
> that, then they're willing to keep accepting pull requests until July
> 2019.
>
> Refs:
>   https://blog.jupyter.org/2016/07/08/ipython-5-0-released/
>   https://github.com/jupyter/roadmap/blob/master/accepted/migr
> ation-to-python-3-only.md
>
> I suspect that in practice that "end of 2017" date will the
> end-of-support date for most intents and purposes. And for numpy with
> its vaguely defined support periods, I think it makes most sense to
> talk in terms of release dates;


agreed, release dates makes sense. we don't want to be doing some kind of
LTS scheme.


> so if we want to compare
> apples-to-apples, my suggestion is that numpy drops py2 support in
> 2020 and in that sense ipython dropped py2 support in july last year.
>
> >>
> >> So my suggestion would be that numpy make some official announcement
> >> that our plan is to drop support for python 2 immediately after
> >> cpython upstream does.
> >
> >
> > Not quite sure CPython schedule is relevant - important bug fixes haven't
> > been making it into 2.7 for a very long time now, so the only change is
> the
> > rare security patch.
>
> Huh? 2.7 gets tons of changes: https://github.com/python/cpyt
> hon/commits/2.7


You're right. My experience is ending up on bugs.python.org when debugging
and the answer to "can this be backported to 2.7" usually being no - but it
looks like my experience is skewed by distutils, which is not exactly well
maintained.


> Officially CPython has 2 modes for releases: "regular support" and
> "security fixes only". 2.7 is special – it get regular support, and
> then on top of that it also has a special exception to allow certain
> kinds of major changes, like the ssl module backports.


> If you know of important bug fixes that they're missing then I think
> they'd like to know :-).


> Anyway, the reason the CPython schedule is relevant is that once they
> drop support, it *will* stop getting security patches, so it will
> become increasingly impossible to use safely.
>

For web stuff yes, but not all that relevant for scientific work.


>
> >>
> >> If worst comes to worst we can always decide to
> >> extend it at the time... but if we make the announcement now, then
> >> it's less likely that we'll need to :-).
> >
> >
> > I'd be in favor of putting out a schedule in coordination with
> > scipy/pandas/etc, but it probably should look more like
> > - 2020: what's on http://www.python3statement.org/ now
> > - 2021: scipy / pandas / scikit-learn / etc.
>
> Um... pandas is already on python3statement.org right now :-)
>
> > - 2022: numpy
>
> Honestly I don't see why we should plan to support python 2 a day
> longer than our major downstream dependencies. That was the point of
> my first paragraph: for us the main benefit to supporting 2 is to
> avoid forcing our downstream dependencies to pin an old numpy. What's
> that extra year get us if they've already moved on?
>
> The other odd thing about this schedule is that you're suggesting that
> the organizing principle should be that the stack switches from
> top-of-stack to bottom... but then you left out the bottom of the
> stack! :-)
>

I don't think of Python as part of the stack, because it's not upgradeable
for most users (except for with conda). It's more like having a base
platform (OS + compilers + Python version) on which you install a
scientific stack which has numpy as its lowest level component.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20170416/00674a08/attachment.html>