[Python-Dev] PEP 408 -- Standard library __preview__ package

[Nick Coghlan]

On Sat, Jan 28, 2012 at 3:22 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Executive summary:
>
> If the promise to remove the module from __preview__ is credible (ie,
> strictly kept), then __preview__ will have a specific audience in
> those who want the stdlib candidate code and are willing to deal with
> a certain amount of instability in that code.

People need to remember there's another half to this equation: the
core dev side.

The reason *regex* specifically isn't in the stdlib already is largely
due to (perhaps excessive) concerns about the potential maintenance
burden. It's not a small chunk of code and we don't want to deal with
another bsddb.

That's the main roadblock to inclusion. Not lack of user demand. Not
blindness to the problems with re. Just concerns about
maintainability. Add to that some niggling concerns about backwards
compatibility in obscure corner cases that may not be exercised by
current users. And so we have an impasse. Matthew has indicated he's
happy to include it and maintain it as part of the core, but it hasn't
really gone anywhere because we don't currently have a good way to
address those maintainability concerns (aside from saying "you're
worrying about it too much", which isn't what I would call
persuasive).

That's what __preview__ gives us: a way to deal with the *negative*
votes that keep positive additions out of the standard library. Most
of the PEP's arguments for due diligence etc are actually talking
about why we want things in the standard library in the first place,
rather than about __preview__ in particular.

The core idea behind the __preview__ namespace is to allow *3*
possible responses when a module is proposed for stdlib inclusion:

1. Yes, that's a good idea, we'll add it (cf. lzma for 3.3)
2. Maybe, so we'll add it to __preview__ for a release and see if it
blows up in our face (hopefully at least regex for 3.3, maybe ipaddr
and daemon as well)
3. No, not going to happen.

Currently, anything where we would answer "2" ends up becoming a "3"
by default, and that's not a good thing for the long-term health of
the language.

The reason this will be more effective in building core developer
confidence than third party distribution via PyPI is due to a few
different things:
- we all run the test suite, so we get to see that the software builds
and tests effectively
- we know what our own buildbots cover, so we know it's passing on all
those platforms
- we'll get to see more of the related discussions in channels we
monitor *anyway* (i.e. the bug tracker, python-dev)

As far as the criteria for failing to graduate goes, I'd say something
that ends up in __preview__ will almost always make it into the main
part of the standard library, with the following exceptions:
- excessive build process, test suite and buildbot instability.
Whether this is due to fragile test cases or fragile code, we don't
want to deal with another bsddb. If the test suite can't be stabilised
over the course of an entire feature release, then the module would
most likely be rejected rather than allowing it to graduate to the
standard library.
- strongly negative (or just plain confused) user feedback. We deal
with feedback on APIs all the time. Sometimes we add new ones, or
tweak the existing ones. Occasionally we'll judge them to be
irredeemably broken and just plain remove them (cf. CObject,
contextlib.nested, Bastion, rexec). This wouldn't change just because
a module was in __preview__ - instead, we'd just have another option
available to us (i.e. rejecting the module for stdlib inclusion
post-preview rather than trying to fix it).

Really, the main benefit for end users doesn't lie in __preview__
itself: it lies in the positive effect __preview__ will have on the
long term evolution of the standard library, as it aims to turn
python-dev's inherent conservatism (which is a good thing!) into a
speed bump rather than a road block.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia