[Python-ideas] Add from __experimental__ import bla [was: Should we move to replace re with regex?]

[Nick Coghlan]

On Mon, Aug 29, 2011 at 11:29 AM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
>  > The net effect is that a Googler who happens to be the first
>  > to want to use a certain 3rd party package has to do a lot of work;
>  > it's *not* just a matter of downloading and installing the package. I
>  > imagine it's the same within many other organizations -- some better,
>  > some worse.
>
> But the friction imposed by this required is presumably considered a
> net good thing by Google, no?  Some benefit is received, or some cost
> avoided, right?  So does __experimental__ just reduce friction for
> carefully selected proposed modules, and so increase net benefit, or
> does reduced friction have adverse impacts (ie, side effects of too-
> easily imported modules on the rest of the organization) that might
> outweigh the reduction of friction, too?

I'd say if we went down this path, a key motivator would be
specifically to make it easier for downstream organisations to point
to it and say "this is covered by our existing license and packaging
review for the Python standard library". ActiveState/Enthought/etc
would ship it with their installers, Debian/Fedora/etc would package
it, and so on and so forth. That's a harder case to make for arbitrary
PyPI packages, since there's far more variation in the licensing and
the code isn't tested on our buildbot suite or covered by our source
control and review processes. "We vetted the licensing for valid PSF
redistributions rights, combined its test suite with ours, and
regularly run those tests on our buildbots" really *is* a
significantly stronger statement on our part than "we're considering
including this module, you can get it from PyPI".

I'm actually coming around to the idea, especially for cases like the
ones Guido mentioned where we think the functionality is definitely
desirable but aren't sure we have the API right yet. Currently, in
those cases we have to choose between publishing on PyPI and locking
in (potentially for a very long time) an API design that we're not
entirely sure is right. (Possible case in point: we ignored some of
Glyph's advice in the design of the concurrent.futures API by having
the executors directly manipulate the future objects instead of
introducing an additional level of indirection so that the object the
executors were playing with wasn't the one clients had a direct
reference to. Even if this turns out to have been a mistake, the API
is unlikely to change within the 3.x series).

An explicit "__experimental__" namespace would be a way to promise
"this functionality will exist in *some* form in future versions of
Python, but the API may change in backwards incompatible ways in the
process of getting there". It reminds me in some ways of the journey
of the set type from its initial addition as "sets.Set" to the final
version as the "set" builtin.

ImportEngine (PEP published to import-sig, I'll push it to python.org
soonish), for example, would be a prime candidate for experimental
status in 3.3 - it's too closely coupled to the interpreter details to
make sense to distribute separately, so it would be good to have a
mechanism to distribute it in a form that would allow feedback to be
gathered over a full release cycle before we commit to a final API in
3.4 (assuming the PEP is accepted at all, of course, although I'm
fairly confident I can persuade people it's a good idea).

Here's another potentially useful litmus test question for Guido: if
it was spelled "from __experimental__ import ipaddr", would you be
more inclined to approve PEP 3144 (IP address library) for 3.3?

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia