[Python-ideas] Add from __experimental__ import bla [was: Should we move to replace re with regex?]

[Nick Coghlan]

On Mon, Aug 29, 2011 at 8:05 AM, ron3200 <ron3200 at gmail.com> wrote:
> It looks to me, that Importing from __future__ can do things that a regular
> import can't do, like add or change core syntax.  So they tend to be python
> core features rather than library features.

If we want to, we can make __experimental__ just as magical as
__future__. That's orthogonal to its semantics in the library case,
which is the point of interest at the moment. If people fear relying
on experimental features, that's OK - if this idea happens at all,
allowing people that aren't prepared to cope with the risk of breakage
to easily avoid it would be a key benefit.

The essential idea here is to be able to add a feature, but flag the
API as potentially unstable for an entire release cycle before
promising to maintain the API in perpetuity. Early adopters would get
*most* of python-dev's usual guarantees (which most PyPI packages
don't offer), while more conservative types can use more stable
alternatives.

The benefit on our side is that we'd get an full 18-24 months of
feedback on a feature from a broad audience before finally locking the
API down in the subsequent release. The more I think about this idea,
the more I think it could help alleviate some of the concerns that can
currently hold up the evolution of the standard library. (The costs of
making a mistake with standard library API design are rather high due
to the backwards compatibility promise, which means iterative design
mostly needs to happen *before* the module is included. PyPI and the
PEP process are both useful for that, but they don't really compare to
the feedback available from being part of a full standard lib
release).

We'd still need to be careful not to throw any old rubbish in there,
but as a phased introduction for things like "from __experimental__
import ipaddr" (PEP 3144) and "from __experimental__ import re" (re
replacement with regex, assuming someone steps up to write the PEP and
eventual drop-in replacement looks feasible), the idea definitely has
potential.

> I describe them as "early" because they either aren't quite ready for the
> current release cycle or they are waiting for something else to be
> depreciated or done before they can included.

No, that's *not* what __future__ means. __future__ changes are exactly
what will become default behaviour in a future version - they aren't
going to change, so code that uses them won't need to change. What may
need changing in the case of __future__ is *old* code - the only
reason we use a __future__ flag is when old code might break.

The semantics of the new marker package would be to indicate that
stuff is a little undercooked, but we've decided that it won't get
enough exposure through PyPI (either through being too small to
overcome NIH syndrome or else too closely coupled to a specific
version of the interpreter). That's a completely different meaning, so
we shouldn't reuse the same word.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia