[Python-Dev] re performance

[Nick Coghlan]

On 28 January 2017 at 18:07, Barry Warsaw <barry at python.org> wrote:
> On Jan 28, 2017, at 03:43 PM, Nick Coghlan wrote:
>
>>I still think it could be a good candidate for a first "bundled"
>>module, where we don't migrate it fully into the CPython development
>>process, but *do* officially bless it and provide it by default in the
>>form of a bundled wheel file (similar to what we do with pip).
>
> How would that work exactly.  I.e. is there a PEP?

There's no PEP, and I don't think it would make sense to have one
without a specific concrete example to drive the discussion.

I think there are 3 main candidates that could fit that bill:

- requests
- setuptools
- regex

Using requests to drive it would bring in a lot of incidental
complexity around SSL/TLS certificate management and integrating with
platform trust stores, which I think would overshadow the core policy
discussion of switching from a pure co-development model for the
standard library to one where some components are independently
versioned with recent snapshots being included in CPython feature and
maintenance branches.

For setuptools, one of our main goals in PyPA is to make it just one
build system option amongst many, as well as ensuring that it's easy
to do service deployments that don't have any build system present at
all, so bundling it by default with the stdlib would run counter to
those goals

regex by contrast is a relatively simple standalone module that offers
a more actively developed alternative to the standard library's re
module, but would still require a PEP to address the following
technical and policy questions that are normally handled by copying
code wholesale into the CPython tree under a contributor license
agreement:

- what licensing and other IP assurances would the PSF need to be
comfortable redistributing the bundled component?
- what maintenance sustainability questions would need to be
addressed? (e.g. succession planning for the ability to publish new
releases)
- how would bundled modules be handled in the CPython test suite?
- how would bundled modules be handled during local development,
during installation on supported OSes, and when creating virtual
environments?
- how does platform support in bundled modules relate to the platform
support guidelines given in PEP 11?
- how are additional build requirements for bundled modules (e.g. C++
compilers) handled?
- would bundled modules be permitted to gain features in CPython
maintenance releases, or would we expect any bundled modules to offer
conservative maintenance branches for the life of a given CPython
feature release?
- how would maintenance collaboration and issue escalation work? Would
bundled module maintainers need to follow both their own issue tracker
and the CPython one?

In many respects, PEP 453 and the bundling of pip already offers
precedent for how these are handled (e.g. it's collectively maintained
by PyPA, and that group is as closely associated with the PSF from a
legal and support perspective as python-dev is), so any PEP along
these lines would be about deciding which parts of what we did for pip
were a technical and policy precedent for 3rd party module bundling in
general, and which are unique to pip.

> While I think it could be a good idea to bundle (more?) third party packages
> for a variety of reasons, I want to make sure it's done in a way that's still
> friendly to downstream repackagers, as I'm sure you do to. :)

Indeed :)

> For Debian/Ubuntu, we already have some additional machinations to make
> ensurepip and such work properly in a distro packaging environment.  We'd want
> to be able to do the same for any bundled packages as well.

Yeah, I suspect any development along these lines would make it even
more important to have something like PEP 534 in place so that the
bundled modules could be omitted by default in some cases (such as in
virtual environments), and require people to depend on them explicitly
if they needed them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia