[Distutils] Standard packaging API? (was Re: Are there any plans to move to pip/wheels in buildout?)

[Donald Stufft]

> On Dec 1, 2014, at 8:44 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 2 December 2014 at 05:54, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> AFAIK pip does use distlib (it is vendored by pip), but only for some
>> ancillary functions such as pre-release version checks.
> 
> The current direction things appear to be going is:
> 
> * use distlib to get a "batteries included" approach to packaging
> infrastructure (which may include some distlib specific capabilities)
> * use the reborn "packaging" if you want a minimalist "just the
> approved PEPs, thanks” API

This is how I view the two projects yes. The only things I view as
acceptable for the packaging lib that isn’t directly from a PEP are
“supporting” things.

An example would be we have a Version class which implements PEP 440
style versions, that comes straight from the PEP. However we also have
a LegacyVersion class which implements the setuptools style version
numbers, this doesn’t come from a PEP but it’s needed as a companion
to Version in order to sanely handle versions in the real world. An
additional example of what wouldn’t be allowed is a SemanticVersion
class which implements https://semver.org/ (unless of course a PEP was
written that detailed how to use that within the python packaging
ecosystem).

At some point it may make sense to move the packaging lib into the
stdlib. However I don’t really feel a compelling need to do this
since both 2.x and 3.x have ensurepip now and the state of the ecosystem
is changing rapidly enough that I don’t suspect the stdlib version to
be very useful to many people. This is particularly true since it’s
a pure Python lib which is super easy to install.

> 
> The latter is at https://pypi.python.org/pypi/packaging, and
> maintained by Donald & Marcus as a way of factoring out a supported
> pip independent library API for the interoperable PEP based behaviour.
> 
> The current focus is on ensuring it provides a suitable API to
> implement the PEP 440 support in both pip and setuptools, so I'm not
> sure if it can be considered stable just yet.

Right. The status of packaging right now is more or less we’ll change
things as we need to. Probably we should mark specific APIs as provisional
or not. For instance we have releases of setuptools/pip that support
PEP 440 using the packaging lib I consider those particular APIs to require
a fairly strong level of backwards compatibility.

> 
> Longer term, it may potentially make sense to have distlib also depend
> on the lower level packaging API, but there's also value in having two
> somewhat independent implementations to help reduce the chance of the
> overall ecosystem coming to rely on implementation defined behaviour
> again.
> 
>> I'm not sure it's a good idea to use pip's internal API (as it's internal,
>> and I don't believe it's been designed for use as a library by external
>> code).
> 
> Agreed - the components intended for external use are the ones being
> factored out into the packaging.

Historically our answer has been that pip’s API is internal and subject
to change at any time. However we’re willing to listen to specific
requests for particular APIs to be made public and given real backwards
compatibility promises. At this point though I think you’d need to make
a pretty compelling argument about why it needed to be done immediately
and couldn’t wait for a standards based implementation to be created
that pip itself would switch to.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA