[Distutils] PEP 439 and pip bootstrap updated

[Vinay Sajip]

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Some day pip will get a "wheel only" mode, and that's the step that will kill
> off the need to run setup.py on production machines even when using the
> Python specific tools.

> Blessing both setuptools and pip as the "obvious way to do it" is designed to
> give us the wedge we need to start a gradual transition to that world without
> facing the initial barriers to adoption that were part of what scuttled the
> distutils2 effort.

I think wheel is a good part of that wedge. Considering the barriers to
adoption of distutils2:

1. Distutils2 expected people to migrate their setup.py to setup.cfg while
   providing only minimal help in doing so. I have gotten quite far in
   addressing the migration issue, in that I already have fully declarative
   metadata, *automatically* generated from setup.py / setup.cfg, and distil
   can do dependency resolution and installation using that metadata for a
   large number of distributions currently existing on PyPI. The automatic
   process might not be perfected yet, but it already does much of what one
   might expect given that it doesn't do e.g. exhaustive analysis of a setup.py
   to determine all possible code paths, etc. so it can't capture all
   environment- dependent info.
2. Distutils2 did not do any dependency resolution (not having any index
   metadata it could rely on for dependency information), but that's not the
   case with distlib. While it's not a full-blown solver, distlib's dependency
   resolution appears at least as good as setuptools'.
3. Windows seemed to be an afterthought for distutils2 - that's not the case
   with distlib. Although it may not be necessary because of the existence of
   the Python launcher for Windows, distlib has provision for e.g. native
   executables on Windows, just as setuptools does.
4. Distutils2 did not provide some functionality that setuptools users have
   come to rely on - e.g. entry points and package resources functionality.
   Distlib makes good many of these omissions, to the point where an
   implementation of pip using distlib to replace pkg_resources functionality
   has been developed and passes the pip test suite.
5. Distutils2 did not support the version scheme that setuptools does, but
   only the PEP 386 version scheme, which was another migration roadblock.
   Distlib supports PEP 440 versioning *and* setuptools versioning, so that
   barrier to adoption is no longer present.
6. Distutils2 did not provide "editable" installations for developers, but
   distil does (using ordinary .pth files, not setuptools-style "executable"
   ones).
7. Because wheel was not available in the distutils2 world, it would be hard
   for distutils to provide a build infrastructure as mature as distutils / 
   setuptools extensions as provided by NumPy, SciPy etc. However, now that
   the wheel specification exists, and wheels can be built using setup.py and
   installed using distlib, there's much less of a reason to require
   setuptools and pip at installation time, and more of a reason to give
   developers reasons to provide their distributions in wheel format.

While I'm not claiming that distlib is feature-complete, and while it doesn't
have the benefit of being battle-tested through widespread usage, I'm asserting
that it removes the barriers to adoption which distutils2 had, at least those
identified above. I'm hoping that those who might disagree will speak up by
identifying other barriers to adoption which I've failed to identify, or any
requirements that I've failed to address satisfactorily in distlib.

Regards,

Vinay Sajip