[Distutils] PEP 438, pip and --allow-external (was: "pip: cdecimal an externally hosted file and may be unreliable" from python-dev)

[Paul Moore]

On 12 May 2014 21:31, Donald Stufft <donald at stufft.io> wrote:
> This is slightly confusing but pip will always be able to go from an sdist to
> an installed system. It'll just build a Wheel first and then install the Wheel
> (at least that's the idea). This is a sort of vague idea right now but it's the
> direction we want to go in.

MAL's concern (if I read his comments correctly) is that people who
use complex setup.py arrangements to do things like extend commands in
setuptools-incompatible ways may not support build to wheel then
install as a viable approach, whether it's done manually or as an
internal step in pip. This is one of the fundamental issues with
distutils, and is the source of much of the old corrosive "distutils
must die" rhetoric that hurt previous distutils-sig debates.

The current approach is to solve 90% of the problem by noting that
nearly all projects don't take advantage of any of the (usually
undocumented) flexibility that distutils allows. This has thus far
been a great success, in terms of getting people on board with the new
tools and technologies. The problem is that it does nothing for that
remaining 10%[1]

We're now at a point where people like MAL, who are in that 10%, are
getting excluded and we don't have a good answer for them. That's a
big problem - particularly as the people in question are quite often
those to whom we owe a lot of the progress that distutils gave us (I
remember the pre-distutils world, and it *SUCKED*). I wish we had an
answer here. I'm particularly worried that some proportion of the 10%
may be complex in-house environments that we'll never hear about till
we break them horribly. But I don't know what we can do. We need to be
able to move forwards, and the lack of encapsulation in distutils
means we will break things when we do.

Now I'm depressed...

Paul

[1] 95% of all statistics quoted on the internet were made up on the spot.