[Distutils] A possible refactor/streamlining of PEP 517

[Donald Stufft]

> On Jul 14, 2017, at 2:24 PM, Nathaniel Smith <njs at pobox.com> wrote:
> 
> On Fri, Jul 14, 2017 at 4:00 AM, Nick Coghlan <ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>> wrote:
>> On 14 July 2017 at 20:18, Nathaniel Smith <njs at pobox.com> wrote:
>>> On Fri, Jul 7, 2017 at 8:27 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> Some possible problems that I've seen mentioned in the thread include:
>>> 
>>> - pip doesn't trust build systems to properly support incremental
>>> builds, so it wants to force it to throw away the build artifacts
>>> after every build
>> 
>> It's less that, and more pip wanting to ensure that the default
>> publisher experience is reasonable close to the default end user
>> experience, in order to increase the likelihood that publishers ship
>> working sdists and wheel files, even if they haven't learned about the
>> full suite of available pre-release testing tools yet (or have chosen
>> not to use them).
> 
> This is exactly the opposite of what Paul says in his message posted
> at about the same time as yours... AFAICT his argument is that build
> artifact leakage is exactly what pip is worried about, and if anything
> he's annoyed at me because he thinks I'm downplaying the problem :-).
> (My impression is that Donald's position might be closer to what you
> wrote, though.)
> 
>>> - pip wants to enforce builds going via sdists
>> 
>> This isn't a requirement, just an option we want to enable (and can't
>> avoid enabling).
> 
> Donald sure seemed to be insisting on it as a requirement earlier, at
> least. And the *entire cause* of this whole discussion was conflict
> between this requirement and flit not wanting to support building
> sdists from unpacked sdists, so characterizing it as some automatic
> thing that doesn't even need to be talked about seems really weird.


Just to point out here that like many things, it’s complicated (tm). We have a number of issues that regularly come in on the pip side which really boil down to the build tool did something wrong and unexpected. Over time we’ve built up a number of strategies to attempt to work around those cases and as we look at a new system, we’re thinking of those cases while we evaluate it and going “Is this going to make those cases, better, worse, or not effect them?” and trying to push the design towards things that make those cases better rather than worse.

There is some differences I suspect in what specific problems me and Paul are each thinking of as well as well as what specific solutions to solving them we might like or even find acceptable. For better or worse pip doesn’t have a BDFL and for direction we generally follow rough consensus and working code. So “pip” doesn’t really have an official position on what we want from this, but Paul has a position and so do I.

To try and break down my thinking some more:

* I think it is a requirement that any PEP that proposes a mechanism for removing ``setup.py`` supports sdist and wheel building in it’s interface. Otherwise it sends a message that either one is an optional part of Python packaging, which is not a message we should be sending.

* I would *like* to remove a fairly common error case where we have a difference in the files that get included in a sdist and the files that get installed such that ``pip install .`` causes different files to be installed than ``python setup.py sdist && pip install dist/*.tar.gz``.

* I would *like* to remove another common error case where two subsequent builds end up affecting each other (at least by default) because of detritus left behind from the first build process. One example I can think of is if you have a library that can optionally link against some other library, you might ``pip install .``, notice it didn’t link against that (because you didn’t have it installed) and then install it and ``pip install .`` again. Nothing would have changed in the directory to trigger a rebuild, so you’d get the behavior where it wouldn’t rebuild.

I have opinions about how the above things get solved, but I think only the first one is a hard and fast requirement where if the PEP doesn’t include it at all I simply cannot support the PEP at all. The other things are things I personally would very much prefer to solve, because it solves a common stumbling block for both new and experienced users alike.

—
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170714/d463efd6/attachment.html>