[Distutils] PEP 517 again

[Donald Stufft]

> On Aug 28, 2017, at 3:58 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
> On 28 August 2017 at 20:47, Donald Stufft <donald at stufft.io> wrote:
>> I also believe it is fundamentally impossible for the backends to guarantee
>> consistency if they have a separate list for what gets installed vs what
>> gets put into a sdist without literally building a sdist (or something
>> similar)— and as I understand it one of the issues with that is that the
>> tools don’t want to mandate being able to do that same logic in all
>> situations.
> 
> And yet that's precisely what xoviat is intending to do, based on what
> the PEP says is needed. Are you saying he shouldn't? And the
> setuptools should simply expose the same consistency problems we've
> been dealing with until now? And the solution is for pip to switch to
> sdist->wheel build as we've always planned in order to resolve this
> issue? And any other future frontends will have to go through the same
> cycle that pip did?


I had some non-OSS stuff come up so I lost track of the discussion part way through so maybe I missed this recommendation, but I think that is going to be a bad path to go down because it is intermixing things in a way that I feel is going to result in a less optimal outcome. The problem is the backend doesn’t really know why we’re asking it to build a wheel to know whether we care or not about consistency guarantees. 

Future front ends can make their own decisions as it makes sense for their workflow. Something like Debian might choose to skip the sdist option (when they go straight from a VCS archive or w/e) because they have a human being sitting there ensuring that the install looks correct, and they are unlikely to want to go through sdist. 

> 
> In which case, isn't the issue here that we've failed to argue
> sufficiently persuasively that all backends must provide a
> non-optional build_sdist hook, so we can do that cleanly?


I’m not sure what you mean by non-optional— Do you mean a build_Sdist hook that does not have a “not implemented” flag? That would probably be my preference, but I’m happy to compromise on the not implemented return value/exception/whatever as a middle ground. Honestly this PEP has been drawn out enough with enough hypotheticals and corner cases that one of the main things I’m trying to optimize for is flexibility so we can handle different possible scenarios without going through this again.


> 
>> So I don’t think it works to say “You *must* ensure a consistent
>> output, and I think the only thing we can do is say that you *SHOULD* try to
>> be consistent, and leave it up to front ends to decide how seriously they
>> take that as a requirement.
> 
> But how do frontends decide that? They have to base the decision on
> experience with backends. And it looks like the current crop of
> backends are split down the middle. Remember that if pip decides to go
> sdist->wheel, then that strongly motivates xoviat to abandon his
> current approach to ensure consistency, as it's duplicating the work
> pip will do. At which point the setuptools backend can't be assumed to
> be consistent, and the flit backend must be assumed to be (because we
> can't guarantee it can make a sdist).
> 
> 


Well I don’t think it’s solely based on their experience with backends, but also what the general goal of their tool is. Something that is designed to run in CI might take a very strict stance to be extra sure to catch errors, something that runs in cases where you care more about performance than correctness in edge cases or specialized tools where they’ve done extra work and aren’t processing arbitrary Packages might just skip straight to building wheels.


—
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170828/dd1c15b2/attachment.html>