[Distutils] A possible refactor/streamlining of PEP 517

[Donald Stufft]

> On Jul 5, 2017, at 12:08 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
> I have to say I still have deep reservations about flit's approach of
> assuming/requiring that you're using VCS (git) to maintain your
> project. I know that in practical terms most people will be, but it
> still seems like a strong assumption to make. One of the consequences
> is that flit doesn't handle scenarios like "I unpacked a sdist" or "I
> downloaded the project archive from github and unpacked that" well.
> And the result of *that* is that we're putting in mechanisms in the
> PEP to manage that approach.


I think it’s possible for us to generically handle the “I unpacked a sdist” case in pip by just using the current shutil.copytree approach. The bad case for that (large .git repositories, .tox, etc) generally only happen in actual development environments and not inside of a sdist itself. The only thing we would need is some mechanism for a front end to determine if something is an unpacked sdist or not.

Of course the backend could also implement it a similar way, by just tarring up the entire directory if it detects it’s in an unpacked sdist (which means it would need to add a mechanism to detect if it’s in an unpacked sdist, rather than us needing to add one to the PEP itself).

I don’t have a strong preference for how we do that, and we could even do both things TBH, and reuse Nathaniel’s idea of using the NotImplemented singleton and make the logic (from pip’s POV):

* From a VCS/random directory that ISNT an unpacked sdist, call build_sdist and if that fails or says it’s not implemented, then fail the build and surface an error to the user to indicate why.
* From a unpacked sdist (as determined by some hypothetical mechanism we would add to the PEP) call build_sdist and if that fails surface an error to the user to indicate why, but if it returns NotImplemented then fall back to just using shutil.copytree.

That would allow a tool like flit to just completely ignore the unpacked sdist case (other than to return a NotImplemented) and have things still work fine BUT it also allows another tool to do something extra in that case (such as verifying all of the file hashes against a record of known hashes and munging the version if any differ to indicate the version has changed).

That doesn’t solve the “I downloaded a archive from GitHub” or “I mounted my VCS checkout into a docker container without my VCS installed”, but those can only be solved by the backend tool itself and IMO it’s perfectly acceptable for the backend to fail with an appropriate error message if it needs something that the current directory or environment doesn’t provide.

—
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170705/48aa4867/attachment-0001.html>