[Distutils] PEP 517 again

[Thomas Kluyver]

Nathaniel seems to be busy with other things at the moment, so I hope he
won't mind me passing on this list of things he'd like to resolve with
the draft PEP. I'll quote his comments and put my responses inline.

> - needs some sort of NotImplemented(Error) handling specified

We've gone round return-value vs exception a few times, but I don't
think that debate is particularly heated, so it probably just needs
someone to say "this is what we're doing". I can be that someone if
needs be. But does anyone feel strongly about it?

> - The out-of-tree build example (that makes an sdist and then builds
> it) seems like exactly the kind of implementation that Donald says he
> doesn't want to see? OTOH I think Nick said he wants to see a more
> elaborated specification of out-of-tree build strategies with this
> specifically as one of them.
> - Nick has suggested that the to-be-defined NotImplemented(Error)
> handling can be used by build_wheel to indicate that it can't do
> out-of-tree builds, so maybe the frontend should try an in-tree build
> instead. I don't know how to convert this into a real spec, though --
> like in general, if I'm a frontend and I call `hook(*args, **kwargs)`
> and it says "can't do that", then how do I know what the problem is
> and what I should try instead?
> - What happens if prepare_build_metadata returns NotImplemented /
> raises NotImplementedError?
> - I don't understand how out-of-tree builds and prepare_build_metadata
> are supposed to interact.
> - Also, AFAICT the out-of-tree build feature has no motivation
> anymore. The only mentioned use case is to support incremental build
> features in automatic build pipelines like Travis, but there are a
> number of these build pipelines, and lots of existing build systems
> that support out-of-tree builds, and AFAICT no-one actually uses them
> together like this. (I think it's because most build systems'
> out-of-tree build features are designed on the assumption that there's
> a human running the show who will deal with any edge cases.)

I believe the out-of-tree build option Nathaniel is referring to is the
build_directory parameter to build_wheel(). His proposed APIs remove
that parameter, and elsewhere in his email he describes that no-one
seems to want it: everyone thinks someone else wants it.

By my understanding, the reasons for including the build_directory
parameter are:

1. Without an explicit build directory, the developers of pip are
concerned that build backends will modify the source tree and cause
issues which users report as bugs to pip.
2. A frontend-controlled build directory could be used for caching
intermediate build artifacts. This was a secondary argument after we had
the idea, and we've never really fleshed out how we expect it to work.
There's also a concern that if the first round of frontends always use
an empty tempdir, backends will end up relying on that. I think this
second argument is a weak one unless we spend the time to figure out the
details.

Do other people see the situation in a similar way? How might we move
forwards on this?

> - needs some sort of conclusion to the srcdir-on-sys.path issue.

While Nathaniel is in the minority regarding srcdir-on-sys.path, he
argues that most of us are assuming a default position without really
thinking through it, which is certainly true of me. I don't feel
strongly about this topic, but I do want to come to a conclusion. As
Nathaniel does feel strongly about it, does anyone object to either:

A. Leaving the source dir off sys.path when loading the hooks, and
making a special backend which exposes hooks from the CWD.
B. Adding another key in the TOML file to control whether hooks can be
loaded from the source dir.

I can live with either, but I prefer A, because it means fewer options
to describe in the spec.

Thomas