[Distutils] A possible refactor/streamlining of PEP 517

[Daniel Holth]

-1 on _for_ ; why should a common prefix plus extra typing be any clearer
than common suffix? Or rearrange the words without _for_.

For the --config options, there's a double dash on the key and the value, I
found that confusing. I suppose the theory, not thoroughly explained, is
that the value is sent to the command line of the build backend. I think
"--build-option=toves=slithy" would be more clear.
--build-option="--build-option1"

A backwards compatible setup.py shim will have to convert
prepare_wheel_metadata
to .egg-info for pip, which is pretty easy.

I didn't anticipate get_requires_* to require running the whole (designed
for a single invocation per process) build backend. Suppose anything's
possible.

On Mon, Jul 3, 2017 at 10:04 AM Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 1 July 2017 at 20:53, Nathaniel Smith <njs at pobox.com> wrote:
> > Hi all,
> >
> > I just attempted an experimental refactor/streamlining of PEP 517, to
> > match what I think it should look like :-). I haven't submitted it as
> > a PR to the PEPs repository yet since I don't know if others will
> > agree with the changes, but I've pasted the full text below, or you
> > can see the text online at:
> >
> >
> https://github.com/njsmith/peps/blob/517-refactor-streamline/pep-0517.txt
> >
> > and the diff at:
> >
> >
> https://github.com/python/peps/compare/master...njsmith:517-refactor-streamline?expand=1
> >
> > Briefly, the changes are:
> >
> > - Rearrange text into (hopefully) better signposted sections with
> >   better organization
>
> This is definitely needed, so thanks for that.
>
> > - Clarify a number of details that have come up in discussion (e.g.,
> >   be more explicit that the hooks are run with the process working
> >   directory set to the source tree, and why)
>
> This is again very helpful (we also need to be clearer on which sets
> of dependencies must be installed prior to running the various hooks)
>
> > - Drop prepare_wheel_metadata and prepare_wheel_build_files (for now);
> >   add detailed rationale for why we might want to add them back later.
>
> I want prepare_wheel_metadata there as a straightforward way for
> backends to expose the equivalent of "setup.py egg_info". Losing that
> would be a significant regression relative to the status quo, since
> pip relies on it during the dependency resolution process (and a
> backtracking resolver will be even more dependent on being able to do
> that efficiently).
>
> For the build preparation hook, Thomas already rejected promising that
> build_sdist will always work in flit when given only an unpacked
> sdist, while Donald and Paul rejected relying on full tree copies as a
> fallback for build_sdist failing.
>
> Since I'm not interested in rehashing that debate, the build
> preparation hook also stays.
>
> However, I'd be fine with renaming these hooks using the
> "prepare_X_for_Y" scheme you suggest below:
>
> - prepare_metadata_for_build_wheel
> - prepare_input_for_build_wheel
>
> That makes it clearer where they fit into the overall wheel building
> process, as well as which particular step they implement within that
> process.
>
> > - Add an "extensions" hook namespace to allow prototyping of future
> >   extensions.
>
> No, on grounds of YAGNI. You can't reject the two hooks we've actually
> identified as needed (aka the matrix multiplication operators of this
> situation), and then turn around and argue in favour of an overly
> general scheme that supports arbitrary extension hooks (which would be
> somewhat akin to having taken the PEP 225 approach to resolving the
> syntactic ambiguity between elementwise multiplication and matrix
> multiplication).
>
> We're not fumbling around based on a complete absence of information
> here - we're formalising a backend build interface based on more than
> a decade of experience with the successes and challenges of the
> existing system based on distutils, setuptools and PyPI.
>
> Besides, if frontends want to invoke arbitrary interfaces that are
> specific to particular backends, we already have ways for them to do
> that: Python's regular import system, and plugin management approaches
> like pkg_resources entry points.
>
> > - Rename get_build_*_requires -> get_requires_for_build_* to make the
> >   naming parallelism more obvious
>
> This sounds like a good change to me.
>
> > - Add the option to declare an operation unsupported by returning
> >   NotImplemented
>
> No. Supported operations for a backend must be identifiable without
> installing any dependencies, and without running any backend code.
>
> To tighten that requirement up even further: if the backend's
> capabilities can't be accurately determined using
> "inspect.getattr_static", then the backend is not compliant with the
> spec. The build frontend/backend API is not a space where we want
> people to try to be clever - we want them to be completely dull and
> boring and use the most mundane code they can possibly come up with.
>
> However, it's fine for operations to fail with an exception if there
> are external dependencies that haven't been satisfied (e.g. requiring
> a C/C++/FORTRAN/Rust/Go/etc compiler for extension modules when there
> isn't one available, or requiring some other non-Python dependency
> that needs to be set up prior to requesting the sdist or wheel build).
> Backends are also free to raise NotImplementedError when it seems
> appropriate to do so (e.g. when the project doesn't support the
> current CPU architecture) - frontends will handle the same way they
> will any other backend exception.
>
> > - Instead of defining a default value for get_requires_for_build_*,
> >   make it mandatory for get_require_for_build_* and build_* to appear
> >   together; this seems simpler now that we have multiple high-level
> >   operations defined in the same PEP, and also simplifies the
> >   definition of the NotImplemented semantics.
>
> I don't see the value in requiring four lines of boiler plate in every
> backend definition that doesn't support dynamic dependencies:
>
>     def get_requires_for_build_sdist(*args, **kwargs):
>         return ()
>
>     def get_requires_for_build_wheel(*args, **kwargs):
>         return ()
>
> That's just pointless noise when we can instead say "Leave those hooks
> out of the backend implementation entirely if all build dependencies
> will be declared in pyproject.toml (and if you really want your
> backend work to support dynamic build dependencies, please make sure
> you have a really good rationale for doing so that isn't already
> covered by pyproject.toml and the environment marker system defined in
> PEP 508)"
>
> > - Update title to better match the scope we ended up with
>
> I'd prefer to retain the current title, as it describes the desired
> *outcome* of the change, whereas the proposed revision only describes
> one of the technical pre-requiisites of achieving that outcome (i.e.
> eliminating the historical dependency on setup.py as the build
> system's executable interface).
>
> > - Add a TODO to decide how to handle backends that don't want to have
> >   multiple hooks called from the same process, including some
> >   discussion of the options.
>
> I don't think that's a TODO: I'm happy with the option of restricting
> frontends to "one hook invocation per subprocess call".
>
> It only becomes an open question in this revised draft by virtue of
> making the get_requires_* hooks mandatory, and I have a different
> resolution for that: keep those hooks optional, so that only backends
> that genuinely support dynamic build time dependencies will define
> them (others should either just get users to list any additional
> static build dependencies in pyproject.toml, or else list any
> always-needed static dependencies in the backend's own
> install_requires).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170703/ab0c8ea8/attachment-0001.html>