[Distutils] PEP: Build system abstraction for pip/conda etc

[Paul Moore]

On 11 February 2016 at 14:02, David Cournapeau <cournape at gmail.com> wrote:
>>> On Wed, Feb 10, 2016 at 1:52 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>>>
>>>> We should probably also check with the flit people that the proposed
>>>> approach works for them. (Are there any other alternative build
>>>> systems apart from flit that exist at present?)
>>>
>>> I am not working on it ATM, but bento was fairly complete and could
>>> interoperate w/ pip (a few years ago at least):
>>> https://cournape.github.io/Bento/
>>
>> I plan to test with Bento (I'm still using it almost daily to work on
>> Scipy) when an implementation is proposed for pip. The interface in the PEP
>> is straightforward though, I don't see any fundamental reason why it
>> wouldn't work for Bento if it works for flit.
>
> It should indeed work, I was just pointing at an alternative build system ;)

Yes, I knew there was something other than flit, thanks for the reminder.

> I am a bit worried about making a PEP for interfacing before we have a few
> decent alternative implementations. Having the official interface is not
> necessary to actually interoperate, even if it is ugly.

Well, a lot of people have complained that setuptools is a problem.
We've only really seen bento and now flit appear as alternatives, the
only conclusion we've been able to draw is that the barrier to
creating alternative build systems is the need to emulate setuptools.
This PEP (hopefully!) removes that barrier, but I agree we need some
validation that people who want to create alternative build systems
(or have done so) can work with the interface in the PEP.

There is some value to the PEP even if it doesn't enable new build
tools (we can fix the problem of install_requires triggering
easy_install) but the key has to be the evolution of (one or more)
replacements for setuptools.

You suggest getting more alternative build systems before implementing
the PEP. That would be nice, but how would we get that to happen?
People have been wanting alternatives to setuptools for years, but
no-one has delivered anything (as far as I know) except for you and
the flit guys. So my preference is to implement the PEP (and remove
the "behave like setuptools" pain point), and then wait to see what
level of adoption flit/bento achieve with the simpler interface and
automated use of the declared build tool. At that stage, all of flit,
bento and setuptools, as well as any newly developed tools, will be
competing on an equal footing, and either one will emerge as the
victor, or people will be free to choose their preferred tool without
concern about whether it is going to work with pip.

I'm not sure what we gain by waiting (incremental improvements to the
spec can me made over time - as long as the basic form of the PEP is
acceptable to *current* tool developers, I think that's sufficient).

Paul