[Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils

[Donald Stufft]

On October 2, 2015 at 8:38:45 AM, David Cournapeau (cournape at gmail.com) wrote:
>  
> A major feature of the proposal is to allow alternative build/packaging
> tools.
>  
> If that proposal is not acceptable in its current form, how would you
> envision interoperability between pip and new systems. For example, is it
> realistic to encode which commands and options a setup.py would need to
> support to be pip-installable (without the setup.py using distutils) ?
>  

I think it depends if you mean in the short term or the long term and what exactly you envision as the scope of an alternative build/packaging tool.

In the short term we have two real options that I can think of off the top of my head.

One is as you mentioned, define the interface to setup.py that any build tool is expected to follow. However I’m not sure if that’s a great idea because you end up having a bootstrapping problem since pip won’t know that it needs X available to execute a particular setup.py (a common problem with numpy.distutils, as I know you’re aware!). We could do a minimal extension and add another defacto-ish standard of allowing pip and setuptools to process additional setup_requires like arguments from a setup.cfg to solve that problem though. The flip side to this is that since it involves new capabilities in pip/setuptools/any other installer is that it you’ll have several years until you can depend on setup.cfg based setup_requires from being able to be depended on.

Another short term option is to simply say that using something that isn’t distutils/setuptools isn’t supported, but that if you want to do something else, you should rely on setuptools ability to be extended. This means you’d need a very minimal ``setup.py`` [1], but that you could then do pretty much anything you wanted, since setuptools lets you override a good bit of it’s own logic, pbr uses this to have completely static metadata [2]. This option would just work and require no changes in pip or setuptools so then you could start depending on it right away and it would just work, the downside of course is that you’re tied to extending setuptools and the APIs it provides rather than being able to more freely do what you want, but the ability to do that in setuptools is pretty extensive.

Longer term, I think the answer is sdist 2.0 which has proper metadata inside of it (name, version, dependencies, etc) but which also includes a hook like this PEP has to specify the build system that should be used to build a wheel out of this source distribution.

[1] Example for pbr: https://github.com/testing-cabal/mock/blob/master/setup.py
[2] https://github.com/testing-cabal/mock/blob/master/setup.cfg

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA