[SciPy-Dev] Making it easier to pin numpy in pyproject.toml

[Ralf Gommers]

On Tue, Nov 26, 2019 at 1:52 AM Thomas Robitaille <
thomas.robitaille at gmail.com> wrote:

> Hi all,
>
> As a developer of several packages that have numpy as a build-time
> dependency, I've been having to specify pinned versions of numpy in
> pyproject.toml files with environment markers, as is done by scipy:
>
> https://github.com/scipy/scipy/blob/master/pyproject.toml#L6
>
> I've also noticed that different packages are sometimes inconsistent
> in what exact versions they use, and the scipy pinnings are also a
> little more advanced since they take into account the platform too
> (specifically the AIX case at the moment).
>
> I decided to experiment with an idea that might make the whole process
> easier and reduce burden on developers every time a new Python version
> is released. The basic idea is to create a meta-package that has a
> run-time dependency which is a pinned Numpy version, using the same
> pinnings. For now I've called this package 'oldest-supported-numpy':
>
> https://github.com/astrofrog/oldest-supported-numpy
> https://pypi.org/project/oldest-supported-numpy/
>
> The idea is that with this, a pyproject.toml file can just use:
>
> [build-system]
> requires = ["wheel", "setuptools", "oldest-supported-numpy"]
>
> and there would then be one centralized place where we keep the
> pinnings, and can easily release a new version when a new Python
> version comes out.
>
> I wanted to check whether I might be missing some subtleties that mean
> that there will be issues with this approach?


I think so. Looking at your setup.cfg, you're using install_requires. These
are build-time dependencies though, so you should be using setup_requires
instead (or as well).

Also, your meta-package does itself not have a pyproject.toml. Pip does not
understand anything about setup_requires, so I think any build-time
dependency you handle via setup.cfg will necessarily invoke easy_install (I
think, didn't test), which is something to be avoided.

If not, then is this
> something that the SciPy community would be interested in using and
> maintaining? (since it seems a more natural home than just my user
> account). I'm also not particularly attached to the package name, so
> this could easily be changed to e.g. pep518-numpy or other options.
>

Maintaining, perhaps yes - if it works (see above). Getting these
dependencies right is hard, so if we can make it easier for packages that
want to stay in sync with how SciPy handles the NumPy dependency, that
sounds like a sensible idea.

On the other hand, for SciPy itself I'd rather just stay with
pyproject.toml in its current form I think. A metapackage dependency comes
with other costs I suspect.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20191126/4ef04118/attachment.html>