[AstroPy] package template setup file

[Erik Bray]

On 05/10/2015 10:29 AM, Andrew Hearin wrote:
> Hi everyone,
>
> I have a question about the setup.py configurations in the astropy package
> template. In the arguments to “setup”, the astropy package template defaults
> has a keyword argument for “requires” as well as “install_requires”. If I
> correctly understand the description on pythonhosted.org, the
> “install_requires” argument will make sure that all previously uninstalled
> dependencies will automatically be pip installed upon call to "python
> setup.py install.” However, what is the independent purpose of the “requires”
> argument? Under what circumstances would these arguments have different
> values?

Good question. I've made a little noise about this in the past but not enough to 
do anything about.

To skip to the punchline: the requires argument to setup() should not be used 
*at all* and the example of its usage should be removed from the 
package-template IMO.

Now for the longer version, the reasons for the dual existence of requires and 
install_requires is historical.  The *requires* keyword is provided by the 
original distutils implementation (along with provides and obsoletes) to specify 
information about what Python *modules* (i.e. the name you actually import) a 
distribution requires to work:

https://docs.python.org/2/distutils/setupscript.html#relationships-between-distributions-and-packages

These were included in the original distutils metadata specification in 
anticipation of automated package management tools that might make use of them 
(but did not yet exist; at the time there was no setuptools, easy_install, pip, 
or even PyPI).  At the end of the day, no tools that used these metadata 
keywords were ever implemented.  Part of the reason for that is that it was not 
considered a robust way to specify requirements for a distribution.  For 
example, say you had:

     setup(
         ...
         requires=['tools'],
         ...
     )

Well, what 'tools' module does that refer to?  There is no top-level namespace 
of globally unique Python module names.  This requirement could be satisfied if 
you ran

$ touch tools.py

before running the above setup.py.

However, with the creation of PyPI (née The Cheeseshop) a sort of "global 
namespace" was created.  However, the keys in that namespace are not module 
names, but *distribution* names.  In many cases the name of a Python 
distribution coincide with the name of a (single) module included in that 
distribution.  But that doesn't have to be the case--a single distribution can 
include multiple modules.  For example the distribution named "setuptools" 
contains both the `setuptools` and `pkg_resources` modules.

So when setuptools was created around the same time as PyPI (I can't remember 
which came first), it made more sense to add a setup() keyword that specifies 
unique distributions that are requirements for a distribution.  Since 
requires=[...] was already taken, and had different semantics, setuptools added 
install_requires.

In practice install_requires ended up being far more useful, and is the only 
field used to specify (Python) requirements of a Python distribution.  Very few 
distributions still actually use requires=[...], and many that do use it 
incorrectly.  In fact, the old requires field is deprecated in PEP 345:

https://www.python.org/dev/peps/pep-0345/

However, most tools (i.e. setuptools) are not currently generating PEP 
345-compatible metadata as a practical measure for backwards compatibility (this 
is changing, but very slowly), so it's not very obvious (there are no warnings) 
that these metadata fields will be deprecated, yet.  But the community consensus 
(albeit confusing to someone who doesn't follow these issues, since the 
distutils docs haven't even been updated yet), is that these keywords 
(`requires`, `provides`, etc.) should not be used.

The last time I could find that this was discussed extensively online was in 
this thread:

https://mail.python.org/pipermail/distutils-sig/2013-October/022859.html

However, after all that, it seems no immediate action ended up being taken (I 
think it was just harmless enough that it fell to the backburner). 
Nevertheless, it's clear that confusion about these keywords still exists.

Erik