[Distutils] backwards compatibility for packaging tools

[zooko]

When thinking about compatibility, keep in mind the distinction  
between two use cases:

1.  You are using a tool (e.g. distutils or setuptools) to package  
your work.  You might consider switching to a new tool, either one  
included in the Python Standard Library or a separately-shipped one,  
to build, package, and distribute your software.

2.  You are re-using other people's work that they have packaged and  
distributed.  You might consider switching to a new tool (either  
Python Standard or separate) to re-use their work.

Backwards compatibility in the first case is not overwhelmingly  
important.  Some people will be willing to entirely rewrite their  
setup.py scripts to use a new tool, other people will be willing to  
use a new tool only if they can keep using their old setup.py  
scripts, and still other people will continue to package and  
distribute their software with Python 2.5 and the distutils that came  
with it for the forseeable future (let's say, for the next 5 years).   
We can't prevent them from doing that, but also we don't need to  
persuade them to change tools in order to benefit from their work.

Compatibility in the second case is overwhelmingly important.  If you  
offer me a new tool for re-using other people's source code, and this  
new tool does *not* give me access to the thousands and thousands of  
Python packages that are already out there and the dozens of hundreds  
of new ones that are appearing every month, then this new tool is  
completely uninteresting to me.

So we should focus on documenting and standardizing the metadata that  
allows code re-use -- the interface between the author of one package  
and the author of another package -- not the interface between the  
author of a package and the tool that he uses to build and distribute  
his own package.

We've already got a pretty good start on this -- the distutils in  
Python 2.5 emits PKG-INFO and .egg-info in a format that is  
understood by all of the current crop of packaging tools  
(easy_install, pyinstall, distribute, yolk, bbfreeze, etc. etc. etc.).

Also of course setuptools produces .egg-info metadata in a way that  
those tools can understand.

We also got past the problem that we had for awhile that Linux  
distributions like Fedora and Debian were deleting those .egg-info  
files.  They don't do that anymore.

The next piece that is missing, from my experience in packaging Tahoe  
and its numerous Python dependencies [1], is for more tools to start  
emitting "requires" metadata in a compatible way.  We already have a  
de jure standard for how spell "I require zope.interface" -- PEPs 314  
and 345.  However, I have never seen metadata of this format in the  
wild, and for all I know there are no tools that actually produce or  
consume this metadata and no packages that are actually labelled with  
this metadata.

We also have a de facto standard that is widely used by a large and  
growing library of packages and is supported by a large and growing  
set of tools -- the way that setuptools spells "I require  
zope.interface" in its .egg-info files.

So, I have a simple and urgent request:

Extend distutils so that when the author of package passes  
install_requires=['zope.interface'] to distutils.core.setup(), then  
it emits an entry named "requires" with body "zope.interface" in the  
resulting .egg-info.  That's all.  It won't hurt, and it will  
probably help quite a lot to facilitate interoperation of future  
Python packaging tools.

Regards,

Zooko

[1] http://mail.python.org/pipermail/distutils-sig/2008-September/ 
010081.html
---
http://allmydata.org -- Tahoe, the Least-Authority Filesystem
http://allmydata.com -- back up all your files for $5/month