[SciPy-dev] what the heck is an egg ;-)

[David Cournapeau]

Joe Harrington wrote:
> A lot of the add-on software for numpy/scipy is distributed using novel
> Python install processes (eggs, setup.py) 
setup.py and using distutils is not novel: this is the standard way to 
distribute python packages to be built for years (distutils was included 
in python in 2000, 2001 ?). It offers a common set of rules to build 
python packages, including compiled extensions, on a wide range of 
platforms. Eggs is a different matter.

Think setup.py as the configure script for python packages
> rather than tarballs or the
> preferred OS-native installers (dpkg, rpm, etc.).  
Building binary packages is totally different than distributing source 
tarballs. Work on that is always welcome, but packaging is not a funny 
thing to do, the rewards mostly consisting in getting complaints when it 
does not work :) Several people develop rpm and deb (Andrew Straw has 
debian packages, for example, I started developing rpm for FC and 
openSUSE using the open suse build system), but ideally, this should be 
included upstream in the different distributions (this is the case for 
debian packages at least, I think: it is in debian and ubuntu).
> I'm sure they are
> described, perhaps even well, in other places, but, since scipy.org is
> our portal, I think it would be good to have a few-line description of
> each method on the Download page and a link to more detailed
> descriptions elsewhere (or on subsidiary pages).  An example of
> installing a package many will want, like mayavi2, would be great.
>
> In particular, many sysadmins (who might be considering a user's request
> for an install and know nothing about python) get nervous when package
> managers other than the native one for the OS start mucking around in
> the system directories, and are hesitant to use something like eggs.
> Some statements describing what they do and where they put stuff would
> be good (like, a guarrantee that they only tread in certain
> directories).  
IMHO, eggs are a bad idea and a mess; I avoid them and setuptools like 
the plague. I don't like them, but they solve real problems for many 
people. If you have complaints about them, bugs reports and ideas on how 
to improve, the right place is the distutils ML, though.

But except for windows, I don't think eggs are distributed for 
numpy/scipy, at least not officially. Where do you see eggs on scipy.org ?
> How to update and how to completely remove a package
> would be good.  Is there a way to have them check periodically for
> updates?  Of course, a statement near the top of why these methods are
> used rather than the native OS installers would help a lot.
Some of the things eggs are supposed to give are:
    - handle dependencies
    - handle several versions
Many OS "native" packaging systems do not enable those (handling 
dependency is not really in the culture of Mac OS X and windows, for 
example; I don't know any linux packaging system which enables several 
versions to be installed at the same time; at least deb and rpm do not 
make that possible, and this covers may 90 % of linux users). Mac OS X 
packaging system does not support uninstalling features either (at least 
officially); I don't know if eggs are uninstallable, though.

I think the only real solution is to push for binary packages to be 
included upstream for linux distributions, and some people are working 
on that now. For other platforms, the one thing really missing is maybe 
binary packages for mac os X, but there is discussion now to solve this. 
Again, this is a manpower problem: producing good binary packages is 
hard and takes time, is not so funny, and most people able to do it do 
not need them (installing from source being easier).

David