Setuptools, build and install dependencies (was: eggs considered harmful)

[Harry George]

Ben Finney <bignose+hates-spam at benfinney.id.au> writes:

> Harry George <harry.g.george at boeing.com> writes:
> 
> > Historically, python packages played well in this context.  Install
> > was a simple download, untar, setup.py build/install.
> >
> > Eggs and with other setuptools-inspired install processes break this
> > paradigm.  The tarballs are incomplete in the first place.  The builds
> > sometimes wander off to the internet looking for more downloads.  The
> > installs sometimes wander off to the internet looking for
> > compatibility conditions.  (Or rather they try to do so and fail
> > because I don't let themn through the firewall.)
> 
> If you provide the build and install script with all the dependencies
> already present (in the current directory), my experience is that
> setuptools does not do any network actions.
> 
> -- 
>  \           "Self-respect: The secure feeling that no one, as yet, is |
>   `\                                 suspicious."  -- Henry L. Mencken |
> _o__)                                                                  |
> Ben Finney

Thanks for the idea.  It doesn't work so well in our context, since
many dependencies are installed long before a particular egg is
attempted.  

We need to know the dependencies, install them in dependency order,
and expect the next package to find them.  "configure" does this for
hundreds of packages.  cmake, scons, and others also tackle this
problem.  Python's old setup.py seems to be able to do it.  

However, as I understand it, setuptools can't detect previously
installed python packages if they were not installed via eggs.  Thus,
my ZSI install was failing on "PyXML>=8.3", even though PyXML 8.4 is
installed.  I can't afford to drag copies of all the dependent source
tarballs into an egg's currdir just so it can find them.  (We have 6 GB
of tarballs -- who knows how much untarred source that would be.)

I just found hints that you should not attempt to install ZSI form
tarball, but should rather install from an egg.  So I was able to
install ZSI for py2.4.

Unfortunately, that means I would have to carry
python-version-dependent renditions of every egg.  We have people
running on py23, py24, and py25, thus tripling the number of
tarballs/eggs to manage.  This is the very reason we went to a
*source* based repository.

-- 
Harry George
PLM Engineering Architecture