[Distutils] setuptools in a cross-compilation packaging environment

[M.-A. Lemburg]

[Some comments on your strategy...]

Phillip J. Eby wrote:
>>The new setuptools is all nice and easy for end user, but as a package
>>maintainer, I'd like to have the option of building a binary package without
>>all the dependencies.
> 
> In the long run, this should be done by packaging the result of bdist_egg, 
> and by default doing bdist_rpm will do this now.  In the short term, unless 
> you're switching to an all-egg distribution, you'll probably want to use 
> legacy/unmanaged mode.

I think you are missing his point here:

As package maintainer you *have* to be able to build a distribution
package without all the dependency checks being applied - how else
would you be able to bootstrap the package in case you have circular
dependencies ?

>>Is it possible? How about the above distutils
>>features, are there any equivalent?
> 
> All of the old 'install' options are available, if you use the appropriate 
> command line flags.  Try "setup.py install --help" and look for the rather 
> long option name about doing legacy "unmanaged" installation.  With that 
> option, a "classic" installation will be done, without an egg being 
> built.  However, even if you don't specify that option, all of the options 
> you listed above should work anyway.
> 
> Please note, by the way, that some packages simply *cannot* work using the 
> "unmanaged" mode, and never will.  Such packages *must* be installed as 
> eggs, period.  Among these are setuptools itself, and any packages that are 
> plugins for Trac or Python Paste, or any other extensible system using 
> setuptools to manage plugins.
>
> What's more, packages that explicitly use pkg_resources to request their 
> dependencies will not recognize unmanaged packages as fulfilling the 
> dependency, which means that over time there will be increasing demand for 
> packages to be installed as eggs to start with.

I don't think that eggs are the solution to everything, so
you should at least extend the dependency checking code to
have it detect already installed packages (by trying import
and looking at __version__ strings) or having an option
to tell the system: "this dependency is satisfied, trust me".

BTW, I haven't looked at your bdist_egg, but since we're
currently fighting through binaries releases for our
eGenix packages, I thought I'd drop in a note:

Please make sure that your eggs catch all possible
Python binary build dimensions:

* Python version
* Python Unicode variant (UCS2, UCS4)
* OS name
* OS version
* Platform architecture (e.g. 32-bit vs. 64-bit)

and please also make this scheme extendable, so that
it is easy to add more dimensions should they become
necessary in the future.

To make things easier for the user, the install system
should be capable of detecting all these dimensions
and use appropriate defaults when looking for an egg.

BTW, have you had a look at the ActiveState ppm system
for add-on packages ? It looks a lot like your egg system.

> The simplest way to deal with this is to install vendor-packaged eggs to 
> wherever the distribution normally installs packaged Python packages, and 
> install a corresponding '.pth' file that points to the installed egg.  This 
> will ensure that the package is available and can be imported as if it were 
> installed the old way, so it doesn't break any non-setuptools-based 
> packages.  It also does not require pkg_resources or setuptools to be 
> installed unless a given package actually uses them.  (If the package 
> contains C extensions, the .egg can be expanded as a directory, so that it 
> isn't necessary to extract the .so files at runtime.)

Please reconsider your use of .pth files - these cause the
Python interpreter startup time to increase significantly.

If you just have one of those files pointing to your
managed installation path used for eggs, that should
be fine (although adding that path to PYTHONPATH still
beats having a .pth to parse everytime the interpreter
fires up).

If you however install a .pth file for every
egg, you'll soon end up with an unreasonable startup time
which slows down your whole Python installation - including
applications that don't use setuptools or any of the eggs.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::