Splitting large packages with distutils

[Joost Molenaar]

Hello list,

suppose I have three packages like this:

ingredients-base/
    ingredients/
        __init__.py
    setup.py   <-- this one only references package ingredients

ingredients-spam/
    ingredients/
        __init__.py
        spam/
            __init__.py
            recipe.py
    setup.py   <-- this one only references package ingredients.spam

ingredients-eggs/
    ingredients/
        __init__.py
        eggs/
            __init__.py
            recipe.py
    setup.py   <-- this one only references package ingredients.eggs

Now I can install these like this:

virtualenv env
env/bin/pip install file:///path/to/ingredients-base
env/bin/pip install file:///path/to/ingredients-spam
env/bin/pip install file:///path/to/ingredients-eggs

Now I have one source tree with the packages ingredients, ingredients.spam
and ingredients.eggs all rolled into one, so that works OK.

But when I run pip uninstall for ingredients.spam, it also removes the
source files for ingredients and ingredients.eggs. That seems a bit wrong,
but maybe I'm using distutils/pip/setuptools the wrong way. I found out
that if I modify top_level.txt in each of the egg-info directories that are
installed so that they include the full package name, pip doesn't uninstall
all the code. (And I didn't see any bad effects on the sys.path.) But I
haven't found a non-hackish way to make top_level.txt contain the 'right'
package name, so I think I'm not supposed to touch it, or even know that
it's there.

I wasn't able to find much documentation on this top_level.txt file, but
what I found suggested that it's used for run-time conflict resolution, not
for package management. [1]

So my question is, how to make this scenario work? Just use different
top-level package names and install them side-by-side? Looking at big
projects such as Zope, Django or Twisted, they all seem to have gone the
non-distutils route.

Before anyone asks why I want this; I want to split up some framework-type
code and some utility-type code I have from some application-specific code,
and I hope that this way I'll be able to accurately specify and install the
dependencies between projects without running ever more risk of clashing
with a top-level module name. But I'm open to the suggestion that my idea
is totally misguided. :-)

Salutation,

Joost Molenaar

[1] http://svn.python.org/projects/sandbox/trunk/setuptools/doc/formats.txt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20121030/d34d8f1d/attachment.html>