[SciPy-dev] Package organization

[Robert Kern]

Arnd Baecker wrote:
> On Thu, 13 Oct 2005, Fernando Perez wrote:
> 
> [...]
> 
>>3.  The scipy.{kits|tools} namespace (or whatever the chosen name).  This is
>>where third parties can drop their own packages, which can depend either only
>>(1) or on the full (2) system (their level of dependency should be explicitly
>>stated).
>>
>>The kits namespace may ship empty by default, or it could be populated with a
>>few things from current scipy if it is decided they are best moved there.
>>
>>The only thing required for projects to live in the .kits namespace is really
>>to avoid top-level name collisions, so it would perhaps be worth having an
>>informal policy of people checking with scipy-dev for a name before using it.
>>
>>This layout would allow the core team to work with relative freedom at the
>>top-level namespace, without worrying about toolkits taking names they may
>>need in the future.  Similarly, toolkit authors will have a well-defined API
>>to build upon.
> 
> To me this looks as if there is essentially no difference
> to people distributing their projects independently of scipy
> (and outside of its name-space).
> I still like this for two reasons
> a) it would bring scientific python projects under one umbrella
> b) the (hopefully) simple user installation:

I'm not entirely convinced that everything needs to live in the scipy
namespace, though. I think I overstated the build-time benefits of being
in scipy itself. I think we can make most of those benefits accessible
to packages that aren't living in scipy.* . I haven't been keeping up
with the AstroPy discussions, but I doubt they'd really want to do

  from scipy.kits.astropy import wcs

etc. "Flat is better than nested," and all that.

I *think* a package can get all the benefits of "being in scipy" simply
by depending on parts of scipy and using the tools appropriately. But to
really figure this out, we need to come down to cases. Let's get our
house in order first, and then we can talk about how we deal with guests.

>>If this is combined with a CPAN-like system (eggs, PyPi, whatever), it should
>>be very easy for users, once they have the basic layers in place, to grab a
>>toolkit by issuing a single command or going to a website.  I'd suggest, if
>>this were adopted, keeping a simple page at scipy with brief descriptions for
>>each toolkit, even if they are developed/distributed externally.
> 
> Sounds very nice! I have no experience with eggs:
> where do these go when a user cannot install them as root?
> (Of course, a  PYTHONPATH modification should do the job.)

Primarily, you just add the egg itself to sys.path in any of the various
ways one might do that. If the egg isn't "zip-safe" because it needs to
access real files-on-the-filesystem, then you need to unpack the egg
first. There's a tool built on top of eggs called easy_install which
manages a .pth file to control which packages are enabled. This scheme
will be changing to something more flexible and controllable shortly,
I'm told. Yes, Fernando, this change should solve all of the problems
you and Prabhu have been having with eggs. I think. I hope.

http://peak.telecommunity.com/DevCenter/PythonEggs
http://peak.telecommunity.com/DevCenter/EasyInstall
http://peak.telecommunity.com/DevCenter/setuptools

-- 
Robert Kern
rkern at ucsd.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter