[SciPy-dev] Some thoughts on packaging SciPy

[Prabhu Ramachandran]

hi,

While I believe that its nice to have everything in one big package, I
dont think that it fits the bill all the time.  For instance I think
weave, all the plot utilities, gui_thread and scipy_distutils and
others do not belong in the scipy namespace.  They are useful in
scopes far beyond scipy.  This is surely obvious to you all but I'd
like to belabour it.  scipy is of use to scientists and engineers but
even we do not do scientific programming all the time.  Many modules
have use outside the scientific world.  These should not be locked
into scipy.  OTOH, anything that is related to science/numerics should
be part of scipy.  It only makes sense.  Consider plt.  I'm sure its
possible that a sys admin would like to use plt to plot stuff (maybe
log files?).  There is nothing scipy specific to wanting to plot data.
Similarly with weave and maybe others.  These are modules that should
not be part of the scipy package.  However, they should definitely be
bundled as part of the scipy distribution.  Hence the distinction
between the scipy python package/namespace and the scipy distribution.
I think they are different things and need to be noted.

However, as I write this I realize that one viewpoint would be, "Why
dont you just install the whole of scipy and just use what you want?"
The problem with this approach is that its not possible to install
scipy without other requirements.  If someone wants just plt, they'd
have to figure out the atlas related issues or any others that will
arise in the future (and these will not be easy issues).  I think
there is only one way of dealing with this.

We need a tool like cpan or ciphon.  Or maybe not even that.
Something like this might suffice.

 (1) Each module has a set of dependencies which it advertises in some
 file.

 (2) setup.py has a config file (or command line switches) that lets
 you select what packages you want installed.

 (3) setup.py also does not install anything that fails dependencies.
 For instance if someone wanted plt alone they select plt. plt ropes
 in gui_thread and requires wxPython.  If wxPython is avlbl. plt and
 gui_thread are installed if not it is not installed.  So the users
 can pick and choose what they want and install things with the least
 hassle.

Once this is done it really does not matter if the modules are
installed inside the scipy python package/namespace or outside it.
The idea is somewhat like rpms and debs.  There is one big source
package from which smaller components are installable separately.

I think this is definitely implementable but dont know how hard it
should be to do.

My apologies for not contributing any code that addresses these
things.  I thought that the least I can do is contribute ideas.

prabhu