[SciPy-dev] Restructuring of SciPy

[Perry Greenfield]

On Aug 17, 2005, at 4:31 PM, Travis Oliphant wrote:

>
> After the 0.3.2 release, scipy will be getting a face-lift.
>
> As part of that, I'm wondering about the restructuring process.  For a 
> long time, we've been operating under the "big-scipy-package" with 
> capability for small packages to be released separately model.  It has 
> had limited success and I don't know of any small packages actually 
> being released like that.
>
> With things like Enthon out there that act as the "batteries included 
> super distribution" anyway.  It makes me wonder if we shouldn't move 
> scipy to the "scipy-modules" perspective directly and develop and 
> think about scipy has a collection of modules that work together 
> smoothly (i.e. naming conventions, etc.)
>
> scipy_core + available instructions could provide all the 
> infrastructure someone needs to contribute packages to scipy.  This 
> would also allow people to develop (and maintain) their own modules 
> (which they seem to like to do anyway).  If we could just get them to 
> play nice with the rest of scipy_core, it would all be for benefit.
>
> We already have the infrastructure in place for modules to play nicely 
> with the overall package.  I'm just wondering if we shouldn't 
> emphasize more the modular approach by the way the svn tree is layed 
> out (so individual modules can be checked out independently, for 
> example).
>
> I've also spoken to people who find it hard to rely on their scipy in 
> their code they pass on because you have to get the whole package, 
> even if you only use a few parts.
> Matlab also uses modules effectively.
> I think we should take advantage of the restructuring process to think 
> about this.
>
> Comments?
>
> -Travis

It's probably already obvious what I think but I don't see any harm in 
reiterating it.

I think the biggest obstacle to scipy in its current form is that many 
users find it hard to install. Yes, some of you may have no problems, 
but I've run into many that have (and some of them are pretty 
experienced at doing installations). It is essential that whatever the 
core is, it be an easy install for the vast majority of users. I think 
Travis is on the right track here. My only variance is that I wouldn't 
object to including more popular modules in the core distribution *so 
long as they are easy installs on all popular platforms*. Anything that 
isn't shouldn't be in the core.

But there are other good reasons for splitting them that Travis alludes 
to. Some module may have a faster pace of development and don't want to 
be synced to scipy.core releases. Not everything will be at the same 
level of maturity, testing, or documentation. The core should have 
higher standards than what optional models have in that respect.

Also, nothing prevents larger binary bundles (assuming someone is 
willing to put the work into making them).

But there is no denying that doing this introduces dependency issues 
that must be managed as well.

Perry