[SciPy-dev] MCMC, Kalman Filtering, AI for SciPy?

[Robert Kern]

Charles Harris wrote:

[snip]

> I agree that search and indexing are the best ways to find stuff, but I 
> am mostly concerned as to where to commit stuff. Clustering, where does 
> that go?

scipy.cluster I would imagine.  ;-)

> Lattice methods, where do they go? How about useful data 
> structures or combinatorics? So on and so forth. I think the upper level 
> GAMS categories cover sufficient range that most things can be put into 
> a directory without embarrassment. As to the detailed breakdown in the 
> GAMS sub-classifications, I am not so sure.

To make the discussion a bit more concrete, here is an example directory 
structure corresponding to the top-level GAMS classifications. The names 
are all my own, so feel free to pretend they are something more to your 
liking.

scipy/
   analysis/
   numbertheory/
   functions/
   linalg/
   interpolation/
   rootfinding/
   optimization/
   calculus/
   diffeq/
   integraltransforms/
   approximation/
   probstat/
   simulation/
   datahandling/
   symbolic/
   geometry/
   graphics/
   service/
   develop/
   other/

Now that I see it, it is somewhat appealing. I would probably want to 
break up some of those into two or more top-level groups. I definitely 
don't want to see too many subpackages under each of the top-level 
groups ("Flat is better than nested.").

Fernando, could you give an example or two where you would want to 
replicate a function across sub-packages? I'm wary of doing so as there 
is already the enormous amount of replication with respect to, at least, 
the base Numeric functions. Try scipy.special.<tab> in IPython. I 
realize what you're proposing doesn't even come close to that, but I'd 
like an example in any case.

And since we are talking about re-organization, is there anything we can 
do about the problem I just mentioned? It wreaks havoc with not only 
tab-completion but also automatic documentation generation [1]. Is it 
practical to be careful about what we import into __init__.py? By which 
I mean not doing "from foo import *" in __init__.py where foo.py does 
"from scipy_base import *". On the other hand, explicitly listing all of 
the names in special is gonna be a major pain and fragile to boot.

[1] http://www.scipy.org/documentation/apidocs/scipy/scipy.special.html

-- 
Robert Kern
rkern at ucsd.edu

"In the fields of hell where the grass grows high
  Are the graves of dreams allowed to die."
   -- Richard Harter