[SciPy-dev] MCMC, Kalman Filtering, AI for SciPy?

[Charles Harris]

From: scipy-dev-bounces at scipy.net on behalf of eric jones
> 
>Robert Kern wrote:

>> Charles Harris wrote:
>>
>> [snip]
>>>
>>> I agree that search and indexing are the best ways to find stuff, but 
>>> I am mostly concerned as to where to commit stuff. Clustering, where 
>>> does that go?
>>
>>
>> scipy.cluster I would imagine.  ;-)
>>
>>> Lattice methods, where do they go? How about useful data structures 
>>> or combinatorics? So on and so forth. I think the upper level GAMS 
>>> categories cover sufficient range that most things can be put into a 
>>> directory without embarrassment. As to the detailed breakdown in the 
>>> GAMS sub-classifications, I am not so sure.
>>
>>
>> To make the discussion a bit more concrete, here is an example 
>> directory structure corresponding to the top-level GAMS 
>> classifications. The names are all my own, so feel free to pretend 
>> they are something more to your liking.
>>
>> <snip hierarchy>

>> Now that I see it, it is somewhat appealing. I would probably want to 
>> break up some of those into two or more top-level groups. I definitely 
>> don't want to see too many subpackages under each of the top-level 
>> groups ("Flat is better than nested.").
>>
>>Here is where the current SciPy modules would likely get lumped in the 
>>GAMS hierarchy.

>scipy/
>  analysis/
>  numbertheory/
>  functions/           special
>  linalg/              linalg, sparse
>  interpolation/       interpolate
>  rootfinding/
>  optimization/        optimize, ga
>  calculus/            integrate
>  diffeq/             
>  integraltransforms/  fftpack
>  approximation/
>  probstat/            stats
>  simulation/
>  datahandling/        io
>  symbolic/
>  geometry/
>  graphics/            xplt, gplt, plt
>  service/             gui_thread
>  develop/
>  other/               cow, cluster ??, signal  ??
 
>(Cluster and signal didn't fit anywhere obvious to me)

Gams has clustering under probstat. The union/find algorithm could also go under data
where they have trees, etc. Lattice stuff goes into numbertheory. Service also contains
the machine parameters, eps and such. Remez algorithm goes into approximation (L_inf). Weave
into develop. Hmm, signal processing is missing somewhere. Markov chain into simulation.
Grey codes, permutations into other, although GAMS has permutations under data.

We could split diffeq into ode, pde, but it is probably ok as is. Control is under 
optimization (for optimal control) but could be brought to the top. The addition of
rootfinding is good. Convolutions and such are under integraltransforms, Haar transforms
would go there also. Filters are under probstats in time series analysis, so it might make
sense to create a signal (time series?) directory, probstats seems to be an overloaded
GAMS category that could use some upperlevel subdivision. 


>The naming conventions are often quite similar.  The SciPy names are 
>generally shorter which is nice for typing.  Where SciPy has multiple 
>packages [(linalg, sparse), (optimize, ga), etc.], it is likely a good 
>idea.  Like you, I don't want to see a deep nesting in the package 
>structure. 

>Looking at this, I don't see any real reason to reorganize top level 
>package names.  Are any of them that bad or misleading?  On the other 
>hand, I do think we should reorganize the functions within them some to 
>fix the places where they are organized based on "build" convenience 
>instead of actual function.  This will probably necessitate the addition 
>of new top level groups and maybe the pruning of one of the current 
>ones.  I've made a Wiki page to keep suggestions that people have:

>    http://www.scipy.org/wikis/featurerequests/PackageReorganization

>If you update the page, you might also post to python-dev so that people 
>know to go check on the Wiki (that is so painful...).  We can obviously 
>also just discuss it here and then transfer to the Wiki later. [side 
>note: this using a wiki and a mailing list for communication is also a 
>little painful].

>> Fernando, could you give an example or two where you would want to 
>> replicate a function across sub-packages? I'm wary of doing so as 
>> there is already the enormous amount of replication with respect to, 
>> at least, the base Numeric functions. Try scipy.special.<tab> in 
>> IPython. I realize what you're proposing doesn't even come close to 
>> that, but I'd like an example in any case.

>I don't like the replication idea very well.  I think things should live 
>in one place.  Otherwise people will wonder if two functions that are 
>actually the same have different purposes, implementation, etc.

>>
>> And since we are talking about re-organization, is there anything we 
>> can do about the problem I just mentioned? It wreaks havoc with not 
>> only tab-completion but also automatic documentation generation [1]. 
>> Is it practical to be careful about what we import into __init__.py? 
>> By which I mean not doing "from foo import *" in __init__.py where 
>> foo.py does "from scipy_base import *". On the other hand, explicitly 
>> listing all of the names in special is gonna be a major pain and 
>> fragile to boot.
 
<snip>


_______________________________________________
Scipy-dev mailing list
Scipy-dev at scipy.net
http://www.scipy.net/mailman/listinfo/scipy-dev


-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 5402 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20040930/8f123fff/attachment.bin>