[SciPy-dev] State of stats modules?

[eric]

Hey Mangus,

About stats.py:

Most of the stats module was written by Gary Strangman
(http://www.nmr.mgh.harvard.edu/Neural_Systems_Group/gary/).  As far as I
know, it's the most full featured stats module around, and has very good
doc-strings.  Gary developed it for his own research, and, as such, it is
somewhat specialized to his field.  Still, it is very usable, and, at least
for the functions I have used, reliable.


Gary's work was/is an excellent starting point for SciPy's statistics
capabilities.  Most of the work needed is actually trimming out extra
functionality not needed or duplicated, adding unit test functions, and
assuring that functions behave similarly in calling convention to other
Numeric/SciPy functions.  The new_stats.py module is the beginnings of this
effort, but it hasn't had any attention in a while.  There are also the
beginnings of some unit testing in the stats/tests directory.  Hopefully a
full compliment of unit tests will develop so there are fewer questions
about result vailidity.

aanova and collapse:

I haven't used these, and don't know much about them.  I'll forward this to
Gary and see if he has any comments.

> Anyone caring for this?


The stats module deserves some attention, but isn't receiving any right now.
Any takers?

see ya,
eric


----- Original Message -----
From: "Magnus Lyckå" <magnus at thinkware.se>
To: <scipy-user at scipy.net>; <scipy-dev at scipy.net>
Sent: Saturday, November 03, 2001 9:47 AM
Subject: [SciPy-dev] State of stats modules?


> Hi, I'm a new scipy user, and I'm still a little lost...
> I've looked through the mailing list archeives and
> the web site, but I guess I might have missed some info
> I should have spotted... Anyway, I'll ask some questions
> here... I start with this one:
>
> What's the state of the stats module?
>
> Just at random, I tried the aanova function and get:
>
> Traceback (most recent call last):
>    File "<interactive input>", line 1, in ?
>    File "G:\Python21\scipy\stats\stats.py", line 3933, in aanova
>      M = pstat.collapse(data,Bscols,-1,None,None,mean)
> TypeError: collapse() takes at most 5 arguments (6 given)
>
> collapse wants:
> def collapse (listoflists,keepcols,collapsecols,sterr=None,ns=None):
>
> In other words... No mean...
>
> Removing the last parameter in lines 3933, 3988 and 4372 I get
> a result, but I won't hang my neck out and say that it's correct
> (or not).
>
> Also, what happens is that the aanova function uses "print" to
> display the result. Fine for interactive use if I just want to
> see how things stand, but less amusing if I want to use the
> results from the anova programmatically in further computations,
> or for plotting or what ever. The return value from the function
> is None. :-(
>
> There is obviously code in ststs.py that assumes that collapse can
> work with other stuff than means, as it seems to do now:
>
> collapsed = pstat.collapse(M,btwcols,-1,None,len,mean)
> # Obviously needed for-loop to get source cell-means embedded in collapse
fcns
> contrastmns = pstat.collapse(collapsed,btwsourcecols,-2,sterr,len,mean)
> # Collapse again, this time SUMMING instead of averaging (to get cell Ns)
> contrastns = pstat.collapse(collapsed,btwsourcecols,-1,None,None,
>                              sum)
> # Collapse again, this time calculating harmonicmeans (for hns)
> contrasthns = pstat.collapse(collapsed,btwsourcecols,-1,None,None,
>                               harmonicmean)
>
> I assume that it's the following snippet in pstat.collapse that should
> use the sixth parameter:
>
>      if keepcols == []:
>          means = [0]*len(collapsecols)
>          for i in range(len(collapsecols)):
>              avgcol = colex(listoflists,collapsecols[i])
>              means[i] = stats.mean(avgcol) <=== supplied function here?
>          return means
>
> But I don't feel that I understand enough of statistics or the plan here
to
> patch things on my own...
>
> Anyone caring for this?
>
>
> --
> Magnus Lyckå, Thinkware AB
> Älvans väg 99, SE-907 50 UMEÅ
> tel 070-582 80 65, fax: 070-612 80 65
> http://www.thinkware.se/  mailto:magnus at thinkware.se
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.net
> http://www.scipy.net/mailman/listinfo/scipy-dev
>