[Numpy-discussion] Using matplotlib's prctile on masked arrays

[Gökhan Sever]

On Tue, Oct 27, 2009 at 8:25 AM, <josef.pktd at gmail.com> wrote:

> This should not be the correct results if you use
> scipy.stats.scoreatpercentile,
> it doesn't have correct missing value handling, it treats nans or
> mask/fill values as regular numbers sorted to the end.
>
> stats.mstats.scoreatpercentile  is the corresponding function for
> masked arrays.
>
>
Thanks for the suggestion. I forgot the existence of such module. It yields
better results.

I[14]: st.mstats.scoreatpercentile(r, per=25)
O[14]:
masked_array(data = 0.401055201111,
             mask = False,
       fill_value = 1e+20)

I[17]: st.scoreatpercentile(r, per=25)
O[17]:
masked_array(data = --,
             mask = True,
       fill_value = 1e+20)

I usually fall into traps using masked arrays. Hopefully I will figure out
these before I make funnier mistakes in my analysis.

Besides, it would be nice to have the "per" argument accepts a sequence
instead of a one item. Like matplotlib's prctile. Using it as: ...(array,
per=[5,25,50,75,95]) in a one call.


> (BTW I wasn't able to quickly copy and past your example because
> MaskedArrays don't seem to have a constructive __repr__, i.e.
> no commas)
>
>
You can copy and paste the sample data from this link. When I copied from a
txt file into gmail into somehow distorted the original look of the data.

http://code.google.com/p/ccnworks/source/browse/trunk/sample.data



> I don't know anything about the matplotlib story.
>
> Josef
>
> >
> > I[55]: stats.scoreatpercentile(am/bm, per=5)
> > O[55]: 0.40877012449846228
> >
> > I[49]: stats.scoreatpercentile(am/bm, per=25)
> > O[49]:
> > masked_array(data = --,
> >              mask = True,
> >        fill_value = 1e+20)
> >
> > I[56]: stats.scoreatpercentile(am/bm, per=95)
> > O[56]:
> > masked_array(data = --,
> >              mask = True,
> >        fill_value = 1e+20)
> >
> >
> > Any confirmation?
> >
> >
> >
> >
> >
> >
> >
> > --
> > Gökhan
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
Gökhan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20091028/9ec81ce1/attachment.html>