[Numpy-discussion] Medians that ignore values

[David Cournapeau]

Peter Saffrey wrote:
>
> I've posted my test code below, which gives me the results:
>
> $ ./arrayspeed3.py
> list build time: 0.01
> list median time: 0.01
> array nanmedian time: 0.36
>
> I must have done something wrong to hobble nanmedian in this way... I'm quite
> new to numpy, so feel free to point out any obviously egregious errors.

Ok: it is "pathological", and can be done better :)

First:

> for natest in natests:
> 	thismed = nanmedian(natest, axis=1)
> 	namedians.append(thismed)

^^^ Here, you are doing nanmedian on a direction with 3 elements: this
will be slow in numpy, because numpy involves some relatively heavy
machinery to run on arrays. The machinery pays off for 'big' arrays, but
for really small arrays like here, list can (and often are) be faster.

Still, it is indeed really slow for your case; when I fixed nanmean and
co, I did not know much about numpy, I just wanted them to give the
right answer :) I think this can be made faster, specially for your case
(where the axis along which the median is computed is really small).

I opened a bug:

http://scipy.org/scipy/scipy/ticket/740

cheers,

David