[Numpy-discussion] np.ma.mean is not working?
Chao YUE
chaoyuejoy at gmail.com
Tue Oct 18 17:25:06 EDT 2011
I would say pandas is really cool. More people need to know it. and we
should have better documentation.
cheers,
Chao
2011/10/18 Bruce Southey <bsouthey at gmail.com>
> **
> On 10/18/2011 09:12 AM, Chao YUE wrote:
>
> thanks. Olivier. I see.
>
> Chao
>
> 2011/10/18 Olivier Delalleau <shish at keba.be>
>
>> As far as I can tell ma.mean() is working as expected here: it computes
>> the mean only over non-masked values.
>> If you want to get rid of any mean that was computed over a series
>> containing masked value you can do:
>>
>> b = a.mean(0)
>> b.mask[a.mask.any(0)] = True
>>
>> Then b will be:
>>
>> masked_array(data = [5.0 -- -- 8.0 9.0 -- 11.0 12.0 -- 14.0],
>> mask = [False True True False False True False False True
>> False],
>> fill_value = 1e+20)
>>
>> -=- Olivier
>>
>> 2011/10/18 Chao YUE <chaoyuejoy at gmail.com>
>>
>>> Dear all,
>>>
>>> previoulsy I think np.ma.mean() will automatically filter the masked
>>> (missing) value but it's not?
>>> In [489]: a=np.arange(20.).reshape(2,10)
>>>
>>> In [490]:
>>> a=np.ma.masked_array(a,(a==2)|(a==5)|(a==11)|(a==18),fill_value=np.nan)
>>>
>>> In [491]: a
>>> Out[491]:
>>> masked_array(data =
>>> [[0.0 1.0 -- 3.0 4.0 -- 6.0 7.0 8.0 9.0]
>>> [10.0 -- 12.0 13.0 14.0 15.0 16.0 17.0 -- 19.0]],
>>> mask =
>>> [[False False True False False True False False False False]
>>> [False True False False False False False False True False]],
>>> fill_value = nan)
>>>
>>> In [492]: a.mean(0)
>>> Out[492]:
>>> masked_array(data = [5.0 1.0 12.0 8.0 9.0 15.0 11.0 12.0 8.0 14.0],
>>> mask = [False False False False False False False False
>>> False False],
>>> fill_value = 1e+20)
>>>
>>> In [494]: np.ma.mean(a,0)
>>> Out[494]:
>>> masked_array(data = [5.0 1.0 12.0 8.0 9.0 15.0 11.0 12.0 8.0 14.0],
>>> mask = [False False False False False False False False
>>> False False],
>>> fill_value = 1e+20)
>>>
>>> In [495]: np.ma.mean(a,0)==a.mean(0)
>>> Out[495]:
>>> masked_array(data = [ True True True True True True True True
>>> True True],
>>> mask = False,
>>> fill_value = True)
>>>
>>> only use a.filled().mean(0) can I get the result I want:
>>> In [496]: a.filled().mean(0)
>>> Out[496]: array([ 5., NaN, NaN, 8., 9., NaN, 11., 12., NaN,
>>> 14.])
>>>
>>> I am doing this because I tried to have a small fuction from the web to
>>> do moving average for data:
>>>
>>> import numpy as np
>>> def rolling_window(a, window):
>>> if window < 1:
>>> raise ValueError, "`window` must be at least 1."
>>> if window > a.shape[-1]:
>>> raise ValueError, "`window` is too long."
>>> shape = a.shape[:-1] + (a.shape[-1] - window + 1, window)
>>> strides = a.strides + (a.strides[-1],)
>>> return np.lib.stride_tricks.as_strided(a, shape=shape,
>>> strides=strides)
>>>
>>> def move_ave(a,window):
>>> temp=rolling_window(a,window)
>>> pre=int(window)/2
>>> post=int(window)-pre-1
>>> return
>>> np.concatenate((a[...,0:pre],np.mean(temp,-1),a[...,-post:]),axis=-1)
>>>
>>>
>>> In [489]: a=np.arange(20.).reshape(2,10)
>>>
>>> In [499]: move_ave(a,4)
>>> Out[499]:
>>> masked_array(data =
>>> [[ 0. 1. 1.5 2.5 3.5 4.5 5.5 6.5 7.5 9. ]
>>> [ 10. 11. 11.5 12.5 13.5 14.5 15.5 16.5 17.5 19. ]],
>>> mask =
>>> False,
>>> fill_value = 1e+20)
>>>
>>> thanks,
>>>
>>> Chao
>>>
>>> --
>>>
>>> ***********************************************************************************
>>> Chao YUE
>>> Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL)
>>> UMR 1572 CEA-CNRS-UVSQ
>>> Batiment 712 - Pe 119
>>> 91191 GIF Sur YVETTE Cedex
>>> Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16
>>>
>>> ************************************************************************************
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
>
> --
>
> ***********************************************************************************
> Chao YUE
> Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL)
> UMR 1572 CEA-CNRS-UVSQ
> Batiment 712 - Pe 119
> 91191 GIF Sur YVETTE Cedex
> Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16
>
> ************************************************************************************
>
>
> _______________________________________________
> NumPy-Discussion mailing listNumPy-Discussion at scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> Looked at pandas for your rolling window functionality:
> http://pandas.sourceforge.net
>
> *"Time series*-specific functionality: date range generation and frequency conversion, moving window statistics, moving window linear regressions, date shifting and lagging, etc."
>
> Bruce
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
--
***********************************************************************************
Chao YUE
Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL)
UMR 1572 CEA-CNRS-UVSQ
Batiment 712 - Pe 119
91191 GIF Sur YVETTE Cedex
Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16
************************************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20111018/c06486c4/attachment.html>
More information about the NumPy-Discussion
mailing list