[Numpy-discussion] np.ma.mean is not working?

[Bruce Southey]

On 10/18/2011 09:12 AM, Chao YUE wrote:
> thanks. Olivier. I see.
>
> Chao
>
> 2011/10/18 Olivier Delalleau <shish at keba.be <mailto:shish at keba.be>>
>
>     As far as I can tell ma.mean() is working as expected here: it
>     computes the mean only over non-masked values.
>     If you want to get rid of any mean that was computed over a series
>     containing masked value you can do:
>
>     b = a.mean(0)
>     b.mask[a.mask.any(0)] = True
>
>     Then b will be:
>
>     masked_array(data = [5.0 -- -- 8.0 9.0 -- 11.0 12.0 -- 14.0],
>                  mask = [False  True  True False False  True False
>     False  True False],
>            fill_value = 1e+20)
>
>     -=- Olivier
>
>     2011/10/18 Chao YUE <chaoyuejoy at gmail.com
>     <mailto:chaoyuejoy at gmail.com>>
>
>         Dear all,
>
>         previoulsy I think np.ma.mean() will automatically filter the
>         masked (missing) value but it's not?
>         In [489]: a=np.arange(20.).reshape(2,10)
>
>         In [490]:
>         a=np.ma.masked_array(a,(a==2)|(a==5)|(a==11)|(a==18),fill_value=np.nan)
>
>         In [491]: a
>         Out[491]:
>         masked_array(data =
>          [[0.0 1.0 -- 3.0 4.0 -- 6.0 7.0 8.0 9.0]
>          [10.0 -- 12.0 13.0 14.0 15.0 16.0 17.0 -- 19.0]],
>                      mask =
>          [[False False  True False False  True False False False False]
>          [False  True False False False False False False  True False]],
>                fill_value = nan)
>
>         In [492]: a.mean(0)
>         Out[492]:
>         masked_array(data = [5.0 1.0 12.0 8.0 9.0 15.0 11.0 12.0 8.0
>         14.0],
>                      mask = [False False False False False False False
>         False False False],
>                fill_value = 1e+20)
>
>         In [494]: np.ma.mean(a,0)
>         Out[494]:
>         masked_array(data = [5.0 1.0 12.0 8.0 9.0 15.0 11.0 12.0 8.0
>         14.0],
>                      mask = [False False False False False False False
>         False False False],
>                fill_value = 1e+20)
>
>         In [495]: np.ma.mean(a,0)==a.mean(0)
>         Out[495]:
>         masked_array(data = [ True  True  True  True  True  True 
>         True  True  True  True],
>                      mask = False,
>                fill_value = True)
>
>         only use a.filled().mean(0) can I get the result I want:
>         In [496]: a.filled().mean(0)
>         Out[496]: array([  5.,  NaN,  NaN,   8.,   9.,  NaN,  11., 
>         12.,  NaN,  14.])
>
>         I am doing this because I tried to have a small fuction from
>         the web to do moving average for data:
>
>         import numpy as np
>         def rolling_window(a, window):
>             if window < 1:
>                 raise ValueError, "`window` must be at least 1."
>             if window > a.shape[-1]:
>                 raise ValueError, "`window` is too long."
>             shape = a.shape[:-1] + (a.shape[-1] - window + 1, window)
>             strides = a.strides + (a.strides[-1],)
>             return np.lib.stride_tricks.as_strided(a, shape=shape,
>         strides=strides)
>
>         def move_ave(a,window):
>             temp=rolling_window(a,window)
>             pre=int(window)/2
>             post=int(window)-pre-1
>             return
>         np.concatenate((a[...,0:pre],np.mean(temp,-1),a[...,-post:]),axis=-1)
>
>
>         In [489]: a=np.arange(20.).reshape(2,10)
>
>         In [499]: move_ave(a,4)
>         Out[499]:
>         masked_array(data =
>          [[  0.    1.    1.5   2.5   3.5   4.5   5.5   6.5   7.5   9. ]
>          [ 10.   11.   11.5  12.5  13.5  14.5  15.5  16.5  17.5  19. ]],
>                      mask =
>          False,
>                fill_value = 1e+20)
>
>         thanks,
>
>         Chao
>
>         -- 
>         ***********************************************************************************
>         Chao YUE
>         Laboratoire des Sciences du Climat et de l'Environnement
>         (LSCE-IPSL)
>         UMR 1572 CEA-CNRS-UVSQ
>         Batiment 712 - Pe 119
>         91191 GIF Sur YVETTE Cedex
>         Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16
>         ************************************************************************************
>
>
>         _______________________________________________
>         NumPy-Discussion mailing list
>         NumPy-Discussion at scipy.org <mailto:NumPy-Discussion at scipy.org>
>         http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
>     _______________________________________________
>     NumPy-Discussion mailing list
>     NumPy-Discussion at scipy.org <mailto:NumPy-Discussion at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
>
> -- 
> ***********************************************************************************
> Chao YUE
> Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL)
> UMR 1572 CEA-CNRS-UVSQ
> Batiment 712 - Pe 119
> 91191 GIF Sur YVETTE Cedex
> Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16
> ************************************************************************************
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
Looked at pandas for your rolling window functionality:
http://pandas.sourceforge.net

*"Time series*-specific functionality: date range generation and frequency conversion, moving window statistics, moving window linear regressions, date shifting and lagging, etc."

Bruce



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20111018/4774552e/attachment.html>