[SciPy-User] How to ignore NaN values and -32767 in numpy array

[questions anon]

Thank you, what you suggested worked but now I don't think that is my
problem.
Within the dataset I am trying to calculate the mean from it appears there
are some hours with no data, the output is:

[[[-- -- -- ..., -- -- --]
  [-- -- -- ..., -- -- --]
  [-- -- -- ..., -- -- --]
  ...,
  [-- -- -- ..., -- -- --]
  [-- -- -- ..., -- -- --]
  [-- -- -- ..., -- -- --]]]

So I would assume these would be ignored when I calculate the mean but when
I make all my files/times into one big array these blanks turn into -32767.
Is there some way to avoid this?
Thanks


On Wed, Aug 17, 2011 at 4:17 PM, questions anon <questions.anon at gmail.com>wrote:

> I am trying to run simple stats on a bunch of monthly netcdfs files with
> hourly temperature data. With help from this list I am able to loop through
> a calculate the mean, but in doing this I have discovered that there are a
> some hours that have no values or -32767. I am sure there are some cases
> where I could slice out the section (if I know where they are) but is there
> a way I could just ignore these hours and calculate the mean?
> I have found something called "numpy.isnan" but this does not seem to work.
>
>
from netCDF4 import Dataset
> import matplotlib.pyplot as plt
> import numpy as N
> from mpl_toolkits.basemap import Basemap
> import os
>
> MainFolder=r"E:/temp_samples/"
>
> all_TSFC=[]
> for (path, dirs, files) in os.walk(MainFolder):
>     for dir in dirs:
>         print dir
>     path=path+'/'
>     for ncfile in files:
>         if ncfile[-3:]=='.nc':
>             ncfile=os.path.join(path,ncfile)
>             ncfile=Dataset(ncfile, 'r+', 'NETCDF4')
>             TSFC=ncfile.variables['T_SFC'][:]
>             LAT=ncfile.variables['latitude'][:]
>             LON=ncfile.variables['longitude'][:]
>             TIME=ncfile.variables['time'][:]
>             fillvalue=ncfile.variables['T_SFC']._FillValue
>             ncfile.close()
>
> #combine all TSFC to make one array for analyses
>             all_TSFC.append(TSFC)
>
> big_array=N.concatenate(all_TSFC)
> Mean=big_array.mean(axis=0)
> print "the mean is", Mean
>
>  #plot output summary stats
> map = Basemap(projection='merc',llcrnrlat=-40,urcrnrlat=-33,
>
> llcrnrlon=139.0,urcrnrlon=151.0,lat_ts=0,resolution='i')
> x,y=map(*N.meshgrid(LON,LAT))
> CS = map.contourf(x,y,Mean, cmap=plt.cm.jet)
> l,b,w,h =0.1,0.1,0.8,0.8
> cax = plt.axes([l+w+0.025, b, 0.025, h])
> plt.colorbar(CS,cax=cax, drawedges=True)
>
> plt.show()
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20110819/f019bd0d/attachment.html>