[Numpy-discussion] loop through values in a array and find maximum as looping

[Derek Homeier]

On 07.12.2011, at 5:07AM, Olivier Delalleau wrote:

> I *think* it may work better if you replace the last 3 lines in your loop by:
> 
>             a=all_TSFC[0]
>             if len(all_TSFC) > 1:
>                 N.maximum(a, TSFC, out=a)
> 
> Not 100% sure that would work though, as I'm not entirely confident I understand your code.
> 
> -=- Olivier
> 
> 2011/12/6 questions anon <questions.anon at gmail.com>
> Something fancier I think, 
> I am able to compare the result with my previous method so I can easily see I am doing something wrong.
> see code below:
> 
> 
> all_TSFC=[]
> for (path, dirs, files) in os.walk(MainFolder):
>     for dir in dirs:
>         print dir
>     path=path+'/'
>     for ncfile in files:
>         if ncfile[-3:]=='.nc':
>             print "dealing with ncfiles:", ncfile
>             ncfile=os.path.join(path,ncfile)
>             ncfile=Dataset(ncfile, 'r+', 'NETCDF4')
>             TSFC=ncfile.variables['T_SFC'][:]
>             fillvalue=ncfile.variables['T_SFC']._FillValue
>             TSFC=MA.masked_values(TSFC, fillvalue)
>             ncfile.close()
>             all_TSFC.append(TSFC)
>             a=TSFC[0]
>             for b in TSFC[1:]:
>                 N.maximum(a,b,out=a)
> 
I also understood TSFC is already the array you want to work on, so above 
you'd just take a slice and overwrite the result in the next file iteration anyway. 
Iterating over the list all_TSFC should be correct, but I understood you 
don't want to load the entire input into memory in you working code.
Then you can simply skip the list, just need to take care of initial conditions - 
something like the following should do:

    path=path+'/'
    a = None
    for ncfile in files:
        if ncfile[-3:]=='.nc':
            print "dealing with ncfiles:", ncfile
            ncfile=os.path.join(path,ncfile)
            ncfile=Dataset(ncfile, 'r+', 'NETCDF4')
            TSFC=ncfile.variables['T_SFC'][:]
            fillvalue=ncfile.variables['T_SFC']._FillValue
            TSFC=MA.masked_values(TSFC, fillvalue)
            ncfile.close()
            if not is instance(a,N.ndarray):
                a=TSFC
            else:
                N.maximum(a, TSFC, out=a)

HTH,
					Derek

> big_array=N.ma.concatenate(all_TSFC)
> Max=big_array.max(axis=0)
> print "max is", Max,"a is", a
>