[SciPy-User] Scipy.io.netcdf: does nc.close release memory?

Julien Salort lists at juliensalort.org
Wed Jul 23 06:20:12 EDT 2014


Aronne Merrelli <aronne.merrelli at gmail.com> wrote:

> I think this is supposed to be safe, but it is usually the case that the
> variable returned from:
> 
> >> var = nc.variables['varname'].data
> 
> ultimately points to a "memmap" object. Try checking the .base attribute of
> the ndarray variable, to see if this is the case. You may need to go a
> couple levels to see the memmap (e.g., var.base.base). Since that seems to
> be causing problems, you could try something like the following:
> 
> >> var = nc.variables['varname'].data.copy()
> 
> This will make sure var is a full copy in memory with no reference back to
> the file. In this case the .base attribute of var should be None; Note this
> might not be desirable if the variable has a huge memory footprint. I often
> do this, I recall having problems in the past where I was looping through a
> large number of netCDF files and then producing an IOError related to "too
> may open files" or something like that.

Indeed, the code works much better with copy().
Thanks,

Julien

-- 
http://www.juliensalort.org




More information about the SciPy-User mailing list