[Matrix-SIG] Re: Memory leaks in netCDFmodule?

[Konrad Hinsen]

> One of the leaks is associated with the netCDFmodule.  When I open a
> file, e.g:
> 
> a1file=NetCDFFile(file, 'r')
> 
> the "C" routine netcdf_variable_new is called for each variable.
> However, when I'm done with this file, the memory for these variables
> should go away, and it doesn't.

Right, because there are circular references (the file object keeps a
list of variables, and each variable has a reference to the file
object). This has been a known problem since the first version, but it
was never important for me, as I never use more than two netCDF files
in a program. But of course it should be fixed ultimately.

> I did find an interesting workaround.
> 
> Immediately after creating a netCDF file object, 
> >>> sys.getrefcount(a1file)
> 12
> 
> Reading the source code, some of these references are caused by creating
> the variables that reside in the file.  If I explicitly free each
> variable, using code like:
> 
>     for v in a1file.variables.keys():
>         a1file.variables[v] = None

I don't think this is sufficient in general. There might be other
references to the variable objects elsewhere, for example in user
code.

On the other hand, the variable objects are useless once the file has
been closed, so one could remove all references from the variables to
the file without loss of functionality. Then all objects (file and
variables) would be freed unless references remain in user code. And
that's all one could hope for anyway.

> I've attached a context diff that fixes 1 memory leak in netcdfmodule.c,
> I'm still chasing others.

Thanks, I'll try to work them into the current version. Which is in
fact already available (but not yet advertised) on my FTP server:
ftp://dirac.cnrs-orleans.fr/pub/ScientificPython-2.0a5.tar.gz
(I decided to integrate the netCDF module into the ScientificPython
package in order to have fewer distributions to worry about).

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------