[Matrix-SIG] Memory leaks in netCDFmodule?

[Joe Van Andel]

This is a multi-part message in MIME format.
--------------27E54049BB21808E8F01A259
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I'm using Konrad Hinsen's netCDFmodule-1.0.3 with Python 1.5.2 and
Numeric Python (LLNLDistribution11).  Now that my application is nearly
completed, I'm trying to find and fix the various memory leaks, such
that I can process an arbitrary number of files without running out of
memory.

One of the leaks is associated with the netCDFmodule.  When I open a
file, e.g:

a1file=NetCDFFile(file, 'r')

the "C" routine netcdf_variable_new is called for each variable.
However, when I'm done with this file, the memory for these variables
should go away, and it doesn't.
(PyNetCDFVariableObject_dealloc is never called.)

  
I did find an interesting workaround.

Immediately after creating a netCDF file object, 
>>> sys.getrefcount(a1file)
12

Reading the source code, some of these references are caused by creating
the variables that reside in the file.  If I explicitly free each
variable, using code like:

    for v in a1file.variables.keys():
        a1file.variables[v] = None

Would everyone agree that the <netcdf>.close() method should explicitly
free these variables, so all memory is recovered?




Thanks much.

I've attached a context diff that fixes 1 memory leak in netcdfmodule.c,
I'm still chasing others.




-- 
Joe VanAndel  	          
National Center for Atmospheric Research
http://www.atd.ucar.edu/~vanandel/
Internet: vanandel@ucar.edu
--------------27E54049BB21808E8F01A259
Content-Type: text/plain; charset=us-ascii;
 name="cdiff"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="cdiff"

*** netcdfmodule.orig	Wed Jun 16 14:52:57 1999
--- netcdfmodule.c.clean	Wed Jun 16 14:54:00 1999
***************
*** 171,176 ****
--- 171,177 ----
       int nattrs;
  {
    char name[MAX_NC_NAME];
+ 
    nc_type type;
    int length;
    int py_type;
***************
*** 186,191 ****
--- 187,193 ----
            ncattget(fileid, varid, name, s);
            s[length] = '\0';
            string = PyString_FromString(s);
+           free(s);
            if (string != NULL) {
                PyDict_SetItemString(attributes, name, string);
                Py_DECREF(string);
***************
*** 273,278 ****
--- 275,282 ----
  PyNetCDFFileObject_dealloc(self)
       PyNetCDFFileObject *self;
  {
+ 
+     
    if (self->open)
      PyNetCDFFile_Close(self);
    Py_XDECREF(self->dimensions);
***************
*** 809,816 ****
  {
    if (self->dimids != NULL)
      free(self->dimids);
!   if (self->name != NULL)
      free(self->name);
    Py_XDECREF(self->file);
    PyMem_DEL(self);
  }
--- 813,822 ----
  {
    if (self->dimids != NULL)
      free(self->dimids);
!   if (self->name != NULL) {
        free(self->name);
+   }
+   
    Py_XDECREF(self->file);
    PyMem_DEL(self);
  }
***************
*** 827,835 ****
--- 833,843 ----
       int *dimids;
       int nattrs;
  {
+   
    PyNetCDFVariableObject *self;
    int recdim;
    int i;
+   
    if (check_if_open(file, -1)) {
      self = PyObject_NEW(PyNetCDFVariableObject, &PyNetCDFVariable_Type);
      if (self == NULL)

--------------27E54049BB21808E8F01A259--