[Matrix-SIG] Freeing malloc'd array in Python

[Scott M. Ransom]

Hello,

I am trying to wrap a large pulsar data analysis library written in C so
that I can use it in Python.  I am using the Numeric extension to
represent many vectors and matrices that I use.

I am new to Python and do not (yet!) have a good grasp of Python's
reference counting method of garbage collection -- as you will probably
see from my code snippet below... ;)

The problem that I am running into is that many of my routines allocate
arrays using malloc() from 'C' and then return these newly created
arrays to the user.  In my 'C' routines I simple free() the array in the
main program when I don't need it anymore.

When I wrap these functions onto Numeric arrays, though, Python does not
seem to want to free the data that I malloc'd.  For example, using the
code attached at the bottom of this note and compiled (on Linux) using:

gcc -shared -I/usr/local/include/python1.5 -Wall -fpic -g -O my_range.c
-o my_range.so

and then imported into Python,  I get the following:

Python 1.5.1 (#8, Sep  3 1998, 19:02:41)  [GCC egcs-2.90.29 980515 (egc
on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> from Numeric import *
>>> from my_range import *
>>> from sys import *
>>> a = my_range(10)
>>> getrefcount(a)
2
>>> a
array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.])
>>> del(a)
>>> getrefcount(a)
Traceback (innermost last):
  File "<stdin>", line 1, in ?
NameError: a
>>> a = my_range(100000)
>>> del(a)
>>>

del(a) obviously deletes the PyArrayObject, but examining the memory
usage before and after the a = my_range(100000) and the corresponding
del(a), it is obvious that Python is not touching the actual data in the
PyArrayObject -- a bigtime memory leak.

I examined the code in arrayobject.c and arrayobject.h and my initial
impressions are that I need to somehow manually free the data portion of
the PyArrayObject when I am finished with it in Python -- before I del()
the PyArrayObject.  I unfortunately have no idea how to do this.  Do I
need another helper function in my wrapper code to free() the data in
the PyArrayObject()?  Or am I missing something even easier?

It seems that I have also been warned (from arrayobject.h) not to use
PyArray_FromDimsAndData() to create my PyArrayObject (any ideas on how
to use something else?)

Thanks in advance for your help,

Scott Ransom

---------my_range.c------------
#include <stdlib.h>
#include <stdio.h>
#include "Python.h"
#include "arrayobject.h"

/* Here is an example library function that returns an array (a 1D
   vector).  This is just representative -- the actual functions in my
   library are much more complicated.  They are all malloc'd,
   though. */

double * my_range(long n)
{
  double *v;
  long i;

  v = (double *) malloc((size_t) (sizeof(double) * n));
  if (!v) {
    printf("\nAllocation error in my_range()\n");
    exit(1);
  }
  for (i=0; i<n; i++) v[i] = (double) i;
  return v;
}

/* Here is a representative wrapper function that I am using to
   interface the above routine with Python (using Numeric arrays). */

static PyObject *wrap_my_range(PyObject *self, PyObject *args)
{
  PyObject *obj;
  PyArrayObject *arr;
  double *result;
  long n;

  self = self;
  if(!PyArg_ParseTuple(args,"O:wrap_my_range",&obj))
    return NULL;
  n = PyInt_AsLong((PyObject *)obj);
  result = my_range(n);
  arr = (PyArrayObject *)PyArray_FromDimsAndData(1, (int *)&n,
PyArray_DOUBLE, \
                                                 (char *) result);
  if (arr == NULL) return NULL;
  return (PyObject *)arr;
}

/* Here are the module initialization functions */

static PyObject *ErrorObject;
static PyMethodDef my_range_method[] = {
  { "my_range", wrap_my_range, 1 },
  { NULL, NULL }
};

void initmy_range()
{
  PyObject *m, *d;

  m = Py_InitModule("my_range", my_range_method);
  d = PyModule_GetDict(m);
  import_array();
  ErrorObject = PyString_FromString("my_range.error");
  PyDict_SetItemString(d, "error", ErrorObject);
  if (PyErr_Occurred())
    Py_FatalError("can't initialize module my_range");
}


--
Scott M. Ransom
Phone:  (580) 536-7215             Address:  703 SW Chaucer Cir.
email:  ransom@cfa.harvard.edu               Lawton, OK  73505
PGP Fingerprint: D2 0E D0 10 CD 95 06 DA  EF 78 FE 2B CB 3A D3 53