lists and c-extensions (Swig), almost there, but still need help
Marcus Stojek
stojek at part-gmbh.de
Mon Feb 3 10:58:51 EST 2003
Hi,
sorry for bothering you again, but this is getting a real problem for
me. I need to write some c extensions that will receive large lists
(float only) do some calcs and return other lists with the results.
I have learned a lot about Swig and typemaps and below you find an
example of what I have right now. If I pass an list argument (input)
to my c extension it is transformed into a contiguous Numeric array
and a second var holding its size. The pointer to the data field of
this array and the size var are passed to the c-function.
If the results have to be stored in another list, I have to pass the
size of this resultlist to my python function. The second typemap
allocates a memory block of this size and passes the pointer and the
size to the c-function.
When returning from the c-function the "out-lists" are transformed
into python Numeric array objects and are attached to the result
tuple.
so i can write:
import Numeric
import ctest
result=ctest([1,2,3,4],8) #inputlist, size of resultlist
and will receive:
(30, [1,2,3,4,2,4,6,8]) #return value of c function and resultlist
all this is really wonderful. But if I do:
for i in range(1000):
result=ctest(longlist,largeresult)
i can watch my heap grow and grow. I have played around with some
Py_DECREF here and there but this isn't getting me anywhere.
I can't get rid of these memory leaks and I have no idea what to do
now.
Please,please, if someone has written something similar or knows what
to do ... let me know.
Thanks
marcus
P.S. There is a remark of Konrad Hinsen that PyArray_FromDimsAndData()
might never be freed. Although I don't understand the reason I tried
PyArray_FromDims with copying the data one by one. -> Slow and didn't
help.
#--------------------snip-------------------
%module ctest
%{
#include "ctest.h"
%}
//-----------------------------------------------------------------------------
typedef double d_list_in;
typedef double d_list_out;
//-----------------------------------------------------------------------------
%include "typemaps.i"
//-----------------------------------------------------------------------------
// doublelist coming in
%typemap(in) (d_list_in *, int) {
int size;
if (!PySequence_Check($input)) {
PyErr_SetString(PyExc_ValueError,"Expected a sequence as double
(in)");
return NULL;
}
size=PySequence_Length($input);
PyArrayObject *data;
data = (PyArrayObject *) PyArray_ContiguousFromObject($input,
PyArray_DOUBLE, 1, 0);
if (data == NULL){
PyErr_SetString(PyExc_ValueError,"PyArray_ContiguousFromObject
returns NULL");
return NULL;
}
$1= (double *) data->data;
$2= size;
}
%typemap(freearg) double * {
if ($1) free($1);
}
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
// result list that will be filled with doubles
%typemap(in) ( d_list_out *, int) {
if (!PyInt_Check($input)) {
PyErr_SetString(PyExc_ValueError, "Expecting an integer (size
of double resultlist)");
return NULL;
}
$2 = PyInt_AsLong($input);
if ($2 < 0) {
PyErr_SetString(PyExc_ValueError, "Positive integer expected");
return NULL;
}
$1 = (double *) malloc($2*sizeof(double));
}
//
%typemap(freearg) d_list_out * {
if ($1) free($1);
}
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//filled out list that will be appended to current result
%typemap(argout) (d_list_out *, int) {
PyObject *retarray;
PyObject *o2, *o3;
int size[1];
size[0] = $2
size[1] = 0;
retarray = (PyObject *)
PyArray_FromDimsAndData(1,size,PyArray_DOUBLE,(char *) $1);
if (retarray == NULL){
PyErr_SetString(PyExc_ValueError,"PyArray_FromDims returns
NULL");
return NULL;
}
if ((!$result) || ($result == Py_None)) {
$result = retarray;
} else {
if (!PyTuple_Check($result)) {
PyObject *o2 = $result;
$result = PyTuple_New(1);
PyTuple_SET_ITEM($result,0,o2);
}
o3 = PyTuple_New(1);
PyTuple_SetItem(o3,0,retarray);
o2 = $result;
$result = PySequence_Concat(o2,o3);
Py_DECREF(o2);
Py_DECREF(o3);
}
}
//-----------------------------------------------------------------------------
//include header file or enter declarations
//-----------------------------------------------------------------------------
double calclist (d_list_in *, int, d_list_out *, int);
//-----------------------------------------------------------------------------
More information about the Python-list
mailing list