Weird memory consumption problem.

Bo Peng bpeng at rice.edu
Tue Dec 20 02:27:50 EST 2005


Dear list,

I spent the last 12 hours in catching this bug (?) and what I found out 
is very difficult to explain:

Basically, I need to call a user-provided function many times, with a 
tuple as parameter.

C/C++ side: (class A, constructed using a python function m_func)

    // create in the constructor, used to hold the parameter
    m_numArray = PyTuple_New(m_len);
    ...

   // member function, will be called many times
   void fun(){
    ....
    // set the tuple
    for( j=0; j<m_len; ++j)
      PyTuple_SetItem(m_numArray, j, PyInt_FromLong(some_value));
    // call function m_func(m_numArray)
    PyObject* arglist = Py_BuildValue("(O)", m_numArray );
    PyObject* result = PyEval_CallObject(m_func, arglist);
    Py_DECREF(arglist);
    // check if result == NULL ...
    // get value from result, ...
    Py_DECREF(result)
    return the value of result
   }

Python side: pyFun1, pyFun2 will be passed to C/C++ as m_func

    def pyFun1(val):
      return 0.5

    def pyFun2(val):
      return 0.5*val[0]

For objects A(pyFun1) and A(pyFun2), member function fun() will be 
called many times. The weird thing is that calling A(pyFun1).fun() 
endlessly goes well with constant memory consumption; while calling 
A(pyFun2).fun() will use more and more memory until core dump. I tried 
"valgrind --tool=memcheck --check-leak=yes" but did not find any sign of 
memory leak.

The real code is much more complicated and I can not possibly explain 
the details here. Can anyone *guess* what might go wrong? My guess is 
that access to val[0] increases the ref_cnt of val[0] and make its value 
stay in memory, but this is unlikely the case.

Many thanks in advance.
Bo



More information about the Python-list mailing list