How to identify memory leaks in extensions?

[Travis Oliphant]

> > The short answer is, everything must be DECREF'd.  
> 
> okay. i am confused now. even if one is __returning__ the array?
>

I'm sorry for the confusion.  Of course you should NOT DECREF arrays you
are going to return (unless you INCREF'd them for some reason in your
code), or your process will lose the memory previously allocated for them.  

***
*** I meant you should DECREF everything you don't want to have a
*** increased reference for when your done with the subroutine.
***
 
> ==============================
> ...
>     indep = (PyArrayObject *) PyArray_FromDims( IND_DIM, &dims[1], PyArray_DOUBLE);
>     dep  =  (PyArrayObject *) PyArray_FromDims( DEP_DIM, dims,     PyArray_DOUBLE);
> 
>     indepPtr = (double *)indep->data;
> 
>     /* now let PyArray_As2D re-arrange, if necessary (?),  */
>     /* our array and set the pointers to the rows of array */
>     PyArray_As2D( (PyObject**)&dep, (char ***) &numarr, 
> 		  &dims[0], &dims[1], PyArray_DOUBLE ); 
> 
> 
>     /* do stuff */
> 
>     return Py_BuildValue("(OO)", indep, dep);
> ==============================
> 
> okay, from your post those 'OO' should be 'NN'. but even then ...
> 

Yes, no need to DECREF if you use "(NN)" (since you do want these
variables to stay around after you leave this subroutine).

> > Whenever you use a PyArray_XXXXXXX construct that gives you back a
> > PyArrayObject, you own a reference.  You must DECREF before leaving
> > the subroutine or you have a memory leak.

Change this to.  You must DECREF before leaving the subroutine if you
did not intend to have an increased reference count.

> 
> you're saying i have to DECREF indep and dep before i return????? its
> confusing. i allocate arrays with PyArrayXXX to do stuff with and
> return them, in my python code i set references to them. somehow the
> 'setting reference in the python code' is an INCREF over and above the
> PyArrayXXX? 

No I don't think so.  I didn't mean to imply that.  If you want to return
the array don't DECREF it (unless you use "O" in Py_BuildValue).

> 
> so, lets see. every PyArray does one INCREF. every Py_BuildValue does
> another one if you use the 'O', and when you set a refernece equal to
> it in a return value, thats yet another?????? or maybe i am confusing
> the last INCREF with the first one.

I don't think the last INCREF happens, so don't let my quick explanation
confuse you.

> 
> but if i go 
> 
> x = 3
> y = x
> 
> that y = adds a reference to the 3 object.
> 
> thanks

This is my understanding of this sequence:

x=3:

   "3" is evaluated (because "3" is an integer between -1 and 100, the
       the reference count to the integer 3 is incremented and a
       pointer to the object is returned).  So the evaluation does the 
       INCREF.

   Since x is a name it binds this name to the returned object (no further 
   reference count increase).

y = x

   The reference count of x is incremented during evaluation of x and 
   a reference is returned.  This reference is bound to y.  Again, the 
   INCREF happens at evaluation.

x = 4

   "4" is evaluated as before, INCREF'ing the reference count.  Now since
    x is already bound, the object x references ("3") is DECREF'd.  Then,
   the new object (with it's already increased reference count) is bound
   to the name ("x").


The language reference leads me to believe that this is sort of how it
works.  I have not carefully looked through this part of the Python source
code, however.  This model lets me write code that does not have memory 
problems, however.

Good luck,

Travis