[Numpy-discussion] Memory leak in array protocol numarray<--numpy

[Todd Miller]

I agree with all of Travis' comments below and committed the suggested 
changes to numarray CVS.     I found one other numarray change needed 
for Francesc's examples to run (apparently) leak-free:

Py_INCREF(obj)
Py_XDECREF(a->base)
a->base = obj
Py_DECREF(cobj)

Thanks Travis!

Regards,
Todd

Travis Oliphant wrote:
> Francesc Altet wrote:
>   
>> Hi,
>>
>> I was tracking down a memory leak in PyTables and it boiled down to a problem 
>> in the array protocol. The issue is easily exposed by:
>>
>> for i in range(1000000):
>>     numarray.array(numpy.zeros(dtype=numpy.float64, shape=3))
>>
>> and looking at the memory consumption of the process. The same happens with:
>>
>> for i in range(1000000):
>>     numarray.asarray(numpy.zeros(dtype=numpy.float64, shape=3))
>>
>> However, the numpy<--numarray sense seems to work well.
>>
>> for i in range(1000000):
>>     numpy.array(numarray.zeros(type="Float64", shape=3))
>>
>> Using numarray 1.5.1 and numpy 1.0b1
>>
>> I think this is a relatively important problem, because it somewhat prevents a 
>> smooth transition from numarray to NumPy. 
>>
>>   
>>     
>
> I tracked the leak to the numarray function
>
> NA_FromDimsStridesDescrAndData
>
> This function calls NA_NewAllFromBuffer  with a brand-new buffer object 
> when data is passed in (like in the case with the array protocol).  That 
> function then takes  a reference to the buffer object but then the 
> calling function never releases the reference it already holds.  This 
> creates the leak.
>
> I added the line
>
> if (data) {Py_DECREF(buf);}
>
> right after the call to NA_NewAllFromBuffer and the leak disappeared. 
>
> For what it's worth, I also think the base object for the new numarray 
> object should be the object passed in and not the C-object that is 
> created from it. 
>
> In other words in the NA_FromArrayStruct  function
>
> a->base = cobj
>
> should be replaced with
>
> Py_INCREF(obj)
> a->base = obj
> Py_DECREF(cobj)
>
>
> Best,
>
>
> -Travis
>
>
>
>
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>