[Numpy-discussion] reference leacks in numpy.asarray

[Lisandro Dalcin]

Ups, I forgot to mention I was using gc.collect(), I accidentally
cleaned it my mail

Anyway, the following

import sys, gc
import numpy

def test():
    a = numpy.zeros(5, dtype=float)
    while 1:
        gc.collect()
        b = numpy.asarray(a, dtype=float); del b
        gc.collect()
        print sys.gettotalrefcount()

test()

shows in mi box alway 1 more totalrefcount in each pass, so always
increasing. IMHO, I still think there is a leak somewere.

And now, I am not sure if PyArray_FromAny is the source of the problem.



On 8/2/07, Timothy Hochberg <tim.hochberg at ieee.org> wrote:
>
>
>
> On 8/2/07, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> > using numpy-1.0.3, I believe there are a reference leak somewhere.
> > Using a debug build of Python 2.5.1 (--with-pydebug), I get the
> > following
> >
> > import sys, gc
> > import numpy
> >
> > def testleaks(func, args=(), kargs={}, repeats=5):
> >     for i in xrange(repeats):
> >         r1 = sys.gettotalrefcount()
> >         func(*args,**kargs)
> >         r2 = sys.gettotalrefcount()
> >         rd = r2-r1
> >         print 'before: %d, after: %d, diff: [%d]' % (r1, r2, rd)
> >
> > def npy_asarray_1():
> >     a = numpy.zeros(5, dtype=int)
> >     b = numpy.asarray(a, dtype=float)
> >     del a, b
> >
> > def npy_asarray_2():
> >     a = numpy.zeros(5, dtype=float)
> >     b = numpy.asarray(a, dtype=float)
> >     del a, b
> >
> > if __name__ == '__main__':
> >     testleaks(npy_asarray_1)
> >     testleaks(npy_asarray_2)
> >
> >
> > $ python npyleaktest.py
> > before: 84531, after: 84532, diff: [1]
> > before: 84534, after: 84534, diff: [0]
> > before: 84534, after: 84534, diff: [0]
> > before: 84534, after: 84534, diff: [0]
> > before: 84534, after: 84534, diff: [0]
> > before: 84531, after: 84533, diff: [2]
> > before: 84535, after: 84536, diff: [1]
> > before: 84536, after: 84537, diff: [1]
> > before: 84537, after: 84538, diff: [1]
> > before: 84538, after: 84539, diff: [1]
> >
> > It seems npy_asarray_2() is leaking a reference. I am  missing
> > something here?. The same problem is found in C, using
> > PyArray_FROM_OTF (no time to go inside to see what's going on, sorry)
> >
> > If this is know and solved in SVN, please forget me.
>
> I don't have a debug build handy to test this on, but this might not be a
> reference leak. Since you are checking the count before and after each
> cycle, it could be that there are cycles being created that are subsequently
> cleaned up by the garbage collector.
>
> Can you try instead to look at the difference between the reference count at
> the end of each cycle with the reference count before the first cycle? If
> that goes up indefinitely, then it's probably a leak. If it bounces around
> or levels off, then probably not. You'd probably want to run a bunch of
> repeats just to be sure.
> regards,
> -tim
>
> > Regards,
> >
> >
> > --
> > Lisandro Dalcín
> > ---------------
> > Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
> > Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
> > Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
> > PTLC - Güemes 3450, (3000) Santa Fe, Argentina
> > Tel/Fax: +54-(0)342-451.1594
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at scipy.org
> >
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
> >
>
>
>
> --
> .  __
> .   |-\
> .
> .   tim.hochberg at ieee.org
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594