[Numpy-discussion] Error in deallocation ?

[David Cournapeau]

Matthieu Brucher wrote:
>
>     There are two types of errors that can occur with reference
>     counting on
>     data-types.
>
>     1) There are too many DECREF's --- this gets us to the error
>     quickly and
>     is usually easy to reproduce
>     2) There are too many INCREF's (the reference count keeps going up
>     until
>     the internal counter wraps around to 0 and deallocation is attempted)
>     --- this error is harder to reproduce and usually takes a while before
>     it happens in the code.
>
>
> The error showed up again with a simple operation :
>   File 
> "/home/brucher/local/src/include/toolbox/stats/kernels/gaussian.py", 
> line 60, in __call__
>     xp = (x-self.loc) * (1/self.scale)
> Where x and self.loc are arrays abd self.scale is an integer.
>
> The error as it was given :
>
> *** Reference count error detected:
> an attempt was made to deallocate 7 (l) ***
> *** Reference count error detected:
> an attempt was made to deallocate 7 (l) ***
> *** Reference count error detected:
> an attempt was made to deallocate 7 (l) ***
>
> I don't know if it is too many INCREF, but I doubt it. It happens in a 
> loop where I treat a bunch of 3D images each time (64*64*64*3), and it 
> crashes after some dozens of them. I have no problem when I treat 
> small 2D images (128*128), but far more samples.
Can't you emulate this behaviour with signals different than images ? 
(say random signal of 64*64*64*3 samples). If the process does not 
require a long processing time (say a couple of minutes), then you may 
be able to use massif tool from valgrind, which may be helpful to detect 
too many INCREF. For DECREF, then the default memory checker from 
valgrind should be useful as well.

If it does take only a couple of minutes, it is then relatively easy to 
"bisect" the code to spot the error (once you get through the C level, 
it is easy to see the problem, if you can reproduce it, in my experience).

cheers,

David