[SciPy-Dev] memory leak in scipy.fftpack.ifft2?

[Brian Toby]

Matt Terry <matt.terry <at> gmail.com> writes:
> 
> I'm assuming that you are expecting the address of CC to remain
> constant.  As written, it should not.  fft2 returns a new array, as
> does ifftshift and ifft2.  You can fill an existing array with the
> answer by creating ffta and CC outside the loop and then filling them
> with the CC[:,:] = blah() syntax.  The modified script is attached.
> 
> WIth these changes, the script still uses a lot of memory (high water
> mark of 2.4 GB), but the memory usage does not grow without bound.  At
> least for me, using the same platform (mac, epd 7.2).
> 
> -matt

That is a very nice trick to force reuse of memory, but it makes even more clear
there is a memory leak in scipy.fftpack. I was expecting in my previous code
that python would garbage collect and delete unreferenced objects, but with your
change arrays are reused so even that is not needed. 

Either way this code should not require any additional memory after the first
iteration, particularly now, since ffta and CC are reused. However, this is
clearly not what I see. Below is a map of the major allocated memory use. Note
that it increases by 320Mb after each iteration. 

I have found a work-around: use of the numpy.fft routines in place of
scipy.fftpack. When I make this change, the memory use stays constant at 432Mb
after every iteration. 

Brian

BHT3:~ toby$ vmmap 65978| grep MALLOC| grep "[GM]]" # (after 1st iteration)
MALLOC_LARGE           0492c000-2296c000 [480.2M] 
MALLOC_LARGE           2a96c000-3e96c000 [320.0M] 
MALLOC                  [  815.8M]

BHT3:~ toby$ vmmap 65978| grep MALLOC| grep "[GM]]" # (after 2nd iteration)
MALLOC_LARGE           0492c000-2296c000 [480.2M] 
MALLOC_LARGE           2a96c000-3e96c000 [320.0M] 
MALLOC_LARGE           4296c000-5696c000 [320.0M] 
MALLOC                  [    1.1G]

BHT3:~ toby$ vmmap 65978| grep MALLOC| grep "[GM]]" # (after 3rd iteration)
MALLOC_LARGE           0492c000-2296c000 [480.2M] 
MALLOC_LARGE           2a96c000-3e96c000 [320.0M] 
MALLOC_LARGE           4296c000-6a96c000 [640.0M] 
MALLOC                  [    1.4G]

BHT3:~ toby$ vmmap 65978| grep MALLOC| grep "[GM]]" # (after 4th iteration)
MALLOC_LARGE           0492c000-2296c000 [480.2M] 
MALLOC_LARGE           2a96c000-3e96c000 [320.0M] 
MALLOC_LARGE           4296c000-7e96c000 [960.0M] 
MALLOC                  [    1.7G]

BHT3:~ toby$ vmmap 65978| grep MALLOC| grep "[GM]]" # (after 5th iteration)
MALLOC_LARGE           0492c000-0e96c000 [160.2M] 
MALLOC_LARGE           1296c000-2296c000 [256.0M] 
MALLOC_LARGE           2a96c000-3e96c000 [320.0M] 
MALLOC_LARGE           4296c000-8296c000 [  1.0G] 
MALLOC_LARGE           c0000000-d0000000 [256.0M] 
MALLOC                  [    2.0G]