[Numpy-discussion] numpy FFT memory accumulation

[Charles R Harris]

On 11/1/07, Ray Schumacher <subscriber100 at rjs.org> wrote:
> At 11:55 PM 10/31/2007, Travis wrote:
> >Ray S wrote:
> > > I am using
> > > fftRes = abs(fft.rfft(data_array[end-2**15:end]))
> > >
> >At first glance, I would say that I don't expect memory to be growing
> >here, so it looks like a problem with rfft that deserves looking into.
>
> I saw that Numeric did also (I still use Numeric for smaller array
> speed) but much more slowly.
> I will try to repeat with a small demo and post.
>
> >Does data_array keep growing?
>
> no, it is a 64k circular buffer
> Which reminds me, I've been wanting to try to build a Numeric
> circular buffer object; one that, when sliced or assigned to,
> auto-magically wraps as needed, exposing only an extra pointer
> attribute for the "end". I currently always do it with Python if-s
> and pointer vars for these products that only do "real time" data
analysis.
>

In Python, collections.deque makes a pretty good circular buffer. Numpy will
make an array out of it, which involves a copy, but it might be better than
what you are doing now.

In [14]: from collections import deque

In [15]: a = deque([1,2,3,4])

In [16]: fft(a)
Out[16]: array([ 10.+0.j,  -2.+2.j,  -2.+0.j,  -2.-2.j])

In [17]: a.append(5)

In [18]: a.popleft()
Out[18]: 1

In [19]: fft(a)
Out[19]: array([ 14.+0.j,  -2.+2.j,  -2.+0.j,  -2.-2.j])


Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20071101/2174ef15/attachment.html>