[SciPy-user] FFTW python bindings again

[Jochen]

On Tue, 2009-01-27 at 12:43 +0900, David Cournapeau wrote:
> Jochen wrote:
> > Hi all, 
> > about starting a new thread, stupid gmail does not show my posts to the
> > list. Anyways I noticed a big mistake in how I was allocating the
> > aligned memory thus it was actually not guaranteed to be 16byte aligned.
> > I guess it didn't show up because I was mainly testing using complex
> > numbers and malloc just took the next free block, which happened to be
> > aligned because I had just allocated a large chunk of aligned data.
> >   
> 
> I believe fftw automatically detects whether your array is aligned or
> not - problem appear when you create your plan with aligned pointers,
> but use other pointers later. That's one reason why fftw backend was not
> that fast in scipy BTW, because the simplest way to ensure this was to
> copy data into aligned buffers.

Yes I understand that. I was using a somewhat hackish way of creating
the memory aligned array, i.e. I was casting the pointer returned from
ctypes to a bytes array and then passed that to ndarray.__new__ as a
buffer. I didn't realise was that in the process I was allocating new
memory, which when I tested manually was still aligned because I had
just allocated aligned array (I was only using small arrays). I now use 
PyBuffer_FromReadWriteMemory to create a buffer object to pass to
ndarray.__new__ in order to create the aligned memory.


> At least on linux, allocating big buffers with malloc is almost
> guaranteed not to be aligned, because of its use of mmap above a certain
> threshold. We discovered this fact a while ago:
> 
> http://projects.scipy.org/pipermail/scipy-dev/2007-August/007591.html
> 
I think I stumbled accross that thread when I was looking for fftw
bindings. 

> Those are some of the reasons why we decided to drop fftw support: to
> use it efficiently is not that easy, because we would first need
> guarantees about aligned allocator (once you take into accout that numpy
> also uses realloc, just using posix_memalign is not enough).
> 
> On the other hand, I think it would be very nice to have fftw wrappers
> outside scipy. For some technical aspects, I answered to you in your
> other post,
> 
> David

Thanks for the comments
Cheers
Jochen

> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-user