[SciPy-dev] fftpack2 for review

[Pearu Peterson]

On Tue, 1 Oct 2002, eric jones wrote:

> Hey Pearu,
> 
> Overall, first tests went fairly smoothly.  Even without djbfft and
> fftw, things appeared to install correctly on win32.  *very* nice job on
> system checking and handling the lack of libraries.
> 
> All tests passed.  Sure am glad to finally have some unit tests for fft
> stuff! 
> 
> I do not have fftw installed.  I tested once without djbfft and once
> with it.  I didn't notice any material difference in timings (the
> non-djbfft set given below).  My tests are on a 850 MHz PIII and they
> are slower than yours.  This leads me to believe, even when trying to
> use djbfft, I'm not getting it linked correctly.  Setup said it detected
> my djbfft libraries though (I defined the environment variable pointing
> to my libs to get this to work).

Note that measuring time is currently reliable only on Linux platform
where real jiffies [*] are used. Under windows and other unices either
real jiffies are not available or I don't know how to use them, and then
time.time is used instead.

[*] jiffies are used to measure how much the given process uses CPU time,
unit is 1/100th of a second. Under Linux one can get user jiffies and
system jiffies, in scipy bench tests the last ones are ignored when
measuring performance of algorithms.

>   Also freqshift puts the Nyquist component at the end of the result
>   while fftshift puts it at the beginning of the result. This is for
>   the consistency among varios functions in the fftpack module.

Let me add here that it would be good for performance not to fix the data
storage order and provide helper functions such as dftfreq function that
return this information, also provide functions for manipulating fft
results. For example, djbfft uses a rather strange ordering (that may be
the key for its good performance) and to transform the results from djbfft
routines to "standard" convention (and back), two nontrivial copying of
arrays is needed. And this may be the performance killer when using
the djbfft library. 
If no one will object this, I'll put it in to my todo list.
But it is fine with me to stick to one and only one convention if users
really want that.

> And now some installation notes about what I did to get djbfft running:
> 
> Install notes:
> 
> 1.  
> win32 (cygwin and mingw) required  #include "errno.h" in any djbfft file
> that used errno.

So, we need to patch djbfft. Probably D.J.B. does not like this.

> 2.
> I installed the latest f2py but still got the warning error:
> 
> WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!
> 
>   fftpack2 requires F2PY version 2.23.190-1367 or higher but got
> 2.23.190-1354
> 
> WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!
> 
> I'm hitting the hay right now, so I'll check out why this happened
> tomorrow.

That is strange. I would expect that Python will crash (with
segfault) when running convolve tests if older F2PY version than
2.23.190-1367 was used for building. Any change that you have several
f2py2e packages lying around in your system?


> 3.
> How hard would it be to get setup.py to build the djbfft library if one
> wasn't detected.  The makefile builds a lot of the files on the fly
> using stuff like:
> 
> 4c0.c: \
> idea/c0.c pentium/c0.c ppro/c0.c sparc/c0.c auto_opt
> 	sed 1s/PRE/pre4.c/ `cat auto_opt`/c0.c > 4c0.c
> 
> 4c0.o: \
> compile 4c0.c pre4.c fftc4.h complex4.h real4.h fftr4.h real4.h 4i.c
> 	./compile 4c0.c
> 
> I'm not sure what all is going on here, but it seems like we could
> either do the same in python or auto-generate a generic version of this
> stuff and include it in SciPy.  Of course, since djbfft isn't required,
> this isn't necessary.  

In principle, this could be done. However, D.J.B. certainly has made it
more difficult for us to accomplish that because djbfft uses a rather
unique building process, and also the conditions of altering this
software are rather restrictive.

I think that having setup.py to build djbfft library does not have a very
high priority right now, especially if the performance boost is not so
great on all platforms and arhitectures. So, I would put it into the todo
list and deal with it in future.

Pearu