[SciPy-user] fft numerical precision

[David M. Cooke]

Robert Kern <robert.kern at gmail.com> writes:

> Ryan Krauss wrote:
>> Sorry to bring this back up again (this thread has been dormant since
>> 12/21/05), but I think this is the key road block between me and
>> really understanding this problem (and I think it went over my head
>> when it was originally posted).  If double precision numbers use 52
>> bits for the mantissa, what does it mean to say that 2**-52 is the
>> approximate fractional precision of double precision floats?  How does
>> this affect optimization routines like fzero and fmin so that their
>> accuracy seems limited to 2**-52 when the smallest possible floating
>> point number should be more like 2**-1074.  Does this somehow relate
>> to how floating point subtraction is done (as relates to my post from
>> earlier today).
>
> I don't have time right now to go into a lecture on condition numbers and other
> such stuff, but let give you an example to chew on and a good reference:
>
[snip example]
>
> The first volume of _Numerical Computation_ by Christoph W. Ueberhuber
> (_Computer-Numerik 1_ in the original German) covers these topics quite well and
> extensively.

Another reference: I've been working through _Accuracy and Stability
of Numerical Algorithms_ (2nd ed.), by Nicholas J. Higham, which I'm
finding to be pretty good. Ryan mentioned that he was working through
"What Every Computer Scientist Should Know About Floating Point" in
another thread, which is good place to start.

Looking at the stuff on William Kahan's site at
http://www.cs.berkeley.edu/~wkahan/ can give you a good idea of what
problems you can run into with floating point.

Basically, when doing numerical computations, there are two levels of
analysis: numerical analysis, where you're concerned about which
algorithm to use, how stable it is, etc., and then the nitty-gritty
things about floating point. For the last bit, you mainly worry about
if rounding errors, etc. are big enough to worry about :) (In my
experience, most people don't do that bit, but they're usually ok,
except when they're not.)

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|cookedm at physics.mcmaster.ca