[SciPy-user] fft numerical precision

[Robert Kern]

Ryan Krauss wrote:
> I have run into this as well.  1e-16 seems large to me for double
> precision.  Robert can you explain this a bit more please.  Is there
> an IEEE spec or something that specifies how much of the 64bits is for
> exponent and how much is for the mantissa (I think that is the right
> word)?   I was playing with some FORTRAN code and it seems like there
> was a big difference with complex vs. double complex.  It seems like
> 1e-16 was the magnitude floor of complex and double complex was
> better.

IEEE-754 double precision has 11 bits for exponent, 1 bit for sign, and 52 for
the rest of the mantissa. 2.**-52 ~= 2.22e-16

Goldberg, David. What Every Computer Scientist Should Know About Floating Point
Arithmetic. 1991.
http://www.physics.ohio-state.edu/~dws/grouplinks/floating_point_math.pdf

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter