Why the nonsense number appears?

Steve Horsley steve.horsley at gmail.com
Tue Nov 1 14:48:08 EST 2005


Dan Bishop wrote:

> That's A source of error, but it's only part of the story.  The
> double-precision binary representation of 0.039 is 5620492334958379 *
> 2**(-57), which is in error by 1/18014398509481984000.  By contrast,
> Johnny Lee's answer is in error by 9/262144000, which is more than 618
> billion times the error of simply representing 0.039 in floating point
> -- a loss of 39 bits.
> 
> The problem here is catastrophic cancellation.
> 
> 1130748744.500 ~= 4742703982051328 * 2**(-22)
> 1130748744.461 ~= 4742703981887750 * 2**(-22)
> 
> Subtracting gives 163578 * 2**(-22), which has only 18 significant bits.
> 

Hmm. Good point.




More information about the Python-list mailing list