Why the nonsense number appears?
Steve Horsley
steve.horsley at gmail.com
Tue Nov 1 14:48:08 EST 2005
Dan Bishop wrote:
> That's A source of error, but it's only part of the story. The
> double-precision binary representation of 0.039 is 5620492334958379 *
> 2**(-57), which is in error by 1/18014398509481984000. By contrast,
> Johnny Lee's answer is in error by 9/262144000, which is more than 618
> billion times the error of simply representing 0.039 in floating point
> -- a loss of 39 bits.
>
> The problem here is catastrophic cancellation.
>
> 1130748744.500 ~= 4742703982051328 * 2**(-22)
> 1130748744.461 ~= 4742703981887750 * 2**(-22)
>
> Subtracting gives 163578 * 2**(-22), which has only 18 significant bits.
>
Hmm. Good point.
More information about the Python-list
mailing list