Floating point subtraction rounding error (NOT display error)

Keflavich keflavich at gmail.com
Thu Dec 13 17:30:18 EST 2007


Hey, I have a bit of code that died on a domain error when doing an
arcsin, and apparently it's because floating point subtraction is
having problems.  I know about the impossibility of storing floating
point numbers precisely, but I was under the impression that the
standard used for that last digit would prevent subtraction errors
from compounding.

Is there a simple solution to this problem, or do I need to run some
sort of check at every subtraction to make sure that my float does not
deviate?  I'm not sure I know even how to do that.

A sample of the failure:
ipdb>1.0000000000000001 == 1
True
ipdb>R
0.69999999999999996
ipdb>R==.7
True
ipdb>y2
3.2999999999999998
ipdb>y2 == 3.3
True
ipdb>cirY-y2
0.70000000000000018
ipdb>cirY-y2 == .7
False

I was unable to find solutions when searching the web because all of
the hits I got were discussing display issues, which I'm not concerned
with.

Thanks,
Adam



More information about the Python-list mailing list