Float precision and float equality

Raymond Hettinger python at rcn.com
Thu Dec 10 20:23:10 EST 2009


[Carl Banks]
> > You're talking about machine epsilon?  I think everyone else here is
> > talking about a number that is small relative to the expected smallest
> > scale of the calculation.

That was also my reading of the OP's question.

The suggestion to use round() was along the
lines of performing a quantize or snap-to-grid
operation after each step in the calculation.
That approach parallels the recommendation for how
to use the decimal module for fixed point calculations:
http://docs.python.org/library/decimal.html#decimal-faq


Raymond



More information about the Python-list mailing list