Float precision and float equality

Tim Roberts timr at probo.com
Sun Dec 6 02:42:28 EST 2009


Raymond Hettinger <python at rcn.com> wrote:
>
>   if not round(x - y, 6): ...

That's a dangerous suggestion.  It only works if x and y happen to be
roughly in the range of integers.

For example, here x and y are within roundoff error of each other, but
round doesn't know it:
   >>> x=1e32
   >>> y=x+1e16
   >>> x-y
   -18014398509481984.0
   >>> round(x-y,6)
   -18014398509481984.0

It fails in the other direction when the numbers are small:
   >>> x=.0000000123
   >>> y=.0000000234
   >>> x-y
   -1.1100000000000002e-008
   >>> round(x-y,6)
   0.0

Mark's solution is the generically correct one, which takes into account
the rough range of the values.
-- 
Tim Roberts, timr at probo.com
Providenza & Boekelheide, Inc.



More information about the Python-list mailing list