HELP: restore my faith in Python

[Johann Hibschman]

Holton, Steven [NCRTP:6125:EXCH] writes:
> Can anyone explaing to me why index 6 and 7 aren't what I'm expecting 
> them to be?

> 6 / 5.0 is 1.2   (when we're dealing with floats)
> 6 / 5 is 1       (when we're dealing with ints)
> 1.2 - 1 = 0.2 
> 0.2 * 5 = 1.0
> int( 1.0 ) = 1   (don't it?)

You're hitting rounding error.  Floats are inherently imprecise; if
you want to pull tricks like this, you should either use a rational
class, or include an allowed "epsilon" for rounding to 1.

You can see what's going on by asking python to show you more decimal
places:


>>> "%.20f" % (6/5.0) 
'1.19999999999999995559'

>>> "%.20f" % ((6/5.0)-1.0)
'0.19999999999999995559'

>>> "%.20f" % (((6/5.0)-1.0)*5)
'0.99999999999999977796'

0.999... is less than 1, so int truncates it back down to 0.

In short, you can't trust floats to give precise answers, if the last
few decimal places matter.  If you're truncating to int, rather than
rounding, then those last few places do matter.  This is true in any
language.  Just don't do it.  ;-)

--Johann

-- 
Johann Hibschman                           johann at physics.berkeley.edu