Strange rounding problem

[Dan Bishop]

Steven Taschuk <staschuk at telusplanet.net> wrote in message news:<mailman.1047761005.17660.python-list at python.org>...
> Quoth Grant Edwards:
> > In article <TrCcnZMtctRJ_u6jXTWc3Q at comcast.com>, Marc wrote:
>  [...]
> > > So I still don't understand that if I enter a number to be an exact value of
> > > .00000096, why it can't be stored as 9.60000000e-007?
> > 
> > Becausing you entered it in base 10, and it's stored in base 2.  Just
> > because it's an exact value in base 10 it doesn't mean it can be represented
> > exactly in base 2.
> 
> For concreteness, it may be interesting to note that
>     x = 9.6e-7
>     y = 9.5999999999999991e-7
> are in binary approximately:
>     x = 1.000000011011001010110010100110100100011010010010101101100e-20
>     y = 1.000000011011001010110010100110100100011010010010101011111e-20
> These values first differ at the 53rd significant bit, here: ^.
> 
> Even if the value in question had a terminating binary expansion
> (which it doesn't), it takes a lot of precision to distinguish
> these two values.  It's *just* within the capacity of IEEE 754
> double-precision floats, I think.

Normalized IEEE double has 53 bits of precision (including the
implicit leading 1), so you would be right if it weren't for the fact
that both values get rounded to
1.0000000110110010101100101001101001000110100100101011e-20.