Why the nonsense number appears?

Sybren Stuvel sybrenUSE at YOURthirdtower.com.imagination
Mon Oct 31 05:23:37 EST 2005


Ben O'Steen enlightened us with:
> I think that the previous poster was asking something different.

It all boils down to floating point inprecision.

> If
>
>>>> t1 = 0.500
>>>> t2 = 0.461
>>>> print t1-t2
> 0.039
>
> Then why:
>
>>>> t1 += 12345678910
>>>> t2 += 12345678910
>>>> # Note, both t1 and t2 have been incremented by the same amount.
>>>> print t1-t2
> 0.0389995574951

It's easier to explain in decimals. Just assume you only have memory
to keep three decimals.  12345678910.500 is internally stored as
something like 1.23456789105e10. Strip that to three decimals, and you
have 1.234e10. In that case, t1 - t2 = 1.234e10 - 1.234e10 = 0.

> Using decimal as opposed to float sorts out this error as floats are
> not built to handle the size of number used here.

They can handle the size just fine. What they can't handle is 1/1000th
precision when using numbers in the order of 1e10.

Sybren
-- 
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself? 
                                             Frank Zappa



More information about the Python-list mailing list