Weird arithmetic in Python

[Thomas Wouters]

On Mon, Oct 30, 2000 at 02:13:37PM +0000, rzantow at my-deja.com wrote:
> In article <slrn8vpahn.be6.kc5tja at garnet.armored.net>,

> > Round-off error.  The computer is incapable of representing certain
> > numbers with full fidelity.  Evaluating 0.1*3 loses some precision over
> > just typing in 0.3, so the difference in precision is the result you see
> > above (5.55111512313e-17).

> I comprehend that roundoff error could readily produce non-zero answers.
> But what sort of roundoff causes .1 * 3 - .3 to be greater than 5?

You forget that computers are binary. 0.1 and 0.3 can't be represented in
binary, and most common floating-point formats can't do much better either.
Here's what 0.1 and 0.3 are really are, on Pentium2's running BSDI:

>>> 0.1
0.10000000000000001
>>> 0.3
0.29999999999999999

So, as you can see, 0.1 is a bit more than 1/10th, and 0.3 is a bit less tan
3/10th. That difference is what you see when you do '0.1 * 3 - 0.3'. 

It has nothing to do with the rounding we are used to, so it's really better
to call it an an 'approximation' error. 

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!