[Tutor] floating point rounding inconsistency

[Dave Angel]

On 09/19/2012 02:27 PM, Andrew Tritt wrote:
> Hello,
>
> I am new to python, and I was experimenting with the round function, and
> came across what appears to be a bug in floating point rounding. I am
> guessing there is a valid explanation for it.

Welcome to the Python tutor;  hope you enjoy learning and using python.

Not a bug, and not specific to Python.  And not new - many threads
already discuss this.

>
> When I round the floating points 10.6[0-9]5 to two decimal places, it
> rounds as expected for 6 of the 10, but not for the other 4. When I try the
> same for 10.7[0-9]5, I experience the same problem, but for 5 of the 10
> possibilties, and not for the analogous floats.
>
> Also, python storing the numbers as they are represented at the prompt.
> i.e. 10.665 is stored as 10.665, not something like 10.665000001 or
> 10.66499999999.
>
> Can anyone explain to me what's happening?

Yep, Python does not store the number 10.065 in any of the forms you
appear to expect.  it instead converts it to binary floating point, and
stores the nearest value that can represent.  Sometimes such a value
will convert back to the decimal value you started with, sometimes not.

This is not a new problem - it was highlighted in the textbook i used
for Fortran in 1967.  But it was more or less made permanent by the
Intel 8087 chip, which did binary floating point, and did not do decimal
floating point.  The IEEE committee then standardized pretty much on
what was already done, and that's what's on all the Intel and AMD chips
we see now.

Generally, the way to avoid the problem is to mask it while displaying. 
Use some form of formatting to represent the precision you expect.

See the article Oscar pointed you to:
    

http://docs.python.org/tutorial/floatingpoint.html




-- 

DaveA