[Tutor] Fraction - differing interpretations for number and string

[Steven D'Aprano]

On Thu, Apr 16, 2015 at 01:52:51AM -0700, Danny Yoo wrote:

> It's this last supposition that should be treated most seriously.  Most
> computers use "floating point", a representation of numbers that use a
> fixed set of bits.  This uniformity allows floating point math to be
> implemented quickly.  But it also means that it's inaccurate.  You're
> seeing evidence of the inaccuracy.

Hmmm. I wouldn't describe it as "inaccurate". The situation is a lot 
more subtle and complicated than mere inaccuracy, especially these days.

Back when dinosaurs ruled the earth, it would be accurate to describe 
floating point arithmetic as "inaccurate", pun intended. Some of the 
biggest companies in computing back in the 1970s had floating point 
arithmetic which was horrible. I'm aware of at least one system where 
code like this:

    if x != 0:
        print 1/x

could crash with a Divide By Zero error. And worse! But since IEEE-754 
floating point semantics has become almost universal, the situation is 
quite different. IEEE-754 guarantees that the four basic arithmetic 
operations + - * / will give the closest possible result to the exact 
mathematical result, depending on the rounding mode and available 
precision. If an IEEE-754 floating point system gives a result for some 
operation like 1/x, you can be sure that this result is the closest you 
can possibly get to the true mathematical result -- and is often exact.

The subtlety is that the numbers you type in decimal are not always the 
numbers the floating point system is actually dealing with, because they 
cannot be. What you get though, is the number closest possible to what 
you want.

Let me explain with an analogy. We all know that the decimal for 1/3 is 
0.33333-repeating, with an infinite number of threes after the decimal 
point. That means any decimal number you can possibly write down is not 
1/3 exactly, it will either be a tiny bit less, or a tiny bit more.

    0.333333333      # pretty close
    0.33333333333    # even closer
    0.3333333333333  # closer, but still not exact
    0.3333333333334  # too big

There is no way to write 1/3 exactly as a decimal, and no way to 
calculate it exactly as a decimal either. If you ask for 1/3 you will 
get something either a tiny bit smaller than 1/3 or a tiny bit bigger.

Computer floating point numbers generally use base 2, not base 10. That 
means that fractions like 1/2, 1/4, 1/8 and similar can be represented 
exactly (up to the limit of available precision) but many decimal 
numbers are like 1/3 in decimal and have an infinitely repeating binary 
form. Since we don't have an infinite amount of memory, we cannot 
represent them *exactly* as binary floats.

So the decimal fraction 0.5 means 5/10 or if you prefer, (5 * 1/10). 
That is the same as "1/2" or (1 * 1/2), which means that in base-2 we 
can write it as "0.1".

0.75 in decimal means (7 * 1/10 + 5 * 1/100). With a bit of simple 
arithmetic, you should be able to work out that 0.75 is also equal to 
(1/2 + 1/4), or to put it another way, (1 * 1/2 + 1 * 1/4) which can be 
written as "0.11" in base-2.

But the simple decimal number 0.1 cannot be written in an exact base-2 
form:

1/10 is smaller than 1/2, so base-2 "0.1" is too big;
1/10 is smaller than 1/4, so base-2 "0.01" is too big;
1/10 is smaller than 1/8, so base-2 "0.001" is too big;

1/10 is bigger than 1/16, so base-2 "0.0001" is too small;
1/10 is bigger than 1/16 + 1/32, so base-2 "0.00011" is too small;
1/10 is smaller than 1/16 + 1/32 + 1/64, so base-2 "0.000111" 
     is too big;

likewise base-2 "0.0001101" is too big (1/16 + 1/32 + 1/128);
base-2 "0.00011001" is too small (1/16 + 1/32 + 1/256);
and so on.

What we actually need is the infinitely repeating binary number:

0.00011001100110011001100110011...

where the 0011s repeat forever. But we cannot do that, since floats only 
have a fixed number of bits. We have to stop the process somewhere, and 
get something a tiny bit too small:

0.0001100110011001100110

or a tiny bit too big:

0.0001100110011001100111

depending on exactly how many bits we have available.

Is this "inaccurate"? Well, in the sense that it is not the exact true 
mathematical result, yes it is, but that term can be misleading if you 
think of it as "a mistake". In another sense, it's not inaccurate, it is 
as accurate as possible (given the limitation of only having a certain 
fixed number of bits). 



-- 
Steve