Floating point bug?

[Steven D'Aprano]

On Fri, 15 Feb 2008 17:30:03 -0800, Jeff Schwab wrote:

>> But good advice becomes a superstition when it becomes treated as a
>> law: "never test floats for equality". That's simply not true. For
>> example, floats are exact for whole numbers, up to the limits of
>> overflow.
> 
> That's not true.  Epsilon becomes large (in absolute terms) for large
> numbers.  For 64-bit IEEE floats, eps > 1 at about 10**16.

Ah yes, good point. Sorry for my brain-fart, I was conflating integer 
floats with the comp data type, as used in the Standard Apple Numerics 
Environment (SANE). It used a floating point data type to store what was 
effectively a 64-bit integer, which was quite significant in the days 
when PCs used 16-bit integers!

Nevertheless, I stand by my claim, and support it by quoting from 
Professor W Kahan of Berkley, who wrote in the forward to the Apple 
Numerics Manual (2nd Edition):

[quote]
... because so many computers in the 1960's and 1970's possessed so many 
different arithmetic anomalies, computational lore has become encumbered 
with a vast body of superstition purporting to cope with them. One such 
superstitious rule is "*Never* ask whether floating-point numbers are 
exactly equal."
[end quote]

In practice, you often do want to test floating point values within some 
tolerance. But it isn't a universal law: see also "What Every Computer 
Scientist Should Know About Floating Point Arithmetic".

[quote]
Incidentally, some people think that the solution to such anomalies is 
never to compare floating-point numbers for equality, but instead to 
consider them equal if they are within some error bound E. This is hardly 
a cure-all because it raises as many questions as it answers.
[end quote]

http://docs.sun.com/source/806-3568/ncg_goldberg.html


Not directly related to this issue, but to get a great overview of some 
of the problems with floating point, you could do a lot worse than to 
read the following interview with Kahan:

http://www.ddj.com/184410314

and this wonderful lecture:

http://www.cs.berkeley.edu/~wkahan/ieee754status/ieee754.ps

where he details how optimizing compilers cause arithmetic errors, how 
and why the facilities provided by IEEE arithmetic are underused, and 
finally gives his idea of some things that could be done to turn the 
situation around.

Sadly this lecture was given almost twelve years ago, and things have 
barely changed. Compilers still do the wrong thing, especially optimizing 
ones; computations that would be easy with NANs and infinities are 
needlessly difficult; and benchmarks still over-value speed and under-
value getting the right answer, let alone simplicity of programming.


>> If you know your floats are whole numbers, and still writing
>> something like this:
>> 
>> x = 1234.0
>> y = 1000.0 + 200.0 + 30.0 + 4.0
>> if abs(x-y) < 1e-12:
>>     print "x and y are equal"
>> 
>> then you are wasting your time and guilty of superstitious behaviour.
>
> In what way?  It's true that you hard-coded integers for which the 
> margin of error happens to be < 1,

No, in this case the error is *precisely* zero. There simply are no 
rounding errors in this calculation, and the error is zero. If you wrote 
the test as "if abs(x-y) < 1.0" you'd still be wasting your time.

It's true that I gave a hard-coded example, but it is hardly a special 
case. There are many problem domains that don't require the full range of 
floats and the type of rounding error you give can't occur.

(If your application calculates the amount of concrete needed to build a 
house, say, then you pretty much know the amounts aren't going to be 
measured in the trillions of megatonnes. If a user insists on specifying 
that the house has 1.73e820 stories built on a base measuring 1.82e-87 
metres squared, then roundoff is the least of your problems: the 
algorithms you are applying will no longer be valid.)

Or possible you have already rounded the numbers yourself, earlier:

>>> x = round(1.5678, 2)
>>> x == 1.57
True

Why would you do this instead?

>>> abs(x - 1.57) < 1e-12
True


The important thing is that your numbers have appropriately similar 
scales. If you know that it going to be the case, then you know that 
addition won't cause the sort of round-off error. I'm talking about. 
Naturally there may be other forms of round-off, but rounding error 
doesn't just appear from nowhere, it is predictable and understandable.


-- 
Steven