Short-circuit Logic

[Steven D'Aprano]

On Thu, 30 May 2013 10:22:02 +0300, Jussi Piitulainen wrote:

> I wonder why floating-point errors are not routinely discussed in terms
> of ulps (units in last position). There is a recipe for calculating the
> difference of two floating point numbers in ulps, and it's possible to
> find the previous or next floating point number, but I don't know of any
> programming language having built-in support for these.

That is an excellent question!

I think it is because the traditional recipes for "close enough" equality 
either pre-date any standardization of floating point types, or because 
they're written by people who are thinking about abstract floating point 
numbers and not considering the implementation.

Prior to most compiler and hardware manufacturers standardizing on IEEE 
754, there was no real way to treat float's implementation  in a machine 
independent way. Every machine laid their floats out differently, or used 
different number of bits. Some even used decimal, and in the case of a 
couple of Russian machines, trinary. (Although that's going a fair way 
back.)

But we now have IEEE 754, and C has conquered the universe, so it's 
reasonable for programming languages to offer an interface for accessing 
floating point objects in terms of ULPs. Especially for a language like 
Python, which only has a single float type.

I have a module that works with ULPs. I may clean it up and publish it. 
Would there be interest in seeing it in the standard library?


> Why isn't this considered the most natural measure of a floating point
> result being close to a given value? The meaning is roughly this: how
> many floating point numbers there are between these two.

There are some subtleties here also. Firstly, how many ULP should you 
care about? Three, as you suggest below, is awfully small, and chances 
are most practical, real-world calculations could not justify 3 ULP. 
Numbers that we normally care about, like "0.01mm", probably can justify 
thousands of ULP when it comes to C-doubles, which Python floats are.

Another subtlety: small-but-positive numbers are millions of ULP away 
from small-but-negative numbers. Also, there are issues to do with +0.0 
and -0.0, NANs and the INFs.


-- 
Steven