Short-circuit Logic

[Jussi Piitulainen]

Steven D'Aprano writes:

> On Thu, 30 May 2013 10:22:02 +0300, Jussi Piitulainen wrote:
> 
> > I wonder why floating-point errors are not routinely discussed in
> > terms of ulps (units in last position). There is a recipe for
> > calculating the difference of two floating point numbers in ulps,
> > and it's possible to find the previous or next floating point
> > number, but I don't know of any programming language having
> > built-in support for these.

...

> But we now have IEEE 754, and C has conquered the universe, so it's
> reasonable for programming languages to offer an interface for
> accessing floating point objects in terms of ULPs. Especially for a
> language like Python, which only has a single float type.

Yes, that's what I'm thinking, that there is now a ubiquitous floating
point format or two, so the properties of the format could be used.

> I have a module that works with ULPs. I may clean it up and publish it. 
> Would there be interest in seeing it in the standard library?

Yes, please.

> There are some subtleties here also. Firstly, how many ULP should
> you care about? Three, as you suggest below, is awfully small, and
> chances are most practical, real-world calculations could not
> justify 3 ULP.  Numbers that we normally care about, like "0.01mm",
> probably can justify thousands of ULP when it comes to C-doubles,
> which Python floats are.

I suppose this depends on the complexity of the process and the amount
of data that produced the numbers of interest. Many individual
floating point operations are required to be within an ulp or two of
the mathematically correct result, I think, and the rounding error
when parsing a written representation of a number should be similar.
Either these add up to produce large errors, or the computation is
approximate in other ways in addition to using floating point.

One could develop a kind of sense for such differences. Ulps could be
a tangible measure when comparing different algorithms. (That's what I
tried to do with them in the first place. And that's how I began to
notice their absence when floating point errors are discussed.)

> Another subtlety: small-but-positive numbers are millions of ULP
> away from small-but-negative numbers. Also, there are issues to do
> with +0.0 and -0.0, NANs and the INFs.

The usual suspects ^_^ and no reason to dismiss the ulp when the
competing kinds of error have their corresponding subtleties. A matter
of education, I'd say.

Thank you much for an illuminating discussion.