Float precision and float equality

[dbd]

On Dec 6, 1:48 pm, sturlamolden <sturlamol... at yahoo.no> wrote:
> On 6 Des, 21:52, r0g <aioe.... at technicalbloke.com> wrote:
>
> > > .> Right.  Using abs(x-y) < eps is the way to go.
> > > .>
> > > .> Raymond
>
> > > This only works when abs(x) and abs(y) are larger that eps, but not
> > > too much larger.
>
> > Okay, I'm confused now... I thought them being larger was entirely the
> > point.
>
> Yes. dbd got it wrong. If both a smaller than eps, the absolute
> difference is smaller than eps, so they are considered equal.

Small x,y failure case:
eps and even eps squared are representable as floats. If you have
samples of a sine wave with peak amplitude of one half eps, the "abs(x-
y) < eps" test would report all values on the sine wave as equal to
zero. This would not be correct.
Large x,y failure case:
If you have two calculation paths that symbolically should produce the
same value of size one over eps, valid floating point implementations
may differ by an lsb or more. An single lsb error would be 1, much
greater than the test allows as 'nearly equal' for floating point
comparison.

1.0 + eps is the smallest value greater than 1.0, distinguishable from
1.0. Long chains of floating point calculations that would
symbolically be expected to produce a value of 1.0 many be expected to
produce errors of an eps or more due to the inexactness of floating
point representation. These errors should be allowed in floating point
equality comparison. The value of the minimum representable error will
scale as the floating point number varies. A constant comparison value
is not appropriate.

Mark was right, DaveA's discussion explains a strategy to use.

Dale B. Dalrymple