Float precision and float equality

[Dave Angel]

Carl Banks wrote:
> On Dec 6, 11:34 am, Anton81 <gerenu... at googlemail.com> wrote:
>   
>> I do some linear algebra and whenever the prefactor of a vector turns
>> out to be zero, I want to remove it.
>>
>> I'd like to keep the system comfortable. So basically I should write a
>> new class for numbers that has it's own __eq__ operator?
>> Is there an existing module for that?
>>     
>
> I highly recommend against it; among other things it invalidates the
> transitive property of equality:
>
> "If a =b and b == c, then a == c."
>
> It will also make the number non-hashable, and have several other
> negative consequences.  Plus, it's not something that's never
> foolproof.  What numbers are close enought to be condidered "equal"
> depends on the calculations.
>
> (I remember once struggling in a homework assignment over seemingly
> large discrepancies in a calculation I was doing, until i realized
> that the actual numbers were on the scale of 10**11, and the
> difference was around 10**1, so it really didn't matter.)
>
>
>
> Carl Banks
>
>   
A few decades ago I implemented the math package (microcode) under the 
machine language for a proprietary processor (this is when a processor 
took 5 boards of circuitry to implement).  I started with floating point 
add and subtract, and continued all the way through the trig, log, and 
even random functions.  Anyway, a customer called asking whether a 
particular problem he had was caused by his logic, or by errors in our 
math.  He was calculating the difference in height between an 
always-level table and a perfectly flat table (between an arc of a great 
circle around the earth, and a flat table that doesn't follow the 
curvature.)  In a couple of hundred feet of table, the difference was 
measured in millionths of an inch, as I recall.  Anyway it turned out 
his calculation was effectively subtracting
     (8000 miles plus a little bit) - (8000 miles)
and if he calculated it three different ways, he got three different 
results, one was off in about the 3rd place, while the other was only 
half the value.  I was able to show him another way (through geometrical 
transformations) to solve the problem that got the exact answer, or at 
least to more digits than he could possibly measure.  I think I recall 
that the new solution also cancelled out the need for trig.  Sometimes 
the math package shouldn't hide the problem, but give it to you straight.

DaveA