Decimal arithmatic, was Re: Python GUI app to impress the boss?

[Chris Gonnerman]

----- Original Message ----- 
From: "Dennis Lee Bieber" <wlfraed at ix.netcom.com>


> Chris Gonnerman fed this fish to the penguins on Sunday 29 September 
> 2002 03:33 pm:
> 
> 
> > You ask how it is a different problem.  Simple.  A human,
> > training in decimal math in grade school, knows that
> > 
> >     1/3 = 0.3333333...
> > 
> > and would *expect* it to round to 0.33 (still in two 
> > decimal
> 
>         So, in 2 decimal place arithmetic
> 
>         a = 1.00 / 3.00
>         b = a * 3.00
> 
> gives
>         b = 0.99
> 
> which is NOT equal to 1... But this is okay because it's done 
> is "decimal arithmetic"? <G>

It's not "OK"... it's EXPECTED.

>         The telling phrase is "training in DECIMAL math"... 
> Most computers don't work in decimal math, so the 
> user /should/ be ready to learn the ins&outs of binary 
> floating point...

Incorrect.  The *user* expects the dang computer to produce
the same results as for a manual calculation.  In other words,
truly serious businesspeople expect to be able to hand-check
the numbers.  The occasional (but far too common) binary float
anomalies are unacceptable in that environment.

> 
>         For every example of "what is wrong with binary 
> floating point" someone else can find a similar case when 
> using decimal math.

True, but the decimal anomalies are *expected* and the binary
anomalies are *surprising*...  I've been a programmer for 
a long time, working in everything from a scientific lab to 
a retail store.  I still don't understand the ins and outs,
as you put it, of binary floating point.

> Even VB's "currency" type (which looks to be a scaled 64-bit 
> integer) carries 4 decimal places -- which allows for 
> "proper" results when displaying final results in two decimal 
> places with rounding.
> 
>         1/3 => 0.3333   internally
>         3*0.3333 => 0.9999
>         1.00 for display after rounding.

I hate to give anything to Microsoft, but they seem to have a
good, workable, not-often-surprising idea here.

I still prefer the FixedPoint.py module's programmer-controlled
scaling.

Chris Gonnerman -- chris.gonnerman at newcenturycomputers.net
http://newcenturycomputers.net