Truncation error

[Peter J. Holzer]

On 2020-10-07 07:53:55 +0200, Marco Sulla wrote:
> If you want to avoid float problems, you can use Decimal:

Decimal doesn't avoid floating point problems, because it is a floating
point format. For example:

    Python 3.8.5 (default, Jul 28 2020, 12:59:40)
    [GCC 9.3.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from decimal import *
    >>> a = Decimal(3)
    >>> a
    Decimal('3')
    >>> b = Decimal(1E50)
    >>> b
    Decimal('100000000000000007629769841091887003294964970946560')
    >>> c = Decimal(2)
    >>> a + b - c - b
    Decimal('8112996705035029053440')
    >>> b - b + a - c
    Decimal('1')
    >>> a + (b - b) - c
    Decimal('1')
    >>> a + b - b - c
    Decimal('8112996705035029053438')
    >>>

Mathematically, all four expressions should have the result 1, but with
floating point numbers they don't because intermediate results are
rounded.

For comparison, here are the results with float:

    >>> a + b - c - b
    0.0
    >>> b - b + a - c
    1.0
    >>> a + (b - b) - c
    1.0
    >>> a + b - b - c
    -2.0
    >>>

One could argue that these are at least closer to the truth, although I
think that's just luck,

Decimal does have two advantages over float: 

a) It's precision is configurable and even by default higher. So on
   average, the error is smaller (but still not zero).

b) It uses decimal numbers like we learned in school. So it will make
   the same errors as we make when we use pencil and paper, which is
   less confusing to laypersons than the seemingly arbitrary errors from
   converting from decimal to binary and back.


The disadvantages are of course higher memory consumption and lower
speed. Also, I'm very confident that the engineers at Intel and AMD knew
what they were doing when they designed the FP units of their
processors. I'm slightly less confident about the authors of the Decimal
module. And I'm much less confident that the average Python programmer
can implement a matrix multiplication with Decimal which is as
numerically stable as what Matlab or Pandas provide using IEEE-754
arithmetic.

        hp

PS: I recently read an interesting article on the Android calculator.
    That goes to extreme lengths to avoid unexpected rounding errors.
    It is also very slow, but still faster than a human can look, so it
    doesn't matter.

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20201010/98608d44/attachment.sig>