Truncation error

[Peter J. Holzer]

On 2020-10-10 13:31:34 -0400, Dennis Lee Bieber wrote:
> On Sat, 10 Oct 2020 17:58:34 +0200, "Peter J. Holzer" <hjp-python at hjp.at>
> declaimed the following:
> 
> >On 2020-10-07 07:53:55 +0200, Marco Sulla wrote:
> >> If you want to avoid float problems, you can use Decimal:
> >
> >Decimal doesn't avoid floating point problems, because it is a floating
> >point format. For example:
> >
> 
> >    >>> b = Decimal(1E50)
> >    >>> b
> >    Decimal('100000000000000007629769841091887003294964970946560')
> 
> 	That one's a red herring... The "problem" occurs with the 1E50 /float/
> before conversion to decimal

No. At least not the problem I wanted to demonstrate which is that like
any floating-point format, Decimal has limited precision and hence 
A + B - A is in general not equal to B.

I could have written 
b = Decimal('100000000000000007629769841091887003294964970946560')
instead, but b = Decimal(1E50) was less to type.
Also, with Decimal('1E150') the result is less spectacular, but also
wrong.

Decimal is especially devious here, because while it will happily store
a value with 51 digits, it will round results of operations to the
configured precision. So by mixing operands with different precision you
get results which are very hard to predict.

At least with float you know the precision[1], so you can reason about the
results.

        hp

[1] This is actually not always true. Some architectures (like x87) use
    a higher precision for intermediate results, which is generally
    good but makes it really hard to estimate errors unless you know
    exactly what code the compiler generates.

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20201012/a6490b09/attachment.sig>