Truncation error

[Peter J. Holzer]

On 2020-10-11 01:40:42 -0000, Grant Edwards wrote:
> On 2020-10-10, Peter J. Holzer <hjp-python at hjp.at> wrote:
> > On 2020-10-07 07:53:55 +0200, Marco Sulla wrote:
> >> If you want to avoid float problems, you can use Decimal:
> >
> > Decimal doesn't avoid floating point problems, because it is a floating
> > point format. For example:
> >     [...]
> 
> >     >>> from decimal import *
> >     >>> a = Decimal(3)
> >     >>> a
> >     Decimal('3')
> >     >>> b = Decimal(1E50)
> >     >>> b
> >     Decimal('100000000000000007629769841091887003294964970946560')
> >     [...]
> 
> There are two problems with your code:
> 
>  1. You meant Decimal('1e50').

No. I meant Decimal(1E50)

>  What you typed creates a Decimal value
>     from the IEEE 64-bit floating point value closest to 1e50.

Which is what I wanted. I gave me some "random" digits at the end
without having to type them. But I realize that I should have used
Decimal('1e50') to remove that red herring - the problem can be
demonstrated with Decimal('1e50') just as well, it's just less
spectacular. (OTOH using Decimal(1E50) has the advantage of
demonstrating the dangers of mixing operands of different precision, but
again, that may just be confusing).


>  2. You need to increase the context precision.  It defaults to 28,
>     and you're example needs it to be at least 51:

That helps for that specific example, but not in general.
> 
> >>> getcontext().prec = 100

So then maybe there is an operand with 150 digits. Or you want to add
1E70 to 1E-70.

The fact is that any non-trivial computation will always exceed the
precision (even a simple division like Decimal('1') / Decimal('3') needs
an infinite number of digits, but you can't set precision to infinite)
and require rounding. You can make the error very small, but it will
still be there. You have to be aware of that. Using Decimal will not
give you mathematically correct results - it will just give you
different (und usually, but not always, as I've demonstrated) errors.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20201012/f90431ed/attachment.sig>