Precision Tail-off?

Greg Ewing greg.ewing at canterbury.ac.nz
Fri Feb 17 20:37:44 EST 2023


On 18/02/23 7:42 am, Richard Damon wrote:
> On 2/17/23 5:27 AM, Stephen Tucker wrote:
>> None of the digits in RootNZZZ's string should be different from the
>> corresponding digits in RootN.
> 
> Only if the storage format was DECIMAL.

Note that using decimal wouldn't eliminate this particular problem,
since 1/3 isn't exactly representable in decimal either.

To avoid it you would need to use an algorithm that computes nth
roots directly rather than raising to the power 1/n.

-- 
Greg


More information about the Python-list mailing list