[Tutor] precision?

Peter Otten __peter__ at web.de
Wed Aug 10 10:39:07 CEST 2011


Steven D'Aprano wrote:

> Shwinn Ricci wrote:
>> When comparing a given value with a database of values, but allowing for
>> freedom due to variation at say, the thousandth digit, how does one
>> generalize the precision to this degree? I don't want to truncate, so is
>> there a round() function that takes into account what decimal place the
>> value (database value) needs to be rounded to?
> 
> Thousandth digit??? Python doesn't support floating point numbers with a
> thousand digits! I think about seventeen is about the limit.

Not only that. The idea of n-digit precision is flawed because assuming an 
actual value of 1.00000 0.999 is closer than 1.01 but the latter has two 
correct digits and the former has none.

> Yes, there is a round function:
> 
> 
>  >>> x = 123.45678
>  >>> y = 123.45731
>  >>> x == y
> False
>  >>> round(x, 3) == round(y, 3)
> True

I prefer 

abs(x-y) < eps

for similar reasons as stated above. 1.8 is as close to 1.4 as 1.0, but 
round(1.8) != round(1.4) whereas round(1.4) == round(1.0).



More information about the Tutor mailing list