What is precision of a number representation?

Terry Reedy tjreedy at udel.edu
Tue Jul 12 00:14:42 EDT 2016


On 7/11/2016 5:51 PM, Chris Angelico wrote:

> This is why it's important to be able to record precisions of
> arbitrary numbers. If I then measure the width of this corridor with a
> laser, I could get an extremely precise answer - say, 2,147
> millimeters, with a precision of four significant digits, and
> excellent accuracy. But if I multiply those numbers together to
> establish the floor area of the corridor, the result does NOT have
> four significant figures. It would be 64 square meters (not 64.41),
> and the accuracy would be pretty low (effectively, the *in*accuracies
> of both measurements get combined). But on the other hand, if you want
> to know whether your new fridge will fit, you could measure it with
> the same laser and come up with a figure of 1,973 mm (four sig fig),
> which would mean your clearance is 174mm (four sig fig).

Wrong as to the last point. 2147 - 1973 = 174 has only 3 sig. figures. 
2147 - 2144 = 3 has 1 significant figure.  Subtraction of nearly equal 
numbers can cancel sig. figures.

 > How do you
> record this? Is it 174.0? 0174? "174 with four significant figures"?

None of these.

-- 
Terry Jan Reedy




More information about the Python-list mailing list