What is precision of a number representation? (was: Curious Omission In New-Style Formats)

Chris Angelico rosuav at gmail.com
Mon Jul 11 17:51:23 EDT 2016


On Tue, Jul 12, 2016 at 6:56 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Precision is not a property of the number. It is a property of the
> *representation* of that number.
>
> The representation “1×10²” has a precision of one digit.
> The representation “100” has a precision of three digits.
> The representation “00100” has a precision of five digits.
> The representation “100.00” also has a precision of five digits.
>
> Those can all represent the same number; or maybe some of them represent
> “one hundred” and others represent “one hundred and a millionth”.
>

Yep. Precision is also a property of a measurement, the same way that
a unit is. If I pace out the length of the main corridor in my house,
I might come up with a result of thirty meters. The number is "30";
the unit is "meters", the precision is two significant digits, and the
accuracy depends on how good I am at pacing distance.

This is why it's important to be able to record precisions of
arbitrary numbers. If I then measure the width of this corridor with a
laser, I could get an extremely precise answer - say, 2,147
millimeters, with a precision of four significant digits, and
excellent accuracy. But if I multiply those numbers together to
establish the floor area of the corridor, the result does NOT have
four significant figures. It would be 64 square meters (not 64.41),
and the accuracy would be pretty low (effectively, the *in*accuracies
of both measurements get combined). But on the other hand, if you want
to know whether your new fridge will fit, you could measure it with
the same laser and come up with a figure of 1,973 mm (four sig fig),
which would mean your clearance is 174mm (four sig fig). How do you
record this? Is it 174.0? 0174? "174 with four significant figures"?

ChrisA



More information about the Python-list mailing list