What is precision of a number representation?

Steven D'Aprano steve+comp.lang.python at pearwood.info
Tue Jul 12 02:21:38 EDT 2016


On Tuesday 12 July 2016 08:17, Ethan Furman wrote:

> So, so far there is no explanation of why leading zeroes make a number
> more precise.


Obviously it doesn't, just as trailing zeroes doesn't make a number more 
precise. Precision in the sense used by scientists is a property of how the 
measurement was done, not of the number itself. We can arbitrarily add zeroes 
to the end of 0.5 until the cows come home, and its still just a half, no more, 
no less.

But scientists have a useful convention that you can indicate the measurement 
precision by showing trailing zeroes:

0.5 is short-hand for 0.5 ± 0.05;

0.50 is short-hand for 0.5 ± 0.005;

0.5000 is short-hand for 0.5 ± 0.00005;


etc. That's just a convention, although its a useful one. But its not the only 
useful convention:

01 is a byte with value 1;

0001 is a double-byte quantity (short int?) with value 1;

00000001 is a 32-bit quantity with value 1;

etc. We say that these examples differ in their "number of digits" instead of 
"number of decimal places", so the difference doesn't quite map to the 
scientist's meaning of the word "precision". But why should we let that stop us 
from using the "precision" field of a format string to represent number of 
digits?

If not, then what are the alternatives? Using str.format, how would you get the 
same output as this?


py> "%8.4d" % 25
'    0025'



-- 
Steve




More information about the Python-list mailing list