Decimal arithmatic, was Re: Python GUI app to impress the boss?
Tim White
timNOSPAMwhite at rogers.com
Wed Oct 2 16:07:26 EDT 2002
Actually, using the implied precision...
0.7 is a number between 0.65 and 0.75 and
0.05 is a number between 0.045 and 0.055,
which multiplied together give a number between 0.02925 and 0.04125
or 0.035 +/- 0.006 if we are not loosing any precision.
This can be expressed as 0.04 with one significant digit (as were the
original numbers) - but looses some precision.
Tim
Dennis Lee Bieber wrote:
> When I took science classes, I'd have been dinged a few points if I
> turned in 0.035 -- the rule was to report results to the same
> significance as the inputs.
>
> 0.7 * 0.05 => 0.0 in those science classes. The assumption here being
> that the accuracy of measurements is such that the last supplied
> decimal place is already an estimate -- one does not add places which
> imply increased precision.
More information about the Python-list
mailing list