rounding problem
Thomas Bartkus
tom at dtsam.com
Thu Feb 24 10:05:37 EST 2005
"Dan Bishop" <danb_83 at yahoo.com> wrote in message
news:1109221152.118191.136510 at f14g2000cwb.googlegroups.com...
> tom wrote:
<snip>
> > That last digit will *always* contain some arithmetic slop.
>
> Your statement is misleading, because it suggests that your processor
> stores digits. It doesn't; it stores *bits*.
Your explanation is much clearer and more than I knew. And I am sorry you
found my statement a bit misleading. But I never implied details about how
things were stored internally.
No matter what computer, calculater, slide rule or programming language - a
floating point number will always exhibit "arithmetic slop" at the last
significant digit. This is a property inherent to floating point numbers and
has nothing to do with how it is stored on any machine.
Thomas Bartkus
More information about the Python-list
mailing list