rounding problem

Thomas Bartkus tom at dtsam.com
Thu Feb 24 10:05:37 EST 2005


"Dan Bishop" <danb_83 at yahoo.com> wrote in message
news:1109221152.118191.136510 at f14g2000cwb.googlegroups.com...
> tom wrote:
<snip>
> > That last digit will *always* contain some arithmetic slop.
>
> Your statement is misleading, because it suggests that your processor
> stores digits.  It doesn't; it stores *bits*.

Your explanation is much clearer and more than I knew. And I am sorry you
found my statement a bit misleading. But I never implied details about how
things were stored internally.

No matter what computer, calculater, slide rule or programming language - a
floating point number will always exhibit "arithmetic slop" at the last
significant digit. This is a property inherent to floating point numbers and
has nothing to do with how it is stored on any machine.

Thomas Bartkus





More information about the Python-list mailing list