rounding problem

[tom]

On Wed, 23 Feb 2005 19:04:47 -0600, Andy Leszczynski wrote:

> It is on Windows, Linux, Python 2.3:
> 
> [GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2 Type "help",
> "copyright", "credits" or "license" for more information.
>  >>> a=1.1
>  >>> a
> 1.1000000000000001
>  >>>
>  >>>
>  >>>
> Is it normal?

Yes. Quite normal.

Floating point numbers are by nature approximations. When you enter your
variable "a" at the prompt, you are asking for a very raw (i.e totally
unformatted!) dump of what your variable "a" contains. It is the
most precise representation of the float 1.1 that your computer can store.
 It is probably the best precision your hardware math processor subsystem
has to offer. And, incidentally, that ain't too shabby since a 1 in the
16th decimal place ain't worth spit!

Try putting a print statement in front of your variable.
>>> print a
>>> 1.1

The print statement ,by itself, adds the most minimial formatting, which
amounts to dropping the last significant digit your math processor holds.
That last digit will *always* contain some arithmetic slop.

If you do any serious work with floats, you will want to learn about
significant figures and the use of the very essential round() function in
whatever language you are using. If you do, you will find that this
situation is really a math thing and has little to do with computers or
any particular language. It's the exact same problem one runs into when
using a slide rule.

If anyone remembers those things :-)
Thomas Bartkus