Strange rounding problem

[Marc]

I have another question for the penguins. I understand now how this works.
I'm a little confused as to why. I'm sorry If I'm going over retreat ground,
but I can't find a good explanation of this anywhere.

I completely understand that if you evalute 1/3 then you get a number that
goes off into infinity asymptotically approaching a value of 1. And then
when you evaluate that number the system will print as many numbers as you
can handle, and then round.

But I didn't make a calculation to achieve this number. I entered it to be
an exact value. Now I understand that when a float is used it must be
represented by a fixed number of binary digits. In the things I have read,
this has been cited as the reason for why things get magically rounded. By I
know that binary digits aren't incapable of holding exact numbers. If that
were true, there wouldn't be an accurate computer in the world. So I still
don't understand that if I enter a number to be an exact value of .00000096,
why it can't be stored as 9.60000000e-007?

Thanks,
Marc


"Dennis Lee Bieber" <wlfraed at ix.netcom.com> wrote in message
news:st0bk-2j3.ln1 at beastie.ix.netcom.com...
> Marc fed this fish to the penguins on Friday 14 March 2003 03:40 pm:
>
> >
> > This appears to be confusing. The variable is set to .00000096 but is
> > passed through as 9.5999999999999991e-007. It appears accurate when
> > printed before the value is returned, is inaccurate when the full
> > return list is printed, accurate when it's printed by itself after the
> > return, and then inaccurate again when the division is made. Why does
> > it do this, and how can I avoid it?
> >
>         Because it is a binary floating point value. You run into the
binary
> equivalent of decimal 1/3.
>
>         Is 1/3
> = 0.3
> = 0.33
> = 0.3333333
> = 0.333333333333333333333333333333333333333333333333333333
>
> If you multiply back by 3 you get 0.9, 0.99, 0.99999999999...., but you
> do not get 1.0!
>
>         How to avoid it? NEVER test a floating point for equality to some
> value. Test for the difference between the two values being LESS than
> some predetermined epsilon.
>
> >>> gap = .00000096
> >>> gap
> 9.5999999999999991e-07
>
>         As you can see, it was NOT set .00000096 from the beginning. You
only
> thought it was because str() tries to be nice, and rounds the value to
> something with fewer decimal places.
>
> >>> print "GAP IN ROUTINE " + repr(gap)
> GAP IN ROUTINE 9.5999999999999991e-07
> >>> print "GAP IN ROUTINE " + str(gap)
> GAP IN ROUTINE 9.6e-07
> >>>
>
> --
>  > ============================================================== <
>  >   wlfraed at ix.netcom.com  | Wulfraed  Dennis Lee Bieber  KD6MOG <
>  >      wulfraed at dm.net     |       Bestiaria Support Staff       <
>  > ============================================================== <
>  >        Bestiaria Home Page: http://www.beastie.dm.net/         <
>  >            Home Page: http://www.dm.net/~wulfraed/             <
>