array precision

[Barry Drake]

Should have added one more line to my previous post.  Here is the
entire story:

Python 2.1.1 (#20, Jul 26 2001, 11:38:51) [MSC 32 bit (Intel)] on
win32
Type "copyright", "credits" or "license" for more information.
IDLE 0.8 -- press F1 for help
>>> import math
>>> n = 10
>>> nf = 10.0
>>> a = 1.0
>>> b = 0.5
>>> a/n
0.10000000000000001
>>> b + a/n
0.59999999999999998
>>> a/nf
0.10000000000000001
>>> b + a/nf
0.59999999999999998
>>> b
0.5
>>> 1.0 + (0.6 - (b + a/n))
1.0
>>> 

This last line is the crucial one.  It's says that within the
numerical precision of the machine b + a/n "looks and behaves as close
as I can make it" like 0.6.  Remember 64 bit doubles are finite.

Barry Drake

"Hugo Martires" <hmartires at cultalg.pt> wrote in message news:<st888.7971$oI4.33917985 at newsserver.ip.pt>...
> but i don´t need all the numbers, i only need 0.6
> 
> 
> "Jason Orendorff" <jason at jorendorff.com> wrote in message
> news:mailman.1012956250.23108.python-list at python.org...
> > Hugo Martires writes:
> > > suposing that n=10 we get: myArray[0] = 0.600000023 ("something
> > > like this")
> > >
> > > but if i do: myArray[i] + (1.0/n) it give's me 0.6
> > >
> > > what's the problem of the array, and how can i solve this ?
> >
> > It's only half of the normal precision.  From the documentation:
> >
> >     The following type codes are defined:
> >         Type code   C Type             Minimum size in bytes
> >         'f'         floating point     4
> >         'd'         floating point     8
> >
> > Try using 'd' instead of 'f'.
> >
> > ## Jason Orendorff    http://www.jorendorff.com/
> >
> >