Decimal() instead of float?
Carsten Haese
carsten at uniqsys.com
Fri Nov 17 17:20:49 EST 2006
On Fri, 2006-11-17 at 16:51 -0500, Michael B. Trausch wrote:
> On Fri, 2006-11-17 at 21:25 +0100, Fredrik Lundh wrote:
> > > Some of the lat/long pairs that I have used seem to come out fine, but
> > > some do not. Because the mathmatics used with them involve complex
> > > equations when determining distance and the like, any error gets
> > > massively compounded before a final result is evident.
> >
> > sorry, but I don't think you have the slightest idea what you're doing,
> > really.
> >
>
> Sure, I do. Let's say that I want to work with the latitude
> 33.6907570. In Python, that number can not be stored exactly without
> the aid of decimal.Decimal().
>
> >>> 33.6907570
> 33.690756999999998
> >>>
>
> As you can see, it loses accuracy after the 6th decimal place. That's
> not good enough: I have 8 numbers that need to be exact, and I am
> only getting six.
Really?
>>> x = 33.6907570
>>> for i in range(7,16): print "%.*f" % (i,x)
...
33.6907570
33.69075700
33.690757000
33.6907570000
33.69075700000
33.690757000000
33.6907570000000
33.69075700000000
33.690756999999998
Looks to me like you're getting, uh, 14 decimal digits, for a total of
16 significant digits. What kinds of operations are you performing that
compound this error by seven orders of magnitude?
-Carsten
More information about the Python-list
mailing list