Decimal() instead of float?

Carsten Haese carsten at uniqsys.com
Fri Nov 17 17:20:49 EST 2006


On Fri, 2006-11-17 at 16:51 -0500, Michael B. Trausch wrote:
> On Fri, 2006-11-17 at 21:25 +0100, Fredrik Lundh wrote: 
> > > Some of the lat/long pairs that I have used seem to come out fine, but 
> > > some do not.  Because the mathmatics used with them involve complex 
> > > equations when determining distance and the like, any error gets 
> > > massively compounded before a final result is evident.
> > 
> > sorry, but I don't think you have the slightest idea what you're doing, 
> > really.
> > 
> 
> Sure, I do.  Let's say that I want to work with the latitude
> 33.6907570.  In Python, that number can not be stored exactly without
> the aid of decimal.Decimal().
> 
> >>> 33.6907570
> 33.690756999999998
> >>> 
> 
> As you can see, it loses accuracy after the 6th decimal place.  That's
> not good enough:  I have 8 numbers that need to be exact, and I am
> only getting six.

Really?

>>> x = 33.6907570
>>> for i in range(7,16): print "%.*f" % (i,x)
... 
33.6907570
33.69075700
33.690757000
33.6907570000
33.69075700000
33.690757000000
33.6907570000000
33.69075700000000
33.690756999999998

Looks to me like you're getting, uh, 14 decimal digits, for a total of
16 significant digits. What kinds of operations are you performing that
compound this error by seven orders of magnitude?

-Carsten





More information about the Python-list mailing list