precision problems in base conversion of rational numbers

mensanator at aol.com mensanator at aol.com
Wed Jul 6 13:32:22 EDT 2005



Raymond Hettinger wrote:
> [Terry Hancock]
> > > Needless to say, the conventional floating point numbers in Python
> > > are actually stored as *binary*, which is why there is a "decimal"
> > > module (which is new).
> > >
> > > If you're going to be converting between bases anyway, it probably
> > > makes little difference whether you are using the decimal module
> > > or not, of course.  You'll have the same problems the conventional
> > > float numbers do.
>
> Not really.  floats won't do because they may not have sufficient
> precision to differentiate rational values falling close the split
> between representable values in a given base.  The decimal module
> offers arbitrarily large precision for making sure the error-term is
> small enough to not make a difference.
>
>
>
> [Brian van den Broek]
> > Thanks. mensanator provided the actual formula for my case. I had a
> > "magic number" in my code by which I multiplied my desired level of
> > "number of places in the representation" to obtain the value for
> > decimal.getcontext.prec.
> >
> > mensanator wrote:
> >
> > > The value you want for x is log(17)/log(10) =
> > > 1.2304489213782739285401698943283
> >
> > where x was my "magic number". I've not had a chance to think it
> > through yet, but I feel confident that given the formula, I'll be able
> > to work out *why* that is the formula I need.
>
> That "formula" just gives a starting point estimate.  The required
> decimal precision may be much higher.  If the rational falls very close
> to the half-way point between two representable numbers, the
> calculation needs to be retried with increased precision until the
> split-point is definitive (when the error-term becomes less than the
> distance to the next representable value).
>
> For a simple example, convert both 10247448370872321 and
> 10247448370872319 from base ten to 4 digits of hex.  The calculations
> need to be carried out to 15 places of hex (or 17 places of decimal)
> just to determine whether the fourth hex digit is a 7 or 8:
>
>     >>> hex(10247448370872321)
>     '0x24680000000001L'
>     >>> hex(10247448370872319)
>     '0x2467ffffffffffL'

I don't understand your simple example.

log(10)/log(16) = 0.83048202372184058696757985737235

Multiplying by the number of decimal digits (17) gives

14.11819440327128997844885757533

Rounding up to an integer digit count, we need need 15 hex digits.
Isn't that exactly what you concluded? Where does that 4 hex digit
stuff come from?

>
> For an example of using decimal with iteratively increasing precision,
> see the dsum() recipe at
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/393090 .
> 
> 
> Raymond




More information about the Python-list mailing list