pricision of string.atof?

[Raymond Hettinger]

"Elvis Chen" <chene at cs.queensu.ca> wrote
> I'm working on some numerical analysis that requires rather good precision
> on calculation.

<snip>

> >>> A
> 0.42399999999999999
> >>> print A
> 0.424
>
> My question is, what is the TRUE value of A when I use it in calculation?
> Is it indeed 0.424 or not?

Neither is the TRUE value.  Variable A is stored in a fixed length floating
point format and .424 is not exactly representable so it is pushed to a
neighboring representable (sometimes the nearest, sometimes the next
lower, or sometimes the next higher depending on your implementation
of floating point).

That limitation, my friend, is the subject of the first chapter of every
numerical analyis book I've ever seen.  It is the core conceptual
challenge in a field full of challenges.

>  If not, what can I do to work around this
> problem?

If you value that last digit of precision more than you value time,
use Python's long ints to represent a scaled version of the number
to as many digits are you prefer.  For example, to compute many
decimal places of 31.0 / 7.0:

>>> digits = 30
>>> scale = 10L ** digits
>>> num, den = 31, 7
>>> print (num*scale) / den
4428571428571428571428571428571
>>> print float(num)/ den
4.42857142857


Raymond Hettinger