Why the nonsense number appears?
Steve Horsley
steve.horsley at gmail.com
Mon Oct 31 15:50:18 EST 2005
Ben O'Steen wrote:
> On Mon, October 31, 2005 10:23, Sybren Stuvel said:
>> Ben O'Steen enlightened us with:
>>> Using decimal as opposed to float sorts out this error as floats are
>>> not built to handle the size of number used here.
>> They can handle the size just fine. What they can't handle is 1/1000th
>> precision when using numbers in the order of 1e10.
>>
>
> I used the word 'size' here incorrectly, I intended to mean 'length'
> rather than numerical value. Sorry for the confusion :)
>
Sybren is right. The problem is not the length or the size, it's
the fact that 0.039 cannot be represented exactly in binary, in
just the same way that 1/3 cannot be represented exactly in
decimal. They both give recurring numbers. If you truncate those
recurring numbers to a finite number of digits, you lose
precision. And this shows up when you convert the inaccurate
number from binary into decimal representation where an exact
representation IS possible.
Steve
More information about the Python-list
mailing list