Why the nonsense number appears?

Steve Horsley steve.horsley at gmail.com
Mon Oct 31 15:50:18 EST 2005


Ben O'Steen wrote:
> On Mon, October 31, 2005 10:23, Sybren Stuvel said:
>> Ben O'Steen enlightened us with:
>>> Using decimal as opposed to float sorts out this error as floats are
>>> not built to handle the size of number used here.
>> They can handle the size just fine. What they can't handle is 1/1000th
>> precision when using numbers in the order of 1e10.
>>
> 
> I used the word 'size' here incorrectly, I intended to mean 'length'
> rather than numerical value. Sorry for the confusion :)
> 

Sybren is right. The problem is not the length or the size, it's 
the fact that 0.039 cannot be represented exactly in binary, in 
just the same way that 1/3 cannot be represented exactly in 
decimal. They both give recurring numbers. If you truncate those 
recurring numbers to a finite number of digits, you lose 
precision. And this shows up when you convert the inaccurate 
number from binary into decimal representation where an exact 
representation IS possible.

Steve



More information about the Python-list mailing list