Floating point errors?

[David Lees]

John,

You are correct that my terminology is wrong (too many years since 9th
grade).  The problem is that 3/10, when divided out in base 10 has a
repeating decimal value of zero.  However, when you do the division base
2, the repeating decimal is non-zero, so you end up having to truncate
at however many bits go into the floating point mantissa.

david lees

"John J. Lee" wrote:
> 
> On Thu, 10 May 2001, David Lees wrote:
> 
> > Complexity has nothing to do with rounding.  Try:
> >
> > >>> 2.3-2.0
> > 0.29999999999999982
> >
> >
> > Rounding is exactly the issue and this looks like the old classic
> > problem, namely rational numbers in one base (say 10) can be irrational
> > in another base (say 2)
> 
> Rationality is a property of numbers, not their representation.  0.3 is a
> rational number because you can represent it as a ratio of two integers,
> which is as true in binary as it is in decimal.  I know what you mean,
> though.
> 
> John