Strange rounding problem

[Bengt Richter]

TOn 15 Mar 2003 17:55:33 GMT, Grant Edwards <grante at visi.com> wrote:

>In article <TrCcnZMtctRJ_u6jXTWc3Q at comcast.com>, Marc wrote:
>
>> But I didn't make a calculation to achieve this number. I entered it to be
>> an exact value. [...]
>> By I know that binary digits aren't incapable of holding exact numbers.
>
>Wrong. Partly.  Some numbers that are exactly representable in base 10 are
>not exactly representble in base 2.  For example, the number 0.1 (base 10)
>can't be exactly represented in base 2.  Just like 0.1 (base 3) can't be
>exactly represented as a decimal value in base 10: it's 0.33333 (repeats
>forever).
>
>> If that were true, there wouldn't be an accurate computer in the world.
>
>There arent.  When dealing with floating point, you must assume that
>_no_value_is_exact_.

Please qualify such statements, so you don't encourage superstitions ;-)

As I'm sure you know, all legal (w.r.t. some standard like IEEE 754)
floating point numbers are represented as specific bit patterns, and
those patterns correspond EXACTLY to specific EXACT real numbers.

It may be that no bit pattern is available to represent the exact
number one wants to represent in floating point, but that doesn't
mean that the nearest available numbers aren't exact in themselves.

>
>> So I still don't understand that if I enter a number to be an exact value of
>> .00000096, why it can't be stored as 9.60000000e-007?
Actually, it can, but that is not what happens if you store it as a floating double.
>
>Becausing you entered it in base 10, and it's stored in base 2.  Just
>because it's an exact value in base 10 it doesn't mean it can be represented
>exactly in base 2.
>
Right. It never ends ;-) Sigh.

Regards,
Bengt Richter