[Tutor] Rounding a float to n significant digits

Dick Moores rdm at rcblue.com
Fri Dec 1 05:44:26 CET 2006


At 02:57 PM 11/30/2006, wesley chun wrote:
>i think on a 32-bit platform, C doubles (IEEE754) are limited to 10 **
>308.25 which is pretty close to 2 ** 1024
>
>-wesley

Yes, that's close.

Did some pinching down:

======================================
 >>> 2**1023.99999999999994
1.7976931348621742e+308
 >>> 2**1023.99999999999995
Traceback (most recent call last):
   File "<string>", line 1, in <string>
OverflowError: (34, 'Result too large')

 >>> 10**308.254715559916718359
1.7976931348620926e+308
 >>> 10**308.254715559916718360
Traceback (most recent call last):
   File "<string>", line 1, in <string>
OverflowError: (34, 'Result too large')
==========================================

But isn't there a PRECISE answer to my question? Or is it OT?

Dick






More information about the Tutor mailing list