[Tutor] Rounding a float to n significant digits
Dick Moores
rdm at rcblue.com
Fri Dec 1 05:44:26 CET 2006
At 02:57 PM 11/30/2006, wesley chun wrote:
>i think on a 32-bit platform, C doubles (IEEE754) are limited to 10 **
>308.25 which is pretty close to 2 ** 1024
>
>-wesley
Yes, that's close.
Did some pinching down:
======================================
>>> 2**1023.99999999999994
1.7976931348621742e+308
>>> 2**1023.99999999999995
Traceback (most recent call last):
File "<string>", line 1, in <string>
OverflowError: (34, 'Result too large')
>>> 10**308.254715559916718359
1.7976931348620926e+308
>>> 10**308.254715559916718360
Traceback (most recent call last):
File "<string>", line 1, in <string>
OverflowError: (34, 'Result too large')
==========================================
But isn't there a PRECISE answer to my question? Or is it OT?
Dick
More information about the Tutor
mailing list