[Tutor] Rounding a float to n significant digits
Dick Moores
rdm at rcblue.com
Thu Nov 30 21:52:32 CET 2006
At 11:19 PM 11/27/2006, Dick Moores wrote:
>I just dug this Tim Smith creation out of the Tutor archive.
>
>def round_to_n(x, n):
> """
> Rounds float x to n significant digits, in scientific notation.
> Written by Tim Peters. See his Tutor list post of 7/3/04 at
> http://mail.python.org/pipermail/tutor/2004-July/030324.html
> """
> if n < 1:
> raise ValueError("number of significant digits must be >= 1")
> return "%.*e" % (n-1, x)
>
>Thought others might find it of use.
>
>Dick Moores
I've run into the limitation on the size of an int that can be
converted to a float. Out of curiosity I've tried to close in on what
that limit is:
>>> round_to_n(2*10**308,4)
Traceback (most recent call last):
File "E:\Program Files\Wing IDE Personal
2.1\src\debug\server\_sandbox.py", line 1, in <module>
# Used internally for debug sandbox under external interpreter
File "e:\Python25\Lib\site-packages\mine\mycalc.py", line 35, in round_to_n
return "%.*e" % (n-1, x)
TypeError: float argument required
>>> round_to_n(1.75*10**308,4)
'1.750e+308'
>>> round_to_n(1.2*10**308,4)
'1.200e+308'
>>> round_to_n(1.87*10**308,4)
'1.#IOe+000'
>>> round_to_n(1.81*10**308,4)
'1.#IOe+000'
>>> round_to_n(1.78*10**308,4)
'1.780e+308'
So I've run across the curious '1.#IOe+000'. What is that? And what
is the exact limit? I suppose it's some power of 2? But 2**1023 is
too low; 2**1024 is too high; so is 2**1024-1.
Dick Moores
More information about the Tutor
mailing list