Floating point "g" format not stripping trailing zeros

[Mark Lawrence]

On 12/02/2015 23:46, Ian Kelly wrote:
> On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nikšić <hniksic at gmail.com> wrote:
>>>>>>>> from decimal import Decimal as D
>>>>>>>> x = D(1)/D(999)
>>>>>>>> '{:.15g}'.format(x)
>>>>>
>>>>> '0.00100100100100100'
>> [...]
>>>> I'd say it's a bug.  P is 15, you've got 17 digits after the decimal place
>>>> and two of those are insignificant trailing zeros.
>>>
>>> Actually it's the float version that doesn't match the documentation.
>>> In the decimal version, sure there are 17 digits after the decimal
>>> place there, but the first two -- which are leading zeroes -- would
>>> not normally be considered significant.
>>
>> {:.15g} is supposed to give 15 digits of precision, but with trailing
>> zeros removed.
>
> The doc says with "insignificant" trailing zeros removed, not all
> trailing zeros.
>
>> For example, '{:.15g}'.format(Decimal('0.5')) should
>> yield '0.5', not '0.500000000000000' -- and, it indeed does.  It is
>> only for some numbers that trailing zeros are not removed, which looks
>> like a bug.  The behavior of floats matches both the documentation and
>> other languages using the 'g' decimal format, such as C.
>
> Ah, I see now what's going on here. With floats, there is really no
> notion of significant digits. The values 0.5 and 0.50000 are
> completely equivalent. With decimals, that's not exactly true; if you
> give the decimal a trailing zero then you are telling it that the zero
> is significant.
>
>>>> Decimal('0.5')
> Decimal('0.5')
>>>> Decimal('0.50000')
> Decimal('0.50000')
>>>> Decimal('0.5').as_tuple()
> DecimalTuple(sign=0, digits=(5,), exponent=-1)
>>>> Decimal('0.50000').as_tuple()
> DecimalTuple(sign=0, digits=(5, 0, 0, 0, 0), exponent=-5)
>
> These are distinct; the decimal knows how many significant digits you
> passed it. As a result, these are also distinct:
>
>>>> '{:.4g}'.format(Decimal('0.5'))
> '0.5'
>>>> '{:.4g}'.format(Decimal('0.50000'))
> '0.5000'
>
> Now what happens in your original example of 1/999? The default
> decimal context uses 28 digits of precision, so the result of that
> calculation will have 28 significant digits in it.
>
>>>> decimal.getcontext().prec
> 28
>>>> Decimal(1) / Decimal(999)
> Decimal('0.001001001001001001001001001001')
>
> When you specify the a precision of 15 in your format string, you're
> telling it to take the first 15 of those. It doesn't care that the
> last couple of those are zeros, because as far as it's concerned,
> those digits are significant.
>

I still think it's a bug as the 'p' being referred to in the OP's 
original message is "The precision is a decimal number indicating how 
many digits should be displayed after the decimal point for a floating 
point value formatted with 'f' and 'F', or before and after the decimal 
point for a floating point value formatted with 'g' or 'G'".  In other 
words is has nothing to do with the precision of the underlying number.


-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence