Floating point "g" format not stripping trailing zeros

[Ian Kelly]

On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nikšić <hniksic at gmail.com> wrote:
>> >>>>> from decimal import Decimal as D
>> >>>>> x = D(1)/D(999)
>> >>>>> '{:.15g}'.format(x)
>> >>
>> >> '0.00100100100100100'
> [...]
>> > I'd say it's a bug.  P is 15, you've got 17 digits after the decimal place
>> > and two of those are insignificant trailing zeros.
>>
>> Actually it's the float version that doesn't match the documentation.
>> In the decimal version, sure there are 17 digits after the decimal
>> place there, but the first two -- which are leading zeroes -- would
>> not normally be considered significant.
>
> {:.15g} is supposed to give 15 digits of precision, but with trailing
> zeros removed.

The doc says with "insignificant" trailing zeros removed, not all
trailing zeros.

> For example, '{:.15g}'.format(Decimal('0.5')) should
> yield '0.5', not '0.500000000000000' -- and, it indeed does.  It is
> only for some numbers that trailing zeros are not removed, which looks
> like a bug.  The behavior of floats matches both the documentation and
> other languages using the 'g' decimal format, such as C.

Ah, I see now what's going on here. With floats, there is really no
notion of significant digits. The values 0.5 and 0.50000 are
completely equivalent. With decimals, that's not exactly true; if you
give the decimal a trailing zero then you are telling it that the zero
is significant.

>>> Decimal('0.5')
Decimal('0.5')
>>> Decimal('0.50000')
Decimal('0.50000')
>>> Decimal('0.5').as_tuple()
DecimalTuple(sign=0, digits=(5,), exponent=-1)
>>> Decimal('0.50000').as_tuple()
DecimalTuple(sign=0, digits=(5, 0, 0, 0, 0), exponent=-5)

These are distinct; the decimal knows how many significant digits you
passed it. As a result, these are also distinct:

>>> '{:.4g}'.format(Decimal('0.5'))
'0.5'
>>> '{:.4g}'.format(Decimal('0.50000'))
'0.5000'

Now what happens in your original example of 1/999? The default
decimal context uses 28 digits of precision, so the result of that
calculation will have 28 significant digits in it.

>>> decimal.getcontext().prec
28
>>> Decimal(1) / Decimal(999)
Decimal('0.001001001001001001001001001001')

When you specify the a precision of 15 in your format string, you're
telling it to take the first 15 of those. It doesn't care that the
last couple of those are zeros, because as far as it's concerned,
those digits are significant.