[Python-Dev] PEP 550 v4: Decimal examples and performance (was: Re: PEP 550 v4)

[Yury Selivanov]

On Sat, Aug 26, 2017 at 7:45 AM, Stefan Krah <stefan at bytereef.org> wrote:
>
> Hi,
>
> thanks, on the whole this is *much* easier to understand.

Thanks!

> I'll add some comments on the decimal examples. The thing is, decimal
> is already quite tricky and people do read PEPs long after they have
> been accepted, so they should probably reflect best practices.

Agree.

[..]
>> Some languages, that support coroutines or generators, recommend
>> passing the context manually as an argument to every function, see [1]_
>> for an example.  This approach, however, has limited use for Python,
>> where there is a large ecosystem that was built to work with a TLS-like
>> context.  Furthermore, libraries like ``decimal`` or ``numpy`` rely
>> on context implicitly in overloaded operator implementations.
>
> I'm not sure why this approach has limited use for decimal:
>
>
> from decimal import *
>
> def fractions(precision, x, y):
>     ctx = Context(prec=precision)
>     yield ctx.divide(Decimal(x), Decimal(y))
>     yield ctx.divide(Decimal(x), Decimal(y**2))
>
> g1 = fractions(precision=2, x=1, y=3)
> g2 = fractions(precision=6, x=2, y=3)
> print(list(zip(g1, g2)))

Because you have to know the limitations of implicit decimal context
to make this choice. Most people don't (at least from my experience).

> This is the first thing I'd do when writing async-safe code.

Because you know the decimal module very well :)

>
> Again, people do read PEPs.  So if an asyncio programmer without any
> special knowledge of decimal reads the PEP, he probably assumes that
> localcontext() is currently the only option, while the safer and
> easy-to-reason-about context methods exist.

I agree.

>
>
>> Now, let's revisit the decimal precision example from the `Rationale`_
>> section, and see how the execution context can improve the situation::
>>
>>     import decimal
>>
>>     decimal_prec = new_context_var()  # create a new context variable
>>
>>     # Pre-PEP 550 Decimal relies on TLS for its context.
>>     # This subclass switches the decimal context storage
>>     # to the execution context for illustration purposes.
>>     #
>>     class MyDecimal(decimal.Decimal):
>>         def __init__(self, value="0"):
>>             prec = decimal_prec.lookup()
>>             if prec is None:
>>                 raise ValueError('could not find decimal precision')
>>             context = decimal.Context(prec=prec)
>>             super().__init__(value, context=context)
>
> As I understand it, the example creates a context with a custom precision
> and attempts to use that context to create a Decimal.
>
> This doesn't switch the actual decimal context. Secondly, the precision in
> the context argument to the Decimal() constructor has no effect --- the
> context there is only used for error handling.
>
> Lastly, if the constructor *did* use the precision, one would have to be
> careful about double rounding when using MyDecimal().
>
>
> I get that this is supposed to be for illustration only, but please let's
> be careful about what people might take away from that code.

In the next iteration of the PEP we'll remove decimal examples and
replace them with something with simpler semantics.  This is clearly
the best choice now.

>> This generic caching approach is similar to what the current C
>> implementation of ``decimal`` does to cache the the current decimal
>> context, and has similar performance characteristics.
>
> I think it'll work, but can we agree on hard numbers like max 2% slowdown
> for the non-threaded case and 4% for applications that only use threads?

I'd be *very* surprised if wee see any noticeable slowdown at all. The
way ContextVars will implement caching is very similar to the trick
you use now.

Yury