[Python-Dev] PEP 550 v4: Decimal examples and performance (was: Re: PEP 550 v4)

[Stefan Krah]

Hi,

thanks, on the whole this is *much* easier to understand.

I'll add some comments on the decimal examples. The thing is, decimal
is already quite tricky and people do read PEPs long after they have
been accepted, so they should probably reflect best practices.


On Fri, Aug 25, 2017 at 06:32:22PM -0400, Yury Selivanov wrote:
> Unfortunately, TLS does not work well for programs which execute
> concurrently in a single thread.  A Python generator is the simplest
> example of a concurrent program.  Consider the following::
> 
>     def fractions(precision, x, y):
>         with decimal.localcontext() as ctx:
>             ctx.prec = precision
>             yield Decimal(x) / Decimal(y)
>             yield Decimal(x) / Decimal(y**2)
> 
>     g1 = fractions(precision=2, x=1, y=3)
>     g2 = fractions(precision=6, x=2, y=3)
> 
>     items = list(zip(g1, g2))
> 
> The expected value of ``items`` is::

"Many people (wrongly) expect the values of ``items`` to be::" ;)

> 
>     [(Decimal('0.33'), Decimal('0.666667')),
>      (Decimal('0.11'), Decimal('0.222222'))]


> Some languages, that support coroutines or generators, recommend
> passing the context manually as an argument to every function, see [1]_
> for an example.  This approach, however, has limited use for Python,
> where there is a large ecosystem that was built to work with a TLS-like
> context.  Furthermore, libraries like ``decimal`` or ``numpy`` rely
> on context implicitly in overloaded operator implementations.

I'm not sure why this approach has limited use for decimal:


from decimal import *

def fractions(precision, x, y):
    ctx = Context(prec=precision)
    yield ctx.divide(Decimal(x), Decimal(y))
    yield ctx.divide(Decimal(x), Decimal(y**2))

g1 = fractions(precision=2, x=1, y=3)
g2 = fractions(precision=6, x=2, y=3)
print(list(zip(g1, g2)))


This is the first thing I'd do when writing async-safe code.

Again, people do read PEPs.  So if an asyncio programmer without any
special knowledge of decimal reads the PEP, he probably assumes that
localcontext() is currently the only option, while the safer and
easy-to-reason-about context methods exist.


> Now, let's revisit the decimal precision example from the `Rationale`_
> section, and see how the execution context can improve the situation::
> 
>     import decimal
> 
>     decimal_prec = new_context_var()  # create a new context variable
> 
>     # Pre-PEP 550 Decimal relies on TLS for its context.
>     # This subclass switches the decimal context storage
>     # to the execution context for illustration purposes.
>     #
>     class MyDecimal(decimal.Decimal):
>         def __init__(self, value="0"):
>             prec = decimal_prec.lookup()
>             if prec is None:
>                 raise ValueError('could not find decimal precision')
>             context = decimal.Context(prec=prec)
>             super().__init__(value, context=context)

As I understand it, the example creates a context with a custom precision
and attempts to use that context to create a Decimal.

This doesn't switch the actual decimal context. Secondly, the precision in
the context argument to the Decimal() constructor has no effect --- the
context there is only used for error handling.

Lastly, if the constructor *did* use the precision, one would have to be
careful about double rounding when using MyDecimal().


I get that this is supposed to be for illustration only, but please let's
be careful about what people might take away from that code.


> This generic caching approach is similar to what the current C
> implementation of ``decimal`` does to cache the the current decimal
> context, and has similar performance characteristics.

I think it'll work, but can we agree on hard numbers like max 2% slowdown
for the non-threaded case and 4% for applications that only use threads?

I'm a bit cautious because other C-extension state-managing PEPs didn't
come close to these figures.


Stefan Krah