[Python-Dev] PEP 550 v4

[Eric Snow]

Thanks for the update.  Comments in-line below.

-eric

On Fri, Aug 25, 2017 at 4:32 PM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> [snip]
>
> This PEP adds a new generic mechanism of ensuring consistent access
> to non-local state in the context of out-of-order execution, such
> as in Python generators and coroutines.
>
> Thread-local storage, such as ``threading.local()``, is inadequate for
> programs that execute concurrently in the same OS thread.  This PEP
> proposes a solution to this problem.

> [snip]
>
> In regular, single-threaded code that doesn't involve generators or
> coroutines, context variables behave like globals::
>
> [snip]
>
> In multithreaded code, context variables behave like thread locals::
>
> [snip]
>
> In generators, changes to context variables are local and are not
> visible to the caller, but are visible to the code called by the
> generator.  Once set in the generator, the context variable is
> guaranteed not to change between iterations::

With threads we have a directed graph of execution, rooted at the root
thread, branching with each new thread and merging with each .join().
Each thread gets its own copy of each threading.local, regardless of
the relationship between branches (threads) in the execution graph.

With async (and generators) we also have a directed graph of
execution, rooted in the calling thread, branching with each new async
call.  Currently there is no equivalent to threading.local for the
async execution graph.  This proposal involves adding such an
equivalent.

However, the proposed solution isn''t quite equivalent, right?  It
adds a concept of lookup on the chain of namespaces, traversing up the
execution graph back to the root.  threading.local does not do this.
Furthermore, you can have more than one threading.local per thread.
>From what I read in the PEP, each node in the execution graph has (at
most) one Execution Context.

The PEP doesn't really say much about these differences from
threadlocals, including a rationale.  FWIW, I think such a COW
mechanism could be useful.  However, it does add complexity to the
feature.  So a clear explanation in the PEP of why it's worth it would
be valuable.

> [snip]
>
> The following new Python APIs are introduced by this PEP:
>
> 1. The ``sys.new_context_var(name: str='...')`` function to create
>    ``ContextVar`` objects.
>
> 2. The ``ContextVar`` object, which has:
>
>    * the read-only ``.name`` attribute,
>    * the ``.lookup()`` method which returns the value of the variable
>      in the current execution context;
>    * the ``.set()`` method which sets the value of the variable in
>      the current execution context.
>
> 3. The ``sys.get_execution_context()`` function, which returns a
>    copy of the current execution context.
>
> 4. The ``sys.new_execution_context()`` function, which returns a new
>    empty execution context.
>
> 5. The ``sys.new_logical_context()`` function, which returns a new
>    empty logical context.
>
> 6. The ``sys.run_with_execution_context(ec: ExecutionContext,
>    func, *args, **kwargs)`` function, which runs *func* with the
>    provided execution context.
>
> 7. The ``sys.run_with_logical_context(lc:LogicalContext,
>    func, *args, **kwargs)`` function, which runs *func* with the
>    provided logical context on top of the current execution context.

#1-4 are consistent with a single EC per Python thread.  However, #5-7
imply that more than one EC per thread is supported, but only one is
active in the current execution stack (notably the EC is rooted at the
calling frame).  threading.local provides a much simpler mechanism but
does not support the chained context (COW) semantics...