[Python-Dev] PEP 550 v4: coroutine policy

[Yury Selivanov]

On Tue, Aug 29, 2017 at 2:40 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 28 Aug 2017 17:24:29 -0400
> Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>> Long story short, I think we need to rollback our last decision to
>> prohibit context propagation up the call stack in coroutines.  In PEP
>> 550 v3 and earlier, the following snippet would work just fine:
>>
>>    var = new_context_var()
>>
>>    async def bar():
>>        var.set(42)
>>
>>    async def foo():
>>        await bar()
>>        assert var.get() == 42   # with previous PEP 550 semantics
>>
>>    run_until_complete(foo())
>>
>> But it would break if a user wrapped "await bar()" with "wait_for()":
>>
>>    var = new_context_var()
>>
>>    async def bar():
>>        var.set(42)
>>
>>    async def foo():
>>        await wait_for(bar(), 1)
>>        assert var.get() == 42  # AssertionError !!!
>>
>>    run_until_complete(foo())
>>
> [...]
>
>> So I guess we have no other choice other than reverting this spec
>> change for coroutines.  The very first example in this email should
>> start working again.
>
> What about the second one?

Just to be clear: in the next revision of the PEP, the first example
will work without an AssertionError; second example will keep raising
an AssertionError.

> Why wouldn't the bar() coroutine inherit
> the LC at the point it's instantiated (i.e. where the synchronous bar()
> call is done)?

We want tasks to have their own isolated contexts.  When a task
is started, it runs its code in parallel with its "parent" task.  We want
each task to have its own isolated EC (OS thread/TLS vs
async task/EC analogy), otherwise the EC of "foo()" will be randomly
changed by the tasks it spawned.

wait_for() in the above example creates an asyncio.Task implicitly,
and that's why we don't see 'var' changed to '42' in foo().

This is a slightly complicated case, but it's addressable with a good
documentation and recommended best practices.

>
>> This means that PEP 550 will have a caveat for async code: don't rely
>> on context propagation up the call stack, unless you are writing
>> __aenter__ and __aexit__ that are guaranteed to be called without
>> being wrapped into a Task.
>
> Hmm, sorry for being a bit slow, but I'm not sure what this
> sentence implies.  How is the user supposed to know whether something
> will be wrapped into a Task (short of being an expert in asyncio
> internals perhaps)?
>
> Actually, if could whip up an example of what you mean here, it would
> be helpful I think :-)

__aenter__ won't ever be wrapped in a task because its called by
the interpreter.

    var = new_context_var()

    class MyAsyncCM:

            def __aenter__(self):
                   var.set(42)

    async with MyAsyncCM():
          assert var.get() == 42

The above snippet will always work as expected.

We'll update the PEP with thorough explanation of all these
nuances in the semantics.

Yury