[Python-Dev] PEP 550 v4

[Yury Selivanov]

On Mon, Aug 28, 2017 at 7:16 PM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> On Mon, Aug 28, 2017 at 6:56 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Yury Selivanov wrote:
>>>
>>> I saying that the following should not work:
>>>
>>>     def nested_gen():
>>>         set_some_context()
>>>         yield
>>>
>>>     def gen():
>>>        # some_context is not set
>>>        yield from nested_gen()
>>>        # use some_context ???
>>
>>
>> And I'm saying it *should* work, otherwise it breaks
>> one of the fundamental principles on which yield-from
>> is based, namely that 'yield from foo()' should behave
>> as far as possible as a generator equivalent of a
>> plain function call.
>>
>
> Consider the following generator:
>
>
>       def gen():
>          with decimal.context(...):
>             yield
>
>
> We don't want gen's context to leak to the outer scope -- that's one
> of the reasons why PEP 550 exists.  Even if we do this:
>
>      g = gen()
>      next(g)
>      # the decimal.context won't leak out of gen
>
> So a Python user would have a mental model: context set in generators
> doesn't leak.
>
> Not, let's consider a "broken" generator:
>
>      def gen():
>           decimal.context(...)
>           yield
>
> If we iterate gen() with next(), it still won't leak its context.  But
> if "yield from" has semantics that you want -- "yield from" to be just
> like function call -- then calling
>
>      yield from gen()
>
> will corrupt the context of the caller.
>
> I simply want consistency.  It's easier for everybody to say that
> generators never leaked their context changes to the outer scope,
> rather than saying that "generators can sometimes leak their context".

Adding to the above: there's a fundamental reason why we can't make
"yield from" transparent for EC modifications.

While we want "yield from" to have semantics close to a function call,
in some situations we simply can't. Because you can manually iterate a
generator and then 'yield from' it, you can have this weird
'partial-function-call' semantics.  For example:

     var = new_context_var()

     def gen():
         var.set(42)
         yield
         yield

Now, we can partially iterate the generator (1):

     def main():
         g = gen()
         next(g)

         # we don't want 'g' to leak its EC changes,
         # so var.get() is None here.
         assert var.get() is None

and then we can "yield from" it (2):

     def main():
         g = gen()
         next(g)

         # we don't want 'g' to leak its EC changes,
         # so var.get() is None here.
         assert var.get() is None

         yield from g
         # at this point it's too late for us to let var leak into
         # main().__logical_context__

For (1) we want the context change to be isolated.  For (2) you say
that the context change should propagate to the caller.  But it's
impossible: 'g' already has its own LC({var: 42}), and we can't merge
it with the LC of "main()".

"await" is fundamentally different, because it's not possible to
partially iterate the coroutine before awaiting it (asyncio will break
if you call "coro.send(None)" manually).

Yury