[Python-ideas] Yielding through context managers

[Guido van Rossum]

On Sun, Jan 6, 2013 at 9:47 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Mon, Jan 7, 2013 at 2:24 AM, Guido van Rossum <guido at python.org> wrote:
>> On Sunday, January 6, 2013, Nick Coghlan wrote:
>>>
>>> On Sun, Jan 6, 2013 at 8:20 PM, Laurens Van Houtven <_ at lvh.cc> wrote:
>>> > Hi Nick,
>>> >
>>> >
>>> > When you say "high latency" (in __exit__), what does "high" mean? Is
>>> > that
>>> > order of magnitude what __exit__ usually means now, or network IO
>>> > included?
>>> >
>>> > (Use case: distributed locking and remotely stored locks: it doesn't
>>> > take a
>>> > long time on network scales, but it can take a long time on CPU scales.)
>>>
>>> The status quo can only be made to work for in-memory locks. If the
>>> release step involves network access, then it's closer to the
>>> "database transaction" use case, because the __exit__ method may need
>>> to block.
>>
>> But you don't need to wait for the release. You can do that asynchronously.
>
> Ah, true, I hadn't thought of that. So yes, any case where the
> __exit__ method can be "fire-and-forget" is also straightforward to
> implement with just PEP 3156. That takes us back to things like
> database transactions being the only ones where

And 'yielding' wouldn't do anything about this, would it?

>> Also, have you given the implementation of your 'yielding' proposal any
>> thought yet?
>
> Not in depth. Off the top of my head, I'd suggest:
>   - make "yielding" a new kind of node in the grammar (so you can't
> write "yielding expr" in arbitrary locations, but only in those that
> are marked as allowing it)
>   - flag for loops and with statements as accepting these nodes as
> iterables and context managers respectively
>   - create a new Yielding AST node (with a single Expr node as the child)
>   - emit different bytecode in the affected compound statements based
> on whether the relevant subnode is an ordinary expression (thus
> invoking the special methods as "obj.__method__()") or a yielding one
> (thus invoking the special methods as "yield from obj.__method__()").
>
> I'm not seeing any obvious holes in that strategy, but I haven't
> looked closely at the compiler code in a while, so there may be
> limitations I haven't accounted for.

So would 'yielding' insert the equivalent of 'yield from' or the
equivalent of 'yield' in the code?

-- 
--Guido van Rossum (python.org/~guido)