[Python-Dev] PEP 492: What is the real goal?

[Paul Moore]

On 29 April 2015 at 19:42, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> Here's how it might look like (this is the same pseudo-code
> but tailored for PEP 492, not a real something)
>
>   q = asyncio.Queue(maxsize=100)
>
>   async def produce():
>       # you might want to wrap it all in 'while True'

I think the "loop" in the Wikipedia pseudocode was intended to be the
"while True" here, not part of the "while" on the next line.

>
>       while not q.full():
>           item = create_item()
>           await q.put(item)
>
>   async def consume():
>       while not q.empty():
>           item = await q.get()
>           process_item(item)

Thanks for that. That does look pretty OK. One question, though - it
uses an asyncio Queue. The original code would work just as well with
a list, or more accurately, something that wasn't designed for async
use. So the translation isn't completely equivalent. Also, can I run
the produce/consume just by calling produce()? My impression is that
with asyncio I need an event loop - which "traditional" coroutines
don't need. Nevertheless, the details aren't so important, it was only
a toy example anyway.

However, just to make my point precise, here's a more or less direct
translation of the Wikipedia code into Python. It doesn't actually
work, because getting the right combinations of yield and send stuff
is confusing to me. Specifically, I suspect that "yield
produce.send(None)" isn't the right way to translate "yield to
produce". But it gives the idea.

data = [1,2,3,4,5,6,7,8,9,10]

q = []

def produce():
    while True:
        while len(q) < 10:
            if not data:
                return
            item = data.pop()
            print("In produce - got", item)
            q.append(item)
        yield consume.send(None)

total = 0
def consume():
    while True:
        while q:
            item = q.pop()
            print("In consume - handling", item)
            global total
            total += item
        yield produce.send(None)

# Prime the coroutines
produce = produce()
consume = consume()
next(produce)
print(total)

The *only* bits of this that are related to coroutines are:

1. yield consume.send(None) (and the same for produce)
2. produce = produce() (and the same for consume) priming the coroutines
3. next(produce) to start the coroutines

I don't think this is at all related to PEP 492 (which is about async)
but it's what is traditionally meant by coroutines. It would be nice
to have a simpler syntax for these "traditional" coroutines, but it's
a very niche requirement, and probably not worth it.

But the use of "coroutine" in PEP 492 for the functions introduced by
"async def" is confusing - at least to me - because I think of the
above, and not of async. Why not just call them "async functions" and
leave the term coroutine for the above flow control construct, which
is where it originated?

But maybe that ship has long sailed - the term "coroutine" is pretty
entrenched in the asyncio documentation. If so, then I guess we have
to live with the consequences.

Paul