[Python-Dev] Using async/await in place of yield expression

[Caleb Hattingh]

On 27 November 2017 at 14:53, Yury Selivanov <yselivanov.ml at gmail.com>
wrote:

> It is correct.  While 'yield from coro()', where 'coro()' is an 'async
> def' coroutine would make sense in some contexts, it would require
> coroutines to implement the iteration protocol.  That would mean that
> you could write 'for x in coro()', which is meaningless for coroutines
> in all contexts.  Therefore, coroutines do not implement the iterator
> protocol.


The two worlds (iterating vs awaiting) collide in an interesting way when
one plays with custom Awaitables.
>From your PEP, an awaitable is either a coroutine, or an object
implementing __await__,
*and* that __await__  returns an iterator.

The PEP only says that __await__ must return an iterator, but it turns out
that it's also required that that iterator
should not return any intermediate values.  This requirement is only
enforced in the event loop, not
in the `await` call itself.  I was surprised by that:

>>> class A:
...     def __await__(self):
...         for i in range(3):
...             yield i               # <--- breaking the rules, returning
a value
...         return 123

>>> async def cf():
...     x = await A()
...     return x

>>> c = cf()
>>> c.send(None)
0
>>> c.send(None)
1
>>> c.send(None)
2
>>> c.send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: 123
123


So we drive the coroutine manually using send(), and we see that
intermediate calls return the illegally-yielded values.  I broke the rules
because my __await__ iterator is returning values (via `yield i`) on each
iteration, and that isn't allowed because the event loop wouldn't know what
to do with these intermediate values; it only knows that "awaiting" is
finished when a value is returned via StopIteration.  However, you only
find out that it isn't allowed if you use the loop to run the coroutine
function:

>>> import asyncio
>>> loop = asyncio.get_event_loop()
>>> loop.run_until_complete(f())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.6/asyncio/base_events.py", line 467, in
run_until_complete
    return future.result()
  File "<stdin>", line 2, in f
  File "<stdin>", line 4, in __await__
RuntimeError: Task got bad yield: 0
Task got bad yield: 0


I found this quite confusing when I first came across it, before I
understood how asyncio/async/await was put together. The __await__ method
implementation must return an iterator that specifically doesn't return any
intermediate values.  This should probably be explained in the docs. I'm
happy to help with any documentation improvements if help is desired.

rgds
Caleb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171127/69b80a13/attachment.html>