[issue39085] Improve docs for await expression

[Kyle Stanley]

Kyle Stanley <aeros167 at gmail.com> added the comment:

> Sorry, my English is bad; I cannot help with docs too much.

No problem. Your feedback is still incredibly helpful and very much appreciated either way. (: 

> Anyway, technically an awaited coroutine *can* be suspended but the suspension is not always necessary. The most deep awaited function decides.

Ah, I see. It took a bit of experimentation for me to understand how this works, but I think that I get it now. Specifically, the suspension occurs when the deepest coroutine function awaits an awaitable object and a `yield` is reached (usually through a defined __await__ method that returns an iterator). When that awaitable object is completed and returns, the coroutine function with the `await` (and everything else that directly awaited it) is resumed. A good example of this is `asyncio.sleep(0)`, as it just awaits `__sleep0()`, which is just a generator-based coroutine with a bare `yield`.

```
import asyncio
import inspect

tracked_coro = None

async def main():
  # loop isn't entirely needed, just used to track event loop time
  loop = asyncio.get_running_loop()
  global tracked_coro
  tracked_coro = coro(loop)
  await asyncio.gather(tracked_coro, other_coro(loop))


# This is the coroutine being tracked
async def coro(loop):
  print(loop.time())
  print("Start of coro():",
           inspect.getcoroutinestate(tracked_coro))
  await nested_coro(loop)


async def nested_coro(loop):
  print(loop.time())
  # coro() is not suspended yet, because we did not reach a `yield`
  print("Start of nested_coro():",
           inspect.getcoroutinestate(tracked_coro))
  # This will call await `__sleep0()`, reaching a `yield` which suspends `coro()` and `nested_coro()`
  await asyncio.sleep(0)
  print(loop.time())
  print("After the await, coro() is resumed:",
           inspect.getcoroutinestate(tracked_coro))


async def other_coro(loop):
  print(loop.time())
  print("Start of other_coro():",
           inspect.getcoroutinestate(tracked_coro))

asyncio.run(main())
```

Output:

```
8687.907528533
Start of coro(): CORO_RUNNING
8687.907800424
Start of nested_coro(): CORO_RUNNING
8687.912218812
Start of other_coro(): CORO_SUSPENDED
8687.912291694
After the await, coro() is resumed: CORO_RUNNING
```

> For example, if you want to read 16 bytes from a stream and these bytes are already fetched there is no suspension at this point  (at least libraries are designed in this way)

After realizing that the suspend only occurs when `yield` is reached, I think I understand how this works for `StreamReader.read()`. 

In sum, a `yield` is reached when `read()` is called with an empty buffer, resulting in `await self._wait_for_data('read')`. Specifically within `_wait_for_data()`, the `yield` is reached within `await self._waiter` (because _waiter is a Future, which defines an __await__ method with a `yield`). However, if `read()` is called after the bytes were fetched and are contained in the buffer, the bytes are read from the buffer and returned directly without ever reaching a `yield`; thus there is no suspension that occurs.

Is my interpretation mostly correct? I want to make sure that I have a good understanding of how await really works, as that will both help with improving the documentation of the await expression and improve my understanding of asyncio.

> Also, technical speaking about awaits is hard without telling that a coroutine is a specialized version of generator object with (partially) overlapped methods and properties, e.g. send() and throw().

Good point. If I understand correctly, send() and throw() were specifically added to the generator API in PEP 342 for the purpose of implementing coroutines in the first place, so it makes sense to explain how they relate to await. 

> To run a coroutine you need a framework which calls these methods depending on the framework rules, the rules for asyncio are different from trio.

That's mainly why I added Nathaniel to the nosy list. I wanted to make sure that we describe the await expression in a way that's as accurate and informative as possible for both, as well as any other async library that uses await.

> Not sure how long should be the section but looking on `yield expressions` https://docs.python.org/3/reference/expressions.html#yield-expressions above I expect that awaits can take two-three times longer.

That would be a great goal to move towards, but I think that might have to be completed in multiple steps over a longer period of time rather than in a single change. Even if it ends up being not quite as long as 2-3 times the length of the reference for the yield expression, I think we can still make a substantial improvement to the existing version.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue39085>
_______________________________________