asyncio awaitable object

[Ian Kelly]

On Fri, Dec 8, 2017 at 2:08 AM, ast <nomail at com.invalid> wrote:
> Hello,
>
> According to: https://www.python.org/dev/peps/pep-0492/#await-expression
> an awaitable object is:
>
> - A native coroutine object returned from a native coroutine function
> - A generator-based coroutine object returned from a function decorated with
> types.coroutine()
> - An object with an __await__ method returning an iterator
>
> I dont understand the last one.
>
> For example in instruction "res = await obj"
>
> where obj has a __await__ method returning an iterator
>
> What kind of data this generator is supposed to provide when next() is
> applied to it and what are these data becoming ?
>
> what res contains when the iterator has finished to iterate ?
>
> It seems that PEP492 documentation says nothing about it (or I dont
> understand, english is not my native language)

I believe the reason that PEP492 doesn't specify that is because it
only creates the keywords and specifies where they can be used. What
values are passed by the iterator is an implementation detail of the
asyncio framework, and the intention is that async/await should be
usable by other async frameworks, not just asyncio. So the proper
answer to the question "what kind of data should the iterator provide"
is "it depends on framework".

An example implementation of an object with an __await__ method is an
asyncio.Future object. If you look at the definition of
asyncio.Future, you can find its __await__ method. Here it is:

    __await__ = __iter__

Huh? To understand this, bear in mind that anywhere you can use
"await", you would previously have used "yield from". To "yield from"
something, that something must be iterable. So to "yield from" asyncio
Futures, they must be iterable, which means they must have an __iter__
method. The protocol that asyncio uses for __await__ is the same that
it uses __iter__, so when __await__ was added it returns an iterator
which made __await__ literally a drop-in replacement of __iter__. So
what does Future.__iter__ do?

    def __iter__(self):
        if not self.done():
            self._asyncio_future_blocking = True
            yield self  # This tells Task to wait for completion.
        assert self.done(), "yield from wasn't used with future"
        return self.result()  # May raise too.

It returns a generator that first checks if the future is already
done. If it is, it just returns its result (which raises a
StopIteration) without ever sleeping. Otherwise, it yields itself,
exactly once. The chain of "yield from" / "await"s ultimately pass
this future all the way down to the asyncio event loop, which adds it
to the scheduler. When the future is done, the task is resumed and the
flow of control goes back up the chain all the way to the futures,
which asserts for sanity that it is now done and returns its result.

So the answer to "what should the iterator yield for asyncio" is that
it should yield unfinished asyncio Futures one at a time, with the
expectation that they will be done when the iterator resumes.