How the heck does async/await work in Python 3.5

[Ian Kelly]

On Fri, Feb 19, 2016 at 10:24 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> Less snarkily looks like a series of bolt-ons after bolt-ons
>
> IMHO Guido's (otherwise) uncannily sound intuitions have been wrong right from
> 2001 when he overloaded def for generators.
> And after that its been slippery-slope down: reusing generator-yield (statement)
> for coroutine-yield (expression)
> Most recently choosing these async-await keywords instead of the more symmetric
> suggestions of Greg Ewing

Two out of three of those are fine. IMO generators should have
required a separate keyword to distinguish them from ordinary
functions, but it's a minor complaint and they're otherwise a great
addition to the language.

I do think it's pretty clear at this point though that PEP 342 was a
mistake. We should never have introduced generator-based coroutines;
it's an abstraction with too much leakage. It doesn't make sense that
generator-based coroutines implement the iterator protocol, because
there's no reason to ever try to iterate over them. It also doesn't
make sense to ever iterate over a Future, and yet asyncio Futures,
which aren't even coroutines, are nevertheless *forced* to be iterable
just because that's how coroutines work. Otherwise you couldn't "yield
from" a Future.

As another point that happens to be fresh in my mind, awaiting a
Future on which an exception gets set is supposed to propagate the
exception. I recently found that this breaks if the exception in
question happens to be StopIteration (granted not one that should
generally be allowed to propagate anyway, but that's a separate
discussion) for the simple reason that raising StopIteration in a
generator is equivalent to returning None. The awaiting coroutine thus
gets a non-exceptional result of None rather than the expected
exception. The irritating thing to me is that this even plagues PEP
492 coroutines using "await" rather than "yield from", because the two
are basically the same under the covers. The former gets an iterator
by calling __await__ instead of __iter__, but it's still implemented
using an iterator. If you look at the asyncio.Future implementation,
__await__ is actually just a synonym of the __iter__ method.

My hope is that someday we can get rid of PEP 342 coroutines entirely.
Then maybe we can get a coroutine implementation that's actually sane.
PEP 492 with async/await and non-iterable semantics is a step in the
right direction, but ultimately I think we also need an underlying
implementation that *isn't* fundamentally based on iteration.