How the heck does async/await work in Python 3.5

[Chris Angelico]

On Sat, Feb 20, 2016 at 7:14 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Sat, Feb 20, 2016 at 12:57 AM, Chris Angelico <rosuav at gmail.com> wrote:
>> On Sat, Feb 20, 2016 at 6:48 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>>> As another point that happens to be fresh in my mind, awaiting a
>>> Future on which an exception gets set is supposed to propagate the
>>> exception. I recently found that this breaks if the exception in
>>> question happens to be StopIteration (granted not one that should
>>> generally be allowed to propagate anyway, but that's a separate
>>> discussion) for the simple reason that raising StopIteration in a
>>> generator is equivalent to returning None.
>>
>> Solved by PEP 479. Use "from __future__ import generator_stop" to save
>> yourself the pain.
>
> Nope.
>
> py> from __future__ import generator_stop
> py> import asyncio
> py> async def test_coro():
> ...     fut = asyncio.Future()
> ...     fut.set_exception(StopIteration())
> ...     print('received %r' % await fut)
> ...
> py> list(test_coro().__await__())
> received None
> []
>
> I think because __future__ imports are per-file, and
> asyncio.Future.__iter__ is defined in a file outside my control that
> doesn't have the __future__ import.

You need the future directive in the file that defines the function
that raises, so I guess you'd need to apply that to an asyncio call.
The tricky bit here is that it's a backward compatibility change, but
since asyncio is flagged provisional, I suspect the future directive
could be added (anyone who's depending on
set_exception(StopIteration()) to terminate without an exception will
have to change code).

Actually, that mightn't be a bad thing. Maybe raise that as a tracker
issue? I just tested, and slapping "from __future__ import
generator_stop" at the top of Lib/asyncio/futures.py causes your
example to raise an exception instead of returning None, and doesn't
seem to break the test suite.

> I suppose that when the generator_stop behavior becomes standard then
> it will work, but still that will just cause a RuntimeError to
> propagate instead of the desired StopIteration.

That then becomes a pretty minor wart, on par with hash() never
returning -1 (which I came across recently, but only by snooping the
source) - it'll hack it to -2 instead, because -1 is used as an error
signal. In the same way, StopIteration is special-cased as a return
signal (because there needs to be _some_ mechanism for distinguishing
between yield and raise and return; normally, the difference between
"has a value to return" and "has no value to return" is indicated by
raising in the latter case, but now we need even more
distinguishments), it can't actually be raised per se. Since the
exception chains, you can't get confused.

> It's not really that big a deal since there is a code smell to it, but
> it's surprising since intuitively StopIteration should have no special
> meaning to a PEP 492 coroutine (it's not an iterator, wink wink, nudge
> nudge), and the thing being awaited is a Future, which also doesn't
> intuitively look like an iterator. Note that if you just call
> Future.result(), then the exception propagates as expected; it's just
> awaiting it that doesn't work.

Definitely seems like it should be fixed, then; the current behaviour
is that Future.result() raises RuntimeError if you raise
StopIteration, so having await do the same would make sense.

ChrisA