[Async-sig] PEP: asynchronous generators

[Brett Cannon]

On Fri, 29 Jul 2016 at 10:46 Yury Selivanov <yselivanov at gmail.com> wrote:

> Thanks a lot for the feedback, Brett!  Comments inlined below:
>
> > On Jul 29, 2016, at 1:25 PM, Brett Cannon <brett at python.org> wrote:
> >
> [..]
> >
> > Performance is an additional point for this proposal: in our testing of
> > the reference implementation, asynchronous generators are *2x* faster
> > than an equivalent implemented as an asynchronous iterator.
> >
> > Another motivation is that types.coroutine becomes purely a
> backwards-compatibility/low-level thing that is no longer required for
> event loop frameworks to use on their generators which provide their async
> API. So now an async framework can be written entirely in terms of def and
> async def and not def and @types.coroutine.
>
> A slight misunderstanding here: @types.coroutine turns a normal old-style
> generator into an awaitable object.  That isn’t directly related to
> asynchronous iteration.  Frameworks like curio will continue using
> @types.coroutine to implement future-like objects in event loops.
>

Ah, OK. So that would be yet another PEP to make that kind of change (and
another keyword).


>
> [..]
> >
> > Support for Asynchronous Iteration Protocol
> > -------------------------------------------
> >
> > The protocol requires two special methods to be implemented:
> >
> > 1. An ``__aiter__`` method returning an *asynchronous iterator*.
> > 2. An ``__anext__`` method returning an *awaitable* object, which uses
> >    ``StopIteration`` exception to "yield" values, and
> >    ``StopAsyncIteration`` exception to signal the end of the iteration.
> >
> > I assume this is in place of __iter__() and __next__(), i.e. an async
> generator won't have both defined?
>
> Correct, async generators don’t have __iter__ and __next__. They can only
> be iterated with an `async for`.
>
> [..]
> >
> > To solve this problem we propose to do the following:
> >
> > 1. Implement an ``aclose`` method on asynchronous generators
> >    returning a special *awaitable*.  When awaited it
> >    throws a ``GeneratorExit`` into the suspended generator and
> >    iterates over it until either a ``GeneratorExit`` or
> >    a ``StopAsyncIteration`` occur.
> >
> >    This is very similar to what the ``close()`` method does to regular
> >    Python generators, except that an event loop is required to execute
> >    ``aclose()``.
> >
> > I'm going to ask this now instead of later when there's more motivation
> behind this question: do we need to append "a" to every async method we
> have? If asynchronous generators won't have a normal close() then why can't
> it just be close(), especially if people are not going to be calling it
> directly and instead it will be  event loops? I'm just leery of codifying
> this practice of prepending "a" to every async method or function and
> ending up in a future where I get tired of a specific letter of the
> alphabet.
>
> I decided to use the prefix because we already use it in magic method
> names: __anext__ and __aiter__.  I think it also makes it easier to
> understand the API of async generators (and understand how it’s different
> from sync generators API).
>
> And while it’s entirely possible to drop the ‘a’ for async generator API,
> it’s not so simple for other cases.  Later, for Python 3.7, we might
> consider adding ‘aiter()’ and ‘anext()’ builtins, for which we’d have to
> use ‘a’ or ‘async’ prefix (we can’t reuse 'iter()' and 'next()’ for async
> generators).
>

I guess we just need to decide as a group that an 'a' prefix is what we
want to signify something is asynchronous vs some other prefix like 'a_' or
'async', or 'async_' as people will follow this style choice in their own
code going forward.


>
> [..]
>
> >
> > 3. ``agen.anext(val)``: Returns an *awaitable*, that pushes the
> >    ``val`` object in the ``agen`` generator.  When the ``agen`` has
> >    not yet been iterated, ``val`` must be ``None``.
> >
> > How is this different from an async send()? I see an asend() used in the
> example below but not anext().
>
> Good catch!  This is a typo, it should read ``agen.asend(val)``.
>
> Similarly to sync generators, where ‘__next__()’ call is equivalent of
> ‘.send(None)’, ‘__anext__()’ awaitable is equivalent to ‘.asend(None)'
>
> [..]
> >
> > 7. ``agen.ag_await``: The object that ``agen`` is currently awaiting on,
> >    or ``None``.
> >
> > That's an interesting addition. I like it! There's no equivalent on
> normal generators, correct?
>
> We actually added that in 3.5 (last minute!).
>
> For sync generators the field is called ‘.gi_yieldfrom’, for coroutines
> it’s ‘.cr_await’, and for proposed async generators it will be ‘.ag_await’.
>

I would also clarify that "waiting on" means "what `await` has been called
on (if anything as an `await` call might not have been used)" and not what
the last yielded object happened to be (which is what my brain initially
thought it was simply because the async generator is paused on the event
loop returning based on what was yielded).


>
> Thanks!
> Yury
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20160729/d7aa630c/attachment.html>