[Python-ideas] PEP 525: Asynchronous Generators

[Andrew Svetlov]

As maintainer of aiohttp (asyncio-compatible HTTP client/server) and aiopg
(PostgreSQL asyncio driver) I totally support the pep.

The problem is: I'm able to write async iterator class which utilizes
__aiter__/__anext__.
But users of my libraries are not.

Writing synchronous generator functions for data processing is very common
pattern.

But class based approach requires saving all context not in local variables
but in instance vars, you know.

It's a nightmare in practice, as consequence people writes more complex and
error prone code than they could write with this PEP.

https://www.python.org/dev/peps/pep-0255/ for simple generators is 15 years
old, I believe everybody understand why we need it.

Async generators extend the same concept to asyncio world by solving the
same problems but with async code.

Honestly I personally don't feel a need for two-way generators with
`asend()`/`athrow()` but as Yury explained he need them internally for
`anext`/`aclose` anyway.

On Thu, Aug 4, 2016 at 6:18 PM Yury Selivanov <yselivanov.ml at gmail.com>
wrote:

> Hi Sven,
>
> On 2016-08-04 9:20 AM, Sven R. Kunze wrote:
> > Hey Yury,
> >
> > that's a great proposal!
> >
> >
> > On 03.08.2016 00:31, Yury Selivanov wrote:
> >>
> >>     class Ticker:
> >>         """Yield numbers from 0 to `to` every `delay` seconds."""
> >>
> >>         def __init__(self, delay, to):
> >>             self.delay = delay
> >>             self.i = 0
> >>             self.to = to
> >>
> >>         def __aiter__(self):
> >>             return self
> >>
> >>         async def __anext__(self):
> >>             i = self.i
> >>             if i >= self.to:
> >>                 raise StopAsyncIteration
> >>             self.i += 1
> >>             if i:
> >>                 await asyncio.sleep(self.delay)
> >>             return i
> >>
> >>
> >> The same can be implemented as a much simpler asynchronous generator::
> >>
> >>     async def ticker(delay, to):
> >>         """Yield numbers from 0 to `to` every `delay` seconds."""
> >>         for i in range(to):
> >>             yield i
> >>             await asyncio.sleep(delay)
> >>
> >
> > That's a great motivational example. +1
> >
> > Especially when reading the venerable PEP 255 (Q/A part), this also
> > gets rid of the "low-level detail": "raise StopAsyncIteration". This
> > could also be worth mentioning in the motivation part for PEP 525.
>
> Thanks!
>
> [..]
> > From your answer to Stefan, I get the impression that the reason why
> > we actual need all those a* methods (basically a duplication of the
> > existing gen protocol), is the fact that normal generators can be
> > converted to coroutines. That means, 'yield' still can be used in both
> > ways.
> >
> > So, it's a technical symptom of the backwards-compatibility rather
> > than something that cannot be avoided by design. Is this correct?
> >
>
> async/await in Python is implemented on top of the generator protocol.
> Any 'await' is either awaiting on a coroutine or on a Future-like
> object.  Future-like objects are defined by implementing the __await__
> method, which should return a generator.
>
> So coroutines and generators are very intimately tied to each other, and
> that's *by design*.
>
> Any coroutine that iterates over an asynchronous generator uses the
> generator protocol behind the scenes.  So we have to multiplex the async
> generaotor's "yields" into the generator protocol in such a way, that it
> stays isolated, and does not interfere with the "yields" that drive
> async/await.
>
> >
> >
> > If it's correct, would you think it would make sense to get rid of the
> > a* in a later iteration of the async capabilities of Python? So, just
> > using the normal generator protocol again?
>
> Because async generators will contain 'await' expressions, we have to
> have a* methods (although we can name them without the "a" prefix, but I
> believe that would be confusing for many users).
>
> >
> >
> > One note on all examples but the last. Reading those examples, it
> > creates the illusion of actual working code which is not the case,
> > right? One would always need to 1) wrap module-level statements into
> > its own coroutine, 2) create an event-loop and 3) run it. Do you think
> > clarifying this in the PEP makes sense?
>
> I'll think about this, thanks!  Maybe I can add a line in the beginning
> of the "Specification" section.
>
> >
> >
> > Thanks again for your hard work here. Async generators definitely
> > completes the picture.
>
> Thank you, Sven
>
> Yury
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
Thanks,
Andrew Svetlov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160805/ca960f14/attachment.html>