[Python-ideas] Are there asynchronous generators?

[Stefan Behnel]

[Fixing the messed-up reply quoting order]

Adam Bartoš schrieb am 28.06.2015 um 12:30:
> On Sun, Jun 28, 2015 at 12:07 PM, Andrew Svetlov wrote:
>> On Sun, Jun 28, 2015 at 1:02 PM, Adam Bartoš wrote:
>>> There is also the problem that one cannot easily feed a queue,
>>> asynchronous
>>> generator, or any asynchronous iterator to a simple synchronous consumer
>>> like sum() or list() or "".join(). It would be nice if there was a way to
>>> wrap them to asynchronous ones when needed – something like (async
>>> sum)(asynchronously_produced_numbers()).
>> 
>> I afraid the last will never possible -- you cannot push async
>> coroutines into synchronous convention call.
>> Your example should be converted into `await
>> async_sum(asynchronously_produced_numbers())` which is possible right
>> now. (asynchronously_produced_numbers should be *iterator* with
>> __aiter__/__anext__ methods, not generator with yield expressions
>> inside.
> 
> I understand that it's impossible today, but I thought that if asynchronous
> generators were going to be added, some kind of generalized generator
> mechanism allowing yielding to multiple different places would be needed
> anyway. So in theory no special change to synchronous consumers would be
> needed – when the asynschronous generator object is created, it gets a link
> to the scheduler from the caller, then it's given as an argument to sum();
> when sum wants next item it calls next() and the asynchronous generator can
> either yield the next value to sum or it can yield a future to the
> scheduler and suspend execution of whole task. But since it's a good idea
> to be explicit and mark each asyncronous call, some wrapper like (async
> sum) would be used.

Stackless might eventually support something like that.

That being said, note that by design, the scheduler (or I/O loop, if that's
what you're using) always lives *outside* of the whole asynchronous call
chain, at its very end, but can otherwise be controlled by arbitrary code
itself, and that is usually synchronous code. In your example, it could
simply be moved between the first async function and its synchronous
consumer ("sum" in your example). Doing that is entirely possible. What is
not possible (unless you're using a design like Stackless) is that this
scheduler controls its own controller, e.g. that it starts interrupting the
execution of the synchronous code that called it.

Stefan