[Async-sig] Asynchronous cleanup is a problem

[Yury Selivanov]

> On Jul 6, 2016, at 3:54 PM, David Beazley <dave at dabeaz.com> wrote:
> 
> 
>> However, as far as I know curio doesn’t have the ability to schedule an operation in a synchronous manner by means of something like a Future. Is that correct? If there is no way in curio to spawn a task to occur later without having to await on it, then clearly there is no option but to allow coroutines in __aexit__ and finally: how else could curio operate?
>> 
> 
> Yes, Curio does not utilize Futures or callback functions for that matter.   However, I think the real issue at hand might be much more subtle and weird.
> 
> If I understand things correctly, the goal is to implement an asynchronous generator for feeding asynchronous iteration.  This is a new kind of generator that runs inside of a coroutine (so, imagine a coroutine inside of another coroutine).   Without seeing much more, I'd guess it would look something roughly akin to this:
> 
> async def agen():
>       ... some sort of async generator ...
>       async yield value     # ???? Syntax ????

I my current WIP branch I just use ‘yield’ inside ‘async def’.  That’s what I was going to propose to do in the PEP.

> 
> async def main():
>       async for x in agen():
>              ...
> 
> There's some kind of underlying protocol driving the async iteration, but it's *different* than what is being used by the enclosing coroutines.   Yes, there is a scheduler kernel (or event loop) that makes the outer coroutines run, but that scheduler is not driving the underlying iteration protocol of the async generator part.   So, things get weird when stuff like this happens:
> 
> async def main():
>       async for x in agen():
>               if x == STOP:
>                     break

Good catch.

[..]

> Since forgetting that last close() step would be easy, naturally an async generator should support the asynchronous context-management protocol.
> 
> async def main():
>       async with agen() as items:
>              async for x in items:
>                      if x == STOP:
>                            break

This is an interesting idea, but I wonder if instead of using ‘async with’ we can actually augment ‘async for’ to do the async finalization.

We can add an __aiter_close__ special method which will return an awaitable.  In this case, ‘async for’ can always look for that method and call it at the end of the iteration.  Async generators will implement the method to make sure that ‘finally’ is always executed (any number of awaits in ‘finally’ is OK; ‘yield’ expressions cannot be used).

Yury