[Async-sig] Asynchronous cleanup is a problem

[Glyph Lefkowitz]

> On Jul 5, 2016, at 5:56 PM, Nathaniel Smith <njs at pobox.com> wrote:
> 
> Starting axiom: async functions / async generators need to be prepared
> for the case where they get garbage collected before being run to
> completion, because, well... this is a thing that can happen.

This thread is really confusing, so I'm going to try to just attack this axiom (which isn't really axiomatic, it's a conclusion drawn from a few other properties of the interpreter and the event loop) and see if it holds :-).

Yury already pointed out that coroutines run under a scheduler that keeps strong references to them.  Backing that up a little bit - by definition, if your coroutine is not strongly referenced by a scheduler, it can't be doing anything interesting; nobody's going to call .send on it.  This is why 'foo()' is a no-op whereas 'await foo()' actually causes something to happen.  Similarly, this is why you need Task() to do something in parallel, and you can't just toss a coroutine out into the void.

Furthermore, Task.cancel()'s documentation holds another clue about asynchronous cleanup:

"Unlike Future.cancel(), this does not guarantee that the task will be cancelled: the exception might be caught and acted upon, delaying cancellation of the task or preventing cancellation completely".

Deferred.cancel() behaves in much the same way, for the same reason.  It is explicitly allowed that an asynchronous task do asynchronous clean-up.

Now, if you have a task that was scheduled but has come forcibly detached from its scheduler, you are in a bit of a mess.  But this is an inherently unresolvable mess, for the same reason that Thread.kill() is an inherently unresolvable mess: you cannot forcibly terminate a thread of execution and end up with your program in a known state.  It's OK to kill a generator (as distinct from a 'coroutine' in the sense that it does not expect .send to be called on it) from the GC because a generator is just yielding values and since it doesn't expect .send it can't expect to keep running, and it can do purely synchronous try/finally.  But as soon as you are allowed to invoke asynchronous work, you have to be allowed to let that asynchronous work complete.

So: async functions should not and cannot be prepared for the case where they get garbage collected; this is not a thing that can happen unless the coroutine scheduler is damaged beyond repair and it's time to crash your process.

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20160706/3d49b430/attachment.html>