Asyncio tasks getting cancelled

[philip.m at tura-home.de]

On Mon, Nov 05, 2018 at 01:57:56PM -0700, Ian Kelly wrote:
> > Which is what I want in this case. Scheduling a new (long-running) task
> > as a side effect, but returning early oneself. The new task can't be
> > awaited right there, because the creating one should return already.
> 
> If you want to do this in the asyncio.run main coroutine, then that
> seems like a problematic design. Once the main coroutine returns, the
> event loop should be considered no longer running, and any still
> pending callbacks or futures won't resolve.

This is only true for the small example I provided. In the actual code
this is somewhere deep in the hirarchy.

> > > If the goal here is for the task created by main() to complete before
> > > the loop exits, then main() should await it, and not just create it
> > > without awaiting it.
> >
> > So if this happens somewhere deep in the hirarchy of your application
> > you would need some mechanism to pass the created tasks back up the
> > chain to the main function?
> 
> I haven't used asyncio.run yet myself, so take all this with a grain
> of salt, but it seems to me that anything that you want to resolve
> before the event loop terminates should be awaited either directly or
> indirectly by the main coroutine. From the documentation:
> 
> """
> This function always creates a new event loop and closes it at the
> end. It should be used as a main entry point for asyncio programs, and
> should ideally only be called once.
> """
> 
> So I think part of the idea with this is that the asyncio.run main
> coroutine is considered the main function of your async app. Once it
> returns, the program should be effectively done. For example, maybe
> the main coroutine spins up a web server and returns when the web
> server shuts down.

Again sorry for the confusion, but I don't think this is an issue with
restarting loops, as this isn't happening in my application.
For context:
https://github.com/ldo/dbussy/issues/13
https://gist.github.com/tu500/3232fe03bd1d85b1529c558f920b8e43

It really feels like asyncio is loosing strong references to scheduled
tasks, as excplicitly keeping them around helps. Also, the error
messages I'm getting are the ones from here:
https://github.com/python/cpython/blob/16c8a53490a22bd4fcde2efaf4694dd06ded882b/Lib/asyncio/tasks.py#L145
Which indicates that the tasks actually weren't even started at all?

> If that doesn't suit your program, for instance there's no core task
> to await, but you want to schedule a lot of things that need to
> resolve and that the main coroutine has no way to know about, then it
> may be the case that asyncio.run is not right for your use case and
> you should use loop.run_forever() instead. You'll still need some
> criteria for figuring out when to exit though, and it seems to me that
> whatever that is you could just bundle it up in a coroutine and await
> it from main.

Though not really related with my actual problem, so getting off topic,
but I can imagine an architecture where that would be "There aren't any
running tasks any more." or even "Never."
Also, I may be overlooking things, but I haven't found a way to add a
task before calling run_forever(), as asyncio will then say the loop
isn't running yet. So I'm not sure how you would jumpstart in that case.