[issue32591] Deprecate sys.set_coroutine_wrapper and replace it with more focused API(s)

[Nathaniel Smith]

Nathaniel Smith <njs at pobox.com> added the comment:

Right, I *knew* I should be nervous about calling into Python from a C-level destructor... what's happening is:

- Somehow, that coroutine object is managing to survive until the very, very end of the shutdown sequence (PyImport_Cleanup). I'm not sure how exactly; Yury suspects it somehow involves _asynciomodule.c holding a reference.

- At the very end of PyImport_Cleanup, there's a final call to the cycle collector, which destroys the coroutine object

- _PyGen_Finalize notices it's unawaited, and calls _PyErr_WarnUnawaitedCoroutine

- _PyErr_WarnUnawaitedCoroutine attempts to call warnings._warn_unawaited_coroutine, in a very careful and defensive manner

- But it doesn't matter how careful and defensive you are, because at this stage in the shutdown, we have sys.module = None, which freaks out the import system so badly that when we try to look up the warnings module, it doesn't even raise an error, it just abort()s the whole interpreter.

We can get a similar crash by doing:

import sys
async def f(): pass
sys.corocycle = [f]
sys.corocycle.append(sys.corocycle)

If you run the same code on 3.6, then it gets collected at the same time, and it issues a warning using the regular PyErr_WarnEx. It turns out that code is even *more* careful and defensive and notices that the interpreter is being shutdown, so it just skips printing the warning entirely.

I guess what we have to do is add a similar check to _PyErr_WarnUnawaitedCoroutine.

You can imagine how excited I am that I started working on this patch so I could make sure people get more reliable notice of unawaited coroutines (bpo-30491), and not only has that been rejected, but now I'm writing code to explicitly hide unawaited coroutine warnings. Just saying'...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32591>
_______________________________________