[New-bugs-announce] [issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

[Nathaniel Smith]

New submission from Nathaniel Smith:

A common problem when working with async functions is to attempt to call them but forget the 'await', which eventually leads to a 'Warning: coroutine ... was never awaited' (possibly buried in the middle of a bunch of traceback shrapnel caused by follow-on errors). This can be confusing, lead to spuriously passing tests, and generally isn't a great experience for the dev who makes the mistake. To improve this, I'd like to do things like have a test harness reliably detect when this has happened inside a test case, or have trio's main loop check occasionally to see if this has happened and immediately raise a useful error. (Unfortunately, it *doesn't* work to promote the warning to an error using the usual warnings filter machinery, because the warning is raised inside a __del__ method, which suppresses exception propagation.)

In principle this is possible with sys.setcoroutinewrapper, but that adds non-trivial runtime overhead to every async function invocation, which is something we'd like to avoid. (It's OK for a "debug mode", but "modes" are also a poor dev experience: they certainly have their place, but whenever possible it's much nicer to give a proper error in the first place instead of relying on the dev to realize they need to enable debug mode and then remember how to do it.)

Therefore I propose that CPython keep a thread-local counter of how many coroutine objects currently exist in a created-but-not-yet-started state -- so corofn.__call__ would increment this counter, and the first call to coroobj.__next__/send/throw would decrement it. And there's some way to access it e.g. via a magic function in the sys module. Then test harnesses can assert that this is zero, trio could occasionally poll this from its run loop and assert that it's zero, etc., with very low overhead.

(This is a slight modification of the version I discussed with Yury and others at PyCon last week; the previous request was to keep a count of how many times the "coroutine '...' was never awaited" warning had been emitted. The problem with the original idea is that the warning message doesn't fire until the garbage collector has collected the coroutine object, and that might not happen at a convenient time if we're using PyPy, or if there are cycles, or in a test where the missing 'await' eventually leads to an exception whose traceback pins the coroutine object in memory just long enough for the warning to be detected on the *next* test and confuse everyone. Thanks to Matthias Bussonnier for spurring the new idea, and see discussion here: https://github.com/python-trio/trio/issues/79#issuecomment-304364010)

----------
components: asyncio
messages: 294584
nosy: giampaolo.rodola, haypo, ncoghlan, njs, yselivanov
priority: normal
severity: normal
status: open
title: Add a lightweight mechanism for detecting un-awaited coroutine objects
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue30491>
_______________________________________