[New-bugs-announce] [issue31061] asyncio segfault when using threadpool and "_asyncio" native module

[Alexander Mohr]

New submission from Alexander Mohr:

I have a project in a prod environment which heavily uses asyncio and a threadpool.  It uses the threadpool to run CPU heavy tasks (in this case populating a defaultdict) to avoid blocking the main thread (no async code in thread).  For some time now my service has been randomly crashing at the same place in the thread which does the dict updating.  I've finally got both the python and native stack traces, and based on the information presented it looked very similar to the issue found by the devs at home-assistant (https://github.com/home-assistant/home-assistant/issues/7752#issuecomment-305100009, which points to https://github.com/home-assistant/home-assistant/pull/7848).  So I tried their fix of disabling the "_asyncio" module, and lo and behold python no longer segfaults.

Per the stacktrace it's crashing in PyObject_GC_Del, and the only place this is used in the asyncio module seems to be here: https://github.com/python/cpython/blob/master/Modules/_asynciomodule.c#L996

does anyone have any idea why it's crashing on this line?  Are there thread protections missing in this file?

I'm trying to reproduce this in a testcase but it's proving very difficult as I'm guessing it's timing related.

----------
components: asyncio
files: native___python_crash_stacks.txt
messages: 299346
nosy: thehesiod, yselivanov
priority: normal
severity: normal
status: open
title: asyncio segfault when using threadpool and "_asyncio" native module
type: crash
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file47043/native___python_crash_stacks.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31061>
_______________________________________