[New-bugs-announce] [issue33996] Crash in gen_send_ex(): _PyErr_GetTopmostException() returns freed memory

[STINNER Victor]

New submission from STINNER Victor <vstinner at redhat.com>:

Attached dagpool_test.py script triggers a crash on Python 3.7, but works well on Python 3.6. The script is a highly simplified code of tests/dagpool_tests.py of eventlet.

To reproduce the crash, you only need a single dependency: greenlet. Example using a venv:

    python3.7 -m venv ENV
    ENV/bin/python -m pip install greenlet

Then run the script:

    $ test_venv/bin/python dagpool_test.py 
    Segmentation fault (core dumped)

eventlet bug report:
https://github.com/eventlet/eventlet/issues/475


I suspected that the bug is caused by the new exc_info attribute of PyThreadState: commit ae3087c6382011c47db82fea4d05f8bbf514265d.

$ gdb -args test_venv/bin/python -X faulthandler dagpool_test.py
(gdb) run

Program received signal SIGSEGV, Segmentation fault.
0x000000000056c9d2 in PyErr_SetObject (exception=<type at remote 0x9709a0>, value=0x0) at Python/errors.c:101
101	        Py_INCREF(exc_value);
(gdb) where
#0  0x000000000056c9d2 in PyErr_SetObject (exception=<type at remote 0x9709a0>, value=0x0) at Python/errors.c:101
#1  0x000000000056cd4e in PyErr_SetNone (exception=<type at remote 0x9709a0>) at Python/errors.c:162
#2  0x000000000067cb0c in gen_send_ex (gen=0x7fffea651d78, arg=0x0, exc=0, closing=0) at Objects/genobject.c:241
#3  0x000000000067dd86 in gen_iternext (gen=0x7fffea651d78) at Objects/genobject.c:542
#4  0x00000000005461b1 in _PyEval_EvalFrameDefault (...)
...

(gdb) p tstate
$1 = (PyThreadState *) 0xa132a0

(gdb) p tstate->exc_info
$2 = (_PyErr_StackItem *) 0x7fffea651930

(gdb) p *tstate->exc_info
$3 = {
  exc_type = 0x0, 
  exc_value = 0x0, 
  exc_traceback = 0x0, 
  previous_item = 0x7fffea651d20
}

(gdb) p *tstate->exc_info->previous_item
$4 = {
  exc_type = <unknown at remote 0xdbdbdbdbdbdbdbdb>, 
  exc_value = <unknown at remote 0xdbdbdbdbdbdbdbdb>, 
  exc_traceback = <unknown at remote 0xdbdbdbdbdbdbdbdb>, 
  previous_item = 0xdbdbdbdbdbdbdbdb
}

----------
components: Interpreter Core
files: dagpool_test.py
keywords: 3.7regression
messages: 320712
nosy: Mark.Shannon, ned.deily, vstinner, yselivanov
priority: release blocker
severity: normal
status: open
title: Crash in gen_send_ex(): _PyErr_GetTopmostException() returns freed memory
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47658/dagpool_test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33996>
_______________________________________