[New-bugs-announce] [issue39958] Deadlock in _PyInterpreterState_DeleteExceptMain with HEAD_LOCK(runtime)

[Atle Solbakken]

New submission from Atle Solbakken <atle.solbakken at gmail.com>:

In _PyInterpreterState_DeleteExceptMain() aqcuires lock with HEAD_LOCK(runtime). With the lock still held and if a interpreter state is to be cleared, _PyInterpreterState_Clear() is called. HEAD_LOCK(runtime) is then called again (from within Clear()), causing deadlock.

Backtrace is from 3.8.2-1. Looks like this is also present in Python-3.8.2rc2 and Python-3.9.0a4.

#0  futex_abstimed_wait_cancelable (private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x555555619cc0) at ../sysdeps/unix/sysv/linux/futex-internal.h:208
#1  do_futex_wait (sem=sem at entry=0x555555619cc0, abstime=0x0, clockid=0) at sem_waitcommon.c:112
#2  0x00007ffff7f08488 in __new_sem_wait_slow (sem=sem at entry=0x555555619cc0, abstime=0x0, clockid=0) at sem_waitcommon.c:184
#3  0x00007ffff7f08501 in __new_sem_wait (sem=sem at entry=0x555555619cc0) at sem_wait.c:42
#4  0x00007ffff6ee6cf3 in PyThread_acquire_lock_timed (lock=0x555555619cc0, microseconds=<optimized out>, intr_flag=0) at ../Python/thread_pthread.h:471
#5  0x00007ffff6ef6789 in _PyInterpreterState_Clear (interp=0x555555630310, runtime=<optimized out>, runtime=<optimized out>) at ../Python/pystate.c:261
#6  0x00007ffff6ef8018 in _PyInterpreterState_DeleteExceptMain (runtime=0x7ffff7297a80 <_PyRuntime>) at ../Python/pystate.c:380
#7  0x00007ffff6e49912 in PyOS_AfterFork_Child () at ../Modules/posixmodule.c:472
#8  0x00007ffff7f9681c in __fork_main_tstate_callback (arg=<optimized out>) at python3.c:626
#9  0x00007ffff7f989d1 in rrr_py_with_global_tstate_do (callback=0x7ffff7f96800 <__fork_main_tstate_callback>, arg=0x0) at python3.c:1222
#10 0x00007ffff7f8b563 in rrr_socket_with_lock_do (callback=callback at entry=0x7ffff7f98a00 <__fork_callback>, arg=arg at entry=0x0) at rrr_socket.c:169
#11 0x00007ffff7f96d07 in __rrr_py_fork_intermediate (function=function at entry=0x7ffff632a700, fork_data=fork_data at entry=0x555555639e40, child_method=child_method at entry=0x7ffff7f965f0 <__rrr_py_start_persistent_thread_rw_child>) at python3.c:653
#12 0x00007ffff7f97075 in __rrr_py_start_persistent_rw_thread_intermediate (function=function at entry=0x7ffff632a700, fork=fork at entry=0x555555639e40) at python3.c:749
#13 0x00007ffff7f972bd in __rrr_py_start_thread (result_fork=0x5555555d2dd8, rrr_objects=0x5555555d2de8, module_name=0x5555556a70e0 "testing", function_name=0x5555555fb6d0 "process", start_method=0x7ffff7f97040 <__rrr_py_start_persistent_rw_thread_intermediate>) at python3.c:855
#14 0x00007ffff65dc3a0 in ?? ()
#15 0x0000000000000000 in ?? ()

----------
components: C API
messages: 364120
nosy: Atle Solbakken
priority: normal
severity: normal
status: open
title: Deadlock in _PyInterpreterState_DeleteExceptMain with HEAD_LOCK(runtime)
type: crash
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue39958>
_______________________________________