[New-bugs-announce] [issue30395] deadlocked child process after forking on pystate.c's head_mutex

[Louis Brandy]

New submission from Louis Brandy:

A forked process (via os.fork) can inherit a locked `head_mutex` from its parent process and will promptly deadlock in this stack (on my linux box):

Child Process (deadlocked):

#0  0x00007f1a4da82e3c in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x7f1a4c2964e0) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  0x00007f1a4da82e3c in do_futex_wait (sem=sem at entry=0x7f1a4c2964e0, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f1a4da82efc in __new_sem_wait_slow (sem=0x7f1a4c2964e0, abstime=0x0) at sem_waitcommon.c:181
#3  0x00007f1a4da82fb9 in __new_sem_wait (sem=<optimized out>) at sem_wait.c:29
#4  0x00007f1a4de4c605 in PyThread_acquire_lock_timed (lock=0x7f1a4c2964e0, microseconds=-1, intr_flag=0) at Python/thread_pthread.h:352
#5  0x00007f1a4de4c4b4 in PyThread_acquire_lock (lock=<optimized out>, waitflag=waitflag at entry=1) at Python/thread_pthread.h:556
#6  0x00007f1a4dd59e08 in _PyThreadState_DeleteExcept (tstate=0x7f19f2301800) at Python/pystate.c:483
#7  0x00007f1a4dd46af4 in PyEval_ReInitThreads () at Python/ceval.c:326
#8  0x00007f1a4dd78b0b in PyOS_AfterFork () at ./Modules/signalmodule.c:1608


The parent process has a race between one thread calling `os.fork` (and holding the GIL) and another (in our case C++) thread trying to use PyGILState_Ensure. PyGILState_Ensure will grab the head_mutex before it tries to get the GIL. So if a fork happens in this critical section, the child process will get the locked head_mutex. 

The path from PyGILState_Ensure -> head_mutex looks like this:

#0  new_threadstate (interp=0x7fb5fd483d80, init=init at entry=1) at Python/pystate.c:183
#1  0x00007fb5ff149027 in PyThreadState_New (interp=<optimized out>) at Python/pystate.c:250
#2  0x00007fb5ff006ac7 in PyGILState_Ensure () at Python/pystate.c:838


----

Possible fix?

A simple fix would be to, inside PyOS_AfterFork, reset/unlock pystate.c's head_mutex if it's already locked.

Unclear if this is related to: https://bugs.python.org/issue28812

----------
components: Interpreter Core
messages: 293906
nosy: Louis Brandy
priority: normal
severity: normal
status: open
title: deadlocked child process after forking on pystate.c's head_mutex
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue30395>
_______________________________________