[Python-checkins] [3.11] GH-102126: fix deadlock at shutdown when clearing thread state… (#102234)

[kumaraditya303]

https://github.com/python/cpython/commit/026faf20cc9d1d5913ff7c01a93d8934594d7fec
commit: 026faf20cc9d1d5913ff7c01a93d8934594d7fec
branch: 3.11
author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com>
committer: kumaraditya303 <59607654+kumaraditya303 at users.noreply.github.com>
date: 2023-02-25T21:00:05+05:30
summary:

[3.11] GH-102126: fix deadlock at shutdown when clearing thread state… (#102234)

[3.11] GH-102126: fix deadlock at shutdown when clearing thread states (GH-102222)

(cherry picked from commit 5f11478ce7fda826d399530af4c5ca96c592f144)

files:
A Misc/NEWS.d/next/Core and Builtins/2023-02-24-17-59-39.gh-issue-102126.HTT8Vc.rst
M Python/pystate.c

diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-02-24-17-59-39.gh-issue-102126.HTT8Vc.rst b/Misc/NEWS.d/next/Core and Builtins/2023-02-24-17-59-39.gh-issue-102126.HTT8Vc.rst
new file mode 100644
index 000000000000..68c43688c3df
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2023-02-24-17-59-39.gh-issue-102126.HTT8Vc.rst	
@@ -0,0 +1 @@
+Fix deadlock at shutdown when clearing thread states if any finalizer tries to acquire the runtime head lock. Patch by Kumar Aditya.
diff --git a/Python/pystate.c b/Python/pystate.c
index c0d161f894c9..dfca3f5fd713 100644
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -396,11 +396,19 @@ interpreter_clear(PyInterpreterState *interp, PyThreadState *tstate)
         _PyErr_Clear(tstate);
     }
 
+    // Clear the current/main thread state last.
     HEAD_LOCK(runtime);
-    for (PyThreadState *p = interp->threads.head; p != NULL; p = p->next) {
+    PyThreadState *p = interp->threads.head;
+    HEAD_UNLOCK(runtime);
+    while (p != NULL) {
+        // See https://github.com/python/cpython/issues/102126
+        // Must be called without HEAD_LOCK held as it can deadlock
+        // if any finalizer tries to acquire that lock.
         PyThreadState_Clear(p);
+        HEAD_LOCK(runtime);
+        p = p->next;
+        HEAD_UNLOCK(runtime);
     }
-    HEAD_UNLOCK(runtime);
 
     Py_CLEAR(interp->audit_hooks);