[issue17263] crash when tp_dealloc allows other threads

[Albert Zeyer]

Albert Zeyer added the comment:

> > Wouldn't it be better to expose and re-use the HEAD_LOCK and HEAD_UNLOCK
> > macros from pystate.c?

> I don't like holding locks before calling "alien" code, it's a recipe
> for deadlocks: for example, if another thread-local object was
> deallocated as part of the deallocation chain, we would call back into
> local_clear(), and deadlock.

Ah, yes. Right now, the head-lock is acquired while the GIL is held. So while the head-lock is held, we must not unlock the GIL. So this wouldn't work.

Btw., I think it also does happen already. While playing around with this test case, I sometimes encountered a deadlock at quit. I was thinking that it was the result of some badly written memory.

But I just saw this code (PyInterpreterState_Clear):

    HEAD_LOCK();
    for (p = interp->tstate_head; p != NULL; p = p->next)
        PyThreadState_Clear(p);
    HEAD_UNLOCK();

So, if something inside PyThreadState_Clear unlocks the GIL and some other thread acquires the GIL and then tries to HEAD_LOCK (for example, at thread exit), you have a classic deadlock.

A solution would be: Only acquire the head-mutex while the GIL is not held. Then, after you held the head-mutex, also acquire the GIL.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17263>
_______________________________________