Lock acquisition by the same thread - deadlock protection

[Barry Scott]

> On 9 Mar 2020, at 22:53, Yonatan Goldschmidt <yon.goldschmidt at gmail.com> wrote:
> 
> I recently debugged a program hang, eventually finding out it's a deadlock of a single thread,
> resulting from my usage of 2 libraries. One of them - call it library A - is reentrant & runs code in
> GC finalizers, while the other - library B - is not reentrant at all.
> Library B held one of its `threading.Lock` locks, and during this period, GC was invoked, running
> finalizers of library A which call back into library B, now attempting to take the lock again,
> locking the thread forever.
> 
> Considering how relatively common this scenario might be (Python, by design, can preempt any user code
> to run some other user code, due to GC finalizers), I was surprised Python code is not protected
> from this simple type of deadlock. It makes sense that while `threading.RLock` allows for recursive
> locking, `threading.Lock` will prevent it - raising an exception if you attempt it.
> 
> I might be missing something, but why isn't it the status? Why taking a `threading.Lock` twice from
> the same thread just hangs, instead of raising a friendly exception?
> I ran a quick search in bpo but found nothing about this topic. I also tried to
> search this mailing list but couldn't find how to, so I grepped a few random archives
> but found nothing about it.
> 
> Would be happy if anyone could shed some light on it...

threading.Lock is not reentrant and its implementation does not allow detection of the problem
from what I recall.

In this case the code might want to use the threading.RLock that is reentrant.
Of course there may be other issues in the code that prevent the finalizers working if it holds the lock.

Barry