Python C/API based multithread python program locks

[Anders J. Munch]

"kanji" <kanji_rama at yahoo.com> wrote:
>
> The issue is it works most of the time, but occassionally (I mean this
> is quite random ) it hangs and it does not generate any errors as
> such. While trying to debug, sometimes even the gdb hangs, but i
> managed to get a backtrace of a hung thread:
>
> #0  0xb75ebc32 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
> #1  0xb75d11ee in pthread_cond_wait@@GLIBC_2.3.2 ()
> from/lib/tls/libpthread.so.0
> #2  0x0809bb3f in PyThread_acquire_lock ()
> #3  0x0809e45c in _PyObject_GC_Del ()

My guess is that the problem is not in this thread: Some other thread is
hung or crashed, while holding the GIL, and this thread here is just waiting
on the GIL.

> So just to weed out the possibility that it is not because of some
> error in the code, I iteratively called the same function (which
> creates say 100 threads) in a for loop - for 500 times. I found that
> it tends to hang at different iterations -- say may be at iteration
> #480  or #12 or sometimes it sails smoothly.

You have a race condition.  There may be some shared resource that is not
accessed in a thread-safe manner. Perhaps some C global variable used by
your code or a library that your code calls?

Or perhaps you just have a blocking I/O-call waiting for something that
never happens.

Good luck - you're gonna need it :/

- Anders