embedded Python: one thread hangs while trying to get global Python lock

[Létezõ]

I wrote an embedded python application 2 years ago using python 1.5.2. I had
similar hangups with my multithreaded (3..34 threads were active at the same
time) app that I wasn't able to debug. I spent houndreds of hours to debug
my code, but serial communication failed in an almost random way!
Conclusion: Sometimes some usage patterns of external APIs can block python
threads in unexpected ways.

I suppose your embedded app is not time-critical, since it has been written
in python. I know that this is not the real solution to your problem, but as
my opinion you shouldn't use multiple threads. Instead try to use an event
loop that waits for an event (using select(), poll() or something similar)
then call it's small non-blocking handlers, or try to fork() your
application and communicate on TCP/IP or pipes between the parts. The later
solution also gives you scalability. Synchronization problems should also go
away with this technique.

Everybody else with a real solution?

- Complex -

----- Original Message -----
From: "Gernot Hillier" <ghillie at suse.de>
Newsgroups: comp.lang.python
To: <python-list at python.org>
Sent: Monday, February 03, 2003 10:43 AM
Subject: embedded Python: one thread hangs while trying to get global Python
lock


> Hi!
>
> I experience a weird problem in my multithreaded telecommunication
> application with embedded Python. It does roughly the following:
>
> - thread A is running continuously and reads a python script and starts a
> function def'd in there in regular intervals
>
> - thread B is created on demand (when a call comes in), runs another
python
> script, calls a function def'd in there and finishes after that.
>
> When thread A + B run at the same time, thread A suddenly hangs forever on
> the following operation:
>
> (gdb) bt
> #0  0x4026cea9 in sigsuspend () from /lib/libc.so.6
> #1  0x4003bd48 in __pthread_wait_for_restart_signal () from
> /lib/libpthread.so.0
> #2  0x4003814b in pthread_cond_wait () from /lib/libpthread.so.0
> #3  0x080eb25f in PyThread_acquire_lock ()
> #4  0x080c8dff in PyEval_RestoreThread ()
> #5  0x0806a984 in capisuite_disconnect (args=0x824544c) at
> capisuitemodule.cpp:422
> #6  0x080a7963 in PyCFunction_Call ()
> #7  0x080cb66d in eval_frame ()
> #8  0x080cc04e in PyEval_EvalCodeEx ()
> #9  0x080cd584 in fast_function ()
> #10 0x080cb5b2 in eval_frame ()
> #11 0x080cc04e in PyEval_EvalCodeEx ()
> #12 0x0810373c in function_call ()
> #13 0x08092b17 in PyObject_Call ()
> #14 0x080cd219 in PyEval_CallObjectWithKeywords ()
> #15 0x08092ace in PyObject_CallObject ()
> #16 0x08066106 in PythonScript::run() (this=0x8163f80) at
> pythonscript.cpp:80
> #17 0x08067fae in IdleScript::run() (this=0x8163f80) at idlescript.cpp:59
> #18 0x40090ee1 in ost::ThreadImpl::ThreadExecHandler(ost::Thread*) () from
> /usr/lib/libccgnu2-0.99.so.0
> #19 0x4009025c in ccxx_exec_handler.1 () from /usr/lib/libccgnu2-0.99.so.0
> #20 0x400391b0 in pthread_start_thread () from /lib/libpthread.so.0
>
> To explain the backtrace:
> - the def'd function is called (#15)
> - it starts a function implemented in C (#6/#5)
> - This function is doing:
>
>         Py_UNBLOCK_THREADS
>         // do some blocking operation
>         Py_BLOCK_THREADS  <---------- This is line 422 where it hangs
>
> So I firstly thought it was clear I made an error somewhere in thread B
thus
> not releasing the python global lock, so thread A can't get it any more.
>
> But I can now start thread B over and over again and it runs w/o problems
> while thread A still hangs until it's killed.
>
> So if I understood things right, it's very unlikely the global lock is
held
> by someone, otherwise thread B won't be able to run also.
>
> So... Has anyone any idea what the reason for this problem could be? Or
> experienced anything like this? Or any insights in Python / pthreads that
> could help?
>
> I'm appreciating any suggestions on how to find the culprit.
>
> BTW: This is Python 2.2.1. Have browsed through the WhatsNew of 2.2.2 but
> couldn't find any signs of a multithreaded problem fixed there so I didn't
> try it so far...
>
> --
> Ciao,
>
> Gernot
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>