Problem with embedded python

[Tom Cocagne]

 From looking at your example, it looks like you're making the problem FAR
more difficult than it needs to be. The main thing to keep in mind is that
Python threads do not correspond to operating system threads. In an
application using a single OS-level thread, you can use as many Python
threads as you like. The thread-switching mechanism in Python is to
transfer control between threads after the execution of 100 bytecodes. The
GIL is used to insure that only 1 operating system thread is executing
bytecodes at any given time. This allows os threads to sleep on blocking
calls (usually I/O) without necessarily stalling the interpreter.

 Your example appears to be written under the assumption that the spawned
Python thread will operate concurrently with your main thread. That will
not be the case. The PyRun_SimpleString() call will operate entirely within
the main thread and will not return until the Python thread has completed
operation (so no GIL manipulation is needed). The subsequent Sleep() call
will simply halt the process for 300ms. 

 Regular threading is difficult enough to get right without throwing the
Python-thread / OS-thread interaction into the mix. It's a tough nut to
crack so don't feel bad if it takes a while to get everything straight ;-)
The quickest and most practical approach would probably be to read through
some code that's similar to what you're shooting for. I'd suggest taking a
look at some of the modules in the Python source code (particularly the
network related ones) to get a better handle on how threads are managed. 

 Cheers,

 Cocagne


Ugo Di Girolamo wrote:

> I have the following code, that seems to make sense to me.
> 
> 
> However, it crashes about 1/3 of the times.
> 
> 
> My platform is Python 2.4.1 on WXP (I tried the release version from
> the msi and the debug version built by me, both downloaded today to
> have the latest version).
> 
> 
> The crash happens while the main thread is in Py_Finalize.
> I traced the crash to _Py_ForgetReference(op) in object.c at line 1847,
> where I have op->_ob_prev == NULL.
> 
> 
> What am I doing wrong? I'm definitely not too sure about the way I'm
> handling the GIL.
> 
> 
> Thanks in adv for any suggestion/ comment
> 
> 
> Cheers and ciao
> 
> 
> Ugo
> 
> 
> ////////////////////////// TestPyThreads.py //////////////////////////
> #include <windows.h>
> #include "Python.h"
> 
> 
> int main()
> { 
>         PyEval_InitThreads();
>         Py_Initialize();
>         PyGILState_STATE main_restore_state = PyGILState_UNLOCKED;
>         PyGILState_Release(main_restore_state);
> 
> 
>         // start the thread
>         { 
>                 PyGILState_STATE state = PyGILState_Ensure();
>                 int trash = PyRun_SimpleString(
>                                 "import thread\n"
>                                 "import time\n"
>                                 "def foo():\n"
>                                 "  f = open('pippo.out', 'w', 0)\n"
>                                 "  i = 0;\n"
>                                 "  while 1:\n"
>                                 "    f.write('%d\\n'%i)\n"
>                                 "    time.sleep(0.01)\n"
>                                 "    i += 1\n"
>                                 "t = thread.start_new_thread(foo, ())\n"
>                                 );
>                 PyGILState_Release(state);
>         }
> 
> 
>         // wait 300 ms
>         Sleep(300);
> 
> 
>         PyGILState_Ensure();
>         Py_Finalize();
>         return 0;
> 
> }