xmlrpc idea for getting around the GIL

[Daniel Fetchinson]

>> icating) the multiprocessing module would be ideal.
>>> > The problem is that the OP has a embedded application running threads.
>>> > multiprocssing doesn't help there.
>>>
>>> that's right. I cannot make CPython calls from my original C-based
>>> threads.
>>
>>
>> It's quite possible to do that.  A thread started from C can make
>> calls to Python if it first calls PyGILState_Ensure, although you'd
>> have to make sure that the Python interpreter has been previously
>> initialized.  See PEP 311 for details.
>>
>> http://www.python.org/dev/peps/pep-0311/
>>
>> I also suggest that if you want people to be more receptive to write
>> your replies after the quoted text.  That is the custom in this
>> newsgroup/mailing list.
>>
>
> What I meant was that I am *not allowed* to make calls to the CPython
> API from the threads I currently have because these threads are high
> priority and are never allowed to make blocking calls. Fortunately,
> each of these threads can have a completely separate interpreter, so
> my idea was to create a daemon process for each thread. This daemon
> would contain the libpython symbols and would make the CPython calls.
> This would keep my current threads from having to contend over the
> single GIL.
>
> My question was whether or not anyone has done anything like this in
> C/C++. This situation is slightly unique in that I am trying to
> maintain separate interpreters within a single app (which is currently
> kind of hacked out using a single interpreter, but it's ugly), but I
> could see how this sort of thing would be useful for other C/C++ apps
> that implement an embedded scripting engine.
>
> My reference to multiprocessing was based on the idea that the library
> hides the details fo the process management, shared memory, and rpc
> mechanisms. Again, I can't use multiprocessing because it runs *in*
> python I need this to be implemented *outside* of python to avoid
> acquiring the GIL. complex, I know.
>
> Naturally, the most intimidating part of perusing this kind of idea is
> the overhead of the processess management and the RPC. Further,
> libpython executes callable objects using C function pointers, and I
> can't think of a way that this would be able to re-gain access to the
> original app's respective functions if the interpreter was living in
> another processes.
>
> That's not to mention the impossibility of debugging.
>
> Damn you, Gil.


By the way, you might be interested in this thread as well:

http://groups.google.com/group/comp.lang.python/browse_thread/thread/2d537ad8df9dab67/cc4cb2b493c98170

HTH,
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown