Ctypes and C Infinite Callback Loops

[Thomas Dimson]

Hello,

I have quite a complex issue that is arising with regards to using
ctypes to hook into some legacy code. The legacy code is in infinite
loop - I can not touch this. It does some listening, and periodically
calls a specific callback function.

What I would like to be able to do is spawn a Python thread to handle
this infinite loop, and continue on my merry way. This works to an
extent, however if I try to raise the SystemExit exception (or any
other one) inside of this thread I get an error message of
"AssertionError: cannot join current thread".

I assume there is some issue with the global interpreter lock or that
you can't exit the infinite loop from above Python. Any suggestions on
how I can design this so the thread will be able to issue exits/raise
exceptions just like a regular thread? Is there a way of terminating
this thread from the python interpreter or ctypes.pythonapi?

I have also tried being sneaky by using a pthread in the C code, but I
had issues when I tried to create a new thread state using
ctypes.pythonapi (well, I had issues swapping it in when I get to the
callback). If this is the best solution, how do I create/swap in the
thread state from ctypes?

For some cooked up sample code that simulates this:

main.c (main.o -> main.so )
#include <stdio.h>
void loop( void (*callback)() )
{
    while( 1 )
    {
        callback();
        sleep(1);
    }
}

void testLoop( void (*callback)() )
{
    loop( callback );
}


************************************************
test.py:
import threading,ctypes,time,sys,os

soPath = os.path.join( "/home/tdimson/ctypes/main.so" )

class callLoop( threading.Thread ):
    def callback( self ):
        sys.exit()

    def run( self ):
        ctypes.cdll.LoadLibrary( soPath )
        mainLib = ctypes.CDLL( soPath )
        _callback =  ctypes.CFUNCTYPE( None )( self.callback )

        mainLib.testLoop( _callback )

loopThread = callLoop()
loopThread.start()

while 1:
    print "Not blocking"
    time.sleep(10)

********************************************

Then I execute: python test.py and get
Not blocking
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python2.4/threading.py", line 634, in __exitfunc
    t.join()
  File "/usr/lib/python2.4/threading.py", line 532, in join
    assert self is not currentThread(), "cannot join current thread"
AssertionError: cannot join current thread
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python2.4/threading.py", line 634, in __exitfunc
    t.join()
  File "/usr/lib/python2.4/threading.py", line 532, in join
    assert self is not currentThread(), "cannot join current thread"
AssertionError: cannot join current thread


Thanks for even reading this much :)

-Thomas Dimson