calldll and threads...

[Mark Hammond]

Steve wrote:
> I could benefit from words of wisdom... this is the situation:
> 
> I want to use Python as a high level language to write a test suite
> for a complex shared library (I've a windows DLL acting as proxy to
> remote server(s)) which has several blocking calls, and for which the
> interleaving and overlapping of calls is very significant. I need to
> drive a client to demonstrate code coverage on the server side - there
> is a large degree of server side interaction between concurrently
> executing client actions.
> 
> In Python, my first attempt used thread.start_new_thread() when
> executing a python wrapper for a call which could block.. which I
> later rewrote to use threading.Thread.  Initially all seemed well
> until I found that the whole python program blocked waiting for the
> first blocking call to the DLL.  Then I read about the "Global
> Interpreter Lock" and went on to look at the source for calldll.pyd -
> which didn't seem to address threading.  I tried following the advice
> in the python docs:
> 
> Py_BEGIN_ALLOW_THREADS
> //...Do some blocking I/O operation... making sure not to use
> return...
> Py_END_ALLOW_THREADS
> 
> Which I used to wrapper all calls to dll functions with a customised
> calldll... the above code massaged into call_foreign_function()... 
> but I now find that while the very first call to my modified calldll
> seems to work fine - subsequent calls crash the python interpreter.
> 
> Has anyone else had similar problems with calldll or anything like it?
> Can anyone explain what I've done wrong with Py_BEGIN_ALLOW_THREADS
> and friend? Do other people use python to script similar regression
> test scenarios? Is there a standard mechanism to make a blocking call
> from a python thread?

This is a common problem, not just with calldll.  Although you released 
the lock OK, you will probably find that you did not re-acquire the lock 
as calldll made callbacks into your code.  What was the callstack when 
you crashed?

Mark.