How to force a thread to stop

[Carl J. Van Arsdall]

bryanjugglercryptographer at yahoo.com wrote:
> Carl J. Van Arsdall wrote:
> [...]
>   
>> My problem with the fact that python doesn't have some type of "thread
>> killer" is that again, the only solution involves some type of polling
>> loop.
>>     
>
> A polliing loop is neither required nor helpful here.
>
> [...]
>   
>> #Just pretend for the sake of arguement that 'op' actually means
>> something and is a lengthy operation
>> def func_to_thread():
>>   os.system('op 1')
>>   os.system('op 2')
>>   os.system('op 3')
>>     
>
> What good do you think killing that thread would do? The
> process running 'op n' has no particular binding to the thread
> that called os.system(). If 'op n' hangs, it stays hung.
>
> The problem here is that os.system doesn't give you enough
> control. It doesn't have a timeout and doesn't give you a
> process ID or handle to the spawned process.
>
> Running os.system() in multiple threads strikes me as
> kind of whacked. Won't they all compete to read and write
> stdin/stdout simultaneously?
>   
Unfortunately this is due to the nature of the problem I am tasked with 
solving.  I have a large computing farm, these os.system calls are often 
things like ssh that do work on locations remote from the initial python 
task.  I suppose eventually I'll end up using a framework like twisted 
but, as with many projects, I got thrown into this thing and threading 
is where we ended up.  So now there's the rush to make things work 
before we can really look at a proper solution.

>   
>> #In order to make this killable with reasonable response time we have to
>> organize each of our ops into a function or something equally annoying
>>
>> op_1():
>>   os.system('op 1')
>>
>> op_2():
>>   os.system('op 2')
>>
>> op_3():
>>   os.system('op 3')
>>
>> opList(op_1, op_2, op_3)
>> def to_thread():
>>   for op in opList:
>>     checkMessageQueue()
>>     op()
>>     
>
> Nonsense. If op() hangs, you never get to checkMessageQueue().
>   
Yea, understood.  At the same time, I can't use a timeout either, I 
don't know how long op_1 or op_2 will be.  This is why I want something 
that is triggered on an event.


> Now suppose op has a timeout. We could write
>
>   def opcheck(thing):
>       result = op(thing)
>       if result == there_was_a_timeout:
>           raise some_timeout_exception
>
> How is:
>
>   def func_to_thread():
>       opcheck('op 1')
>       opcheck('op 2')
>       opcheck('op 3')
>
> any less managable than your version of func_to_thread?
>
>   
Again, the problem I'm trying to solve doesn't work like this.  I've 
been working on a framework to be run across a large number of 
distributed nodes (here's where you throw out the "duh, use a 
distributed technology" in my face).  The thing is, I'm only writing the 
framework, the framework will work with modules, lots of them, which 
will be written by other people.  Its going to be impossible to get 
people to write hundreds of modules that constantly check for status 
messages.  So, if I want my thread to "give itself up" I have to tell it 
to give up.  In order to tell it to give up I need some mechanism to 
check messages that is not going to piss off a large team of 
programmers.  At the same time, do I really want to rely on other people 
to make things work?  Not really, I'd much rather let my framework 
handle all control and not leave that up to programmers.

So the problem is, I have something linearly executed a large list of 
python functions of various sizes ranging from short to long.  Its not 
about killing the thread so much as how do I make the thread listen to 
control messages without polling.



>> So with this whole "hey mr. nice thread, please die for me" concept gets
>> ugly quickly in complex situations and doesn't scale well at all.
>> Furthermore, say you have a complex systems where users can write
>> pluggable modules.  IF a module gets stuck inside of some screwed up
>> loop and is unable to poll for messages there's no way to kill the
>> module without killing the whole system.  Any of you guys thought of a
>> way around this scenario?
>>     
>
> Threadicide would not solve the problems you actually have, and it
> tends to create other problems. What is the condition that makes
> you want to kill the thread? Make the victim thread respond to that
> condition itself.
>
>   

I feel like this is something we've established multiple times.  Yes, we 
want the thread to kill itself.  Alright, now that we agree on that, 
what is the best way to do that.  Right now people keep saying we must 
send the thread a message.  That's fine and I completely understand 
that, but right now the only mechanism I see is some type of polling 
loop (or diving into the C API to force exceptions).  So far I've not 
seen any other method though.  If you want to send a thread a control 
message you must wait until that thread is able to check for a control 
message.  If something hangs in your thread you are totally screwed, 
similarly, if your thread ends up in some excessively lengthy IO (IO 
that could be interrupted or whatever) you have to wait for that IO to 
finish before your thread can process any control messages.




-- 

Carl J. Van Arsdall
cvanarsdall at mvista.com
Build and Release
MontaVista Software