Terminating a Python program that uses multi-process, multi-threading

[MRAB]

akineko wrote:
> Hello Python experts,
> 
> I have a program that uses three processes (invoked by
> multiprocessing) and several threads.
> The program is terminated when ^C is typed (KeyboardInterrupt).
> The main process takes the KeyboardInterrupt Exception and it orderly
> shutdown the program.
> 
> It works fine in normal situation.
> 
> However, KeyboardInterrupt is not accepted when, for example, the
> program is stuck somewhere due to error in network. I understand that
> the KeyboardInterrupt won't be processed until the program moves out
> from an atomic step in a Python program.
> 
> Probably this is partly my fault as I use some blocking statements
> (without timeout) as they should not block the program in normal
> situation.
> 
> As the program freezes up, I cannot do anything except killing three
> processes using kill command.
> So, I cannot tell which statement is actually blocking when this
> occurs (and there are many suspects).
> 
> Is there any good way to deal with this kind of problem?
> Killing three processes when it hangs is not a fun thing to do.
> 
> Any suggestions will be greatly appreciated.
> 
The scripts could write to a log around where they could block (write to 
the log just before and just after), so that you can then see where they 
blocked.

A hacky way of killing the processes would be for the processes to write 
their process IDs to the log and then you can write another script which 
will parse the log to get the process IDs and then kill the processes 
with those IDs. (Left as an exercise for the reader. :-))