How do i : Python Threads + KeyboardInterrupt exception

[Brendon Costa]

Hi all,

I have a small python project i am working on. Basically i always have
two threads. A "Read" thread that sits in a loop reading a line at a
time from some input (Usually stdin) and then generating events to be
processed and a "Proc" thread that processes incoming events from a
queue. There will be additional threads as well that asynchronously
insert events into the queue to be processed, but they are not a part
of this so i have omitted them.

What i want to know is: "What is the standard/best way of implementing
such a pattern that works in the presence of errors and particularly
with the KeyboardInterrupt exception?"

Some sample code is shown below. This code works as is, except in the
case where the "Proc" thread wants to initiate the exit of the
application.

For example:
* running the code below and pressing Ctrl + C works fine as the Read
thread is initiating the shutdown.
* running the code below and entering:
   pquit
   some other data
   <Ctrl + D>

will cause oytput:

Processing: pquit
Proc: Initiating quit

and then it HANGS waiting for the Read thread to exit.

Some questions i have that are:
* KeyboardInterrupt exception seems to only be recieved by the main
thread. Is this ALWAYS the case across all UNIX + windows platforms
(not so worried about others)?
* Can i somehow get the Proc thread to force the Read thread to
generate a KeyboardInterrupt or somehow exit its blocking "for line in
fin:" call?


Thanks,
Brendon


--- SNIP ---
# Two or more threads
#
# proc : Is a processing thread that basically reads events from a
event queue and processes them
# read : Is a thread reading in a loop from stdin and generating
events for "proc"
# * : Other additional threads that may asynchronously add events to
the queue to be processed

import Queue
import threading
import sys

def Read(queue, fin, fout):
   ret = (1, 'Normal Exit')
   try:
      for line in fin:
         queue.put((0, line))
         #raise Exception("Blah")
         #raise "Blah"
   except KeyboardInterrupt:  ret = (1, 'KeyboardInterrupt')
   except Exception, e:       ret = (1, 'ERROR: ' + str(e))
   except:                    ret = (1, 'UNKNOWN-ERROR')

   # Notify Proc thread that we are exiting.
   queue.put(ret)
   print >>fout, 'Read: Initiating quit'


def Proc(queue, fout, ignore):
   quit = False
   while not quit:
      (evt_type, evt_data) = queue.get()

      if   evt_type == 0:
         print >>fout, 'Processing: ' + str(evt_data)
         if evt_data.startswith('pquit'):
            print >>fout, 'Proc: Initiating quit'
            quit = True

      elif evt_type == 1:
         print >>fout, 'Quit: ' + str(evt_data)
         quit = True

class MyThread(threading.Thread):
   def __init__(self, func, queue, file1, file2, *args, **kwds):
      threading.Thread.__init__(self, *args, **kwds)
      self.func = func
      self.queue = queue
      self.file1 = file1
      self.file2 = file2
      self.start()

   def run(self):
      return self.func(self.queue, self.file1, self.file2)


if __name__ == '__main__':
   queue = Queue.Queue()

   # Read thread is the main thread and seems to get the
KeyboardInterrupt exception.
   t = MyThread(Proc, queue, sys.stderr, None)
   Read(queue, sys.stdin, sys.stderr)

   # Read thread is NOT the main thread and never seems to get the
KeyboardInterrupt exception.
   # This doesnt work for that reason.
   #t = MyThread(Read, queue, sys.stdin, sys.stderr)
   #Proc(queue, sys.stderr, None)


   # @@@Brendon How do we notify the Read thread that they should
exit?
   # If the Read thread initiated the quit then all is fine.
   # If the Proc thread initiated the quit then i need to get the
Read
   # thread to exit too somehow. But it is currently blocking in a
read
   # on an input file.
   print >>sys.stderr, 'Joining thread.'
   t.join()