threading.Event usage causing intermitent exception

[akameswaran at gmail.com]

Tim Peters wrote:
> [akameswaran at gmail.com]
> > Admittedly this problem causes no actual functional issues aside from
> > an occasional error message when the program exits.  The error is:
> >
> > Unhandled exception in thread started by
> > Error in sys.excepthook:
> > Original exception was:
> >
> > Yes all that info is blank.
>
> That's typical when the interpreter has torn so much of itself down
> that there's not enough left in the `sys` module even to print
> exception info gracefully.  The easiest way to stop that is to stop
> /trying/ to run Python code while the interpreter is tearing itself
> down.
>
> > The application is a console application that is waiting for some
> > condition on the machine to happen.  However, I leave open the
> > possiblitiy to cancel by a single key press at which
> > point the program terminates.  Suffice it to say, I cannot perform both
> > checks without invoking threads as the key press gets "missed"
> > sometimes.  Below is a simplification of the code
> >
> > canceled = False
> > myEvent = threading.Event()
> >
> > def watchForCancel()
> >     global canceled
> >     # turn of terminal buffering and capture key presses here
> >     canceled = True
> >     myEvent.set()
>
> Presumably this is some kind of poll-and-sleep loop?  If so, add a
> check to get out of the loop if myEvent.isSet().  Or if not, make it
> some kind of poll-and-sleep loop ;-)

after setting up the terminal correctly, ie changing buffering settings
etc. (I want to cancel with a single key-press with no "return" hit.
This was suprisingly difficult to do.  The wait code is simply:

    while keyPressed != 'q':
        keyPressed = sys.stdin.read(1)

The really annoying thing here, is that I cannot put in any other steps
in the while loop without sometimes missing the key press.  I have to
unbuffer the terminal(otherwise the read doesn't return until the enter
key is pressed), and for whatever reason performing any action no
matter how trivial leads to missing the key stroke on occasion - that's
why I pursued the daemon route.
>
> > def watchForCondition()
> >     # do a bunch of stuff checking the system
> >     myEvent.set()
>
> Ditto.
>
> > cancelThread = threading.Thread(target = watchForCancel)
> > cancelThread.setDaemon(True)  # so I can exit the program when I want to
>
The conditional thread will never terminate without making the threads
aware of eachother.  Setting them daemon allows the interpreter to
shutdown with one of them running - and ocaisonally cuases the error.
What I find interesting, the behavior only happens when I cancel, not
when the condition is reached.

> And get rid of that.  The comment doesn't make sense to me, and
> forcing a thread to be a daemon is exactly what /allows/ the thread to
> keep running while the interpreter is tearing itself down.  That's why
> "daemonism" isn't the default:  it's at best delicate.  I don't see a
> real reason for wanting this here.
>
> > cancelThread.start()
> > conditionThread = threading.Thread(target = watchForCondition)
> > conditionThread.setDaemon(True)
>
> Ditto.
>
> > conditionThread.start()
> >
> > myEvent.wait()
> >
> > if cancelled:
> >     sys.exit(2)
> >
> > # do more stuff if the condition returned instead of cancel and then
> > I'm done
> >
> >
> > I've left out most of the active code, just cuz I think it muddies the
> > water.  Now about 9 out of 10 times this works just fine.  However,
> > every once in a while I get the exceptions mentioned above, but only
> > when I cancel out of the operation.  I think the conditionThread is in
> > the process of shutting down and gets hosed up somehow and spits out an
> > exception, but the interpreter no longer has access to the info since
> > it is shutting down.
>
> At this point it's likely that even sys.stdout and sys.stderr no
> longer exist.  The "Unhandled exception" message is printed directly
> to the C-level `stderr` instead.
>

exactly - giving me no way to swallow it.

> > ...
> > I suppose I could make the threads aware of each other, but that just
> > seems stupid.  Any suggestions on how to eliminate this intermittent
> > error?
>
> Stop forcing them to be daemon threads.  The interpreter won't start
> to tear itself down then before both threads terminate on their own.
> To arrange for that, it's not necessary for the threads to become
> aware of each other, but it is necessary for the threads to become
> aware of another (shared) reason /for/ exiting.  The most natural way
> to do that, given what you said above, is to make both threads aware
> that their shared myEvent event may get set "externally", and to stop
> when they find it has been set.

Due to the oddities of capturing that keypress at the console, I
suspect I can un-daemonize the conditional thread and check for
cancelled=True and have it return.  Since the error only occurs when
the conditional thread is still running, undaemonizing it will prevent
that and allow the interpreter to shut down more gracefully.

Thanks for the advice.