KeyboardInterrupt eats my error and then won't be caught

[Piet van Oostrum]

>>>>> greg <greg at cosc.canterbury.ac.nz> (g) wrote:

>g> Philip Semanchuk wrote:
>>> try:
>>> sem.acquire()   # User hits Ctrl + C while this is waiting
>>> except:
>>> print "********* I caught it!"

>>> Instead a KeyboardInterrupt error is propagated up to the interpreter
>>> and the process is killed as if the try/except wasn't there at all.

>g> Not sure exactly what's happening, but I think I can guess.
>g> Python installs a signal handler for Ctrl-C that sets a
>g> flag in the interpreter. Every so many bytecodes executed,
>g> the flag is checked and KeyboardInterrupt raised if it's
>g> set.

>g> So there can be a short delay between the Ctrl-C signal
>g> being received and KeyboardInterrupt being raised, and it
>g> seems that this delay results in it happening after the
>g> try-except has exited.

I think you are approaching the cause of the problem.
Your answer triggered the following thought in my head:

There are actually two exceptions occurring: One is the Ctrl-C, which as
you correctly say, will probably delayed until there is a `check' in the
interpreter (see also David Beazley's wonderful presentation on the
GIL). The other one is the exception that is generated in the IPC code
by returning a NULL. This one should caught by the except clause.

As the call to sem.acquire releases and reacquires the GIL, I think
signal processing will be done immediately. This causes the
KeyboardInterrupt exception to occur immediately i.e. to interrupt the
handling of the other exception.

>g> You could try using signal.signal() to install a handler
>g> for Ctrl-C that does nothing in a section around the
>g> sem.acquire call(). That should prevent the KeyboardInterrupt
>g> flag from being set, but the signal will still be occurring
>g> at the Unix level, so the system call will get interrupted.

Your suggestion seems to work:

import posix_ipc
import signal

sem = posix_ipc.Semaphore(None, posix_ipc.O_CREX)
signal.signal(signal.SIGINT, lambda sig, frame: None)
status = []
try:
    status.append("Trying")
    sem.acquire()   # User hits Ctrl + C while this is waiting
    status.append("Acquired")
except:
    status.append("I caught it!")
print status
    
sem.close()
sem.unlink()

prints: ['Trying', 'I caught it!']

I also tried some other variants, catching the KeyboardInterrupt at
various places:

This one prints: ['Trying', 'Keyboard Interrupt']
This suggests to me that the first exception handling is aborted by the
Ctrl-C handling.

import posix_ipc

sem = posix_ipc.Semaphore(None, posix_ipc.O_CREX)

status = []
try:
    try:
        status.append("Trying")
        sem.acquire()   # User hits Ctrl + C while this is waiting
        status.append("Acquired")
    except:
        status.append("I caught it!")
except KeyboardInterrupt:
    status.append("Keyboard Interrupt")
print status
    
sem.close()
sem.unlink()

And this one prints: ['Trying', 'I caught it!']

import posix_ipc

sem = posix_ipc.Semaphore(None, posix_ipc.O_CREX)

status = []
try:
    status.append("Trying")
    try:
        sem.acquire()   # User hits Ctrl + C while this is waiting
        status.append("Acquired")
    except KeyboardInterrupt:
        status.append("Interrupt")
except:
    status.append("I caught it!")
print status
    
sem.close()
sem.unlink()

I was actually a bit surprised that the addition of the try/except
KeyboardInterrupt helps solve the problem but that apparently the
exception handler is not executed. 

Folding the two try's into one with two except clauses will not help as
there are indeed two exceptions to be handled.

I also added traceback printout in the outer exception handler and it
points to the sem.acquire line. 

My conclusion is that if there are two exceptions at the same time, the
inner exception handler is interrupted by the other exception even
before the except clause can be entered. And only the outer one is
really executed. This explains the behaviour that the OP described.

I think you can only have two exceptions at the same time if at least
one of them is a signal.
-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org