Synchronous signals vs. robust code

[Mark Mitchell]

Python provides synchronous signals, even though signals are
asynchronous events at the operating system level.  In many ways,
this is a Good Thing.  For example, in C you can't do much of anything
from within a signal handler since you don't know much about the
current state of the application, but in Python you can do just
about anything.

In particular, you can throw an exception from a Python signal handler,
which makes handling signals simple.  In C, you generally have to
set a flag from within the signal handler, and then poll the flag
elsewhere; in Python you simply have the signal handler thrown an
exception and then provide an appropriate exception handler.

But, I'm having a hard time figuring out how to write signal-safe
code in the presence of synchronous signals.  Consider, for example,
the following:

  pipe = os.pipe()

  try:
    # Do stuff with the pipe.
  finally:
    # Close the ends of the pipe.
    os.close(pipe[0])
    os.close(pipe[1])

This code is supposed to make sure that the file descriptors are
not leaked; when we exit this scope, the descriptors will be closed,
even if an exception occurs.  [Note that I am assuming that os.close
will never thrown an exception when given a valid file descriptor.
That's not strictly speaking true on some operating systems, but
let's pretend that it is true.  For the purposes of this discussion,
the particular functions involved don't matter.]

This code, however, is not signal-safe.  For example, if a signal
occurs between the two calls to os.close, we will end up not closing
the second descriptor.  The equivalent C++ code *would* be signal safe;
an exception cannot be thrown from the signal-handler, so if the program
is still executing we can be sure that both calls to os.close will
occur.

One solution, of course, is to make sure that the signal-handlers do
not throw exceptions.  But that's not a reasonable solution in library
code; the library can't go around deciding what signal handlers should
be doing.

Another solution is to install an alternate signal handler around the
entire try-block.  This alternate signal handler would remember the
signal; we could then regenerate it at the end of the try-block.  This,
however, is not very robust; what about multiple signals occurring?  And
if the signal that occurs is important, we're ignoring it until the end
of the try-block, which isn't polite.

Another solution would be to provide a module that provides access to
sigprocmask.  Then, one could explicitly block signals during parts
of the code where these problems could occur.  For example:

  pipe = os.pipe()

  try:
    # Do stuff with the pipe.
  finally:
    # Block signals.
    sigprocmask(...)
    # Close the ends of the pipe.
    os.close(pipe[0])
    os.close(pipe[1])
    # Unblock signals.
    sigprocmask(...)

This does not work either; there are still at least two race conditions.
A signal that arrives after we have entered the finally clause, but
before we have called sigprocmask, would still result in a failure to
close the descriptors.  Similarly, a signal that occurs after the call
to os.pipe, but before the try-block is entered would result in us
skipping the finally clause.  (And moving the try earlier is not valid;
we can't clean up the descriptors unless we know that the pipe has been
created.)

Here is another try:

  # Block signals.
  sigprocmask(...)

  try:
    pipe = os.pipe()

    try:
      # Do stuff with the pipe.
    finally:
      # Close the ends of the pipe.
      os.close(pipe[0])
      os.close(pipe[1])
  finally:
    # Unblock signals.
    sigprocmask(...)

This variant is robust; we are now guaranteed that if the pipe is
created, the descriptors will be closed.

However, it is still unsatisfactory in that it results in the signal
being delayed for an arbitrary amount of time; if a time-critical
signal happens during the try-block, we will not process it until
much later.

Another solution is to require that the code be run in a thread other
than the main thread; since only the main thread receives signals,
the signal-safety problem does not occur.  However, it seems excessive
to require threads simply to write signal-safe code!  And if you have
code that depends on, say, getting SIGPIPE, then you have to have some
way of communicating the signal from the main thread to the thread
running the code above.

What do Python programmers that need to write truly robust code do
about this problem?  Is there a solution that I have missed?

If not, would the Powers That Be be amenable to making changes to the
language to support signal-safety?  One possible fix would be
except/finally clauses that blocks signals on entry.  For example,
if there were a finally_block_signals clause, you could do:

  # Block signals.
  sigprocmask(...)

  try:
    pipe = os.pipe()
  except:
    # Unblock signals.
    sigprocmask(...)
    # Reraise the exception.
    raise

  try:
    # Unblock signals.
    sigprocmask(...)
    # Do stuff with the pipe.
  finally_block_signals:
    # Close the pipe ends.
    os.close(pipe[0])
    os.close(pipe[1])
    # Unblock signals.
    sigprocmask(...) # There needs to be a way of getting the signal
                     # mask on entry to the finally clause.

This is not particularly pretty, but it would work.

Thoughts?

-- 
Mark Mitchell                mark at codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com