Synchronous signals vs. robust code

[Peter Milliken]

Why not use a language that provides your requirements of signal safety etc
and do a Python API for it? (assuming one doesn't exist) i.e. I find it
somewhat difficult to reconcile your statement about responding to time
critical signals when you propose use of an interpretive language..... :-)
Sounds like you would ideally like real-time language features in a language
that isn't designed for it (well, I'm not sure Guido had visions of Python
being used in real-time applications :-)).

Peter

"Mark Mitchell" <mark at codesourcery.com> wrote in message
news:mailman.1013998768.20770.python-list at python.org...
> Python provides synchronous signals, even though signals are
> asynchronous events at the operating system level.  In many ways,
> this is a Good Thing.  For example, in C you can't do much of anything
> from within a signal handler since you don't know much about the
> current state of the application, but in Python you can do just
> about anything.
>
> In particular, you can throw an exception from a Python signal handler,
> which makes handling signals simple.  In C, you generally have to
> set a flag from within the signal handler, and then poll the flag
> elsewhere; in Python you simply have the signal handler thrown an
> exception and then provide an appropriate exception handler.
>
> But, I'm having a hard time figuring out how to write signal-safe
> code in the presence of synchronous signals.  Consider, for example,
> the following:
>
>   pipe = os.pipe()
>
>   try:
>     # Do stuff with the pipe.
>   finally:
>     # Close the ends of the pipe.
>     os.close(pipe[0])
>     os.close(pipe[1])
>
> This code is supposed to make sure that the file descriptors are
> not leaked; when we exit this scope, the descriptors will be closed,
> even if an exception occurs.  [Note that I am assuming that os.close
> will never thrown an exception when given a valid file descriptor.
> That's not strictly speaking true on some operating systems, but
> let's pretend that it is true.  For the purposes of this discussion,
> the particular functions involved don't matter.]
>
> This code, however, is not signal-safe.  For example, if a signal
> occurs between the two calls to os.close, we will end up not closing
> the second descriptor.  The equivalent C++ code *would* be signal safe;
> an exception cannot be thrown from the signal-handler, so if the program
> is still executing we can be sure that both calls to os.close will
> occur.
>
> One solution, of course, is to make sure that the signal-handlers do
> not throw exceptions.  But that's not a reasonable solution in library
> code; the library can't go around deciding what signal handlers should
> be doing.
>
> Another solution is to install an alternate signal handler around the
> entire try-block.  This alternate signal handler would remember the
> signal; we could then regenerate it at the end of the try-block.  This,
> however, is not very robust; what about multiple signals occurring?  And
> if the signal that occurs is important, we're ignoring it until the end
> of the try-block, which isn't polite.
>
> Another solution would be to provide a module that provides access to
> sigprocmask.  Then, one could explicitly block signals during parts
> of the code where these problems could occur.  For example:
>
>   pipe = os.pipe()
>
>   try:
>     # Do stuff with the pipe.
>   finally:
>     # Block signals.
>     sigprocmask(...)
>     # Close the ends of the pipe.
>     os.close(pipe[0])
>     os.close(pipe[1])
>     # Unblock signals.
>     sigprocmask(...)
>
> This does not work either; there are still at least two race conditions.
> A signal that arrives after we have entered the finally clause, but
> before we have called sigprocmask, would still result in a failure to
> close the descriptors.  Similarly, a signal that occurs after the call
> to os.pipe, but before the try-block is entered would result in us
> skipping the finally clause.  (And moving the try earlier is not valid;
> we can't clean up the descriptors unless we know that the pipe has been
> created.)
>
> Here is another try:
>
>   # Block signals.
>   sigprocmask(...)
>
>   try:
>     pipe = os.pipe()
>
>     try:
>       # Do stuff with the pipe.
>     finally:
>       # Close the ends of the pipe.
>       os.close(pipe[0])
>       os.close(pipe[1])
>   finally:
>     # Unblock signals.
>     sigprocmask(...)
>
> This variant is robust; we are now guaranteed that if the pipe is
> created, the descriptors will be closed.
>
> However, it is still unsatisfactory in that it results in the signal
> being delayed for an arbitrary amount of time; if a time-critical
> signal happens during the try-block, we will not process it until
> much later.
>
> Another solution is to require that the code be run in a thread other
> than the main thread; since only the main thread receives signals,
> the signal-safety problem does not occur.  However, it seems excessive
> to require threads simply to write signal-safe code!  And if you have
> code that depends on, say, getting SIGPIPE, then you have to have some
> way of communicating the signal from the main thread to the thread
> running the code above.
>
> What do Python programmers that need to write truly robust code do
> about this problem?  Is there a solution that I have missed?
>
> If not, would the Powers That Be be amenable to making changes to the
> language to support signal-safety?  One possible fix would be
> except/finally clauses that blocks signals on entry.  For example,
> if there were a finally_block_signals clause, you could do:
>
>   # Block signals.
>   sigprocmask(...)
>
>   try:
>     pipe = os.pipe()
>   except:
>     # Unblock signals.
>     sigprocmask(...)
>     # Reraise the exception.
>     raise
>
>   try:
>     # Unblock signals.
>     sigprocmask(...)
>     # Do stuff with the pipe.
>   finally_block_signals:
>     # Close the pipe ends.
>     os.close(pipe[0])
>     os.close(pipe[1])
>     # Unblock signals.
>     sigprocmask(...) # There needs to be a way of getting the signal
>                      # mask on entry to the finally clause.
>
> This is not particularly pretty, but it would work.
>
> Thoughts?
>
> --
> Mark Mitchell                mark at codesourcery.com
> CodeSourcery, LLC            http://www.codesourcery.com
>