Python, threads, and signals (oh my!)

[Jason Lowe]

Donn Cave <donn at u.washington.edu> writes:

>I ran some experiments on other UNIX platforms, and Solaris is not
>alone if signals aren't handled well when there are multiple concurrent
>threads.  In general, signals were trapped but Python handler functions
>weren't executed.

I am also seeing unstable situations on Linux with threading and
signals, but I haven't looked into those problems as much as with the
Solaris case.

>My take is that it's complicated.  Does it make sense to split out
>readline's special problems as an issue that could be addressed
>separately?  In my experiments, without readline, there were still
>these issues, and even without readline, signal handling at the
>interpreter prompt differed from an interpreted disk file.

I don't think the readline module issue with threads is separate,
although there might be ways to make the readline module more robust
in the presence of threading without addressing the whole
threads/signals issue.  Specifically, the readline module is using
longjmp() to get back into the call_readline() context.  If the wrong
thread handles the SIGINT signal and performs the longjmp(), we're
hosed on many thread implementations.

>I can't think of any reason why it would hurt anything to block
>signals by default per non-main thread.  I don't think it will
>work for many platforms, but if it works for anyone's I guess
>that's better than nothing.  (And the one I would be most concerned
>about, BeOS, doesn't seem to need it, as signals seem to work more
>or less as intended in multithreaded programs there.)

Agreed.  The POSIX thread books I've seen all indicate that you want
this setup anyway.  It appears the only way to safely handle signals and 
threads is to make sure all asynchronous signals are delivered to only
one thread.  They even go so far as to recommend a separate thread whose
only purpose is to catch signals synchronously with sigwait().  The
signal-handling thread can then dispatch work to other threads using
condition variables, mutexes, and other thread
communication/synchronization primitives.

>I think the ability to block signals in general should be added,
>in the form of the POSIX 1003.1 sigprocmask function.  It would
>be easy enough to do, if anyone cared enough.  Is that how you
>masked off the signal from the new thread - do new pthreads copy
>their signal handling from the parent at creation?

I didn't use sigprocmask(), as I'm not sure that's technically portable
in a POSIX thread environment.  Instead, I used pthread_sigmask(),
although I suspect sigprocmask() == pthread_sigmask() when threads are
used on most platforms.

New pthreads do indeed inherit their signal mask from the thread that
creates them.  I masked off all signals before creating the new thread,
then restored them after the new thread was created.  Therefore, if the
main thread creates the new thread, the main thread will have its
original signal mask restored but the new thread will have all signals
blocked.  If another Python thread creates a thread, all signals for
both threads will be blocked.

The ability to control signal masks from Python would be great, although 
I suspect this has portability concerns leading to its current
omission from the standard library. If signal mask controls aren't 
added to the Python standard library, it would be nice if Python could
create new threads with all signals masked on platforms where this is 
appropriate, so signals are only handled by one thread (the main thread
or a signal-handling thread).  This would be desirable on any platform
that uses POSIX threads, in accordance with the POSIX thread
documentation.

Thoughts?

Jason Lowe
--
Jason Lowe                                    Urbana Design Center
Motorola Personal Communications Sector       1800 South Oak Street
jlowe at urbana.css.mot.com                      Champaign, IL  61820-6947
                                              (217) 384-8513