Python, threads, and signals (oh my!)

[Rick Lee]

On a related note about threads.  Does anyone have any experience on which
unix system is the fastest with threads, by supporting native thread
switching?  It seems that Linux does not have native thread switching, and
in an environment where the processor is fairly heavily loaded running
several threads, it is much slower compared to WinNT on the same hardware.

- Rick Lee

Jason Lowe wrote:

> I've been playing around with Python and threads, and I've noticed some
> odd and often unstable behavior.  In particular, on my Solaris 8 box I
> can get Python 1.5.2, 1.6, or 2.0 to core dump every time with the
> following sequence.  I've also seen this happen on Solaris 6.
>
> 1. Enter the following code into the interactive interpreter:
> --
> import threading
>
> def loopingfunc():
>   while 1: pass
>
> threading.Thread(target=loopingfunc).start()
> --
>
> 2. Send a SIGINT signal (usually Ctrl-C, your terminal settings may
>    vary).  "Keyboard Interrupt" is displayed and so far everything looks
>    fine.
>
> 3. Now simply press the <Enter> key to enter a blank line in the
>    interpreter.  For my Solaris 8 box with the GNU readline 2.2 module
>    present, this always ends up in a core dump.  It may take a while,
>    since at this point the readline signal handler is being re-entered
>    recursively until the stack overflows.
>
> Looking more into this, it appears that on Solaris, more than one
> thread is processing the signal handler installed by the readline
> module (according to truss output of the process).  Unfortunately, the
> readline signal handlers don't support being re-entered, as they have
> global data that's not protected.
>
> Now granted, signals and threads are a dangerous business.  However, it
> would be nice if the user sending SIGINT to a script wouldn't cause
> instability.  Looking at the Python module sources, I noticed that the
> signal module is somewhat "thread aware" -- it allows signal
> handlers to be installed only by the main thread.  However, I found it a
> little odd that the thread support, both in the core interpreter and in
> the thread module, had no support code for signals -- even in the
> specific thread cases like POSIX (pthreads).
>
> According to various pthread documentation, it appears the "right" way
> to handle the often volatile signal/thread mix is to mask all signals
> except in one thread (usually the main thread) so that only one thread
> will receive the signal.  Unfortunately, I couldn't find any Python
> module that would allow one to change the signal mask of a thread
> (pthread_sigmask) or even a process (sigmask, sigprocmask, etc.).  I'm
> assuming the lack of this interface has to do with portability across
> platforms.
>
> I was able to solve the problem by modifying Python/thread_pthread.h's
> PyThread_start_new_thread() to block all signals before creating the new
> thread and then restoring the signal mask after the new thread was off
> and running.  Therefore, all threads created by Python except the
> initial thread will have all signals masked with this change.  Masking
> all signals in new Python threads makes sense to me, given that the
> signal module doesn't like other threads installing handlers anyway.
>
> As a side note, I think part of the problem with sending SIGINT at the
> interactive prompt while threading is aggravated by the Python readline
> module's signal handler.  It setjmp()'s and longjmp()'s to do its dirty
> work, and many thread implementations require the longjmp() be performed
> by the thread that did the setjmp().  If signals aren't masked in all
> threads except the one doing the readline() call, then this isn't
> guaranteed.
>
> So what's everyone else's take on this?  Has anyone else experienced
> problems in Python when using threads and receiving signals (like
> SIGINT)?  Should the thread code be at least a little more "signal
> aware", so that new Python threads have all signals masked on platforms
> that support this?
>
> Thanks in advance.
>
> Jason Lowe
> --
> Jason Lowe                                    Urbana Design Center
> Motorola Personal Communications Sector       1800 South Oak Street
> jlowe at urbana.css.mot.com                      Champaign, IL  61820-6947
>                                               (217) 384-8513