[issue1975] signals in thread problem

[Andriy Pylypenko]

New submission from Andriy Pylypenko:

Hello,

This issue is actually a follow up of issue 960406. The patch applied
there brought in a problem at least under FreeBSD. The problem is
extremely annoying so I made an effort to uncover the source of it.

Here is an example code that shows the problem:

    some_time = 6000000 # seconds

    class MyThread(Thread):
        def run(self):
            while (True):
                time.sleep(some_time)

    t = MyThread()
    t.start()
    while(True):
        select.select(None, None, None, some_time)

Start this script, then try to interrupt it by hitting Control-C. If you
run this code under Linux you won't see any problem. But under FreeBSD
the script won't stop until the timeout in main thread occurs or some
activity takes place on descriptors passed to the select().

My investigation showed that the source of the problem is in that how
signals are processed. FreeBSD processes signals in opposite order than
Linux. For example suppose we have a program that starts one user thread
and allows both main and user threads to receive signals. Under Linux
the signal handler always fires up in context of the main thread, but
under FreeBSD the signal handler runs in context of the user thread. 
POSIX doesn't state which behavior is correct so both behaviors should
be assumed to be correct and Python should be aware of them both. Before
the patch from 960406 the Python made effort to deny signal handling in
user threads but the patch dropped this code and all threads are allowed
to handle signals.

Let's return to the script. When running the script under Linux the
select() call is the one that gets interrupted by the signal and this
allows the script to shutdown quickly. But under FreeBSD the sleep()
call is interrupted by the signal leaving the main thread to wait on
select() until timeout.

The description of issue 960406 states:

"This is a patch which will correct the issues some people have with
python's handling of signal handling in threads. It allows any thread to
initially catch the signal mark it as triggered, allowing the main
thread to later process it."

And yes it behaves exactly as described but this behavior is
inconsistent between different OSes.

To make things predictable I've restored the code that ensures that
signal handler will run in context of main thread only:

long
PyThread_start_new_thread(void (*func)(void *), void *arg)
{
    ...
+   sigset_t set, oset;
    ...
+   sigfillset(&set);
+   SET_THREAD_SIGMASK(SIG_BLOCK, &set, &oset);
    pthread_create(...)
+   SET_THREAD_SIGMASK(SIG_SETMASK, &oset, NULL);
    ...

and this works perfectly for me under FreeBSD and Linux at least for my
needs. It doesn't bring any visible changes to readline behavior either.
I'm using the 2.5.1 version of Python. In attach you can find this patch
against the trunk.

I'm not Python guru but let me try to display my vision of the situation.

As I understand, my change does nothing for applications written in pure
Python and running under Linux (without user modules written in C and
using special thread and signal handling). Signals under Linux have
absolutely no chance to be caught from within user threads as Python
doesn't provide any way to alter the signal mask and with the default
signal mask the signals always arrive to the main thread. So explicit
prohibition to handle signals from within user thread doesn't change
anything. On the other hand this change ensures that under FreeBSD
things go exactly like under Linux.

Of course this change can possibly break some C-written module that
relies on signal handling in context of user thread (as the signal mask
of the user thread must be modified explicitly now). But anyway this is
how things are meant to work in order to be portable. So I'm considering
this possibility as highly unlikely.

I suppose the Mac OS X is affected also as it's based on FreeBSD.

----------
components: Interpreter Core
files: pthread_sig.diff
messages: 61870
nosy: bamby
severity: normal
status: open
title: signals in thread problem
type: behavior
versions: Python 2.4, Python 2.5
Added file: http://bugs.python.org/file9333/pthread_sig.diff

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue1975>
__________________________________