[Python-Dev] Questions about signal handling.

[Eric Snow]

On Mon, Sep 24, 2018 at 11:14 AM Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> On Fri, Sep 21, 2018 at 7:04 PM Eric Snow <ericsnowcurrently at gmail.com> wrote:
> > 1. Why do we restrict calls to signal.signal() to the main thread?
> > 2. Why must signal handlers run in the main thread?
> > 3. Why does signal handling operate via the "pending calls" machinery
> > and not distinctly?
>
> Here's my take on this:
>
> Handling signals in a multi-threaded program is hard. Some signals can
> be delivered to an arbitrary thread, some to the one that caused them.
> Posix provides lots of mechanisms to tune how signals are received (or
> blocked) by individual threads, but (a) Python doesn't expose those
> APIs, (b) using those APIs correctly is insanely hard.  By restricting
> that we can only receive signals in the main thread we remove all that
> complexity.

Just to be clear, I'm *not* suggesting that we allow folks to specify
in which Python (or kernel) thread a Python-level signal handler is
called.

The reason I've asked about signals is because of the semantics under
subinterpreters (where there is no "main" thread).  However, I don't
have any plans to introduce per-interpreter signal handlers.  Mostly I
want to understand about the "main" thread restriction for the
possible impact on subinterpreters.

FWIW, I'm also mildly curious about the value of the "main" thread
restriction currently.  From what I can tell the restriction was made
early on and there are hints in the C code that it's no longer needed.
I suspect we still have the restriction solely because no one has
bothered to change it.  However, I wasn't sure so I figured I'd ask.
:)

> Restricting that signal.signal() can only be called from
> the main thread just makes this API more consistent

Yeah, that's what I thought.

> (and also IIRC
> avoids weird sigaction() behaviour when it is called from different
> threads within one program).

Is there a good place where this weirdness is documented?

> Next, you can only call reentrant functions in your signal handlers.
> For instance, printf() function isn't safe to use.  Therefore one
> common practice is to set a flag that a signal was received and check
> it later (exactly what we do with the pending calls machinery).

We don't actually use the pending calls machinery for signals though.
The only thing we do is *always* call PyErr_CheckSignals() before
making any pending calls.  Wouldn't it be equivalent if we called
PyErr_CheckSignals() at the beginning of the eval loop right before
calling Py_MakePendingCalls()?

This matters to me because I'd like to use "pending" calls for
subinterpreters, which means dealing with signals *in*
Py_MakePendingCalls() is problematic.  Pulling the
PyErr_CheckSignals() call out would eliminate that problem.

> Therefore, IMO, the current way we handle signals in Python is the
> safest, most predictable, and most cross-platform option there is.
> And changing how Python signals API works with threads in any way will
> actually break the world.

I agree that the way we deal with signals (i.e. set a flag that is
later handled in PyErr_CheckSignals(), protected by the GIL) shouldn't
change.  My original 3 questions do not relate to that.  Looks like
that wasn't terribly clear. :)

-eric