Shutting down a cross-platform multithreaded app

James Harris james.harris.1 at gmail.com
Fri Sep 18 13:17:27 EDT 2015


Well, this is fun ... for some definition of the word. ;-(

I have a multithreaded app that I want to be able to shut down easily 
such as by hitting control-c or sending it a signal. What follows is the 
way I have come up with given the common elements of different 
environments. Suggestions for improvement would be welcome or you may 
just find the convolutions and machinations interesting.

The first issue is that only the main thread can receive signals, 
according to

  https://docs.python.org/2/library/signal.html

It says: "the main thread will be the only one to receive signals (this 
is enforced by the Python signal module, even if the underlying thread 
implementation supports sending signals to individual threads)".

That's OK. I can get the main thread to accept suitable signals but then 
I need some way for it to tell the other threads to shut themselves down 
too. In most (probably all) cases they will be sitting waiting for 
network IO.

I could have the main thread set a value in a global variable and then 
have the sub-threads check the global in between accesses of the network 
in a polling loop (using select() with a timeout). But as has already 
been pointed out to me in the thread "Signal SIGINT ignored during 
socket.accept" such polling does not sit well with hosted OSes and it 
can keep the CPU from remaining at rest in a low-power mode.

I can, however, use select() to monitor two file descriptors. One would 
be the socket the thread is using to communicate with the client. The 
other would be a control connection from the master thread.

Now to make this cross platform.... According to the opening paragraph 
in the following link Windows select() won't work on arbitrary file 
descriptors but only works for sockets.

  https://docs.python.org/2/library/select.html

Well, that can be dealt with. I thought of using AF_UNIX or something 
else but it seems there is nothing else which could be considered 
universal and, according to the next link, if socket.AF_UNIX is not 
defined then even the Unix protocol is not supported.

  https://docs.python.org/2/library/socket.html

Needless to say, on a test Windows machine AF_UNIX is not present. The 
only cross-platform option, therefore, seems to be to use each 
subthread's select()s to monitor two AF_INET sockets: the one to the 
client and a control one from the master thread. I would seem to need IP 
socket pairs between the master thread and the subthreads. If the master 
thead receives a shutdown signal it will send a shutdown command to each 
subthread.

The above seems logical but would use quite a few IP sockets. I cannot 
think of a better way, though. Any comments on the ideas above?

James




More information about the Python-list mailing list