Daemonization / Popen / pipe issue

[John O'Hagan]

On Fri, 16 Mar 2012 22:12:14 -0400
Lee Clemens <java at leeclemens.net> wrote:


> 
> I have a multi-threaded application, each thread has an instance of a class
> which calls Popen. The command(s) being executed (shell=True) include pipes.
> The errors I have seen involve "broken pipe" and unexepected output (as
> demonstrated in the test case).
> 
> This issues only seem to occur when the application is "daemonized", using a
> double-fork and os.dup2, as shown here:
> http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/
> 
> I have provided a test-case here: https://gist.github.com/2054194
> 
> Please test how flipping DAEMONIZE between True and False yield different
> results.


I haven't looked at your test case yet, but a possible cause is the fact that
the main application (or main thread) exits if it has finished executing and
only daemon threads remain. This can  abruptly terminate threads which may be
busy, for example, communicating via a pipe. The best solution IMO is not to
use daemon threads, but to give all threads a way to terminate cleanly before
the main thread does, even if this means using a flag or the like.

HTH,

John