python-daemon interaction with multiprocessing (secure-smtpd)

[Grant Edwards]

On 2014-05-07, Grant Edwards <invalid at invalid.invalid> wrote:

> With Python 2.7.5, I'm trying to use the python-daemon 1.6 and its
> DaemonRunner helper with the seucre-smtpd 1.1.9 which appears to use
> multiprocessing and a process pool under the covers.  There seem to be
> a couple process issues:
>
>  1) The pid file created by DaemonRunner dissappears. [...]
>
>  2) When DaemonRunner kills the "lead" process (the parent of the
>     worker pool), the worker pools stays alive and continues to handle
>     accept and handle requests. [...]

I've tracked both these problems down to a single bug in secure_smtpd.

The secure_smtpd server is a normal asyncore server until the first
connection arrives.  At that point, it creates a bunch of
multiprocessor worker processes _without_ the daemon flag which all
loop doing blocking accept() calls.  The parent process then shuts
down the asyncore server and returns.

When the parent process returns, DaemonRunner removes the pidfile,
since it thinks the server has terminated.  However, the parent
process never actually dies -- it just hangs until all the children
terminate.

At this point, killing the parent process (whose PID _was_ in the
pidfile, and is now idle) does nothing: even though the parent process
gets killed, the worker processes keeps working.

The fix is to 

 1) Change secure_smtpd to create the worker processes with daemon=True

 2) After asyncore.loop() returns (which means the children have been
    created), do a while 1: time.sleep(1) to wait for SIGTERM.

-- 
Grant Edwards               grant.b.edwards        Yow! Maybe we could paint
                                  at               GOLDIE HAWN a rich PRUSSIAN
                              gmail.com            BLUE --