Bug help: CGI forking

[Robin Munn]

Jonathan Hayward <jonathan.hayward at pobox.com> wrote:
> I am trying to have a CGI script which forks a daemon the first time,
> and on subsequent times asks the daemon to do all the work. The daemon
> and the CGI script are run by the same code.
> 
> I'm trying to have a class that will handle daemon multithreading, the
> "hollow shell" CGI script starting up the daemon, and the CGI script
> talking with the daemon. I tried to use recipes from O'Reilly's
> _Python Cookbook_, but I must have messed up. The script usually
> hangs, and when it doesn't hang, the daemon sometimes isn't started. I
> think I have a fork bug, at least.
> 
> Any bugfixes and/or clarifications welcome:
> 
> class multitasking_manager(ancestor):
>     """Class to handle multithreading and multiprocessing material."""
>     def __init__(self):
>         ancestor.__init__(self)
>         thread_specific_storage = {}
>         thread_specific_storage_lock = thread.allocate_lock()
>     def check_and_start_daemon(self):
>         if not self.is_daemon_running():
>             self.start_daemon()
[snip some code]
>     def start_daemon(self):
>         try:
>             first_pid = os.fork()
>         except OSError, e:
>             log_error("Failed to make first fork for daemon. Error: "
> + \
>               e.strerror)
>             return
>         if first_pid == 0:
>             os.chdir("/")
>             os.setsid()
>             os.umask(066)
>             try:
>                 second_pid = os.fork()
>             except OSError, e:
>                 log_error("Failed to make second fork for daemon.
> Error: " + \
>                   e.strerror)
>                 return
>             if second_pid == 0:
>                 self.run_daemon()

I believe the start_daemon routine may be your problem.

First, you may want to look at this recipe in the online Python Cookbook:

    http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66012

I believe that's the recipe you're trying to follow in the paper copy;
the online version, though, includes some useful comments about things
like decoupling from stdin, stdout and stderr, which might cause
problems if you don't do it.

I'm noticing one omission here: the sys.exit(0) calls after the two
forks. I understand not putting in a sys.exit(0) after the first fork,
if you want your CGI script to continue running after forking off the
daemon, but after the second fork, the parent should close. It might
also be a good idea to split this out into two scripts:

--- begin cgiscript.py ---
def is_daemon_running():
    try:
        # Communicate with daemon via socket.
        return True
    except socket.error:
        return False

def start_daemon():
    os.system('daemon.py parameters')

def check_and_start_daemon():
    if not is_daemon_running():
        start_daemon()

def do_work():
    check_and_start_daemon()
    # Whatever else needs to happen...
---- end cgiscript.py ----


--- begin daemon.py ---
def daemonize_self():
    def start_daemon(self):
        try:
            first_pid = os.fork()
        except OSError, e:
            log_error("Failed to make first fork for daemon. Error: " + \
              e.strerror)
            return
        if first_pid > 0:
            # Exit first parent
            sys.exit(0)
        else:
            os.chdir("/")
            os.setsid()
            os.umask(066)
            try:
                second_pid = os.fork()
            except OSError, e:
                log_error("Failed to make second fork for daemon. Error: " + \
                  e.strerror)
                return
            if second_pid > 0:
                # Exit second parent
                sys.exit(0)
            else:
                # Optional but recommended: disconnect stdin, stdout, stderr
                new_stdin = file('/dev/null', 'r')
                new_stdout = file('/dev/null', 'a+')
                new_stderr = file('/dev/null', 'a+', 0)
                os.dup2(new_stdin.fileno(), sys.stdin.fileno())
                os.dup2(new_stdout.fileno(), sys.stdout.fileno())
                os.dup2(new_stderr.fileno(), sys.stderr.fileno())
                # Now we're ready
                run_daemon()

def run_daemon():
    # Insert processing here
---- end daemon.py ----

This FAQ might also be useful:

     http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16

-- 
Robin Munn <rmunn at pobox.com> | http://www.rmunn.com/ | PGP key 0x6AFB6838
-----------------------------+-----------------------+----------------------
"Remember, when it comes to commercial TV, the program is not the product.
YOU are the product, and the advertiser is the customer." - Mark W. Schumann