[Web-SIG] daemon tools

[Jim Fulton]

On Mar 5, 2007, at 1:38 PM, Robert Brewer wrote:
...
> What several people have asked for is the ability to combine
> applications (and WSGI components) from a variety of frameworks into a
> single "website". What I'm proposing is that we standardize on a  
> set of
> topics/channels/events/signals that are "site-wide" events, like  
> start,
> stop, restart and graceful. If we collaborated on a tool to manage
> those, we could potentially make the codebases of each project  
> smaller,
> not just by removing the event manager, but by collaborating on a  
> set of
> standard event handlers, one of which could be a "daemonize me"  
> handler.

Agreed.

>
> What we have now:
>
>     CherryPy              Zope              Django
>     --------             ------             -------
>       ???                events             signals
>        |                    |                  |
>     autoreload             ???             autoreload
>        |                    |                  |
>     engine                zdrun               ???
>        |                    |                  |
>       ???                 zdctl               ???
>
> What we could have instead:
>
>                       webctl     modpython_gateway
>                          |           /
>          ------------ pywebd ------------
>         /                |               \
>     --------          ------           ------
>     CherryPy           Zope            Django
>
>
> ...where the "pywebd" module:
>
>  1. Composes the WSGI stack (provides a library to do so at least),
>  2. Notifies frameworks of site-wide events (like start, stop, restart
> and graceful),
>  3. Provides plugins that frameworks can "notify"; for example, adding
> files to an autoreload plugin.

This sounds great to me.

>> I think your "sitewide container" is the main program that loads
>> the WSGI components.  This might be Apache, if mod_python is
>> used, or some Python script/program.
>
> Apache itself is not going to be the chunk of code that loads the WSGI
> components. In my head, a modpython_gateway module (or something
> similar) would ask pywebd to do that.

Right.

>> I was discussing a tool that managed the main program in the
>> latter case. Something that started and restarted it, provided
>> status information, helped it to run as a proper daemon and so on.
>
> Sure, something like zdctl? But zdctl doesn't do the actual fork,  
> zdrun
> does...so what does "help run as a proper daemon" mean?

(zdrun is really an internal implementation detail of zdaemon.  The  
latest version of zdaemon hides this much more than earlier versions. )

Logically, zdctl runs zdrun, which forks and execs the application  
process. (In the latest version, there is just one script, zdaemon,  
that loads either the zdctl or zdrun entry point when it is run.)  
zdrun does the deamonizing steps:

  - disconnecting from the controlling terminal, and

  - changing to a different user if requested

before forking and execing the application.

I see a division of responsibilities between:

- A facility for managing an application process

   - start/stop/status/etc

   - passing environment variables, providing some logging support if  
necessary (especially for applications that spew to standard err/out).

   - Optionally providing other daemon behaviors like disconnecting  
from the controlling terminal, changing user, etc.  zdaemon provides  
this service on behalf of applications.

- A main program that provides common application-level services like  
the ones you describe above.

   - Optionally providing other daemon behaviors like disconnecting  
from the controlling terminal, changing user, etc.  ll.deamon  
provides some of these services within an application.

A question is whether to provide the daemonizing support in the main  
program or in the controlling program.  Note that in answering this  
question, we probably need to have an idea how this will work on  
windows.  If Unix-specific daemonizing code is in the main  
application, then the application won't be portable. Of course, if  
the main program is generic, it might not be a big deal to have  
separate versions for Windows and Unix.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org