making a Windows NT service out of a python program

[Bill Tutt]

> From: Christian Tismer [mailto:tismer at appliedbiometrics.com]
> 
> 
> This is all great stuff if you have understood it already.
> For me, it is still unclear what makes up a service, and
> what I need at the minimum to turn a Python program into
> an NT service.
> 

Not knowing ANYTHING about Medusa... I'll provide another example service
right here, and try and explain things as I go... 

The Win32 support for services is encompassed by 3 major pieces.
* win32service.pyd
This provides the standard "Glue Win32 API functions to Python" code.
* PythonService.exe
NT services are executables that must follow special rules. 
PythonService.exe follows these special rules. 
It also exports Python bindings to the "servicemanager" module.
This module exports the following methods:
	* CoInitializeEx (COM initialization)
	* CoUnitialize (COM unitilialization)
	* RegisterServiceCtrlHandler 
	  (This is the lynchpin function of getting 
         a Python class to run as a service.)
	* LogMsg, LogInfoMsg, LogErrorMsg, LogWarningMsg
	   (These all log messages to the eventlog, and mark them as
		coming from the "Python Service Manager") To make eventlog
messages
        appear to come from your own application you'll need to use Python's
        other eventlog support.
	* PumpWaitingMessages
	   (Dispatches any incoming window messages, 
	    and returns 1 if WM_QUIT was recieve, otherwise returns 0.)
	   Most services shouldn't need this, I take it Mark stuck it here
for
       some wierd service he wrote that actually needs access to the
desktop.	* Debugging
	   (This returns 1 if the service is running in debug mode.
          More on this later....)
* win32serviceutil.py
	This sucker provides the following sub-groups of useful code:
	* ServiceFramework
  	    This is just a useful base class to build services from.
	* Provides a handly command line handling function that handles:
	    Installing, Removing, Updating, Starting, Stoping, Restarting,
	    pausing, setting the user the service logs on as, specifying
        auto, manual, or disabled startup policy, and something about 
	    perfmon stuff. 
	* Handy wrapper functions to the ugly API interfaces in
win32service.pyd.
	    (See QueryServiceStatus for a good example.)
	* Handy utility functions for storing configuration data in the 
	  registry.
	    (SetServiceCustomOption, and GetServiceCustomOption)

Ok, so those are the pieces, but Bill, how does all of this nonsense fit
together? Good question. :)

NT Services are managed by the SCM (Service Control Manager).
The SCM is what manages almost everything about services...
Installing, removing, starting, stopping, etc...
The SCM is also in charge of dealing with the service interdependancy chain.
Classic example:
  IIS is composed of multiple services: iisadmin, w3svc, ftpsvc, etc....
  In order for w3svc, and ftpsvc to run correctly iisadmin must already
  have been started. The SCM handles all of this detail for us.

An NT service must have special entry points inorder to get along with
the SCM nicely. All of these entry points for most Python services
should just be in PythonService.exe to make your life simple.

* main: Most NT services are console applications, so this is where things
start.
	PythonService.exe also has some code to handle command line
arguments.  i.e. register, and debug. register handles keeping track
of some python/eventlog bookkeeping it needs internally. After
handling any command line arguments, main calls the NT API
StartServiceCtrlDispatcher function, which connects the main thread of
PythonService.exe to the SCM. The main thread is the thread on which
the service management code gets called on.

* ServiceMain: This what gets passed into StartServiceCtrlDispatcher
	As soon as this gets called, we call RegisterServiceCtrlHandler()
to let NT know what C function of ours is going to handle dealing with
notifications from the SCM. We then tell NT that we're in the process
of starting up, chill out and wait a bit while we initliaze ourselves,
and suggest to NT that 5 seconds might be a nice length to wait for us
to finish starting up before complaining too loudly.
	ServiceMain then calls the Python class that you've designated's
"SvcRun" method. "SvcRun" must not return until the service is ready
to terminate. The default "SvcRun" in win32serviceutil.py tells NT
that the service is up and running, calls your definition of
"SvcDoRun".

* ServiceCtrlHandler:
	This handles communications between the SCM and the rest of your
code. This function gets called by the SCM when it wants you do
something, or wants to inform you of something. Examples: start, stop,
pause/continue, machine shutdown, update your status, power event
(Win2k only), etc....

NOTE: ServiceCtrlHandler gets called on the orignal main thread of your
process. SvcRun is actually running on a seperate thread, so you must use
some kind of synchronization object between your registered
ServiceCtrlHandler function, and your "SvcDoRun" function. An example of
this is in the attached TCPServerService class.

Hopefully this helps explain an overview of how NT services need to work.
Now, lets get into the nitty gritty of this example service I cooked up.

The necessary files in this little example are:
* edna.py: Greg Stein's Edna
    A simple streaming MP3 server, and a subclass of BaseHTTPServer) (only
a simple change to handle how configuration is handled was all that
was necessary....)  See http://www.lyra.org/greg/edna for more
details.

* TCPServerService.py: This contains two useful classes you need to inherit
from.
	* TCPServerService:
      This provides all the hairy glue of meshing the Python service
framework, and SocketServer.TCPServer subclasses together.

	* SvcTrackingThreadMixin:
      This is almost identical to the normal threading mixin, except
that it keeps track of thread handles, so shutdown can be postponed
until all of our pending requests are done.

* ednaNTSvc.py:
    This is all that is necessary to fill in the rest of the details
to enable an NT service. All this module does is implement policy for
when the service should shut down with pending requests, and handle
installation/configuration issues.

* win32eventmodule_win32.cpp:
    It turned out that the Win32 extensions were missing one API
function. WSAEventSelect(), so its been added to the win32event
module. WSAEventSelect allows you to do a WaitForMultipleObjects on
sockets which is nice sine you can't use select() on NT events. :)

How to setup this cool example so that you can try it out.
1. Alter edna.conf to taste.
2. Do the following command: python ednaNTSvc.py -c ...\edna.conf install
    This saves the location of the configuration file in the registry, and
installs the service, and marks it as not starting automatically.  To
make it start automatically when your machine starts add "--startup
auto" before "install" in the command line.

3. Run it.
You can run it more than one way.
	* ednaNTSvc.py start
		This kicks off the service, and then ednaNTSvc.py exits.

	* ednaNTSvc.py debug
		This kicks off PythonService.exe passing in -debug on the
command line, this has the nice effect of being able to see the output
of stdout/stderr in your console window, and being interuptible via
Ctrl-C.

	* net start edna (or any other way to start an NT service)
		This just let NT do its thing.

Since there are so many files that are involved, you can pick up the
whole lot of them, including an updated win32event.pyd, at:
http://lima.mudlib.org/~rassilon/ednaSvc.zip

Mark:
The .pyd hasn't changed, but the rest of the files have been
excessively commented. Feel free to drop this note & the
TCPServerService

The rest of the code is heavily commented, so it should be fairly
simple to see whats going on, if not scream and yell with specific
questions/comments and I will endeavor to answer... :)

I hope this helps,
Bill