server bootstrapping upon connection (WARNING: LONG)

[ralf at brainbot.com]

fortepianissimo at yahoo.com.tw (Fortepianissimo) writes:

> Here is the situation: I want my server started up upon connection.
> When the first connection comes in, the server is not running. The
> client realizes the fact, and then starts up the server and tries to
> connect again. This of course all happens on the same machine (local
> connection only).
>
> The connections can come in as fast as 30+/sec, so the server is
> threaded (using SocketServer.ThreadingTCPServer). Further, the server
> initialization is threaded into two: one is to do the real, lengthy
> setup, the other is start lisenting ASAP.
>
> The problem: I need to prevent multiple copies of the server being
> started. I did this by using file locking (fcntl.lockf()). However,
> not every time the code successfully prevented the server being
> started up more than one time. Here is the relevant code:

When using fcntl.lockf different FooClient instances in the same
process will be able lock the file and start another server. You could
either use fcntl.flock to prevent that or use some global flag.
Also be sure to keep the file open by keeping a reference to it
(i.e. self.serverStartLock=open(...)). 
For debugging purposes, remove that 'if self.connect(): return' and
I think you'll see much more servers being started.

>
> ---- CLIENT CODE SNIPPET STARTS ----
> from fooConsts import *
>
> import socket,fcntl,os,sys,time,logging
>
> FOO_CLIENT_DEBUG=1
>
> POLL_SERVER_INTVL=0.5
>
> if FOO_CLIENT_DEBUG:
>     myPID=os.getpid()
>     logger = logging.getLogger('FOO Client')
>     hdlr = logging.FileHandler(TMP_PATH+'fooClient.log')
>     formatter = logging.Formatter('%(asctime)s %(message)s')
>     hdlr.setFormatter(formatter)
>     logger.addHandler(hdlr) 
>     logger.setLevel(logging.INFO)
>
>     def log (info):
>         logger.info('%d: %s'%(myPID,info))
>
>
> class FooClient:
>     def __init__ (self, startServer=True):
>         """Connects to FooServer if it exists, otherwise starts it and
> connects to it"""
>         self.connected=True
>         if self.connect(): return
>         elif not startServer:
>             if FOO_CLIENT_DEBUG: log('connection failed 1')
>             self.connected=False
>             return
>
>         # try to obtain the right to start a server; if we can't,
> someone else
>         # must be doing it - try to reconnect
>         try:
>             if FOO_CLIENT_DEBUG: log('try to get right to start
> server')
>             serverStartLock=open(TMP_PATH+'serverStart.lock','w')
>             fcntl.lockf(serverStartLock.fileno(),fcntl.LOCK_EX|fcntl.LOCK_NB)
>         except:
>             if FOO_CLIENT_DEBUG: log('someone else is doing it; wait
> 2')
>             while 1:
>                 time.sleep(POLL_SERVER_INTVL)
>                 if self.connect(): return
>
>         # safe to start a server and connect to it
>         if FOO_CLIENT_DEBUG: log('start server')
>         exitCode=os.system('python -OO "%sfooServer.py" &'%FOO_PATH)
>         if exitCode==0:
>             if FOO_CLIENT_DEBUG: log('server is being started; wait
> 3')
>             while 1:
>                 time.sleep(POLL_SERVER_INTVL)
>                 if self.connect(): return
>         else:
>             if FOO_CLIENT_DEBUG: log('sever bootstrapping failed')
>             self.connected=False
>             raise "Cannot start FOOServer"
>     
>     def connect (self):
>         """Attempts to connect to FOOServer from PORT_MIN to
> PORT_MAX"""
>         if FOO_CLIENT_DEBUG: log('connection attempt')
>         port=PORT_MIN
>         while port<=PORT_MAX:
>             try:
>                 self.socket=socket.socket(socket.AF_INET,
> socket.SOCK_STREAM)
>                 self.socket.connect(('',port))
>                 break
>             except:
>                 self.socket.close()
>                 port+=1
>
>         if port>PORT_MAX:
>             if FOO_CLIENT_DEBUG: log('connection failed 2')
>             return False
>         if FOO_CLIENT_DEBUG: log('connection succeeded at port
> %d'%port)
>         return True
> ...
>
> FooClient()
> ---- CLIENT CODE SNIPPET ENDS ----
>
> From the log (when problem occurred) I see even *AFTER* the server was
> started and accepted connections (several connections came and went
> happily), a connection would come in and hit the "connection failed 1"
> log line. This shouldn't have happened as the default value of
> startServer for FooClient.__init__() is True. In the very same

Well, maybe too many connection attempts are pending...

> incident, FooServer was started twice, but in the log there's no trace
> of two "start server" lines, but only the "connection failed 1" line
> in the very end of the log.
>
> I realize this is a rather complex question to ask for help, but I'm
> really at wits end. Feel free to skip the detailed description, and
> just throw me some general tips and hints. Thank you VERY MUCH!

-- 
brainbot technologies ag
boppstrasse 64 . 55118 mainz . germany
fon +49 6131 211639-1 . fax +49 6131 211639-2
http://brainbot.com/  mailto:ralf at brainbot.com