[execnet-dev] Socket gateway windows service
holger krekel
holger at merlinux.eu
Thu Jul 8 18:33:58 CEST 2010
Hi Charles,
am quite busy with releasing some bits before i leave for a couple of days
but i am wondering:
a) could you provide a full example on how to use your code?
b) why does the IOJoiner need to be aware of (Un)Serializer details?
cheers,
holger
On Thu, Jul 08, 2010 at 10:39 -0500, Charles Solar wrote:
> Ok I have a solution that works for my purposes. I rethought my
> original idea and figured out how to proxy the io instances to the new
> popengateway that socketserver creates, so no modifications to execnet
> internals was necessary.
> I am attaching my solution, but there are some lines I would like you
> to look at and probably fix marked by XXX
> For those lines I either used a trick that could break at any update,
> or I had to assume things that I am not sure should be assumed.
>
> Look forward to seeing what you do with it :)
>
> Charles
>
> On Sat, Jul 3, 2010 at 3:02 AM, holger krekel <holger at merlinux.eu> wrote:
> > Hi Charles,
> >
> > On Fri, Jul 02, 2010 at 16:31 -0500, Charles Solar wrote:
> >> Ah, a few hours after sending this I found the scripts folder, and here I
> >> thought socketserver.py was just an example.. oh well.
> >
> > Your code still makes sense - an installable windows service would be great.
> >
> >> Anyway after using this a bit I am afraid I need to modify how the
> >> Socketgateway works and I would like a tip or two about a couple things.
> >>
> >> I noticed that socketgateways will run the remote code in their own world,
> >> which is a bit different from ssh gateways where the ssh session terminates
> >> when the job is done. The sshd remains untainted because the remote server
> >> spawned a nice python process for us.
> >> However the socketgateways run the remote code in themselves and thus if the
> >> remote code dirties up the interpreter the whole daemon is bad.
> >
> > True.
> >
> >> In my case I am starting a twisted reactor on the remote server, and once a
> >> twisted reactor has been created its expected that it is alive as long as
> >> the python process itself. Because of this, the socket gateway daemon is
> >> only good for 1 connection, then it crashes because twisted's reactor has
> >> issues.
> >>
> >> The solution to this problem would be to execute the incoming remote code in
> >> a new popen gateway on the remote server instead of inside the socket
> >> gateway instance itself. I have been skimming a few files but I am not
> >> completely sure how or where would be a good place to put the popen gateway.
> >> My gut tells me the proxy should be put in SocketIO, but I figured it might
> >> save some time to send out an email to find out how these systems interact.
> >
> > not sure, but for now i'd rather not try to nest execnet gateways,
> > mostly because it will be fun to debug (there is some logging and
> > nested gateways generally work though). Also, a nice property of
> > the socketgateway server is that it's rather independent from execnet
> > impl details.
> >
> > Rather, the socketgateway.py service could learn to act just like
> > the ssh-daemon by using subprocess.Popen and allowing multiple connections.
> > IMHO just using threads that own a socket and proxy IO to their subprocess'ed
> > gateway could be straightforward.
> >
> > In any case, I am happy to review and integrate your code into the next release
> > if you go down this route, both for the service and the socketserver issue.
> >
> > cheers,
> > holger
> >
> >
> >>
> >> Thanks
> >>
> >> On Fri, Jul 2, 2010 at 11:31 AM, Charles Solar <charlessolar at gmail.com>wrote:
> >>
> >> > I wrote up a windows service script for starting a socket gateway. Thought
> >> > other people might like to use it. Its pretty basic, but gets the job
> >> > done. PyWin32 is required.
> >> >
> >> > Thanks for the great library.
> >> >
> >> > Charles
> >> >
> >
> >> _______________________________________________
> >> execnet-dev mailing list
> >> execnet-dev at codespeak.net
> >> http://codespeak.net/mailman/listinfo/execnet-dev
> >
> >
> > --
> >
> """
> Windows service for handling incomming socket gateways
> """
>
> import threading, SocketServer
>
> import win32serviceutil
> import win32service
> import win32event
> import servicemanager
>
> import execnet
> from execnet.gateway_socket import SocketIO
>
> class IOJoiner():
> """
> Joins two io instances so when one wants to read, the incoming data is
> sent straight to the other io. Useful for tieing the Popen gateway IO
> and the SocketIO.
> """
> def __init__( self, primaryIO, secondaryIO ):
> self.prim = primaryIO
> self.sec = secondaryIO
>
> def read( self, numbytes ):
> buf = self.prim.read( numbytes )
> self.sec.write( buf )
> # 'L' corresponds to the 'NoneType' in the Unserializer
> # Do this so the person who called read does not freak out
> # XXX I should return an object ACTUALLY representing NoneType, but not sure atm
> # how to access that data.
> return 'L'
>
> def write( self, data ):
> # XXX I do not see any reason for the other io to worry
> # About things the primary is writing, I could be wrong though.
> self.prim.write( data )
>
> def close_read( self ):
> self.prim.close_read()
>
> def close_write( self ):
> self.prim.close_write()
>
>
> class PythonService(win32serviceutil.ServiceFramework):
> _svc_name_ = "PythonSocketServer"
> _svc_display_name_ = "Python execnet socket server"
> _svc_description_ = "Gateway server to allow execnet to connect to windows machines"
> def __init__(self, args):
> win32serviceutil.ServiceFramework.__init__(self, args)
> # Create an event which we will use to wait on.
> # The "service stop" request will set this event.
> self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
>
> def SvcStop(self):
> # Before we do anything, tell the SCM we are starting the stop process.
> self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
> # And set my event.
> win32event.SetEvent(self.hWaitStop)
>
> def SvcDoRun(self):
>
> servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,servicemanager.PYS_SERVICE_STARTED,(self._svc_name_, ''))
>
> self.main()
>
> while True:
> rc = win32event.WaitForSingleObject(self.hWaitStop, 1000)
> if rc == win32event.WAIT_OBJECT_0:
> self._server.shutdown()
> self._serverThread.join()
> servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,servicemanager.PYS_SERVICE_STOPPED,(self._svc_name_, ''))
> break
>
> def main( self ):
>
> class TCPHandler( SocketServer.StreamRequestHandler ):
> def handle( self ):
> source = self.rfile.readline().rstrip()
>
> # Spawn a child python process for the new connection
> # XXX It would be nice if there was a way to remotely request a new ironpython or jython instance instead of a generic cpython instance
> # but that would require modifications to SocketGateway I believe.
> gw = execnet.PopenGateway(python='python')
>
> sockio = SocketIO( self.request )
> gwio = gw._io
>
> source = source.replace( "io = SocketIO(clientsock)", "io = joiner" )
>
> g = { 'joiner': IOJoiner( sockio, gwio ), 'address': self.client_address }
> gw._io = IOJoiner( gwio, sockio )
>
> source = eval(source)
>
> if source:
> co = compile( source+'\n', source, 'exec' )
>
> try:
> exec co in g
> except Exception as e:
> servicemanager.LogErrorMsg( "Execution of received source code raised the following exception: %r" % e )
>
> self._server = SocketServer.ThreadingTCPServer( ('0.0.0.0', 8888), TCPHandler )
> self._serverThread = threading.Thread( target=self._server.serve_forever )
> self._serverThread.setDaemon( True )
> self._serverThread.start()
>
> if __name__ == '__main__':
> win32serviceutil.HandleCommandLine(PythonService)
--
More information about the execnet-dev
mailing list