[execnet-dev] Socket gateway windows service
Charles Solar
charlessolar at gmail.com
Thu Jul 8 19:03:27 CEST 2010
Well its a windows service so on any windows machine you would just do
python socketserver.py install
python socketserver.py start
to get the socketserver running on port 8888.
You can connect to it from any machine with
import execnet
gw=execnet.makegateway( "socket=[hostname]:8888" )
on the socket server it will spawn a new popen gateway and connect the
socketIO to the popenIO through IOJoiner.
IOJoiner replaces the io that the gateways on the server side use to
serve requests. So when the slave socket gateway calls io.read(1) in
the serve loop, its actually calling IOJoiner.read(). IOJoiner.read()
will then read from the socket, and write all incoming data to the
popen gateway.
However, the socket slave gateway still wants a return value.
Returning what we just read from the socket will produce incorrect
results, so I have it just return the Unserializer's None type, which
will cause the serve loop to not do anything, exactly what I wanted.
On the popen end, I write the incoming socket data to the master
gateway's io, which is then read by the slave gateway and used. Also,
all data read from the popen slave gateway is written to the socket
io, same as how socket io passes to popen io.
IOJoiner needed to use Unserializer types because I did not want the
slave socket gateway or the master popen io do anything, as their only
purpose is to facilitate communication between the other io gateways.
I have no doubt there is a better way to do this, but with my limited
understanding of execnet internals at the moment, this seems like the
easiest solution.
Charles
On Thu, Jul 8, 2010 at 11:33 AM, holger krekel <holger at merlinux.eu> wrote:
> Hi Charles,
>
> am quite busy with releasing some bits before i leave for a couple of days
> but i am wondering:
>
> a) could you provide a full example on how to use your code?
>
> b) why does the IOJoiner need to be aware of (Un)Serializer details?
>
> cheers,
> holger
>
> On Thu, Jul 08, 2010 at 10:39 -0500, Charles Solar wrote:
>> Ok I have a solution that works for my purposes. I rethought my
>> original idea and figured out how to proxy the io instances to the new
>> popengateway that socketserver creates, so no modifications to execnet
>> internals was necessary.
>> I am attaching my solution, but there are some lines I would like you
>> to look at and probably fix marked by XXX
>> For those lines I either used a trick that could break at any update,
>> or I had to assume things that I am not sure should be assumed.
>>
>> Look forward to seeing what you do with it :)
>>
>> Charles
>>
>> On Sat, Jul 3, 2010 at 3:02 AM, holger krekel <holger at merlinux.eu> wrote:
>> > Hi Charles,
>> >
>> > On Fri, Jul 02, 2010 at 16:31 -0500, Charles Solar wrote:
>> >> Ah, a few hours after sending this I found the scripts folder, and here I
>> >> thought socketserver.py was just an example.. oh well.
>> >
>> > Your code still makes sense - an installable windows service would be great.
>> >
>> >> Anyway after using this a bit I am afraid I need to modify how the
>> >> Socketgateway works and I would like a tip or two about a couple things.
>> >>
>> >> I noticed that socketgateways will run the remote code in their own world,
>> >> which is a bit different from ssh gateways where the ssh session terminates
>> >> when the job is done. The sshd remains untainted because the remote server
>> >> spawned a nice python process for us.
>> >> However the socketgateways run the remote code in themselves and thus if the
>> >> remote code dirties up the interpreter the whole daemon is bad.
>> >
>> > True.
>> >
>> >> In my case I am starting a twisted reactor on the remote server, and once a
>> >> twisted reactor has been created its expected that it is alive as long as
>> >> the python process itself. Because of this, the socket gateway daemon is
>> >> only good for 1 connection, then it crashes because twisted's reactor has
>> >> issues.
>> >>
>> >> The solution to this problem would be to execute the incoming remote code in
>> >> a new popen gateway on the remote server instead of inside the socket
>> >> gateway instance itself. I have been skimming a few files but I am not
>> >> completely sure how or where would be a good place to put the popen gateway.
>> >> My gut tells me the proxy should be put in SocketIO, but I figured it might
>> >> save some time to send out an email to find out how these systems interact.
>> >
>> > not sure, but for now i'd rather not try to nest execnet gateways,
>> > mostly because it will be fun to debug (there is some logging and
>> > nested gateways generally work though). Also, a nice property of
>> > the socketgateway server is that it's rather independent from execnet
>> > impl details.
>> >
>> > Rather, the socketgateway.py service could learn to act just like
>> > the ssh-daemon by using subprocess.Popen and allowing multiple connections.
>> > IMHO just using threads that own a socket and proxy IO to their subprocess'ed
>> > gateway could be straightforward.
>> >
>> > In any case, I am happy to review and integrate your code into the next release
>> > if you go down this route, both for the service and the socketserver issue.
>> >
>> > cheers,
>> > holger
>> >
>> >
>> >>
>> >> Thanks
>> >>
>> >> On Fri, Jul 2, 2010 at 11:31 AM, Charles Solar <charlessolar at gmail.com>wrote:
>> >>
>> >> > I wrote up a windows service script for starting a socket gateway. Thought
>> >> > other people might like to use it. Its pretty basic, but gets the job
>> >> > done. PyWin32 is required.
>> >> >
>> >> > Thanks for the great library.
>> >> >
>> >> > Charles
>> >> >
>> >
>> >> _______________________________________________
>> >> execnet-dev mailing list
>> >> execnet-dev at codespeak.net
>> >> http://codespeak.net/mailman/listinfo/execnet-dev
>> >
>> >
>> > --
>> >
>
>> """
>> Windows service for handling incomming socket gateways
>> """
>>
>> import threading, SocketServer
>>
>> import win32serviceutil
>> import win32service
>> import win32event
>> import servicemanager
>>
>> import execnet
>> from execnet.gateway_socket import SocketIO
>>
>> class IOJoiner():
>> """
>> Joins two io instances so when one wants to read, the incoming data is
>> sent straight to the other io. Useful for tieing the Popen gateway IO
>> and the SocketIO.
>> """
>> def __init__( self, primaryIO, secondaryIO ):
>> self.prim = primaryIO
>> self.sec = secondaryIO
>>
>> def read( self, numbytes ):
>> buf = self.prim.read( numbytes )
>> self.sec.write( buf )
>> # 'L' corresponds to the 'NoneType' in the Unserializer
>> # Do this so the person who called read does not freak out
>> # XXX I should return an object ACTUALLY representing NoneType, but not sure atm
>> # how to access that data.
>> return 'L'
>>
>> def write( self, data ):
>> # XXX I do not see any reason for the other io to worry
>> # About things the primary is writing, I could be wrong though.
>> self.prim.write( data )
>>
>> def close_read( self ):
>> self.prim.close_read()
>>
>> def close_write( self ):
>> self.prim.close_write()
>>
>>
>> class PythonService(win32serviceutil.ServiceFramework):
>> _svc_name_ = "PythonSocketServer"
>> _svc_display_name_ = "Python execnet socket server"
>> _svc_description_ = "Gateway server to allow execnet to connect to windows machines"
>> def __init__(self, args):
>> win32serviceutil.ServiceFramework.__init__(self, args)
>> # Create an event which we will use to wait on.
>> # The "service stop" request will set this event.
>> self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
>>
>> def SvcStop(self):
>> # Before we do anything, tell the SCM we are starting the stop process.
>> self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
>> # And set my event.
>> win32event.SetEvent(self.hWaitStop)
>>
>> def SvcDoRun(self):
>>
>> servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,servicemanager.PYS_SERVICE_STARTED,(self._svc_name_, ''))
>>
>> self.main()
>>
>> while True:
>> rc = win32event.WaitForSingleObject(self.hWaitStop, 1000)
>> if rc == win32event.WAIT_OBJECT_0:
>> self._server.shutdown()
>> self._serverThread.join()
>> servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,servicemanager.PYS_SERVICE_STOPPED,(self._svc_name_, ''))
>> break
>>
>> def main( self ):
>>
>> class TCPHandler( SocketServer.StreamRequestHandler ):
>> def handle( self ):
>> source = self.rfile.readline().rstrip()
>>
>> # Spawn a child python process for the new connection
>> # XXX It would be nice if there was a way to remotely request a new ironpython or jython instance instead of a generic cpython instance
>> # but that would require modifications to SocketGateway I believe.
>> gw = execnet.PopenGateway(python='python')
>>
>> sockio = SocketIO( self.request )
>> gwio = gw._io
>>
>> source = source.replace( "io = SocketIO(clientsock)", "io = joiner" )
>>
>> g = { 'joiner': IOJoiner( sockio, gwio ), 'address': self.client_address }
>> gw._io = IOJoiner( gwio, sockio )
>>
>> source = eval(source)
>>
>> if source:
>> co = compile( source+'\n', source, 'exec' )
>>
>> try:
>> exec co in g
>> except Exception as e:
>> servicemanager.LogErrorMsg( "Execution of received source code raised the following exception: %r" % e )
>>
>> self._server = SocketServer.ThreadingTCPServer( ('0.0.0.0', 8888), TCPHandler )
>> self._serverThread = threading.Thread( target=self._server.serve_forever )
>> self._serverThread.setDaemon( True )
>> self._serverThread.start()
>>
>> if __name__ == '__main__':
>> win32serviceutil.HandleCommandLine(PythonService)
>
>
> --
>
More information about the execnet-dev
mailing list