From matdrapeau at gmail.com  Mon Aug  2 21:40:40 2010
From: matdrapeau at gmail.com (Mathieu Drapeau)
Date: Mon, 2 Aug 2010 15:40:40 -0400
Subject: [execnet-dev] how to remotely open a program on win box?
In-Reply-To: <AANLkTi=Sj1Z+SpYbJPjLoZ+rc=xfaza_MbGH_NkyaiN7@mail.gmail.com>
References: <AANLkTinMDtns6Ta46_ZLKKHtbTMNRa_ue6HZSwbSeoOY@mail.gmail.com>
	<AANLkTimr=Fc0KgcthpX-TYh162t21FY15X8FSyvkzbJ7@mail.gmail.com>
	<AANLkTin=rHc5HooJ638+FvhbP8FDAYcoPV8d+wNu7AE7@mail.gmail.com>
	<AANLkTimVKELwEpo7tRtfwc44eYiGfiO-wkAaUpwruUie@mail.gmail.com>
	<AANLkTinE8ReX_6Y7eyjRQV6Dp0NuZdvz3cbKk9_qEqRW@mail.gmail.com>
	<AANLkTi=ncA5oBt3xQ+sPo_OrQ7jZBe5ymJGoX-KbjwHs@mail.gmail.com>
	<AANLkTim6PzY2JwCLbKr2CwcBKET=si-BA1F0vrAU9YMQ@mail.gmail.com>
	<AANLkTikXOw-2NWOkAWjf1fjeeB0==qeQapZtfc7ng_JQ@mail.gmail.com>
	<AANLkTikgJm_dVwpuvwW7KV8PkYpbdyMqsA-EmNixeBbX@mail.gmail.com>
	<AANLkTim+17+nnv2_fS1sKV1k_DntVjhmxW9=qswVr1UC@mail.gmail.com>
	<AANLkTi=Sj1Z+SpYbJPjLoZ+rc=xfaza_MbGH_NkyaiN7@mail.gmail.com>
Message-ID: <AANLkTinCRzG7_v_=OrH0RmeE+X3rUbZFV83am3rgm6kS@mail.gmail.com>

Hi Charles,
after some modifications to the script it appears that what is causing the
problem is really running the script as a service!
When I commented what you suggested, the server is working correctly, it
doesn't jam at the "channel.receive" call.

I am not familiar enough with Windows to find what could cause this error.
If you have any idea where I could look, please tell me.

Thanks,
Mathieu

2010/7/31 Charles Solar <charlessolar at gmail.com>

> I only vaguely remember trying to use the event manager as a debugging
> tools with little to no success.  I would strongly recommend debugging
> in a console window instead of a service instance.
> Source is a string, but printing it out will not help you very much,
> it just contains all the gateway bootstrapping code.  Actually, I bet
> you are getting that error because the source string is to large.
>
> To remove the windows service part from that file, what you need to do
> is comment out all the __init__ code in the service class, and under
> if __name__ == "-_main__" replace process args with something like
> if __name__ == "__main__":
>     svr=ServiceClass( None )
>     svr.main()
>    while True:
>        time.sleep(0.1)
>
> then you can put print statements where ever you want and they will
> print to the console window you started the daemon in.
>
> On Fri, Jul 30, 2010 at 11:07 AM, Mathieu Drapeau <matdrapeau at gmail.com>
> wrote:
> > Hi Charles,
> > I don't think it is the makegateway call that causes problems, I did try
> > bunch of different configuration without any help...
> > But the "source" variable seems weird. The way I understand it, it should
> be
> > a string and I should be able to output in the logs but everytime I try
> to
> > print its value, I get an error like this:
> >
> > "RegisterEventSource/ReportEvent", the parameter is incorrect
> >
> > Here is how I am trying to output the value of source:
> >
> > try:
> >     servicemanager.LogErrorMsg("source = %s" % source)
> > except Exception as e:
> >     servicemanager.LogErrorMsg("error = %s" % e)
> >
> > Any idea?
> >
> > Thanks,
> > Mathieu
> >
> > 2010/7/29 Charles Solar <charlessolar at gmail.com>
> >>
> >> When I was debugging that script what I did was modify it so it was
> >> not a service, but just a little daemon that ran in a console.  Then I
> >> could put print commands in there and find out what was going on.
> >> Something I think you should try is instead of adding the
> >> "popen//python=", just add the python directory to your computers path
> >> and remove that line.  I dont know, but maybe running makegateway like
> >> that is opening a new python interactively, which would probably break
> >> the popen io connection.
> >> If that is all you changed then that is really all I can guess at what
> >> the problem might be.  If you add some print statements and find out
> >> what paths the service is taking when it fails I might be able to help
> >> some more.
> >>
> >> Charles
> >>
> >> On Thu, Jul 29, 2010 at 6:36 PM, Mathieu Drapeau <matdrapeau at gmail.com>
> >> wrote:
> >> > Yes I am currently using the other script and everything works fine.
> >> > Do you know how I can troubleshoot / debug your script?
> >> >
> >> > Thanks,
> >> > Mathieu
> >> >
> >> > 2010/7/29 Charles Solar <charlessolar at gmail.com>
> >> >>
> >> >> mm, well having a python window pop up is a bit suspicious.  Maybe
> the
> >> >> popen io is not connecting correctly.
> >> >> Maybe before trying the advanced process spawning service I wrote up,
> >> >> try the basic 1 python instance service that is in execnet's script
> >> >> directory.
> >> >>
> >> >> Charles
> >> >>
> >> >>
> >> >> On Thu, Jul 29, 2010 at 4:18 PM, Mathieu Drapeau <
> matdrapeau at gmail.com>
> >> >> wrote:
> >> >> > Hi Charles,
> >> >> >
> >> >> > In the source of your server script, I did change the makegateway
> >> >> > definition
> >> >> > to this:
> >> >> >
> >> >> > gw = execnet.makegateway("popen//pytho=C:\Python26\python.exe")
> >> >> >
> >> >> > When I connect to the server, a python box pop and when I try to
> send
> >> >> > basic
> >> >> > command to the server, I never receive any answers. The client just
> >> >> > wait
> >> >> > for
> >> >> > something.
> >> >> > Here is what I tried:
> >> >> >     ch = gw.remote_exec("import os; channel.send(os.getcwd())")
> >> >> >     ch.receive()
> >> >> >
> >> >> > Do you know what could cause this problem?
> >> >> >
> >> >> > Thanks,
> >> >> > Mathieu
> >> >> >
> >> >> >
> >> >> > 2010/7/29 Mathieu Drapeau <matdrapeau at gmail.com>
> >> >> >>
> >> >> >> Hi Charles,
> >> >> >> I was able to install it finally, I needed to change the name of
> the
> >> >> >> file
> >> >> >> (which was socketserver.py and causes conflict).
> >> >> >>
> >> >> >> My problem now when I am trying to do a remote_exec, I am not able
> >> >> >> to
> >> >> >> debug what's happening in the background. Do you have an idea
> >> >> >> where/how
> >> >> >> I
> >> >> >> can see the trace?
> >> >> >>
> >> >> >> thanks,
> >> >> >> Mathieu
> >> >> >>
> >> >> >>
> >> >> >> 2010/7/29 Charles Solar <charlessolar at gmail.com>
> >> >> >>>
> >> >> >>> well, i would check the process list, see if pythonservice.exe is
> >> >> >>> running.  You may need to run it as admin if you are not.  Open
> >> >> >>> services.msc to make sure the service is actually installed (
> named
> >> >> >>> 'python service' creatively enough).
> >> >> >>> If all that is working, then make sure you are connecting to it
> as
> >> >> >>> a
> >> >> >>> socket gateway, not a ssh gateway.
> >> >> >>>
> >> >> >>> Another thing you may want to check, make sure you have the
> latest
> >> >> >>> version I posted which should have a class def for IOJoiner at
> the
> >> >> >>> top.  That version has a few improvements and should work better
> >> >> >>> than
> >> >> >>> the first one i posted.
> >> >> >>> Also, there is a script in the execnet scripts folder called
> >> >> >>> socketservice.py or something which works similarly and might
> work
> >> >> >>> better for you.
> >> >> >>>
> >> >> >>> Charles
> >> >> >>>
> >> >> >>> On Thu, Jul 29, 2010 at 9:54 AM, Mathieu Drapeau
> >> >> >>> <matdrapeau at gmail.com>
> >> >> >>> wrote:
> >> >> >>> > Charles,
> >> >> >>> > Like you mentionned the problem was with sshd which didn't have
> >> >> >>> > the
> >> >> >>> > option
> >> >> >>> > to "interact with the desktop".
> >> >> >>> > This option could be enabled in the windows services panel.
> >> >> >>> > Unfortunately,
> >> >> >>> > this can only be done in Windows XP pre-SP! Since then
> Microsoft
> >> >> >>> > has
> >> >> >>> > disabled this option for security reasons.
> >> >> >>> >
> >> >> >>> > So, I decided to switch to a socket server. I did see the
> socket
> >> >> >>> > server
> >> >> >>> > script you posted but I am not able to make it work.
> >> >> >>> > I do install and start the server but I am not able to connect
> to
> >> >> >>> > it.
> >> >> >>> > Seems
> >> >> >>> > it is not running because when I try to stop it right after, it
> >> >> >>> > says
> >> >> >>> > the
> >> >> >>> > service has not been started.
> >> >> >>> >
> >> >> >>> > Do you have an updated version of your script? Do you have an
> >> >> >>> > idea
> >> >> >>> > why
> >> >> >>> > the
> >> >> >>> > script seems not to be running?
> >> >> >>> >
> >> >> >>> > thanks,
> >> >> >>> > Mathieu
> >> >> >>> >
> >> >> >>> > 2010/7/27 Charles Solar <charlessolar at gmail.com>
> >> >> >>> >>
> >> >> >>> >> I suspect that cygwin's sshd probably operates in
> >> >> >>> >> non-interactive
> >> >> >>> >> mode.  Check the process list on the computer, does calc.exe
> >> >> >>> >> show
> >> >> >>> >> up?
> >> >> >>> >> If so, it just means that sshd is indeed non-interactive which
> >> >> >>> >> means
> >> >> >>> >> it will not spawn graphical windows for logged in users.
>  There
> >> >> >>> >> is
> >> >> >>> >> probably some way for it to do so, but it would be a cygwin
> >> >> >>> >> thing,
> >> >> >>> >> not
> >> >> >>> >> execnet specificly =/
> >> >> >>> >>
> >> >> >>> >> Charles
> >> >> >>> >>
> >> >> >>> >> On Tue, Jul 27, 2010 at 4:36 PM, Mathieu Drapeau
> >> >> >>> >> <matdrapeau at gmail.com>
> >> >> >>> >> wrote:
> >> >> >>> >> > Hi,
> >> >> >>> >> > I am trying to connect to a windows box and execute a
> program
> >> >> >>> >> > which
> >> >> >>> >> > requires
> >> >> >>> >> > the display using execnet. The windows box has cygwin
> >> >> >>> >> > installed
> >> >> >>> >> > where
> >> >> >>> >> > sshd
> >> >> >>> >> > is running.
> >> >> >>> >> > Here is a small example of what I am trying to achieve.
> >> >> >>> >> >
> >> >> >>> >> >
> >> >> >>> >> > # remote.py script which should open the windows calculator
> >> >> >>> >> > utility
> >> >> >>> >> > from subprocess import call
> >> >> >>> >> > if __name__ == '__channelexec__':
> >> >> >>> >> >     call(["calc"])
> >> >> >>> >> >
> >> >> >>> >> >
> >> >> >>> >> > #code to run locally
> >> >> >>> >> > import execnet, remote
> >> >> >>> >> > gw = execnet.makegateway("ssh=%s" % WINBOX_IP)
> >> >> >>> >> > ch = gw.remote_exec (remote)
> >> >> >>> >> > ch.waitclose ()
> >> >> >>> >> >
> >> >> >>> >> >
> >> >> >>> >> > Looking on the screen of my windows box, I don't see any
> >> >> >>> >> > calculator
> >> >> >>> >> > program!
> >> >> >>> >> > Is it expected?
> >> >> >>> >> >
> >> >> >>> >> > Thanks,
> >> >> >>> >> > Mathieu
> >> >> >>> >> >
> >> >> >>> >> > _______________________________________________
> >> >> >>> >> > execnet-dev mailing list
> >> >> >>> >> > execnet-dev at codespeak.net
> >> >> >>> >> > http://codespeak.net/mailman/listinfo/execnet-dev
> >> >> >>> >> >
> >> >> >>> >> >
> >> >> >>> >
> >> >> >>> >
> >> >> >>
> >> >> >
> >> >> >
> >> >
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/execnet-dev/attachments/20100802/52a453d1/attachment.html>

From charlessolar at gmail.com  Mon Aug  2 23:10:25 2010
From: charlessolar at gmail.com (Charles Solar)
Date: Mon, 2 Aug 2010 16:10:25 -0500
Subject: [execnet-dev] how to remotely open a program on win box?
In-Reply-To: <AANLkTinCRzG7_v_=OrH0RmeE+X3rUbZFV83am3rgm6kS@mail.gmail.com>
References: <AANLkTinMDtns6Ta46_ZLKKHtbTMNRa_ue6HZSwbSeoOY@mail.gmail.com> 
	<AANLkTimr=Fc0KgcthpX-TYh162t21FY15X8FSyvkzbJ7@mail.gmail.com> 
	<AANLkTin=rHc5HooJ638+FvhbP8FDAYcoPV8d+wNu7AE7@mail.gmail.com> 
	<AANLkTimVKELwEpo7tRtfwc44eYiGfiO-wkAaUpwruUie@mail.gmail.com> 
	<AANLkTinE8ReX_6Y7eyjRQV6Dp0NuZdvz3cbKk9_qEqRW@mail.gmail.com> 
	<AANLkTi=ncA5oBt3xQ+sPo_OrQ7jZBe5ymJGoX-KbjwHs@mail.gmail.com> 
	<AANLkTim6PzY2JwCLbKr2CwcBKET=si-BA1F0vrAU9YMQ@mail.gmail.com> 
	<AANLkTikXOw-2NWOkAWjf1fjeeB0==qeQapZtfc7ng_JQ@mail.gmail.com> 
	<AANLkTikgJm_dVwpuvwW7KV8PkYpbdyMqsA-EmNixeBbX@mail.gmail.com> 
	<AANLkTim+17+nnv2_fS1sKV1k_DntVjhmxW9=qswVr1UC@mail.gmail.com> 
	<AANLkTi=Sj1Z+SpYbJPjLoZ+rc=xfaza_MbGH_NkyaiN7@mail.gmail.com> 
	<AANLkTinCRzG7_v_=OrH0RmeE+X3rUbZFV83am3rgm6kS@mail.gmail.com>
Message-ID: <AANLkTimHL3G4zFKsYk9ssE2pu2CS1OzBpcSqJBGp3Zxx@mail.gmail.com>

Well a freeze on channel receive means the remote code was not sending
anything.  This would tell me that the IO instances are not being
joined correctly, as the instances seem unable to communicate with
each other.
The question is what would cause this to happen.  I would guess there
would be some sort of permission error or some problem with setup.
I guess what you could do for a quick test is to try sending
remote_exec code to the new popen gateway before replacing the IO
instance in the service script.  Just do a quick send/receive with
that popen gateway and make sure channel.receive does not freeze with
the popen gateway.

If that works then I would make sure the user that the service is
running as is an admin, and can spawn processes correctly..

Charles

On Mon, Aug 2, 2010 at 2:40 PM, Mathieu Drapeau <matdrapeau at gmail.com> wrote:
> Hi Charles,
> after some modifications to the script it appears that what is causing the
> problem is really running the script as a service!
> When I commented what you suggested, the server is working correctly, it
> doesn't jam at the "channel.receive" call.
>
> I am not familiar enough with Windows to find what could cause this error.
> If you have any idea where I could look, please tell me.
>
> Thanks,
> Mathieu
>
> 2010/7/31 Charles Solar <charlessolar at gmail.com>
>>
>> I only vaguely remember trying to use the event manager as a debugging
>> tools with little to no success. ?I would strongly recommend debugging
>> in a console window instead of a service instance.
>> Source is a string, but printing it out will not help you very much,
>> it just contains all the gateway bootstrapping code. ?Actually, I bet
>> you are getting that error because the source string is to large.
>>
>> To remove the windows service part from that file, what you need to do
>> is comment out all the __init__ code in the service class, and under
>> if __name__ == "-_main__" replace process args with something like
>> if __name__ == "__main__":
>> ? ? svr=ServiceClass( None )
>> ? ? svr.main()
>> ? ?while True:
>> ? ? ? ?time.sleep(0.1)
>>
>> then you can put print statements where ever you want and they will
>> print to the console window you started the daemon in.
>>
>> On Fri, Jul 30, 2010 at 11:07 AM, Mathieu Drapeau <matdrapeau at gmail.com>
>> wrote:
>> > Hi Charles,
>> > I don't think it is the makegateway call that causes problems, I did try
>> > bunch of different configuration without any help...
>> > But the "source" variable seems weird. The way I understand it, it
>> > should be
>> > a string and I should be able to output in the logs but everytime I try
>> > to
>> > print its value, I get an error like this:
>> >
>> > "RegisterEventSource/ReportEvent", the parameter is incorrect
>> >
>> > Here is how I am trying to output the value of source:
>> >
>> > try:
>> > ??? servicemanager.LogErrorMsg("source = %s" % source)
>> > except Exception as e:
>> > ??? servicemanager.LogErrorMsg("error = %s" % e)
>> >
>> > Any idea?
>> >
>> > Thanks,
>> > Mathieu
>> >
>> > 2010/7/29 Charles Solar <charlessolar at gmail.com>
>> >>
>> >> When I was debugging that script what I did was modify it so it was
>> >> not a service, but just a little daemon that ran in a console. ?Then I
>> >> could put print commands in there and find out what was going on.
>> >> Something I think you should try is instead of adding the
>> >> "popen//python=", just add the python directory to your computers path
>> >> and remove that line. ?I dont know, but maybe running makegateway like
>> >> that is opening a new python interactively, which would probably break
>> >> the popen io connection.
>> >> If that is all you changed then that is really all I can guess at what
>> >> the problem might be. ?If you add some print statements and find out
>> >> what paths the service is taking when it fails I might be able to help
>> >> some more.
>> >>
>> >> Charles
>> >>
>> >> On Thu, Jul 29, 2010 at 6:36 PM, Mathieu Drapeau <matdrapeau at gmail.com>
>> >> wrote:
>> >> > Yes I am currently using the other script and everything works fine.
>> >> > Do you know how I can troubleshoot / debug your script?
>> >> >
>> >> > Thanks,
>> >> > Mathieu
>> >> >
>> >> > 2010/7/29 Charles Solar <charlessolar at gmail.com>
>> >> >>
>> >> >> mm, well having a python window pop up is a bit suspicious. ?Maybe
>> >> >> the
>> >> >> popen io is not connecting correctly.
>> >> >> Maybe before trying the advanced process spawning service I wrote
>> >> >> up,
>> >> >> try the basic 1 python instance service that is in execnet's script
>> >> >> directory.
>> >> >>
>> >> >> Charles
>> >> >>
>> >> >>
>> >> >> On Thu, Jul 29, 2010 at 4:18 PM, Mathieu Drapeau
>> >> >> <matdrapeau at gmail.com>
>> >> >> wrote:
>> >> >> > Hi Charles,
>> >> >> >
>> >> >> > In the source of your server script, I did change the makegateway
>> >> >> > definition
>> >> >> > to this:
>> >> >> >
>> >> >> > gw = execnet.makegateway("popen//pytho=C:\Python26\python.exe")
>> >> >> >
>> >> >> > When I connect to the server, a python box pop and when I try to
>> >> >> > send
>> >> >> > basic
>> >> >> > command to the server, I never receive any answers. The client
>> >> >> > just
>> >> >> > wait
>> >> >> > for
>> >> >> > something.
>> >> >> > Here is what I tried:
>> >> >> > ??? ch = gw.remote_exec("import os; channel.send(os.getcwd())")
>> >> >> > ??? ch.receive()
>> >> >> >
>> >> >> > Do you know what could cause this problem?
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Mathieu
>> >> >> >
>> >> >> >
>> >> >> > 2010/7/29 Mathieu Drapeau <matdrapeau at gmail.com>
>> >> >> >>
>> >> >> >> Hi Charles,
>> >> >> >> I was able to install it finally, I needed to change the name of
>> >> >> >> the
>> >> >> >> file
>> >> >> >> (which was socketserver.py and causes conflict).
>> >> >> >>
>> >> >> >> My problem now when I am trying to do a remote_exec, I am not
>> >> >> >> able
>> >> >> >> to
>> >> >> >> debug what's happening in the background. Do you have an idea
>> >> >> >> where/how
>> >> >> >> I
>> >> >> >> can see the trace?
>> >> >> >>
>> >> >> >> thanks,
>> >> >> >> Mathieu
>> >> >> >>
>> >> >> >>
>> >> >> >> 2010/7/29 Charles Solar <charlessolar at gmail.com>
>> >> >> >>>
>> >> >> >>> well, i would check the process list, see if pythonservice.exe
>> >> >> >>> is
>> >> >> >>> running. ?You may need to run it as admin if you are not. ?Open
>> >> >> >>> services.msc to make sure the service is actually installed (
>> >> >> >>> named
>> >> >> >>> 'python service' creatively enough).
>> >> >> >>> If all that is working, then make sure you are connecting to it
>> >> >> >>> as
>> >> >> >>> a
>> >> >> >>> socket gateway, not a ssh gateway.
>> >> >> >>>
>> >> >> >>> Another thing you may want to check, make sure you have the
>> >> >> >>> latest
>> >> >> >>> version I posted which should have a class def for IOJoiner at
>> >> >> >>> the
>> >> >> >>> top. ?That version has a few improvements and should work better
>> >> >> >>> than
>> >> >> >>> the first one i posted.
>> >> >> >>> Also, there is a script in the execnet scripts folder called
>> >> >> >>> socketservice.py or something which works similarly and might
>> >> >> >>> work
>> >> >> >>> better for you.
>> >> >> >>>
>> >> >> >>> Charles
>> >> >> >>>
>> >> >> >>> On Thu, Jul 29, 2010 at 9:54 AM, Mathieu Drapeau
>> >> >> >>> <matdrapeau at gmail.com>
>> >> >> >>> wrote:
>> >> >> >>> > Charles,
>> >> >> >>> > Like you mentionned the problem was with sshd which didn't
>> >> >> >>> > have
>> >> >> >>> > the
>> >> >> >>> > option
>> >> >> >>> > to "interact with the desktop".
>> >> >> >>> > This option could be enabled in the windows services panel.
>> >> >> >>> > Unfortunately,
>> >> >> >>> > this can only be done in Windows XP pre-SP! Since then
>> >> >> >>> > Microsoft
>> >> >> >>> > has
>> >> >> >>> > disabled this option for security reasons.
>> >> >> >>> >
>> >> >> >>> > So, I decided to switch to a socket server. I did see the
>> >> >> >>> > socket
>> >> >> >>> > server
>> >> >> >>> > script you posted but I am not able to make it work.
>> >> >> >>> > I do install and start the server but I am not able to connect
>> >> >> >>> > to
>> >> >> >>> > it.
>> >> >> >>> > Seems
>> >> >> >>> > it is not running because when I try to stop it right after,
>> >> >> >>> > it
>> >> >> >>> > says
>> >> >> >>> > the
>> >> >> >>> > service has not been started.
>> >> >> >>> >
>> >> >> >>> > Do you have an updated version of your script? Do you have an
>> >> >> >>> > idea
>> >> >> >>> > why
>> >> >> >>> > the
>> >> >> >>> > script seems not to be running?
>> >> >> >>> >
>> >> >> >>> > thanks,
>> >> >> >>> > Mathieu
>> >> >> >>> >
>> >> >> >>> > 2010/7/27 Charles Solar <charlessolar at gmail.com>
>> >> >> >>> >>
>> >> >> >>> >> I suspect that cygwin's sshd probably operates in
>> >> >> >>> >> non-interactive
>> >> >> >>> >> mode. ?Check the process list on the computer, does calc.exe
>> >> >> >>> >> show
>> >> >> >>> >> up?
>> >> >> >>> >> If so, it just means that sshd is indeed non-interactive
>> >> >> >>> >> which
>> >> >> >>> >> means
>> >> >> >>> >> it will not spawn graphical windows for logged in users.
>> >> >> >>> >> ?There
>> >> >> >>> >> is
>> >> >> >>> >> probably some way for it to do so, but it would be a cygwin
>> >> >> >>> >> thing,
>> >> >> >>> >> not
>> >> >> >>> >> execnet specificly =/
>> >> >> >>> >>
>> >> >> >>> >> Charles
>> >> >> >>> >>
>> >> >> >>> >> On Tue, Jul 27, 2010 at 4:36 PM, Mathieu Drapeau
>> >> >> >>> >> <matdrapeau at gmail.com>
>> >> >> >>> >> wrote:
>> >> >> >>> >> > Hi,
>> >> >> >>> >> > I am trying to connect to a windows box and execute a
>> >> >> >>> >> > program
>> >> >> >>> >> > which
>> >> >> >>> >> > requires
>> >> >> >>> >> > the display using execnet. The windows box has cygwin
>> >> >> >>> >> > installed
>> >> >> >>> >> > where
>> >> >> >>> >> > sshd
>> >> >> >>> >> > is running.
>> >> >> >>> >> > Here is a small example of what I am trying to achieve.
>> >> >> >>> >> >
>> >> >> >>> >> >
>> >> >> >>> >> > # remote.py script which should open the windows calculator
>> >> >> >>> >> > utility
>> >> >> >>> >> > from subprocess import call
>> >> >> >>> >> > if __name__ == '__channelexec__':
>> >> >> >>> >> > ??? call(["calc"])
>> >> >> >>> >> >
>> >> >> >>> >> >
>> >> >> >>> >> > #code to run locally
>> >> >> >>> >> > import execnet, remote
>> >> >> >>> >> > gw = execnet.makegateway("ssh=%s" % WINBOX_IP)
>> >> >> >>> >> > ch = gw.remote_exec (remote)
>> >> >> >>> >> > ch.waitclose ()
>> >> >> >>> >> >
>> >> >> >>> >> >
>> >> >> >>> >> > Looking on the screen of my windows box, I don't see any
>> >> >> >>> >> > calculator
>> >> >> >>> >> > program!
>> >> >> >>> >> > Is it expected?
>> >> >> >>> >> >
>> >> >> >>> >> > Thanks,
>> >> >> >>> >> > Mathieu
>> >> >> >>> >> >
>> >> >> >>> >> > _______________________________________________
>> >> >> >>> >> > execnet-dev mailing list
>> >> >> >>> >> > execnet-dev at codespeak.net
>> >> >> >>> >> > http://codespeak.net/mailman/listinfo/execnet-dev
>> >> >> >>> >> >
>> >> >> >>> >> >
>> >> >> >>> >
>> >> >> >>> >
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>
>


From matdrapeau at gmail.com  Tue Aug  3 00:00:16 2010
From: matdrapeau at gmail.com (Mathieu Drapeau)
Date: Mon, 2 Aug 2010 18:00:16 -0400
Subject: [execnet-dev] how to remotely open a program on win box?
In-Reply-To: <AANLkTimHL3G4zFKsYk9ssE2pu2CS1OzBpcSqJBGp3Zxx@mail.gmail.com>
References: <AANLkTinMDtns6Ta46_ZLKKHtbTMNRa_ue6HZSwbSeoOY@mail.gmail.com>
	<AANLkTimr=Fc0KgcthpX-TYh162t21FY15X8FSyvkzbJ7@mail.gmail.com>
	<AANLkTin=rHc5HooJ638+FvhbP8FDAYcoPV8d+wNu7AE7@mail.gmail.com>
	<AANLkTimVKELwEpo7tRtfwc44eYiGfiO-wkAaUpwruUie@mail.gmail.com>
	<AANLkTinE8ReX_6Y7eyjRQV6Dp0NuZdvz3cbKk9_qEqRW@mail.gmail.com>
	<AANLkTi=ncA5oBt3xQ+sPo_OrQ7jZBe5ymJGoX-KbjwHs@mail.gmail.com>
	<AANLkTim6PzY2JwCLbKr2CwcBKET=si-BA1F0vrAU9YMQ@mail.gmail.com>
	<AANLkTikXOw-2NWOkAWjf1fjeeB0==qeQapZtfc7ng_JQ@mail.gmail.com>
	<AANLkTikgJm_dVwpuvwW7KV8PkYpbdyMqsA-EmNixeBbX@mail.gmail.com>
	<AANLkTim+17+nnv2_fS1sKV1k_DntVjhmxW9=qswVr1UC@mail.gmail.com>
	<AANLkTi=Sj1Z+SpYbJPjLoZ+rc=xfaza_MbGH_NkyaiN7@mail.gmail.com>
	<AANLkTinCRzG7_v_=OrH0RmeE+X3rUbZFV83am3rgm6kS@mail.gmail.com>
	<AANLkTimHL3G4zFKsYk9ssE2pu2CS1OzBpcSqJBGp3Zxx@mail.gmail.com>
Message-ID: <AANLkTinHt99rEJ=+43z0gZ-DOzv5YWAXNN9wmatjpXAa@mail.gmail.com>

Thanks Charles,
I think the problem is coming from ghost process that never gets killed when
I call exit on the gateway. Then subsequent connection are all messed up
with the previous one.
Based on the fact that sometimes it works and sometimes not even for a
standalone version or the service version.

And I guess a CTL-C in the windows shell doesn't destroy the python gateway
either.

Thanks,
Mathieu

2010/8/2 Charles Solar <charlessolar at gmail.com>

> Well a freeze on channel receive means the remote code was not sending
> anything.  This would tell me that the IO instances are not being
> joined correctly, as the instances seem unable to communicate with
> each other.
> The question is what would cause this to happen.  I would guess there
> would be some sort of permission error or some problem with setup.
> I guess what you could do for a quick test is to try sending
> remote_exec code to the new popen gateway before replacing the IO
> instance in the service script.  Just do a quick send/receive with
> that popen gateway and make sure channel.receive does not freeze with
> the popen gateway.
>
> If that works then I would make sure the user that the service is
> running as is an admin, and can spawn processes correctly..
>
> Charles
>
> On Mon, Aug 2, 2010 at 2:40 PM, Mathieu Drapeau <matdrapeau at gmail.com>
> wrote:
> > Hi Charles,
> > after some modifications to the script it appears that what is causing
> the
> > problem is really running the script as a service!
> > When I commented what you suggested, the server is working correctly, it
> > doesn't jam at the "channel.receive" call.
> >
> > I am not familiar enough with Windows to find what could cause this
> error.
> > If you have any idea where I could look, please tell me.
> >
> > Thanks,
> > Mathieu
> >
> > 2010/7/31 Charles Solar <charlessolar at gmail.com>
> >>
> >> I only vaguely remember trying to use the event manager as a debugging
> >> tools with little to no success.  I would strongly recommend debugging
> >> in a console window instead of a service instance.
> >> Source is a string, but printing it out will not help you very much,
> >> it just contains all the gateway bootstrapping code.  Actually, I bet
> >> you are getting that error because the source string is to large.
> >>
> >> To remove the windows service part from that file, what you need to do
> >> is comment out all the __init__ code in the service class, and under
> >> if __name__ == "-_main__" replace process args with something like
> >> if __name__ == "__main__":
> >>     svr=ServiceClass( None )
> >>     svr.main()
> >>    while True:
> >>        time.sleep(0.1)
> >>
> >> then you can put print statements where ever you want and they will
> >> print to the console window you started the daemon in.
> >>
> >> On Fri, Jul 30, 2010 at 11:07 AM, Mathieu Drapeau <matdrapeau at gmail.com
> >
> >> wrote:
> >> > Hi Charles,
> >> > I don't think it is the makegateway call that causes problems, I did
> try
> >> > bunch of different configuration without any help...
> >> > But the "source" variable seems weird. The way I understand it, it
> >> > should be
> >> > a string and I should be able to output in the logs but everytime I
> try
> >> > to
> >> > print its value, I get an error like this:
> >> >
> >> > "RegisterEventSource/ReportEvent", the parameter is incorrect
> >> >
> >> > Here is how I am trying to output the value of source:
> >> >
> >> > try:
> >> >     servicemanager.LogErrorMsg("source = %s" % source)
> >> > except Exception as e:
> >> >     servicemanager.LogErrorMsg("error = %s" % e)
> >> >
> >> > Any idea?
> >> >
> >> > Thanks,
> >> > Mathieu
> >> >
> >> > 2010/7/29 Charles Solar <charlessolar at gmail.com>
> >> >>
> >> >> When I was debugging that script what I did was modify it so it was
> >> >> not a service, but just a little daemon that ran in a console.  Then
> I
> >> >> could put print commands in there and find out what was going on.
> >> >> Something I think you should try is instead of adding the
> >> >> "popen//python=", just add the python directory to your computers
> path
> >> >> and remove that line.  I dont know, but maybe running makegateway
> like
> >> >> that is opening a new python interactively, which would probably
> break
> >> >> the popen io connection.
> >> >> If that is all you changed then that is really all I can guess at
> what
> >> >> the problem might be.  If you add some print statements and find out
> >> >> what paths the service is taking when it fails I might be able to
> help
> >> >> some more.
> >> >>
> >> >> Charles
> >> >>
> >> >> On Thu, Jul 29, 2010 at 6:36 PM, Mathieu Drapeau <
> matdrapeau at gmail.com>
> >> >> wrote:
> >> >> > Yes I am currently using the other script and everything works
> fine.
> >> >> > Do you know how I can troubleshoot / debug your script?
> >> >> >
> >> >> > Thanks,
> >> >> > Mathieu
> >> >> >
> >> >> > 2010/7/29 Charles Solar <charlessolar at gmail.com>
> >> >> >>
> >> >> >> mm, well having a python window pop up is a bit suspicious.  Maybe
> >> >> >> the
> >> >> >> popen io is not connecting correctly.
> >> >> >> Maybe before trying the advanced process spawning service I wrote
> >> >> >> up,
> >> >> >> try the basic 1 python instance service that is in execnet's
> script
> >> >> >> directory.
> >> >> >>
> >> >> >> Charles
> >> >> >>
> >> >> >>
> >> >> >> On Thu, Jul 29, 2010 at 4:18 PM, Mathieu Drapeau
> >> >> >> <matdrapeau at gmail.com>
> >> >> >> wrote:
> >> >> >> > Hi Charles,
> >> >> >> >
> >> >> >> > In the source of your server script, I did change the
> makegateway
> >> >> >> > definition
> >> >> >> > to this:
> >> >> >> >
> >> >> >> > gw = execnet.makegateway("popen//pytho=C:\Python26\python.exe")
> >> >> >> >
> >> >> >> > When I connect to the server, a python box pop and when I try to
> >> >> >> > send
> >> >> >> > basic
> >> >> >> > command to the server, I never receive any answers. The client
> >> >> >> > just
> >> >> >> > wait
> >> >> >> > for
> >> >> >> > something.
> >> >> >> > Here is what I tried:
> >> >> >> >     ch = gw.remote_exec("import os; channel.send(os.getcwd())")
> >> >> >> >     ch.receive()
> >> >> >> >
> >> >> >> > Do you know what could cause this problem?
> >> >> >> >
> >> >> >> > Thanks,
> >> >> >> > Mathieu
> >> >> >> >
> >> >> >> >
> >> >> >> > 2010/7/29 Mathieu Drapeau <matdrapeau at gmail.com>
> >> >> >> >>
> >> >> >> >> Hi Charles,
> >> >> >> >> I was able to install it finally, I needed to change the name
> of
> >> >> >> >> the
> >> >> >> >> file
> >> >> >> >> (which was socketserver.py and causes conflict).
> >> >> >> >>
> >> >> >> >> My problem now when I am trying to do a remote_exec, I am not
> >> >> >> >> able
> >> >> >> >> to
> >> >> >> >> debug what's happening in the background. Do you have an idea
> >> >> >> >> where/how
> >> >> >> >> I
> >> >> >> >> can see the trace?
> >> >> >> >>
> >> >> >> >> thanks,
> >> >> >> >> Mathieu
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> 2010/7/29 Charles Solar <charlessolar at gmail.com>
> >> >> >> >>>
> >> >> >> >>> well, i would check the process list, see if pythonservice.exe
> >> >> >> >>> is
> >> >> >> >>> running.  You may need to run it as admin if you are not.
>  Open
> >> >> >> >>> services.msc to make sure the service is actually installed (
> >> >> >> >>> named
> >> >> >> >>> 'python service' creatively enough).
> >> >> >> >>> If all that is working, then make sure you are connecting to
> it
> >> >> >> >>> as
> >> >> >> >>> a
> >> >> >> >>> socket gateway, not a ssh gateway.
> >> >> >> >>>
> >> >> >> >>> Another thing you may want to check, make sure you have the
> >> >> >> >>> latest
> >> >> >> >>> version I posted which should have a class def for IOJoiner at
> >> >> >> >>> the
> >> >> >> >>> top.  That version has a few improvements and should work
> better
> >> >> >> >>> than
> >> >> >> >>> the first one i posted.
> >> >> >> >>> Also, there is a script in the execnet scripts folder called
> >> >> >> >>> socketservice.py or something which works similarly and might
> >> >> >> >>> work
> >> >> >> >>> better for you.
> >> >> >> >>>
> >> >> >> >>> Charles
> >> >> >> >>>
> >> >> >> >>> On Thu, Jul 29, 2010 at 9:54 AM, Mathieu Drapeau
> >> >> >> >>> <matdrapeau at gmail.com>
> >> >> >> >>> wrote:
> >> >> >> >>> > Charles,
> >> >> >> >>> > Like you mentionned the problem was with sshd which didn't
> >> >> >> >>> > have
> >> >> >> >>> > the
> >> >> >> >>> > option
> >> >> >> >>> > to "interact with the desktop".
> >> >> >> >>> > This option could be enabled in the windows services panel.
> >> >> >> >>> > Unfortunately,
> >> >> >> >>> > this can only be done in Windows XP pre-SP! Since then
> >> >> >> >>> > Microsoft
> >> >> >> >>> > has
> >> >> >> >>> > disabled this option for security reasons.
> >> >> >> >>> >
> >> >> >> >>> > So, I decided to switch to a socket server. I did see the
> >> >> >> >>> > socket
> >> >> >> >>> > server
> >> >> >> >>> > script you posted but I am not able to make it work.
> >> >> >> >>> > I do install and start the server but I am not able to
> connect
> >> >> >> >>> > to
> >> >> >> >>> > it.
> >> >> >> >>> > Seems
> >> >> >> >>> > it is not running because when I try to stop it right after,
> >> >> >> >>> > it
> >> >> >> >>> > says
> >> >> >> >>> > the
> >> >> >> >>> > service has not been started.
> >> >> >> >>> >
> >> >> >> >>> > Do you have an updated version of your script? Do you have
> an
> >> >> >> >>> > idea
> >> >> >> >>> > why
> >> >> >> >>> > the
> >> >> >> >>> > script seems not to be running?
> >> >> >> >>> >
> >> >> >> >>> > thanks,
> >> >> >> >>> > Mathieu
> >> >> >> >>> >
> >> >> >> >>> > 2010/7/27 Charles Solar <charlessolar at gmail.com>
> >> >> >> >>> >>
> >> >> >> >>> >> I suspect that cygwin's sshd probably operates in
> >> >> >> >>> >> non-interactive
> >> >> >> >>> >> mode.  Check the process list on the computer, does
> calc.exe
> >> >> >> >>> >> show
> >> >> >> >>> >> up?
> >> >> >> >>> >> If so, it just means that sshd is indeed non-interactive
> >> >> >> >>> >> which
> >> >> >> >>> >> means
> >> >> >> >>> >> it will not spawn graphical windows for logged in users.
> >> >> >> >>> >>  There
> >> >> >> >>> >> is
> >> >> >> >>> >> probably some way for it to do so, but it would be a cygwin
> >> >> >> >>> >> thing,
> >> >> >> >>> >> not
> >> >> >> >>> >> execnet specificly =/
> >> >> >> >>> >>
> >> >> >> >>> >> Charles
> >> >> >> >>> >>
> >> >> >> >>> >> On Tue, Jul 27, 2010 at 4:36 PM, Mathieu Drapeau
> >> >> >> >>> >> <matdrapeau at gmail.com>
> >> >> >> >>> >> wrote:
> >> >> >> >>> >> > Hi,
> >> >> >> >>> >> > I am trying to connect to a windows box and execute a
> >> >> >> >>> >> > program
> >> >> >> >>> >> > which
> >> >> >> >>> >> > requires
> >> >> >> >>> >> > the display using execnet. The windows box has cygwin
> >> >> >> >>> >> > installed
> >> >> >> >>> >> > where
> >> >> >> >>> >> > sshd
> >> >> >> >>> >> > is running.
> >> >> >> >>> >> > Here is a small example of what I am trying to achieve.
> >> >> >> >>> >> >
> >> >> >> >>> >> >
> >> >> >> >>> >> > # remote.py script which should open the windows
> calculator
> >> >> >> >>> >> > utility
> >> >> >> >>> >> > from subprocess import call
> >> >> >> >>> >> > if __name__ == '__channelexec__':
> >> >> >> >>> >> >     call(["calc"])
> >> >> >> >>> >> >
> >> >> >> >>> >> >
> >> >> >> >>> >> > #code to run locally
> >> >> >> >>> >> > import execnet, remote
> >> >> >> >>> >> > gw = execnet.makegateway("ssh=%s" % WINBOX_IP)
> >> >> >> >>> >> > ch = gw.remote_exec (remote)
> >> >> >> >>> >> > ch.waitclose ()
> >> >> >> >>> >> >
> >> >> >> >>> >> >
> >> >> >> >>> >> > Looking on the screen of my windows box, I don't see any
> >> >> >> >>> >> > calculator
> >> >> >> >>> >> > program!
> >> >> >> >>> >> > Is it expected?
> >> >> >> >>> >> >
> >> >> >> >>> >> > Thanks,
> >> >> >> >>> >> > Mathieu
> >> >> >> >>> >> >
> >> >> >> >>> >> > _______________________________________________
> >> >> >> >>> >> > execnet-dev mailing list
> >> >> >> >>> >> > execnet-dev at codespeak.net
> >> >> >> >>> >> > http://codespeak.net/mailman/listinfo/execnet-dev
> >> >> >> >>> >> >
> >> >> >> >>> >> >
> >> >> >> >>> >
> >> >> >> >>> >
> >> >> >> >>
> >> >> >> >
> >> >> >> >
> >> >> >
> >> >> >
> >> >
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/execnet-dev/attachments/20100802/4b501e01/attachment.html>

From charlessolar at gmail.com  Tue Aug  3 00:24:03 2010
From: charlessolar at gmail.com (Charles Solar)
Date: Mon, 2 Aug 2010 17:24:03 -0500
Subject: [execnet-dev] how to remotely open a program on win box?
In-Reply-To: <AANLkTinHt99rEJ=+43z0gZ-DOzv5YWAXNN9wmatjpXAa@mail.gmail.com>
References: <AANLkTinMDtns6Ta46_ZLKKHtbTMNRa_ue6HZSwbSeoOY@mail.gmail.com> 
	<AANLkTimr=Fc0KgcthpX-TYh162t21FY15X8FSyvkzbJ7@mail.gmail.com> 
	<AANLkTin=rHc5HooJ638+FvhbP8FDAYcoPV8d+wNu7AE7@mail.gmail.com> 
	<AANLkTimVKELwEpo7tRtfwc44eYiGfiO-wkAaUpwruUie@mail.gmail.com> 
	<AANLkTinE8ReX_6Y7eyjRQV6Dp0NuZdvz3cbKk9_qEqRW@mail.gmail.com> 
	<AANLkTi=ncA5oBt3xQ+sPo_OrQ7jZBe5ymJGoX-KbjwHs@mail.gmail.com> 
	<AANLkTim6PzY2JwCLbKr2CwcBKET=si-BA1F0vrAU9YMQ@mail.gmail.com> 
	<AANLkTikXOw-2NWOkAWjf1fjeeB0==qeQapZtfc7ng_JQ@mail.gmail.com> 
	<AANLkTikgJm_dVwpuvwW7KV8PkYpbdyMqsA-EmNixeBbX@mail.gmail.com> 
	<AANLkTim+17+nnv2_fS1sKV1k_DntVjhmxW9=qswVr1UC@mail.gmail.com> 
	<AANLkTi=Sj1Z+SpYbJPjLoZ+rc=xfaza_MbGH_NkyaiN7@mail.gmail.com> 
	<AANLkTinCRzG7_v_=OrH0RmeE+X3rUbZFV83am3rgm6kS@mail.gmail.com> 
	<AANLkTimHL3G4zFKsYk9ssE2pu2CS1OzBpcSqJBGp3Zxx@mail.gmail.com> 
	<AANLkTinHt99rEJ=+43z0gZ-DOzv5YWAXNN9wmatjpXAa@mail.gmail.com>
Message-ID: <AANLkTinAwuUM27D-m2v3w8q-QV7LDy-9nTeEWdOUos_n@mail.gmail.com>

Yes this is certainly true.  From what I have seen if the new python
instance is busy doing busy work and not properly quitting, it will be
just be orphaned when the parent process dies.  The child will
continue to do work in the background but I have not really seen this
behavior break IO joining like you are seeing.  However I do still
keep a close eye on my process list for zombie python instances on
windows.
I should mention that there are plans for a new type of gateway that
does this sort of intermediary work built in to execnet and not hacked
in via my script.  For the time being my script works for me on a few
select machines but I definitely would not trust it for production use
at this point.

Charles


On Mon, Aug 2, 2010 at 5:00 PM, Mathieu Drapeau <matdrapeau at gmail.com> wrote:
> Thanks Charles,
> I think the problem is coming from ghost process that never gets killed when
> I call exit on the gateway. Then subsequent connection are all messed up
> with the previous one.
> Based on the fact that sometimes it works and sometimes not even for a
> standalone version or the service version.
>
> And I guess a CTL-C in the windows shell doesn't destroy the python gateway
> either.
>
> Thanks,
> Mathieu
>
> 2010/8/2 Charles Solar <charlessolar at gmail.com>
>>
>> Well a freeze on channel receive means the remote code was not sending
>> anything. ?This would tell me that the IO instances are not being
>> joined correctly, as the instances seem unable to communicate with
>> each other.
>> The question is what would cause this to happen. ?I would guess there
>> would be some sort of permission error or some problem with setup.
>> I guess what you could do for a quick test is to try sending
>> remote_exec code to the new popen gateway before replacing the IO
>> instance in the service script. ?Just do a quick send/receive with
>> that popen gateway and make sure channel.receive does not freeze with
>> the popen gateway.
>>
>> If that works then I would make sure the user that the service is
>> running as is an admin, and can spawn processes correctly..
>>
>> Charles
>>
>> On Mon, Aug 2, 2010 at 2:40 PM, Mathieu Drapeau <matdrapeau at gmail.com>
>> wrote:
>> > Hi Charles,
>> > after some modifications to the script it appears that what is causing
>> > the
>> > problem is really running the script as a service!
>> > When I commented what you suggested, the server is working correctly, it
>> > doesn't jam at the "channel.receive" call.
>> >
>> > I am not familiar enough with Windows to find what could cause this
>> > error.
>> > If you have any idea where I could look, please tell me.
>> >
>> > Thanks,
>> > Mathieu
>> >
>> > 2010/7/31 Charles Solar <charlessolar at gmail.com>
>> >>
>> >> I only vaguely remember trying to use the event manager as a debugging
>> >> tools with little to no success. ?I would strongly recommend debugging
>> >> in a console window instead of a service instance.
>> >> Source is a string, but printing it out will not help you very much,
>> >> it just contains all the gateway bootstrapping code. ?Actually, I bet
>> >> you are getting that error because the source string is to large.
>> >>
>> >> To remove the windows service part from that file, what you need to do
>> >> is comment out all the __init__ code in the service class, and under
>> >> if __name__ == "-_main__" replace process args with something like
>> >> if __name__ == "__main__":
>> >> ? ? svr=ServiceClass( None )
>> >> ? ? svr.main()
>> >> ? ?while True:
>> >> ? ? ? ?time.sleep(0.1)
>> >>
>> >> then you can put print statements where ever you want and they will
>> >> print to the console window you started the daemon in.
>> >>
>> >> On Fri, Jul 30, 2010 at 11:07 AM, Mathieu Drapeau
>> >> <matdrapeau at gmail.com>
>> >> wrote:
>> >> > Hi Charles,
>> >> > I don't think it is the makegateway call that causes problems, I did
>> >> > try
>> >> > bunch of different configuration without any help...
>> >> > But the "source" variable seems weird. The way I understand it, it
>> >> > should be
>> >> > a string and I should be able to output in the logs but everytime I
>> >> > try
>> >> > to
>> >> > print its value, I get an error like this:
>> >> >
>> >> > "RegisterEventSource/ReportEvent", the parameter is incorrect
>> >> >
>> >> > Here is how I am trying to output the value of source:
>> >> >
>> >> > try:
>> >> > ??? servicemanager.LogErrorMsg("source = %s" % source)
>> >> > except Exception as e:
>> >> > ??? servicemanager.LogErrorMsg("error = %s" % e)
>> >> >
>> >> > Any idea?
>> >> >
>> >> > Thanks,
>> >> > Mathieu
>> >> >
>> >> > 2010/7/29 Charles Solar <charlessolar at gmail.com>
>> >> >>
>> >> >> When I was debugging that script what I did was modify it so it was
>> >> >> not a service, but just a little daemon that ran in a console. ?Then
>> >> >> I
>> >> >> could put print commands in there and find out what was going on.
>> >> >> Something I think you should try is instead of adding the
>> >> >> "popen//python=", just add the python directory to your computers
>> >> >> path
>> >> >> and remove that line. ?I dont know, but maybe running makegateway
>> >> >> like
>> >> >> that is opening a new python interactively, which would probably
>> >> >> break
>> >> >> the popen io connection.
>> >> >> If that is all you changed then that is really all I can guess at
>> >> >> what
>> >> >> the problem might be. ?If you add some print statements and find out
>> >> >> what paths the service is taking when it fails I might be able to
>> >> >> help
>> >> >> some more.
>> >> >>
>> >> >> Charles
>> >> >>
>> >> >> On Thu, Jul 29, 2010 at 6:36 PM, Mathieu Drapeau
>> >> >> <matdrapeau at gmail.com>
>> >> >> wrote:
>> >> >> > Yes I am currently using the other script and everything works
>> >> >> > fine.
>> >> >> > Do you know how I can troubleshoot / debug your script?
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Mathieu
>> >> >> >
>> >> >> > 2010/7/29 Charles Solar <charlessolar at gmail.com>
>> >> >> >>
>> >> >> >> mm, well having a python window pop up is a bit suspicious.
>> >> >> >> ?Maybe
>> >> >> >> the
>> >> >> >> popen io is not connecting correctly.
>> >> >> >> Maybe before trying the advanced process spawning service I wrote
>> >> >> >> up,
>> >> >> >> try the basic 1 python instance service that is in execnet's
>> >> >> >> script
>> >> >> >> directory.
>> >> >> >>
>> >> >> >> Charles
>> >> >> >>
>> >> >> >>
>> >> >> >> On Thu, Jul 29, 2010 at 4:18 PM, Mathieu Drapeau
>> >> >> >> <matdrapeau at gmail.com>
>> >> >> >> wrote:
>> >> >> >> > Hi Charles,
>> >> >> >> >
>> >> >> >> > In the source of your server script, I did change the
>> >> >> >> > makegateway
>> >> >> >> > definition
>> >> >> >> > to this:
>> >> >> >> >
>> >> >> >> > gw = execnet.makegateway("popen//pytho=C:\Python26\python.exe")
>> >> >> >> >
>> >> >> >> > When I connect to the server, a python box pop and when I try
>> >> >> >> > to
>> >> >> >> > send
>> >> >> >> > basic
>> >> >> >> > command to the server, I never receive any answers. The client
>> >> >> >> > just
>> >> >> >> > wait
>> >> >> >> > for
>> >> >> >> > something.
>> >> >> >> > Here is what I tried:
>> >> >> >> > ??? ch = gw.remote_exec("import os; channel.send(os.getcwd())")
>> >> >> >> > ??? ch.receive()
>> >> >> >> >
>> >> >> >> > Do you know what could cause this problem?
>> >> >> >> >
>> >> >> >> > Thanks,
>> >> >> >> > Mathieu
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > 2010/7/29 Mathieu Drapeau <matdrapeau at gmail.com>
>> >> >> >> >>
>> >> >> >> >> Hi Charles,
>> >> >> >> >> I was able to install it finally, I needed to change the name
>> >> >> >> >> of
>> >> >> >> >> the
>> >> >> >> >> file
>> >> >> >> >> (which was socketserver.py and causes conflict).
>> >> >> >> >>
>> >> >> >> >> My problem now when I am trying to do a remote_exec, I am not
>> >> >> >> >> able
>> >> >> >> >> to
>> >> >> >> >> debug what's happening in the background. Do you have an idea
>> >> >> >> >> where/how
>> >> >> >> >> I
>> >> >> >> >> can see the trace?
>> >> >> >> >>
>> >> >> >> >> thanks,
>> >> >> >> >> Mathieu
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> 2010/7/29 Charles Solar <charlessolar at gmail.com>
>> >> >> >> >>>
>> >> >> >> >>> well, i would check the process list, see if
>> >> >> >> >>> pythonservice.exe
>> >> >> >> >>> is
>> >> >> >> >>> running. ?You may need to run it as admin if you are not.
>> >> >> >> >>> ?Open
>> >> >> >> >>> services.msc to make sure the service is actually installed (
>> >> >> >> >>> named
>> >> >> >> >>> 'python service' creatively enough).
>> >> >> >> >>> If all that is working, then make sure you are connecting to
>> >> >> >> >>> it
>> >> >> >> >>> as
>> >> >> >> >>> a
>> >> >> >> >>> socket gateway, not a ssh gateway.
>> >> >> >> >>>
>> >> >> >> >>> Another thing you may want to check, make sure you have the
>> >> >> >> >>> latest
>> >> >> >> >>> version I posted which should have a class def for IOJoiner
>> >> >> >> >>> at
>> >> >> >> >>> the
>> >> >> >> >>> top. ?That version has a few improvements and should work
>> >> >> >> >>> better
>> >> >> >> >>> than
>> >> >> >> >>> the first one i posted.
>> >> >> >> >>> Also, there is a script in the execnet scripts folder called
>> >> >> >> >>> socketservice.py or something which works similarly and might
>> >> >> >> >>> work
>> >> >> >> >>> better for you.
>> >> >> >> >>>
>> >> >> >> >>> Charles
>> >> >> >> >>>
>> >> >> >> >>> On Thu, Jul 29, 2010 at 9:54 AM, Mathieu Drapeau
>> >> >> >> >>> <matdrapeau at gmail.com>
>> >> >> >> >>> wrote:
>> >> >> >> >>> > Charles,
>> >> >> >> >>> > Like you mentionned the problem was with sshd which didn't
>> >> >> >> >>> > have
>> >> >> >> >>> > the
>> >> >> >> >>> > option
>> >> >> >> >>> > to "interact with the desktop".
>> >> >> >> >>> > This option could be enabled in the windows services panel.
>> >> >> >> >>> > Unfortunately,
>> >> >> >> >>> > this can only be done in Windows XP pre-SP! Since then
>> >> >> >> >>> > Microsoft
>> >> >> >> >>> > has
>> >> >> >> >>> > disabled this option for security reasons.
>> >> >> >> >>> >
>> >> >> >> >>> > So, I decided to switch to a socket server. I did see the
>> >> >> >> >>> > socket
>> >> >> >> >>> > server
>> >> >> >> >>> > script you posted but I am not able to make it work.
>> >> >> >> >>> > I do install and start the server but I am not able to
>> >> >> >> >>> > connect
>> >> >> >> >>> > to
>> >> >> >> >>> > it.
>> >> >> >> >>> > Seems
>> >> >> >> >>> > it is not running because when I try to stop it right
>> >> >> >> >>> > after,
>> >> >> >> >>> > it
>> >> >> >> >>> > says
>> >> >> >> >>> > the
>> >> >> >> >>> > service has not been started.
>> >> >> >> >>> >
>> >> >> >> >>> > Do you have an updated version of your script? Do you have
>> >> >> >> >>> > an
>> >> >> >> >>> > idea
>> >> >> >> >>> > why
>> >> >> >> >>> > the
>> >> >> >> >>> > script seems not to be running?
>> >> >> >> >>> >
>> >> >> >> >>> > thanks,
>> >> >> >> >>> > Mathieu
>> >> >> >> >>> >
>> >> >> >> >>> > 2010/7/27 Charles Solar <charlessolar at gmail.com>
>> >> >> >> >>> >>
>> >> >> >> >>> >> I suspect that cygwin's sshd probably operates in
>> >> >> >> >>> >> non-interactive
>> >> >> >> >>> >> mode. ?Check the process list on the computer, does
>> >> >> >> >>> >> calc.exe
>> >> >> >> >>> >> show
>> >> >> >> >>> >> up?
>> >> >> >> >>> >> If so, it just means that sshd is indeed non-interactive
>> >> >> >> >>> >> which
>> >> >> >> >>> >> means
>> >> >> >> >>> >> it will not spawn graphical windows for logged in users.
>> >> >> >> >>> >> ?There
>> >> >> >> >>> >> is
>> >> >> >> >>> >> probably some way for it to do so, but it would be a
>> >> >> >> >>> >> cygwin
>> >> >> >> >>> >> thing,
>> >> >> >> >>> >> not
>> >> >> >> >>> >> execnet specificly =/
>> >> >> >> >>> >>
>> >> >> >> >>> >> Charles
>> >> >> >> >>> >>
>> >> >> >> >>> >> On Tue, Jul 27, 2010 at 4:36 PM, Mathieu Drapeau
>> >> >> >> >>> >> <matdrapeau at gmail.com>
>> >> >> >> >>> >> wrote:
>> >> >> >> >>> >> > Hi,
>> >> >> >> >>> >> > I am trying to connect to a windows box and execute a
>> >> >> >> >>> >> > program
>> >> >> >> >>> >> > which
>> >> >> >> >>> >> > requires
>> >> >> >> >>> >> > the display using execnet. The windows box has cygwin
>> >> >> >> >>> >> > installed
>> >> >> >> >>> >> > where
>> >> >> >> >>> >> > sshd
>> >> >> >> >>> >> > is running.
>> >> >> >> >>> >> > Here is a small example of what I am trying to achieve.
>> >> >> >> >>> >> >
>> >> >> >> >>> >> >
>> >> >> >> >>> >> > # remote.py script which should open the windows
>> >> >> >> >>> >> > calculator
>> >> >> >> >>> >> > utility
>> >> >> >> >>> >> > from subprocess import call
>> >> >> >> >>> >> > if __name__ == '__channelexec__':
>> >> >> >> >>> >> > ??? call(["calc"])
>> >> >> >> >>> >> >
>> >> >> >> >>> >> >
>> >> >> >> >>> >> > #code to run locally
>> >> >> >> >>> >> > import execnet, remote
>> >> >> >> >>> >> > gw = execnet.makegateway("ssh=%s" % WINBOX_IP)
>> >> >> >> >>> >> > ch = gw.remote_exec (remote)
>> >> >> >> >>> >> > ch.waitclose ()
>> >> >> >> >>> >> >
>> >> >> >> >>> >> >
>> >> >> >> >>> >> > Looking on the screen of my windows box, I don't see any
>> >> >> >> >>> >> > calculator
>> >> >> >> >>> >> > program!
>> >> >> >> >>> >> > Is it expected?
>> >> >> >> >>> >> >
>> >> >> >> >>> >> > Thanks,
>> >> >> >> >>> >> > Mathieu
>> >> >> >> >>> >> >
>> >> >> >> >>> >> > _______________________________________________
>> >> >> >> >>> >> > execnet-dev mailing list
>> >> >> >> >>> >> > execnet-dev at codespeak.net
>> >> >> >> >>> >> > http://codespeak.net/mailman/listinfo/execnet-dev
>> >> >> >> >>> >> >
>> >> >> >> >>> >> >
>> >> >> >> >>> >
>> >> >> >> >>> >
>> >> >> >> >>
>> >> >> >> >
>> >> >> >> >
>> >> >> >
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>
>


From charlessolar at gmail.com  Wed Aug  4 22:34:56 2010
From: charlessolar at gmail.com (Charles Solar)
Date: Wed, 4 Aug 2010 15:34:56 -0500
Subject: [execnet-dev] RSync Issue
In-Reply-To: <AANLkTin=JyPRKW8RcDSOsng8iRtSAXKVaJiJbwhXx4d6@mail.gmail.com>
References: <AANLkTilS-Xxx1uxslsFvb0Wpu9bOJZjm-dFtGpPHkFJm@mail.gmail.com> 
	<20100711103618.GT14601@trillke.net>
	<AANLkTikXHQd1PUR8DGSnpveOqkHKWkXpi1SzNOGRGEIF@mail.gmail.com> 
	<20100727174748.GM14601@trillke.net>
	<AANLkTin=JyPRKW8RcDSOsng8iRtSAXKVaJiJbwhXx4d6@mail.gmail.com>
Message-ID: <AANLkTinRezN-nd70Lu=NQZnnWVnf9cYQt0YOPska1EiK@mail.gmail.com>

Ok I got that fail case for you, I uploaded it to rapidshare since my
last attachment did not work
http://rapidshare.com/files/411056579/rsync_fail.tar.gz.html

Also, I am going to submit a change to you on bitbucket, I added a new
option for sshgateways to programatically use ssh options without
having to give a ssh config file.  I found sending an actual file was
not very platform independent with paths and all that.
I am using it to tell ssh to shutup about rsa identification like so
gw = execnet.makegateway(
"ssh=%s@%s//ssh_options=StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null"
% ( host['user'], host['ip'] ))

Charles

On Tue, Jul 27, 2010 at 5:14 PM, Charles Solar <charlessolar at gmail.com> wrote:
> Sorry could not get around to testing this out today, i will get this
> installed next time I have a chance to run my app.
> Oh yea, completely forgot about the test case, I will get a small
> package setup to illustrate too.
>
> Charles
>
> On Tue, Jul 27, 2010 at 12:47 PM, holger krekel <holger at merlinux.eu> wrote:
>> Hi Charles,
>>
>> so i think that rsyncing symlinks should now work reaonsably well.
>> I used your patches as a starting point but also added tests and made
>> sure permissions are also transferred for directories. ?Could you verify
>> that after installing
>>
>> ? ?http://hudson.testrun.org/view/pytest/job/execnet-sdist/lastSuccessfulBuild/artifact/.tox/dist/execnet-1.0.8a1.zip
>>
>> working with symlinks works for you?
>>
>> Also, i am still interested in a small failing test case for the other issue
>> where you could not use RSync later on.
>>
>> thanks & best,
>> holger
>>
>> On Mon, Jul 12, 2010 at 17:16 -0500, Charles Solar wrote:
>>> I'll make up a small test case but just so I do not forgot I found
>>> another bug dealing with symlinks.
>>> I think that all relative symlinks are broken, or else I do not
>>> understand how they could have worked before.
>>> I saw it be having a structure like this rsync:
>>>
>>> lib/
>>> ? ?libnet.so -> libnet.so.1.0.0
>>> ? ?libnet.so.1.0.0
>>>
>>> What it created on the remote side was
>>>
>>> lib/
>>> ? libnet.so -> /tmp/rsynctemp/libnet.so.1.0.0
>>> ? libnet.so.1.0.0
>>>
>>> The link on the remote side did not include the lib subdir in the
>>> link. ?After looking at the code I fixed it up but I do not think they
>>> should be working at all since the code atm only works if the symlink
>>> is in the root dest directory.
>>>
>>> Ill send in a patch for you to look at.
>>>
>>> Charles
>>>
>>> On Sun, Jul 11, 2010 at 5:36 AM, holger krekel <holger at merlinux.eu> wrote:
>>> > Hi Charles,
>>> >
>>> > On Fri, Jul 09, 2010 at 10:03 -0500, Charles Solar wrote:
>>> >> It would appear that RSync will only work with brand new gateways that
>>> >> have not had any other operations run on them. ?Is this intentional?
>>> >
>>> > No. ?Sounds like a bug.
>>> >
>>> >> For example, I do something like
>>> >>
>>> >> gw = execnet.makegateway( "ssh=foobar" )
>>> >> gw.remote_exec( mybootstrapper )
>>> >>
>>> >> ... then later I want to rsync ...
>>> >>
>>> >> rsync = execnet.RSync( "/tmp/foobar" )
>>> >> rsync.add_target( gw, "/tmp/barfoo" )
>>> >> rsync.send()
>>> >>
>>> >> I get remote EOFErrors
>>> >>
>>> >> File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 126, in send
>>> >> ? ? self._end_of_channel(channel)
>>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 44, in _end_of_channel
>>> >> ? ? channel.waitclose()
>>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/gateway_base.py", line 377, in waitclose
>>> >> ? ? raise error
>>> >> exceptions.EOFError: expected 1 bytes, got 0
>>> >>
>>> >> If I however create a brand new gateway to give to rsync it works fine.
>>> >>
>>> >> I would think that since rsync just creates a new channel on the
>>> >> gateway that it should work on existing 'old' gateways.
>>> >
>>> > yip. Myself i only used it for doing things at the beginning, it seems.
>>> > Do you have or could you try to find an repeatable minimal example?
>>> >
>>> > best,
>>> >
>>> > holger
>>> >
>>>
>>
>> --
>>
>


From holger at merlinux.eu  Thu Aug  5 18:58:37 2010
From: holger at merlinux.eu (holger krekel)
Date: Thu, 5 Aug 2010 18:58:37 +0200
Subject: [execnet-dev] RSync Issue
In-Reply-To: <AANLkTinRezN-nd70Lu=NQZnnWVnf9cYQt0YOPska1EiK@mail.gmail.com>
References: <AANLkTilS-Xxx1uxslsFvb0Wpu9bOJZjm-dFtGpPHkFJm@mail.gmail.com>
	<20100711103618.GT14601@trillke.net>
	<AANLkTikXHQd1PUR8DGSnpveOqkHKWkXpi1SzNOGRGEIF@mail.gmail.com>
	<20100727174748.GM14601@trillke.net>
	<AANLkTin=JyPRKW8RcDSOsng8iRtSAXKVaJiJbwhXx4d6@mail.gmail.com>
	<AANLkTinRezN-nd70Lu=NQZnnWVnf9cYQt0YOPska1EiK@mail.gmail.com>
Message-ID: <20100805165837.GV1914@trillke.net>

Hi Charles, 

On Wed, Aug 04, 2010 at 15:34 -0500, Charles Solar wrote:
> Ok I got that fail case for you, I uploaded it to rapidshare since my
> last attachment did not work
> http://rapidshare.com/files/411056579/rsync_fail.tar.gz.html

with execnet-1.0.8 (containing the rsync fixes) this seems to work for me 
unless i am missing something? 
 
> Also, I am going to submit a change to you on bitbucket, I added a new
> option for sshgateways to programatically use ssh options without
> having to give a ssh config file.  I found sending an actual file was
> not very platform independent with paths and all that.
> I am using it to tell ssh to shutup about rsa identification like so
> gw = execnet.makegateway(
> "ssh=%s@%s//ssh_options=StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null"
> % ( host['user'], host['ip'] ))

sounds good.  I wonder if it's also possible to append the option 
to the actual ssh=* spec. We'd need a safe separator but maybe 
";" is reasonably safe so it could read:

    ssh=user at host;StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null

looking forward to your patch :)

best,
holger

> 
> Charles
> 
> On Tue, Jul 27, 2010 at 5:14 PM, Charles Solar <charlessolar at gmail.com> wrote:
> > Sorry could not get around to testing this out today, i will get this
> > installed next time I have a chance to run my app.
> > Oh yea, completely forgot about the test case, I will get a small
> > package setup to illustrate too.
> >
> > Charles
> >
> > On Tue, Jul 27, 2010 at 12:47 PM, holger krekel <holger at merlinux.eu> wrote:
> >> Hi Charles,
> >>
> >> so i think that rsyncing symlinks should now work reaonsably well.
> >> I used your patches as a starting point but also added tests and made
> >> sure permissions are also transferred for directories. ?Could you verify
> >> that after installing
> >>
> >> ? ?http://hudson.testrun.org/view/pytest/job/execnet-sdist/lastSuccessfulBuild/artifact/.tox/dist/execnet-1.0.8a1.zip
> >>
> >> working with symlinks works for you?
> >>
> >> Also, i am still interested in a small failing test case for the other issue
> >> where you could not use RSync later on.
> >>
> >> thanks & best,
> >> holger
> >>
> >> On Mon, Jul 12, 2010 at 17:16 -0500, Charles Solar wrote:
> >>> I'll make up a small test case but just so I do not forgot I found
> >>> another bug dealing with symlinks.
> >>> I think that all relative symlinks are broken, or else I do not
> >>> understand how they could have worked before.
> >>> I saw it be having a structure like this rsync:
> >>>
> >>> lib/
> >>> ? ?libnet.so -> libnet.so.1.0.0
> >>> ? ?libnet.so.1.0.0
> >>>
> >>> What it created on the remote side was
> >>>
> >>> lib/
> >>> ? libnet.so -> /tmp/rsynctemp/libnet.so.1.0.0
> >>> ? libnet.so.1.0.0
> >>>
> >>> The link on the remote side did not include the lib subdir in the
> >>> link. ?After looking at the code I fixed it up but I do not think they
> >>> should be working at all since the code atm only works if the symlink
> >>> is in the root dest directory.
> >>>
> >>> Ill send in a patch for you to look at.
> >>>
> >>> Charles
> >>>
> >>> On Sun, Jul 11, 2010 at 5:36 AM, holger krekel <holger at merlinux.eu> wrote:
> >>> > Hi Charles,
> >>> >
> >>> > On Fri, Jul 09, 2010 at 10:03 -0500, Charles Solar wrote:
> >>> >> It would appear that RSync will only work with brand new gateways that
> >>> >> have not had any other operations run on them. ?Is this intentional?
> >>> >
> >>> > No. ?Sounds like a bug.
> >>> >
> >>> >> For example, I do something like
> >>> >>
> >>> >> gw = execnet.makegateway( "ssh=foobar" )
> >>> >> gw.remote_exec( mybootstrapper )
> >>> >>
> >>> >> ... then later I want to rsync ...
> >>> >>
> >>> >> rsync = execnet.RSync( "/tmp/foobar" )
> >>> >> rsync.add_target( gw, "/tmp/barfoo" )
> >>> >> rsync.send()
> >>> >>
> >>> >> I get remote EOFErrors
> >>> >>
> >>> >> File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 126, in send
> >>> >> ? ? self._end_of_channel(channel)
> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 44, in _end_of_channel
> >>> >> ? ? channel.waitclose()
> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/gateway_base.py", line 377, in waitclose
> >>> >> ? ? raise error
> >>> >> exceptions.EOFError: expected 1 bytes, got 0
> >>> >>
> >>> >> If I however create a brand new gateway to give to rsync it works fine.
> >>> >>
> >>> >> I would think that since rsync just creates a new channel on the
> >>> >> gateway that it should work on existing 'old' gateways.
> >>> >
> >>> > yip. Myself i only used it for doing things at the beginning, it seems.
> >>> > Do you have or could you try to find an repeatable minimal example?
> >>> >
> >>> > best,
> >>> >
> >>> > holger
> >>> >
> >>>
> >>
> >> --
> >>
> >
> 

-- 


From charlessolar at gmail.com  Thu Aug  5 19:46:46 2010
From: charlessolar at gmail.com (Charles Solar)
Date: Thu, 5 Aug 2010 12:46:46 -0500
Subject: [execnet-dev] RSync Issue
In-Reply-To: <20100805165837.GV1914@trillke.net>
References: <AANLkTilS-Xxx1uxslsFvb0Wpu9bOJZjm-dFtGpPHkFJm@mail.gmail.com> 
	<20100711103618.GT14601@trillke.net>
	<AANLkTikXHQd1PUR8DGSnpveOqkHKWkXpi1SzNOGRGEIF@mail.gmail.com> 
	<20100727174748.GM14601@trillke.net>
	<AANLkTin=JyPRKW8RcDSOsng8iRtSAXKVaJiJbwhXx4d6@mail.gmail.com> 
	<AANLkTinRezN-nd70Lu=NQZnnWVnf9cYQt0YOPska1EiK@mail.gmail.com> 
	<20100805165837.GV1914@trillke.net>
Message-ID: <AANLkTikD=-vhUO59R6BApLzY9aW6Lm6mWQ5Ms-PbJg1e@mail.gmail.com>

Oh my, it appears in my haste to get a fail test case I mis-remembered
what you were actually asking for.  I had completely forgotten about
the original rsync on an old gateway does not work problem I was
having.  That file was meant to demonstrate the symlink error :P

Here is a python file that demonstrates the ORIGINAL rsync problem I
was having.  Turns out its not exactly 'an old gateway' like I said
earlier.  Its a gateway where a remote channel is still executing
code.  In my programs, I start a twisted reactor on the remote
machines, so the gateway's remote code never really 'completes.'  This
causes rsync to fail to use that gateway for the rsync channel.
In this demo file you can see this by first running it and watching it
freeze on rsync send.  Then, uncomment the chan.send and chan.receive
lines and it will work.

hopefully this attachment works.

Charles

On Thu, Aug 5, 2010 at 11:58 AM, holger krekel <holger at merlinux.eu> wrote:
> Hi Charles,
>
> On Wed, Aug 04, 2010 at 15:34 -0500, Charles Solar wrote:
>> Ok I got that fail case for you, I uploaded it to rapidshare since my
>> last attachment did not work
>> http://rapidshare.com/files/411056579/rsync_fail.tar.gz.html
>
> with execnet-1.0.8 (containing the rsync fixes) this seems to work for me
> unless i am missing something?
>
>> Also, I am going to submit a change to you on bitbucket, I added a new
>> option for sshgateways to programatically use ssh options without
>> having to give a ssh config file. ?I found sending an actual file was
>> not very platform independent with paths and all that.
>> I am using it to tell ssh to shutup about rsa identification like so
>> gw = execnet.makegateway(
>> "ssh=%s@%s//ssh_options=StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null"
>> % ( host['user'], host['ip'] ))
>
> sounds good. ?I wonder if it's also possible to append the option
> to the actual ssh=* spec. We'd need a safe separator but maybe
> ";" is reasonably safe so it could read:
>
> ? ?ssh=user at host;StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null
>
> looking forward to your patch :)
>
> best,
> holger
>
>>
>> Charles
>>
>> On Tue, Jul 27, 2010 at 5:14 PM, Charles Solar <charlessolar at gmail.com> wrote:
>> > Sorry could not get around to testing this out today, i will get this
>> > installed next time I have a chance to run my app.
>> > Oh yea, completely forgot about the test case, I will get a small
>> > package setup to illustrate too.
>> >
>> > Charles
>> >
>> > On Tue, Jul 27, 2010 at 12:47 PM, holger krekel <holger at merlinux.eu> wrote:
>> >> Hi Charles,
>> >>
>> >> so i think that rsyncing symlinks should now work reaonsably well.
>> >> I used your patches as a starting point but also added tests and made
>> >> sure permissions are also transferred for directories. ?Could you verify
>> >> that after installing
>> >>
>> >> ? ?http://hudson.testrun.org/view/pytest/job/execnet-sdist/lastSuccessfulBuild/artifact/.tox/dist/execnet-1.0.8a1.zip
>> >>
>> >> working with symlinks works for you?
>> >>
>> >> Also, i am still interested in a small failing test case for the other issue
>> >> where you could not use RSync later on.
>> >>
>> >> thanks & best,
>> >> holger
>> >>
>> >> On Mon, Jul 12, 2010 at 17:16 -0500, Charles Solar wrote:
>> >>> I'll make up a small test case but just so I do not forgot I found
>> >>> another bug dealing with symlinks.
>> >>> I think that all relative symlinks are broken, or else I do not
>> >>> understand how they could have worked before.
>> >>> I saw it be having a structure like this rsync:
>> >>>
>> >>> lib/
>> >>> ? ?libnet.so -> libnet.so.1.0.0
>> >>> ? ?libnet.so.1.0.0
>> >>>
>> >>> What it created on the remote side was
>> >>>
>> >>> lib/
>> >>> ? libnet.so -> /tmp/rsynctemp/libnet.so.1.0.0
>> >>> ? libnet.so.1.0.0
>> >>>
>> >>> The link on the remote side did not include the lib subdir in the
>> >>> link. ?After looking at the code I fixed it up but I do not think they
>> >>> should be working at all since the code atm only works if the symlink
>> >>> is in the root dest directory.
>> >>>
>> >>> Ill send in a patch for you to look at.
>> >>>
>> >>> Charles
>> >>>
>> >>> On Sun, Jul 11, 2010 at 5:36 AM, holger krekel <holger at merlinux.eu> wrote:
>> >>> > Hi Charles,
>> >>> >
>> >>> > On Fri, Jul 09, 2010 at 10:03 -0500, Charles Solar wrote:
>> >>> >> It would appear that RSync will only work with brand new gateways that
>> >>> >> have not had any other operations run on them. ?Is this intentional?
>> >>> >
>> >>> > No. ?Sounds like a bug.
>> >>> >
>> >>> >> For example, I do something like
>> >>> >>
>> >>> >> gw = execnet.makegateway( "ssh=foobar" )
>> >>> >> gw.remote_exec( mybootstrapper )
>> >>> >>
>> >>> >> ... then later I want to rsync ...
>> >>> >>
>> >>> >> rsync = execnet.RSync( "/tmp/foobar" )
>> >>> >> rsync.add_target( gw, "/tmp/barfoo" )
>> >>> >> rsync.send()
>> >>> >>
>> >>> >> I get remote EOFErrors
>> >>> >>
>> >>> >> File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 126, in send
>> >>> >> ? ? self._end_of_channel(channel)
>> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 44, in _end_of_channel
>> >>> >> ? ? channel.waitclose()
>> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/gateway_base.py", line 377, in waitclose
>> >>> >> ? ? raise error
>> >>> >> exceptions.EOFError: expected 1 bytes, got 0
>> >>> >>
>> >>> >> If I however create a brand new gateway to give to rsync it works fine.
>> >>> >>
>> >>> >> I would think that since rsync just creates a new channel on the
>> >>> >> gateway that it should work on existing 'old' gateways.
>> >>> >
>> >>> > yip. Myself i only used it for doing things at the beginning, it seems.
>> >>> > Do you have or could you try to find an repeatable minimal example?
>> >>> >
>> >>> > best,
>> >>> >
>> >>> > holger
>> >>> >
>> >>>
>> >>
>> >> --
>> >>
>> >
>>
>
> --
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fail.py
Type: text/x-python
Size: 340 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/execnet-dev/attachments/20100805/ef74b5e5/attachment.py>

From holger at merlinux.eu  Thu Aug  5 20:31:52 2010
From: holger at merlinux.eu (holger krekel)
Date: Thu, 5 Aug 2010 20:31:52 +0200
Subject: [execnet-dev] RSync Issue
In-Reply-To: <AANLkTikD=-vhUO59R6BApLzY9aW6Lm6mWQ5Ms-PbJg1e@mail.gmail.com>
References: <AANLkTilS-Xxx1uxslsFvb0Wpu9bOJZjm-dFtGpPHkFJm@mail.gmail.com>
	<20100711103618.GT14601@trillke.net>
	<AANLkTikXHQd1PUR8DGSnpveOqkHKWkXpi1SzNOGRGEIF@mail.gmail.com>
	<20100727174748.GM14601@trillke.net>
	<AANLkTin=JyPRKW8RcDSOsng8iRtSAXKVaJiJbwhXx4d6@mail.gmail.com>
	<AANLkTinRezN-nd70Lu=NQZnnWVnf9cYQt0YOPska1EiK@mail.gmail.com>
	<20100805165837.GV1914@trillke.net>
	<AANLkTikD=-vhUO59R6BApLzY9aW6Lm6mWQ5Ms-PbJg1e@mail.gmail.com>
Message-ID: <20100805183152.GW1914@trillke.net>

Hey Charles, 

On Thu, Aug 05, 2010 at 12:46 -0500, Charles Solar wrote:
> Oh my, it appears in my haste to get a fail test case I mis-remembered
> what you were actually asking for.  I had completely forgotten about
> the original rsync on an old gateway does not work problem I was
> having.  That file was meant to demonstrate the symlink error :P

np, so we can consider the rsync symlink issue solved i guess :) 

> Here is a python file that demonstrates the ORIGINAL rsync problem I
> was having.  Turns out its not exactly 'an old gateway' like I said
> earlier.  Its a gateway where a remote channel is still executing
> code.  In my programs, I start a twisted reactor on the remote
> machines, so the gateway's remote code never really 'completes.'  This
> causes rsync to fail to use that gateway for the rsync channel.
> In this demo file you can see this by first running it and watching it
> freeze on rsync send.  Then, uncomment the chan.send and chan.receive
> lines and it will work.

It's expected behaviour.  If you do: 

    gw.remote_init_threads(num=2)

just after creating the gateway you can have two concurrent executions
and the rsync will work.

best,
holger

> hopefully this attachment works.
> 
> Charles
> 
> On Thu, Aug 5, 2010 at 11:58 AM, holger krekel <holger at merlinux.eu> wrote:
> > Hi Charles,
> >
> > On Wed, Aug 04, 2010 at 15:34 -0500, Charles Solar wrote:
> >> Ok I got that fail case for you, I uploaded it to rapidshare since my
> >> last attachment did not work
> >> http://rapidshare.com/files/411056579/rsync_fail.tar.gz.html
> >
> > with execnet-1.0.8 (containing the rsync fixes) this seems to work for me
> > unless i am missing something?
> >
> >> Also, I am going to submit a change to you on bitbucket, I added a new
> >> option for sshgateways to programatically use ssh options without
> >> having to give a ssh config file. ?I found sending an actual file was
> >> not very platform independent with paths and all that.
> >> I am using it to tell ssh to shutup about rsa identification like so
> >> gw = execnet.makegateway(
> >> "ssh=%s@%s//ssh_options=StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null"
> >> % ( host['user'], host['ip'] ))
> >
> > sounds good. ?I wonder if it's also possible to append the option
> > to the actual ssh=* spec. We'd need a safe separator but maybe
> > ";" is reasonably safe so it could read:
> >
> > ? ?ssh=user at host;StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null
> >
> > looking forward to your patch :)
> >
> > best,
> > holger
> >
> >>
> >> Charles
> >>
> >> On Tue, Jul 27, 2010 at 5:14 PM, Charles Solar <charlessolar at gmail.com> wrote:
> >> > Sorry could not get around to testing this out today, i will get this
> >> > installed next time I have a chance to run my app.
> >> > Oh yea, completely forgot about the test case, I will get a small
> >> > package setup to illustrate too.
> >> >
> >> > Charles
> >> >
> >> > On Tue, Jul 27, 2010 at 12:47 PM, holger krekel <holger at merlinux.eu> wrote:
> >> >> Hi Charles,
> >> >>
> >> >> so i think that rsyncing symlinks should now work reaonsably well.
> >> >> I used your patches as a starting point but also added tests and made
> >> >> sure permissions are also transferred for directories. ?Could you verify
> >> >> that after installing
> >> >>
> >> >> ? ?http://hudson.testrun.org/view/pytest/job/execnet-sdist/lastSuccessfulBuild/artifact/.tox/dist/execnet-1.0.8a1.zip
> >> >>
> >> >> working with symlinks works for you?
> >> >>
> >> >> Also, i am still interested in a small failing test case for the other issue
> >> >> where you could not use RSync later on.
> >> >>
> >> >> thanks & best,
> >> >> holger
> >> >>
> >> >> On Mon, Jul 12, 2010 at 17:16 -0500, Charles Solar wrote:
> >> >>> I'll make up a small test case but just so I do not forgot I found
> >> >>> another bug dealing with symlinks.
> >> >>> I think that all relative symlinks are broken, or else I do not
> >> >>> understand how they could have worked before.
> >> >>> I saw it be having a structure like this rsync:
> >> >>>
> >> >>> lib/
> >> >>> ? ?libnet.so -> libnet.so.1.0.0
> >> >>> ? ?libnet.so.1.0.0
> >> >>>
> >> >>> What it created on the remote side was
> >> >>>
> >> >>> lib/
> >> >>> ? libnet.so -> /tmp/rsynctemp/libnet.so.1.0.0
> >> >>> ? libnet.so.1.0.0
> >> >>>
> >> >>> The link on the remote side did not include the lib subdir in the
> >> >>> link. ?After looking at the code I fixed it up but I do not think they
> >> >>> should be working at all since the code atm only works if the symlink
> >> >>> is in the root dest directory.
> >> >>>
> >> >>> Ill send in a patch for you to look at.
> >> >>>
> >> >>> Charles
> >> >>>
> >> >>> On Sun, Jul 11, 2010 at 5:36 AM, holger krekel <holger at merlinux.eu> wrote:
> >> >>> > Hi Charles,
> >> >>> >
> >> >>> > On Fri, Jul 09, 2010 at 10:03 -0500, Charles Solar wrote:
> >> >>> >> It would appear that RSync will only work with brand new gateways that
> >> >>> >> have not had any other operations run on them. ?Is this intentional?
> >> >>> >
> >> >>> > No. ?Sounds like a bug.
> >> >>> >
> >> >>> >> For example, I do something like
> >> >>> >>
> >> >>> >> gw = execnet.makegateway( "ssh=foobar" )
> >> >>> >> gw.remote_exec( mybootstrapper )
> >> >>> >>
> >> >>> >> ... then later I want to rsync ...
> >> >>> >>
> >> >>> >> rsync = execnet.RSync( "/tmp/foobar" )
> >> >>> >> rsync.add_target( gw, "/tmp/barfoo" )
> >> >>> >> rsync.send()
> >> >>> >>
> >> >>> >> I get remote EOFErrors
> >> >>> >>
> >> >>> >> File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 126, in send
> >> >>> >> ? ? self._end_of_channel(channel)
> >> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 44, in _end_of_channel
> >> >>> >> ? ? channel.waitclose()
> >> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/gateway_base.py", line 377, in waitclose
> >> >>> >> ? ? raise error
> >> >>> >> exceptions.EOFError: expected 1 bytes, got 0
> >> >>> >>
> >> >>> >> If I however create a brand new gateway to give to rsync it works fine.
> >> >>> >>
> >> >>> >> I would think that since rsync just creates a new channel on the
> >> >>> >> gateway that it should work on existing 'old' gateways.
> >> >>> >
> >> >>> > yip. Myself i only used it for doing things at the beginning, it seems.
> >> >>> > Do you have or could you try to find an repeatable minimal example?
> >> >>> >
> >> >>> > best,
> >> >>> >
> >> >>> > holger
> >> >>> >
> >> >>>
> >> >>
> >> >> --
> >> >>
> >> >
> >>
> >
> > --
> >

> import execnet
> 
> print "New rsync"
> rsync = execnet.RSync( "." )
> print "Making gateway"
> gw = execnet.makegateway()
> print "Execute echo"
> chan=gw.remote_exec( "channel.send( channel.receive() )" )
> #chan.send( "Echo" )
> #assert chan.receive() == "Echo"
> print "Add rsync target"
> rsync.add_target( gw, "/tmp/rsync" )
> print "Rsync send"
> rsync.send()


-- 


From charlessolar at gmail.com  Thu Aug  5 22:45:23 2010
From: charlessolar at gmail.com (Charles Solar)
Date: Thu, 5 Aug 2010 15:45:23 -0500
Subject: [execnet-dev] RSync Issue
In-Reply-To: <20100805183152.GW1914@trillke.net>
References: <AANLkTilS-Xxx1uxslsFvb0Wpu9bOJZjm-dFtGpPHkFJm@mail.gmail.com> 
	<20100711103618.GT14601@trillke.net>
	<AANLkTikXHQd1PUR8DGSnpveOqkHKWkXpi1SzNOGRGEIF@mail.gmail.com> 
	<20100727174748.GM14601@trillke.net>
	<AANLkTin=JyPRKW8RcDSOsng8iRtSAXKVaJiJbwhXx4d6@mail.gmail.com> 
	<AANLkTinRezN-nd70Lu=NQZnnWVnf9cYQt0YOPska1EiK@mail.gmail.com> 
	<20100805165837.GV1914@trillke.net>
	<AANLkTikD=-vhUO59R6BApLzY9aW6Lm6mWQ5Ms-PbJg1e@mail.gmail.com> 
	<20100805183152.GW1914@trillke.net>
Message-ID: <AANLkTikwRrjJDV2sma+vi6PCwLvcPDA+9aewSCsecGT_@mail.gmail.com>

Ah, that makes sense. However, twisted's reactor requires that its run
in the main app loop, not a new thread.  Is there a way to get my
remote_exec module to run in the main app thread and still get a
thread for serving rync data?

( I tried using  that code and twisted actually spit back
"exceptions.ValueError: signal only works in main thread" at me )
Charles

On Thu, Aug 5, 2010 at 1:31 PM, holger krekel <holger at merlinux.eu> wrote:
> Hey Charles,
>
> On Thu, Aug 05, 2010 at 12:46 -0500, Charles Solar wrote:
>> Oh my, it appears in my haste to get a fail test case I mis-remembered
>> what you were actually asking for. ?I had completely forgotten about
>> the original rsync on an old gateway does not work problem I was
>> having. ?That file was meant to demonstrate the symlink error :P
>
> np, so we can consider the rsync symlink issue solved i guess :)
>
>> Here is a python file that demonstrates the ORIGINAL rsync problem I
>> was having. ?Turns out its not exactly 'an old gateway' like I said
>> earlier. ?Its a gateway where a remote channel is still executing
>> code. ?In my programs, I start a twisted reactor on the remote
>> machines, so the gateway's remote code never really 'completes.' ?This
>> causes rsync to fail to use that gateway for the rsync channel.
>> In this demo file you can see this by first running it and watching it
>> freeze on rsync send. ?Then, uncomment the chan.send and chan.receive
>> lines and it will work.
>
> It's expected behaviour. ?If you do:
>
> ? ?gw.remote_init_threads(num=2)
>
> just after creating the gateway you can have two concurrent executions
> and the rsync will work.
>
> best,
> holger
>
>> hopefully this attachment works.
>>
>> Charles
>>
>> On Thu, Aug 5, 2010 at 11:58 AM, holger krekel <holger at merlinux.eu> wrote:
>> > Hi Charles,
>> >
>> > On Wed, Aug 04, 2010 at 15:34 -0500, Charles Solar wrote:
>> >> Ok I got that fail case for you, I uploaded it to rapidshare since my
>> >> last attachment did not work
>> >> http://rapidshare.com/files/411056579/rsync_fail.tar.gz.html
>> >
>> > with execnet-1.0.8 (containing the rsync fixes) this seems to work for me
>> > unless i am missing something?
>> >
>> >> Also, I am going to submit a change to you on bitbucket, I added a new
>> >> option for sshgateways to programatically use ssh options without
>> >> having to give a ssh config file. ?I found sending an actual file was
>> >> not very platform independent with paths and all that.
>> >> I am using it to tell ssh to shutup about rsa identification like so
>> >> gw = execnet.makegateway(
>> >> "ssh=%s@%s//ssh_options=StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null"
>> >> % ( host['user'], host['ip'] ))
>> >
>> > sounds good. ?I wonder if it's also possible to append the option
>> > to the actual ssh=* spec. We'd need a safe separator but maybe
>> > ";" is reasonably safe so it could read:
>> >
>> > ? ?ssh=user at host;StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null
>> >
>> > looking forward to your patch :)
>> >
>> > best,
>> > holger
>> >
>> >>
>> >> Charles
>> >>
>> >> On Tue, Jul 27, 2010 at 5:14 PM, Charles Solar <charlessolar at gmail.com> wrote:
>> >> > Sorry could not get around to testing this out today, i will get this
>> >> > installed next time I have a chance to run my app.
>> >> > Oh yea, completely forgot about the test case, I will get a small
>> >> > package setup to illustrate too.
>> >> >
>> >> > Charles
>> >> >
>> >> > On Tue, Jul 27, 2010 at 12:47 PM, holger krekel <holger at merlinux.eu> wrote:
>> >> >> Hi Charles,
>> >> >>
>> >> >> so i think that rsyncing symlinks should now work reaonsably well.
>> >> >> I used your patches as a starting point but also added tests and made
>> >> >> sure permissions are also transferred for directories. ?Could you verify
>> >> >> that after installing
>> >> >>
>> >> >> ? ?http://hudson.testrun.org/view/pytest/job/execnet-sdist/lastSuccessfulBuild/artifact/.tox/dist/execnet-1.0.8a1.zip
>> >> >>
>> >> >> working with symlinks works for you?
>> >> >>
>> >> >> Also, i am still interested in a small failing test case for the other issue
>> >> >> where you could not use RSync later on.
>> >> >>
>> >> >> thanks & best,
>> >> >> holger
>> >> >>
>> >> >> On Mon, Jul 12, 2010 at 17:16 -0500, Charles Solar wrote:
>> >> >>> I'll make up a small test case but just so I do not forgot I found
>> >> >>> another bug dealing with symlinks.
>> >> >>> I think that all relative symlinks are broken, or else I do not
>> >> >>> understand how they could have worked before.
>> >> >>> I saw it be having a structure like this rsync:
>> >> >>>
>> >> >>> lib/
>> >> >>> ? ?libnet.so -> libnet.so.1.0.0
>> >> >>> ? ?libnet.so.1.0.0
>> >> >>>
>> >> >>> What it created on the remote side was
>> >> >>>
>> >> >>> lib/
>> >> >>> ? libnet.so -> /tmp/rsynctemp/libnet.so.1.0.0
>> >> >>> ? libnet.so.1.0.0
>> >> >>>
>> >> >>> The link on the remote side did not include the lib subdir in the
>> >> >>> link. ?After looking at the code I fixed it up but I do not think they
>> >> >>> should be working at all since the code atm only works if the symlink
>> >> >>> is in the root dest directory.
>> >> >>>
>> >> >>> Ill send in a patch for you to look at.
>> >> >>>
>> >> >>> Charles
>> >> >>>
>> >> >>> On Sun, Jul 11, 2010 at 5:36 AM, holger krekel <holger at merlinux.eu> wrote:
>> >> >>> > Hi Charles,
>> >> >>> >
>> >> >>> > On Fri, Jul 09, 2010 at 10:03 -0500, Charles Solar wrote:
>> >> >>> >> It would appear that RSync will only work with brand new gateways that
>> >> >>> >> have not had any other operations run on them. ?Is this intentional?
>> >> >>> >
>> >> >>> > No. ?Sounds like a bug.
>> >> >>> >
>> >> >>> >> For example, I do something like
>> >> >>> >>
>> >> >>> >> gw = execnet.makegateway( "ssh=foobar" )
>> >> >>> >> gw.remote_exec( mybootstrapper )
>> >> >>> >>
>> >> >>> >> ... then later I want to rsync ...
>> >> >>> >>
>> >> >>> >> rsync = execnet.RSync( "/tmp/foobar" )
>> >> >>> >> rsync.add_target( gw, "/tmp/barfoo" )
>> >> >>> >> rsync.send()
>> >> >>> >>
>> >> >>> >> I get remote EOFErrors
>> >> >>> >>
>> >> >>> >> File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 126, in send
>> >> >>> >> ? ? self._end_of_channel(channel)
>> >> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 44, in _end_of_channel
>> >> >>> >> ? ? channel.waitclose()
>> >> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/gateway_base.py", line 377, in waitclose
>> >> >>> >> ? ? raise error
>> >> >>> >> exceptions.EOFError: expected 1 bytes, got 0
>> >> >>> >>
>> >> >>> >> If I however create a brand new gateway to give to rsync it works fine.
>> >> >>> >>
>> >> >>> >> I would think that since rsync just creates a new channel on the
>> >> >>> >> gateway that it should work on existing 'old' gateways.
>> >> >>> >
>> >> >>> > yip. Myself i only used it for doing things at the beginning, it seems.
>> >> >>> > Do you have or could you try to find an repeatable minimal example?
>> >> >>> >
>> >> >>> > best,
>> >> >>> >
>> >> >>> > holger
>> >> >>> >
>> >> >>>
>> >> >>
>> >> >> --
>> >> >>
>> >> >
>> >>
>> >
>> > --
>> >
>
>> import execnet
>>
>> print "New rsync"
>> rsync = execnet.RSync( "." )
>> print "Making gateway"
>> gw = execnet.makegateway()
>> print "Execute echo"
>> chan=gw.remote_exec( "channel.send( channel.receive() )" )
>> #chan.send( "Echo" )
>> #assert chan.receive() == "Echo"
>> print "Add rsync target"
>> rsync.add_target( gw, "/tmp/rsync" )
>> print "Rsync send"
>> rsync.send()
>
>
> --
>


From holger at merlinux.eu  Thu Aug  5 23:17:29 2010
From: holger at merlinux.eu (holger krekel)
Date: Thu, 5 Aug 2010 23:17:29 +0200
Subject: [execnet-dev] RSync Issue
In-Reply-To: <AANLkTikwRrjJDV2sma+vi6PCwLvcPDA+9aewSCsecGT_@mail.gmail.com>
References: <AANLkTilS-Xxx1uxslsFvb0Wpu9bOJZjm-dFtGpPHkFJm@mail.gmail.com>
	<20100711103618.GT14601@trillke.net>
	<AANLkTikXHQd1PUR8DGSnpveOqkHKWkXpi1SzNOGRGEIF@mail.gmail.com>
	<20100727174748.GM14601@trillke.net>
	<AANLkTin=JyPRKW8RcDSOsng8iRtSAXKVaJiJbwhXx4d6@mail.gmail.com>
	<AANLkTinRezN-nd70Lu=NQZnnWVnf9cYQt0YOPska1EiK@mail.gmail.com>
	<20100805165837.GV1914@trillke.net>
	<AANLkTikD=-vhUO59R6BApLzY9aW6Lm6mWQ5Ms-PbJg1e@mail.gmail.com>
	<20100805183152.GW1914@trillke.net>
	<AANLkTikwRrjJDV2sma+vi6PCwLvcPDA+9aewSCsecGT_@mail.gmail.com>
Message-ID: <20100805211729.GA1914@trillke.net>

On Thu, Aug 05, 2010 at 15:45 -0500, Charles Solar wrote:
> Ah, that makes sense. However, twisted's reactor requires that its run
> in the main app loop, not a new thread.  Is there a way to get my
> remote_exec module to run in the main app thread and still get a
> thread for serving rync data?
> 
> ( I tried using  that code and twisted actually spit back
> "exceptions.ValueError: signal only works in main thread" at me )

Hum, we would need to refactor the remote_init_thread and threadpool 
implementation which might make sense also for other reasons (i'd like
to see the execnet-green experiment evolve and maybe merge with
execnet at some point). 

I cannot think of an immediate way - is it maybe possible for you to 
rather do the rsync before starting twisted or to just
open another (short-lived) gateway? 

best,
holger

> Charles
> 
> On Thu, Aug 5, 2010 at 1:31 PM, holger krekel <holger at merlinux.eu> wrote:
> > Hey Charles,
> >
> > On Thu, Aug 05, 2010 at 12:46 -0500, Charles Solar wrote:
> >> Oh my, it appears in my haste to get a fail test case I mis-remembered
> >> what you were actually asking for. ?I had completely forgotten about
> >> the original rsync on an old gateway does not work problem I was
> >> having. ?That file was meant to demonstrate the symlink error :P
> >
> > np, so we can consider the rsync symlink issue solved i guess :)
> >
> >> Here is a python file that demonstrates the ORIGINAL rsync problem I
> >> was having. ?Turns out its not exactly 'an old gateway' like I said
> >> earlier. ?Its a gateway where a remote channel is still executing
> >> code. ?In my programs, I start a twisted reactor on the remote
> >> machines, so the gateway's remote code never really 'completes.' ?This
> >> causes rsync to fail to use that gateway for the rsync channel.
> >> In this demo file you can see this by first running it and watching it
> >> freeze on rsync send. ?Then, uncomment the chan.send and chan.receive
> >> lines and it will work.
> >
> > It's expected behaviour. ?If you do:
> >
> > ? ?gw.remote_init_threads(num=2)
> >
> > just after creating the gateway you can have two concurrent executions
> > and the rsync will work.
> >
> > best,
> > holger
> >
> >> hopefully this attachment works.
> >>
> >> Charles
> >>
> >> On Thu, Aug 5, 2010 at 11:58 AM, holger krekel <holger at merlinux.eu> wrote:
> >> > Hi Charles,
> >> >
> >> > On Wed, Aug 04, 2010 at 15:34 -0500, Charles Solar wrote:
> >> >> Ok I got that fail case for you, I uploaded it to rapidshare since my
> >> >> last attachment did not work
> >> >> http://rapidshare.com/files/411056579/rsync_fail.tar.gz.html
> >> >
> >> > with execnet-1.0.8 (containing the rsync fixes) this seems to work for me
> >> > unless i am missing something?
> >> >
> >> >> Also, I am going to submit a change to you on bitbucket, I added a new
> >> >> option for sshgateways to programatically use ssh options without
> >> >> having to give a ssh config file. ?I found sending an actual file was
> >> >> not very platform independent with paths and all that.
> >> >> I am using it to tell ssh to shutup about rsa identification like so
> >> >> gw = execnet.makegateway(
> >> >> "ssh=%s@%s//ssh_options=StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null"
> >> >> % ( host['user'], host['ip'] ))
> >> >
> >> > sounds good. ?I wonder if it's also possible to append the option
> >> > to the actual ssh=* spec. We'd need a safe separator but maybe
> >> > ";" is reasonably safe so it could read:
> >> >
> >> > ? ?ssh=user at host;StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null
> >> >
> >> > looking forward to your patch :)
> >> >
> >> > best,
> >> > holger
> >> >
> >> >>
> >> >> Charles
> >> >>
> >> >> On Tue, Jul 27, 2010 at 5:14 PM, Charles Solar <charlessolar at gmail.com> wrote:
> >> >> > Sorry could not get around to testing this out today, i will get this
> >> >> > installed next time I have a chance to run my app.
> >> >> > Oh yea, completely forgot about the test case, I will get a small
> >> >> > package setup to illustrate too.
> >> >> >
> >> >> > Charles
> >> >> >
> >> >> > On Tue, Jul 27, 2010 at 12:47 PM, holger krekel <holger at merlinux.eu> wrote:
> >> >> >> Hi Charles,
> >> >> >>
> >> >> >> so i think that rsyncing symlinks should now work reaonsably well.
> >> >> >> I used your patches as a starting point but also added tests and made
> >> >> >> sure permissions are also transferred for directories. ?Could you verify
> >> >> >> that after installing
> >> >> >>
> >> >> >> ? ?http://hudson.testrun.org/view/pytest/job/execnet-sdist/lastSuccessfulBuild/artifact/.tox/dist/execnet-1.0.8a1.zip
> >> >> >>
> >> >> >> working with symlinks works for you?
> >> >> >>
> >> >> >> Also, i am still interested in a small failing test case for the other issue
> >> >> >> where you could not use RSync later on.
> >> >> >>
> >> >> >> thanks & best,
> >> >> >> holger
> >> >> >>
> >> >> >> On Mon, Jul 12, 2010 at 17:16 -0500, Charles Solar wrote:
> >> >> >>> I'll make up a small test case but just so I do not forgot I found
> >> >> >>> another bug dealing with symlinks.
> >> >> >>> I think that all relative symlinks are broken, or else I do not
> >> >> >>> understand how they could have worked before.
> >> >> >>> I saw it be having a structure like this rsync:
> >> >> >>>
> >> >> >>> lib/
> >> >> >>> ? ?libnet.so -> libnet.so.1.0.0
> >> >> >>> ? ?libnet.so.1.0.0
> >> >> >>>
> >> >> >>> What it created on the remote side was
> >> >> >>>
> >> >> >>> lib/
> >> >> >>> ? libnet.so -> /tmp/rsynctemp/libnet.so.1.0.0
> >> >> >>> ? libnet.so.1.0.0
> >> >> >>>
> >> >> >>> The link on the remote side did not include the lib subdir in the
> >> >> >>> link. ?After looking at the code I fixed it up but I do not think they
> >> >> >>> should be working at all since the code atm only works if the symlink
> >> >> >>> is in the root dest directory.
> >> >> >>>
> >> >> >>> Ill send in a patch for you to look at.
> >> >> >>>
> >> >> >>> Charles
> >> >> >>>
> >> >> >>> On Sun, Jul 11, 2010 at 5:36 AM, holger krekel <holger at merlinux.eu> wrote:
> >> >> >>> > Hi Charles,
> >> >> >>> >
> >> >> >>> > On Fri, Jul 09, 2010 at 10:03 -0500, Charles Solar wrote:
> >> >> >>> >> It would appear that RSync will only work with brand new gateways that
> >> >> >>> >> have not had any other operations run on them. ?Is this intentional?
> >> >> >>> >
> >> >> >>> > No. ?Sounds like a bug.
> >> >> >>> >
> >> >> >>> >> For example, I do something like
> >> >> >>> >>
> >> >> >>> >> gw = execnet.makegateway( "ssh=foobar" )
> >> >> >>> >> gw.remote_exec( mybootstrapper )
> >> >> >>> >>
> >> >> >>> >> ... then later I want to rsync ...
> >> >> >>> >>
> >> >> >>> >> rsync = execnet.RSync( "/tmp/foobar" )
> >> >> >>> >> rsync.add_target( gw, "/tmp/barfoo" )
> >> >> >>> >> rsync.send()
> >> >> >>> >>
> >> >> >>> >> I get remote EOFErrors
> >> >> >>> >>
> >> >> >>> >> File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 126, in send
> >> >> >>> >> ? ? self._end_of_channel(channel)
> >> >> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 44, in _end_of_channel
> >> >> >>> >> ? ? channel.waitclose()
> >> >> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/gateway_base.py", line 377, in waitclose
> >> >> >>> >> ? ? raise error
> >> >> >>> >> exceptions.EOFError: expected 1 bytes, got 0
> >> >> >>> >>
> >> >> >>> >> If I however create a brand new gateway to give to rsync it works fine.
> >> >> >>> >>
> >> >> >>> >> I would think that since rsync just creates a new channel on the
> >> >> >>> >> gateway that it should work on existing 'old' gateways.
> >> >> >>> >
> >> >> >>> > yip. Myself i only used it for doing things at the beginning, it seems.
> >> >> >>> > Do you have or could you try to find an repeatable minimal example?
> >> >> >>> >
> >> >> >>> > best,
> >> >> >>> >
> >> >> >>> > holger
> >> >> >>> >
> >> >> >>>
> >> >> >>
> >> >> >> --
> >> >> >>
> >> >> >
> >> >>
> >> >
> >> > --
> >> >
> >
> >> import execnet
> >>
> >> print "New rsync"
> >> rsync = execnet.RSync( "." )
> >> print "Making gateway"
> >> gw = execnet.makegateway()
> >> print "Execute echo"
> >> chan=gw.remote_exec( "channel.send( channel.receive() )" )
> >> #chan.send( "Echo" )
> >> #assert chan.receive() == "Echo"
> >> print "Add rsync target"
> >> rsync.add_target( gw, "/tmp/rsync" )
> >> print "Rsync send"
> >> rsync.send()
> >
> >
> > --
> >
> 

-- 


From charlessolar at gmail.com  Thu Aug  5 23:57:06 2010
From: charlessolar at gmail.com (Charles Solar)
Date: Thu, 5 Aug 2010 16:57:06 -0500
Subject: [execnet-dev] RSync Issue
In-Reply-To: <20100805211729.GA1914@trillke.net>
References: <AANLkTilS-Xxx1uxslsFvb0Wpu9bOJZjm-dFtGpPHkFJm@mail.gmail.com> 
	<20100711103618.GT14601@trillke.net>
	<AANLkTikXHQd1PUR8DGSnpveOqkHKWkXpi1SzNOGRGEIF@mail.gmail.com> 
	<20100727174748.GM14601@trillke.net>
	<AANLkTin=JyPRKW8RcDSOsng8iRtSAXKVaJiJbwhXx4d6@mail.gmail.com> 
	<AANLkTinRezN-nd70Lu=NQZnnWVnf9cYQt0YOPska1EiK@mail.gmail.com> 
	<20100805165837.GV1914@trillke.net>
	<AANLkTikD=-vhUO59R6BApLzY9aW6Lm6mWQ5Ms-PbJg1e@mail.gmail.com> 
	<20100805183152.GW1914@trillke.net>
	<AANLkTikwRrjJDV2sma+vi6PCwLvcPDA+9aewSCsecGT_@mail.gmail.com> 
	<20100805211729.GA1914@trillke.net>
Message-ID: <AANLkTi=orWo2Aiv9b+JLOxD9NyFPhfV=YmWWN19uHnGa@mail.gmail.com>

Yea opening up a small temporary gateway when I rsync is what I have
been doing.  Works fine, but on windows it can get a little clunky
when I try syncing to 10 different machines and end up with 20 ssh
processes haha.

I want to spend more time with execnet adding and tweaking some of the
things I have encountered while developing my app with it.
Unfortunately its almost always crunch time where I work and my app
needs to be up and running asap.  Once my app is stable and everyone
loves it :) I will be able to devote more time to fixing some of the
back end stuff.  Right now it just needs to work.

Which speaking of windows, what are your opinions on using a library
like paramiko to connect to remote sshd machines instead of popen ssh?
(http://www.lag.net/paramiko/)
I found on windows opening ssh gateways can be a real pain in the ass.
 Plink does not setup the new session as I wanted so it could not be a
simple drop in replacement for ssh, which forced me to install cygwin,
something I really did not want to do.

While searching for plink alternatives I found this library and
thought that it would be very nice for execnet to be able to open ssh
gateways on windows without cygwin or plink.

Charles


On Thu, Aug 5, 2010 at 4:17 PM, holger krekel <holger at merlinux.eu> wrote:
> On Thu, Aug 05, 2010 at 15:45 -0500, Charles Solar wrote:
>> Ah, that makes sense. However, twisted's reactor requires that its run
>> in the main app loop, not a new thread. ?Is there a way to get my
>> remote_exec module to run in the main app thread and still get a
>> thread for serving rync data?
>>
>> ( I tried using ?that code and twisted actually spit back
>> "exceptions.ValueError: signal only works in main thread" at me )
>
> Hum, we would need to refactor the remote_init_thread and threadpool
> implementation which might make sense also for other reasons (i'd like
> to see the execnet-green experiment evolve and maybe merge with
> execnet at some point).
>
> I cannot think of an immediate way - is it maybe possible for you to
> rather do the rsync before starting twisted or to just
> open another (short-lived) gateway?
>
> best,
> holger
>
>> Charles
>>
>> On Thu, Aug 5, 2010 at 1:31 PM, holger krekel <holger at merlinux.eu> wrote:
>> > Hey Charles,
>> >
>> > On Thu, Aug 05, 2010 at 12:46 -0500, Charles Solar wrote:
>> >> Oh my, it appears in my haste to get a fail test case I mis-remembered
>> >> what you were actually asking for. ?I had completely forgotten about
>> >> the original rsync on an old gateway does not work problem I was
>> >> having. ?That file was meant to demonstrate the symlink error :P
>> >
>> > np, so we can consider the rsync symlink issue solved i guess :)
>> >
>> >> Here is a python file that demonstrates the ORIGINAL rsync problem I
>> >> was having. ?Turns out its not exactly 'an old gateway' like I said
>> >> earlier. ?Its a gateway where a remote channel is still executing
>> >> code. ?In my programs, I start a twisted reactor on the remote
>> >> machines, so the gateway's remote code never really 'completes.' ?This
>> >> causes rsync to fail to use that gateway for the rsync channel.
>> >> In this demo file you can see this by first running it and watching it
>> >> freeze on rsync send. ?Then, uncomment the chan.send and chan.receive
>> >> lines and it will work.
>> >
>> > It's expected behaviour. ?If you do:
>> >
>> > ? ?gw.remote_init_threads(num=2)
>> >
>> > just after creating the gateway you can have two concurrent executions
>> > and the rsync will work.
>> >
>> > best,
>> > holger
>> >
>> >> hopefully this attachment works.
>> >>
>> >> Charles
>> >>
>> >> On Thu, Aug 5, 2010 at 11:58 AM, holger krekel <holger at merlinux.eu> wrote:
>> >> > Hi Charles,
>> >> >
>> >> > On Wed, Aug 04, 2010 at 15:34 -0500, Charles Solar wrote:
>> >> >> Ok I got that fail case for you, I uploaded it to rapidshare since my
>> >> >> last attachment did not work
>> >> >> http://rapidshare.com/files/411056579/rsync_fail.tar.gz.html
>> >> >
>> >> > with execnet-1.0.8 (containing the rsync fixes) this seems to work for me
>> >> > unless i am missing something?
>> >> >
>> >> >> Also, I am going to submit a change to you on bitbucket, I added a new
>> >> >> option for sshgateways to programatically use ssh options without
>> >> >> having to give a ssh config file. ?I found sending an actual file was
>> >> >> not very platform independent with paths and all that.
>> >> >> I am using it to tell ssh to shutup about rsa identification like so
>> >> >> gw = execnet.makegateway(
>> >> >> "ssh=%s@%s//ssh_options=StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null"
>> >> >> % ( host['user'], host['ip'] ))
>> >> >
>> >> > sounds good. ?I wonder if it's also possible to append the option
>> >> > to the actual ssh=* spec. We'd need a safe separator but maybe
>> >> > ";" is reasonably safe so it could read:
>> >> >
>> >> > ? ?ssh=user at host;StrictHostKeyChecking=no;UserKnownHostsFile=/dev/null
>> >> >
>> >> > looking forward to your patch :)
>> >> >
>> >> > best,
>> >> > holger
>> >> >
>> >> >>
>> >> >> Charles
>> >> >>
>> >> >> On Tue, Jul 27, 2010 at 5:14 PM, Charles Solar <charlessolar at gmail.com> wrote:
>> >> >> > Sorry could not get around to testing this out today, i will get this
>> >> >> > installed next time I have a chance to run my app.
>> >> >> > Oh yea, completely forgot about the test case, I will get a small
>> >> >> > package setup to illustrate too.
>> >> >> >
>> >> >> > Charles
>> >> >> >
>> >> >> > On Tue, Jul 27, 2010 at 12:47 PM, holger krekel <holger at merlinux.eu> wrote:
>> >> >> >> Hi Charles,
>> >> >> >>
>> >> >> >> so i think that rsyncing symlinks should now work reaonsably well.
>> >> >> >> I used your patches as a starting point but also added tests and made
>> >> >> >> sure permissions are also transferred for directories. ?Could you verify
>> >> >> >> that after installing
>> >> >> >>
>> >> >> >> ? ?http://hudson.testrun.org/view/pytest/job/execnet-sdist/lastSuccessfulBuild/artifact/.tox/dist/execnet-1.0.8a1.zip
>> >> >> >>
>> >> >> >> working with symlinks works for you?
>> >> >> >>
>> >> >> >> Also, i am still interested in a small failing test case for the other issue
>> >> >> >> where you could not use RSync later on.
>> >> >> >>
>> >> >> >> thanks & best,
>> >> >> >> holger
>> >> >> >>
>> >> >> >> On Mon, Jul 12, 2010 at 17:16 -0500, Charles Solar wrote:
>> >> >> >>> I'll make up a small test case but just so I do not forgot I found
>> >> >> >>> another bug dealing with symlinks.
>> >> >> >>> I think that all relative symlinks are broken, or else I do not
>> >> >> >>> understand how they could have worked before.
>> >> >> >>> I saw it be having a structure like this rsync:
>> >> >> >>>
>> >> >> >>> lib/
>> >> >> >>> ? ?libnet.so -> libnet.so.1.0.0
>> >> >> >>> ? ?libnet.so.1.0.0
>> >> >> >>>
>> >> >> >>> What it created on the remote side was
>> >> >> >>>
>> >> >> >>> lib/
>> >> >> >>> ? libnet.so -> /tmp/rsynctemp/libnet.so.1.0.0
>> >> >> >>> ? libnet.so.1.0.0
>> >> >> >>>
>> >> >> >>> The link on the remote side did not include the lib subdir in the
>> >> >> >>> link. ?After looking at the code I fixed it up but I do not think they
>> >> >> >>> should be working at all since the code atm only works if the symlink
>> >> >> >>> is in the root dest directory.
>> >> >> >>>
>> >> >> >>> Ill send in a patch for you to look at.
>> >> >> >>>
>> >> >> >>> Charles
>> >> >> >>>
>> >> >> >>> On Sun, Jul 11, 2010 at 5:36 AM, holger krekel <holger at merlinux.eu> wrote:
>> >> >> >>> > Hi Charles,
>> >> >> >>> >
>> >> >> >>> > On Fri, Jul 09, 2010 at 10:03 -0500, Charles Solar wrote:
>> >> >> >>> >> It would appear that RSync will only work with brand new gateways that
>> >> >> >>> >> have not had any other operations run on them. ?Is this intentional?
>> >> >> >>> >
>> >> >> >>> > No. ?Sounds like a bug.
>> >> >> >>> >
>> >> >> >>> >> For example, I do something like
>> >> >> >>> >>
>> >> >> >>> >> gw = execnet.makegateway( "ssh=foobar" )
>> >> >> >>> >> gw.remote_exec( mybootstrapper )
>> >> >> >>> >>
>> >> >> >>> >> ... then later I want to rsync ...
>> >> >> >>> >>
>> >> >> >>> >> rsync = execnet.RSync( "/tmp/foobar" )
>> >> >> >>> >> rsync.add_target( gw, "/tmp/barfoo" )
>> >> >> >>> >> rsync.send()
>> >> >> >>> >>
>> >> >> >>> >> I get remote EOFErrors
>> >> >> >>> >>
>> >> >> >>> >> File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 126, in send
>> >> >> >>> >> ? ? self._end_of_channel(channel)
>> >> >> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/rsync.py", line 44, in _end_of_channel
>> >> >> >>> >> ? ? channel.waitclose()
>> >> >> >>> >> ? File "execnet-1.0.6-py2.7.egg/execnet/gateway_base.py", line 377, in waitclose
>> >> >> >>> >> ? ? raise error
>> >> >> >>> >> exceptions.EOFError: expected 1 bytes, got 0
>> >> >> >>> >>
>> >> >> >>> >> If I however create a brand new gateway to give to rsync it works fine.
>> >> >> >>> >>
>> >> >> >>> >> I would think that since rsync just creates a new channel on the
>> >> >> >>> >> gateway that it should work on existing 'old' gateways.
>> >> >> >>> >
>> >> >> >>> > yip. Myself i only used it for doing things at the beginning, it seems.
>> >> >> >>> > Do you have or could you try to find an repeatable minimal example?
>> >> >> >>> >
>> >> >> >>> > best,
>> >> >> >>> >
>> >> >> >>> > holger
>> >> >> >>> >
>> >> >> >>>
>> >> >> >>
>> >> >> >> --
>> >> >> >>
>> >> >> >
>> >> >>
>> >> >
>> >> > --
>> >> >
>> >
>> >> import execnet
>> >>
>> >> print "New rsync"
>> >> rsync = execnet.RSync( "." )
>> >> print "Making gateway"
>> >> gw = execnet.makegateway()
>> >> print "Execute echo"
>> >> chan=gw.remote_exec( "channel.send( channel.receive() )" )
>> >> #chan.send( "Echo" )
>> >> #assert chan.receive() == "Echo"
>> >> print "Add rsync target"
>> >> rsync.add_target( gw, "/tmp/rsync" )
>> >> print "Rsync send"
>> >> rsync.send()
>> >
>> >
>> > --
>> >
>>
>
> --
>


From holger at merlinux.eu  Fri Aug  6 11:41:59 2010
From: holger at merlinux.eu (holger krekel)
Date: Fri, 6 Aug 2010 11:41:59 +0200
Subject: [execnet-dev] RSync Issue
In-Reply-To: <AANLkTi=orWo2Aiv9b+JLOxD9NyFPhfV=YmWWN19uHnGa@mail.gmail.com>
References: <AANLkTikXHQd1PUR8DGSnpveOqkHKWkXpi1SzNOGRGEIF@mail.gmail.com>
	<20100727174748.GM14601@trillke.net>
	<AANLkTin=JyPRKW8RcDSOsng8iRtSAXKVaJiJbwhXx4d6@mail.gmail.com>
	<AANLkTinRezN-nd70Lu=NQZnnWVnf9cYQt0YOPska1EiK@mail.gmail.com>
	<20100805165837.GV1914@trillke.net>
	<AANLkTikD=-vhUO59R6BApLzY9aW6Lm6mWQ5Ms-PbJg1e@mail.gmail.com>
	<20100805183152.GW1914@trillke.net>
	<AANLkTikwRrjJDV2sma+vi6PCwLvcPDA+9aewSCsecGT_@mail.gmail.com>
	<20100805211729.GA1914@trillke.net>
	<AANLkTi=orWo2Aiv9b+JLOxD9NyFPhfV=YmWWN19uHnGa@mail.gmail.com>
Message-ID: <20100806094159.GC1914@trillke.net>

Hi Charles, 

On Thu, Aug 05, 2010 at 16:57 -0500, Charles Solar wrote:
> Yea opening up a small temporary gateway when I rsync is what I have
> been doing.  Works fine, but on windows it can get a little clunky
> when I try syncing to 10 different machines and end up with 20 ssh
> processes haha.
> 
> I want to spend more time with execnet adding and tweaking some of the
> things I have encountered while developing my app with it.
> Unfortunately its almost always crunch time where I work and my app
> needs to be up and running asap.  Once my app is stable and everyone
> loves it :) I will be able to devote more time to fixing some of the
> back end stuff.  Right now it just needs to work.

sure, feel free to continue to ask for any workaround you need :) 
Also happy to collaborate on improving things later ... 
(right now i am about to drop to holiday for 2 weeks with occassional
email connect)
 
> Which speaking of windows, what are your opinions on using a library
> like paramiko to connect to remote sshd machines instead of popen ssh?
> (http://www.lag.net/paramiko/)
> I found on windows opening ssh gateways can be a real pain in the ass.
>  Plink does not setup the new session as I wanted so it could not be a
> simple drop in replacement for ssh, which forced me to install cygwin,
> something I really did not want to do.
> 
> While searching for plink alternatives I found this library and
> thought that it would be very nice for execnet to be able to open ssh
> gateways on windows without cygwin or plink.

I guess paramiko could be especially helpful on windows.  On Linux/OSX 
using the ssh binary goes a long way and allows us to treat things in 
a black-box manner, i.e. remain blissfully ignorant :) 
ASFAIK paramiko is a lower level and will need looking into APIs, 
handling config, starting threads (?) and such.  But i haven't looked deep.

best,
holger