From holger at merlinux.eu Thu Aug 18 10:05:58 2011 From: holger at merlinux.eu (holger krekel) Date: Thu, 18 Aug 2011 08:05:58 +0000 Subject: [execnet-dev] Internal errors calling channel.send In-Reply-To: <9418D43F-C247-4CC7-B3E5-38BA0F97D0E0@sandia.gov> References: <43D2F479-6897-49FD-8904-91F081A37AAB@sandia.gov> <9418D43F-C247-4CC7-B3E5-38BA0F97D0E0@sandia.gov> Message-ID: <20110818080558.GG12151@merlinux.eu> Hi Timothy, i presume this problem does not occur with CPython or PyPy versions, does it? I wonder if Jython maybe has some problem with socket-sending/receiving code or if there are race conditions between the receiving and sending thread (Jython has free threading - the other python implementations do not). Not sure i can help much but could you send the jython and java versions you are using? best, holger On Thu, Jul 28, 2011 at 22:26 +0000, Shead, Timothy wrote: > > On Jul 28, 2011, at 4:14 PM, Shead, Timothy wrote: > > > > > On Jul 28, 2011, at 3:11 PM, Shead, Timothy wrote: > > > >> Hello, List! > >> > >> I'm using exenet 1.0.9 with Python 2.7 on a Mac OSX workstation. Execnet has generally been very reliable and easy to use, but I'm having trouble using it to send large "jobs" to worker processes. The size of the serialized data for each job varies, but using EXECNET_DEBUG=2 I see that they're all around 2500000 bytes. I get an intermittent (doesn't happen for the same job) error in the receiving thread: > > > > Forgot to mention that my main application is running in Python, while the worker processes are running in Jython. Are there any known issues connecting to Jython? > > Sure 'nuff, here's a test case that always hangs in the same spot: > > def stress_server(channel): > for request in channel: > channel.send(request) > > gateway = execnet.makegateway("popen//python=jython") > channel = gateway.remote_exec(stress_server) > > for size in range(31): > sys.stderr.write("Sending string length %d\n" % 2**size) > original = "a" * (2**size) > channel.send(original) > response = channel.receive() > self.assertEqual(original, response) > > ... running it with EXECNET_DEBUG=2 yields the following: > > [68941] RECEIVERTHREAD: starting to run > [68941] sent > Sending string length 1 > [68941] sent > [68947] creating slavegateway on > [68947] RECEIVERTHREAD: starting to run > [68947] received > [68947] execution starts[1]: 'def stress_server(channel):\n for request in cha > [68947] received > [68947] calling stress_server(** {}) > [68947] sent > [68941] received > Sending string length 2 > [68941] sent > [68947] received > [68947] sent > [68941] received > Sending string length 4 > [68941] sent > [68947] received > [68941] received > [68947] sent > Sending string length 8 > [68941] sent > [68947] received > [68941] received > [68947] sent > Sending string length 16 > [68941] sent > [68947] received > [68947] sent > [68941] received > Sending string length 32 > [68941] sent > [68947] received > [68941] received > [68947] sent > Sending string length 64 > [68941] sent > [68947] received > [68941] received > [68947] sent > Sending string length 128 > [68941] sent > [68947] received > [68941] received > [68947] sent > Sending string length 256 > [68941] sent > [68947] received > [68941] received > [68947] sent > Sending string length 512 > [68941] sent > [68947] received > [68941] received > [68947] sent > Sending string length 1024 > [68941] sent > [68947] received > [68941] received > [68947] sent > Sending string length 2048 > [68941] sent > [68947] received > [68941] received > Sending string length 4096 > [68941] sent > [68947] sent > [68947] received > [68941] received > [68947] sent > Sending string length 8192 > [68941] sent > [68947] received > [68941] received > [68947] sent > Sending string length 16384 > [68941] sent > > Any thoughts? > > Cheers, > Tim > > Timothy M. Shead > Sandia National Laboratories > 1461, Scalable Analysis and Visualization > > > _______________________________________________ > execnet-dev mailing list > execnet-dev at codespeak.net > http://codespeak.net/mailman/listinfo/execnet-dev > From tshead at sandia.gov Fri Aug 19 17:19:18 2011 From: tshead at sandia.gov (Shead, Timothy) Date: Fri, 19 Aug 2011 15:19:18 +0000 Subject: [execnet-dev] Internal errors calling channel.send In-Reply-To: <20110818080558.GG12151@merlinux.eu> References: <43D2F479-6897-49FD-8904-91F081A37AAB@sandia.gov> <9418D43F-C247-4CC7-B3E5-38BA0F97D0E0@sandia.gov> <20110818080558.GG12151@merlinux.eu> Message-ID: <7FC78FC6-EAE0-4AA0-9D35-4B0CB02ABFA4@sandia.gov> On Aug 18, 2011, at 2:05 AM, holger krekel wrote: >> Sure 'nuff, here's a test case that always hangs in the same spot: >> >> def stress_server(channel): >> for request in channel: >> channel.send(request) >> >> gateway = execnet.makegateway("popen//python=jython") >> channel = gateway.remote_exec(stress_server) >> >> for size in range(31): >> sys.stderr.write("Sending string length %d\n" % 2**size) >> original = "a" * (2**size) >> channel.send(original) >> response = channel.receive() >> self.assertEqual(original, response) > Hi Timothy, > > i presume this problem does not occur with CPython or PyPy versions, does it Correct, this problem only manifests when using CPython <--> Jython. If I test CPython <--> CPython, I'm able to pass huge amounts of data back-and-forth without problem. > I wonder if Jython maybe has some problem with socket-sending/receiving code > or if there are race conditions between the receiving and sending thread > (Jython has free threading - the other python implementations do not). > > Not sure i can help much but could you send the jython and java versions > you are using? Sure, this is Jython running on an OSX 10.6 workstation, installed with MacPorts: Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) [Java HotSpot(TM) 64-Bit Server VM (Apple Inc.)] on java1.6.0_24 Cheers, Tim Timothy M. Shead Sandia National Laboratories 1461, Scalable Analysis and Visualization From holger at merlinux.eu Sat Aug 20 18:46:17 2011 From: holger at merlinux.eu (holger krekel) Date: Sat, 20 Aug 2011 16:46:17 +0000 Subject: [execnet-dev] Internal errors calling channel.send In-Reply-To: <7FC78FC6-EAE0-4AA0-9D35-4B0CB02ABFA4@sandia.gov> References: <43D2F479-6897-49FD-8904-91F081A37AAB@sandia.gov> <9418D43F-C247-4CC7-B3E5-38BA0F97D0E0@sandia.gov> <20110818080558.GG12151@merlinux.eu> <7FC78FC6-EAE0-4AA0-9D35-4B0CB02ABFA4@sandia.gov> Message-ID: <20110820164617.GL12151@merlinux.eu> On Fri, Aug 19, 2011 at 15:19 +0000, Shead, Timothy wrote: > On Aug 18, 2011, at 2:05 AM, holger krekel wrote: > > >> Sure 'nuff, here's a test case that always hangs in the same spot: > >> > >> def stress_server(channel): > >> for request in channel: > >> channel.send(request) > >> > >> gateway = execnet.makegateway("popen//python=jython") > >> channel = gateway.remote_exec(stress_server) > >> > >> for size in range(31): > >> sys.stderr.write("Sending string length %d\n" % 2**size) > >> original = "a" * (2**size) > >> channel.send(original) > >> response = channel.receive() > >> self.assertEqual(original, response) > > > > Hi Timothy, > > > > i presume this problem does not occur with CPython or PyPy versions, does it > > Correct, this problem only manifests when using CPython <--> Jython. If I test CPython <--> CPython, I'm able to pass huge amounts of data back-and-forth without problem. > > > I wonder if Jython maybe has some problem with socket-sending/receiving code > > or if there are race conditions between the receiving and sending thread > > (Jython has free threading - the other python implementations do not). > > > > Not sure i can help much but could you send the jython and java versions > > you are using? > > Sure, this is Jython running on an OSX 10.6 workstation, installed with MacPorts: > > Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) > [Java HotSpot(TM) 64-Bit Server VM (Apple Inc.)] on java1.6.0_24 Could you maybe try to come up with an example where two threads concurrently send/receive large data over a socket using Jython? I suspect this will already trigger the errors and would be something that could be pushed to the Jython devs. Alternatively you could drop to the #jython channel and ask around there - maybe it's a known problem anyway. best, holger > Cheers, > Tim > > Timothy M. Shead > Sandia National Laboratories > 1461, Scalable Analysis and Visualization >