From holger at merlinux.eu Sat Dec 5 19:41:34 2009 From: holger at merlinux.eu (holger krekel) Date: Sat, 5 Dec 2009 19:41:34 +0100 Subject: [execnet-dev] execnet-1.0.1: more robust and rapid python deployment Message-ID: <20091205184134.GA19640@trillke.net> Hi everybody, Just uploaded execnet-1.0.1 featuring a new motto: execnet is about rapid-python deployment, be it for multiple CPUs, different platforms or python versions. This release brings a bunch of refinements and most importantly more robust termination, handling of CTRL-C and automatically tested documentation:: http://codespeak.net/execnet have fun, holger 1.0.1 -------------------------------- - revamp and better structure documentation - new method: gateway.hasreceiver() returns True if the gateway is still receive-active. remote_status now only carries information about remote execution status. - new: execnet.MultiChannel provides basic iteration/contain interface - new: execnet.Group can be indexed by integer - new: group.makegateway() uses group.default_spec if no spec is given and the execnet.default_group uses ``popen`` as a default spec. - have popen-gateways use imports instead of source-strings, also improves debugging/tracebacks, as a side effect popen-gateway startup can be substantially faster (>30%) - refine internal gateway exit/termination procedure and introduce group.terminate(timeout) which will attempt to kill all subprocesses that did not terminate within time. - EOFError on channel.receive/waitclose if the other side unexpectedly went away. When a gateway exits it now internally sends an explicit termination message instead of abruptly closing. - introduce a timeout parameter to channel.receive() and default to periodically internally wake up to let KeyboardInterrupts pass through. - EXECNET_DEBUG=2 will cause tracing to go to stderr, which with popen slave gateways will relay back tracing to the instantiator process. 1.0.0 -------------------------------- * introduce execnet.Group for managing gateway creation and termination. Introduce execnet.default_group through which all "global" calls are routed. cleanup gateway termination. All Gateways get an id through which they can be retrieved from a group object. * deprecate execnet.XYZGateway in favour of direct makegateway() calls. * refine socketserver-examples, experimentally introduce a way to indirectly setup a socket server ("installvia") through a gateway url. * refine and automatically test documentation examples 1.0.0b3 -------------------------------- * fix EXECNET_DEBUG to work with win32 * add support for serializing longs, sets and frozensets (thanks Benjamin Peterson) * introduce remote_status() method which on the low level gives information about the remote side of a gateway * disallow explicit close in remote_exec situation * perform some more detailed tracing with EXECNET_DEBUG 1.0.0b2 -------------------------------- * make internal protocols more robust against serialization failures * fix a seralization bug with nested tuples containing empty tuples (thanks to ronny for discovering it) * setting the environment variable EXECNET_DEBUG will generate per process trace-files for debugging 1.0.0b1 ---------------------------- * added new examples for NumPy, Jython, IronPython * improved documentation * include apipkg.py for lazy-importing * integrated new serializer code from Benjamin Peterson * improved support for Jython-2.5.1 1.0.0alpha2 ---------------------------- * improve documentation, new website * use sphinx for documentation, added boilerplate files and setup.py * fixes for standalone usage, adding boilerplate files * imported py/execnet and made it work standalone ----- End forwarded message ----- -- From holger at merlinux.eu Sun Dec 6 18:18:47 2009 From: holger at merlinux.eu (holger krekel) Date: Sun, 6 Dec 2009 18:18:47 +0100 Subject: [execnet-dev] nltk and execnet In-Reply-To: References: Message-ID: <20091206171846.GE20380@trillke.net> Hi Jacob, many thanks for the NTLK link! Was a bit busy with 1.0.1 and other bits but happily read your entry and also link to it now from the home page :) I hope that the new execnet works even better for you. Maybe http://codespeak.net/execnet/example/test_group.html http://codespeak.net/execnet/example/test_multi.html could be interesting to you, btw. cheers, holger On Sun, Nov 29, 2009 at 10:05 -0800, Jacob Perkins wrote: > In honor of the 1.0.0 release, I just published an article on using NLTK > with execnet. > > http://streamhacker.com/2009/11/29/distributed-nltk-execnet/ > > Jacob > -- > http://streamhacker.com/ > _______________________________________________ > execnet-dev mailing list > execnet-dev at codespeak.net > http://codespeak.net/mailman/listinfo/execnet-dev -- From Adam.Schmalhofer at gmx.de Sat Dec 12 13:38:27 2009 From: Adam.Schmalhofer at gmx.de (Adam) Date: Sat, 12 Dec 2009 13:38:27 +0100 Subject: [execnet-dev] execnet in Debian In-Reply-To: <20091205184134.GA19640@trillke.net> References: <20091205184134.GA19640@trillke.net> Message-ID: <20091212133827.481f388d@deepthought> I packaged execnet for debian and it just got off the new-queue: http://packages.debian.org/sid/python-execnet --Adam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 828 bytes Desc: not available URL: From holger at merlinux.eu Mon Dec 14 10:20:13 2009 From: holger at merlinux.eu (holger krekel) Date: Mon, 14 Dec 2009 10:20:13 +0100 Subject: [execnet-dev] execnet in Debian In-Reply-To: <20091212133827.481f388d@deepthought> References: <20091205184134.GA19640@trillke.net> <20091212133827.481f388d@deepthought> Message-ID: <20091214092013.GK3516@trillke.net> Hi Adam, On Sat, Dec 12, 2009 at 13:38 +0100, Adam wrote: > I packaged execnet for debian and it just got off the new-queue: > > http://packages.debian.org/sid/python-execnet great, thanks! holger From holger at merlinux.eu Wed Dec 23 21:40:51 2009 From: holger at merlinux.eu (holger krekel) Date: Wed, 23 Dec 2009 21:40:51 +0100 Subject: [execnet-dev] execnet-1.0.2: channel-over-channels / bug fixes Message-ID: <20091223204051.GG3516@trillke.net> execnet is a small stable pure-python library for working with local or remote clusters of Python interpreters, with ease. The 1.0.2 release is fully backward compatible and: - introduces a generalized way to send channels over channels - makes gateways more resilient against callback failures - speeds up local gateway creation, now at <50ms on an old 1.5GHZ machine - fixes a bug in channel.receive() which could wrongly timeout More info, new tested examples and links to blog entries here: http://codespeak.net/execnet cheers and i wish you all relaxed days of this decade, holger changes in 1.0.2 -------------------------------- - generalize channel-over-channel sending: you can now have channels anywhere in a data structure (i.e. as an item of a container type). Add according examples. - automatically close a channel when a remote callback raises an exception, makes communication more robust because until now failing callbacks rendered the receiverthread unuseable leaving the remote side in-accessible. - internally split socket gateways, speeds up popen-gateways by 10% (now at <50 milliseconds per-gateway on a 1.5 GHZ machine) - fix bug in channel.receive() that would wrongly raise a TimeoutError after 1000 seconds (thanks Ronny Pfannschmidt)