From jsibre at chironsys.com Thu Feb 12 18:54:45 2004 From: jsibre at chironsys.com (Jason Sibre) Date: Mon Feb 16 10:02:11 2004 Subject: [Medusa-dev] Poor upload (POST) performance with Quixote/Medusa/xmlrpc_handler.collector Message-ID: I'm really targeting the Quixote folks with this message, but Medusa folks may be interested also, so I'm posting to both lists. I've noticed that the upload performance with Medusa and Quixote can be very bad... Especially with larger files (more than a couple hundred KB). I've dug around, and found that the medusa_http.QuixoteHandler uses the medusa.xmlrpc_handler.collector for it's collection dirty work... Well, medusa.xmlrpc_handler.collector uses the 'anti-pattern' of "string = string + string_part" when collecting data, which accounts for the crappy upload performance I've observed. I've modified my copy of medusa.xmlrpc_handler.collector to join a list of strings instead (*), and it's much (MUCH!) better. However, I'm not sure modifying part of Medusa is the best way to go... Maybe we should just provide our own collector class in quixote.medusa_http (they're quite trivial), and leave Medusa alone. OTOH, if anyone uses medusa for XMLRPC... They'd probably appreciate the better POST performance. A possible third option would be to modify QuixoteHandler to use the medusa.script_handler.collector instead, which uses StringIO as the buffer. This might be best (reusing as much existing code as possible, and all that), but I haven't looked into it very hard. Some of the work medusa.xmlrpc_handler.collector does would have to be shifted into the QuixoteHandler class. I'll be happy to provide the patches, whichever way we go, but I'm looking for opinions/preferences on which route to take. Thanks, Jason Sibre * I did some impromptu benchmarking, and learned that the list.append()/"".join(list) technique actually outperforms StringIO and cStringIO for large increments, and falls in between for small increments. For Medusa's purposes (I've observed 4KB chunks going to the collector) the list joining technique is the fastest. I've copied my test results below if anyone cares: 100000 iterations of adding 1 bytes Doing list joiner : 0.45186 seconds (100000 bytes returned) Doing str concat-er: 7.30244 seconds (100000 bytes returned) Doing StringIO : 1.03356 seconds (100000 bytes returned) Doing cStringIO : 0.20682 seconds (100000 bytes returned) 20000 iterations of adding 5 bytes Doing list joiner : 0.09181 seconds (100000 bytes returned) Doing str concat-er: 1.89697 seconds (100000 bytes returned) Doing StringIO : 0.21069 seconds (100000 bytes returned) Doing cStringIO : 0.03587 seconds (100000 bytes returned) 10000 iterations of adding 10 bytes Doing list joiner : 0.04546 seconds (100000 bytes returned) Doing str concat-er: 1.02731 seconds (100000 bytes returned) Doing StringIO : 0.10039 seconds (100000 bytes returned) Doing cStringIO : 0.01793 seconds (100000 bytes returned) 10000 iterations of adding 50 bytes Doing list joiner : 0.04703 seconds (500000 bytes returned) Doing str concat-er: 15.52068 seconds (500000 bytes returned) Doing StringIO : 0.11230 seconds (500000 bytes returned) Doing cStringIO : 0.02438 seconds (500000 bytes returned) 1000 iterations of adding 500 bytes Doing list joiner : 0.00532 seconds (500000 bytes returned) Doing str concat-er: 1.56356 seconds (500000 bytes returned) Doing StringIO : 0.01097 seconds (500000 bytes returned) Doing cStringIO : 0.00969 seconds (500000 bytes returned) 1000 iterations of adding 1000 bytes Doing list joiner : 0.00734 seconds (1000000 bytes returned) Doing str concat-er: 3.49345 seconds (1000000 bytes returned) Doing StringIO : 0.01283 seconds (1000000 bytes returned) Doing cStringIO : 0.02518 seconds (1000000 bytes returned) These are all 'wallclock' measurements, just what an anxious web-surfer is paying attention to, and the measurements include the "".join(list) or x.getvalue() calls (to be fair). From fawcett at uwindsor.ca Mon Feb 16 09:24:40 2004 From: fawcett at uwindsor.ca (Graham Fawcett) Date: Mon Feb 16 10:02:12 2004 Subject: [Medusa-dev] Re: [Quixote-users] Poor upload (POST) performance with Quixote/Medusa/xmlrpc_handler.collector In-Reply-To: References: Message-ID: <4030D2A8.6040607@uwindsor.ca> Jason Sibre wrote: >I'm really targeting the Quixote folks with this message, but Medusa folks >may be interested also, so I'm posting to both lists. > >I've noticed that the upload performance with Medusa and Quixote can be very >bad... Especially with larger files (more than a couple hundred KB). > >I've dug around, and found that the medusa_http.QuixoteHandler uses the >medusa.xmlrpc_handler.collector for it's collection dirty work... Well, >medusa.xmlrpc_handler.collector uses the 'anti-pattern' of "string = string >+ string_part" when collecting data, which accounts for the crappy upload >performance I've observed. > >I've modified my copy of medusa.xmlrpc_handler.collector to join a list of >strings instead (*), and it's much (MUCH!) better. > >However, I'm not sure modifying part of Medusa is the best way to go... >Maybe we should just provide our own collector class in quixote.medusa_http >(they're quite trivial), and leave Medusa alone. OTOH, if anyone uses >medusa for XMLRPC... They'd probably appreciate the better POST performance. > > Good catch, Jason! Would you mind patching medusa_http, adding your new handler, and posting it? -- Graham From jsibre at sibre.org Mon Feb 16 13:26:51 2004 From: jsibre at sibre.org (Jason E. Sibre) Date: Mon Feb 16 13:27:24 2004 Subject: [Medusa-dev] RE: [Quixote-users] Poor upload (POST) performance... [PATCH] In-Reply-To: <4030D2A8.6040607@uwindsor.ca> Message-ID: > Would you mind patching medusa_http, adding your new handler, and > posting it? Ok, here's a fix revolving aroung the medusa_http.py file (in Quixote/servers). Medusa is unaffected by this patch. The patch adds a Collector class to medusa_http (which is a fixed version of the xmlrpc_handler.collector), and utilizes it. Since this will only affect Quixote users, no attempt has been made to provide backward-compatibility on Collector.data attribute accesses. (I can't see where a Qx user would ever need to mess with the collector directly, so it seems silly to worry about it.) This patch is written on top of the last patch I submitted against medusa_http, so if the line numbers don't match, it's because you're missing the other patch :) Jason -------------- next part -------------- A non-text attachment was scrubbed... Name: medusa_http.py.patch Type: application/octet-stream Size: 1917 bytes Desc: not available Url : http://mail.python.org/pipermail/medusa-dev/attachments/20040216/b0724b1d/medusa_http.py.obj From amk at amk.ca Mon Feb 16 13:38:12 2004 From: amk at amk.ca (A.M. Kuchling) Date: Mon Feb 16 13:39:49 2004 Subject: [Medusa-dev] Re: [Quixote-users] Poor upload (POST) performance... [PATCH] In-Reply-To: <20040216183616.GA13618@mems-exchange.org> References: <4030D2A8.6040607@uwindsor.ca> <20040216183616.GA13618@mems-exchange.org> Message-ID: <20040216183812.GA6049@rogue.amk.ca> On Mon, Feb 16, 2004 at 01:36:16PM -0500, Neil Schemenauer wrote: > Your patch looks fine to me. However, would it be better to patch > xmlrpc_handler? Perhaps Andrew can comment. I plan to accept the patch for Medusa, and will also apply it to the version of Medusa in Graham's experimental patch for Quixote. (At least, I'll do that once my tree is in a decent state again; I'm giving up on tla for now and switching back to Subversion.) --amk From jsibre at sibre.org Mon Feb 16 15:20:52 2004 From: jsibre at sibre.org (Jason E. Sibre) Date: Mon Feb 16 15:21:24 2004 Subject: [Medusa-dev] Re: [Quixote-users] Poor upload (POST) performance...[PATCH] In-Reply-To: <20040216183812.GA6049@rogue.amk.ca> Message-ID: > On Mon, Feb 16, 2004 at 01:36:16PM -0500, Neil Schemenauer wrote: > > Your patch looks fine to me. However, would it be better to patch > > xmlrpc_handler? Perhaps Andrew can comment. > > I plan to accept the patch for Medusa, and will also apply it to > the version > of Medusa in Graham's experimental patch for Quixote. > > (At least, I'll do that once my tree is in a decent state again; > I'm giving > up on tla for now and switching back to Subversion.) Ok, I've given it some thought, and I've about convinced myself that we aren't likely to have problems involving backwards compatibility on this one. Even for medusa users not using quixote, I really can't see why they would ever have felt the need to diddle with medusa.xmlrpc_handler.collector's data attribute, and as such, submit the attached patch. If anyone should think that it would be better to attempt to preserve some level of backwards compatibility on the collector.data attribute, let me know. I have something that I was working on that emulated a string (sorta) while actually being a list. I decided the additional performance overhead, complexity (and resulting chance of stupid bugs), and line count just wasn't worth it for this situation. Jason -------------- next part -------------- A non-text attachment was scrubbed... Name: xmlrpc_handler.py.patch Type: application/octet-stream Size: 1070 bytes Desc: not available Url : http://mail.python.org/pipermail/medusa-dev/attachments/20040216/1a0e4099/xmlrpc_handler.py.obj From sugino at brandeis.edu Mon Feb 16 19:55:28 2004 From: sugino at brandeis.edu (Ken Sugino) Date: Mon Feb 16 19:55:29 2004 Subject: [Medusa-dev] Re: [Quixote-users] Poor upload (POST) performance... [PATCH] In-Reply-To: <20040216183812.GA6049@rogue.amk.ca> References: <4030D2A8.6040607@uwindsor.ca> <20040216183616.GA13618@mems-exchange.org> <20040216183812.GA6049@rogue.amk.ca> Message-ID: == Related issue: Poor GET performace for large files: 1) http_request.push uses "simple_producer" which copies string each time "more" is called. Instead "scanning_producer" should be used: --- medusa-0.5.4/http_server.py 2003-12-06 01:34:27.000000000 -0500 +++ medusa-0.5.4.mod1/http_server.py 2004-02-16 18:46:52.000000000 -0500 @@ -157,8 +157,8 @@ def push (self, thing): if type(thing) == type(''): - self.outgoing.append(producers.simple_producer (thing)) + self.outgoing.append(producers.scanning_producer (thing)) + # 2003.04.29 KS changed from simple producer else: self.outgoing.append(thing) 2) Sometimes (e.g. from script_handler in thread/thread_handler.py), "http_channel.push" is called directly which is actually inherited from "asynchat.async_chat.push" which uses again "simple_producer". "http_channel" should probably override this "push" to use "scanning_producer": @@ -536,6 +539,7 @@ + def push (self, data): + # 20031202 KS override this async_chat method to use scanning_producer + self.producer_fifo.push (producers.scanning_producer(data)) + self.initiate_send() == Threading problem on windows xp (os.name=='nt') "asyncore.dispatcher.listen" restricts number of connection to one if requested connection is more than 5. | def listen(self, num): | self.accepting = 1 | if os.name == 'nt' and num > 5: | num = 1 | return self.socket.listen(num) "http_server.__init__" calls self.listen(1024) which on windows ends up with only one connection and affect threaded connections. Ideally, asyncore.py should be modified but following change is OK as well. @@ -562,8 +565,14 @@ self.bind ((ip, port)) # lower this to 5 if your OS complains - self.listen (1024) - + #self.listen (1024) + #KS + if os.name == 'nt': + self.listen(5) + else: + self.listen(1024) + #/KS + host, port = self.socket.getsockname() if not ip: self.log_info('Computing default hostname', 'warning') == Other bugs related to thread_handler.py 1) thread/thread_handler.py line 165 'put', 'post' -> 'PUT', 'POST' since crack_request in http_server.py dropped .lower() 2) thread/thread_handler.py line 214 lines=lines[:-1] should be commented out for binary data transfer. --- medusa-0.5.4/thread/thread_handler.py 2003-12-06 01:36:23.000000000 -0500 +++ medusa-0.5.4.mod1/thread/thread_handler.py 2004-02-16 18:46:52.000000000 -0500 @@ -161,7 +161,8 @@ except KeyError: pass - if request.command in ('put', 'post'): + if request.command in ('PUT', 'POST', 'OPTIONS'): # KS, OPTIONS: explorer? webdav? # PUT data requires a correct Content-Length: header # (though I bet with http/1.1 we can expect chunked encoding) request.collector = collector (self, request, env) @@ -209,7 +210,10 @@ self.buffer = self.buffer + data lines = string.split (self.buffer, '\n') # ignore the last piece, it is either empty, or a partial line - lines = lines[:-1] + + #lines = lines[:-1] + # (KS) need last line if you are sending binary data + # look for something un-header-like for i in range(len(lines)): li = lines[i] I've been using medusa with these modifications for a couple of months now w/o problem. (http://egret.bio.brandeis.edu/nikki/NiKKi) Ciao, Ken From jim at deltaxresearch.com Mon Feb 16 16:02:53 2004 From: jim at deltaxresearch.com (Jim Dukarm) Date: Wed Feb 18 18:13:34 2004 Subject: [Medusa-dev] Re: Poor upload (POST) performance...[PATCH] In-Reply-To: References: Message-ID: <93504035765.20040216130253@deltaxresearch.com> > If anyone should think that it would be better to attempt to preserve some > level of backwards compatibility on the collector.data attribute, let me > know. I don't think it is a big deal. The script_handler collector not only uses a StringIO as its data, but it also reverses the order of the arguments in its call to handler.continue_request(), so it seems evident that there was not a lot of expectation that anyone would be subclassing the collector or even using it apart from its handler. That being the case, it seems reasonable that a QuixoteHandler should have its own Collector, too. After seeing your initial heads-up on the use of string concatenation in the xmlrpc_handler.collector, I wrote my own "QCollector" which uses StringIO for its data but otherwise looks just like the xmlrpc one, so your Collector patch, which is the same thing except for using list.join instead of StringIO, seems just fine to me. Jim Dukarm DELTA-X RESEARCH Victoria BC Canada From jsibre at chironsys.com Mon Feb 16 16:32:36 2004 From: jsibre at chironsys.com (Jason Sibre) Date: Wed Feb 18 18:13:39 2004 Subject: [Medusa-dev] [PATCH] medusa.http_server to handle multiple response headers with same name Message-ID: Folks, Attached is a patch for medusa, (and a patch for quixote to make it work ideally with the patched medusa) which enables medusa to have more than one header in a reply with the same name. The existing implementation uses a dict, which is easy, but it prevents multiple headers of the same name from being set. This is an issue (for instance) when trying to set two cookies in the course of a single request: Only one will make it to the browser (this is what bit me). This repair is made more complicated than it might otherwise be, because the api in the medusa request object reveals the dict orientation of the header storage, and in fact, some operations are impossible without working directly with the dict object(removing a header, for example). What I've done is add a list object for handling the headers, and then added a rich(er) api for working with them. I've also left the original implementation intact, so that someone upgrading from an old medusa to this patched version will not notice a difference (although, they still won't be able to have multiple headers of the same name). My theory being that if they upgrade, unaware of the changes, then even if they're doing fancy things directly with the old dict object, things will keep on working, but if they want to take advantage of the new feature, it only requires a slight modification of how headers are added to a request object. The new api will look at the existing dict object, and merge it's header info with the list object's header info when called to generate the header portion of the request. This means that even if accesses to the headers go through both the old and new methods, things should work as intended. I say 'should', because may be possible to break things if you really try. Internally, the medusa http_server module uses the new api. Since that part merges the dict with the list, things will work nicely whichever way a developer goes with writing headers. Well... The patch is littered with comments, so I'll shut up and let the code speak for itself. Jason -------------- next part -------------- A non-text attachment was scrubbed... Name: http_server.py.multiHeaderPatch Type: application/octet-stream Size: 6840 bytes Desc: not available Url : http://mail.python.org/pipermail/medusa-dev/attachments/20040216/6852949a/http_server.py.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: medusa_http.py.multiHeaderPatch Type: application/octet-stream Size: 671 bytes Desc: not available Url : http://mail.python.org/pipermail/medusa-dev/attachments/20040216/6852949a/medusa_http.py.obj From nas at mems-exchange.org Mon Feb 16 13:36:16 2004 From: nas at mems-exchange.org (Neil Schemenauer) Date: Wed Feb 18 18:13:44 2004 Subject: [Medusa-dev] Re: [Quixote-users] Poor upload (POST) performance... [PATCH] In-Reply-To: References: <4030D2A8.6040607@uwindsor.ca> Message-ID: <20040216183616.GA13618@mems-exchange.org> On Mon, Feb 16, 2004 at 12:26:51PM -0600, Jason E. Sibre wrote: > Ok, here's a fix revolving aroung the medusa_http.py file (in > Quixote/servers). Medusa is unaffected by this patch. The patch adds a > Collector class to medusa_http (which is a fixed version of the > xmlrpc_handler.collector), and utilizes it. Your patch looks fine to me. However, would it be better to patch xmlrpc_handler? Perhaps Andrew can comment. Neil From jsibre at sibre.org Thu Feb 26 20:15:13 2004 From: jsibre at sibre.org (Jason E. Sibre) Date: Thu Feb 26 20:15:20 2004 Subject: [Medusa-dev] RE: [Quixote-users] request.get_url issue [PATCH] In-Reply-To: <20040226225507.GA4016@mems-exchange.org> Message-ID: > > Unfortunately, our own REQUEST_URI is decoded (at least with medusa it > > is...) > > It's not if you use Apache. I don't think it should be decoded. > > Neil Ok, again, I agree. And now I have the patch(es) ready. These patches fix medusa.http_server, and quixote.server.medusa_http so that uri unquoting is moved out of medusa and into quixote's handler for medusa (so that quixote can have the clean URI provided by the client in REQUEST_URI and can put an unquoted version in PATH_INFO). I was a little worried about doing this (Other sharp eyes have looked at this recently), until I saw that the default hander provided with Medusa (medusa.default_handler.default_handler) actually takes care of it's own URI unquoting, leading me to believe that this is the correct (or sanctioned) approach, and that http_server shouldn't be doing it internally. I especially feel this way since medusa only has one version of that information available to it internally at that point, and if it unquotes it, there ain't no goin' back! Sure, you can requote it, but can you really be sure you'll end up with the same thing you started with? Better just to leave it alone, and unquote a copy of it. I've tested medusa with it's own default handler against the mythical "foo%26%3fbar" file, and it works, and of course it's working nicely with Quixote. Jason -------------- next part -------------- A non-text attachment was scrubbed... Name: http_server.unquote_uri.patch Type: application/octet-stream Size: 805 bytes Desc: not available Url : http://mail.python.org/pipermail/medusa-dev/attachments/20040226/a37bfa67/http_server.unquote_uri.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: medusa_http.unquote_uri.patch Type: application/octet-stream Size: 725 bytes Desc: not available Url : http://mail.python.org/pipermail/medusa-dev/attachments/20040226/a37bfa67/medusa_http.unquote_uri.obj From laforge49 at yahoo.co.in Fri May 7 10:58:46 2004 From: laforge49 at yahoo.co.in (=?iso-8859-1?q?Bill=20la=20Forge?=) Date: Fri May 7 10:58:51 2004 Subject: [Medusa-dev] Do you accept submissions? Message-ID: <20040507145846.87464.qmail@web8310.mail.in.yahoo.com> Just downloaded and reviewed Medusa. Looks great. Easy-to-read code, if you're familiar with asynchat/asyncore. I'd like to write a handler making use of compstrm for coroutine-like sessions (threadless): http://compstrm.sourceforge.net I figure it'll take one or two weeks. So, do you accept submissions? Is there a FAQ somewhere I missed? (I'm not using coroutines really--just jazzed up generator functions.) Thanks! Bill la Forge http://www.geocities.com/laforge49/ Yahoo! India Matrimony: Find your partner online. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/medusa-dev/attachments/20040507/d4e8e544/attachment.html From laforge49 at yahoo.co.in Fri May 7 23:00:06 2004 From: laforge49 at yahoo.co.in (=?iso-8859-1?q?Bill=20la=20Forge?=) Date: Fri May 7 23:00:14 2004 Subject: [Medusa-dev] Where are the Medusa patches? Message-ID: <20040508030006.90769.qmail@web8309.mail.in.yahoo.com> I downloaded the Medusa 0.5.4 release (9/3/03), but I notice the archive contains PATCHES email (without the attachments!) dated 2/26/04. Where are these patches? Thankyou, Bill la Forge Yahoo! India Matrimony: Find your partner online. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/medusa-dev/attachments/20040508/b9146688/attachment.html From laforge49 at yahoo.co.in Fri May 7 23:04:12 2004 From: laforge49 at yahoo.co.in (=?iso-8859-1?q?Bill=20la=20Forge?=) Date: Fri May 7 23:04:18 2004 Subject: [Medusa-dev] appologies for my foolishness Message-ID: <20040508030412.85535.qmail@web8304.mail.in.yahoo.com> The archive says the attachment was scrubbed and not available, but then gives a working URL to the patch. Sorry for (my) confusion. Bill la Forge Yahoo! India Matrimony: Find your partner online. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/medusa-dev/attachments/20040508/6bf44439/attachment.html From laforge49 at yahoo.co.in Fri May 7 08:37:52 2004 From: laforge49 at yahoo.co.in (=?iso-8859-1?q?Bill=20la=20Forge?=) Date: Mon May 10 08:04:02 2004 Subject: [Medusa-dev] do you accept submissions? Message-ID: <20040507123752.41826.qmail@web8312.mail.in.yahoo.com> I just downloaded Medusa, and spent 3 hours reviewing the code. I find it easy to understand and easily extended. Simply wonderful! (Familiarity with asyncore/asynchat helped a lot, of course! ;-) I very much want to add a handler for doing long-running (threadless/asynchronous) stateful processes which interact with a user via forms based on the compstrm project: http://compstrm.sourceforge.net/ --which already contains one integration with asynchat (asyncs). So, do you accept submissions? (Is there a FAQ somewhere that I missed?) Thanks! Bill la Forge http://www.geocities.com/laforge49/ Yahoo! India Matrimony: Find your partner online. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/medusa-dev/attachments/20040507/281ae6bb/attachment.html From abo at minkirri.apana.org.au Mon May 10 20:44:38 2004 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Mon May 10 20:44:46 2004 Subject: [Medusa-dev] do you accept submissions? In-Reply-To: <20040507123752.41826.qmail@web8312.mail.in.yahoo.com> References: <20040507123752.41826.qmail@web8312.mail.in.yahoo.com> Message-ID: <1084236278.1434.4.camel@schizo> On Fri, 2004-05-07 at 22:37, Bill la Forge wrote: > I just downloaded Medusa, and spent 3 hours reviewing the code. I find > it easy to understand and easily extended. Simply wonderful! > (Familiarity with asyncore/asynchat helped a lot, of course! ;-) Before you commit to medusa, it is worth having a look at twisted. Twisted is another Python async framework, but it is newer and more actively developed. However, medusa is much more stable (API is pretty much set in concrete, as opposed to twisted which has a history of refactoring), so it depends on what you want. > So, do you accept submissions? (Is there a FAQ somewhere that I > missed?) sure, post to the list. -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From laforge49 at yahoo.co.in Wed May 12 05:43:32 2004 From: laforge49 at yahoo.co.in (=?iso-8859-1?q?Bill=20la=20Forge?=) Date: Wed May 12 05:43:45 2004 Subject: [Medusa-dev] Integration with CompStrm, bug report Message-ID: <20040512094332.17413.qmail@web8310.mail.in.yahoo.com> http://compstrm.sourceforge.net/medusa.html (The integration provides an alternative to threads for long-running scripts. I'll note that the use of threads works only when they can be executed quickly, but that they are not a scalable solution when dealing with numerous long-running scripts typical of distributed applications.) While working with Medusa, I found two bugs in medusa.script_handler.persistent_script_handler, which seemed to be oversights. 1. self.hits was not incrimented 2. the status method was missing. I'll note that in both cases, medusa.script_handler.script_handler contained the correct code. Bill la Forge Yahoo! India Matrimony: Find your partner online. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/medusa-dev/attachments/20040512/f01910d9/attachment.html From jeff at jab.org Fri Jun 11 22:58:55 2004 From: jeff at jab.org (Jeff Breidenbach) Date: Fri Jun 11 22:59:13 2004 Subject: [Medusa-dev] Medusa in Debian Message-ID: Hi all, I am a Debian developer, and I would like to see a Medusa package inside Debian main. With a little bit of hunting, I found a Debian Medusa package on Georg Bauer's website. http://simon.bofh.ms/~gb/ Looking more closely, I see things like: This package was debianized by A.M. Kuchling on Fri, 22 Aug 2003 08:54:11 -0400. Kuchling, do you have any interest in this? Would you like to be involved? For example by joining the Debian project, or having a Debian developer such as myself sponsor your packaging work? Is anyone else working on this sort of thing? Please let me know what makes the most sense. I've already contacted Georg; he would be happy to see the package inside Debian but has no plans to get further involved. Cheers, Jeff From jeff at jab.org Sat Jun 12 00:57:40 2004 From: jeff at jab.org (Jeff Breidenbach) Date: Sat Jun 12 00:57:53 2004 Subject: [Medusa-dev] Medusa on Debian Message-ID: I cleaned up the existing Debian package a little bit and submitted it for inclusion in the distribution a few minutes ago. New packages take a little bit of time before they can propogate, but I expect Medusa to be generally available in Debian's unstable branch within a week or so. I'd be delighted if amk or another member of the medusa team wants to do or help with the package maintenance - and I'm happy to help as desired. Otherwise, I guess I'll just do it myself. Cheers, Jeff From amk at amk.ca Wed Jul 7 14:38:25 2004 From: amk at amk.ca (A.M. Kuchling) Date: Wed Jul 7 14:38:54 2004 Subject: [Medusa-dev] Error conditions and select() Message-ID: <20040707123825.GA7947@rogue.amk.ca> Jim Jewett sent me the following in private mail: =========== In poll(), error_list e is always empty. Should all items in map be added to e? r = []; w = []; e = [] for fd, obj in map.items(): if obj.readable(): r.append(fd) if obj.writable(): w.append(fd) # # add this line? # # e.append(fd) if [] == r == w == e: # # but then maybe not select if e is the only list? time.sleep(timeout) =========== poll2(), which uses select.poll(), calls obj.handle_expt_event() when the POLLERR bit is set, so it seems natural to do the same in the select-using version of poll(). Seem reasonable to everyone? --amk From greg at electricrain.com Fri Jul 9 05:33:40 2004 From: greg at electricrain.com (Gregory P. Smith) Date: Fri Jul 9 05:33:45 2004 Subject: [Medusa-dev] Error conditions and select() In-Reply-To: <20040707123825.GA7947@rogue.amk.ca> References: <20040707123825.GA7947@rogue.amk.ca> Message-ID: <20040709033340.GJ30677@zot.electricrain.com> On Wed, Jul 07, 2004 at 08:38:25AM -0400, A.M. Kuchling wrote: > Jim Jewett sent me the following in private mail: > =========== > In poll(), error_list e is always empty. > Should all items in map be added to e? > > r = []; w = []; e = [] > for fd, obj in map.items(): > if obj.readable(): > r.append(fd) > if obj.writable(): > w.append(fd) > # # add this line? > # # e.append(fd) > if [] == r == w == e: > # # but then maybe not select if e is the only list? > time.sleep(timeout) > > =========== > > poll2(), which uses select.poll(), calls obj.handle_expt_event() when > the POLLERR bit is set, so it seems natural to do the same in the > select-using version of poll(). Seem reasonable to everyone? > > --amk Yes, that looks right. It is valid to select only looking for exceptions. Underneath it all time.sleep is most likely calling select with zero fds for its high resolution sleep anyways so why not always call select with the timeout and look for exceptions ourselves. Does this imply that Jim actually has a use for socket/fd exceptions? They're only raised for out of band data on a socket or control status info on a pseudo-terminal in packet mode. Neither of which I'd expect to see used very much. -g From skumar at datec-systems.com Wed Aug 11 14:13:41 2004 From: skumar at datec-systems.com (Santosh Kumar) Date: Wed Aug 11 14:15:19 2004 Subject: [Medusa-dev] Telnelib used with asyncore Message-ID: <411A0D75.2050706@datec-systems.com> Hi, I am using a telnetlib.Telnet object to create a telnet session to a host. After i instantiate the telnet object i pass the fd used by telnetlib object to an asyncore.dispatcher object to watch this fd for data ( i cannot use the telnetlib.expect or select to watch this fd for some design reasons). When i pass data over the telnetlib object by calling its .write(data) method, the data is read by the host and the command is executed remotely. I am also able to see the results being echoed back to the asyncore.dispatcher's handle_read() method. Now the problem is that the asyncore also echoes the comand which i have sent to the host. I know that this happens because the same fd is used by telnetlib for writing and the asyncore to watch. Is there a way by which i can send the data to the host without echoing it locally (as local data echo will be read by asyncore). I want the following: - Pass the command to Telnet's write method eg TelnetObject.write( mycommand) - The host will see this command and will act upon it, results echoed (not the myCommand) to the asyncore's handle_read() In general is there a way to supress local echoing while using the telnetlib? Regards, SK -- Santosh Kumar K. Tel: +91-471-2527520, 21, 22, 23 Ext 216 Fax: +91-471-2527524 Cell: 9847496484 Email: skumar@datec-systems.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/medusa-dev/attachments/20040811/c1d1d6ca/attachment.htm