From jims@s... Sun Jan 7 09:00:08 2001 From: jims@s... (jims@s...) Date: Sun, 07 Jan 2001 09:00:08 -0000 Subject: Newbie quesiton: how to pass parameters to a .mpy script as uri queries? Message-ID: <939b6o+44jh@e...> Hi all I'm a bit of a newbie, and I'm wondering how to go about writing a Python script (I'm currently using script_handler, and I don;t know if that's the proper handler to use) that accepts as input parameters passed in on the URI of a GET request. For example: http://loopy.metrics.com/gimme.mpy?category=42&item=white would invoke the gimme.mpy script and pass the argument "category" with a value of "42" and the argument "item" with the value "white". I tried writing the .mpy using the Python cgi module (the usual has_key stuff, see the cgi module docs on python.org), and that does not seem to work. When I tinker with script_handler.py I see that the query is being split off from the path by split_path, but nothing is being done with the query. It seems to me that there should be a simple way to do this? thanks jim From weissblitz@y... Fri Jan 12 19:48:49 2001 From: weissblitz@y... (weissblitz@y...) Date: Fri, 12 Jan 2001 19:48:49 -0000 Subject: IPC between Medusa Modules? Message-ID: <93nn31+fgcp@e...> I'm developing a messaging/chat client/server application using Medusa for the server side. It is composed of HTTP, FTP, messaging, chat (IRC like), and a user manager to manage user profile information. I can have any number of these components running, some on different machines accross the network. These components need to connect to the user manager component for user authentication and profile information. What would be the most efficient way of IPC between the modules? Named pipes? Sockets? SOAP? RPC? Also, whare can I get additional information on how to use the monitor server. I can connect to it and get the Python prompt but i would like to know what things I can do with this. Thanks in advance, Carlos :) PS: Love Medusa! Thanks to all who put their time and effort into developing and improving Medusa. Keep up the good work. From rprice@u... Fri Jan 12 14:03:26 2001 From: rprice@u... (ray) Date: Fri, 12 Jan 2001 06:03:26 -0800 Subject: [medusa] IPC between Medusa Modules? References: <93nn31+fgcp@e...> Message-ID: <3A5F0EAE.7E4BC678@u...> I've created a similar system using the RPC module distributed by Sam / egroups. The only problem, which is a real problem, is that once a server goes away, the client needs to be rebooted. I just can't figure out how to get the client to reconnect. Fortunately this only happens once every couple of weeks. You also need to remember that if one reboots a server, all clients need to be rebooted. Any suggestions on how to get the client to reconnect? It's running on Redhat Linux. Ray weissblitz@y... wrote: > I'm developing a messaging/chat client/server application using > Medusa for the server side. It is composed of HTTP, FTP, messaging, > chat (IRC like), and a user manager to manage user profile > information. I can have any number of these components running, some > on different machines accross the network. These components > need to connect to the user manager component for user authentication > and profile information. What would be the most efficient way of IPC > between the modules? Named pipes? Sockets? SOAP? RPC? > > Also, whare can I get additional information on how to use the > monitor server. I can connect to it and get the Python prompt but i > would like to know what things I can do with this. > > Thanks in advance, > Carlos :) > > PS: Love Medusa! Thanks to all who put their time and effort into > developing and improving Medusa. Keep up the good work. From lucifer@e... Sat Jan 13 02:12:28 2001 From: lucifer@e... (Nicolas Echaniz) Date: Fri, 12 Jan 2001 23:12:28 -0300 Subject: =?ISO-8859-1?Q?Re:_[medusa]_IPC_between_Medusa_Modules=3F?= In-Reply-To: <93nn31+fgcp@e...> Message-ID: <200101130212.XAA29959@w...> Hi Carlos, I'm interested in your proyect. I'm building something quite similar. What are you planning to use on the client side? java ? Is there any preview release of your work? Maybe we could join our efforts some time. What are your plans for this application? PS: where are you from? On Fri, 12 Jan 2001 19:48:49 -0000, you wrote: > I'm developing a messaging/chat client/server application using > Medusa for the server side. It is composed of HTTP, FTP, messaging, > chat (IRC like), and a user manager to manage user profile > information. I can have any number of these components running, some > on different machines accross the network. These components > need to connect to the user manager component for user authentication > and profile information. What would be the most efficient way of IPC > between the modules? Named pipes? Sockets? SOAP? RPC? > > Also, whare can I get additional information on how to use the > monitor server. I can connect to it and get the Python prompt but i > would like to know what things I can do with this. > > Thanks in advance, > Carlos :) > > PS: Love Medusa! Thanks to all who put their time and effort into > developing and improving Medusa. Keep up the good work. _______________________________________________________________ Nicolás Echániz | RAKAR Producciones | Buenos Aires | Argentina From nhytro@c... Mon Jan 15 08:20:26 2001 From: nhytro@c... (nhytro@c...) Date: Mon, 15 Jan 2001 08:20:26 -0000 Subject: Medusa Installation and Documentation Message-ID: <93ubsa+c595@e...> Hi there! is there some kind person out there that can help me with=20 the installation and usage of MEDUSA on WIN32 machines? I could=B4nt=20 make anything out of the docs. Thanks in advance Sharriff From akuchlin@c... Thu Jan 18 21:43:27 2001 From: akuchlin@c... (akuchlin@c...) Date: Thu, 18 Jan 2001 21:43:27 -0000 Subject: Patch to use poll() support in Python 2.0 In-Reply-To: <200012210340.WAA00993@2...> Message-ID: <947o1v+gva0@e...> --- In medusa@egroups.com, "A.M. Kuchling" wrote: > This patch adds the ability to use the poll() support that was added > to the select module in Python 2.0. Only lightly tested ... please > proofread this patch carefully. I posted that patch Dec. 22. With Python 2.1alpha1 rumbling toward us, I thought I'd check if it was accepted, and if a new Medusa release is going to be made fairly soon. Sam, any comments? --amk From kens@s... Tue Jan 23 22:07:43 2001 From: kens@s... (Ken Seehof) Date: Tue, 23 Jan 2001 22:07:43 -0000 Subject: Simple subclassing of http-server In-Reply-To: <8t3ouq+m9so@e...> Message-ID: <94kvbf+ks4j@e...> I need the same thing. Have you found anything out yet? I seems to be one of those things where the simplest and most typical usage doesn't get documented. I guess I should complain since it is free, but I am frustrated. Is there a book out there that is recommended? --- In medusa@egroups.com, thomas@c... wrote: > I want to subclass medusas http-server-class so that I can write my > own request handler. The server is going to serve xml. > > I've looked at the docs/tutorial but I cannot figure out where to put > my own handler, so that I can pick the requested url apart, use the > submitted data to find what data to send back to the client. > > The simpleHTTPServer in the standard python distro has a simpler or > at least more understandable approach, so if anybody could show me > how to get something similar to that to work with I'd be glad. > > The only thing I need is to have a place to insert my method and let > it return data based on the url the user typed in the browser. If I > can sublclass something and do it in my own subclass that would be > the best solution. It would be nice to leave the medusa-code alone. > > Thanks From lucifer@e... Wed Jan 24 00:42:36 2001 From: lucifer@e... (Nicolas Echaniz) Date: Tue, 23 Jan 2001 21:42:36 -0300 Subject: [medusa] Re: Simple subclassing of http-server In-Reply-To: <94kvbf+ks4j@e...> Message-ID: <200101240042.VAA08248@w...> Hi guys, I'm no Medusa expert. But I think that if what you need is to add your own handler to medusa, you need to write a new handler and then install it using start_medusa.py. The http_server.py will then use your handler for the type of requests it should handle. See this code (from the http_server): it starts on line 470 (but my base code is a little bit mangled :) for h in self.server.handlers: if h.match (r): try: self.current_request = r h.handle_request (r) I wrote a modified version of the script_handler, so I modified the start_medusa.py as follows: scr = ec_script_handler.ec_script_handler () hs.install_handler (scr) this installs my handler (ec_script_handler), so next time I start medusa, it will handle .py requests with my ec_script_handler. Hope it helps. PS: if you don't know how to write your own handlers, just look at the code from the other handlers ;) On Tue, 23 Jan 2001 22:07:43 -0000, you wrote: > I need the same thing. Have you found anything out yet? > > I seems to be one of those things where the simplest and most typical > usage doesn't get documented. I guess I should complain since it is > free, but I am frustrated. Is there a book out there that is > recommended? > > --- In medusa@egroups.com, thomas@c... wrote: > > I want to subclass medusas http-server-class so that I can write my > > own request handler. The server is going to serve xml. > > > > I've looked at the docs/tutorial but I cannot figure out where to > put > > my own handler, so that I can pick the requested url apart, use the > > submitted data to find what data to send back to the client. > > > > The simpleHTTPServer in the standard python distro has a simpler or > > at least more understandable approach, so if anybody could show me > > how to get something similar to that to work with I'd be glad. > > > > The only thing I need is to have a place to insert my method and > let > > it return data based on the url the user typed in the browser. If I > > can sublclass something and do it in my own subclass that would be > > the best solution. It would be nice to leave the medusa-code alone. > > > > Thanks _______________________________________________________________ Nicolás Echániz | RAKAR Producciones | Buenos Aires | Argentina From greg@e... Fri Jan 26 10:19:26 2001 From: greg@e... (Gregory P. Smith) Date: Fri, 26 Jan 2001 02:19:26 -0800 Subject: detecting connect() failures on windows Message-ID: <20010126021926.C3554@z...> Has anybody had any success in getting asyncore under python on windows to detect connect() failures? On real OSes (linux and bsd) I get nice easy to understand behavior, the attempt to connect to a host on a port its not listening on produces this output: (code below) ...... creating foo to port 81 log: adding channel asyncore loop starting: handle_connect Exception during self.send("") in connect: (32, 'Broken pipe') handle_read Exception during self.recv in read: (111, 'Connection refused') handle_write handle_read handle_close log: closing channel 4: data read: exited asyncore loop ...... On windows, it just hangs for a long time in select. (win2000 is my main test case but it behaves just as badly on win98) thanks, -Greg Here's my simple test program: import asyncore import socket class foo(asyncore.dispatcher): writ = 1 buf = 'GET /non.existant.file HTTP/1.0\r\n\r\n' def handle_write(self): print 'handle_write' if not self.writ: return self.send(self.buf) self.writ = 0 def writable(self): return self.writ def readable(self): return 1 def handle_read(self): print 'handle_read' try: data = self.recv(4096) except socket.error, e: print 'Exception during self.recv in read:', e self.writ = 0 # read failed? we're closed now return print 'data read:' print data def handle_close(self): print 'handle_close' self.close() #raise asyncore.ExitNow, 'handle_close called, exiting' def handle_connect(self): print 'handle_connect' try: data = self.send("") except socket.error, e: print 'Exception during self.send("") in connect:', e return def __init__(self, host, port): asyncore.dispatcher.__init__(self) self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.connect((host, port)) print 'creating foo to port 80' f1 = foo('www.mad-scientist.com', 80) print 'asyncore loop starting:' asyncore.loop() print 'exited asyncore loop' print 'creating foo to port 81' f2 = foo('www.mad-scientist.com', 81) print 'asyncore loop starting:' asyncore.loop() print 'exited asyncore loop' From abo@m... Sat Jan 27 10:10:22 2001 From: abo@m... (Donovan Baarda) Date: Sat, 27 Jan 2001 21:10:22 +1100 Subject: asynchat ready() method patch. In-Reply-To: <3.0.6.32.20001209180505.007d5d60@p...>; from support@i... on Sat, Dec 09, 2000 at 06:05:05PM -0500 References: <3.0.6.32.20001209180505.007d5d60@p...> Message-ID: <20010127211021.D13503@m...> ---------------------- multipart/mixed attachment G'day, I've recieved one note of interest in my patch for medusa to allow producers to block using a ready() method. The gist was "this is a good idea, but it doesn't work". At the time I was focusing on the ftp server part of medusa, so I hadn't checked the http side much at all. I also noticed that asyncore.py and asynchat.py were distributed without producers.py in python itself. My original changes made asynchat.py dependant on producers.py, which was probably a bad idea. So I have gone right over my changes and produced a new, more thoroughly tested patch that tries to minimize the side affects. I have tested the changes as they are incrementaly applied to ensure that my changes don't break old code that doesn't use the ready() feature. This patch contains all changes made to all files, but you only need to accept the changes to some files to use ready() The main change is to asynchat.py, where support for the ready() method has been added. Also added are a variety of producer base classes that allow new producers with ready() support to be easily derived (and removed the asynchat.py dependancy on producers.py that my early patch introduced). The main async_chat class is actualy significantly simpler for the change, mostly due to replacing the fifo class with a fifo_producer class. The find_prefix_at_end procedure for finding terminators has also been simplified and speeded up. Overall, asynchat.py is only 23 lines bigger dispite all the additional functionality. The good news is, the changes to asynchat.py are fully backwards compatible and medusa runs fine with all other files unchanged, _except_ that filesys.py and status_handler.py have conflicting ready() methods in producers. These ready() methods are unused in the original medusa, but cause the new asynchat.py to think the producers never complete. The patch to these two files simply removes these ready() methods so medusa can run with the new asynchat. The next lot of serious changes are to producers.py and http_server.py. These changes are not needed to run medusa as it stands, but are needed if you start using blocking producers. The changes primarily make the producers inherited from the base producer classes defined in asynchat.py. This ensures that producers that wrap other producers will block correctly. The inheritance of features makes producers.py 62 lines smaller than it was. All the changes to producers.py are backwards compatible _except_ the compound_producer.__init__() method now takes a python list of producers not a fifo. The changes to http_server.py take this into account. The final trivial change is a commented out alternative to use the new async_producer instead of file_producer in ftp_server.py. This allows medusa filesystems to return file types that could block without blocking the whole server. I hope someone gets a chance to try these out, and I would appreciate any form of feedback. I'm tempted to submit my patch to asynchat.py direct to the Python people for inclusion in Python 2.1, but would like to hear more feedback before I do. BTW, ignore the patch to "medusa.prj"... that is a side affect of using prcs to revision control my changes. This patch is to the 20000601 version of medusa. -- ---------------------------------------------------------------------- ABO: finger abo@m... for more info, including pgp key ---------------------------------------------------------------------- ---------------------- multipart/mixed attachment A non-text attachment was scrubbed... Name: medusa-ready_0.3.patch.bz2 Type: application/octet-stream Size: 13627 bytes Desc: not available Url : http://mail.python.org/pipermail-21/medusa-dev/attachments/b01537c6/medusa-ready_0.3.patch.bz2 ---------------------- multipart/mixed attachment-- From abo@m... Sun Jan 28 02:51:53 2001 From: abo@m... (Donovan Baarda) Date: Sun, 28 Jan 2001 13:51:53 +1100 Subject: asynchat ready() method patch. In-Reply-To: <20010127174237.D12344@k...>; from julio@p... on Sat, Jan 27, 2001 at 05:42:37PM -0200 References: <20010127174237.D12344@k...> Message-ID: <20010128135153.A19001@m...> On Sat, Jan 27, 2001 at 05:42:37PM -0200, Julio Maia wrote: > Hi, > > I've modified that test script I've sent to you in order to try your new > ready() patch. Now it seems to be calling ready() and more() properly, > however it takes a long time between calls of ready() (about 30 seconds). > Also, it seems the returned data from more() is not being processed > correctly in the handler because of ready(). I never recieved your test scripts... I think the attachment went missing. I'd be interested to see them though, so send me them again. The 30 second delays are caused by the way asyncore.py works. The ready() methods are called each iteration of the select() loop. The select loop completes an iteration everytime an async event happens, or the select() call times out. The default timeout in asyncore.py is 30 seconds... hence the 30 second delay. One way to prove this is to establish another connection to the medusa server before the 30 sec's expires, and this should kick the select loop to re-calculate the ready() methods. A server that is being hammered will not notice this problem because all the async events will be kicking the event loop all the time. However, to get snapier responces from an under-loaded server using ready(), set the timeout in loop() to something lower, like 1 sec. I dunno about the data not being processed correctly though... I'd have to look at your code to figure it out. I suspect that your ready() methods are not quite correct... ready() should return true and more() should return '' when the producer is finished. If ready() does not return true when the the producer is finished, asynchat will think that it is unfinished and blocked forever, so asynchat will never complete. > My script runs a http server on port 8000 by default. Any hits on it will > (or should) return a pair of timestamps. Try to rename ready() to see how it > should work. I thought of renaming ready() to something else when I found the vestigial ready() methods in filesys.py and status_handler.py. However, the code was pretty complete by the time I found them, and the reality is there will always be a potential problem with name clashes no-matter what name I choose. -- ---------------------------------------------------------------------- ABO: finger abo@m... for more info, including pgp key ---------------------------------------------------------------------- From abo@m... Wed Jan 31 12:18:04 2001 From: abo@m... (Donovan Baarda) Date: Wed, 31 Jan 2001 23:18:04 +1100 Subject: [Zope-dev] asynchat producer ready() method patch. In-Reply-To: <14965.46576.76669.281517@l...>; from dieter@h... on Mon, Jan 29, 2001 at 07:26:56PM +0100 References: <220829293@t...> <20010129185326.A31092@m...> <14965.46576.76669.281517@l...> Message-ID: <20010131231804.C1484@m...> On Mon, Jan 29, 2001 at 07:26:56PM +0100, Dieter Maurer wrote: > Donovan Baarda writes: > > On Sun, Jan 28, 2001 at 09:31:07PM +0100, Dieter Maurer wrote: > > > Donovan Baarda writes: > > > > I remember reading ages ago on a wishlist for ZServer that someone was > > > > thinking of adding blocking support to producers. This patch adds support > > > > for just that in a non-intrusive way. I haven't tried applying this patch to > > > > ZServer, but it _should_ work. > > > What does "blocking support dor producers" mean? > > > > Disclaimer: you must be moderately familiar with ZServer/medusa/asynchat for > > the following to make sense; > > > > Asynchat used by ZServer and medusa use producers as a way to "push" data > > through an async interface. The current version of asynchat requires that > > producers be able to produce data immediately on calling their more() > > method, untill they are finished, when they return ''. > > > > My patch adds support for producers being able to "block" by letting > > asynchat know if they are ready via a ready() method. This allows you to use > > the simple producer model to push data that may not be immediately > > available. An example would be a http-fetch producer that fetches data from > > upstream for a proxy. > > > > This patch is only useful for people creating or modifying things like > > medusa or ZServer. > Thank you for this good explanation. > > By now, I think, it would not be usefull for ZServer, at least > not for standard operations. > > ZServer is a multi-threaded server, where different requests > are serviced by different threads. > The thread may block and does simply not return from "more()". > > On first thought, you patch might free some blocked threads > to do productive work. However, Zope's transaction system > is linked to thread management. If you patch would be used > for that, Zope's transaction subsystem would need to be > rewritten. Hmmm, it's been a while since I looked at ZServer, I might have to take another look. I thought that it pre-spawned a number of threads that each ran their own asyncore.loop event handler. I thought this was a neat way to get the benefits of async event handlers and distributing threads on multiple processors. Unless it spawns new threads when it needs them, it would still be possible for ZServer to get blocked if all the threads were blocked. However, my patch is only useful if you _have_ producers that can block. It is usually possible to re-structure your system so that no producers block (typicaly by using a dispatcher instead of a producer). The ready() block feature is really a shortcut way of modifying an existing producer-based design when you realise that the producer can block; ie an alternative way of implementing it that may or may not be better. In my case I wanted to create a cached-ftp-backend filesystem that medusa could serve via ftp or http. Unfortunately the existing ftp and http server implementations used producers to pull data from the abstract filesystem. Since my "filesystem" could block pretty severly, I either needed to completely re-write the http and ftp servers to use dispatchers instead of producers, or implement blocking for producers (or use a seperate thread for each request, but I thought that defeated the purpose of using an async event loop). I really must look at ZServer to figure out how it handles things... I suspect that my patch by itself would not _hurt_ ZServer, but perhaps if you started using the ready() feature it provides it might cause problems. I'll have a look at ZServer again soon and give you an evaluation of how my patch would interact with it. Who knows, someone might find a need for blocking producers as ZServer evolves, and if people know the patch exists, they might use it instead of re-structuring to get around it :-) -- ---------------------------------------------------------------------- ABO: finger abo@m... for more info, including pgp key ---------------------------------------------------------------------- From kthangavelu@e... Wed Jan 31 19:21:42 2001 From: kthangavelu@e... (ender) Date: Wed, 31 Jan 2001 11:21:42 -0800 Subject: [medusa] Re: [Zope-dev] asynchat producer ready() method patch. In-Reply-To: <20010131231804.C1484@m...> References: <220829293@t...> <14965.46576.76669.281517@l...> <20010131231804.C1484@m...> Message-ID: <01013111214200.00885@l...> On Wednesday 31 January 2001 04:18, Donovan Baarda wrote: > On Mon, Jan 29, 2001 at 07:26:56PM +0100, Dieter Maurer wrote: > > Donovan Baarda writes: > > > On Sun, Jan 28, 2001 at 09:31:07PM +0100, Dieter Maurer wrote: > > > > Donovan Baarda writes: > > > > > I remember reading ages ago on a wishlist for ZServer that > > > > > someone was thinking of adding blocking support to producers. > > > > > This patch adds support for just that in a non-intrusive way. I > > > > > haven't tried applying this patch to ZServer, but it _should_ > > > > > work. > > > > > > > > What does "blocking support dor producers" mean? > > > > > > Disclaimer: you must be moderately familiar with > > > ZServer/medusa/asynchat for the following to make sense; > > > > > > Asynchat used by ZServer and medusa use producers as a way to "push" > > > data through an async interface. The current version of asynchat > > > requires that producers be able to produce data immediately on calling > > > their more() method, untill they are finished, when they return ''. > > > > > > My patch adds support for producers being able to "block" by letting > > > asynchat know if they are ready via a ready() method. This allows you > > > to use the simple producer model to push data that may not be > > > immediately available. An example would be a http-fetch producer that > > > fetches data from upstream for a proxy. > > > > > > This patch is only useful for people creating or modifying things like > > > medusa or ZServer. > > > > Thank you for this good explanation. > > > > By now, I think, it would not be usefull for ZServer, at least > > not for standard operations. > > > > ZServer is a multi-threaded server, where different requests > > are serviced by different threads. > > The thread may block and does simply not return from "more()". > > > > On first thought, you patch might free some blocked threads > > to do productive work. However, Zope's transaction system > > is linked to thread management. If you patch would be used > > for that, Zope's transaction subsystem would need to be > > rewritten. > > Hmmm, it's been a while since I looked at ZServer, I might have to take > another look. I thought that it pre-spawned a number of threads that each > ran their own asyncore.loop event handler. I thought this was a neat way to > get the benefits of async event handlers and distributing threads on > multiple processors. Unless it spawns new threads when it needs them, it > would still be possible for ZServer to get blocked if all the threads were > blocked. disclaimer: insert here. zserver has a main thread which binds and listens, passing requests to a pool of zpublisher threads. the zpublisher threads communicate with the client via the main threads dispatcher, ie output is funneled back into the main thread's dispatcher so as not to tie up threads talking to distant clients. reference: Zserver/medusa/select_trigger.py python has a global interpreter lock, so multiple cpus have little utility. > However, my patch is only useful if you _have_ producers that can block. It > is usually possible to re-structure your system so that no producers block > (typicaly by using a dispatcher instead of a producer). The ready() block > feature is really a shortcut way of modifying an existing producer-based > design when you realise that the producer can block; ie an alternative way > of implementing it that may or may not be better. i've taken a quick look at the patch, i find it interesting in that it provides i think a way to implement a Proactor type server vs the standard medusa Reactor style server. reference: Patterns of System Architecture Vol II. Douglas Schmidt et al reference: http://www.cs.wustl.edu/~schmidt/PDF/proactor.pdf and a webserver that makes use of this pattern http://www.cs.wustl.edu/~jxh/research/ kapil > In my case I wanted to create a cached-ftp-backend filesystem that medusa > could serve via ftp or http. Unfortunately the existing ftp and http server > implementations used producers to pull data from the abstract filesystem. > Since my "filesystem" could block pretty severly, I either needed to > completely re-write the http and ftp servers to use dispatchers instead of > producers, or implement blocking for producers (or use a seperate thread > for each request, but I thought that defeated the purpose of using an async > event loop). > I really must look at ZServer to figure out how it handles things... I > suspect that my patch by itself would not _hurt_ ZServer, but perhaps if > you started using the ready() feature it provides it might cause problems. > I'll have a look at ZServer again soon and give you an evaluation of how my > patch would interact with it. > Who knows, someone might find a need for blocking producers as ZServer > evolves, and if people know the patch exists, they might use it instead of > re-structuring to get around it :-) From jbenninghoff@y... Mon Feb 5 05:40:17 2001 From: jbenninghoff@y... (John Benninghoff) Date: Mon, 05 Feb 2001 05:40:17 -0000 Subject: HTTP load generator Message-ID: <95lec1+ak31@e...> I'm thinking of writing an HTTP load generator using asynchat.py. I want to support a typical approach using scenarios much like LoadRunner and SilkPerformer do it. Has someone already done something like this? From rushing@n... Mon Feb 5 17:12:51 2001 From: rushing@n... (Sam Rushing) Date: Mon, 05 Feb 2001 09:12:51 -0800 Subject: [medusa] HTTP load generator References: <95lec1+ak31@e...> Message-ID: <3A7EDF13.C5BBCD34@n...> John Benninghoff wrote: > I'm thinking of writing an HTTP load generator using asynchat.py. I > want to support a typical approach using scenarios much like > LoadRunner and SilkPerformer do it. Has someone already done > something like this? I've written a few small benchmarking tools, but haven't worked on any separate load-generator projects. Usually it's medusa or asyncore that I'm benchmarking, and I always feel a little uncomfortable benchmarking something with itself. -Sam From petri_savolainen@y... Wed Feb 21 18:02:09 2001 From: petri_savolainen@y... (Petri Savolainen) Date: Wed, 21 Feb 2001 18:02:09 -0000 Subject: HTTP load generator In-Reply-To: <95lec1+ak31@e...> Message-ID: <970vr1+6kos@e...> There is an open-source tool set called OpenSTA (.org) which has a nice GUI, records test scripts as you browse and has a whole testing language. Regards, Petri --- In medusa@y..., "John Benninghoff" wrote: > I'm thinking of writing an HTTP load generator using asynchat.py. I > want to support a typical approach using scenarios much like > LoadRunner and SilkPerformer do it. Has someone already done > something like this? From jbenninghoff@y... Thu Feb 22 03:45:17 2001 From: jbenninghoff@y... (John Benninghoff) Date: Wed, 21 Feb 2001 19:45:17 -0800 (PST) Subject: [medusa] Re: HTTP load generator In-Reply-To: <970vr1+6kos@e...> Message-ID: <20010222034517.23687.qmail@w...> I checked it out. It only runs on NT and I could not find a way to run it on more than one machine despite the fact that it was built using CORBA! Besides, I prefer Python to the yet another specialized language. But thanks for the feedback anyway. --- Petri Savolainen wrote: > There is an open-source tool set called OpenSTA > (.org) which has a > nice GUI, records test scripts as you browse and has > a whole testing > language. > > Regards, > > Petri > > --- In medusa@y..., "John Benninghoff" > wrote: > > I'm thinking of writing an HTTP load generator > using asynchat.py. > I > > want to support a typical approach using scenarios > much like > > LoadRunner and SilkPerformer do it. Has someone > already done > > something like this? > > ===== John Benninghoff It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change. -- Charles Darwin __________________________________________________ Do You Yahoo!? Yahoo! Auctions - Buy the things you want at great prices! http://auctions.yahoo.com/ From reynol@p... Fri Feb 23 04:48:29 2001 From: reynol@p... (Samuel Reynolds) Date: Thu, 22 Feb 2001 21:48:29 -0700 Subject: Medusa on Macintosh [long] In-Reply-To: <3A82E049.40EE0A16@n...> References: Message-ID: ---------------------- multipart/mixed attachment Hi, all. Please go easy on a Python/Medusa newbie. ;-) I just picked up Python (Mac Python 2.0) a few weeks ago, and I'm working on getting Medusa up and running on Macintosh. (I may try to move "up" to Zope, but the Medusa docs say that it "should" work on Macintosh, whereas the Zope docs simply say *nix and Windows. A brief introduction: I'm a professional software developer (mainly OO C++). I've also done extensive scripting (including internet work) in Frontier on Mac and Windows for several years (until very recently, I was a Support Associate for Userland Software, publisher of Frontier). I just finished reading through the medusa@yahoogroups.com archives. In the process, I found I could hack os_filesystem and http_server to make them work, but couldn't get ftp_server to work with my changes. I kept getting bunches of extra colons (the Mac filesystem path separator) in the paths, so ftp_server couldn't find the files. I have exchanged email directly with Sam Rushing about this, but I'd like to open the discussion to the development community. Since os_filesystem is intended to function as a virtual-*nix filesystem, I backed up and took another look. I reverted to the original code (except for the date coercions), and studied filesys.py. It seemed to make an awful lot of calls to os.path (via self.path_module). That's why there were so many glitches when I tried to use it on Mac. It worked okay on Unix and Windows because "/" is recognized as a proper path separator on both of those systems. So I rewrote os_filesystem to always use posixpath (i.e., the "/" separator) except when it had to go to the actual files, and did the translation in only one place (the translate method). The FTP server suddenly worked perfectly, and the http server (actually, default_handler.py) required only a couple of small changes. Other advantages this provides (as long as all file references are processed via the os_filesystem): o External code (servers, etc) only uses virtual-posix paths (e.g., http server need do no translations). o Mapping to os-specific paths is kept internal to the os_filesystem. (Though, since the methods are all public, the translate() method can be used by client code if necessary.) o I duplicated the various path and file/directory methods in os_filesystem, so it can provide the complete file-interaction abstraction. o Except by going through the translate method to get the actual system-specific file path, the client code cannot pop out past the virtual-root directory (i.e., can't pop out past "/"). I've attached the modified filesys.py file, as well as a test program that exercises it. I think it will work on other systems as-is, but I'd appreciate if someone could try it out on Windows and Unix. Any takers? (By all means, read through the changed file first, so you understand exactly what it does.) If I get the chance, I intend to test it on Solaris and WinNT4, but I'd like to get alternate input, as well--I'm sure there's code built on top of Medusa that I'm not aware of, and that the changed os_filesystem might adversely impact. Note: In the test program (filesys_test.py), you may need to add a section for your OS to set the real_* paths as test reference values. ON A SECOND TOPIC: How can I get access to the CVS repository? I saw the commands in the archive, but I'm running the CVS client on a Mac, so it looks a bit different from the Unix command-line access. MacCVS Pro asks for server, path, login, and password. Perhaps I need to contact Sam Rushing directly? - Sam ---------------------- multipart/mixed attachment A non-text attachment was scrubbed... Name: filesys.py Type: application/mac-binhex40 Size: 18341 bytes Desc: not available Url : http://mail.python.org/pipermail-21/medusa-dev/attachments/70e3a2b7/filesys.py ---------------------- multipart/mixed attachment A non-text attachment was scrubbed... Name: filesys_test.py Type: application/mac-binhex40 Size: 9255 bytes Desc: not available Url : http://mail.python.org/pipermail-21/medusa-dev/attachments/9100bfbb/filesys_test.py ---------------------- multipart/mixed attachment ________________________________________ Samuel Reynolds Spinward Stars: http://www.spinwardstars.com/ Reynolds Virtual Workshop: http://www.primenet.com/~reynol reynol@p... samuel_reynolds@c... ---------------------- multipart/mixed attachment-- From reynol@p... Fri Feb 23 15:14:04 2001 From: reynol@p... (Samuel Reynolds) Date: Fri, 23 Feb 2001 08:14:04 -0700 Subject: [medusa] Medusa on Macintosh [long] In-Reply-To: References: <3A82E049.40EE0A16@n...> Message-ID: >I reverted to the original code (except for the date >coercions) To clarify about the date coercions (and get them into the archive): On Unix and Windows, the date epoch is 1970. For dates from this epoch, it's safe to coerce times to ints. On the Mac, on the other hand, the epoch is 1907. As a result, an attempt to coerce to int results in a range overflow. Instead, coerce to long. In particular: o Change all instances of [ int(time.time()) ] to [ long(time.time())] ]. o Change all instances of [ int(time.mktime()) ] to [ long(time.mktime()) ]. In the medusa-src-20000601 tarball, this affects the files eventloop.py, http_date.py, http_server.py, resolver.py, and status_handler.py. Sam Rushing has suggested that there are other date issues, as well, that I haven't run into. - Sam ________________________________________ Samuel Reynolds Spinward Stars: http://www.spinwardstars.com/ Reynolds Virtual Workshop: http://www.primenet.com/~reynol reynol@p... samuel_reynolds@c... From weissblitz@y... Sun Mar 4 15:50:01 2001 From: weissblitz@y... (Weiss Blitz) Date: Sun, 04 Mar 2001 15:50:01 -0000 Subject: How to use the Medusa Monitor Server ??? Message-ID: <97to79+10t00@e...> Can anybody shed some light about how to use the Monitor Server to check and/or update a Medusa server with new code? I'm running Medusa HTTP,FTP and Monitor Servers on an W2k machine. I can login into the monitor server and get the regular Python interpreter prompt: python monitor_client_win32.py (my-ip) 8899 Enter Password: secret Python 2.0 (#8, Oct 19 2000, 11:30:05) [MSC 32 bit (Intel)] Copyright (c) 2000 ActiveState Tool Corp. Copyright (c) 2000 BeOpen.com. All Rights Reserved. Copyright (c) 1995-2000 Corporation for National Research Initiatives. All Rights Reserved. Copyright (c) 1991-1995 Stichting Mathematisch Centrum, Amsterdam. All Rights Reserved. Welcome to >>> dir() ['__builtins__'] >>> print dir() ['_', '__builtins__'] Seems I'm not getting much here besides a virgin Python interpreter. How can I see my Medusa namespace? Do I have to run Medusa differenly? For example, let say I made some changes to the FTP server and now I want to reload that code without stoping/starting the whole Medusa server. Thanks, Carlos :) BTW: Thanks to Sam and Guido and all others who made Medusa and Python possible! Python/Medusa rocks! :) From robin@j... Sun Mar 4 16:38:00 2001 From: robin@j... (Robin Becker) Date: Sun, 4 Mar 2001 16:38:00 +0000 Subject: [medusa] How to use the Medusa Monitor Server ??? In-Reply-To: <97to79+10t00@e...> References: <97to79+10t00@e...> Message-ID: In article <97to79+10t00@e...>, Weiss Blitz writes ... well everything hangs off the __main__ module. So you could try import __main__ dir(__main__) to retry FTP stuff you'd need to find anything that had a reference to the ftp_server and del it and then reload(ftp_server) and redfine all the references. Seems quite hard from the monitor. Perhaps it would be better to just restart medusa somehow. -- Robin Becker From rushing@n... Sun Mar 4 21:40:02 2001 From: rushing@n... (Sam Rushing) Date: Sun, 04 Mar 2001 13:40:02 -0800 Subject: [medusa] How to use the Medusa Monitor Server ??? References: <97to79+10t00@e...> Message-ID: <3AA2B632.937E0399@n...> Weiss Blitz wrote: > Can anybody shed some light about how to use the Monitor Server to > check and/or update a Medusa server with new code? > > Welcome to de6a54> > >>> dir() > ['__builtins__'] > >>> print dir() > ['_', '__builtins__'] > > Seems I'm not getting much here besides a virgin Python interpreter. > How can I see my Medusa namespace? > Do I have to run Medusa differenly? Try 'from __main__ import *' The non-secure version of the interpreter prints this as a hint in the prompt, it should probably just be done for you. If you access the '__main__' module directly you can change top-level values there [which is a little more dangerous if you don't want to take the server down] > For example, let say I made some changes to the FTP server and now I > want to reload that code without stoping/starting the whole Medusa > server. > Something like this should work: reload(ftp_server) __main__.ftp.close() # close the server socket new_ftp = ftp_server.ftp_server (...) # create and bind a new one __main__.status_objects.remove (__main__.ftp) __main__.ftp = new_ftp __main__.status_objects.append(new_ftp) Note that the 'old' ftp server is still pointing at the old code objects... same with any existing clients. When the last of those closes, all references to the old code will go away and you'll be fully 'upgraded'. -Sam From lucifer@e... Tue Mar 6 21:52:06 2001 From: lucifer@e... (Lucifer) Date: Tue, 6 Mar 2001 17:52:06 -0400 Subject: SSL on medusa In-Reply-To: Message-ID: <200103062152.RAA01446@L...> Is there any SSL support on medusa ? Thanks. Nicolás Echániz Rakar Producciones Argentina From rushing@n... Wed Mar 7 00:44:16 2001 From: rushing@n... (Sam Rushing) Date: Tue, 06 Mar 2001 16:44:16 -0800 Subject: [medusa] SSL on medusa References: <200103062152.RAA01446@L...> Message-ID: <3AA58460.CC4404A@n...> Lucifer wrote: > Is there any SSL support on medusa ? 1) On nightmare.com I think you'll find a link to a description of how to use stunnel (I think that's the name of it). 2) Python 2.0 supports SSL sockets directly, it's probably just a matter of figuring out how to support a different flavor of socket object. I remember building the SSL support a few months back, but couldn't figure out the certificate stuff very quickly, didn't go any further. -Sam From lucifer@e... Thu Mar 8 21:27:25 2001 From: lucifer@e... (Lucifer) Date: Thu, 8 Mar 2001 17:27:25 -0400 Subject: kill open port In-Reply-To: <200103062152.RAA01446@L...> Message-ID: <200103082127.RAA02205@L...> (Thanks for your answer on the SSL subject Sam.) I have the following problem. I've been writing an html_chat_server and it's working nice now (on Netscape and Explorer as well). I've built it on top of the chat_server. But for some reason, the port on which I normally run it has now been left open and I don't know how to close it. So I keep getting an 'Address already in use' error every time I try to run the server. How do I explicitly close a port that's been left open? and how did this happen ? It's been open for several days now and I don't know why because the chat server is not running. Any hints ? Nicolás Echániz RAKAR Producciones Argentina From kthangavelu@e... Fri Mar 9 03:24:29 2001 From: kthangavelu@e... (ender) Date: Thu, 8 Mar 2001 19:24:29 -0800 Subject: [medusa] kill open port In-Reply-To: <200103082127.RAA02205@L...> References: <200103082127.RAA02205@L...> Message-ID: <01030819242903.13837@l...> >>But for some reason, the port on which I normally run it has now been left >> open and I don't know how to close it. So I keep getting an 'Address >> already in use' error every time I try to run the server. >> >>How do I explicitly close a port that's been left open? and how did this >> happen ? >> >>It's been open for several days now and I don't know why because the chat >> server is not running. in your chat server after you create the socket (before you bind) you should do yoursocket.set_reuse_addr() else ths os won't let you reclaim, how long it won't depends on your os and your configuration. my linux box lets me do it after a couple of minutes, but its pretty depedent on your system. cheers kapil thangavelu From eugene.leitl@l... Fri Mar 9 16:24:18 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Fri, 9 Mar 2001 17:24:18 +0100 (MET) Subject: Medusa/Bobo nonblocking? Message-ID: I don't need the high performance of Medusa, but I need it to be nonblocking, as my cgi-bin queries can take awhile. From seconds to minutes. I would also rather like to be able embed Python into server-side HTML as well as cgi-bin which doesn't invoke an external instance of Python. The server doesn't need to be high-performance, but I'd like it to be relatively small. It would be nice if the solution was also resistant to memory leaks, but that's a little too much to ask for, given above contraints. Does Medusa/Bobo fit the ticket? It's for a commercial product, btrw. Or should I just take a good small http server written in C, and extend Python with it? Thanks for your thoughts. TIA, -- Eugene From rushing@n... Fri Mar 9 20:08:58 2001 From: rushing@n... (Sam Rushing) Date: Fri, 09 Mar 2001 12:08:58 -0800 Subject: [medusa] Medusa/Bobo nonblocking? References: Message-ID: <3AA9385A.FD57541C@n...> Eugene Leitl wrote: > I don't need the high performance of Medusa, but I need it to be > nonblocking, as my cgi-bin queries can take awhile. From seconds to > minutes. I would also rather like to be able embed Python into server-side > HTML as well as cgi-bin which doesn't invoke an external instance of > Python. The server doesn't need to be high-performance, but I'd like it to > be relatively small. It would be nice if the solution was also resistant > to memory leaks, but that's a little too much to ask for, given above > contraints. > > Does Medusa/Bobo fit the ticket? It's for a commercial product, btrw. > > Or should I just take a good small http server written in C, and extend > Python with it? Depends on the nature of you CGI scripts. If they are taking a long time because they're doing work that would cause a non-blocking process to block for long periods of time (say, long SQL queries), then you either won't get any advantage, or you'll have to rewrite your cgi to be event-driven. Another approach to this is to use an async http server like squid as a reverse proxy. (check out http://pasadena.wr.usgs.gov/stans/slashdot.html for an explanation of this setup) -Sam From eugene.leitl@l... Fri Mar 9 20:45:04 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Fri, 09 Mar 2001 21:45:04 +0100 Subject: [medusa] Medusa/Bobo nonblocking? References: <3AA9385A.FD57541C@n...> Message-ID: <3AA940D0.D8748B54@l...> Sam Rushing wrote: > Depends on the nature of you CGI scripts. If they are taking a long time > because they're doing work that would cause a non-blocking process to block > for long periods of time (say, long SQL queries), then you either won't get Yeah, they're doing a database search, and producing a tree of .png images plus a few user controls mapped to a (actually, they'll be producing a piece of XML which is rendered by another (socket-decoupled) piece of code). Alternatively, this is going to be one big .png image with server-side clickable maps (would require a web server which understand clickable maps, the Tk tree widget and maintaining a list of clickable rectangles plus be heavy on bandwidth use for dialup users, though). > any advantage, or you'll have to rewrite your cgi to be event-driven. Care to outline the principle, or provide a pointer? No problem if you don't have the time nor the inclination, "event-driven" and "cgi" plugged into Google will eventually come up with something (so far, with surprisingly little, though). > Another approach to this is to use an async http server like squid as a Uh, I'd rather stay with something with a small footprint (a few pages of code) and Python or ANSI C. > reverse proxy. (check out http://pasadena.wr.usgs.gov/stans/slashdot.html for > an explanation of this setup) From rushing@n... Fri Mar 9 21:31:56 2001 From: rushing@n... (Sam Rushing) Date: Fri, 09 Mar 2001 13:31:56 -0800 Subject: [medusa] Medusa/Bobo nonblocking? References: <3AA9385A.FD57541C@n...> <3AA940D0.D8748B54@l...> Message-ID: <3AA94BCC.13E932C@n...> eugene.leitl@l... wrote: > Yeah, they're doing a database search, and producing a tree of .png images > plus a few user controls mapped to a
(actually, they'll be producing > a piece of XML which is rendered by another (socket-decoupled) piece of code). > Alternatively, this is going to be one big .png image with server-side > clickable maps (would require a web server which understand clickable maps, > the Tk tree widget and maintaining a list of clickable rectangles plus be > heavy on bandwidth use for dialup users, though). The big-files-through-small-pipes issue is what a reverse proxy is meant to address... if you have heavy-weight resources on the server that can nonetheless do their processing quickly, then the server dumps everything into the proxy quickly, which handles these tedious doling out of bytes to dialup users. This frees up the heavyweight resource (think apache process) to move on to the next job. > > > any advantage, or you'll have to rewrite your cgi to be event-driven. > Care to outline the principle, or provide a pointer? No problem if you > don't have the time nor the inclination, "event-driven" and "cgi" plugged > into Google will eventually come up with something (so far, with surprisingly > little, though). > If the back end of the system is accessed via another socket (is this what you mean by 'socket-decoupled'?) then you can implement the protocol that talks to it using an event-driven model. [i.e., state variables, callbacks, etc... look at any of the code that comes with medusa for examples] A common issue with web servers is talking to a database back-end. If you have access to docs for the wire protocol, then you can write an event-driven database client. Using it is tricky because you have that much more state to keep track of, but it can be done. There's a mysql client distributed with medusa, in misc/async_mysql.py -Sam From lucifer@e... Fri Mar 9 22:49:01 2001 From: lucifer@e... (Lucifer) Date: Fri, 9 Mar 2001 18:49:01 -0400 Subject: [medusa] kill open port In-Reply-To: <01030819242903.13837@l...> Message-ID: <200103092249.SAA02771@L...> the set_reuse_addr() was already there. Any other idea ? I´ve scanned the ports on the server and the port I used for the server is still open though there is no chat_server running. On Thu, 8 Mar 2001 19:24:29 -0800, ender wrote: > > >>But for some reason, the port on which I normally run it has now been left > >> open and I don't know how to close it. So I keep getting an 'Address > >> already in use' error every time I try to run the server. > >> > >>How do I explicitly close a port that's been left open? and how did this > >> happen ? > >> > >>It's been open for several days now and I don't know why because the chat > >> server is not running. > > in your chat server after you create the socket (before you bind) you should > do > yoursocket.set_reuse_addr() > > else ths os won't let you reclaim, how long it won't depends on your os and > your configuration. my linux box lets me do it after a couple of minutes, but > its pretty depedent on your system. > > cheers > > kapil thangavelu > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From rushing@n... Fri Mar 9 22:20:54 2001 From: rushing@n... (Sam Rushing) Date: Fri, 09 Mar 2001 14:20:54 -0800 Subject: [medusa] kill open port References: <200103092249.SAA02771@L...> Message-ID: <3AA95746.806A35E6@n...> Lucifer wrote: > the set_reuse_addr() was already there. > > Any other idea ? > > I´ve scanned the ports on the server and the port I used for the server is still open though there is no chat_server running. What happens if you telnet to that port? Some other process may be using it. Try using 'netstat -a'. Depending on your OS, you may be able to follow clues given there (say, inode number) to track down which process is holding the socket. I have a script for linux that may or may not make this easy, let me know if you'd like it... -Sam From greg@e... Fri Mar 9 23:16:30 2001 From: greg@e... (Gregory P. Smith) Date: Fri, 9 Mar 2001 15:16:30 -0800 Subject: [medusa] Medusa/Bobo nonblocking? In-Reply-To: <3AA94BCC.13E932C@n...>; from rushing@n... on Fri, Mar 09, 2001 at 01:31:56PM -0800 References: <3AA9385A.FD57541C@n...> <3AA940D0.D8748B54@l...> <3AA94BCC.13E932C@n...> Message-ID: <20010309151630.A9807@z...> > > > any advantage, or you'll have to rewrite your cgi to be event-driven. > > > Care to outline the principle, or provide a pointer? No problem if you > > don't have the time nor the inclination, "event-driven" and "cgi" plugged > > into Google will eventually come up with something (so far, with surprisingly > > little, though). > > > > If the back end of the system is accessed via another socket (is this what you > mean by 'socket-decoupled'?) then you can implement the protocol that talks to it > using an event-driven model. [i.e., state variables, callbacks, etc... look at any > of the code that comes with medusa for examples] > > A common issue with web servers is talking to a database back-end. If you have > access to docs for the wire protocol, then you can write an event-driven database > client. Using it is tricky because you have that much more state to keep track > of, but it can be done. There's a mysql client distributed with medusa, in > misc/async_mysql.py > > -Sam Depending upon the application the simpler-to-code approach (when no obvious non-blocking database interface already exists) of using an event driven server such as Medusa or TwistedPython to merely accept the requests for initial processing and hand them off to a thread/process pool for the actual blocking work to be done may be good enough. Greg (who wishes everything had a nonblocking interface and that python had continuations) From lucifer@e... Mon Mar 12 23:01:30 2001 From: lucifer@e... (Lucifer) Date: Mon, 12 Mar 2001 19:01:30 -0400 Subject: [medusa] kill open port In-Reply-To: <3AA95746.806A35E6@n...> Message-ID: <200103122301.TAA03179@L...> I'm using Linux Sam. Maybe your script would help. One more thing. I never explicitly kill the process when I restart the server. May that be the reason why the socket stays open even between reboots ? Thanks. On Fri, 09 Mar 2001 14:20:54 -0800, Sam Rushing wrote: > Lucifer wrote: > > > the set_reuse_addr() was already there. > > > > Any other idea ? > > > > I´ve scanned the ports on the server and the port I used for the server is still open though there is no chat_server running. > > What happens if you telnet to that port? Some other process may be using it. Try using 'netstat -a'. Depending on your OS, > you may be able to follow clues given there (say, inode number) to track down which process is holding the socket. I have a > script for linux that may or may not make this easy, let me know if you'd like it... > > -Sam > > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From lucifer@e... Wed Mar 14 07:28:59 2001 From: lucifer@e... (Lucifer) Date: Wed, 14 Mar 2001 03:28:59 -0400 Subject: [medusa] kill open port solved. In-Reply-To: <3AA95746.806A35E6@n...> Message-ID: <200103140728.DAA01079@L...> Hi everybody. The open port thing has been magically solved :) I asked the guys from cobaltracks to turn the server off and on again and it's gone. From lucifer@e... Wed Mar 14 07:38:50 2001 From: lucifer@e... (Lucifer) Date: Wed, 14 Mar 2001 03:38:50 -0400 Subject: server as backgroud process In-Reply-To: <3AA95746.806A35E6@n...> Message-ID: <200103140738.DAA01101@L...> Hi. I'm having this problem. When I run the start_medusa script I allways run it like this: python start_medusa.py & so that it runs as a background process, and it works fine. I normally run the start_medusa from a telnet session and then just exit and it keeps running fine. Now I have this web-chat server that I've built on top of the chat server. I run the chat server in the same way I run the medusa script and it works perfectly well. But when I exit the telnet session, it just gets killed after a while. Any hints? Thanks in advance for your time. Nicolás Echániz RAKAR Producciones From reynol@p... Wed Mar 14 15:37:35 2001 From: reynol@p... (Samuel Reynolds) Date: Wed, 14 Mar 2001 08:37:35 -0700 Subject: [medusa] server as backgroud process In-Reply-To: <200103140738.DAA01101@L...> References: <3AA95746.806A35E6@n...> Message-ID: >Hi. > >I'm having this problem. >When I run the start_medusa script I allways run it like this: > python start_medusa.py & >so that it runs as a background process, and it works fine. > >I normally run the start_medusa from a telnet session and then just exit a= nd it keeps running fine. > >Now I have this web-chat server that I've built on top of the chat server. >I run the chat server in the same way I run the medusa script and it works= perfectly well. But when I exit the telnet session, it just gets killed af= ter a while. > >Any hints? > >Thanks in advance for your time. > >Nicol=E1s Ech=E1niz >RAKAR Producciones nohup python start_medusa.py & - Sam ________________________________________ Samuel Reynolds Spinward Stars: http://www.spinwardstars.com/ Reynolds Virtual Workshop: http://www.primenet.com/~reynol reynol@p... samuel_reynolds@c... From lucifer@e... Wed Mar 14 22:29:20 2001 From: lucifer@e... (Lucifer) Date: Wed, 14 Mar 2001 18:29:20 -0400 Subject: [medusa] server as backgroud process In-Reply-To: Message-ID: <200103142229.SAA02707@L...> Thanks Sam. On Wed, 14 Mar 2001 08:37:35 -0700, Samuel Reynolds wrote: > >Hi. > > > >I'm having this problem. > >When I run the start_medusa script I allways run it like this: > > python start_medusa.py & > >so that it runs as a background process, and it works fine. > > > >I normally run the start_medusa from a telnet session and then just exit and it keeps running fine. > > > >Now I have this web-chat server that I've built on top of the chat server. > >I run the chat server in the same way I run the medusa script and it works perfectly well. But when I exit the telnet session, it just gets killed after a while. > > > >Any hints? > > > >Thanks in advance for your time. > > > >Nicolás Echániz > >RAKAR Producciones > > nohup python start_medusa.py & > > - Sam > > ________________________________________ > Samuel Reynolds > Spinward Stars: http://www.spinwardstars.com/ > Reynolds Virtual Workshop: http://www.primenet.com/~reynol > reynol@p... > samuel_reynolds@c... > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From weissblitz@y... Thu Mar 15 15:29:29 2001 From: weissblitz@y... (Weiss Blitz) Date: Thu, 15 Mar 2001 15:29:29 -0000 Subject: How to run Python/Medusa as a Win2000 service? Message-ID: <98qn4p+nc3p@e...> Does anyone have a tip on how to run Python/Medusa as a Win2000 service? Thanks in advance! From cwagner@s... Thu Mar 15 18:41:27 2001 From: cwagner@s... (Chuck Wagner) Date: Thu, 15 Mar 2001 12:41:27 -0600 Subject: [medusa] How to run Python/Medusa as a Win2000 service? References: <98qn4p+nc3p@e...> Message-ID: <003501c0ad7f$8d8417b0$9501c30a@b...> Well, as a starter on Python apps as a service, there are examples in Hammond's Python Programming on Win32 book. If you don't have that, you absolutely need it. Using those examples as a basis we were able to run our python app as a service on Win2k, and it's based on Sam's Asyncore/Asynchat architecture. We did have to put in a hack to asyncore to check the event flag after every timeout so that we could stop the service. I imagine that running medusa would involve a similar effort. Chuck ----- Original Message ----- From: "Weiss Blitz" To: Sent: Thursday, March 15, 2001 9:29 AM Subject: [medusa] How to run Python/Medusa as a Win2000 service? > Does anyone have a tip on how to > run Python/Medusa as a Win2000 service? > > Thanks in advance! > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From lucifer@e... Thu Mar 15 20:00:44 2001 From: lucifer@e... (Lucifer) Date: Thu, 15 Mar 2001 16:00:44 -0400 Subject: [medusa] kill open port In-Reply-To: <3AAFEE90.35718157@n...> Message-ID: <200103152000.QAA01276@L...> Thank you Sam· It will be useful if this happens again. On Wed, 14 Mar 2001 14:20:01 -0800, Sam Rushing wrote: > Lucifer wrote: > > > I'm using Linux Sam. Maybe your script would help. > > Better late than never. 8^) > (just checked it, appears to still work on a 2.4 kernel) > > [root@s... rushing]# python ./who_owns.py -s 890 udp > inode: 17987 > Name: rpc.statd > State: S (sleeping) > Pid: 1137 > PPid: 1 > TracerPid: 0 > Uid: 0 0 0 0 > Gid: 0 0 0 0 > FDSize: 32 > Groups: 0 1 2 3 4 6 10 11 > VmSize: 1176 kB > VmLck: 0 kB > VmRSS: 0 kB > VmData: 36 kB > VmStk: 12 kB > VmExe: 20 kB > VmLib: 1080 kB > SigPnd: 0000000000000000 > SigBlk: 0000000000000000 > SigIgn: 0000000000000000 > SigCgt: 0000000000004003 > CapInh: 0000000000000000 > CapPrm: 00000000fffffeff > CapEff: 00000000fffffeff > > -Sam > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > #!/usr/local/bin/python > # -*- Mode: Python; tab-width: 4 -*- > > import os > import regex > import string > import sys > > # linux-specific. > # find out what process is listening on a particular socket > > def inode_of_socket (port, type='tcp', server_only=1, remote=0): > target_port = string.upper ('%04x' % (string.atoi (port))) > lines = open ('/proc/net/%s' % type).readlines() > inode = 0 > for line in lines[1:]: > fields = string.split (line) > if remote: > index = 2 > else: > index = 1 > [addr, port] = string.split (fields[index], ':') > if port == target_port: > if (not server_only) or (addr == '00000000'): > inode = string.atoi (fields[9]) > break > return inode > > pid_regex = regex.compile ('\([0-9]+\)') > > def process_of_inode (inode): > #inode = '[0000]:%d' % inode > inode = 'socket:[%d]' % inode > cwd = os.getcwd() > try: > pids = filter ( > lambda x: pid_regex.match (x) == len(x), > os.listdir('/proc') > ) > pids.remove (str(os.getpid())) > for pid in pids: > fd_dir = '/proc/%s/fd' % pid > if os.path.isdir (fd_dir): > try: > os.chdir (fd_dir) > except OSError: > # hmm, a zombie? > if string.split (open('/proc/%s/stat' % pid).read())[2] == 'Z': > continue > links = map ( > os.readlink, > os.listdir ('.') > ) > if inode in links: > return pid > finally: > os.chdir (cwd) > return 0 > > def getopt (s): > import sys > if s in sys.argv: > sys.argv.remove (s) > return 1 > else: > return 0 > > if __name__ == '__main__': > if len(sys.argv) < 2: > print 'Usage: %s [-r|-s] [tcp|udp]' % sys.argv[0] > else: > remote = getopt ('-r') > server = getopt ('-s') > if len(sys.argv) < 3: > socket_type='tcp' > else: > socket_type=sys.argv[2] > port = sys.argv[1] > inode = inode_of_socket (port, socket_type, server_only=server, remote=remote) > if not inode: > print "Couldn't find inode for socket on %s port %s" % ( > socket_type, > port > ) > sys.exit(0) > else: > print 'inode: %d' % inode > process = process_of_inode (inode) > if not process: > print "Couldn't find process for inode %d" % inode > else: > print open('/proc/%s/status' % process).read() > > From eugene.leitl@l... Tue Mar 27 14:16:06 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Tue, 27 Mar 2001 14:16:06 -0000 Subject: freezing Medusa Message-ID: <99q7b6+chgh@e...> Anything running wrong when freezing Medusa? As root under Linux, I can run start_medusa.py unmodified as it comes out of the tarball. [root@p... medusa]# python start_medusa.py warning: Cannot do reverse lookup info: Medusa (V3.39) started at Tue Mar 27 16:12:58 2001 Hostname: 205.160.176.5 Port:8080 info: FTP server started at Tue Mar 27 16:12:58 2001 Authorizer: Hostname: pc27.infochem.de Port: 8021 info: Monitor Server (V1.28) started on port 9999 Chat Server (V1.6) started on port 8888 But not the frozen result: [root@p... medusa]# ./start_medusa warning: Cannot do reverse lookup info: Medusa (V3.39) started at Tue Mar 27 16:14:03 2001 Hostname: 205.160.176.5 Port:8080 info: FTP server started at Tue Mar 27 16:14:03 2001 Authorizer: Hostname: pc27.infochem.de Port: 8021 info: Monitor Server (V1.28) started on port 9999 Chat Server (V1.6) started on port 8888 Traceback (most recent call last): File "start_medusa.py", line 193, in ? os.setegid (gid) OSError: [Errno 1] Operation not permitted Huh? The culprit seems to be becoming nobody: # become 'nobody' if os.name == 'posix': import os if hasattr (os, 'seteuid'): # look in ~medusa/patches for {set,get}euid. import pwd [uid, gid] = pwd.getpwnam ('nobody')[2:4] os.seteuid (uid) os.setegid (gid) Can anybody clue me in? From eugene.leitl@l... Tue Mar 27 14:46:51 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Tue, 27 Mar 2001 16:46:51 +0200 (MET DST) Subject: [medusa] freezing Medusa In-Reply-To: <99q7b6+chgh@e...> Message-ID: It does run now. One-banana fix: I commented out the offending section below. Another question: if I'm doing object publishing, what is the best way to parse arguments and GET/POST requests, preferably a la Zope/Bobo, *without* requiring Zope/Bobo? I've tried Guido's example for SimpleHTTPServer.py import StringIO import SimpleHTTPServer BaseClass = SimpleHTTPServer.SimpleHTTPRequestHandler CounterTemplate = """

Server Statistics

This server has been accessed %d times. """ count = 0 class MyRequestHandler(BaseClass): def do_GET(self): global count count = count + 1 BaseClass.do_GET(self) def send_head(self): if self.path == "/counter.html": return self.send_counter() else: return BaseClass.send_head(self) def send_counter(self): self.send_response(200) self.send_header("Content-type", "text/html") self.end_headers() text = CounterTemplate % count return StringIO.StringIO(text) def test(): SimpleHTTPServer.test(MyRequestHandler) test() It does what I need, so in principle I don't need Medusa, but I don't have any clue how to pass arguments to the methods and parse GET/PUT requests the way CGI module does (packing them into dictionary). Confused in Krautland, Eugene On Tue, 27 Mar 2001 Eugene.Leitl@l... wrote: > > Anything running wrong when freezing Medusa? [...] > Huh? The culprit seems to be becoming nobody: > > # become 'nobody' > if os.name == 'posix': > import os > if hasattr (os, 'seteuid'): > # look in ~medusa/patches for {set,get}euid. > import pwd > [uid, gid] = pwd.getpwnam ('nobody')[2:4] > os.seteuid (uid) > os.setegid (gid) > > Can anybody clue me in? From eugene.leitl@l... Mon Apr 2 16:33:56 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Mon, 2 Apr 2001 17:33:56 +0200 (MET DST) Subject: any Medusa users out there? Message-ID: I'm hacking Medusa right, and would like to discuss a few issues with other Medusa users. It seems that medusa@yahoogroups.com is largely silent these days. Are any of you using Medusa still, or have you switched to Zope? From eugene.leitl@l... Mon Apr 2 17:06:25 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Mon, 2 Apr 2001 18:06:25 +0200 (MET DST) Subject: fighting handlers Message-ID: I've added the following handlers to start_medusa.py : import script_handler import persistent sch = script_handler.script_handler (fs) ph = script_handler.persistent_script_handler() ph.add_module ('per', persistent) hs.install_handler (sch) hs.install_handler (ph) However, this knocked out the default status handler: # Create a status handler. By default it binds to the URI '/status'... sh = status_handler.status_extension(status_objects) # ... and install it on the web server. hs.install_handler (sh) Since it's quite useful, I wanted to keep it. Is there a way, or is there just one handler allowed at a time? From t.c.jones@a... Mon Apr 2 17:53:49 2001 From: t.c.jones@a... (t.c.jones@a...) Date: Mon, 02 Apr 2001 16:53:49 +0000 Subject: [medusa] any Medusa users out there? Message-ID: <20010402165349.SOHD18668.mtiwmhc22.worldnet.att.net@w...> my site still uses medusa ..tom > > I'm hacking Medusa right, and would like to discuss a few issues with > other Medusa users. It seems that medusa@yahoogroups.com is largely silent > these days. > > Are any of you using Medusa still, or have you switched to Zope? > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > From kbd@i... Mon Apr 2 18:03:43 2001 From: kbd@i... (Davidson, Keith) Date: Mon, 2 Apr 2001 13:03:43 -0400 Subject: [medusa] any Medusa users out there? Message-ID: <3DCF393159D8D3119C1C00508B62B90622F259@n...> still using it too... -----Original Message----- From: t.c.jones@a... [mailto:t.c.jones@a...] Sent: Monday, April 02, 2001 12:54 PM To: medusa@yahoogroups.com Subject: Re: [medusa] any Medusa users out there? my site still uses medusa ..tom > > I'm hacking Medusa right, and would like to discuss a few issues with > other Medusa users. It seems that medusa@yahoogroups.com is largely silent > these days. > > Are any of you using Medusa still, or have you switched to Zope? > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ From weissblitz@y... Tue Apr 3 17:57:20 2001 From: weissblitz@y... (Weiss Blitz) Date: Tue, 03 Apr 2001 16:57:20 -0000 Subject: No subject Message-ID: <9acvdg+n9ao@e...> I'm running Medusa on a dual-processor Compaq Proliant server running Win-2K Advanced Server. When the application starts (even when idle) the CPU utilization jumps to 50% right off the bat. Does anyone knows if this is normal or if there is a way to improve CPU utilization? Thanks in advance, Carlos Perez From weissblitz@y... Tue Apr 3 17:58:25 2001 From: weissblitz@y... (Weiss Blitz) Date: Tue, 03 Apr 2001 16:58:25 -0000 Subject: CPU utilization optimization? Message-ID: <9acvfh+t5tj@e...> I'm running Medusa on a dual-processor Compaq Proliant server running Win-2K Advanced Server. When the application starts (even when idle) the CPU utilization jumps to 50% right off the bat. Does anyone knows if this is normal or if there is a way to improve CPU utilization? Thanks in advance, Carlos Perez From chrisk@a... Tue Apr 3 17:57:23 2001 From: chrisk@a... (Christopher Kolp) Date: Tue, 3 Apr 2001 12:57:23 -0400 Subject: [medusa] (unknown) In-Reply-To: <9acvdg+n9ao@e...> Message-ID: <001c01c0bc5f$280587e0$9865fea9@c...> Don't use Winblows as your OS. I could never get Medusa to run right on NT... > -----Original Message----- > From: Weiss Blitz [mailto:weissblitz@y...] > Sent: Tuesday, April 03, 2001 12:57 PM > To: medusa@yahoogroups.com > Subject: [medusa] (unknown) > > > I'm running Medusa on a dual-processor Compaq Proliant server running > Win-2K Advanced Server. When the application starts (even when idle) > the CPU utilization jumps to 50% right off the bat. Does anyone > knows if this is normal or if there is a way to improve CPU > utilization? > > Thanks in advance, > Carlos Perez > > > > ------------------------ Yahoo! Groups Sponsor > ---------------------~-~> > Do you have 128-bit SSL encryption server security? > Get VeriSign's FREE Guide, "Securing Your > Web Site for Business." Get it now! > http://us.click.yahoo.com/EVNB7A/c.WCAA/bT0EAA/CxaWlB/TM > -------------------------------------------------------------- > -------_-> > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > From billtut@m... Tue Apr 3 18:03:40 2001 From: billtut@m... (Bill Tutt) Date: Tue, 3 Apr 2001 10:03:40 -0700 Subject: [medusa] CPU utilization optimization? Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A626E@r...> Python isn't completely thread safe. It has a lock that must be acquired before any Python C API may be called. This is what causes the behavior you're seeing. The easiest way to work around the problem is to move more of your code into Python extensions written in C/C++/whatever that don't rely on Python API calls to do the majority of your work. (Assuming the performance you're getting isn't good enough.) Bill -----Original Message----- From: Weiss Blitz [mailto:weissblitz@y...]=20 Sent: Tuesday, April 03, 2001 9:58 AM To: medusa@yahoogroups.com Subject: [medusa] CPU utilization optimization? I'm running Medusa on a dual-processor Compaq Proliant server running=20 Win-2K Advanced Server. When the application starts (even when idle)=20 the CPU utilization jumps to 50% right off the bat. Does anyone=20 knows if this is normal or if there is a way to improve CPU=20 utilization? Thanks in advance, Carlos Perez =20 Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/=20 From billtut@m... Tue Apr 3 18:16:41 2001 From: billtut@m... (Bill Tutt) Date: Tue, 3 Apr 2001 10:16:41 -0700 Subject: [medusa] (unknown) Message-ID: <58C671173DB6174A93E9ED88DCB0883D02667B84@r...> Now now, the same thing will happen on any dual processor box regardless of the OS, it's a fundamental problem with the implementation of Python. Got medusa working quite well on NT, Bill -----Original Message----- From: Christopher Kolp [mailto:chrisk@a...]=20 Sent: Tuesday, April 03, 2001 9:57 AM To: medusa@yahoogroups.com Subject: RE: [medusa] (unknown) Don't use Winblows as your OS. I could never get Medusa to run right on NT... > -----Original Message----- > From: Weiss Blitz [mailto:weissblitz@y...] > Sent: Tuesday, April 03, 2001 12:57 PM > To: medusa@yahoogroups.com > Subject: [medusa] (unknown) >=20 >=20 > I'm running Medusa on a dual-processor Compaq Proliant server running=20 > Win-2K Advanced Server. When the application starts (even when idle)=20 > the CPU utilization jumps to 50% right off the bat. Does anyone=20 > knows if this is normal or if there is a way to improve CPU=20 > utilization? >=20 > Thanks in advance, > Carlos Perez >=20 >=20 >=20 > ------------------------ Yahoo! Groups Sponsor=20 > ---------------------~-~> > Do you have 128-bit SSL encryption server security? > Get VeriSign's FREE Guide, "Securing Your > Web Site for Business." Get it now! > http://us.click.yahoo.com/EVNB7A/c.WCAA/bT0EAA/CxaWlB/TM > -------------------------------------------------------------- > -------_-> >=20 >=20=20 >=20 > Your use of Yahoo! Groups is subject to=20 > http://docs.yahoo.com/info/terms/=20 >=20 >=20 =20 Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/=20 From greg@e... Tue Apr 3 21:18:04 2001 From: greg@e... (Gregory P. Smith) Date: Tue, 3 Apr 2001 13:18:04 -0700 Subject: [medusa] 50% cpu usage on dual processor box In-Reply-To: <58C671173DB6174A93E9ED88DCB0883D02667B84@r...>; from billtut@m... on Tue, Apr 03, 2001 at 10:16:41AM -0700 References: <58C671173DB6174A93E9ED88DCB0883D02667B84@r...> Message-ID: <20010403131804.A27258@z...> On Tue, Apr 03, 2001 at 10:16:41AM -0700, Bill Tutt wrote: > Now now, the same thing will happen on any dual processor box regardless > of the OS, it's a fundamental problem with the implementation of Python. > > Got medusa working quite well on NT, > Bill On a dual processor linux box I've never noticed this behavior. Why would python acquiring a lock on a dual processor windows box cause 50% cpu utilization? There is no need for it to be a spinlock. curious, Greg -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From weissblitz@y... Wed Apr 4 17:37:42 2001 From: weissblitz@y... (Weiss Blitz) Date: Wed, 04 Apr 2001 16:37:42 -0000 Subject: Medusa hangings waiting on keystroke!!! Message-ID: <9afikm+l46n@e...> I noticed Medusa stops running and if I press a key it will keep on running. Seems for some reason it's waiting on a kbd input. This is running on a dual-processor w2k Advanced server box with 260Mb of RAM. Any ideas on this? Thanks in advance, Carlos From eugene.leitl@l... Wed Apr 4 22:30:32 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Wed, 04 Apr 2001 23:30:32 +0200 Subject: [medusa] Medusa hangings waiting on keystroke!!! References: <9afikm+l46n@e...> Message-ID: <3ACB9278.C5C7DFAD@l...> Weiss Blitz wrote: > > I noticed Medusa stops running and if I press a key it will keep on > running. Seems for some reason it's waiting on a kbd input. This is > running on a dual-processor w2k Advanced server box with 260Mb of > RAM. Any ideas on this? I'm new to Medusa, but I've never seen such behaviour so far. I think it's a rather safe bet the problems are with the w2k server box. Try it with *nix instead. From greg@e... Wed Apr 4 21:17:56 2001 From: greg@e... (Gregory P. Smith) Date: Wed, 4 Apr 2001 13:17:56 -0700 Subject: [medusa] Medusa hangings waiting on keystroke!!! In-Reply-To: <9afikm+l46n@e...>; from weissblitz@y... on Wed, Apr 04, 2001 at 04:37:42PM -0000 References: <9afikm+l46n@e...> Message-ID: <20010404131756.A18614@z...> On Wed, Apr 04, 2001 at 04:37:42PM -0000, Weiss Blitz wrote: > I noticed Medusa stops running and if I press a key it will keep on > running. Seems for some reason it's waiting on a kbd input. This is > running on a dual-processor w2k Advanced server box with 260Mb of > RAM. Any ideas on this? > > Thanks in advance, > Carlos Perhaps its not medusa related at all: If you're running anything in a text window on windows and select a portion with the mouse or scroll back in the window I've noticed that it hangs the process within until you hit a key (causing the it to go out of selection or scroll back mode). bad behaviour on windows part. Greg From rushing@n... Wed Apr 4 21:36:05 2001 From: rushing@n... (Sam Rushing) Date: Wed, 04 Apr 2001 13:36:05 -0700 Subject: [medusa] Medusa hangings waiting on keystroke!!! References: <9afikm+l46n@e...> Message-ID: <3ACB85B5.6C9FDDE0@n...> Weiss Blitz wrote: > I noticed Medusa stops running and if I press a key it will keep on > running. Seems for some reason it's waiting on a kbd input. This is > running on a dual-processor w2k Advanced server box with 260Mb of > RAM. Any ideas on this? The resource kit used to come with tools for turning console-mode programs into services. Another resource would be Zope, I'm sure they've addressed techniques for running on nt. -Sam From z30303@h... Wed Apr 4 22:18:53 2001 From: z30303@h... (z30303@h...) Date: Wed, 04 Apr 2001 21:18:53 -0000 Subject: I have it working Message-ID: <9ag33t+eivo@e...> OK, not hard but a little annoying to set this up. Find instsrv.exe and srvany.exe (typically from Windows 2000 Resource Disk or NT Resource Disk). Usually, I put them into C:\WINNT\System32 at command enter 'instsrv.exe medusa "C:\WINNT\System32\srvany.exe"' fire up a registry editor (run->regedit) we will modify HKLM.SYSTEM.CurrentControlSet.Services.Medusa add a new Key, called "Parameters" under Parameters add a String called "Application", with a value of "C:\Python20\python.exe" under Parameters add a String called "AppParameters", with a value of "C:\Medusa\start_medusa.py" under Parameters add a String called "AppDirectory", with a value of "C:\Medusa" all quotes are optional (as far as i know), and all paths are relative to your particular installation. now you can issue "net start medusa" from command line, and it will run. "net stop medusa" will stop the service. You can also use the Management console. Good Luck From z30303@h... Wed Apr 4 22:22:42 2001 From: z30303@h... (z30303@h...) Date: Wed, 04 Apr 2001 21:22:42 -0000 Subject: How to run Python/Medusa as a Win2000 service? In-Reply-To: <003501c0ad7f$8d8417b0$9501c30a@b...> Message-ID: <9ag3b2+cm7p@e...> Check out the "I have it working" post I sent in a few minutes ago --- In medusa@y..., "Chuck Wagner" wrote: > Well, as a starter on Python apps as a service, there are examples in > Hammond's Python Programming on Win32 book. If you don't have that, you > absolutely need it. > > Using those examples as a basis we were able to run our python app as a > service on Win2k, and it's based on Sam's Asyncore/Asynchat architecture. We > did have to put in a hack to asyncore to check the event flag after every > timeout so that we could stop the service. > > I imagine that running medusa would involve a similar effort. > > Chuck > ----- Original Message ----- > From: "Weiss Blitz" > To: > Sent: Thursday, March 15, 2001 9:29 AM > Subject: [medusa] How to run Python/Medusa as a Win2000 service? > > > > Does anyone have a tip on how to > > run Python/Medusa as a Win2000 service? > > > > Thanks in advance! > > > > > > > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > > > > From weissblitz@y... Thu Apr 5 00:35:27 2001 From: weissblitz@y... (Weiss Blitz) Date: Wed, 04 Apr 2001 23:35:27 -0000 Subject: I have it working In-Reply-To: <9ag33t+eivo@e...> Message-ID: <9agb3v+9mgl@e...> Thanks for the info :) I have been using this method for about a week now. But srvany.exe opens python on a console window. Is there a way to prevent this console window from showing? --- In medusa@y..., z30303@h... wrote: > OK, not hard but a little annoying to set this up. > > Find instsrv.exe and srvany.exe (typically from Windows 2000 Resource > Disk or NT Resource Disk). Usually, I put them into C:\WINNT\System32 > > at command enter 'instsrv.exe medusa "C:\WINNT\System32\srvany.exe"' > > fire up a registry editor (run->regedit) > > we will modify HKLM.SYSTEM.CurrentControlSet.Services.Medusa > add a new Key, called "Parameters" > under Parameters add a String called "Application", with a value > of "C:\Python20\python.exe" > under Parameters add a String called "AppParameters", with a value > of "C:\Medusa\start_medusa.py" > under Parameters add a String called "AppDirectory", with a value > of "C:\Medusa" > > all quotes are optional (as far as i know), and all paths are > relative to your particular installation. > > now you can issue "net start medusa" from command line, and it will > run. "net stop medusa" will stop the service. You can also use the > Management console. > > Good Luck From cwagner@s... Thu Apr 5 00:59:26 2001 From: cwagner@s... (Chuck Wagner) Date: Wed, 4 Apr 2001 18:59:26 -0500 Subject: [medusa] Re: How to run Python/Medusa as a Win2000 service? References: <9ag3b2+cm7p@e...> Message-ID: <006101c0bd63$47fc8f80$9501c30a@b...> If you create the service using Hammond's guidelines, it to doesn't open any console windows - it runs using the PythonService.exe. ----- Original Message ----- From: To: Sent: Wednesday, April 04, 2001 4:22 PM Subject: [medusa] Re: How to run Python/Medusa as a Win2000 service? > Check out the "I have it working" post I sent in a few minutes ago > > --- In medusa@y..., "Chuck Wagner" wrote: > > Well, as a starter on Python apps as a service, there are examples > in > > Hammond's Python Programming on Win32 book. If you don't have that, > you > > absolutely need it. > > > > Using those examples as a basis we were able to run our python app > as a > > service on Win2k, and it's based on Sam's Asyncore/Asynchat > architecture. We > > did have to put in a hack to asyncore to check the event flag after > every > > timeout so that we could stop the service. > > > > I imagine that running medusa would involve a similar effort. > > > > Chuck > > ----- Original Message ----- > > From: "Weiss Blitz" > > To: > > Sent: Thursday, March 15, 2001 9:29 AM > > Subject: [medusa] How to run Python/Medusa as a Win2000 service? > > > > > > > Does anyone have a tip on how to > > > run Python/Medusa as a Win2000 service? > > > > > > Thanks in advance! > > > > > > > > > > > > > > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > > > > > > > > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From weissblitz@y... Thu Apr 5 01:07:42 2001 From: weissblitz@y... (Weiss Blitz) Date: Thu, 05 Apr 2001 00:07:42 -0000 Subject: CPU utilization optimization? PROBLEM FOUND! In-Reply-To: <58C671173DB6174A93E9ED88DCB0883D0A626E@r...> Message-ID: <9agd0e+bj2u@e...> Ok, I think I found the problem... I developed a Medusa based instant messaging server running on a dual- processor Compaq Proliant box under Win2000 Server. I noticed Medusa idled at 1-2%, one connection comes up and the CPU utilization jumps to 100%! Found that the asyncore.py select loop was being pegged as soon as the first connection was made. Also noticed this was so because the asynchat.writable method was always returning a '1' even when the ac_out_buffer output buffer was empty. I replaced the 'is' operator with the '==' and it now seems to work normally. Following is a copy of the original and the modified version of the asynchat.writable method: def writable (self): "predicate for inclusion in the writable for select()" # return len(self.ac_out_buffer) or len(self.producer_fifo) or (not self.connected) # this is about twice as fast, though not as clear. return not ( (self.ac_out_buffer is '') and self.producer_fifo.is_empty() and self.connected ) def writable (self): "predicate for inclusion in the writable for select()" # return len(self.ac_out_buffer) or len(self.producer_fifo) or (not self.connected) # this is about twice as fast, though not as clear. return not ( (self.ac_out_buffer == '') and self.producer_fifo.is_empty() and self.connected ) Not sure if this is the way it is on the original Medusa distribution, but at least that's the way mine was until now. Anyways, thanks to all who have spent time and efforts to make Medusa and Python even better tools! Carlos :) From weissblitz@y... Fri Apr 6 21:32:11 2001 From: weissblitz@y... (Weiss Blitz) Date: Fri, 06 Apr 2001 20:32:11 -0000 Subject: HELP! too many file descriptors in select()!!!! Message-ID: <9al94b+hkqb@e...> I got the following message from Python20/Medusa running on a Win2k Advanced Server: Traceback (most recent call last): File "zulu_start.py", line 254, in ? asyncore.loop() File "d:\\zuluforumserver\asyncore.py", line 139, in loop poll_fun (timeout, map) File "d:\\zuluforumserver\asyncore.py", line 67, in poll r,w,e = select.select (r,w,e, timeout) ValueError: too many file descriptors in select() Only seem to be around 50 to 60 socket connections when this happened ! Does anyone knows what's the limit on the selector descriptors?! Or a way around this? Carlos :( From greg@e... Fri Apr 6 21:44:44 2001 From: greg@e... (Gregory P. Smith) Date: Fri, 6 Apr 2001 13:44:44 -0700 Subject: [medusa] HELP! too many file descriptors in select()!!!! In-Reply-To: <9al94b+hkqb@e...>; from weissblitz@y... on Fri, Apr 06, 2001 at 08:32:11PM -0000 References: <9al94b+hkqb@e...> Message-ID: <20010406134444.A30921@z...> On Fri, Apr 06, 2001 at 08:32:11PM -0000, Weiss Blitz wrote: > I got the following message from Python20/Medusa running on a Win2k > Advanced Server: > > Traceback (most recent call last): > File "zulu_start.py", line 254, in ? > asyncore.loop() > File "d:\\zuluforumserver\asyncore.py", line 139, in loop > poll_fun (timeout, map) > File "d:\\zuluforumserver\asyncore.py", line 67, in poll > r,w,e = select.select (r,w,e, timeout) > ValueError: too many file descriptors in select() > > Only seem to be around 50 to 60 socket connections when this happened > ! > Does anyone knows what's the limit on the selector descriptors?! > Or a way around this? > > Carlos :( You need to recompile the _select.pyd Python module with FS_SETSIZE set to a reasonable number (such as 256). Windows, by default, only supports up to 64 file descriptors in a call to select. I believe they have changed the default compilation to python in python 2.1 to use 256 or 512. (you'll still hit this problem at those higher numbers of sockets; at that point you should look into making your application not require as many open sockets at once) Greg -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From greg@e... Fri Apr 6 21:46:16 2001 From: greg@e... (Gregory P. Smith) Date: Fri, 6 Apr 2001 13:46:16 -0700 Subject: [medusa] HELP! too many file descriptors in select()!!!! In-Reply-To: <20010406134444.A30921@z...>; from greg@e... on Fri, Apr 06, 2001 at 01:44:44PM -0700 References: <9al94b+hkqb@e...> <20010406134444.A30921@z...> Message-ID: <20010406134616.B30921@z...> > You need to recompile the _select.pyd Python module with FS_SETSIZE err that should read.. FD_SETSIZE or FD_SET_SIZE, double check. :) From jim@d... Sat Apr 7 18:09:33 2001 From: jim@d... (Jim Fulton) Date: Sat, 07 Apr 2001 13:09:33 -0400 Subject: [medusa] Re: CPU utilization optimization? PROBLEM FOUND! References: <9agd0e+bj2u@e...> Message-ID: <3ACF49CD.1AE9EE83@d...> Weiss Blitz wrote: > > Ok, I think I found the problem... > > I developed a Medusa based instant messaging server running on a dual- > processor Compaq Proliant box under Win2000 Server. > > I noticed Medusa idled at 1-2%, one connection comes up and the CPU > utilization jumps to 100%! > > Found that the asyncore.py select loop was being pegged as soon as > the first connection was made. Also noticed this was so because the > asynchat.writable method was always returning a '1' even when the > ac_out_buffer output buffer was empty. I replaced the 'is' operator > with the '==' and it now seems to work normally. Interesting. Did you have the select timeout set very low? Jim -- Jim Fulton mailto:jim@d... Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org From weissblitz@y... Sat Apr 7 19:45:15 2001 From: weissblitz@y... (Weiss Blitz) Date: Sat, 07 Apr 2001 18:45:15 -0000 Subject: HELP! too many file descriptors in select()!!!! In-Reply-To: <20010406134444.A30921@z...> Message-ID: <9ann7r+hqso@e...> Yikes!!!... Is 512 the maximum possible number of file descriptors under Win2k Advanced Server?! What about under Linux? Do I hear 32767 or higher? Please,please,please?!! This will greately limit the number of open sockets that Medusa can handle concurrently! My application is an instant-messenger server and needs to maintain an open socket connection with each client! Any suggestions? Carlos :( --- In medusa@y..., "Gregory P. Smith" wrote: > On Fri, Apr 06, 2001 at 08:32:11PM -0000, Weiss Blitz wrote: > > I got the following message from Python20/Medusa running on a Win2k > > Advanced Server: > > > > Traceback (most recent call last): > > File "zulu_start.py", line 254, in ? > > asyncore.loop() > > File "d:\\zuluforumserver\asyncore.py", line 139, in loop > > poll_fun (timeout, map) > > File "d:\\zuluforumserver\asyncore.py", line 67, in poll > > r,w,e = select.select (r,w,e, timeout) > > ValueError: too many file descriptors in select() > > > > Only seem to be around 50 to 60 socket connections when this happened > > ! > > Does anyone knows what's the limit on the selector descriptors?! > > Or a way around this? > > > > Carlos :( > > You need to recompile the _select.pyd Python module with FS_SETSIZE > set to a reasonable number (such as 256). Windows, by default, only > supports up to 64 file descriptors in a call to select. > > I believe they have changed the default compilation to python in > python 2.1 to use 256 or 512. > > (you'll still hit this problem at those higher numbers of sockets; at > that point you should look into making your application not require as > many open sockets at once) > > Greg > > -- > Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ > C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From greg@e... Sat Apr 7 20:06:17 2001 From: greg@e... (Gregory P. Smith) Date: Sat, 7 Apr 2001 12:06:17 -0700 Subject: [medusa] Re: HELP! too many file descriptors in select()!!!! In-Reply-To: <9ann7r+hqso@e...>; from weissblitz@y... on Sat, Apr 07, 2001 at 06:45:15PM -0000 References: <20010406134444.A30921@z...> <9ann7r+hqso@e...> Message-ID: <20010407120617.A10369@z...> On Sat, Apr 07, 2001 at 06:45:15PM -0000, Weiss Blitz wrote: > Yikes!!!... > > Is 512 the maximum possible number of file descriptors under Win2k > Advanced Server?! > > What about under Linux? > Do I hear 32767 or higher? Please,please,please?!! I never said 512 was the maximum, just suggested it as a possible value that's better than 64. You'll have to ask microsoft what the max is... (linux defaults to 1024 per process but its dynamically changable to high values using the /proc filesystem). > > This will greately limit the number of open sockets that Medusa can > handle concurrently! My application is an instant-messenger server > and needs to maintain an open socket connection with each client! > > Any suggestions? > Carlos :( Holding an open connection to each client even while idle is not necessarily good (but i agree, its much simpler). Is there any reason you have to? In mojonation for our message based protocol running on top of TCP, we close connections after an idle period or when nearning your limit of max connections and reestablish them as needed. We use relay servers for people who are behind NAT or firewalls that can't accept incoming connections when another party needs to reestablish a connection with them (but can make outgoing ones which if open are preferred over relay). Greg -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From zmbq@a... Mon Apr 9 08:56:01 2001 From: zmbq@a... (Itay Zandbank) Date: Mon, 9 Apr 2001 09:56:01 +0200 Subject: [medusa] Re: CPU utilization optimization? PROBLEM FOUND! References: <9agd0e+bj2u@e...> <3ACF49CD.1AE9EE83@d...> Message-ID: <005b01c0c0ca$8532ce40$05010101@s...> > > Found that the asyncore.py select loop was being pegged as soon as > > the first connection was made. Also noticed this was so because the > > asynchat.writable method was always returning a '1' even when the > > ac_out_buffer output buffer was empty. I replaced the 'is' operator > > with the '==' and it now seems to work normally. > Interesting. Did you have the select timeout set very low? The timeout in his case was irrelevant, since select always returned instantly. I once had the same problem, when I didn't close a dispatcher down properly. It took me a while to figure out why /tmp filled up with debug printings... From ecrespo@u... Tue Apr 10 12:19:55 2001 From: ecrespo@u... (Ernesto Nadir Crespo Avila) Date: Tue, 10 Apr 2001 07:19:55 -0400 Subject: Trouble with the CGI Message-ID: <01041007195500.01222@z...> I Can=B4t get the values passed to a web page, even that I tried the method= s Get and Post, the results were: With POST: The Page=B4s structure it=B4s the following:
The cgi "ej4.mpy" is: # -*- Mode: Python; tab-width: 4 -*- # send and process a POST form import sys, cgi print '' print '' print '' print print ' A Phyton Example' print '' print '' datos =3D sys.stdin.read() if datos: print '

The name is: %s

' % repr(cgi.parse_qs(datos) else: print '

There=B4s no data .

' print '' print '' The message returned by this method Error response Error code 500. Message: Internal Server Error. The Log message: 127.0.0.1:1119 - - [10/Apr/2001:01:21:59 -0400] "POST /webconflrp/cgi-bin/ej4.mpy HTTP/1.1" 500 289 ***************************************************************************= ** ***** The Method Get The Page=B4s structure it=B4s the following:
The cgi "ej4.mpy" is: # -*- Mode: Python; tab-width: 4 -*- # send and process a POST form import cgi print '' print '' print '' print print ' A Cgi example with Phyton' print '' print '' datos =3D cgi.FieldStorage() if datos.has_key('name'): print '

Your data: %s

' %datos['name'] else: print 'there=B4s no data' print '' print '' The page waits a lot of time for loading and the log returned: 127.0.0.1:1138 - - [10/Apr/2001:01:49:49 -0400] "GET /webconflrp/cgi-bin/ej5.mpy?name=3Dernesto HTTP/1.1" 500 116 How can I solve this? Thank you all in advance! Bye, Ernesto Crespo From rprice@u... Tue Apr 10 17:24:13 2001 From: rprice@u... (rprice@u...) Date: Tue, 10 Apr 2001 09:24:13 -0700 (PDT) Subject: [medusa] Trouble with the CGI In-Reply-To: <01041007195500.01222@z...> References: <01041007195500.01222@z...> Message-ID: <986919853.3ad333addcb46@w...> It's been a while since I've workd with cgi.py but I believe your problem is in reading datos['name']. Try datos['name'].value instead. datos['name'] may contain a list or string, so adapt your code accordingly. Ray Quoting Ernesto Nadir Crespo Avila : > I Can´t get the values passed to a web page, even that I tried the > methods > Get and Post, the results were: > > > With POST: > > The Page´s structure it´s the following: > >
name="name" size="15" maxlength="15"> > > > > The cgi "ej4.mpy" is: > > # -*- Mode: Python; tab-width: 4 -*- > # send and process a POST form > import sys, cgi > > print '' > print '' > print '' > print > print ' A Phyton Example' > print '' > print '' > datos = sys.stdin.read() > > if datos: > print '

The name is: %s

' % repr(cgi.parse_qs(datos) > else: > print '

There´s no data .

' > print '' > print '' > > The message returned by this method > Error response > Error code 500. > Message: Internal Server Error. > > The Log message: > 127.0.0.1:1119 - - [10/Apr/2001:01:21:59 -0400] "POST > /webconflrp/cgi-bin/ej4.mpy HTTP/1.1" 500 289 > > ***************************************************************************** > ***** The Method Get > The Page´s structure it´s the following: > >
> > > > > The cgi "ej4.mpy" is: > > # -*- Mode: Python; tab-width: 4 -*- > # send and process a POST form > import cgi > print '' > print '' > print '' > print > print ' A Cgi example with Phyton' > print '' > print '' > datos = cgi.FieldStorage() > > if datos.has_key('name'): > print '

Your data: %s

' %datos['name'] > else: > print 'there´s no data' > print '' > print '' > > The page waits a lot of time for loading and the log returned: > 127.0.0.1:1138 - - [10/Apr/2001:01:49:49 -0400] "GET > /webconflrp/cgi-bin/ej5.mpy?name=ernesto HTTP/1.1" 500 116 > > How can I solve this? > Thank you all in advance! > > Bye, Ernesto Crespo > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > From ecrespo@u... Wed Apr 11 04:16:25 2001 From: ecrespo@u... (Ernesto Nadir Crespo Avila) Date: Tue, 10 Apr 2001 23:16:25 -0400 Subject: [medusa] Trouble with the CGI In-Reply-To: <986919853.3ad333addcb46@w...> References: <01041007195500.01222@z...> <986919853.3ad333addcb46@w...> Message-ID: <01041023162500.13529@z...> El Mar 10 Abr 2001 12:24, escribiste: > It's been a while since I've workd with cgi.py but I believe your problem > is in reading datos['name']. Try datos['name'].value instead.=20 > datos['name'] may contain a list or string, so adapt your code accordingl= y. Thank you. But, now just appear: there=B4s no data And FieldStorage(None, None, [])=20=20 Bye, Ernesto Crespo # -*- Mode: Python; tab-width: 4 -*- # send and process a POST form import cgi =20 print '' print '' print '' print print ' A Cgi example with Phyton' print '' print '' datos =3D cgi.FieldStorage() =20 if datos.has_key('name'): print '

Your data: %s

' %datos['name'].value else: print '

there=B4s no data

' print datos print '' print '' From lou@h... Thu Apr 12 23:19:05 2001 From: lou@h... (lou@h...) Date: Thu, 12 Apr 2001 22:19:05 -0000 Subject: extended logging Message-ID: <9b59kp+q4f1@e...> I am using the version of medusa that came bundled with zope. Has anyone extended logging as to add the host header information. I have found the log routine but I do not know what variable name to reference this. Thanks in advance. Lou From rprice@u... Thu Apr 12 23:53:30 2001 From: rprice@u... (rprice@u...) Date: Thu, 12 Apr 2001 15:53:30 -0700 (PDT) Subject: medusa and apache proxy Message-ID: <987116010.3ad631ea5993a@w...> When a user accesses medusa directly (not through a proxy) it takes less than 1 second to display the page. When a user accesses the same medusa server page via an apache proxy server, the pages seem to hang at 66% loaded. It can then take anywhere from 2 to 60 seconds for the page to finally load. Another problem when accessing through the proxy is that the browser occasionally indicates 'contacting server' for many seconds before any information is transmitted. Any ideas why this would happen? FYI I am using the most current version of medusa along with the old, orignal CGIResponse.py ,cgi_module_publisher.py files. Ray From rushing@n... Fri Apr 13 06:12:45 2001 From: rushing@n... (Sam Rushing) Date: Thu, 12 Apr 2001 22:12:45 -0700 Subject: [medusa] medusa and apache proxy References: <987116010.3ad631ea5993a@w...> Message-ID: <3AD68ACD.ED5BAAB9@n...> rprice@u... wrote: > When a user accesses medusa directly (not through a proxy) it takes less than 1 > second to display the page. > > When a user accesses the same medusa server page via an apache proxy server, the > pages seem to hang at 66% loaded. It can then take anywhere from 2 to 60 > seconds for the page to finally load. Another problem when accessing through > the proxy is that the browser occasionally indicates 'contacting server' for > many seconds before any information is transmitted. > > Any ideas why this would happen? None yet... a couple of things to try: 1) what happens if you try squid instead of apache proxy? 2) what protocol versions (i.e., http/1.0 or 1.1) are being used between browser, proxy, and medusa? -Sam From rprice@u... Fri Apr 13 22:03:34 2001 From: rprice@u... (rprice@u...) Date: Fri, 13 Apr 2001 14:03:34 -0700 (PDT) Subject: [medusa] medusa and apache proxy In-Reply-To: <3AD68ACD.ED5BAAB9@n...> References: <987116010.3ad631ea5993a@w...> <3AD68ACD.ED5BAAB9@n...> Message-ID: <987195814.3ad769a6419fb@w...> I have more information, unfortunately not what Sam asked for, but hopefully helpful! BROWSER PLATFM PROXY? REPLY Netscape Linux no proxy no delay Netscape Win98 no proxy no delay Explorer Win98 no proxy no delay LWP Perl Linux no proxy 30 second delay lynx -dump Linux no proxy 30 second delay LWP Perl Linux with proxy 30 second delay lynx -dump Linux with proxy 30 second delay Netscape Linux with proxy 30 second delay Netscape Win98 with proxy no delay !!! Ray Quoting Sam Rushing : > rprice@u... wrote: > > > When a user accesses medusa directly (not through a proxy) it takes > less than 1 > > second to display the page. > > > > When a user accesses the same medusa server page via an apache proxy > server, the > > pages seem to hang at 66% loaded. It can then take anywhere from 2 to > 60 > > seconds for the page to finally load. Another problem when accessing > through > > the proxy is that the browser occasionally indicates 'contacting > server' for > > many seconds before any information is transmitted. > > > > Any ideas why this would happen? > > None yet... a couple of things to try: > > 1) what happens if you try squid instead of apache proxy? > 2) what protocol versions (i.e., http/1.0 or 1.1) are being used > between browser, > proxy, and medusa? > > -Sam > > > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > From lou@h... Fri Apr 13 22:49:22 2001 From: lou@h... (lou@h...) Date: Fri, 13 Apr 2001 21:49:22 -0000 Subject: extended logging In-Reply-To: <9b59kp+q4f1@e...> Message-ID: <9b7s92+prrr@e...> I found the answer in a zope archive: http://zope.nipltd.com/public/lists/zope-archive.nsf/242bb7cc2b2c343d802568ab003585d4/b5bad84d0e217ff280256994000ace5f?OpenDocument In ZServer/medusa/http_server.py, the "log" method of the class "http_request" can be modified to report the request hostname which is readily available via the get_header method in the same class. Here is my modified "log" method: def log (self, bytes): user_agent=self.get_header('user-agent') if not user_agent: user_agent='' referer=self.get_header('referer') #ADD host=self.get_header('host') if not referer: referer='' self.channel.server.logger.log ( self.channel.addr[0], #ADD %S ' - - [%s] "%s" %d %d "%s" "%s" %s\n' % ( self.log_date_string (time.time()), self.request, self.reply_code, bytes, referer, user_agent, # ADD host host ) ) This will add the host portion of the http header to the end of each log entry. where can I find the complete list of get_header returns??? Lou --- In medusa@y..., lou@h... wrote: > I am using the version of medusa that came bundled with zope. Has > anyone extended logging as to add the host header information. I have > found the log routine but I do not know what variable name to > reference this. Thanks in advance. > > Lou From rprice@u... Fri Apr 13 23:08:25 2001 From: rprice@u... (rprice@u...) Date: Fri, 13 Apr 2001 15:08:25 -0700 (PDT) Subject: No subject Message-ID: <987199705.3ad778d9410f2@w...> Latest information from the sysadmin working on this: I haven't tried with squid. I'm interested in doing so, but with my experience with squid (zero) I don't expect that to be a fair test. The protocol was HTTP/1.1 for all browsers and HTTP/1.0 for the request from apache to medusa. Using "lynx -head -dump" shows "HTTP/1.0 200 OK", but I believe that that is due to the proxied HTTP/1.0 request since apache is proxying for a name based virtual. Apache: 1.3.19 Medusa: 20000601 (right?) Browsers: (See other email for varying results) Internet Explorer 5.00.3105.0106 (Win 98) Netscape Communicator 4.75 (Win 98) Netscape Communicator 4.76 (Mandrake 7.2) Lynx Version 2.8.4dev.8 (Mandrake 7.2) Lynx Version 2.8.3dev.18 (Red Hat 6.2) LWP UserAgent.pm,v 1.73 (Mandrake 7.2) LWP UserAgent.pm,v 1.77 (Red Hat 6.2) Ray From lucifer@e... Mon Apr 16 23:41:05 2001 From: lucifer@e... (lucifer@e...) Date: Mon, 16 Apr 2001 22:41:05 -0000 Subject: Trouble with the CGI In-Reply-To: <01041023162500.13529@z...> Message-ID: <9bfse1+5bsd@e...> I've written my own code to replace cgi.Fieldstorage() but I don't know if it will be ok for you. Has anybody ever got cgi.FieldStorage() to work inside medusa ? --- In medusa@y..., Ernesto Nadir Crespo Avila wrote: >=20 >=20 > El Mar 10 Abr 2001 12:24, escribiste: > > It's been a while since I've workd with cgi.py but I believe your problem > > is in reading datos['name']. Try datos['name'].value instead.=20 > > datos['name'] may contain a list or string, so adapt your code accordingly. > Thank you. >=20 > But, now just appear: >=20 > there=B4s no data >=20 > And >=20 > FieldStorage(None, None, [])=20=20 >=20 > Bye, Ernesto Crespo >=20 >=20 > # -*- Mode: Python; tab-width: 4 -*- > # send and process a POST form > import cgi >=20=20 > print '' > print '' > print '' > print > print ' A Cgi example with Phyton' > print '' > print '' > datos =3D cgi.FieldStorage() >=20=20 > if datos.has_key('name'): > print '

Your data: %s

' %datos['name'].value > else: > print '

there=B4s no data

' > print datos > print '' > print '' From rprice@u... Tue Apr 17 00:16:49 2001 From: rprice@u... (rprice@u...) Date: Mon, 16 Apr 2001 16:16:49 -0700 (PDT) Subject: [medusa] Re: Trouble with the CGI In-Reply-To: <9bfse1+5bsd@e...> References: <9bfse1+5bsd@e...> Message-ID: <987463009.3adb7d61cdbb3@w...> I have never used it from within medusa. I use the old bobo code to interact with my Python scripts. Ray Quoting lucifer@e...: > I've written my own code to replace cgi.Fieldstorage() but I don't > know if it will be ok for you. > > Has anybody ever got cgi.FieldStorage() to work inside medusa ? > > > --- In medusa@y..., Ernesto Nadir Crespo Avila wrote: > > > > > > El Mar 10 Abr 2001 12:24, escribiste: > > > It's been a while since I've workd with cgi.py but I believe your > problem > > > is in reading datos['name']. Try datos['name'].value instead. > > > datos['name'] may contain a list or string, so adapt your code > accordingly. > > Thank you. > > > > But, now just appear: > > > > there´s no data > > > > And > > > > FieldStorage(None, None, []) > > > > Bye, Ernesto Crespo > > > > > > # -*- Mode: Python; tab-width: 4 -*- > > # send and process a POST form > > import cgi > > > > print '' > > print '' > > print '' > > print > > print ' A Cgi example with Phyton' > > print '' > > print '' > > datos = cgi.FieldStorage() > > > > if datos.has_key('name'): > > print '

Your data: %s

' %datos['name'].value > > else: > > print '

there´s no data

' > > print datos > > print '' > > print '' > > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > From rprice@u... Tue Apr 17 00:35:29 2001 From: rprice@u... (rprice@u...) Date: Mon, 16 Apr 2001 23:35:29 -0000 Subject: medusa and apache proxy -more info In-Reply-To: <3AD68ACD.ED5BAAB9@n...> Message-ID: <9bfvk1+72vu@e...> With more testing we've found out that medusa works differenetly on Solaris (sunOS5) and Linux (I'm using Mandrake 7.1) at least when using the Linux telnet client: On solaris using telnet the server closes the connection immediately after the when using HTTP/1.0. When using HTTP/1.1 it will keep connected. Both as expected. On linux the Medusa server will wait 30 seconds before closing the session when using HTTP/1.0 and keeps the connection open much longer using HTTP/1.1. I've done these tests on Medusa versions 3.39 and 3.19 on linux and ONLY on 3.19 on sunOS5. Ray --- In medusa@y..., Sam Rushing wrote: > rprice@u... wrote: > > > When a user accesses medusa directly (not through a proxy) it takes less than 1 > > second to display the page. > > > > When a user accesses the same medusa server page via an apache proxy server, the > > pages seem to hang at 66% loaded. It can then take anywhere from 2 to 60 > > seconds for the page to finally load. Another problem when accessing through > > the proxy is that the browser occasionally indicates 'contacting server' for > > many seconds before any information is transmitted. > > > > Any ideas why this would happen? > > None yet... a couple of things to try: > > 1) what happens if you try squid instead of apache proxy? > 2) what protocol versions (i.e., http/1.0 or 1.1) are being used between browser, > proxy, and medusa? > > -Sam From abo@m... Tue Apr 17 02:27:51 2001 From: abo@m... (Donovan Baarda) Date: Tue, 17 Apr 2001 11:27:51 +1000 (EST) Subject: [medusa] Re: medusa and apache proxy -more info In-Reply-To: <9bfvk1+72vu@e...> References: <9bfvk1+72vu@e...> Message-ID: <987470871.3adb9c17659a5@m...> Quoting rprice@u...: > With more testing we've found out that medusa works differenetly on > Solaris (sunOS5) and Linux (I'm using Mandrake 7.1) at least when > using the Linux telnet client: > > On solaris using telnet the server closes the connection immediately > after the when using HTTP/1.0. When using HTTP/1.1 it will > keep connected. Both as expected. > > On linux the Medusa server will wait 30 seconds before closing the > session when using HTTP/1.0 and keeps the connection open much longer > using HTTP/1.1. The 30 sec delay sounds like the default medusa polling time on select. I don't know _why_ it would require another iteration through the loop for your application, but is it possible you have readable/writeable status changing around about the time you service the final push() for the connection. When the delay happens, it might not be quite ready the select, so it is excluded until the next iteration, when it gets selected immediately. Remember the readable/writeable test is performed once per iteration through the loop, so if something is not ready at the start of the loop, it will be excluded from the select until the next iteration. I suspect that if you change your polling time from the default, you can "adjust" the delay. Also, establishing another connection while one is "delayed" will kick the select loop, causing the delay to end immediately. Adjusting the poll time might be a quick fix, but I suspect you need to look at your readable/writeable tests. -- ABO: finger abo@m... for more information. From eugene.leitl@l... Tue Apr 17 08:21:09 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Tue, 17 Apr 2001 09:21:09 +0200 Subject: [medusa] Re: Trouble with the CGI References: <9bfse1+5bsd@e...> <987463009.3adb7d61cdbb3@w...> Message-ID: <3ADBEEE5.F9CEF8E3@l...> rprice@u... wrote: > > I have never used it from within medusa. I use the old bobo code to interact > with my Python scripts. Could you please mail me a tarball of your old bobo code? I've looked far and wide, and was unable to find it. I tried prying it from ZServer, but I couldn't understand it. From lucifer@e... Tue Apr 17 16:25:09 2001 From: lucifer@e... (lucifer@e...) Date: Tue, 17 Apr 2001 15:25:09 -0000 Subject: medusa and apache proxy -more info In-Reply-To: <9bfvk1+72vu@e...> Message-ID: <9bhn8l+cjom@e...> I don't know if this will lead to anything better but I have had some problems with apache proxy server too. I've built an html chat system over medusa and it works fine, but on our network, if my clients connect through an apache proxy it does not work as expected. The proxy seems to buffer a lot of data before transmitting it to the client. This behaviour has only been observed on apache proxy, no other proxy makes the system fail till now. --- In medusa@y..., rprice@u... wrote: > With more testing we've found out that medusa works differenetly on > Solaris (sunOS5) and Linux (I'm using Mandrake 7.1) at least when > using the Linux telnet client: > > On solaris using telnet the server closes the connection immediately > after the when using HTTP/1.0. When using HTTP/1.1 it will > keep connected. Both as expected. > > On linux the Medusa server will wait 30 seconds before closing the > session when using HTTP/1.0 and keeps the connection open much longer > using HTTP/1.1. > > I've done these tests on Medusa versions 3.39 and 3.19 on linux and > ONLY on 3.19 on sunOS5. > > Ray > > --- In medusa@y..., Sam Rushing wrote: > > rprice@u... wrote: > > > > > When a user accesses medusa directly (not through a proxy) it > takes less than 1 > > > second to display the page. > > > > > > When a user accesses the same medusa server page via an apache > proxy server, the > > > pages seem to hang at 66% loaded. It can then take anywhere from > 2 to 60 > > > seconds for the page to finally load. Another problem when > accessing through > > > the proxy is that the browser occasionally indicates 'contacting > server' for > > > many seconds before any information is transmitted. > > > > > > Any ideas why this would happen? > > > > None yet... a couple of things to try: > > > > 1) what happens if you try squid instead of apache proxy? > > 2) what protocol versions (i.e., http/1.0 or 1.1) are being used > between browser, > > proxy, and medusa? > > > > -Sam From rushing@n... Tue Apr 17 18:42:35 2001 From: rushing@n... (Sam Rushing) Date: Tue, 17 Apr 2001 10:42:35 -0700 Subject: [medusa] Re: medusa and apache proxy -more info References: <9bfvk1+72vu@e...> <987470871.3adb9c17659a5@m...> Message-ID: <3ADC808B.CEDEB472@n...> Donovan Baarda wrote: > I suspect that if you change your polling time from the default, you > can "adjust" the delay. Also, establishing another connection while one > is "delayed" will kick the select loop, causing the delay to end immediately. > Adjusting the poll time might be a quick fix, but I suspect you need to look at > your readable/writeable tests. Yup, methinks there's a problem with readable/writable. The easiest way to debug this kind of problem is to print the read/write sets just before the call to select(). I'd also attach a print of some kind to the socket close() of the http client... -Sam From rprice@u... Tue Apr 17 18:46:16 2001 From: rprice@u... (rprice@u...) Date: Tue, 17 Apr 2001 10:46:16 -0700 (PDT) Subject: [medusa] Re: medusa and apache proxy -more info In-Reply-To: <987470871.3adb9c17659a5@m...> References: <9bfvk1+72vu@e...> <987470871.3adb9c17659a5@m...> Message-ID: <987529576.3adc8168a65b9@w...> Quoting Donovan Baarda : > Quoting rprice@u...: > > > With more testing we've found out that medusa works differenetly on > > Solaris (sunOS5) and Linux (I'm using Mandrake 7.1) at least when > > using the Linux telnet client: > > > > On solaris using telnet the server closes the connection immediately > > after the when using HTTP/1.0. When using HTTP/1.1 it will > > keep connected. Both as expected. > > > > On linux the Medusa server will wait 30 seconds before closing the > > session when using HTTP/1.0 and keeps the connection open much longer > > using HTTP/1.1. > > The 30 sec delay sounds like the default medusa polling time on select. Yes it is. I temporarily 'fixed' the problem like this: asyncore.loop(timeout=.5) I just don't know how this well work when we have 100 concurrent users. > I don't > know _why_ it would require another iteration through the loop for your > application, but is it possible you have readable/writeable status > changing > around about the time you service the final push() for the connection. > When the > delay happens, it might not be quite ready the select, so it is excluded > until > the next iteration, when it gets selected immediately. Remember the > readable/writeable test is performed once per iteration through the > loop, so if > something is not ready at the start of the loop, it will be excluded > from the > select until the next iteration. > > I suspect that if you change your polling time from the default, you > can "adjust" the delay. Also, establishing another connection while one > is "delayed" will kick the select loop, causing the delay to end > immediately. Yes, another user jumping on will immediately close the connection to the prior user... if HTTP/1.0. Maybe I shoud increase the timeout to several seconds? > Adjusting the poll time might be a quick fix, but I suspect you need to > look at > your readable/writeable tests. > > -- > ABO: finger abo@m... for more information. > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > Thanks for your help, Ray From abo@m... Wed Apr 18 02:38:48 2001 From: abo@m... (Donovan Baarda) Date: Wed, 18 Apr 2001 11:38:48 +1000 (EST) Subject: asynchat producer ready() method patch. In-Reply-To: <20010417230813.91311.qmail@w...> References: <20010417230813.91311.qmail@w...> Message-ID: <987557928.3adcf0285f11c@m...> Quoting Ted Shab : > Hello, > > Saw this from the pipermail list. Has this patch been > applied to the standard python distribution or is it > pipermail specific? If the later, can you let me know > where I might find the fix (I'm not really interested > in pipermail). Hmmm, someone is using this patch? Wow, (warm fuzzies :-). I'm Cc'ing this to the medusa list incase anyone else is interested. AFAIK, no-one was using this patch apart from one person experimenting with it. There is nothing wrong with it, there just didn't seem to be much interest. I don't believe it has been applied anywhere apart from in my own projects. I'm hungry for any feedback on it and keen to support/promote it if there is interest. The patch is against medusa-src-20000601.tar.gz, and can be found in; ftp://minkirri.apana.org.au/pub/python/ Note that parts of medusa are distributed with python (asyncore.py and asynchat.py), and the relevant parts of the patch are designed to be compatible with the components in the python distribution (ie, there are no introduced dependancies and they are backwards compatible). I would like to think that the changes introduced also clean up the code a bit and provide a useful inheritance hirachy for producers. Performance heads might argue that the inheritance hurts performance. Comments about that are welcome too :-) The only tips about this patch from experience are; - be aware of how ready() is meant to work. It must return true whenever a producer is ready for more() to be called. That means when it has more data _or_ the producer is finished. Calling more() when a producer is not ready() is an error, and the producer base clases provided will raise an exception when this happens. - beware of producers that already have ready() methods. The original medusa had quite a few vestigial ready() methods that behaved incorrectly. The patch includes changes to fix/remove these methods in medusa. - a change in ready() status is not an event that will trigger select(). The ready() method is called before select() inside the main polling loop. This means a change in ready() status will not be immediately evaluated, but will be delayed until the next iteration through the loop. If no events trigger the select(), then this means it will be delayed untill the select() times out. The simple solution is to use a short timeout value. The complex solution is to make the change in ready() status trigger the select(), typicaly by putting an async object inside the producer. Note that this behaviour also affects the normal readable() and writeable() methods, but typicaly these change status as a result of something that triggers select() so you don't notice. -- ABO: finger abo@m... for more information. From abo@m... Wed Apr 18 02:39:25 2001 From: abo@m... (Donovan Baarda) Date: Wed, 18 Apr 2001 11:39:25 +1000 (EST) Subject: [medusa] Re: medusa and apache proxy -more info In-Reply-To: <987529576.3adc8168a65b9@w...> References: <9bfvk1+72vu@e...> <987470871.3adb9c17659a5@m...> <987529576.3adc8168a65b9@w...> Message-ID: <987557965.3adcf04de0a3c@m...> Quoting rprice@u...: > > Quoting Donovan Baarda : > > > Quoting rprice@u...: [...] > > The 30 sec delay sounds like the default medusa polling time on > select. > Yes it is. I temporarily 'fixed' the problem like this: > asyncore.loop(timeout=.5) I just don't know how this well work when we > have 100 > concurrent users. the beauty of the short timeout hack is it doesn't affect things when the load is high, only when it is low. At high loads, the select loop will be kicking around very fast so the timeout doesn't come into it (and any delays caused by poor readable()/writeable() implementation wont be noticed either). At low loads, the select loop will be mainly kicked around by the timeout. So a short timeout will increase the work for low loads, but will not increase the work at high loads. -- ABO: finger abo@m... for more information. From abo@m... Wed Apr 18 05:36:26 2001 From: abo@m... (Donovan Baarda) Date: Wed, 18 Apr 2001 14:36:26 +1000 (EST) Subject: asynchat producer ready() method patch. In-Reply-To: <20010418023428.19012.qmail@w...> References: <20010418023428.19012.qmail@w...> Message-ID: <987568586.3add19ca68aa6@m...> G'day all, in case I stuffed up the url and others are having the same problem finding it, I've Cc'd this to the medusa list. Quoting Ted Shab : > Mostly just interested in how you did this - is the > source to patch available - didn't see it on the site > you led me to. > > Thanks very much. The patch is a bzip2 compressed unified diff against medusa-src- 20000601.tar.gz, available at; ftp://minkirri.apana.org.au/pub/python/medusa-ready_0.3.patch.bz2 -- ABO: finger abo@m... for more information. From reynol@p... Wed Apr 18 18:00:34 2001 From: reynol@p... (Samuel Reynolds) Date: Wed, 18 Apr 2001 11:00:34 -0600 Subject: Medusa on Macintosh Message-ID: <5.1.0.14.0.20010418104349.00a48710@p...> ---------------------- multipart/mixed attachment Okay. I finally got back to the Mac-port/filesys.py changes testing. I've updated my modified filesys.py and tested it on MacOS 9, Windows NT, and SunOS (Solaris?) 5.7, and it works like a champ. I even dropped it into Zope/ZServer/medusa and ran Zope on top of it, without a hitch. The changes are as described in a previous post (http://groups.yahoo.com/group/medusa/message/286). I have made some additional tweaks to os_filesystem.translate, and improved the test script, so the files attached to the earlier message are outdated. While there is platform-specific code (unavoidable, I think), it is only in os_filesystem.translate. I've attached two files: o filesys.py -- The modified file, tested on MacOS, WinNT, and Sun. o filesys_test.py -- The test script. These files have Windows-style (\r\n) line endings, so you may need to convert them for your system (e.g., using dos2unix on Unices). Is anyone other than me interested in a Mac port and/or improving filesys.py? Or should I keep changes like this to myself in the future? If there is interest, I'd appreciate if others would try this out. (I renamed the original filesys.py to filesys_original.py and dropped the modified file in its place; that makes it easy to revert.) If you try it, I suggest you start by running the test script. It generates absolute (native file system) paths from standard relative paths within the Python tree for posix, Dos/Windows/OS2, or Mac, as appropriate, then exercises os_filesystem and compares the results to the reference paths. If you test on any other system, you may need to add a new block to generate the native absolute paths for your system, and any changes to filesys.py will affect only os_filesystem.translate. - Sam ---------------------- multipart/mixed attachment # filesys_test.py # # SCR 2001.04.17 # import os import filesys import string # Utility for converting path separator def ConvertSep( s, fromSep, toSep ): return string.join( string.split( s, fromSep ), toSep ) print "**** Testing filesys.os_filesystem ****" print "" # Get working directory workingDir = os.getcwd() #print "Working directory:", workingDir # Must be run from within Python directory if string.find( workingDir, "Python" ) < 0: raise RuntimeError, "Test must be run from within or below the Python main directory" # Determine Python directory path # We will use this as the root for the virtual filesystem. bits = string.split( workingDir, os.sep ) #print "***" #print bits #print "***" while string.find( bits[-1], "Python" ) < 0: bits = bits[:-1] pythonDir = string.join( bits, os.sep ) if os.name == 'mac': pythonDir = pythonDir + os.sep #print "pythonDir =", pythonDir #print " " # Create filesys for testing fs = filesys.os_filesystem( pythonDir, '/' ) # Pick some subpaths to work with. relPaths = ["Lib", "Lib/distutils", "Lib/distutils/command", "Lib/asyncore.py"] realPaths = [] # Pull out longest directory path for cd-up/cd-down testing deepPath = "" deepPathDepth = 0 for relPath in relPaths: if ( fs.isfile(relPath) ): continue tempL = string.split( relPath, '/' ) tempD = len(tempL) if ( tempD > deepPathDepth ): deepPathDepth = tempD deepPath = relPath # Since we're testing the conversion code, we have to # determine the absolute paths the hard way for comparison. if os.name == 'posix': # Just concatenate root & path for relPath in relPaths: realPaths = realPaths \ + [ pythonDir + sub_lib ] elif os.name == 'mac': # Explicitly add trailing separator for directory paths # (with prior knowledge that the last path is a file) onlyFilePath = relPaths[-1] for relPath in relPaths: temp = pythonDir + ConvertSep( relPath, '/', os.sep ) + os.sep if relPath == onlyFilePath: temp = temp[:-1] #if relPath == onlyFilePath: # realPaths = realPaths \ # + [ pythonDir + ConvertSep( relPath, '/', os.sep ) ] #else: # realPaths = realPaths \ # + [ pythonDir + ConvertSep( relPath, '/', os.sep ) + os.sep ] realPaths = realPaths + [ temp ] elif os.name == 'nt': # Explicitly add trailing separator to root directory path. for relPath in relPaths: realPaths = realPaths \ + [ pythonDir + os.sep + ConvertSep( relPath, '/', os.sep ) ] else: # os.name not specifically addressed. # This is a test error! raise RuntimeError, "Test paths not computed for os.name = " + os.name exit print "** Reference paths for test **" print "%-21s ==> %s" % ( "Python directory", pythonDir ) for n in range( len(relPaths) ): print "%-21s ==> %s" % ( relPaths[n], realPaths[n] ) print " " # Core test & statistics routines global passCount passCount = 0 global failCount failCount = 0 def CHKEQ( desc, testValue, refValue ): global passCount global failCount passesTest = ( testValue == refValue ) if passesTest: print "CHKEQ: %-50s - PASS" % desc passCount += 1 else: print "CHKEQ: %-50s - FAIL" % desc print "\tExpected:", refValue print "\tActual :", testValue failCount += 1 return ( passesTest ) def CHKNE( desc, testValue, refValue ): global passCount global failCount passesTest = ( testValue != refValue ) if passesTest: print "CHKNE: %-50s - PASS" % desc passCount += 1 else: print "CHKNE: %-50s - FAIL" % desc print "\tValue:", testValue failCount += 1 return ( passesTest ) def CHKPASS( desc ): global passCount print "CHKPASS: %-50s - PASS" % desc passCount += 1 def CHKFAIL( desc ): global failCount print "CHKFAIL: %-50s - FAIL" % desc failCount += 1 #### Translation from virtual to absolute path def CHK_TRANSLATE( subPath, refValue ): desc = "%-25s ==> absolute" % subPath temp = fs.translate( subPath ) CHKEQ( desc, temp, refValue ) for n in range( len(relPaths) ): CHK_TRANSLATE( relPaths[n], realPaths[n] ) #### Relative movement within directory structure # First, check our starting point CHKEQ( "Initial virtual directory", fs.current_directory(), "/" ) # From '/', cwd('/') should leave us where we are fs.cwd( '/' ) CHKEQ( "After cwd('/')", fs.current_directory(), "/" ) # CD into sub_lib, then back to top for relPath in relPaths: fs.cwd( relPath ) if ( fs.isfile( relPath) ): # Shouldn't change working directory CHKEQ( "After cwd('"+relPath+"')", fs.current_directory(), "/" ) else: # Should have changed working directory CHKEQ( "After cwd('"+relPath+"')", fs.current_directory(), "/" + relPath ) # Change working directory back to root fs.cwd('/') CHKEQ( "After cwd('/')", fs.current_directory(), "/" ) # Create up/down paths list bits = string.split( deepPath, '/' ) upDownPaths1 = [] upDownPaths2 = [] onePath = "/" for bit in bits: upDownPaths1 += [ bit ] onePath += bit upDownPaths2 += [ onePath ] onePath += '/' # Start at virtual root fs.cwd( '/' ) # CD up one level, and up two levels (should not move) fs.cwd( ".." ) CHKEQ( "After cwd('..') from '/' ==> '/'", fs.current_directory(), "/" ) fs.cwd( '/' ) fs.cwd( "../.." ) CHKEQ( "After cwd('../..') from '/' ==> '/'", fs.current_directory(), "/" ) print "------------" step = 1 for idx in range(0, len(upDownPaths1), step): fs.cwd( upDownPaths1[idx] ); CHKEQ( "After cwd('"+upDownPaths1[idx]+"')", fs.current_directory(), \ upDownPaths2[idx] ) fs.cwd('/') # CD into sub_lib_distutils_cmd as virtual "absolute" path #fs.cwd( '/' + sub_lib_distutils_cmd ) #CHKEQ( "After cwd('/"+sub_lib_distutils_cmd+"')", \ # fs.current_directory(), "/" + sub_lib_distutils_cmd ) # CD up two levels #fs.cwd( "../.." ) #CHKEQ( "After cwd('../..')", \ # fs.current_directory(), "/" + sub_lib ) # CD back to top #fs.cwd('/') #CHKEQ( "After cwd('/')", fs.current_directory(), "/" ) # Try to CD above root #fs.cwd('/') #fs.cwd('..') #CHKEQ( "After cwd('..') when wd='/'", fs.current_directory(), "/" ) # CD up two levels above root #fs.cwd('/') #fs.cwd( "../.." ) #CHKEQ( "After cwd('../..') when wd='/'", fs.current_directory(), "/" ) # CD up past root from subdirectory #fs.cwd('/' + sub_lib) ##CHKEQ( "After cwd('../..') when wd='/" + sub_lib + "'", \ # fs.current_directory(), "/" ) #### Informational Tests # isfile/isdir for relPath in relPaths: if ( fs.isfile(relPath) ): CHKEQ( "%-30s isdir:%d isfile:%d" % \ (relPath, fs.isdir(relPath), fs.isfile(relPath)), \ "d:%d f:%d" % (fs.isdir(relPath), fs.isfile(relPath)), \ "d:0 f:1" ) else: CHKEQ( "%-30s isdir:%d isfile:%d" % \ (relPath, fs.isdir(relPath), fs.isfile(relPath)), \ "d:%d f:%d" % (fs.isdir(relPath), fs.isfile(relPath)), \ "d:1 f:0" ) # exists for relPath in relPaths: # Real paths CHKEQ( "%-30s exists:%d" % ( relPath, fs.exists(relPath) ), fs.exists(relPath), 1 ) # Fake paths CHKEQ( "%-30s exists:%d" % ( relPath+'_XX', fs.exists(relPath+'_XX') ), fs.exists(relPath+'_XX'), 0 ) # stat for relPath in relPaths: # Real paths try: temp = fs.stat( relPath ) CHKEQ( "stat('%s') returns tuple" % relPath, type(temp), type(()) ) except: CHKFAIL( "stat('%s') returns tuple" % relPath ) # Fake paths try: temp = fs.stat( relPath+'_XX' ) CHKEQ( "stat('%s') throws" % (relPath+'_XX'), type(temp), None ) except: CHKPASS( "stat('%s') throws" % (relPath+'_XX') ) print "" print ' ' + '=' * 11 print " PASS: %5d" % passCount print " FAIL: %5d" % failCount print ' ' + '=' * 11 print "" ---------------------- multipart/mixed attachment # -*- Mode: Python; tab-width: 4 -*- # $Id: filesys.py,v 1.9 1998/06/22 05:36:05 rushing Exp $ # Author: Sam Rushing # # Generic filesystem interface. # # We want to provide a complete wrapper around any and all # filesystem operations. # this class is really just for documentation, # identifying the API for a filesystem object. # opening files for reading, and listing directories, should # return a producer. import posixpath class abstract_filesystem: def __init__ (self): pass def current_directory (self): "Return a string representing the current directory." pass def listdir (self, path, long=0): """Return a listing of the directory at 'path' The empty string indicates the current directory. If 'long' is set, instead return a list of (name, stat_info) tuples """ pass def open (self, path, mode): "Return an open file object" pass def stat (self, path): "Return the equivalent of os.stat() on the given path." pass def isdir (self, path): "Does the path represent a directory?" pass def isfile (self, path): "Does the path represent a plain file?" pass def cwd (self, path): "Change the working directory." pass def cdup (self): "Change to the parent of the current directory." pass def longify (self, path): """Return a 'long' representation of the filename [for the output of the LIST command]""" pass # standard wrapper around a unix-like filesystem, with a 'false root' # capability. # security considerations: can symbolic links be used to 'escape' the # root? should we allow it? if not, then we could scan the # filesystem on startup, but that would not help if they were added # later. We will probably need to check for symlinks in the cwd method. # what to do if wd is an invalid directory? import os import stat import string def safe_stat (path): try: return (path, os.stat (path)) except: return None import regex import regsub import glob class os_filesystem: path_module = os.path # set this to zero if you want to disable pathname globbing. # [we currently don't glob, anyway] do_globbing = 1 def __init__ (self, root, wd='/'): self.root = root self.wd = wd ### Provide replacements for path methods def abspath (self, path): return self.normalize (posixpath.join (self.wd, path)) def basename (self, path): return posixpath.basename (path) def commonprefix (self, pathlist): return posixpath.commonprefix (pathlist) def dirname (self, path): return posixpath.dirname (path) def exists (self, path): p = self.abspath (path) return self.path_module.exists (self.translate(p)) def expanduser (self, path): p = self.abspath (path) return self.path_module.expanduser (self.translate(p)) def expandvars (self, path): p = self.abspath (path) return self.path_module.expandvars (self.translate(p)) def getatime (self, path): p = self.abspath (path) return self.path_module.getatime (self.translate(p)) def getmtime (self, path): p = self.abspath (path) return self.path_module.getmtime (self.translate(p)) def getsize (self, path): p = self.abspath (path) return self.path_module.getsize (self.translate(p)) def isabs (self, path): return posixpath.isabs (path) def isdir (self, path): p = self.abspath (path) return self.path_module.isdir (self.translate(p)) def isfile (self, path): p = self.abspath (path) return self.path_module.isfile (self.translate(p)) def islink (self, path): p = self.abspath (path) return self.path_module.islink (self.translate(p)) def ismount (self, path): p = self.abspath (path) return self.path_module.ismount (self.translate(p)) def normcase (self, path): return posixpath.normcase (path) def normpath (self, path): return posixpath.normpath (path) def samefile (self, p1, p2): p1 = self.abspath (p1) p2 = self.abspath (p2) return self.path_module.samefile (self.translate(p1), self.translate(p2)) def sameopenfile (self, fp1, fp2): return self.path_module.sameopenfile (fp1, fp2) def samestat (self, s1, s2): return self.path_module.samestat (s1, s2) def split (self, path): return posixpath.split (path) def stat (self, path): p = self.abspath (path) p = self.translate(p) st = os.stat (p) return st # splitdrive NOT IMPLEMENTED def splittext (self, path): return posixpath.splittext (path) ### Provide Unix filesystem commands def cwd (self, path): p = self.abspath (path) translated_path = self.translate(p) if not self.path_module.isdir (translated_path): return 0 else: old_dir = os.getcwd() # temporarily change to that directory, in order # to see if we have permission to do so. try: can = 0 try: os.chdir (translated_path) can = 1 self.wd = p except: pass finally: if can: os.chdir (old_dir) return can def cdup (self): return self.cwd ('..') def current_directory (self): return self.wd def listdir (self, path, long=0): p = self.translate (path) # I think we should glob, but limit it to the current # directory only. ld = os.listdir (p) if not long: return list_producer (ld, 0, None) else: old_dir = os.getcwd() try: os.chdir (p) # if os.stat fails we ignore that file. result = filter (None, map (safe_stat, ld)) finally: os.chdir (old_dir) return list_producer (result, 1, self.longify) # TODO: implement a cache w/timeout for stat() def open (self, path, mode): p = self.translate (path) return open (p, mode) def unlink (self, path): p = self.translate (path) return os.unlink (p) def mkdir (self, path): p = self.translate (path) return os.mkdir (p) def rmdir (self, path): p = self.translate (path) return os.rmdir (p) # utility methods def normalize (self, path): # watch for the ever-sneaky '/+' path element path = regsub.gsub ('/+', '/', path) p = posixpath.normpath (path) # remove 'dangling' cdup's. if len(p) > 2 and p[:3] == '/..': p = '/' return p def translate (self, path): # Get full (virtual-unix) path p = self.abspath (path) # Convert to absolute path recognizable by current OS if ( os.name == 'posix' ): if ( p[:1] == os.sep ): p = p[1:] p = self.path_module.join( self.root, p ) else: # Make platform-specific pre-adjustments, if any if ( os.sep != '/' ): p = string.join( string.split( p, '/' ), os.sep ) if (os.name == 'nt' or os.name == 'dos' or os.name == 'os2'): if ( p[:1] == os.sep ): p = p[1:] p = self.path_module.join( self.root, p ) # Make platform-specific post-adjustments, if any if (os.name == 'mac'): if self.path_module.isdir( p ) and ( p[-1] <> os.sep ): p = p + os.sep return p def longify (self, (path, stat_info)): return unix_longify (path, stat_info) def __repr__ (self): return '' % ( self.root, self.wd ) if os.name == 'posix': class unix_filesystem (os_filesystem): pass class schizophrenic_unix_filesystem (os_filesystem): PROCESS_UID = os.getuid() PROCESS_EUID = os.geteuid() PROCESS_GID = os.getgid() PROCESS_EGID = os.getegid() def __init__ (self, root, wd='/', persona=(None, None)): os_filesystem.__init__ (self, root, wd) self.persona = persona def become_persona (self): if self.persona is not (None, None): uid, gid = self.persona # the order of these is important! os.setegid (gid) os.seteuid (uid) def become_nobody (self): if self.persona is not (None, None): os.seteuid (self.PROCESS_UID) os.setegid (self.PROCESS_GID) # cwd, cdup, open, listdir def cwd (self, path): try: self.become_persona() return os_filesystem.cwd (self, path) finally: self.become_nobody() def cdup (self, path): try: self.become_persona() return os_filesystem.cdup (self) finally: self.become_nobody() def open (self, filename, mode): try: self.become_persona() return os_filesystem.open (self, filename, mode) finally: self.become_nobody() def listdir (self, path, long=0): try: self.become_persona() return os_filesystem.listdir (self, path, long) finally: self.become_nobody() # This hasn't been very reliable across different platforms. # maybe think about a separate 'directory server'. # # import posixpath # import fcntl # import FCNTL # import select # import asyncore # # # pipes /bin/ls for directory listings. # class unix_filesystem (os_filesystem): # pass # path_module = posixpath # # def listdir (self, path, long=0): # p = self.translate (path) # if not long: # return list_producer (os.listdir (p), 0, None) # else: # command = '/bin/ls -l %s' % p # print 'opening pipe to "%s"' % command # fd = os.popen (command, 'rt') # return pipe_channel (fd) # # # this is both a dispatcher, _and_ a producer # class pipe_channel (asyncore.file_dispatcher): # buffer_size = 4096 # # def __init__ (self, fd): # asyncore.file_dispatcher.__init__ (self, fd) # self.fd = fd # self.done = 0 # self.data = '' # # def handle_read (self): # if len (self.data) < self.buffer_size: # self.data = self.data + self.fd.read (self.buffer_size) # #print '%s.handle_read() => len(self.data) == %d' % (self, len(self.data)) # # def handle_expt (self): # #print '%s.handle_expt()' % self # self.done = 1 # # def ready (self): # #print '%s.ready() => %d' % (self, len(self.data)) # return ((len (self.data) > 0) or self.done) # # def more (self): # if self.data: # r = self.data # self.data = '' # elif self.done: # self.close() # self.downstream.finished() # r = '' # else: # r = None # #print '%s.more() => %s' % (self, (r and len(r))) # return r # For the 'real' root, we could obtain a list of drives, and then # use that. Doesn't win32 provide such a 'real' filesystem? # [yes, I think something like this "\\.\c\windows"] class msdos_filesystem (os_filesystem): def longify (self, (path, stat_info)): return msdos_longify (path, stat_info) # A merged filesystem will let you plug other filesystems together. # We really need the equivalent of a 'mount' capability - this seems # to be the most general idea. So you'd use a 'mount' method to place # another filesystem somewhere in the hierarchy. # Note: this is most likely how I will handle ~user directories # with the http server. class merged_filesystem: def __init__ (self, *fsys): pass # this matches the output of NT's ftp server (when in # MSDOS mode) exactly. def msdos_longify (file, stat_info): if stat.S_ISDIR (stat_info[stat.ST_MODE]): dir = '' else: dir = ' ' date = msdos_date (stat_info[stat.ST_MTIME]) return '%s %s %8d %s' % ( date, dir, stat_info[stat.ST_SIZE], file ) def msdos_date (t): try: info = time.gmtime (t) except: info = time.gmtime (0) # year, month, day, hour, minute, second, ... if info[3] > 11: merid = 'PM' info[3] = info[3] - 12 else: merid = 'AM' return '%02d-%02d-%02d %02d:%02d%s' % ( info[1], info[2], info[0]%100, info[3], info[4], merid ) months = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'] mode_table = { '0':'---', '1':'--x', '2':'-w-', '3':'-wx', '4':'r--', '5':'r-x', '6':'rw-', '7':'rwx' } import time def unix_longify (file, stat_info): # for now, only pay attention to the lower bits mode = ('%o' % stat_info[stat.ST_MODE])[-3:] mode = string.join (map (lambda x: mode_table[x], mode), '') if stat.S_ISDIR (stat_info[stat.ST_MODE]): dirchar = 'd' else: dirchar = '-' date = ls_date (long(time.time()), stat_info[stat.ST_MTIME]) return '%s%s %3d %-8d %-8d %8d %s %s' % ( dirchar, mode, stat_info[stat.ST_NLINK], stat_info[stat.ST_UID], stat_info[stat.ST_GID], stat_info[stat.ST_SIZE], date, file ) # Emulate the unix 'ls' command's date field. # it has two formats - if the date is more than 180 # days in the past, then it's like this: # Oct 19 1995 # otherwise, it looks like this: # Oct 19 17:33 def ls_date (now, t): try: info = time.gmtime (t) except: info = time.gmtime (0) # 15,600,000 == 86,400 * 180 if (now - t) > 15600000: return '%s %2d %d' % ( months[info[1]-1], info[2], info[0] ) else: return '%s %2d %02d:%02d' % ( months[info[1]-1], info[2], info[3], info[4] ) # =========================================================================== # Producers # =========================================================================== class list_producer: def __init__ (self, file_list, long, longify): self.file_list = file_list self.long = long self.longify = longify self.done = 0 def ready (self): if len(self.file_list): return 1 else: if not self.done: self.done = 1 return 0 return (len(self.file_list) > 0) # this should do a pushd/popd def more (self): if not self.file_list: return '' else: # do a few at a time bunch = self.file_list[:50] if self.long: bunch = map (self.longify, bunch) self.file_list = self.file_list[50:] return string.joinfields (bunch, '\r\n') + '\r\n' ---------------------- multipart/mixed attachment-- From eugene.leitl@l... Mon Apr 23 14:34:19 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Mon, 23 Apr 2001 15:34:19 +0200 (MET DST) Subject: XML-RPC with a http server Message-ID: Um, I just realized something embarrassing. Please correct me if I'm spouting nonsense. I want to run a janusian server based on medusa (pardon the mythology melange): talking to the users HTML via a http port, and via XML-RPC to other services, running on the same machine, or different ones. Now the wunderbar Medusa usually uses something like this, when starting a server (this ripped from xmlrpc_handler.py): class rpc_demo (xmlrpc_handler): def call (self, method, params): print 'method="%s" params=%s' % (method, params) return "Sure, that works" import asyncore import http_server hs = http_server.http_server ('', 8000) rpc = rpc_demo() hs.install_handler (rpc) asyncore.loop() There can be only one handler in the asyncore.loop(), correct? I can't run medusa serving web pages (applets, cgi-bin python) at one end, and have an xml-rpc running within the same binary, on an alternative port? Or can I have medusa serving html on, say, port 80, and the xml-rpc handler be automatically invoked, when a POST /RPC2 HTTP/1.0 request comes in on same standard port? I'm a bit fuzzy on the concepts, somebody please clue me in. From rushing@n... Mon Apr 23 17:33:05 2001 From: rushing@n... (Sam Rushing) Date: Mon, 23 Apr 2001 09:33:05 -0700 Subject: [medusa] XML-RPC with a http server References: Message-ID: <3AE45941.FEA61EE@n...> Eugene Leitl wrote: > There can be only one handler in the asyncore.loop(), correct? I can't > run medusa serving web pages (applets, cgi-bin python) at one end, and > have an xml-rpc running within the same binary, on an alternative port? The 'handler' is an http_server-specific thing. You install multiple 'handlers' on the http server to map to different URI's. (they're usually matched using a regular expression). You can have as many different kinds of servers as you like running. I've used servers that spoke 4 or 5 different protocols. The easiest way to do that of course, is to have each one on a different port. 8^) > Or can I have medusa serving html on, say, port 80, and the xml-rpc > handler be automatically invoked, when a POST /RPC2 HTTP/1.0 request comes > in on same standard port? It should just work as you expect. Just call 'hs.install_handler' in whatever order makes sense. The handlers are maintained in the order they are installed, and they're tried in that order. This allows you to install a 'match-all' handler at the end of the chain. Take a look at the 'match' method of the xmlrpc_handler... -Sam From mj@d... Wed Apr 25 15:59:45 2001 From: mj@d... (mj@d...) Date: Wed, 25 Apr 2001 14:59:45 -0000 Subject: Bug in log format Message-ID: <9c6op1+lmcc@e...> Hi all, Currently, the file logging writes all remote addreses, wether DNS resolved or not, with a trailing colon (':'): (address): - - [(date)] "(request)" (status) (bytes) The colon does not belong there, at least, not according to the CLF specs. Webalizer will not resolve ip addresses because it doesn't recognize them as ips, for example. The colon seems to be a remnant of a change made in reference to a post to this list in 1999: http://groups.yahoo.com/group/medusa/message/33 which removed the client port number from the log, but *not* the colon that separates the two. The following patch removes the colon: --- logger.py 2000/06/02 15:36:39 1.8 +++ logger.py 2001/04/25 14:57:23 @@ -227,7 +227,7 @@ def __call__ (self, host, ttl, answer): if not answer: answer = host - self.logger.log ('%s:%s' % (answer, self.message)) + self.logger.log ('%s%s' % (answer, self.message)) def log (self, ip, message): self.resolver.resolve_ptr ( @@ -244,7 +244,7 @@ self.logger = logger def log (self, ip, message): - self.logger.log ('%s:%s' % (ip, message)) + self.logger.log ('%s%s' % (ip, message)) def strip_eol (line): Martijn Pieters Digital Creations From johs@c... Sat Apr 28 09:33:39 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Sat, 28 Apr 2001 10:33:39 +0200 Subject: Medusa, Python 2.0. Message-ID: <20010428103339.A2991@u...> Seems Medusa doesn't quite like Python 2.0. Some library incompatibilities. The auth-handler, for example, excepts re-expressions to have a group-function, which it did, in 1.5.2, but not anymore. (Yes, I know, the regex-module in 2.0 gives me this.) Anyway, is someone working on making it Python 2.0-compatible? -- johs From fredrik@p... Sat Apr 28 10:00:54 2001 From: fredrik@p... (Fredrik Lundh) Date: Sat, 28 Apr 2001 11:00:54 +0200 Subject: [medusa] Medusa, Python 2.0. References: <20010428103339.A2991@u...> Message-ID: <009601c0cfc1$c09aebb0$e46940d5@h...> Johannes Grřdem wrote: > Seems Medusa doesn't quite like Python 2.0. Some library incompatibilities. > The auth-handler, for example, excepts re-expressions to have a > group-function, which it did, in 1.5.2, but not anymore. there has never been a group function in the re module. there's a group *method* on match objects, but it's still there in 2.0. Cheers /F Python 1.5.2 (#0, May 9 2000, 14:04:03) >>> import re >>> re.group() Traceback (innermost last): File "", line 1, in ? AttributeError: group >>> p = re.compile("x") >>> p.group() Traceback (innermost last): File "", line 1, in ? AttributeError: group >>> m = p.match("x") >>> m.group() 'x' Python 2.0 (#8, Jan 4 2001, 12:33:41) >>> import re >>> re.group Traceback (most recent call last): File "", line 1, in ? AttributeError: group >>> p = re.compile("x") >>> p.group() Traceback (most recent call last): File "", line 1, in ? AttributeError: group >>> m = p.match("x") >>> m.group() 'x' From johs@c... Sat Apr 28 09:50:58 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Sat, 28 Apr 2001 10:50:58 +0200 Subject: Medusa, Python 2.0. Message-ID: <20010428105058.B2991@u...> > Seems Medusa doesn't quite like Python 2.0. Some library incompatibilities. > The auth-handler, for example, excepts re-expressions to have a > group-function, which it did, in 1.5.2, but not anymore. (Yes, I know, the > regex-module in 2.0 gives me this.) Um. The re-module in 1.5.2 didn't do this, I see, upon closer inspection. So it's just a bug, then? -- johs From rushing@n... Sat Apr 28 10:32:39 2001 From: rushing@n... (Sam Rushing) Date: Sat, 28 Apr 2001 02:32:39 -0700 Subject: medusa minus regex References: <20010428103339.A2991@u...> Message-ID: <3AEA8E37.AAA103E2@n...> Johannes Grřdem wrote: > Seems Medusa doesn't quite like Python 2.0. Some library incompatibilities. > The auth-handler, for example, excepts re-expressions to have a > group-function, which it did, in 1.5.2, but not anymore. (Yes, I know, the > regex-module in 2.0 gives me this.) > > Anyway, is someone working on making it Python 2.0-compatible? I made a first stab at it last week or so. Haven't heard any complaints yet... http://www.nightmare.com/medusa/medusa-20010416.tar.gz There were a lot more changes required than I thought, so caveat upgrader! -Sam From johs@c... Sun Apr 29 15:22:54 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Sun, 29 Apr 2001 16:22:54 +0200 Subject: URL-parsing. Message-ID: <20010429162254.A13375@u...> The crack_request-function in http_server.py breaks when URLs contain spaces. I'm not really too steady in regexps, so I'm not quite sure how to best fix it. -- johs From fredrik@p... Sun Apr 29 19:49:36 2001 From: fredrik@p... (Fredrik Lundh) Date: Sun, 29 Apr 2001 20:49:36 +0200 Subject: [medusa] URL-parsing. References: <20010429162254.A13375@u...> Message-ID: <009401c0d0dd$26086c90$e46940d5@h...> Johannes Grřdem wrote: > The crack_request-function in http_server.py breaks when URLs contain spaces. > I'm not really too steady in regexps, so I'm not quite sure how to best fix it. according to RFC 2396, URLs cannot contain spaces: "The space character is excluded because significant spaces may disappear and insignificant spaces may be introduced when URI are transcribed or typeset or subjected to the treatment of word- processing programs. Whitespace is also used to delimit URI in many contexts." or do you mean that it doesn't work if the URI contains %20 ? Cheers /F From johs@c... Mon Apr 30 11:50:18 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Mon, 30 Apr 2001 12:50:18 +0200 Subject: URL-parsing. Message-ID: <20010430125018.A7046@u...> > or do you mean that it doesn't work if the URI contains %20? I tried with %20, too, but it seems crack_request doesn't receive %20. The %20 is converted to a space before it reaches crack_request. -- johs From johs@c... Thu May 3 14:57:06 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Thu, 3 May 2001 15:57:06 +0200 Subject: URL-parsing. Message-ID: <20010503155705.A5594@u...> >> or do you mean that it doesn't work if the URI contains %20? > I tried with %20, too, but it seems crack_request doesn't receive %20. > The %20 is converted to a space before it reaches crack_request. Ok, I haven't read the RFC thoroughly, but wouldn't this be the correct way to do it? --- http_server.py.old Thu May 3 15:34:28 2001 +++ http_server.py Thu May 3 15:34:45 2001 @@ -457,10 +457,10 @@ # unquote path if necessary (thanks to Skip Montaro for pointing # out that we must unquote in piecemeal fashion). - if '%' in request: - request = unquote (request) command, uri, version = crack_request (request) + if '%' in uri: + uri = unquote (uri) header = join_headers (lines[1:]) r = http_request (self, request, command, uri, version, header) From weissblitz@y... Thu May 3 16:35:26 2001 From: weissblitz@y... (Weiss Blitz) Date: Thu, 03 May 2001 15:35:26 -0000 Subject: Looking for an XML to Python sequence source code Message-ID: <9crtru+lv6c@e...> Does anyone knows of any Python code to convert from XML to a Python sequence object? Like XML to a list? From akuchlin@m... Thu May 3 17:15:01 2001 From: akuchlin@m... (Andrew Kuchling) Date: Thu, 3 May 2001 12:15:01 -0400 Subject: [medusa] Looking for an XML to Python sequence source code In-Reply-To: <9crtru+lv6c@e...>; from weissblitz@y... on Thu, May 03, 2001 at 03:35:26PM -0000 References: <9crtru+lv6c@e...> Message-ID: <20010503121501.A17964@u...> On Thu, May 03, 2001 at 03:35:26PM -0000, Weiss Blitz wrote: >Does anyone knows of any Python code to convert from XML to a Python >sequence object? Like XML to a list? The PyXML package includes an xml.marshal subpackage with a bunch of different marshallers: a generic one, WDDX, and XML-RPC. For example: >>> from xml.marshal import generic >>> print generic.dumps( [1.0, pow(2, 75L), 'this is a string'] ) 1.0 37778931862957161709568this is a string >>> --amk From skip@p... Sun May 6 19:20:02 2001 From: skip@p... (skip@p...) Date: Sun, 6 May 2001 13:20:02 -0500 Subject: SIGRT_0 signal caught by ZServer/medusa? Message-ID: <15093.38354.488789.63286@b...> I'm using ZServer from the Zope 2.2.0 distribution as a backend to Apache on a Linux machine. The architecture (if you can call it that anymore) of the system is this: apache <--http proxy--> zserver <--xmlrpc--> database server Actually, behind the database server sits the real database - MySQL, but I don't think it's germane to this discussion. Performance of this system is sometimes abysmal at times when I can't account for it by some cause like system load. I monitor things from time-to-time using tools like strace. Once started, there are typically four process ids associated with zserver, presumably four different threads. I can watch the thread that's doing all the socket activity: select(19, [9 14 18], [], [], {10, 0}) = 1 (in [14], left {9, 970000}) read(14, "x", 8192) = 1 select(19, [9 14 18], [18], [], {10, 0}) = 1 (out [18], left {10, 0}) send(18, "HTTP/1.0 200 OK\r\nServer: Zope/Zo"..., 7087, 0) = 7087 ... Every so often, strace quits with something like this: kill(11754, SIGRT_0) = 0 select(15, [9 10 14], [], [], {10, 0}) = 1 (in [9], left {1, 530000}) rt_sigprocmask(SIG_SETMASK, NULL, [RT_0], 8) = 0 rt_sigsuspend([] --- SIGRT_0 (Real-time signal 0) --- The strace'd process doesn't actually croak, however. Frequently, the kill call is executed without terminating strace. If I strace the database server (also written in Python, but not using medusa), I never see the kill calls or the strace termination, so my naive assumption is that somehow the behavior I see in ZServer is somehow Zope- or Medusa-related. Any idea what this mysterious SIGRT_0 signal is (I can see it's a "real-time signal", but that doesn't tell me much) and why it would be getting sent to my ZServer? I can't find mention of it in the man pages. Thanks, -- Skip Montanaro (skip@p...) (847)971-7098 From eugene.leitl@l... Thu May 10 17:06:37 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Thu, 10 May 2001 18:06:37 +0200 (MET DST) Subject: Medusa: global name 'split_path' is not defined: file: status_handler.pyline: 63 Message-ID: This is Python 2.1 and medusa-20010416.tar.gz I'm getting error: Server Error: exceptions.NameError, global name 'split_path' is not defined: file: status_handler.py line: 63 when trying to access http://localhost:8080/status I'm also having trouble getting Bobo running, which also tells me stuff about split_path: Traceback (most recent call last): File "start_medusa.py", line 110, in ? import bobo_handler File "bobo_handler.py", line 58, in ? split_path = default_handler.split_path AttributeError: 'default_handler' module has no attribute 'split_path' Any idea what is going on? Something changed in 2.1? From cm@b... Sun May 13 05:09:16 2001 From: cm@b... (cm@b...) Date: Sun, 13 May 2001 04:09:16 -0000 Subject: Gracefully quitting Medusa? Message-ID: <9dl1dc+cqqh@e...> ^C seems a bit harsh. Is there a better way? Thanks :) From eugene.leitl@l... Mon May 14 15:34:22 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Mon, 14 May 2001 14:34:22 -0000 Subject: Medusa: global name 'split_path' is not defined: file: status_handler.pyline: 63 In-Reply-To: Message-ID: <9doqde+cho0@e...> --- In medusa@y..., Eugene Leitl wrote: > > This is Python 2.1 and medusa-20010416.tar.gz > > I'm getting > > error: Server Error: exceptions.NameError, global name 'split_path' is > not defined: file: status_handler.py line: 63 > > when trying to access http://localhost:8080/status > > I'm also having trouble getting Bobo running, which also tells me > stuff about split_path: > > Traceback (most recent call last): > File "start_medusa.py", line 110, in ? > import bobo_handler > File "bobo_handler.py", line 58, in ? > split_path = default_handler.split_path > AttributeError: 'default_handler' module has no attribute 'split_path' > > > Any idea what is going on? Something changed in 2.1? From eugene.leitl@l... Mon May 14 15:33:53 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Mon, 14 May 2001 14:33:53 -0000 Subject: Medusa: global name 'split_path' is not defined: file: status_handler.pyline: 63 In-Reply-To: Message-ID: <9doqch+kjvq@e...> Er. Disregard my last message (well, ok, you already did that ;) I went back to Python 1.5.2 and medusa-src-20000601.tar.gz and everything works. Bobo included. /status even. And it freezes vewy vewy nicely. Under ancient AIX, to wit. I've got a form with a submit button to supply arguments to hello.py (see bobo-howto). I'm happy. (Now for the hard part: application front end and XML-RPC back end. Urgh. I feel I need a vacation already). I don't know what weird combinations of versions I tried before which broke all over the palce, but above combination should work. Probably. Rushing, you made another guy out there very happy. Medusa r0xx, officially. --- In medusa@y..., Eugene Leitl wrote: > > This is Python 2.1 and medusa-20010416.tar.gz > > I'm getting > > error: Server Error: exceptions.NameError, global name 'split_path' is > not defined: file: status_handler.py line: 63 > > when trying to access http://localhost:8080/status > > I'm also having trouble getting Bobo running, which also tells me > stuff about split_path: > > Traceback (most recent call last): > File "start_medusa.py", line 110, in ? > import bobo_handler > File "bobo_handler.py", line 58, in ? > split_path = default_handler.split_path > AttributeError: 'default_handler' module has no attribute 'split_path' > > > Any idea what is going on? Something changed in 2.1? From rushing@n... Mon May 14 17:33:58 2001 From: rushing@n... (Sam Rushing) Date: Mon, 14 May 2001 09:33:58 -0700 Subject: [medusa] Re: Medusa: global name 'split_path' is not defined: file: status_handler.pyline: 63 References: <9doqch+kjvq@e...> Message-ID: <3B0008F6.E296A809@n...> eugene.leitl@l... wrote: > Er. Disregard my last message (well, ok, you already did that ;) > [...] Oops, sorry about that. 8^) > > --- In medusa@y..., Eugene Leitl wrote: > > > > This is Python 2.1 and medusa-20010416.tar.gz > > > > I'm getting > > > > error: Server Error: exceptions.NameError, global name 'split_path' > is > > not defined: file: status_handler.py line: 63 > > The 'split_path' function (which was copied around to several other modules) was replaced with a 'split_uri()' method on the request object, which also saves a little effort by memoizing the result. To fix, just remove all references to the split_path function, changing " = split_path (path)" to " = request.split_uri (path)" Let me know if you have any trouble with this... -Sam From weissblitz@y... Wed May 16 22:19:51 2001 From: weissblitz@y... (Weiss Blitz) Date: Wed, 16 May 2001 21:19:51 -0000 Subject: Asyncore's socket 'send' method not found?! Message-ID: <9duqtn+710s@e...> My Medusa server is crashing with the following error: (exceptions.AttributeError:send [asyncore.py|poll|79] [asyncore.py|handle_read_event|337] [asynchat.py|handle_read|128] [zulu_pager_server.py|found_terminator|192] [zulu_pager_server.py|process_command|287] [zulu_pager_server.py|cmd_login|511] [zulu_pager_server.py|send_response|213] [zulu_pager_server.py|push_packet|201] [asynchat.py|push_with_producer|155] [asynchat.py|initiate_send|212] [asyncore.py|send|272]) also get this error sometimes: (exceptions.AttributeError:'int' object has no attribute 'send' [asyncore.py|poll|79] [asyncore.py|handle_read_event|337] [asynchat.py|handle_read|80] [asyncore.py|recv|294] [zulu_chat_server.py|handle_close|48] [zulu_chat_server.py|close|53] [zulu_chat_server.py|push_packet|388] [zulu_chat_server.py|push_packet|73] [asynchat.py|push_with_producer|155] [asynchat.py|initiate_send|212] [asyncore.py|send|272] [|send|1]) Seems it's not finding the asyncore's socket 'send' method when this happens.. Anyone have seen this before? From eugene.leitl@l... Thu May 17 17:45:37 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Thu, 17 May 2001 18:45:37 +0200 (MET DST) Subject: medusa/bobo html and cookies Message-ID: Okay, it's time for the next inevitable round of questions. I don't expect too many answers, as we're apparently not many (most people are probably hacking Zope right now), and busy with our own projects. Nevertheless I do intend to use Medusa/Bobo for me privately as well as at work, and I'll stick around here to answer beginner's questions. Also, what is needed is making the small changes to keep Medusa/Bobo running on latest versions of Python. While the rapid development *is* a good thing, I'm starting to run into incompatibilities between several pieces of software, especially if several of them are present in a single project (right now in my case GadFly, Bobo, Medusa and xmlrpclib). This can be incredibly frustrating, especially for a beginner with not too much time on one's hands. Anyways, I intend to stick around and see what I can do to address these points. So whatever information you toss my way won't vanish below an event horizont, okay? I'm going to pass on the torch, and stuff. So, what I'm missing is a library of demo code for the Medusa/Bobo framework, doing both simple things and complicated things. Right now I'm looking for image generation, cookie manipulation (beyond of what is in the archive -- surely Bobo has extras for cookie mangling?) and XML-RPC. If you have such sources, and are willing to share, that would be great. I haven't checked with my employer, but I'm going to publish the framework of the application I'm currently writing anyway, as it is commercially worthless without the proprietary dbase. From rushing@n... Fri May 18 20:53:50 2001 From: rushing@n... (Sam Rushing) Date: Fri, 18 May 2001 12:53:50 -0700 Subject: [medusa] Asyncore's socket 'send' method not found?! References: <9duqtn+710s@e...> Message-ID: <3B057DCE.B06A216B@n...> Weiss Blitz wrote: > My Medusa server is crashing with the following error: > > (exceptions.AttributeError:send [asyncore.py|poll|79] Just a guess, but I'd make sure you have only one copy of asyn{core,char}.py in your python path. This sounds a little like an accidental mixing of old and new versions... Another thing to double-check is that you've called the __init__ method of the parent class... -Sam From jim@d... Sat May 19 18:00:23 2001 From: jim@d... (Jim Fulton) Date: Sat, 19 May 2001 13:00:23 -0400 Subject: Bug in parsing request paths, and patch Message-ID: <3B06A6A7.BF4B1831@d...> The HTTP server sort of follows RFC 1808 when parsing paths. RFC 1808 specified a URL as: :///;?# The HTTP request line contains the part after the net_loc, which the Medusa HTTP server parses into 4 parts, path, params, query, and fragment. It ignores (tosses) the params and fragment. RFC 2396 supercedes RFC 1808 and incorporates 'params' into the path. Loosely, a path is a sequence of path segments separated by slashes. Each path segment has a name followed by 0 or more parameters, where each parameter is set off by a semicolon, as in: /namex/namey;p1=v2;p2=v2/namez;p4/.... I've never seen this syntax actually used, but I'm considering using it in Zope to provide more explicit control over name lookup. Medusa's current request parsing breaks this syntax and, technically, violates RFC 2396. It lops off everything after a semicolon up to a query string. For example, the above path becomes: /namex/namey I suggest that, especially given RFC 2396, the parsing and interpretation of parameters should be left to individual handlers. The simplest fix would be to change the parser to ignore the semicolons in paths and return an empty string for the "params" part. The patch is included below. Jim diff -c -r1.25 http_server.py *** http_server.py 2001/05/01 12:49:04 1.25 --- http_server.py 2001/05/19 16:53:10 *************** *** 95,105 **** # split a uri # -------------------------------------------------- ! # ;?# path_regex = re.compile ( ! # path params query fragment ! r'([^;?#]*)(;[^?#]*)?(\?[^#]*)?(#.*)?' ! ) def split_uri (self): if self._split_uri is None: --- 95,105 ---- # split a uri # -------------------------------------------------- ! # ?# path_regex = re.compile ( ! # path query fragment ! r'([^?#]*)(\?[^#]*)?(#.*)?' ! ) def split_uri (self): if self._split_uri is None: *************** *** 107,113 **** if m.end() != len(self.uri): raise ValueError, "Broken URI" else: ! self._split_uri = m.groups() return self._split_uri def get_header_with_regex (self, head_reg, group): --- 107,114 ---- if m.end() != len(self.uri): raise ValueError, "Broken URI" else: ! m=m.groups() ! self._split_uri = m[0], '', m[1], m[2] return self._split_uri def get_header_with_regex (self, head_reg, group): -- Jim Fulton mailto:jim@d... Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org From rcspython@y... Sun May 20 16:46:01 2001 From: rcspython@y... (rcspython@y...) Date: Sun, 20 May 2001 15:46:01 -0000 Subject: Does Medusa's FTP Server have built in triggers? Message-ID: <9e8orp+ff1e@e...> Hello All, I am a total newbie to Python and Medusa for that matter. I want to use the ftpserver, however, I have to trigger some server side scriping after a file(s) is uploaded. If this is not supported at this time, how difficult would you say it is to add this feature? Best Regards! Ron From sav@u... Mon May 21 06:11:17 2001 From: sav@u... (Alexander Semenov) Date: Mon, 21 May 2001 09:11:17 +0400 Subject: [medusa] Does Medusa's FTP Server have built in triggers? References: <9e8orp+ff1e@e...> Message-ID: <00dd01c0e1b4$771a9e90$8220a8c0@u...> > Hello All, > I am a total newbie to Python and Medusa for that matter. I want to > use the ftpserver, however, I have to trigger some server side > scriping after a file(s) is uploaded. Better (more pythonic) use overloading instead callbacks. You should look into ftp_server.py I think you are interested in overloading recv_channel.handle_close() or whole class recv_channel (about 40 lines of code) if you want alternate file storing. You can, of cause, simply hack ftp_server.py. WBR, Alexander Semenov From rushing@n... Mon May 21 08:57:42 2001 From: rushing@n... (Sam Rushing) Date: Mon, 21 May 2001 00:57:42 -0700 Subject: [medusa] Does Medusa's FTP Server have built in triggers? References: <9e8orp+ff1e@e...> Message-ID: <3B08CA76.C9110746@n...> rcspython@y... wrote: > I am a total newbie to Python and Medusa for that matter. I want to > use the ftpserver, however, I have to trigger some server side > scriping after a file(s) is uploaded. > > If this is not supported at this time, how difficult would you say it > is to add this feature? Should be pretty easy to do... I'd just derive from the ftp_channel class and override the cmd_stor method... -Sam From lucifer@e... Mon May 21 17:17:38 2001 From: lucifer@e... (lucifer@e...) Date: Mon, 21 May 2001 16:17:38 -0000 Subject: Instant messaging & ICQ Message-ID: <9ebf32+3nk1@e...> Hi I'm one of those who are trying to build an instant messenger based on medusa. I was wondering if anyone has tried to communicate with ICQ. I know there is a C library, but no python port. I'm no C programmer, so I cant port it, not even using SWING I believe, but I was planning to study the protocol and build my own library. Has anyone tried something similar ? Or tried to port the C library ? From eugene.leitl@l... Wed May 23 13:53:39 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Wed, 23 May 2001 14:53:39 +0200 (MET DST) Subject: FYI: XML-RPC with Medusa Message-ID: http://aspn.activestate.com/ASPN/Python/Cookbook/Recipe/52226 Title: XML-RPC with Medusa By: Jeff Bauer Last Updated: 2001/03/11 Version no: 1.0 1 vote(s) Description: This section demonstrates remote method calls between two machines (or processes) using the XML-RPC protocol. A complete example of working client/server code is provided. Source: Text Source # xmlrpc_server.py from socket import gethostname from medusa.xmlrpc_handler import xmlrpc_handler from medusa.http_server import http_server from medusa import asyncore class xmlrpc_server(xmlrpc_handler): """The xmlrpc_server class demonstrates a simple implementation of Userland's XML-RPC protocol. You must download and install xmlrpclib and Medusa to run this code. Obtain Sam Rushing's Medusa library from http://www.nightmare.com Download Fredrik Lundh's xmlrpclib at http://www.pythonware.com""" def __init__(self, host=None, port=8182): if host is None: host = gethostname() hs = http_server(host, port) hs.install_handler(self) asyncore.loop() def add(self, op1, op2): return op1 + op2 def call(self, method, params): print "call method: %s, params: %s" % (method, str(params)) if method == 'add': return apply(self.add, params) return "method not found: %s" % method if __name__ == '__main__': server = xmlrpc_server() --- # xmlrpc_client.py from socket import gethostname from xmlrpclib import Transport, dumps class xmlrpc_connection: """The xmlrpc_connection class tests the xmlrpc_server. You must download and install the medusa and xmlrpclib libraries to run this code: http://www.nightmare.com http://www.pythonware.com""" def __init__(self, host=None, port=8182): if host is None: host = gethostname() self.host = "%s:%s" % (host, port) self.transport = Transport() def remote(self, method, params=()): """remote invokes the server with the method name and an optional set of parameters. The return value is always a tuple.""" response = self.transport.request(self.host, '/RPC2', dumps(params, method)) return response if __name__ == '__main__': connection = xmlrpc_connection() (answer,) = connection.remote("add", (40, 2)) print "The answer is:", answer Discussion: XML-RPC is one of the easier ways to handle distributed processing tasks. There's no messing around with the low-level socket details, nor is it necessary to write an interface definition. The protocol is platform-neutral and language-neutral. The XML-RPC specification can be found at: http://www.xml-rpc.com To run the example above, it is necessary to download xmlrpclib from http://www.pythonware.com and the Medusa library from http://www.nightmare.com. [ Aside: Perhaps someone from ActiveState can provide an additional client example in Perl, to demonstrate how multiple languages can be used together with xmlrpc. -Jeff ] From eugene.leitl@l... Sun May 27 13:37:16 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Sun, 27 May 2001 14:37:16 +0200 Subject: [Fwd: Re: Distributed computing in Python - Callback howto ?] Message-ID: <3B10F4FC.FCAFF7FC@l...> -------- Original Message -------- From: Doug Fort Subject: Re: Distributed computing in Python - Callback howto ? To: python-list@p... Newsgroups: comp.lang.python Sasa Zivkov wrote: > Hi, > > What options one have for distributed computing in Python ? > Check out Frederik Lund's implementation of xmlrpc -- http://www.pythonware.com/products/xmlrpc/index.htm If you want to see some sample code, we use it in our agent infrastructure http://pyagent.sourceforge.net -- Doug Fort Senior Meat Manager Downright Software LLC http://www.downright.com ______________________________________________________________________ Posted Via Uncensored-News.Com - Still Only $9.95 - http://www.uncensored-news.com With Seven Servers In California And Texas - The Worlds Uncensored News Source -- http://mail.python.org/mailman/listinfo/python-list From altis@y... Wed May 30 20:55:12 2001 From: altis@y... (altis@y...) Date: Wed, 30 May 2001 19:55:12 -0000 Subject: Instant messaging & ICQ In-Reply-To: <9ebf32+3nk1@e...> Message-ID: <9f3j70+f3e5@e...> You might want to check out the Jabber infrastructure. The Jabber protocol justs passes around XML until it hits a Jabber gateway which can then talk to ICQ, Yahoo Messenger, MSN Messenger, and AIM, or any other defined gateway on the server. http://www.jabber.org/ http://www.jabbercentral.org/ http://www.jabber.com/ There are numerous clients and servers available for different platforms as well as documentation on the protocol, etc. so that should help get you going. Since there are already Python libraries to do the XML parsing for you, it should simplify development. ka --- In medusa@y..., lucifer@e... wrote: > Hi > > I'm one of those who are trying to build an instant messenger based on > medusa. > I was wondering if anyone has tried to communicate with ICQ. > I know there is a C library, but no python port. > I'm no C programmer, so I cant port it, not even using SWING I > believe, but I was planning to study the protocol and build my own > library. > Has anyone tried something similar ? Or tried to port the C library ? From altis@s... Wed May 30 20:59:20 2001 From: altis@s... (Kevin Altis) Date: Wed, 30 May 2001 12:59:20 -0700 Subject: turning off DNS resolves, ftp, chat Message-ID: I searched for an answer to the DNS issue, but didn't see a response. I just want to log IP addresses and skip the DNS resolve. Also, it appears there may be a problem connecting to medusa if you simply try connecting on the local machine as http://localhost/ Also, I just installed Medusa and don't need the FTP or Chat servers. Do I just need to comment out those lines in the startup? ka From rprice@u... Wed May 30 21:04:05 2001 From: rprice@u... (rprice@u...) Date: Wed, 30 May 2001 13:04:05 -0700 (PDT) Subject: [medusa] turning off DNS resolves, ftp, chat In-Reply-To: References: Message-ID: <991253045.3b1552352510c@w...> Quoting Kevin Altis : > I searched for an answer to the DNS issue, but didn't see a response. I > just > want to log IP addresses and skip the DNS resolve. > Set the rs variable like this to only log IP addresses: rs = None > Also, it appears there may be a problem connecting to medusa if you > simply > try connecting on the local machine as http://localhost/ Is is set to run on port 80? You may need to access it like this: http://localhost:8080/ which is the default.. as I remember. > > Also, I just installed Medusa and don't need the FTP or Chat servers. Do > I > just need to comment out those lines in the startup? Yes, and besure to comment all occurances of the ftp and cs variables. > > ka > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > Ray From altis@y... Wed May 30 23:13:42 2001 From: altis@y... (altis@y...) Date: Wed, 30 May 2001 22:13:42 -0000 Subject: turning off DNS resolves, ftp, chat In-Reply-To: <991253045.3b1552352510c@w...> Message-ID: <9f3ram+7nvj@e...> --- In medusa@y..., rprice@u... wrote: > > Also, it appears there may be a problem connecting to medusa if you > > simply > > try connecting on the local machine as http://localhost/ > Is is set to run on port 80? You may need to access it like this: > http://localhost:8080/ > which is the default.. as I remember. Thanks for the info. I think there is actually a bug here. I'm using Python 2.1 (ActivePython build 210) on Windows 2000 Pro. I'm using port 80. If I configure as: IP_ADDRESS = '127.0.0.1' then I can connect with IE from the same machine using http://localhost/ as expected. However, I am then unable to connect to the server from either the local machine or another machine on the LAN using either the IP address or DNS name of the server machine. If I configure the IP_ADDRESS to use the actual IP address, then I can't connect as localhost or via 127.0.0.1 which is the same thing. Somehow the binding is broken. I tested using a simple telnet connection to port 80 and got the same results. Normally, a simple server would do something like: from socket import * # get socket constructor and constants myHost = '' # server machine, '' means local host myPort = 80 # port to listen on sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object sockobj.bind((myHost, myPort)) # bind it to server port number sockobj.listen(5) # listen, allow 5 pending connects I haven't looked at the code for medusa yet, but the above code definitely works for localhost or using the real IP address or DNS name. ka From rprice@u... Wed May 30 23:28:47 2001 From: rprice@u... (rprice@u...) Date: Wed, 30 May 2001 15:28:47 -0700 (PDT) Subject: [medusa] Re: turning off DNS resolves, ftp, chat In-Reply-To: <9f3ram+7nvj@e...> References: <9f3ram+7nvj@e...> Message-ID: <991261727.3b15741fc96e1@w...> What happens if you set IP_ADDRESS = '' Ray Quoting altis@y...: > --- In medusa@y..., rprice@u... wrote: > > > Also, it appears there may be a problem connecting to medusa if > you > > > simply > > > try connecting on the local machine as http://localhost/ > > Is is set to run on port 80? You may need to access it like this: > > http://localhost:8080/ > > which is the default.. as I remember. > > Thanks for the info. I think there is actually a bug here. I'm using > Python 2.1 (ActivePython build 210) on Windows 2000 Pro. I'm using > port 80. If I configure as: > IP_ADDRESS = '127.0.0.1' > then I can connect with IE from the same machine using > http://localhost/ as expected. However, I am then unable to connect > to the server from either the local machine or another machine on the > LAN using either the IP address or DNS name of the server machine. If > I configure the IP_ADDRESS to use the actual IP address, then I can't > connect as localhost or via 127.0.0.1 which is the same thing. > Somehow the binding is broken. I tested using a simple telnet > connection to port 80 and got the same results. > > Normally, a simple server would do something like: > > from socket import * # get socket constructor and constants > myHost = '' # server machine, '' means local host > myPort = 80 # port to listen on > > sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object > sockobj.bind((myHost, myPort)) # bind it to server port number > sockobj.listen(5) # listen, allow 5 pending connects > > I haven't looked at the code for medusa yet, but the above code > definitely works for localhost or using the real IP address or DNS > name. > > ka > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > From altis@y... Thu May 31 00:37:45 2001 From: altis@y... (altis@y...) Date: Wed, 30 May 2001 23:37:45 -0000 Subject: turning off DNS resolves, ftp, chat In-Reply-To: <991261727.3b15741fc96e1@w...> Message-ID: <9f4089+tvi4@e...> --- In medusa@y..., rprice@u... wrote: > What happens if you set > IP_ADDRESS = '' > Doh! That works for localhost, 127.0.0.1, actual IP address and DNS name, so I guess that is the appropriate thing to use. Thanks I am seeing log entries like 127.0.0.1:2009 but I think that bug was discussed in a different thread. ka From lucifer@e... Thu May 31 16:31:11 2001 From: lucifer@e... (lucifer@e...) Date: Thu, 31 May 2001 15:31:11 -0000 Subject: Instant messaging & ICQ In-Reply-To: <9f3j70+f3e5@e...> Message-ID: <9f5o3v+8i8h@e...> Thanks ka, I will check it out. --- In medusa@y..., altis@y... wrote: > You might want to check out the Jabber infrastructure. The Jabber > protocol justs passes around XML until it hits a Jabber gateway which > can then talk to ICQ, Yahoo Messenger, MSN Messenger, and AIM, or any > other defined gateway on the server. > > http://www.jabber.org/ > http://www.jabbercentral.org/ > http://www.jabber.com/ > > There are numerous clients and servers available for different > platforms as well as documentation on the protocol, etc. so that > should help get you going. Since there are already Python libraries > to do the XML parsing for you, it should simplify development. > > ka > --- In medusa@y..., lucifer@e... wrote: > > Hi > > > > I'm one of those who are trying to build an instant messenger based > on > > medusa. > > I was wondering if anyone has tried to communicate with ICQ. > > I know there is a C library, but no python port. > > I'm no C programmer, so I cant port it, not even using SWING I > > believe, but I was planning to study the protocol and build my own > > library. > > Has anyone tried something similar ? Or tried to port the C > library ? From altis@s... Fri Jun 1 04:53:48 2001 From: altis@s... (Kevin Altis) Date: Thu, 31 May 2001 20:53:48 -0700 Subject: directory listing produces 404 error Message-ID: I'm sure this is just another newbie question, but how do I turn on directory listings in medusa for http requests? Right now, if I go to a directory that has an index.html file it returns that file, but other directories produce a 404 error. ka From eugene.leitl@l... Fri Jun 1 17:25:31 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Fri, 1 Jun 2001 18:25:31 +0200 (MET DST) Subject: Webware for Python Message-ID: Anyone checked this out yet? http://webware.sourceforge.net/ If yes, how does this compare to Bobo/Medusa combo? -- Eugene ______________________________________________________________ ICBMTO : N48 10'07'' E011 33'53'' http://www.lrz.de/~ui22204 57F9CFD3: ED90 0433 EB74 E4A9 537F CFF5 86E7 629B 57F9 CFD3 From altis@y... Fri Jun 1 20:18:14 2001 From: altis@y... (altis@y...) Date: Fri, 01 Jun 2001 19:18:14 -0000 Subject: Webware for Python In-Reply-To: Message-ID: <9f8ppm+38b7@e...> If you read the various papers and docs on the WebWare site you'll see that the design is more like WebObjects http://www.apple.com/webobjects/ than a simple CGI framework. Consequently, the additional overhead of using an application server would make it overkill for simple CGIs, but it should do quite nicely as the complexity of your web application increases. At this point, it is unclear to me how much inconvenience the framework causes when you want to mix it with static pages, images, regular CGIs, servlets, etc. I only started looking at it a few days ago, but I'm impressed by the amount of work that has gone into it and the quality of documentation. It is certainly worth experimenting with. ka --- In medusa@y..., Eugene Leitl wrote: > > Anyone checked this out yet? > > http://webware.sourceforge.net/ > > If yes, how does this compare to Bobo/Medusa combo? > > -- Eugene > > ______________________________________________________________ > ICBMTO : N48 10'07'' E011 33'53'' http://www.lrz.de/~ui22204 > 57F9CFD3: ED90 0433 EB74 E4A9 537F CFF5 86E7 629B 57F9 CFD3 From p_pillai_p@r... Wed Jun 6 18:48:41 2001 From: p_pillai_p@r... (p_pillai_p@r...) Date: Wed, 06 Jun 2001 17:48:41 -0000 Subject: FTP Server Message-ID: <9flqdp+4qqi@e...> hi I want to develop a FTP server starting from the beginning.I wish that the server supports simple functions like PUT and GET.Also it should be multitasking. Can someone guide me in this regard. From cgale1@h... Fri Jun 8 10:02:58 2001 From: cgale1@h... (Van Gale) Date: Fri, 8 Jun 2001 02:02:58 -0700 Subject: [medusa] FTP Server References: <9flqdp+4qqi@e...> Message-ID: <001501c0eff9$d24b0180$0200a8c0@e...> ----- Original Message ----- From: To: Sent: Wednesday, June 06, 2001 10:48 AM Subject: [medusa] FTP Server > I want to develop a FTP server starting from the beginning.I wish > that the server supports simple functions like PUT and GET.Also it > should be multitasking. Can someone guide me in this regard. I'm not sure why you are asking this question. Even if you want to develop a new FTP server from the beginning, why is it you do not want to use the medusa ftp_server.py module to guide you? Van Gale From cgale1@h... Fri Jun 8 10:17:38 2001 From: cgale1@h... (Van Gale) Date: Fri, 8 Jun 2001 02:17:38 -0700 Subject: Possible small bugs in counter.py and auth_handler.py? Message-ID: <001b01c0effb$e0da3340$0200a8c0@e...> I've really only been using Python since version 2.1, so I'm not aware of all the issues involved in porting older code, however I *think* I have found an inconsequential bug in counter.py related to differences in long() vs. string.atol. In counter.py, it seems the __str__ method is chopping off the last digit of the counter because of a [:-1] slice on the string returned by str(long()). If I change the line >>> return str(long(self.value))[:-1] to >>> return str(long(self.value)) printing the counter now works correctly. The method "make_nonce" in auth_handler.py seems to be doing the same thing (found by simply doing a grep for [:-1]) Van Gale From skip@p... Fri Jun 8 15:55:46 2001 From: skip@p... (Skip Montanaro) Date: Fri, 8 Jun 2001 09:55:46 -0500 Subject: [medusa] Possible small bugs in counter.py and auth_handler.py? In-Reply-To: <001b01c0effb$e0da3340$0200a8c0@e...> References: <001b01c0effb$e0da3340$0200a8c0@e...> Message-ID: <15136.59250.965374.487184@b...> Van> If I change the line >>>> return str(long(self.value))[:-1] Van> to >>>> return str(long(self.value)) Van> printing the counter now works correctly. Yes, str's handling of longs changed in 1.6, so if you are using longs in 1.6 or beyond, you have to check/change such code. In 1.5 and earlier versions, str(long(1)) would have returned "1L" instead of just "1". -- Skip Montanaro (skip@p...) (847)971-7098 From rushing@n... Fri Jun 8 20:16:48 2001 From: rushing@n... (Sam Rushing) Date: Fri, 08 Jun 2001 12:16:48 -0700 Subject: [medusa] Possible small bugs in counter.py and auth_handler.py? References: <001b01c0effb$e0da3340$0200a8c0@e...> Message-ID: <3B2124A0.D0CFF9A9@n...> Van Gale wrote: > I've really only been using Python since version 2.1, so I'm not aware of > all the issues involved in porting older code, however I *think* I have > found an inconsequential bug in counter.py related to differences in long() > vs. string.atol. > The method "make_nonce" in auth_handler.py seems to be doing the same thing > (found by simply doing a grep for [:-1]) The [:-1] was to slice off the 'L' on the end of the string representation of long numbers. This must have been changed since python1.x... -Sam From abo@m... Mon Jun 11 06:59:30 2001 From: abo@m... (Donovan Baarda) Date: Mon, 11 Jun 2001 15:59:30 +1000 Subject: medusa ready() patch In-Reply-To: <0106080204430E.11391@l...>; from kthangavelu@e... on Fri, Jun 08, 2001 at 02:05:24AM -0700 References: <0106080204430E.11391@l...> Message-ID: <20010611155930.B17879@m...> On Fri, Jun 08, 2001 at 02:05:24AM -0700, ender wrote: > Hello Donovan, > > I was curious if there was any further developments regarding your patch to > add a ready methods to producers. Namely if you had developed the patches > further or used them in production and if there had been any discussion with > sam rushing regarding inclusion with medusa? I have not had a chance to actualy use it in anger myself yet, but I have had some feedback from a few people playing with it. One post pointed out that I seemed to have re-invented a slightly different design-pattern to the one medusa was based on... a total accident :-) I have also failed to keep it up to date with medusa. Sam has put out at least one new version since I did my patch, so I need to update it. Interestingly, at least one of the things I did in my patch seems to have been adopted and improved on in Sam's new version (removing the cache for find_prefix_at_end), though I think it was independantly developed without looking at my patch. Certainly I have not had any feedback from Sam about it. I really don't know how much demand there would be for something like this in medusa, or even the async bits in Python. Everything it does can be done probably better by using proper async objects rather than blocking producers. The only reason I did it was because the existing medusa achitecture assumes that its virtual filesystem files are non-blocking, and making producers block was a quicker fix than making the whole medusa virtual filesystem asynchronos. I did make my changes backwards compatible, so it shouldn't hurt too much if they were included in medusa itself. However, some of my other changes to things like the producer class heirachy might have been too heavy handed for someone like Sam to comfortably accept. If enough people did feel it was worth including, and Sam was willing, I'd be more than happy to do what is needed to make it suitable for inclusion into medusa itself. I am still planning on using it soon, so I will be updating it. I'm also willing to support it as an add-on patch to medusa, tracking medusa development. I'm open to any other suggestions too :-) -- ---------------------------------------------------------------------- ABO: finger abo@m... for more info, including pgp key ---------------------------------------------------------------------- From tshab@y... Tue Jun 12 05:10:55 2001 From: tshab@y... (tshab@y...) Date: Tue, 12 Jun 2001 04:10:55 -0000 Subject: Streaming Data Socket Server Message-ID: <9g44of+9qov@e...> I'm trying to consolidate two similar programs (one in C, one that uses National Instruments Datasockets - www.ni.com/datasocket), and someone suggested Medusa. Basically, I'm trying to collect data from multiple streaming socket producers, which will then be connected via the server switchboard to one or more socket consumers. Also, the channel of the currently connected consumers needs to be listed on a dynamically created web page. Right now, the protocol is almost non-existent, just an 's' for producers, or the channel number for consumers. I looked at asynchat, because it has the nice channel handling, but I don't know how to get around the set_terminator issue (there is no delimiter on the data). So, any pointers would be appreciated on: a. Mapping channels to multiple consumers b. Handling streamed data in asynchat correctly c. Dynamic web page generation (should I use one of the templating packages)? Finally, I'm hoping to extend this design in the future to support things such as encryption and compression of the streams. Would I be able to use producers on streams of data? Thanks in advance for any advice. Best, Ted From andreas@d... Mon Jul 9 16:10:28 2001 From: andreas@d... (andreas@d...) Date: Mon, 09 Jul 2001 15:10:28 -0000 Subject: asyncore: 'unhandled connect event' Message-ID: <9ichh4+rh6g@e...> For some FTP clients (e.g. WSFTP) Zope 2.3/2.4 logs an "unhandled connect event". This message is generated in asyncore module of Python 2.1 in handle_connect(). This message confuses lots of people because the message is logged for every FTP command. To avoid this "mis"-behaviour of Medusa I suggest to add a dummy handle_connect() function to the xmit_channel class to suppress the warning. Andreas From max@a... Thu Jul 12 18:58:59 2001 From: max@a... (max@a...) Date: Thu, 12 Jul 2001 17:58:59 -0000 Subject: default_with_post_handler Message-ID: <9ikoh3+cmrj@e...> I got the latest drop of Medusa (20010416) and am attempting to implement a non-standard HTTP server with it (for the Web game Interstelen, if anyone's interested: http://www.interstelen.com/). The server should accept both GET and POST (GET to start with for debugging, POST for a cleaner interface unfettered by query size limits). I started implementing an HTTP server myself (but still using the asyncore/asynchat framework) and decided my time would be better spent using a preexisting, known reliable HTTP server that's in Medusa. So I looked to default_handler.py to find a POST example, and sure enough: # support for handling POST requests is available in the derived # class , defined below. But ... emf@s...:/usr/local/lib/python2.0/site-packages/medusa% grep default_with_post_handler *.py default_handler.py:# class , defined below. That's the only reference to that class I can find anywhere. Where is this class? As a parting comment, I find asyncore/asynchat/Medusa very powerful and very convenient, but I haven't really been able to find any documentation beyond sketchy READMEs and tutorials. Even something along the lines of HappyDoc-generated API documentation based solely on the class hierarchy, the methods in classes, and the docstrings that appear here and there would be helpful. Does any exist somewhere? From max@a... Thu Jul 12 21:55:16 2001 From: max@a... (max@a...) Date: Thu, 12 Jul 2001 20:55:16 -0000 Subject: Invalid protocol in query results in exception Message-ID: <9il2rl+auph@e...> Sending GET / 1.0\r\n\r\n instead of GET / HTTP/1.0\r\n\r\n to Medusa 3.41 results in: error: uncaptured python exception, closing channel (exceptions.AttributeError:'None' object has no attribute 'end' [/usr/local/lib/python2.0/site-packages/medusa/asyncore.py|poll|96] [/usr/local/lib/python2.0/site-packages/medusa/asy ncore.py|handle_read_event|384] [/usr/local/lib/python2.0/site-packages/medusa/asy nchat.py|handle_read|131] [/usr/local/lib/python2.0/site-packages/medusa/htt p_server.py|found_terminator|463] [/usr/local/lib/python2.0/site-packages/medusa/htt p_server.py|crack_request|692]) which closes the HTTP connection rather than just responding with a 400 (Bad Request). From rushing@n... Fri Jul 13 09:04:29 2001 From: rushing@n... (Sam Rushing) Date: Fri, 13 Jul 2001 01:04:29 -0700 Subject: [medusa] default_with_post_handler References: <9ikoh3+cmrj@e...> Message-ID: <3B4EAB8D.4FF71E78@n...> max@a... wrote: > So I looked to default_handler.py to find a POST example, and sure > enough: > > # support for handling POST requests is available in the derived > # class , defined below. > > But ... > > emf@s...:/usr/local/lib/python2.0/site-packages/medusa% grep > default_with_post_handler *.py > default_handler.py:# class , defined below. Ahh... an outdated comment. I would recommend looking at the post-handling capabilities of the 'script_handler' class in script_handler.py, there should be enough info there to help. You might event want to use one of the script-handling classes, since they support what most folks would want to do with a medusa-based web server. > That's the only reference to that class I can find anywhere. Where is > this class? > > As a parting comment, I find asyncore/asynchat/Medusa very powerful > and very convenient, but I haven't really been able to find any > documentation beyond sketchy READMEs and tutorials. Even something > along the lines of HappyDoc-generated API documentation based solely > on the class hierarchy, the methods in classes, and the docstrings > that appear here and there would be helpful. Does any exist > somewhere? That's interesting, I've just been discussing the lack of documentation with someone else. IMHO when the code is short, it should serve as the documentation. It's less likely to be misleading, and much more likely to be written. So one of my most important goals when writing code is to keep it short enough to not require documentation. 8^) For me, the most important part of a library are the examples. Tons of documentation can actually be a hindrance without short examples to get you started. Anyway, enough weak excuses for my lack of motivation to document... [Of course, none of this excuses the lack of proper doc strings...] -Sam From rushing@n... Fri Jul 13 09:11:48 2001 From: rushing@n... (Sam Rushing) Date: Fri, 13 Jul 2001 01:11:48 -0700 Subject: [medusa] Invalid protocol in query results in exception References: <9il2rl+auph@e...> Message-ID: <3B4EAD44.ECF106AA@n...> ---------------------- multipart/alternative attachment max@a... wrote: > Sending > GET / 1.0\r\n\r\n > instead of > GET / HTTP/1.0\r\n\r\n > to Medusa 3.41 results in: > [...] > which closes the HTTP connection rather than just responding with a > 400 (Bad Request). Here's a quick fix: Index: http_server.py =================================================================== RCS file: /usr/local/cvsroot/medusa/http_server.py,v retrieving revision 3.41 retrieving revision 3.43 diff -r3.41 -r3.43 9c9 < RCS_ID = '$Id: http_server.py,v 3.41 2001/04/16 03:41:29 rushing Exp $' --- > RCS_ID = '$Id: http_server.py,v 3.43 2001/07/13 08:02:38 rushing Exp $' 107c107 < if m.end() != len(self.uri): --- > if not m or m.end() != len(self.uri): 116c116 < if m.end() == len(line): --- > if m and m.end() == len(line): 692c692 < if m.end() == len(r): --- > if m and m.end() == len(r): -Sam ---------------------- multipart/alternative attachment An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail-21/medusa-dev/attachments/3abacd13/attachment.html ---------------------- multipart/alternative attachment-- From max@a... Mon Jul 16 22:50:01 2001 From: max@a... (max@a...) Date: Mon, 16 Jul 2001 21:50:01 -0000 Subject: Invalid protocol in query results in exception In-Reply-To: <3B4EAD44.ECF106AA@n...> Message-ID: <9ivni9+ingb@e...> --- In medusa@y..., Sam Rushing wrote: > max@a... wrote: > > > Sending > > GET / 1.0\r\n\r\n > > instead of > > GET / HTTP/1.0\r\n\r\n > > to Medusa 3.41 results in: > > [...] > > > which closes the HTTP connection rather than just responding with > > a > > 400 (Bad Request). > > Here's a quick fix: > > Index: http_server.py > =================================================================== > RCS file: /usr/local/cvsroot/medusa/http_server.py,v > retrieving revision 3.41 > retrieving revision 3.43 > diff -r3.41 -r3.43 ... Cool. For the record, I wasn't requesting a fix, just informing you guys of the bug. I happened to notice it in my testing, but it isn't a particularly urgent issue for me at the moment. Just wanted to let you know. From max@a... Mon Jul 16 22:54:28 2001 From: max@a... (max@a...) Date: Mon, 16 Jul 2001 21:54:28 -0000 Subject: default_with_post_handler In-Reply-To: <3B4EAB8D.4FF71E78@n...> Message-ID: <9ivnqk+3762@e...> --- In medusa@y..., Sam Rushing wrote: > max@a... wrote: > > Ahh... an outdated comment. > I would recommend looking at the post-handling capabilities of the > 'script_handler' class in script_handler.py, there should be enough > info > there to help. You might event want to use one of the > script-handling > classes, since they support what most folks would want to do with a > medusa-based web server. Yep, that ends up being exactly what I did, using script_handler as a guide for how to do POST. It works fine. > That's interesting, I've just been discussing the lack of > documentation with > someone else. IMHO when the code is short, it should serve as the > documentation. It's less likely to be misleading, and much more > likely to > be written. So one of my most important goals when writing code is > to keep > it short enough to not require documentation. 8^) That's certainly what I did. It's just that there are some relatively important concepts that don't seem to be documented anywhere in the tutorials or readmes (at least that I found). For instance, when I was using asynchat, it took a bit of digging to find out about the push_with_producer(None) trick. > For me, the most important part of a library are the examples. Tons > of > documentation can actually be a hindrance without short examples to > get you > started. Anyway, enough weak excuses for my lack of motivation to > document... [Of course, none of this excuses the lack of proper doc > strings...] Just thought I'd suggest it in case anybody hadn't prodded you, but more importantly, I just wanted to make sure I wasn't spectacularly missing some obvious documentation somewhere. From tusharwagle@y... Tue Jul 17 19:42:07 2001 From: tusharwagle@y... (tusharwagle@y...) Date: Tue, 17 Jul 2001 18:42:07 -0000 Subject: Adding per-directory security to ftp server (how to?) Message-ID: <9j20tv+hga9@e...> Hello, I've just started looking at Medusa and would like some pointers into the 'best' way to do the following: I want to check access at each directory in a filesystem which is served over both http and ftp. I've written a handler for the http server, but then realised that I'd have to modify it for the ftp server. Is there a recommended way to apply this kind of authentication to both http and ftp servers? Any help appreaciated! - Tushar P.S. Many Many thanks to Sam for releasing this as free software! :) From rushing@n... Tue Jul 17 20:06:43 2001 From: rushing@n... (Sam Rushing) Date: Tue, 17 Jul 2001 12:06:43 -0700 Subject: [medusa] Adding per-directory security to ftp server (how to?) References: <9j20tv+hga9@e...> Message-ID: <3B548CC3.949F03B8@n...> tusharwagle@y... wrote: > I want to check access at each directory in a filesystem > which is served over both http and ftp. > Is there a recommended way to apply this kind of > authentication to both http and ftp servers? I guess it depends on where the auth info is stored and how you access it. Not knowing too much more about it I would extend one of the classes in filesys.py to support your access check. The ftp server and the http default_handler both expect a 'filesystem' object. The ftp server currently retrieves that object from an authorizer, but you could change this, depending on whether you want to extend normal ftp auth or replace it. -Sam From jbarham@o... Tue Jul 31 00:20:00 2001 From: jbarham@o... (John E. Barham) Date: Mon, 30 Jul 2001 16:20:00 -0700 Subject: Multi-threaded web-server design issues Message-ID: ---------------------- multipart/alternative attachment Hi all, I'm trying to build a multi-threaded Web-server using Medusa's http_server. (I need threads because our Web app. has some long-lived CGI scripts w/ heavy database access.) I based the initial design on the example HTTP server from the thread_handler.py code in the thread dir. of the Medusa distro. However, that code obviously hasn't been touched in a while (I had to change it to use the "new" http_request.split_uri(), used the standard Queue module etc. etc.) so am wondering if there's a better approach. In particular, to simplify things, I'm thinking about creating a separate output queue (instead of the select_trigger stuff which has been giving me weird errors) where all of the CGI worker threads would put their output when they were finished. A separate response handling thread would then pull the responses off this queue and feed them back to Medusa. Is this a good approach? Also, from what I can see from the code, the only http_request method that isn't thread-safe is done(). Is this correct? That would mean that each CGI thread could safely manipulate the request object itself, leaving the response-handling queue to (serially) invoke done(). Thanks in advance for any feedback. John --- John E. Barham Senior Software Developer jbarham@o... www.octopuswireless.com Phone: (604) 434-1321 Fax: (604) 430-1419 Octopus Wireless Inc. 5317 Lane Street Burnaby, BC V5H 2H4 ---------------------- multipart/alternative attachment An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail-21/medusa-dev/attachments/e6fcc562/attachment.html ---------------------- multipart/alternative attachment-- From rushing@n... Tue Jul 31 09:44:46 2001 From: rushing@n... (Sam Rushing) Date: Tue, 31 Jul 2001 01:44:46 -0700 Subject: [medusa] Multi-threaded web-server design issues References: Message-ID: <3B666FFD.977FAB3F@n...> ---------------------- multipart/alternative attachment "John E. Barham" wrote: > I'm trying to build a multi-threaded Web-server using Medusa's > http_server. (I need threads because our Web app. has some long-lived > CGI scripts w/ heavy database access.) I based the initial design on > the example HTTP server from the thread_handler.py code in the thread > dir. of the Medusa distro. However, that code obviously hasn't been > touched in a while (I had to change it to use the "new" > http_request.split_uri(), used the standard Queue module etc. etc.) so > am wondering if there's a better approach. In particular, to simplify > things, I'm thinking about creating a separate output queue (instead > of the select_trigger stuff which has been giving me weird errors) > where all of the CGI worker threads would put their output when they > were finished. A separate response handling thread would then pull > the responses off this queue and feed them back to Medusa. Is this a > good approach? The purpose of the trigger is to provide a way to 'wake up' the select() call that is pending in the main asyncore event loop. Even if you had a separate thread that pulled out of a central queue, you'd still need to address this issue... > Also, from what I can see from the code, the only http_request method > that isn't thread-safe is done(). Is this correct? That would mean > that each CGI thread > could safely manipulate the request object itself, leaving the > response-handling queue to (serially) invoke done(). As long as you processed the requests carefully - for example, let's say that you were sent three pipelined requests on an http/1.1 channel. If you're not careful, the requests might get retired out-of-order, and thus push the responses out of order. If you disable http/1.1 (or just pipelining) then it's probably not an issue. -Sam ---------------------- multipart/alternative attachment An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail-21/medusa-dev/attachments/529b9b2d/attachment.html ---------------------- multipart/alternative attachment-- From abo@m... Wed Aug 1 01:26:32 2001 From: abo@m... (Donovan Baarda) Date: Wed, 01 Aug 2001 10:26:32 +1000 (EST) Subject: [medusa] Multi-threaded web-server design issues In-Reply-To: References: Message-ID: <996625592.3b674cb88a64b@m...> Quoting "John E. Barham" : > Hi all, > > I'm trying to build a multi-threaded Web-server using Medusa's > http_server. > (I need threads because our Web app. has some long-lived CGI scripts w/ > heavy database access.) I based the initial design on the example HTTP > server from the thread_handler.py code in the thread dir. of the Medusa > distro. However, that code obviously hasn't been touched in a while (I > had > to change it to use the "new" http_request.split_uri(), used the > standard > Queue module etc. etc.) so am wondering if there's a better approach. > In > particular, to simplify things, I'm thinking about creating a separate > output queue (instead of the select_trigger stuff which has been giving > me > weird errors) where all of the CGI worker threads would put their output > when they were finished. A separate response handling thread would then > pull the responses off this queue and feed them back to Medusa. Is this > a > good approach? You could have a look at ZServer that comes with Zope. This branched from medusa some time ago. It uses threads to get around the delays associated with fetching objects from remote databases. It uses quite a few nice tricks that you can use in medusa (callback producers etc). Another alternative is to use my patch that allows producers to block using a ready() method. This way you can embed the cgi thread into a producer that you can push strait onto medusa. If you want this, let me know. I'm yet to integrate my patch with the latest release of medusa, and a show of interest will encorage me to get around to it. -- ABO: finger abo@m... for more information. From Thomas@g... Thu Aug 2 12:29:44 2001 From: Thomas@g... (Thomas Weholt) Date: Thu, 2 Aug 2001 13:29:44 +0200 Subject: Benefits from using ftp to download vs. http ?? Message-ID: <3BC7333BE30CD511AEB700B0D0AB2F5803D246@L...> Hi, I've written a simple file-sharing app using http as web-frontend and ftp to serve the actual files. Are there any benefits from this; do I ease the "pressure" on the http-interface so it can handle more clients by doing all downloading using ftp? Thomas From rushing@n... Thu Aug 2 17:41:22 2001 From: rushing@n... (Sam Rushing) Date: Thu, 02 Aug 2001 09:41:22 -0700 Subject: [medusa] Benefits from using ftp to download vs. http ?? References: <3BC7333BE30CD511AEB700B0D0AB2F5803D246@L...> Message-ID: <3B6982B2.C0B0939B@n...> Thomas Weholt wrote: > I've written a simple file-sharing app using http as web-frontend and ftp to > serve the actual files. Are there any benefits from this; do I ease the > "pressure" on the http-interface so it can handle more clients by doing all > downloading using ftp? FTP is the more complicated protocol; it uses two TCP connections per download rather than one. HTTP is your better bet, unless you need the extra capabilities of FTP for some reason (access control, mostly). -Sam From Thomas@g... Tue Aug 7 08:48:40 2001 From: Thomas@g... (Thomas Weholt) Date: Tue, 7 Aug 2001 09:48:40 +0200 Subject: Adding/removing handlers dynamically Message-ID: <3BC7333BE30CD511AEB700B0D0AB2F5803D262@L...> I need to add/remove handlers dynamically. I got a modular web-server that searches a given folder for new handlers, when it finds one it would be cool to have it install it without restarting the server, all automatic. If I could build a server that search for new handlers at a given interval, installs all new, removes all handlers that have been removed, it would open for some really cool features. Any hints on how to implement a way of loading handlers dynamically would also be great. ( This is for a distributed network model, where each node/server runs several user-specified/user-built web-services. Communication between the nodes are done using xml-rpc. The development is moving along nicely using Medusa as foundation. thanks to everyone involved in the project. If I just could get dynamic loading/reload of modules/handlers to work it would really be a cool killer-app written in python. In my own humble opionon. ;-) ) Thomas From jeremy@z... Fri Aug 10 15:35:05 2001 From: jeremy@z... (jeremy@z...) Date: Fri, 10 Aug 2001 10:35:05 -0400 (EDT) Subject: update to asyncore in Python library Message-ID: <200108101435.KAA02722@m...> I just made a few small changes to the asyncore module in the Python source tree, which I thought I'd mention here. It's hard to tell if this is the only drift between the two copies of asyncore, because we've "normalized whitespace" in our copy and diff isn't too helpful anymore. The key change is to remove the definintions of EWOULDBLOCK and company for os.name == "nt". On Windows, the errno module gets these definitions from winsock.h. So errno should be correct across platforms. Is there any Windows variant where this wouldn't be true? I also wrapped some long lines. Jeremy From jeremy@z... Fri Aug 10 15:46:32 2001 From: jeremy@z... (Jeremy Hylton) Date: Fri, 10 Aug 2001 10:46:32 -0400 (EDT) Subject: uncaught EAGAIN in recv() Message-ID: <15219.62408.345661.185367@w...> We have used asyncore to implement ZEO, a client-server subsystem for the ZODB object database. I think the details of the application are unimportant, but you can find more info at http://www.zope.org/Wikis/ZODB/ One of our users has a problem with an uncaught socket error on recv(). Logging output here: Logging [ZEO]: uncaptured python exception, closing channel (socket.error:(11, 'Resource temporarily unavailable') [/usr/local/stow/python21_debug/lib/python2.1/asyncore.py|poll|95] [/usr/local/stow/python21_debug/lib/python2.1/asyncore.py|handle_read_event|383] [/home/jheintz/lib/python/ZEO/smac.py|handle_read|119] [/usr/local/stow/python21_debug/lib/python2.1/asyncore.py|recv|349]) This is certainly a standard error that recv() can raise, although I am surprised to see it after poll() says that the socket is ready to read. The Linux man page for recv() says: EAGAIN The socket is marked non-blocking and the receive operation would block, or a receive timeout had been set and the timeout expired before data was received. It seems to me that asyncore should be prepared to catch this error on recv(), but it does not. Is this any oversight? Would it make sense to fix in the Python source tree? Jeremy From rushing@n... Sat Aug 11 04:04:57 2001 From: rushing@n... (Sam Rushing) Date: Fri, 10 Aug 2001 20:04:57 -0700 Subject: [medusa] Adding/removing handlers dynamically References: <3BC7333BE30CD511AEB700B0D0AB2F5803D262@L...> Message-ID: <3B74A0D9.B17BAFDC@n...> Thomas Weholt wrote: > I need to add/remove handlers dynamically. I got a modular web-server that > searches a given folder for new handlers, when it finds one it would be cool > to have it install it without restarting the server, all automatic. If I > could build a server that search for new handlers at a given interval, > installs all new, removes all handlers that have been removed, it would open > for some really cool features. > > Any hints on how to implement a way of loading handlers dynamically would > also be great. The top-level handlers list is just that, a list. The position in the list controls the priority of a match. You can remove handlers by calling 'handlers.remove()'... You could make a new kind of top-level handler that implemented your behavior - so for example, your 'match' method would search its own list/dict/whatever for matches under that portion of the tree. Hmm... a dictionary would probably work pretty well. Then your top-level handler could be installed in a 'subdirectory', or perhaps as a default/fallback handler that you would install at the end of the list. -Sam From max@a... Mon Aug 13 23:50:25 2001 From: max@a... (max@a...) Date: Mon, 13 Aug 2001 22:50:25 -0000 Subject: SMTP delivery service for Medusa? Message-ID: <9l9ljh+qmu5@e...> In the Web-based game that I'm working on (Interstelen, http://www.interstelen.com/), which is written in Python and Medusa, I'm going to have the need to deliver email to players for such things as account registration, notifications, and so on. For the time being I'm just using smtplib which works, of course, but isn't select-friendly and so will block the whole server if there is some glitch in the communication stream with the SMTP server. Is there an SMTP delivery module available which uses Medusa (or just asyncore/asynchat of course) to do its delivery and properly defers things in nice friendly ways? I can obviously write my own, but would prefer not to if there's no need. Note that in my case I only need to do SMTP _delivery_ (as a client), I don't have to act as an SMTP server. Thanks. From jim@z... Tue Aug 14 13:44:15 2001 From: jim@z... (Jim Fulton) Date: Tue, 14 Aug 2001 08:44:15 -0400 Subject: [medusa] SMTP delivery service for Medusa? References: <9l9ljh+qmu5@e...> Message-ID: <3B791D1F.A954D834@z...> max@a... wrote: > > In the Web-based game that I'm working on (Interstelen, > http://www.interstelen.com/), which is written in Python and Medusa, > I'm going to have the need to deliver email to players for such things > as account registration, notifications, and so on. For the time being > I'm just using smtplib which works, of course, but isn't > select-friendly and so will block the whole server if there is some > glitch in the communication stream with the SMTP server. > > Is there an SMTP delivery module available which uses Medusa (or just > asyncore/asynchat of course) to do its delivery and properly defers > things in nice friendly ways? I can obviously write my own, but would > prefer not to if there's no need. I would like this too. > Note that in my case I only need to do SMTP _delivery_ (as a client), > I don't have to act as an SMTP server. Note that Python 2.1 includes an asyncore-based smtp server in the standard library. Jim -- Jim Fulton mailto:jim@d... Python Powered! Technical Director (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From spex66@y... Tue Aug 21 17:00:29 2001 From: spex66@y... (spex66@y...) Date: Tue, 21 Aug 2001 16:00:29 -0000 Subject: Blindshot: C-Python-extension thread problems under bobo_handler 1.03 Message-ID: <9lu0it+pduh@e...> Hi, strange things happend and exceeded my knowledge... so a short description: (1) Python module with import a c-python-extension (made with SWIG, target is database called Gigabase) (2) Test module "mbase" for publishing has objects w/ methods which throws HTML in return (3) Testing this module "mbase" in python interpreter works fine... database-accesses are correct (4) start_my_medusa with folowing snippet: # ============================================================== # BOBO Handler # ============================================================== import bobo_handler import mbase bfh = bobo_handler.bobo_handler(mbase, debug=1) hs.install_handler(bfh) (5) same access at database went now down into a core-dump. And the only hint that I have is that a start of a transaction with new thread went wrong sorry for this rough overview, but as mentioned as a blindshot has anyone an idea, if this problem is a collision from i.e. asyncore and a C-Extension??? thanks for any tip Peter (=PA=) And I've thought the "bloody" parts are over... fixing regex to re, till I got my POST variables in my methods... gnaaag I'm using Python 2.1, W2k, SWIG 1.3.6 with MS-VC6 with Multithreaded Debugging -MTd option From max@a... Sat Aug 25 20:14:49 2001 From: max@a... (max@a...) Date: Sat, 25 Aug 2001 19:14:49 -0000 Subject: SMTP delivery service for Medusa? In-Reply-To: <3B791D1F.A954D834@z...> Message-ID: <9m8tf9+lfrf@e...> --- In medusa@y..., Jim Fulton wrote: > Note that Python 2.1 includes an asyncore-based smtp server in the > standard > library. Oh really? I didn't realize that. Interesting. I'm looking at the library reference for 2.1.1. All that jumps out at me is smtplib, which is the legacy SMTP client (and which I presume still does not use asyncore). What's the one you're referring to? Thanks. From jims@s... Sun Jan 7 09:00:08 2001 From: jims@s... (jims@s...) Date: Sun, 07 Jan 2001 09:00:08 -0000 Subject: Newbie quesiton: how to pass parameters to a .mpy script as uri queries? Message-ID: <939b6o+44jh@e...> Hi all I'm a bit of a newbie, and I'm wondering how to go about writing a Python script (I'm currently using script_handler, and I don;t know if that's the proper handler to use) that accepts as input parameters passed in on the URI of a GET request. For example: http://loopy.metrics.com/gimme.mpy?category=42&item=white would invoke the gimme.mpy script and pass the argument "category" with a value of "42" and the argument "item" with the value "white". I tried writing the .mpy using the Python cgi module (the usual has_key stuff, see the cgi module docs on python.org), and that does not seem to work. When I tinker with script_handler.py I see that the query is being split off from the path by split_path, but nothing is being done with the query. It seems to me that there should be a simple way to do this? thanks jim From weissblitz@y... Fri Jan 12 19:48:49 2001 From: weissblitz@y... (weissblitz@y...) Date: Fri, 12 Jan 2001 19:48:49 -0000 Subject: IPC between Medusa Modules? Message-ID: <93nn31+fgcp@e...> I'm developing a messaging/chat client/server application using Medusa for the server side. It is composed of HTTP, FTP, messaging, chat (IRC like), and a user manager to manage user profile information. I can have any number of these components running, some on different machines accross the network. These components need to connect to the user manager component for user authentication and profile information. What would be the most efficient way of IPC between the modules? Named pipes? Sockets? SOAP? RPC? Also, whare can I get additional information on how to use the monitor server. I can connect to it and get the Python prompt but i would like to know what things I can do with this. Thanks in advance, Carlos :) PS: Love Medusa! Thanks to all who put their time and effort into developing and improving Medusa. Keep up the good work. From rprice@u... Fri Jan 12 14:03:26 2001 From: rprice@u... (ray) Date: Fri, 12 Jan 2001 06:03:26 -0800 Subject: [medusa] IPC between Medusa Modules? References: <93nn31+fgcp@e...> Message-ID: <3A5F0EAE.7E4BC678@u...> I've created a similar system using the RPC module distributed by Sam / egroups. The only problem, which is a real problem, is that once a server goes away, the client needs to be rebooted. I just can't figure out how to get the client to reconnect. Fortunately this only happens once every couple of weeks. You also need to remember that if one reboots a server, all clients need to be rebooted. Any suggestions on how to get the client to reconnect? It's running on Redhat Linux. Ray weissblitz@y... wrote: > I'm developing a messaging/chat client/server application using > Medusa for the server side. It is composed of HTTP, FTP, messaging, > chat (IRC like), and a user manager to manage user profile > information. I can have any number of these components running, some > on different machines accross the network. These components > need to connect to the user manager component for user authentication > and profile information. What would be the most efficient way of IPC > between the modules? Named pipes? Sockets? SOAP? RPC? > > Also, whare can I get additional information on how to use the > monitor server. I can connect to it and get the Python prompt but i > would like to know what things I can do with this. > > Thanks in advance, > Carlos :) > > PS: Love Medusa! Thanks to all who put their time and effort into > developing and improving Medusa. Keep up the good work. From lucifer@e... Sat Jan 13 02:12:28 2001 From: lucifer@e... (Nicolas Echaniz) Date: Fri, 12 Jan 2001 23:12:28 -0300 Subject: =?ISO-8859-1?Q?Re:_[medusa]_IPC_between_Medusa_Modules=3F?= In-Reply-To: <93nn31+fgcp@e...> Message-ID: <200101130212.XAA29959@w...> Hi Carlos, I'm interested in your proyect. I'm building something quite similar. What are you planning to use on the client side? java ? Is there any preview release of your work? Maybe we could join our efforts some time. What are your plans for this application? PS: where are you from? On Fri, 12 Jan 2001 19:48:49 -0000, you wrote: > I'm developing a messaging/chat client/server application using > Medusa for the server side. It is composed of HTTP, FTP, messaging, > chat (IRC like), and a user manager to manage user profile > information. I can have any number of these components running, some > on different machines accross the network. These components > need to connect to the user manager component for user authentication > and profile information. What would be the most efficient way of IPC > between the modules? Named pipes? Sockets? SOAP? RPC? > > Also, whare can I get additional information on how to use the > monitor server. I can connect to it and get the Python prompt but i > would like to know what things I can do with this. > > Thanks in advance, > Carlos :) > > PS: Love Medusa! Thanks to all who put their time and effort into > developing and improving Medusa. Keep up the good work. _______________________________________________________________ Nicolás Echániz | RAKAR Producciones | Buenos Aires | Argentina From nhytro@c... Mon Jan 15 08:20:26 2001 From: nhytro@c... (nhytro@c...) Date: Mon, 15 Jan 2001 08:20:26 -0000 Subject: Medusa Installation and Documentation Message-ID: <93ubsa+c595@e...> Hi there! is there some kind person out there that can help me with=20 the installation and usage of MEDUSA on WIN32 machines? I could=B4nt=20 make anything out of the docs. Thanks in advance Sharriff From akuchlin@c... Thu Jan 18 21:43:27 2001 From: akuchlin@c... (akuchlin@c...) Date: Thu, 18 Jan 2001 21:43:27 -0000 Subject: Patch to use poll() support in Python 2.0 In-Reply-To: <200012210340.WAA00993@2...> Message-ID: <947o1v+gva0@e...> --- In medusa@egroups.com, "A.M. Kuchling" wrote: > This patch adds the ability to use the poll() support that was added > to the select module in Python 2.0. Only lightly tested ... please > proofread this patch carefully. I posted that patch Dec. 22. With Python 2.1alpha1 rumbling toward us, I thought I'd check if it was accepted, and if a new Medusa release is going to be made fairly soon. Sam, any comments? --amk From kens@s... Tue Jan 23 22:07:43 2001 From: kens@s... (Ken Seehof) Date: Tue, 23 Jan 2001 22:07:43 -0000 Subject: Simple subclassing of http-server In-Reply-To: <8t3ouq+m9so@e...> Message-ID: <94kvbf+ks4j@e...> I need the same thing. Have you found anything out yet? I seems to be one of those things where the simplest and most typical usage doesn't get documented. I guess I should complain since it is free, but I am frustrated. Is there a book out there that is recommended? --- In medusa@egroups.com, thomas@c... wrote: > I want to subclass medusas http-server-class so that I can write my > own request handler. The server is going to serve xml. > > I've looked at the docs/tutorial but I cannot figure out where to put > my own handler, so that I can pick the requested url apart, use the > submitted data to find what data to send back to the client. > > The simpleHTTPServer in the standard python distro has a simpler or > at least more understandable approach, so if anybody could show me > how to get something similar to that to work with I'd be glad. > > The only thing I need is to have a place to insert my method and let > it return data based on the url the user typed in the browser. If I > can sublclass something and do it in my own subclass that would be > the best solution. It would be nice to leave the medusa-code alone. > > Thanks From lucifer@e... Wed Jan 24 00:42:36 2001 From: lucifer@e... (Nicolas Echaniz) Date: Tue, 23 Jan 2001 21:42:36 -0300 Subject: [medusa] Re: Simple subclassing of http-server In-Reply-To: <94kvbf+ks4j@e...> Message-ID: <200101240042.VAA08248@w...> Hi guys, I'm no Medusa expert. But I think that if what you need is to add your own handler to medusa, you need to write a new handler and then install it using start_medusa.py. The http_server.py will then use your handler for the type of requests it should handle. See this code (from the http_server): it starts on line 470 (but my base code is a little bit mangled :) for h in self.server.handlers: if h.match (r): try: self.current_request = r h.handle_request (r) I wrote a modified version of the script_handler, so I modified the start_medusa.py as follows: scr = ec_script_handler.ec_script_handler () hs.install_handler (scr) this installs my handler (ec_script_handler), so next time I start medusa, it will handle .py requests with my ec_script_handler. Hope it helps. PS: if you don't know how to write your own handlers, just look at the code from the other handlers ;) On Tue, 23 Jan 2001 22:07:43 -0000, you wrote: > I need the same thing. Have you found anything out yet? > > I seems to be one of those things where the simplest and most typical > usage doesn't get documented. I guess I should complain since it is > free, but I am frustrated. Is there a book out there that is > recommended? > > --- In medusa@egroups.com, thomas@c... wrote: > > I want to subclass medusas http-server-class so that I can write my > > own request handler. The server is going to serve xml. > > > > I've looked at the docs/tutorial but I cannot figure out where to > put > > my own handler, so that I can pick the requested url apart, use the > > submitted data to find what data to send back to the client. > > > > The simpleHTTPServer in the standard python distro has a simpler or > > at least more understandable approach, so if anybody could show me > > how to get something similar to that to work with I'd be glad. > > > > The only thing I need is to have a place to insert my method and > let > > it return data based on the url the user typed in the browser. If I > > can sublclass something and do it in my own subclass that would be > > the best solution. It would be nice to leave the medusa-code alone. > > > > Thanks _______________________________________________________________ Nicolás Echániz | RAKAR Producciones | Buenos Aires | Argentina From greg@e... Fri Jan 26 10:19:26 2001 From: greg@e... (Gregory P. Smith) Date: Fri, 26 Jan 2001 02:19:26 -0800 Subject: detecting connect() failures on windows Message-ID: <20010126021926.C3554@z...> Has anybody had any success in getting asyncore under python on windows to detect connect() failures? On real OSes (linux and bsd) I get nice easy to understand behavior, the attempt to connect to a host on a port its not listening on produces this output: (code below) ...... creating foo to port 81 log: adding channel asyncore loop starting: handle_connect Exception during self.send("") in connect: (32, 'Broken pipe') handle_read Exception during self.recv in read: (111, 'Connection refused') handle_write handle_read handle_close log: closing channel 4: data read: exited asyncore loop ...... On windows, it just hangs for a long time in select. (win2000 is my main test case but it behaves just as badly on win98) thanks, -Greg Here's my simple test program: import asyncore import socket class foo(asyncore.dispatcher): writ = 1 buf = 'GET /non.existant.file HTTP/1.0\r\n\r\n' def handle_write(self): print 'handle_write' if not self.writ: return self.send(self.buf) self.writ = 0 def writable(self): return self.writ def readable(self): return 1 def handle_read(self): print 'handle_read' try: data = self.recv(4096) except socket.error, e: print 'Exception during self.recv in read:', e self.writ = 0 # read failed? we're closed now return print 'data read:' print data def handle_close(self): print 'handle_close' self.close() #raise asyncore.ExitNow, 'handle_close called, exiting' def handle_connect(self): print 'handle_connect' try: data = self.send("") except socket.error, e: print 'Exception during self.send("") in connect:', e return def __init__(self, host, port): asyncore.dispatcher.__init__(self) self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.connect((host, port)) print 'creating foo to port 80' f1 = foo('www.mad-scientist.com', 80) print 'asyncore loop starting:' asyncore.loop() print 'exited asyncore loop' print 'creating foo to port 81' f2 = foo('www.mad-scientist.com', 81) print 'asyncore loop starting:' asyncore.loop() print 'exited asyncore loop' From abo@m... Sat Jan 27 10:10:22 2001 From: abo@m... (Donovan Baarda) Date: Sat, 27 Jan 2001 21:10:22 +1100 Subject: asynchat ready() method patch. In-Reply-To: <3.0.6.32.20001209180505.007d5d60@p...>; from support@i... on Sat, Dec 09, 2000 at 06:05:05PM -0500 References: <3.0.6.32.20001209180505.007d5d60@p...> Message-ID: <20010127211021.D13503@m...> ---------------------- multipart/mixed attachment G'day, I've recieved one note of interest in my patch for medusa to allow producers to block using a ready() method. The gist was "this is a good idea, but it doesn't work". At the time I was focusing on the ftp server part of medusa, so I hadn't checked the http side much at all. I also noticed that asyncore.py and asynchat.py were distributed without producers.py in python itself. My original changes made asynchat.py dependant on producers.py, which was probably a bad idea. So I have gone right over my changes and produced a new, more thoroughly tested patch that tries to minimize the side affects. I have tested the changes as they are incrementaly applied to ensure that my changes don't break old code that doesn't use the ready() feature. This patch contains all changes made to all files, but you only need to accept the changes to some files to use ready() The main change is to asynchat.py, where support for the ready() method has been added. Also added are a variety of producer base classes that allow new producers with ready() support to be easily derived (and removed the asynchat.py dependancy on producers.py that my early patch introduced). The main async_chat class is actualy significantly simpler for the change, mostly due to replacing the fifo class with a fifo_producer class. The find_prefix_at_end procedure for finding terminators has also been simplified and speeded up. Overall, asynchat.py is only 23 lines bigger dispite all the additional functionality. The good news is, the changes to asynchat.py are fully backwards compatible and medusa runs fine with all other files unchanged, _except_ that filesys.py and status_handler.py have conflicting ready() methods in producers. These ready() methods are unused in the original medusa, but cause the new asynchat.py to think the producers never complete. The patch to these two files simply removes these ready() methods so medusa can run with the new asynchat. The next lot of serious changes are to producers.py and http_server.py. These changes are not needed to run medusa as it stands, but are needed if you start using blocking producers. The changes primarily make the producers inherited from the base producer classes defined in asynchat.py. This ensures that producers that wrap other producers will block correctly. The inheritance of features makes producers.py 62 lines smaller than it was. All the changes to producers.py are backwards compatible _except_ the compound_producer.__init__() method now takes a python list of producers not a fifo. The changes to http_server.py take this into account. The final trivial change is a commented out alternative to use the new async_producer instead of file_producer in ftp_server.py. This allows medusa filesystems to return file types that could block without blocking the whole server. I hope someone gets a chance to try these out, and I would appreciate any form of feedback. I'm tempted to submit my patch to asynchat.py direct to the Python people for inclusion in Python 2.1, but would like to hear more feedback before I do. BTW, ignore the patch to "medusa.prj"... that is a side affect of using prcs to revision control my changes. This patch is to the 20000601 version of medusa. -- ---------------------------------------------------------------------- ABO: finger abo@m... for more info, including pgp key ---------------------------------------------------------------------- ---------------------- multipart/mixed attachment A non-text attachment was scrubbed... Name: medusa-ready_0.3.patch.bz2 Type: application/octet-stream Size: 13627 bytes Desc: not available Url : http://mail.python.org/pipermail-21/medusa-dev/attachments/ae91f9ca/medusa-ready_0.3.patch.bz2 ---------------------- multipart/mixed attachment-- From abo@m... Sun Jan 28 02:51:53 2001 From: abo@m... (Donovan Baarda) Date: Sun, 28 Jan 2001 13:51:53 +1100 Subject: asynchat ready() method patch. In-Reply-To: <20010127174237.D12344@k...>; from julio@p... on Sat, Jan 27, 2001 at 05:42:37PM -0200 References: <20010127174237.D12344@k...> Message-ID: <20010128135153.A19001@m...> On Sat, Jan 27, 2001 at 05:42:37PM -0200, Julio Maia wrote: > Hi, > > I've modified that test script I've sent to you in order to try your new > ready() patch. Now it seems to be calling ready() and more() properly, > however it takes a long time between calls of ready() (about 30 seconds). > Also, it seems the returned data from more() is not being processed > correctly in the handler because of ready(). I never recieved your test scripts... I think the attachment went missing. I'd be interested to see them though, so send me them again. The 30 second delays are caused by the way asyncore.py works. The ready() methods are called each iteration of the select() loop. The select loop completes an iteration everytime an async event happens, or the select() call times out. The default timeout in asyncore.py is 30 seconds... hence the 30 second delay. One way to prove this is to establish another connection to the medusa server before the 30 sec's expires, and this should kick the select loop to re-calculate the ready() methods. A server that is being hammered will not notice this problem because all the async events will be kicking the event loop all the time. However, to get snapier responces from an under-loaded server using ready(), set the timeout in loop() to something lower, like 1 sec. I dunno about the data not being processed correctly though... I'd have to look at your code to figure it out. I suspect that your ready() methods are not quite correct... ready() should return true and more() should return '' when the producer is finished. If ready() does not return true when the the producer is finished, asynchat will think that it is unfinished and blocked forever, so asynchat will never complete. > My script runs a http server on port 8000 by default. Any hits on it will > (or should) return a pair of timestamps. Try to rename ready() to see how it > should work. I thought of renaming ready() to something else when I found the vestigial ready() methods in filesys.py and status_handler.py. However, the code was pretty complete by the time I found them, and the reality is there will always be a potential problem with name clashes no-matter what name I choose. -- ---------------------------------------------------------------------- ABO: finger abo@m... for more info, including pgp key ---------------------------------------------------------------------- From abo@m... Wed Jan 31 12:18:04 2001 From: abo@m... (Donovan Baarda) Date: Wed, 31 Jan 2001 23:18:04 +1100 Subject: [Zope-dev] asynchat producer ready() method patch. In-Reply-To: <14965.46576.76669.281517@l...>; from dieter@h... on Mon, Jan 29, 2001 at 07:26:56PM +0100 References: <220829293@t...> <20010129185326.A31092@m...> <14965.46576.76669.281517@l...> Message-ID: <20010131231804.C1484@m...> On Mon, Jan 29, 2001 at 07:26:56PM +0100, Dieter Maurer wrote: > Donovan Baarda writes: > > On Sun, Jan 28, 2001 at 09:31:07PM +0100, Dieter Maurer wrote: > > > Donovan Baarda writes: > > > > I remember reading ages ago on a wishlist for ZServer that someone was > > > > thinking of adding blocking support to producers. This patch adds support > > > > for just that in a non-intrusive way. I haven't tried applying this patch to > > > > ZServer, but it _should_ work. > > > What does "blocking support dor producers" mean? > > > > Disclaimer: you must be moderately familiar with ZServer/medusa/asynchat for > > the following to make sense; > > > > Asynchat used by ZServer and medusa use producers as a way to "push" data > > through an async interface. The current version of asynchat requires that > > producers be able to produce data immediately on calling their more() > > method, untill they are finished, when they return ''. > > > > My patch adds support for producers being able to "block" by letting > > asynchat know if they are ready via a ready() method. This allows you to use > > the simple producer model to push data that may not be immediately > > available. An example would be a http-fetch producer that fetches data from > > upstream for a proxy. > > > > This patch is only useful for people creating or modifying things like > > medusa or ZServer. > Thank you for this good explanation. > > By now, I think, it would not be usefull for ZServer, at least > not for standard operations. > > ZServer is a multi-threaded server, where different requests > are serviced by different threads. > The thread may block and does simply not return from "more()". > > On first thought, you patch might free some blocked threads > to do productive work. However, Zope's transaction system > is linked to thread management. If you patch would be used > for that, Zope's transaction subsystem would need to be > rewritten. Hmmm, it's been a while since I looked at ZServer, I might have to take another look. I thought that it pre-spawned a number of threads that each ran their own asyncore.loop event handler. I thought this was a neat way to get the benefits of async event handlers and distributing threads on multiple processors. Unless it spawns new threads when it needs them, it would still be possible for ZServer to get blocked if all the threads were blocked. However, my patch is only useful if you _have_ producers that can block. It is usually possible to re-structure your system so that no producers block (typicaly by using a dispatcher instead of a producer). The ready() block feature is really a shortcut way of modifying an existing producer-based design when you realise that the producer can block; ie an alternative way of implementing it that may or may not be better. In my case I wanted to create a cached-ftp-backend filesystem that medusa could serve via ftp or http. Unfortunately the existing ftp and http server implementations used producers to pull data from the abstract filesystem. Since my "filesystem" could block pretty severly, I either needed to completely re-write the http and ftp servers to use dispatchers instead of producers, or implement blocking for producers (or use a seperate thread for each request, but I thought that defeated the purpose of using an async event loop). I really must look at ZServer to figure out how it handles things... I suspect that my patch by itself would not _hurt_ ZServer, but perhaps if you started using the ready() feature it provides it might cause problems. I'll have a look at ZServer again soon and give you an evaluation of how my patch would interact with it. Who knows, someone might find a need for blocking producers as ZServer evolves, and if people know the patch exists, they might use it instead of re-structuring to get around it :-) -- ---------------------------------------------------------------------- ABO: finger abo@m... for more info, including pgp key ---------------------------------------------------------------------- From kthangavelu@e... Wed Jan 31 19:21:42 2001 From: kthangavelu@e... (ender) Date: Wed, 31 Jan 2001 11:21:42 -0800 Subject: [medusa] Re: [Zope-dev] asynchat producer ready() method patch. In-Reply-To: <20010131231804.C1484@m...> References: <220829293@t...> <14965.46576.76669.281517@l...> <20010131231804.C1484@m...> Message-ID: <01013111214200.00885@l...> On Wednesday 31 January 2001 04:18, Donovan Baarda wrote: > On Mon, Jan 29, 2001 at 07:26:56PM +0100, Dieter Maurer wrote: > > Donovan Baarda writes: > > > On Sun, Jan 28, 2001 at 09:31:07PM +0100, Dieter Maurer wrote: > > > > Donovan Baarda writes: > > > > > I remember reading ages ago on a wishlist for ZServer that > > > > > someone was thinking of adding blocking support to producers. > > > > > This patch adds support for just that in a non-intrusive way. I > > > > > haven't tried applying this patch to ZServer, but it _should_ > > > > > work. > > > > > > > > What does "blocking support dor producers" mean? > > > > > > Disclaimer: you must be moderately familiar with > > > ZServer/medusa/asynchat for the following to make sense; > > > > > > Asynchat used by ZServer and medusa use producers as a way to "push" > > > data through an async interface. The current version of asynchat > > > requires that producers be able to produce data immediately on calling > > > their more() method, untill they are finished, when they return ''. > > > > > > My patch adds support for producers being able to "block" by letting > > > asynchat know if they are ready via a ready() method. This allows you > > > to use the simple producer model to push data that may not be > > > immediately available. An example would be a http-fetch producer that > > > fetches data from upstream for a proxy. > > > > > > This patch is only useful for people creating or modifying things like > > > medusa or ZServer. > > > > Thank you for this good explanation. > > > > By now, I think, it would not be usefull for ZServer, at least > > not for standard operations. > > > > ZServer is a multi-threaded server, where different requests > > are serviced by different threads. > > The thread may block and does simply not return from "more()". > > > > On first thought, you patch might free some blocked threads > > to do productive work. However, Zope's transaction system > > is linked to thread management. If you patch would be used > > for that, Zope's transaction subsystem would need to be > > rewritten. > > Hmmm, it's been a while since I looked at ZServer, I might have to take > another look. I thought that it pre-spawned a number of threads that each > ran their own asyncore.loop event handler. I thought this was a neat way to > get the benefits of async event handlers and distributing threads on > multiple processors. Unless it spawns new threads when it needs them, it > would still be possible for ZServer to get blocked if all the threads were > blocked. disclaimer: insert here. zserver has a main thread which binds and listens, passing requests to a pool of zpublisher threads. the zpublisher threads communicate with the client via the main threads dispatcher, ie output is funneled back into the main thread's dispatcher so as not to tie up threads talking to distant clients. reference: Zserver/medusa/select_trigger.py python has a global interpreter lock, so multiple cpus have little utility. > However, my patch is only useful if you _have_ producers that can block. It > is usually possible to re-structure your system so that no producers block > (typicaly by using a dispatcher instead of a producer). The ready() block > feature is really a shortcut way of modifying an existing producer-based > design when you realise that the producer can block; ie an alternative way > of implementing it that may or may not be better. i've taken a quick look at the patch, i find it interesting in that it provides i think a way to implement a Proactor type server vs the standard medusa Reactor style server. reference: Patterns of System Architecture Vol II. Douglas Schmidt et al reference: http://www.cs.wustl.edu/~schmidt/PDF/proactor.pdf and a webserver that makes use of this pattern http://www.cs.wustl.edu/~jxh/research/ kapil > In my case I wanted to create a cached-ftp-backend filesystem that medusa > could serve via ftp or http. Unfortunately the existing ftp and http server > implementations used producers to pull data from the abstract filesystem. > Since my "filesystem" could block pretty severly, I either needed to > completely re-write the http and ftp servers to use dispatchers instead of > producers, or implement blocking for producers (or use a seperate thread > for each request, but I thought that defeated the purpose of using an async > event loop). > I really must look at ZServer to figure out how it handles things... I > suspect that my patch by itself would not _hurt_ ZServer, but perhaps if > you started using the ready() feature it provides it might cause problems. > I'll have a look at ZServer again soon and give you an evaluation of how my > patch would interact with it. > Who knows, someone might find a need for blocking producers as ZServer > evolves, and if people know the patch exists, they might use it instead of > re-structuring to get around it :-) From jbenninghoff@y... Mon Feb 5 05:40:17 2001 From: jbenninghoff@y... (John Benninghoff) Date: Mon, 05 Feb 2001 05:40:17 -0000 Subject: HTTP load generator Message-ID: <95lec1+ak31@e...> I'm thinking of writing an HTTP load generator using asynchat.py. I want to support a typical approach using scenarios much like LoadRunner and SilkPerformer do it. Has someone already done something like this? From rushing@n... Mon Feb 5 17:12:51 2001 From: rushing@n... (Sam Rushing) Date: Mon, 05 Feb 2001 09:12:51 -0800 Subject: [medusa] HTTP load generator References: <95lec1+ak31@e...> Message-ID: <3A7EDF13.C5BBCD34@n...> John Benninghoff wrote: > I'm thinking of writing an HTTP load generator using asynchat.py. I > want to support a typical approach using scenarios much like > LoadRunner and SilkPerformer do it. Has someone already done > something like this? I've written a few small benchmarking tools, but haven't worked on any separate load-generator projects. Usually it's medusa or asyncore that I'm benchmarking, and I always feel a little uncomfortable benchmarking something with itself. -Sam From petri_savolainen@y... Wed Feb 21 18:02:09 2001 From: petri_savolainen@y... (Petri Savolainen) Date: Wed, 21 Feb 2001 18:02:09 -0000 Subject: HTTP load generator In-Reply-To: <95lec1+ak31@e...> Message-ID: <970vr1+6kos@e...> There is an open-source tool set called OpenSTA (.org) which has a nice GUI, records test scripts as you browse and has a whole testing language. Regards, Petri --- In medusa@y..., "John Benninghoff" wrote: > I'm thinking of writing an HTTP load generator using asynchat.py. I > want to support a typical approach using scenarios much like > LoadRunner and SilkPerformer do it. Has someone already done > something like this? From jbenninghoff@y... Thu Feb 22 03:45:17 2001 From: jbenninghoff@y... (John Benninghoff) Date: Wed, 21 Feb 2001 19:45:17 -0800 (PST) Subject: [medusa] Re: HTTP load generator In-Reply-To: <970vr1+6kos@e...> Message-ID: <20010222034517.23687.qmail@w...> I checked it out. It only runs on NT and I could not find a way to run it on more than one machine despite the fact that it was built using CORBA! Besides, I prefer Python to the yet another specialized language. But thanks for the feedback anyway. --- Petri Savolainen wrote: > There is an open-source tool set called OpenSTA > (.org) which has a > nice GUI, records test scripts as you browse and has > a whole testing > language. > > Regards, > > Petri > > --- In medusa@y..., "John Benninghoff" > wrote: > > I'm thinking of writing an HTTP load generator > using asynchat.py. > I > > want to support a typical approach using scenarios > much like > > LoadRunner and SilkPerformer do it. Has someone > already done > > something like this? > > ===== John Benninghoff It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change. -- Charles Darwin __________________________________________________ Do You Yahoo!? Yahoo! Auctions - Buy the things you want at great prices! http://auctions.yahoo.com/ From reynol@p... Fri Feb 23 04:48:29 2001 From: reynol@p... (Samuel Reynolds) Date: Thu, 22 Feb 2001 21:48:29 -0700 Subject: Medusa on Macintosh [long] In-Reply-To: <3A82E049.40EE0A16@n...> References: Message-ID: ---------------------- multipart/mixed attachment Hi, all. Please go easy on a Python/Medusa newbie. ;-) I just picked up Python (Mac Python 2.0) a few weeks ago, and I'm working on getting Medusa up and running on Macintosh. (I may try to move "up" to Zope, but the Medusa docs say that it "should" work on Macintosh, whereas the Zope docs simply say *nix and Windows. A brief introduction: I'm a professional software developer (mainly OO C++). I've also done extensive scripting (including internet work) in Frontier on Mac and Windows for several years (until very recently, I was a Support Associate for Userland Software, publisher of Frontier). I just finished reading through the medusa@yahoogroups.com archives. In the process, I found I could hack os_filesystem and http_server to make them work, but couldn't get ftp_server to work with my changes. I kept getting bunches of extra colons (the Mac filesystem path separator) in the paths, so ftp_server couldn't find the files. I have exchanged email directly with Sam Rushing about this, but I'd like to open the discussion to the development community. Since os_filesystem is intended to function as a virtual-*nix filesystem, I backed up and took another look. I reverted to the original code (except for the date coercions), and studied filesys.py. It seemed to make an awful lot of calls to os.path (via self.path_module). That's why there were so many glitches when I tried to use it on Mac. It worked okay on Unix and Windows because "/" is recognized as a proper path separator on both of those systems. So I rewrote os_filesystem to always use posixpath (i.e., the "/" separator) except when it had to go to the actual files, and did the translation in only one place (the translate method). The FTP server suddenly worked perfectly, and the http server (actually, default_handler.py) required only a couple of small changes. Other advantages this provides (as long as all file references are processed via the os_filesystem): o External code (servers, etc) only uses virtual-posix paths (e.g., http server need do no translations). o Mapping to os-specific paths is kept internal to the os_filesystem. (Though, since the methods are all public, the translate() method can be used by client code if necessary.) o I duplicated the various path and file/directory methods in os_filesystem, so it can provide the complete file-interaction abstraction. o Except by going through the translate method to get the actual system-specific file path, the client code cannot pop out past the virtual-root directory (i.e., can't pop out past "/"). I've attached the modified filesys.py file, as well as a test program that exercises it. I think it will work on other systems as-is, but I'd appreciate if someone could try it out on Windows and Unix. Any takers? (By all means, read through the changed file first, so you understand exactly what it does.) If I get the chance, I intend to test it on Solaris and WinNT4, but I'd like to get alternate input, as well--I'm sure there's code built on top of Medusa that I'm not aware of, and that the changed os_filesystem might adversely impact. Note: In the test program (filesys_test.py), you may need to add a section for your OS to set the real_* paths as test reference values. ON A SECOND TOPIC: How can I get access to the CVS repository? I saw the commands in the archive, but I'm running the CVS client on a Mac, so it looks a bit different from the Unix command-line access. MacCVS Pro asks for server, path, login, and password. Perhaps I need to contact Sam Rushing directly? - Sam ---------------------- multipart/mixed attachment A non-text attachment was scrubbed... Name: filesys.py Type: application/mac-binhex40 Size: 18341 bytes Desc: not available Url : http://mail.python.org/pipermail-21/medusa-dev/attachments/f26084b8/filesys.py ---------------------- multipart/mixed attachment A non-text attachment was scrubbed... Name: filesys_test.py Type: application/mac-binhex40 Size: 9255 bytes Desc: not available Url : http://mail.python.org/pipermail-21/medusa-dev/attachments/72d724f0/filesys_test.py ---------------------- multipart/mixed attachment ________________________________________ Samuel Reynolds Spinward Stars: http://www.spinwardstars.com/ Reynolds Virtual Workshop: http://www.primenet.com/~reynol reynol@p... samuel_reynolds@c... ---------------------- multipart/mixed attachment-- From reynol@p... Fri Feb 23 15:14:04 2001 From: reynol@p... (Samuel Reynolds) Date: Fri, 23 Feb 2001 08:14:04 -0700 Subject: [medusa] Medusa on Macintosh [long] In-Reply-To: References: <3A82E049.40EE0A16@n...> Message-ID: >I reverted to the original code (except for the date >coercions) To clarify about the date coercions (and get them into the archive): On Unix and Windows, the date epoch is 1970. For dates from this epoch, it's safe to coerce times to ints. On the Mac, on the other hand, the epoch is 1907. As a result, an attempt to coerce to int results in a range overflow. Instead, coerce to long. In particular: o Change all instances of [ int(time.time()) ] to [ long(time.time())] ]. o Change all instances of [ int(time.mktime()) ] to [ long(time.mktime()) ]. In the medusa-src-20000601 tarball, this affects the files eventloop.py, http_date.py, http_server.py, resolver.py, and status_handler.py. Sam Rushing has suggested that there are other date issues, as well, that I haven't run into. - Sam ________________________________________ Samuel Reynolds Spinward Stars: http://www.spinwardstars.com/ Reynolds Virtual Workshop: http://www.primenet.com/~reynol reynol@p... samuel_reynolds@c... From weissblitz@y... Sun Mar 4 15:50:01 2001 From: weissblitz@y... (Weiss Blitz) Date: Sun, 04 Mar 2001 15:50:01 -0000 Subject: How to use the Medusa Monitor Server ??? Message-ID: <97to79+10t00@e...> Can anybody shed some light about how to use the Monitor Server to check and/or update a Medusa server with new code? I'm running Medusa HTTP,FTP and Monitor Servers on an W2k machine. I can login into the monitor server and get the regular Python interpreter prompt: python monitor_client_win32.py (my-ip) 8899 Enter Password: secret Python 2.0 (#8, Oct 19 2000, 11:30:05) [MSC 32 bit (Intel)] Copyright (c) 2000 ActiveState Tool Corp. Copyright (c) 2000 BeOpen.com. All Rights Reserved. Copyright (c) 1995-2000 Corporation for National Research Initiatives. All Rights Reserved. Copyright (c) 1991-1995 Stichting Mathematisch Centrum, Amsterdam. All Rights Reserved. Welcome to >>> dir() ['__builtins__'] >>> print dir() ['_', '__builtins__'] Seems I'm not getting much here besides a virgin Python interpreter. How can I see my Medusa namespace? Do I have to run Medusa differenly? For example, let say I made some changes to the FTP server and now I want to reload that code without stoping/starting the whole Medusa server. Thanks, Carlos :) BTW: Thanks to Sam and Guido and all others who made Medusa and Python possible! Python/Medusa rocks! :) From robin@j... Sun Mar 4 16:38:00 2001 From: robin@j... (Robin Becker) Date: Sun, 4 Mar 2001 16:38:00 +0000 Subject: [medusa] How to use the Medusa Monitor Server ??? In-Reply-To: <97to79+10t00@e...> References: <97to79+10t00@e...> Message-ID: In article <97to79+10t00@e...>, Weiss Blitz writes ... well everything hangs off the __main__ module. So you could try import __main__ dir(__main__) to retry FTP stuff you'd need to find anything that had a reference to the ftp_server and del it and then reload(ftp_server) and redfine all the references. Seems quite hard from the monitor. Perhaps it would be better to just restart medusa somehow. -- Robin Becker From rushing@n... Sun Mar 4 21:40:02 2001 From: rushing@n... (Sam Rushing) Date: Sun, 04 Mar 2001 13:40:02 -0800 Subject: [medusa] How to use the Medusa Monitor Server ??? References: <97to79+10t00@e...> Message-ID: <3AA2B632.937E0399@n...> Weiss Blitz wrote: > Can anybody shed some light about how to use the Monitor Server to > check and/or update a Medusa server with new code? > > Welcome to de6a54> > >>> dir() > ['__builtins__'] > >>> print dir() > ['_', '__builtins__'] > > Seems I'm not getting much here besides a virgin Python interpreter. > How can I see my Medusa namespace? > Do I have to run Medusa differenly? Try 'from __main__ import *' The non-secure version of the interpreter prints this as a hint in the prompt, it should probably just be done for you. If you access the '__main__' module directly you can change top-level values there [which is a little more dangerous if you don't want to take the server down] > For example, let say I made some changes to the FTP server and now I > want to reload that code without stoping/starting the whole Medusa > server. > Something like this should work: reload(ftp_server) __main__.ftp.close() # close the server socket new_ftp = ftp_server.ftp_server (...) # create and bind a new one __main__.status_objects.remove (__main__.ftp) __main__.ftp = new_ftp __main__.status_objects.append(new_ftp) Note that the 'old' ftp server is still pointing at the old code objects... same with any existing clients. When the last of those closes, all references to the old code will go away and you'll be fully 'upgraded'. -Sam From lucifer@e... Tue Mar 6 21:52:06 2001 From: lucifer@e... (Lucifer) Date: Tue, 6 Mar 2001 17:52:06 -0400 Subject: SSL on medusa In-Reply-To: Message-ID: <200103062152.RAA01446@L...> Is there any SSL support on medusa ? Thanks. Nicolás Echániz Rakar Producciones Argentina From rushing@n... Wed Mar 7 00:44:16 2001 From: rushing@n... (Sam Rushing) Date: Tue, 06 Mar 2001 16:44:16 -0800 Subject: [medusa] SSL on medusa References: <200103062152.RAA01446@L...> Message-ID: <3AA58460.CC4404A@n...> Lucifer wrote: > Is there any SSL support on medusa ? 1) On nightmare.com I think you'll find a link to a description of how to use stunnel (I think that's the name of it). 2) Python 2.0 supports SSL sockets directly, it's probably just a matter of figuring out how to support a different flavor of socket object. I remember building the SSL support a few months back, but couldn't figure out the certificate stuff very quickly, didn't go any further. -Sam From lucifer@e... Thu Mar 8 21:27:25 2001 From: lucifer@e... (Lucifer) Date: Thu, 8 Mar 2001 17:27:25 -0400 Subject: kill open port In-Reply-To: <200103062152.RAA01446@L...> Message-ID: <200103082127.RAA02205@L...> (Thanks for your answer on the SSL subject Sam.) I have the following problem. I've been writing an html_chat_server and it's working nice now (on Netscape and Explorer as well). I've built it on top of the chat_server. But for some reason, the port on which I normally run it has now been left open and I don't know how to close it. So I keep getting an 'Address already in use' error every time I try to run the server. How do I explicitly close a port that's been left open? and how did this happen ? It's been open for several days now and I don't know why because the chat server is not running. Any hints ? Nicolás Echániz RAKAR Producciones Argentina From kthangavelu@e... Fri Mar 9 03:24:29 2001 From: kthangavelu@e... (ender) Date: Thu, 8 Mar 2001 19:24:29 -0800 Subject: [medusa] kill open port In-Reply-To: <200103082127.RAA02205@L...> References: <200103082127.RAA02205@L...> Message-ID: <01030819242903.13837@l...> >>But for some reason, the port on which I normally run it has now been left >> open and I don't know how to close it. So I keep getting an 'Address >> already in use' error every time I try to run the server. >> >>How do I explicitly close a port that's been left open? and how did this >> happen ? >> >>It's been open for several days now and I don't know why because the chat >> server is not running. in your chat server after you create the socket (before you bind) you should do yoursocket.set_reuse_addr() else ths os won't let you reclaim, how long it won't depends on your os and your configuration. my linux box lets me do it after a couple of minutes, but its pretty depedent on your system. cheers kapil thangavelu From eugene.leitl@l... Fri Mar 9 16:24:18 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Fri, 9 Mar 2001 17:24:18 +0100 (MET) Subject: Medusa/Bobo nonblocking? Message-ID: I don't need the high performance of Medusa, but I need it to be nonblocking, as my cgi-bin queries can take awhile. From seconds to minutes. I would also rather like to be able embed Python into server-side HTML as well as cgi-bin which doesn't invoke an external instance of Python. The server doesn't need to be high-performance, but I'd like it to be relatively small. It would be nice if the solution was also resistant to memory leaks, but that's a little too much to ask for, given above contraints. Does Medusa/Bobo fit the ticket? It's for a commercial product, btrw. Or should I just take a good small http server written in C, and extend Python with it? Thanks for your thoughts. TIA, -- Eugene From rushing@n... Fri Mar 9 20:08:58 2001 From: rushing@n... (Sam Rushing) Date: Fri, 09 Mar 2001 12:08:58 -0800 Subject: [medusa] Medusa/Bobo nonblocking? References: Message-ID: <3AA9385A.FD57541C@n...> Eugene Leitl wrote: > I don't need the high performance of Medusa, but I need it to be > nonblocking, as my cgi-bin queries can take awhile. From seconds to > minutes. I would also rather like to be able embed Python into server-side > HTML as well as cgi-bin which doesn't invoke an external instance of > Python. The server doesn't need to be high-performance, but I'd like it to > be relatively small. It would be nice if the solution was also resistant > to memory leaks, but that's a little too much to ask for, given above > contraints. > > Does Medusa/Bobo fit the ticket? It's for a commercial product, btrw. > > Or should I just take a good small http server written in C, and extend > Python with it? Depends on the nature of you CGI scripts. If they are taking a long time because they're doing work that would cause a non-blocking process to block for long periods of time (say, long SQL queries), then you either won't get any advantage, or you'll have to rewrite your cgi to be event-driven. Another approach to this is to use an async http server like squid as a reverse proxy. (check out http://pasadena.wr.usgs.gov/stans/slashdot.html for an explanation of this setup) -Sam From eugene.leitl@l... Fri Mar 9 20:45:04 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Fri, 09 Mar 2001 21:45:04 +0100 Subject: [medusa] Medusa/Bobo nonblocking? References: <3AA9385A.FD57541C@n...> Message-ID: <3AA940D0.D8748B54@l...> Sam Rushing wrote: > Depends on the nature of you CGI scripts. If they are taking a long time > because they're doing work that would cause a non-blocking process to block > for long periods of time (say, long SQL queries), then you either won't get Yeah, they're doing a database search, and producing a tree of .png images plus a few user controls mapped to a
(actually, they'll be producing a piece of XML which is rendered by another (socket-decoupled) piece of code). Alternatively, this is going to be one big .png image with server-side clickable maps (would require a web server which understand clickable maps, the Tk tree widget and maintaining a list of clickable rectangles plus be heavy on bandwidth use for dialup users, though). > any advantage, or you'll have to rewrite your cgi to be event-driven. Care to outline the principle, or provide a pointer? No problem if you don't have the time nor the inclination, "event-driven" and "cgi" plugged into Google will eventually come up with something (so far, with surprisingly little, though). > Another approach to this is to use an async http server like squid as a Uh, I'd rather stay with something with a small footprint (a few pages of code) and Python or ANSI C. > reverse proxy. (check out http://pasadena.wr.usgs.gov/stans/slashdot.html for > an explanation of this setup) From rushing@n... Fri Mar 9 21:31:56 2001 From: rushing@n... (Sam Rushing) Date: Fri, 09 Mar 2001 13:31:56 -0800 Subject: [medusa] Medusa/Bobo nonblocking? References: <3AA9385A.FD57541C@n...> <3AA940D0.D8748B54@l...> Message-ID: <3AA94BCC.13E932C@n...> eugene.leitl@l... wrote: > Yeah, they're doing a database search, and producing a tree of .png images > plus a few user controls mapped to a
(actually, they'll be producing > a piece of XML which is rendered by another (socket-decoupled) piece of code). > Alternatively, this is going to be one big .png image with server-side > clickable maps (would require a web server which understand clickable maps, > the Tk tree widget and maintaining a list of clickable rectangles plus be > heavy on bandwidth use for dialup users, though). The big-files-through-small-pipes issue is what a reverse proxy is meant to address... if you have heavy-weight resources on the server that can nonetheless do their processing quickly, then the server dumps everything into the proxy quickly, which handles these tedious doling out of bytes to dialup users. This frees up the heavyweight resource (think apache process) to move on to the next job. > > > any advantage, or you'll have to rewrite your cgi to be event-driven. > Care to outline the principle, or provide a pointer? No problem if you > don't have the time nor the inclination, "event-driven" and "cgi" plugged > into Google will eventually come up with something (so far, with surprisingly > little, though). > If the back end of the system is accessed via another socket (is this what you mean by 'socket-decoupled'?) then you can implement the protocol that talks to it using an event-driven model. [i.e., state variables, callbacks, etc... look at any of the code that comes with medusa for examples] A common issue with web servers is talking to a database back-end. If you have access to docs for the wire protocol, then you can write an event-driven database client. Using it is tricky because you have that much more state to keep track of, but it can be done. There's a mysql client distributed with medusa, in misc/async_mysql.py -Sam From lucifer@e... Fri Mar 9 22:49:01 2001 From: lucifer@e... (Lucifer) Date: Fri, 9 Mar 2001 18:49:01 -0400 Subject: [medusa] kill open port In-Reply-To: <01030819242903.13837@l...> Message-ID: <200103092249.SAA02771@L...> the set_reuse_addr() was already there. Any other idea ? I´ve scanned the ports on the server and the port I used for the server is still open though there is no chat_server running. On Thu, 8 Mar 2001 19:24:29 -0800, ender wrote: > > >>But for some reason, the port on which I normally run it has now been left > >> open and I don't know how to close it. So I keep getting an 'Address > >> already in use' error every time I try to run the server. > >> > >>How do I explicitly close a port that's been left open? and how did this > >> happen ? > >> > >>It's been open for several days now and I don't know why because the chat > >> server is not running. > > in your chat server after you create the socket (before you bind) you should > do > yoursocket.set_reuse_addr() > > else ths os won't let you reclaim, how long it won't depends on your os and > your configuration. my linux box lets me do it after a couple of minutes, but > its pretty depedent on your system. > > cheers > > kapil thangavelu > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From rushing@n... Fri Mar 9 22:20:54 2001 From: rushing@n... (Sam Rushing) Date: Fri, 09 Mar 2001 14:20:54 -0800 Subject: [medusa] kill open port References: <200103092249.SAA02771@L...> Message-ID: <3AA95746.806A35E6@n...> Lucifer wrote: > the set_reuse_addr() was already there. > > Any other idea ? > > I´ve scanned the ports on the server and the port I used for the server is still open though there is no chat_server running. What happens if you telnet to that port? Some other process may be using it. Try using 'netstat -a'. Depending on your OS, you may be able to follow clues given there (say, inode number) to track down which process is holding the socket. I have a script for linux that may or may not make this easy, let me know if you'd like it... -Sam From greg@e... Fri Mar 9 23:16:30 2001 From: greg@e... (Gregory P. Smith) Date: Fri, 9 Mar 2001 15:16:30 -0800 Subject: [medusa] Medusa/Bobo nonblocking? In-Reply-To: <3AA94BCC.13E932C@n...>; from rushing@n... on Fri, Mar 09, 2001 at 01:31:56PM -0800 References: <3AA9385A.FD57541C@n...> <3AA940D0.D8748B54@l...> <3AA94BCC.13E932C@n...> Message-ID: <20010309151630.A9807@z...> > > > any advantage, or you'll have to rewrite your cgi to be event-driven. > > > Care to outline the principle, or provide a pointer? No problem if you > > don't have the time nor the inclination, "event-driven" and "cgi" plugged > > into Google will eventually come up with something (so far, with surprisingly > > little, though). > > > > If the back end of the system is accessed via another socket (is this what you > mean by 'socket-decoupled'?) then you can implement the protocol that talks to it > using an event-driven model. [i.e., state variables, callbacks, etc... look at any > of the code that comes with medusa for examples] > > A common issue with web servers is talking to a database back-end. If you have > access to docs for the wire protocol, then you can write an event-driven database > client. Using it is tricky because you have that much more state to keep track > of, but it can be done. There's a mysql client distributed with medusa, in > misc/async_mysql.py > > -Sam Depending upon the application the simpler-to-code approach (when no obvious non-blocking database interface already exists) of using an event driven server such as Medusa or TwistedPython to merely accept the requests for initial processing and hand them off to a thread/process pool for the actual blocking work to be done may be good enough. Greg (who wishes everything had a nonblocking interface and that python had continuations) From lucifer@e... Mon Mar 12 23:01:30 2001 From: lucifer@e... (Lucifer) Date: Mon, 12 Mar 2001 19:01:30 -0400 Subject: [medusa] kill open port In-Reply-To: <3AA95746.806A35E6@n...> Message-ID: <200103122301.TAA03179@L...> I'm using Linux Sam. Maybe your script would help. One more thing. I never explicitly kill the process when I restart the server. May that be the reason why the socket stays open even between reboots ? Thanks. On Fri, 09 Mar 2001 14:20:54 -0800, Sam Rushing wrote: > Lucifer wrote: > > > the set_reuse_addr() was already there. > > > > Any other idea ? > > > > I´ve scanned the ports on the server and the port I used for the server is still open though there is no chat_server running. > > What happens if you telnet to that port? Some other process may be using it. Try using 'netstat -a'. Depending on your OS, > you may be able to follow clues given there (say, inode number) to track down which process is holding the socket. I have a > script for linux that may or may not make this easy, let me know if you'd like it... > > -Sam > > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From lucifer@e... Wed Mar 14 07:28:59 2001 From: lucifer@e... (Lucifer) Date: Wed, 14 Mar 2001 03:28:59 -0400 Subject: [medusa] kill open port solved. In-Reply-To: <3AA95746.806A35E6@n...> Message-ID: <200103140728.DAA01079@L...> Hi everybody. The open port thing has been magically solved :) I asked the guys from cobaltracks to turn the server off and on again and it's gone. From lucifer@e... Wed Mar 14 07:38:50 2001 From: lucifer@e... (Lucifer) Date: Wed, 14 Mar 2001 03:38:50 -0400 Subject: server as backgroud process In-Reply-To: <3AA95746.806A35E6@n...> Message-ID: <200103140738.DAA01101@L...> Hi. I'm having this problem. When I run the start_medusa script I allways run it like this: python start_medusa.py & so that it runs as a background process, and it works fine. I normally run the start_medusa from a telnet session and then just exit and it keeps running fine. Now I have this web-chat server that I've built on top of the chat server. I run the chat server in the same way I run the medusa script and it works perfectly well. But when I exit the telnet session, it just gets killed after a while. Any hints? Thanks in advance for your time. Nicolás Echániz RAKAR Producciones From reynol@p... Wed Mar 14 15:37:35 2001 From: reynol@p... (Samuel Reynolds) Date: Wed, 14 Mar 2001 08:37:35 -0700 Subject: [medusa] server as backgroud process In-Reply-To: <200103140738.DAA01101@L...> References: <3AA95746.806A35E6@n...> Message-ID: >Hi. > >I'm having this problem. >When I run the start_medusa script I allways run it like this: > python start_medusa.py & >so that it runs as a background process, and it works fine. > >I normally run the start_medusa from a telnet session and then just exit a= nd it keeps running fine. > >Now I have this web-chat server that I've built on top of the chat server. >I run the chat server in the same way I run the medusa script and it works= perfectly well. But when I exit the telnet session, it just gets killed af= ter a while. > >Any hints? > >Thanks in advance for your time. > >Nicol=E1s Ech=E1niz >RAKAR Producciones nohup python start_medusa.py & - Sam ________________________________________ Samuel Reynolds Spinward Stars: http://www.spinwardstars.com/ Reynolds Virtual Workshop: http://www.primenet.com/~reynol reynol@p... samuel_reynolds@c... From lucifer@e... Wed Mar 14 22:29:20 2001 From: lucifer@e... (Lucifer) Date: Wed, 14 Mar 2001 18:29:20 -0400 Subject: [medusa] server as backgroud process In-Reply-To: Message-ID: <200103142229.SAA02707@L...> Thanks Sam. On Wed, 14 Mar 2001 08:37:35 -0700, Samuel Reynolds wrote: > >Hi. > > > >I'm having this problem. > >When I run the start_medusa script I allways run it like this: > > python start_medusa.py & > >so that it runs as a background process, and it works fine. > > > >I normally run the start_medusa from a telnet session and then just exit and it keeps running fine. > > > >Now I have this web-chat server that I've built on top of the chat server. > >I run the chat server in the same way I run the medusa script and it works perfectly well. But when I exit the telnet session, it just gets killed after a while. > > > >Any hints? > > > >Thanks in advance for your time. > > > >Nicolás Echániz > >RAKAR Producciones > > nohup python start_medusa.py & > > - Sam > > ________________________________________ > Samuel Reynolds > Spinward Stars: http://www.spinwardstars.com/ > Reynolds Virtual Workshop: http://www.primenet.com/~reynol > reynol@p... > samuel_reynolds@c... > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From weissblitz@y... Thu Mar 15 15:29:29 2001 From: weissblitz@y... (Weiss Blitz) Date: Thu, 15 Mar 2001 15:29:29 -0000 Subject: How to run Python/Medusa as a Win2000 service? Message-ID: <98qn4p+nc3p@e...> Does anyone have a tip on how to run Python/Medusa as a Win2000 service? Thanks in advance! From cwagner@s... Thu Mar 15 18:41:27 2001 From: cwagner@s... (Chuck Wagner) Date: Thu, 15 Mar 2001 12:41:27 -0600 Subject: [medusa] How to run Python/Medusa as a Win2000 service? References: <98qn4p+nc3p@e...> Message-ID: <003501c0ad7f$8d8417b0$9501c30a@b...> Well, as a starter on Python apps as a service, there are examples in Hammond's Python Programming on Win32 book. If you don't have that, you absolutely need it. Using those examples as a basis we were able to run our python app as a service on Win2k, and it's based on Sam's Asyncore/Asynchat architecture. We did have to put in a hack to asyncore to check the event flag after every timeout so that we could stop the service. I imagine that running medusa would involve a similar effort. Chuck ----- Original Message ----- From: "Weiss Blitz" To: Sent: Thursday, March 15, 2001 9:29 AM Subject: [medusa] How to run Python/Medusa as a Win2000 service? > Does anyone have a tip on how to > run Python/Medusa as a Win2000 service? > > Thanks in advance! > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From lucifer@e... Thu Mar 15 20:00:44 2001 From: lucifer@e... (Lucifer) Date: Thu, 15 Mar 2001 16:00:44 -0400 Subject: [medusa] kill open port In-Reply-To: <3AAFEE90.35718157@n...> Message-ID: <200103152000.QAA01276@L...> Thank you Sam· It will be useful if this happens again. On Wed, 14 Mar 2001 14:20:01 -0800, Sam Rushing wrote: > Lucifer wrote: > > > I'm using Linux Sam. Maybe your script would help. > > Better late than never. 8^) > (just checked it, appears to still work on a 2.4 kernel) > > [root@s... rushing]# python ./who_owns.py -s 890 udp > inode: 17987 > Name: rpc.statd > State: S (sleeping) > Pid: 1137 > PPid: 1 > TracerPid: 0 > Uid: 0 0 0 0 > Gid: 0 0 0 0 > FDSize: 32 > Groups: 0 1 2 3 4 6 10 11 > VmSize: 1176 kB > VmLck: 0 kB > VmRSS: 0 kB > VmData: 36 kB > VmStk: 12 kB > VmExe: 20 kB > VmLib: 1080 kB > SigPnd: 0000000000000000 > SigBlk: 0000000000000000 > SigIgn: 0000000000000000 > SigCgt: 0000000000004003 > CapInh: 0000000000000000 > CapPrm: 00000000fffffeff > CapEff: 00000000fffffeff > > -Sam > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > #!/usr/local/bin/python > # -*- Mode: Python; tab-width: 4 -*- > > import os > import regex > import string > import sys > > # linux-specific. > # find out what process is listening on a particular socket > > def inode_of_socket (port, type='tcp', server_only=1, remote=0): > target_port = string.upper ('%04x' % (string.atoi (port))) > lines = open ('/proc/net/%s' % type).readlines() > inode = 0 > for line in lines[1:]: > fields = string.split (line) > if remote: > index = 2 > else: > index = 1 > [addr, port] = string.split (fields[index], ':') > if port == target_port: > if (not server_only) or (addr == '00000000'): > inode = string.atoi (fields[9]) > break > return inode > > pid_regex = regex.compile ('\([0-9]+\)') > > def process_of_inode (inode): > #inode = '[0000]:%d' % inode > inode = 'socket:[%d]' % inode > cwd = os.getcwd() > try: > pids = filter ( > lambda x: pid_regex.match (x) == len(x), > os.listdir('/proc') > ) > pids.remove (str(os.getpid())) > for pid in pids: > fd_dir = '/proc/%s/fd' % pid > if os.path.isdir (fd_dir): > try: > os.chdir (fd_dir) > except OSError: > # hmm, a zombie? > if string.split (open('/proc/%s/stat' % pid).read())[2] == 'Z': > continue > links = map ( > os.readlink, > os.listdir ('.') > ) > if inode in links: > return pid > finally: > os.chdir (cwd) > return 0 > > def getopt (s): > import sys > if s in sys.argv: > sys.argv.remove (s) > return 1 > else: > return 0 > > if __name__ == '__main__': > if len(sys.argv) < 2: > print 'Usage: %s [-r|-s] [tcp|udp]' % sys.argv[0] > else: > remote = getopt ('-r') > server = getopt ('-s') > if len(sys.argv) < 3: > socket_type='tcp' > else: > socket_type=sys.argv[2] > port = sys.argv[1] > inode = inode_of_socket (port, socket_type, server_only=server, remote=remote) > if not inode: > print "Couldn't find inode for socket on %s port %s" % ( > socket_type, > port > ) > sys.exit(0) > else: > print 'inode: %d' % inode > process = process_of_inode (inode) > if not process: > print "Couldn't find process for inode %d" % inode > else: > print open('/proc/%s/status' % process).read() > > From eugene.leitl@l... Tue Mar 27 14:16:06 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Tue, 27 Mar 2001 14:16:06 -0000 Subject: freezing Medusa Message-ID: <99q7b6+chgh@e...> Anything running wrong when freezing Medusa? As root under Linux, I can run start_medusa.py unmodified as it comes out of the tarball. [root@p... medusa]# python start_medusa.py warning: Cannot do reverse lookup info: Medusa (V3.39) started at Tue Mar 27 16:12:58 2001 Hostname: 205.160.176.5 Port:8080 info: FTP server started at Tue Mar 27 16:12:58 2001 Authorizer: Hostname: pc27.infochem.de Port: 8021 info: Monitor Server (V1.28) started on port 9999 Chat Server (V1.6) started on port 8888 But not the frozen result: [root@p... medusa]# ./start_medusa warning: Cannot do reverse lookup info: Medusa (V3.39) started at Tue Mar 27 16:14:03 2001 Hostname: 205.160.176.5 Port:8080 info: FTP server started at Tue Mar 27 16:14:03 2001 Authorizer: Hostname: pc27.infochem.de Port: 8021 info: Monitor Server (V1.28) started on port 9999 Chat Server (V1.6) started on port 8888 Traceback (most recent call last): File "start_medusa.py", line 193, in ? os.setegid (gid) OSError: [Errno 1] Operation not permitted Huh? The culprit seems to be becoming nobody: # become 'nobody' if os.name == 'posix': import os if hasattr (os, 'seteuid'): # look in ~medusa/patches for {set,get}euid. import pwd [uid, gid] = pwd.getpwnam ('nobody')[2:4] os.seteuid (uid) os.setegid (gid) Can anybody clue me in? From eugene.leitl@l... Tue Mar 27 14:46:51 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Tue, 27 Mar 2001 16:46:51 +0200 (MET DST) Subject: [medusa] freezing Medusa In-Reply-To: <99q7b6+chgh@e...> Message-ID: It does run now. One-banana fix: I commented out the offending section below. Another question: if I'm doing object publishing, what is the best way to parse arguments and GET/POST requests, preferably a la Zope/Bobo, *without* requiring Zope/Bobo? I've tried Guido's example for SimpleHTTPServer.py import StringIO import SimpleHTTPServer BaseClass = SimpleHTTPServer.SimpleHTTPRequestHandler CounterTemplate = """

Server Statistics

This server has been accessed %d times. """ count = 0 class MyRequestHandler(BaseClass): def do_GET(self): global count count = count + 1 BaseClass.do_GET(self) def send_head(self): if self.path == "/counter.html": return self.send_counter() else: return BaseClass.send_head(self) def send_counter(self): self.send_response(200) self.send_header("Content-type", "text/html") self.end_headers() text = CounterTemplate % count return StringIO.StringIO(text) def test(): SimpleHTTPServer.test(MyRequestHandler) test() It does what I need, so in principle I don't need Medusa, but I don't have any clue how to pass arguments to the methods and parse GET/PUT requests the way CGI module does (packing them into dictionary). Confused in Krautland, Eugene On Tue, 27 Mar 2001 Eugene.Leitl@l... wrote: > > Anything running wrong when freezing Medusa? [...] > Huh? The culprit seems to be becoming nobody: > > # become 'nobody' > if os.name == 'posix': > import os > if hasattr (os, 'seteuid'): > # look in ~medusa/patches for {set,get}euid. > import pwd > [uid, gid] = pwd.getpwnam ('nobody')[2:4] > os.seteuid (uid) > os.setegid (gid) > > Can anybody clue me in? From eugene.leitl@l... Mon Apr 2 16:33:56 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Mon, 2 Apr 2001 17:33:56 +0200 (MET DST) Subject: any Medusa users out there? Message-ID: I'm hacking Medusa right, and would like to discuss a few issues with other Medusa users. It seems that medusa@yahoogroups.com is largely silent these days. Are any of you using Medusa still, or have you switched to Zope? From eugene.leitl@l... Mon Apr 2 17:06:25 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Mon, 2 Apr 2001 18:06:25 +0200 (MET DST) Subject: fighting handlers Message-ID: I've added the following handlers to start_medusa.py : import script_handler import persistent sch = script_handler.script_handler (fs) ph = script_handler.persistent_script_handler() ph.add_module ('per', persistent) hs.install_handler (sch) hs.install_handler (ph) However, this knocked out the default status handler: # Create a status handler. By default it binds to the URI '/status'... sh = status_handler.status_extension(status_objects) # ... and install it on the web server. hs.install_handler (sh) Since it's quite useful, I wanted to keep it. Is there a way, or is there just one handler allowed at a time? From t.c.jones@a... Mon Apr 2 17:53:49 2001 From: t.c.jones@a... (t.c.jones@a...) Date: Mon, 02 Apr 2001 16:53:49 +0000 Subject: [medusa] any Medusa users out there? Message-ID: <20010402165349.SOHD18668.mtiwmhc22.worldnet.att.net@w...> my site still uses medusa ..tom > > I'm hacking Medusa right, and would like to discuss a few issues with > other Medusa users. It seems that medusa@yahoogroups.com is largely silent > these days. > > Are any of you using Medusa still, or have you switched to Zope? > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > From kbd@i... Mon Apr 2 18:03:43 2001 From: kbd@i... (Davidson, Keith) Date: Mon, 2 Apr 2001 13:03:43 -0400 Subject: [medusa] any Medusa users out there? Message-ID: <3DCF393159D8D3119C1C00508B62B90622F259@n...> still using it too... -----Original Message----- From: t.c.jones@a... [mailto:t.c.jones@a...] Sent: Monday, April 02, 2001 12:54 PM To: medusa@yahoogroups.com Subject: Re: [medusa] any Medusa users out there? my site still uses medusa ..tom > > I'm hacking Medusa right, and would like to discuss a few issues with > other Medusa users. It seems that medusa@yahoogroups.com is largely silent > these days. > > Are any of you using Medusa still, or have you switched to Zope? > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ From weissblitz@y... Tue Apr 3 17:57:20 2001 From: weissblitz@y... (Weiss Blitz) Date: Tue, 03 Apr 2001 16:57:20 -0000 Subject: No subject Message-ID: <9acvdg+n9ao@e...> I'm running Medusa on a dual-processor Compaq Proliant server running Win-2K Advanced Server. When the application starts (even when idle) the CPU utilization jumps to 50% right off the bat. Does anyone knows if this is normal or if there is a way to improve CPU utilization? Thanks in advance, Carlos Perez From weissblitz@y... Tue Apr 3 17:58:25 2001 From: weissblitz@y... (Weiss Blitz) Date: Tue, 03 Apr 2001 16:58:25 -0000 Subject: CPU utilization optimization? Message-ID: <9acvfh+t5tj@e...> I'm running Medusa on a dual-processor Compaq Proliant server running Win-2K Advanced Server. When the application starts (even when idle) the CPU utilization jumps to 50% right off the bat. Does anyone knows if this is normal or if there is a way to improve CPU utilization? Thanks in advance, Carlos Perez From chrisk@a... Tue Apr 3 17:57:23 2001 From: chrisk@a... (Christopher Kolp) Date: Tue, 3 Apr 2001 12:57:23 -0400 Subject: [medusa] (unknown) In-Reply-To: <9acvdg+n9ao@e...> Message-ID: <001c01c0bc5f$280587e0$9865fea9@c...> Don't use Winblows as your OS. I could never get Medusa to run right on NT... > -----Original Message----- > From: Weiss Blitz [mailto:weissblitz@y...] > Sent: Tuesday, April 03, 2001 12:57 PM > To: medusa@yahoogroups.com > Subject: [medusa] (unknown) > > > I'm running Medusa on a dual-processor Compaq Proliant server running > Win-2K Advanced Server. When the application starts (even when idle) > the CPU utilization jumps to 50% right off the bat. Does anyone > knows if this is normal or if there is a way to improve CPU > utilization? > > Thanks in advance, > Carlos Perez > > > > ------------------------ Yahoo! Groups Sponsor > ---------------------~-~> > Do you have 128-bit SSL encryption server security? > Get VeriSign's FREE Guide, "Securing Your > Web Site for Business." Get it now! > http://us.click.yahoo.com/EVNB7A/c.WCAA/bT0EAA/CxaWlB/TM > -------------------------------------------------------------- > -------_-> > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > From billtut@m... Tue Apr 3 18:03:40 2001 From: billtut@m... (Bill Tutt) Date: Tue, 3 Apr 2001 10:03:40 -0700 Subject: [medusa] CPU utilization optimization? Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A626E@r...> Python isn't completely thread safe. It has a lock that must be acquired before any Python C API may be called. This is what causes the behavior you're seeing. The easiest way to work around the problem is to move more of your code into Python extensions written in C/C++/whatever that don't rely on Python API calls to do the majority of your work. (Assuming the performance you're getting isn't good enough.) Bill -----Original Message----- From: Weiss Blitz [mailto:weissblitz@y...]=20 Sent: Tuesday, April 03, 2001 9:58 AM To: medusa@yahoogroups.com Subject: [medusa] CPU utilization optimization? I'm running Medusa on a dual-processor Compaq Proliant server running=20 Win-2K Advanced Server. When the application starts (even when idle)=20 the CPU utilization jumps to 50% right off the bat. Does anyone=20 knows if this is normal or if there is a way to improve CPU=20 utilization? Thanks in advance, Carlos Perez =20 Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/=20 From billtut@m... Tue Apr 3 18:16:41 2001 From: billtut@m... (Bill Tutt) Date: Tue, 3 Apr 2001 10:16:41 -0700 Subject: [medusa] (unknown) Message-ID: <58C671173DB6174A93E9ED88DCB0883D02667B84@r...> Now now, the same thing will happen on any dual processor box regardless of the OS, it's a fundamental problem with the implementation of Python. Got medusa working quite well on NT, Bill -----Original Message----- From: Christopher Kolp [mailto:chrisk@a...]=20 Sent: Tuesday, April 03, 2001 9:57 AM To: medusa@yahoogroups.com Subject: RE: [medusa] (unknown) Don't use Winblows as your OS. I could never get Medusa to run right on NT... > -----Original Message----- > From: Weiss Blitz [mailto:weissblitz@y...] > Sent: Tuesday, April 03, 2001 12:57 PM > To: medusa@yahoogroups.com > Subject: [medusa] (unknown) >=20 >=20 > I'm running Medusa on a dual-processor Compaq Proliant server running=20 > Win-2K Advanced Server. When the application starts (even when idle)=20 > the CPU utilization jumps to 50% right off the bat. Does anyone=20 > knows if this is normal or if there is a way to improve CPU=20 > utilization? >=20 > Thanks in advance, > Carlos Perez >=20 >=20 >=20 > ------------------------ Yahoo! Groups Sponsor=20 > ---------------------~-~> > Do you have 128-bit SSL encryption server security? > Get VeriSign's FREE Guide, "Securing Your > Web Site for Business." Get it now! > http://us.click.yahoo.com/EVNB7A/c.WCAA/bT0EAA/CxaWlB/TM > -------------------------------------------------------------- > -------_-> >=20 >=20=20 >=20 > Your use of Yahoo! Groups is subject to=20 > http://docs.yahoo.com/info/terms/=20 >=20 >=20 =20 Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/=20 From greg@e... Tue Apr 3 21:18:04 2001 From: greg@e... (Gregory P. Smith) Date: Tue, 3 Apr 2001 13:18:04 -0700 Subject: [medusa] 50% cpu usage on dual processor box In-Reply-To: <58C671173DB6174A93E9ED88DCB0883D02667B84@r...>; from billtut@m... on Tue, Apr 03, 2001 at 10:16:41AM -0700 References: <58C671173DB6174A93E9ED88DCB0883D02667B84@r...> Message-ID: <20010403131804.A27258@z...> On Tue, Apr 03, 2001 at 10:16:41AM -0700, Bill Tutt wrote: > Now now, the same thing will happen on any dual processor box regardless > of the OS, it's a fundamental problem with the implementation of Python. > > Got medusa working quite well on NT, > Bill On a dual processor linux box I've never noticed this behavior. Why would python acquiring a lock on a dual processor windows box cause 50% cpu utilization? There is no need for it to be a spinlock. curious, Greg -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From weissblitz@y... Wed Apr 4 17:37:42 2001 From: weissblitz@y... (Weiss Blitz) Date: Wed, 04 Apr 2001 16:37:42 -0000 Subject: Medusa hangings waiting on keystroke!!! Message-ID: <9afikm+l46n@e...> I noticed Medusa stops running and if I press a key it will keep on running. Seems for some reason it's waiting on a kbd input. This is running on a dual-processor w2k Advanced server box with 260Mb of RAM. Any ideas on this? Thanks in advance, Carlos From eugene.leitl@l... Wed Apr 4 22:30:32 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Wed, 04 Apr 2001 23:30:32 +0200 Subject: [medusa] Medusa hangings waiting on keystroke!!! References: <9afikm+l46n@e...> Message-ID: <3ACB9278.C5C7DFAD@l...> Weiss Blitz wrote: > > I noticed Medusa stops running and if I press a key it will keep on > running. Seems for some reason it's waiting on a kbd input. This is > running on a dual-processor w2k Advanced server box with 260Mb of > RAM. Any ideas on this? I'm new to Medusa, but I've never seen such behaviour so far. I think it's a rather safe bet the problems are with the w2k server box. Try it with *nix instead. From greg@e... Wed Apr 4 21:17:56 2001 From: greg@e... (Gregory P. Smith) Date: Wed, 4 Apr 2001 13:17:56 -0700 Subject: [medusa] Medusa hangings waiting on keystroke!!! In-Reply-To: <9afikm+l46n@e...>; from weissblitz@y... on Wed, Apr 04, 2001 at 04:37:42PM -0000 References: <9afikm+l46n@e...> Message-ID: <20010404131756.A18614@z...> On Wed, Apr 04, 2001 at 04:37:42PM -0000, Weiss Blitz wrote: > I noticed Medusa stops running and if I press a key it will keep on > running. Seems for some reason it's waiting on a kbd input. This is > running on a dual-processor w2k Advanced server box with 260Mb of > RAM. Any ideas on this? > > Thanks in advance, > Carlos Perhaps its not medusa related at all: If you're running anything in a text window on windows and select a portion with the mouse or scroll back in the window I've noticed that it hangs the process within until you hit a key (causing the it to go out of selection or scroll back mode). bad behaviour on windows part. Greg From rushing@n... Wed Apr 4 21:36:05 2001 From: rushing@n... (Sam Rushing) Date: Wed, 04 Apr 2001 13:36:05 -0700 Subject: [medusa] Medusa hangings waiting on keystroke!!! References: <9afikm+l46n@e...> Message-ID: <3ACB85B5.6C9FDDE0@n...> Weiss Blitz wrote: > I noticed Medusa stops running and if I press a key it will keep on > running. Seems for some reason it's waiting on a kbd input. This is > running on a dual-processor w2k Advanced server box with 260Mb of > RAM. Any ideas on this? The resource kit used to come with tools for turning console-mode programs into services. Another resource would be Zope, I'm sure they've addressed techniques for running on nt. -Sam From z30303@h... Wed Apr 4 22:18:53 2001 From: z30303@h... (z30303@h...) Date: Wed, 04 Apr 2001 21:18:53 -0000 Subject: I have it working Message-ID: <9ag33t+eivo@e...> OK, not hard but a little annoying to set this up. Find instsrv.exe and srvany.exe (typically from Windows 2000 Resource Disk or NT Resource Disk). Usually, I put them into C:\WINNT\System32 at command enter 'instsrv.exe medusa "C:\WINNT\System32\srvany.exe"' fire up a registry editor (run->regedit) we will modify HKLM.SYSTEM.CurrentControlSet.Services.Medusa add a new Key, called "Parameters" under Parameters add a String called "Application", with a value of "C:\Python20\python.exe" under Parameters add a String called "AppParameters", with a value of "C:\Medusa\start_medusa.py" under Parameters add a String called "AppDirectory", with a value of "C:\Medusa" all quotes are optional (as far as i know), and all paths are relative to your particular installation. now you can issue "net start medusa" from command line, and it will run. "net stop medusa" will stop the service. You can also use the Management console. Good Luck From z30303@h... Wed Apr 4 22:22:42 2001 From: z30303@h... (z30303@h...) Date: Wed, 04 Apr 2001 21:22:42 -0000 Subject: How to run Python/Medusa as a Win2000 service? In-Reply-To: <003501c0ad7f$8d8417b0$9501c30a@b...> Message-ID: <9ag3b2+cm7p@e...> Check out the "I have it working" post I sent in a few minutes ago --- In medusa@y..., "Chuck Wagner" wrote: > Well, as a starter on Python apps as a service, there are examples in > Hammond's Python Programming on Win32 book. If you don't have that, you > absolutely need it. > > Using those examples as a basis we were able to run our python app as a > service on Win2k, and it's based on Sam's Asyncore/Asynchat architecture. We > did have to put in a hack to asyncore to check the event flag after every > timeout so that we could stop the service. > > I imagine that running medusa would involve a similar effort. > > Chuck > ----- Original Message ----- > From: "Weiss Blitz" > To: > Sent: Thursday, March 15, 2001 9:29 AM > Subject: [medusa] How to run Python/Medusa as a Win2000 service? > > > > Does anyone have a tip on how to > > run Python/Medusa as a Win2000 service? > > > > Thanks in advance! > > > > > > > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > > > > From weissblitz@y... Thu Apr 5 00:35:27 2001 From: weissblitz@y... (Weiss Blitz) Date: Wed, 04 Apr 2001 23:35:27 -0000 Subject: I have it working In-Reply-To: <9ag33t+eivo@e...> Message-ID: <9agb3v+9mgl@e...> Thanks for the info :) I have been using this method for about a week now. But srvany.exe opens python on a console window. Is there a way to prevent this console window from showing? --- In medusa@y..., z30303@h... wrote: > OK, not hard but a little annoying to set this up. > > Find instsrv.exe and srvany.exe (typically from Windows 2000 Resource > Disk or NT Resource Disk). Usually, I put them into C:\WINNT\System32 > > at command enter 'instsrv.exe medusa "C:\WINNT\System32\srvany.exe"' > > fire up a registry editor (run->regedit) > > we will modify HKLM.SYSTEM.CurrentControlSet.Services.Medusa > add a new Key, called "Parameters" > under Parameters add a String called "Application", with a value > of "C:\Python20\python.exe" > under Parameters add a String called "AppParameters", with a value > of "C:\Medusa\start_medusa.py" > under Parameters add a String called "AppDirectory", with a value > of "C:\Medusa" > > all quotes are optional (as far as i know), and all paths are > relative to your particular installation. > > now you can issue "net start medusa" from command line, and it will > run. "net stop medusa" will stop the service. You can also use the > Management console. > > Good Luck From cwagner@s... Thu Apr 5 00:59:26 2001 From: cwagner@s... (Chuck Wagner) Date: Wed, 4 Apr 2001 18:59:26 -0500 Subject: [medusa] Re: How to run Python/Medusa as a Win2000 service? References: <9ag3b2+cm7p@e...> Message-ID: <006101c0bd63$47fc8f80$9501c30a@b...> If you create the service using Hammond's guidelines, it to doesn't open any console windows - it runs using the PythonService.exe. ----- Original Message ----- From: To: Sent: Wednesday, April 04, 2001 4:22 PM Subject: [medusa] Re: How to run Python/Medusa as a Win2000 service? > Check out the "I have it working" post I sent in a few minutes ago > > --- In medusa@y..., "Chuck Wagner" wrote: > > Well, as a starter on Python apps as a service, there are examples > in > > Hammond's Python Programming on Win32 book. If you don't have that, > you > > absolutely need it. > > > > Using those examples as a basis we were able to run our python app > as a > > service on Win2k, and it's based on Sam's Asyncore/Asynchat > architecture. We > > did have to put in a hack to asyncore to check the event flag after > every > > timeout so that we could stop the service. > > > > I imagine that running medusa would involve a similar effort. > > > > Chuck > > ----- Original Message ----- > > From: "Weiss Blitz" > > To: > > Sent: Thursday, March 15, 2001 9:29 AM > > Subject: [medusa] How to run Python/Medusa as a Win2000 service? > > > > > > > Does anyone have a tip on how to > > > run Python/Medusa as a Win2000 service? > > > > > > Thanks in advance! > > > > > > > > > > > > > > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > > > > > > > > > > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > > From weissblitz@y... Thu Apr 5 01:07:42 2001 From: weissblitz@y... (Weiss Blitz) Date: Thu, 05 Apr 2001 00:07:42 -0000 Subject: CPU utilization optimization? PROBLEM FOUND! In-Reply-To: <58C671173DB6174A93E9ED88DCB0883D0A626E@r...> Message-ID: <9agd0e+bj2u@e...> Ok, I think I found the problem... I developed a Medusa based instant messaging server running on a dual- processor Compaq Proliant box under Win2000 Server. I noticed Medusa idled at 1-2%, one connection comes up and the CPU utilization jumps to 100%! Found that the asyncore.py select loop was being pegged as soon as the first connection was made. Also noticed this was so because the asynchat.writable method was always returning a '1' even when the ac_out_buffer output buffer was empty. I replaced the 'is' operator with the '==' and it now seems to work normally. Following is a copy of the original and the modified version of the asynchat.writable method: def writable (self): "predicate for inclusion in the writable for select()" # return len(self.ac_out_buffer) or len(self.producer_fifo) or (not self.connected) # this is about twice as fast, though not as clear. return not ( (self.ac_out_buffer is '') and self.producer_fifo.is_empty() and self.connected ) def writable (self): "predicate for inclusion in the writable for select()" # return len(self.ac_out_buffer) or len(self.producer_fifo) or (not self.connected) # this is about twice as fast, though not as clear. return not ( (self.ac_out_buffer == '') and self.producer_fifo.is_empty() and self.connected ) Not sure if this is the way it is on the original Medusa distribution, but at least that's the way mine was until now. Anyways, thanks to all who have spent time and efforts to make Medusa and Python even better tools! Carlos :) From weissblitz@y... Fri Apr 6 21:32:11 2001 From: weissblitz@y... (Weiss Blitz) Date: Fri, 06 Apr 2001 20:32:11 -0000 Subject: HELP! too many file descriptors in select()!!!! Message-ID: <9al94b+hkqb@e...> I got the following message from Python20/Medusa running on a Win2k Advanced Server: Traceback (most recent call last): File "zulu_start.py", line 254, in ? asyncore.loop() File "d:\\zuluforumserver\asyncore.py", line 139, in loop poll_fun (timeout, map) File "d:\\zuluforumserver\asyncore.py", line 67, in poll r,w,e = select.select (r,w,e, timeout) ValueError: too many file descriptors in select() Only seem to be around 50 to 60 socket connections when this happened ! Does anyone knows what's the limit on the selector descriptors?! Or a way around this? Carlos :( From greg@e... Fri Apr 6 21:44:44 2001 From: greg@e... (Gregory P. Smith) Date: Fri, 6 Apr 2001 13:44:44 -0700 Subject: [medusa] HELP! too many file descriptors in select()!!!! In-Reply-To: <9al94b+hkqb@e...>; from weissblitz@y... on Fri, Apr 06, 2001 at 08:32:11PM -0000 References: <9al94b+hkqb@e...> Message-ID: <20010406134444.A30921@z...> On Fri, Apr 06, 2001 at 08:32:11PM -0000, Weiss Blitz wrote: > I got the following message from Python20/Medusa running on a Win2k > Advanced Server: > > Traceback (most recent call last): > File "zulu_start.py", line 254, in ? > asyncore.loop() > File "d:\\zuluforumserver\asyncore.py", line 139, in loop > poll_fun (timeout, map) > File "d:\\zuluforumserver\asyncore.py", line 67, in poll > r,w,e = select.select (r,w,e, timeout) > ValueError: too many file descriptors in select() > > Only seem to be around 50 to 60 socket connections when this happened > ! > Does anyone knows what's the limit on the selector descriptors?! > Or a way around this? > > Carlos :( You need to recompile the _select.pyd Python module with FS_SETSIZE set to a reasonable number (such as 256). Windows, by default, only supports up to 64 file descriptors in a call to select. I believe they have changed the default compilation to python in python 2.1 to use 256 or 512. (you'll still hit this problem at those higher numbers of sockets; at that point you should look into making your application not require as many open sockets at once) Greg -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From greg@e... Fri Apr 6 21:46:16 2001 From: greg@e... (Gregory P. Smith) Date: Fri, 6 Apr 2001 13:46:16 -0700 Subject: [medusa] HELP! too many file descriptors in select()!!!! In-Reply-To: <20010406134444.A30921@z...>; from greg@e... on Fri, Apr 06, 2001 at 01:44:44PM -0700 References: <9al94b+hkqb@e...> <20010406134444.A30921@z...> Message-ID: <20010406134616.B30921@z...> > You need to recompile the _select.pyd Python module with FS_SETSIZE err that should read.. FD_SETSIZE or FD_SET_SIZE, double check. :) From jim@d... Sat Apr 7 18:09:33 2001 From: jim@d... (Jim Fulton) Date: Sat, 07 Apr 2001 13:09:33 -0400 Subject: [medusa] Re: CPU utilization optimization? PROBLEM FOUND! References: <9agd0e+bj2u@e...> Message-ID: <3ACF49CD.1AE9EE83@d...> Weiss Blitz wrote: > > Ok, I think I found the problem... > > I developed a Medusa based instant messaging server running on a dual- > processor Compaq Proliant box under Win2000 Server. > > I noticed Medusa idled at 1-2%, one connection comes up and the CPU > utilization jumps to 100%! > > Found that the asyncore.py select loop was being pegged as soon as > the first connection was made. Also noticed this was so because the > asynchat.writable method was always returning a '1' even when the > ac_out_buffer output buffer was empty. I replaced the 'is' operator > with the '==' and it now seems to work normally. Interesting. Did you have the select timeout set very low? Jim -- Jim Fulton mailto:jim@d... Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org From weissblitz@y... Sat Apr 7 19:45:15 2001 From: weissblitz@y... (Weiss Blitz) Date: Sat, 07 Apr 2001 18:45:15 -0000 Subject: HELP! too many file descriptors in select()!!!! In-Reply-To: <20010406134444.A30921@z...> Message-ID: <9ann7r+hqso@e...> Yikes!!!... Is 512 the maximum possible number of file descriptors under Win2k Advanced Server?! What about under Linux? Do I hear 32767 or higher? Please,please,please?!! This will greately limit the number of open sockets that Medusa can handle concurrently! My application is an instant-messenger server and needs to maintain an open socket connection with each client! Any suggestions? Carlos :( --- In medusa@y..., "Gregory P. Smith" wrote: > On Fri, Apr 06, 2001 at 08:32:11PM -0000, Weiss Blitz wrote: > > I got the following message from Python20/Medusa running on a Win2k > > Advanced Server: > > > > Traceback (most recent call last): > > File "zulu_start.py", line 254, in ? > > asyncore.loop() > > File "d:\\zuluforumserver\asyncore.py", line 139, in loop > > poll_fun (timeout, map) > > File "d:\\zuluforumserver\asyncore.py", line 67, in poll > > r,w,e = select.select (r,w,e, timeout) > > ValueError: too many file descriptors in select() > > > > Only seem to be around 50 to 60 socket connections when this happened > > ! > > Does anyone knows what's the limit on the selector descriptors?! > > Or a way around this? > > > > Carlos :( > > You need to recompile the _select.pyd Python module with FS_SETSIZE > set to a reasonable number (such as 256). Windows, by default, only > supports up to 64 file descriptors in a call to select. > > I believe they have changed the default compilation to python in > python 2.1 to use 256 or 512. > > (you'll still hit this problem at those higher numbers of sockets; at > that point you should look into making your application not require as > many open sockets at once) > > Greg > > -- > Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ > C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From greg@e... Sat Apr 7 20:06:17 2001 From: greg@e... (Gregory P. Smith) Date: Sat, 7 Apr 2001 12:06:17 -0700 Subject: [medusa] Re: HELP! too many file descriptors in select()!!!! In-Reply-To: <9ann7r+hqso@e...>; from weissblitz@y... on Sat, Apr 07, 2001 at 06:45:15PM -0000 References: <20010406134444.A30921@z...> <9ann7r+hqso@e...> Message-ID: <20010407120617.A10369@z...> On Sat, Apr 07, 2001 at 06:45:15PM -0000, Weiss Blitz wrote: > Yikes!!!... > > Is 512 the maximum possible number of file descriptors under Win2k > Advanced Server?! > > What about under Linux? > Do I hear 32767 or higher? Please,please,please?!! I never said 512 was the maximum, just suggested it as a possible value that's better than 64. You'll have to ask microsoft what the max is... (linux defaults to 1024 per process but its dynamically changable to high values using the /proc filesystem). > > This will greately limit the number of open sockets that Medusa can > handle concurrently! My application is an instant-messenger server > and needs to maintain an open socket connection with each client! > > Any suggestions? > Carlos :( Holding an open connection to each client even while idle is not necessarily good (but i agree, its much simpler). Is there any reason you have to? In mojonation for our message based protocol running on top of TCP, we close connections after an idle period or when nearning your limit of max connections and reestablish them as needed. We use relay servers for people who are behind NAT or firewalls that can't accept incoming connections when another party needs to reestablish a connection with them (but can make outgoing ones which if open are preferred over relay). Greg -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From zmbq@a... Mon Apr 9 08:56:01 2001 From: zmbq@a... (Itay Zandbank) Date: Mon, 9 Apr 2001 09:56:01 +0200 Subject: [medusa] Re: CPU utilization optimization? PROBLEM FOUND! References: <9agd0e+bj2u@e...> <3ACF49CD.1AE9EE83@d...> Message-ID: <005b01c0c0ca$8532ce40$05010101@s...> > > Found that the asyncore.py select loop was being pegged as soon as > > the first connection was made. Also noticed this was so because the > > asynchat.writable method was always returning a '1' even when the > > ac_out_buffer output buffer was empty. I replaced the 'is' operator > > with the '==' and it now seems to work normally. > Interesting. Did you have the select timeout set very low? The timeout in his case was irrelevant, since select always returned instantly. I once had the same problem, when I didn't close a dispatcher down properly. It took me a while to figure out why /tmp filled up with debug printings... From ecrespo@u... Tue Apr 10 12:19:55 2001 From: ecrespo@u... (Ernesto Nadir Crespo Avila) Date: Tue, 10 Apr 2001 07:19:55 -0400 Subject: Trouble with the CGI Message-ID: <01041007195500.01222@z...> I Can=B4t get the values passed to a web page, even that I tried the method= s Get and Post, the results were: With POST: The Page=B4s structure it=B4s the following:
The cgi "ej4.mpy" is: # -*- Mode: Python; tab-width: 4 -*- # send and process a POST form import sys, cgi print '' print '' print '' print print ' A Phyton Example' print '' print '' datos =3D sys.stdin.read() if datos: print '

The name is: %s

' % repr(cgi.parse_qs(datos) else: print '

There=B4s no data .

' print '' print '' The message returned by this method Error response Error code 500. Message: Internal Server Error. The Log message: 127.0.0.1:1119 - - [10/Apr/2001:01:21:59 -0400] "POST /webconflrp/cgi-bin/ej4.mpy HTTP/1.1" 500 289 ***************************************************************************= ** ***** The Method Get The Page=B4s structure it=B4s the following:
The cgi "ej4.mpy" is: # -*- Mode: Python; tab-width: 4 -*- # send and process a POST form import cgi print '' print '' print '' print print ' A Cgi example with Phyton' print '' print '' datos =3D cgi.FieldStorage() if datos.has_key('name'): print '

Your data: %s

' %datos['name'] else: print 'there=B4s no data' print '' print '' The page waits a lot of time for loading and the log returned: 127.0.0.1:1138 - - [10/Apr/2001:01:49:49 -0400] "GET /webconflrp/cgi-bin/ej5.mpy?name=3Dernesto HTTP/1.1" 500 116 How can I solve this? Thank you all in advance! Bye, Ernesto Crespo From rprice@u... Tue Apr 10 17:24:13 2001 From: rprice@u... (rprice@u...) Date: Tue, 10 Apr 2001 09:24:13 -0700 (PDT) Subject: [medusa] Trouble with the CGI In-Reply-To: <01041007195500.01222@z...> References: <01041007195500.01222@z...> Message-ID: <986919853.3ad333addcb46@w...> It's been a while since I've workd with cgi.py but I believe your problem is in reading datos['name']. Try datos['name'].value instead. datos['name'] may contain a list or string, so adapt your code accordingly. Ray Quoting Ernesto Nadir Crespo Avila : > I Can´t get the values passed to a web page, even that I tried the > methods > Get and Post, the results were: > > > With POST: > > The Page´s structure it´s the following: > >
name="name" size="15" maxlength="15"> > > > > The cgi "ej4.mpy" is: > > # -*- Mode: Python; tab-width: 4 -*- > # send and process a POST form > import sys, cgi > > print '' > print '' > print '' > print > print ' A Phyton Example' > print '' > print '' > datos = sys.stdin.read() > > if datos: > print '

The name is: %s

' % repr(cgi.parse_qs(datos) > else: > print '

There´s no data .

' > print '' > print '' > > The message returned by this method > Error response > Error code 500. > Message: Internal Server Error. > > The Log message: > 127.0.0.1:1119 - - [10/Apr/2001:01:21:59 -0400] "POST > /webconflrp/cgi-bin/ej4.mpy HTTP/1.1" 500 289 > > ***************************************************************************** > ***** The Method Get > The Page´s structure it´s the following: > >
> > > > > The cgi "ej4.mpy" is: > > # -*- Mode: Python; tab-width: 4 -*- > # send and process a POST form > import cgi > print '' > print '' > print '' > print > print ' A Cgi example with Phyton' > print '' > print '' > datos = cgi.FieldStorage() > > if datos.has_key('name'): > print '

Your data: %s

' %datos['name'] > else: > print 'there´s no data' > print '' > print '' > > The page waits a lot of time for loading and the log returned: > 127.0.0.1:1138 - - [10/Apr/2001:01:49:49 -0400] "GET > /webconflrp/cgi-bin/ej5.mpy?name=ernesto HTTP/1.1" 500 116 > > How can I solve this? > Thank you all in advance! > > Bye, Ernesto Crespo > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > From ecrespo@u... Wed Apr 11 04:16:25 2001 From: ecrespo@u... (Ernesto Nadir Crespo Avila) Date: Tue, 10 Apr 2001 23:16:25 -0400 Subject: [medusa] Trouble with the CGI In-Reply-To: <986919853.3ad333addcb46@w...> References: <01041007195500.01222@z...> <986919853.3ad333addcb46@w...> Message-ID: <01041023162500.13529@z...> El Mar 10 Abr 2001 12:24, escribiste: > It's been a while since I've workd with cgi.py but I believe your problem > is in reading datos['name']. Try datos['name'].value instead.=20 > datos['name'] may contain a list or string, so adapt your code accordingl= y. Thank you. But, now just appear: there=B4s no data And FieldStorage(None, None, [])=20=20 Bye, Ernesto Crespo # -*- Mode: Python; tab-width: 4 -*- # send and process a POST form import cgi =20 print '' print '' print '' print print ' A Cgi example with Phyton' print '' print '' datos =3D cgi.FieldStorage() =20 if datos.has_key('name'): print '

Your data: %s

' %datos['name'].value else: print '

there=B4s no data

' print datos print '' print '' From lou@h... Thu Apr 12 23:19:05 2001 From: lou@h... (lou@h...) Date: Thu, 12 Apr 2001 22:19:05 -0000 Subject: extended logging Message-ID: <9b59kp+q4f1@e...> I am using the version of medusa that came bundled with zope. Has anyone extended logging as to add the host header information. I have found the log routine but I do not know what variable name to reference this. Thanks in advance. Lou From rprice@u... Thu Apr 12 23:53:30 2001 From: rprice@u... (rprice@u...) Date: Thu, 12 Apr 2001 15:53:30 -0700 (PDT) Subject: medusa and apache proxy Message-ID: <987116010.3ad631ea5993a@w...> When a user accesses medusa directly (not through a proxy) it takes less than 1 second to display the page. When a user accesses the same medusa server page via an apache proxy server, the pages seem to hang at 66% loaded. It can then take anywhere from 2 to 60 seconds for the page to finally load. Another problem when accessing through the proxy is that the browser occasionally indicates 'contacting server' for many seconds before any information is transmitted. Any ideas why this would happen? FYI I am using the most current version of medusa along with the old, orignal CGIResponse.py ,cgi_module_publisher.py files. Ray From rushing@n... Fri Apr 13 06:12:45 2001 From: rushing@n... (Sam Rushing) Date: Thu, 12 Apr 2001 22:12:45 -0700 Subject: [medusa] medusa and apache proxy References: <987116010.3ad631ea5993a@w...> Message-ID: <3AD68ACD.ED5BAAB9@n...> rprice@u... wrote: > When a user accesses medusa directly (not through a proxy) it takes less than 1 > second to display the page. > > When a user accesses the same medusa server page via an apache proxy server, the > pages seem to hang at 66% loaded. It can then take anywhere from 2 to 60 > seconds for the page to finally load. Another problem when accessing through > the proxy is that the browser occasionally indicates 'contacting server' for > many seconds before any information is transmitted. > > Any ideas why this would happen? None yet... a couple of things to try: 1) what happens if you try squid instead of apache proxy? 2) what protocol versions (i.e., http/1.0 or 1.1) are being used between browser, proxy, and medusa? -Sam From rprice@u... Fri Apr 13 22:03:34 2001 From: rprice@u... (rprice@u...) Date: Fri, 13 Apr 2001 14:03:34 -0700 (PDT) Subject: [medusa] medusa and apache proxy In-Reply-To: <3AD68ACD.ED5BAAB9@n...> References: <987116010.3ad631ea5993a@w...> <3AD68ACD.ED5BAAB9@n...> Message-ID: <987195814.3ad769a6419fb@w...> I have more information, unfortunately not what Sam asked for, but hopefully helpful! BROWSER PLATFM PROXY? REPLY Netscape Linux no proxy no delay Netscape Win98 no proxy no delay Explorer Win98 no proxy no delay LWP Perl Linux no proxy 30 second delay lynx -dump Linux no proxy 30 second delay LWP Perl Linux with proxy 30 second delay lynx -dump Linux with proxy 30 second delay Netscape Linux with proxy 30 second delay Netscape Win98 with proxy no delay !!! Ray Quoting Sam Rushing : > rprice@u... wrote: > > > When a user accesses medusa directly (not through a proxy) it takes > less than 1 > > second to display the page. > > > > When a user accesses the same medusa server page via an apache proxy > server, the > > pages seem to hang at 66% loaded. It can then take anywhere from 2 to > 60 > > seconds for the page to finally load. Another problem when accessing > through > > the proxy is that the browser occasionally indicates 'contacting > server' for > > many seconds before any information is transmitted. > > > > Any ideas why this would happen? > > None yet... a couple of things to try: > > 1) what happens if you try squid instead of apache proxy? > 2) what protocol versions (i.e., http/1.0 or 1.1) are being used > between browser, > proxy, and medusa? > > -Sam > > > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > From lou@h... Fri Apr 13 22:49:22 2001 From: lou@h... (lou@h...) Date: Fri, 13 Apr 2001 21:49:22 -0000 Subject: extended logging In-Reply-To: <9b59kp+q4f1@e...> Message-ID: <9b7s92+prrr@e...> I found the answer in a zope archive: http://zope.nipltd.com/public/lists/zope-archive.nsf/242bb7cc2b2c343d802568ab003585d4/b5bad84d0e217ff280256994000ace5f?OpenDocument In ZServer/medusa/http_server.py, the "log" method of the class "http_request" can be modified to report the request hostname which is readily available via the get_header method in the same class. Here is my modified "log" method: def log (self, bytes): user_agent=self.get_header('user-agent') if not user_agent: user_agent='' referer=self.get_header('referer') #ADD host=self.get_header('host') if not referer: referer='' self.channel.server.logger.log ( self.channel.addr[0], #ADD %S ' - - [%s] "%s" %d %d "%s" "%s" %s\n' % ( self.log_date_string (time.time()), self.request, self.reply_code, bytes, referer, user_agent, # ADD host host ) ) This will add the host portion of the http header to the end of each log entry. where can I find the complete list of get_header returns??? Lou --- In medusa@y..., lou@h... wrote: > I am using the version of medusa that came bundled with zope. Has > anyone extended logging as to add the host header information. I have > found the log routine but I do not know what variable name to > reference this. Thanks in advance. > > Lou From rprice@u... Fri Apr 13 23:08:25 2001 From: rprice@u... (rprice@u...) Date: Fri, 13 Apr 2001 15:08:25 -0700 (PDT) Subject: No subject Message-ID: <987199705.3ad778d9410f2@w...> Latest information from the sysadmin working on this: I haven't tried with squid. I'm interested in doing so, but with my experience with squid (zero) I don't expect that to be a fair test. The protocol was HTTP/1.1 for all browsers and HTTP/1.0 for the request from apache to medusa. Using "lynx -head -dump" shows "HTTP/1.0 200 OK", but I believe that that is due to the proxied HTTP/1.0 request since apache is proxying for a name based virtual. Apache: 1.3.19 Medusa: 20000601 (right?) Browsers: (See other email for varying results) Internet Explorer 5.00.3105.0106 (Win 98) Netscape Communicator 4.75 (Win 98) Netscape Communicator 4.76 (Mandrake 7.2) Lynx Version 2.8.4dev.8 (Mandrake 7.2) Lynx Version 2.8.3dev.18 (Red Hat 6.2) LWP UserAgent.pm,v 1.73 (Mandrake 7.2) LWP UserAgent.pm,v 1.77 (Red Hat 6.2) Ray From lucifer@e... Mon Apr 16 23:41:05 2001 From: lucifer@e... (lucifer@e...) Date: Mon, 16 Apr 2001 22:41:05 -0000 Subject: Trouble with the CGI In-Reply-To: <01041023162500.13529@z...> Message-ID: <9bfse1+5bsd@e...> I've written my own code to replace cgi.Fieldstorage() but I don't know if it will be ok for you. Has anybody ever got cgi.FieldStorage() to work inside medusa ? --- In medusa@y..., Ernesto Nadir Crespo Avila wrote: >=20 >=20 > El Mar 10 Abr 2001 12:24, escribiste: > > It's been a while since I've workd with cgi.py but I believe your problem > > is in reading datos['name']. Try datos['name'].value instead.=20 > > datos['name'] may contain a list or string, so adapt your code accordingly. > Thank you. >=20 > But, now just appear: >=20 > there=B4s no data >=20 > And >=20 > FieldStorage(None, None, [])=20=20 >=20 > Bye, Ernesto Crespo >=20 >=20 > # -*- Mode: Python; tab-width: 4 -*- > # send and process a POST form > import cgi >=20=20 > print '' > print '' > print '' > print > print ' A Cgi example with Phyton' > print '' > print '' > datos =3D cgi.FieldStorage() >=20=20 > if datos.has_key('name'): > print '

Your data: %s

' %datos['name'].value > else: > print '

there=B4s no data

' > print datos > print '' > print '' From rprice@u... Tue Apr 17 00:16:49 2001 From: rprice@u... (rprice@u...) Date: Mon, 16 Apr 2001 16:16:49 -0700 (PDT) Subject: [medusa] Re: Trouble with the CGI In-Reply-To: <9bfse1+5bsd@e...> References: <9bfse1+5bsd@e...> Message-ID: <987463009.3adb7d61cdbb3@w...> I have never used it from within medusa. I use the old bobo code to interact with my Python scripts. Ray Quoting lucifer@e...: > I've written my own code to replace cgi.Fieldstorage() but I don't > know if it will be ok for you. > > Has anybody ever got cgi.FieldStorage() to work inside medusa ? > > > --- In medusa@y..., Ernesto Nadir Crespo Avila wrote: > > > > > > El Mar 10 Abr 2001 12:24, escribiste: > > > It's been a while since I've workd with cgi.py but I believe your > problem > > > is in reading datos['name']. Try datos['name'].value instead. > > > datos['name'] may contain a list or string, so adapt your code > accordingly. > > Thank you. > > > > But, now just appear: > > > > there´s no data > > > > And > > > > FieldStorage(None, None, []) > > > > Bye, Ernesto Crespo > > > > > > # -*- Mode: Python; tab-width: 4 -*- > > # send and process a POST form > > import cgi > > > > print '' > > print '' > > print '' > > print > > print ' A Cgi example with Phyton' > > print '' > > print '' > > datos = cgi.FieldStorage() > > > > if datos.has_key('name'): > > print '

Your data: %s

' %datos['name'].value > > else: > > print '

there´s no data

' > > print datos > > print '' > > print '' > > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > From rprice@u... Tue Apr 17 00:35:29 2001 From: rprice@u... (rprice@u...) Date: Mon, 16 Apr 2001 23:35:29 -0000 Subject: medusa and apache proxy -more info In-Reply-To: <3AD68ACD.ED5BAAB9@n...> Message-ID: <9bfvk1+72vu@e...> With more testing we've found out that medusa works differenetly on Solaris (sunOS5) and Linux (I'm using Mandrake 7.1) at least when using the Linux telnet client: On solaris using telnet the server closes the connection immediately after the when using HTTP/1.0. When using HTTP/1.1 it will keep connected. Both as expected. On linux the Medusa server will wait 30 seconds before closing the session when using HTTP/1.0 and keeps the connection open much longer using HTTP/1.1. I've done these tests on Medusa versions 3.39 and 3.19 on linux and ONLY on 3.19 on sunOS5. Ray --- In medusa@y..., Sam Rushing wrote: > rprice@u... wrote: > > > When a user accesses medusa directly (not through a proxy) it takes less than 1 > > second to display the page. > > > > When a user accesses the same medusa server page via an apache proxy server, the > > pages seem to hang at 66% loaded. It can then take anywhere from 2 to 60 > > seconds for the page to finally load. Another problem when accessing through > > the proxy is that the browser occasionally indicates 'contacting server' for > > many seconds before any information is transmitted. > > > > Any ideas why this would happen? > > None yet... a couple of things to try: > > 1) what happens if you try squid instead of apache proxy? > 2) what protocol versions (i.e., http/1.0 or 1.1) are being used between browser, > proxy, and medusa? > > -Sam From abo@m... Tue Apr 17 02:27:51 2001 From: abo@m... (Donovan Baarda) Date: Tue, 17 Apr 2001 11:27:51 +1000 (EST) Subject: [medusa] Re: medusa and apache proxy -more info In-Reply-To: <9bfvk1+72vu@e...> References: <9bfvk1+72vu@e...> Message-ID: <987470871.3adb9c17659a5@m...> Quoting rprice@u...: > With more testing we've found out that medusa works differenetly on > Solaris (sunOS5) and Linux (I'm using Mandrake 7.1) at least when > using the Linux telnet client: > > On solaris using telnet the server closes the connection immediately > after the when using HTTP/1.0. When using HTTP/1.1 it will > keep connected. Both as expected. > > On linux the Medusa server will wait 30 seconds before closing the > session when using HTTP/1.0 and keeps the connection open much longer > using HTTP/1.1. The 30 sec delay sounds like the default medusa polling time on select. I don't know _why_ it would require another iteration through the loop for your application, but is it possible you have readable/writeable status changing around about the time you service the final push() for the connection. When the delay happens, it might not be quite ready the select, so it is excluded until the next iteration, when it gets selected immediately. Remember the readable/writeable test is performed once per iteration through the loop, so if something is not ready at the start of the loop, it will be excluded from the select until the next iteration. I suspect that if you change your polling time from the default, you can "adjust" the delay. Also, establishing another connection while one is "delayed" will kick the select loop, causing the delay to end immediately. Adjusting the poll time might be a quick fix, but I suspect you need to look at your readable/writeable tests. -- ABO: finger abo@m... for more information. From eugene.leitl@l... Tue Apr 17 08:21:09 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Tue, 17 Apr 2001 09:21:09 +0200 Subject: [medusa] Re: Trouble with the CGI References: <9bfse1+5bsd@e...> <987463009.3adb7d61cdbb3@w...> Message-ID: <3ADBEEE5.F9CEF8E3@l...> rprice@u... wrote: > > I have never used it from within medusa. I use the old bobo code to interact > with my Python scripts. Could you please mail me a tarball of your old bobo code? I've looked far and wide, and was unable to find it. I tried prying it from ZServer, but I couldn't understand it. From lucifer@e... Tue Apr 17 16:25:09 2001 From: lucifer@e... (lucifer@e...) Date: Tue, 17 Apr 2001 15:25:09 -0000 Subject: medusa and apache proxy -more info In-Reply-To: <9bfvk1+72vu@e...> Message-ID: <9bhn8l+cjom@e...> I don't know if this will lead to anything better but I have had some problems with apache proxy server too. I've built an html chat system over medusa and it works fine, but on our network, if my clients connect through an apache proxy it does not work as expected. The proxy seems to buffer a lot of data before transmitting it to the client. This behaviour has only been observed on apache proxy, no other proxy makes the system fail till now. --- In medusa@y..., rprice@u... wrote: > With more testing we've found out that medusa works differenetly on > Solaris (sunOS5) and Linux (I'm using Mandrake 7.1) at least when > using the Linux telnet client: > > On solaris using telnet the server closes the connection immediately > after the when using HTTP/1.0. When using HTTP/1.1 it will > keep connected. Both as expected. > > On linux the Medusa server will wait 30 seconds before closing the > session when using HTTP/1.0 and keeps the connection open much longer > using HTTP/1.1. > > I've done these tests on Medusa versions 3.39 and 3.19 on linux and > ONLY on 3.19 on sunOS5. > > Ray > > --- In medusa@y..., Sam Rushing wrote: > > rprice@u... wrote: > > > > > When a user accesses medusa directly (not through a proxy) it > takes less than 1 > > > second to display the page. > > > > > > When a user accesses the same medusa server page via an apache > proxy server, the > > > pages seem to hang at 66% loaded. It can then take anywhere from > 2 to 60 > > > seconds for the page to finally load. Another problem when > accessing through > > > the proxy is that the browser occasionally indicates 'contacting > server' for > > > many seconds before any information is transmitted. > > > > > > Any ideas why this would happen? > > > > None yet... a couple of things to try: > > > > 1) what happens if you try squid instead of apache proxy? > > 2) what protocol versions (i.e., http/1.0 or 1.1) are being used > between browser, > > proxy, and medusa? > > > > -Sam From rushing@n... Tue Apr 17 18:42:35 2001 From: rushing@n... (Sam Rushing) Date: Tue, 17 Apr 2001 10:42:35 -0700 Subject: [medusa] Re: medusa and apache proxy -more info References: <9bfvk1+72vu@e...> <987470871.3adb9c17659a5@m...> Message-ID: <3ADC808B.CEDEB472@n...> Donovan Baarda wrote: > I suspect that if you change your polling time from the default, you > can "adjust" the delay. Also, establishing another connection while one > is "delayed" will kick the select loop, causing the delay to end immediately. > Adjusting the poll time might be a quick fix, but I suspect you need to look at > your readable/writeable tests. Yup, methinks there's a problem with readable/writable. The easiest way to debug this kind of problem is to print the read/write sets just before the call to select(). I'd also attach a print of some kind to the socket close() of the http client... -Sam From rprice@u... Tue Apr 17 18:46:16 2001 From: rprice@u... (rprice@u...) Date: Tue, 17 Apr 2001 10:46:16 -0700 (PDT) Subject: [medusa] Re: medusa and apache proxy -more info In-Reply-To: <987470871.3adb9c17659a5@m...> References: <9bfvk1+72vu@e...> <987470871.3adb9c17659a5@m...> Message-ID: <987529576.3adc8168a65b9@w...> Quoting Donovan Baarda : > Quoting rprice@u...: > > > With more testing we've found out that medusa works differenetly on > > Solaris (sunOS5) and Linux (I'm using Mandrake 7.1) at least when > > using the Linux telnet client: > > > > On solaris using telnet the server closes the connection immediately > > after the when using HTTP/1.0. When using HTTP/1.1 it will > > keep connected. Both as expected. > > > > On linux the Medusa server will wait 30 seconds before closing the > > session when using HTTP/1.0 and keeps the connection open much longer > > using HTTP/1.1. > > The 30 sec delay sounds like the default medusa polling time on select. Yes it is. I temporarily 'fixed' the problem like this: asyncore.loop(timeout=.5) I just don't know how this well work when we have 100 concurrent users. > I don't > know _why_ it would require another iteration through the loop for your > application, but is it possible you have readable/writeable status > changing > around about the time you service the final push() for the connection. > When the > delay happens, it might not be quite ready the select, so it is excluded > until > the next iteration, when it gets selected immediately. Remember the > readable/writeable test is performed once per iteration through the > loop, so if > something is not ready at the start of the loop, it will be excluded > from the > select until the next iteration. > > I suspect that if you change your polling time from the default, you > can "adjust" the delay. Also, establishing another connection while one > is "delayed" will kick the select loop, causing the delay to end > immediately. Yes, another user jumping on will immediately close the connection to the prior user... if HTTP/1.0. Maybe I shoud increase the timeout to several seconds? > Adjusting the poll time might be a quick fix, but I suspect you need to > look at > your readable/writeable tests. > > -- > ABO: finger abo@m... for more information. > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > Thanks for your help, Ray From abo@m... Wed Apr 18 02:38:48 2001 From: abo@m... (Donovan Baarda) Date: Wed, 18 Apr 2001 11:38:48 +1000 (EST) Subject: asynchat producer ready() method patch. In-Reply-To: <20010417230813.91311.qmail@w...> References: <20010417230813.91311.qmail@w...> Message-ID: <987557928.3adcf0285f11c@m...> Quoting Ted Shab : > Hello, > > Saw this from the pipermail list. Has this patch been > applied to the standard python distribution or is it > pipermail specific? If the later, can you let me know > where I might find the fix (I'm not really interested > in pipermail). Hmmm, someone is using this patch? Wow, (warm fuzzies :-). I'm Cc'ing this to the medusa list incase anyone else is interested. AFAIK, no-one was using this patch apart from one person experimenting with it. There is nothing wrong with it, there just didn't seem to be much interest. I don't believe it has been applied anywhere apart from in my own projects. I'm hungry for any feedback on it and keen to support/promote it if there is interest. The patch is against medusa-src-20000601.tar.gz, and can be found in; ftp://minkirri.apana.org.au/pub/python/ Note that parts of medusa are distributed with python (asyncore.py and asynchat.py), and the relevant parts of the patch are designed to be compatible with the components in the python distribution (ie, there are no introduced dependancies and they are backwards compatible). I would like to think that the changes introduced also clean up the code a bit and provide a useful inheritance hirachy for producers. Performance heads might argue that the inheritance hurts performance. Comments about that are welcome too :-) The only tips about this patch from experience are; - be aware of how ready() is meant to work. It must return true whenever a producer is ready for more() to be called. That means when it has more data _or_ the producer is finished. Calling more() when a producer is not ready() is an error, and the producer base clases provided will raise an exception when this happens. - beware of producers that already have ready() methods. The original medusa had quite a few vestigial ready() methods that behaved incorrectly. The patch includes changes to fix/remove these methods in medusa. - a change in ready() status is not an event that will trigger select(). The ready() method is called before select() inside the main polling loop. This means a change in ready() status will not be immediately evaluated, but will be delayed until the next iteration through the loop. If no events trigger the select(), then this means it will be delayed untill the select() times out. The simple solution is to use a short timeout value. The complex solution is to make the change in ready() status trigger the select(), typicaly by putting an async object inside the producer. Note that this behaviour also affects the normal readable() and writeable() methods, but typicaly these change status as a result of something that triggers select() so you don't notice. -- ABO: finger abo@m... for more information. From abo@m... Wed Apr 18 02:39:25 2001 From: abo@m... (Donovan Baarda) Date: Wed, 18 Apr 2001 11:39:25 +1000 (EST) Subject: [medusa] Re: medusa and apache proxy -more info In-Reply-To: <987529576.3adc8168a65b9@w...> References: <9bfvk1+72vu@e...> <987470871.3adb9c17659a5@m...> <987529576.3adc8168a65b9@w...> Message-ID: <987557965.3adcf04de0a3c@m...> Quoting rprice@u...: > > Quoting Donovan Baarda : > > > Quoting rprice@u...: [...] > > The 30 sec delay sounds like the default medusa polling time on > select. > Yes it is. I temporarily 'fixed' the problem like this: > asyncore.loop(timeout=.5) I just don't know how this well work when we > have 100 > concurrent users. the beauty of the short timeout hack is it doesn't affect things when the load is high, only when it is low. At high loads, the select loop will be kicking around very fast so the timeout doesn't come into it (and any delays caused by poor readable()/writeable() implementation wont be noticed either). At low loads, the select loop will be mainly kicked around by the timeout. So a short timeout will increase the work for low loads, but will not increase the work at high loads. -- ABO: finger abo@m... for more information. From abo@m... Wed Apr 18 05:36:26 2001 From: abo@m... (Donovan Baarda) Date: Wed, 18 Apr 2001 14:36:26 +1000 (EST) Subject: asynchat producer ready() method patch. In-Reply-To: <20010418023428.19012.qmail@w...> References: <20010418023428.19012.qmail@w...> Message-ID: <987568586.3add19ca68aa6@m...> G'day all, in case I stuffed up the url and others are having the same problem finding it, I've Cc'd this to the medusa list. Quoting Ted Shab : > Mostly just interested in how you did this - is the > source to patch available - didn't see it on the site > you led me to. > > Thanks very much. The patch is a bzip2 compressed unified diff against medusa-src- 20000601.tar.gz, available at; ftp://minkirri.apana.org.au/pub/python/medusa-ready_0.3.patch.bz2 -- ABO: finger abo@m... for more information. From reynol@p... Wed Apr 18 18:00:34 2001 From: reynol@p... (Samuel Reynolds) Date: Wed, 18 Apr 2001 11:00:34 -0600 Subject: Medusa on Macintosh Message-ID: <5.1.0.14.0.20010418104349.00a48710@p...> ---------------------- multipart/mixed attachment Okay. I finally got back to the Mac-port/filesys.py changes testing. I've updated my modified filesys.py and tested it on MacOS 9, Windows NT, and SunOS (Solaris?) 5.7, and it works like a champ. I even dropped it into Zope/ZServer/medusa and ran Zope on top of it, without a hitch. The changes are as described in a previous post (http://groups.yahoo.com/group/medusa/message/286). I have made some additional tweaks to os_filesystem.translate, and improved the test script, so the files attached to the earlier message are outdated. While there is platform-specific code (unavoidable, I think), it is only in os_filesystem.translate. I've attached two files: o filesys.py -- The modified file, tested on MacOS, WinNT, and Sun. o filesys_test.py -- The test script. These files have Windows-style (\r\n) line endings, so you may need to convert them for your system (e.g., using dos2unix on Unices). Is anyone other than me interested in a Mac port and/or improving filesys.py? Or should I keep changes like this to myself in the future? If there is interest, I'd appreciate if others would try this out. (I renamed the original filesys.py to filesys_original.py and dropped the modified file in its place; that makes it easy to revert.) If you try it, I suggest you start by running the test script. It generates absolute (native file system) paths from standard relative paths within the Python tree for posix, Dos/Windows/OS2, or Mac, as appropriate, then exercises os_filesystem and compares the results to the reference paths. If you test on any other system, you may need to add a new block to generate the native absolute paths for your system, and any changes to filesys.py will affect only os_filesystem.translate. - Sam ---------------------- multipart/mixed attachment # filesys_test.py # # SCR 2001.04.17 # import os import filesys import string # Utility for converting path separator def ConvertSep( s, fromSep, toSep ): return string.join( string.split( s, fromSep ), toSep ) print "**** Testing filesys.os_filesystem ****" print "" # Get working directory workingDir = os.getcwd() #print "Working directory:", workingDir # Must be run from within Python directory if string.find( workingDir, "Python" ) < 0: raise RuntimeError, "Test must be run from within or below the Python main directory" # Determine Python directory path # We will use this as the root for the virtual filesystem. bits = string.split( workingDir, os.sep ) #print "***" #print bits #print "***" while string.find( bits[-1], "Python" ) < 0: bits = bits[:-1] pythonDir = string.join( bits, os.sep ) if os.name == 'mac': pythonDir = pythonDir + os.sep #print "pythonDir =", pythonDir #print " " # Create filesys for testing fs = filesys.os_filesystem( pythonDir, '/' ) # Pick some subpaths to work with. relPaths = ["Lib", "Lib/distutils", "Lib/distutils/command", "Lib/asyncore.py"] realPaths = [] # Pull out longest directory path for cd-up/cd-down testing deepPath = "" deepPathDepth = 0 for relPath in relPaths: if ( fs.isfile(relPath) ): continue tempL = string.split( relPath, '/' ) tempD = len(tempL) if ( tempD > deepPathDepth ): deepPathDepth = tempD deepPath = relPath # Since we're testing the conversion code, we have to # determine the absolute paths the hard way for comparison. if os.name == 'posix': # Just concatenate root & path for relPath in relPaths: realPaths = realPaths \ + [ pythonDir + sub_lib ] elif os.name == 'mac': # Explicitly add trailing separator for directory paths # (with prior knowledge that the last path is a file) onlyFilePath = relPaths[-1] for relPath in relPaths: temp = pythonDir + ConvertSep( relPath, '/', os.sep ) + os.sep if relPath == onlyFilePath: temp = temp[:-1] #if relPath == onlyFilePath: # realPaths = realPaths \ # + [ pythonDir + ConvertSep( relPath, '/', os.sep ) ] #else: # realPaths = realPaths \ # + [ pythonDir + ConvertSep( relPath, '/', os.sep ) + os.sep ] realPaths = realPaths + [ temp ] elif os.name == 'nt': # Explicitly add trailing separator to root directory path. for relPath in relPaths: realPaths = realPaths \ + [ pythonDir + os.sep + ConvertSep( relPath, '/', os.sep ) ] else: # os.name not specifically addressed. # This is a test error! raise RuntimeError, "Test paths not computed for os.name = " + os.name exit print "** Reference paths for test **" print "%-21s ==> %s" % ( "Python directory", pythonDir ) for n in range( len(relPaths) ): print "%-21s ==> %s" % ( relPaths[n], realPaths[n] ) print " " # Core test & statistics routines global passCount passCount = 0 global failCount failCount = 0 def CHKEQ( desc, testValue, refValue ): global passCount global failCount passesTest = ( testValue == refValue ) if passesTest: print "CHKEQ: %-50s - PASS" % desc passCount += 1 else: print "CHKEQ: %-50s - FAIL" % desc print "\tExpected:", refValue print "\tActual :", testValue failCount += 1 return ( passesTest ) def CHKNE( desc, testValue, refValue ): global passCount global failCount passesTest = ( testValue != refValue ) if passesTest: print "CHKNE: %-50s - PASS" % desc passCount += 1 else: print "CHKNE: %-50s - FAIL" % desc print "\tValue:", testValue failCount += 1 return ( passesTest ) def CHKPASS( desc ): global passCount print "CHKPASS: %-50s - PASS" % desc passCount += 1 def CHKFAIL( desc ): global failCount print "CHKFAIL: %-50s - FAIL" % desc failCount += 1 #### Translation from virtual to absolute path def CHK_TRANSLATE( subPath, refValue ): desc = "%-25s ==> absolute" % subPath temp = fs.translate( subPath ) CHKEQ( desc, temp, refValue ) for n in range( len(relPaths) ): CHK_TRANSLATE( relPaths[n], realPaths[n] ) #### Relative movement within directory structure # First, check our starting point CHKEQ( "Initial virtual directory", fs.current_directory(), "/" ) # From '/', cwd('/') should leave us where we are fs.cwd( '/' ) CHKEQ( "After cwd('/')", fs.current_directory(), "/" ) # CD into sub_lib, then back to top for relPath in relPaths: fs.cwd( relPath ) if ( fs.isfile( relPath) ): # Shouldn't change working directory CHKEQ( "After cwd('"+relPath+"')", fs.current_directory(), "/" ) else: # Should have changed working directory CHKEQ( "After cwd('"+relPath+"')", fs.current_directory(), "/" + relPath ) # Change working directory back to root fs.cwd('/') CHKEQ( "After cwd('/')", fs.current_directory(), "/" ) # Create up/down paths list bits = string.split( deepPath, '/' ) upDownPaths1 = [] upDownPaths2 = [] onePath = "/" for bit in bits: upDownPaths1 += [ bit ] onePath += bit upDownPaths2 += [ onePath ] onePath += '/' # Start at virtual root fs.cwd( '/' ) # CD up one level, and up two levels (should not move) fs.cwd( ".." ) CHKEQ( "After cwd('..') from '/' ==> '/'", fs.current_directory(), "/" ) fs.cwd( '/' ) fs.cwd( "../.." ) CHKEQ( "After cwd('../..') from '/' ==> '/'", fs.current_directory(), "/" ) print "------------" step = 1 for idx in range(0, len(upDownPaths1), step): fs.cwd( upDownPaths1[idx] ); CHKEQ( "After cwd('"+upDownPaths1[idx]+"')", fs.current_directory(), \ upDownPaths2[idx] ) fs.cwd('/') # CD into sub_lib_distutils_cmd as virtual "absolute" path #fs.cwd( '/' + sub_lib_distutils_cmd ) #CHKEQ( "After cwd('/"+sub_lib_distutils_cmd+"')", \ # fs.current_directory(), "/" + sub_lib_distutils_cmd ) # CD up two levels #fs.cwd( "../.." ) #CHKEQ( "After cwd('../..')", \ # fs.current_directory(), "/" + sub_lib ) # CD back to top #fs.cwd('/') #CHKEQ( "After cwd('/')", fs.current_directory(), "/" ) # Try to CD above root #fs.cwd('/') #fs.cwd('..') #CHKEQ( "After cwd('..') when wd='/'", fs.current_directory(), "/" ) # CD up two levels above root #fs.cwd('/') #fs.cwd( "../.." ) #CHKEQ( "After cwd('../..') when wd='/'", fs.current_directory(), "/" ) # CD up past root from subdirectory #fs.cwd('/' + sub_lib) ##CHKEQ( "After cwd('../..') when wd='/" + sub_lib + "'", \ # fs.current_directory(), "/" ) #### Informational Tests # isfile/isdir for relPath in relPaths: if ( fs.isfile(relPath) ): CHKEQ( "%-30s isdir:%d isfile:%d" % \ (relPath, fs.isdir(relPath), fs.isfile(relPath)), \ "d:%d f:%d" % (fs.isdir(relPath), fs.isfile(relPath)), \ "d:0 f:1" ) else: CHKEQ( "%-30s isdir:%d isfile:%d" % \ (relPath, fs.isdir(relPath), fs.isfile(relPath)), \ "d:%d f:%d" % (fs.isdir(relPath), fs.isfile(relPath)), \ "d:1 f:0" ) # exists for relPath in relPaths: # Real paths CHKEQ( "%-30s exists:%d" % ( relPath, fs.exists(relPath) ), fs.exists(relPath), 1 ) # Fake paths CHKEQ( "%-30s exists:%d" % ( relPath+'_XX', fs.exists(relPath+'_XX') ), fs.exists(relPath+'_XX'), 0 ) # stat for relPath in relPaths: # Real paths try: temp = fs.stat( relPath ) CHKEQ( "stat('%s') returns tuple" % relPath, type(temp), type(()) ) except: CHKFAIL( "stat('%s') returns tuple" % relPath ) # Fake paths try: temp = fs.stat( relPath+'_XX' ) CHKEQ( "stat('%s') throws" % (relPath+'_XX'), type(temp), None ) except: CHKPASS( "stat('%s') throws" % (relPath+'_XX') ) print "" print ' ' + '=' * 11 print " PASS: %5d" % passCount print " FAIL: %5d" % failCount print ' ' + '=' * 11 print "" ---------------------- multipart/mixed attachment # -*- Mode: Python; tab-width: 4 -*- # $Id: filesys.py,v 1.9 1998/06/22 05:36:05 rushing Exp $ # Author: Sam Rushing # # Generic filesystem interface. # # We want to provide a complete wrapper around any and all # filesystem operations. # this class is really just for documentation, # identifying the API for a filesystem object. # opening files for reading, and listing directories, should # return a producer. import posixpath class abstract_filesystem: def __init__ (self): pass def current_directory (self): "Return a string representing the current directory." pass def listdir (self, path, long=0): """Return a listing of the directory at 'path' The empty string indicates the current directory. If 'long' is set, instead return a list of (name, stat_info) tuples """ pass def open (self, path, mode): "Return an open file object" pass def stat (self, path): "Return the equivalent of os.stat() on the given path." pass def isdir (self, path): "Does the path represent a directory?" pass def isfile (self, path): "Does the path represent a plain file?" pass def cwd (self, path): "Change the working directory." pass def cdup (self): "Change to the parent of the current directory." pass def longify (self, path): """Return a 'long' representation of the filename [for the output of the LIST command]""" pass # standard wrapper around a unix-like filesystem, with a 'false root' # capability. # security considerations: can symbolic links be used to 'escape' the # root? should we allow it? if not, then we could scan the # filesystem on startup, but that would not help if they were added # later. We will probably need to check for symlinks in the cwd method. # what to do if wd is an invalid directory? import os import stat import string def safe_stat (path): try: return (path, os.stat (path)) except: return None import regex import regsub import glob class os_filesystem: path_module = os.path # set this to zero if you want to disable pathname globbing. # [we currently don't glob, anyway] do_globbing = 1 def __init__ (self, root, wd='/'): self.root = root self.wd = wd ### Provide replacements for path methods def abspath (self, path): return self.normalize (posixpath.join (self.wd, path)) def basename (self, path): return posixpath.basename (path) def commonprefix (self, pathlist): return posixpath.commonprefix (pathlist) def dirname (self, path): return posixpath.dirname (path) def exists (self, path): p = self.abspath (path) return self.path_module.exists (self.translate(p)) def expanduser (self, path): p = self.abspath (path) return self.path_module.expanduser (self.translate(p)) def expandvars (self, path): p = self.abspath (path) return self.path_module.expandvars (self.translate(p)) def getatime (self, path): p = self.abspath (path) return self.path_module.getatime (self.translate(p)) def getmtime (self, path): p = self.abspath (path) return self.path_module.getmtime (self.translate(p)) def getsize (self, path): p = self.abspath (path) return self.path_module.getsize (self.translate(p)) def isabs (self, path): return posixpath.isabs (path) def isdir (self, path): p = self.abspath (path) return self.path_module.isdir (self.translate(p)) def isfile (self, path): p = self.abspath (path) return self.path_module.isfile (self.translate(p)) def islink (self, path): p = self.abspath (path) return self.path_module.islink (self.translate(p)) def ismount (self, path): p = self.abspath (path) return self.path_module.ismount (self.translate(p)) def normcase (self, path): return posixpath.normcase (path) def normpath (self, path): return posixpath.normpath (path) def samefile (self, p1, p2): p1 = self.abspath (p1) p2 = self.abspath (p2) return self.path_module.samefile (self.translate(p1), self.translate(p2)) def sameopenfile (self, fp1, fp2): return self.path_module.sameopenfile (fp1, fp2) def samestat (self, s1, s2): return self.path_module.samestat (s1, s2) def split (self, path): return posixpath.split (path) def stat (self, path): p = self.abspath (path) p = self.translate(p) st = os.stat (p) return st # splitdrive NOT IMPLEMENTED def splittext (self, path): return posixpath.splittext (path) ### Provide Unix filesystem commands def cwd (self, path): p = self.abspath (path) translated_path = self.translate(p) if not self.path_module.isdir (translated_path): return 0 else: old_dir = os.getcwd() # temporarily change to that directory, in order # to see if we have permission to do so. try: can = 0 try: os.chdir (translated_path) can = 1 self.wd = p except: pass finally: if can: os.chdir (old_dir) return can def cdup (self): return self.cwd ('..') def current_directory (self): return self.wd def listdir (self, path, long=0): p = self.translate (path) # I think we should glob, but limit it to the current # directory only. ld = os.listdir (p) if not long: return list_producer (ld, 0, None) else: old_dir = os.getcwd() try: os.chdir (p) # if os.stat fails we ignore that file. result = filter (None, map (safe_stat, ld)) finally: os.chdir (old_dir) return list_producer (result, 1, self.longify) # TODO: implement a cache w/timeout for stat() def open (self, path, mode): p = self.translate (path) return open (p, mode) def unlink (self, path): p = self.translate (path) return os.unlink (p) def mkdir (self, path): p = self.translate (path) return os.mkdir (p) def rmdir (self, path): p = self.translate (path) return os.rmdir (p) # utility methods def normalize (self, path): # watch for the ever-sneaky '/+' path element path = regsub.gsub ('/+', '/', path) p = posixpath.normpath (path) # remove 'dangling' cdup's. if len(p) > 2 and p[:3] == '/..': p = '/' return p def translate (self, path): # Get full (virtual-unix) path p = self.abspath (path) # Convert to absolute path recognizable by current OS if ( os.name == 'posix' ): if ( p[:1] == os.sep ): p = p[1:] p = self.path_module.join( self.root, p ) else: # Make platform-specific pre-adjustments, if any if ( os.sep != '/' ): p = string.join( string.split( p, '/' ), os.sep ) if (os.name == 'nt' or os.name == 'dos' or os.name == 'os2'): if ( p[:1] == os.sep ): p = p[1:] p = self.path_module.join( self.root, p ) # Make platform-specific post-adjustments, if any if (os.name == 'mac'): if self.path_module.isdir( p ) and ( p[-1] <> os.sep ): p = p + os.sep return p def longify (self, (path, stat_info)): return unix_longify (path, stat_info) def __repr__ (self): return '' % ( self.root, self.wd ) if os.name == 'posix': class unix_filesystem (os_filesystem): pass class schizophrenic_unix_filesystem (os_filesystem): PROCESS_UID = os.getuid() PROCESS_EUID = os.geteuid() PROCESS_GID = os.getgid() PROCESS_EGID = os.getegid() def __init__ (self, root, wd='/', persona=(None, None)): os_filesystem.__init__ (self, root, wd) self.persona = persona def become_persona (self): if self.persona is not (None, None): uid, gid = self.persona # the order of these is important! os.setegid (gid) os.seteuid (uid) def become_nobody (self): if self.persona is not (None, None): os.seteuid (self.PROCESS_UID) os.setegid (self.PROCESS_GID) # cwd, cdup, open, listdir def cwd (self, path): try: self.become_persona() return os_filesystem.cwd (self, path) finally: self.become_nobody() def cdup (self, path): try: self.become_persona() return os_filesystem.cdup (self) finally: self.become_nobody() def open (self, filename, mode): try: self.become_persona() return os_filesystem.open (self, filename, mode) finally: self.become_nobody() def listdir (self, path, long=0): try: self.become_persona() return os_filesystem.listdir (self, path, long) finally: self.become_nobody() # This hasn't been very reliable across different platforms. # maybe think about a separate 'directory server'. # # import posixpath # import fcntl # import FCNTL # import select # import asyncore # # # pipes /bin/ls for directory listings. # class unix_filesystem (os_filesystem): # pass # path_module = posixpath # # def listdir (self, path, long=0): # p = self.translate (path) # if not long: # return list_producer (os.listdir (p), 0, None) # else: # command = '/bin/ls -l %s' % p # print 'opening pipe to "%s"' % command # fd = os.popen (command, 'rt') # return pipe_channel (fd) # # # this is both a dispatcher, _and_ a producer # class pipe_channel (asyncore.file_dispatcher): # buffer_size = 4096 # # def __init__ (self, fd): # asyncore.file_dispatcher.__init__ (self, fd) # self.fd = fd # self.done = 0 # self.data = '' # # def handle_read (self): # if len (self.data) < self.buffer_size: # self.data = self.data + self.fd.read (self.buffer_size) # #print '%s.handle_read() => len(self.data) == %d' % (self, len(self.data)) # # def handle_expt (self): # #print '%s.handle_expt()' % self # self.done = 1 # # def ready (self): # #print '%s.ready() => %d' % (self, len(self.data)) # return ((len (self.data) > 0) or self.done) # # def more (self): # if self.data: # r = self.data # self.data = '' # elif self.done: # self.close() # self.downstream.finished() # r = '' # else: # r = None # #print '%s.more() => %s' % (self, (r and len(r))) # return r # For the 'real' root, we could obtain a list of drives, and then # use that. Doesn't win32 provide such a 'real' filesystem? # [yes, I think something like this "\\.\c\windows"] class msdos_filesystem (os_filesystem): def longify (self, (path, stat_info)): return msdos_longify (path, stat_info) # A merged filesystem will let you plug other filesystems together. # We really need the equivalent of a 'mount' capability - this seems # to be the most general idea. So you'd use a 'mount' method to place # another filesystem somewhere in the hierarchy. # Note: this is most likely how I will handle ~user directories # with the http server. class merged_filesystem: def __init__ (self, *fsys): pass # this matches the output of NT's ftp server (when in # MSDOS mode) exactly. def msdos_longify (file, stat_info): if stat.S_ISDIR (stat_info[stat.ST_MODE]): dir = '' else: dir = ' ' date = msdos_date (stat_info[stat.ST_MTIME]) return '%s %s %8d %s' % ( date, dir, stat_info[stat.ST_SIZE], file ) def msdos_date (t): try: info = time.gmtime (t) except: info = time.gmtime (0) # year, month, day, hour, minute, second, ... if info[3] > 11: merid = 'PM' info[3] = info[3] - 12 else: merid = 'AM' return '%02d-%02d-%02d %02d:%02d%s' % ( info[1], info[2], info[0]%100, info[3], info[4], merid ) months = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'] mode_table = { '0':'---', '1':'--x', '2':'-w-', '3':'-wx', '4':'r--', '5':'r-x', '6':'rw-', '7':'rwx' } import time def unix_longify (file, stat_info): # for now, only pay attention to the lower bits mode = ('%o' % stat_info[stat.ST_MODE])[-3:] mode = string.join (map (lambda x: mode_table[x], mode), '') if stat.S_ISDIR (stat_info[stat.ST_MODE]): dirchar = 'd' else: dirchar = '-' date = ls_date (long(time.time()), stat_info[stat.ST_MTIME]) return '%s%s %3d %-8d %-8d %8d %s %s' % ( dirchar, mode, stat_info[stat.ST_NLINK], stat_info[stat.ST_UID], stat_info[stat.ST_GID], stat_info[stat.ST_SIZE], date, file ) # Emulate the unix 'ls' command's date field. # it has two formats - if the date is more than 180 # days in the past, then it's like this: # Oct 19 1995 # otherwise, it looks like this: # Oct 19 17:33 def ls_date (now, t): try: info = time.gmtime (t) except: info = time.gmtime (0) # 15,600,000 == 86,400 * 180 if (now - t) > 15600000: return '%s %2d %d' % ( months[info[1]-1], info[2], info[0] ) else: return '%s %2d %02d:%02d' % ( months[info[1]-1], info[2], info[3], info[4] ) # =========================================================================== # Producers # =========================================================================== class list_producer: def __init__ (self, file_list, long, longify): self.file_list = file_list self.long = long self.longify = longify self.done = 0 def ready (self): if len(self.file_list): return 1 else: if not self.done: self.done = 1 return 0 return (len(self.file_list) > 0) # this should do a pushd/popd def more (self): if not self.file_list: return '' else: # do a few at a time bunch = self.file_list[:50] if self.long: bunch = map (self.longify, bunch) self.file_list = self.file_list[50:] return string.joinfields (bunch, '\r\n') + '\r\n' ---------------------- multipart/mixed attachment-- From eugene.leitl@l... Mon Apr 23 14:34:19 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Mon, 23 Apr 2001 15:34:19 +0200 (MET DST) Subject: XML-RPC with a http server Message-ID: Um, I just realized something embarrassing. Please correct me if I'm spouting nonsense. I want to run a janusian server based on medusa (pardon the mythology melange): talking to the users HTML via a http port, and via XML-RPC to other services, running on the same machine, or different ones. Now the wunderbar Medusa usually uses something like this, when starting a server (this ripped from xmlrpc_handler.py): class rpc_demo (xmlrpc_handler): def call (self, method, params): print 'method="%s" params=%s' % (method, params) return "Sure, that works" import asyncore import http_server hs = http_server.http_server ('', 8000) rpc = rpc_demo() hs.install_handler (rpc) asyncore.loop() There can be only one handler in the asyncore.loop(), correct? I can't run medusa serving web pages (applets, cgi-bin python) at one end, and have an xml-rpc running within the same binary, on an alternative port? Or can I have medusa serving html on, say, port 80, and the xml-rpc handler be automatically invoked, when a POST /RPC2 HTTP/1.0 request comes in on same standard port? I'm a bit fuzzy on the concepts, somebody please clue me in. From rushing@n... Mon Apr 23 17:33:05 2001 From: rushing@n... (Sam Rushing) Date: Mon, 23 Apr 2001 09:33:05 -0700 Subject: [medusa] XML-RPC with a http server References: Message-ID: <3AE45941.FEA61EE@n...> Eugene Leitl wrote: > There can be only one handler in the asyncore.loop(), correct? I can't > run medusa serving web pages (applets, cgi-bin python) at one end, and > have an xml-rpc running within the same binary, on an alternative port? The 'handler' is an http_server-specific thing. You install multiple 'handlers' on the http server to map to different URI's. (they're usually matched using a regular expression). You can have as many different kinds of servers as you like running. I've used servers that spoke 4 or 5 different protocols. The easiest way to do that of course, is to have each one on a different port. 8^) > Or can I have medusa serving html on, say, port 80, and the xml-rpc > handler be automatically invoked, when a POST /RPC2 HTTP/1.0 request comes > in on same standard port? It should just work as you expect. Just call 'hs.install_handler' in whatever order makes sense. The handlers are maintained in the order they are installed, and they're tried in that order. This allows you to install a 'match-all' handler at the end of the chain. Take a look at the 'match' method of the xmlrpc_handler... -Sam From mj@d... Wed Apr 25 15:59:45 2001 From: mj@d... (mj@d...) Date: Wed, 25 Apr 2001 14:59:45 -0000 Subject: Bug in log format Message-ID: <9c6op1+lmcc@e...> Hi all, Currently, the file logging writes all remote addreses, wether DNS resolved or not, with a trailing colon (':'): (address): - - [(date)] "(request)" (status) (bytes) The colon does not belong there, at least, not according to the CLF specs. Webalizer will not resolve ip addresses because it doesn't recognize them as ips, for example. The colon seems to be a remnant of a change made in reference to a post to this list in 1999: http://groups.yahoo.com/group/medusa/message/33 which removed the client port number from the log, but *not* the colon that separates the two. The following patch removes the colon: --- logger.py 2000/06/02 15:36:39 1.8 +++ logger.py 2001/04/25 14:57:23 @@ -227,7 +227,7 @@ def __call__ (self, host, ttl, answer): if not answer: answer = host - self.logger.log ('%s:%s' % (answer, self.message)) + self.logger.log ('%s%s' % (answer, self.message)) def log (self, ip, message): self.resolver.resolve_ptr ( @@ -244,7 +244,7 @@ self.logger = logger def log (self, ip, message): - self.logger.log ('%s:%s' % (ip, message)) + self.logger.log ('%s%s' % (ip, message)) def strip_eol (line): Martijn Pieters Digital Creations From johs@c... Sat Apr 28 09:33:39 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Sat, 28 Apr 2001 10:33:39 +0200 Subject: Medusa, Python 2.0. Message-ID: <20010428103339.A2991@u...> Seems Medusa doesn't quite like Python 2.0. Some library incompatibilities. The auth-handler, for example, excepts re-expressions to have a group-function, which it did, in 1.5.2, but not anymore. (Yes, I know, the regex-module in 2.0 gives me this.) Anyway, is someone working on making it Python 2.0-compatible? -- johs From fredrik@p... Sat Apr 28 10:00:54 2001 From: fredrik@p... (Fredrik Lundh) Date: Sat, 28 Apr 2001 11:00:54 +0200 Subject: [medusa] Medusa, Python 2.0. References: <20010428103339.A2991@u...> Message-ID: <009601c0cfc1$c09aebb0$e46940d5@h...> Johannes Grřdem wrote: > Seems Medusa doesn't quite like Python 2.0. Some library incompatibilities. > The auth-handler, for example, excepts re-expressions to have a > group-function, which it did, in 1.5.2, but not anymore. there has never been a group function in the re module. there's a group *method* on match objects, but it's still there in 2.0. Cheers /F Python 1.5.2 (#0, May 9 2000, 14:04:03) >>> import re >>> re.group() Traceback (innermost last): File "", line 1, in ? AttributeError: group >>> p = re.compile("x") >>> p.group() Traceback (innermost last): File "", line 1, in ? AttributeError: group >>> m = p.match("x") >>> m.group() 'x' Python 2.0 (#8, Jan 4 2001, 12:33:41) >>> import re >>> re.group Traceback (most recent call last): File "", line 1, in ? AttributeError: group >>> p = re.compile("x") >>> p.group() Traceback (most recent call last): File "", line 1, in ? AttributeError: group >>> m = p.match("x") >>> m.group() 'x' From johs@c... Sat Apr 28 09:50:58 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Sat, 28 Apr 2001 10:50:58 +0200 Subject: Medusa, Python 2.0. Message-ID: <20010428105058.B2991@u...> > Seems Medusa doesn't quite like Python 2.0. Some library incompatibilities. > The auth-handler, for example, excepts re-expressions to have a > group-function, which it did, in 1.5.2, but not anymore. (Yes, I know, the > regex-module in 2.0 gives me this.) Um. The re-module in 1.5.2 didn't do this, I see, upon closer inspection. So it's just a bug, then? -- johs From rushing@n... Sat Apr 28 10:32:39 2001 From: rushing@n... (Sam Rushing) Date: Sat, 28 Apr 2001 02:32:39 -0700 Subject: medusa minus regex References: <20010428103339.A2991@u...> Message-ID: <3AEA8E37.AAA103E2@n...> Johannes Grřdem wrote: > Seems Medusa doesn't quite like Python 2.0. Some library incompatibilities. > The auth-handler, for example, excepts re-expressions to have a > group-function, which it did, in 1.5.2, but not anymore. (Yes, I know, the > regex-module in 2.0 gives me this.) > > Anyway, is someone working on making it Python 2.0-compatible? I made a first stab at it last week or so. Haven't heard any complaints yet... http://www.nightmare.com/medusa/medusa-20010416.tar.gz There were a lot more changes required than I thought, so caveat upgrader! -Sam From johs@c... Sun Apr 29 15:22:54 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Sun, 29 Apr 2001 16:22:54 +0200 Subject: URL-parsing. Message-ID: <20010429162254.A13375@u...> The crack_request-function in http_server.py breaks when URLs contain spaces. I'm not really too steady in regexps, so I'm not quite sure how to best fix it. -- johs From fredrik@p... Sun Apr 29 19:49:36 2001 From: fredrik@p... (Fredrik Lundh) Date: Sun, 29 Apr 2001 20:49:36 +0200 Subject: [medusa] URL-parsing. References: <20010429162254.A13375@u...> Message-ID: <009401c0d0dd$26086c90$e46940d5@h...> Johannes Grřdem wrote: > The crack_request-function in http_server.py breaks when URLs contain spaces. > I'm not really too steady in regexps, so I'm not quite sure how to best fix it. according to RFC 2396, URLs cannot contain spaces: "The space character is excluded because significant spaces may disappear and insignificant spaces may be introduced when URI are transcribed or typeset or subjected to the treatment of word- processing programs. Whitespace is also used to delimit URI in many contexts." or do you mean that it doesn't work if the URI contains %20 ? Cheers /F From johs@c... Mon Apr 30 11:50:18 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Mon, 30 Apr 2001 12:50:18 +0200 Subject: URL-parsing. Message-ID: <20010430125018.A7046@u...> > or do you mean that it doesn't work if the URI contains %20? I tried with %20, too, but it seems crack_request doesn't receive %20. The %20 is converted to a space before it reaches crack_request. -- johs From johs@c... Thu May 3 14:57:06 2001 From: johs@c... (=?iso-8859-1?Q?Johannes_Gr=F8dem?=) Date: Thu, 3 May 2001 15:57:06 +0200 Subject: URL-parsing. Message-ID: <20010503155705.A5594@u...> >> or do you mean that it doesn't work if the URI contains %20? > I tried with %20, too, but it seems crack_request doesn't receive %20. > The %20 is converted to a space before it reaches crack_request. Ok, I haven't read the RFC thoroughly, but wouldn't this be the correct way to do it? --- http_server.py.old Thu May 3 15:34:28 2001 +++ http_server.py Thu May 3 15:34:45 2001 @@ -457,10 +457,10 @@ # unquote path if necessary (thanks to Skip Montaro for pointing # out that we must unquote in piecemeal fashion). - if '%' in request: - request = unquote (request) command, uri, version = crack_request (request) + if '%' in uri: + uri = unquote (uri) header = join_headers (lines[1:]) r = http_request (self, request, command, uri, version, header) From weissblitz@y... Thu May 3 16:35:26 2001 From: weissblitz@y... (Weiss Blitz) Date: Thu, 03 May 2001 15:35:26 -0000 Subject: Looking for an XML to Python sequence source code Message-ID: <9crtru+lv6c@e...> Does anyone knows of any Python code to convert from XML to a Python sequence object? Like XML to a list? From akuchlin@m... Thu May 3 17:15:01 2001 From: akuchlin@m... (Andrew Kuchling) Date: Thu, 3 May 2001 12:15:01 -0400 Subject: [medusa] Looking for an XML to Python sequence source code In-Reply-To: <9crtru+lv6c@e...>; from weissblitz@y... on Thu, May 03, 2001 at 03:35:26PM -0000 References: <9crtru+lv6c@e...> Message-ID: <20010503121501.A17964@u...> On Thu, May 03, 2001 at 03:35:26PM -0000, Weiss Blitz wrote: >Does anyone knows of any Python code to convert from XML to a Python >sequence object? Like XML to a list? The PyXML package includes an xml.marshal subpackage with a bunch of different marshallers: a generic one, WDDX, and XML-RPC. For example: >>> from xml.marshal import generic >>> print generic.dumps( [1.0, pow(2, 75L), 'this is a string'] ) 1.0 37778931862957161709568this is a string >>> --amk From skip@p... Sun May 6 19:20:02 2001 From: skip@p... (skip@p...) Date: Sun, 6 May 2001 13:20:02 -0500 Subject: SIGRT_0 signal caught by ZServer/medusa? Message-ID: <15093.38354.488789.63286@b...> I'm using ZServer from the Zope 2.2.0 distribution as a backend to Apache on a Linux machine. The architecture (if you can call it that anymore) of the system is this: apache <--http proxy--> zserver <--xmlrpc--> database server Actually, behind the database server sits the real database - MySQL, but I don't think it's germane to this discussion. Performance of this system is sometimes abysmal at times when I can't account for it by some cause like system load. I monitor things from time-to-time using tools like strace. Once started, there are typically four process ids associated with zserver, presumably four different threads. I can watch the thread that's doing all the socket activity: select(19, [9 14 18], [], [], {10, 0}) = 1 (in [14], left {9, 970000}) read(14, "x", 8192) = 1 select(19, [9 14 18], [18], [], {10, 0}) = 1 (out [18], left {10, 0}) send(18, "HTTP/1.0 200 OK\r\nServer: Zope/Zo"..., 7087, 0) = 7087 ... Every so often, strace quits with something like this: kill(11754, SIGRT_0) = 0 select(15, [9 10 14], [], [], {10, 0}) = 1 (in [9], left {1, 530000}) rt_sigprocmask(SIG_SETMASK, NULL, [RT_0], 8) = 0 rt_sigsuspend([] --- SIGRT_0 (Real-time signal 0) --- The strace'd process doesn't actually croak, however. Frequently, the kill call is executed without terminating strace. If I strace the database server (also written in Python, but not using medusa), I never see the kill calls or the strace termination, so my naive assumption is that somehow the behavior I see in ZServer is somehow Zope- or Medusa-related. Any idea what this mysterious SIGRT_0 signal is (I can see it's a "real-time signal", but that doesn't tell me much) and why it would be getting sent to my ZServer? I can't find mention of it in the man pages. Thanks, -- Skip Montanaro (skip@p...) (847)971-7098 From eugene.leitl@l... Thu May 10 17:06:37 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Thu, 10 May 2001 18:06:37 +0200 (MET DST) Subject: Medusa: global name 'split_path' is not defined: file: status_handler.pyline: 63 Message-ID: This is Python 2.1 and medusa-20010416.tar.gz I'm getting error: Server Error: exceptions.NameError, global name 'split_path' is not defined: file: status_handler.py line: 63 when trying to access http://localhost:8080/status I'm also having trouble getting Bobo running, which also tells me stuff about split_path: Traceback (most recent call last): File "start_medusa.py", line 110, in ? import bobo_handler File "bobo_handler.py", line 58, in ? split_path = default_handler.split_path AttributeError: 'default_handler' module has no attribute 'split_path' Any idea what is going on? Something changed in 2.1? From cm@b... Sun May 13 05:09:16 2001 From: cm@b... (cm@b...) Date: Sun, 13 May 2001 04:09:16 -0000 Subject: Gracefully quitting Medusa? Message-ID: <9dl1dc+cqqh@e...> ^C seems a bit harsh. Is there a better way? Thanks :) From eugene.leitl@l... Mon May 14 15:34:22 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Mon, 14 May 2001 14:34:22 -0000 Subject: Medusa: global name 'split_path' is not defined: file: status_handler.pyline: 63 In-Reply-To: Message-ID: <9doqde+cho0@e...> --- In medusa@y..., Eugene Leitl wrote: > > This is Python 2.1 and medusa-20010416.tar.gz > > I'm getting > > error: Server Error: exceptions.NameError, global name 'split_path' is > not defined: file: status_handler.py line: 63 > > when trying to access http://localhost:8080/status > > I'm also having trouble getting Bobo running, which also tells me > stuff about split_path: > > Traceback (most recent call last): > File "start_medusa.py", line 110, in ? > import bobo_handler > File "bobo_handler.py", line 58, in ? > split_path = default_handler.split_path > AttributeError: 'default_handler' module has no attribute 'split_path' > > > Any idea what is going on? Something changed in 2.1? From eugene.leitl@l... Mon May 14 15:33:53 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Mon, 14 May 2001 14:33:53 -0000 Subject: Medusa: global name 'split_path' is not defined: file: status_handler.pyline: 63 In-Reply-To: Message-ID: <9doqch+kjvq@e...> Er. Disregard my last message (well, ok, you already did that ;) I went back to Python 1.5.2 and medusa-src-20000601.tar.gz and everything works. Bobo included. /status even. And it freezes vewy vewy nicely. Under ancient AIX, to wit. I've got a form with a submit button to supply arguments to hello.py (see bobo-howto). I'm happy. (Now for the hard part: application front end and XML-RPC back end. Urgh. I feel I need a vacation already). I don't know what weird combinations of versions I tried before which broke all over the palce, but above combination should work. Probably. Rushing, you made another guy out there very happy. Medusa r0xx, officially. --- In medusa@y..., Eugene Leitl wrote: > > This is Python 2.1 and medusa-20010416.tar.gz > > I'm getting > > error: Server Error: exceptions.NameError, global name 'split_path' is > not defined: file: status_handler.py line: 63 > > when trying to access http://localhost:8080/status > > I'm also having trouble getting Bobo running, which also tells me > stuff about split_path: > > Traceback (most recent call last): > File "start_medusa.py", line 110, in ? > import bobo_handler > File "bobo_handler.py", line 58, in ? > split_path = default_handler.split_path > AttributeError: 'default_handler' module has no attribute 'split_path' > > > Any idea what is going on? Something changed in 2.1? From rushing@n... Mon May 14 17:33:58 2001 From: rushing@n... (Sam Rushing) Date: Mon, 14 May 2001 09:33:58 -0700 Subject: [medusa] Re: Medusa: global name 'split_path' is not defined: file: status_handler.pyline: 63 References: <9doqch+kjvq@e...> Message-ID: <3B0008F6.E296A809@n...> eugene.leitl@l... wrote: > Er. Disregard my last message (well, ok, you already did that ;) > [...] Oops, sorry about that. 8^) > > --- In medusa@y..., Eugene Leitl wrote: > > > > This is Python 2.1 and medusa-20010416.tar.gz > > > > I'm getting > > > > error: Server Error: exceptions.NameError, global name 'split_path' > is > > not defined: file: status_handler.py line: 63 > > The 'split_path' function (which was copied around to several other modules) was replaced with a 'split_uri()' method on the request object, which also saves a little effort by memoizing the result. To fix, just remove all references to the split_path function, changing " = split_path (path)" to " = request.split_uri (path)" Let me know if you have any trouble with this... -Sam From weissblitz@y... Wed May 16 22:19:51 2001 From: weissblitz@y... (Weiss Blitz) Date: Wed, 16 May 2001 21:19:51 -0000 Subject: Asyncore's socket 'send' method not found?! Message-ID: <9duqtn+710s@e...> My Medusa server is crashing with the following error: (exceptions.AttributeError:send [asyncore.py|poll|79] [asyncore.py|handle_read_event|337] [asynchat.py|handle_read|128] [zulu_pager_server.py|found_terminator|192] [zulu_pager_server.py|process_command|287] [zulu_pager_server.py|cmd_login|511] [zulu_pager_server.py|send_response|213] [zulu_pager_server.py|push_packet|201] [asynchat.py|push_with_producer|155] [asynchat.py|initiate_send|212] [asyncore.py|send|272]) also get this error sometimes: (exceptions.AttributeError:'int' object has no attribute 'send' [asyncore.py|poll|79] [asyncore.py|handle_read_event|337] [asynchat.py|handle_read|80] [asyncore.py|recv|294] [zulu_chat_server.py|handle_close|48] [zulu_chat_server.py|close|53] [zulu_chat_server.py|push_packet|388] [zulu_chat_server.py|push_packet|73] [asynchat.py|push_with_producer|155] [asynchat.py|initiate_send|212] [asyncore.py|send|272] [|send|1]) Seems it's not finding the asyncore's socket 'send' method when this happens.. Anyone have seen this before? From eugene.leitl@l... Thu May 17 17:45:37 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Thu, 17 May 2001 18:45:37 +0200 (MET DST) Subject: medusa/bobo html and cookies Message-ID: Okay, it's time for the next inevitable round of questions. I don't expect too many answers, as we're apparently not many (most people are probably hacking Zope right now), and busy with our own projects. Nevertheless I do intend to use Medusa/Bobo for me privately as well as at work, and I'll stick around here to answer beginner's questions. Also, what is needed is making the small changes to keep Medusa/Bobo running on latest versions of Python. While the rapid development *is* a good thing, I'm starting to run into incompatibilities between several pieces of software, especially if several of them are present in a single project (right now in my case GadFly, Bobo, Medusa and xmlrpclib). This can be incredibly frustrating, especially for a beginner with not too much time on one's hands. Anyways, I intend to stick around and see what I can do to address these points. So whatever information you toss my way won't vanish below an event horizont, okay? I'm going to pass on the torch, and stuff. So, what I'm missing is a library of demo code for the Medusa/Bobo framework, doing both simple things and complicated things. Right now I'm looking for image generation, cookie manipulation (beyond of what is in the archive -- surely Bobo has extras for cookie mangling?) and XML-RPC. If you have such sources, and are willing to share, that would be great. I haven't checked with my employer, but I'm going to publish the framework of the application I'm currently writing anyway, as it is commercially worthless without the proprietary dbase. From rushing@n... Fri May 18 20:53:50 2001 From: rushing@n... (Sam Rushing) Date: Fri, 18 May 2001 12:53:50 -0700 Subject: [medusa] Asyncore's socket 'send' method not found?! References: <9duqtn+710s@e...> Message-ID: <3B057DCE.B06A216B@n...> Weiss Blitz wrote: > My Medusa server is crashing with the following error: > > (exceptions.AttributeError:send [asyncore.py|poll|79] Just a guess, but I'd make sure you have only one copy of asyn{core,char}.py in your python path. This sounds a little like an accidental mixing of old and new versions... Another thing to double-check is that you've called the __init__ method of the parent class... -Sam From jim@d... Sat May 19 18:00:23 2001 From: jim@d... (Jim Fulton) Date: Sat, 19 May 2001 13:00:23 -0400 Subject: Bug in parsing request paths, and patch Message-ID: <3B06A6A7.BF4B1831@d...> The HTTP server sort of follows RFC 1808 when parsing paths. RFC 1808 specified a URL as: :///;?# The HTTP request line contains the part after the net_loc, which the Medusa HTTP server parses into 4 parts, path, params, query, and fragment. It ignores (tosses) the params and fragment. RFC 2396 supercedes RFC 1808 and incorporates 'params' into the path. Loosely, a path is a sequence of path segments separated by slashes. Each path segment has a name followed by 0 or more parameters, where each parameter is set off by a semicolon, as in: /namex/namey;p1=v2;p2=v2/namez;p4/.... I've never seen this syntax actually used, but I'm considering using it in Zope to provide more explicit control over name lookup. Medusa's current request parsing breaks this syntax and, technically, violates RFC 2396. It lops off everything after a semicolon up to a query string. For example, the above path becomes: /namex/namey I suggest that, especially given RFC 2396, the parsing and interpretation of parameters should be left to individual handlers. The simplest fix would be to change the parser to ignore the semicolons in paths and return an empty string for the "params" part. The patch is included below. Jim diff -c -r1.25 http_server.py *** http_server.py 2001/05/01 12:49:04 1.25 --- http_server.py 2001/05/19 16:53:10 *************** *** 95,105 **** # split a uri # -------------------------------------------------- ! # ;?# path_regex = re.compile ( ! # path params query fragment ! r'([^;?#]*)(;[^?#]*)?(\?[^#]*)?(#.*)?' ! ) def split_uri (self): if self._split_uri is None: --- 95,105 ---- # split a uri # -------------------------------------------------- ! # ?# path_regex = re.compile ( ! # path query fragment ! r'([^?#]*)(\?[^#]*)?(#.*)?' ! ) def split_uri (self): if self._split_uri is None: *************** *** 107,113 **** if m.end() != len(self.uri): raise ValueError, "Broken URI" else: ! self._split_uri = m.groups() return self._split_uri def get_header_with_regex (self, head_reg, group): --- 107,114 ---- if m.end() != len(self.uri): raise ValueError, "Broken URI" else: ! m=m.groups() ! self._split_uri = m[0], '', m[1], m[2] return self._split_uri def get_header_with_regex (self, head_reg, group): -- Jim Fulton mailto:jim@d... Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org From rcspython@y... Sun May 20 16:46:01 2001 From: rcspython@y... (rcspython@y...) Date: Sun, 20 May 2001 15:46:01 -0000 Subject: Does Medusa's FTP Server have built in triggers? Message-ID: <9e8orp+ff1e@e...> Hello All, I am a total newbie to Python and Medusa for that matter. I want to use the ftpserver, however, I have to trigger some server side scriping after a file(s) is uploaded. If this is not supported at this time, how difficult would you say it is to add this feature? Best Regards! Ron From sav@u... Mon May 21 06:11:17 2001 From: sav@u... (Alexander Semenov) Date: Mon, 21 May 2001 09:11:17 +0400 Subject: [medusa] Does Medusa's FTP Server have built in triggers? References: <9e8orp+ff1e@e...> Message-ID: <00dd01c0e1b4$771a9e90$8220a8c0@u...> > Hello All, > I am a total newbie to Python and Medusa for that matter. I want to > use the ftpserver, however, I have to trigger some server side > scriping after a file(s) is uploaded. Better (more pythonic) use overloading instead callbacks. You should look into ftp_server.py I think you are interested in overloading recv_channel.handle_close() or whole class recv_channel (about 40 lines of code) if you want alternate file storing. You can, of cause, simply hack ftp_server.py. WBR, Alexander Semenov From rushing@n... Mon May 21 08:57:42 2001 From: rushing@n... (Sam Rushing) Date: Mon, 21 May 2001 00:57:42 -0700 Subject: [medusa] Does Medusa's FTP Server have built in triggers? References: <9e8orp+ff1e@e...> Message-ID: <3B08CA76.C9110746@n...> rcspython@y... wrote: > I am a total newbie to Python and Medusa for that matter. I want to > use the ftpserver, however, I have to trigger some server side > scriping after a file(s) is uploaded. > > If this is not supported at this time, how difficult would you say it > is to add this feature? Should be pretty easy to do... I'd just derive from the ftp_channel class and override the cmd_stor method... -Sam From lucifer@e... Mon May 21 17:17:38 2001 From: lucifer@e... (lucifer@e...) Date: Mon, 21 May 2001 16:17:38 -0000 Subject: Instant messaging & ICQ Message-ID: <9ebf32+3nk1@e...> Hi I'm one of those who are trying to build an instant messenger based on medusa. I was wondering if anyone has tried to communicate with ICQ. I know there is a C library, but no python port. I'm no C programmer, so I cant port it, not even using SWING I believe, but I was planning to study the protocol and build my own library. Has anyone tried something similar ? Or tried to port the C library ? From eugene.leitl@l... Wed May 23 13:53:39 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Wed, 23 May 2001 14:53:39 +0200 (MET DST) Subject: FYI: XML-RPC with Medusa Message-ID: http://aspn.activestate.com/ASPN/Python/Cookbook/Recipe/52226 Title: XML-RPC with Medusa By: Jeff Bauer Last Updated: 2001/03/11 Version no: 1.0 1 vote(s) Description: This section demonstrates remote method calls between two machines (or processes) using the XML-RPC protocol. A complete example of working client/server code is provided. Source: Text Source # xmlrpc_server.py from socket import gethostname from medusa.xmlrpc_handler import xmlrpc_handler from medusa.http_server import http_server from medusa import asyncore class xmlrpc_server(xmlrpc_handler): """The xmlrpc_server class demonstrates a simple implementation of Userland's XML-RPC protocol. You must download and install xmlrpclib and Medusa to run this code. Obtain Sam Rushing's Medusa library from http://www.nightmare.com Download Fredrik Lundh's xmlrpclib at http://www.pythonware.com""" def __init__(self, host=None, port=8182): if host is None: host = gethostname() hs = http_server(host, port) hs.install_handler(self) asyncore.loop() def add(self, op1, op2): return op1 + op2 def call(self, method, params): print "call method: %s, params: %s" % (method, str(params)) if method == 'add': return apply(self.add, params) return "method not found: %s" % method if __name__ == '__main__': server = xmlrpc_server() --- # xmlrpc_client.py from socket import gethostname from xmlrpclib import Transport, dumps class xmlrpc_connection: """The xmlrpc_connection class tests the xmlrpc_server. You must download and install the medusa and xmlrpclib libraries to run this code: http://www.nightmare.com http://www.pythonware.com""" def __init__(self, host=None, port=8182): if host is None: host = gethostname() self.host = "%s:%s" % (host, port) self.transport = Transport() def remote(self, method, params=()): """remote invokes the server with the method name and an optional set of parameters. The return value is always a tuple.""" response = self.transport.request(self.host, '/RPC2', dumps(params, method)) return response if __name__ == '__main__': connection = xmlrpc_connection() (answer,) = connection.remote("add", (40, 2)) print "The answer is:", answer Discussion: XML-RPC is one of the easier ways to handle distributed processing tasks. There's no messing around with the low-level socket details, nor is it necessary to write an interface definition. The protocol is platform-neutral and language-neutral. The XML-RPC specification can be found at: http://www.xml-rpc.com To run the example above, it is necessary to download xmlrpclib from http://www.pythonware.com and the Medusa library from http://www.nightmare.com. [ Aside: Perhaps someone from ActiveState can provide an additional client example in Perl, to demonstrate how multiple languages can be used together with xmlrpc. -Jeff ] From eugene.leitl@l... Sun May 27 13:37:16 2001 From: eugene.leitl@l... (eugene.leitl@l...) Date: Sun, 27 May 2001 14:37:16 +0200 Subject: [Fwd: Re: Distributed computing in Python - Callback howto ?] Message-ID: <3B10F4FC.FCAFF7FC@l...> -------- Original Message -------- From: Doug Fort Subject: Re: Distributed computing in Python - Callback howto ? To: python-list@p... Newsgroups: comp.lang.python Sasa Zivkov wrote: > Hi, > > What options one have for distributed computing in Python ? > Check out Frederik Lund's implementation of xmlrpc -- http://www.pythonware.com/products/xmlrpc/index.htm If you want to see some sample code, we use it in our agent infrastructure http://pyagent.sourceforge.net -- Doug Fort Senior Meat Manager Downright Software LLC http://www.downright.com ______________________________________________________________________ Posted Via Uncensored-News.Com - Still Only $9.95 - http://www.uncensored-news.com With Seven Servers In California And Texas - The Worlds Uncensored News Source -- http://mail.python.org/mailman/listinfo/python-list From altis@y... Wed May 30 20:55:12 2001 From: altis@y... (altis@y...) Date: Wed, 30 May 2001 19:55:12 -0000 Subject: Instant messaging & ICQ In-Reply-To: <9ebf32+3nk1@e...> Message-ID: <9f3j70+f3e5@e...> You might want to check out the Jabber infrastructure. The Jabber protocol justs passes around XML until it hits a Jabber gateway which can then talk to ICQ, Yahoo Messenger, MSN Messenger, and AIM, or any other defined gateway on the server. http://www.jabber.org/ http://www.jabbercentral.org/ http://www.jabber.com/ There are numerous clients and servers available for different platforms as well as documentation on the protocol, etc. so that should help get you going. Since there are already Python libraries to do the XML parsing for you, it should simplify development. ka --- In medusa@y..., lucifer@e... wrote: > Hi > > I'm one of those who are trying to build an instant messenger based on > medusa. > I was wondering if anyone has tried to communicate with ICQ. > I know there is a C library, but no python port. > I'm no C programmer, so I cant port it, not even using SWING I > believe, but I was planning to study the protocol and build my own > library. > Has anyone tried something similar ? Or tried to port the C library ? From altis@s... Wed May 30 20:59:20 2001 From: altis@s... (Kevin Altis) Date: Wed, 30 May 2001 12:59:20 -0700 Subject: turning off DNS resolves, ftp, chat Message-ID: I searched for an answer to the DNS issue, but didn't see a response. I just want to log IP addresses and skip the DNS resolve. Also, it appears there may be a problem connecting to medusa if you simply try connecting on the local machine as http://localhost/ Also, I just installed Medusa and don't need the FTP or Chat servers. Do I just need to comment out those lines in the startup? ka From rprice@u... Wed May 30 21:04:05 2001 From: rprice@u... (rprice@u...) Date: Wed, 30 May 2001 13:04:05 -0700 (PDT) Subject: [medusa] turning off DNS resolves, ftp, chat In-Reply-To: References: Message-ID: <991253045.3b1552352510c@w...> Quoting Kevin Altis : > I searched for an answer to the DNS issue, but didn't see a response. I > just > want to log IP addresses and skip the DNS resolve. > Set the rs variable like this to only log IP addresses: rs = None > Also, it appears there may be a problem connecting to medusa if you > simply > try connecting on the local machine as http://localhost/ Is is set to run on port 80? You may need to access it like this: http://localhost:8080/ which is the default.. as I remember. > > Also, I just installed Medusa and don't need the FTP or Chat servers. Do > I > just need to comment out those lines in the startup? Yes, and besure to comment all occurances of the ftp and cs variables. > > ka > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > Ray From altis@y... Wed May 30 23:13:42 2001 From: altis@y... (altis@y...) Date: Wed, 30 May 2001 22:13:42 -0000 Subject: turning off DNS resolves, ftp, chat In-Reply-To: <991253045.3b1552352510c@w...> Message-ID: <9f3ram+7nvj@e...> --- In medusa@y..., rprice@u... wrote: > > Also, it appears there may be a problem connecting to medusa if you > > simply > > try connecting on the local machine as http://localhost/ > Is is set to run on port 80? You may need to access it like this: > http://localhost:8080/ > which is the default.. as I remember. Thanks for the info. I think there is actually a bug here. I'm using Python 2.1 (ActivePython build 210) on Windows 2000 Pro. I'm using port 80. If I configure as: IP_ADDRESS = '127.0.0.1' then I can connect with IE from the same machine using http://localhost/ as expected. However, I am then unable to connect to the server from either the local machine or another machine on the LAN using either the IP address or DNS name of the server machine. If I configure the IP_ADDRESS to use the actual IP address, then I can't connect as localhost or via 127.0.0.1 which is the same thing. Somehow the binding is broken. I tested using a simple telnet connection to port 80 and got the same results. Normally, a simple server would do something like: from socket import * # get socket constructor and constants myHost = '' # server machine, '' means local host myPort = 80 # port to listen on sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object sockobj.bind((myHost, myPort)) # bind it to server port number sockobj.listen(5) # listen, allow 5 pending connects I haven't looked at the code for medusa yet, but the above code definitely works for localhost or using the real IP address or DNS name. ka From rprice@u... Wed May 30 23:28:47 2001 From: rprice@u... (rprice@u...) Date: Wed, 30 May 2001 15:28:47 -0700 (PDT) Subject: [medusa] Re: turning off DNS resolves, ftp, chat In-Reply-To: <9f3ram+7nvj@e...> References: <9f3ram+7nvj@e...> Message-ID: <991261727.3b15741fc96e1@w...> What happens if you set IP_ADDRESS = '' Ray Quoting altis@y...: > --- In medusa@y..., rprice@u... wrote: > > > Also, it appears there may be a problem connecting to medusa if > you > > > simply > > > try connecting on the local machine as http://localhost/ > > Is is set to run on port 80? You may need to access it like this: > > http://localhost:8080/ > > which is the default.. as I remember. > > Thanks for the info. I think there is actually a bug here. I'm using > Python 2.1 (ActivePython build 210) on Windows 2000 Pro. I'm using > port 80. If I configure as: > IP_ADDRESS = '127.0.0.1' > then I can connect with IE from the same machine using > http://localhost/ as expected. However, I am then unable to connect > to the server from either the local machine or another machine on the > LAN using either the IP address or DNS name of the server machine. If > I configure the IP_ADDRESS to use the actual IP address, then I can't > connect as localhost or via 127.0.0.1 which is the same thing. > Somehow the binding is broken. I tested using a simple telnet > connection to port 80 and got the same results. > > Normally, a simple server would do something like: > > from socket import * # get socket constructor and constants > myHost = '' # server machine, '' means local host > myPort = 80 # port to listen on > > sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object > sockobj.bind((myHost, myPort)) # bind it to server port number > sockobj.listen(5) # listen, allow 5 pending connects > > I haven't looked at the code for medusa yet, but the above code > definitely works for localhost or using the real IP address or DNS > name. > > ka > > > > > Your use of Yahoo! Groups is subject to > http://docs.yahoo.com/info/terms/ > > > From altis@y... Thu May 31 00:37:45 2001 From: altis@y... (altis@y...) Date: Wed, 30 May 2001 23:37:45 -0000 Subject: turning off DNS resolves, ftp, chat In-Reply-To: <991261727.3b15741fc96e1@w...> Message-ID: <9f4089+tvi4@e...> --- In medusa@y..., rprice@u... wrote: > What happens if you set > IP_ADDRESS = '' > Doh! That works for localhost, 127.0.0.1, actual IP address and DNS name, so I guess that is the appropriate thing to use. Thanks I am seeing log entries like 127.0.0.1:2009 but I think that bug was discussed in a different thread. ka From lucifer@e... Thu May 31 16:31:11 2001 From: lucifer@e... (lucifer@e...) Date: Thu, 31 May 2001 15:31:11 -0000 Subject: Instant messaging & ICQ In-Reply-To: <9f3j70+f3e5@e...> Message-ID: <9f5o3v+8i8h@e...> Thanks ka, I will check it out. --- In medusa@y..., altis@y... wrote: > You might want to check out the Jabber infrastructure. The Jabber > protocol justs passes around XML until it hits a Jabber gateway which > can then talk to ICQ, Yahoo Messenger, MSN Messenger, and AIM, or any > other defined gateway on the server. > > http://www.jabber.org/ > http://www.jabbercentral.org/ > http://www.jabber.com/ > > There are numerous clients and servers available for different > platforms as well as documentation on the protocol, etc. so that > should help get you going. Since there are already Python libraries > to do the XML parsing for you, it should simplify development. > > ka > --- In medusa@y..., lucifer@e... wrote: > > Hi > > > > I'm one of those who are trying to build an instant messenger based > on > > medusa. > > I was wondering if anyone has tried to communicate with ICQ. > > I know there is a C library, but no python port. > > I'm no C programmer, so I cant port it, not even using SWING I > > believe, but I was planning to study the protocol and build my own > > library. > > Has anyone tried something similar ? Or tried to port the C > library ? From altis@s... Fri Jun 1 04:53:48 2001 From: altis@s... (Kevin Altis) Date: Thu, 31 May 2001 20:53:48 -0700 Subject: directory listing produces 404 error Message-ID: I'm sure this is just another newbie question, but how do I turn on directory listings in medusa for http requests? Right now, if I go to a directory that has an index.html file it returns that file, but other directories produce a 404 error. ka From eugene.leitl@l... Fri Jun 1 17:25:31 2001 From: eugene.leitl@l... (Eugene Leitl) Date: Fri, 1 Jun 2001 18:25:31 +0200 (MET DST) Subject: Webware for Python Message-ID: Anyone checked this out yet? http://webware.sourceforge.net/ If yes, how does this compare to Bobo/Medusa combo? -- Eugene ______________________________________________________________ ICBMTO : N48 10'07'' E011 33'53'' http://www.lrz.de/~ui22204 57F9CFD3: ED90 0433 EB74 E4A9 537F CFF5 86E7 629B 57F9 CFD3 From altis@y... Fri Jun 1 20:18:14 2001 From: altis@y... (altis@y...) Date: Fri, 01 Jun 2001 19:18:14 -0000 Subject: Webware for Python In-Reply-To: Message-ID: <9f8ppm+38b7@e...> If you read the various papers and docs on the WebWare site you'll see that the design is more like WebObjects http://www.apple.com/webobjects/ than a simple CGI framework. Consequently, the additional overhead of using an application server would make it overkill for simple CGIs, but it should do quite nicely as the complexity of your web application increases. At this point, it is unclear to me how much inconvenience the framework causes when you want to mix it with static pages, images, regular CGIs, servlets, etc. I only started looking at it a few days ago, but I'm impressed by the amount of work that has gone into it and the quality of documentation. It is certainly worth experimenting with. ka --- In medusa@y..., Eugene Leitl wrote: > > Anyone checked this out yet? > > http://webware.sourceforge.net/ > > If yes, how does this compare to Bobo/Medusa combo? > > -- Eugene > > ______________________________________________________________ > ICBMTO : N48 10'07'' E011 33'53'' http://www.lrz.de/~ui22204 > 57F9CFD3: ED90 0433 EB74 E4A9 537F CFF5 86E7 629B 57F9 CFD3 From p_pillai_p@r... Wed Jun 6 18:48:41 2001 From: p_pillai_p@r... (p_pillai_p@r...) Date: Wed, 06 Jun 2001 17:48:41 -0000 Subject: FTP Server Message-ID: <9flqdp+4qqi@e...> hi I want to develop a FTP server starting from the beginning.I wish that the server supports simple functions like PUT and GET.Also it should be multitasking. Can someone guide me in this regard. From cgale1@h... Fri Jun 8 10:02:58 2001 From: cgale1@h... (Van Gale) Date: Fri, 8 Jun 2001 02:02:58 -0700 Subject: [medusa] FTP Server References: <9flqdp+4qqi@e...> Message-ID: <001501c0eff9$d24b0180$0200a8c0@e...> ----- Original Message ----- From: To: Sent: Wednesday, June 06, 2001 10:48 AM Subject: [medusa] FTP Server > I want to develop a FTP server starting from the beginning.I wish > that the server supports simple functions like PUT and GET.Also it > should be multitasking. Can someone guide me in this regard. I'm not sure why you are asking this question. Even if you want to develop a new FTP server from the beginning, why is it you do not want to use the medusa ftp_server.py module to guide you? Van Gale From cgale1@h... Fri Jun 8 10:17:38 2001 From: cgale1@h... (Van Gale) Date: Fri, 8 Jun 2001 02:17:38 -0700 Subject: Possible small bugs in counter.py and auth_handler.py? Message-ID: <001b01c0effb$e0da3340$0200a8c0@e...> I've really only been using Python since version 2.1, so I'm not aware of all the issues involved in porting older code, however I *think* I have found an inconsequential bug in counter.py related to differences in long() vs. string.atol. In counter.py, it seems the __str__ method is chopping off the last digit of the counter because of a [:-1] slice on the string returned by str(long()). If I change the line >>> return str(long(self.value))[:-1] to >>> return str(long(self.value)) printing the counter now works correctly. The method "make_nonce" in auth_handler.py seems to be doing the same thing (found by simply doing a grep for [:-1]) Van Gale From skip@p... Fri Jun 8 15:55:46 2001 From: skip@p... (Skip Montanaro) Date: Fri, 8 Jun 2001 09:55:46 -0500 Subject: [medusa] Possible small bugs in counter.py and auth_handler.py? In-Reply-To: <001b01c0effb$e0da3340$0200a8c0@e...> References: <001b01c0effb$e0da3340$0200a8c0@e...> Message-ID: <15136.59250.965374.487184@b...> Van> If I change the line >>>> return str(long(self.value))[:-1] Van> to >>>> return str(long(self.value)) Van> printing the counter now works correctly. Yes, str's handling of longs changed in 1.6, so if you are using longs in 1.6 or beyond, you have to check/change such code. In 1.5 and earlier versions, str(long(1)) would have returned "1L" instead of just "1". -- Skip Montanaro (skip@p...) (847)971-7098 From rushing@n... Fri Jun 8 20:16:48 2001 From: rushing@n... (Sam Rushing) Date: Fri, 08 Jun 2001 12:16:48 -0700 Subject: [medusa] Possible small bugs in counter.py and auth_handler.py? References: <001b01c0effb$e0da3340$0200a8c0@e...> Message-ID: <3B2124A0.D0CFF9A9@n...> Van Gale wrote: > I've really only been using Python since version 2.1, so I'm not aware of > all the issues involved in porting older code, however I *think* I have > found an inconsequential bug in counter.py related to differences in long() > vs. string.atol. > The method "make_nonce" in auth_handler.py seems to be doing the same thing > (found by simply doing a grep for [:-1]) The [:-1] was to slice off the 'L' on the end of the string representation of long numbers. This must have been changed since python1.x... -Sam From abo@m... Mon Jun 11 06:59:30 2001 From: abo@m... (Donovan Baarda) Date: Mon, 11 Jun 2001 15:59:30 +1000 Subject: medusa ready() patch In-Reply-To: <0106080204430E.11391@l...>; from kthangavelu@e... on Fri, Jun 08, 2001 at 02:05:24AM -0700 References: <0106080204430E.11391@l...> Message-ID: <20010611155930.B17879@m...> On Fri, Jun 08, 2001 at 02:05:24AM -0700, ender wrote: > Hello Donovan, > > I was curious if there was any further developments regarding your patch to > add a ready methods to producers. Namely if you had developed the patches > further or used them in production and if there had been any discussion with > sam rushing regarding inclusion with medusa? I have not had a chance to actualy use it in anger myself yet, but I have had some feedback from a few people playing with it. One post pointed out that I seemed to have re-invented a slightly different design-pattern to the one medusa was based on... a total accident :-) I have also failed to keep it up to date with medusa. Sam has put out at least one new version since I did my patch, so I need to update it. Interestingly, at least one of the things I did in my patch seems to have been adopted and improved on in Sam's new version (removing the cache for find_prefix_at_end), though I think it was independantly developed without looking at my patch. Certainly I have not had any feedback from Sam about it. I really don't know how much demand there would be for something like this in medusa, or even the async bits in Python. Everything it does can be done probably better by using proper async objects rather than blocking producers. The only reason I did it was because the existing medusa achitecture assumes that its virtual filesystem files are non-blocking, and making producers block was a quicker fix than making the whole medusa virtual filesystem asynchronos. I did make my changes backwards compatible, so it shouldn't hurt too much if they were included in medusa itself. However, some of my other changes to things like the producer class heirachy might have been too heavy handed for someone like Sam to comfortably accept. If enough people did feel it was worth including, and Sam was willing, I'd be more than happy to do what is needed to make it suitable for inclusion into medusa itself. I am still planning on using it soon, so I will be updating it. I'm also willing to support it as an add-on patch to medusa, tracking medusa development. I'm open to any other suggestions too :-) -- ---------------------------------------------------------------------- ABO: finger abo@m... for more info, including pgp key ---------------------------------------------------------------------- From tshab@y... Tue Jun 12 05:10:55 2001 From: tshab@y... (tshab@y...) Date: Tue, 12 Jun 2001 04:10:55 -0000 Subject: Streaming Data Socket Server Message-ID: <9g44of+9qov@e...> I'm trying to consolidate two similar programs (one in C, one that uses National Instruments Datasockets - www.ni.com/datasocket), and someone suggested Medusa. Basically, I'm trying to collect data from multiple streaming socket producers, which will then be connected via the server switchboard to one or more socket consumers. Also, the channel of the currently connected consumers needs to be listed on a dynamically created web page. Right now, the protocol is almost non-existent, just an 's' for producers, or the channel number for consumers. I looked at asynchat, because it has the nice channel handling, but I don't know how to get around the set_terminator issue (there is no delimiter on the data). So, any pointers would be appreciated on: a. Mapping channels to multiple consumers b. Handling streamed data in asynchat correctly c. Dynamic web page generation (should I use one of the templating packages)? Finally, I'm hoping to extend this design in the future to support things such as encryption and compression of the streams. Would I be able to use producers on streams of data? Thanks in advance for any advice. Best, Ted From andreas@d... Mon Jul 9 16:10:28 2001 From: andreas@d... (andreas@d...) Date: Mon, 09 Jul 2001 15:10:28 -0000 Subject: asyncore: 'unhandled connect event' Message-ID: <9ichh4+rh6g@e...> For some FTP clients (e.g. WSFTP) Zope 2.3/2.4 logs an "unhandled connect event". This message is generated in asyncore module of Python 2.1 in handle_connect(). This message confuses lots of people because the message is logged for every FTP command. To avoid this "mis"-behaviour of Medusa I suggest to add a dummy handle_connect() function to the xmit_channel class to suppress the warning. Andreas From max@a... Thu Jul 12 18:58:59 2001 From: max@a... (max@a...) Date: Thu, 12 Jul 2001 17:58:59 -0000 Subject: default_with_post_handler Message-ID: <9ikoh3+cmrj@e...> I got the latest drop of Medusa (20010416) and am attempting to implement a non-standard HTTP server with it (for the Web game Interstelen, if anyone's interested: http://www.interstelen.com/). The server should accept both GET and POST (GET to start with for debugging, POST for a cleaner interface unfettered by query size limits). I started implementing an HTTP server myself (but still using the asyncore/asynchat framework) and decided my time would be better spent using a preexisting, known reliable HTTP server that's in Medusa. So I looked to default_handler.py to find a POST example, and sure enough: # support for handling POST requests is available in the derived # class , defined below. But ... emf@s...:/usr/local/lib/python2.0/site-packages/medusa% grep default_with_post_handler *.py default_handler.py:# class , defined below. That's the only reference to that class I can find anywhere. Where is this class? As a parting comment, I find asyncore/asynchat/Medusa very powerful and very convenient, but I haven't really been able to find any documentation beyond sketchy READMEs and tutorials. Even something along the lines of HappyDoc-generated API documentation based solely on the class hierarchy, the methods in classes, and the docstrings that appear here and there would be helpful. Does any exist somewhere? From max@a... Thu Jul 12 21:55:16 2001 From: max@a... (max@a...) Date: Thu, 12 Jul 2001 20:55:16 -0000 Subject: Invalid protocol in query results in exception Message-ID: <9il2rl+auph@e...> Sending GET / 1.0\r\n\r\n instead of GET / HTTP/1.0\r\n\r\n to Medusa 3.41 results in: error: uncaptured python exception, closing channel (exceptions.AttributeError:'None' object has no attribute 'end' [/usr/local/lib/python2.0/site-packages/medusa/asyncore.py|poll|96] [/usr/local/lib/python2.0/site-packages/medusa/asy ncore.py|handle_read_event|384] [/usr/local/lib/python2.0/site-packages/medusa/asy nchat.py|handle_read|131] [/usr/local/lib/python2.0/site-packages/medusa/htt p_server.py|found_terminator|463] [/usr/local/lib/python2.0/site-packages/medusa/htt p_server.py|crack_request|692]) which closes the HTTP connection rather than just responding with a 400 (Bad Request). From rushing@n... Fri Jul 13 09:04:29 2001 From: rushing@n... (Sam Rushing) Date: Fri, 13 Jul 2001 01:04:29 -0700 Subject: [medusa] default_with_post_handler References: <9ikoh3+cmrj@e...> Message-ID: <3B4EAB8D.4FF71E78@n...> max@a... wrote: > So I looked to default_handler.py to find a POST example, and sure > enough: > > # support for handling POST requests is available in the derived > # class , defined below. > > But ... > > emf@s...:/usr/local/lib/python2.0/site-packages/medusa% grep > default_with_post_handler *.py > default_handler.py:# class , defined below. Ahh... an outdated comment. I would recommend looking at the post-handling capabilities of the 'script_handler' class in script_handler.py, there should be enough info there to help. You might event want to use one of the script-handling classes, since they support what most folks would want to do with a medusa-based web server. > That's the only reference to that class I can find anywhere. Where is > this class? > > As a parting comment, I find asyncore/asynchat/Medusa very powerful > and very convenient, but I haven't really been able to find any > documentation beyond sketchy READMEs and tutorials. Even something > along the lines of HappyDoc-generated API documentation based solely > on the class hierarchy, the methods in classes, and the docstrings > that appear here and there would be helpful. Does any exist > somewhere? That's interesting, I've just been discussing the lack of documentation with someone else. IMHO when the code is short, it should serve as the documentation. It's less likely to be misleading, and much more likely to be written. So one of my most important goals when writing code is to keep it short enough to not require documentation. 8^) For me, the most important part of a library are the examples. Tons of documentation can actually be a hindrance without short examples to get you started. Anyway, enough weak excuses for my lack of motivation to document... [Of course, none of this excuses the lack of proper doc strings...] -Sam From rushing@n... Fri Jul 13 09:11:48 2001 From: rushing@n... (Sam Rushing) Date: Fri, 13 Jul 2001 01:11:48 -0700 Subject: [medusa] Invalid protocol in query results in exception References: <9il2rl+auph@e...> Message-ID: <3B4EAD44.ECF106AA@n...> ---------------------- multipart/alternative attachment max@a... wrote: > Sending > GET / 1.0\r\n\r\n > instead of > GET / HTTP/1.0\r\n\r\n > to Medusa 3.41 results in: > [...] > which closes the HTTP connection rather than just responding with a > 400 (Bad Request). Here's a quick fix: Index: http_server.py =================================================================== RCS file: /usr/local/cvsroot/medusa/http_server.py,v retrieving revision 3.41 retrieving revision 3.43 diff -r3.41 -r3.43 9c9 < RCS_ID = '$Id: http_server.py,v 3.41 2001/04/16 03:41:29 rushing Exp $' --- > RCS_ID = '$Id: http_server.py,v 3.43 2001/07/13 08:02:38 rushing Exp $' 107c107 < if m.end() != len(self.uri): --- > if not m or m.end() != len(self.uri): 116c116 < if m.end() == len(line): --- > if m and m.end() == len(line): 692c692 < if m.end() == len(r): --- > if m and m.end() == len(r): -Sam ---------------------- multipart/alternative attachment An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail-21/medusa-dev/attachments/192ba923/attachment.html ---------------------- multipart/alternative attachment-- From max@a... Mon Jul 16 22:50:01 2001 From: max@a... (max@a...) Date: Mon, 16 Jul 2001 21:50:01 -0000 Subject: Invalid protocol in query results in exception In-Reply-To: <3B4EAD44.ECF106AA@n...> Message-ID: <9ivni9+ingb@e...> --- In medusa@y..., Sam Rushing wrote: > max@a... wrote: > > > Sending > > GET / 1.0\r\n\r\n > > instead of > > GET / HTTP/1.0\r\n\r\n > > to Medusa 3.41 results in: > > [...] > > > which closes the HTTP connection rather than just responding with > > a > > 400 (Bad Request). > > Here's a quick fix: > > Index: http_server.py > =================================================================== > RCS file: /usr/local/cvsroot/medusa/http_server.py,v > retrieving revision 3.41 > retrieving revision 3.43 > diff -r3.41 -r3.43 ... Cool. For the record, I wasn't requesting a fix, just informing you guys of the bug. I happened to notice it in my testing, but it isn't a particularly urgent issue for me at the moment. Just wanted to let you know. From max@a... Mon Jul 16 22:54:28 2001 From: max@a... (max@a...) Date: Mon, 16 Jul 2001 21:54:28 -0000 Subject: default_with_post_handler In-Reply-To: <3B4EAB8D.4FF71E78@n...> Message-ID: <9ivnqk+3762@e...> --- In medusa@y..., Sam Rushing wrote: > max@a... wrote: > > Ahh... an outdated comment. > I would recommend looking at the post-handling capabilities of the > 'script_handler' class in script_handler.py, there should be enough > info > there to help. You might event want to use one of the > script-handling > classes, since they support what most folks would want to do with a > medusa-based web server. Yep, that ends up being exactly what I did, using script_handler as a guide for how to do POST. It works fine. > That's interesting, I've just been discussing the lack of > documentation with > someone else. IMHO when the code is short, it should serve as the > documentation. It's less likely to be misleading, and much more > likely to > be written. So one of my most important goals when writing code is > to keep > it short enough to not require documentation. 8^) That's certainly what I did. It's just that there are some relatively important concepts that don't seem to be documented anywhere in the tutorials or readmes (at least that I found). For instance, when I was using asynchat, it took a bit of digging to find out about the push_with_producer(None) trick. > For me, the most important part of a library are the examples. Tons > of > documentation can actually be a hindrance without short examples to > get you > started. Anyway, enough weak excuses for my lack of motivation to > document... [Of course, none of this excuses the lack of proper doc > strings...] Just thought I'd suggest it in case anybody hadn't prodded you, but more importantly, I just wanted to make sure I wasn't spectacularly missing some obvious documentation somewhere. From tusharwagle@y... Tue Jul 17 19:42:07 2001 From: tusharwagle@y... (tusharwagle@y...) Date: Tue, 17 Jul 2001 18:42:07 -0000 Subject: Adding per-directory security to ftp server (how to?) Message-ID: <9j20tv+hga9@e...> Hello, I've just started looking at Medusa and would like some pointers into the 'best' way to do the following: I want to check access at each directory in a filesystem which is served over both http and ftp. I've written a handler for the http server, but then realised that I'd have to modify it for the ftp server. Is there a recommended way to apply this kind of authentication to both http and ftp servers? Any help appreaciated! - Tushar P.S. Many Many thanks to Sam for releasing this as free software! :) From rushing@n... Tue Jul 17 20:06:43 2001 From: rushing@n... (Sam Rushing) Date: Tue, 17 Jul 2001 12:06:43 -0700 Subject: [medusa] Adding per-directory security to ftp server (how to?) References: <9j20tv+hga9@e...> Message-ID: <3B548CC3.949F03B8@n...> tusharwagle@y... wrote: > I want to check access at each directory in a filesystem > which is served over both http and ftp. > Is there a recommended way to apply this kind of > authentication to both http and ftp servers? I guess it depends on where the auth info is stored and how you access it. Not knowing too much more about it I would extend one of the classes in filesys.py to support your access check. The ftp server and the http default_handler both expect a 'filesystem' object. The ftp server currently retrieves that object from an authorizer, but you could change this, depending on whether you want to extend normal ftp auth or replace it. -Sam From jbarham@o... Tue Jul 31 00:20:00 2001 From: jbarham@o... (John E. Barham) Date: Mon, 30 Jul 2001 16:20:00 -0700 Subject: Multi-threaded web-server design issues Message-ID: ---------------------- multipart/alternative attachment Hi all, I'm trying to build a multi-threaded Web-server using Medusa's http_server. (I need threads because our Web app. has some long-lived CGI scripts w/ heavy database access.) I based the initial design on the example HTTP server from the thread_handler.py code in the thread dir. of the Medusa distro. However, that code obviously hasn't been touched in a while (I had to change it to use the "new" http_request.split_uri(), used the standard Queue module etc. etc.) so am wondering if there's a better approach. In particular, to simplify things, I'm thinking about creating a separate output queue (instead of the select_trigger stuff which has been giving me weird errors) where all of the CGI worker threads would put their output when they were finished. A separate response handling thread would then pull the responses off this queue and feed them back to Medusa. Is this a good approach? Also, from what I can see from the code, the only http_request method that isn't thread-safe is done(). Is this correct? That would mean that each CGI thread could safely manipulate the request object itself, leaving the response-handling queue to (serially) invoke done(). Thanks in advance for any feedback. John --- John E. Barham Senior Software Developer jbarham@o... www.octopuswireless.com Phone: (604) 434-1321 Fax: (604) 430-1419 Octopus Wireless Inc. 5317 Lane Street Burnaby, BC V5H 2H4 ---------------------- multipart/alternative attachment An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail-21/medusa-dev/attachments/bbbe8d11/attachment.html ---------------------- multipart/alternative attachment-- From rushing@n... Tue Jul 31 09:44:46 2001 From: rushing@n... (Sam Rushing) Date: Tue, 31 Jul 2001 01:44:46 -0700 Subject: [medusa] Multi-threaded web-server design issues References: Message-ID: <3B666FFD.977FAB3F@n...> ---------------------- multipart/alternative attachment "John E. Barham" wrote: > I'm trying to build a multi-threaded Web-server using Medusa's > http_server. (I need threads because our Web app. has some long-lived > CGI scripts w/ heavy database access.) I based the initial design on > the example HTTP server from the thread_handler.py code in the thread > dir. of the Medusa distro. However, that code obviously hasn't been > touched in a while (I had to change it to use the "new" > http_request.split_uri(), used the standard Queue module etc. etc.) so > am wondering if there's a better approach. In particular, to simplify > things, I'm thinking about creating a separate output queue (instead > of the select_trigger stuff which has been giving me weird errors) > where all of the CGI worker threads would put their output when they > were finished. A separate response handling thread would then pull > the responses off this queue and feed them back to Medusa. Is this a > good approach? The purpose of the trigger is to provide a way to 'wake up' the select() call that is pending in the main asyncore event loop. Even if you had a separate thread that pulled out of a central queue, you'd still need to address this issue... > Also, from what I can see from the code, the only http_request method > that isn't thread-safe is done(). Is this correct? That would mean > that each CGI thread > could safely manipulate the request object itself, leaving the > response-handling queue to (serially) invoke done(). As long as you processed the requests carefully - for example, let's say that you were sent three pipelined requests on an http/1.1 channel. If you're not careful, the requests might get retired out-of-order, and thus push the responses out of order. If you disable http/1.1 (or just pipelining) then it's probably not an issue. -Sam ---------------------- multipart/alternative attachment An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail-21/medusa-dev/attachments/812f7f4b/attachment.html ---------------------- multipart/alternative attachment-- From abo@m... Wed Aug 1 01:26:32 2001 From: abo@m... (Donovan Baarda) Date: Wed, 01 Aug 2001 10:26:32 +1000 (EST) Subject: [medusa] Multi-threaded web-server design issues In-Reply-To: References: Message-ID: <996625592.3b674cb88a64b@m...> Quoting "John E. Barham" : > Hi all, > > I'm trying to build a multi-threaded Web-server using Medusa's > http_server. > (I need threads because our Web app. has some long-lived CGI scripts w/ > heavy database access.) I based the initial design on the example HTTP > server from the thread_handler.py code in the thread dir. of the Medusa > distro. However, that code obviously hasn't been touched in a while (I > had > to change it to use the "new" http_request.split_uri(), used the > standard > Queue module etc. etc.) so am wondering if there's a better approach. > In > particular, to simplify things, I'm thinking about creating a separate > output queue (instead of the select_trigger stuff which has been giving > me > weird errors) where all of the CGI worker threads would put their output > when they were finished. A separate response handling thread would then > pull the responses off this queue and feed them back to Medusa. Is this > a > good approach? You could have a look at ZServer that comes with Zope. This branched from medusa some time ago. It uses threads to get around the delays associated with fetching objects from remote databases. It uses quite a few nice tricks that you can use in medusa (callback producers etc). Another alternative is to use my patch that allows producers to block using a ready() method. This way you can embed the cgi thread into a producer that you can push strait onto medusa. If you want this, let me know. I'm yet to integrate my patch with the latest release of medusa, and a show of interest will encorage me to get around to it. -- ABO: finger abo@m... for more information. From Thomas@g... Thu Aug 2 12:29:44 2001 From: Thomas@g... (Thomas Weholt) Date: Thu, 2 Aug 2001 13:29:44 +0200 Subject: Benefits from using ftp to download vs. http ?? Message-ID: <3BC7333BE30CD511AEB700B0D0AB2F5803D246@L...> Hi, I've written a simple file-sharing app using http as web-frontend and ftp to serve the actual files. Are there any benefits from this; do I ease the "pressure" on the http-interface so it can handle more clients by doing all downloading using ftp? Thomas From rushing@n... Thu Aug 2 17:41:22 2001 From: rushing@n... (Sam Rushing) Date: Thu, 02 Aug 2001 09:41:22 -0700 Subject: [medusa] Benefits from using ftp to download vs. http ?? References: <3BC7333BE30CD511AEB700B0D0AB2F5803D246@L...> Message-ID: <3B6982B2.C0B0939B@n...> Thomas Weholt wrote: > I've written a simple file-sharing app using http as web-frontend and ftp to > serve the actual files. Are there any benefits from this; do I ease the > "pressure" on the http-interface so it can handle more clients by doing all > downloading using ftp? FTP is the more complicated protocol; it uses two TCP connections per download rather than one. HTTP is your better bet, unless you need the extra capabilities of FTP for some reason (access control, mostly). -Sam From Thomas@g... Tue Aug 7 08:48:40 2001 From: Thomas@g... (Thomas Weholt) Date: Tue, 7 Aug 2001 09:48:40 +0200 Subject: Adding/removing handlers dynamically Message-ID: <3BC7333BE30CD511AEB700B0D0AB2F5803D262@L...> I need to add/remove handlers dynamically. I got a modular web-server that searches a given folder for new handlers, when it finds one it would be cool to have it install it without restarting the server, all automatic. If I could build a server that search for new handlers at a given interval, installs all new, removes all handlers that have been removed, it would open for some really cool features. Any hints on how to implement a way of loading handlers dynamically would also be great. ( This is for a distributed network model, where each node/server runs several user-specified/user-built web-services. Communication between the nodes are done using xml-rpc. The development is moving along nicely using Medusa as foundation. thanks to everyone involved in the project. If I just could get dynamic loading/reload of modules/handlers to work it would really be a cool killer-app written in python. In my own humble opionon. ;-) ) Thomas From jeremy@z... Fri Aug 10 15:35:05 2001 From: jeremy@z... (jeremy@z...) Date: Fri, 10 Aug 2001 10:35:05 -0400 (EDT) Subject: update to asyncore in Python library Message-ID: <200108101435.KAA02722@m...> I just made a few small changes to the asyncore module in the Python source tree, which I thought I'd mention here. It's hard to tell if this is the only drift between the two copies of asyncore, because we've "normalized whitespace" in our copy and diff isn't too helpful anymore. The key change is to remove the definintions of EWOULDBLOCK and company for os.name == "nt". On Windows, the errno module gets these definitions from winsock.h. So errno should be correct across platforms. Is there any Windows variant where this wouldn't be true? I also wrapped some long lines. Jeremy From jeremy@z... Fri Aug 10 15:46:32 2001 From: jeremy@z... (Jeremy Hylton) Date: Fri, 10 Aug 2001 10:46:32 -0400 (EDT) Subject: uncaught EAGAIN in recv() Message-ID: <15219.62408.345661.185367@w...> We have used asyncore to implement ZEO, a client-server subsystem for the ZODB object database. I think the details of the application are unimportant, but you can find more info at http://www.zope.org/Wikis/ZODB/ One of our users has a problem with an uncaught socket error on recv(). Logging output here: Logging [ZEO]: uncaptured python exception, closing channel (socket.error:(11, 'Resource temporarily unavailable') [/usr/local/stow/python21_debug/lib/python2.1/asyncore.py|poll|95] [/usr/local/stow/python21_debug/lib/python2.1/asyncore.py|handle_read_event|383] [/home/jheintz/lib/python/ZEO/smac.py|handle_read|119] [/usr/local/stow/python21_debug/lib/python2.1/asyncore.py|recv|349]) This is certainly a standard error that recv() can raise, although I am surprised to see it after poll() says that the socket is ready to read. The Linux man page for recv() says: EAGAIN The socket is marked non-blocking and the receive operation would block, or a receive timeout had been set and the timeout expired before data was received. It seems to me that asyncore should be prepared to catch this error on recv(), but it does not. Is this any oversight? Would it make sense to fix in the Python source tree? Jeremy From rushing@n... Sat Aug 11 04:04:57 2001 From: rushing@n... (Sam Rushing) Date: Fri, 10 Aug 2001 20:04:57 -0700 Subject: [medusa] Adding/removing handlers dynamically References: <3BC7333BE30CD511AEB700B0D0AB2F5803D262@L...> Message-ID: <3B74A0D9.B17BAFDC@n...> Thomas Weholt wrote: > I need to add/remove handlers dynamically. I got a modular web-server that > searches a given folder for new handlers, when it finds one it would be cool > to have it install it without restarting the server, all automatic. If I > could build a server that search for new handlers at a given interval, > installs all new, removes all handlers that have been removed, it would open > for some really cool features. > > Any hints on how to implement a way of loading handlers dynamically would > also be great. The top-level handlers list is just that, a list. The position in the list controls the priority of a match. You can remove handlers by calling 'handlers.remove()'... You could make a new kind of top-level handler that implemented your behavior - so for example, your 'match' method would search its own list/dict/whatever for matches under that portion of the tree. Hmm... a dictionary would probably work pretty well. Then your top-level handler could be installed in a 'subdirectory', or perhaps as a default/fallback handler that you would install at the end of the list. -Sam From max@a... Mon Aug 13 23:50:25 2001 From: max@a... (max@a...) Date: Mon, 13 Aug 2001 22:50:25 -0000 Subject: SMTP delivery service for Medusa? Message-ID: <9l9ljh+qmu5@e...> In the Web-based game that I'm working on (Interstelen, http://www.interstelen.com/), which is written in Python and Medusa, I'm going to have the need to deliver email to players for such things as account registration, notifications, and so on. For the time being I'm just using smtplib which works, of course, but isn't select-friendly and so will block the whole server if there is some glitch in the communication stream with the SMTP server. Is there an SMTP delivery module available which uses Medusa (or just asyncore/asynchat of course) to do its delivery and properly defers things in nice friendly ways? I can obviously write my own, but would prefer not to if there's no need. Note that in my case I only need to do SMTP _delivery_ (as a client), I don't have to act as an SMTP server. Thanks. From jim@z... Tue Aug 14 13:44:15 2001 From: jim@z... (Jim Fulton) Date: Tue, 14 Aug 2001 08:44:15 -0400 Subject: [medusa] SMTP delivery service for Medusa? References: <9l9ljh+qmu5@e...> Message-ID: <3B791D1F.A954D834@z...> max@a... wrote: > > In the Web-based game that I'm working on (Interstelen, > http://www.interstelen.com/), which is written in Python and Medusa, > I'm going to have the need to deliver email to players for such things > as account registration, notifications, and so on. For the time being > I'm just using smtplib which works, of course, but isn't > select-friendly and so will block the whole server if there is some > glitch in the communication stream with the SMTP server. > > Is there an SMTP delivery module available which uses Medusa (or just > asyncore/asynchat of course) to do its delivery and properly defers > things in nice friendly ways? I can obviously write my own, but would > prefer not to if there's no need. I would like this too. > Note that in my case I only need to do SMTP _delivery_ (as a client), > I don't have to act as an SMTP server. Note that Python 2.1 includes an asyncore-based smtp server in the standard library. Jim -- Jim Fulton mailto:jim@d... Python Powered! Technical Director (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From spex66@y... Tue Aug 21 17:00:29 2001 From: spex66@y... (spex66@y...) Date: Tue, 21 Aug 2001 16:00:29 -0000 Subject: Blindshot: C-Python-extension thread problems under bobo_handler 1.03 Message-ID: <9lu0it+pduh@e...> Hi, strange things happend and exceeded my knowledge... so a short description: (1) Python module with import a c-python-extension (made with SWIG, target is database called Gigabase) (2) Test module "mbase" for publishing has objects w/ methods which throws HTML in return (3) Testing this module "mbase" in python interpreter works fine... database-accesses are correct (4) start_my_medusa with folowing snippet: # ============================================================== # BOBO Handler # ============================================================== import bobo_handler import mbase bfh = bobo_handler.bobo_handler(mbase, debug=1) hs.install_handler(bfh) (5) same access at database went now down into a core-dump. And the only hint that I have is that a start of a transaction with new thread went wrong sorry for this rough overview, but as mentioned as a blindshot has anyone an idea, if this problem is a collision from i.e. asyncore and a C-Extension??? thanks for any tip Peter (=PA=) And I've thought the "bloody" parts are over... fixing regex to re, till I got my POST variables in my methods... gnaaag I'm using Python 2.1, W2k, SWIG 1.3.6 with MS-VC6 with Multithreaded Debugging -MTd option From max@a... Sat Aug 25 20:14:49 2001 From: max@a... (max@a...) Date: Sat, 25 Aug 2001 19:14:49 -0000 Subject: SMTP delivery service for Medusa? In-Reply-To: <3B791D1F.A954D834@z...> Message-ID: <9m8tf9+lfrf@e...> --- In medusa@y..., Jim Fulton wrote: > Note that Python 2.1 includes an asyncore-based smtp server in the > standard > library. Oh really? I didn't realize that. Interesting. I'm looking at the library reference for 2.1.1. All that jumps out at me is smtplib, which is the legacy SMTP client (and which I presume still does not use asyncore). What's the one you're referring to? Thanks.