From jdhardy at gmail.com Fri Dec 5 02:57:12 2008 From: jdhardy at gmail.com (Jeff Hardy) Date: Thu, 4 Dec 2008 18:57:12 -0700 Subject: [Web-SIG] NWSGI 0.7 released Message-ID: NWSGI 0.7 is now available. This version adds support for ASP.NET sessions, fixes Unicode handling, and adds support for wildcard mappings. It is available at http://www.codeplex.com/NWSGI/Release/ProjectReleases.aspx?ReleaseId=20189. What is NWSGI? NWSGI is an ASP.NET HttpHandler that implements the Python WSGI spec (PEP 333 - http://www.python.org/dev/peps/pep-0333/) for interfacing web servers and web frameworks using IronPython. There are still some IronPYthon bugs that prevent everything from working, but some things do. For more details on this release, see http://jdhardy.blogspot.com/2008/12/nwsgi-07-released.html. - Jeff From graham.dumpleton at gmail.com Fri Dec 12 12:46:38 2008 From: graham.dumpleton at gmail.com (Graham Dumpleton) Date: Fri, 12 Dec 2008 22:46:38 +1100 Subject: [Web-SIG] wsgiref.validate allows wsgi.input.read() with no argument Message-ID: <88e286470812120346q2d93a64ao70552708e79a3933@mail.gmail.com> Just noticed that although WSGI PEP doesn't specifically mention that argument to read() on wsgi.input is optional, wsgiref.validate allows calling read() with no argument. >From wsgiref.validate: """ * That wsgi.input is used properly: - .read() is called with zero or one argument class InputWrapper: def read(self, *args): assert_(len(args) <= 1) v = self.input.read(*args) assert_(type(v) is type("")) return v """ Of course, the issue is still that WSGI PEP says: """The server is not required to read past the client's specified Content-Length, and ***is allowed to simulate an end-of-file condition if the application attempts to read past that point***.""" Thus, simulating end-of-file condition is not mandatory, and so if something does call read() with no argument, not guaranteed it will actually return. The problem case here being where HTTP/1.1 request pipelining is being used and WSGI adapter is using raw socket as wsgi.input. The socket isn't going to be closed in this case as potentially another request to yet be sent over same socket. Thus application call of read() with no arguments will hang until client decides to close socket connection. Anyway, I thought it was interesting that wsgiref.validate accepts that argument read() can be optional. Supports further that this part of WSGI PEP needs to be clarified and end-of-file condition perhaps made mandatory. Graham From manlio_perillo at libero.it Fri Dec 12 13:46:21 2008 From: manlio_perillo at libero.it (Manlio Perillo) Date: Fri, 12 Dec 2008 13:46:21 +0100 Subject: [Web-SIG] wsgiref.validate allows wsgi.input.read() with no argument In-Reply-To: <88e286470812120346q2d93a64ao70552708e79a3933@mail.gmail.com> References: <88e286470812120346q2d93a64ao70552708e79a3933@mail.gmail.com> Message-ID: <49425D1D.1010904@libero.it> Graham Dumpleton ha scritto: > Just noticed that although WSGI PEP doesn't specifically mention that > argument to read() on wsgi.input is optional, wsgiref.validate allows > calling read() with no argument. > wsgiref.validate makes also other assumptions about a WSGI application that are not required by the WSGI PEP. As an example it reports as an error the presence in the environ dictionary of HTTP_CONTENT_TYPE and HTTP_CONTENT_LENGTH, but the PEP says nothing about this, and CGI [1] says: """"The server may exclude any headers which it has already processed, such as Authorization, Content-type, and Content-length. If necessary, the server may choose to exclude any or all of these headers if including them would exceed any system environment limits.""" [1] http://hoohoo.ncsa.uiuc.edu/cgi/env.html P.S.: the link "http://cgi-spec.golux.com/draft-coar-cgi-v11-03.txt" is broken. > [...] Regards Manlio Perillo From ianb at colorstudy.com Fri Dec 12 18:58:48 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 12 Dec 2008 11:58:48 -0600 Subject: [Web-SIG] wsgiref.validate allows wsgi.input.read() with no argument In-Reply-To: <88e286470812120346q2d93a64ao70552708e79a3933@mail.gmail.com> References: <88e286470812120346q2d93a64ao70552708e79a3933@mail.gmail.com> Message-ID: <4942A658.2090207@colorstudy.com> Graham Dumpleton wrote: > Just noticed that although WSGI PEP doesn't specifically mention that > argument to read() on wsgi.input is optional, wsgiref.validate allows > calling read() with no argument. > >>From wsgiref.validate: > > """ > * That wsgi.input is used properly: > > - .read() is called with zero or one argument > > class InputWrapper: > > def read(self, *args): > assert_(len(args) <= 1) > v = self.input.read(*args) > assert_(type(v) is type("")) > return v > """ > > Of course, the issue is still that WSGI PEP says: > > """The server is not required to read past the client's specified > Content-Length, and ***is allowed to simulate an end-of-file condition > if the application attempts to read past that point***.""" An application that relies on the server to simulate end-of-file will be a broken application on some servers. This is not an uncommon problem. Therefore the validator tests for this case; if you want an application that actually works consistently, you shouldn't do environ['wsgi.input'].read(). -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From graham.dumpleton at gmail.com Fri Dec 12 23:52:09 2008 From: graham.dumpleton at gmail.com (Graham Dumpleton) Date: Sat, 13 Dec 2008 09:52:09 +1100 Subject: [Web-SIG] wsgiref.validate allows wsgi.input.read() with no argument In-Reply-To: <4942A658.2090207@colorstudy.com> References: <88e286470812120346q2d93a64ao70552708e79a3933@mail.gmail.com> <4942A658.2090207@colorstudy.com> Message-ID: <88e286470812121452i3992df64lbc94370921805922@mail.gmail.com> 2008/12/13 Ian Bicking : > Graham Dumpleton wrote: >> >> Just noticed that although WSGI PEP doesn't specifically mention that >> argument to read() on wsgi.input is optional, wsgiref.validate allows >> calling read() with no argument. >> >>> From wsgiref.validate: >> >> """ >> * That wsgi.input is used properly: >> >> - .read() is called with zero or one argument >> >> class InputWrapper: >> >> def read(self, *args): >> assert_(len(args) <= 1) >> v = self.input.read(*args) >> assert_(type(v) is type("")) >> return v >> """ >> >> Of course, the issue is still that WSGI PEP says: >> >> """The server is not required to read past the client's specified >> Content-Length, and ***is allowed to simulate an end-of-file condition >> if the application attempts to read past that point***.""" > > An application that relies on the server to simulate end-of-file will be a > broken application on some servers. This is not an uncommon problem. > Therefore the validator tests for this case; if you want an application > that actually works consistently, you shouldn't do > environ['wsgi.input'].read(). The validator does not test for that case, that is what I am pointing out. The validator allows read() to be called with no argument. Graham From ianb at colorstudy.com Sun Dec 14 01:08:10 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Sat, 13 Dec 2008 18:08:10 -0600 Subject: [Web-SIG] wsgiref.validate allows wsgi.input.read() with no argument In-Reply-To: <88e286470812121452i3992df64lbc94370921805922@mail.gmail.com> References: <88e286470812120346q2d93a64ao70552708e79a3933@mail.gmail.com> <4942A658.2090207@colorstudy.com> <88e286470812121452i3992df64lbc94370921805922@mail.gmail.com> Message-ID: <49444E6A.8090705@colorstudy.com> Graham Dumpleton wrote: >> An application that relies on the server to simulate end-of-file will be a >> broken application on some servers. This is not an uncommon problem. >> Therefore the validator tests for this case; if you want an application >> that actually works consistently, you shouldn't do >> environ['wsgi.input'].read(). > > The validator does not test for that case, that is what I am pointing > out. The validator allows read() to be called with no argument. Ah, sorry, I wasn't paying attention... okay, then yes, I agree -- the validator should be more restrictive. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From manlio_perillo at libero.it Sun Dec 14 11:23:37 2008 From: manlio_perillo at libero.it (Manlio Perillo) Date: Sun, 14 Dec 2008 11:23:37 +0100 Subject: [Web-SIG] handling URLs with ending slash Message-ID: <4944DEA9.5090105@libero.it> Hi. In my WSGI applications I always have an ending slash to the URLs. This means that an URL without the ending slash will cause the underlying resource to return 404 Not Found HTTP response. What is the best method to handle this, using a regex based URL dispatcher? I'm planning to add an option to my URL dispatcher to force any URL to have an ending slash (as an example requesting an HTTP redirect - either 302 or 301, or by just internally modifying the URL), but I'm not sure this is the best solution. Thanks Manlio Perillo From t.broyer at gmail.com Sun Dec 14 15:45:13 2008 From: t.broyer at gmail.com (Thomas Broyer) Date: Sun, 14 Dec 2008 15:45:13 +0100 Subject: [Web-SIG] handling URLs with ending slash In-Reply-To: <4944DEA9.5090105@libero.it> References: <4944DEA9.5090105@libero.it> Message-ID: On Sun, Dec 14, 2008 at 11:23 AM, Manlio Perillo wrote: > > In my WSGI applications I always have an ending slash to the URLs. > This means that an URL without the ending slash will cause the underlying > resource to return 404 Not Found HTTP response. > > What is the best method to handle this, using a regex based URL dispatcher? I would add some kind of "catch-all entry" to dispatch to a "trailing slash redirector" WSGI app: routes.add("[^/]$", force_trailing_slash) or eventually add a WSGI middleware to each mapped application (...that need such a treatment, could be all of them) that would issue a redirect to the "slash-appended" URL when needed, or just pass through to the application otherwise: routes.add(, force_trailing_slash(my_application)) > I'm planning to add an option to my URL dispatcher to force any URL to have > an ending slash (as an example requesting an HTTP redirect - either 302 or > 301, or by just internally modifying the URL), but I'm not sure this is the > best solution. A 301 is what webservers (such as Apache) generally do for resources served from file systems (when asked for a directory/folder). Internally modifying the URL could easily yield bad results, as it has an impact on relative URLs resolution. Anyway, I wouldn't build it inside the dispatcher (just a matter of taste; it's useful, whichever dispatcher you're using) -- Thomas Broyer From randy at rcs-comp.com Sun Dec 14 19:54:03 2008 From: randy at rcs-comp.com (Randy Syring) Date: Sun, 14 Dec 2008 13:54:03 -0500 Subject: [Web-SIG] handling URLs with ending slash In-Reply-To: <4944DEA9.5090105@libero.it> References: <4944DEA9.5090105@libero.it> Message-ID: <4945564B.9020403@rcs-comp.com> Manilo, Here is a thread on this topic, well a partial thread, start reading about half way down: http://groups.google.com/group/pylons-discuss/browse_thread/thread/6888b790239b488b I found it informative. -------------------------------------- Randy Syring RCS Computers & Web Solutions 502-644-4776 http://www.rcs-comp.com "Whether, then, you eat or drink or whatever you do, do all to the glory of God." 1 Cor 10:31 Manlio Perillo wrote: > Hi. > > In my WSGI applications I always have an ending slash to the URLs. > This means that an URL without the ending slash will cause the > underlying resource to return 404 Not Found HTTP response. > > What is the best method to handle this, using a regex based URL > dispatcher? > > I'm planning to add an option to my URL dispatcher to force any URL to > have an ending slash (as an example requesting an HTTP redirect - > either 302 or 301, or by just internally modifying the URL), but I'm > not sure this is the best solution. > > > Thanks Manlio Perillo > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/randy%40rcs-comp.com > From manlio_perillo at libero.it Sun Dec 14 20:24:04 2008 From: manlio_perillo at libero.it (Manlio Perillo) Date: Sun, 14 Dec 2008 20:24:04 +0100 Subject: [Web-SIG] handling URLs with ending slash In-Reply-To: <4945564B.9020403@rcs-comp.com> References: <4944DEA9.5090105@libero.it> <4945564B.9020403@rcs-comp.com> Message-ID: <49455D54.9050609@libero.it> Randy Syring ha scritto: > Manilo, > Manlio not Manilo, please! > Here is a thread on this topic, well a partial thread, start reading > about half way down: > > http://groups.google.com/group/pylons-discuss/browse_thread/thread/6888b790239b488b > > > I found it informative. > Thanks, it is interesting. Regards Manlio Perillo From manlio_perillo at libero.it Sun Dec 14 20:28:45 2008 From: manlio_perillo at libero.it (Manlio Perillo) Date: Sun, 14 Dec 2008 20:28:45 +0100 Subject: [Web-SIG] handling URLs with ending slash In-Reply-To: References: <4944DEA9.5090105@libero.it> Message-ID: <49455E6D.6030400@libero.it> Thomas Broyer ha scritto: > On Sun, Dec 14, 2008 at 11:23 AM, Manlio Perillo wrote: >> In my WSGI applications I always have an ending slash to the URLs. >> This means that an URL without the ending slash will cause the underlying >> resource to return 404 Not Found HTTP response. >> >> What is the best method to handle this, using a regex based URL dispatcher? > > I would add some kind of "catch-all entry" to dispatch to a "trailing > slash redirector" WSGI app: > > routes.add("[^/]$", force_trailing_slash) > I not sure I like this. > or eventually add a WSGI middleware to each mapped application The URL dispatcher is a WSGI middleware, so it is ok for me to do this in the url dispatcher. I would like to keep the numbers of middleware to a minimun (function calls in Python are not cheap). > (...that need such a treatment, could be all of them) that would issue > a redirect to the "slash-appended" URL when needed, or just pass > through to the application otherwise: > > routes.add(, force_trailing_slash(my_application)) > Yes, that's an idea. Note that this is a special case of an "URL normalizer" middleware. A middleware can be used as a function decorator. This is a point in favour to use a dedicated middleware for this. Thanks Manlio Perillo From manlio_perillo at libero.it Wed Dec 17 17:10:53 2008 From: manlio_perillo at libero.it (Manlio Perillo) Date: Wed, 17 Dec 2008 17:10:53 +0100 Subject: [Web-SIG] logging support in a multiprocess web server Message-ID: <4949248D.5070709@libero.it> Hi. I have noted that some WSGI based web applications use the standard logging module, for logging. However I have some doubts about how this works when the application is embedded in a web server that uses multiple processes (like Nginx or Apache with prefork). Thanks Manlio Perillo From graham.dumpleton at gmail.com Wed Dec 17 21:43:18 2008 From: graham.dumpleton at gmail.com (Graham Dumpleton) Date: Thu, 18 Dec 2008 07:43:18 +1100 Subject: [Web-SIG] logging support in a multiprocess web server In-Reply-To: <4949248D.5070709@libero.it> References: <4949248D.5070709@libero.it> Message-ID: <88e286470812171243s4fd366b8q70309b3d380bd25c@mail.gmail.com> 2008/12/18 Manlio Perillo : > Hi. > > I have noted that some WSGI based web applications use the standard logging > module, for logging. > > However I have some doubts about how this works when the application is > embedded in a web server that uses multiple processes (like Nginx or Apache > with prefork). The logging module principally provides the interface for logging. If none of the supplied output logging handlers are suitable for what you want then you create and register your own. What underlying logging handler you use isn't going to change the high level interface that WSGI based web applications are going to use. For multiple process applications, one option is logging.handlers.SysLogHandler. In Apache/mod_wsgi, since sys.stderr is already linked to Apache error logs and deals with multiple processes, you could just use a handler which outputs to sys.stderr. Only issue with Apache/mod_wsgi is that sys.stderr gets logged in Apache as ERROR level only, with logging module log levels being a secondary layer within that. Graham From manlio_perillo at libero.it Thu Dec 18 14:52:50 2008 From: manlio_perillo at libero.it (Manlio Perillo) Date: Thu, 18 Dec 2008 14:52:50 +0100 Subject: [Web-SIG] setup_testing_defaults and SERVER_PROTOCOL Message-ID: <494A55B2.7020204@libero.it> Isn't more appropriate for wsgiref.util.setup_testing_defaults function to set SERVER_PROTOCOL to HTTP/1.1, instead of HTTP/1.0, since HTTP/1.1 is the current version of the protocol? Thanks Manlio Perillo From davidgshi at yahoo.co.uk Thu Dec 18 16:44:44 2008 From: davidgshi at yahoo.co.uk (David Shi) Date: Thu, 18 Dec 2008 15:44:44 +0000 (GMT) Subject: [Web-SIG] Fw: Extracting data from XML containing CDATA and store data in a .dbf file Message-ID: <334465.27200.qm@web26302.mail.ukl.yahoo.com> From: David Shi Subject: Extracting data from XML containing CDATA and store data in a .dbf file To: help at python.org Date: Thursday, 18 December, 2008, 3:41 PM I am looking for advice/efficient script to extract data from XML containing CDATA. ? I wish to extract all the data and store it in a .dbf file. ? Regards. ? David -------------- next part -------------- An HTML attachment was scrubbed... URL: From dima at hlabs.spb.ru Tue Dec 23 22:56:46 2008 From: dima at hlabs.spb.ru (Dmitry Vasiliev) Date: Wed, 24 Dec 2008 00:56:46 +0300 Subject: [Web-SIG] WSGI and Py3k Message-ID: <49515E9E.3010206@hlabs.spb.ru> Hi! Currently wsgiref package in Py3k just broken so I've filled ticket #4718 (http://bugs.python.org/issue4718). Actually there are more than one ticket about the problem - see the link for details. In a comment to the ticket Phillip said we need to open discussion here then update PEP-333 and only then fix the package. I think it's important to fix wsgiref *somehow* (for example we can temporarily disable it) in the upcoming Python 3.0.1 release so there is the main question: - Which Py3k data type to use as "strings" mentioned in the PEP? For now the PEP explicitly say to *always* use "str" type (http://www.python.org/dev/peps/pep-0333/#unicode-issues) but in Py3k "str" is just like "unicode" in Python 2.*. Moreover from Py3k's point of view there is a conflict in the PEP because of 'strings must be of type str, and must not be of type unicode'. The type corresponding to "str" from Python 2.* in Py3k is "bytes" so at first it seems convenient to use "bytes". But it violates the current version of the PEP and 2to3 utility (for now?) convert old "str" as new "str", not "bytes". However I believe it's not too hard to encode new "str" (converted by 2to3) to "bytes" before returning from an WSGI application. Also we can support both new "str" and "bytes" in some transitional period. -- Dmitry Vasiliev http://hlabs.spb.ru From graham.dumpleton at gmail.com Tue Dec 23 23:34:53 2008 From: graham.dumpleton at gmail.com (Graham Dumpleton) Date: Wed, 24 Dec 2008 09:34:53 +1100 Subject: [Web-SIG] WSGI and Py3k In-Reply-To: <49515E9E.3010206@hlabs.spb.ru> References: <49515E9E.3010206@hlabs.spb.ru> Message-ID: <88e286470812231434v4a154d42g8ce0c2a476da8cd9@mail.gmail.com> 2008/12/24 Dmitry Vasiliev : > Hi! > > Currently wsgiref package in Py3k just broken so I've filled ticket > #4718 (http://bugs.python.org/issue4718). Actually there are more than > one ticket about the problem - see the link for details. In a comment to > the ticket Phillip said we need to open discussion here then update > PEP-333 and only then fix the package. I think it's important to fix > wsgiref *somehow* (for example we can temporarily disable it) in the > upcoming Python 3.0.1 release so there is the main question: > > - Which Py3k data type to use as "strings" mentioned in the PEP? > > For now the PEP explicitly say to *always* use "str" type > (http://www.python.org/dev/peps/pep-0333/#unicode-issues) but in Py3k > "str" is just like "unicode" in Python 2.*. Moreover from Py3k's point > of view there is a conflict in the PEP because of 'strings must be of > type str, and must not be of type unicode'. > > The type corresponding to "str" from Python 2.* in Py3k is "bytes" so at > first it seems convenient to use "bytes". But it violates the current > version of the PEP and 2to3 utility (for now?) convert old "str" as new > "str", not "bytes". However I believe it's not too hard to encode new > "str" (converted by 2to3) to "bytes" before returning from an WSGI > application. > > Also we can support both new "str" and "bytes" in some transitional period. Have you read: http://www.wsgi.org/wsgi/Amendments_1.0 As much as could be done was worked out quite a while back. I don't quite understand why PJE is ignoring/forgetting about that list in the amendments page. I haven't read the bug reports properly yet, but some of the comments in the thread says certain things are broken yet that is what was worked out already as way of doing it. Have you tried out mod_wsgi (source from subversion repository), which also has been ported to Python 3.0 already as per proposals in that amendments page. When get chance I will read bug reports. Graham From pje at telecommunity.com Wed Dec 24 21:42:08 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 24 Dec 2008 15:42:08 -0500 Subject: [Web-SIG] WSGI and Py3k In-Reply-To: <88e286470812231434v4a154d42g8ce0c2a476da8cd9@mail.gmail.co m> References: <49515E9E.3010206@hlabs.spb.ru> <88e286470812231434v4a154d42g8ce0c2a476da8cd9@mail.gmail.com> Message-ID: <20081224204025.E32B13A40B2@sparrow.telecommunity.com> At 09:34 AM 12/24/2008 +1100, Graham Dumpleton wrote: >Have you read: > > http://www.wsgi.org/wsgi/Amendments_1.0 > >As much as could be done was worked out quite a while back. I don't >quite understand why PJE is ignoring/forgetting about that list in the >amendments page. Because I'm getting forgetful in my old age, apparently. Especially since I believe I was the one who drafted at least part of that amendment. ;-) More likely, it's just that Python 3 is a lot less on my mind these days than finishing the book I'm writing. Even checking email is just something I do to take a break. From jdhardy at gmail.com Sun Dec 28 01:36:18 2008 From: jdhardy at gmail.com (Jeff Hardy) Date: Sat, 27 Dec 2008 17:36:18 -0700 Subject: [Web-SIG] NWSGI 1.0 Released Message-ID: I'm very pleased to announce the release of NWSGI 1.0. NWSGI is an implementation of the WSGI specification (PEP 333) for IronPython. It is implemented as an ASP.NET HttpHandler and is supported on IIS 6 and IIS 7 (and should work on any other ASP.NET-capable web server). NWSGI allows the integration of Python applications with existing ASP.NET facilities such as caching, sessions, and user management. NWSGI 1.0 can be downloaded from http://www.codeplex.com/NWSGI/Release/ProjectReleases.aspx?ReleaseId=18995. More details are available at http://jdhardy.blogspot.com/2008/12/nwsgi-10.html - Jeff