From ianb at colorstudy.com  Thu Aug 16 04:55:53 2007
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 15 Aug 2007 21:55:53 -0500
Subject: [Web-SIG] WebOb
Message-ID: <46C3BCB9.6010708@colorstudy.com>

Howdy all.

Lately I got on a kick and extracted/refined/reimplemented a bunch of 
stuff from Paste.  The result is the not-quite-released WebOb (I don't 
want to do a release until I think people should use it instead of 
Paste, to the degree the two overlap -- and it's not *quite* ready for 
that).

Anyway, I'd be interested in feedback.  We've talked a little about a 
shared request object -- only a little, and I don't know if it is really 
a realistic goal to even try.  But I think this request object is a 
considerably higher quality than any other request objects out there. 
The response object provides a nice symmetry, as well as facilitating 
testing.  And it's also a very nice response object.

They are both fairly reasonable to subclass, if there are minor naming 
issues (if there's really missing features, I'd like to add them 
directly -- though for the response object in particular it's likely 
you'll want to subclass to give application defaults, like a default 
content type).

It's based strictly on WSGI, with the request object an almost-stateless 
wrapper around a WSGI environment, and the response object a WSGI 
application that contains mutable status/headers/app_iter.

Almost all the defined HTTP headers are available as attributes on the 
request and/or response.  I try to parse these in as sensible a way as 
possible, e.g., req.if_modified_since is a datetime object (of course 
unparsed access is also available).  Several objects like 
response.cache_control are a bit more complex, since there's no data 
structure that exactly represents them.  I've tried to make them as easy 
to use as possible for realistic web tasks.

I'm very interested to get any feedback, especially right now when there 
are no backward compatibility concerns.  Right now no critique is too 
large or small.

It's in svn at:
   http://svn.pythonpaste.org/Paste/WebOb/trunk

And there are fairly complete docs at:
   http://pythonpaste.org/webob/

A quick summary of differences in the API and some other 
request/response objects out there:
   http://pythonpaste.org/webob/differences.html
I'd include more frameworks, if you can point me to their 
request/response API documentation (e.g., I looked but couldn't find any 
for Zope 3).

WebOb has a lot more methods and attributes than other libraries, but 
this document points out only things where there are differing names or 
things not in WebOb.  Most other such objects also don't have the same 
WSGI-oriented scope (with the exception of Yaro and paste.wsgiwrappers).

The Request and Response API (extracted docs):
   http://pythonpaste.org/webob/class-webob.Request.html
   http://pythonpaste.org/webob/class-webob.Response.html



-- 
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
             : Write code, do good : http://topp.openplans.org/careers

From cliff at develix.com  Mon Aug 20 23:08:59 2007
From: cliff at develix.com (Cliff Wells)
Date: Mon, 20 Aug 2007 14:08:59 -0700
Subject: [Web-SIG] WSGI diagram
Message-ID: <1187644140.2956.8.camel@portable-evil>

Hi,

There's someone working on a mod_wsgi module for the Nginx HTTP
server/proxy and Nginx's author was asking if there is a diagram
somewhere outlining the WSGI protocol (I think perhaps he'd help if he
understood WSGI).  I google'd a bit but came up empty-handed.

Anyone know of anything?

TIA,

Cliff


From graham.dumpleton at gmail.com  Tue Aug 21 00:33:52 2007
From: graham.dumpleton at gmail.com (Graham Dumpleton)
Date: Tue, 21 Aug 2007 08:33:52 +1000
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <1187644140.2956.8.camel@portable-evil>
References: <1187644140.2956.8.camel@portable-evil>
Message-ID: <88e286470708201533x701b1c0dobe3b5584dd47ba42@mail.gmail.com>

On 21/08/07, Cliff Wells <cliff at develix.com> wrote:
> Hi,
>
> There's someone working on a mod_wsgi module for the Nginx HTTP
> server/proxy and Nginx's author was asking if there is a diagram
> somewhere outlining the WSGI protocol (I think perhaps he'd help if he
> understood WSGI).  I google'd a bit but came up empty-handed.
>
> Anyone know of anything?

Start with the PEP:

  http://www.python.org/dev/peps/pep-0333/

Further information may be found on:

  http://www.wsgi.org/wsgi

The official site for the Apache mod_wsgi even references this
information, although the www.wsgi.org site seems to be unavailable as
I write this.

Other than that, the only thing is the various little articles and
blog entries that people have written. There is no exhaustive or
complete description of WSGI with even the PEP leaving various things
unanswered. Generally people ask questions on this list to fill out
the details on what is meant to occur, so perusing the archives would
also be useful.

Some will no doubt say I am biased and trying to protect my turf, but
from what I understand of the aims of Nginx HTTP, in as much as it is
trying to create the fastest web server around, embedding a Python
interpreter within the main server processes would work against that.
This is because supporting Python in an embedded manner may impact
performance in serving static files etc.

Thus, it may make more sense to be suggesting people use the existing
FASTCGI support in Nginx HTTP for hosting Python WSGI applications.
That way you aren't necessarily burdening the main server processes
with the extra overhead. The same argument applies to lighttpd as
well. As to Apache, many have the opinion that it is getting quite
bloated anyway, so adding more stuff into the main server processes
doesn't seem to worry people as much. It does though need to be used
appropriately though.

Also note that WSGI isn't a wire protocol like FASTCGI is. It is more
an programmatic interface definition for adapting a Python application
on top of a specific web server.  If you specifically want to run WSGI
applications in a separate daemon process, then you would need to come
up with your own wire protocol for communicating between the main
server processes and the daemon process. The daemon mode of the Apache
mod_wsgi does this, but since this is what FASTCGI defines anyway,
that is why it may make more sense to use the existing FASTCGI support
in Nginx HTTP and not bother with a separate module. In Apache
mod_wsgi, a new FASTCGI like daemon mode was specifically implemented
to try and come up with an improvement over the sometimes hard to
configure and get working FASTCGI implementations for Apache.

BTW, I hope you would call it something other than mod_wsgi so as not
to cause confusion with the Apache implementation.

Graham

From pje at telecommunity.com  Tue Aug 21 00:42:37 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 20 Aug 2007 18:42:37 -0400
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <1187644140.2956.8.camel@portable-evil>
References: <1187644140.2956.8.camel@portable-evil>
Message-ID: <20070820224015.9D6C33A4072@sparrow.telecommunity.com>

At 02:08 PM 8/20/2007 -0700, Cliff Wells wrote:
>Hi,
>
>There's someone working on a mod_wsgi module for the Nginx HTTP
>server/proxy and Nginx's author was asking if there is a diagram
>somewhere outlining the WSGI protocol (I think perhaps he'd help if he
>understood WSGI).  I google'd a bit but came up empty-handed.
>
>Anyone know of anything?

I don't know of anything.  But it might help to explain that WSGI 
isn't really a protocol, unless you consider e.g. Java servlets to be 
a "protocol".

To implement WSGI in a non-Python web server, you need to invoke 
Python code via the Python C API.

Pardon me if any of this is already obvious, but from your 
description it sounds like one or more parties might not be aware of 
these points.


From cliff at develix.com  Tue Aug 21 01:18:06 2007
From: cliff at develix.com (Cliff Wells)
Date: Mon, 20 Aug 2007 16:18:06 -0700
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <20070820224015.9D6C33A4072@sparrow.telecommunity.com>
References: <1187644140.2956.8.camel@portable-evil>
	<20070820224015.9D6C33A4072@sparrow.telecommunity.com>
Message-ID: <1187651886.2956.37.camel@portable-evil>

On Mon, 2007-08-20 at 18:42 -0400, Phillip J. Eby wrote:

> Pardon me if any of this is already obvious, but from your 
> description it sounds like one or more parties might not be aware of 
> these points.

One of the parties (Igor, Nginx's author) is certainly not aware of
these points (hence his seeking a schematic so he could get a quick
overview).  The author of the upcoming module is aware but didn't offer
much in the way of explanation when Igor queried him.

Regards,
Cliff



From cliff at develix.com  Tue Aug 21 01:18:11 2007
From: cliff at develix.com (Cliff Wells)
Date: Mon, 20 Aug 2007 16:18:11 -0700
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <88e286470708201533x701b1c0dobe3b5584dd47ba42@mail.gmail.com>
References: <1187644140.2956.8.camel@portable-evil>
	<88e286470708201533x701b1c0dobe3b5584dd47ba42@mail.gmail.com>
Message-ID: <1187651891.2956.39.camel@portable-evil>

On Tue, 2007-08-21 at 08:33 +1000, Graham Dumpleton wrote:
> On 21/08/07, Cliff Wells <cliff at develix.com> wrote:
> > Hi,
> >
> > There's someone working on a mod_wsgi module for the Nginx HTTP
> > server/proxy and Nginx's author was asking if there is a diagram
> > somewhere outlining the WSGI protocol (I think perhaps he'd help if he
> > understood WSGI).  I google'd a bit but came up empty-handed.
> >

> Some will no doubt say I am biased and trying to protect my turf, but
> from what I understand of the aims of Nginx HTTP, in as much as it is
> trying to create the fastest web server around, embedding a Python
> interpreter within the main server processes would work against that.
> This is because supporting Python in an embedded manner may impact
> performance in serving static files etc.

The author of the module says the recommended deployment method will be
running a second Nginx server dedicated to the WSGI process and proxy to
that with a main Nginx server, probably for this very reason.

Personally I think he'd do better implementing a mod_ajp and then
utilizing the ajp-wsgi [1] server since this would probably be easier to
implement and I suspect ajp-wsgi has a smaller footprint than Nginx
(even though Nginx is pretty small), but it's not my project =)

> Thus, it may make more sense to be suggesting people use the existing
> FASTCGI support in Nginx HTTP for hosting Python WSGI applications.
> That way you aren't necessarily burdening the main server processes
> with the extra overhead. 

Lots of people do this or proxy to a Python HTTP server.  I think the
idea is to get rid of one layer of relatively inefficient processing (a
HTTP server in Python) by doing it in C.  How much performance impact
this will actually have in real world applications I don't know
(although Apache's mod_wsgi should give a clue, if there were any
benchmarks published).

> The same argument applies to lighttpd as
> well. As to Apache, many have the opinion that it is getting quite
> bloated anyway, so adding more stuff into the main server processes
> doesn't seem to worry people as much. It does though need to be used
> appropriately though.

Since it's a module, I don't think it will worry people too much ;-)
Nginx has plenty of optional stuff already.  The people who need or want
it will use it, everyone else will skip it. 

> Also note that WSGI isn't a wire protocol like FASTCGI is. It is more
> an programmatic interface definition for adapting a Python application
> on top of a specific web server.  If you specifically want to run WSGI
> applications in a separate daemon process, then you would need to come
> up with your own wire protocol for communicating between the main
> server processes and the daemon process. The daemon mode of the Apache
> mod_wsgi does this, but since this is what FASTCGI defines anyway,
> that is why it may make more sense to use the existing FASTCGI support
> in Nginx HTTP and not bother with a separate module. 

Except that this still requires a fcgi->wsgi gateway that would probably
be just as complicated as direct WSGI support (or done inefficiently in
Python, such as flup).

> In Apache
> mod_wsgi, a new FASTCGI like daemon mode was specifically implemented
> to try and come up with an improvement over the sometimes hard to
> configure and get working FASTCGI implementations for Apache.

Personally, I think FastCGI is too difficult on every server and that
may be another motivation for the author.

> BTW, I hope you would call it something other than mod_wsgi so as not
> to cause confusion with the Apache implementation.

Again, I'm not the author, but I'll bring this up.  However, I'm not
certain it would be any worse than the plethora of mod_fcgi's or
mod_proxy's or mod_xxx's in existing webservers.

Regards,
Cliff

[1] http://www.saddi.com/software/news/archives/49-ajp-wsgi.html




From ianb at colorstudy.com  Tue Aug 21 01:21:04 2007
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 20 Aug 2007 18:21:04 -0500
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <1187644140.2956.8.camel@portable-evil>
References: <1187644140.2956.8.camel@portable-evil>
Message-ID: <46CA21E0.9090503@colorstudy.com>

Cliff Wells wrote:
> Hi,
> 
> There's someone working on a mod_wsgi module for the Nginx HTTP
> server/proxy and Nginx's author was asking if there is a diagram
> somewhere outlining the WSGI protocol (I think perhaps he'd help if he
> understood WSGI).  I google'd a bit but came up empty-handed.
> 
> Anyone know of anything?

I've seen a few I think, but I'm not sure -- probably nothing that would 
help him that much.  There's some links on 
http://wsgi.org/wsgi/Learn_WSGI -- but I think they are all more for 
applications, not for people implementing servers.

If Nginx wanted to support Python really well, I think doing process 
management would be the most ideal way to do that.  That's kind of what 
FastCGI does, but with lots of different options and some general 
confusion.  Nginx could potentially implement FastCGI (or a protocol 
like SCGI or any other over-the-wire protocol they thought best) with 
intelligently chosen process management, potentially with a client 
library (maybe a customized Flup) that invoked everything properly.

It's really more of an integration issue than a protocol issue.  But if 
Nginx makes the integration clear, easy, and robust then they'd create 
something quite useful.  And probably applicable to other languages like 
Ruby as well.


-- 
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
             : Write code, do good : http://topp.openplans.org/careers

From graham.dumpleton at gmail.com  Tue Aug 21 01:29:09 2007
From: graham.dumpleton at gmail.com (Graham Dumpleton)
Date: Tue, 21 Aug 2007 09:29:09 +1000
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <1187651886.2956.37.camel@portable-evil>
References: <1187644140.2956.8.camel@portable-evil>
	<20070820224015.9D6C33A4072@sparrow.telecommunity.com>
	<1187651886.2956.37.camel@portable-evil>
Message-ID: <88e286470708201629o5b53f240p1058c8160a64c547@mail.gmail.com>

On 21/08/07, Cliff Wells <cliff at develix.com> wrote:
> On Mon, 2007-08-20 at 18:42 -0400, Phillip J. Eby wrote:
>
> > Pardon me if any of this is already obvious, but from your
> > description it sounds like one or more parties might not be aware of
> > these points.
>
> One of the parties (Igor, Nginx's author) is certainly not aware of
> these points (hence his seeking a schematic so he could get a quick
> overview).  The author of the upcoming module is aware but didn't offer
> much in the way of explanation when Igor queried him.

If the person doing this is called Igor, then they aren't the only
person trying to implement an mod_wsgi for Nginx HTTP. After dealing
with this email, found a message in my spam box from someone else
doing the same thing. This other persons name is Manlio Perillo. I
have pointed them to this discussion on Google groups, so maybe they
will explain what they are up to and all parties can get together. I
have already indicated to Manlio that I may be interested in providing
him with assistance in doing for Nginx HTTP what I have already done
in Apache mod_wsgi.

Graham

From ianb at colorstudy.com  Tue Aug 21 01:32:50 2007
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 20 Aug 2007 18:32:50 -0500
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <1187651891.2956.39.camel@portable-evil>
References: <1187644140.2956.8.camel@portable-evil>	<88e286470708201533x701b1c0dobe3b5584dd47ba42@mail.gmail.com>
	<1187651891.2956.39.camel@portable-evil>
Message-ID: <46CA24A2.3040901@colorstudy.com>

Cliff Wells wrote:
>> In Apache
>> mod_wsgi, a new FASTCGI like daemon mode was specifically implemented
>> to try and come up with an improvement over the sometimes hard to
>> configure and get working FASTCGI implementations for Apache.
> 
> Personally, I think FastCGI is too difficult on every server and that
> may be another motivation for the author.

If they ship their own supported FastCGI Python client, then this 
shouldn't be a problem.  Because of WSGI they don't really need to 
support different FastCGI Python clients; or, for that matter, if they 
can get the FastCGI system to work well then they don't need to worry 
about other ways of embedding Python.

The problem with FastCGI is when it's ad hoc recipes and supports 
multiple clients of different quality or features.  But there's nothing 
stopping them from skipping that, supporting one FastCGI Python client, 
and having a very simple system as a result.


-- 
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
             : Write code, do good : http://topp.openplans.org/careers

From arnarbi at gmail.com  Tue Aug 21 02:07:42 2007
From: arnarbi at gmail.com (Arnar Birgisson)
Date: Tue, 21 Aug 2007 00:07:42 +0000
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <88e286470708201629o5b53f240p1058c8160a64c547@mail.gmail.com>
References: <1187644140.2956.8.camel@portable-evil>
	<20070820224015.9D6C33A4072@sparrow.telecommunity.com>
	<1187651886.2956.37.camel@portable-evil>
	<88e286470708201629o5b53f240p1058c8160a64c547@mail.gmail.com>
Message-ID: <28012bc60708201707j762a28lafa7b60db3c6dc11@mail.gmail.com>

On 8/20/07, Graham Dumpleton <graham.dumpleton at gmail.com> wrote:
> On 21/08/07, Cliff Wells <cliff at develix.com> wrote:
> > On Mon, 2007-08-20 at 18:42 -0400, Phillip J. Eby wrote:
> >
> > > Pardon me if any of this is already obvious, but from your
> > > description it sounds like one or more parties might not be aware of
> > > these points.
> >
> > One of the parties (Igor, Nginx's author) is certainly not aware of
> > these points (hence his seeking a schematic so he could get a quick
> > overview).  The author of the upcoming module is aware but didn't offer
> > much in the way of explanation when Igor queried him.
>
> If the person doing this is called Igor, then they aren't the only
> person trying to implement an mod_wsgi for Nginx HTTP. After dealing
> with this email, found a message in my spam box from someone else
> doing the same thing. This other persons name is Manlio Perillo.

>From the Nginx ML archives [1] it seems Manlio is the person Cliff
refers to. Igor is the author of Nginx - not the one writing
"mod_wsgi" for it.

Just to clarify..

Arnar

[1] http://news.gmane.org/gmane.comp.web.nginx.english

From cliff at develix.com  Tue Aug 21 02:09:31 2007
From: cliff at develix.com (Cliff Wells)
Date: Mon, 20 Aug 2007 17:09:31 -0700
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <88e286470708201629o5b53f240p1058c8160a64c547@mail.gmail.com>
References: <1187644140.2956.8.camel@portable-evil>
	<20070820224015.9D6C33A4072@sparrow.telecommunity.com>
	<1187651886.2956.37.camel@portable-evil>
	<88e286470708201629o5b53f240p1058c8160a64c547@mail.gmail.com>
Message-ID: <1187654972.2956.47.camel@portable-evil>

On Tue, 2007-08-21 at 09:29 +1000, Graham Dumpleton wrote:
> On 21/08/07, Cliff Wells <cliff at develix.com> wrote:
> > On Mon, 2007-08-20 at 18:42 -0400, Phillip J. Eby wrote:
> >
> > > Pardon me if any of this is already obvious, but from your
> > > description it sounds like one or more parties might not be aware of
> > > these points.
> >
> > One of the parties (Igor, Nginx's author) is certainly not aware of
> > these points (hence his seeking a schematic so he could get a quick
> > overview).  The author of the upcoming module is aware but didn't offer
> > much in the way of explanation when Igor queried him.
> 
> If the person doing this is called Igor, then they aren't the only
> person trying to implement an mod_wsgi for Nginx HTTP. 

No, Igor isn't trying to implement it.  He's the main Nginx author.  I
think he was just trying to understand what it is people are trying to
implement (and perhaps to understand if it's something he'd want to
accept as part of Nginx proper - but I doubt that will happen, for the
very reasons you outlined earlier).

> After dealing
> with this email, found a message in my spam box from someone else
> doing the same thing. This other persons name is Manlio Perillo. I
> have pointed them to this discussion on Google groups, so maybe they
> will explain what they are up to and all parties can get together. I
> have already indicated to Manlio that I may be interested in providing
> him with assistance in doing for Nginx HTTP what I have already done
> in Apache mod_wsgi.

Manlio is, in fact, the person I've been speaking of.   Your help would
be terrific.  I'd actually meant to approach you about it before, but
didn't know how interested you'd be in a Nginx port.

Cliff


From graham.dumpleton at gmail.com  Tue Aug 21 02:19:52 2007
From: graham.dumpleton at gmail.com (Graham Dumpleton)
Date: Tue, 21 Aug 2007 10:19:52 +1000
Subject: [Web-SIG] WSGI diagram
In-Reply-To: <1187654972.2956.47.camel@portable-evil>
References: <1187644140.2956.8.camel@portable-evil>
	<20070820224015.9D6C33A4072@sparrow.telecommunity.com>
	<1187651886.2956.37.camel@portable-evil>
	<88e286470708201629o5b53f240p1058c8160a64c547@mail.gmail.com>
	<1187654972.2956.47.camel@portable-evil>
Message-ID: <88e286470708201719t44c6ca60ncbb87c6e9fe92ff0@mail.gmail.com>

On 21/08/07, Cliff Wells <cliff at develix.com> wrote:
> On Tue, 2007-08-21 at 09:29 +1000, Graham Dumpleton wrote:
> > After dealing
> > with this email, found a message in my spam box from someone else
> > doing the same thing. This other persons name is Manlio Perillo. I
> > have pointed them to this discussion on Google groups, so maybe they
> > will explain what they are up to and all parties can get together. I
> > have already indicated to Manlio that I may be interested in providing
> > him with assistance in doing for Nginx HTTP what I have already done
> > in Apache mod_wsgi.
>
> Manlio is, in fact, the person I've been speaking of.   Your help would
> be terrific.  I'd actually meant to approach you about it before, but
> didn't know how interested you'd be in a Nginx port.

Because Apache mod_wsgi is to a degree quite wedded to the Apache
APIs, not sure I'd really call it a port, although there are certainly
things that could be learned from Apache mod_wsgi.

As I mentioned to Manlio one of the decisions that probably has to be
made early on is whether you want to try and support multiple Python
sub interpreters in a single process like mod_python and mod_wsgi do.
Although this can be useful in some circumstances it isn't always
practical as some third party extension modules will not work in
secondary Python sub interpreters.

Thus, to make it a lot simpler, you might only support use of the main
Python interpreter like would be the case if using pure Python web
server. Doing this would eliminate a lot of the complexity around
management of Python thread APIs and creation/destruction of Python
sub interpreters.

But then, if you cut things back and go for a simpler design, it is
arguable as to whether you may as well just use standard Python
program and pure Python web server behind a proxy, of flup behind
FASTCGI. Therefore comes down to what you want to achieve in doing it
and whether the benefit is really there and that it offers something
extra over these other existing approaches.

Graham

From fumanchu at aminus.org  Sat Aug 25 08:45:03 2007
From: fumanchu at aminus.org (Robert Brewer)
Date: Fri, 24 Aug 2007 23:45:03 -0700
Subject: [Web-SIG] Another WSGI gotcha to be aware of
Message-ID: <9BBC2D2B2CCF7E4DA0E212D1BD0CC6FA1FEECF@ex10.hostedexchange.local>

Hi all,

I just found another corner case in the WSGI spec
that I thought I'd share so you all can check your
WSGI components for similar problems. Basically,
it comes down to error handling. Here's a simple
example:


class Middleware(object):
    
    def __init__(self, nextapp, environ, start_response):
        try:
            self.response = nextapp(environ, start_response)
            self.iter_response = iter(self.response)
            return
        except SomeException:
            self.close()
    
    def close(self):
        if hasattr(self.response, "close"):
            self.response.close()
    
    def __iter__(self):
        return self
    
    def next(self):
        try:
            return self.iter_response.next()
        except AnotherException:
            self.close()
            raise StopIteration

As you know, all WSGI middleware (that doesn't just pass
through the response from "nextapp") must itself possess
a "close" method so that the response from "nextapp" can
have *its* close method called. In this example, that
would be "self.response.close".

However, and here's the rub, if nextapp() raises an
exception, **self.response is never bound**, and
therefore we have no handle to the object we need
to close. Note that this is not a middleware-only
problem; servers can run into this too.

The spec says, "In general, applications *should* try to
trap their own, internal errors"; we might want to make
that a MUST in future versions. Alternately, we could
require that every application provide its resource-
releasing endpoint via some means other than a successful
response. I'm sure you all can come up with other solutions.


Robert Brewer
fumanchu at aminus.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/web-sig/attachments/20070824/e9f7a2a7/attachment.htm 

From graham.dumpleton at gmail.com  Sat Aug 25 11:24:42 2007
From: graham.dumpleton at gmail.com (Graham Dumpleton)
Date: Sat, 25 Aug 2007 19:24:42 +1000
Subject: [Web-SIG] Another WSGI gotcha to be aware of
In-Reply-To: <9BBC2D2B2CCF7E4DA0E212D1BD0CC6FA1FEECF@ex10.hostedexchange.local>
References: <9BBC2D2B2CCF7E4DA0E212D1BD0CC6FA1FEECF@ex10.hostedexchange.local>
Message-ID: <88e286470708250224o47932516rd3ac0378826ca76f@mail.gmail.com>

On 25/08/07, Robert Brewer <fumanchu at aminus.org> wrote:
>
>
>
> Hi all,
>
>  I just found another corner case in the WSGI spec
>  that I thought I'd share so you all can check your
>  WSGI components for similar problems. Basically,
>  it comes down to error handling. Here's a simple
>  example:
>
>
>  class Middleware(object):
>
>      def __init__(self, nextapp, environ, start_response):
>          try:
>              self.response = nextapp(environ, start_response)
>              self.iter_response = iter(self.response)
>              return
>          except SomeException:
>              self.close()
>
>      def close(self):
>          if hasattr(self.response, "close"):
>              self.response.close()
>
>      def __iter__(self):
>          return self
>
>      def next(self):
>          try:
>              return self.iter_response.next()
>          except AnotherException:
>              self.close()
>              raise StopIteration

A question related to this example. In what ways is using
__iter__/yield better or worse than defining next() in terms of
performance, compatibility with different Python versions etc. For
example:

class Generator:
    def __init__(self, generator, callback, environ):
        self.__generator = generator
        self.__callback = callback
        self.__environ = environ
    def __iter__(self):
        for item in self.__generator:
            yield item
    def close(self):
        if hasattr(self.__generator, 'close'):
            self.__generator.close()
        self.__callback(self.__environ)

class ExecuteOnCompletion:
    def __init__(self, application, callback):
        self.__application = application
        self.__callback = callback
    def __call__(self, environ, start_response):
        try:
            result = self.__application(environ, start_response)
        except:
            self.__callback(self.__environ)
            raise
        return Generator(result, self.__callback, environ)

Just curious.

Graham

From pje at telecommunity.com  Sat Aug 25 18:34:25 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 25 Aug 2007 12:34:25 -0400
Subject: [Web-SIG] Another WSGI gotcha to be aware of
In-Reply-To: <9BBC2D2B2CCF7E4DA0E212D1BD0CC6FA1FEECF@ex10.hostedexchange
	.local>
References: <9BBC2D2B2CCF7E4DA0E212D1BD0CC6FA1FEECF@ex10.hostedexchange.local>
Message-ID: <20070825163159.A7F5C3A403D@sparrow.telecommunity.com>

At 11:45 PM 8/24/2007 -0700, Robert Brewer wrote:
>However, and here's the rub, if nextapp() raises an
>exception, **self.response is never bound**, and
>therefore we have no handle to the object we need
>to close. Note that this is not a middleware-only
>problem; servers can run into this too.
>
>The spec says, "In general, applications *should* try to
>trap their own, internal errors"; we might want to make
>that a MUST in future versions. Alternately, we could
>require that every application provide its resource-
>releasing endpoint via some means other than a successful
>response. I'm sure you all can come up with other solutions.

I don't see a problem here to solve.  If the application didn't 
return a response, the middleware naturally isn't obligated to call 
close() on it.


From fumanchu at aminus.org  Sat Aug 25 23:09:41 2007
From: fumanchu at aminus.org (Robert Brewer)
Date: Sat, 25 Aug 2007 14:09:41 -0700
Subject: [Web-SIG] Another WSGI gotcha to be aware of
References: <9BBC2D2B2CCF7E4DA0E212D1BD0CC6FA1FEECF@ex10.hostedexchange.local>
	<20070825163159.A7F5C3A403D@sparrow.telecommunity.com>
Message-ID: <9BBC2D2B2CCF7E4DA0E212D1BD0CC6FA1FEED1@ex10.hostedexchange.local>

Phillip J. Eby wrote:
> At 11:45 PM 8/24/2007 -0700, Robert Brewer wrote:
> >However, and here's the rub, if nextapp() raises an
> >exception, **self.response is never bound**, and
> >therefore we have no handle to the object we need
> >to close. Note that this is not a middleware-only
> >problem; servers can run into this too.
> >
> >The spec says, "In general, applications *should* try to
> >trap their own, internal errors"; we might want to make
> >that a MUST in future versions. Alternately, we could
> >require that every application provide its resource-
> >releasing endpoint via some means other than a successful
> >response. I'm sure you all can come up with other solutions.
> 
> I don't see a problem here to solve.  If the application
> didn't return a response, the middleware naturally isn't
> obligated to call close() on it.

Sorry; I didn't mean to imply that WSGI server interfaces
need to be fixed in an way.

As the author of a WSGI application interface, it means
you have to call your own resource cleanup code if an
exception is raised while you're being called (as opposed
to having the response iterated over):

def __call__(self, environ, start_response):
    try:
        return self.respond(environ, start_response)
    except:
        self.cleanup()
        raise

Some applications don't do this, so my primary message
is to authors of WSGI application interfaces (including
all middleware) to check your code and make sure you do
it, rather than just let errors propagate out.

Some application authors may have chosen to not do this
because the spec says, "should try", not "must". So I'm
thinking of ways to improve that situation, all the way
from "do nothing" to "clarify the language in the spec"
to different conversation models in future specs.


Robert Brewer
fumanchu at aminus.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/web-sig/attachments/20070825/930a92c1/attachment.htm