[Web-SIG] Returned application object and fileno.

[Phillip J. Eby]

At 06:08 PM 9/1/04 +0100, Alan Kennedy wrote:
>I would summarise the position of the current spec as "you must return an 
>iterable, except when you want to return a file object, which will work 
>fine under cpython 2.2+, because files are iterable under cpython 2.2+, 
>even though they don't need to be iterable when they have fileno()".

No, it's just that you must return an iterable, *period*.  The fact that 
2.2 allows this to be a file object is irrelevant, as is the fact that 2.1 
doesn't allow this to be a file object.  There are thousands of classes out 
there for both 2.1 and 2.2 that either are, or aren't, iterable, and that's 
equally irrelevant.


>I'm just afraid that people will steam ahead writing WSGI middleware 
>applications which return file-objects,

The part you keep leaving out is that such middleware is thereby targeted 
at Python 2.2, not 2.1.  The spec explicitly mentions this.


>which will fail on jython simply because putting the following lines in my 
>code is a violation of the spec
>
>if type(app_return) is types.FileType:
>   do_file_stuff(app_return)

What you're doing here isn't a "violation", IMO, merely "out of 
scope".  It's not up to the spec to explain how to make Python 2.1 support 
2.2 features; IMO, that's all you're doing here, and it doesn't hurt anybody.


>[Alan Kennedy]
> >> 2. The spec, as is, explicitly permits authors of cpython applications
> >> to return file-like objects,
>
>[Phillip J. Eby]
> > Only if they are *iterable*, which is only true of the 'file' object in
> > 2.2 and up.
>
>Which seems to me an arbitrary criterion, especially in the light that the 
>iterator nature of the file object will possibly (likely) not be actually 
>used, as described in the snippet above.

The CGI runner won't use fileno(), and neither will many other servers.  I 
don't see how the "iterable" criterion is arbitrary because some objects 
are iterable and others aren't.  Any criterion we choose will by definition 
include some objects and not others.


>In cpython frameworks, the code would look like this
>
>if hasattr(app_object, 'fileno'):
>   do_file_stuff(app_object.fileno())
>else:
>   do_iterator_stuff(app_object)
>
>On jython
>
>if type(app_object) is types.FileType:
>   do_file_stuff(app_object)
>else:
>   do_iterator_stuff(app_object)
>
>Is that so difficult to accept?

But that's exactly what the spec says to do *now*, except that it doesn't 
explicitly bless the type check.  If you really want to have that blessing 
written into the spec, so be it.  I just don't see it as a matter that's in 
scope of the spec, because it's not only Jython-specific, but specific to 
your server as well.  Document your extension as you would any such 
extension.  There's no law against being more *permissive* than the spec 
requires.

I do not see any reason to burden *other* server authors by requiring them 
to support your extension, because no use cases have been presented for 
this for any situation *other* than a Jython 2.1 server trying to run a 
Python 2.2 application.


>[Alan Kennedy]
> >> I think we should explicitly allow return of a file-like object, and
> >> thus freedom to use the read() method, etc.
>
>[Phillip J. Eby]
> > You are trying to let 2.2 code run in a 2.1 Python.
>
>Well, I see it as WSGI forcing me to jump through hoops in order to 
>support the notion of iterability, even when that notion is NOT 
>universally applicable, as the fileno() exception proves.

It's stretching a 2.2 spec to work with older versions of Python, largely 
intended for your benefit, as you were the first person who presented a 
strong use case for supporting *any* pre-2.2 version of Python.


>However, I think the absolute insistence on return objects being iterable 
>is slightly arbitrary and unnecessarily constraining.

For whom?  I've given numerous examples of how trivial it is for code 
targeted to 2.1 or earlier to support making files and even file-like 
objects into iterables.  This is a small burden for those who want their 
code to be portable to such versions.

Similarly, it's not an unreasonable burden for your server to support 
extensions to 2.1 behavior in order to accommodate code not written for 
Python 2.1 compatibility.

It *is* unreasonable to expand the spec to place those burdens on people 
who don't care about supporting 2.1, or who don't care about supporting 2.2 
code under 2.1


>Maybe I'm just wasting my time? Maybe I'm the only one who is interested 
>in seeing a jython WSGI server into which users can drop universal WSGI 
>components and have them just work? Is anyone else interested in such a 
>jython WSGI container? Or should I just toddle off back to J2EE servlets?

I agree with your intentions; I just don't agree that *other* server 
authors should be forced to duplicate your efforts if they don't have that 
use case.  Iterability is the single simplest protocol that is universally 
accessible in any Python used in the last several years.  It doesn't 
require any introspection.  Currently, the common case code for a server 
looks like this:

         result = application(environ, start_response)
         try:
             for data in result:
                 write(data)
         finally:
             if hasattr(result,'close'):
                 result.close()

This is perfectly valid implementation under the spec.  Changing the spec 
to allow the application to return anything *but* iterables means 
complicating *every* server, for the sole benefit of applications that want 
to use 2.2 idioms under Python 2.1.

If I were a server author targeting 2.2 and up (and I will be), I would 
rightly object to adding extra introspection to the above, when it will not 
benefit me or any user in my target audience.  If my server requires 2.2, 
then obviously applications running under it can safely use 2.2 
idioms.  And if they're written for 2.1 they also work.

So here's the resolution: I will slightly expand the section on supporting 
older versions of Python, to explicitly allow a 2.1 server to 
"forward-compatibly" check for 2.2 idioms such as returning a file object.

I'd prefer not to do that, but not because I dislike the approach.  We're 
in "violent agreement" on what *your* server should do about this, and I 
encourage you to implement it.  Our disagreement (as I understand it) is that:

1. I think this is a "server-specific extension" that's outside the spec's 
scope to rule on the validity of, and

2. I don't think that requiring others to do what your code will be doing 
is a good idea, because they don't *need* to, unless they're trying to run 
2.2 code on a 2.1 Python, which should *definitely* not be a requirement of 
the spec.