[Web-SIG] WSGI, cgi.FieldStorage incompatibility

[James Y Knight]

On Sep 29, 2006, at 3:31 PM, Guido van Rossum wrote:

> On 9/29/06, Michael Kerrin <michael.kerrin at openapp.biz> wrote:
>>   But the current implementation of cgi.FieldStorage in the 2.4.4  
>> branch
>> and on Python 2.5 does call readline with the size argument. It has
>> started to do this in response to the Python bug #1112549 -
>> cgi.FieldStorage memory usage can spike in line-oriented ops. See
>> http://sourceforge.net/tracker/index.php? 
>> func=detail&aid=1112549&group_id=5470&atid=105470
>>
>>   Since it is reasonable for a WSGI application to use  
>> cgi.FieldStorage
>> I am wondering whether cgi.FieldStorage or the WSGI specification  
>> needs
>> to changed in order to solve this incompatibility.
>>
>>   Originally I thought it was cgi.FieldStorage that needs to be  
>> changed,
>> and hence tried to fix it by wrapping the input stream so that the
>> readline method always uses the read method on the input stream.  
>> While
>> this seems to work for me it introduces a level of complexity in the
>> cgi.py file, and possible some other bugs, that makes me think that
>> adding the size argument for readline into the WSGI specification  
>> isn't
>> such bad idea after all.
>
> Since that change to cgi.py was a security fix I would strongly
> recommend not to remove it and to change the WSGI spec instead.

Given that this change is now part of python 2.4.4 and python 2.5, it  
seems to me it is now a defacto requirement that all WSGI server  
implementations must support readline with a size argument in order  
to run any interesting software, despite the spec explicitly saying  
that you shouldn't. I suspect simply modifying the spec to follow the  
current reality would be the least bad option.

But this kind of destabilizing breakage really shouldn't be allowed  
to happen again. Once the error was discovered, the cgi.py change  
should have been immediately reverted until either a decision was  
made to change the WSGI spec, or else the change fixed to not break  
WSGI compliant servers. This limbo situation is pretty bad.

James