[Web-SIG] WSGI, cgi.FieldStorage incompatibility

[Jim Fulton]

James Y Knight wrote:
> On Sep 29, 2006, at 3:31 PM, Guido van Rossum wrote:
> 
>> On 9/29/06, Michael Kerrin <michael.kerrin at openapp.biz> wrote:
>>>   But the current implementation of cgi.FieldStorage in the 2.4.4  
>>> branch
>>> and on Python 2.5 does call readline with the size argument. It has
>>> started to do this in response to the Python bug #1112549 -
>>> cgi.FieldStorage memory usage can spike in line-oriented ops. See
>>> http://sourceforge.net/tracker/index.php? 
>>> func=detail&aid=1112549&group_id=5470&atid=105470
>>>
>>>   Since it is reasonable for a WSGI application to use  
>>> cgi.FieldStorage
>>> I am wondering whether cgi.FieldStorage or the WSGI specification  
>>> needs
>>> to changed in order to solve this incompatibility.
>>>
>>>   Originally I thought it was cgi.FieldStorage that needs to be  
>>> changed,
>>> and hence tried to fix it by wrapping the input stream so that the
>>> readline method always uses the read method on the input stream.  
>>> While
>>> this seems to work for me it introduces a level of complexity in the
>>> cgi.py file, and possible some other bugs, that makes me think that
>>> adding the size argument for readline into the WSGI specification  
>>> isn't
>>> such bad idea after all.
>> Since that change to cgi.py was a security fix I would strongly
>> recommend not to remove it and to change the WSGI spec instead.
> 
> Given that this change is now part of python 2.4.4 and python 2.5, it  
> seems to me it is now a defacto requirement that all WSGI server  
> implementations must support readline with a size argument in order  
> to run any interesting software, despite the spec explicitly saying  
> that you shouldn't. I suspect simply modifying the spec to follow the  
> current reality would be the least bad option.

Yes and updating the server implementations, of course, where necessary.

> But this kind of destabilizing breakage really shouldn't be allowed  
> to happen again. Once the error was discovered, the cgi.py change  
> should have been immediately reverted until either a decision was  
> made to change the WSGI spec, or else the change fixed to not break  
> WSGI compliant servers. This limbo situation is pretty bad.

Agreed.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org