Jitterbug-Id: 381
Submitted-By: naris@ensim.com
Date: Mon,  3 Jul 2000 21:29:25 -0400 (EDT)
Version: 1.5.2
OS: RedHat 6.1


read_lines_to_outerboundary chews up memory.
there's a while (1) loop that does a readline(), which
is probably not good, as you could have a binary file that
doesn't have an endline til the end, and thus readline would
read the entire file into memory.  i did the following:
dd'd a 100MB /dev/zero file and used zope to upload it.
it chewed up a large amount of memory.

there seems to be bug reports about text files and windows
systems (and needing to use python -u), but this problem
is unrelated.  i am using a unix browser -> unix web server.



====================================================================
Audit trail:
Tue Jul 11 08:24:22 2000	guido	moved from incoming to open

there's a bug i submitted to zope that deals with a second problem
with read_lines_to_outerboundary, which will also cause a memory
leak.

the read_lines_to_outerboundary has an unnecessary line 
self.lines.append(line).  removing will fix it this second problem.

Agreed that readlines() isn't the best thing to use in the face of large binary uploads.  I've added this feature request to PEP 42 (and am closing this bug report).

Removing self.lines might be more problematic, since although it's not documented as part of the public interface for FieldStorage, it is possible that existing code is using it, and that code would break.

i feel the self.lines.append() is is a pretty serious problem...have you got any
second opinions on how to proceed?

if its not part of the public interface for FieldStorage, then wouldn't it be somewhat safe to remove it (you might anger some people, but they could always stick to using the old cgi.py)?

my argument is that if they changed the socket connect/bind functions to reject connect(host,port) in favor of connect((host,port)), then we should certainly be able to remove this self.lines.append() :-)

what do you think?

We've decided to remove the self.lines attribute for Python 2.1.  This part of the bug report is identical to 119806.  No one can remember why self.lines was there in the first place, so we'll remove it and see who complains.