[ python-Bugs-1441530 ] socket read() can cause MemoryError in Windows

[SourceForge.net]

Bugs item #1441530, was opened at 2006-03-02 10:27
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1441530&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: taukki (taukki)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket read() can cause MemoryError in Windows

Initial Comment:
When a big mail (>10MB) is fetched MemoryError may
result (reproduced in Windows XP/2003, Python 2.41):

Traceback (most recent call last):
  File "t5.py", line 9, in ?
    data = f.read(26872159)
  File "c:\python24\lib\socket.py", line 307, in read
    data = self._sock.recv(recv_size)
MemoryError

The problem is not actually in imaplib (though it
requests the whole mail in one read), but probably in
combination of socket core code and the implementation
of read() function (he way it buffers the received data
in a list?). I can easily reproduce the problem e.g
with these snippets:

------------CLIENT----------------->
import socket
import time

s = socket.socket()

s.connect(("127.0.0.1", 8037))
f = s.makefile("r", 200000)
while 1:
  data = f.read(26872159) # Using smaller value here  
helps, 100000 seems to work forever...
  #data = s.recv(26872159) # THIS WORKS OK
  print len(data)

---------SERVER--------------------->
import SocketServer
import time

PORT = 8037
time.sleep(0.2)

s = "X" * 1823
class
TimeRequestHandler(SocketServer.StreamRequestHandler):
    def handle(self):
      for x in range(200000):
        self.wfile.write(s)
        print x


server = SocketServer.TCPServer(("", PORT),
TimeRequestHandler)
print "listening on port", PORT
server.serve_forever()





----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1441530&group_id=5470