[issue1441530] socket read() can cause MemoryError in Windows

[Charles-François Natali]

Charles-François Natali <neologix at free.fr> added the comment:

> Patch looks ok. Is 3.x also affected? The I/O stack changed quite a bit in
> 3.x.

I think it's not affected, but I can't reproduce this behaviour with
glibc/eglibc, so don't just take my word for it.
The reason is that in py3k, imaplib uses a buffered reader to read
from the socket (obtained through sock.makefile()), and IIUC, reading
from a buffered reader n bytes will always return n bytes (unless EOF
is encountered or the underlying object is set to non-blocking),
contrarily to reading directly from a socket. So the resizing is done
from the buffered io module, which only grows the result buffer,
freeing the buffers returned by socket's recv() one at a time.

Note that if I'm right, another way to solve this would be to wrap the
SSL socket with a call to makefile, like what's already done for the
non-SSL case (and in py3k). This would also lead to cleaner and
simpler code, because we wouldn't need to handle partial reads
ourself.
That's what the patch attached does.
Note that if I'm correct, then the current py3k imaplib read() method
could also be simplified:

    def read(self, size):
        """Read 'size' bytes from remote."""
        chunks = []
        read = 0
        while read < size:
            data = self.file.read(min(size-read, 4096))
            if not data:
                break
            read += len(data)
            chunks.append(data)
        return b''.join(chunks)

It would also be a bit faster.
But since I can't reproduce this issue, I'd rather make sure it's
correct before making a such change.

----------
Added file: http://bugs.python.org/file22051/imaplib_ssl_makefile.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1441530>
_______________________________________
-------------- next part --------------
diff -r b3f410dbab25 Lib/imaplib.py
--- a/Lib/imaplib.py	Fri May 20 11:49:06 2011 -0500
+++ b/Lib/imaplib.py	Sat May 21 18:24:01 2011 +0200
@@ -1158,28 +1158,17 @@
             self.port = port
             self.sock = socket.create_connection((host, port))
             self.sslobj = ssl.wrap_socket(self.sock, self.keyfile, self.certfile)
+            self.file = self.sslobj.makefile('rb')
 
 
         def read(self, size):
             """Read 'size' bytes from remote."""
-            # sslobj.read() sometimes returns < size bytes
-            chunks = []
-            read = 0
-            while read < size:
-                data = self.sslobj.read(min(size-read, 16384))
-                read += len(data)
-                chunks.append(data)
-
-            return ''.join(chunks)
+            return self.file.read(n)
 
 
         def readline(self):
             """Read line from remote."""
-            line = []
-            while 1:
-                char = self.sslobj.read(1)
-                line.append(char)
-                if char in ("\n", ""): return ''.join(line)
+            return self.file.readline()
 
 
         def send(self, data):