[issue26499] http.client.IncompleteRead from HTTPResponse read after part reading file

[Martin Panter]

Martin Panter added the comment:

Thanks for the report and the patch. It looks okay as far as it goes, but I think there are other related bugs:

## read1() doesn’t update length either ##

>>> handle = urlopen("https://www.python.org/")
>>> len(handle.read1())
7374
>>> handle.read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/proj/python/cpython/Lib/http/client.py", line 462, in read
    s = self._safe_read(self.length)
  File "/home/proj/python/cpython/Lib/http/client.py", line 614, in _safe_read
    raise IncompleteRead(b''.join(s), amt)
http.client.IncompleteRead: IncompleteRead(39583 bytes read, 7374 more expected)

## readline() and read1() blindly read beyond Content-Length ##

>>> conn = HTTPSConnection("www.python.org")
>>> conn.request("GET", "/")
>>> resp = conn.getresponse()
>>> resp.readlines()  # Hangs until the connection is closed

I wonder if anyone has considered implementing HTTPResponse by wrapping or subclassing BufferedReader; that way you get well-tested readline() etc functionality for free. The HTTP protocol (Connection: close, Content-Length, chunked decoding) could be handled by an internal RawIOBase.readinto() method.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26499>
_______________________________________