[New-bugs-announce] [issue39371] http.client.HTTPResponse raises IncompleteRead on chunked encoding

[Arden]

New submission from Arden <olyx32 at gmail.com>:

http.client.HTTPResponse._readinto_chunked has two problems in python 3.8 - 3.9.

1. _safe_readinto assumes that self.fp.readinto(b) will read exactly len(b) bytes. This is not always true, especially in case of SSL traffic. But _safe_readinto raises IncompleteRead if less len(b) bytes was read.

So, _safe_readinto should be removed and substituted with self.fp.readinto

2. _readinto_chunked may lose chunked block boundary because of this line:
  self.chunk_left = 0
it should be changed to
  self.chunk_left = chunk_left - n
in order to self._get_chunk_left() be able to find real chunk boundary

Corrected function looks like this:
def _readinto_chunked(self, b):
    assert self.chunked != _UNKNOWN
    total_bytes = 0
    mvb = memoryview(b)
    try:
        while True:
            chunk_left = self._get_chunk_left()
            if chunk_left is None:
                return total_bytes

            if len(mvb) <= chunk_left:
                n = self.fp.readinto(mvb)
                self.chunk_left = chunk_left - n
                return total_bytes + n

            temp_mvb = mvb[:chunk_left]
            n = self.fp.readinto(temp_mvb)
            mvb = mvb[n:]
            total_bytes += n
            self.chunk_left = chunk_left - n

    except IncompleteRead:
        raise IncompleteRead(bytes(b[0:total_bytes]))

----------
components: Library (Lib)
messages: 360199
nosy: Arden
priority: normal
severity: normal
status: open
title: http.client.HTTPResponse raises IncompleteRead on chunked encoding
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue39371>
_______________________________________