[New-bugs-announce] [issue38482] BUG in codecs.BufferedIncrementalDecoder

[Jim Carroll]

New submission from Jim Carroll <jim at carroll.com>:

While working with codecs.iterdecode(), encountered "can't concat int to bytes". The source of the problem is BufferedIncrementalDecoder stores it's internal buffer as a bytes (ie: b""), but decode() can be passed either a byte string or in the case of iterdecode() an int.  The solution is to test for this in the decode and if passed an int to coerce to bytes (see attach patch)

Platform: Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32

Code to demonstrate the issue:

>>> import codecs
>>> source = ''.join([chr(x) for x in range(256)])
>>> enc = b''.join(codecs.iterencode(source, 'utf-8'))
>>> list(''.join(codecs.iterdecode(enc, 'utf-8')))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python37\lib\codecs.py", line 1048, in iterdecode
    output = decoder.decode(input)
  File "C:\Python37\lib\codecs.py", line 321, in decode
    data = self.buffer + input
TypeError: can't concat int to bytes

----------
components: Library (Lib)
files: codecs.patch
keywords: patch
messages: 354707
nosy: jamercee
priority: normal
severity: normal
status: open
title: BUG in codecs.BufferedIncrementalDecoder
versions: Python 3.7
Added file: https://bugs.python.org/file48661/codecs.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38482>
_______________________________________