[issue34935] Misleading error message in str.decode()
Walter Dörwald
report at bugs.python.org
Mon Oct 8 11:38:20 EDT 2018
New submission from Walter Dörwald <walter at livinglogic.de>:
The following code issues a misleading exception message:
>>> b'\xed\xa0\xbd\xed\xb3\x9e'.decode("utf-8")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 0: invalid continuation byte
The cause for the exception is *not* an invalid continuation byte, but UTF-8 encoded surrogates. In fact using the 'surrogatepass' error handler doesn't raise an exception:
>>> b'\xed\xa0\xbd\xed\xb3\x9e'.decode("utf-8", "surrogatepass")
'\ud83d\udcde'
I would have expected an exception message like:
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 0-2: surrogates not allowed
(Note that the input bytes are an improperly UTF-8 encoded version of U+1F4DE (telephone receiver))
----------
components: Unicode
messages: 327357
nosy: doerwalter, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: Misleading error message in str.decode()
versions: Python 3.7
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34935>
_______________________________________
More information about the Python-bugs-list
mailing list