[Pytest-commit] Issue #628: UnicodeDecodeError (INTERNALERROR) when doctests contain Unicode (hpk42/pytest)

[Jason R. Coombs]

New issue 628: UnicodeDecodeError (INTERNALERROR) when doctests contain Unicode
https://bitbucket.org/hpk42/pytest/issue/628/unicodedecodeerror-internalerror-when

Jason R. Coombs:

Consider this test:

```
def foo():
    """
    >>> name = 'с'
    'anything'
    """
```

Note that isn't the letter 'c' but instead Cyrillic 's'.

Save it as mod.py, then invoke `pytest --doctest-modules`. It will fail with this output:

```
> py.test --doctest-modules
============================= test session starts =============================

platform win32 -- Python 3.4.2 -- py-1.4.26 -- pytest-2.6.4
collected 1 items

mod.py
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\main.py", line 84, in wrap_session
INTERNALERROR>     doit(config, session)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\main.py", line 122, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\core.py", line 413, in __call__
INTERNALERROR>     return self._docall(methods, kwargs)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\core.py", line 424, in _docall
INTERNALERROR>     res = mc.execute()
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\core.py", line 315, in execute
INTERNALERROR>     res = method(**kwargs)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\main.py", line 142, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\core.py", line 413, in __call__
INTERNALERROR>     return self._docall(methods, kwargs)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\core.py", line 424, in _docall
INTERNALERROR>     res = mc.execute()
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\core.py", line 315, in execute
INTERNALERROR>     res = method(**kwargs)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\runner.py", line 65, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\runner.py", line 75, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\runner.py", line 111, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)

INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\main.py", line 167, in call_matching_hooks
INTERNALERROR>     return hookmethod.pcall(plugins, **kwargs)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\core.py", line 417, in pcall
INTERNALERROR>     return self._docall(methods, kwargs)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\core.py", line 424, in _docall
INTERNALERROR>     res = mc.execute()
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\core.py", line 315, in execute
INTERNALERROR>     res = method(**kwargs)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\runner.py", line 214, in pytest_runtest_makereport
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "c:\python\lib\site-packages\pytest-2.6.4-py3.4.egg\_pytest\doctest.py", line 62, in repr_failure
INTERNALERROR>     filelines = py.path.local(filename).readlines(cr=0)
INTERNALERROR>   File "c:\python\lib\site-packages\py-1.4.26-py3.4.egg\py\_path\common.py", line 141, in readlines
INTERNALERROR>     content = self.read('rU')
INTERNALERROR>   File "c:\python\lib\site-packages\py-1.4.26-py3.4.egg\py\_path\common.py", line 135, in read
INTERNALERROR>     return f.read()
INTERNALERROR>   File "c:\python\lib\encodings\cp1252.py", line 23, in decode
INTERNALERROR>     return codecs.charmap_decode(input,self.errors,decoding_table)[0]
INTERNALERROR> UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 36: character maps to <undefined>

==============================  in 0.30 seconds ===============================
```