[Python-ideas] Fix default encodings on Windows

[Adam Bartoš]

Eryk Sun wrote:

> IMO, Python needs a C implementation of the win_unicode_console
> module, using the wide-character APIs ReadConsoleW and WriteConsoleW.
> Note that this sets sys.std*.encoding as UTF-8 and transcodes, so
> Python code never has to work directly with UTF-16 encoded text.
>
>
The transcoding wrappers with 'utf-8' encoding are used just as a work
around the fact that Python tokenizer cannot use utf-16-le and that the
readlinehook machinery is unfortunately bytes-based. The tanscoding wrapper
just has encoding 'utf-8' and no buffer attribute, so there is no actual
transcoding in sys.std* objects. It's just a signal for PyOS_Readline
consumers, and the transcoding occurs in a custom readline hook. Nothing
like this would be needed if PyOS_Readline was replaced by some Python API
wrapper around sys.readlinehook that would be Unicode string based.

Adam Bartoš
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160811/c2bb029c/attachment-0001.html>