[New-bugs-announce] [issue23901] Force console stdout to use UTF8 on Windows

[Paul Moore]

New submission from Paul Moore:

Console code page issues are a consistent source of problems on Windows. It would be nice, given that the Windows console has Unicode support, if Python could write the full range of Unicode to the console by default.

The MSVC runtime appears to have a flag that can be set via _setmode(), _O_U8TEXT, which "enables Unicode mode" (see https://msdn.microsoft.com/en-us/library/tw4k6df8%28v=vs.100%29.aspx?f=255&MSPPError=-2147217396, in particular the second example). It seems as if Python could set U8TEXT mode on sys.stdout on startup (assuming it's a console) and set the encoding to UTF8, and then Unicode output would "just work".

I don't have code that implements this yet, but if I can get my head round the IO stack and the Python startup code, I'll give it a go.

Steve - any comments on whether this might work? I've never seen any real-world code using U8TEXT which makes me wonder if it's reliable (doing msvcrt.setmode(sys.stdout.fileno(), 0x40000) in Python 3.4 causes Python to crash, which is worrying, but it works in 3.5...).

----------
assignee: paul.moore
components: Windows
messages: 240354
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Force console stdout to use UTF8 on Windows
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23901>
_______________________________________