[issue6203] 3.x locale does not default to C, contrary to the documentation and to 2.x behavior

[STINNER Victor]

STINNER Victor <victor.stinner at haypocalc.com> added the comment:

> To add a little bit more analysis: posix.device_encoding requires that
> the LC_CTYPE is set. Setting it just in this function would not be
> possible, as setlocale is not thread-safe.

open() does indirectly (locale.getpreferredencoding()) change temporary the locale (set LC_CTYPE to "") if the file is not a TTY (if it is a TTY, device_encoding() calls nl_langinfo(CODESET) without changing the current locale). If setlocale() is not thread-safe we have (maybe?) a problem here. See also #11022: report of an user not understanding why setlocale() doesn't impact open() (TextIOWrapper) encoding). A quick solution is to call locale.getpreferredencoding(False) which doesn't change the locale.

Do you really need os.device_encoding()? If we change TextIOWrapper to call locale.getpreferredencoding(False), os.device_encoding() and locale.getpreferredencoding(False) will give the same result. Except on Windows: os.device_encoding() uses GetConsoleCP() if fd==0 and GetConsoleOutputCP() if fd in (1, 2). But we can use GetConsoleCP() and GetConsoleOutputCP() directly in initstdio(). If someone closes sys.std* and recreate them later: os.device_encoding() can be use explicitly to keep the previous behaviour.

> It would still be better it is was unset afterwards. Third-party
> extensions could have LC_CTYPE-dependent behaviour.

If Python is embeded, it should not change the locale. Even if it is not embeded, it is maybe better to never set LC_CTYPE.

It is too late to touch such critical point in Python 3.2, but we may change it in Python 3.3.

----------
nosy: +haypo
versions: +Python 3.3 -Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6203>
_______________________________________