[issue37413] Deprecate sys._enablelegacywindowsfsencoding()?

[STINNER Victor]

New submission from STINNER Victor <vstinner at redhat.com>:

sys._enablelegacywindowsfsencoding() was added late in PEP 529 design "just in case" if something goes wrong. But I'm not aware of anyone using it. Do we want to keep supporting the *legacy* Windows filesystem encoding (ANSI code page) forever? IMHO using UTF-8 is a way more practical solution to design portable applications working unmodified on Windows *and* Unix. Well, it's the purpose of the PEP 529.

I propose to deprecate sys._enablelegacywindowsfsencoding() and PYTHONLEGACYWINDOWSFSENCODING environment variable in Python 3.9 and remove it from Python 3.10. Calling sys._enablelegacywindowsfsencoding() would emit a DeprecationWarning in 3.9.

I dislike sys._enablelegacywindowsfsencoding() because it can lead to mojibake: filenames decoded from the ANSI code page but then encoded to UTF-8. In the PEP 587 "Python Initialization Configuration" I tried to ensure that encodings are set early: in a new "pre-initialization" phase. Encodings should not change after the pre-initialization.

--

By the way, I'm not aware of any issue with io._WindowsConsoleIO. Should we also deprecated PYTHONLEGACYWINDOWSSTDIO environment variable which opt-out from the new io._WindowsConsoleIO?

Extract of open() code in Modules/_io/_iomodule.c:

    /* Create the Raw file stream */
    {
        PyObject *RawIO_class = (PyObject *)&PyFileIO_Type;
#ifdef MS_WINDOWS
        PyConfig *config = &_PyInterpreterState_GET_UNSAFE()->config;
        if (!config->legacy_windows_stdio && _PyIO_get_console_type(path_or_fd) != '\0') {
            RawIO_class = (PyObject *)&PyWindowsConsoleIO_Type;
            encoding = "utf-8";
        }
#endif
        raw = PyObject_CallFunction(RawIO_class,
                                    "OsiO", path_or_fd, rawmode, closefd, opener);
    }

----------
components: Windows
messages: 346631
nosy: ZackerySpytz, eryksun, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: Deprecate sys._enablelegacywindowsfsencoding()?
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37413>
_______________________________________