[issue36945] Add _PyPreConfig.configure_locale: allow to leave LC_CTYPE unchanged when embedding Python

[STINNER Victor]

New submission from STINNER Victor <vstinner at redhat.com>:

Py_Initiliaze() always call setlocale(LC_CTYPE, "") on all platforms to set the LC_CTYPE locale to the user preferred locale.

That's fine when Py_Main() is used: when Python is the only "owner" of a process.

I'm not sure that it's fine when Python is embedded into an application.

I propose to add _PyPreConfig.configure_locale: when set to 0, it leaves the LC_CTYPE locale unchanged. The caller is responsible to choose its favorite locale.

I'm not sure if it's real issue in practice. Maybe users learnt how to workaround this limitation.

By the way, I modified Python to always call setlocale(LC_CTYPE, "") on Windows, bpo-34485:

commit 177d921c8c03d30daa32994362023f777624b10d
Author: Victor Stinner <vstinner at redhat.com>
Date:   Wed Aug 29 11:25:15 2018 +0200

    bpo-34485, Windows: LC_CTYPE set to user preference (GH-8988)
    
    On Windows, the LC_CTYPE is now set to the user preferred locale at
    startup: _Py_SetLocaleFromEnv(LC_CTYPE) is now called during the
    Python initialization. Previously, the LC_CTYPE locale was "C" at
    startup, but changed when calling setlocale(LC_CTYPE, "") or
    setlocale(LC_ALL, "").
    
    pymain_read_conf() now also calls _Py_SetLocaleFromEnv(LC_CTYPE) to
    behave as _Py_InitializeCore(). Moreover, it doesn't save/restore the
    LC_ALL anymore.
    
    On Windows, standard streams like sys.stdout now always use
    surrogateescape error handler by default (ignore the locale).

This change caused a performance regression: bpo-35195 "[Windows] Python 3.7 initializes LC_CTYPE locale at startup, causing performance issue on msvcrt isdigit()".

Attached PR implements proposed change.

----------
components: Interpreter Core
messages: 342677
nosy: ncoghlan, steve.dower, twouters, vstinner
priority: normal
severity: normal
status: open
title: Add _PyPreConfig.configure_locale: allow to leave LC_CTYPE unchanged when embedding Python
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36945>
_______________________________________