[Python-Dev] Late Python 3.7.1 changes to fix the C locale coercion (PEP 538) implementation

[Nick Coghlan]

On Thu, 20 Sep 2018 at 06:48, Ned Deily <nad at python.org> wrote:
>
> On Sep 19, 2018, at 15:08, Victor Stinner <vstinner at redhat.com> wrote:
> > Le mardi 18 septembre 2018, Victor Stinner <vstinner at redhat.com> a écrit :
> > > Hi Unicode and locales lovers,
> > >
> > > tl; dr Nick, Ned, INADA-san: I modified 3.7.1 to add a new "-X
> > > coerce_c_locale=value" option and make sure that the C locale coercion
> > > cannot be when Python in embedded: are you ok with these changes?
> >
> > Nick asked me to revert, which means that no, he is not ok with these changes.
> >
> > I reverted my change in 3.7.
>
> Thank you, Victor!
>
> Nick, with regard to this does the current state of the 3.7 branch look acceptable now for a 3.7.1?

It's still broken relative to the PEP in the following respects:

- Py_Initialize() coerces the C locale to C.UTF-8, even though it's
not supposed to
- Py_Main() coerces the C locale to C.UTF-8, even though it's not supposed to
- PYTHONCOERCECLOCALE=0 doesn't work if -E or -I are passed on the
command line (but it's supposed to)
- PYTHONCOERCECLOCALE=warn doesn't work if -E or -I are passed on the
command line (it's nominally supposed to do this too, but I'm less
concerned about this one)

The problem with Victor's patch is that instead of reverting to the
as-designed-and-accepted PEP the way my PR (mostly) does, it instead
introduces a whole new command line option (which then needs to be
documented and tested), and still coerces *far* too late (not until
Py_Initialise is already running, after who knows how much code in the
embedding application has already executed).

I don't have the time required to push through Victor's insistence
that -E and -I are sacrosanct and must always be respected (despite
PEP 538 explicitly saying that they won't be where PYTHONCOERCECLOCALE
is concerned), and so we can't *possibly* change back to having the
locale coercion work the way I originally implemented it, so I wrote
the 3.7.x series off as a lost cause, and decided to devote my
energies to getting things back to the way they were supposed to be
for 3.8+.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia