[issue31900] localeconv() should decode numeric fields from LC_NUMERIC encoding, not from LC_CTYPE encoding

STINNER Victor report at bugs.python.org
Mon Jan 15 08:05:27 EST 2018


STINNER Victor <victor.stinner at gmail.com> added the comment:

Example of Fedora 27 and Python 3.6:

vstinner at apu$ env -i LC_NUMERIC=uk_UA.koi8u python3 -c 'import locale; print(locale.setlocale(locale.LC_ALL, "")); print(locale.getpreferredencoding(), ascii(locale.localeconv()["thousands_sep"]))'
LC_CTYPE=C.UTF-8;LC_NUMERIC=uk_UA.koi8u;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;LC_IDENTIFICATION=C
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.6/locale.py", line 110, in localeconv
    d = _localeconv()
UnicodeDecodeError: 'locale' codec can't decode byte 0x9a in position 0: Invalid or incomplete multibyte or wide character

"env -i" starts Python in an empty environment. It seems like LC_CTYPE defaults to C.UTF-8 in this case.

* LC_CTYPE = C.UTF-8, encoding = UTF-8
* LC_NUMERIC = uk_UA.koi8u, encoding = KOI8-U


With my PR, it works:

vstinner at apu$ env -i LC_NUMERIC=uk_UA.koi8u ./python -c 'import locale; print(locale.setlocale(locale.LC_ALL, "")); print(locale.getpreferredencoding(), ascii(locale.localeconv()["thousands_sep"]))'
LC_CTYPE=C.UTF-8;LC_NUMERIC=uk_UA.koi8u;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;LC_IDENTIFICATION=C
UTF-8 '\xa0'

=> thousands_sep byte string b'\x9A' is decoded as the Uniode string '\xa0'.


vstinner at apu$ env -i LC_NUMERIC=uk_UA.koi8u ./python -c 'import locale; locale.setlocale(locale.LC_ALL, ""); print(ascii(f"{1234:n}"))'
'1\xa0234'

=> the number is properly formatted


vstinner at apu$ env -i LC_NUMERIC=uk_UA.koi8u ./python -c 'import locale; locale.setlocale(locale.LC_ALL, ""); print(f"{1234:n}")'
1 234

It's possible to display the result using print().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue31900>
_______________________________________


More information about the Python-bugs-list mailing list