[New-bugs-announce] [issue20485] Enable 'import <Non-ASCII>.pyd'

[Suzumizaki]

New submission from Suzumizaki:

Currently, the name of .pyd modules is limited within 7 bit US-ASCII.

I want to do "import X" to import X.pyd, where X contains Unicode characters.
I tried to make the patch and my work seems to be done successfully.
I will post the patch with this issue, and next what should I do?

About the solution:
To make the export entry 'PyInit_xxx' kept inside 7 bit, I use the simple encoding 'szm62' for unicode, called in the patch.

1) 'szm62' is used once per module(.pyd), only for 'PyInit_xxx'.
2) 'szm62' is used only when non-ASCII characters are in the name of the module.
3) 'szm62' generates short string as UTF-8, except 0-9A-Za-z are encoded to 2 bytes.
4) 'szm62' is very simple, much easier than UTF-8.
5) I tested it only with MS VC++, but I believe highly compatible with other environments.
6) 'szm62' also can decode 7 bits to unicode, but only the encoding is used for this issue.

Notes:
The simplicity is important for the project like Cython -- it generates .pyd files.
The codepoints over 16bits are also simply supported. They will be encoded to 4 alnum(0-9A-Za-z) characters.
0-9A-Za-z are (always) encoded to 2 alnums. They will be simply prefixed with '0'(U+000030).
When the generating 'Non-ASCII.pyd' with MSVC toolkit, the report 'LINK : warning LNK4232' will be raised on linking. But no problem. The warning says "When someone try to link with LoadLibraryEx'A' function, it may or may not fail, depends on the user locale." Our Python.exe uses always LoadLibraryEx'W', it never fail with locale issue.

Or if you have any question, please tell me that.

Regards,
Suzumizaki-Kimitaka

----------
components: Interpreter Core
files: 20140202_patch_for_python_default_branch_88885.patch
keywords: patch
messages: 209988
nosy: Suzumizaki
priority: normal
severity: normal
status: open
title: Enable 'import <Non-ASCII>.pyd'
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file33866/20140202_patch_for_python_default_branch_88885.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20485>
_______________________________________