[issue17588] runpy cannot run Unicode path on Windows

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc added the comment:

The issue is actually with compile():
  compile('x=1', '\u222b.py', 'exec')
fails on my Western Windows machine (mbcs = cp1252).
This conversion should not be necessary, since the filename is only used for error messages (and decoded again!)

But unfortunately the various API functions used by compile() are documented to take a filename encoded with the filesystem encoding:
http://docs.python.org/dev/c-api/veryhigh.html#Py_CompileStringExFlags
This API is unfortunate; on Windows Python should never have to convert filenames unless bytes strings are explicitly used.

I can see two ways to fix the issue:
- build another set of APIs which take unicode strings for the filename, or at least encoded to UTF-8.
- use some trick for unencodable filenames; filename.encode('mbcs', 'backslashreplace') works, but does not round-trip (and cannot fetch source code in tracebacks). I don't know if there is some variant of surrogateescape that we could use.

----------
nosy: +amaury.forgeotdarc

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17588>
_______________________________________