[issue3975] PyTraceBack_Print() doesn't respect # coding: xxx header

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc <amauryfa at gmail.com> added the comment:

> Ok, O_BINARY and GIL fixed.
which patch? ;-)

> I just realized that sys.path is considered as a bytes string
> (PyBytes_Check) whereas Python3 uses an unicode string!
> 
> ['', '/home/haypo/prog/python-ptrace', (...)]
> >>> sys.path.append(b'bytes'); sys.path
> ['', '/home/haypo/prog/python-ptrace', '(...)', b'bytes']
> 
> Oops, so it's possible to mix strings and bytes. But the normal case
is a list 
> of unicode strings. Another fix is needed :-/

You could do like the import function, which converts unicode with the
fs encoding. See the code in import.c, below "PyList_GetItem(path, i)"

> I don't know how to get into "if (fd < 0)" (code using sys.path).
> Any clue?

The .pyc stores the full path of the .py, at compilation time. If the
directory is moved, or accessed with another name (via mounts or soft
links), the traceback displays the filename as stored in the .pyc file,
but the content can still be displayed. It does not work every time,
though, for example when the directory is shared between Windows & Unix.

> > Besides this, the "char* filename" is relevant only on utf8-encoded
> > filesystems, and will not work on windows with non-ascii filenames.
> > But this is another issue.
> 
> I don't know the Windows API enough to answer.

Windows interprets the char* variables with its system-set "mbcs"
encoding. On my machine, it is equivalent to the cp1252 encoding (almost
equivalent latin-1). Since the 'filename' comes from a call to
_PyUnicode_AsString(tb->tb_frame->f_code->co_filename), it will be wrong
with any accented letter.

> Doesn't Python provide "high level" functions to open a file?
io.open()?

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3975>
_______________________________________