[issue3187] os.listdir can return byte strings

[Martin v. Löwis]

Martin v. Löwis <martin at v.loewis.de> added the comment:

I'd like to propose yet another approach: make sure that conversion
according to the file system encoding always succeeds. If an
unconvertable byte is detected, map it into some private-use character.
To reduce the chance of conflict with other people's private-use
characters, we can use some of the plane 15 private-use characters, e.g.
map byte 0xPQ to U+F30PQ (in two-byte Unicode mode, this would result in
a surrogate pair).

This would make all file names accessible to all text processing
(including glob and friends); UI display would typically either report
an encoding error, or arrange for some replacement glyph to be shown.

There are certain variations of the approach possible, in case there is
objection to a specific detail.

----------
nosy: +loewis

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3187>
_______________________________________