[issue19977] Use "surrogateescape" error handler for sys.stdin and sys.stdout on UNIX for the C locale

[STINNER Victor]

STINNER Victor added the comment:

os.fsencode(text) always fail if text cannot be encoded to sys.getfilesystemencoding(). surrogateescape doesn't help here.

Your example is "artificial", you should not get 'ä'. All OS data is decoded from the filesystem encoding using the surrogateescape error handler (except on Windows, where strict is used, but it's a different story, Python uses Unicode functions when available so don't worry). So all these data can always be encoded back to bytes using os.fsencode().

More generally, os.fsencode(os.fsdecode(read_data)) == read_data is always true on Unix, with any filesystem (locale) encoding.

You may get Unicode data from other sources like files or a GUI, but I don't see what can be done here.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19977>
_______________________________________