[issue29409] Implement PEP 529 for io.FileIO

[Eryk Sun]

New submission from Eryk Sun:

PEP 529 isn't implemented for io.FileIO, and I think it should be. If a UTF-8 path is passed to open(), it ends up calling C _open instead of decoding the path and calling C _wopen. Also, if a pathlike object is passed to io.FileIO, it calls PyUnicode_FSConverter on it, converting it to UTF-8 and then passing it to _open. For example:

    >>> p = r'C:\Temp\αβψδ'
    >>> os.path.exists(p)
    True

    >>> open(p.encode())
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    FileNotFoundError: [Errno 2] No such file or directory: b'C:\\Temp\\\xce\xb1\xce\xb2\xcf\x88\xce\xb4'

    >>> io.FileIO(pathlib.Path(p))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    FileNotFoundError: [Errno 2] No such file or directory: WindowsPath('C:/Temp/αβψδ')

The Windows implementation should mirror the POSIX implementation via PyUnicode_FSDecoder.

----------
components: IO, Windows
messages: 286585
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Implement PEP 529 for io.FileIO
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29409>
_______________________________________