[Python-Dev] Emit a BytesWarning on bytes filenames on Windows

[Mark Hammond]

On 29/10/2011 9:52 AM, Victor Stinner wrote:
> Hi,
>
> I am not more conviced that raising a UnicodeEncodeError on unencodable
> characters is the right fix for the issue #13247. The problem with this
> solution is that you have to wait until an user get a UnicodeEncodeError.
>
> I have yet another proposition: emit a warning when a bytes filename is used.
> So it doesn't affect the default behaviour, but you can use -Werror to test if
> your program is fully Unicode compliant on Windows (without having to test
> invalid filenames).
>
> I don't know if a BytesWarning or a DeprecationWarning is more apropriate. It
> depends if we plan to drop support of bytes filenames on Windows later (in
> Python 3.5 or later).

When previously discussing this issue, I was under the impression that 
the problem was unencodable bytes passed from the Python code to Windows 
- but the reverse is true - only the data coming back from Windows isn't 
encodable.

This changes my opinion significantly :)  I don't think raising an error 
is the right choice - there are almost certainly use-cases where the 
current behaviour works OK and we would break them (eg, not all files in 
a directory are likely to be unencodable).  As the data came externally, 
the only solution the programmer has is to change to the unicode version 
of the api - so we recommend the bytes version not be used by anyone, 
anytime - which means it is conceptually deprecated already.

Therefore, as you imply, I think the solution to this issue is to start 
the process of deprecating the bytes version of the api in py3k with a 
view to removing it completely - possibly with a less aggressive 
timeline than normal.  In Python 2.7, I think documenting the issue and 
a recommendation to always use unicode is sufficient (ie, we can't 
deprecate it and a new BytesWarning seems gratuitous.)

Cheers,

Mark