[issue3187] os.listdir can return byte strings

[STINNER Victor]

STINNER Victor <victor.stinner at haypocalc.com> added the comment:

I implemented the "invalid filename" class feature:
 - by default, os.listdir() raise an error (UnicodeDecodeError) on 
invalid filename. The previous behaviour was to return bytes object 
instead of str.
 - if invalid_filename=True: create an InvalidFilename class instance

InvalidFilename is not a bytes string, it's not a str string, it's a 
new class. It has three attributes:
 - bytes: the real filename
 - charset: charset (type str)
 - str: fake filename (type str) used by __str__() method

My patch also fixes os.path.join() to accept InvalidFilename: if at 
last one argument is an InvalidFilename, use InvalidFilename.join() 
(class method).

os.listdir() and os.unlink() are patched to accept InvalidFilename. 
unlink always accept InvalidFilename whereas listdir() only produces 
InvalidFilename is os.listdir(path, invalid_filename=True) is used.

I added an optional argument "invalid_filename" to shutil.rmtree(), 
default value is *True*.

To sum up, visible changes:
 - os.listdir() raise an error on invalid filename instead of return a 
mixed list of str and bytes
 - shutil.rmtree() manipulate str and InvalidFilename instead of str 
and bytes

Added file: http://bugs.python.org/file11210/invalid_filename.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3187>
_______________________________________