[Python-Dev] file system path protocol PEP

[Ethan Furman]

On 05/11/2016 09:55 PM, Serhiy Storchaka wrote:

> [...]  But for example os.walk() was significantly
> boosted with using os.scandir(), it would be sad to make it slower
> again.

scandir's speed improvement is due to not not throwing away data the OS 
was already giving us.

> os.path is used in number of files, sometimes in loops, sometimes
> indirectly. It is hard to find all examples.

Currently, any of these functions that already take a string have to do 
a couple pointer comparisons to make sure they have a string; any of 
these functions that take both a string and a bytes have to do a couple 
pointer comparisons to make sure they have a string or a bytes;  the 
only difference if this PEP is accepted is the fall-back path when those 
first checks fail.

As an example: if os.walk is called with a Path, it converts the Path to 
a string (once!) and then uses that string to generate more strings and 
return strings.  When os.walk calls os.path.join or os.path.split it 
will be with strings (or bytes), not with the original Path object.

> Such functions as glob.glob() calls split() and join() for every
> component, but they also use string or bytes operations with paths. So
> they need to convert argument to str or bytes before start iteration,
> and always call os.path functions only with str or bytes.

Exactly.

> Additional
> conversion in every os.path function is redundant.

And won't happen, since the fast-path checks will confirm that the 
argument is a string or bytes object.

> I suppose most other
> high-level functions that manipulates paths in a loop also should
> convert arguments once at the start and don't need the support of path
> protocol in os.path functions.

Human's can call the os.path functions; therefore, the os.path functions 
need to support the __fspath__ protocol.

> I'm for adding conversions in C implemented path consuming APIs and may
> be in high-level path manipulation functions like os.walk(), but left
> low-level API of os.path, fnmatch and glob unchanged.

So instead of the easy to remember "Path doesn't work with the rest of 
the standard library" we'll have "Path works with some APIs, but not 
others -- guess you better look it up" ?  That is not an improvement.

--
~Ethan~