Why exception from os.path.exists()?

[Steven D'Aprano]

On Fri, 01 Jun 2018 11:58:42 -0400, Richard Damon wrote:

> I would say that one way to look at it is that os.path.exists
> fundamentally (at the OS level) expects a parameter of the 'type' of
> either a nul terminated string or a file descriptor (aka fixed width
> integer). One issue we have is that these 'types' don't directly map to
> Python types.


What a strange and unhelpful way to look at it. When calling 
os.path.exists() from Python, are you interacting directly with the OS in 
low-level C or assembly, or using a high-level language like Python?

That's not a rhetorical question. I would like to know what you think we 
are doing when we type into the Python interpreter

import os
os.path.exists("pathname")

Well actually no I lie. Of course its a rhetorical question: I'm sure you 
know full well that you're using Python, a high-level language.


> We can basically make a call to os.path.exists with 4 different types of
> parameter:
> 
> 1) The parameter has a totally wrong type of type that just doesn't map
> to one of the expected type. This gives a TypeError exeception.
> 
> 2) The parameter has a Python type that maps to right OS 'type' but has
> a value that prevents us from properly converting it to a corresponding
> value of that type. This could be a integral value out of range for the
> fixed width type used, 

That is a reasonable case for either ValueError or OverflowError, 
OverflowError being a more specific (and therefore useful) exception to 
raise. Being a low-level detail, file descriptors are inherently limited 
to a fixed width and it truly is an error to supply something outside of 
that width.

> or a string which contains an embedded nul.

The fact that this is illegal under POSIX is an irrelevant platform-
dependent detail. Irrelevant in the sense that it shouldn't change the 
API of the function. "<" doesn't raise ValueError on Windows, and "" 
doesn't raise ValueError on any platform. Why should POSIX nulls be 
treated as special?


> Currently these generate an OverflowError for out of range integer and a
> ValueError for a bad string

Incorrect: as Paul and MRAB (and myself) have pointed out, bad strings 
return False, with the surprising exception of null-embedded strings 
under POSIX.

Illegal strings like "<" on Windows return False. Excessive long strings, 
or strings with too many path components, return False. The empty string 
returns False.

There is no good reason to raise ValueError on strings with null in them.


> 3) The parameter can be mapped to the proper type but the value is
> somehow illegal (the number fits the type, but isn't legal for a file
> descriptor, or a string has a value that can't represent a real file).
> In this case, os.path.exists doesn't try to validate the parameter but
> just passes it along and returns a value based on the answer it gets.

Right. And if the answer it gets is "illegal value", it returns False.



> 4) The parameter represents a legal value of a right type, so as above
> we pass the value and get back the answer.
> 
> The fundamental question is about case 2. Should os.path.exist, having
> been give a value of the right 'Python Type' but not matching the type
> of the operating system parameter identify this as an error (as it
> currently does), 

That's wrong, that is not what it does, except in the surprising case of 
nulls under POSIX.


> or should it be changed to decide that if it could
> somehow get that parameter to the os, then it would say that the file
> doesn't exist, and so return false.

Of course.


> I would say that if you accept that,
> should we also say that if we pass a totally wrong type, why shouldn't
> we again return false instead of a TypeError, after all, if we pass it a
> dictionary, they certainly is no file like that in existence,

Because os.path.exists is documented as accepting strings, bytes and 
ints, and everything else is a TypeError.

If you want to make a case for relaxing the type restrictions on 
os.path.exists, go right ahead, but I won't be supporting you.



-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson