Why exception from os.path.exists()?

[Peter J. Holzer]

On 2018-06-11 12:24:54 +0000, Steven D'Aprano wrote:
> On Mon, 11 Jun 2018 12:31:09 +0200, Peter J. Holzer wrote:
> > On 2018-06-11 01:06:37 +0000, Steven D'Aprano wrote:
> >> On Sun, 10 Jun 2018 23:57:35 +0200, Peter J. Holzer wrote:
> > 
> > [Note: I was talking about os.stat here, not os.path.exists. I agree
> > that os.path.exists (and the other boolean functions) should simply
> > return false]
> 
> o_O
> 
> Well... I don't know what to say. In a thread about os.path.exists, 
> you're talking about os.stat. I see. I don't understand why,

Easy to explain. Marko wrote:

| It may even be that the fix needs to go to os.stat(). That's for the
| Python gods to decide.

And to this I replied:

| I'm not a Python god, but I don't think os.stat() should be changed.
[and then explained why]

I do take some care to quote only what is necessary for context and
phrase my replies in a way which make it clear what I'm replying to.
Apparently I wasn't successful in this case (even though the message is
relatively short). If you have suggestions on how I might have been
clearer, I'm all ears.


> but if you want to derail the thread to discuss something else,

I wasn't derailing the thread, I was replying to a specific suggestion.

> okay, I'll play along.
> 
> 
> [...]
> > We are talking about platform-specific code here.
> 
> Are we? I thought we were talking about Python and the os module. The 
> very first paragraph of the documentation for os says:
> 
>     This module provides a portable way of using operating
>     system dependent functionality.

Note the magic words "system dependent functionality". The way you call
os.stat() is the same on all systems, but what it actually does and what
it returns depends on the OS. For example, the description of class
os.stat_result says:

|  st_ino
|
|    Platform dependent, but if non-zero, uniquely identifies the file
|    for a given value of st_dev.
[...]
| On some Unix systems (such as Linux), the following attributes may also
| be available:
[...]
| On other Unix systems (such as FreeBSD), the following attributes may be
| available (but may be only filled out if root tries to use them):
[...]
| On Mac OS systems, the following attributes may also be available:
[...]
| On Windows systems, the following attribute is also available:

So, quite some variation dependent on the system type.

> It also clearly states:
> 
>     All functions in this module raise OSError in the case of
>     invalid or inaccessible file names and paths, or other
>     arguments that have the correct type, but are not accepted
>     by the operating system. 
> 
> You know... like strings with NUL in them.

Ok. I missed that. So either the documentation or the implementation
should be fixed. 

In any case, if the implementation is changed, I still think that
OSError(ENOENT) is wrong. It would have to be OSError(None, "embedded
null byte"), or, if that is not possible (I haven't checked)
OSError(EINVAL, "embedded null byte"), although that is slightly
misleading (it implies that the OS returned EINVAL, which it didn't).

The same check for NUL is also in other functions (e.g. open()), so
those would have to be changed as well.


> > On POSIX systems, there IS NO WAY to pass a filename with an
> > embedded NUL byte to the OS.
> 
> Mac OS X is certified as POSIX-compliant. As I pointed out in a previous 
> email, OS X also provides APIs that are perfectly capable of dealing with 
> NULs in file names.

I wasn't entirely clear here. What I meant is that POSIX systems, as a
group, provide no such way. Some specific systems may have such an API,
but it isn't in POSIX and it not even likely to be the same on different
systems, so it cannot be used portably across POSIX systems or even the
subset of systems which allow NUL in filenames (unless that subset
happens to contain only one OS).


> POSIX specifies a *minimum* set of functionality, not a maximum.

But when writing an application for POSIX systems you are interested in
that minimum set of functionality. Plus maybe some common extensions.

Oh, and from the test results Gregory Ewing posted, it looks like Python
does use the POSIX API (and not the Carbon API) on MacOS. So you won't
ever see a filename containing NUL in a Python program on MacOS and you
won't be able to create one.

        hp

-- 
   _  | Peter J. Holzer    | we build much bigger, better disasters now
|_|_) |                    | because we have much more sophisticated
| |   | hjp at hjp.at         | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20180613/f4a8d126/attachment.sig>