Why exception from os.path.exists()?

[Chris Angelico]

On Thu, May 31, 2018 at 11:38 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Chris Angelico <rosuav at gmail.com>:
>> Do you have an actual use-case where it is correct for an invalid path
>> to be treated as not existing?
>
> Note that os.path.exists() returns False for other types of errors
> including:
>
>  * File might exist but you have no access rights
>
>  * The pathname is too long for the file system
>
>  * The pathname is a broken symbolic link
>
>  * The pathname is a circular symbolic link
>
>  * The hard disk ball bearings are chipped

All of those are conceptually valid filenames, and it's perfectly
reasonable to ask if the file exists. Running the same program inside
a chroot might result in a True.

> I'm not aware of any other kind of a string argument that would trigger
> an exception except the presence of a NUL byte.

With a zero byte in the file name, it is not a valid file name under
any Unix-based OS. Regardless of the file system, "\0" is not valid.

> The reason for the different treatment is that the former errors are
> caught by the kernel and converted to False by os.path.exists(). The NUL
> byte check is carried out by Python's standard library.

That's because the kernel, having declared that zero bytes are
invalid, uses ASCIIZ filenames. It's way simpler that way. So the
Python string cannot validly be turned into input for the kernel. It's
on par with trying to represent 2**53+1.0 - it's not representable and
will behave differently. With floats, you get something close to the
requested value; with strings, they'd be truncated. But either way,
you absolutely cannot represent the file name "spam\0ham" to any Unix
kernel, because the file name is fundamentally invalid.

Can someone on Windows see if there are other path names that raise
ValueError there? Windows has a whole lot more invalid characters, and
invalid names as well.

ChrisA