Undocumented issue: Open system call blocks on named pipes (and a feature request)

[Daniel Ojalvo]

Thank you for the advice!

I haven't used the opener argument before, but I'll keep it for future reference. I think it's still a little kludge-y, but it works.

I agree that previous behavior shouldn't be changed, but I would suggest updating the documentation to point it out as a footnote. The current behavior is correct just unclear. Most people just learning about the open command wouldn't have this expectation. I came across the issue when I had a program that would open up all the files in a directory to read a few bytes from the beginning. My concern would be someone just making a named pipe over a file that a program would open. Arguably, anyone affected by that would be shooting themselves in the foot to begin with, but I think there are "security" concerns because someone could cause a bit of mischief that would be difficult to diagnose.

That all being said, I think I would like to put in a feature request for a non-blocking option. How should I go about doing so?

Thanks again,
Dan

-----Original Message-----
From: Chris Angelico <rosuav at gmail.com> 
Sent: Thursday, December 27, 2018 7:10 PM
To: python-list at python.org
Subject: Re: Undocumented issue: Open system call blocks on named pipes (and a feature request)

On Fri, Dec 28, 2018 at 1:38 PM Daniel Ojalvo via Python-list <python-list at python.org> wrote:
>
> Hello,
>
> I've been working on a python3 project and I came across an issue with the open system call that, at the very least, isn't documented. In my humble opinion, the documentation<https://docs.python.org/3/library/functions.html#open> should be updated because folks wouldn't expect open to be a blocking operation and simply error out. Worse yet, open doesn't have an option to make itself non-blocking. You have to use the os system calls to kludge a solution.
>

Hmm. I disagree that the docs are deceptive here; I would normally expect open() to block if it needs to. But looking at this as a feature request, it seems reasonable. Actually, it's not even that hard to do, since open() is already pluggable:

rosuav at sikorsky:~/tmp$ rm rene_magritte
rosuav at sikorsky:~/tmp$ mkfifo rene_magritte rosuav at sikorsky:~/tmp$ ls -l rene_magritte
prw-r--r-- 1 rosuav rosuav 0 Dec 28 14:05 rene_magritte rosuav at sikorsky:~/tmp$ python3 Python 3.8.0a0 (heads/master:8b9c33ea9c, Nov 20 2018, 02:18:50) [GCC 6.3.0 20170516] on linux Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> def nonblock(fn, mode): return os.open(fn, mode | os.O_NONBLOCK)
...
>>> open("rene_magritte", opener=nonblock)
<_io.TextIOWrapper name='rene_magritte' mode='r' encoding='UTF-8'>
>>> _.read(1)
''

> Here is how I reproduced the issue:
>
> root at beefy:~/sandbox# mkfifo this_is_a_pipe

(my example file name is a more subtle reference...)

> I'm doing this to get a fileobject and make it error out if we do have a blocking special file:
> with os.fdopen(os.open(<filename>, os.O_RDONLY| os.O_NONBLOCK) , mode='rb') as file_obj:
>
> I think this is mostly a documentation bug because this wouldn't be expected behavior to someone reading the docs, but open is behaving as the fifo man page<http://man7.org/linux/man-pages/man7/fifo.7.html> is documented. The feature request would be to add a non-blocking option to the default open system call.
>

Honestly, I don't think there's a problem with it blocking by default.
Most of Python works that way. But it would be pretty straight-forward to add "nonblocking=False" as another keyword-only parameter, and for compatibility with existing versions (back as far as 3.3), the opener should work just fine.

ChrisA