[Python-Dev] PEP 446 (make FD non inheritable) ready for a final review

[Charles-François Natali]

Hello,

A couple remarks:

> The following functions are modified to make newly created file descriptors non-inheritable by default:
> [...]
> os.dup()

then

> os.dup2() has a new optional inheritable parameter: os.dup2(fd, fd2, inheritable=True). fd2 is created inheritable by default, but non-inheritable if inheritable is False.

Why does dup2() create inheritable FD, and not dup()?

I think a hint is given a little later:

> Applications using the subprocess module with the pass_fds parameter or using os.dup2() to redirect standard streams should not be affected.

But that's overly-optimistic.

For example, a lot of code uses the guarantee that dup()/open()...
returns the lowest numbered file descriptor available, so code like
this:

r, w = os.pipe()
if os.fork() == 0:
    # child
    os.close(r)
    os.close(1)
    dup(w)

*will break*

And that's a lot of code (e.g. that's what _posixsubprocess.c uses,
but since it's implemented in C it's wouldn't be affected).

We've already had this discussion, and I stand by my claim that
changing the default *will break* user code.
Furthermore, many people use Python for system programming, and this
change would be highly surprising.

So no matter what the final decision on this PEP is, it must be kept in mind.

> The programming languages Go, Perl and Ruby make newly created file descriptors non-inheritable by default: since Go 1.0 (2009), Perl 1.0 (1987) and Ruby 2.0 (2013).

OK, but do they expose OS file descriptors?
I'm sure such a change would be fine for Java, which doesn't expose
FDs and fork(), but Python's another story.

Last time, I said that to me, the FD inheritance issue is solved on
POSIX by the subprocess module which passes close_fds. In my own code,
I use subprocess, which is the "official", portable and safe way to
create child processes in Python. Someone using fork() + exec() should
know what he's doing, and be able to deal with the consequences: I'm
not only talking about FD inheritance, but also about
async-signal/multi-threaded safety ;-)

As for Windows, since it doesn't have fork(), it would make sense to
make its FD non heritable by default. And then use what you describe
here to selectively inherit FDs (i.e. implement keep_fds):
"""
Since Windows Vista, CreateProcess() supports an extension of the
STARTUPINFO struture: the STARTUPINFOEX structure. Using this new
structure, it is possible to specify a list of handles to inherit:
PROC_THREAD_ATTRIBUTE_HANDLE_LIST. Read Programmatically controlling
which handles are inherited by new processes in Win32 (Raymond Chen,
Dec 2011) for more information.
"""

cf