[Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Victor Stinner]

Hi,

Guido van Rossum and others asked me details on how file descriptors
and handles are inherited on Windows, for the PEP 446.
http://www.python.org/dev/peps/pep-0446/

I hacked Python 3.4 to add a os.get_cloexec() function (extracted from
my implementation of the PEP 433), here are some results.

Python functions open(), os.open() and os.dup() create file
descriptors with the HANDLE_FLAG_INHERIT flag set (cloexec=False),
whereas os.pipe() creates 2 file descriptors with the
HANDLE_FLAG_INHERIT flag unset (cloexec=False, see also issue #4708).

Even if the HANDLE_FLAG_INHERIT flag is set, all handles are closed if
subprocess is used with close_fds=True (which is the default value of
the parameter), and all file descriptors are closed except 0, 1 and 2.

If close_fds=False, handles with the HANDLE_FLAG_INHERIT flag set are
inherited, but all file descriptors are still closed except 0, 1 and
2.

(I didn't check if file descriptors 0, 1 and 2 are inherited,
duplicated or new file descriptors.)

The PEP 466 allows you to control which handles are inherited to child
process when you use subprocess with close_fds=False. (The subprocess
parameter should be called "close_handles" on Windows to avoid
confusion.)

Said differently: the HANDLE_FLAG_INHERIT flag only has an effect on
*handles*, as indicated in its name. On Windows, file *descriptors*
are never inherited (are always closed) in child processes. I don't
think that it is possible to inherit file descriptors on Windows.

By the way, using pass_fds on Windows raises an assertion error
("pass_fds not supported on Windows").

Another example in Python:
---
import subprocess, sys

code = """
import os, sys
fd = int(sys.argv[1])
f = os.fdopen(fd, "rb")
print(f.read())
"""

f = open(__file__, "rb")
fd = f.fileno()
subprocess.call([sys.executable, "-c", code, str(fd)], close_fds=False)
---

On Unix, the child process will write the script into stdout. On
Windows, you just get an OSError(9, "Bad file descriptor") exception.

To fix this example on Windows, you have to:

* Retrieve the handle of the file using msvcrt.get_osfhandle() ;
* Pass the handle, instead of the file descriptor, to the child ;
* Create a file descriptor from the handle using
msvcrt.open_osfhandle() in the child.

The fix would be simpler if Python would provide the handle of a file
object (ex: in a method) and if open() supported opening a handle as
it does with file descriptors on UNIX.

Victor