[Python-checkins] peps: PEP 446: add a section "Performances of Closing All File Descriptors"

[victor.stinner]

http://hg.python.org/peps/rev/3dbfe0c66c7e
changeset:   5036:3dbfe0c66c7e
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Aug 07 01:31:07 2013 +0200
summary:
  PEP 446: add a section "Performances of Closing All File Descriptors"

files:
  pep-0446.txt |  45 +++++++++++++++++++++++++++++++++++----
  1 files changed, 40 insertions(+), 5 deletions(-)


diff --git a/pep-0446.txt b/pep-0446.txt
--- a/pep-0446.txt
+++ b/pep-0446.txt
@@ -14,11 +14,15 @@
 ========
 
 Leaking file descriptors in child processes causes various annoying
-issues and is a known major security vulnerability. This PEP proposes to
-make all file descriptors created by Python non-inheritable by default
-to reduces the risk of these issues. This PEP fixes also a race
-condition in multithreaded applications on operating systems supporting
-atomic flags to create non-inheritable file descriptors.
+issues and is a known major security vulnerability. Using the
+``subprocess`` module with the *close_fds* parameter set to ``True`` is
+not possible in some cases, and has poor performances on some platforms.
+
+This PEP proposes to make all file descriptors created by Python
+non-inheritable by default to reduces the risk of these issues. This PEP
+fixes also a race condition in multithreaded applications on operating
+systems supporting atomic flags to create non-inheritable file
+descriptors.
 
 
 Rationale
@@ -276,6 +280,27 @@
   subprocess are inherited
 
 
+Performances of Closing All File Descriptors
+--------------------------------------------
+
+On UNIX, the subprocess module closes almost all file descriptors in the
+child process. This operation require MAXFD system calls where MAXFD is
+the maximum number of file descriptors, even if there are few open file
+descriptors. This maximum can be get using: ``sysconf("SC_OPEN_MAX")``.
+
+The operation can be slow if MAXFD is large. For example, on a FreeBSD
+buildbot with ``MAXFD=655,000``, the operation took 0.3 second: see
+`issue #11284: slow close file descriptors
+<http://bugs.python.org/issue11284#msg132668>`_).
+
+On Linux, Python gets the list of all open file descriptors from
+``/proc/<PID>/fd/``, and so performances depends on the number of open
+file descriptors, not on MAXFD.
+
+See also the `issue #1663329: subprocess close_fds perform poor if
+SC_OPEN_MAX is high <http://bugs.python.org/issue1663329>`_.
+
+
 Proposal
 ========
 
@@ -335,6 +360,16 @@
   (stdin), ``1`` (stdout) or ``2`` (stderr) and *fd2* is different than
   *fd*.
 
+Since Python should only create non-inheritable file descriptors, it is
+safe to use subprocess with the *close_fds* parameter set to ``False``.
+Not closing explicitly file descriptors is faster, especially on
+platform with a large maximum number of file descriptors.
+
+The default value of the *close_fds* parameter is unchanged, because
+third party modules, especially extensions implemented in C, may not
+conform immediatly to the PEP 446 (still create inheritable file
+descriptors).
+
 
 Backward Compatibility
 ======================

-- 
Repository URL: http://hg.python.org/peps