[issue37790] subprocess.Popen() is sometimes slower in python3 under illumos

[John Levon]

John Levon <movement at users.sourceforge.net> added the comment:

I checked, and the supposition this is due to lack of closefrom() doesn't seem to be correct. Running the test case and looking at 'truss' output, there is no large number of close() that one would expect if this was the issue.

I don't see Alexander's 2-time speedup however:

root at piano:/export/src/cpython# /export/python3/bin/python3 ./1.py
11.679689645767212
root at piano:/export/src/cpython# vi 1.p^C
root at piano:/export/src/cpython# /export/python3/bin/python3 ./1.py foo
10.402687549591064
root at piano:/export/src/cpython# python2.7 ./1.py 
10.0434100628

Any difference doesn't seem to be distinguishable from noise on my system.

If I run this:

from __future__ import print_function
import os
import sys
import time

def getoutput(cmd):
        # Hand-crafted variant
        if len(sys.argv) >1:
                import shlex, tempfile
                f, fpath = tempfile.mkstemp()
                status = os.system("{ " + cmd + "; } >" +
                        shlex.quote(fpath) + " 2>&1")
                with os.fdopen(f, "r") as tfile:
                        output = tfile.read()
                os.unlink(fpath)
                if output[-1:] == '\n':
                        output = output[:-1]
                return output
        else:
                import subprocess
                p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=None, close_fds=True)
                return p.communicate()[0]
        

t = time.time()
for file in getoutput("find /usr/bin -type f 2>/dev/null").decode().split('\n'):
        diff = getoutput("/usr/bin/objdump '%s' 2>/dev/null" % file)


then I get more variation than can be measured by changing close_fds.

Running something similar (no decode()) under python 2.7 is *slower* than python3.


So, something other than closefrom() is going on here.

----------
title: subprocess.Popen() is extremely slow (with close_fds=True which is the default) on Illumos -> subprocess.Popen() is sometimes slower in python3 under illumos

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37790>
_______________________________________