【Regarding Performance of a Python Script....】

[Peter J. Holzer]

On 2020-02-28 19:24:18 -0500, Kenzi wrote:
>  I have a question regarding a simple code snippet in Python:
> 
> from subprocess import check_output
> for i in range(1024):
>     check_output(['/bin/bash', '-c', 'echo 42'], close_fds=True)
> 
> *I wonder why running it in Python 3.7 is much faster than Python 2.7? *
> (Python 3.7 is still faster, after I used *xrange * in Python 2.7)

I think almost all of the time is spent in the child processes, so it
doesn't matter whether you use range or xrange.

On my laptop, the program takes about 2.1 seconds with python 2.7 and
1.6 seconds with python 3.6. So the difference is 0.5 seconds overall or
about 500 µs per execution.

strace shows that python 2.7 explicitely closes all unneeded file
descriptors below 1024 (even though most of them aren't actually open)
in the child before execing bash, python 3 doesn't do that.

I can see no other obvious difference. 

So that's ~ 1020 extra system calls. If this is indeed the only
difference, that's about 500 ns per system call. That sounds plausible.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20200229/8f99b9d3/attachment.sig>