More CPUs doen't equal more speed

Terry Reedy tjreedy at udel.edu
Fri May 24 01:27:10 EDT 2019


On 5/23/2019 2:39 PM, Bob van der Poel wrote:
> I've got a short script that loops though a number of files and processes
> them one at a time. I had a bit of time today and figured I'd rewrite the
> script to process the files 4 at a time by using 4 different instances of
> python.

As others have said, you give no evidence that you are doing that.

The python test suite runner has an argument to use multiple cores.  For 
me, 'python -m test -j0' runs about 6 times faster than 'python -m 
test'.  The speedup would be faster except than there is one test file 
that takes over two minutes, and may run at least a minute after all 
other processes have quit.  (I have suggested than long running files be 
split to even out the load, but that has not gained favor yet.)  For 
this use, more CPUs *does* equal more speed.

-- 
Terry Jan Reedy




More information about the Python-list mailing list