Python 3: Launch multiple commands(subprocesses) in parallel (but upto 4 any time at same time) AND store each of their outputs into a variable

lax.clarke at gmail.com lax.clarke at gmail.com
Tue Aug 23 23:15:09 EDT 2016


Hi,

I've been reading various forums and python documentation on subprocess, multithreading, PIPEs, etc.  But I cannot seem to mash together several of my requirements into working code.

I am trying to:

1) Use Python 3+ (specifically 3.4 if it matters)
2) Launch N commands in background (e.g., like subprocess.call would for individual commands)
3) But only limit P commands to run at same time
4) Wait until all N commands are done
5) Have an array of N strings with the stdout+stderr of each command in it.

What is the best way to do this?
There are literally many variations of things in the Python documentation and Stackoverflow that I am unable to see the forest from trees (for my problem).

Thank you very much!



More information about the Python-list mailing list