need something like threads, or just multiple simulateous exec's

Jeff Epler jepler at unpythonic.net
Mon Jan 26 10:41:31 EST 2004


I'm not sure why you need to use threads here.  popen()ed processes
execute asynchronously, blocking when they create enough unread output.
In this case, they'll generate very little output.  Why not try this?

# Create all processes at once
pipes = [os.popen(...) for ip in hosts]

# Read their output
output = [pipe.read for pipe in pipes]

Suppose you decide you want to run only N os.popen()s at a time.  Here
you'll want to use select() instead of threads.  Something like this:

remaining_hosts = hosts[:]
pipes = []
output = []
for i in range(nthreads):
while remaining_hosts or pipes:
	while remaining_hosts and len(pipes) < npipes:
		host = remaining_hosts.pop()
		pipes.append(os.popen(...))
	ready = select.select([pipes], [], [])[0]
	for pipe in ready:
		# Assuming output arrives all at once
		# (it should since it's one line)
		output.append(pipe.read())
		pipes.remove(pipe)
		pipe.close()

Jeff




More information about the Python-list mailing list