draining pipes simultaneously

Dmitry Teslenko dteslenko at gmail.com
Wed Mar 5 04:33:49 EST 2008


Hello!
Here's my implementation of a function that executes some command and
drains stdout/stderr invoking other functions for every line of
command output:

def __execute2_drain_pipe(queue, pipe):
	for line in pipe:
		queue.put(line)
	return

def execute2(command, out_filter = None, err_filter = None):
	p = subprocess.Popen(command , shell=True, stdin = subprocess.PIPE, \
		stdout = subprocess.PIPE, stderr = subprocess.PIPE, \
		env = os.environ)

	qo = Queue.Queue()
	qe = Queue.Queue()

	to = threading.Thread(target = __execute2_drain_pipe, \
		args = (qo, p.stdout))
	to.start()
	time.sleep(0)
	te = threading.Thread(target = __execute2_drain_pipe, \
		args = (qe, p.stderr))
	te.start()

	while to.isAlive() or te.isAlive():
		try:
			line = qo.get()
			if out_filter:
				out_filter(line)
			qo.task_done()
		except Queue.Empty:
			pass

		try:
			line = qe.get()
			if err_filter:
				err_filter(line)
			qe.task_done()
		except Queue.Empty:
			pass

	to.join()
	te.join()
	return p.wait()

Problem is my implementation is buggy and function hungs when there's
empty stdout/stderr. Can I have your feedback?



More information about the Python-list mailing list