subprocess.Popen and thread module

[Danny Wong (dannwong)]

I did read that portion of the doc, then I change it to communicate and it still hangs. So I reverted back to "wait" while launching one thread to see if I could isolate the problem, but it still hangs regardless of using "wait" or "communicate".

-----Original Message-----
From: chris at rebertia.com [mailto:chris at rebertia.com] On Behalf Of Chris Rebert
Sent: Tuesday, August 09, 2011 11:53 PM
To: Danny Wong (dannwong)
Cc: python-list at python.org
Subject: Re: subprocess.Popen and thread module

> On Tue, Aug 9, 2011 at 11:38 PM, Danny Wong (dannwong)
> <dannwong at cisco.com> wrote:
>> Hi All,
>>   I'm trying to execute some external commands from multiple database.
>> I'm using threads and subprocess.Popen ( from docs, all the popen*
>> functions are deprecated and I was told to use subprocess.Popen) to
>> execute the external commands in parallel, but the commands seems to
>> hang.
>
> What's your Popen() call look like?

On Tue, Aug 9, 2011 at 11:48 PM, Danny Wong (dannwong)
<dannwong at cisco.com> wrote:
>    try:
>        cmd_output = subprocess.Popen(['scm', 'load', '--force', '-r', nickname, '-d', directory, project], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>        status = cmd_output.wait()

Er, did you read the warning about Popen.wait() in the docs? (emphasis added):
"""
Popen.wait()
    Wait for child process to terminate. Set and return returncode attribute.
    Warning: ***This will deadlock*** when using stdout=PIPE and/or
stderr=PIPE and the child process generates enough output to a pipe
such that it blocks waiting for the OS pipe buffer to accept more
data. Use communicate() to avoid that.
"""
– http://docs.python.org/library/subprocess.html#subprocess.Popen.wait

Cheers,
Chris