python script hangs when run from subprocess

[Peter Otten]

Larry.Martell at gmail.com wrote:

> I have a python script and when I run it directly from the command line it
> runs to completion. But I need to run it from another script. I do that
> like this:
> 
> p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
> stderr=subprocess.STDOUT) rv = p.wait()
> out_buf = p.stdout.read()
> 
> When I do this, wait never returns. If I trace the underlying script it's
> always in the same write to stderr that never seems to complete:
> 
> write(2, "/KA22/05Feb12/Images/12063LBO003"..., 24457
> 
> I run many other scripts and commands in the same manner, and they all
> complete, it's just this one. Anyone have any ideas why this is happening,
> and how I can further debug or fix this?

The script writes to an OS buffer, and when that buffer is full it blocks 
forever. p.wait() in turn then waits forever for the script to terminate...

As a fix try

out_buf = subprocess.Popen(...).communicate()[0]


This uses threads or select (depending on the OS) to avoid the problem -- 
and is prominently mentionend in the documentation:

http://docs.python.org/2/library/subprocess.html#subprocess.Popen.wait