read and readline hanging

[Steve Holden]

Olivier Lefevre wrote:
> I am calling subprocess.Popen with
> p = Popen(args, bufsize=-1, stdin=PIPE, stdout=PIPE, stderr=PIPE)
> and after sending come command to the process, I try to read
> from p.stdout but after a few calls I hang. What is the correct
> way to do this, i.e., to read everything w/o getting stuck? I am
> not familiar with all these low-level functions and their annoying
> blocking behaviour. Note that I don't want to close the process
> because I want to send more commands to it after that, read the
> results, and so on and on.
> 
While I realise it isn't easy to determine what's going on in a hung 
program, I might ask what you have tried already? I can think of at 
least two reasons why your program might hang:

1. The subprocess has stopped producing output. If you are only reading 
its standard output, are you sure that the subprocess is flushing its 
buffers so you can recognize it's time to provide more input? Are you 
simply failing to provide more input?

2. The subprocess has blocked because it has filled its stderr buffer 
and is waiting for something (i.e. your program) to read it.

To eliminate the second possibility use stderr=STDOUT, which will then 
have both streams written to standard output.

In general subprocess management is much tricker and subtler than most 
people expect when they first dive into it, and I suspect your problem 
is a manifestation of that fact.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/