I don't understand popen2 :(

[Penfold]

This is simply the usual nightmare of dealing with pipes between processes.
The problem is simple ... both the
po_out.read() and pe_err.read() are *blocking* reads ... further more since
you used read and not read(100) eg,
they will block until they can read to the end of file (ie the whole output
of the command).

In your case when you do a po_read first it works since the comand has
produced all its output, you read it, then it dies
and you read the error stream, all well and cool.  In the latter case, you
try and read the error stream first ... but your
child process refuses to die until its output pipe has been read ... hence
it idles doing nothing and you idle since you've just
entered a blocking read on the error pipe == deadlock.  Bummer :-)  Or
something like that.

Solution ... I tend to alter the pipes to no longer be blocking ... I'm not
sure what other people do in these cases.  You
can use fctl to do this but I forget the flags you want, something like
NDELAY and O_NOBLOCK ... I have none of
my sample code to hand.  Look at man fcntl.

Alternatively, just read the pipes in a sensible order :-)

D.

<marco at atmosp.physics.utoronto.ca> wrote in message
news:96ejto$4q3$1 at news.netmar.com...
>
> Hello All,
>
> I'm a python newbie, happy so far, but somewhat bewildered :(
>
> My problem is with popen2.popen3. It sometimes gets stuck when
> reading the process std_out/err. Here is an example:
>
> >>> (po, pi, pe) = popen2.popen3('ssh computer "ls"')
> >>> po_out = po.read()
> >>> pe_err = pe.read()
> >>> pe_err
> ''
> >>> po_out
> [Directory listing OK]
>
> Alternatively (notice the read order):
>
> >>> (po, pi, pe) = popen2.popen3('ssh computer "ls"')
> >>> pe_err = pe.read()
> >>> po_out = po.read()
> >>> pe_err
> ''
> >>> po_out
> [Directory listing OK]
>
> Fine. Now I try something slightly different:
>
> >>> (po, pi, pe) = popen2.popen3('ssh computer "find /home/marco"')
> >>> po_out = po.read()
> >>> pe_err = pe.read()
> >>> pe_err
> ''
> >>> po_out
> [snip find results]
>
> which is fine, BUT if I do:
>
> >>> (po, pi, pe) = popen2.popen3('ssh computer "find /home/marco"')
> >>> pe_err = pe.read()
>
> [freezes!! I then Ctrl-C]
>
> Traceback (innermost last):
>   File "<stdin>", line 1, in ?
> KeyboardInterrupt
>
> >>> po_out = po.read()
> >>> po_out
> [snip find results]
>
> So, in this last case if I pe.read() first
> my script freezes, and I have to Ctrl-C. Sure, I
> could always po.read() first, but my biggest problem
> is that sometimes it's the other way around! That is,
> performing po.read() first causes a freeze! Unfortunately,
> I cannot reliably reproduce this last behaviour, but here
> is the result from a script I had to Ctrl-C:
>
> prompt> ./myprog.py
>
> [frozen -- Ctrl-C]
>
> Traceback (innermost last):
>   File "./myprog.py", line 1245, in ?
>     get_files()
>   File "./myprog.py", line 1102, in get_files
>     po_out = po.read()
> KeyboardInterrupt
>
> line #
>  1101     (po, pi, pe) = popen2.popen3(scp_from_remote_IP)
>  1102     po_out = po.read()
>  1103     pe_err = pe.read()
>
> Sadly, the scp operation *does* seem to work most of the time,
> why does it sometimes get stuck? Thoughts? Ideas?
>
> I'm using:
> Python 1.5.2 (#1, Feb  1 2000, 16:32:16)  [GCC egcs-2.91.66 19990314/Linux
> (egcs- on linux-i386)]
>
> Thanks for any help!
> (Oh, and please CC me a reply if possible -- Deja's undergoing
> transition pains to Google right now)
>
>  -----  Posted via NewsOne.Net: Free (anonymous) Usenet News via the
eb  -----
>   http://newsone.net/ -- Free reading and anonymous posting to 60,000+
groups
>    NewsOne.Net prohibits users from posting spam.  If this or other posts
> made through NewsOne.Net violate posting guidelines, email
abuse at newsone.net