Bug/Weak Implementation? popen* routines can't handle simultaneous read/write?

[Nick Craig-Wood]

dmoore <damienlmoore at gmail.com> wrote:
>  I've seen the following issue come up in multiple posts to this
>  mailing list:
> 
>  I have a python program that spawns a child process with popen or
>  popen2 or popen3 or popen2.popen2 etc.
>  the child process is interactive: it asks for input then spits out
>  some output, asks for more input then spits out some output. for
>  example, consider the trivial child program:
> 
>  print "welcome"
>  print ">",
>  s=raw_input()
>  while s!='exit':
>      print "you entered:",s
>      print ">",
>      s=raw_input()
> 
>  Now I may be completely wrong about this (I did play with popen for a
>  very long time before writing this message), but it appears that none
>  of the popen variants allow for a sequence of reads and writes to/from
>  this child. that is, if I read from the open pipe's output I will
>  never be able to write the input for the child because the parent
>  program will block on read until eof (I will have similar blocking
>  problems if I start with write - using readline does not seem to
>  help).

You are correct.

>  the standard proposed remedy I have seen on this list is to use Unix-
>  only select/fctl, or otherwise dig into the bowls of the win32 api, or
>  download some half-complete sourceforge process control project.

If you are referring to pexpect I've found it works well - unix only
though.  I've not noticed it being half complete.

There is also a recipe for a non-blocking subprocess - see

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440554

> All well and good, but unsatisfying for writing platform independent code.
> 
>  it turns out that there is at least one open source multi-platform
>  (read: win32/linux) api that does handle synchronous I/O with the
>  child: wxWidgets and wxPython using the class wxProcess. Now the
>  wxWidgets implementation is far from perfect, but it at least allows a
>  program to test for new input on the child's stdout and read stdout/
>  write stdout in a non-blocking way.

Interesting I didn't know about that - I shall try it!

>  However, I find it frustrating
>  that I have to import wx just to have useable interactive pipes in my
>  python scripts when I would expect this to be part of the native
>  python implementation.
> 
>  Anybody have any thoughts on this? Do I have my story straight? (the
>  popen variants can't handle this case and there are no other
>  alternatives in the standard python distro) Is there some place I can
>  submit this as a feature request? (Python dev?)

The non-blocking subprocess would make a good start for a stdlib
submission.

It should really optionally use ptys under unix too otherwise you'll
never be able to script passwd etc.  An interface a bit like pexpect
wpuld be useful too (ie scan for these regexps or timeout and return a
match object).

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick