Simple process IO capture (Was: "shell-commands" and python!) - process.py (0/1)

[Greg Weeks]

Paul Moore (gustav at morpheus.demon.co.uk) wrote:

: What do people think? Is this a useful idea?

I think it would be great if this functionality was available, slick, and
standard.  But that might be pretty difficult:

1.  The "re" module allows access to a *language* that describes regular
expressions.  This language is then available within Python.  In general,
this "language within a language" approach is kind of icky.  But, as "re"
illustrates, it is sometimes the right solution.  Could that be the case
here?  And if so, what language?  In any case, on Unix systems, os.system()
*currently* implements this approach with the POSIX shell as the language.
Is this sufficiently nonideal to merit the effort to do better?

2.  I'm not a nut for efficiency, but I have been nagged *conceptually* by
the inefficiency of building standard I/O redirection on top of popen2.
For example, a traditional Unix pipe command (eg, "foo | bar") requires one
pipe.  Implementing this with popen2 requires two pipes which then need to
be "welded" together somehow.  (By the way, you have the same problem in
Java and Perl.)  I normally wouldn't niggle about such a thing; but it
might be worth thinking about if we're really shooting for a standard
module.

3.  Another minor problem with using popen2 to implement redirection:
Suppose the process you invoke starts up a daemon process.  Then a read()
from child_stdout can hang, never reaching end-of-file.  [I think.]

So, I just don't know.  For now, though, I'm living with what is
essentially os.system() with the following additions:

  - The shell is called with the -e option.
  - ksh is called instead of sh (alas!) because "sh -e -c ..." doesn't work
    on my system.  (-e is ignored.)
  - A nonzero exit code causes a "ShellError" to be thrown.  The stderr
    text is stored in the ShellError.  Indeed, this is the only way to see
    the stderr text (which, I admit, is dicey, given how some programs use
    stderr).
  - There is of course the option of capturing stdout to a string.

Greg