question about subprocess and shells

[Nobody]

On Fri, 04 Dec 2009 13:38:11 -0800, Ross Boylan wrote:

> If one uses subprocess.Popen(args, ..., shell=True, ...)
> 
> When args finishes execution, does the shell terminate?  Either way
> seems problematic.

That depends upon what "args" is. On Unix, if args ends with a "&", the
shell will terminate as soon as it has started the command. Typically, the
shell will terminate once it has executed the last command, where
"executed" means fork()+exec() for a background command and
fork()+exec()+wait() for a non-background command.

> If it does not terminate, then it seems as if calls like wait and
> communicate would never return.  It also seems the subprocess would
> never die, and that most of the examples with the shell True would leave
> processes lying around.

That would be the case if the shell didn't terminate.

> If it does terminate, then how can you stuff new commands down the pipe
> to the subprocesses stdin?

You don't.

shell=True executes ["/bin/sh", "-c", args], not an interactive shell
reading commands from stdin (unless you explicitly run an interactive
shell with e.g. Popen(args = "sh", ...)).

> Does this module contemplate receiving multiple commands for the shell
> to execute?

No. It behaves like an extended version of the Unix popen() function, i.e.
like system() but you get to provide pipes for the standard handles.

> I'm also unsure of the semantics of the pipes for the processes standard
> file handles.  Do they need to be closed (judging from the examples,
> no)?

If args is a command which reads from stdin, then you will need to close
stdin, otherwise it will never terminate. Otherwise, there's no need.

> When reads/writes to them return, and what state is the stream in
> at the time?

Any streams created when std{in,out,err}= are specified as PIPE or as a
descriptor number are unbuffered by default, but this can be changed by
the bufsize parameter to the Popen() constructor.

For an unbuffered stream, read() and write() will return when the
underlying system calls return. write() to stdin will return once the data
has been written to the pipe (not necessarily when the child has consumed
it), while read() from stdout/stderr will return once the requested
amount of data has been read from the pipe (or EOF is reached).