os.system() or os.execv() w/stdout redirect and PID tracking

[Michael P. Reilly]

Michael Olivier <dmtech at iname.com> wrote:
: Hi folks,

: What's the best way from a python program to start another program in
: the background and:
: - redirect stdout/stderr to a file
: - get the process ID of the started program.

If you put it in the background, then I guess you don't want the PID
(backgrounding the process means that you are ignoring when it completes).
I will assume that you want to be notified when it completes and continue
processing.

: It seems that:

: - os.system() call can start a program in the background, but doesn't
: return the pid (process ID) of the started program -- returns 0

:     os.system("find / -name mic* -print > /tmp/find-cmd 2>&1 &")

:     This does what I want but don't get the PID back for it

This calls "/bin/sh -c 'find / -name mic* -print > /tmp/find-cmd 2>&1 &'"
Which spawns two processes, the `find' and the '/bin/sh', of which only
the '/bin/sh' would be retrievable.  Besides which, you are placing the
find in the background.

: - if I call os.fork() and then os.execv(), the pid is trackable,
: but I don't know a way to redirect stdout/stderr

You must replace fd1 and fd2 (stdout and stderr respectively) with the
output file using the POSIX functions in the os module.


For much of what you seem to want, you could use the Popen3 class in
the popen2 module.

  import popen2
  # this has the same semantics as os.system(), which calls /bin/sh and
  # attempts to evaluate the wildcard
  finder = popen2.Popen3("find / -name 'mic*' -print >/tmp/find-cmd 2>&1")
  pid = finder.pid
  while finder.poll() < -1:
    do_something_else_for_a_short_time()
  return_code = finder.sts  # status value from wait

or
  # it might be good to put the read/write loop in a thread
  import popen2
  finder = popen2.Popen3("find / -name 'mic*' -print")
  file = open('/tmp/find-cmd', 'w')
  # if you didn't want the pid, you could use os.popen()
  pid = finder.pid
  line = finger.fromchild.readline()
  while line:
    file.write(line)
    line = finder.fromchild.readline()
  file.close()
  return_code = finder.wait()

Or the old-fashioned method:
  import os, signal

  pid = os.fork()
  return_code = None
  if pid == 0:
    outfile = os.open('/tmp/find-cmd', os.WRONLY)
    # redirect stdout and stderr to '/tmp/find-cmd'
    os.close(1)
    os.dup(outfile)
    os.close(2)
    os.dup(outfile)
    os.close(outfile)
    # note that this does NOT evaluate the wildcard
    os.execlp('find', 'find', '/', '-name', 'mic*' -print')
    os._exit(255)
  def process_completion(signum, frame, pid=pid):
    global return_code
    wpid, rc = os.waitpid(pid, os.WNOHANG)
    if wpid == pid:
      return_code = rc
  signal.signal(signal.SIGCHLD, process_completion)
  do_something_else_for_a_short_time()

Using the raw POSIX calls is the most flexible, but not the easiest or
most portable; the functions in popen2 are generally better to use.
But then it all depends on what you want to do.

  -Arcege