polling for output from a subprocess module

[Thomas Bellman]

jakub.hrozek at gmail.com wrote:

>       try:
>             test = Popen(test_path,
>                                   stdout=PIPE,
>                                   stderr=PIPE,
>                                   close_fds=True,
>                                   env=test_environ)

>             while test.poll() == None:
>                 ready = select.select([test.stderr], [], [])

>                 if test.stderr in ready[0]:
>                     t_stderr_new = test.stderr.readlines()
>                     if t_stderr_new != []:
>                         print "STDERR:", "\n".join(t_stderr_new)
>                         t_stderr.extend(t_stderr_new)
[...]
> The problem is, that it seems that all the output from the subprocess
> seems to be coming at once. Do I need to take a different approach?

The readlines() method will read until it reaches end of file (or
an error occurs), not just what is available at the moment.  You
can see that for your self by running:

    $ python -c 'import sys; print sys.stdin.readlines()'

The call to sys.stdin.readlines() will not return until you press
Ctrl-D (or, I think, Ctrl-Z if you are using MS-Windows).

However, the os.read() function will only read what is currently
available.  Note, though, that os.read() does not do line-based
I/O, so depending on the timing you can get incomplete lines, or
multiple lines in one read.


-- 
Thomas Bellman,   Lysator Computer Club,   Linköping University,  Sweden
"Adde parvum parvo magnus acervus erit"       ! bellman @ lysator.liu.se
          (From The Mythical Man-Month)       ! Make Love -- Nicht Wahr!