polling for output from a subprocess module

[Ivo]

Thomas Bellman wrote:
> jakub.hrozek at gmail.com wrote:
> 
>>       try:
>>             test = Popen(test_path,
>>                                   stdout=PIPE,
>>                                   stderr=PIPE,
>>                                   close_fds=True,
>>                                   env=test_environ)
> 
>>             while test.poll() == None:
>>                 ready = select.select([test.stderr], [], [])
> 
>>                 if test.stderr in ready[0]:
>>                     t_stderr_new = test.stderr.readlines()
>>                     if t_stderr_new != []:
>>                         print "STDERR:", "\n".join(t_stderr_new)
>>                         t_stderr.extend(t_stderr_new)
> [...]
>> The problem is, that it seems that all the output from the subprocess
>> seems to be coming at once. Do I need to take a different approach?
> 
> The readlines() method will read until it reaches end of file (or
> an error occurs), not just what is available at the moment.  You
> can see that for your self by running:
> 
>     $ python -c 'import sys; print sys.stdin.readlines()'
> 
> The call to sys.stdin.readlines() will not return until you press
> Ctrl-D (or, I think, Ctrl-Z if you are using MS-Windows).
> 
> However, the os.read() function will only read what is currently
> available.  Note, though, that os.read() does not do line-based
> I/O, so depending on the timing you can get incomplete lines, or
> multiple lines in one read.
> 
> 
be carefull that you specify how much you want to read at a time, 
otherwise it cat be that you keep on reading.

Specify read(1024) or somesuch.

In case of my PPCEncoder I recompiled the mencoder subprocess to deliver 
me lines that end with \n.

If anyone can tell me how to read a continues stream than I am really 
interested.

cya