os.popen and lengthy operations

[Tommy Nordgren]

On 20 sep 2007, at 08.31, Dmitry Teslenko wrote:

> Hello!
> I'm using os.popen to perform lengthy operation such as building some
> project from source.
> It looks like this:
> def execute_and_save_output( command, out_file, err_file):
>
> import os
>
> def execute_and_save_output( command, out_file, err_file):
> 	(i,o,e) = os.popen3( command )
> 	try:
> 		for line in o:
> 			out_file.write( line )
>
> 		for line in e:
> 			err_file.write( line )
> 	finally:
> 		i.close()
> 		o.close()
> 		e.close()
>
> ...
> execute_and_save_output( '<some long to run command>', out_file,  
> err_file)
>
> Problem is that script hangs on operations that take long to execute
> and have lots of output such as building scripts.
> -- 
	Your problem is that you are not reading the standard output and  
standard error streams in the correct way.
You need to do the reading of standard out and standard err in  
parallell rather than sequentially.
The called process can't proceed when it's output buffers are full.  
Pipes is limited by small buffers (for examples 512 bytes
on Mac OS X)
-------------------------------------
This sig is dedicated to the advancement of Nuclear Power
Tommy Nordgren
tommy.nordgren at comhem.se