Help replacing os.system call with subprocess call

[David Pratt]

Hi Matt. My apologies, I was away a good part of the day and evening 
today. Here are the numbers for the different methods:

original os.system call:  29.685759 s
buffered stdout call:    213.370982 s  (with 1mb buffer)
direct to stdout call:    33.378756 s

The second method worked great and is essentially equivalent in 
execution time to original call :-). This has got me smiling. Many 
thanks Matt for your help, particularly working through the second 
example that provided equivalent speed.

Regards,
David

David Pratt wrote:
> Hi Matt. Many thanks. Sorry I had not seen your second post. I'll give 
> this a try and time the completion to compare the differences and post 
> back later today to show os.system, buffered imput and using a file 
> directly for stdout.
> 
> Regards,
> David
> 
> Matt Nordhoff wrote:
>> David Pratt wrote:
>>> Hi David and Matt. I appreciate your help which has got me moving
>>> forward again so many thanks for your reply. I have been using
>>> subprocess.Popen a fair bit but this was the first time I had to use
>>> subprocess to capture large file output. The trouble I was having was
>>> with the process would just hang. Chunking was the solution. I guess I
>>> assumed this would be taken care of in the internals.
>>>
>>> Overall, I wish subprocess had some better documentation since it is
>>> definitely not a drop in replacement for os.system. In other
>>> circumstances I am using subprocess.call() for simple calls which works
>>> fine.
>>>
>>> The speed of this solution is slower than os.system. Would a queue of
>>> some kind be needed to speed this up? Has anyone implemented something
>>> like this? Many thanks.
>>>
>>> Regards,
>>> David
>> Did you see my second message? That should help performance. If not, I'm
>> totally out of my depth and have no idea at all. Sorry.
>>
>> (How much slower? 10%? 200%?)