[Baypiggies] Subprocess: Common problem/common pattern
Glen Jarvis
glen at glenjarvis.com
Sat Oct 2 18:44:05 CEST 2010
I've now seen a common problem come up several times. And, I imagine there
is a common solution to this problem that I don't know.
About a year ago, I wrote an automatic script that would automatically do an
'svn export' of certain branches of a tree depending upon what menu option
the customer chose. When I did the svn export, I used subprocess.Popen.
The pattern was similar to the following:
print """This output is being buffered so I can read the version number.
.... I'm not stuck, just busy exporting files....
"""
.....
process = subprocess.Popen(['svn', 'export', repository,
repo_name], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdoutdata, stderrdata = process.communicate()
I printed "please wait" and then printed the data given when the process was
done (stdoutdata). It wasn't ideal but, it was sufficient for the time. If I
were to have gone for the best fix, I would probably have learned the API
for subversion to integrate directly into python.
However, another BayPIGgie is having the same issue. He is using a command
to start a CD burner from the command line and wants to print the output as
it is being created from the command line.
I can see that to solve this we wouldn't use the communicate() convenience
function. A few 'hackish' ways that may solve this, but I'm looking for the
common pattern that is used when other pythonista run up against this
problem. I also want to ensure that I don't have a 'hack' that causes a
deadlock only to discover this much later after I've implemented the pattern
a few times.
To help keep the conversation more focused, I've created two tiny test
programs for a test case:
1) A C command line program that we have no control over changing within
python, and
2) A Python program that calls that the c-program (baypiggies_demo):
Compile command line so output is same as expected by Python program:
gcc -pedantic baypiggies_demo.c -o baypiggies_demo
---- start of c program: File: baypiggies_demo.c ---
#include <stdio.h>
#include <unistd.h>
int
main()
{
int i = 0;
printf("Silly output for monitoring...\n");
for (i = 0; i <= 10; i++) {
printf("Counting... %d\n", i);
sleep(1);
}
}
--- end of c program ---
--- start of python program to demonstrate. File baypiggies.py ---
import subprocess
print "Just waiting...."
process = subprocess.Popen(['./baypiggies_demo'],
stdout=subprocess.PIPE)
stdoutdata, stderrdata = process.communicate()
print "Well, NOW I get it.. :( "
print stdoutdata
--- end baypiggies.py --
Has anyone else ran into this before? What's the classic pattern used?
Thanks in advance,
Glen
--
Whatever you can do or imagine, begin it;
boldness has beauty, magic, and power in it.
-- Goethe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/baypiggies/attachments/20101002/30c711f7/attachment.html>
More information about the Baypiggies
mailing list