Popen question (redundant processes)

[Gabriel Genellina]

En Sun, 30 Aug 2009 17:25:40 -0300, Chris Rebert <clp2 at rebertia.com>  
escribió:

> On Sun, Aug 30, 2009 at 12:33 PM, Sebastian<sebas0 at gmail.com> wrote:
>> Hello World!
>> This is my first post on the list and I'm hoping it is the right forum  
>> and
>> not OT, I've searched
>> a bit on this, but, none-the-wiser!
>>
>> My question is on the Popen method, here is my snippet:
>>
>>> p1 = Popen(["cat", "georgi_ddr7_allmag_kcor_in_test.dat"], stdout=PIPE  
>>> )
>>> p2 = Popen(["fit_coeffs"], stdin=p1.stdout, stdout=PIPE)
>>> p3 = Popen(["reconstruct_maggies"], stdin=p2.stdout,stdout=PIPE)
>>> output_maggies_z=p3.communicate()[0]
>>>
>>> p1 = Popen(["cat", "georgi_ddr7_allmag_kcor_in_test.dat"], stdout=PIPE  
>>> )
>>> p2 = Popen(["fit_coeffs"], stdin=p1.stdout, stdout=PIPE)
>>> p4 = Popen(["reconstruct_maggies", "--band-shift", "0.1", "--redshift",
>>> "0."], stdin=p2.stdout,stdout=PIPE)
>>> output_maggies_z0=p4.communicate()[0]
>>>
>>
>> That is, p1 and p2 are the same, but p3 and p4 which they are passed  
>> to, are
>> different.
>> Is there a way to pass p1 and p2 to p3 AND p4 simultaneously, so as to  
>> not
>> need to
>> run p1 and p2 twice, as above?
>> What arguments would I need to achieve this?
>>
>> NOTE: "georgi_ddr7_allmag_kcor_in_test.dat" is a very large file (~1E6
>> records)
>
> Send the output of p2 through the unix command "tee"
> (http://unixhelp.ed.ac.uk/CGI/man-cgi?tee). Then put the output of tee
> into p3 and set p4's input to the file you specified to tee.

In addition to that, you can avoid the cat command (and omit p1  
completely), just pass the georgi file as p2 stdin:
p2 = Popen(["fit_coeffs"], stdin=open("georgi..."), ...

In other words, the original shell commands are:

cat georgi.dat | fit | reconstruct
cat georgi.dat | fit | reconstruct --otherargs

and we suggest doing:

fit < georgi.dat | tee /tmp/foo | reconstruct
reconstruct  --otherargs < /tmp/foo

-- 
Gabriel Genellina