Python open a named pipe == hanging?

Rochester rochester1976 at gmail.com
Sun Aug 6 23:09:21 EDT 2006


Thanks Alex, now I think I understand much better the fifo/pipe mechanism  
and how Python treats them.

For those who are interested, I would like to restate the problem I was  
tring to solve and a working solution (inspired by Alex Martelli's code),  
feel free to criticize it:

The problem:

I have an external program which takes two matrices (two text files) as  
input, and produces an output to stdout, you can take diff as an example.   
I want to call this program from within Python, without writing any  
temporary file (for the performance reason).

In Bash, this task would be best written as:

#!/bin/bash

diff <(step1) <(step2) | step3

Where step1, step2 and step3 have to be independent external programs.

Now in Python, since there is no exact equivalence of <() magic a.k.a.  
process substitution, I figured taht the best solution should be to create  
a pair of fifos and do something like this:

#!/bin/bash

mkfifo fifo1 fifo2
step1 > fifo1 &
step2 > fifo2 &
diff step1 step2 | step3

And a working Python equivalent code is:

#!/usr/bin/python

import os

# do step1 and step2 in Python, say we end up with something like these:

s1 = "some string\n second line."      # results from step1
s2 = "some string\n a different line." # results from step2

os.mkfifo('fifo1')
os.mkfifo('fifo2')

op = os.popen(' '.join(['diff', 'fifo1', 'fifo2'])) # this step is crucial!
print >> open('fifo1', 'w'), s1
print >> open('fifo2', 'w'), s2

os.unlink('fifo1')
os.unlink('fifo2')

x = op.read()

# Now do something about x

print x

P.S.: process substitution is a Bash hack which uses /dev/fd/<n> to send  
(usually more than one) processes output to a program which takes (more  
than one) filename as input.  Heuristically speaking, this is like a  
multiway pipe.  You can find more about this mechanism here:

http://www.tldp.org/LDP/abs/html/process-sub.html



More information about the Python-list mailing list