Piping processes works with 'shell = True' but not otherwise.

[Luca Cerone]

Hi everybody, 
I am new to the group (and relatively new to Python)
so I am sorry if this issues has been discussed (although searching for topics in the group I couldn't find a solution to my problem).

I am using Python 2.7.3 to analyse the output of two 3rd parties programs that can be launched in a linux shell as:

 program1 | program2

To do this I have written a function that pipes program1 and program2 (using subprocess.Popen) and the stdout of the subprocess, and a function that parses the output:

A basic example:

from subprocess import Popen, STDOUT, PIPE
def run():
  p1 = Popen(['program1'], stdout = PIPE, stderr = STDOUT)
  p2 = Popen(['program2'], stdin = p1.stdout, stdout = PIPE, stderr = STDOUT)
  p1.stdout.close()
  return p2.stdout


def parse(out):
  for row in out:
    print row
    #do something else with each line
  out.close()
  return parsed_output
   

# main block here

pout = run()

parsed = parse(pout)

#--- END OF PROGRAM ----#

I want to parse the output of 'program1 | program2' line by line because the output is very large.

When running the code above, occasionally some error occurs (IOERROR: [Errno 0]). However this error doesn't occur if I code the run() function as:

def run():
  p = Popen('program1 | program2', shell = True, stderr = STDOUT, stdout = PIPE)
  return p.stdout

I really can't understand why the first version causes errors, while the second one doesn't.

Can you please help me understanding what's the difference between the two cases? 

Thanks a lot in advance for the help,
Cheers, Luca