[Python-Dev] A question about the subprocess implementation

[Vinay Sajip]

The subprocess.Popen constructor takes stdin, stdout and stderr keyword
arguments which are supposed to represent the file handles of the child process.
The object also has stdin, stdout and stderr attributes, which one would naively
expect to correspond to the passed in values, except where you pass in e.g.
subprocess.PIPE (in which case the corresponding attribute would be set to an
actual stream or descriptor).

However, in common cases, even when keyword arguments are passed in, the
corresponding attributes are set to None. The following script

import os
from subprocess import Popen, PIPE
import tempfile

cmd = 'ls /tmp'.split()

p = Popen(cmd, stdout=open(os.devnull, 'w+b'))
print('process output streams: %s, %s' % (p.stdout, p.stderr))
p = Popen(cmd, stdout=tempfile.TemporaryFile())
print('process output streams: %s, %s' % (p.stdout, p.stderr))

prints

process output streams: None, None
process output streams: None, None

under both Python 2.7 and 3.2. However, if subprocess.PIPE is passed in, then
the corresponding attribute *is* set: if the last four lines are changed to

p = Popen(cmd, stdout=PIPE)
print('process output streams: %s, %s' % (p.stdout, p.stderr))
p = Popen(cmd, stdout=open(os.devnull, 'w+b'), stderr=PIPE)
print('process output streams: %s, %s' % (p.stdout, p.stderr))

then you get

process output streams: <open file '<fdopen>', mode 'rb' at 0x2088660>, None
process output streams: None, <open file '<fdopen>', mode 'rb' at 0x2088e40>

under Python 2.7, and

process output streams: <_io.FileIO name=3 mode='rb'>, None
process output streams: None, <_io.FileIO name=5 mode='rb'>

This seems to me to contradict the principle of least surprise. One would
expect, when an file-like object is passed in as a keyword argument, that it be
placed in the corresponding attribute. That way, if one wants to do
p.stdout.close() (which is necessary in some cases), one doesn't hit an
AttributeError because NoneType has no attribute 'close'.

This seems like it might be a bug, but if so it does seem rather egregious: can
someone tell me if there is a good design reason for the current behaviour? If
there isn't one, I'll raise an issue.

Regards,

Vinay Sajip