Need feedback on subprocess-using function

[ryles]

On Oct 4, 9:46 pm, Nobody <nob... at nowhere.com> wrote:
> On Sat, 03 Oct 2009 13:21:00 +0000, gb345 wrote:
> > I'm relatively new to Python, and I'm trying to get the hang of
> > using Python's subprocess module.  As an exercise, I wrote the Tac
> > class below, which can prints output to a file "in reverse order",
> > by piping it through the Unix tac utility.  (The idea is to delegate
> > the problem of managing the memory for an arbitrarily large task
> > to tac.)
> >         self.pipe = subprocess.Popen(['tac'], stdout=out,
> >                                      stdin=subprocess.PIPE,
> >                                      stderr=subprocess.PIPE)
> > This works OK, as far as I can tell, but I'm not sure that I've
> > dotted all the i's and crossed all the t's...  E.g., I had to add
> > the line "p.stdin.close()" to the close method when I when I ran
> > into sporadic deadlock at the p.stderr.read() statement.  Are there
> > other similar problems lurking in this code?
>
> Yep. If the process writes more than a buffer-full of data to stderr, it
> will deadlock. tac will block trying to write to stderr, and won't be
> reading its stdin, so your program will block trying to write to tac.
>
> This is why the POSIX popen() call only lets you attach a pipe to stdin or
> stdout but not both.
>
> If you want a "double-ended" slave process, you need to use polling or
> non-blocking I/O or asynchronous I/O or multiple threads. I'm not aware of
> any single solution which works on all platforms.
>
> The easiest way around this problem is to redirect stderr to a temporary
> file and read in the file's contents in the close() method.

There is also Popen.communicate():

http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate