subprocess woes

[kj]

In <d87065db-f51f-4afe-924c-f9e4a1eb0c70 at g23g2000vbr.googlegroups.com> Mike Driscoll <kyosohma at gmail.com> writes:

>On Sep 15, 2:26=A0pm, kj <no.em... at please.post> wrote:
>> I'm trying to write a function, sort_data, that takes as argument
>> the path to a file, and sorts it in place, leaving the last "sentinel"
>> line in its original position (i.e. at the end). =A0Here's what I
>> have (omitting most error-checking code):
>>
>> def sort_data(path, sentinel=3D'.\n'):
>> =A0 =A0 tmp_fd, tmp =3D tempfile.mkstemp()
>> =A0 =A0 out =3D os.fdopen(tmp_fd, 'wb')
>> =A0 =A0 cmd =3D ['/usr/local/bin/sort', '-t', '\t', '-k1,1', '-k2,2']
>> =A0 =A0 p =3D Popen(cmd, stdin=3DPIPE, stdout=3Dout)
>> =A0 =A0 in_ =3D file(path, 'r')
>> =A0 =A0 while True:
>> =A0 =A0 =A0 =A0 line =3D in_.next()
>> =A0 =A0 =A0 =A0 if line !=3D sentinel:
>> =A0 =A0 =A0 =A0 =A0 =A0 p.stdin.write(line)
>> =A0 =A0 =A0 =A0 else:
>> =A0 =A0 =A0 =A0 =A0 =A0 break
>> =A0 =A0 in_.close()
>> =A0 =A0 p.stdin.close()
>> =A0 =A0 retcode =3D p.wait()
>> =A0 =A0 if retcode !=3D 0:
>> =A0 =A0 =A0 =A0 raise CalledProcessError(retcode, cmd)
>> =A0 =A0 out.write(sentinel)
>> =A0 =A0 out.close()
>> =A0 =A0 shutil.move(tmp, path)
>>
>> This works OK, except that it does not catch the stderr from the
>> called sort process. =A0The problem is how to do this. =A0I want to to
>> avoid having to create a new file just to capture this stderr
>> output. =A0I would like instead to capture it to an in-memory buffer.
>> Therefore I tried using a StringIO object as the stderr parameter
>> to Popen, but this resulted in the error "StringIO instance has no
>> attribute 'fileno'".
>>
>> How can I capture stderr in the scenario depicted above?
>>
>> TIA!
>>
>> kynn

>According to the docs for subprocess module (which you don't appear to
>be using even though that's what you used for your subject line),

Sorry, I should have been clearer.  I *am* using subprocess; that's
were Popen and PIPE come from.  I omitted the import lines along
with much else to keep the code concise.  Maybe I overdid it.

>you can set stderr to stdout:

This won't do: I'm already using stdout to collect the output of
sort.

>You can use cStringIO to create a "file-like" object in memory:

Nope.  I get the same error I get when I try this idea using
StringIO (i.e. "no attribute 'fileno'").

kj