how to simulate tar filename substitution across piped subprocess.Popen() calls?

[Hans Mulder]

On 8/11/12 19:05:11, jkn wrote:
> Hi All
>     i am trying to build up a set of subprocess.Ponen calls to
> replicate the effect of a horribly long shell command. I'm not clear
> how I can do one part of this and wonder if anyone can advise. I'm on
> Linux, fairly obviously.
> 
> I have a command which (simplified) is a tar -c command piped through
> to xargs:
> 
> tar  -czvf myfile.tgz -c $MYDIR mysubdir/ | xargs -I '{}' sh -c "test -
> f $MYDIR/'{}'"
> 
> (The full command is more complicated than this; I got it from a shell
> guru).
> 
> IIUC, when called like this, the two occurences of '{}' in the xargs
> command will get replaced with the file being added to the tarfile.
> 
> Also IIUC, I will need two calls to subprocess.Popen() and use
> subprocess.stdin on the second to receive the output from the first.
> But how can I achive the substitution of the '{}' construction across
> these two calls?

That's what 'xargs' will do for you.  All you need to do, is invoke
xargs with arguments containing '{}'.  I.e., something like:

cmd1 = ['tar', '-czvf', 'myfile.tgz', '-c', mydir, 'mysubdir']
first_process = subprocess.Popen(cmd1, stdout=subprocess.PIPE)

cmd2 = ['xargs', '-I', '{}', 'sh', '-c', "test -f %s/'{}'" % mydir]
second_process = subprocess.Popen(cmd2, stdin=first_process.stdout)

> Apologies if I've made any howlers in this description - it's very
> likely...

I think the second '-c' argument to tar should have been a '-C'.

I'm not sure I understand what the second command is trying to
achieve.  On my system, nothing happens, because tar writes the
names of the files it is adding to stderr, so xargs receives no
input at all.  If I send the stderr from tar to the stdin of
xargs, then it still doesn't seem to do anything sensible.

Perhaps your real xargs command is more complicated and more
sensible.



Hope this helps,

-- HansM