how to simulate tar filename substitution across piped subprocess.Popen() calls?

[jkn]

Hi Hans
    thanks a lot for your reply:

> That's what 'xargs' will do for you.  All you need to do, is invoke
> xargs with arguments containing '{}'.  I.e., something like:
>
> cmd1 = ['tar', '-czvf', 'myfile.tgz', '-c', mydir, 'mysubdir']
> first_process = subprocess.Popen(cmd1, stdout=subprocess.PIPE)
>
> cmd2 = ['xargs', '-I', '{}', 'sh', '-c', "test -f %s/'{}'" % mydir]
> second_process = subprocess.Popen(cmd2, stdin=first_process.stdout)
>

Hmm - that's pretty much what I've been trying. I will have to
experiment a bit more and post the results in a bit more detail.

> > Apologies if I've made any howlers in this description - it's very
> > likely...
>

> I think the second '-c' argument to tar should have been a '-C'.

You are correct, thanks. Serves me right for typing the simplified
version in by hand. I actually use the equivalent "--directory=..." in
the actual code.

> I'm not sure I understand what the second command is trying to
> achieve.  On my system, nothing happens, because tar writes the
> names of the files it is adding to stderr, so xargs receives no
> input at all.  If I send the stderr from tar to the stdin of
> xargs, then it still doesn't seem to do anything sensible.

That's interesting ... on my system, and all others that I know about,
the file list goes to stdout.

> Perhaps your real xargs command is more complicated and more
> sensible.

Yes, in fact the output from xargs is piped to a third process. But I
realise this doesn't alter the result of your experiment; the xargs
process should filter a subset of the files being fed to it.

I will experiment a bit more and hopefully post some results. Thanks
in the meantime...

    Regards
    Jon N