creating size-limited tar files

[Ian Kelly]

On Tue, Nov 13, 2012 at 3:31 AM, andrea crotti
<andrea.crotti.0 at gmail.com> wrote:
> but it's a bit ugly.  I wonder if I can use the subprocess PIPEs to do
> the same thing, is it going to be as fast and work in the same way??

It'll look something like this:

>>> p1 = subprocess.Popen(cmd1, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> p2 = subprocess.Popen(cmd2, shell=True, stdin=p1.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> p1.communicate()
('', '')
>>> p2.communicate()
('', '')
>>> p1.wait()
0
>>> p2.wait()
0

Note that there's a subtle potential for deadlock here.  During the
p1.communicate() call, if the p2 output buffer fills up, then it will
stop accepting input from p1 until p2.communicate() can be called, and
then if that buffer also fills up, p1 will hang.  Additionally, if p2
needs to wait on the parent process for some reason, then you end up
effectively serializing the two processes.

Solution would be to poll all the open-ended pipes in a select() loop
instead of using communicate(), or perhaps make the two communicate
calls simultaneously in separate threads.