subprocess call is not waiting.

[Jean-Michel Pichavant]

----- Original Message -----
> I have a subprocess.call which tries to download a data from a remote
> server using HTAR. I put the call in a while loop, which tests to
> see if the download was successful, and if not, loops back around up
> to five times, just in case my internet connection has a hiccup.
> 
> Subprocess.call is supposed to wait.
> 
> But it doesn't work as intended. The loop quickly runs 5 times,
> starting a new htar command each time. After five times around, my
> program tells me my download failed, because the target file doesn't
> yet exist. But it turns out that the download is still
> happening---five times.
> 
> When I run htar from the shell, I don't get a shell prompt again
> until after the download is complete. How come control is returned
> to python before the htar command is through?
> 
> I've tried using Popen with wait and/or communicate, but no waiting
> ever happens. This is troublesome not only because I don't get to
> post process my data, but because when I run this script for
> multiple datasets (checking to see whether I have local copies), I
> quickly get a "Too many open files" error. (I began working on that
> by trying to use Popopen with fds_close, etc.)
> 
> Should I just go back to os.system?
> --
> http://mail.python.org/mailman/listinfo/python-list
> 

A related subset of code would be useful. 

You can use subprocess.PIPE to redirect stdout & stderr et get them with communicate, something like:

proc = subprocess.Popen(['whatever'], stdout=subprocess.PIPE, stdout=subprocess.PIPE)
stdout, stderr = proc.communicate()
print stdout
print stderr

Just by looking at stdout and stderr, you should be able to see why htar is returning so fast.

JM

PS : if you see nothing wrong, is it possible that htar runs asynchronously ?