subprocess call is not waiting.

[MRAB]

On 2012-09-13 16:34, Jean-Michel Pichavant wrote:
> ----- Original Message -----
>> I have a subprocess.call which tries to download a data from a remote
>> server using HTAR. I put the call in a while loop, which tests to
>> see if the download was successful, and if not, loops back around up
>> to five times, just in case my internet connection has a hiccup.
>>
>> Subprocess.call is supposed to wait.
>>
>> But it doesn't work as intended. The loop quickly runs 5 times,
>> starting a new htar command each time. After five times around, my
>> program tells me my download failed, because the target file doesn't
>> yet exist. But it turns out that the download is still
>> happening---five times.
>>
>> When I run htar from the shell, I don't get a shell prompt again
>> until after the download is complete. How come control is returned
>> to python before the htar command is through?
>>
>> I've tried using Popen with wait and/or communicate, but no waiting
>> ever happens. This is troublesome not only because I don't get to
>> post process my data, but because when I run this script for
>> multiple datasets (checking to see whether I have local copies), I
>> quickly get a "Too many open files" error. (I began working on that
>> by trying to use Popopen with fds_close, etc.)
>>
>> Should I just go back to os.system?
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>
> A related subset of code would be useful.
>
> You can use subprocess.PIPE to redirect stdout & stderr et get them with communicate, something like:
>
> proc = subprocess.Popen(['whatever'], stdout=subprocess.PIPE, stdout=subprocess.PIPE)
> stdout, stderr = proc.communicate()
> print stdout
> print stderr
>
> Just by looking at stdout and stderr, you should be able to see why htar is returning so fast.
>
> JM
>
> PS : if you see nothing wrong, is it possible that htar runs asynchronously ?
>
The OP says that it waits when run from the shell.