Subprocess does not return for longer-running process

[Nobody]

On Fri, 24 Sep 2010 15:18:47 +1200, Lawrence D'Oliveiro wrote:

>>>> And I can't think of any reason why you should use os.waitpid() or
>>>> similar; use the .wait() method.
>>> 
>>> I have used WNOHANG to poll for completion of a subprocess while
>>> providing progress updates to the user.
>> 
>> This can be done via the .poll() method.
> 
> And what do you think the poll method uses?

1. Not relevant; use the defined interface.

2. It currently uses waitpid() on Unix, WaitForSingleObject() on Windows,
maybe other functions if subprocess gets ported to other platforms or
extended. If you use the .poll() method, you don't need to worry about any
of this.

More importantly the .poll() method sets the .returncode attribute. Why is
this important? Because if the .returncode attribute is None,
Popen.__del__ will add the object to the _active list. Not only does this
prevent the object from being finalised, it will continue to poll for
termination via _cleanup().

But if you have manually reaped the child, its PID becomes available for
re-use. This can result in a subsequent _cleanup() reaping some other
child process. In the 2.6 implementation, manually setting the .returncode
attribute will solve this. In future implementations ... who knows?

Again: use the .wait() or .poll() methods. These are portable, can be
assumed to do the right thing for subclasses of subprocess.Popen(), and
will perform any necessary maintenance of "internal" state.

If those methods don't suffice, you can't reliably use subprocess. E.g. if
you need to wait until any child terminates, using os.wait() will cause
the child to be reaped, which will confuse Popen().

POSIX (and Linux since 2.6.9) has waitid(), which allows you to wait for a
child *without* reaping it (via the WNOWAIT flag), but Python doesn't
include an interface for this.