[issue37424] subprocess.run timeout does not function if shell=True and capture_output=True

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

On Windows, the following pattern _can_ hang:

proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
try:
  return proc.communicate(timeout=1.0)
except TimeoutExpired:
  proc.kill()
  return proc.communicate() # <== HERE

Even if the first child process is killed, communicate() waits until the pipe is closed. If the child process spawned a 3rd process before being killed, the second .communicate() calls hangs until the 3rd process exit or close its stdout.

I'm not sure if subprocess.run() should do anything about this case, but it was at least for your information.

I'm fighting against this issue in bpo-37531.

IMHO it's an issue of the Windows implementation of Popen.communicate(): it's implemented with a blocking call to stdout.read() run in a thread. The thread cannot be interrupted in any way and will until complete once stdout is closed.

Again, if the direct child process spawned other processes, stdout is only closed in the main parent process once all child processes exit or at least closed their stdout.

Maybe another issue should be opened to avoid blocking with the following pattern on Windows:

proc.kill()
proc.communicate()

----------
nosy: +vstinner

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37424>
_______________________________________