[Python-Dev] Strange behavior of subprocess.Popen._get_handles under Windows

[Alexey Borzenkov]

For a long time I was surprised why if I have a script testme.py:

    import subprocess
    subprocess.call("echo Something", shell=True)

and I try to execute it like this:

    python testme.py >testme.txt

I get the output:

    The handle is invalid.

Strange failures randomly happened with different programs, so I
thought maybe this was intended (mis)behavior, and I found that I can
workaround this by explicitly specifying std* handles:

    subprocess.call(...., stdin=sys.stdin, stdout=sys.stdout, stderr=sys.stderr)

I lived with this until I understood that if something substitutes
sys.std* with a proxy (examples: running under IDLE or wxPython, or
when there is a custom logger on stdout/stderr, etc), this will no
longer work:

      File "C:\Programs\Python25\Lib\subprocess.py", line 698, in _get_handles
        p2cread = msvcrt.get_osfhandle(stdin.fileno())
    AttributeError: fileno

Now I finally found that my problem are these two lines in subprocess.py:

                if stdin is None and stdout is None and stderr is None:
                    return (None, None, None, None, None, None)

These lines add an interesting quirk: if I explicitly specify any
single channel (i.e. stdout=1) the problem suddenly disappears, and if
I just comment these lines altogether my problem vanishes completely!
(i.e. output redirection works absolutely well)

Further investigations showed that it seems to be some strange OS
quirk/bug, i.e. this behavior is caused when STARTF_USESTDHANDLES is
not used, and crt's spawnl* behaves incorrectly too:

    os.spawnl(r"C:\WINDOWS\system32\cmd.exe",
r"C:\WINDOWS\system32\cmd.exe /c echo Something")

So the question is should these two lines be removed to workaround this OS bug?

To my thinking, this will not change behavior when any std* arguments
are passed to Popen, and the only case when it kicks in is when no
std* arguments are specified and the resulting side effects are
previously failing cases start working correctly. Or am I wrong here,
are there any side effects of using STARTF_USESTDHANDLES that I'm
missing here?

Best regards,
Alexey.