subprocess "handle is invalid" error

[Thomas Heller]

Grant Edwards schrieb:
> I'm trying to use the py-gnuplot module on windows, and have
> been unable to get it to work reliably under Win2K and WinXP.
> 
> By default, it uses popen(gnuplotcmd,'w'), but in some
> situations that consistently gets an "invalid operand" IOError
> when write() is called on the pipe.
> 
> So I switched to subprocess.  It works fine when executed
> "normally" (e.g.  "python progname.py"), but when bundled by
> py2exe, it always does this:
> 
>   Traceback (most recent call last):
>     File "surfedit.py", line 28, in ?
>     File "Gnuplot\_Gnuplot.pyc", line 178, in __init__
>     File "Gnuplot\gp_win32.pyc", line 117, in __init__
>     File "subprocess.pyc", line 533, in __init__
>     File "subprocess.pyc", line 607, in _get_handles
>     File "subprocess.pyc", line 634, in _make_inheritable
>   WindowsError: [Errno 6] The handle is invalid
> 
> How does one troubleshoot errors that happen three layers deep
> in the subprocess module?
> 

I think this is a subprocess bug.  It is often attributed to py2exe because
usually developers do never run the script in pythonW.exe instead of python.exe,
and later build a *windows* program with py2exe (the *windows* program has no
console, and that triggers the bug).

Consider ths little script:

"""
import os, sys, subprocess

sys.stderr = open("errors.txt", "w")

if os.path.exists("output.txt"):
    os.remove("output.txt")

proc = subprocess.Popen("dir", shell=True,
                        stdout=subprocess.PIPE,
##                        stderr=subprocess.PIPE,
##                        stdin=subprocess.PIPE,
                        )
##proc.stderr.close()
##proc.stdin.close()
data = proc.stdout.read()

open("output.txt", "w").write(data)
"""

It calls 'dir' in the current directory, and writes the output
to the file 'output.txt'. Any errors are written to 'errors.txt'.

When you run this script with python.exe, everything works.
If the script is run with pythonW.exe, nothing works and 'errors.txt'
contains this:

c:\svn\theller>type errors.txt
Traceback (most recent call last):
  File "test_subproc.py", line 9, in <module>
    stdout=subprocess.PIPE,
  File "c:\python25\lib\subprocess.py", line 586, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "c:\python25\lib\subprocess.py", line 699, in _get_handles
    p2cread = self._make_inheritable(p2cread)
  File "c:\python25\lib\subprocess.py", line 744, in _make_inheritable
    DUPLICATE_SAME_ACCESS)
WindowsError: [Error 6] Das Handle ist ungültig
c:\svn\theller>

The error message, translated to english, is 'the handle is invalid'.

The script can be made to work correctly even with pythonW.exe (and also
as py2exe'd windows program, I just checked it out) when the 4 commented out
lines are uncommented.  subprocess cannot inherit the standard handles when
the process has no console, you have to create pipes for all 3 channels, and
close those that are not needed.

I thought that this bug was fixed in Python2.5.1 (the release candidate),
but it seems it wasn't.  The bug is at 
http://sourceforge.net/tracker/index.php?func=detail&aid=1124861&group_id=5470&atid=105470

If all this is correct, I hope that someone adds a section to the py2exe wiki;
and reopens the above bug report.

Thomas