[python-win32] Inheriting a command line - or how to efficiently replace Unix' os.execvpe

[Eryk Sun]

On 5/12/21, Sebastian M. Ernst <ernst at pleiszenburg.de> wrote:
>
> It starts the new process more or less like a child, but does not allow it to
> control the command line. Instead, both the Python process and the child
> die. The child merely manages to produce a few lines of output before that.

First let's go over some basic information about consoles in Windows.
Each console session [1] is implemented by a host process, which
currently is either conhost.exe or openconsole.exe (used by Windows
Terminal). Each Windows process can attach to one and only one console
session at a time. A console session doesn't always have a window
(i.e. the CREATE_NO_WINDOW process creation flag). Windows 10 also
allows the console host to operate in pseudoconsole mode, which relays
I/O to another application that implements the console/terminal user
interface, or no UI at all. For example, each tab in Windows Terminal
(wt.exe) is a pseudoconsole session that's hosted by an instance of
openconsole.exe.

A console session doesn't implement the POSIX concept of foreground
and background process groups. Any process that's attached to a
console can read and write from it. Generally when one starts a child
process in the same console session, one either avoids using console
I/O until the child exits, or one simply blocks and does nothing while
waiting for the child to exit. Typically a command-line shell (e.g.
cmd.exe) implements the latter.

> `os.execvpe` is present in the Python standard library on Windows, but
> it does not replace the original (Python) process. Windows is apparently
> lacking relevant POSIX semantics.

os.execve() is implemented internally as the equivalent of
os.spawnve() with P_OVERLAY. I suggest you pretend that it does not
exist. Windows does not support overlaying a new executable image on
the current process, which is what callers reasonably expect from an
exec() call. Instead, with P_OVERLAY the parent process spawns the
child in a new process and then exits. If a grandparent process is
waiting on the parent process, it will resume when the parent exits.
The grandparent and its grandchild will then compete with each other
for console I/O, which is an interleaved mess. You'll get a prompt
from the grandparent, but the text you enter will be read by the
grandchild. Then you'll get the grandchild's prompt, and so on.

> `ssh` opens into a second, new `cmd.exe` window and can be interacted
> with. The original `cmd.exe` window with Python in it remains open,
> Python itself quits, returning control to `cmd.exe` itself.

You're confusing things. cmd.exe is just a console client application.
It has nothing to do with implementing a console session. If `ssh` is
"ssh.bat", then it will execute via cmd.exe. But if it's an executable
binary, either it will allocate a new console session if it doesn't
inherit one (i.e. either the parent has no console, or the process was
created with the CREATE_NEW_CONSOLE or CREATE_NO_WINDOW creation flag)
or it will execute without a console if it's a detached process (i.e.
the process was created with the DETACHED_PROCESS creation flag). You
can pass these creation flags via the `creationflags` parameter of
subprocess.Popen(). CREATE_NO_WINDOW is ignored if paired with either
CREATE_NEW_CONSOLE or DETACHED_PROCESS, and the latter two obviously
cannot be paired with each other.

If you want to reuse the current console session, you'll have to spawn
ssh, wait for it to exit, and proxy its exit status. That's the
closest you can get to using exec*() in a POSIX system.

---

[1] There's an unrelated concept of the active "Console" session in
Windows terminal services. A Windows session contains a "WinSta0"
interactive window station, which has input devices and a display and
contains desktops such as the "Default" desktop, which contain
windows. The "Console" session is the one that's connected to the
physical console, i.e. the physical display, keyboard, and mouse. A
Windows session also has the concept of a graphical shell, which is
typically Windows Explorer. Generally speaking, however, the term
"console" in Windows refers to a sub-session (i.e. subsystem) for
text-based applications, which can exist in any Windows session, even
in the non-interactive "Services" session.