Bug/Weak Implementation? popen* routines can't handle simultaneous read/write?

[Noah]

On Jun 7, 9:01 am, dmoore <damienlmo... at gmail.com> wrote:

popen and friends will never do what you want it to do. Down that path
lies bitter disappointment.
You need pseduo-ttys and non-blocking IO. I don't know how to do this
on Windows, but I know it's possible.
Cygwin does it.

> Anybody have any thoughts on this? Do I have my story straight? (the
> popen variants can't handle this case and there are no other
> alternatives in the standard python distro) Is there some place I can
> submit this as a feature request? (Python dev?)

Try Pexpect http://pexpect.sourceforge.net/
It's been around for a long time and is quite complete and stable.

The catch is that it's UNIX-only. If you want to tease out the magic
code from wxProcess that
does non-blocking reads on win32 then I'd be happy to integrate that
into the current development branch
of Pexpect. If someone can provide a pure Python drop-in replacement
for the read_nonblocking() function
I use for UNIX systems then it would be easy. I have a whole test
framework that I can run it through to
see if it performs the same as the UNIX flavor.

I'm sure this is feasible without any C extensions -- it might require
some ctypes hacking.
I know Windows has pretty decent async IO, but I don't know what they
have as an equivalent for a pty.
Maybe it isn't necessary. A pty is only necessary on UNIX because the
standard c library, stdio, behaves
differently when it's talking to a plain pipe versus a terminal -- it
switches buffering
between block and line oriented buffer. You don't want block buffering
on interactive applications.
This is why popen eventually breaks down. No, there is no way to
select this behavior from
the calling side... unless you can trick it into dynamically linking
to your specially hacked libc.
But that's getting ahead because that's what happens on UNIX -- it
might be a non-issue on Windows.

The read_nonblocking() function I use has an interface like this:
<pre>
    def read_nonblocking (self, size = 1, timeout = -1):

        """This reads at most size characters from the child
application. It
        includes a timeout. If the read does not complete within the
timeout
        period then a TIMEOUT exception is raised. If the end of file
is read
        then an EOF exception will be raised. If a log file was set
using
        setlog() then all data will also be written to the log file.

        If timeout is None then the read may block indefinitely. If
timeout is -1
        then the self.timeout value is used. If timeout is 0 then the
child is
        polled and if there was no data immediately ready then this
will raise
        a TIMEOUT exception.

        The timeout refers only to the amount of time to read at least
one
        character. This is not effected by the 'size' parameter, so if
you call
        read_nonblocking(size=100, timeout=30) and only one character
is
        available right away then one character will be returned
immediately.
        It will not wait for 30 seconds for another 99 characters to
come in.

        This is a wrapper around os.read(). It uses select.select() to
        implement the timeout. """
</pre>

Yours,
Noah