Bug/Weak Implementation? popen* routines can't handle simultaneous read/write?

[dmoore]

thanks for all of your responses. i'll look more closely at pexpect
(The version I originally saw was much much older). I do want a cross-
platform solution. I'm pretty busy so getting the wxProcess magic into
useful shape will take me some time (at least on the order of weeks)
so other interest parties feel free to step up.

cheers
Damien

On Jun 7, 8:59 pm, Noah <n... at noah.org> wrote:
> On Jun 7, 9:01 am, dmoore <damienlmo... at gmail.com> wrote:
>
> popen and friends will never do what you want it to do. Down that path
> lies bitter disappointment.
> You need pseduo-ttys and non-blocking IO. I don't know how to do this
> on Windows, but I know it's possible.
> Cygwin does it.
>
> > Anybody have any thoughts on this? Do I have my story straight? (the
> > popen variants can't handle this case and there are no other
> > alternatives in the standard python distro) Is there some place I can
> > submit this as a feature request? (Python dev?)
>
> Try Pexpecthttp://pexpect.sourceforge.net/
> It's been around for a long time and is quite complete and stable.
>
> The catch is that it's UNIX-only. If you want to tease out the magic
> code from wxProcess that
> does non-blocking reads on win32 then I'd be happy to integrate that
> into the current development branch
> of Pexpect. If someone can provide a pure Python drop-in replacement
> for the read_nonblocking() function
> I use for UNIX systems then it would be easy. I have a whole test
> framework that I can run it through to
> see if it performs the same as the UNIX flavor.
>
> I'm sure this is feasible without any C extensions -- it might require
> some ctypes hacking.
> I know Windows has pretty decent async IO, but I don't know what they
> have as an equivalent for a pty.
> Maybe it isn't necessary. A pty is only necessary on UNIX because the
> standard c library, stdio, behaves
> differently when it's talking to a plain pipe versus a terminal -- it
> switches buffering
> between block and line oriented buffer. You don't want block buffering
> on interactive applications.
> This is why popen eventually breaks down. No, there is no way to
> select this behavior from
> the calling side... unless you can trick it into dynamically linking
> to your specially hacked libc.
> But that's getting ahead because that's what happens on UNIX -- it
> might be a non-issue on Windows.
>
> The read_nonblocking() function I use has an interface like this:
> <pre>
>     def read_nonblocking (self, size = 1, timeout = -1):
>
>         """This reads at most size characters from the child
> application. It
>         includes a timeout. If the read does not complete within the
> timeout
>         period then a TIMEOUT exception is raised. If the end of file
> is read
>         then an EOF exception will be raised. If a log file was set
> using
>         setlog() then all data will also be written to the log file.
>
>         If timeout is None then the read may block indefinitely. If
> timeout is -1
>         then the self.timeout value is used. If timeout is 0 then the
> child is
>         polled and if there was no data immediately ready then this
> will raise
>         a TIMEOUT exception.
>
>         The timeout refers only to the amount of time to read at least
> one
>         character. This is not effected by the 'size' parameter, so if
> you call
>         read_nonblocking(size=100, timeout=30) and only one character
> is
>         available right away then one character will be returned
> immediately.
>         It will not wait for 30 seconds for another 99 characters to
> come in.
>
>         This is a wrapper around os.read(). It uses select.select() to
>         implement the timeout. """
> </pre>
>
> Yours,
> Noah