Nonblocking IO on popen pipe (isn't nonblocking)

[Noah]

Ah... some things are  more clear now.
I did have something like this working before,
but it was running on Solaris. Now I'm working on OpenBSD.
What I have discovered is that apparently the ftp 
on OpenBSD writes the "Password:" message directly to the tty.
It also reads the password back direct from the tty device.
AFTER that it switches back to stdin and stdout.
FTP was never putting anything into the pipe
because it never got past the login. After this
the fix was simple. So now I have non-blocking 
pipes wrapped with a select and all is happy
(why do this? So I can control the timeout without
having to poll inside of a loop)

The quick fix was to make use of the fact that
the OpenBSD version allows you to invoke FTP with 
the password on the command line (which seems
oddly unsecure). Now it works perfectly.
Expect python here I come!

Long term, I would like something more portable,
so I guess I will have to start to look at that
pty stuff now. I'm trying to figure out the pty library,
but it lacks samples (except for test_pty.py)
and documentation on the library is slim. 
Hopefully pty will work under OpenBSD. I looked
at the source code and there is support for BSD
in there, but the comments say it was only tested
on SGI and Linux.

It would be nice to have a 100% Python expect.

Yours,
Noah

-----Original Message-----
From: python-list-admin at python.org
[mailto:python-list-admin at python.org]On Behalf Of Donn Cave
Sent: Thursday, April 11, 2002 3:16 PM
To: python-list at python.org
Subject: RE: Nonblocking IO on popen pipe (isn't nonblocking)


Quoth "Noah" <noah at noah.org>:
| Hmm... This sounds suspicious... If FTP won't write its buffered
| output then no application could see it -- that includes the shell!

When you use ftp in a normal session, the shell doesn't see its
output.  You're interacting with the ftp process through the
tty driver.  This phenomenon occurs when output is a pipe, not
when it's a tty.

| Python shouldn't be any different in what it sees.
| If I start FTP from the shell, then how does FTP know 
| when to flush the buffer so that you can see it in a shell?

C stdio I/O, which ftp presumably uses, buffers its output differently
depending on the device.  If it sees that it's writing to a tty, then
it "line buffers" output.  Otherwise, it "block buffers".  That's why
the solution is a device that acts like a pipe but looks like a tty.
(Of course all this is assuming that ftp actually does use stdio.  It
doesn't have to, and I haven't checked.)

	Donn Cave, donn at u.washington.edu