problems with blocking pipes, threading

[Donn Cave]

Quoth "Sven Drescher" <Sven.Drescher at dlr.de>:
[... re inter process I/O with pipes ]
| But in my further work I start processes, where I cannont know, how the
| standard output is realized. Is there a possibility to flush the child's
| stdout??? Or is there a complete other way to catch the child's stdout in
| 'real-time'???

It's hard.  If a process is written so as to buffer its output, when
writing to a pipe - and most are written that way, thanks to the default
behavior of C library I/O - there is just nothing you can do about it.

The only solution is to choose another output device, an exotic kind
of pipe called a ``pseudo-tty''.  On UNIX this device is used to
implement things like the telnet service.  One end of this pipe, the
end that you present to the external process, is a terminal device;
your other end is more or less like a socket.  It's a very important
device for things like the telnet service, but unusual in the kind
of general interprocess communication you're doing.  The system API
for pseudo-ttys is not at all standardized, hardly two UNIX implementations
do it the same way.  The exposure of these devices in the /dev filesystem
gives them extra headaches for security.  They're usually much more
limited resources than pipes or sockets, the system can run out and
then no one can telnet in.  Their behavior under stress isn't really
ideal for IPC - in particular, a terminal device can drop output if
too much arrives and it's not read fast enough.

But because the application sees a terminal, the C library uses line
buffering.  The Python module for a pseudo-tty devices is "pty", and
you can look at it for more details - it's a Python module.  It supports
only a few types of UNIX pseudo-tty implementations, but if you're lucky
it may work for you.

	Donn Cave, University Computing Services, University of Washington
	donn at u.washington.edu