Unbuffer stdout?

[Francis Avila]

"Donn Cave" <donn at drizzle.com> wrote in message
news:1048659589.577001 at yasure...
> Quoth "Francis Avila" <franga at shentel.net>:
> | >>> sys.stdout.write( os.popen2("yes", 'r', 0)[1].readline() )
> | y
> | >>>
>
> It would, if the pipe ever closed, but the above doesn't make that
> happen, I imagine because some internal references keep the file
> object from being deleted.  By the way, I don't think your parameters
> to popen2() are right.

You're right.  It's meaningless to specify 'r' or 'w' on popen?(), since
they return separate file objects for reading and writing.  However, the
function doesn't seem to complain....

I'm still curious as to why popen2() is not closing the pipe in this
context, while popen() is.

> Sort of.  What do you think read() is supposed to do?  Check it out:
>
>   >>> print sys.stdin.read.__doc__
>   read([size]) -> read at most size bytes, returned as a string.
>
>   If the size argument is negative or omitted, read until EOF is reached.

I realized this soon after I posted.  I had somehow got it stuck in my head
that read() should stream, or something, if the pipe were opened with
bufsize=0.  But of course that doesn't even make sense, since you can only
return once.  (Pipes are so _magical_ on the command line, I never really
thought about implementation.)

> | What I'm looking for is to pipe the stdout of a child process directly
to
> | the stdout of the parent process (i.e., the python script), without any
> | buffering along the pipeline.  As I see it, this is either not possible
in
> | python, or I'm going about this the wrong way.
>
> Well, it really is not possible to control this where it counts, which
> is on the child process end.  It will work only when the child process
> takes it upon itself to flush its output buffer;  some applications,
> notable the shell, already do this, but most don't.

With further investigation it seems that this is indeed my problem, since
the app blocks any sort of pipe, even at the shell, even after being
terminated.  However, it doesn't seem to have any problem with writing
directly to a terminal.  I suppose I'm going to have to start digging into
its code to see what's going on.

> If you don't want to read a line at a time but just whatever input is
> available, then I think it makes more sense (as usual) to forget the
> file object and use the pipe file descriptor directly --
>   fp = os.popen('yes')
>   fd = fp.fileno()
>   while 1:
>       data = os.read(fd, 16000)
>       if data:
>           os.write(1, data)
>       else:
>           break
>   fp.close()

Is there any appreciable advantage to reading from the file descriptor
directly verses passing an argument to read() or readline()?  It seems as
though:

p = os.popen('yes')
while 1:
    data = p.read(16000)
        if data:
            sys.stdout.write(data)
        else:
            break

would do much the same, and be more intuitive.  I guess it'd be faster,
since there are fewer layers to cut through.

> Because popen is in the os module?  Built-in basically means that the
> os module was written in C.  popen is a C library function, so you
> might find some documentation for it with "man 3 popen".

Ah, I didn't realize "built-in" was that general (i.e., anything written in
C).  I thought it only referred to core functions, which is how the term
seems to be used in the Library Reference.

Thanks aplenty, you've really helped me think things straight.

--
Francis Avila