select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))

[Chris McDonough]

On Wed, 2004-08-04 at 11:20, Guido van Rossum wrote:
> > Doesn't select() effectively busy-wait on "real" files (pipes and file
> > descriptors obtained via open(), as opposed to network sockets) on most
> > (all?) UNIXen?  At least this has been my finding under Linux.
> 
> select() doesn't make sense for regular files, so tail -f can't use
> it.  For Unix pipes, it works as advertised.  On Windows, it *only*
> applies to sockets.

Aha.  I think I understand now.

Sorry for going off-topic, but this program (UNIX-only):

import select
import errno
import fcntl
import os

def go(r, timeout=1):
    while 1:
        try:
            r, w, x = select.select(r, [], [], timeout)
            print r
        except select.error, err:
            if err[0] != errno.EINTR:
                raise
            r = w = x = []

p_in, p_out = os.pipe()

for fd in p_in, p_out:
    flags = fcntl.fcntl(fd, fcntl.F_GETFL)
    fcntl.fcntl(fd, fcntl.F_SETFL,  flags | os.O_NDELAY)

os.close(p_in)

go([p_out])

... will busywait in select (p_out is always in the ready state; the
select timeout is never reached).

But if you comment out the "os.close(p_in)" line, it no longer busywaits
(the select timeout is reached on every iteration).  At least this is
the behavior under Linux.

This is a little unfortunate because the normal dance when communicating
between parent and child in order to capture the output of a child
process seems to be:

1) In the parent process, create a set of pipes that will represent
stdin/stdout/stderr of the child.

2)  fork

3) In the parent process, close the "unused" fds representing the ends
of the pipes that the child will use as fds for stdin/stderr/stdout.

4) In the child process, dup these same fds to stdin, stdout, and
stderr.

... so select seems to be unuseful here, unless step 3 isn't actually
necessary.

Still a bit confused and researching...

- C