select.select example?

[Donn Cave]

Quoth marco at atmosp.physics.utoronto.ca:
| A few days ago I posted a little problem I had regarding
| the blocking of UNIX pipes. Basically, when I ran:
|
| (po, pi, pe) = popen2.popen3('some command')
|
| and then:
|
| po_out = po.read()
| pe_err = pe.read()
...

| The other option was using the select module. If I understand correctly,
| select.select will retrieve the input, output and error (exceptional
| condition?) without me having to worry which to read first. If none
| is available a timeout can also be specified. If this is the case
| it's probably what I want, but unfortunately the documentation is
| very scarce. From what I can gather:
|
| >>> import popen2, select
| >>> (po, pi, pe) = popen2.popen3('some command')
| >>> (spo, spi, spe) = select.select([po], [pi], [pe], 10)
| >>> spo[0].read()
|
| seems to work, although according to the Python 1.5.2 Library
| Reference the order of the lists should be:
| select.select(input, output, error, 10)
| which doesn't seem to be working for me.

Meaning in this context:
   input:  file descriptors to check for readability
   output: file descriptors to check for writability
   error:  file descriptors to check for I/O errors (not error I/O!)

So you were close, but it should really have been
   rds, wrs, ers = select.select([po, pe], [pi], [pi, po, pe], 10)

You could eventually be interested in whether your pipes are
ready to have more data written to them (and in fact you had
better learn to be interested in that if you want reliable
general purpose 2 way pipe communications between processes.
But by then, you will have weaned yourself off of file objects
too.)  For now, since you're interested only in readability,
you may leave the second set empty.  The errors that you
might get from the third set are actually fairly exotic, but
it doesn't hurt to check.

| ... Furthermore, if
| 'some command' worked OK I would expect spe[0].read() to
| be '' but instead I get the error:
| IOError: [Errno 9] Bad file descriptor

Can't say I see how that happened - certainly, anything
select() reports in its third, error set will not likely
be a valid file descriptor for more I/O, after the error
that got it into that set.  But I don't know right off
hand what that error would have been.

| On the other hand, when 'some command' *does* raise an error, then
| spo[0].read() gives me '' (which is expected), but spe[0].read()
| gives the error:
| IndexError: list index out of range
| instead of std_err.

That's fine, one doesn't expect I/O errors.

About file objects and select(), please bear in mind that the
file object is a C stdio buffer.  Output can be buffered and
not written to the pipe.  Input can be buffered and be ready
to read but not visible to select().  Reads can block until
the amount requested appears in the pipe, which is usually
not convenient.  You can ameliorate some of this brain damage
by turning off the buffering, but it's even better to just
get the file descriptor back out of the file object and use
it for everything.

    po, pi, pe = popen2.popen3('some command')
    pod = po.fileno()
    pid = pi.fileno()
    ped = pi.fileno()

    while 1:
        if data:
            ofs = [pid]
        else:
            ofs = []
        ifd, ofd, efd = select.select([pod, ped], ofs, [pod, ped, pid], 10)
        if efd:
            print >> sys.stderr, "error on units", efd
            break
        if pod in ifd:
            output = os.read(pod, 16384)
            if not output:
                print >> sys.stderr, "EOF from process"
                break
            else:
                print "process output", repr(output)
        if ped in ifd:
            errors = os.read(ped, 16384)
            if not errors:
                print >> sys.stderr, "EOF from process"
                break
            else:
                print "process errors", repr(errors)
        if data and pid in ofs:
            n = os.write(pid, data)
            data = data[n:]

I haven't checked to see if that actually works, but it looks OK to me.

	Donn Cave, donn at oz.net