Tuning a select() loop for os.popen3()

[Donn Cave]

In article <mailman.2735.1135972050.18701.python-list at python.org>,
 Christopher DeMarco <cmd at alephant.net> wrote:

> I've written a class to provide an interface to popen; I've included
> the actual select() loop below.  I'm finding that "sometimes" popen'd
> processes take "a really long time" to complete and "other times" I
> get incomplete stdout.

...

> My first question, then, is paranoid: I've run all these benchmarks
> because the application using this code saw a HUGE performance hit
> when we started using popen'd commands which generated "lots of"
> output.
> 
> Is there anything wrong with the logic in my code?!

I tried a modified version with 'ls -R .', which yields about
1 Mb of data, and saw no problems on MacOS X.  Same data, and
about the same time as 'ls -R .' from the shell, maybe 5% longer.

But I modified it a lot.  I removed every "continue", I removed
the "break", and I made readList the condition for the while loop.
With these changes, a 0.1 second timeout is about the same as no
timeout, but at 0.01 second I do see a little slow down.  Still
no loss of data.

I suspect there is indeed something wrong with your logic, but
I'm not going to try to figure it out.  If you're sure it's
right, I think you should post again with the actual code for
a program that demonstrates your problem(s).  Your goal for the
revised logic should be 1) avoid gratuitous branches in the flow
of control, 2) reduce number of state variables that you have to
account for, and 3) express your intentions clearly with respect
to the timeouts -- what do you do when it times out, and why?

   Donn Cave, donn at u.washington.edu