Maxmium bufsize using open2?

[Piet van Oostrum]

>>>>> Maurice <bertvansesamstraat at hotmail.com> (M) wrote:

M> Dear all,
M> I've a problem using popen2 when using large files.
M> When I use small input files everything works well but when I feed large
M> inputfile to the pipe nothing happens. Is there a maximum bufsize for using
M> the pipe. The code I use is written down here below. Do I need to specify a
M> waittime between line 4 and 5 ?


M> o,i =popen2('command '))
M> fh=open(os.path.join(self.dirname, self.filename),'r')
M> i.write(fh.read())
M> i.close()
M> self.StringData=o.read()
M> o.close()
	
With popen2 (and popen3 and popen4) you can manage yourself in a deadlock.
This is apparently happening to you. The problem is that the O.S. has only
a limited buffering in the pipes. Now when the command generates much
output and the buffer is full, the command blocks until you have read some
of its output. But you are still busy feeding it more input. A waittime
doesn't solve it. The only solutions are: using different threads for
input and output, or redirecting either stdin or stdout to a file.

I don't think the buffersize parameter will solve it, because this
specifies the in-process buffer, not the O.S. buffer (I think). The
in-process buffer will be flushed by i.close() so you are back to the
situation without buffer.
-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP]
Private email: P.van.Oostrum at hccnet.nl