Help need with subprocess communicate

[Daniel Klein]

On Tue, 3 Jun 2008 23:48:38 -0700 (PDT), rdabane at gmail.com wrote:

>On Jun 3, 11:23 pm, Dennis Lee Bieber <wlfr... at ix.netcom.com> wrote:
>> On Tue, 3 Jun 2008 18:04:40 -0700 (PDT), rdab... at gmail.com declaimed the
>> following in comp.lang.python:
>>
>>
>>
>> > Hi Daniel,
>> > Thanks for your reply..
>> > I've done exactly as you suggested...but I'm still having problem with
>> > the read...it just gets stuck in
>> > the read ( I think because its a blocking read...)
>>
>>         And it is likely blocking because the subprocess is doing buffered
>> output -- ie, nothing is available to be read because the output has not
>> been flushed.
>>
>>         This is a problem with most programs when run as a subprocess -- it
>> is common for stdout, when routed to a pipe or file, to behave as a
>> buffered stream that only flushes when some x-bytes have been written;
>> unlike stdout to a console which gets flushed on each new-line.
>> --
>>         Wulfraed        Dennis Lee Bieber               KD6MOG
>>         wlfr... at ix.netcom.com              wulfr... at bestiaria.com
>>                 HTTP://wlfraed.home.netcom.com/
>>         (Bestiaria Support Staff:               web-a... at bestiaria.com)
>>                 HTTP://www.bestiaria.com/
>
>
>Is there way to configure the stdout buffer size so that it flushes
>earlier..
>Is there a way to make above mentioned piece code working?

I'm not so sure it is a buffer problem. To test this out I first created a
'p2.py' script...

from subprocess import *
import os
p=Popen('ConsoleApplication1.exe',stdin=PIPE,stdout=PIPE,universal_newlines=True)
print p.stdout.readline()[:-1] # strip \n from end of line
p.stdin.write('hi' + os.linesep)
print p.stdout.readline()[:-1]
p.stdin.write('bye' + os.linesep)
print p.stdout.readline()[:-1]
p.stdin.close()
p.stdout.close()

I then created the following VB console application (this is the 'process'
that is being 'Popen'd and is in my %PATH%)...

Module Module1
    Dim x As String
    Sub Main()
        Console.WriteLine("Process started...")
        x = Console.ReadLine()
        Console.WriteLine(x)
        x = Console.ReadLine()
        Console.WriteLine(x)
    End Sub
End Module

Here is the output when I run it...

C:\home\python>python p2.py
Process started...
hi
bye

Note that I didn't have to 'flush()' anything.

I got the same thing working with a C program. I don't know why it won't
work with a similar python script...

import sys
sys.stdout.write('process started...\n')
r = sys.stdin.readline()
sys.stdout.write(r + '\n')
s = sys.stdin.readline()
sys.stdout.write(s + '\n')

I called this 'p3.py'. When I plug this into the 'p2.py' script I get
nothing, it just hangs. So maybe there is something else I am missing.

I normally don't do things this way cos there are os size limits to what you
can send/recv, so I use my own protocol (similar to netstrings)  for
communication.

Daniel Klein