pipe problem on FreeBSD....

[Steve Spicklemire]

Thanks Jarkko! 

       OK... I think I see now how my program was flawed. However
there is still something fishy..... 

when I use *your* program I can do

./jarrkki-tryit | more

and it behaves quite well. I also tried with threads enabled
and without. On the other hand the python version dies with
the broken pipe error (on FreeBSD only..). I'm assuming that
either python is doing something fundamentally different than
the c program, or there is some other library that is being
used by python but not by the c program (that is broken on
FreeBSD, but OK on linux......) whew!

anyway.. thanks for the input.... I've really got to track
this down.. and any insights are most welcome!

-steve

>>>>> "Jarkko" == Jarkko Torppa <torppa at polykoira.megabaud.fi> writes:

    Jarkko> In article <200001082039.PAA12903 at acer.spvi.com>, Steve Spicklemire wrote:
    Jarkko> >   Hmmm.. what I thought started out as a "Zope Bug" isn't. Let me 
    Jarkko> >forgo all the history here and cut to the current analysis. There
    Jarkko> >seems to be a problem on FreeBSD with python sending large amounts of 
    Jarkko> >data through a pipe that blocks.
    Jarkko> ...
    Jarkko> >Here is the reference to that problem report:
    Jarkko> >http://lists.zope.org/pipermail/zope/1999-December/015874.html
    Jarkko> ...
    Jarkko> >I think I've now simplified it to it's most basic form:
    Jarkko> 
    Jarkko> This is quite different matter that what you are seeing in Zope.
    Jarkko> >This python script:
    Jarkko> ...
    Jarkko> >fails on FreeBSD when invoked thusly:
    Jarkko> >
    Jarkko> >pluto.spvi.com> python tryit.py | head -3
    Jarkko> ...
    Jarkko> >IOError: [Errno 32] Broken pipe
    Jarkko> >where this c program does not:
    Jarkko> ..
    Jarkko> 
    Jarkko> Here is c program that does tries more or less match python's 
    Jarkko> behaviour. 
    Jarkko> ----
    Jarkko> #include <stdio.h>
    Jarkko> #include <err.h>
    Jarkko> #include <signal.h>
    Jarkko> 
    Jarkko> int main() {
    Jarkko>   size_t charCount,c;
    Jarkko>   char buffer[1024];
    Jarkko>   FILE *f;
    Jarkko> 
    Jarkko>   signal(SIGPIPE,SIG_IGN);
    Jarkko> 
    Jarkko>   if((f = fopen("blah","r")) != NULL) {
    Jarkko>     do {
    Jarkko>       charCount = fread(buffer,1,sizeof(buffer),f);
    Jarkko>       c=fwrite(buffer,1,charCount,stdout);
    Jarkko>       if ( c != charCount )
    Jarkko> 	err(1,"WriteFailed");
    Jarkko>     } while(charCount == sizeof(buffer));
    Jarkko>     close(f);
    Jarkko>   }
    Jarkko>   else
    Jarkko>     printf("Can't open file... blah\n");
    Jarkko> }
    Jarkko> ---
    Jarkko> tuhnu[2]%./tryit | head -3
    Jarkko> ...
    Jarkko> tryit: WriteFailed: Broken pipe
    Jarkko> 
    Jarkko> >My thinking is that they should behave in the same way. (This also
    Jarkko> >happens with 'more' and other such blocking programs.) Where in python
    Jarkko> >should I look for the difference between tryit.c and tryit.py? Of
    Jarkko> >course on linux both programs *do* behave the same way... I just
    Jarkko> >created the 'c' version to see if it was deep in FreeBSD or somewhere
    Jarkko> >in the python port. The python and c version work correctly in linux.
    Jarkko> ><dang!>
    Jarkko> 
    Jarkko> By default SIGPIPE terminates the program, also your cprogram
    Jarkko> did not check writes return value so it could not have
    Jarkko> caught the error anyhow. 
    Jarkko> 
    Jarkko> Behaviour you are seeing is correct. But this is different from
    Jarkko> blocking it should not give SIGPIPE, writing thread should
    Jarkko> just stall untill there is space in pipe to put the data.
    Jarkko> 
    Jarkko> This python piece does blocking reads on stdin
    Jarkko> 
    Jarkko> import time,sys
    Jarkko> while 1:
    Jarkko>     r=sys.stdin.read(1024)
    Jarkko>     if len(r) == 0:
    Jarkko>         break
    Jarkko>     sys.stdout.write(r)
    Jarkko>     time.sleep(30)
    Jarkko> 
    Jarkko> -- 
    Jarkko>  Jarkko Torppa                torppa at staff.megabaud.fi
    Jarkko>   Megabaud Internet-palvelut
    Jarkko> -- 
    Jarkko> http://www.python.org/mailman/listinfo/python-list
    Jarkko> 
    Jarkko>