Having trouble with tail -f standard input

[Diez B. Roggisch]

sab schrieb:
> Hello,
> 
> I have been working on a python script to parse a continuously growing
> log file on a UNIX server.  The input is the standard in, piped in
> from the log file.  The application works well for the most part, but
> the problem is when attempting to continuously pipe information into
> the application via the tail -f command.  The command line looks
> something like this:
> 
> tail -f <logfile> | grep <search string> | python parse.py
> 
> If I don't pipe the standard in to the python script, it displays any
> new entries immediately on the screen.  However, if I pipe the
> information into the script, the sys.stdin.readline() doesn't get any
> new data until a buffer fills, after which it parses a block of new
> information all at once (output is fine).  I need it to read the data
> in real-time instead of waiting for the buffer to fill.  I have tried
> running the script with the -u parameter but that doesn't seem to be
> doing anything.  Also, if I run the program against a text file and
> add a line to the text file (via cat >> <text file>) it picks it up
> right away.  I'm sure that it's just a simple parameter that needs to
> be passed or something along those lines but have been unable to find
> the answer.  Any ideas would be appreciated.

Get rid of tail, it's useless here anyway and most probably causing the 
problem.

If for whatever reason you can't get rid of it, try and see if there is 
any other way of skipping most of the input file - maybe creating *one* 
python script to seek to the end, grep & parse.

You can't do anything in python though - the buffering and potential 
flushing is courtesy of the upper end of the pipe - not python.

Diez