[Tutor] Re: manipulating a file

[Danny Yoo]

> >>This simplifies the code down to:
> >>
> >>###
> >>srcfile = open('/var/log/httpd-access.log.bak', 'r')
> >>dstfile = open('/var/log/httpd-access.log', 'w')
> >>for line in srcfile:
> >>    if len(line) < 2086:
> >>        dstfile.write(line)
> >>srcfile.close()
> >>dstfile.close()
> >>###
> >>
> >>
> I am actually mv log to bak and write the non offening entries back to
> log.


Hi Reed,

Oh, so the web server is still continuing to write things onto the open
log file then.  Out of curiosity, what web server are you using?

One remote possiblility for the weirdness might depend on how your web
server writes new log messages into the file: perhaps it can automatically
detect log file rotation, and may be overwriting http-access.log?  It's
very hard to say.  I'll talk about this a little more below.


> I can not be working on log as it needs to be able to accept new entries
> as the webserver is accessed.  Plus I am learning python, I could have
> sh scripted it easy but am broadening my horizons...

Sure, I think that's perfectly fine.  I think there was some confusion on
the list earlier, so I'm glad you cleared that up.


I'm still trying to figure out what could be causing the problem.  You
mentioned that you were running the programs as root, so permission
problems are probably not an issue.  I don't think we have enough
information to debug this, and I hate shooting arrows in random
directions.

I feel a little nervous programs that try to do 'in-place' changes.
Conceptually, we're trying to swap out httpd_access out from underneath
the httpd process's nose, and that might not work.  If you're doing a
'mv', then the log messages should continue to log to the backup file, if
I understand how apache works.

Let's test something.  Can you try writing the truncated log to another
file, like '/var/log/http-access.truncated.log'?  This at least should
prevent any wacky writing conflicst between the httpd process and our
log-filtering program.



Also, what happens if instead of opening up explicit files you use
standard input and output?  Here's a modified program that uses sys.stdin
and sys.stdout:

######
"""Cut really long lines out of the output."""
import sys
for line in sys.stdin:
    if len(line) < 2086:
        sys.stdout.write(line)
sys.stdin.close()
sys.stdout.close()
######

This can be equivalent to the previous program if we use the shell's file
redirection.  It also allows us to run the program on different input and
output files with ease.  Try it on some other files first and see that it
does work on regular files first.

Best of wishes to you!