[Tutor] script follows a file into zip?

[Peter Otten]

richard kappler wrote:

> I have a script that checks a file and if there are new additions to the
> file, parses and sends the new additions. This log rolls over and zips at
> midnight creating a new log of the same name. The problem is, when the
> file rolls over, the new file seems to follow the old file the way the
> script is currently written.
> 
> Current script (edited for brevity):
> #######################################
> from time import sleep
> 
> f1 = open('theFile.log', 'r')
> 
> f1.seek(0,2)
> eof = f1.tell()
> 
> While True:
>     try:
>         #check file size to see if grown, set neweof
>         f1.seek(0,2)
>         neweof = f1.tell()
>     except ValueError:
>         f1 = open(rd1, 'r')
>         f1.seek(0,2)
>         neweof = f1.tell()
> 
>     #if the file is larger...
>     if neweof > eof:
>         do a bunch of stuff
> 
>         # update log.txt file size
>         eof = neweof
>         time.sleep(10)
> 
>     elif neweof < eof:
>         # this resets eof at night when old log file zipped and new log
> file started
>         eof = 0
>         time.sleep(10)
> 
>     elif neweof == eof:
>          # file hasn't changed, do nothing
>          time.sleep(10)
> #############################################
> 
> The script works great all day until the logrotate at midnight. I would
> expect to get the elif neweof < eof bit then, but my log shows I'm getting
> elif neweof == eof for the rest of the night, which is what leads me to
> believe that the script, as written, is following the zipped file, not
> looking at the new file.
> 
> I have two thoughts on this, not sure which to use, or if something else
> completely might be appropriate.
> 1. Instead of
> 
> f1 = open('theFile.log', 'r')
> 
> while True:
> ....
> 
> Would it make a difference if I used
> 
> with open('theFile.log', 'r') as f1:
>     while True:
> and so on

No. The problem is that you keep the file open. When the file is overwritten 
by another application only the entry in the file system is replaced while 
the underlying file (identified by its inode) is kept.

The file you see has now no entry in the file system and therefore could 
only be changed by you; as you only read it no more changes will happen.

To see the file that is  is currently registered under the name 
"theFile.log" you have to reopen:

while True:
    with open("theFile.log") as f1:
        ...

 
> 2. Get rid of the elif neweof == eof statement and just make it
> else:
>    pass
> 
> to send it back to the beginning of the loop (doesn't make sense to me,
> but wanted to throw it out there).
> 
> regards, Richard