avoid script running twice

[Jeff McNeil]

I've got a rather large log processing job here that has the same
requirement.  I process and sort Apache logs from an 8-way cluster. I
sort and calculate statistics in 15-minute batch jobs. Only one copy
should run at once.

I open a file and lock it via something like this:

import fcntl

fhandle = file("ourlockfile.txt", "w")

try:
    fcntl.lockf(fhandle.fileno(), fcntl.LOCK_EX|fcntl.LOCK_NB)
except IOError, e:
    if e.errno == errno.EAGAIN:
        print >>sys.stderr, "exiting, another copy currently running"
    else:
        raise

I've got it wrapped in a 'FileBasedLock' class that quacks like Lock
objects in the threading module.

If the system bombs and reboots, the locks are cleared. If the first
process dies via 'kill -XX', the locks it owns are released.  Of
course, external influences can screw it all up, but that's not so
much a concern in our environment.

While I have no idea how portable this is, it works great for what I'm
doing on RHES 5.

-Jeff

On 6/18/07, Tim Williams <tim at tdw.net> wrote:
> On 18/06/07, Nick Craig-Wood <nick at craig-wood.com> wrote:
> > Tim Williams <tim at tdw.net> wrote:
> > >  You can also do this by holding a file open in write mode until the
> > >  script has finished.
> > >
> > >        try:
> > >             open('lock.txt','w')
> > >             my_script()
> > >       except:
> > >            #print script is already running
> >
> > That only works under windows
> >
> >  >>> f=open('lock.txt','w')
> >  >>> g=open('lock.txt','w')
> >  >>> f.write('hi')
> >  >>> g.write('ho')
> >  >>> f.close()
> >  >>> g.close()
> >  >>> open('lock.txt').read()
> >  'ho'
> >  >>>
> >
> > The best cross platform way to create a lock is creating a directory.
> > It is atomic on both windows and linux anyway.
> >
> >  try:
> >    os.mkdir("lock")
> >  except OSError:
> >    print "locked!"
> >  else:
> >    try:
> >      do_stuff()
> >    finally:
> >      os.rmdir("lock")
> >
> > (untested)
> >
>
> Being a windows-only person, I didn't know that :)
>
> Actually I think I did,  this thread has happened before - a few months ago :)
>
> I would be worried with the directory-exists option for the same
> reason I don't use the file-exists method currently.  It is possible
> for the directory to exist when the script isn't running, thus
> preventing the script from running again until someone notices.
>
> On Windows the open-a-file-for-writing method works well, but as *nix
> doesn't work the same way then maybe the socket solution is the best
> cross-platform option.  The socket can't exist when the script isn't
> running, and if you try and create a duplicate socket you catch the
> exception and exit.
>
> IMHO of course.
>
> :)
> --
> http://mail.python.org/mailman/listinfo/python-list
>