Python and file locking - NFS or MySQL?

[Fredrik Lundh]

Christopher DeMarco wrote:

> 2.  open(2) is atomic on a local FS, I've read discussions that imply
> that link(2) is atomic over NFS (is it?), so I can link from local
> lockfile to remote target atomically.  I don't grok this; open(2) +
> link(2) + stat(2) == 3 calls on my fingers.  HTH is this supposed to
> work?

1) generate a "globally unique" file name (e.g. use a UUID
library, or create a string containing the machine name, pid,
timestamp, possibly a random number, the FQDN, etc). e.g.

    tempname = os.uname()[1] + "." + str(os.getpid())

    lockfile = os.path.join(directory, "mylock")
    tempfile = os.path.join(directory, tempname)

2) create a file with that name in the shared directory

    f = open(tempfile, "w")
    f.close()

3) create the lock file as a hard link to the file you just
created.

       os.link(tempfile, lockfile) # atomic!

5) check the number of links to each file

       n = os.stat(tempfile)[3]
       m = os.stat(lockfile)[3]

6) if n == m, you own the lockfile.  you can now remove
the tempfile and do whatever you need to do.  remove the
lockfile when you're done (this is also an atomic operation).

if n != m is something else, remove tempfile, sleep for a few
seconds, and start all over again.

you may want to add some information to the file to be able to
identify stale locks, add proper error handling (if link or stat fails,
retry), and perhaps some kind of "incremental backoff" to con-
troll the sleep length.

</F>