Advanced lockfiles

[Carl J. Van Arsdall]

David Hirschfield wrote:
> I want some kind of lockfile implementation that will allow one process 
> to lock a file (or create an appropriately named lockfile that other 
> processes will find and understand the meaning of), but there are some 
> important requirements:
>
> 1. Multiple processes will be attempting to grab a lock on the file, and 
> they must not freeze up if they can't get a lock
> 2. The processes can be on different hosts on a network, attempting to 
> grab a lock on a file somewhere in network storage
> 3. All processes involved will know about the locking system, so no need 
> to worry about rogue processes that don't care about whatever setup we have
> 4. The locking process has to be "crash safe" such that if the process 
> that locked a file dies, the lock is released quickly, or other 
> processes can find out if the lock is held by a dead process and force a 
> release
>
> I've tried a bunch of ideas, looked online, and still don't have a good 
> way to make a system that meets all the requirements above, but I'm not 
> too well-read on this kind of synchronicity problem.
>   
It just so turns out I had to do a project with similar requirements.  
In our environment we had a common file available to 70 machines via 
NFS.  We wanted to make sure that only a single machine at a time could 
access this file.  So we did this using a combination of a couple 
technologies(this is in linux btw):

1. fcntl system calls - these are the calls to do the locking
2. rpc/lockd - this handles our locking over nfs

there is an fcntl that you can import from the python standard library.  
I found that I couldn't get these to work with my existing system setup 
and eventually gave up on them.  What I ended up doing what writing a C 
extension, this allows for blocking and non-blocking calls to get a lock 
on a file.  Another thing I had a problem with was in order to lock the 
file a python program had to open a pointer to it to test for the lock.  
After having multiple processes on 80 machines do this I ran into 
problems where the shell complained about too many open files.  I 
attempted to solve this by having the module open the file when its 
loaded and close it when its unloaded.  Maybe not the right way, but I 
haven't had the problem come up since.

I also don't know much about lockd, that was installed by another member 
of my team but I'm sure you can find good information on google

This wasn't written to be the omni-solution but a solution to my 
problem.  Here's the module:

#include "Python.h"
#include <fcntl.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>

/*Globals*/
int fp; /*Both functions need access to this but we can't open and close the file within the boundries of any
        /single function, this will be opened once by the module initializer */

/*getLock() will make fnctl calls to do nonblocking or blocking locks depending on the arguements passed*/

static PyObject * lock_getLock(PyObject *self, PyObject *args)
{
  int blockingCall, retVal;
  struct flock myLock;

  if (!PyArg_ParseTuple(args,"i", &blockingCall))
  {
    return NULL; /*NULL indicating arg failure*/
  }

  if(fp < 0)
  {
    perror("Error Opening lock file, check initialization functions");
  /*insert sys.exit(1) python call here*/  
  }

  /*populate mylock*/
  myLock.l_type = F_WRLCK;
  myLock.l_whence = SEEK_SET;
  myLock.l_start = 0;
  myLock.l_len = 0;
  myLock.l_pid = 0;



  if(blockingCall)
  {
    /*It turns out python is incredibly sensitive, in order to make these blocking functions play well*/
    /*With threading in python, we need some happy macros*/
    Py_BEGIN_ALLOW_THREADS /*release global interpreter lock*/
    retVal = fcntl(fp,F_SETLKW,&myLock);
    Py_END_ALLOW_THREADS /*aquire global interpreter lock*/
    return Py_BuildValue("i", retVal);
  }

  else /*non blocking call*/
  {
    retVal = fcntl(fp,F_SETLK,&myLock);
    return Py_BuildValue("i", retVal);
  }

}

/*releaseLock() will release any lock on a file*/
static PyObject * lock_releaseLock(PyObject *self, PyObject *args)
{
  int retVal;
  struct flock myflock;

  if(fp < 0)
  {
    perror("Error with File Pointer, there is a problem in the initialization of this module");
    return Py_BuildValue("i",-1);
  }

 

  myflock.l_type = F_UNLCK;
  myflock.l_whence = SEEK_SET;
  myflock.l_start = 0;
  myflock.l_len = 0;
  myflock.l_pid = 0;

  retVal=fcntl(fp,F_SETLK,&myflock);
  return Py_BuildValue("i", retVal);

}

static PyMethodDef lock_methods[] = 
{
  {"getLock", lock_getLock, METH_VARARGS, "calls fcntl to get a lock on lockfile"},
  {"releaseLock", lock_releaseLock, METH_VARARGS, "releases lock on lockfile"},
  {NULL, NULL}
};


/*Close the file to be neat about things*/
void cleanupModule(void)
{
  close(fp);
}


/*Init function that python needs to load this as a module*/
void initlock() 
{
  fp = open("/home/build/bin/resourceManager/lock", O_RDWR);
  (void) Py_InitModule("lock", lock_methods);
  Py_AtExit((void*)cleanupModule);

}







-- 

Carl J. Van Arsdall
cvanarsdall at mvista.com
Build and Release
MontaVista Software