Mutex a file?

[Sheila King]

On Thu, 14 Feb 2002 16:12:51 -0500, "MDK" <mdk at mdk.com> wrote in comp.lang.python in
article <a4h98k$ao2f$1 at ID-98166.news.dfncis.de>:

> I have a file that is to be read to and written from by many Python scripts
> running at the same time.
> 
> Is there a way to put a Mutex on it so that the scripts wait their turn
> before tyring to open it?

This is a problem I worked on some time back. On most Unices, it is not 
hard to lock the file (which is what you need to do). I guess there are 
some problems if you are working across certain types of networks. On 
WinNT and Win2000 (and probably XP) there is also a great deal of control 
over files and file access. It's the Win9x where you loose a lot of this 
ability. I'm not sure what platform you are running on...if Windows, 
you might want to look in more detail at the win32all libraries from 
Mark Hammond.

Anyhow, here are some links to the discussion I initiated on this topic 
some time back. Read the threads. There are many useful comments.
http://groups.google.com/groups?as_umsgid=t7uqot8n8gjm03r2m03md8kfsjuauchqhu%404ax.com
http://groups.google.com/groups?as_umsgid=djsfotomogdeba70gp1cuogjbt5ii3chrb%404ax.com
http://groups.google.com/groups?as_umsgid=bo2ootg61vmposfvsg8gvrks6eqiuss940%404ax.com
http://mail.python.org/pipermail/tutor/2001-August/008366.html


Here is the module I wrote, and am using in a production environment 
on a Linux server for file locking. Advice given to me re: the 
Windows component of this modules is...well, that it wouldn't be 
recommended to use in a production environment. But who would run 
a production server on a Win98 machine anyhow? So, for me it let's 
me test my scripts on my home Windows machine and then upload it 
to my Linux server and expect the same results.

FWIW, I've pasted the code I'm currently running below my sig.

-- 
Sheila King
http://www.thinkspot.net/sheila/

"When introducing your puppy to an adult cat,
restrain the puppy, not the cat." -- Gwen Bailey,
_The Perfect Puppy: How to Raise a Well-behaved Dog_


'''MutexFile.py

wraps WinMutexFile.py and posixMutexFile.py to provide
a single interface for working with mutex files
across win32 and posix platforms.
Does not support fine-grained locking.
Recommended for getting a lock on a sentinel file
before trying to read/write data to a separate data file.

public interface:
obj = MutexFile.MutexFile('somefile.txt')
obj.flock(flag)

where flag is one of the following:
LOCK_SH - get a shared (or read) lock
LOCK_EX - get an exclusive (or write) lock
LOCK_UN - release a lock
'''
import os

if os.name == 'nt':
     from winMutexFile import mutexfile
elif os.name == 'posix':
     from posixMutexFile import mutexfile
else:
    raise ImportError, "MutexFile is not supported on your platform."


class MutexFile(mutexfile):
    pass



''' posixMutexFile.py

class lockfile
supports the same function calls as
winLock.lockfile
export to MutexFile.py: a wrapper module
around the win and posix lockfiles
'''

import os, fcntl

class mutexfile:
    def __init__(self, filename):
        if os.access(filename, os.F_OK):
            self.filename = filename
            self.fd = None
        else:
            errmssg = filename + " does not exist. Can't lock non-existent file."
            raise IOError, errmssg

    def __del__(self):
        try:
            self.unlock()
        except:
            pass
        try:
            self.f.close()
        except:
            pass

    def getReadLock(self):
        self.f = open(self.filename, 'r')
        self.fd = self.f.fileno()
        fcntl.lockf(self.fd, fcntl.LOCK_SH)


    def getWriteLock(self):
        self.f = open(self.filename, 'r+')
        self.fd = self.f.fileno()
        fcntl.lockf(self.fd, fcntl.LOCK_EX)


    def unlock(self):
        fcntl.lockf(self.fd, fcntl.LOCK_UN)
        self.f.close()
        self.fd = None

    def flock(self, flag):
        '''flags are:
        LOCK_UN - unlock
        LOCK_SH - acquire a shared (or read) lock
        LOCK_EX - acquire an exclusive (or write) lock
        '''
        if flag == 'LOCK_SH':
            self.getReadLock()
        elif flag == 'LOCK_EX':
            self.getWriteLock()
        elif flag == 'LOCK_UN':
            self.unlock()
        else:
            errmssg = "The flag " + flag + " is not implemented for flock"
            raise NotImplementedError, errmssg



''' winMutexFile.py

class lockfile
supports the same function calls as
posixLock.lockobject
export to lockobject.py: a wrapper module
around the win and posix lockobjects
'''
import os
from time import time, sleep

try:
    import win32file
    from win32con import GENERIC_READ, GENERIC_WRITE,\
        FILE_SHARE_READ, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL
except ImportError, e:
    print e
    print "winMutexFile.py requires the ActiveState.com Python win32 extensions"
    raise

MAXTIME = 8  # number of secs to retry for a lock before timing out
sleepinterval = 0.1 # number of secs to sleep between retries for lock

class mutexfile:
    def __init__(self, filename):
        if os.access(filename, os.F_OK):
            self.filename = filename
            self.fd = None
        else:
            errmssg = filename + " does not exist. Can't lock non-existent file."
            raise IOError, errmssg

    def __del__(self):
        try:
            self.unlock()
        except:
            pass
        try:
            self.f.close()
        except:
            pass

    def getReadLock(self):
        start_time = time()
        while time() - start_time < MAXTIME:
            try:
                self.fd = win32file.CreateFile(self.filename, GENERIC_READ,\
                                              FILE_SHARE_READ, None, OPEN_EXISTING,\
                                              FILE_ATTRIBUTE_NORMAL, 0)
                return 1
            except:
                pass
            sleep(sleepinterval)
        if not self.fd:
            errmssg = self.filename + " temporarily unavailable"
            raise IOError, errmssg

    def getWriteLock(self):
        start_time = time()
        while time() - start_time < MAXTIME:
            try:
                self.fd = win32file.CreateFile(self.filename, GENERIC_READ,\
                                              0, None, OPEN_EXISTING,\
                                              FILE_ATTRIBUTE_NORMAL, 0)
                return 1
            except:
                pass
            sleep(sleepinterval)
        if not self.fd:
            errmssg = self.filename + " temporarily unavailable"
            raise IOError, errmssg

    def unlock(self):
        win32file.CloseHandle(self.fd)
        self.fd = None

    def flock(self, flag):
        '''flags are:
        LOCK_UN - unlock
        LOCK_SH - acquire a shared (or read) lock
        LOCK_EX - acquire an exclusive (or write) lock
        '''
        if flag == 'LOCK_SH':
            self.getReadLock()
        elif flag == 'LOCK_EX':
            self.getWriteLock()
        elif flag == 'LOCK_UN':
            self.unlock()
        else:
            errmssg = "The flag " + flag + " is not implemented for flock"
            raise NotImplementedError, errmssg