[Python-Dev] Does Python need a file locking module (slightly higher level)?

[M.-A. Lemburg]

On 2007-10-26 05:41, Barry Warsaw wrote:
> On Oct 22, 2007, at 11:30 PM, skip at pobox.com wrote:
> 
>> It's not clear that any of these implementations is going to be  
>> perfect.
>> Maybe none ever will be.
> 
> I would agree with this.  You write a program and know you need to  
> implement some kind of resource locking, so you start looking for  
> some OTS solution.  But then you realize that your application needs  
> somewhat different semantics or needs to work in platforms or  
> environments that the OTS code doesn't handle.  Just a few days ago,  
> I was looking at some locking code that needed to work across  
> multiple invocations of a script on multiple machines, and the only  
> thing they shared was a PostgreSQL connection, so we ended up wanting  
> to use its advisory locks.
> 
>> In his reply Jean-Paul made this comment:
> 
>>     It might be nice to have something like that in the standard  
>> library,
>>     but it's very simple once you know what to do.
> 
>> I'm not so sure about the "very simple" part, especially if you aren't
>> familiar with all the ins and outs of the different platforms.
> 
> I'd totally agree with this.  Locking seems simple, but it's got some  
> really tricky aspects that need to be coded just right or you'll be  
> in a world of hurt.  Mailman's LockFile.py (which you're right is  
> *nix only) is stable now, but has had some really subtle bugs in the  
> past.

You might want to take a look at the FileLock.py module that's
part of the eGenix mx Base distribution (mx.Misc.FileLock).

It works reliably on Unix and Windows, doesn't rely on fcntl and
has been in use for years.

The only downside is that it's application specific,
ie. only applications using the module for locking will
detect the locks - but then again: this is exactly the problem
you typically want to solve.

>> The fact
>> that the first three bits of code I was referred to were  
>> implemented by
>> three significant Python tools/platforms and that all are different  
>> in some
>> significant ways suggests that there is some both an underlying  
>> need for a
>> file locking mechanism but with a lack of consensus about the best  
>> way to
>> implement the mother-of-all-file-locking schemes for Python.  Maybe  
>> the best
>> place for this is in the distribution.  PEP?
> 
> I don't think any one solution will work for everybody.  I'm not even  
> sure we can define a common API a la the DBAPI, but if something were  
> to make it into the standard distribution, that's the direction I'd  
> go in.  Then we can provide various implementations that support the  
> LockingAPI under various environments, constraints, and platforms.   
> If we wanted to distribute them in the stdlib, we could put them all  
> in a package and let the user decide which features they need.
> 
> I'm still planning on de-Mailman-ifying LockFile.py sometime soon.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 26 2007)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611