How to safely maintain a status file

[Gene Heskett]

On Thursday 12 July 2012 23:21:16 Steven D'Aprano did opine:

> On Fri, 13 Jul 2012 12:12:01 +1000, Chris Angelico wrote:
> > On Fri, Jul 13, 2012 at 11:20 AM, Rick Johnson
> > 
> > <rantingrickjohnson at gmail.com> wrote:
> >> On Jul 12, 2:39 pm, Christian Heimes <li... at cheimes.de> wrote:
> >>> Windows's file system layer is not POSIX compatible. For example you
> >>> can't remove or replace a file while it is opened by a process.
> >> 
> >> Sounds like a reasonable fail-safe to me.
> 
> Rick has obviously never tried to open a file for reading when somebody
> else has it opened, also for reading, and discovered that despite
> Windows being allegedly a multi-user operating system, you can't
> actually have multiple users read the same files at the same time.
> 
Chuckle.  That was one of the 'features' that os9 on the trs-80 color 
computer had back in the 80's, and it was clean and well done because of 
the locking model the random block file manager had in OS9 for 6809 cpu's, 
no relation to the Mac OS9 other than a similar name.  That color computer 
has a separate, text only video card I could plug in and display on an 80 
column amber screen monitor.

When I wanted to impress the visiting frogs, I often did something I have 
never been able to do on any other operating system since, start assembling 
a long assembly language file on one of the screens on the color monitor, 
hit the clear key to advance to the amber screen and start a listing on it 
of the assemblers output listing file.

Because the file locking was applied only to the sector (256 bytes on that 
machine) being written at the instant, the listing would fly by till it 
caught up with the assemblers output, running into the lock and then 
dutifully following along, one sector behind the assemblers output, until 
the assembly was finished.  That was in 1986 folks, and in the year of our 
Lord 2012, 26 years later, I still cannot do that in linux.  When I ask why 
not, the replies seem to think I'm from outer space.  Its apparently a 
concept that is not even attempted to be understood by the linux code 
carvers.

Something is drastically wrong with that picture IMO.

> (At least not unless the application takes steps to allow it.)
> 
> Or tried to back-up files while some application has got them opened.

That in fact, ran me out of the amiga business in 1999, a 30Gb drive failed 
on my full blown 040 + 64 megs of dram A2000.  When the warranty drive 
arrived is when I found that due to file locks on the startup files, all of 
them involved with the booting of that machine, my high priced Diavolo Pro 
backup tapes didn't contain a single one of those files.  The linux box 
with Red Hat 5.0 on it that I had built in late 1998 to see what linux was 
all about found space under that desk yet that evening and I never looked 
back.

> Or
> open a file while an anti-virus scanner is oh-so-slooooowly scanning it.
> 
> Opening files for exclusive read *by default* is a pointless and silly
> limitation. It's also unsafe: if a process opens a file for exclusive
> read, and then dies, *no other process* can close that file.
> 
> At least on POSIX systems, not even root can override a mandatory
> exclusive lock (it would be pretty pointless if it could), so a rogue or
> buggy program could wreck havoc with mandatory exclusive file locks.
> That's why Linux, by default, treats exclusive file locks as advisory
> (cooperative), not mandatory.
> 
> In general, file locking is harder than it sounds, with many traps for
> the unwary, and of course the semantics are dependent on both the
> operating system and the file system.
> 
> https://en.wikipedia.org/wiki/File_locking
> 
> > POSIX says that files and file names are independent. I can open a
> > file based on its name, delete the file based on its name, and still
> > have the open file there. When it's closed, it'll be wiped from the
> > disk.
> 
> One neat trick is to open a file, then delete it from disk while it is
> still open. So long as your process is still running, you can write to
> this ghost file, as normal, but no other process can (easily) see it.
> And when your process ends, the file contents is automatically deleted.
> 
> This is remarkably similar to what Python does with namespaces and
> dicts:
> 
> # create a fake "file system"
> ns = {'a': [], 'b': [], 'c': []}
> # open a file
> myfile = ns['a']
> # write to it
> myfile.append('some data')
> # delete it from the "file system"
> del ns['a']
> # but I can still read and write to it
> myfile.append('more data')
> print(myfile[0])
> # but anyone else will get an error if they try
> another_file = ns['a']

Cheers, Gene
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
My web page: <http://coyoteden.dyndns-free.com:85/gene> is up!
You just wait, I'll sin till I blow up!
		-- Dylan Thomas