transaction-like file operations

[Andrae Muys]

moc.q-dnan-p at p-nand-q.com (Gerson Kurz) wrote in message news:<3d50e6c3.100966296 at News.CIS.DFN.DE>...
> I am working on a python program that will run on an embedded
> ppc-linux system. If, during a file write operation (to flash memory),
> the machine is powered off, the file gets corrupted, and is lost on
> the next reboot. (I cannot prevent the user from powering off the
> machine - there is no display, nor any indication if the system is
> right now writing to disk or not. Its an embedded system, after all).
> 
> So, what I need is a transaction-like file operation, that allows me
> to either write the file completely, or keep the old file (so that at
> every point there is at least one set of data available).
> 
> I have made a "homegrown" solution that includes writing to a backup
> file first, then doing two rename operations. I wonder if there exists
> a standard class for transaction-like file operations in python?
> [Note: a database is not an option]

Note that you aren't looking for transaction-like file operations, the
fcntl module provides the basis for that.  What you are looking for is
some sort of data-journaling flash file-system.  Not surprisingly this
is an OS issue, python is too high level to be playing this game well.

Look into what guarrantees you can obtain from different file-systems
and build a C extension module to provide the IO guarrentees you
require.  Note that the standard journalling filesystems won't work
for you, as they tend to overwork the flash memory, seriously
shortening its life.

Once you have the infrastructure in place, your python program can
read/write to its hearts content, relying on the module and probably a
POST-fsck to repair any damage.

Andrae Muys