[Python-Dev] wait time [was: Ext4 data loss]

[Daniel Stutzbach]

On Thu, Mar 12, 2009 at 4:09 PM, "Martin v. Löwis" <martin at v.loewis.de>wrote:

> So when you select "Save" in your application, would you like the data
> to be saved, or would you accept that they get lost? If the latter,
> what kind of interaction would you perform with your application to
> indicate that you *do* want the data to appear on disk?
>

I accept that if the computer crashes at just the wrong moment as I click
Save, my changes will not actually be Saved.  No amount of diligence in the
implementation of close() can prevent that since the computer can crash
before the program calls close().

I oppose applications that lose or corrupt both my new save and my
*previous* save if the computer crashes at the wrong moment.  That would
cause me to lose not only my most recent changes (an inconvenience), but
also all the work I have ever done on the file (a major headache for anyone
who doesn't make regular backups).

However, defaulting to calling fsync() when closing a file will:
1) Cripple performance for the many applications that don't need it (e.g.,
temporary files)
2) Fail to prevent data loss for applications that use the
truncate-and-rewrite paradigm for saving

Consider the following example:

with open('mysavefile', 'w') as f:
    f.write(data)
    f.flush()
    os.fsync(f.fileno())
    f.close()

If the system crashes after the call to open(), but before the call to
fsync(), then both the old and the new mysavefile may be gone.

Since needing to safely replace a file with new data is a moderately common
task, perhaps it would be useful to have a convenience class that looks like
a file, but takes care of the ugly details behind-the-scenes?  Something
vaguely like this flawed and untested class:

class open_for_safe_replacement(file): # needs a better name
    def __init__(self, path, flags):
        if 'w' not in flags:
            raise RuntimeError, 'Writing without writing?'
        self.path = path
        self.tmp_name =
some_function_that_generates_a_safe_temporary_filename() # good luck
        file.__init__(self.tmp_name, flags)

    def close(self):
        self.flush()
        os.fsync(self.fileno())
        self.close()
        os.rename(self.tmp_name, self.path) # won't work on Windows :-(

then we could simply:

with appropriate_module.open_for_safe_replacement('mysavefile', 'w'):
    f.write(data)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/9485fa1c/attachment.htm>