[Python-ideas] `to_file()` method for strings

[Nathaniel Smith]

On Wed, Mar 23, 2016 at 8:10 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, Mar 23, 2016 at 10:14:07AM -0700, Nathaniel Smith wrote:
>
>> POSIX doesn't guarantee anything whatsoever over power loss. Individual
>> filesystem implementations make somewhat stronger guarantees, but it's a
>> mess:
>>   http://danluu.com/file-consistency/
> [...]
>> I don't want to get sucked into a long debate about this; it's entirely
>> likely that adding something like that original recipe to the stdlib would
>> be an improvement, so long as it had *very* detailed docs explaining the
>> exact tradeoffs made. All I want to do is raise a cautionary flag that such
>> an effort would need to tread carefully :-)
>
> And then there are file media which lie to you, and return from a fsync
> before actually syncing, because that makes their benchmarks look good.
> I've seen file corruption on USB sticks that do this, including some
> otherwise "respectable" brands. Worst case I ever saw was a USB stick
> that (fortunately) had a blinking light to show when it was writing. It
> continuing writing for *eight minutes* (I timed it) after returning from
> fsync and the OS had unmounted the stick and it was nominally safe to
> remove.
>
> I'm told that some hard drives will do the same, although I've never
> knowingly seen it.

Not just USB sticks and hard drives -- whole operating systems too.
Ones you might have heard of! Like... OS X.

Their fsync() syscall is intentionally designed and documented to
provide absolutely zero guarantees of anything. (From the man page:
"[after calling fsync], if the drive loses power or the OS crashes,
the application may find that only some or none of their data was
written [...] This is not a theoretical edge case.  This scenario is
easily reproduced with real world workloads".) A real fsync call is
still available, it's just hidden in fcntl.

While we're speaking of it, Python's os.fsync on OS X should probably
call that fcntl instead of fsync. Right now os.fsync is effectively a
no-op on OS X.

(Apparently this is https://bugs.python.org/issue11877 but that issue
seems to have petered out years ago in confusion.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org