[OT] Lying hard drives [was Re: shutil.copyfile is incomplete (truncated)]

[Chris Angelico]

On Sat, Apr 13, 2013 at 1:17 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Sat, 13 Apr 2013 03:33:29 +0100, Nobody wrote:
>> If you want to wait for the data written to be written to the physical
>> disk (in order to obtain specific behaviour with respect to an unclean
>> shutdown), use f.flush() followed by os.fsync(f.fileno()).
>
> If only it were that simple. It has been documented that some disks will
> lie, even when told to sync. When I say "some", I mean *most*. There's
> probably nothing you can do about it, apart from not using that model or
> brand of disk, so you have to just live with the risk.

It's often close to that simple. With most hard disks, you can make
them 100% reliable, but you may have to check some disk parameters (on
Linux, that's just a matter of writing to something in /proc
somewhere, don't remember the details but it's easy to check). The
worst offenders I've met are SSDs...

> USB sticks are especially nasty. I've got quite a few USB thumb drives
> where the "write" light keeps flickering for anything up to five minutes
> after the OS reports that the drive has been unmounted and is safe to
> unplug. I corrupted the data on these quite a few times until I noticed
> the light. And let's not even mention the drives that have no light at
> all...

... but you've met worse.

> But my favourite example of lying hard drives of all time is this:
>
> http://blog.jitbit.com/2011/04/chinese-magic-drive.html
>
> I want one of those!

Awesome! It's the new version of DoubleSpace / DriveSpace!

http://en.wikipedia.org/wiki/DriveSpace

(And its problems, according to that Wikipedia article, actually had
the same root cause - write caching that the user wasn't aware of.
Great.)

ChrisA