[issue15723] Python breaks OS' append guarantee on file writes

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Even if we write in chunks, if we are calling the OS write function
> and O_APPEND is set, wouldn't be satisfying the condition?  Or,
> rather, the OS would be.  That is, I don't really see a guarantee of
> an *atomic* write in the quoted description.

I'm not sure it's guaranteed to be atomic at the hardware level, but as
AFAIU the updates should be atomic as seen from other processes on the
same machine (i.e. filesystem cache coherency).

As a side-note, I've just tested under Linux with the following script:

    with open("foo", "ab") as f:
        f.write(b"abcd")
        f.write(b"x" * (1024 ** 2))

Results:

- on 2.7, the write buffers get sliced up (the glibc's fwrite() doesn't
care about atomicity):
write(3, "abcdxxxxxxxxxxxxxxxxxxxxxxxxxxxx"..., 4096) = 4096
write(3, "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"..., 1044480) = 1044480

- on 3.2 and 3.3, our home-grown buffering respects the original
buffers:
write(3, "abcd", 4)                     = 4
write(3, "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"..., 1048576) = 1048576

(but that's purely out of luck, since we didn't design it with that
goal :-))

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15723>
_______________________________________