strange file.write() behavior on windows: $ConvertToNonresident, $ReplaceAttribute2

[Fredrik Lundh]

"welch" <welch.quietplease at gmail.com> wrote:

> while taking some rough disk performance measures on windows machines,
> and snooping with FileMon, i've noticed some odd behavior

> sometimes, depending on the phase of the moon, instead of
> $ReplaceAttribute2 the write goes to $ConvertToNonresident. within a
> single run of writeTest the write always goes to the same one,
> whichever one that is. these runs are always slower (sometimes greatly)
> than writing the same with 64K blocks, even though many fewer
> file.write()'s are being issued because of the larger block size.
>
> finally, where you'd expect an even 10 of the WRITE
> C:\$ReplaceAttribute2 lines per WRITE C:\writeTest.out lines in the
> example above, instead FileMon reports 8 lines for the first, 6 for the
> second, 8 for the third, 6 for the fourth, etc... i've no idea if this
> means FileMon is missing messages, but this pattern is absolutely
> consistent across every run i've made, on both xp and win2k3 server
> machines.
>
> a look at the python fileobject.c source shed no light for me, anyone
> know what could be going on? the "equivalent" c version of writeTest
> using fwrite() shows a succession of 1K blocks writing to the named
> file (even when fwrite is given 64K blocks), and no mysterious
> $ReplaceAttribute2 lines

you're seeing activities by the Windows caching system (c:\$ stuff is various
NTFS data streams).  unless you're writing low-level drivers, you don't really
have to care about what it does, and when it does what...

</F>