[issue4944] os.fsync() doesn't work as expect in Windows

[Javen Wang]

New submission from Javen Wang <javen72 at gmail.com>:

I encountered a very strange issue in file flush operation in Windows.
Here's the scenario of my application:

    1. The main thread of my application will create makefiles sequentially.
    2. Once a makefile is generated, launch a separate process calling
nmake.exe to run it in parallell. The main thread then create another
makefile until no more makefiles to create.
    3. The number of new processes is limited by command line options.

My application has been running for almost a year without any problem.
But, after I made some changes recently to the content of makefile
generated, "nmake.exe" in separate process sometimes complains that
the makefile was not found. But when I went into the directory, the
makefile was there.

Because I didn't change any thing related to file creation and the new
makefiles are a little bit less than before, I guessed that the
makefile just created hasn't been flushed to disk because of size
change so the new process could not see it in a short time.

So I decided add code to force flush file buffer after writing the
file content (I didn't flush the file explicitly before). I did it
like below:

    Fd = open(File, "w")
    Fd.write(Content)
    Fd.flush()
    os.fsync(Fd.fileno())
    Fd.close()

The strangest thing happened. The "makefile" missing happened more
frequently than no flush operation. I searched the web but no answer
there.

Finally I tried to use Windows file API to create the file via pywin32
extension. The problem's gone.

        import win32file
        Fd = win32file.CreateFile(
                        File,
                        win32file.GENERIC_WRITE,
                        0,
                        None,
                        win32file.CREATE_ALWAYS,
                        win32file.FILE_ATTRIBUTE_NORMAL,
                        None
                        )
        win32file.WriteFile(Fd, str(Content), None)
        win32file.FlushFileBuffers(Fd)
        win32file.CloseHandle(Fd)

I tried writing small python extension in C to make use Windows API to
create file like above. It also works well, even I removed the
FlushFileBuffers() calling.

I think that there's a bug in Python file buffer mechanism.

----------
components: Windows
messages: 79829
nosy: javen72
severity: normal
status: open
title: os.fsync() doesn't work as expect in Windows
type: behavior
versions: Python 2.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4944>
_______________________________________