[issue1425127] os.remove OSError: [Errno 13] Permission denied

[Josh Rosenberg]

Josh Rosenberg added the comment:

Similar reference regarding the same basic behavior: http://blogs.msdn.com/b/oldnewthing/archive/2012/09/07/10347136.aspx

Short version: Indexing and anti-virus tools prevent deletion from occurring.

Longer version: 
DeleteFile (and all the stuff that ultimately devolves to DeleteFile) operate in a funny way on Windows. Internally, it opens a HANDLE to the file, marks it as pending deletion, and closes the HANDLE. If no one snuck in and grabbed another HANDLE to the file during that time, then the file is deleted when DeleteFile's hidden HANDLE is closed. Well designed anti-virus/indexing tools use oplocks ( http://blogs.msdn.com/b/oldnewthing/archive/2013/04/15/10410965.aspx ) so they can open a file, but seamlessly get out of the way if a normal process needs to take exclusive control of a file or delete it. Sadly "well-designed" is not a term usually associated with anti-virus tools, so errors like this are relatively commonplace.

Workarounds like using GetTempFileName() and MoveFile() to move the file out of the way will work, though I believe they introduce their own race conditions (the temp file itself is created but the HANDLE is closed immediately, which could mean a race to open the empty file by the bad anti-virus that would block MoveFile()).

Basically, if you're running on Windows, and you're using unfriendly anti-virus/indexing tools, there is no clean workaround that maintains the same behavior. You can't keep creating and deleting a file of the same name over and over without risking access denied errors.

That said, you could probably get the same results by opening and closing the file only once. Change from the original pseudocode:

while 1:
    with open(myfilename, ...) as myfile:
        myfile.write(...)
    do_stuff_with(myfilename)
    os.remove(myfilename)

to (assuming the default file sharing permissions are amenable):

with open(myfilename, ...) as myfile:
    while 1:
        myfile.write(...)
        myfile.flush()
        myfile.seek(0)
        do_stuff_with(myfilename)
        myfile.truncate()

Same basic pattern, except this time, you're rebuilding the same file over and over without ever leaving it unowned long enough for anti-virus/indexing to swoop in and steal it from you.

----------
nosy: +josh.rosenberg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1425127>
_______________________________________