Permission denied when opening a file that was created concurrently by os.rename (Windows)

[Alexey Izbyshev]

Hello!

I've hit a strange problem that I reduced to the following test case:
* Run several python processes in parallel that spin in the following 
loop:
while True:
   if os.path.isfile(fname):
     with open(fname, 'rb') as f:
       f.read()
     break
* Then, run another process that creates a temporary file and then 
renames it to the name than other processes are expecting
* Now, some of the reading processes occasionally fail with "Permission 
denied" OSError

I was able to reproduce it on two Windows 7 64-bit machines. It seems 
when the file appears on the filesystem it is still unavailable to 
reading, but I have no idea how it can happen. Both source and 
destination files are in the same directory, and the destination doesn't 
exist before calling os.rename. Everything I could find indicates that 
os.rename should be atomic under this conditions even on Windows, so 
nobody should be able to observe the destination in unaccessible state.

I know that I can workaround this problem by removing useless 
os.path.isfile() check and wrapping open() with try-except, but I'd like 
to know the root cause of the problem. Please share you thoughts.

The test case is attached, the main file is test.bat. Python is 
expected to be in PATH. Stderr of readers is redirected to *.log. You 
may need to run several times to hit the issue.

Alexey Izbyshev,
research assistant,
ISP RAS