Why you sometimes get Windows Permission Denied errors

[Colin Brown]

Hi

I have been struggling to solve why an occasional "Permission Denied" error
popped up from the following code fragment in one of my Win2K program
threads:

...
input_file = preprocess(raw_file)
os.system('third_party input_file output_file > error_file')
if os.path.exists(saved_file):
    os.remove(saved_file)
os.rename(input_file,saved_file)
...

The error occurs on the os.rename(). The third_party executable was closing
input_file before terminating and anyway the subshell process has finished
before the os.rename is called! Most baffling.

With the aid of handle.exe from www.sysinternals.com I finally resolved the
problem. To see the problem at first hand try the following script:

import time, thread, os

def rm(file):
    print 'delete x.x'
    os.remove(file)

def wait():
    if os.name == 'nt':
        os.system('pause')
    elif os.name == 'posix':
        os.system('sleep 3')
    print 'end wait'

print 'create x.x'
f = open('x.x','w')
thread.start_new_thread(wait,())
time.sleep(1)
print '\nclose x.x'
f.close()
rm('x.x')

Although this works fine on Linux, I get a Permission Denied error on
Windows. I surmise that an os.system call uses a C fork to create the
subshell  - an exact duplicate of the current process environment including
OPEN FILE HANDLES. Because the os.system call has not returned before the
os.remove is called a "Permission Denied" error occurs. Quite simple really.

Now going back to my original problem, I had other threads doing os.system
calls. When one of these calls happens during the preprocess file write of
my above thread AND takes longer to complete than the os.system call in the
above thread then the file will still be open and the os.rename will return
the Permission Denied error.

What can be done to prevent the problem? I am open to suggestions here. My
solution was to use a threading.Semaphore() lock around all my file
operations to ensure they could only happen sequentially and also around my
os.system calls as below:

mainline>
lock=threading.Semaphore()

threads>
lock.acquire()
try:
    file_operation()
finally:
    lock.release()

subprocess_call>

def child(cmd):
    os.system(cmd)

...
lock.acquire()
try:
    thread.start_new_thread(child,(cmd,))
    time.sleep(1)
finally:
    lock.release()
...

This is definitely not elegant. If you know that your subprocess call will
always finish quickly then maybe you could just trap the error, delay and
retry until the operation suceeds.

I hope this has been helpful.

Colin Brown
PyNZ