Why you sometimes get Windows Permission Denied errors

[Chris Tavares]

"Colin Brown" <cbrown at metservice.com> wrote in message
news:3cc88d2f$1 at news.nz.asiaonline.net...
> Hi
>
> I have been struggling to solve why an occasional "Permission Denied"
error
> popped up from the following code fragment in one of my Win2K program
> threads:
>
> ...
> input_file = preprocess(raw_file)
> os.system('third_party input_file output_file > error_file')
> if os.path.exists(saved_file):
>     os.remove(saved_file)
> os.rename(input_file,saved_file)
> ...
>
> The error occurs on the os.rename(). The third_party executable was
closing
> input_file before terminating and anyway the subshell process has finished
> before the os.rename is called! Most baffling.
>
> With the aid of handle.exe from www.sysinternals.com I finally resolved
the
> problem. To see the problem at first hand try the following script:
>
> import time, thread, os
>
> def rm(file):
>     print 'delete x.x'
>     os.remove(file)
>
> def wait():
>     if os.name == 'nt':
>         os.system('pause')
>     elif os.name == 'posix':
>         os.system('sleep 3')
>     print 'end wait'
>
> print 'create x.x'
> f = open('x.x','w')
> thread.start_new_thread(wait,())
> time.sleep(1)
> print '\nclose x.x'
> f.close()
> rm('x.x')
>
> Although this works fine on Linux, I get a Permission Denied error on
> Windows. I surmise that an os.system call uses a C fork to create the
> subshell  - an exact duplicate of the current process environment
including
> OPEN FILE HANDLES. Because the os.system call has not returned before the
> os.remove is called a "Permission Denied" error occurs. Quite simple
really.
>

Not quite what's happening - Windows doesn't do fork. CreateProcess (the
underlying system call) will start a new process, but it doesn't do the
virtual memory copy thing that Unix does on a fork. However, there is
something called file handle inheritance, so that could be what's happening
here.

> Now going back to my original problem, I had other threads doing os.system
> calls. When one of these calls happens during the preprocess file write of
> my above thread AND takes longer to complete than the os.system call in
the
> above thread then the file will still be open and the os.rename will
return
> the Permission Denied error.
>
> What can be done to prevent the problem? I am open to suggestions here. My
> solution was to use a threading.Semaphore() lock around all my file
> operations to ensure they could only happen sequentially and also around
my
> os.system calls as below:
>

[... snip ... ]

Hmm... one possible solution would be to use win32process.CreateProcess
(part of the win32 extensions) instead of os.system. Depending on how
complex your command lines are (do you do a lot of io redirection?) that may
be either trivial or very difficult. That way you can make sure handles
don't get inherited, and you won't have this problem.

I suspect that the underlying C library system call just isn't set up to be
called in this manner - os.system isn't the Windows Way.

-Chris