When will os.remove fail?

[Steve D'Aprano]

On Mon, 13 Mar 2017 08:47 pm, eryk sun wrote:

> On Sun, Mar 12, 2017 at 5:48 PM, Steve D'Aprano
> <steve+python at pearwood.info> wrote:
>>
>> Does os.remove work like this under Windows too?
> 
> os.remove calls DeleteFile on Windows. 
[...]

Thanks for the unexpectedly detailed explanation! A few follow-up questions:


> One hurdle to getting delete access is the sharing mode. If there are
> existing File objects that reference the file, they all have to share
> delete access. Otherwise the open fails with a sharing violation. This
> is often the show stopper because the C runtime (and thus CPython,
> usually) opens files with read and write sharing but not delete
> sharing.

This is the famous "can't delete a file which is open under Windows"
problem, am I right?

I take it that you *can* delete open files, but only if the process that
opens them takes special care to use "delete sharing". Is that correct?

I don't have a machine to test this on, but I'd like to deal with this
situation in my (cross-platform) code. If I have one Python script do this:


with open("My Documents/foo") as f:
    time.sleep(100000)


and while it is sleeping another script does this:

os.remove("My Documents/foo")


what exception will I get? Is that unique to this situation, or is a generic
exception that could mean anything?

My aim is to do:

try:
    os.remove(thefile)
except SomeError:
    # Could be a virus checker or other transient process.
    time.sleep(0.2)
    os.remove(thefile)  # try again


Does that seem reasonable to you, as a Windows user?


[...]
> OK, that covers the basics.

The basics, he says :-)


> In user mode, a kernel object such as a File instance is referenced as
> a handle. 

Out of curiosity, I only know the term "handle" from classic Macintosh
(pre-OS X) where a handle was a managed pointer to a pointer to a chunk of
memory. Being managed, the OS could move the memory around without the
handles ending up pointing to garbage. Is that the same meaning in Windows
land?


[...]
> When the delete disposition is set, no new File references can be
> instantiated (i.e. any level of access is denied, even just to read
> the file attributes), but the file isn't immediately unlinked. It's
> still listed in the parent directory. Any existing File reference that
> has delete access can unset the delete disposition.
> 
> When all handle and pointer references to a File object are closed,
> the file system decrements the reference count on the FCB. When the
> FCB reference count drops to 0, if the delete disposition is currently
> set, then finally the file is unlinked from the parent directory.

So technically file deletions aren't atomic in Windows? In principle, I
could say:

delete file X

which then returns immediately, and if I try to open(X) it will fail. But I
can still see it if I do a dir() on the parent directory?

Eventually the last reference to X will go away, and then it is unlinked.
What happens if I pull the plug in the meantime? Will the file magically
come back on rebooting?


[...]
> Finally, I'm sure most people are familiar with the read-only file
> attribute. If this attribute is set you can still open a file with
> delete access to rename it, but setting the delete disposition will
> fail with access denied.

That was actually the situation I was thinking about when I started on this
question. From Python code, I was considering writing something like this:


def delete_readonly(thefile):
    try:
        os.remove(thefile)
    except ReadOnlyFileError:  # what is this really?
        try:
            # change the user permissions to unset Read-Only
            old_perms = ... ?
            set_perms(...)
        except OSError:
            pass
        else:
            # We changed Read-Only. Try deleting again. If it fails, 
            # revert the flags and fail.
            try:
                os.remove(thefile)
            except OSError:
                # Restore permissions
                ...
            else:
                # Success on the second attempt!
                return
        raise



That's intended to emulate the behaviour on Unix where deleting a file that
you own will succeed even if you don't have write access to the file.

(The bash rm command will ask you before deleting, but Python's os.remove
just removes it.)


Does this seem reasonable? Or overly complicated for not enough benefit?





-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.