Remove directory tree without following symlinks

[Albert-Jan Roskam]

> From: eryksun at gmail.com
> Date: Sat, 23 Apr 2016 15:22:35 -0500
> Subject: Re: Remove directory tree without following symlinks
> To: python-list at python.org
> 
> On Sat, Apr 23, 2016 at 4:34 AM, Albert-Jan Roskam
>  wrote:
>>
>>> From: eryksun at gmail.com
>>> Date: Fri, 22 Apr 2016 13:28:01 -0500
>>> On Fri, Apr 22, 2016 at 12:39 PM, Albert-Jan Roskam
>>>  wrote:
>>>> FYI, Just today I found out that shutil.rmtree raises a WindowsError if
>>>> the dir is read-only (or its contents). Using 'ignore_errors', won't help.
>>>> Sure, no error is raised, but the dir is not deleted either! A 'force' option
>>>> would be a nice improvement.
>>>
>>> Use the onerror handler to call os.chmod(path, stat.S_IWRITE). For
>>> example, see pip's rmtree_errorhandler:
>>>
>>> https://github.com/pypa/pip/blob/8.1.1/pip/utils/__init__.py#L105
>>
>> Thanks, that looks useful indeed. I thought about os.chmod, but with
>> os.walk. That seemed expensive. So I used subprocess.call('rmdir "%s" /s /q'
>> % dirname). That's Windows only, of course, but aside of that, is using
>> subprocess less preferable?
> 
> I assume you used shell=True in the above call, and not an external
> rmdir.exe. There are security concerns with using the shell if you're
> not in complete control of the command line.
> 
> As to performance, cmd's rmdir wins without question, not only because
> it's implemented in C, but also because it uses the stat data from the
> WIN32_FIND_DATA returned by FindFirstFile/FindNextFile to check for
> FILE_ATTRIBUTE_DIRECTORY and FILE_ATTRIBUTE_READONLY.
> 
> On the other hand, Python wins when it comes to working with deeply
> nested directories. Paths in cmd are limited to MAX_PATH characters.
> rmdir uses DOS 8.3 short names (i.e. cAlternateFileName in
> WIN32_FIND_DATA), but that could still exceed MAX_PATH for a deeply
> nested tree, or the volume may not even have 8.3 DOS filenames.
> shutil.rmtree allows you to work around the DOS limit by prefixing the
> path with "\\?\". For example:
> 
>    >>> subprocess.call(r'rmdir /q/s Z:\Temp\long', shell=True)
>     The path Z:\Temp\long\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>     aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>     aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>     aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>     aaaaa is too long.
>     0
> 
>    >>> shutil.rmtree(r'\\?\Z:\Temp\long')
>    >>> os.path.exists(r'Z:\Temp\long')
>     False
> 
> Using "\\?\" requires a path that's fully qualified, normalized
> (backslash only), and unicode (i.e. decode a Python 2 str).

Aww, I kinda forgot about that already, but I came across this last year [1]. Apparently, 
shutil.rmtree(very_long_path) failed under Win 7, even with the "silly prefix". I believe very_long_path was a Python2-str.
It seems useful if shutil or os.path would automatically prefix paths with "\\?\". It is rarely really needed, though.
(in my case it was needed to copy a bunch of MS Outlook .msg files, which automatically get the subject line as the filename, and perhaps
the first sentence of the mail of the mail has no subject).

[1] https://mail.python.org/pipermail/python-list/2015-June/693156.html