Pip upgrade causing issues in 3.10

[Eryk Sun]

On 7/20/22, Mike Dewhirst <miked at dewhirst.com.au> wrote:
> On 20/07/2022 4:43 am, David Raymond wrote:
>> C:\Program Files\Python310\Scripts>..\python.exe -m pip install --upgrade
>> pip
>> ERROR: Could not install packages due to an OSError: [WinError 32] The
>> process cannot access the file because it is being used by another
>> process: 'c:\\program files\\python310\\scripts\\'
> There's your problem. The 'other' process is your cmd.exe within which
> you are typing etc.
>
> Python scripts dir should be on the path so you don't have to execute
> anything from within it. Windows is obviously tripping over its own toes
> trying to delete and install something in the same dir while you also
> have your foot in it.

This should only occur if uninstalling pip deletes all of the files in
the "Scripts" directory. In this case, pip will 'compress' the
individual delete operations to remove the entire directory. It begins
by trying to 'stash' the "Scripts" directory, i.e. by renaming it to a
temporary adjacent name. It uses shutil.move() for this, which tries
os.rename() and falls back on shutil.copytree() and shutil.rmtree().

Of course this fails with a sharing violation if the directory is open
as the working directory in any process, including the current Python
process, because an open for a working directory doesn't share
delete/rename access. pip fails to handle this error. It crashes and
leaves a mess. The empty "Scripts" directory and the copytree() copy
aren't rolled back properly, and neither are the stashed (renamed)
"pip" and "pip*dist-info" directories in site packages.

This is not user error. It is a bug in pip. It should be able to
recover gracefully from failing to delete the directory. Moreover, it
shouldn't even have to roll back the operations if it deleted the
directory due to compression of explicit removals from the install
"RECORD". In particular there is no reason to delete the "Scripts"
directory, even if it's left empty.