pip hangs after successful operation

[Ryan Johnson]

Hello, I am seeking some quick help, and probably am reporting bugs along the way. I apologize that this is a long email. Please let me know what I should do in the future.

On Windows, pip hangs and does not install packages in the proper location (or perhaps, at all), even if pip claims it installed successfully. Upon hanging, I can use Ctrl+C to terminate individual processes that have succeeded, and the results show up in “pip list”, but this bug affects any packages that call pip within their install script, which makes this a critical bug, since the package install scripts will never complete.

Filed bug report here: https://github.com/pypa/pip/issues/5850

My environment:
➢ Python 3.7.0 x64
➢ pip 18.0
➢ Windows 10 Pro x64

Steps taken to resolve, so far:
➢ Repaired Installation.
➢ Removed and Reinstalled Installation.
➢ Rebooted computer as last resort.

This appears to be a bug similar to https://github.com/pypa/pip/issues/4588 

However, unlike in that bug report, pip hangs after all of these commands:
• pip list
• pip install
• pip install --upgrade pip

However, when called using “py -m pip”, these commands do not hang, and, upon inspection of the package list, via “pip list” and “py -m pip list” methods, there is a discrepancy. This is the culprit, but I don’t know how to fix the problem. Output:

• PS C:\Users\Ryan> py -V
Python 3.7.0

• PS C:\Users\Ryan> pip list
Package     Version
----------- -------
Click       7.0
mysqlclient 1.3.13
pip         18.0
pip-tools   3.0.0
setuptools  40.4.3
six         1.11.0

• PS C:\Users\Ryan> py -m pip list
Package    Version
---------- -------
pip        18.0
setuptools 40.4.3

Notice the package lists are different. The outputs above are from the exact same pip executable:

• PS C:\Users\Ryan> pip -V
pip 18.0 from c:\users\ryan\appdata\local\programs\python\python37\lib\site-packages\pip (python 3.7)

• PS C:\Users\Ryan> py -m pip -V
pip 18.0 from C:\Users\Ryan\AppData\Local\Programs\Python\Python37\lib\site-packages\pip (python 3.7)

I have another machine running Python 3.7.0 x64 that does not experience this issue.

Both machines have the same set-up:
➢ Python 3.7.0 from x64 installer (local user install, not All Users install, with PATH option checked. pip runs directly in terminal.)
➢ Python 3.6.6 from Visual Studio 2017 installer (pip does not run unless called from py -m pip or py -3.6 m pip)
➢ Anaconda from installer (not used yet)
➢ MySQL Connector Python 8.0.12 from MySQL Installer (only installs for Python 3.6.6, not compatible with Python 3.7.0)
➢ Cygwin
➢ MinGW/MSYS

I have exhausted all online help documentation, and none of the bug reports contain answers that explain why one computer fails while the other does not.

Further testing reveals more bugs:

This bug does not affect the older Python 3.6.6 pip on either machine:

• py -3.6 -m pip -V
pip 10.0.1 from < blah path to Python36_64 package dir >

• py -3.6 -m pip install matplotlib
MySQL-connector-python 8.0.12 requires protobuf>=3.0.0, which is not installed.

The installation [also] fails, but at least it doesn’t hang.
However it is not possible to install protobuf, as this also fails:

• py -3.6 -m pip install protobuf
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: C:\\ ….. \\six.py

And it cannot upgrade:

• py -3.6 -m pip install --upgrade pip
Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: ‘c:\\program...\\...\\pip-10.0.1.dist-info\\entry_points.txt’
Consider using the `--user` option or check the permissions.

Used the suggestion, which looked more like a warning:

• Py -3.6 -m pip install --upgrade pip --user
Successfully installed

Permission denied is a bad excuse to fail, when a command line argument can change this. I do not like adding command line switches to something that should work without them (since I specifically call this user-specific resource in the command statement), because there is no telling what the application will do from a black-box engineering viewpoint. This error message / hint should be more elaborate, explaining what the switch does, as it is counterintuitive for anyone familiar with Linux to believe for one second that “--user” is supposed get around a “permission denied” problem by elevating permissions, after years of telling users they need to use “sudo” to run something as root. This is an advantage for the ignorant coder. If ignorance is an advantage, then that just means the error message is not explanative enough. Requiring users to glean the meaning of “—user” from the “—help” option wastes time and mental resources by adding another layer of complexity to the installation process. It is not obvious that there is a “–help” option for “pip install” (hence the waste of time).

Other bug encountered recently:

Also, there is a bug that I reported approximately a week ago with the mysqlclient package installer, in which an install script for MySQL Connector-C chokes on its own assumptions about where the installation is:

I worked around it by using Link Shell Extension to create an NTFS junction:
       
       From “C:\Program Files\MySQL\MySQL Connector C 6.1” (actual install)
       To “C:\Program Files (x86)\MySQL\MySQL Connector C 6.1” (where script believes install folder is)

The sheer number of bugs and gotchas involving pip for a newcomer to have to deal with is inexcusable, in light of the fact that creating an executable of a python script for deployment on client computers is not straightforward either. The purpose of agile coding languages is to trade off deployment advantages for development speed, but these obstacles in pip absolutely kill this intended benefit of Python. If someone can share an up-to-date resource for making executables, I would appreciate it.

I am single-handedly developing an inventory management solution for a business with no prior Python experience, and these problems are not trivial. They have taken several days to discover the causes and solutions, and every time I fix one, a new one pops up.

Final Straw:

Mysqlclient install exits with errors:

• PS C:\Users\Ryan> py -m pip install mysqlclient --user
o Collecting mysqlclient
o   Using cached https://files.pythonhosted.org/packages/ec/fd/83329b9d3e14f7344d1cb31f128e6dbba70c5975c9e57896815dbb1988ad/mysqlclient-1.3.13.tar.gz
o     Error [WinError 87] The parameter is incorrect while executing command python setup.py egg_info
o Could not install packages due to an EnvironmentError: [WinError 87] The parameter is incorrect

This package correctly installed on my original machine with “pip install mysqlclient”, after linking the MySQL Connector C.

Installing mysqlclient on Python 3.6.6 does not work, either:

• PS C:\Users\Ryan> py -3.6 -m pip install mysqlclient --user
o Collecting mysqlclient
o   Using cached https://files.pythonhosted.org/packages/ec/fd/83329b9d3e14f7344d1cb31f128e6dbba70c5975c9e57896815dbb1988ad/mysqlclient-1.3.13.tar.gz
o     Complete output from command python setup.py egg_info:
o 
o     ----------------------------------------
o Command "python setup.py egg_info" failed with error code 1 in C:\Users\Ryan\AppData\Local\Temp\pip-install-rfj6k92m\mysqlclient\

However, as shown at the beginning of my email, these packages show up in “pip list”, as though the install succeeded, but not in “py -m pip list” or “py -3.6 -m pip list”.

Thanks,
Ryan Johnson


Sent from Mail for Windows 10