[Distutils] Reviving PEP 470 - Removing External Hosting support on PyPI

[Donald Stufft]

On August 27, 2015 at 4:26:28 AM, Antoine Pitrou (solipsis at pitrou.net) wrote:
>  
> Out of curiosity, have you tried to determine if those Unsafely Off
> PyPI projects were either still active or "popular" ?
>  
>

10 Months ago I attempted to figure out how popular or active those projects were, I didn’t redo those numbers but I can if people think it should be in the PEP. I felt the previous versions of the PEP were a bit too much of “well if you look at the data this way you get X but if you look at it this way you get Y” and I tried to narrow it down to just a single measure of impact that took the broadest interpretation of what impact would mean.

Anyways, 10 months ago I parsed the log files for a single day and I looked how often the /simple/foo/ pages got hit for every project on PyPI. This isn’t a great metric since people running an install on multiple machines or using something like tox to run it multiple times on the same machine will be counted multiple times, however I got data that looked like this:

Top Externally Hosted Projects by Requests
-------------------------------------------

This is determined by looking at the number of requests the
``/simple/<project>/`` page had gotten in a single day. The total number of
requests during that day was 10,623,831.

============================== ========
Project                        Requests
============================== ========
PIL                            63869
Pygame                         2681
mysql-connector-python         1562
pyodbc                         724
elementtree                    635
salesforce-python-toolkit      316
wxPython                       295
PyXML                          251
RBTools                        235
python-graph-core              123
cElementTree                   121
============================== ========


Top Externally Hosted Projects by Unique IPs
--------------------------------------------

This is determined by looking at the IP addresses of requests the
``/simple/<project>/`` page had gotten in a single day. The total number of
unique IP addresses during that day was 124,604.

============================== ==========
Project                        Unique IPs
============================== ==========
PIL                            4553
mysql-connector-python         462
Pygame                         202
pyodbc                         181
elementtree                    166
wxPython                       126
RBTools                        114
PyXML                          87
salesforce-python-toolkit      76
pyDes                          76
============================== ==========

I don’t know what those number look like today, but I suspect that they’d be even more weighted towards PIL being the primary project that actual users are pulling down that isn’t hosted on PyPI.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA