[Distutils] Fixing PyPI download stats with real-time log analysis (Was: PyPI Download Counts)

[Donald Stufft]

On Jun 23, 2013, at 9:36 AM, Alex Clark <aclark at aclark.net> wrote:

> Hi Noah,
> 
> 
> Noah Kantrowitz <noah <at> coderanger.net> writes:
> 
>> 
>> 
>> On Jun 22, 2013, at 10:33 PM, anatoly techtonik wrote:
>> 
>>> On Fri, Jun 14, 2013 at 5:05 PM, anatoly techtonik <techtonik <at>
> gmail.com> wrote:
>>> Could you, please, share the log format+example, so that we can
> experiment with it?
>>> 
>>> ping
>> 
>> Additional assistance is not required for this project. Thank you for your
> interest, and stay tuned for
>> future updates.
> 
> 
> WAT.
> 
> 
> Can you explain why "additional assistance is not required for this
> project"? AFAICT:
> 
> - Shit is broken (i.e. download counts)
> - At first, there was no plans to fix it (which I accepted as a cost to
> enable the CDN)
> - Now there are plans to fix it, but no additional assistance is needed? If
> that were the case I'd expect download counts to be fixed. Or at the very
> least, I'd expect a "no thanks we don't need help, but here is where we are
> at [ explanation of situation ]". Not "we don't need help, we'll be in
> touch". I think the latter is a disservice to the community, which (I
> assume) you especially don't want to be responsible for given the
> tremendously positive service you've done by enabling the CDN in the first
> place.)
> 
> 
> We get it: download counts were not as important as CDN. But they *are*
> important, so let's keep talking about how to fix them.

https://en.wikipedia.org/wiki/Brooks%27_law

To further expand. Right now the biggest issue preventing the CDN download counts is a less than ideal configuration on the VM hosting PyPI. I've been working on (and almost have completed) a Chef cookbook that will allow us to easily deploy PyPI to a second VM (and gracefully do the switch over) with minimal downtime.

However there is an issue with launching VM's at the moment that OSUOL is looking into which means even with a completed cookbook we couldn't launch a VM at the moment anyways. If we wanted to correct the configuration in place it would require downtime.

> 
> 
> Thank you,
> 
> 
> Alex
> 
> 
> 
>> 
>> --Noah
>> 
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG <at> python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>> 
>> 
> 
> 
> 
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130623/11b5fe44/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130623/11b5fe44/attachment.pgp>