[Catalog-sig] Pypi cdn for hosted packages

[Noah Kantrowitz]

Reponding from my phone quickly before this gets any further, will write more later. Plan is to have pypi move package download links to a new hostname (probably pypi-download.python.org) and then throw that behind fastly. This sidesteps 100% of issues with dynamic pages, etc. Simple index with be handled secondarily.

Jim Fulton <jim at zope.com> wrote:

>On Thu, Feb 28, 2013 at 5:39 AM, Jesse Noller <jnoller at gmail.com>
>wrote:
>> Thread fork.
>>
>> Anyway. I know we have at least 1 major rep of a cloud provider on
>the list, and I have at least one off in my pocket.
>>
>> I'd like to start discussing (completely ignoring past efforts and
>discussion which got bogged down) how we can start distributing the
>package data we host via CDN rather than the mirroring system.
>
>Woohoo!!!
>
>One issue is how to keep the relatively dynamic stuff up to date.
>One option is to bypass the CDN for dynamic content, but a better
>approach might be to use CDN-provided invalidation APIs.
>
>I'm mainly thinking of the simple index pages, which are relatively
>static, but, when they change, you want to get the change quickly.
>Ideally, when there's an update to package, PyPI would
>invalidate the index page.  Everything should have a really long
>cache interval.
>
>BTW, we use CloudFront fairly extensively for media.  We get a lot
>more origin requests than we expected, given the cache intervals
>we'd set.  (It was still a big win for us.)
>
>Jim
>
>-- 
>Jim Fulton
>http://www.linkedin.com/in/jimfulton
>Jerky is better than bacon! http://zo.pe/Kqm
>_______________________________________________
>Catalog-SIG mailing list
>Catalog-SIG at python.org
>http://mail.python.org/mailman/listinfo/catalog-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130228/675c08b5/attachment.html>