[Catalog-sig] Rewrite PyPI for App Engine?

[Noah Kantrowitz]

On Jun 25, 2010, at 12:21 AM, Antoine Pitrou wrote:

> Almir Karic <almir <at> almirkaric.com> writes:
>> 
>> i would like to help out with the move.
>> 
>> is anyone actually opposed to moving to GAE (either moving the current
>> code base or re-write, whichever seems more appropriate)?
> 
> As I already said, I don't think it's reasonable to do it without first getting
> the community's (and the PSF's) agreement that a vital Python infrastructure can
> be managed under a proprietary API, platform and datastore.
> 
> I would myself be strongly opposed to such a move.
> 
> (and I don't even get the point, technically, of wanting to use GAE, which seems
> to provide a crippled version of Python)

GAE provides a professionally managed, "infinitely" scalable (or at least a heck of a lot more scalable than any other single server is likely to be, still not a substitute for mirrors), battle tested platform. There are already implementations of the GAE APIs that can be run independently, so I don't think it is quite as proprietary as you might think (though you do lose most of the benefits without having their services available, you just end up with yet another not-so-amazing web framework). I'm not saying that I think GAE is 100% the best path forward, but it certainly has a lot going for it. Also, while Google is real company and has its own business to attend to, they have almost always been an ally and partner to the Python community and would likely be willing to work with us moreso than, say, Amazon Web Services (Rackspace is also a big Python proponent though, and has cloud offerings similar to AWS). Similar arguments can be made for things like using S3/Cloudfront for content hosting, it isn't a replacement for mirroring, but it would allow the main server (or possibly a "primary mirror") to take advantage of these powerful services towards better uptime, responsiveness, management, etc. This isn't a slight against the current system, just pointing out that while we have a few volunteers taking care of the PyPI server, Google and GAE have dozens of people who keep GAE running smoothly as their full-time job.

--Noah