[pydotorg-www] PyPI now running on nginx+uwsgi

[Patrick Ben Koetter]

* "Martin v. Löwis" <martin at v.loewis.de>:
> Am 22.11.2011 21:56, schrieb Patrick Ben Koetter:
> > * "Martin v. Löwis" <martin at v.loewis.de>:
> >> I have switched the PyPI installation to nginx and uwsgi.
> >> The start scripts are in /etc/init.d/{nginx,pypi}; the configuration
> >> in /etc/nginx and /data/pypi/config.ini. Log files go to /log as usual.
> > 
> > Out of curiosity: Why?
> 
> I hope that it can deal better with the overload cases that (possibly)
> have constantly caused outages over the last few months. Load average
> would occasionally go to 250, and apache spawn 2400 threads (*), in
> overload situations. With nginx, this shouldn't happen (IIUC), it will
> continue to serve files with low load even if all the PyPI processes
> become unresponsive.
> 
> In addition, I also switched to uwsgi (primarily because it's one
> of nginx's supported integrations of Python apps). I found that uwsgi
> supports a timeout on single requests, which I set to 60s. So any
> PyPI request taking longer than 60s (and I think these actually happened
> in the past) will get killed, which should help to overcome overload
> situations more quickly, and to allow continuing service to "regular"
> users even in abuse situations.
> 
> Of course, whether this actually works out, we will see.

If a web cache helps with PyPi, we could add varnish
<https://www.varnish-cache.org/>. It has been praised all over the place. Ralf
and I happen to know the author. Dunno if that would open a door, but I would
try if we think we need help.

p at rick

-- 
Patrick Ben Koetter <patrick at python.org>
Python.org Postmaster Team