[pydotorg-www] PyPI now running on nginx+uwsgi

["Martin v. Löwis"]

Am 22.11.2011 21:56, schrieb Patrick Ben Koetter:
> * "Martin v. Löwis" <martin at v.loewis.de>:
>> I have switched the PyPI installation to nginx and uwsgi.
>> The start scripts are in /etc/init.d/{nginx,pypi}; the configuration
>> in /etc/nginx and /data/pypi/config.ini. Log files go to /log as usual.
> 
> Out of curiosity: Why?

I hope that it can deal better with the overload cases that (possibly)
have constantly caused outages over the last few months. Load average
would occasionally go to 250, and apache spawn 2400 threads (*), in
overload situations. With nginx, this shouldn't happen (IIUC), it will
continue to serve files with low load even if all the PyPI processes
become unresponsive.

In addition, I also switched to uwsgi (primarily because it's one
of nginx's supported integrations of Python apps). I found that uwsgi
supports a timeout on single requests, which I set to 60s. So any
PyPI request taking longer than 60s (and I think these actually happened
in the past) will get killed, which should help to overcome overload
situations more quickly, and to allow continuing service to "regular"
users even in abuse situations.

Of course, whether this actually works out, we will see.

Regards,
Martin

(*) one may wonder: why did he set the thread limit to 2400 then?
Because when I didn't, people would be locked out of PyPI, when it
said that it couldn't take any additional clients, even though the
CPU load was "low".