High volume websites using Python web server software?

[Elbert Lev]

Michael Sparks <zathras at thwackety.com> wrote in message news:<mailman.5585.1098903028.5135.python-list at python.org>...
> On Wed, 27 Oct 2004, Ian Bicking wrote:
> 
> > Michael Sparks wrote:
> > >>Hi,
> > >>Things like Twisted, medusa, etc.... that claim to be able
> > >>to support hundreds of concurrent connections because of the
> > >>async I/O framework they're based on....

Hi!

For the last several weeks I was occupied with the service for
harvesting/delivering bulk e-mail. Initially the program was written
in Java with one thread per connection. We could support up to 100
threads/connections without performance degradation (which was not
enough). Connections are relatively "long living". The logic of this
piece of software is very complex, so modification and addition of
extra handling, like anti-spam filters, did become real hassle (nobody
wanted to touch this program). In essence the program was "dead."

Facing mentioned above, I decided to rewrite it in Python, using
asyncore module. The only modification required (for performance) was
the replacing of readable()/writable() with flags. The net result was:
500 simultaneous connections are handled with almost no CPU load 5-7%.
(socket I/O only, no text processing).  Still the program has multiple
threads for backend DB processing and file I/O. For comparison: the
same stream of data can be handled by IOCP program in C (Windows) with
0.5-1% CPU consumption. Probaly more optimization could be done for
better performance, but I was sutisfied.

I beleive this numbers prove (at least for me) that
Python-medisa-twisted is a good solution for high load servers (and a
cheap one, and maintainable :)

I'm in no way a Python guru. Just a "beleiver" :)

Good luck