Production Web Deployment?

[Paul Everitt]

Dirk Leas wrote:
> I've read numerous threads regarding how to deploy Python on the web and
> there appears to be no real consensus on what actually performs/scales well
> (about 3M hits/month). Is anybody taking lots of hits with httpdapy,
> PyApache, ASP+Python, fastcgi, or Zope? I've got a customer who's
> development staff is struggling with the complexity of Java Servlets and
> they're already using a large 3rd party, vertical Python application (and
> several folks already use Python). Peliminary testing showed plain CGI was
> just too slow...

Zope.org seems to be doing up to 2M hits/month quite easily, and it's a
pretty complicated site (many of the pages are generated by catalog
queries to the index).  Over the last couple of weeks we had some
inexplicable slow spots, which turned out to be caused by accidentally
setting the object cache way too low (200 instead of 4000).  Note that
Zope.org doesn't use a relational database for any content.

As far as I can tell, performance on Zope.org is quite snappy right
now.  It's running on a sub-$1k machine (a vanilla PIII-550 with 256 Mb
RAM and an IDE drive).  I'd expect it to do four or five times more hits
before it appears sluggish.

As mentioned in another reply, keeping database connections open is
quite important.  Zope does a good job at this and some of the database
adapters are pretty well optimized (Oracle and Sybase particularly).

Hopefully over the next month we'll migrate Zope.org away from PCGI (a
Zope system for getting requests from the web server to the long-running
process) to FastCGI.  This will eliminate execution of small C "wrapper"
executable.

Finally, Digital Creations has a commercial-with-source-plus-consulting
solution called ZEO:

  http://www.digicool.com/Solutions/ZEOFactSheet.pdf

This lets you spread your object system across multiple
machines/locations.

--Paul