help! advocacy resources needed fast

[Kyler Laird]

Geoff Gerrietts <geoff at gerrietts.net> writes:

>We have something approaching 30 servers in play, many of them
>commodity-level boxes (PC architectures with varied processor speeds,
>in the 1GB RAM range, big beefy database server), and at peak traffic
>times, response becomes unacceptably slow. The bottleneck is in Zope;

It sounds like you're running ZEO.  Is that correct?

>some portion of this could be blamed on the dynamic nature and
>complexity of our pages, but some portion is pushing DTML through a
>Python parser on every request.

So it's a problem in the ZEO clients/Zope servers (as
opposed to the ZEO server), right?  Isn't it enough to
throw more hardware at the problem?  If the data used
by these pages doesn't change constantly (thus can be
cached effectively), I'd expect that doubling the
number of servers would come close to doubling the
number of requests it can handle with the current
response time.

>We have spent a year refactoring key components, and building caching
>solutions to minimize the impact of load.

I realize you're trying to solve your problems now,
but I'd enjoy hearing more about this.

>And I need numbers, something to recommend the final solution I want
>to push as most viable in Python beyond "a bunch of Python advocates
>think it's quality shit". Or, I should say, I think I need numbers.

While I understand that you *must* handle the load
with adequate response times, I think there's more
to it than that.

>As dire as I've made things look, every engineer who touches the
>Python code, wants to work in Python. We like Python, we're good at
>Python, and Python has proven to be our engineering department's
>competitive advantage.

This reminds me of the situation where people jump
up and down about Python not running as fast as 
C/asm/whatever.  The *real* benefit is that you can
*create* in it much more efficiently.  Need it to
run faster?  Throw some hardware at it.  Hardware
is cheap and you can always get more.

>I hear lots of "probably mod_python" and now some "check out SCGI",
>but is anyone doing page templating with these things?

I'll pass your message along to someone who does.

--kyler