tuning our site (was: help! advocacy resources needed fast)

[Jeremy Hylton]

On Tue, 2003-03-11 at 15:23, Geoff Gerrietts wrote:
> Now, there's a lot that's atypical about our architecture, most of
> which I won't go into. We're on an old version of Zope with lots of
> little patches. We're on an old version of Python with a couple little
> patches.

There are some changes in newer Pythons and Zopes that might make a
difference.  It depends how old "old" is.  Zope 2.6.1 and ZEO 2.0.1 both
have cache improvements that can make a difference, particularly when
the Zope process is using a lot of memory.

The old Zope object cache had a tendency to accumulate many more objects
than it should.  The object cache in Zope 2.6.1 does a much better job
of limiting memory usage by keeping the cache size under control.

There were also several bug fixes in ZEO 2.0.1 that made the ZEO object
cache more effective.  There's also a tool you can use to analyze cache
behavior and determine what an optimal cache size is.

> - We replaced the asyncore module with a C extension that provided the
>   same functionality.

Do you have any idea how much difference that made?  On the one hand,
asyncore seems quite inefficient to me.  On the other hand, I wouldn't
expect it to have a big overall impact on application performance.

> - Put ZEO's cache on a ramdisk instead of hdd. Zope responds much
>   worse to loads late in the day than it does early in the day, after
>   the log roll and cache clear. We suspect this is because looking
>   through the 20+MB ZEO cache for content hits the disk pretty hard.

20MB sounds like an awfully small ZEO cache.  For some of our big
production sites, we have a 200MB cache.

> - Roll logs more often. The logs get big, and appending to big files
>   takes longer.
> 

Jeremy