[Mailman-Users] CGI alternative?

[kellan]

Hi, I'm part of a team that works on maintaining lists.indymedia.org, we
have a very large number of lists, a lot of traffic, and seemingly a lot
of interest for robots, particularily badly behaved ones.

Several times lately the server has started choaking and dieing, with
loads of 70+ in response to some bot hitting all the listinfo pages, and
firing up dozens and dozens of CGI processes.

I was wondering if anyone else has struggled with this issue.  We don't
want to simply block all robots, google can be essential to finding old
posts on some of the high traffic lists.  So I'm looking for less
resource intensive solutions then CGI.

Has anyone setup the Mailman CGIs under FastCGI, or mod_snake or
something?  How did that go?

Also considering simply sticking Squid in front of it, anybody have tips
on a Mailman friendly Squid config would be great.

Am I missing an obvious solution?

Thanks
Kellan