[Mailman-Developers] locking and big lists

[Brad Knowles]

At 4:37 PM +0200 2004-10-28, Fil wrote:

>  I don't understand this language, sorry. We have just upgraded the I/O
>  capacity of the server, and it's working fine with every other piece of
>  software, no problem with postfix handling tons of emails for example.

	The same sorts of problems generally face Mailman as face other 
programs, and the same sorts of solutions are usually useful for 
Mailman.  So, solid-state disks, high-capacity RAID 1+0 arrays with 
high-speed controllers and large amounts of battery-backed write-back 
cache, etc....

	If you're confused about a lot of these terms, I'd recommend 
reading the book by Nick Christenson entitled _sendmail Performance 
Tuning_, which is linked from that first page I mentioned.

>  But still when we're talking about a 15-60 seconds lock for each subscriber,
>  even if we could have the hardware go twice faster, it's way too slow.

	I think it would be profitable to find out why VirginRunner is 
holding the lock this long.

>  That's why I believe only the database backend can help (or any other
>  software modification). What's especially wierd is that probe sending is
>  very costly, but personalization or VERP-delivery is not that costly.

	There are people who have done database back-end stuff for 
Mailman 2.x, but so far as I know none of that has been integrated 
back into the mainstream code base.  Those are all one-off 
operations.  There will be a database back-end built into Mailman3, 
but it's still in the design phase right now.

	If you can't find out why VirginRunner is holding the locks so 
long, then the best thing to do would be turn off the VERP bounce 
probes.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

   SAGE member since 1995.  See <http://www.sage.org/> for more info.