[Mailman-Users] Large mailing lists

[Brad Knowles]

At 5:40 PM -0500 2005-12-21, Ross Vandegrift wrote:

>  I've got some questions on a mailman server that's currently
>  performing non-optimally.

	If you've got a large list, the first thing you need to do is to 
search the FAQ and the archives for the words "performance" and 
"large".

>  1) Organization: the current setup is designed so that a post to the
>  list does not generate overwhelming amounts of traffic at once.  The
>  list owner posts to each list spaced out by a few hours.
>
>  Does a setup like this give any real benefit, or just cause headaches?
>  Should we consider cutting up the lists further?

	One thing you could do is break the list up into multiple 
sub-lists, and then set up larger umbrella lists (whose only 
"subscribers" are the sub-lists).  This will help reduce contention 
for locking the Python "pickle" objects which are used to store the 
current status of the list.  This is likely to help improve your 
performance.

>  2) Recipient expansion: Currently, Mailman is running it's default
>  recipient expansion to postfix, which sends via SMTP.  Since the
>  postfix server only accepts mail locally, I have considered making
>  these changes:
>  	a) Remove any recipient restrictions from postfix smtpd
>  	b) Set Mailman to send a single message with all recipients
>  	c) Set Postfix to limit outgoing messages to <20 recipients

	In this case, "b" is not likely to help you that much.  Again, 
search the FAQ and archives for "performance" for the reasons.

>  The goal of this is two fold:
>  	a) We have far more postfix expertise in-house than Mailman.
>  When there is an issue with Mailman, it's hard to solve.  We've got
>  more people capable of solving issues at the MTA level.

	For a list this large, a lot of your performance issues are going 
to be a result of Mailman and not postfix, and there's just nothing 
you can do to get around that.  Trying to shoe-horn everything into a 
postfix problem is not going to solve your fundamental Mailman 
problems.

>  	b) Postfix keeps logs for each outgoing message, giving us an
>  easy way to keep track of a message that is being sent.

	Mailman logs data for each message, and the logging has gotten 
better with each new version -- 2.1.6 is better than 2.1.5, etc.... 
In your case, I would encourage you to look at the new 2.1.7-beta 
version, and consider giving feedback to Tokio and Mark, which may 
significantly improve the official 2.1.7-release version.

>  3) Comparison: Is anyone else running lists this large?  I've found
>  some list server usage surveys online, but most of them have "30,000
>  and above" as their largest class of addresses.

	You have some of the largest lists I've heard of, but there may 
be larger ones out there.  Certainly, you would do well to check the 
FAQ and the archives.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  LOPSA member since December 2005.  See <http://www.lopsa.org/>.