[Mailman-Users] Mailman 2.1.6 slowness...?

[Brad Knowles]

At 4:22 PM -0400 2005-07-15, Jeff Squyres wrote:

>  However, once we upgraded to 2.1.6, performance on our smaller lists
>  decreased dramatically during peak traffic times -- it can take 30-90
>  minutes before mailman hands off incoming list mail to sendmail.

	Weird.  There is nothing in Mailman 2.1.6 that should cause it to 
be significantly slower.

>  Does mailman somehow throttle/queue its output to the MTA?

	No.

>                                                              Are there
>  any configuration parameters that I can tweak to change this behavior?

	With regards to Mailman itself, there are relatively few 
performance tweaks or changes that you can make.  There are some 
tricks you can do to speed things up some, depending on what type of 
specific performance problems you're having.  I'd encourage you to 
start by going to the Mailman FAQ Wizard at 
<http://www.python.org/cgi-bin/faqw-mm.py> and search for 
"performance".

>  It's quite a bummer, especially since 2.1.5 did not seem to exhibit
>  this behavior.  We see the parameter SMTP_MAX_RCPTS; it's listed in our
>  Defaults.py with a default value of 500 (which seems to be fine).

	One of the things that will really slow down a mailing list is 
trying to hand off a large set of recipients to the MTA, especially 
if the MTA does extensive checking on each and every recipient 
address, and most especially if the MTA delivers each recipient in a 
synchronous mode -- as is typical for sendmail.

	One thing I would suggest is to make sure that you can enable 
list personalization and VERP, and then turn them on for your biggest 
lists -- this way the MTA can handle delivery to each recipient in 
parallel.  This also helps in bounce message management, which is one 
of the biggest problems on large mailing lists.

>                                                                     I'm
>  thinking that this is not our problem (it was 500 in 2.1.5 as well) --
>  indeed, the problem seems to be the rate at which mailman gives mail to
>  sendmail, not how many mails it gives in each MTA transaction.

	Hmm.  That doesn't sound like a typical Mailman performance problem.

	Are your filesystems full?  Most Unix OSes will optimize for 
speed, up until about 90% full.  At that point, they start optimizing 
for space, and this is usually a pretty big performance hit.


	Also, do any of the directories have more than a few hundred 
files in them?  Or have they ever had more than a few hundred files 
in them at any one time?  One of the problems with standard Unix 
filesystems is that you have to do a linear search through all 
possible directory entries before you can create a new file, or 
before you can confirm that a given file doesn't exist.  So, if a 
directory ever ballooned to thousands of files, even if it no longer 
has thousands of files in it now, it still takes a long time to do 
the linear search.

	The solution for this problem is to stop the affected systems, 
move aside the old directories, create new ones with all the same 
ownership and permissions, and then move any files that need to be 
preserved from the old directory to the new one.


	If you do a search of the Mailman FAQ Wizard and the archives of 
the mailman-users mailing list, you should find all these tips and 
more.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

   SAGE member since 1995.  See <http://www.sage.org/> for more info.