[Mailman-Users] Mailman 2.1.6 slowness...?

[Jeff Squyres]

On Jul 16, 2005, at 8:27 AM, Brad Knowles wrote:

>>  We didn't change the value of SMTP_MAX_RCPTS between 2.1.5 and 2.1.6
>>  (500).  Perhaps this is too high (none of our lists have close to 500
>>  subscribers in a single domain).
>
> 	That's not done on a per-domain basis.  Let's say you have 1000 
> subscribers.  With SMTP_MAX_RCPTS set to 500, then you will send out 
> one copy of the message to the first 500 recipients, wait for it to be 
> fully accepted by the MTA, and then send out the second copy of the 
> message to 500 recipients.  The result can extremely "peaky" in 
> nature, kind of like taking a car and shifting the gear box directly 
> from 5th gear to 1st, and then from 1st back to 5th.

Ah -- thanks for clarifying.  This is not really what is implied in the 
FAQ 
(http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.012.htp) 
-- I read it to  imply that mailman is grouping messages to the MTA by 
domain in order to boost minimum number of deliveries to a given target 
domain.  That could simply be my flawed interpretation (I based that 
assumption on the bandwidth discussion in the middle), but perhaps it's 
worth a few clarifying statements in the FAQ...?

> 	If you break that up into smaller chunks, the system can be 
> processing more of those chunks in parallel, and each chunk can be 
> accepted faster.

We have pumped our SMTP_MAX_RCPTS down to 5 (we had never changed it 
before -- the mailman default was 500, even though the FAQ suggests 
that it should be significantly lower), but we'll have to wait for our 
next peak traffic period (likely not until Monday) to see how well it's 
actually working out.

>>                                    We routinely see large sendmail
>>  outgoing queues (and did with 2.1.5, too).
>
> 	Given your configuration, I'm not at all surprised.

Right -- sorry, I didn't mean to imply otherwise.  We were not 
surprised by this, either.  I was trying to say that we've seen this 
behavior for a long time and didn't have any performance issues with 
it.

>>  A comment in one of the prior e-mails made me go check the other 
>> qfiles
>>  directories -- we have a few hundred files in the qfiles/bad 
>> directory
>>  (~200) and a few thousand in qfiles/shunt (~4000).  What are these
>>  files? (pardon the newbie question)
>
> 	The stuff in the "shunt" queue are messages that were malformed in 
> some way, and Mailman shoved them off to the side because it couldn't 
> deal with them.  Whenever you stop and restart Mailman, it will go 
> through the "shunt" queue to see if there is anything it can handle 
> now that previously it could not.  This can really hurt startup times, 
> and a large "shunt" queue can hurt you any time you have to move 
> messages into it.
>
> 	You might want to consider turning off Mailman, moving most of the 
> "shunt" messages to a different directory name (maybe shunt.old, or 
> shunt.2005-07-16, or whatever), and then creating a new shunt queue 
> with the exact same ownership and permissions.

Sounds good -- will do.  Thanks!

>>  Thanks for all your suggestions -- we're checking them in the
>>  background, but it takes a little time to check properly.  Also, 
>> based
>>  on your replies, I think we need to triple check with our sendmail
>>  maintainers and ensure that they really, really didn't change 
>> anything
>>  between when we were running 2.1.5 and 2.1.6 (they swear that they
>>  didn't, but we'll ask again).
>
> 	You definitely want to learn more about your sendmail configuration, 
> yes.

The thought occurs to me that perhaps it wasn't our sendmail guys who 
changed something, but perhaps the guys in the anti-spam/virus-checking 
crew changed something (I believe they also check outgoing mails for 
some insundry list of things that they believe indicates spam/viruses). 
  Hmm.  Need to go ping them, too...

-- 
{+} Jeff Squyres
{+} jsquyres at osl.iu.edu
{+} Post Doctoral Research Associate, Open Systems Lab, Indiana 
University
{+} http://www.osl.iu.edu/