[Mailman-Users] Mailman throughput

[Ivan Fetch]

Hello,

THanks Mark, I appreciate this. MOre below:

On Aug 14, 2011, at 11:39 AM, Mark Sapiro wrote:
> 
> It seems the major hurdle is in processing the 'out' queue. It is
> possible to slice OutgoingRunner to provide some parallelism in this
> process and that may speed things up, but I suspect that a lot of the
> time is in network communications between OutgoingRunner and the
> remote Postfix and that slicing OutgoingRunner may not help much, but
> it would be worth rerunning your benchmark with 2 or 4 outgoing runner
> slices to see if it helps.
> 
BY slicing, do you mean setting MAX_DELIVERY_THREADS in mm_cfg.py (restarting Mailman of course)? I did this, with values of 2 and 4, and if it made a difference for my smaller benchmark of 5000 messages, to one list of 25 recipients, it was only seconds of improvement.


> There are some MTA tuning tips in the FAQ
> <http://wiki.list.org/x/AgA3>, but some are only applicable to Mailman
> 2.0 so be careful.
> 
The only thing I can think of which may help, given that Mailman's Postfix is not delivering to subscribers, is to adjust concurrency or backoff settings for the local delivery agent, which is piping messages into Mailman's post script. I'm not sure whether Postfix still uses the backoff algorythm, when using local and pipe though.


> The main outgoing MTA performance killer is doing DNS verification on
> recipient domains during SMTP from Mailman. This should be avoided.

Using a local DNS cache cut my 5000 messages to a 25 recipient list, from 10 minutes down to 8 & 1/2 minutes. Even avoiding looking up the same hand full of hosts over and over again, helps.


I have to amend my earlier statement about our receiving 68000 posts per day - I was not careful enough when mining the post log; a lot of the posts are Mailman retrying delivery for tempfailed subscribers. So we do not see 68000 distinct posts, but we are doing a lot of redelivery attempts. Apparently we need to tune bounce processing for lists - this can be challenging to get right, and seems to require individual attention per list. I suppose I could have Mailman retry delivery less often, and if we have something like an outage of our own relays, I just trigger a retry by restarting the queue runners.


Thanks,

Ivan.























.