[Mailman-Users] Problem with qrunner and too much incoming mail

[Nigel Metheringham]

[I am replying *only* to the question of how to get this going with 
current Mailman on current exim - I'm not touching future directions 
for any product in this message]

[[I'm also assuming exim because I'm sure a message said this was with 
exim - or there was a message to the exim mailing list.]]

marc_news at valinux.com said:
> The problem is due to qrunner being  single threaded by default and
> having a global lock. Because  some mailing lists  have subscribers in
>  domains where DNS is  slow and unreliable, the  MTA will hang  on
> those rcpt to  until DNS resolves or timeouts, and qrunner won't be
> done in time. After that, it's all downhill from there, more mail
> queues up, qrunner falls even further behind, etc, etc... 

Problem # 1 - DNS is slow

Run a local DNS cache.  I would strongly recommend Dan Bernstein's 
dnscache for this - its small, secure, focussed and doesn't expand to 
steal all your memory.  For exim.org I run dnscache on the Mailman/exim 
box with a significant performance increase despite the hosting ISP 
having local DNS caches (based on bind) very close to the box.
See 
	http://cr.yp.to/djbdns.html
for the djbdns package which includes dnscache.  You also need his 
daemontools package to run it.


Problem # 2 - Exim is checking addresses on receipt.

The Exim/Mailman HOWTO has some performance tweaks - see
	http://www.exim.org/howto/mailman.html
Specifically in there is:-

	# Verify receipient addresses on everything except local injects
	# DO NOT verify addresses from mailman - this would slow down
	# the acceptance of messages dramatically
	receiver_verify_hosts = !127.0.0.1/8:0.0.0.0/0
	sender_verify

That will verify senders (fast - they are local - you could turn that 
off if you wish), but does not verify recipients on stuff that mailman 
is sending out.

You may want the other tweaks on that page too.  However I only run a 
list of around 1K members, know a good bit about mail systems, but 
don't have the facilities to check things for really big lists.  If 
people have specific advice for bigger systems then tell me and I'll 
add them to that document.


marc_news at valinux.com said:
> We're currently playing with  MTAs to optimize this a bit,  but the
> real fix is on the mailing list side. 

No its not - your problem is that you are doing unnecessary work in the 
MTA and holding up Mailman accordingly.   There probably is work needs 
doing to qrunner, but this ain't why.  Its better to dump the mails 
into the MTA and let the MTA send bounces back than to do all this 
synchronous address checking here.

	Nigel.

-- 
[ - Opinions expressed are personal and may not be shared by VData - ]
[ Nigel Metheringham                  Nigel.Metheringham at VData.co.uk ]
[ Phone: +44 1423 850000                         Fax +44 1423 858866 ]