[Mailman-Users] Problem with qrunner and too much incoming mail

[Chuq Von Rospach]

At 11:35 AM -0800 11/3/00, Marc MERLIN wrote:

>We're currently playing with  MTAs to optimize this a bit,  but the real fix
>is on the mailing list side.

what's your MTA? Sendmail? postfix?

>- Forget about qrunner and switch back to direct delivery and queueing only
>   when direct delivery fails.

the other problem with this is peak load problems. Instead of getting 
nailed by a backlogged, you get nailed by trying to delivery all at 
once and having your system croak. You need some way to manage 
delivery to keep the peaks from killing you.

>- Remove the locking in qrunner, run more than one qrunner at once, and hope
>   for the best ;-)

you definitely don't want that. Definitely.

>- Have a multithreaded qrunner that processes 10 or 20 mails at once
>   (talking to 10 or 20 instances of the MTa in parrallel)

something I was going to bring up -- since I'm now running Mailman on 
a 2CPU system, it'd be nice to be able to run multiple qrunners. But 
it actually has some fairly serious implications in the locking areas 
as well, so it doesn't seem to be an easy fix. It's certainly not 
something I'd try to load into the 2.0 timeframe, not if we're RC 
candidate status.

>- Other?

I think it's a combination of factors .The quick fix is tweaking the 
MTA to be less sensitive. On my big server (50+ lists, 40,000+ 
subscribers) I simply haven't see more than a couple of hour delay, 
and that was only on the day when all of the "we've migrated to the 
new server" messages that went to everyone on every list at the same 
time started coming back in as MTA timeouts from the queue. to be 
really honest -- I wonder if the machine you're running this on is 
udnerpowered or misconfigured in some way, but I'd go beating on the 
MTA first.

I agree that multiple qrunners would be nice, but I don't think it's 
a quick fix thing, but a 2.1 thing.

>Now, if I  have X machines that mount /var/local/mailman,  they'd be able to
>service all the lists (config.db would  get locked correctly), but I'd still
>be stuck with only one queue runner because of the global lock.
>That said,  I *could* have mailman/data  and mailman/qfiles be a  symlink to
>somewhere on the local disk, and patch qrunner to put its lock in data.
>This would allow for independant queue  runners, but shared list configs and
>shared locks on the list configs themselves.
>
>Would that work?

I'm glad I'm not trying that on MY machine. I wish you luck...
>

-- 
Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui at plaidworks.com)
Apple Mail List Gnome (mailto:chuq at apple.com)

Be just, and fear not.