[Mailman-Developers] Exim tweak

[J C Lawrence]

On Mon, 9 Dec 2002 23:36:47 -0800 
Marc MERLIN <marc_news@merlins.org> wrote:
> On Mon, Dec 09, 2002 at 08:00:25PM -0800, J C Lawrence wrote:

>> Not a bad side effect for a little idle poking about.

> Indeed, it will increase overall throughput.  However, you are
> sacrificing turnaround time to several minutes or more as a result,
> and this from mere seconds with mm 2.1 (for that matter, with some of
> my lists, when I save my message in vim, by the time I get back to the
> mutt index, the message has been sent, processed by the MTA, handed
> out to mailman, which exploded it and sent it back to my mailbox, so
> my message shows up in my index in the half second that it took mutt
> to come back from vim and rescan my mailbox

Yup.  

> If you ask me, that's pretty cool :-)

I can stand that pain, easily, especially as I generally moderate only
once or twice a day.  Adding a few minutes to a process which averages a
more than 6 hour latency is hardly expensive.

> Now, I'm curious: In your case, you get bursts of messages with lots
> of subscribers on your list.  

Yup, the largest burst I've played with to day this way was 20,000 spool
entries in one slam (I killed SMTP deliveries until qrunner had finished
injecting into the MTA).  It didn't quite drain proportionally to the
number of subscribers, but it wasn't a long way off either.

> How about a list server with many lists so that you always have 50 to
> 100 exims running and trying to clean the queue, deliver mail, or
> talking to mailman...  In that case (i.e. really loaded server), I'm
> not convinced that spooling everything to disk and having exim process
> the message on the next queue run is going to be that much more
> effective. Is it?

I expect that the efficiency *could* be maintained.  There are quite a
few variables.  (I've spent a lot more time tuning Postfix than I have
Exim) It depends on the distribution of the destination MX's (how much
repetition across your lists thus incurring parallel delivery), how much
of the default state of the "loaded mail server" is really repetitious
delivery attempts to slow MX'es (which defines your background sustained
load), the size of the message bursts (in terms of number of messages
prior to moderation, thus defining known parallel delivery
opportunities), the rate of MTA queue runner passes (defining windows of
opportunity for parallel deliveries), and the time required to inject a
burst into the MTA (which thus has a defined intersection rate with the
MTA queue runners).  The ___apparent___ critical bit is to get all (or
most/much of) the burst injected into the MTA before it starts
delivering it and then having a large number of moderated messages thus
being able to take advantage of parallel delivery and reduce the total
delivery time down to (optimally) linearly proportional to the number of
subscribers.

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
claw@kanga.nu               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.