Efficient final message disposition (was Re: [Mailman-Developers] Requirements for a new archiver)

[J C Lawrence]

On Fri, 31 Oct 2003 16:04:43 +0100 
Brad Knowles <brad.knowles at skynet.be> wrote:
> At 8:20 PM -0500 2003/10/30, J C Lawrence wrote:

>> While I don't disagree, this is really an MTA's job, not Mailman's.
>> This is why I've been doing log analysis of MXes and routing mail to
>> customised outbound MTAs on the basis of responsiveness, since early
>> 2000.  Adaptive MX routing is great stuff.

> There is a need for this function, and no MTA available today does it.
> MLMs throughout the history of the Internet have incorporated a
> variety of features for SMTP performance enhancement that are unique
> to mailing lists or are usually found primarily in mailing lists, and
> this is no different.

True.  Its not a very difficult process, and is absurdly expensive the
way I handle it.  At some point in my copious spare time I should whack
another couple config tokens into Exim, just to up the ante.

> If you want to externalize all these functions outside of mailman,
> that's fine.  But then someone has to pick up the ball and start
> hacking on bulk_mailer or some other program to provide these
> features.

Aye, but some care should be taken here defining who the people are,
between the Good-For-Mailman, and Good-For-Large-Mail-Systems camps.
They're related, but not synonymous.

>> Yup.  I did it at the first level with an initial SMTP proxy which
>> routed based on MX response records pulled from a DB.

> Again, this is a feature which is not found on any MTA available
> today, and which is known to have a huge impact on mailing list
> performance.  This feature needs to be provided somewhere, by someone.

True.

> If you go back to Barry's message, he was talking about getting
> even further involved, by doing a mail-merge process.  Since there is
> no MMTP (something that Bryan Costales, Eric Allman, and I had worked
> on for a while, before we realized that it would just make the spam
> problem worse and then dropped all further efforts), there is a need
> for an intermediate program that is called by mailman and then hands
> the messages off to the MTA.

<nod>

Mailmerge and VERP customisation, and the standards for the
communication of those things to the MTA are areas that need attention,
both for Mailman and the rest of the market (tho the IronPort and
related guys might argue).  This would be a good point to get some
cross-MTA discussion going on.

>> We're not in the game of second guessing the MTAs.  That way lies
>> wasted time and madness.

> If there were MLTAs which were optimized for this function, 

IIRC QMail has a (typically DJB) VERP/rewrite handoff method.  I also
recall that it is very bound into QMail's process and IO model, but
perhaps this should be examined?

> I would agree with you.  Since we're trying to take standard MTAs
> which may have only some optimizations that might be generally
> applicable to most situations (including mailing lists), I must
> disagree.

There's that audience problem again.  I actually agree with you in the
general case, and am willing to spend time and effort in that direction.
However I see this as somewhat disjoint from Mailman in specific.

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
claw at kanga.nu               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.