[Mailman-Developers] MM Bouncer

[Chuq Von Rospach]

On 12/6/01 9:44 AM, "Peter C. Norton" <spacey-mailman@lenin.nu> wrote:

>> At the expense of loading the wire, the MTA, *and* the MLM.
>> 
>> How big are your lists, Chuq?  :-)
> 
> Again, I don't think the wire is usually an issue.

It is on my machine, but you don't want to know how much work we've done
into getting the disk I/O under control. And to be honest, I bet if I put
even more disks, or a ram-based disk system on it, I'd speed it up another
20-30%, but it gets more and more expensive for less and less gain.

I expect to lose some performance going to the one-per-address customized
format, which is one reason why I'm going ot the army of smurfs -- but we
feel strongly it's great for the subscriber and gives us some nice usage
advantages, and besides, that system was originally designed to delivery
about 250,000 addresses and hour, and it's now doing about 600K, so I have
some slop before I fall below my performance metrics...

(and that's using sendmail. I expect once I get comfortable with postfix and
migrate my sites to it, it'll get better)

> So, to speculate, a sensible MTA puts metadata in a seperate file.
 
> The re-writing would be done on the way out to the remote host, and it would
> be pretty cheap to implement at this phase.
> 
> Is this about the right idea?

If you're going to go that way, cut the MTA completely out of the loop.
Simply write a delivery agent that writes directly out to the receiver's
SMTP port, so it never actually touches disk unless the first delivery
attempt fails for some reason (if it does, stuff it in a standard MTA and
let it worry about redeliveries, unless you want to keep lots of state
around). I've done some noodling on doing something like this, and if you do
it right (it's a fair amount of work), you can really do some fun stuff,
because you're literally writing the message on the fly out the wire.

But ti's not worth it except for customized, high-volume operations. For 99%
of mailman installations, it'd be hoplessly overkill technology, even if you
want customized messages.