[Mailman-Developers] load balancing with mailman.
Marc MERLIN
marc_news@vasoftware.com
Wed, 30 Jan 2002 18:56:36 -0800
On Tue, Jan 29, 2002 at 02:40:56PM -0500, Barry A. Warsaw wrote:
> MM> Barry: With the new qrunner infrastructure, does qrunner still
> MM> need to lock the lists during delivery? If qrunner doesn't
> MM> modify config.db anymore, could it open config.db read only?
>
> Remember that now, we have usually 7 queues, and each one has its own
Yep.
> runner process. One of the advantages of this is that we can really
> isolate lock acquisition to a finer granularity. In fact,
> OutgoingRunner -- which processes qfiles/out files, and thus is the
> process that actually calls SMTPDirect -- does not lock the lists for
> the normal delivery processing. It simply shovels messages from the
> queue to smtpd and doesn't need to update any list information, as
> that's all done before the message gets to the outgoing queue.
Awesome, just what I was hoping for.
> There's one exception (of course ;). If your smtpd ever returns
> synchronous errors, then Mailman has to lock the list in order to
> register bounces. However, Mailman only does this periodically, and
> this is controllable by the variable DEAL_WITH_PERMFAILURES_EVERY in
> Mailman/Queue/OutgoingRunner.py (it's not a mm_cfg.py variable).
That's not a real problem. Even if there is a race condition somewhere, this
case should be sufficiently rare for the race not to happen.
> By default this is set to 1, but you could crank it up so that the
> culling of the known bounces is done less frequently. OTOH, if your
Good to know.
> MTA is set up to never do recipient tests/deliveries synchonously (and
> you're not delivering to local users), you should never have such
> delivery failures to deal with, thus you'd never need to lock the
> list.
Thanks a bunch.
This should definitely allow for load sharing and some failover capability
(the second server periodically rsyncs the NFS exported mailman tree in case
the first list server, which exports ~mailman minus qfiles which is a
symlink to local disk, dies.
So, if this happens, you can move a symlink and continue with one server and
an rsynced copy of the whole tree that is hopefully not too old)
Marc
--
Microsoft is to operating systems & security ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key