[Mailman-Users] Load-balancing mailman between two servers

[Guy Waugh]

Brad Knowles wrote:
> At 3:03 PM +1100 11/27/06, Guy Waugh quoted Kim Hawtin:
>
>>>  How/where do you share the incoming mail list aliases that sendmail 
>>> checks?
>>>
>>>  Also when you create a new list, how to you update the other hosts 
>>> aliases?
>>>
>>  On each server, in the sendmail aliases file. So, when adding or
>>  removing a list, I have to do the alias changes on each of the two 
>> servers.
>
> Hmm.  With postfix, you can specify multiple alias files, some of 
> which will get auto-rebuilt as necessary by postfix, others which can 
> get manually rebuilt by other processes (like by Mailman, with the 
> standard tools it provides).
>
> It's been a while since I mucked around with sendmail, but I have to 
> believe that the same is possible there.  Indeed, I believe that the 
> technique that is currently used to completely automate this process 
> with postfix was adapted from the technique that previously worked 
> only with sendmail.
>
> You could put that second set of aliases in the NFS shared space, so 
> that you shouldn't have to rebuild that separately on the two boxes.
Yeah, thanks... given I've only got two servers, and we don't 
create/remove a lot of lists, I've never bothered to think about this 8-)
>
>>  One thing I am a bit concerned about is contention on the shared files
>>  between the two servers. If, for example, both servers wanted to update
>>  the same .pck file at the same time, I'm not sure what would happen...
>>  would one server lock the file and the other server wait for the 
>> lock to
>>  be released, does anyone know? Or would chaos ensue?
>
> The developers of Mailman have gone to great lengths to make the 
> system as NFS-proof as possible.  That is to say, it should be 
> possible to share the entire /usr/local/mailman structure via NFS (or 
> wherever you put all the Mailman files, including archives, queue 
> directories, etc...).  It should "just work".
>
> That said, file locking on NFS is problematic during the very best of 
> times, and I believe that the Mailman developers have gone to great 
> lengths to try to work around that.  But, by putting that stuff on 
> NFS, you are increasing the chances that you'll run into a situation 
> where you get lock contention, or maybe stale locks that will need to 
> be cleared out.  You will increase the amount of system maintenance 
> that you have to do -- that's simply unavoidable.
>
> And a very great deal depends on your NFS server.  Putting all this 
> stuff on NFS can cut your throughput that you can handle by a great 
> deal -- orders of magnitude or more, and that's if you've got a 
> high-end mega-expensive dedicated NFS fileserver from the likes of 
> EMC, Auspex, or Network Appliance.  Just keep this in mind as you 
> program for redundancy.
>
>
> I'm not saying that you can't put things on NFS, or that it's even 
> "unwise".  I am saying that everything is a set of trade-offs, and 
> you've got to understand what it is that you're trading for what.
>
OK, thanks Brad...

I'm still wondering whether I should be NFS-sharing the qfiles 
directory. I haven't delved into the Mailman source code to try to 
figure this out, but...

If, for example, a list post is held for (say) moderation in one 
server's qfiles directory (if this is in fact where held posts are 
kept?), and a list administrator accesses the Mailman web interface on 
the other server and approves the post, will the second (web interface) 
server be able to find the held post on the first server? I can't 
imagine that it would be able to find the post, if it is the case that 
posts held for moderation (and for other reasons, like posts by 
non-members to a member-only list etc. etc.) are held in the qfiles 
directory.

Maybe I should change my setup like you outline above, such that the 
*entire* Mailman installation is NFS-shared between both servers? The 
only anomaly to be dealt with here, AFAIK, is that Mailman writes a 
master-qrunner.pid file to the data directory, but I can get around 
that. I'd really prefer to leave it roughly the way I already have it, 
as I can upgrade Mailman on one server and test it while the old version 
is still running on the other server.

Thanks,
Guy.

-- 
Guy Waugh
Unix System Administrator
IT&TS, Southern Cross University
Lismore, NSW, Australia
Email: gwaugh at scu.edu.au
Ph.: +61 2 6620 3196
Fax: +61 2 6620 3033