[Mailman-Users] Hardware Requirements

[J C Lawrence]

On Thu, 11 Jul 2002 08:43:06 -0400 
Scott Courtney <courtney at 4th.com> wrote:
> On Thursday 11 July 2002 12:14 am, J C Lawrence wrote:

>> a) Add more RAM.  Number of queue runners for your MTA

> Here's a silly question: Is it worth considering *really* upping the
> RAM, say to two gigabytes, and then putting /var/spool/mqueue on a RAM
> disk?  

Probably not, for two simple reasons:

  1) Too small.  He has a fairly large number of lists and the high
  probability of flash bursts in traffic, both in terms of numbers of
  messages and total size of outbound spool.  I could trivially see a
  RAM disk that small being exhausted.

  2) No battery backup (the other side of the stability issue you
  mention).  Doing it properly would involve something like a Rupp solid
  state disk (which is also big enough for a spool) with battery backup
  -- which would also happen to blow his budget.

> Another approach, much easier: Buy a lot of RAM, and rely on Linux
> using the excess as disk cache. This is probably a much better idea.

MTAs are limited by physical disk IO.  They do lots of reads and writes
(which can be cached), and more importantly, do lots of calls to sync()
and close() which explicitly flush those caches down to metal.

> I suspect my first paragraph above is not practical. I toss it out
> onto the list not so much in expectation of anyone trying it, but just
> to see if anyone thinks it's even worth experimentation. I would never
> even consider this for "regular" email service, but in many contexts
> lists are considered a lower priority with regard to reliability. And
> even if some outbound messages were lost in the (infrequent) crash
> scenario, they would still be available in the archives.

Ignoring the use of battery backed up solid state disks (which I know of
a few large mail shops using), this assumes that archiving is
synchronous with receipt and broadcast.  This is currently the case with
mailman 2.0 if you still use Pipermail, and IIRC not true with v2.1.

>> b) drop the CPU speed if it will save any money, though I suspect
>> that's as low as you can buy these days.

> With respect, I disagree. The price difference between CPU speeds
> below about 1 GHz is insignificant. 

Err, that's what I said.

> It's nice to have some CPU headroom so that you can do things like
> compile the next version of Mailman on the same machine, in a test
> directory, without impacting production. 

That wouldn't be a problem even with a quite smaller CPU.

> Speaking of which, disabling "locatedb" on this machine is probably a
> good idea.

True, along with other general system tuning.  The big question is still
what basic mail volumes are.  Without that we're just guessing.  

> A second reason for not short-changing this machine's processor: Any
> server in this kind of heavy-duty production will be tricky to upgrade
> from a logistical standpoint. If you build in some headroom, you
> postpone for more years the need to migrate all of this to a new
> machine.

True.  He asked for 3 years.

>> c) go SCSI with /var/spool/MTA, /var/log, and /var/www on different
>> spindles.

> Yes, definitely. Also, for a system that will have this many files on
> it, consider using a journaling filesystem rather than ext2. I have
> had superb success with Reiserfs, but there are also IBM's JFS, SGI's
> XFS, and the ext2-compatible ext3. Reiserfs has significant
> performance improvement over ext2 and ext3, especially on small files,
> and it might be a good choice for this system.

Journalling actually is a loss in this sort of scenario due to the extra
tracking and buffer copy overhead.  The nice thing about ReiserFS and
XFS in particular is that the other optimisations they make more than
make up for that cost.

Please see the large mail system stuff I wrote up in the FAQ last year.

> Just out of curiosity, what are you planning to do for backup media? 
> You may need a secondary SCSI controller channel to prevent contention
> for bus bandwidth during large backup runs. It could be a lower-cost
> SCSI board than the primary, probably.

Ahem.  He doesn't have enough SCSI targets to make that a problem.
Remember how SCSI disconnect works.

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
claw at kanga.nu               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.