[Mailman-Developers] AOL's requirements for spam complaints

[Brad Knowles]

At 8:35 AM -0800 2004/02/02, Chuq Von Rospach wrote:

>  It's definitely useful and a big win. It both clears up general disk
>  I/O, but more importantly (from what I have seen), moves certain key
>  inodes in the delivery file structure off of disk, and since I/O
>  operations have to lock and unlock them for update, the time wasted
>  single-threading through them goes way down

	Those I/O locking operations are called synchronous meta-data updates.


	Imagine trying to clear an entire stadium filled with millions of 
people.  There is one door.  It can open and close very fast, but to 
be sure that everything is happening safely, it has to be closed and 
locked between every person.  So, you unlock the door, open it, step 
through, close the door behind you, lock it, and then the next person 
can come through.

	No matter how fast you can open and close and luck and unlock 
that door, that's going to seriously decrease the throughput you can 
get through the system.

>                                              (this is why, for
>  instance, you should generate a fairly large number of sub-queues
>  in sendmail; if you're trying to do volume and haven't, you're
>  being silly; it spreads the load across more than one inode)

	Right, postfix does this by default, and sendmail can easily be 
configured to do it as well.  There are lots of other nice things 
that postfix does by default and which more modern implementations of 
sendmail (e.g., 8.12 and above) also do, either by default, or are 
easily configured to do.  In particular, envelope splitting for large 
numbers of recipients.


	This is like having multiple doors that can be operated 
simultaneously.  Even if there is a central operating mechanism that 
can handle a single unlock/open/close/lock operation, you can 
interleave the operations between multiple doors, and greatly 
increase the overall throughput.

	Using improved filesystem architectures, such as softupdates, 
would be like using revolving doors instead.

	Using a RAM disk instead of physical drives is like making those 
doors operate at light speed, instead of the limitations imposed on 
normal wood, glass, and steel by the laws of physics.

-- 
Brad Knowles, <brad.knowles at skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)