[Mailman-Users] Problem with archrunner using large %'s of cpu (read faq & archives)

[Brad Knowles]

At 6:21 PM -0500 2003/10/31, Scott Lambert wrote:

>  I haven't looked at the code yet, and probably won't (ENOTIME), but it
>  almost sounds to me like it's not pruning it's list of handled messages
>  and has to walk all of them each time.  I would have expected queue
>  handling to get faster as the queue got smaller due to fewer files
>  in the directory that it needs to search through.  Maybe it's just a
>  function of the python datastructure being used.

	If it's using files as the queue mechanism, then deleting a file 
simply marks the entry in the directory as "available", and it still 
takes just at long to scan the directory afterwards as it did before.

	This is a known problem with many MTAs handling large amounts of 
messages, and is one reason why you should use a hashed directory 
scheme for your mail queue (a la postfix), or you should periodically 
stop the MTA, move the mail queue directory aside, create a new mail 
queue directory (with appropriate ownership and permissions), then 
move what messages may remain from the old queue back into the new 
one (or fire up queue runners to clear the old queue while the new 
one is being used for new mail).

	Mailman could very easily be suffering from the same sort of 
problem -- once you get a directory with a large number of entries in 
it, it takes a long time to scan it even if there are only a few 
files that are currently visible.  Same problem, perhaps the same 
solution?

-- 
Brad Knowles, <brad.knowles at skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)