[Mailman-Users] Mailman locking up, seems to be Spamassassin related

[Richard Barrett]

On 27 Jun 2006, at 10:33, Chris Croome wrote:

> Hi
>
> One a CentOS 4 box, running Mailman that comes with the distro, I'm
> seeing locks up every now and then -- all the lists on the machine  
> stop
> processing mail.
>
> When this happens a restart gets things moving again.
>
> This is the error that is written to /var/log/mailman/error:
>
>  Jun 26 12:04:58 2006 (28052) Uncaught runner exception: (4,  
> 'Interrupted system call')
>  Jun 26 12:04:58 2006 (28052) Traceback (most recent call last):
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 111, in  
> _oneloop
>     self._onefile(msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 167, in  
> _onefile
>     keepqueued = self._dispose(mlist, msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line  
> 130, in _dispose
>     more = self._dopipeline(mlist, msg, msgdata, pipeline)
>   File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line  
> 153, in _dopipeline
>     sys.modules[modname].process(mlist, msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Handlers/SpamAssassin.py", line  
> 76, in process
>     score, symbols = check_message(mlist, str(msg))
>   File "/usr/lib/mailman/Mailman/Handlers/SpamAssassin.py", line  
> 62, in check_message
>     res = connection.check(spamd.SYMBOLS, message)
>   File "/usr/lib/mailman/Mailman/Handlers/spamd.py", line 123, in  
> check
>     response = fp.readline()
>   File "/usr/lib/python2.3/socket.py", line 338, in readline
>     data = self._sock.recv(self._rbufsize)
>  error: (4, 'Interrupted system call')
>
>  Jun 26 12:04:58 2006 (28052) SHUNTING: 1151298261.692024 
> +45c6f4028caac564c9880d1ac3f73dca4b88bfb1
>
> Mailman has been set up to work with Spamassassin using the  
> instructions
> here:
>
>   http://www.jamesh.id.au/articles/mailman-spamassassin/
>
> As far as I can tell the only thing that this involves is this line
> being added to /etc/mailman/mm_cfg.py:
>
>   GLOBAL_PIPELINE.insert(1, 'SpamAssassin')
>
> (I didn't set this machine up however...).
>
> Does anyone have any ideas what could be going wrong here and what  
> might
> be don't to fix it...?
>

I do not use this myself but a quick look at the code available at  
the site you reference above shows that the addition to the Mailman  
pipeline is synchronously passing the message to an external/  
separate process for checking and then awaiting its response.

That pipeline is operated by the IncomingRunner of which there is  
usually only one such queue runner. Hence if there is any delay in  
the external SpamAssassin processing a message all other incoming  
messages will just wait in the queue.

The code communicating with the checking process does not appear to  
protect itself with any timeouts so if the external process fails to  
respond for prolonged periods then your Mailman system will simply  
appear to lock up as far as processing incoming messages is  
concerned. In effect it will lockup the entire Mailman system as  
without new messages to process, the other queue runners have nothing  
to do.

Personally, I would not adopt this approach to running SpamAssassin.  
But if I were to use this approach I would re-engineer the queue  
runner/pipeline to allow multiple instances of the external checking  
process to be utilised and add some timeout protection.

But better, I would integrate SpamAssassin into the MTA delivering to  
the Mailman server. We use the latter approach so that unchecked mail  
never gets near Mailman (or any other MTA/MDA/MUA on our site). This  
should give much higher levels of concurrency when checking incoming  
than this serial approach with Spamassassin in the Mailman pipeline.  
This is in fact what we do.

Incidentally, we also use this check all the mail going off-site from  
our systems via the MTA so that we can avoid any infected machine in  
our network passing its problem on to the outside world.

Maybe you should discuss the problem with the author of the  
Spamassassin integration code.

> Thanks
>
> Chris
>
> -- 
> Chris Croome                               <chris at webarchitects.co.uk>
> web design                             http://www.webarchitects.co.uk/
> web content management                               http://mkdoc.com/