[Mailman-Developers] qrunner infinite loop?

[Barry Warsaw]

On Sun, 2003-08-24 at 20:49, Greg Ward wrote:
> I've just installed the mailman 2.1.2-7 package on a Debian sarge
> (testing) system, and qrunner goes into an apparent infinite loop as
> soon as it starts.  Here's a snapshot from "top" shortly after running
> "/etc/init.d/mailman start":
> 
> 
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  Command           
>  4498 list      19   0  4656 4656 1844 R 99.7  0.5   0:48.36 python            
>  4484 gward      9   0   940  940  748 R  0.3  0.1   0:00.17 top               
> 
> Memory use is stable, hence my assumption that this is a garden-variety
> infinite loop.
> 
> And "ps -fu list" reports this:
> 
> UID        PID  PPID  C STIME TTY          TIME CMD
> list      4492     1  0 02:42 ?        00:00:00 [mailmanctl]
> list      4493  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=ArchRunner:0:1 -s
> list      4494  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=BounceRunner:0:1 -s
> list      4495  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=CommandRunner:0:1 -s
> list      4496  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s
> list      4497  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=NewsRunner:0:1 -s
> list      4498  4492 97 02:42 ?        00:01:01 qrunner /var/lib/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s
> list      4499  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=VirginRunner:0:1 -s
> 
> ... so it's just one qrunner (OutgoingRunner) that's responsible for all
> the CPU sucking.
> 
> Hmmm: there *are* files in the queue, but I'm not sure what they're
> doing there or how they got there (this is a new, fairly inactive system
> with no lists and no incoming traffic, apart from me testing things):
> 
> # cd /var/lib/mailman/
> # find qfiles/ -type f
> qfiles/out/1061532001.524745+03e15ef2a9a7d27904406a659cdb4922a204431b.pck
> qfiles/out/1061497885.874818+cf5f7e2e03cbca8bb84a23a60f055679c16e81fa.pck
> qfiles/out/1061704802.107101+4617d8731ecb303dd756e475b5362bd5a64b1fcf.pck
> qfiles/out/1061497885.871999+3453d7caa05a1aad6b230da61b39d4937405fcc8.pck
> qfiles/out/1061497885.871999+3453d7caa05a1aad6b230da61b39d4937405fcc8.db
> qfiles/out/1061497885.874818+cf5f7e2e03cbca8bb84a23a60f055679c16e81fa.db
> qfiles/out/1061704802.107101+4617d8731ecb303dd756e475b5362bd5a64b1fcf.db
> qfiles/out/1061532001.524745+03e15ef2a9a7d27904406a659cdb4922a204431b.db
> 
> Any clues?  I'll try to investigate more tomorrow evening.

While I have seen qrunners suck up all available cpu occasionally, it's
usually for manageable periods of time.  I haven't actually seen true
infloops, although I won't discount it.  ArchRunner and BounceRunner are
the worst offenders, the former because Pipermail is very inefficient,
and the latter because of the stupid way bounces are registered.  I
actually have a fix for the BounceRunner issue in cvs, which I intend on
installing on python.org to test out <wink>.

OutgoingRunner should be fairly sane since all it's doing is reading the
files from disk and spewing them over port 25 to your local smptd.  It's
also doing logging so you could tail logs/{smtp,smtp-failure,post} to
watch it make progress.  You should also see the qfiles/out directory
grow and shrink as files are consumed and unlinked, or new ones are
prepared for sending out.

-Barry