[Mailman-Developers] Performance problems and MailMan

[Barry A. Warsaw]

--YhIK0mKoxU
Content-Type: text/plain; charset=us-ascii
Content-Description: message body text
Content-Transfer-Encoding: 7bit


>>>>> "JCL" == J C Lawrence <claw@varesearch.com> writes:

    JCL> I believe I've found out how to reliably reproduce the
    JCL> performance problemsI've noticed here at VA and at Kanga.Nu,
    JCL> and which Barry and another (forget name, sorry) have
    JCL> observed as well:

    JCL>   1) Create a moderated list.

    JCL>   2) Subscribe 200 addresses to the list (can be bogus
    JCL> addresses but the local MTA must accept them)

    JCL>   3) Post at least 30 messages of an average of at least 2K
    JCL> size to the list.

    JCL>   4) Go to the moderation page, approve every message, and hit
    JCL> submit.

    JCL>   5) Watch your system load peg and stay there for an
    JCL> obscenely long time.

Just a quick note 'cause I have very little time.  I'm currently
seeing python.org massively pegged, and Guido and I were talking about 
some Python tools we'd like to develop that would help debug
situations like this.  What I wanted was something like gdb's ability
to attach to and print stack traces of running external programs.  We
got into some brainstorming and came up with A Certified Very Cool
Trick[1].

This yielded a traceback for where at least two pegged processes are
spinning.  Seems to make sense, but I'm not very familar with the
archiving guts, so I post this traceback to spur some discussion.
Maybe Scott or Harald can craft a fix.

Here's the traceback:


--YhIK0mKoxU
Content-Type: application/octet-stream
Content-Disposition: attachment;
	filename="foo2"
Content-Transfer-Encoding: base64

ICBGaWxlICIvZXhwb3J0L3BhcnJvdC9tYWlsbWFsdC9zY3JpcHRzL3Bvc3QiLCBsaW5lIDcz
LCBpbiA/CiAgICBtbGlzdC5Qb3N0KG1zZywgYXBwcm92ZWQ9ZnJvbXVzZW5ldCkKICBGaWxl
ICIvZXhwb3J0L3BhcnJvdC9tYWlsbWFsdC9NYWlsbWFuL01haWxMaXN0LnB5IiwgbGluZSAx
MjgzLCBpbiBQb3N0CiAgICBzZWxmLkFyY2hpdmVNYWlsKG1zZykKICBGaWxlICIvZXhwb3J0
L3BhcnJvdC9tYWlsbWFsdC9NYWlsbWFuL0FyY2hpdmVyL0FyY2hpdmVyLnB5IiwgbGluZSAy
MTAsIGluIEFyY2hpdmVNYWlsCiAgICBoLmNsb3NlKCkKICBGaWxlICIvZXhwb3J0L3BhcnJv
dC9tYWlsbWFsdC9NYWlsbWFuL0FyY2hpdmVyL0h5cGVyQXJjaC5weSIsIGxpbmUgOTA2LCBp
biBjbG9zZQogICAgc2VsZi51cGRhdGVfZGlydHlfYXJjaGl2ZXMoKSMgVXBkYXRlIGFsbCBj
aGFuZ2VkIGFyY2hpdmVzCiAgRmlsZSAiL2V4cG9ydC9wYXJyb3QvbWFpbG1hbHQvTWFpbG1h
bi9BcmNoaXZlci9IeXBlckFyY2gucHkiLCBsaW5lIDg3MSwgaW4gdXBkYXRlX2RpcnR5X2Fy
Y2hpdmVzCiAgICBzZWxmLnVwZGF0ZV9hcmNoaXZlKGkpCiAgRmlsZSAiL2V4cG9ydC9wYXJy
b3QvbWFpbG1hbHQvTWFpbG1hbi9BcmNoaXZlci9waXBlcm1haWwucHkiLCBsaW5lIDMzMCwg
aW4gdXBkYXRlX2FyY2hpdmUKICAgIHNlbGYud3JpdGVfaW5kZXhfaGVhZGVyKCkKICBGaWxl
ICIvZXhwb3J0L3BhcnJvdC9tYWlsbWFsdC9NYWlsbWFuL0FyY2hpdmVyL0h5cGVyQXJjaC5w
eSIsIGxpbmUgNzM4LCBpbiB3cml0ZV9pbmRleF9oZWFkZXIKICAgIHNlbGYudXBkYXRlVGhy
ZWFkZWRJbmRleCgpCiAgRmlsZSAiL2V4cG9ydC9wYXJyb3QvbWFpbG1hbHQvTWFpbG1hbi9B
cmNoaXZlci9waXBlcm1haWwucHkiLCBsaW5lIDI1OCwgaW4gdXBkYXRlVGhyZWFkZWRJbmRl
eAogICAgc2VsZi5kYXRhYmFzZS5jbGVhckluZGV4KHNlbGYuYXJjaGl2ZSwgJ3RocmVhZCcp
CiAgRmlsZSAiL2V4cG9ydC9wYXJyb3QvbWFpbG1hbHQvTWFpbG1hbi9BcmNoaXZlci9IeXBl
ckRhdGFiYXNlLnB5IiwgbGluZSAzMDUsIGluIGNsZWFySW5kZXgKICAgIGRlbCBzZWxmLnRo
cmVhZEluZGV4W2tleV0KICBGaWxlICIvZXhwb3J0L3BhcnJvdC9tYWlsbWFsdC9NYWlsbWFu
L0FyY2hpdmVyL0h5cGVyRGF0YWJhc2UucHkiLCBsaW5lIDg1LCBpbiBfX2RlbGl0ZW1fXwog
ICAgc2VsZi5zb3J0ZWQuc29ydCgpCiAgRmlsZSAiPHN0cmluZz4iLCBsaW5lIDEsIGluID8K

--YhIK0mKoxU
Content-Type: text/plain; charset=us-ascii
Content-Description: message body text
Content-Transfer-Encoding: 7bit


Looks like the archiver is doing way too much work for every message
it has to process.  When python.org came back up today, it got slammed 
with incoming mail for a bazillion lists.  Each message spins in this
HyperDatabase.clearIndex() loop.

-Barry

[1] CVCT:

Use gdb to attach to the running Python program, then type this at
the gdb prompt:

(gdb) call PyRun_SimpleString("import sys, traceback; sys.stderr=open('/tmp/tb','w',0); traceback.print_stack()")

Sitting in /tmp/tb will be the stack trace of where the Python program 
was when you stopped it.  There's reason to believe this will not
always work, but it likely will, and you can even detach the program
and let it continue on.

--YhIK0mKoxU--