[Mailman-Developers] [ mailman-Bugs-558988 ] bad performance for big queue dirs

[Norbert Bollow]

Chuq Von Rospach <chuqui@plaidworks.com> wrote:

> >   NB> Now the welcome messages are going out, at a rate of one
> >   NB> message every 6.6 seconds.  This is on inexpensive hardware (a
> >   NB> Cobalt RaQ3) but the server is otherwise idle.
> 
> > There are a lot of unknowns here though.  For example, we don't know
> > if your rate limits are caused by MTA or network throttles.

Trust me, they aren't.  Whenever Mailman would pass one of the
messages to the MTA, it would go out almost instantanously.

> If it's a cobalt, it's probably sendmail.

It's qmail with a couple of patches, including big_concurrency,
which isn't relevant for this test because the MTA gets one
message per 6.6 seconds only.

> Then you need to look at disk I/O and whether you're MTA is thrashing
> because of access contention on the mqueeu directory(s). I'd put the chance
> of qfile issues really low on the list of things to check here.

Well, I deleted those almost 40000 files from welcome messages
(as I didn't have 36 hours of patience :-), added 20000 real
subscribers (_witout_sending_them_welcome_messages_) and it all
works ok now.

This issue is certainly related to the size of Mailman's queues,
in this case the virgin queue.  If it's not a matter of
filesystem (in)efficiency, then it must be Python or Mailman
doing something that scales extremely badly with queue size.

Greetings, Norbert.

-- 
A founder of the http://DotGNU.org project and Steering Committee member
Norbert Bollow, Weidlistr.18, CH-8624 Gruet   (near Zurich, Switzerland)
Tel +41 1 972 20 59         Fax +41 1 972 20 69        http://norbert.ch
List hosting with GNU Mailman on your own domain name   http://cisto.com