[Mailman-Developers] very large lists

[Bob Puff]

---------- Original Message -----------
From: Ian Eiloart <iane at sussex.ac.uk>
To: Bob Puff <bob at nleaudio.com>, Mailman-Developers at python.org
Sent: Tue, 02 Jun 2009 10:49:39 +0100
Subject: Re: [Mailman-Developers] very large lists

> --On 2 June 2009 00:16:39 -0400 Bob Puff <bob at nleaudio.com> wrote:
> 
> > Hi gang,
> >
> > I have a customer who wants a one-way distribution list set up that will
> > handle millions of recipients... like 10 million.  It is a double opt-in,
> > and its not spam.. <g>
> >
> > I'm thinking this is probably too much for MM 2 to handle.  Will MM3 be
> > able to scale to this size?
> >
> > If anyone has suggestions for me for current software that can do this,
> > feel free to email me off-list.  Bounce processing is important, as is
> > user management.
> 
> You've not specified the problem very clearly. How frequently do you 
> expect to send messages to this list? Do you have several lists? How 
> urgent are the messages? Do they need to be delivered in the space 
> of a few minutes, hours, or days?

Hi Ian,

The problem I've seen is that with the "large" lists I have now (up to
10,000), python is the dominate process taking up time.  I recall there being
some discussion about how the list data files are locked during bounce
processing, preventing parallel processes from doing much.

I'm not sure its physically possible to deliver a million emails in a matter
of a few minutes or even a few hours, unless its a bot-net! :-)  Even on a
100mb link, I would expect more on the order of many hours.  Messages would be
sent at a maximum of one per week.
 
> If you don't need personalised messages, then you can do the VERP in 
> your MTA. That should make it much more feasible. Configure Mailman 
> with several parallel queue runners to prevent messages to large 
> lists from holding up messages to small lists.

What and where would be the config statement specifying the number of queue
runners?

Bob