[Mailman-Developers] Patching qrunner to never send duplicates w/ same Message-ID

[Ben Gertzfield]

I'm interested in killing two birds with one stone; I'm presently
moving a company's massive (> 1200 line) aliases file that has grown
unmaintainable to Mailman, so that the sysadmins can let the list
admins maintain their own subscriber lists.

The one problem is that people at the company are used to Cc:ing
multiple aliases and having only *one* copy of the email reach the
final recipient, even if the recipient is a member of multiple aliases
in the Cc: list.  Sendmail would handle this internally with the
old way of using one massive aliases file, because it would expand
all aliases and only send one mail to each final address.

This has been discussed before on mailman-users at
http://mail.python.org/pipermail/mailman-users/2000-September/006880.html
-- the answer there was of course "force the users to use a different
MUA". Yes, I know the proper solution is to have the MUA ignore
duplicate mails with the same Message-ID, but it's impossible to get a
company to all switch to mailers that support this.

I would like to work on a different solution; I think it might be
possible to modify the Mailman queue runner to never send out multiple
messages with the same Message-ID to the same recipient.  Looking
through the source in Mailman/Queue, I see that each mail's
destination is put into its own marshalled .db file in
/var/lib/mailman/qfiles, so we will have to keep state to avoid sending
duplicates.

The way Gnus (the best mail reader on the planet, natch) handles this
is by keeping a cache of the past X Message-IDs (1000 by default,
obviously tuneable); we could pretty easily keep a list of Message-IDs
and recipient pairs, and do one of the following if a message is about
to be sent out to a recipient that has already received an email with
that Message-ID:

1) Do nothing. (to keep current behavior)
2) Add a warning header to let the users filter on their own
3) Don't send out the message.

This could be configurable per-user, and I think it'd be a great
improvement over the current state of things.

Would a patch for this be accepted?  At which point of the queue
runner do we want to deal with this? Mailman.Queue.OutgoingRunner
might be the right place, but I'm not sure.

Ben

-- 
Brought to you by the letters L and J and the number 12.
"A calpac is a large cap."
Debian GNU/Linux maintainer of Gimp and GTK+ -- http://www.debian.org/