[Mailman-Developers] ARC module implementation [was: GSOC 2016]

[Stephen J. Turnbull]

[Aside @Barry:  what's the current state of DMARC handling in Mailman
3?  (I assume below that there isn't any yet.)  I guess porting some
or all of the Mailman 2 facilities be a good GSoC project?]

Please keep GSoC traffic on Mailman-Developers.  Other developers have
need-to-know, as do other students.  Also, there's a good chance one
of the other developers will get around to answering before I do,
especially as I just got a pile of work dumped on me at work (one of
our adjunct faculty got suddenly recalled to the Bank of Japan,
leaving 8 units of courses unstaffed from April 1).  I'll stay current
with Mailman as much as a I can, but for the next few weeks that means
responding to non-work mail in 48 hours.

 > The deliver method in deliver.py is responsible for sending the
 > messages, and the errors are caught here and then classified as
 > permanent or temporary. So currently, if I am not wrong, all the
 > rejections that Mailman faces due to the dmarc reject policy of
 > some domains, permanent error 550 I think, are caught here if there
 > are any.

I don't think so.  You'll not catch them in mta.deliver.py, which
talks to your outgoing SMTP gateway, which normally does not do DMARC
processing.  On the other hand, the outgoing gateway doesn't make
Mailman wait until it knows what its remote peer will do; it queues
the message.  So any rejections you see in mta.deliver.deliver() are
a local matter.  Permanent failures likely indicate a configuration
problem, while temporary failures are probably due to maintenance on
the MTA or network problems or the like.

It's the final recipient MTA[1] that does DMARC processing, perhaps
hours or days after deliver() has returned :-).  At that point a
(new!) bounce message is created and sent back through the mail
system to Sender or Errors-To, which is <list>-bounces.  So this is
handled by the bounce runner, not the outgoing runner.

 > I was curious to know how this specific problem is being handled by
 > mailman right now. As in, how are the cases of subscribers of yahoo
 > using mailman currently being handled?

First, subscribers using Yahoo! are not the problem.  The DMARC
problem is that when anybody from Yahoo! posts, *all* subscribers are
at risk[2] of having their service bounce the post.  If it was just
Yahoo! subscribers, we wouldn't have to do anything except tell them
that their mail service is deliberately sabotaged by Yahoo!  It's the
collateral damage to well-behaved third parties that makes DMARC abuse
by AOL and Yahoo! so horrible.

Second, in Mailman 3, currently nothing is done AFAIK.  The first
patch was submitted against Mailman 2 before release of Mailman 3.0.
People using Mailman 3 at this point are unlikely to have large
populations of p=reject posters.

In Mailman 2, there are a number of options.  The one I recommend is
adding aol.com and yahoo.com to the subscription and poster ban
lists.<0.7 wink/>  This has the advantage (?) of also being usable in
Mailman 3.  A second is to configure the list without transforming the
message at all (no Subject tags, header, or footer), thus preserving
any valid DKIM signature.  This also works in Mailman 3.  These are
standard configuration options which you might use anyway for
completely different reasons.  However, they sort undermine the
argument for using Mailman in the first place for many sites.

The third, more pleasant, alternative is to configure the list to
transform the message by changing the *address* in RFC5322.From to the
list, and putting some amount of user identity in the display name of
the RFC5322.From (eg, "From Aditya Divekar (adityadivekar03 at gmail.com)
via Mailman-Users") of all messages.  A fourth is to treat the message
as a literal forward by including the message as a message/rfc822 part.

These transformations are done by Mailman Handlers.  Each has
disadvantages.  From-munging basically prevents the recipient from
filtering on From.  Few MUAs know how to handle a message "wrapped" in
a MIME part pleasantly, and some (especially AppleMail on iPhone)
choke badly.

Finally, the most sophisticated alternative is to parse the address
out of From, and do the DMARC DNS dance to determine if the sending
domain has a p=reject or p=quarantine policy.  If so, use either the
From-munging strategy or the MIME-wrapping strategy.  These still have
the disadvantages described above, but they only apply to posts From
domains abusing DMARC.

I don't think any of these would be that hard to port to Mailman 3, it
just hasn't been done yet.

Footnotes: 
[1]  This concept is a little ambiguous, in the case of things like
.forward files that point to another mail server.

[2]  If their mail service participates in DMARC (there's no way of
knowing that unless you're a user there), and if they don't do their
own mitigation (eg, GMail treats many messages from p=reject domains
as p=quarantine).