[Mailman-Users] Executive summary of DMARC issues

[Barry S. Finkel]

On 5/14/2014 2:49 PM, Stephen J. Turnbull wrote:
> Gary Algier writes:
>
>   > I have been following the discussion of the DMARC issues and Mailman's
>   > attempts to live with it.  I was wondering if anyone has an "Executive
>   > Summary" of the DMARC issue in a general sense.
>
> How about the following:
>
> DMARC is a set of protocols for Internet mail that are used to by
> participating hosts to exchange information about mail traffic.  This
> information is primarily useful for fighting email fraud, including
> spam and phishing.  The basic procedure is that recipient hosts which
> participate in the protocol check whether the apparent sending domain
> (the domain that appears in the address in "From" header field)
> participates, using a query to the DNS defined by the DMARC protocol.
> If so, the recipient host will send to that domain some information
> about mail which purports to be from the sending host, but fails to
> validate.
>
> The validation refers specifically to, and only to, the domain in the
> "From" header field.  The DMARC protocol itself does not validate the
> mailbox (user), although some originating hosts may do so.  Nor can it
> guarantee that the user's account hasn't been used fraudulently.
>
> For participants in the DMARC protocol, valid mail provides a strong
> guarantee that the domain of the address in the "From" header field
> is authentic, with some strong restrictions required to ensure success
> of validation.  These restrictions include:
>
> (1) When authentic mail is delivered directly from the sending host to
>      the recipient, validation will succeed.
>
> (2) When authentic mail is forwarded verbatim by all intervening
>      Internet hosts (including mailing lists and user mailbox
>      forwarding), validation will succeed.
>
> Authentic mail may fail to validate in many common circumstances,
> where a forwarding host alters certain headers or any part of the body
> of the message.  This may occur for forwarding hosts which add
> advertisements to the footer or the like, or if a forwarding host
> reformats the body (it shouldn't do that, but some do).  There are
> also restrictions on some parts of the mail.  For example, the "From"
> header field must contain exactly one address (multiple author
> addresses are permitted by the mail standards and are occasionally
> useful, though rarely seen nowadays).
>
> Failure to validate is also frequent on mailing lists.  Common
> transformations that *will* cause mail distributed by a mailing list
> to fail validation include (but are not limited to)
>
> - Adding a list tag or sequence number to the Subject header field.
>
> - Adding a footer, typically containing unsubscribe information for
>    public discussion lists or legal disclaimers for corporate lists.
>
> - Removal of banned content, such as executable file attachments or
>    excessively large attachments.
>
> Because many common practices with Internet mail can cause authentic
> mail to fail DMARC validation, although mail which successfully
> validates may be trusted to be from the domain in the address in the
> "From" field, failure to validate in general *does not* imply lack of
> authenticity.
>
> Under some circumstances, a sending domain may be willing to severely
> restrict its users' use of their addresses.  For example, they may not
> be allowed to post to mailing lists, and recipients may be advised to
> *always* keep their addresses updated to the domain where they
> actually read their mail.  This is common practice with banks and
> other financial institutions.  Such organizations can be reasonably
> sure that mail which fails to validate is "fraudulent" (although the
> intent may be merely mischievious rather than truly felonious).  They
> may take advantage of a further feature of the DMARC protocol, the
> "policy", which is the action that the sending domain suggests that
> recipients take with mail that appears to be from that domain but
> fails validation.
>
> The normal policy is "none", that is, the recipient should deliver the
> mail as though the apparent sending domain did not participate in
> DMARC.  (Of course, if the mail fails spam or virus checks, it would
> be quarantined or discarded as usual.)  The second level is
> "quarantine", which means that the mail should not be delivered
> directly to the user's mailbox in the ordinary way, for fear of
> fraudulent use of the sending address (typically phishing).  The third
> level is "reject", which means that the sender recommends that the
> mail not be delivered at all, to completely prevent fraudulent misuse
> of their domain.
>
> Unfortunately, unless the sending domain has truly draconian policies
> about the use of addresses in that domain, use of "quarantine" or
> "reject" makes it virtually certain that some authentic mail will not
> be read in a timely fashion or (in the case of "reject") never
> delivered at all.
>
> The "reject" policy may also result in denial of services to third
> parties (one common case is having delivery disabled from a mailing
> list, or even being unsubscribed), due to rejected mail being
> "bounced" back to the forwarding host.  At present we know of *no*
> hosts which distinguish DMARC bounces from other delivery failures.
> Therefore they are treated as a permanent delivery failure to the
> recipient due to problems at the recipient's host, rather than due to
> a policy of the sender.  As a consequence, those unrelated recipients
> may be dropped from distribution lists.
>
> Editorial comments:
>
> In some cases (where the mail validates according the DKIM, one of the
> protocols used in DMARC validation), the mail can be trusted not to
> have been altered in any way by a "man in the middle".
>
> It is entirely up to the recipient whether to honor the published
> policy or not.  (GMail, for example, does not -- it treats the policy
> as advisory and makes case by case decisions, allowing most list mail
> to be delivered despite failing DMARC validation.)  However, because
> the original use of policy was envisioned to be banks vulnerable to
> phishing (not large freemail providers incapable of protecting their
> users' address books), most recipients do honor the policy published
> by sending domains.
>
> The action of Yahoo! and AOL in publishing policies of "reject"
> basically amounts to deciding that they do not care if their users'
> mail can be reliably delivered.  Nevertheless, Mailman is committed to
> providing mitigation options for mailing list owners that improve the
> reliability of mail delivery for posters from those domains, and
> protect other subscribers from the (unintentional) denial of service
> attacks being conducted by Yahoo! and AOL.
>
> I believe the word "attack" is justified because Yahoo! and AOL are
> now aware that their policies have the effect of denying service to
> third parties (even if not intended by them), but they continue to
> implement them.  It seems likely to me that as mailing lists and
> others implement workarounds that bypass DMARC validation, these
> workarounds will be mimicked by spam and phishing mails to bypass
> DMARC validation, and in response AOL and Yahoo! will take further
> action detrimental to the reliability of everyone's mail service, not
> just their own users'.
>
>   > Of course if someone says that the current MS365 implementation has
>   > addressed this, then that's a different (unfortunate) story.
>
> Seems unlikely to me (this isn't a new variant of a traditional
> problem to be addressed by database updates like virus checking),
> although they probably will roll out something within a couple months
> (counting from April 22).  Nevertheless I doubt it will be as flexible
> as Mailman already is (and most likely will be tuned to protect
> Hotmail users if Hotmail should decide to use the "reject" policy).
>
>   > Nielsen's First Law of Computer Manuals:
>   >      People don't read documentation voluntarily.
>
> How (sadly) apropos!
>
> HTH
>
> Steve

Is this also true?

Users from DMARC-reject domains send mail to mailing lists, and the
resulting mail from the mailing list is rejected.  Enough rejections
can cause the mailing list possibly to be blacklisted for sending lots
of "spam" mail.

--Barry Finkel