[Mailman-Developers] before next release: disable backscatter in default installation

[Ian Eiloart]

--On 1 April 2008 03:11:17 +0200 Mads Kiilerich <mads at kiilerich.com> wrote:

> Mark Sapiro wrote, On 03/31/2008 06:26 PM:
>> So what do I do practically in this case. Calling out to verify the
>> recipient won't help because the recipient is valid.
>
> I think that we - admins and developers - have to realize that times are
> changing.

agreed.

> Once upon a time when the RFCs were written then mail was asynchronous
> store-and-forward. MTAs accepted mails before being 100% certain that
> they could and would deliver them, and as soon as the mail had hit the
> disk then the receiving MTA took over delivery responsibility from the
> sending MTA.

true.

> In such a scenario Mailman "is just" a MTA with bouncing/holding
> filtering and dynamic alias expansion of envelope-rcpt and replacement of
> envelope-from with a robot address.

yes.

> But now in modern times all mail handling has to be synchronous.

that would be nice.

> Spam and
> virus and backscatter does that a MTA should never take responsibility
> for anything from an untrusted sender before the mail has actually been
> taken fully care of by someone - perhaps just by making it the problem of
> the next MTA/MDA. Yes, doing this all the way to the final mailboxes
> might be more expensive than the old way of doing it - but at the same
> time the same amount of work (or less?) has to be done, so it can't be
> that bad ... except for the number of live connections. I think that the
> RFCs allows fully synchronous mail handling, they just have to be used
> another way than they usually are today.

No, they don't allow fully synchronous mail handling. The problem is that 
much spam filtering depends on message content (headers and body). And, 
different recipients have different views on what is acceptable content. 
The RFCs don't allow us handle differences of opinion, because SMTP has no 
way of saying "foo at example.com" will accept this message, but 
"bar at example.net" will not.

In theory, this could be dealt with by adopting LMTP instead of SMTP. LMTP 
does allow us to reject or defer for individual recipients after we've seen 
the message content. But that's always going to be a slow process, and 
it'll be a brave sysadmin who first says "you can only deliver to us using 
LMTP".

The first step to this would be an extension to SMTP which allows a server 
to say "you may switch to LTMP" if you wish. MTA's might be configured to 
switch to single recipient deliveries when LMTP wasn't available. But, what 
spammer would use LMTP if SMTP were available?

It's nice to know that Mailman 3 is likely to have an LMTP engine, because 
it will be possible for lists to use different content-based rules, and 
reject (not bounce) messages to multiple lists depending on individual list 
rules. For example, one list might reject a message with attachments. Of 
course, this will only push the problem back to the local MTA, but that is 
at least some progress.

> In Marks specific case in this scenario: The verification phase has been
> replaced by actually doing the delivery all the way to the final
> recipients, so the recipient ending up being invalid will be handled
> properly.
>
> In this scenario Mailman is a MTA which decouples the synchronous chain.
> Mails are accepted and enqueued by Mailman iff responsibility is taken,
> either for immediate delivery or manual review - all other mails are
> rejected. All filtering thus has to be done synchronously. Delivery to
> list members is a separate process where target mail hosts probably will
> provide synchronous delivery status feedback (perhaps as DSN bounces from
> a trusting "real" MTA) (and if other mail hosts chooses to accept and
> then bounce later then they have a problem, but Mailman can still handle
> receiving bounces).
>
> Lots of details are missing in this description. But do you agree that
> this is the direction we have to take as developers and admins?

That would depend on whether we thought converting the world to LMTP is 
worth the effort. Perhaps it is, but not in my view simply so that we can 
fight spam. The real problem that we have to solve is trusting 
return-paths. At the moment, we can't trust or distrust anyone because it 
is so trivial to forge return-paths. SPF and DKIM offer solutions to 
authenticate senders, and hold domain owners accountable for email that's 
sent.

Once they're widely deployed, we will be able to make reasonable decisions 
about whether to trust the email sender based on the sending domain, or 
even on the individual sender.

At that point, we'll be able to synchronously chain callouts, without 
requiring to look at any message content. Sites will reliably be able to 
blacklist or whitelist based on Mail From.  We'll be able to build trust 
networks (like RBL lists, but with domains and email addresses, not IP 
addresses), and we won't need to scan content at all.

> /Mads
>
> ps: This might be a bad description of trivial observation, but it
> suddenly made sense to me and I wanted to share it, even though I'm
> mostly a lurker on this list. Take it or leave it ;-)
>



-- 
Ian Eiloart
IT Services, University of Sussex
x3148