[Baypiggies] Challenge/Response email systems

[Marilyn Davis]

I'm going to trim a lot out of these messages.  I hope no one feels I've left out any important context.  It is not my intention.

----- On Thursday, June 15, 2006 bob at redivi.com wrote:

>>
>> You did answer the challenge, Guido.  So I got that email twice.
>> Maybe you couldn't resist because it said door.py in the url?
> 
> Could've easily been someone else, given that the challenge URL ended
> up on a public mailing list.
> 

Of course you're right.  Thank you someone.  I'm very glad to have Guido on my white list so he'll never face the challenge again.  In fact, I'll add *@python.org to my white list.

> 
> C/R doesn't scale because spoofed challenges are indistinguishable
> from real ones and you can't filter any of them out without filtering
> all of them. If everyone did C/R you'd simply get hundreds of non-
> filterable challenges a day instead of spam, and you would have no
> idea which ones to respond to.

A spoofed challenge won't get into my email box.  It will be sent a challenge!  If I send someone a message, they are automatically on my white list.  So if they generate a challenge, I'll see and answer it.

----- On Friday, June 16, 2006 ken at seehart.com wrote:

> My understanding is that there are a couple kinds of CR systems.  One
> kind is blind, and sends a
> challenge to anyone not on a whitelist (call this Blind CR), and
> another kind that only sends a challenge
> when the email is:
> 
>   Not on whitelist.
>   Has a moderately high spam score.
>   Doesn't have a super high spam score (just dump these in the spam
> bucket).
> 
> 
> Obviously, the most common objections apply to blind CR systems.  A smart
> CR system would
> only generate a relatively small percentage increase to email traffic
> in exchange for putting spammers
> out of business (a reasonable exchange I think).  In general I would
> receive a challenge only when
> it is likely that the person receiving my mail would have discarded it
> if they had not installed a CR
> system.
> 
> Anyway, this discussion has left me "undecided" on the issue.  Leaning
> against CR at the moment.
> 
> I have two questions:
> 
> 1. Marylin: Is your CR "smart" or "blind"?  I can't imaging why the
> email Guido sent you would have
> gotten a "moderately high" spam score.

Blind so far. 

My boss (son), so far, doesn't want any spam scoring.  We do it blindly for all unknown addreses.

> 
> 2. Does anyone have an answer to Bob's objection?  That looks like a
> killer to me.  Although
> I have not yet received any spoofed CR messages, I could imagine it
> could become a very
> popular technique for spammers.  I don't want to complicate things by
> simply adding another layer
> to the battle (CR spoofs vs. CR spoof filtering).  If Bob's objection
> is not answerable, CR is dead.

Answered above.

> 
> 3. What about metaCRs?  Maybe my CR system sends you a challenge about
> your CR.  Yuck.

Your CR is badly broken if, when you write to someone, that someone isn't added immediately your white list.

> 
> 4. Is there anything as bulletproof as CR, but without the problems? 
> And I am not asking about
> better filters (that's just an ongoing battle against better spam).
> 
> - Ken

> 
> I know the 'designated sender' system(s?) were causing a stir a while back...
> I forget the acroynm but I believe it worked by adding DNS records
> indicating which mail servers were 'authorized' to originate mail for
> a particular domain.  I believe AOL was backing/implementing a version
> of it.  Don't know what the outcome was, but I sure someone on the
> list can shed more light on it.
> 
> Steve

"SPF"  Sender Policy Framework. It is good for proving that mail originating from python.org came from python.org.  But it is not good for proving that mail from python.org is illegimate if python.org provides portible addresses.  Also SPF breaks with forwarding.

Wikipedia has a nice discussion:
http://en.wikipedia.org/wiki/Sender_Policy_Framework#Controversy


----- On Friday, June 16, 2006 DennisR at dair.com wrote:

> At 08:58 AM 6/16/2006, you wrote:
> 
>>4. Is there anything as bulletproof as CR, but without the problems?
>>And I am not asking about
>>better filters (that's just an ongoing battle against better spam).
> 
> I think so and have implemented such in my latest version of SpamAI:
> 
> Guilty until proven innocent
> 
> Only whitelisted email is directed to your inbox.  The rest is reviewed or
> ignored as you see fit.

The review/ignore part isn't practical for me.  My address has been around for so long, and I run so many email lists that I get constantly spammed, I can't review them.

> 
> Because this algo and C/R are both explicit whitelist systems, they are
> equally bulletproof.
> 
> Now on to "the problems"...  A C/R "shares" *your* spam problem with people
> who send to you. 

I need to comment that in all other communication modes, the sender is the one responsible: to put a stamp on mail, to ring the doorbell, to dial the phone.  

> Now, there is going to be an asymmetry in who finds the
> message valuable.  Consider this table based on who finds the message
> important:
> 
> hi importance to sender, lo importance to receiver
> The sender does the work to go through C/R and the recipient gets a message
> they are not interested in.  Sender is irritated but so what?  - for C/R

And I need to comment that the only people who have ever expressed irritation are technical people.  Regular people, so far, think it is totally cool.  I think the "irritation" of a challenge is pretty low compared to all the broken windows stuff that people are used to.

If the mail wasn't important to the sender, why did she send it in the first place? If you are sending stuff that you don't care about, what are you doing? And what is lost if I don't see it?  I really don't want to see mail you send me if you don't mean it.

> 
> low importance to sender, lo importance to receiver
> message is lost. + for C/R
> 
> hi importance to sender, hi importance to receiver
> The sender does the work and recipient receives message but at the cost of
> irritating sender on something both consider important.  Both + and - for C/R
> 
> lo importance to sender, high importance to receiver
> The message is lost and the sender is *really* irritated. - for C/R

The sender is irritated because she didn't answer the challenge and the mail was important to her?  Well, that person is very irritable indeed and maybe I'm better off without having her in my email.

I'll confess I'm a little confused about the usefulness of this analysis.

> 
> Including the ambiguous case, there are 3 - and 2 + for C/R.  C/R works
> less well when there is an asymmetry in the importance of the
> conversation/message.  I think the sender's responsibility ends when they
> send their message.  From that point forward, it is the recipient's sole
> responsibility to act/not act.

Take Guido's case.  Answering the challenge was unimportant to him and I wonder why he cc-ed me.  Losing his private email was inconsequential for me too.  I got it from the list.

However, if Guido really needed me, to teach a class tomorrow night, to fill in for someone, or something, would he stand on his principle if I could solve a problem for him?  If so, I wouldn't know.  It would be his loss.

The issue that no one has brought up is the 'Joe Job', discussed ad nauseum on the exim mailing list.

I have to run now but I'll talk about it later if anyone demonstrates interest.

And then there's the object-model implied, and the socio-political model.

Thanks for discussing.

Marilyn