spam classification breaker

[Tim Peters]

[Robin Becker]
> This article at the BBC reports on what appears to be a genetic
> algorithm or random search method for finding words that apparently
> fool bayesian classifiers every time.
>
> http://news.bbc.co.uk/1/hi/technology/3458457.stm
>
> The author apparently had to include html reporting into the emails to
> allow his mail client to report back automatically.

If I'm a spammer trying to get my pitches seen by you, and you're using a
personal Bayesian classifier, then I need to load my pitches with words that
are very hammy to you.  If I don't have access to your personal training
data (if I do, I already own your machine ...), then I need to *deduce*
what's hammy to you.  One way to do that is, as John Graham-Cumming noted
here, is for me to send you thousands of messages with different piles of
words, and note which ones did and didn't get caught by your filter.   Then
I load my sales pitches with words from the ones that your filter didn't
reject, and avoid words from ones your filter did reject.  In order to do
that, I have to know which messages you did and didn't look at.  That's the
purpose of the HTML "web bug"/"web beacon"s in the thousands of test
messages.  (If your email client renders HTML pages, including fetching
images off the net, a spammer can know when you've rendered their message,
by, e.g., embedding your email address as a parameter in a URL that fetches
a .jpg to display.)

> Of course if he'd used python the whole process of email generation
> and classification could have been done in a single process and would
> probably allow easier generation of the magic words.

I have to deduce *your* magic words, not mine.  I have to send email to you,
and deduce what you did and didn't look at.  This is an expensive process
for the spammer, of course.

> Why Berkshire, Marriot etc should be allowed through is pretty strange :)

It's one reason personal Bayesian classifiers are hard to beat.  Perhaps Dr.
Graham-Cumming attended a conference in Berkshire, and stayed at the
Marriott there.  It's not hammy to me, but it is to him.

There's a much cheaper way to beat these classifiers, which I've seen in the
wild, but very rarely:  I send spam to you by attaching it as a reply to a
message you posted to a public mailing list.  I retain your original subject
line, and the full text of your original message.  It's almost certain that
the things you post to public mailing lists contain words that are hammy to
you.  But this is still much more expensive for me than just blasting a
single email to millions of addresses in bulk, and it's not an effective
*sales* pitch:  it will get through your filter, and everyone looks at
replies to their own messages, but when you see that it's really an
advertisment you're much more likely to be pissed at the sender than to
givee them money <heh>.  Since the real goal of spam is to sell product,
it's not enough just to evade filters.