[Spambayes] Virus Bulletin Anti-spam Testing

[Martijn Grooten]

Dear Amedee, others,

Thank you for your email.

To answer your question: if I understand it correctly, Spambayes takes an email message (header + body) as input and outputs either 'spam' or 'ham'. (I know many products actually output a likeliness percentage, but by setting a threshold this has the same effect.) This does not necessarily have to be a problem: we could possibly install Spambayes on the server which redistributes the email and, once an email is redistributed, make it check whether it thinks the email is ham or spam and store that in the database. Note, however, that for various reasons, we do not 'teach' the filters by providing them with end user feedback.

On the (separate) test where the filters are exposed to the stream of a large spamtrap, we could use any free SMTP daemon. I assume that the actual daemon used should not matter for its performance.

I do have one concern though: we are happy to include free, open source products in our test (and, unlike commercial products, will test them for free). However, we want to be sure that the developers agree on submitting the product. In particular, we don't want to be the source of an argument among developers about submitting a product.

Best Regards,

Martijn Grooten
Anti-spam Test Director
www.virusbtn.com
Email: martijn.grooten at virusbtn.com
Tel: +44 1235 540235 / +44 7872 674989


-----Original Message-----
From: Amedee Van Gasse [mailto:amedee at amedee.be]
Sent: 13 January 2009 22:55
To: Allison Sketchley
Cc: spambayes at python.org; Martijn Grooten
Subject: Re: [Spambayes] Virus Bulletin Anti-spam Testing

Allison Sketchley schreef:
>
> You might have already learnt from other sources that Virus Bulletin is
> planning to implement anti-spam testing in the near future and I am
> writing to you to see if there might be any interest to participate in
> this new and exciting venture with your open source anti-spam product.
>
> I have attached by way of information the article published in the
> January edition of the Virus Bulletin magazine which outlines the
> proposed test methodology, as well as a recent press release announcing
> the new tests.
>
> Please feel free to pass this information onto any other potentially
> interested parties within your organisation and if you, or they, should
> have any questions or feedback please contact Martijn Grooten, Anti-spam
> Test Director (martijn.grooten at virusbtn.com
> <mailto:martijn.grooten at virusbtn.com>) for any technical based queries
> or myself for any financial/administrative queries.
>
> Look forward to hearing from you.

In the proposed test methodology you write the following:

"To take part in Virus Bulletin’s anti-spam tests, products
must be able to accept SMTP transfers and classify email
into two categories: ham and spam."

According to the Spambayes FAQ on
http://spambayes.sourceforge.net/faq.html, Spambayes is not an MTA
(postfix, exchange,...) spam filter. It works by sitting between the MTA
or the MDA (procmail,...) and the MUA (outlook, thunderbird, mutt,...)
as a POP or IMAP proxy. In case of the Outlook plugin and the upcoming
Thunderbird plugin, it even works *after* the MUA.

I know that there have been some people who have tried to implement
Spambayes as a MTA spamfilter, but these attempts were all very
experimental, and they were done by individual Spambayes users. I don't
think that there will ever be an "official" build for Spambayes at the
SMTP. There is always SpamAssassin for those who want that.

However, I am very interested to see how other Software Libre products
perform that do work on SMTP level, like SpamAssassin. How well do they
compare to commercial products?


Disclaimer: I am not a Spambayes developer, I'm just a regular user.
By sending your email to a public mailing list, you have agreed that any
user of Spambayes may read your email, and that any user may reply to it.

Kind regards,
Amedee Van Gasse

Virus Bulletin Ltd, The Pentagon, Abingdon, OX14 3YP, England.
Company Reg No: 2388295. VAT Reg No: GB 532 5598 33.