[Mailman-Users] spam filter

[Dan Wilder]

On Thu, Jan 16, 2003 at 08:27:17AM -0700, Kory Wheatley wrote:
> Hi,
> 
> I would like to discard any email that comes from the from address
> "big at boss.com".  I know you can put a filter value in the mm_cfg.py and
> that will filter mail to be held for moderation; for example,  "from:
> .big at boss.com", but I would rather have it be automatically discarded,
> so there is no need to manually discard these emails when they are held
> for approval.
> 
> I would need this for mailman 2.0.13 and mailman 2.1

Why not stick with the "small sharp tools" philosophy of UNIX and
use the existing tools that more than adequately address the problem,
rather than trying to reinvent a solution?

Procmail (to name just one tool) is very good at sorting out this
kind of thing.  It doesn't take much to set up.  In the 
MTA's aliases file, instead of passing directly to the Mailman
wrapper, use a line like:


---------- snip -------------
listname: "|/usr/bin/procmail -m /etc/procmailrcs/listname"
---------- snip -------------

for a list called "listname".

The /etc/procmailrcs directory is owned either by root.root
or by the list user and group.  The list procmail rcfile

/etc/procmailrcs/listname

is owned by the list user and group.  Near the beginning
is a line that says:

LIST=listname

In it, along with whatever else you might use, you put a recipe:

---------- snip -------------
:0
* ^From: .*\.big at boss\.com
/dev/null
---------- snip -------------

which consigns mail containing a header such as
the one you mention to oblivion.

You then have much more flexible and sophisticated mail sorting
than is reasonable to expect Mailman ever to implement.

You can discard postings with duplicate bodies:

---------- snip -------------
# Passthrough recipe
:0fw
| /usr/lib/mailman/localbin/dup.py /var/log/mailman/$LIST.db

# If preceding inserted X-Duplicate header, consign to oblivion
:0
* ^X-Duplicate: yes
/dev/null
---------- snip -------------

(dup.py near end of this email)

You can invoke SpamAssassin and put things flagged as spam
in your log directory:

---------- snip -------------
:0fw
| spamc

:0:
* ^X-Spam-Status: Yes
/var/log/mailman/$LIST-spam
---------- snip -------------

At the end of the procmail rcfile is the wrapper recipe:

---------- snip -------------
:0
| /usr/lib/mailman/mail/wrapper post $LIST
---------- snip -------------

which delivers anything not disposed of by an earlier recipe.


A python script to tag duplicates, useful in procmail rcfiles:

----------------------- snip ----------------------------
#!/usr/bin/python
#
# Copyright 2002 Specialized Systems Consultants, 
# publishers of Linux Journal
#
# You are licensed to use this software under the provisions
# of the Gnu General Public License, version 2 or later.
#
# Suggestions and patches to: Dan Wilder <dan at ssc.com>
#
# Tue Oct 29 09:24:35 PST 2002 dhw

"""
dup.py -- Filter a piece of email, inserting an

X-Duplicate: yes

header if we've seen this thing before.

Usage: dup.py dbfilename <inputmail >outputmail
"""

import fcntl
import shelve
import md5
from sys import stdin, stdout, argv, exit

if len(argv) < 2:
    print("""
Usage: dup.py dbfilename <inputmail >outputmail
""")
    exit(1)

# Get db file name and related lock file name

dupdb = argv[1]

parts = dupdb.split("/")

duplock = parts[:-1]
duplock.append("." + parts[-1:][0] + ".lock")
duplock = "/".join(duplock)

# Grab the email message
# Break up the mail into head and body text strings.
# Omit X-Duplicate header, if found.
# Thanks, we'll add our own if we want one.

head = []
body = []

for line in stdin.readlines():
    if body or line == "\n":
        body.append(line)
    elif line.find("X-Duplicate: yes") != 0:
        head.append(line)

head = "".join(head)
body = "".join(body)

# calculate body md5sum

sum = md5.new(body).hexdigest()

# Open a shelve under lock.  
# If it doesn't exist create it.

lockfd = open(duplock, "w")
fcntl.lockf(lockfd, fcntl.LOCK_EX)

try:
    dupfd = shelve.open(dupdb, "w")
except:
    dupfd = shelve.open(dupdb, "c")

# If we've seen this sum before we'll add the header,
# if not, write this sum to the shelve file.

if dupfd.has_key(sum):
    duphdr = "X-Duplicate: yes\n"
else:
    duphdr = None
    dupfd[sum] = 1

dupfd.close()

fcntl.lockf(lockfd, fcntl.LOCK_UN)
lockfd.close()

# Print out the mail inserting X-Duplicate header if needed.

stdout.write(head)

if duphdr:        
    stdout.write(duphdr)

stdout.write(body)

exit(0)
----------------------- snip ----------------------------



-- 
-----------------------------------------------------------------
 Dan Wilder <dan at ssc.com>   Technical Manager
 SSC, Inc. P.O. Box 55549   Phone:  206-782-8808
 Seattle, WA  98155-0549    URL http://www.linuxjournal.com/
-----------------------------------------------------------------