[Mailman-Developers] Updated dupe removal patch

Marc MERLIN marc_news@vasoftware.com
Mon, 4 Mar 2002 02:25:09 -0800


--pf9I7BMVVzbSWLtt
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

It took all of my sunday, but I just finished porting Ben Gertzfield's
excellent dupe removal patch to mailman cvs
(I also  had to learn  some python in  the process. I'm starting  to believe
that Mailman is a conspiracy to get people to learn python :-p)

In a nutshell, the patch does two things:
1) it does not send you your list copy if
   - your subscribed Email address is already in the headers
   - you already received the message through another list (Cc accross two
     lists or more on the same site)

2) The new  "nodupes" setting  is really  something you  probably want  as a
   default on all lists. I also had lists were people wanted notmetoo as a
   default too.
   Ben's fix for that is to have a bitfield per list that you can set and
   that states which options newly added users get.

As Ben said,  this breaks the one  patch one functionality rule,  but when I
ported his work to mailman-cvs, I realized that it didn't make sense to take
them apart.
However, Barry,  if that  would stop  you from  merging #1  in CVS,  I could
remove it, but I'm not sure why one would want to.

I've done reasonable tests to make sure I didn't break all of mailman in the
process, and  the core logic hasn't  changed, so the basic  functionality is
the same  that Ben  had written  and that has  been used  for 6-9mo?  on the
debian lists now.
In other words, it  should work (it does for me, and  I'm already running it
on my  production mailman-cvs list server),  but there is always  the chance
that there might be a corner case buglet left somewhere.

Considering this was a  pain to port, and how this puts to  rest many of the
reply-to munging discussions (the only real argument for reply-to munging is
that it "solves" the duplicate mails you other receive when people use reply
to all), I'm hoping that this could make it in (wink, wink :-D)

Thanks,
Marc
-- 
Microsoft is to operating systems & security ....
                                      .... what McDonalds is to gourmet cooking
  
Home page: http://marc.merlins.org/   |   Finger marc_f@merlins.org for PGP key

--pf9I7BMVVzbSWLtt
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="mailman.nodupes.diff"

diff -urN mailman-cvs/Mailman/Cgi/admin.py mailman-cvs-nodupes/Mailman/Cgi/admin.py
--- mailman-cvs/Mailman/Cgi/admin.py	Sun Mar  3 14:29:16 2002
+++ mailman-cvs-nodupes/Mailman/Cgi/admin.py	Sun Mar  3 17:09:38 2002
@@ -843,7 +843,7 @@
         usertable.AddRow([Center(Italic(_('%(allcnt)s members total')))])
     usertable.AddCellInfo(usertable.GetCurrentRowIndex(),
                           usertable.GetCurrentCellIndex(),
-                          colspan=10,
+                          colspan=11,
                           bgcolor=mm_cfg.WEB_ADMINITEM_COLOR)
     # Add the alphabetical links
     if bucket:
@@ -861,17 +861,18 @@
         usertable.AddRow([Center(joiner.join(cells))])
     usertable.AddCellInfo(usertable.GetCurrentRowIndex(),
                           usertable.GetCurrentCellIndex(),
-                          colspan=10,
+                          colspan=11,
                           bgcolor=mm_cfg.WEB_ADMINITEM_COLOR)
     usertable.AddRow([Center(h) for h in (_('unsub'),
                                           _('member address<br>member name'),
                                           _('mod'), _('hide'),
                                           _('nomail<br>[reason]'),
                                           _('ack'), _('not metoo'),
+					  _('nodupes'),
                                           _('digest'), _('plain'),
                                           _('language'))])
     rowindex = usertable.GetCurrentRowIndex()
-    for i in range(10):
+    for i in range(11):
         usertable.AddCellInfo(rowindex, i, bgcolor=mm_cfg.WEB_ADMINITEM_COLOR)
     # Find the longest name in the list
     longest = 0
@@ -907,7 +908,7 @@
             checked = 0
         box = CheckBox('%s_mod' % addr, value, checked)
         cells.append(Center(box).Format())
-        for opt in ('hide', 'nomail', 'ack', 'notmetoo'):
+	for opt in ('hide', 'nomail', 'ack', 'notmetoo', 'nodupes'):
             extra = ''
             if opt == 'nomail':
                 status = mlist.getDeliveryStatus(addr)
@@ -984,6 +985,9 @@
         _('''<b>not metoo</b> -- Does the member avoid copies of their own
         posts?'''))
     legend.AddItem(
+	_('''<b>nodupes</b> -- Does the member avoid duplicates of the same
+	message?'''))
+    legend.AddItem(
         _('''<b>digest</b> -- Does the member get messages in digests?
         (otherwise, individual messages)'''))
     legend.AddItem(
@@ -1328,7 +1332,7 @@
                     mlist.setDeliveryStatus(user, MemberAdaptor.BYADMIN)
             else:
                 mlist.setDeliveryStatus(user, MemberAdaptor.ENABLED)
-            for opt in ('hide', 'ack', 'notmetoo', 'plain'):
+	    for opt in ('hide', 'ack', 'notmetoo', 'nodupes', 'plain'):
                 opt_code = option_info[opt]
                 if cgidata.has_key('%s_%s' % (user, opt)):
                     mlist.setMemberOption(user, opt_code, 1)
diff -urN mailman-cvs/Mailman/Cgi/options.py mailman-cvs-nodupes/Mailman/Cgi/options.py
--- mailman-cvs/Mailman/Cgi/options.py	Sun Mar  3 14:29:16 2002
+++ mailman-cvs-nodupes/Mailman/Cgi/options.py	Sun Mar  3 17:50:46 2002
@@ -424,6 +424,7 @@
                            ('conceal',     mm_cfg.ConcealSubscription),
                            ('remind',      mm_cfg.SuppressPasswordReminder),
                            ('rcvtopic',    mm_cfg.ReceiveNonmatchingTopics),
+                           ('nodupes',	   mm_cfg.DontReceiveDuplicates),
                            ):
             try:
                 newval = int(cgidata.getvalue(item))
@@ -514,9 +515,18 @@
                     global_remind = newval
                     break
 
-        if global_enable is not None or global_remind is not None:
+	global_nodupes = None
+	if cgidata.getvalue('nodupes-globally'):
+	    for flag, newval in newvals:
+		if flag == mm_cfg.DontReceiveDuplicates:
+		    global_nodupes = newval
+		    break
+
+        if (global_enable is not None or global_remind is not None
+	    or global_nodupes is not None):
             for gmlist in lists_of_member(mlist, user):
-                global_options(gmlist, user, global_enable, global_remind)
+                global_options(gmlist, user, global_enable, global_remind, 
+			       global_nodupes)
 
         # Now print the results
         if cantdigest:
@@ -591,6 +601,10 @@
         mlist.FormatOptionButton(mm_cfg.ConcealSubscription, 0, user))
     replacements['<mm-hide-subscription-button>'] = mlist.FormatOptionButton(
         mm_cfg.ConcealSubscription, 1, user)
+    replacements['<mm-dont-receive-duplicates-button>'] = (
+        mlist.FormatOptionButton(mm_cfg.DontReceiveDuplicates, 1, user))
+    replacements['<mm-receive-duplicates-button>'] = (
+        mlist.FormatOptionButton(mm_cfg.DontReceiveDuplicates, 0, user))
     replacements['<mm-unsubscribe-button>'] = (
         mlist.FormatButton('unsub', _('Unsubscribe')) + '<br>' +
         CheckBox('unsubconfirm', 1, checked=0).Format() +
@@ -620,6 +634,8 @@
         CheckBox('deliver-globally', 1, checked=0).Format())
     replacements['<mm-global-remind-button>'] = (
         CheckBox('remind-globally', 1, checked=0).Format())
+    replacements['<mm-global-nodupes-button>'] = (
+        CheckBox('nodupes-globally', 1, checked=0).Format())
 
     days = int(mm_cfg.PENDING_REQUEST_LIFE / mm_cfg.days(1))
     if days > 1:
@@ -806,7 +822,7 @@
 
 
 
-def global_options(mlist, user, global_enable, global_remind):
+def global_options(mlist, user, global_enable, global_remind, global_nodupes):
     def sigterm_handler(signum, frame, mlist=mlist):
         # Make sure the list gets unlocked...
         mlist.Unlock()
@@ -827,6 +843,10 @@
         if global_remind is not None:
             mlist.setMemberOption(user, mm_cfg.SuppressPasswordReminder,
                                   global_remind)
+
+        if global_nodupes is not None:
+            mlist.setMemberOption(user, mm_cfg.DontReceiveDuplicates,
+                                  global_nodupes)
 
         mlist.Save()
     finally:
diff -urN mailman-cvs/Mailman/Defaults.py.in mailman-cvs-nodupes/Mailman/Defaults.py.in
--- mailman-cvs/Mailman/Defaults.py.in	Sun Mar  3 14:42:08 2002
+++ mailman-cvs-nodupes/Mailman/Defaults.py.in	Sun Mar  3 17:08:19 2002
@@ -399,6 +399,7 @@
     'Hold',
     'Tagger',
     'CalcRecips',
+    'AvoidDuplicates',
     'Cleanse',
     'CookHeaders',
     # And now we send the message to the digest mbox file, and to the arch and
@@ -658,6 +659,11 @@
     'privacy', 'bounce',    'archive',  'gateway', 'autoreply', 'topics',
     ]
 
+# See "Bitfield for user options" below; make this a sum of those
+# options, to make all new members of lists start with those options
+# flagged.
+# We assume by default that people don't want to receive two copies of posts
+DEFAULT_LIST_OPTIONS = 256
 
 
 #####
@@ -979,6 +985,7 @@
 TEXTFIELDWIDTH = 40
 
 # Bitfield for user options
+# See DEFAULT_LIST_OPTIONS above to set defaults for all new lists
 Digests             = 0 # handled by other mechanism, doesn't need a flag.
 DisableDelivery     = 1 # Obsolete; use set/getDeliveryStatus()
 DontReceiveOwnPosts = 2 # Non-digesters only
@@ -987,7 +994,8 @@
 ConcealSubscription = 16
 SuppressPasswordReminder = 32
 ReceiveNonmatchingTopics = 64
-Moderate = 128
+Moderate            = 128
+DontReceiveDuplicates = 256
 
 # Authentication contexts.
 #
diff -urN mailman-cvs/Mailman/Gui/GUIBase.py mailman-cvs-nodupes/Mailman/Gui/GUIBase.py
--- mailman-cvs/Mailman/Gui/GUIBase.py	Wed Feb 27 20:00:42 2002
+++ mailman-cvs-nodupes/Mailman/Gui/GUIBase.py	Mon Mar  4 00:55:04 2002
@@ -22,6 +22,7 @@
 from Mailman import mm_cfg
 from Mailman import Utils
 from Mailman import Errors
+from Mailman import MailCommandHandler
 from Mailman.i18n import _
 
 NL = '\n'
@@ -117,36 +118,48 @@
                 continue
             # Unpack the gui item description
             property, wtype, args, deps, desc = item[0:5]
-            # BAW: I know this code is a little crufty but I wanted to
-            # reproduce the semantics of the original code in admin.py as
-            # closely as possible, for now.  We can clean it up later.
-            #
-            # The property may be uploadable...
-            uploadprop = property + '_upload'
-            if cgidata.has_key(uploadprop) and cgidata[uploadprop].value:
-                val = cgidata[uploadprop].value
-            elif not cgidata.has_key(property):
-                return
-            elif isinstance(cgidata[property], ListType):
-                val = [x.value for x in cgidata[property]]
-            else:
-                val = cgidata[property].value
-            # Coerce the value to the expected type, raising exceptions if the
-            # value is invalid
-            try:
-                val = self._getValidValue(mlist, property, wtype, val)
-            except ValueError:
-                doc.addError(_('Invalid value for variable: %(property)s'),
-                             tag=_('Error: '))
-                return
-            # This is the parent of MMBadEmailError and MMHostileAddress
-            except Errors.EmailAddressError:
-                doc.addError(
-                    _('Bad email address for option %(property)s: %(val)s'),
-                    tag=_('Error: '))
-                return
-            # Set the attribute, which will normally delegate to the mlist
-            self._setValue(mlist, property, val, doc)
+	    
+	    if property == 'default_options':
+                checked_defaults = cgidata.getvalue("default_options")
+                i = 0
+                new_defaults = 0
+                for opt in ("hide", "ack", "notmetoo", "plain", "nodupes"):
+                    opt_code = MailCommandHandler.option_info[opt]
+                    if `i` in checked_defaults:
+                        new_defaults = new_defaults | opt_code
+                    i = i + 1
+                mlist.default_options = new_defaults
+	    else:
+		# BAW: I know this code is a little crufty but I wanted to
+		# reproduce the semantics of the original code in admin.py as
+		# closely as possible, for now.  We can clean it up later.
+		#
+		# The property may be uploadable...
+		uploadprop = property + '_upload'
+		if cgidata.has_key(uploadprop) and cgidata[uploadprop].value:
+		    val = cgidata[uploadprop].value
+		elif not cgidata.has_key(property):
+		    return
+		elif isinstance(cgidata[property], ListType):
+		    val = [x.value for x in cgidata[property]]
+		else:
+		    val = cgidata[property].value
+                # Coerce the value to  the expected type, raising exceptions
+                # if the value is invalid
+		try:
+		    val = self._getValidValue(mlist, property, wtype, val)
+		except ValueError:
+		    doc.addError(_('Invalid value for variable: %(property)s'),
+				 tag=_('Error: '))
+		    return
+		# This is the parent of MMBadEmailError and MMHostileAddress
+		except Errors.EmailAddressError:
+		    doc.addError(
+			_('Bad email address for option %(property)s: %(val)s'),
+			tag=_('Error: '))
+		    return
+		# Set the attribute, which will normally delegate to the mlist
+		self._setValue(mlist, property, val, doc)
 
     # Convenience method for handling $-string attributes
     def _convertString(self, mlist, property, alloweds, val, doc):
diff -urN mailman-cvs/Mailman/Gui/General.py mailman-cvs-nodupes/Mailman/Gui/General.py
--- mailman-cvs/Mailman/Gui/General.py	Sun Mar  3 14:29:17 2002
+++ mailman-cvs-nodupes/Mailman/Gui/General.py	Sun Mar  3 20:01:15 2002
@@ -33,6 +33,22 @@
             return None
         WIDTH = mm_cfg.TEXTFIELDWIDTH
 
+        # These are for the default_options checkboxes below.
+        # this should be set in a module somewhere..
+        option_info = {'hide'     : mm_cfg.ConcealSubscription,
+                       'ack'      : mm_cfg.AcknowledgePosts,
+                       'notmetoo' : mm_cfg.DontReceiveOwnPosts,
+                       'plain'    : mm_cfg.DisableMime,
+                       'nodupes'  : mm_cfg.DontReceiveDuplicates
+                       }
+
+        options = ['hide', 'ack', 'notmetoo', 'plain', 'nodupes']
+        option_values = []
+
+        for o in options:
+            option_values.append(mlist.default_options & option_info[o])
+
+
         rtn = [
             _('''Fundamental list characteristics, including descriptive
             info and basic behaviors.'''),
@@ -237,6 +253,12 @@
              _('''Turn this on if you want password reminders to be sent once
              per month to your members.  Note that members may disable their
              own individual password reminders.''')),
+
+            ('default_options', mm_cfg.Checkbox, (options, option_values, 1),
+             0, _('''Default options for all members that join this list.'''),
+	     
+	    _('''This value is a bitfield that lets you set default options
+             for list subscribers.''')),
 
             ('welcome_msg', mm_cfg.Text, (4, WIDTH), 0,
              _('''List-specific text prepended to new-subscriber welcome
diff -urN mailman-cvs/Mailman/HTMLFormatter.py mailman-cvs-nodupes/Mailman/HTMLFormatter.py
--- mailman-cvs/Mailman/HTMLFormatter.py	Wed Feb 20 21:51:57 2002
+++ mailman-cvs-nodupes/Mailman/HTMLFormatter.py	Sun Mar  3 17:55:45 2002
@@ -116,6 +116,7 @@
                 mm_cfg.ConcealSubscription      : 'conceal',
                 mm_cfg.SuppressPasswordReminder : 'remind',
                 mm_cfg.ReceiveNonmatchingTopics : 'rcvtopic',
+                mm_cfg.DontReceiveDuplicates    : 'nodupes',
                 }[option]
         return '<input type=radio name="%s" value="%d"%s>' % (
             name, value, checked)
diff -urN mailman-cvs/Mailman/Handlers/AvoidDuplicates.py mailman-cvs-nodupes/Mailman/Handlers/AvoidDuplicates.py
--- mailman-cvs/Mailman/Handlers/AvoidDuplicates.py	Wed Dec 31 16:00:00 1969
+++ mailman-cvs-nodupes/Mailman/Handlers/AvoidDuplicates.py	Sun Mar  3 22:31:02 2002
@@ -0,0 +1,115 @@
+# Copyright (C) 1998,1999,2000,2001,2002 by the Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software 
+# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+
+"""If the user wishes it, do not send duplicates of the same message.
+
+This module keeps an in-memory dictionary of Message-ID and recipient
+pairs.  If a message with an identical Message-ID is about to be sent
+to someone who has already received a copy, we either drop the message,
+add a duplicate warning header, or pass it through, depending on the
+user's preferences.
+"""
+
+import string
+
+from Mailman import mm_cfg
+from Mailman import Utils
+from Mailman import Message
+from Mailman import Errors
+from Mailman.i18n import _
+from email.Utils import getaddresses
+
+
+
+class DuplicateDetected(Errors.DiscardMessage):
+    """The message would have been sent multiple times to a user who
+    prefers not to receive duplicates."""
+
+# A dictionary of dictionaries, used to store which recipients have received
+# which message IDs.
+recip_msgids = {}
+
+
+
+def process(mlist, msg, msgdata):
+
+    recips = msgdata['recips']
+    msgid = msg.get('message-id')
+    
+    if not recips or not msgid:
+        return
+
+    # This dictionary will hold recips who want their mail to have
+    # the X-Mailman-Duplicate: yes header.
+    if not msgdata.has_key('add-dupe-header'):
+        msgdata['add-dupe-header'] = {}
+
+    external_recips = []
+    for header in ('to', 'cc', 'resent-to', 'resent-cc'):
+	external_recips.extend(getaddresses(msg.get_all(header, [])))
+
+    # Anyone mentioned in the to/cc/resent-to/resent-cc headers should
+    # not get a duplicate of the message.
+    for (name, email) in external_recips:
+
+        # If getaddresses fails, email could be null. Skip those.
+        if not email:
+            continue
+        
+        # Initialize the external recipient's msgid hash if this is the
+        # first email they've received with this message-id.
+        if not recip_msgids.has_key(email):
+            recip_msgids[email] = {}
+
+        # We don't do anything except record that that address has
+        # gotten or will get a copy of this email externally.
+        recip_msgids[email][msgid] = 1
+
+    newrecips = []
+
+    for r in recips:
+        if not recip_msgids.has_key(r):
+            recip_msgids[r] = {}
+
+        # If they have received a message with this message-id already,
+        # see if they don't want duplicates.
+        if recip_msgids[r].has_key(msgid):
+            send_duplicate = 1
+            
+            # If the member wants to receive duplicates, or if the recipient 
+            # is not a member at all, just flag the X-Mailman-Duplicate: yes
+            # header.
+            try:
+                if mlist.getMemberOption(r, mm_cfg.DontReceiveDuplicates):
+                    send_duplicate = 0
+            except Errors.NotAMemberError:
+                pass
+
+            # We'll send a duplicate unless the user doesn't wish it.
+            # If personalization is enabled, the add-dupe-header flag will
+            # add a X-Mailman-Duplicate: yes header for this user's message.
+            if send_duplicate:
+                msgdata['add-dupe-header'][r] = 1
+                newrecips.append(r)
+
+        else:
+            # Otherwise, this is the first time they've been in the recips
+            # list.  Add them to the newrecips list and flag them as having
+            # received this message.
+            recip_msgids[r][msgid] = 1
+            newrecips.append(r)
+
+    msgdata['recips'] = newrecips
diff -urN mailman-cvs/Mailman/Handlers/Personalize.py mailman-cvs-nodupes/Mailman/Handlers/Personalize.py
--- mailman-cvs/Mailman/Handlers/Personalize.py	Tue Nov 20 08:39:38 2001
+++ mailman-cvs-nodupes/Mailman/Handlers/Personalize.py	Sun Mar  3 17:07:42 2002
@@ -45,6 +45,13 @@
             msg['To'] = '%s (%s)' % (member, name)
         else:
             msg['To'] = member
+        # We can flag the mail as a duplicate for each member, if
+        # they've already received that message. (See AvoidDuplicates.py)
+        if msgdata['add-dupe-header'].has_key(member):
+            msg['X-Mailman-Duplicate'] = 'yes'
+        elif msg.has_key('X-Mailman-Duplicate'):
+            del msg['X-Mailman-Duplicate']
+
         # See if we're taking the opportunity to VERP for more reliable bounce
         # processing.
         metadatacopy['verp'] = mm_cfg.VERP_PERSONALIZED_DELIVERIES
@@ -52,6 +59,10 @@
     # Restore the original To: line
     del msg['To']
     msg['To'] = originalto
+
+    # The original message is not a duplicate.
+    if msg.has_key('X-Mailman-Duplicate'):
+        del msg['X-Mailman-Duplicate']
     # Don't let the normal ToOutgoing processing actually send the original
     # copy, otherwise we'll get duplicates.
     del msgdata['recips']
diff -urN mailman-cvs/Mailman/MailCommandHandler.py mailman-cvs-nodupes/Mailman/MailCommandHandler.py
--- mailman-cvs/Mailman/MailCommandHandler.py	Sun Mar  3 14:29:16 2002
+++ mailman-cvs-nodupes/Mailman/MailCommandHandler.py	Sun Mar  3 16:28:01 2002
@@ -80,12 +80,20 @@
 you get digests in MIME format, which are much better if you have a mail
 reader that supports MIME.""")
 
-option_desc = {'hide'    : HIDE,
-               'nomail'  : NOMAIL,
-               'ack'     : ACK,
-               'notmetoo': NOTMETOO,
-               'digest'  : DIGEST,
-               'plain'   : PLAIN,
+NODUPES = _("""When turned on, you do *not* receive duplicate mails if mail is
+sent to multiple lists that you belong to.  This option will let you avoid
+duplicate mails; if you turn it on, you will never receive multiple copies
+of the same message.  Also, if you *and* the list are mentioned explicitly
+in the To: or Cc: headers of a message, you will not receive duplicates if
+this is turned on.""")
+
+option_desc = {'hide'     : HIDE,
+               'nomail'   : NOMAIL,
+               'ack'      : ACK,
+               'notmetoo' : NOTMETOO,
+               'digest'   : DIGEST,
+               'plain'    : PLAIN,
+               'nodupes'  : NODUPES,
                }
 
 # jcrey: and then the real one
@@ -97,10 +105,11 @@
                'notmetoo': mm_cfg.DontReceiveOwnPosts,
                'digest'  : 0,
                'plain'   : mm_cfg.DisableMime,
+               'nodupes' : mm_cfg.DontReceiveDuplicates
                }
 
 # ordered list
-options = ('hide', 'nomail', 'ack', 'notmetoo', 'digest', 'plain')
+options = ('hide', 'nomail', 'ack', 'notmetoo', 'digest', 'plain', 'nodupes')
 
 # strip just the outer layer of quotes
 quotecre = re.compile(r'["\'`](?P<cmd>.*)["\'`]')
diff -urN mailman-cvs/Mailman/MailList.py mailman-cvs-nodupes/Mailman/MailList.py
--- mailman-cvs/Mailman/MailList.py	Sun Mar  3 14:29:16 2002
+++ mailman-cvs-nodupes/Mailman/MailList.py	Sun Mar  3 17:08:03 2002
@@ -257,6 +257,7 @@
         self.language = {}
         self.usernames = {}
         self.passwords = {}
+	self.default_options = mm_cfg.DEFAULT_LIST_OPTIONS
 
         # This stuff is configurable
         self.respond_to_post_requests = 1
diff -urN mailman-cvs/Mailman/OldStyleMemberships.py mailman-cvs-nodupes/Mailman/OldStyleMemberships.py
--- mailman-cvs/Mailman/OldStyleMemberships.py	Wed Feb 20 21:51:58 2002
+++ mailman-cvs-nodupes/Mailman/OldStyleMemberships.py	Mon Mar  4 01:09:18 2002
@@ -207,6 +207,8 @@
         self.setMemberLanguage(member, language)
         if realname:
             self.setMemberName(member, realname)
+        if self.__mlist.default_options:
+            self.__mlist.user_options[member] = self.__mlist.default_options
     
     def removeMember(self, member):
         assert self.__mlist.Locked()
diff -urN mailman-cvs/Mailman/Version.py mailman-cvs-nodupes/Mailman/Version.py
--- mailman-cvs/Mailman/Version.py	Fri Jan 25 13:38:42 2002
+++ mailman-cvs-nodupes/Mailman/Version.py	Sun Mar  3 16:44:50 2002
@@ -15,7 +15,7 @@
 # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
 
 # Mailman version
-VERSION = "2.1a4+"
+VERSION = "2.1a4+-nodupes"
 
 # And as a hex number in the manner of PY_VERSION_HEX
 ALPHA = 0xa
@@ -36,7 +36,7 @@
                (REL_LEVEL << 4)  | (REL_SERIAL << 0))
 
 # config.pck schema version number
-DATA_FILE_VERSION = 54
+DATA_FILE_VERSION = 55
 
 # qfile/*.db schema version number
 QFILE_SCHEMA_VERSION = 3
diff -urN mailman-cvs/Mailman/versions.py mailman-cvs-nodupes/Mailman/versions.py
--- mailman-cvs/Mailman/versions.py	Fri Jan 25 13:38:42 2002
+++ mailman-cvs-nodupes/Mailman/versions.py	Sun Mar  3 17:07:53 2002
@@ -286,6 +286,7 @@
     add_only_if_missing('one_last_digest', {})
     add_only_if_missing('usernames', {})
     add_only_if_missing('personalize', 0)
+    add_only_if_missing('default_options', mm_cfg.DEFAULT_LIST_OPTIONS)
     add_only_if_missing('first_strip_reply_to',
                         mm_cfg.DEFAULT_FIRST_STRIP_REPLY_TO)
     add_only_if_missing('unsubscribe_policy',
diff -urN mailman-cvs/templates/en/help.txt mailman-cvs-nodupes/templates/en/help.txt
--- mailman-cvs/templates/en/help.txt	Fri May 18 14:28:54 2001
+++ mailman-cvs-nodupes/templates/en/help.txt	Sun Mar  3 16:46:25 2002
@@ -79,6 +79,11 @@
             Conceals your address when people look at who is on this
             list.
 
+        nodupes:
+            Turn this on if you do not want to receive duplicate mail
+            from the list, in case you are explicitly in the To: or Cc:
+            fields already or are included in multiple lists in one message.
+
 
     options
         Show the current values of your list options.
diff -urN mailman-cvs/templates/en/options.html mailman-cvs-nodupes/templates/en/options.html
--- mailman-cvs/templates/en/options.html	Sun Mar  3 14:29:21 2002
+++ mailman-cvs-nodupes/templates/en/options.html	Sun Mar  3 17:53:01 2002
@@ -280,6 +280,26 @@
         <mm-receive-nonmatching-topics>Yes
         </td></tr>
 
+    <tr><td bgcolor="#cccccc">
+        <strong>Avoid duplicate copies of messages?</strong><p>
+
+                When you are listed explicitly in the To: or Cc: headers
+                of a list message, or a message is sent to multiple lists
+                that you are a member of, you can opt to not receive another
+                copy from the mailing list.  Select <em>Yes</em> to avoid
+                receiving duplicate copies from the mailing list; select
+                <em>No</em> to receive duplicate copies.  
+
+                <p>If the list has per-message personalization
+                enabled, every duplicate mail will have a
+                <tt>X-Mailman-Duplicate: yes</tt> header added to it.
+
+        </td><td bgcolor="#cccccc">
+        <mm-receive-duplicates-button>No<br>
+        <mm-dont-receive-duplicates-button>Yes<p>
+        <mm-global-nodupes-button><i>Set globally</i>
+        </td></tr>
+
     <tr><TD colspan="2">
         <center><MM-options-Submit-button></center>
         </td></tr>

--pf9I7BMVVzbSWLtt--