[Mailman-Users] duplicates

[Marco Stoecker]

On 10/28/2015 08:57 AM, Marco Stoecker wrote:
> 
> 
> On 10/28/2015 05:49 AM, Mark Sapiro wrote:
>> On 10/27/2015 12:30 AM, Marco Stoecker wrote:
>>> On 10/26/2015 11:32 PM, Mark Sapiro wrote:
>>>> On 10/26/2015 02:14 PM, Marco Stoecker wrote:
>>>>
>>>>> I did attach the header from a recent message I got last week. I'm a
>>>>> member of a
>>>>> list and the sender sent this mail to 5 mailinglists on our server.
>>>>> Each
>>>>> member of these 5 lists got 5 messages.
>>>>>
>>>>> But the email, which I sent last week, with the attachement never made
>>>>> it to this list. How can I send the headers instead, cause this email
>>>>> would be very long?
>>>>
>>>>
>>>> I never saw such a message, even in the moderation queue because of
>>>> size.
>>
>>
>> I and the list have now received it. It was sent during the recent
>> server outage on only reached the server earlier today.
>>
>> It is helpful. I will copy some of the header info below.
>>
>>
>> ...
>>>>
>>>> The most helpful information would be the Postfix log entries from
>>>> several minutes before Oct 22 10:40:33 2015 up to Oct 22 10:44:28 2015,
>>>>
>>> This is the related mail.log entry:
>>>
>>> Oct 22 10:40:05 wakis02 postfix/smtpd[4898]: connect from localhost[::1]
>>> Oct 22 10:40:05 wakis02 postfix/smtpd[4898]: BF9AD1C94:
>> ...
>>
>>
>> This doesn't start early enough. The duplication occurs at 10:37:47. The
>> duplication occurs because of the way your mail is ultimately delivered
>> to mailman. Here is an excerpt from header1.txt in your other mail:
> 
> The entry before this in /var/log/mail.log has a time stamp of 09:35:59
> which is far before the duplicated message was sent.
> 
>>
>>> Received: from imap.1und1.de [212.227.15.188]
>>
>>>  by wakis02.local with IMAP (fetchmail-6.3.26)
>>
>>>  Thu, 22 Oct 2015 10:40:43 +0200 (CEST)
>>
>>> Received: from [212.227.15.41] ([212.227.15.41]) by mx.kundenserver.de
>>
>>>  (mxeue002) with ESMTPS (Nemesis) id 0M1foo-1aeGtV2fLW-00tjIT for
>>
>>>  <mailmanserver at waldorfkindergarten-erlangen.de>; Thu, 22 Oct 2015
>>> 10:37:47
>>
>>>  +0200
>>
>>> Received: from mout.web.de ([212.227.15.3]) by mx.kundenserver.de
>>> (mxeue002)
>>
>>>  with ESMTPS (Nemesis) id 0MVE1V-1a1cd42azt-00YP1m for
>>
>>>  <ak-leiter at waldorfkindergarten-erlangen.de>; Thu, 22 Oct 2015
>>> 10:37:47 +0200
>>
>>> Received: from Klamotten ([84.168.195.183]) by smtp.web.de (mrweb003)
>>> with
>>
>>>  ESMTPSA (Nemesis) id 0MTh7A-1Zy14E1g36-00QRsw; Thu, 22 Oct 2015
>>> 10:37:47
>>
>>>  +0200
>>
>>> From: "Alexandra Kick" <alexandrakick at web.de>
>>
>>> To: "'Alexandra Kick'" <alexandrakick at web.de>,
>>
>>>  <vorstand at waldorfkindergarten-erlangen.de>,
>>
>>>  <kollegium at waldorfkindergarten-erlangen.de>,
>>
>>>  <ak-leiter at waldorfkindergarten-erlangen.de>,
>>
>>>  <gruppensprecher at waldorfkindergarten-erlangen.de>,
>>
>>>  <verwaltung at waldorfkindergarten-erlangen.de>,
>>
>>>  <beisitz at waldorfkindergarten-erlangen.de>
>>
>>
>> Looking at the Received headers in chronological order (bottom to top),
>> the message is received by smtp.web.de and relayed (as mout.web.de) to
>> mx.kundenserver.de for the <ak-leiter at waldorfkindergarten-erlangen.de>
>> list. It is then relayed as is probably all list mail on that server to
>> itself for <mailmanserver at waldorfkindergarten-erlangen.de>. It is then
>> picked up by fetchmail and processed further.
>>
>> The other messages headers all look the same except they are each
>> initially for one of the other lists, but they all get forwarded to
>> <mailmanserver at waldorfkindergarten-erlangen.de>.
>>
>> Now, I'm sure what happens is one message for
>> <ak-leiter at waldorfkindergarten-erlangen.de> gets to
>> <mailmanserver at waldorfkindergarten-erlangen.de> and at some point later
>> the fact that it was originally received just for the
>> <ak-leiter at waldorfkindergarten-erlangen.de> list is lost or ignored and
>> the process looks at the To: header of the mail, sees 5 lists and
>> forwards the mail to all 5 lists.
>>
>> The same thing happens with each of the other 4 messages resulting in 5
>> messages to each of the 5 lists.
>>
>> The answer is your process for delivering mail to Mailman is flawed
>> because it takes a message received for one list only and delivers it to
>> every list mentioned in To: (and maybe Cc:).
> 
> I also posted this problem to the postfix mailing list and today I got
> an indicator that maybe fetchmail is the root of the problem due to some
> probably missing multi-drop support, which I have to double check now.
> But the fetchmail version which is delivered since debian 7 does not
> support time stamp information in the log.
> 
> Thx for your support!
> Marco

Dear Mark,

I think I do now understand why duplication is happening. But have no
idea how to avoid it. So here is how I think it happened.

I send an email to list1 and list2
now on my ISP there is an email forwarding from list1 at ... and list2 at ...
to the email account mailmansrv at mydomain. So here I have 2 emails in
that account which are both to: list1 and list2. Fetchmail now will get
these 2 messages with to: to both lists and even postfix gets 2 messages
with each adressed to two lists. I think this is the reason why
duplication happens.
But now I have no glue how to configure on the ISP side or
postfix(relayhost) or mailman to avoid such duplicates.
Any assistance possible here in this mailing list?

BR
Marco