From 1859011 at bugs.launchpad.net  Thu Jan  9 07:23:27 2020
From: 1859011 at bugs.launchpad.net (Michael Menge)
Date: Thu, 09 Jan 2020 12:23:27 -0000
Subject: [Bug 1859011] [NEW] bounce mail processed multiple times -> oom crash
 of BounceRunner
Message-ID: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>

Public bug reported:

System: RHEL 7.7
Mailman: mailman-2.1.15-26.el7_4.1.x86_64

Description: We have 2 Mailinglist with > 7000 Members which trigger a spike in processed bounces (more than 3000 in an sinle run), followed by an Out of Memory situation in the BounceRunner and an > 20 GB bounce-event-XXXX.pck 
file.

We tried to mitigate the problem by increasing the system memory, running the BoucneRunner 
every minute and limiting the number of mails delivered at ounce by postfix.

But it happened again:

Dec 30 19:53:29 2019 (13392) <BounceRunner at 140395473885088>
processing 4134 queued bounces

...
Dec 30 19:53:59 mx09 kernel: [13392]    41 13392  2755797  1874474    5337   825695             0 python
...
Dec 30 19:53:59 mx09 kernel: Out of memory: Kill process 13392 (python) score 896 or sacrifice child
Dec 30 19:53:59 mx09 kernel: Killed process 13392 (python), UID 41, total-vm:11023188kB, anon-rss:7497896kB, file-rss:0kB, shmem-rss:0kB


We analyzed the bounce-event file, extracting data with "stings".
This time we extracted the postfix mail queue ids from the received headers 
with our listserver. We found the following:

 
cat /tmp/bounce-20191230.txt | sed 's/;//' | sort | uniq -c | sort -n

      1 01A7DE9314
      1 10F6AE9319
      1 18456E930E
      1 27D0BAC960
      1 3B51CE9316
      1 57C2DAC992
      1 5D3B2E9310
      1 5EF11E9311
      1 63054E9312
      1 69377E9313
      1 ED636E930F
      2 29884E9315
      2 49ECEAC98D
      2 99A16A9DA7
      2 (Postfix)
      3 59EE1AC995
      3 CEB61AC996
    192 C12E9D3B48
   3929 F2BEEE9318
   4122 CC58AAC993
   4134 6EFD1A9DA7


As the bounces the last 2 qids are from the original mail send to the list (6EFD1A9DA7), 
and one of mails send by mailman to 500 members of that list (CC58AAC993).

(F2BEEE9318 and C12E9D3B48) are both bounces from members that where deliverd only once 
to /usr/lib/mailman/mail/mailman bounces NAME-OF-HUGE-LIST

So from me it seems that somehow some few bounce where "multiplied" so that ~20 real 
bounce produced 4134 virtual bounce. 

I see the potential of a deny of service attack, as it could be used 
to fill up the disk where the bounce-event files get dumped 
But I don't know if this would warrant marking "This bug is a security vulnerability".

** Affects: mailman
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From mark at msapiro.net  Thu Jan  9 19:54:18 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 10 Jan 2020 00:54:18 -0000
Subject: [Bug 1859104] [NEW] It is possible to mailbomb a third party by
 repeatedly posting the subscribe form.
Message-ID: <157861765903.28078.12796185779470947240.malonedeb@chaenomeles.canonical.com>

Public bug reported:

This can be prevented by refusing to pend a subscription when one is
already pending, but that means if a subscriber loses or doesn't receive
the confirmation request email, she has to wait PENDING_REQUEST_LIFE
(default 3 days) before she can request another.

It can also be avoided by setting the list's subscribe_policy to
Moderate, but that may not be desirable in many cases.

Because of these considerations, I will implement the refusal to pend a
subscription when one is already pending, but make that depend on a new
REFUSE_SECOND_PENDING mm_cfg.py setting.

** Affects: mailman
     Importance: Medium
         Status: In Progress

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859104

Title:
  It is possible to mailbomb a third party by repeatedly posting the
  subscribe form.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859104/+subscriptions

From mark at msapiro.net  Thu Jan  9 20:05:42 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 10 Jan 2020 01:05:42 -0000
Subject: [Bug 1859104] Re: It is possible to mailbomb a third party by
 repeatedly posting the subscribe form.
References: <157861765903.28078.12796185779470947240.malonedeb@chaenomeles.canonical.com>
Message-ID: <157861834337.6078.16171732034490456249.launchpad@gac.canonical.com>

** Changed in: mailman
       Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859104

Title:
  It is possible to mailbomb a third party by repeatedly posting the
  subscribe form.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859104/+subscriptions

From 1859104 at bugs.launchpad.net  Thu Jan  9 20:01:25 2020
From: 1859104 at bugs.launchpad.net (Launchpad Bug Tracker)
Date: Fri, 10 Jan 2020 01:01:25 -0000
Subject: [Bug 1859104] Re: It is possible to mailbomb a third party by
 repeatedly posting the subscribe form.
References: <157861765903.28078.12796185779470947240.malonedeb@chaenomeles.canonical.com>
Message-ID: <157861809178.5156.7371740010560626461.launchpad@ackee.canonical.com>

** Branch linked: lp:mailman/2.1

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859104

Title:
  It is possible to mailbomb a third party by repeatedly posting the
  subscribe form.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859104/+subscriptions

From mark at msapiro.net  Fri Jan 10 18:16:32 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 10 Jan 2020 23:16:32 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157869819324.2454.10020055314058961090.malone@soybean.canonical.com>

We need more information to know what's happening.

You say the MTA log shows only one delivery for, e.g., F2BEEE9318, to
/usr/lib/mailman/mail/mailman bounces NAME-OF-HUGE-LIST yet strings on
the bounce event file finds  F2BEEE9318 a total of 3929 times. Can you
examine the bounce-events-ppppp.pck file with Mailman's bin/dumpdb and
see what the actual bounce messages are and where they all come from?

** Changed in: mailman
       Status: New => Incomplete

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From mark at msapiro.net  Sat Jan 11 00:31:13 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 11 Jan 2020 05:31:13 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157872067375.14604.296035911308470044.malone@wampee.canonical.com>

If it does appear that Mailman is somehow generating these multiple
bounce-events entries, you might compare your BounceRunner.py to the
2.1.15 base at https://bazaar.launchpad.net/~mailman-
coders/mailman/2.1/view/1357/Mailman/Queue/BounceRunner.py to see if
there might be some corruption.

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:03:59 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:03:59 -0000
Subject: [Bug 1859104] Re: It is possible to mailbomb a third party by
 repeatedly posting the subscribe form.
References: <157861765903.28078.12796185779470947240.malonedeb@chaenomeles.canonical.com>
Message-ID: <157879464079.14807.3391573823662188192.launchpad@wampee.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

** Changed in: mailman
     Assignee: (unassigned) => Mark Sapiro (msapiro)

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859104

Title:
  It is possible to mailbomb a third party by repeatedly posting the
  subscribe form.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859104/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:04:33 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:04:33 -0000
Subject: [Bug 1851442] Re: MailList.AddMember crashes in a certain condition
References: <157299423430.2617.9283469890592885768.malonedeb@gac.canonical.com>
Message-ID: <157879467468.2880.12144310359946531352.launchpad@soybean.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1851442

Title:
  MailList.AddMember crashes in a certain condition

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1851442/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:01:28 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:01:28 -0000
Subject: [Bug 1845751] Re: cc modification due to nodup setting breaks DKIM &
 thus DMARC
References: <156963231161.15808.17034191154411525924.malonedeb@gac.canonical.com>
Message-ID: <157879449086.3160.4823066479739602967.launchpad@soybean.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1845751

Title:
  cc modification due to nodup setting breaks DKIM & thus DMARC

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1845751/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:02:15 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:02:15 -0000
Subject: [Bug 1838866] Re: Mailman 2.1: hardcoded site-packages dir prefix
 doesn't work on some 64 bit filesystem layouts
References: <156490528078.28889.17410140899278165529.malonedeb@wampee.canonical.com>
Message-ID: <157879453768.15063.5667023091359091217.launchpad@wampee.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1838866

Title:
  Mailman 2.1: hardcoded site-packages dir prefix doesn't work on some
  64 bit filesystem layouts

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1838866/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:04:48 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:04:48 -0000
Subject: [Bug 1831777] Re: The site list -owner,
 -request and -bounces addresses in virtual domains aren't deliverable.
References: <155975520423.23831.14511102514617290178.malonedeb@chaenomeles.canonical.com>
Message-ID: <157879468970.5239.514695209268088914.launchpad@gac.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1831777

Title:
  The site list -owner, -request and -bounces addresses in virtual
  domains aren't deliverable.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1831777/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:02:32 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:02:32 -0000
Subject: [Bug 1831321] Re: the List-Id field is filled without < and > around
 the list-id if the description contains unsupported characters
References: <155939546935.16310.4695708192037878768.malonedeb@chaenomeles.canonical.com>
Message-ID: <157879455390.5123.17688724743908571857.launchpad@gac.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1831321

Title:
  the List-Id field is filled without < and > around the list-id if the
  description contains unsupported characters

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1831321/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:05:03 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:05:03 -0000
Subject: [Bug 1818205] Re: qrunner --runners=All blocks one CPU core
References: <155143497129.32297.3389252011709275467.malonedeb@wampee.canonical.com>
Message-ID: <157879470523.27526.6573932188094070914.launchpad@chaenomeles.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1818205

Title:
  qrunner --runners=All blocks one CPU core

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1818205/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:02:49 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:02:49 -0000
Subject: [Bug 1818872] Re: Trailing spaces in the email address cause login
 failures.
References: <155189129684.31538.2382575262677645149.malonedeb@wampee.canonical.com>
Message-ID: <157879457084.14692.7399815111851706558.launchpad@wampee.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1818872

Title:
  Trailing spaces in the email address cause login failures.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1818872/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:05:15 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:05:15 -0000
Subject: [Bug 1810098] Re: Some security log messages have missing or
 incorrect information
References: <154618894892.25910.4889341139926585961.malonedeb@chaenomeles.canonical.com>
Message-ID: <157879471644.3101.12250326640221488622.launchpad@soybean.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1810098

Title:
  Some security log messages have missing or incorrect information

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1810098/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:03:28 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:03:28 -0000
Subject: [Bug 1805137] Re: Bounces sent from opensmtpd not parsed
References: <154323533980.15899.18330420192997885292.malonedeb@gac.canonical.com>
Message-ID: <157879460933.2497.6000499436270275692.launchpad@soybean.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1805137

Title:
  Bounces sent from opensmtpd not parsed

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1805137/+subscriptions

From mark at msapiro.net  Sat Jan 11 21:03:42 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 12 Jan 2020 02:03:42 -0000
Subject: [Bug 1785854] Re: TypeError from confirm CGI.
References: <153365987313.17868.5140007562982999428.malonedeb@soybean.canonical.com>
Message-ID: <157879462371.2371.15930755654399342728.launchpad@soybean.canonical.com>

** Changed in: mailman
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1785854

Title:
  TypeError from confirm CGI.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1785854/+subscriptions

From futatuki at poem.co.jp  Sun Jan 12 20:19:28 2020
From: futatuki at poem.co.jp (Yasuhito FUTATSUKI at POEM)
Date: Mon, 13 Jan 2020 01:19:28 -0000
Subject: [Merge] lp:~futatuki/mailman/2.1-ja-translation into lp:mailman/2.1
Message-ID: <157887836463.18260.8854025725682215859.launchpad@ackee.canonical.com>

Yasuhito FUTATSUKI at POEM has proposed merging lp:~futatuki/mailman/2.1-ja-translation into lp:mailman/2.1.

Commit message:
Update Japanese translation

Requested reviews:
  Mailman Coders (mailman-coders)

For more details, see:
https://code.launchpad.net/~futatuki/mailman/2.1-ja-translation/+merge/377482

* Add msgstr for msgid added in rev 1829, in Japansese po file
* also, add translated comment for new setting in Japanse translation of Defaults.py.in 
-- 
Your team Mailman Coders is requested to review the proposed merge of lp:~futatuki/mailman/2.1-ja-translation into lp:mailman/2.1.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: review-diff.txt
Type: text/x-diff
Size: 3233 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-coders/attachments/20200113/14b8d220/attachment.diff>

From 1859011 at bugs.launchpad.net  Mon Jan 13 07:12:57 2020
From: 1859011 at bugs.launchpad.net (Michael Menge)
Date: Mon, 13 Jan 2020 12:12:57 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157891757770.2842.7049185620491839487.malone@soybean.canonical.com>

Thanks for the hint to the dumpdb command.

though the command was killed by the system, it extracted sufficient data, 
and  I was able to extract some useful informations.
I did split the output into files for each object (csplit '/^<----- start object/' '{*}')


1. one bounced mail (qid F2BEEE9318) was multiple times in the bonce-event-13392.pck

2. the dumped objects for the duplicated bounce differ only in the 1st and 3rd line, 
   first line beeing '<----- start object' and second line been the mailinglist

3. I guess the detected mail-address should be in the third line. For the first 
   instance of the duplicated mail this was '\x00', for the other objects it 
   seems to contain "random" data e.g.

   '\x00',
   '\x16\xc9\xeaNq\xf8\xfbU\xe9\xa6\xbd\xbc\x92\x19\xaf\xee.o.\xa3M\x82i\x98g\xaep=\xb3\\\x94S_[\x01',
   'y\x9d\xf5O\x88\x05O\x88\x05O\xb0r/+\xbfQ\xd4<\xb2\xd7\xd2\xec\xf7E\xda\x1f\x86\xa7e\xb0\xa7e\xb0\xa52\xd8\xd3\xf6\xe6\x16{s\x8b\x0c\xb6\xd4\xfe|\xd4\xfe\xdcb\x7f',
   "\x87\xe5\xc7\xe3\xaa\xd2\xc5\xff\x8f\xb6\xbe\x15\xe2\x9b\xb8b\x8a\xa0\x842\xa8\x87\t0\xe5a\xd0\xb1\x9bqY\x1d\xc6\x9e\xddYY\x19\x93\xa7|\x14\x83\x02\xd8\xd9\xb3\x10\x83\xe6\xbb\xd9\x1f'\x13tVk\xb9\xb3Dq\x85\xd8`\xaa-W^!L\x81\xf2\xdew\xde~\x1a\x1f\xcf\xc7\x96\xf9\x9eg\xbe\xb7{^\xe91\xf6",
   

4. The envelope-from of the bounce mail did contain '=' 2 times
   prvs=126700a846=ndr at some.domain while the from address is
   ndr at some.domain


5. the bounced mail included the complete original mail with pdf attachment ~5 MB

There is no change in my BounceRunner.py compared to the linked version.

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From mark at msapiro.net  Tue Jan 14 17:08:40 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 14 Jan 2020 22:08:40 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157903972062.14807.16441296763668796616.malone@wampee.canonical.com>

Each object in the bounce events is a 4-tuple like (list-name, bouncing
address, bounce date, bounce message). The bounce message is a
Mailman.Message.Message object, but should be and apparently is rendered
as the text of the message. So yes, the third line is the bouncing
addresses, and it is clearly corrupted, but I suspect that is a symptom,
not a cause.

I still don't really understand what's happening. Given that the message
contained a large PDF attachment, I suspect this isn't a real bounce,
but rather is spam sent to the list-bounces address or possibly a list
post mis-sent to the list-bounces address. The size of it may have
something to do with the corruption that occurred, although I still
don't understand how the duplication comes about.

What's in Mailman's bounce log from the time this happened. I'm
particularly interested in any 'bounce message w/no discernable
addresses' or other messages just before the "Dec 30 19:53:29 2019
(13392) <BounceRunner at 140395473885088> processing 4134 queued
bounces" message.

Also are there any "VERP_PROBE_REGEXP doesn't yield the right match
groups:" messages in Mailman's error log from this time.

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From 1859011 at bugs.launchpad.net  Wed Jan 15 10:51:40 2020
From: 1859011 at bugs.launchpad.net (Michael Menge)
Date: Wed, 15 Jan 2020 15:51:40 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157910350092.5472.4099035384383417220.malone@gac.canonical.com>

To clarify  some points:

1. the bounce mail is genuine. My logs show that a ~5MB mail was send to the 
list by the owner of the list, it was distributed to the members of the list,
and caused some bounce mails by some recipients. The mail that caused the 
problem was normal non delivery notification with the characteristic that 
the complete original mail was included as message/rfc822.

2. I don't know if the size of the bounced mail has an impact on the number 
of duplications, but the size combined with the duplications resulted 
in the bounce process requesting using ~20 GB of ram 

3. My logs indicate some cases with >2000 processed bounces that didn't crash 
the BounceRunner, most likely there was also a duplication, but the mail 
was smaller

At the moment I try to create a striped down version that I can share 
that reproduces the problem

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From 1859011 at bugs.launchpad.net  Thu Jan 16 05:43:25 2020
From: 1859011 at bugs.launchpad.net (Michael Menge)
Date: Thu, 16 Jan 2020 10:43:25 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157917140537.22264.5046997021593305547.malone@wampee.canonical.com>

** Attachment added: "example that creates 81 bounces out of one bounce"
   https://bugs.launchpad.net/mailman/+bug/1859011/+attachment/5320825/+files/bounce-mail-times-81.eml

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From 1859011 at bugs.launchpad.net  Thu Jan 16 05:43:00 2020
From: 1859011 at bugs.launchpad.net (Michael Menge)
Date: Thu, 16 Jan 2020 10:43:00 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157917138077.14296.15249744507661153955.malone@wampee.canonical.com>

I was able to create a striped down version of the mail.

I also found that the size of the attachment has an influence 
on the number of dulicates

I have created 2 examples

** Attachment added: "example that creates 22 bounces out of one bounce"
   https://bugs.launchpad.net/mailman/+bug/1859011/+attachment/5320823/+files/bounce-mail-times-22.eml

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From mark at msapiro.net  Thu Jan 16 10:01:06 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 16 Jan 2020 15:01:06 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157918686702.4016.783652436838799967.malone@soybean.canonical.com>

Thanks for your work on this. I will use your examples to try to
recreate the issue and diagnose further.

** Changed in: mailman
   Importance: Undecided => Medium

** Changed in: mailman
       Status: Incomplete => Triaged

** Changed in: mailman
    Milestone: None => 2.1.30

** Changed in: mailman
     Assignee: (unassigned) => Mark Sapiro (msapiro)

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From mark at msapiro.net  Thu Jan 16 12:37:20 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 16 Jan 2020 17:37:20 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157919624072.24573.13283712806341531915.malone@chaenomeles.canonical.com>

I know what's going on. The bounce message itself is unusual. It
consists of a simple text/plain body containing the text

----------------------------------------------------------------------------

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients.

One or more of the recipients are not a valid user of the system. Please
check to be sure the email address is correct.

----------------------------------------------------------------------------

followed by a message/rfc822 part containing the entire original
message.

Bounce processing has a number of recognizers that examine the message
and if they recognize it, return a list of the bouncing addresses. One
of these recognizers matches RFC compliant DSNs, but there are many non-
compliant DSNs that we have seen that we try to recognize. Another
'heuristic' recognizer is SimpleMatch. It looks at every line of the
message until it finds a line matching a start pattern. Tnen it
continues to look for lines matching an address pattern until it finds a
line matching a stop pattern.

In this case, the start pattern that matches is

'A message that you( have)? sent could not be delivered'

It then looks for addresses matching

'<(?P<addr>[^>]*)>'

until it finds a line matching

'---'

It never finds the stop pattern so it goes through the entire base64
decoded PDF line by line looking for any string between '<' and '>', and
it finds a bunch of them, each of which is returned as a bouncing
address even though they don't look at all like addresses, and in fact,
the actual bouncing address is nowhere at all in the bounce message.
Then each 'address' is recorded as a bounce in the bounce_events file.

I will change the address patterns so the string inside the angle
brackets has to at least resemble an email address. That should fix
this.

** Changed in: mailman
       Status: Triaged => In Progress

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From 1859011 at bugs.launchpad.net  Thu Jan 16 19:08:42 2020
From: 1859011 at bugs.launchpad.net (Launchpad Bug Tracker)
Date: Fri, 17 Jan 2020 00:08:42 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157921972584.4674.9365957772698969780.launchpad@ackee.canonical.com>

** Branch linked: lp:mailman/2.1

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From mark at msapiro.net  Thu Jan 16 19:11:29 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 17 Jan 2020 00:11:29 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157921989016.5290.3886019930564099371.launchpad@wampee.canonical.com>

** Changed in: mailman
       Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From 1859011 at bugs.launchpad.net  Fri Jan 17 06:13:37 2020
From: 1859011 at bugs.launchpad.net (Michael Menge)
Date: Fri, 17 Jan 2020 11:13:37 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157925961798.22747.893552964533409477.malone@gac.canonical.com>

Thanks for the Fix

am I correct that I would need the patches from

https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1834 and
https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1835

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

From mark at msapiro.net  Fri Jan 17 12:25:08 2020
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 17 Jan 2020 17:25:08 -0000
Subject: [Bug 1859011] Re: bounce mail processed multiple times -> oom crash
 of BounceRunner
References: <157857260765.27735.6900917651298046478.malonedeb@chaenomeles.canonical.com>
Message-ID: <157928190843.22592.11718447185172508969.malone@gac.canonical.com>

Actually, https://bazaar.launchpad.net/~mailman-
coders/mailman/2.1/revision/1834 will fix your issue, but it is not
completely correct and https://bazaar.launchpad.net/~mailman-
coders/mailman/2.1/revision/1835 fixes that, so yes, install both.

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1859011

Title:
  bounce mail processed multiple times -> oom crash of BounceRunner

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions