From turnbull.stephen.fw at u.tsukuba.ac.jp Wed May 1 12:11:46 2019 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 2 May 2019 01:11:46 +0900 Subject: [Mailman-Users] Auto-discard emails In-Reply-To: <20190501025538.GL12773@ando.pearwood.info> References: <20190501025538.GL12773@ando.pearwood.info> Message-ID: <23753.50498.222488.488991@turnbull.sk.tsukuba.ac.jp> Steven D'Aprano writes: Mark's already answered, but I had a couple of minor points to add. > I run an announcement-only mailing list which has become the target of a > lot of spam, so I've set unsubscribed emails to be immediately > discarded. 99% of the discarded emails are spam, but unfortunately there > is a very small number of ham emails that get sent to the list by > non-members, which I need to be made aware of so I can contact them > off-list. I assume that the auto-discard policy is mentioned in the list information, but if not, you should consider adding it. > So I have turned on the option for discard messages to go to the > list-admin (me), and I get sent the "Auto-discard notification" messages > which includes enough information for me to tell at a glance whether I > need to contect the sender. Great! Yup. I believe there's an option for those to be batched once a day. The only thing I could imagine that might be better would be to integrate this with DMARC reporting, but Mailman 2 doesn't provide a querying facility, so that's quite hard to do. Mailman 3 does allow querying through the REST API, so it should be fairly straightforward. Note to self: Maybe an idea for next year's GSoC. Steve From steve at pearwood.info Wed May 1 21:09:25 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 2 May 2019 11:09:25 +1000 Subject: [Mailman-Users] Auto-discard emails In-Reply-To: <6cb6df54-f784-905f-4d5b-2f25f2e2e2c1@msapiro.net> References: <20190501025538.GL12773@ando.pearwood.info> <6cb6df54-f784-905f-4d5b-2f25f2e2e2c1@msapiro.net> Message-ID: <20190502010925.GA5010@ando.pearwood.info> On Tue, Apr 30, 2019 at 08:41:59PM -0700, Mark Sapiro wrote: [...] > I think you have to use > > whitelist_from mailman-bounces at mydomain Thanks, that seems to do the trick. Even though it is called "whitelist" it doesn't actually whitelist the emails, they're still scanned by spam assassin, but they're given what looks like a score of -100 which works well enough for my purposes. Thanks for the help. -- Steven From donnerland at aol.com Mon May 6 09:08:15 2019 From: donnerland at aol.com (Valentin Schwarze) Date: Mon, 6 May 2019 13:08:15 +0000 (UTC) Subject: [Mailman-Users] Rebuild Archives directory structure References: <1591396660.596220.1557148095969.ref@mail.yahoo.com> Message-ID: <1591396660.596220.1557148095969@mail.yahoo.com> Hello, I wanted to free some server disc space, therefore I wanted to delete my mailman archives. Due to a mistake, I accidently deleted the whole /private and /public directorys. Now, when I want to access my lists through the online admin backend, I get an error. Is there any way to rebuild (or reinstall) the archive directory structure? The only documents I found only were about the rebuilding the archive content, but not the directory and file structure itself. Any help would be appriciated. Cheers,Valentin From mark at msapiro.net Mon May 6 11:56:12 2019 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 6 May 2019 08:56:12 -0700 Subject: [Mailman-Users] Rebuild Archives directory structure In-Reply-To: <1591396660.596220.1557148095969@mail.yahoo.com> References: <1591396660.596220.1557148095969.ref@mail.yahoo.com> <1591396660.596220.1557148095969@mail.yahoo.com> Message-ID: <015d0f3a-ed90-a602-4917-eec95f617074@msapiro.net> On 5/6/19 6:08 AM, Valentin Schwarze via Mailman-Users wrote: > Hello, > I wanted to free some server disc space, therefore I wanted to delete my mailman archives. Due to a mistake, I accidently deleted the whole /private and /public directorys. Now, when I want to access my lists through the online admin backend, I get an error. > Is there any way to rebuild (or reinstall) the archive directory structure? The only documents I found only were about the rebuilding the archive content, but not the directory and file structure itself. > Any help would be appriciated. You need to create the public/ directory with mode drwxrwsr-x (2775) and the private/ directory with mode drwxrws--x (2771) and group equal Mailman's group. The public/ directory can be empty. The private/ directory should contain two sub-directories per list with names xxx and xxx.mbox where xxx is the list name and with mode mode drwxrwsr-x (2775), and the private/xxx/ directories should contain an index.html file with content similar to The xxx Archives

The xxx Archives

No messages have been posted to this list yet, so the archives are currently empty. You can get more information about this list.

-- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From qro at xs4all.nl Tue May 7 12:50:29 2019 From: qro at xs4all.nl (qro at xs4all.nl) Date: Tue, 7 May 2019 18:50:29 +0200 Subject: [Mailman-Users] Archive updater Message-ID: Hi, I have moved my email list to a new server. All is working well, email are send out. BUT.... I do have a problem with the archiver. The *.mbox archive is updated after a emails has been send. But the HTML pages are NOT updated. The then latest email is replaced by the send later email. So the list is not growing any more. I tried : bin/arch leden archives/private/*.mbox/*.mbox and rebuild the archive pages. At the end : Pickling archive state into /var/lib/mailman/archives/private/*/pipermail.pck This looks good after (* is replaced by a name :-) and the archive is complete. ?A? new message will got the same behavior and overwrit the last message. And hit wheer to look? Using version 2.1.23 ?Peter From mark at msapiro.net Wed May 8 08:34:35 2019 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 8 May 2019 05:34:35 -0700 Subject: [Mailman-Users] Archive updater In-Reply-To: References: Message-ID: <330881ab-a7db-9f72-950a-bf7eaa85eddd@msapiro.net> On 5/7/19 9:50 AM, qro at xs4all.nl wrote: > Hi, > > I have moved my email list to a new server. > All is working well, email are send out. > BUT.... I do have a problem with the archiver. > The *.mbox archive is updated after a emails has been send. What's in /var/lib/mailman/archives/public/* ? There should only be symlinks to /var/lib/mailman/archives/private/*, not actual content. Sometimes when moving things, the symlinks in archives/public/ are followed rather than copied and you wind up with content in the new archives/public/ which never gets updated. If this isn't the issue, look in Mailman's error log and maybe run Mailman's bin/check_perms. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From doerges at pre-sense.de Thu May 9 06:49:21 2019 From: doerges at pre-sense.de (=?UTF-8?Q?Till_D=c3=b6rges?=) Date: Thu, 9 May 2019 12:49:21 +0200 Subject: [Mailman-Users] Why are mails not being sent to all users (mails seem to disappear)? Message-ID: Hello all, we're using mailman-2.1.29 to operate a list where all users are moderated. The MTA used is postfix 3.2.0. Generally everything seems to be working fine. I.e. when I send out an e-mail I have several confirmations that it arrives under the addresses subscribed to the list. However, (at least) for one user the e-mail does not arrive. And I would at least like to find out the reason. But there's no trace of the address in any of the logfiles that should hold an entry (see below). Generally I'm able to send messages using my MUA to the corresponding e-mail address using the very same postfix also delivering the mailman messages. So I'm guessing it's nothing like DMARC problems or the likes. Like I said, what really puzzles me is that I cannot find the address in either /var/log/mail (postfix) or /var/lib/mailman/logs/* (mailman) /var/lib/mailman/logs/error doesn't contain any entries. /var/lib/mailman/logs/bounce does contain some addresses with a bounce score of 1.0 which are for /other/ e-mail addresses also subscribed to the list in question. /var/log/mail contains entries for /some/ of the addresses subscribed (including some bounces), but *not* for all. /var/lib/mailman/logs/smtp gives me a log entry with the original messages and claims to have sent it to 416 recips (which is correct). But overall I'm only able to find roughly half in either /var/log/mail or /var/lib/mailman/logs/*. Why are not all 416 addresses listed in /var/log/mail, assuming that mailman handed 416 mails to postfix? Do I have any chance of finding out what happened to the mails to the addresses which don't appear in any of the log files? Any hints/insights are greatly appreciated! Thanks -- Till -- PRESENSE Technologies GmbH Sachsenstr. 5, D-20097 HH Gesch?ftsf?hrer/Managing Directors AG Hamburg, HRB 107844 Till D?rges, J?rgen Sander USt-IdNr.: DE263765024 Wir sind wieder auf dem BSI IT-Sicherheitskongress 21.-23. Mai 2019 ? Bonn From mark at msapiro.net Thu May 9 08:31:19 2019 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 9 May 2019 05:31:19 -0700 Subject: [Mailman-Users] Why are mails not being sent to all users (mails seem to disappear)? In-Reply-To: References: Message-ID: <30690786-37a0-4862-2b7e-bbb629218203@msapiro.net> On 5/9/19 3:49 AM, Till D?rges wrote: > > /var/lib/mailman/logs/smtp gives me a log entry with the original messages and claims > to have sent it to 416 recips (which is correct). I assume there's nothing in /var/lib/mailman/logs/smtp-failure since if there were, these would also be recorded as bounces. > But overall I'm only able to find roughly half in either /var/log/mail or > /var/lib/mailman/logs/*. > > > Why are not all 416 addresses listed in /var/log/mail, assuming that mailman handed > 416 mails to postfix? Mailman did complete SMTP to Postfix for all 416 recipients. The /var/lib/mailman/logs/smtp log entry verifies that. What happened after that is a Postfix question. > Do I have any chance of finding out what happened to the mails to the addresses which > don't appear in any of the log files? I don't think this should happen without Postfix logging something, but if I have to guess, I'd guess you are not VERPing delivery to Postfix and the list is not personalized so Mailman is delivering to Postfix in transactions with up to SMTP_MAX_RCPTS (default 500) per transaction and Postfix's smtpd_recipient_limit is less. I suggest you set SMTP_MAX_RCPTS = 10 or 20 in mm_cfg.py or better set VERP_DELIVERY_INTERVAL = 1 to enable VERP on all deliveries. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From doerges at pre-sense.de Thu May 9 17:06:58 2019 From: doerges at pre-sense.de (=?UTF-8?Q?Till_D=c3=b6rges?=) Date: Thu, 9 May 2019 23:06:58 +0200 Subject: [Mailman-Users] Why are mails not being sent to all users (mails seem to disappear)? In-Reply-To: <30690786-37a0-4862-2b7e-bbb629218203@msapiro.net> References: <30690786-37a0-4862-2b7e-bbb629218203@msapiro.net> Message-ID: <8f64ab34-44dc-8ece-f647-16f2194e52a6@pre-sense.de> Hi Mark, thanks for your quick answer! Am 09.05.19 um 14:31 schrieb Mark Sapiro: > I assume there's nothing in /var/lib/mailman/logs/smtp-failure since if > there were, these would also be recorded as bounces. /var/lib/mailman/logs/smtp-failure does not have any entries related to the issue. >> But overall I'm only able to find roughly half in either /var/log/mail or >> /var/lib/mailman/logs/*. [...] > Mailman did complete SMTP to Postfix for all 416 recipients. The > /var/lib/mailman/logs/smtp log entry verifies that. What happened after > that is a Postfix question. Ack. IIUC it seems in deed as if messages were lost between Mailman and Postfix, without any of them noticing/logging it. The only indication I could find that there /might/ be problem was this warning with a suitable timestamp: --- snip --- ... postfix/master...: warning: service ... has reached its process limit "10": new clients may experience noticeable delays --- snap --- >> Do I have any chance of finding out what happened to the mails to the addresses which >> don't appear in any of the log files? > > > I don't think this should happen without Postfix logging something, but > if I have to guess, I'd guess you are not VERPing delivery to Postfix > and the list is not personalized so Mailman is delivering to Postfix in > transactions with up to SMTP_MAX_RCPTS (default 500) per transaction and > Postfix's smtpd_recipient_limit is less. Postfix is configured with 1000: smtpd_recipient_limit = 1000 Neither SMTP_MAX_RCPTS nor VERP_DELIVERY_INTERVAL was set explicitly, so I guess the defaults from /usr/lib/mailman/Mailman/Defaults.py are being used: SMTP_MAX_RCPTS = 500 VERP_DELIVERY_INTERVAL = 0 > I suggest you set SMTP_MAX_RCPTS = 10 or 20 in mm_cfg.py or better set > VERP_DELIVERY_INTERVAL = 1 to enable VERP on all deliveries. I will try with VERP_DELIVERY_INTERVAL set to 1, to see whether all messages are accounted for then. IIRC, setting VERP_DELIVERY_INTERVAL = 1 effectively means to ignore SMTP_MAX_RCPTS and only submit 1 mail at a time to the MTA. Curiously for the message that was not delivered to the specific user I did /not/ find a message by Postfix like the following in /var/log/mail: --- snip --- ... to=, relay=local, delay=0.58, delays=0.01/0/0/0.57, dsn=2.0.0, status=sent (delivered to command: /usr/lib/mailman/mail/mailman post ) --- snap --- To me that looks as if Postfix did in deed not complete sending all messages. I will monitor the situation with these reduced rate settings applied. Thanks again -- Till -- Dipl.-Inform. Till D?rges doerges at pre-sense.de Tel. +49 - 40 - 244 2407 - 0 Fax +49 - 40 - 244 2407 - 24 PRESENSE Technologies GmbH Sachsenstr. 5, D-20097 HH Gesch?ftsf?hrer/Managing Directors AG Hamburg, HRB 107844 Till D?rges, J?rgen Sander USt-IdNr.: DE263765024 Wir sind wieder auf dem BSI IT-Sicherheitskongress 21.-23. Mai 2019 ? Bonn From mark at msapiro.net Thu May 9 17:33:22 2019 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 9 May 2019 14:33:22 -0700 Subject: [Mailman-Users] Why are mails not being sent to all users (mails seem to disappear)? In-Reply-To: <8f64ab34-44dc-8ece-f647-16f2194e52a6@pre-sense.de> References: <30690786-37a0-4862-2b7e-bbb629218203@msapiro.net> <8f64ab34-44dc-8ece-f647-16f2194e52a6@pre-sense.de> Message-ID: On 5/9/19 2:06 PM, Till D?rges wrote: > > I will try with VERP_DELIVERY_INTERVAL set to 1, to see whether all messages are > accounted for then. > > IIRC, setting VERP_DELIVERY_INTERVAL = 1 effectively means to ignore SMTP_MAX_RCPTS > and only submit 1 mail at a time to the MTA. That's correct. > Curiously for the message that was not delivered to the specific user I did /not/ > find a message by Postfix like the following in /var/log/mail: > > --- snip --- > ... to=, relay=local, delay=0.58, delays=0.01/0/0/0.57, dsn=2.0.0, > status=sent (delivered to command: /usr/lib/mailman/mail/mailman post ) > --- snap --- > > To me that looks as if Postfix did in deed not complete sending all messages. That message is delivery of the original incoming post to Mailman. That message should be logged at the time Mailman received the message. I.e., in Mailman's smtp log you see a time stamp and the smtp to list for 416 recipients in n.nnn seconds. The arrival time is a bit before the time stamp - n.nnn seconds. > I will monitor the situation with these reduced rate settings applied. OK. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From csa at web-analysts.net Sat May 11 10:43:36 2019 From: csa at web-analysts.net (Web Analysts) Date: Sat, 11 May 2019 07:43:36 -0700 Subject: [Mailman-Users] mailman start error Message-ID: <013401d50807$ea7a3b10$bf6eb130$@web-analysts.net> I'm now getting an error when I start Mailman: root at primeo:~# /etc/init.d/mailman start Traceback (most recent call last): File "/usr/local/mailman/bin/mailmanctl", line 106, in from Mailman import mm_cfg File "/usr/local/mailman/Mailman/mm_cfg.py", line 54, in add_virtualhost(lists.domainname.tld, lists.domainname.tld) NameError: name 'lists' is not defined root at primeo:~# What did I inadvertently break? -----Original Message----- From: Mailman-Users [mailto:mailman-users-bounces+csa=web-analysts.net at python.org] On Behalf Of Mark Sapiro Sent: Friday, March 22, 2019 9:14 AM To: mailman-users at python.org Subject: Re: [Mailman-Users] iterative loop On 3/22/19 5:42 AM, Web Analysts via Mailman-Users wrote: > Okay here's the output from 'postconf -n' > > > alias_database = hash:/etc/aliases > alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases OK ... > mydestination = mail.domainname.tld, mail1.domainname.tld, > lists.domainname.tld, localhost.domainname.tld, localhost OK ... > relay_domains = $mydestination, lists.domainname.tld lists.domainname.tld is redundant here as it's in $mydestination, but I think this in combination with virtual_transport = lmtp:unix:private/dovecot-lmtp is the issue. I suggest setting relay_domains = mail.domainname.tld, mail1.domainname.tld, localhost.domainname.tld, localhost I.e. everything in $mydestination except lists.domainname.tld. > relay_recipient_maps = hash:/var/lib/mailman/data/virtual-mailman Then this will be unnecessary. If that doesn't fix your issue, look at /var/lib/mailman/data/aliases and ensure that the aliases for each list that look like > LISTNAME: "|/var/lib/mailman/mail/mailman post LISTNAME" > LISTNAME-admin: "|/var/lib/mailman/mail/mailman admin LISTNAME" > LISTNAME-bounces: "|/var/lib/mailman/mail/mailman bounces LISTNAME" > LISTNAME-confirm: "|/var/lib/mailman/mail/mailman confirm LISTNAME" > LISTNAME-join: "|/var/lib/mailman/mail/mailman join LISTNAME" > LISTNAME-leave: "|/var/lib/mailman/mail/mailman leave LISTNAME" > LISTNAME-owner: "|/var/lib/mailman/mail/mailman owner LISTNAME" > LISTNAME-request: "|/var/lib/mailman/mail/mailman request LISTNAME" > LISTNAME-subscribe: "|/var/lib/mailman/mail/mailman subscribe LISTNAME" > LISTNAME-unsubscribe: "|/var/lib/mailman/mail/mailman unsubscribe LISTNAME" In particular, that the quotes around the commands are not missing, although I don't see how they could be unless you edited this by hand. If there is an issue, running Mailman's bin/genaliases as the Mailman user -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan ------------------------------------------------------ Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/csa%40web-analysts.net From MorrillJ at childandfamilyagency.org Fri May 10 15:37:50 2019 From: MorrillJ at childandfamilyagency.org (Jason Morrill) Date: Fri, 10 May 2019 19:37:50 +0000 Subject: [Mailman-Users] utf-8 error with all lists Message-ID: Since the beginning of May I've had a problem with all messages going through Mailman. They are being delivered to the lists properly but copies are staying in qfiles/shunt The errors appearing in the Mailman error.1 log file indicate " May 05 10:16:29 2019 (1167) Uncaught runner exception: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte May 05 10:16:29 2019 (1167) Traceback (most recent call last): File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop self._onefile(msg, msgdata) File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/var/lib/mailman/Mailman/Queue/ArchRunner.py", line 77, in _dispose mlist.ArchiveMail(msg) File "/var/lib/mailman/Mailman/Archiver/Archiver.py", line 214, in ArchiveMail h.processUnixMailbox(f) File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 596, in processUnixMailbox self.add_article(a) File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 640, in add_article author = fixAuthor(article.decoded['author']) File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor while i>0 and (L[i-1][0] in lowercase or UnicodeDecodeError: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte May 05 10:16:29 2019 (1167) SHUNTING: 1557065787.914554+25c37ef6e662cc98d41ac9d45855fe04f151e11a " This is on Debian 9.6 with Python 2.7.13 and Mailman 2.1.23 , all installed from the Debian package manager. I've spent several hours hunting around for a potential solution. Then stumbled upon this: https://mail.python.org/pipermail/mailman-users/2019-March/084243.html Before making the code edits suggested in that post I was performing a simple test using the 'arch' command. With the original code the arch command fails with every single list in my system. The error it throws looks like this: " Updating HTML for article 66 Pickling archive state into /var/lib/mailman/archives/private/jtest/pipermail.pck Traceback (most recent call last): File "./arch", line 201, in main() File "./arch", line 189, in main archiver.processUnixMailbox(fp, start, end) File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 606, in processUnixMailbox self.add_article(a) File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 650, in add_article author = fixAuthor(article.decoded['author']) File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 72, in fixAuthor while i>0 and (L[i-1][0] in lowercase or UnicodeDecodeError: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte " With the modified code the arch command works perfectly with every single list and no errors are generated in the log file. I'm going to leave the hand-edited code in-place for the time being. If anyone would like to work with me to diagnose this mystery problem I'd be happy to assist however I can. Thanks, Jason Morrill IT Director Child and Family Agency 860-443-2896 x1422 http://www.childandfamilyagency.org/ From MorrillJ at childandfamilyagency.org Fri May 10 15:46:40 2019 From: MorrillJ at childandfamilyagency.org (Jason Morrill) Date: Fri, 10 May 2019 19:46:40 +0000 Subject: [Mailman-Users] unshunt does nothing Message-ID: I have messages accumulating in the qfiles/shunt folder. Running unshunt does nothing. Literally. No error message, no progress message. The only change I see is the timestamp on the shunted messages is updated to when I ran the unshunt command. This is on Debian 9.6 with Python 2.7.13 and Mailman 2.1.23 , all installed from the Debian package manager. BTW - I've been using Mailman for 10+ years on this machine. It's not a new install and these are not new lists. Thanks, Jason Morrill IT Director Child and Family Agency 860-443-2896 x1422 http://www.childandfamilyagency.org/ From mark at msapiro.net Sat May 11 13:45:15 2019 From: mark at msapiro.net (Mark Sapiro) Date: Sat, 11 May 2019 10:45:15 -0700 Subject: [Mailman-Users] unshunt does nothing In-Reply-To: References: Message-ID: On 5/10/19 12:46 PM, Jason Morrill wrote: > I have messages accumulating in the qfiles/shunt folder. > > Running unshunt does nothing. Literally. > > No error message, no progress message. The only change I see is the timestamp on the shunted messages is updated to when I ran the unshunt command. This is because the message is restored to it's original queue and processed and throws the same exception that occurred before and is shunted again. Are there new entries in Mailman's error log when you unshunt? > This is on Debian 9.6 with Python 2.7.13 and Mailman 2.1.23 , all installed from the Debian package manager. > > BTW - I've been using Mailman for 10+ years on this machine. It's not a new install and these are not new lists. What changed? Was the Debian package updated? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Sat May 11 14:13:25 2019 From: mark at msapiro.net (Mark Sapiro) Date: Sat, 11 May 2019 11:13:25 -0700 Subject: [Mailman-Users] utf-8 error with all lists In-Reply-To: References: Message-ID: On 5/10/19 12:37 PM, Jason Morrill wrote: > Since the beginning of May I've had a problem with all messages going through Mailman. They are being delivered to the lists properly but copies are staying in qfiles/shunt > > The errors appearing in the Mailman error.1 log file indicate > " > May 05 10:16:29 2019 (1167) Uncaught runner exception: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte ... > File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor > while i>0 and (L[i-1][0] in lowercase or > UnicodeDecodeError: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte > > May 05 10:16:29 2019 (1167) SHUNTING: 1557065787.914554+25c37ef6e662cc98d41ac9d45855fe04f151e11a > > " > > This is on Debian 9.6 with Python 2.7.13 and Mailman 2.1.23 , all installed from the Debian package manager. > > I've spent several hours hunting around for a potential solution. Then stumbled upon this: > https://mail.python.org/pipermail/mailman-users/2019-March/084243.html If you read the entire thread (which is split in the archive with the second part beginning at ), you will see that I think the issue is that whatever changed in your system at the beginning of May is causing the Python from string import lowercase in /var/lib/mailman/Mailman/Archiver/pipermail.py to import a lowercase object which should be the string 'abcdefghijklmnopqrstuvwxyz', but in fact is longer and has non-ascii accented characters following 'z'. The OP in that thread couldn't verify that that was the case, but I am convinced that it is. There doesn't seem to be a Debian patch to mailman 1:2.1.23-1+deb9u4 that would affect this so it is apparently something in Python or the OS. I am certain that if you start with the base /var/lib/mailman/Mailman/Archiver/pipermail.py and change from string import lowercase at line 13 to lowercase = 'abcdefghijklmnopqrstuvwxyz' the problem will be fixed. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Sat May 11 14:21:42 2019 From: mark at msapiro.net (Mark Sapiro) Date: Sat, 11 May 2019 11:21:42 -0700 Subject: [Mailman-Users] mailman start error In-Reply-To: <013401d50807$ea7a3b10$bf6eb130$@web-analysts.net> References: <013401d50807$ea7a3b10$bf6eb130$@web-analysts.net> Message-ID: <5dab669a-649a-9304-443c-546001a961b7@msapiro.net> On 5/11/19 7:43 AM, Web Analysts via Mailman-Users wrote: > I'm now getting an error when I start Mailman: > > root at primeo:~# /etc/init.d/mailman start > Traceback (most recent call last): > File "/usr/local/mailman/bin/mailmanctl", line 106, in > from Mailman import mm_cfg > File "/usr/local/mailman/Mailman/mm_cfg.py", line 54, in > add_virtualhost(lists.domainname.tld, lists.domainname.tld) > NameError: name 'lists' is not defined > root at primeo:~# > > > What did I inadvertently break? In the line add_virtualhost(lists.domainname.tld, lists.domainname.tld) which you added to mm_cfg.py, the domain names need to be quoted as in add_virtualhost('lists.domainname.tld', 'lists.domainname.tld') -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From csa at web-analysts.net Sun May 12 12:16:52 2019 From: csa at web-analysts.net (Web Analysts) Date: Sun, 12 May 2019 09:16:52 -0700 Subject: [Mailman-Users] script alias Message-ID: <001b01d508de$1c78d320$556a7960$@web-analysts.net> Okay. That worked. Now I have upgraded Mailman from an early 2x to 2.1.29. the script alias is not working consistently now across all lists. Some web page admin links are broken requiring me to manually insert the /cgi-bin/ into the web browser to bring up admin pages. Other links work w.o. the cgi-bin. I cannot admin one of my lists too. Any link I click on in the General Options Section takes me back to asking me for the list admin password, i.e. stuck in a loop. Thank you again. -----Original Message----- From: Mailman-Users [mailto:mailman-users-bounces+csa=web-analysts.net at python.org] On Behalf Of Mark Sapiro Sent: Saturday, May 11, 2019 11:22 AM To: mailman-users at python.org Subject: Re: [Mailman-Users] mailman start error On 5/11/19 7:43 AM, Web Analysts via Mailman-Users wrote: > I'm now getting an error when I start Mailman: > > root at primeo:~# /etc/init.d/mailman start > Traceback (most recent call last): > File "/usr/local/mailman/bin/mailmanctl", line 106, in > from Mailman import mm_cfg > File "/usr/local/mailman/Mailman/mm_cfg.py", line 54, in > add_virtualhost(lists.domainname.tld, lists.domainname.tld) > NameError: name 'lists' is not defined > root at primeo:~# > > > What did I inadvertently break? In the line add_virtualhost(lists.domainname.tld, lists.domainname.tld) which you added to mm_cfg.py, the domain names need to be quoted as in add_virtualhost('lists.domainname.tld', 'lists.domainname.tld') -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan ------------------------------------------------------ Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/csa%40web-analysts.net From mark at msapiro.net Sun May 12 19:05:35 2019 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 12 May 2019 16:05:35 -0700 Subject: [Mailman-Users] script alias In-Reply-To: <001b01d508de$1c78d320$556a7960$@web-analysts.net> References: <001b01d508de$1c78d320$556a7960$@web-analysts.net> Message-ID: On 5/12/19 9:16 AM, Web Analysts via Mailman-Users wrote: > Okay. That worked. Now I have upgraded Mailman from an early 2x to 2.1.29. > the script alias is not working consistently now across all lists. Some web > page admin links are broken requiring me to manually insert the /cgi-bin/ > into the web browser to bring up admin pages. Other links work w.o. the > cgi-bin. I cannot admin one of my lists too. Any link I click on in the > General Options Section takes me back to asking me for the list admin > password, i.e. stuck in a loop. See the faq at . This will help with some of this. For the rest, I'm not sure what you are saying. If you are saying you need to access some lists via, e.g., http://www.example.com/mailman/cgi-bin/..., and others via http://www.example.net/mailman/... without the cgi-bin, that's your web server configuration. You need to adjust that to make the ScriptAlias (or whatever your web server uses) directives consistent over all domains. If you are only saying some lists' pages have links with the cgi-bin and other lists' pages have links without the cgi-bin, setting DEFAULT_URL_PATTERN appropriately and running fix_url per the above faq will fix it. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Sun May 12 21:44:14 2019 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 12 May 2019 18:44:14 -0700 Subject: [Mailman-Users] script alias In-Reply-To: References: <001b01d508de$1c78d320$556a7960$@web-analysts.net> Message-ID: <2c3a7d56-9bd1-fa0c-9e44-2da0ed8121dd@msapiro.net> On 5/12/19 4:05 PM, Mark Sapiro wrote: > On 5/12/19 9:16 AM, Web Analysts via Mailman-Users wrote: >> Okay. That worked. Now I have upgraded Mailman from an early 2x to 2.1.29. >> the script alias is not working consistently now across all lists. Some web >> page admin links are broken requiring me to manually insert the /cgi-bin/ >> into the web browser to bring up admin pages. Other links work w.o. the >> cgi-bin. I cannot admin one of my lists too. Any link I click on in the >> General Options Section takes me back to asking me for the list admin >> password, i.e. stuck in a loop. > > > See the faq at . This will help with > some of this. Other FAQs which may be helpful with the latter issue are and -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From MorrillJ at childandfamilyagency.org Mon May 13 10:46:50 2019 From: MorrillJ at childandfamilyagency.org (Jason Morrill) Date: Mon, 13 May 2019 14:46:50 +0000 Subject: [Mailman-Users] utf-8 error with all lists In-Reply-To: References: , , Message-ID: Realizing I didn't respond to the list. So scroll down for my response. Then come back here to continue reading. Now that I have my Python-fu working I'm retesting Mark's suggestion: webserv:~# python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import string >>> string.lowercase 'abcdefghijklmnopqrstuvwxyz' >>> It appears that lowercase does return an apparent properly lowercased string of letters. So, I'm uncertain why the hand-edited code would function differently. Thanks, Jason Morrill IT Director Child and Family Agency 860-443-2896 x1422 http://www.childandfamilyagency.org/ ________________________________________ From: Jason Morrill Sent: Monday, May 13, 2019 10:29 AM To: Mark Sapiro Subject: RE: [Mailman-Users] utf-8 error with all lists Mark, Thanks for the suggestions and link to the remainder of that prior message thread. I too get the same results related to Python and lowercase. Now, I'm not a Python programmer but can certainly stumble through to sort things out. Here's what I'm seeing: webserv:~# python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import string >>> lowercase Traceback (most recent call last): File "", line 1, in NameError: name 'lowercase' is not defined >>> string >>> Per your suggestion I modified /var/lib/mailman/Mailman/Archiver/pipermail.py to set lowercase = 'abcdefghijklmnopqrstuvwxyz' And indeed the issue worked itself out. No more errors and I was able to clear the qfiles/shunt folder too. Per the Python docs, and your info, the string returned by lowercase should be a-z. And it depends upon the locale of the server. Something I've never purposefully changed. In Debian it looks at /etc/locale.gen In that file I see my locale set to be en_US ISO-8859-1 Could that somehow be related to this issue? Thanks, Jason Morrill IT Director Child and Family Agency 860-443-2896 x1422 http://www.childandfamilyagency.org/ ________________________________________ From: Mailman-Users [mailman-users-bounces+morrillj=childandfamilyagency.org at python.org] on behalf of Mark Sapiro [mark at msapiro.net] Sent: Saturday, May 11, 2019 2:13 PM To: mailman-users at python.org Subject: Re: [Mailman-Users] utf-8 error with all lists On 5/10/19 12:37 PM, Jason Morrill wrote: > Since the beginning of May I've had a problem with all messages going through Mailman. They are being delivered to the lists properly but copies are staying in qfiles/shunt > > The errors appearing in the Mailman error.1 log file indicate > " > May 05 10:16:29 2019 (1167) Uncaught runner exception: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte ... > File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor > while i>0 and (L[i-1][0] in lowercase or > UnicodeDecodeError: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte > > May 05 10:16:29 2019 (1167) SHUNTING: 1557065787.914554+25c37ef6e662cc98d41ac9d45855fe04f151e11a > > " > > This is on Debian 9.6 with Python 2.7.13 and Mailman 2.1.23 , all installed from the Debian package manager. > > I've spent several hours hunting around for a potential solution. Then stumbled upon this: > https://mail.python.org/pipermail/mailman-users/2019-March/084243.html If you read the entire thread (which is split in the archive with the second part beginning at ), you will see that I think the issue is that whatever changed in your system at the beginning of May is causing the Python from string import lowercase in /var/lib/mailman/Mailman/Archiver/pipermail.py to import a lowercase object which should be the string 'abcdefghijklmnopqrstuvwxyz', but in fact is longer and has non-ascii accented characters following 'z'. The OP in that thread couldn't verify that that was the case, but I am convinced that it is. There doesn't seem to be a Debian patch to mailman 1:2.1.23-1+deb9u4 that would affect this so it is apparently something in Python or the OS. I am certain that if you start with the base /var/lib/mailman/Mailman/Archiver/pipermail.py and change from string import lowercase at line 13 to lowercase = 'abcdefghijklmnopqrstuvwxyz' the problem will be fixed. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan ------------------------------------------------------ Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/morrillj%40childandfamilyagency.org From MorrillJ at childandfamilyagency.org Mon May 13 11:05:53 2019 From: MorrillJ at childandfamilyagency.org (Jason Morrill) Date: Mon, 13 May 2019 15:05:53 +0000 Subject: [Mailman-Users] unshunt does nothing In-Reply-To: References: , , Message-ID: I believe this issue is related to my other trouble with utf-8. https://mail.python.org/pipermail/mailman-users/2019-May/084435.html Once I hand-edited /var/lib/mailman/Mailman/Archiver/pipermail.py, per the suggestion in that message. I restarted Mailman and ran the unshunt command again. The shunted messages cleared out and no errors appeared. Though there was no positive affirmation the shunted messages were cleared either. Is the 'shunt' command meant to display anything? ==== Per Mark No. All unshunt does is move the .pck files from the shunt queue to the original queue from which they were shunted. It does this silently. The requeued entry is then processed again by the respective qrunner, and this process either succeeds or possibly throws the same exception that caused the shunting in the first place, in which case the message is shunted again. ===== Thanks, Jason Morrill IT Director Child and Family Agency 860-443-2896 x1422 http://www.childandfamilyagency.org/ ________________________________________ From: Mailman-Users [mailman-users-bounces+morrillj=childandfamilyagency.org at python.org] on behalf of Mark Sapiro [mark at msapiro.net] Sent: Saturday, May 11, 2019 1:45 PM To: mailman-users at python.org Subject: Re: [Mailman-Users] unshunt does nothing On 5/10/19 12:46 PM, Jason Morrill wrote: > I have messages accumulating in the qfiles/shunt folder. > > Running unshunt does nothing. Literally. > > No error message, no progress message. The only change I see is the timestamp on the shunted messages is updated to when I ran the unshunt command. This is because the message is restored to it's original queue and processed and throws the same exception that occurred before and is shunted again. Are there new entries in Mailman's error log when you unshunt? > This is on Debian 9.6 with Python 2.7.13 and Mailman 2.1.23 , all installed from the Debian package manager. > > BTW - I've been using Mailman for 10+ years on this machine. It's not a new install and these are not new lists. What changed? Was the Debian package updated? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan ------------------------------------------------------ Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/morrillj%40childandfamilyagency.org From mark at msapiro.net Mon May 13 11:19:41 2019 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 13 May 2019 08:19:41 -0700 Subject: [Mailman-Users] utf-8 error with all lists In-Reply-To: References: Message-ID: On 5/13/19 7:46 AM, Jason Morrill wrote: > > Now that I have my Python-fu working I'm retesting Mark's suggestion: > > webserv:~# python > Python 2.7.13 (default, Sep 26 2018, 18:42:22) > [GCC 6.3.0 20170516] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import string >>>> string.lowercase > 'abcdefghijklmnopqrstuvwxyz' >>>> > > It appears that lowercase does return an apparent properly lowercased string of letters. So, I'm uncertain why the hand-edited code would function differently. This is what we were unable to solve in the original thread starting at for 10 posts and then continuing at for another 13 posts. When the OP put in the logging suggested at , we discovered that in this Mailman/Archiver/pipermail.py at least, string .lowercase returns the iso-8859-1 encoding of 'abcdefghijklmnopqrstuvwxyz?????????????????????????????????' (see ), but we were never able to determine where this came from. If you are curious, you could modify your current /var/lib/mailman/Mailman/Archiver/pipermail.py which i think will contain lowercase = 'abcdefghijklmnopqrstuvwxyz' at line 13. If you add following that, the two lines import string syslog('error', 'string: %r', string) You should then see (after at least one post) messages in Mailman's error log like string: The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From dmaziuk at bmrb.wisc.edu Mon May 13 13:28:22 2019 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Mon, 13 May 2019 12:28:22 -0500 Subject: [Mailman-Users] utf-8 error with all lists In-Reply-To: References: Message-ID: <857b50db-4cd2-6096-b086-395f9de3b4e7@bmrb.wisc.edu> On 5/13/19 10:19 AM, Mark Sapiro wrote: > On 5/13/19 7:46 AM, Jason Morrill wrote: >> >> It appears that lowercase does return an apparent properly lowercased string of letters. So, I'm uncertain why the hand-edited code would function differently. > > > This is what we were unable to solve in the original thread starting at > > for 10 posts and then continuing at > > for another 13 posts. Just pipe it through lambda fixstr s : return unicodedata.normalize('NFKD', s.decode('utf-8')).encode('ascii','ignore')) or something along those lines first and be done with it. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From andrew at hodgson.io Wed May 15 09:45:15 2019 From: andrew at hodgson.io (Andrew Hodgson) Date: Wed, 15 May 2019 13:45:15 +0000 Subject: [Mailman-Users] Training Mailman to find email address from NDR Message-ID: Hi, As part of some upgrade projects I am testing using a transactional email service (Sendgrid) and have eventually got bounce forwarding working. However Mailman is not seeing the address in the bounce so is forwarding it to the list owner. I am probably not going to use Sendgrid long-term, as they seem to have a worse reputation than our static IPs, but wanted to know if I could get Mailman to process this bounce somehow? Here is the bounce message, I know I can enable VERP and this problem should go away, but I wanted to see if I could get this working without VERP as we have a large mail traffic on the list. Thanks. Andrew. Received: from o198-37-151-26.outbound-mail.sendgrid.net ([198.37.151.26]) by samwise.hodgsonfamily.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hQs8e-0000Nl-9y for bcab-bounces at lists.bcab.org.uk; Wed, 15 May 2019 12:32:07 +0100 Date: Wed, 15 May 2019 11:00:09 +0000 To: bcab-bounces at lists.bcab.org.uk From: no-reply at sendgrid.net Subject: Undelivered Mail Returned to Sender X-Spam-Score: -0.3 (/) X-Spam-Report: BAYES_00=-1.9, HELO_STATIC_HOST=-0.001, INVALID_DATE=1.096, MISSING_MID=0.497, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable --66728b7fa14ce3ed Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 7bit Content-Description: Notification This is an automatically generated message from SendGrid. We are sorry to tell you that your email was unable to be delivered to one of its intended recipients. If you require assistance with this, please contact SendGrid support. [TSS04] Messages from 167.89.106.6 temporarily deferred due to user complaint= s - 4.16.55.1; see https://help.yahoo.com/kb/postmaster/SLN3434.html --66728b7fa14ce3ed Content-Type: message/delivery-status Content-Disposition: inline Content-Transfer-Encoding: 7bit Content-Description: Delivery Report X-SendGrid-Sender: bcab-bounces at lists.bcab.org.uk Arrival-Date: 2019-05-05 11-00-00 Final-Recipient: rfc822; redacted at yahoo.com Original-Recipient: rfc822; redacted at yahoo.com Action: failed Status: 4.7.0 Diagnostic-Code: 421 4.7.0 [TSS04] Messages from 167.89.106.6 temporarily def= erred due to user complaints - 4.16.55.1; see https://help.yahoo.com/kb/postm= aster/SLN3434.html --66728b7fa14ce3ed Content-Type: message/rfc822; name=3D"66728b7fa14ce3ed" Content-Disposition: inline; filename=3D"66728b7fa14ce3ed" Content-Transfer-Encoding: binary Content-Description: Undelivered Message DKIM-Signature: v=3D1; a=3Drsa-sha1; c=3Drelaxed/relaxed;=20 d=3Dlists.bcab.org.uk;=20 h=3Dfrom:subject:to:reply-to:mime-version:content-type:content-transfer-enco= ding:list-id:list-unsubscribe:list-archive:list-help:list-subscribe;=20 s=3Ds1; bh=3DHSwHQmEzYnh8RVhCdSgx3+oQCI4=3D; b=3DqbnxUiQh9/93x0IvtPJnds+ UzsUGMTysX0jwFGtM3/aaJ2DkmCOa9RaIOHjhFopdNp2jEdDCN3PvxQeCps2t86G 3s2XXATdnbyBklvfuFDnoZ7kDoDbJbbyh4xKFTdBGWjMhYa1qg6Lz6A4Ey3SYeOr m1uhNPykiWFEriAFs0Bw=3D DKIM-Signature: v=3D1; a=3Drsa-sha1; c=3Drelaxed/relaxed; d=3Dsendgrid.info; = h=3Dfrom:subject:to:reply-to:mime-version:content-type:content-transfer-enco= ding:list-id:list-unsubscribe:list-archive:list-help:list-subscribe;=20 s=3Dsmtpapi; bh=3DHSwHQmEzYnh8RVhCdSgx3+oQCI4=3D; b=3DvLaOMgJkVx4gOD/ISt q2+4i+yYm+BPygXDX+oQSBYKFqD8m6xiSDL6szf/9ggGawmmE7SZXURlPoBd/iEG iL7oUIdYMFmhw7hpf8weXvet3pHpSYjDm/V8x01v5j4e9nldPkIqa0UNRPd/2BaN ggsr2TCLKTtlTmHMm/kJJUGTw=3D Received: by filter0033p3las1.sendgrid.net with SMTP id filter0033p3las1-1221= 3-5CD7FCB8-37 2019-05-12 11:00:09.005418638 +0000 UTC m=3D+233538.834188040 Received: from samwise.hodgsonfamily.org (samwise.hodgsonfamily.org [213.138.= 110.233]) by ismtpd0002p1lon1.sendgrid.net (SG) with ESMTP id z_pYT4dYT-eVrj9RAaq_jQ Sun, 12 May 2019 11:00:08.469 +0000 (UTC) Received: from [::1] (helo=3Dsamwise.hodgsonfamily.org) by samwise.hodgsonfamily.org with esmtp (Exim 4.89) (envelope-from ) id 1hPmD9-0001xg-KD; Sun, 12 May 2019 12:00:07 +0100 From: bcab-request at lists.bcab.org.uk Subject: Bcab Digest, Vol 444, Issue 7 To: bcab at lists.bcab.org.uk Reply-To: bcab at lists.bcab.org.uk Date: Sun, 12 May 2019 11:00:09 +0000 (UTC) Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=3D"us-ascii" Content-Transfer-Encoding: 7bit X-BeenThere: bcab at lists.bcab.org.uk X-Mailman-Version: 2.1.29 Precedence: list List-Id: BCAB Discussion List List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , Errors-To: bcab-bounces at lists.bcab.org.uk X-SG-EID: CdqM+Q1xrT/kSH2coGhbodTclTkyEKQEtO7KLouUOI9+psGGnYF6zXb7HRiZwFI6WBd= pZoQ+G2T2gJ PFi38sgTGhL9o9OeuLakXU5N+LB7ojKmPXJAXw+RJO/rqiCjQwOgxjJ65jrwb1pFdv1g+rgeWpSP= D4 TRJYSwUf9eT74JrfEt3RStrhQgSdkn8r3XNU7nG7mWZlCBBZpZkPiZlN+g=3D=3D [...] --66728b7fa14ce3ed-- =00 From gtaylor at tnetconsulting.net Wed May 15 11:46:25 2019 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 15 May 2019 09:46:25 -0600 Subject: [Mailman-Users] Training Mailman to find email address from NDR In-Reply-To: References: Message-ID: On 5/15/19 7:45 AM, Andrew Hodgson wrote: > Hi, Hi, > Here is the bounce message, I know I can enable VERP and this problem > should go away, but I wanted to see if I could get this working without > VERP as we have a large mail traffic on the list. Did you redact anything significant from the bounce? I don't see the expected boundary="66728b7fa14ce3ed" parameter to the Content-Type: header. I'm also surprised that the Content-Type: is text/plain and not multipart/report. I don't know enough about Mailman's bounce processing to know if it will find and process the message/delivery-status or not. I also question what Mailman would do with a temporary failure, 4.7.0. It looks like Yahoo TempFailed the message because they don't like SendGrid for one reason or another. I'm also somewhat surprised that SendGrid is returning the entire original message instead of just the headers. IMHO that's a good way for a bounced message's content to get trapped by a spam filter. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4008 bytes Desc: S/MIME Cryptographic Signature URL: From andrew at hodgson.io Wed May 15 12:49:25 2019 From: andrew at hodgson.io (Andrew Hodgson) Date: Wed, 15 May 2019 16:49:25 +0000 Subject: [Mailman-Users] Training Mailman to find email address from NDR In-Reply-To: References: , Message-ID: Grant Taylor wrote: >On 5/15/19 7:45 AM, Andrew Hodgson wrote: >> Here is the bounce message, I know I can enable VERP and this problem >> should go away, but I wanted to see if I could get this working without >> VERP as we have a large mail traffic on the list. >Did you redact anything significant from the bounce? I changed the email address of the recipient to redacted at yahoo.com (original recipient was in there), and removed the whole digest message. >I don't see the expected boundary="66728b7fa14ce3ed" parameter to the >Content-Type: header. I noticed that as well after I sent the original message. >I'm also surprised that the Content-Type: is text/plain and not >multipart/report. Ok I will relay this onto them. >I don't know enough about Mailman's bounce processing to know if it will >find and process the message/delivery-status or not. I also question >what Mailman would do with a temporary failure, 4.7.0. I don't care at this stage what Mailman would do with the error, I just want it to be attributed to the correct address. >It looks like Yahoo TempFailed the message because they don't like >SendGrid for one reason or another. Yep, pritty poor actually. My IP has better delivery rates than these shared IPs Sendgrid use. Thanks for some pointers anyway, Andrew. From mark at msapiro.net Wed May 15 12:55:57 2019 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 15 May 2019 09:55:57 -0700 Subject: [Mailman-Users] Training Mailman to find email address from NDR In-Reply-To: References: Message-ID: On 5/15/19 6:45 AM, Andrew Hodgson wrote: > > Here is the bounce message, I know I can enable VERP and this problem should go away, but I wanted to see if I could get this working without VERP as we have a large mail traffic on the list. This message is a disaster. Is this an actual bounce as received? It almost looks like an RFC 3464 compliant DSN except see comment below. > Received: from o198-37-151-26.outbound-mail.sendgrid.net ([198.37.151.26]) by > samwise.hodgsonfamily.org with esmtps > (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from > ) id 1hQs8e-0000Nl-9y for > bcab-bounces at lists.bcab.org.uk; Wed, 15 May 2019 12:32:07 +0100 > Date: Wed, 15 May 2019 11:00:09 +0000 > To: bcab-bounces at lists.bcab.org.uk > From: no-reply at sendgrid.net > Subject: Undelivered Mail Returned to Sender > X-Spam-Score: -0.3 (/) > X-Spam-Report: BAYES_00=-1.9, HELO_STATIC_HOST=-0.001, INVALID_DATE=1.096, > MISSING_MID=0.497, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, > URIBL_BLOCKED=0.001 > MIME-Version: 1.0 > Content-Type: text/plain; charset="utf-8" > Content-Transfer-Encoding: quoted-printable The message body is that of a MIME multipart message, but the main content type is text/plain instead of multipart/mixed; boundary="66728b7fa14ce3ed" so the whole body is just one plain text part and is quoted-printable encoded so that just changing the above Content-Type: won't work because there is quoted-printable encoded content in the sub-parts including sub-part headers. I have seen the other replies in this thread, so I'm not adding much here. Trying to recognize this in Mailman would be a major kludge and not worth the effort. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From gtaylor at tnetconsulting.net Wed May 15 13:06:27 2019 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 15 May 2019 11:06:27 -0600 Subject: [Mailman-Users] Training Mailman to find email address from NDR In-Reply-To: References: Message-ID: <92297922-778d-d895-ff6a-152ea3265073@spamtrap.tnetconsulting.net> On 5/15/19 10:55 AM, Mark Sapiro wrote: > This message is a disaster. Is this an actual bounce as received? It > almost looks like an RFC 3464 compliant DSN except see comment below. ~chuckle~ Agreed. > The message body is that of a MIME multipart message, but the main > content type is text/plain instead of > > multipart/mixed; boundary="66728b7fa14ce3ed" RFC 3464 wants a Content-Type of message/delivery-status. > so the whole body is just one plain text part and is quoted-printable > encoded so that just changing the above Content-Type: won't work because > there is quoted-printable encoded content in the sub-parts including > sub-part headers. > > I have seen the other replies in this thread, so I'm not adding much here. > > Trying to recognize this in Mailman would be a major kludge and not > worth the effort. I would argue against hacking Mailman to recognize this as a failed attempt at an RFC 3464 Delivery Status Notification. RFC 3464 has been out for 17 years. I think it's past time that we stop coddling people that can't conform to it. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4008 bytes Desc: S/MIME Cryptographic Signature URL: From mark at msapiro.net Wed May 15 13:13:44 2019 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 15 May 2019 10:13:44 -0700 Subject: [Mailman-Users] Training Mailman to find email address from NDR In-Reply-To: <92297922-778d-d895-ff6a-152ea3265073@spamtrap.tnetconsulting.net> References: <92297922-778d-d895-ff6a-152ea3265073@spamtrap.tnetconsulting.net> Message-ID: <4d7a7c92-f80d-16bd-f1a2-d1cfb034c09a@msapiro.net> On 5/15/19 10:06 AM, Grant Taylor via Mailman-Users wrote: > On 5/15/19 10:55 AM, Mark Sapiro wrote: > >> The message body is that of a MIME multipart message, but the main >> content type is text/plain instead of >> >> multipart/mixed; boundary="66728b7fa14ce3ed" > > RFC 3464 wants a Content-Type of message/delivery-status. The message/delivery-status part is there as a subpart. I should have said multipart/report; boundary="66728b7fa14ce3ed" rather than multipart/mixed as that is what the outer part's Content-Type should be. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From turnbull.stephen.fw at u.tsukuba.ac.jp Wed May 15 22:25:26 2019 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 16 May 2019 11:25:26 +0900 Subject: [Mailman-Users] Training Mailman to find email address from NDR In-Reply-To: <92297922-778d-d895-ff6a-152ea3265073@spamtrap.tnetconsulting.net> References: <92297922-778d-d895-ff6a-152ea3265073@spamtrap.tnetconsulting.net> Message-ID: <23772.51734.697361.333310@turnbull.sk.tsukuba.ac.jp> Grant Taylor via Mailman-Users writes: > RFC 3464 has been out for 17 years. I think it's past time that we > stop coddling people that can't conform to it. It's not a matter of coddling nonconformers. The only thing we are likely to be able to do about them is "Sleeping Giants" them on hostmaster Twitter. The only thing that would really hurt the nonconformers is to put them out the airlock. But that would hurt *our* users first, fast, and hard. Only if it became a movement would the large providers even notice. I know, you know that already. But a lot of the folks on this list may not. The fact of the matter is that the troublesome miscreants (Yahoo![1], AOL, Sendgrid) are large, they have lots of "captive" end users who would be very unhappy if any service they use took strong action to prevent arrival of nonconformant mail from those sources, and *our* clients are the list owners, postmasters, and hostmasters who manage sites and lists for those end users. AFAICS we're stuck in a position where the people who do the work have to make a decision whether doing the work is worth it for the project. Mark's been doing this a long time, nobody is more expert on that for Mailman 2. I support his decision: however much I'd like to make things a little bit easier for the list owners, his judgment is better than mine (and I don't know how to do it myself -- learning how would mean a lot more work than it would take for Mark! :-( ) Steve Footnotes: [1] I no longer blame Yahoo! and AOL for "p=reject" given that they were exposed to unimaginably large spam campaigns based on hundreds of millions of leaked address books used to spearphish spam and phishing messages at their users' correspondents. The alternative would have been to shut down their mail systems entirely, which would have hurt nearly a billion people directly, not to mention their correspondents. Something needs to be done about making firms with large stocks of PII accountable for leaking any of it, though, up to and including corporate death on the auction block if it's big enough. -- Associate Professor Division of Policy and Planning Science http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull at sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN From per.starback at lingfil.uu.se Fri May 17 08:07:46 2019 From: per.starback at lingfil.uu.se (Per =?iso-8859-1?Q?Starb=E4ck?=) Date: Fri, 17 May 2019 14:07:46 +0200 Subject: [Mailman-Users] Soft deactivation of Digest mode Message-ID: (I'm using Mailman 2.1.29.) I would like to eventually remove Digest mode from a couple of lists, but I don't want to interfere with users who currently use it. Can I remove the choice for new members, but still keep the digests working? I guess just removing the lines about it in the listinfo page and the personal member page for regular subscribers would suffice for me, if that can be achieved. -- Per Starb?ck From mark at msapiro.net Sat May 18 12:19:11 2019 From: mark at msapiro.net (Mark Sapiro) Date: Sat, 18 May 2019 09:19:11 -0700 Subject: [Mailman-Users] Soft deactivation of Digest mode In-Reply-To: References: Message-ID: On 5/17/19 5:07 AM, Per Starb?ck wrote: > (I'm using Mailman 2.1.29.) > > I would like to eventually remove Digest mode from a couple of lists, > but I don't want to interfere with users who currently use it. > > Can I remove the choice for new members, but still keep the digests > working? I guess just removing the lines about it in the listinfo page > and the personal member page for regular subscribers would suffice for > me, if that can be achieved. Unfortunately, setting Digest options -> digestable to No won't do what you want because while this will do all the right things like remove the digest settings from the user options page and the listinfo subscribe form and prevent subscribing in digest mode, e.g. by email, it also prevents messages from being accumulated for the digest. You can go to the list admin "Edit the public HTML pages and text files" link and edit the General list information page and the User specific options page. On the former, you would remove all the lines between the and tags, but not the tags themselves, and on the latter remove the two entries between and , but while this will remove those things from the web UI, it won't prevent subscribing to the digest or setting digest by email command. I suggest you just set digestable to No and edit Mailman/Handlers/ToDigest.py to change lines 82-84 from # Short circuit non-digestable lists. if not mlist.digestable or msgdata.get('isdigest'): return to # Short circuit digests. if msgdata.get('isdigest'): return -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From matt at conundrum.com Sun May 19 16:13:59 2019 From: matt at conundrum.com (Matthew Pounsett) Date: Sun, 19 May 2019 16:13:59 -0400 Subject: [Mailman-Users] DMARC message wrapping not working Message-ID: I've got a mailman 2.1.26 install I've taken over. I've attempted to turn on DMARC message wrapping by setting DEFAULT_DMARC_MODERATION_ACTION in my mm_cfg.py file, but it doesn't seem to have had the desired effect. I'm still seeing messages from p=reject domains going out with their original headers, and subscribers being blocked as a result of the bounces. These are the (non-comment) contents of my mm_cfg.py, with the mail domain anonymized: from Defaults import * DEFAULT_URL_HOST = 'lists.example.com' VIRTUAL_HOSTS.clear() add_virtualhost('lists.example.com', 'lists.example.com') DEFAULT_URL_PATTERN = 'https://%s/mailman/' DEFAULT_DMARC_MODERATION_ACTION = 2 The docs say this sets a minimum bar for the per-list configurations, but when I go to look at those they're all still set to 0/Accept, and in the web UI the full range of options is still available. There is a distinct possibility I've got my hands on the wrong config file. This install isn't using the version from the system package manager.. it's a manually compiled install that looks like it has an install prefix of /usr/local/mailman. This file is /usr/local/mailman/Mailman/mm_cfg.py. I don't see an obvious way to get mailman to spit out the full path it's actually checking for the file, but the output of mailman-config suggests it's probably looking in the right place. % ./mailman-config -h Configuration and build information for Mailman Mailman version: 2.1.26 Build Date: Wed May 2 13:10:19 UTC 2018 prefix: /usr/local/mailman var_prefix: /usr/local/mailman mailman_user: mailman mailman_group: mailman mail_group: mail cgi_group: www-data configure_opts: "--with-mail-gid=mail --with-cgi-gid=www-data" I see no mention of mm_cfg.py in the mailman logs.. so nothing clearly indicating it's failing to find the file (nor positive feedback that it has found the file). But I'm not convinced that's the problem anyway, since mailman clearly is picking up the virtual host and list URL settings. Any suggestions for what else I should be looking at to get Mailman to pick up this config change? Thanks! From mark at msapiro.net Mon May 20 11:34:47 2019 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 20 May 2019 08:34:47 -0700 Subject: [Mailman-Users] DMARC message wrapping not working In-Reply-To: References: Message-ID: On 5/19/19 1:13 PM, Matthew Pounsett wrote: > I've got a mailman 2.1.26 install I've taken over. I've attempted to turn > on DMARC message wrapping by setting DEFAULT_DMARC_MODERATION_ACTION in my > mm_cfg.py file, but it doesn't seem to have had the desired effect. I'm > still seeing messages from p=reject domains going out with their original > headers, and subscribers being blocked as a result of the bounces. Setting that in mm_cfg.py only affects lists created after you make the change. For existing lists, you have to change the list's configuration. > The docs say this sets a minimum bar for the per-list configurations, but > when I go to look at those they're all still set to 0/Accept, and in the > web UI the full range of options is still available. The doc says "Whatever is set as the default here precludes the list owner from setting a lower value." It doesn't say that if you set a value here, existing list settings will be increased automatically. If you go to the web UI for a list and just click Submit Your Changes, you will get "Error: dmarc_moderation_action must be >= the configured default value." at the top of the page and the setting will be changed. Or, I just created (mirrored at ) which you can copy to Mailman's bin/ directory and run via Mialman's bin/withlist -r set_dmarc -a and it will increase dmarc_moderation_action to DEFAULT_DMARC_MODERATION_ACTION if necessary for all lists. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From matt at conundrum.com Tue May 21 09:41:30 2019 From: matt at conundrum.com (Matthew Pounsett) Date: Tue, 21 May 2019 09:41:30 -0400 Subject: [Mailman-Users] DMARC message wrapping not working In-Reply-To: References: Message-ID: On Mon, 20 May 2019 at 11:35, Mark Sapiro wrote: > On 5/19/19 1:13 PM, Matthew Pounsett wrote: > > I've got a mailman 2.1.26 install I've taken over. I've attempted to > turn > > on DMARC message wrapping by setting DEFAULT_DMARC_MODERATION_ACTION in > my > > mm_cfg.py file, but it doesn't seem to have had the desired effect. I'm > > still seeing messages from p=reject domains going out with their original > > headers, and subscribers being blocked as a result of the bounces. > > > Setting that in mm_cfg.py only affects lists created after you make the > change. For existing lists, you have to change the list's configuration. > Ah, I see. I think that would be worth calling out in the documentation. I think the way it's currently written strongly implies that it will bring all lists up to a minimum behaviour. > Or, I just created > (mirrored at ) which > you can copy to Mailman's bin/ directory and run via Mialman's > Thanks. Knowing what the issue is, I can whip up something like that in a few minutes. Thanks for the clue! Matt From mark at msapiro.net Tue May 21 20:20:53 2019 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 21 May 2019 17:20:53 -0700 Subject: [Mailman-Users] DMARC message wrapping not working In-Reply-To: References: Message-ID: <3b931a7e-ec4e-8f03-b3d1-bc0ea42cf5b5@msapiro.net> On 5/21/19 6:41 AM, Matthew Pounsett wrote: > > Ah, I see. I think that would be worth calling out in the documentation. > I think the way it's currently written strongly implies that it will bring > all lists up to a minimum behaviour. I have updated that text. It now says: > # Default action for posts whose From: address domain has a DMARC policy of > # reject or quarantine. See DEFAULT_FROM_IS_LIST below. Whatever is set as > # the default here precludes the list owner from setting a lower value, however > # an existing list won't be changed until the first time "Submit Your Changes" > # is pressed on the list's Privacy options... -> Sender filters page. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From matt at conundrum.com Tue May 21 21:03:54 2019 From: matt at conundrum.com (Matthew Pounsett) Date: Tue, 21 May 2019 21:03:54 -0400 Subject: [Mailman-Users] DMARC message wrapping not working In-Reply-To: <3b931a7e-ec4e-8f03-b3d1-bc0ea42cf5b5@msapiro.net> References: <3b931a7e-ec4e-8f03-b3d1-bc0ea42cf5b5@msapiro.net> Message-ID: On Tue, 21 May 2019 at 20:21, Mark Sapiro wrote: > On 5/21/19 6:41 AM, Matthew Pounsett wrote: > > > > Ah, I see. I think that would be worth calling out in the documentation. > > I think the way it's currently written strongly implies that it will > bring > > all lists up to a minimum behaviour. > > > I have updated that text. It now says: > > > # Default action for posts whose From: address domain has a DMARC policy > of > > # reject or quarantine. See DEFAULT_FROM_IS_LIST below. Whatever is > set as > > # the default here precludes the list owner from setting a lower value, > however > > # an existing list won't be changed until the first time "Submit Your > Changes" > > # is pressed on the list's Privacy options... -> Sender filters page. > Thanks! I think that will make it perfectly clear. From fsantiago at gjb-online.com Tue May 21 20:20:54 2019 From: fsantiago at gjb-online.com (Fabian Aldo Santiago) Date: Tue, 21 May 2019 20:20:54 -0400 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 Message-ID: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> Hello all, I have my postfix mail server running mm 2.1.26 installed via apt on Ubuntu 18.04. I wish to upgrade to 2.1.29 but it?s not in the official repos. I tried installing from source, after first running mailman-config to gather the configure parameters (thanks Lindsay!). It installed no sweat BUT failed to start afterwards. It kept complaining of a pid file being unreadable. So I started over (restored snapshot) and then tried adding a ppa apt source which contains 2.1.29: https://launchpad.net/~mailman-administrivia/+archive/ubuntu/ppa Installed with an IO error over a lock file purported to be from one of my lists but otherwise started ok. But sending an email produced an error complaining: The mail system : Command died with status 2: "/var/lib/mailman/mail/mailman post list". Command output: Group mismatch error. Mailman expected the mail wrapper script to be executed as group "daemon", but the system's mail server executed the mail script as group "list". Try tweaking the mail server to run the script as group "daemon", or re-run configure, providing the command line option `--with-mail-gid=list'. So that gid is what I used when I tried from source. Does anyone have any ideas? Thanks. -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 Quick link to shared public key: https://keys.mailvelope.com/pks/lookup?op=get&search=0xBBB5B16CFB701CD7 From mark at msapiro.net Wed May 22 11:42:27 2019 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 22 May 2019 08:42:27 -0700 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> Message-ID: On 5/21/19 5:20 PM, Fabian Aldo Santiago wrote: > Hello all, > > I have my postfix mail server running mm 2.1.26 installed via apt on Ubuntu 18.04. I wish to upgrade to 2.1.29 but it?s not in the official repos. > > I tried installing from source, after first running mailman-config to gather the configure parameters (thanks Lindsay!). It installed no sweat BUT failed to start afterwards. It kept complaining of a pid file being unreadable. Did you look at ? What was the exact complaint. It's probably easy to fix. > So I started over (restored snapshot) and then tried adding a ppa apt source which contains 2.1.29: > > https://launchpad.net/~mailman-administrivia/+archive/ubuntu/ppa > > Installed with an IO error over a lock file purported to be from one of my lists but otherwise started ok. But sending an email produced an error complaining: > > The mail system > > : Command died with status 2: "/var/lib/mailman/mail/mailman > post list". Command output: Group mismatch error. Mailman expected the > mail wrapper script to be executed as group "daemon", but the system's mail > server executed the mail script as group "list". Try tweaking the mail > server to run the script as group "daemon", or re-run configure, providing > the command line option `--with-mail-gid=list'. This is a group mismatch in the /var/lib/mailman/mail/mailman wrapper. See for more on group mismatch. This may be an issue with the ppa which is maintained by @jimpop who reads this list and may chime in. However, Postfix is running the wrapper as group 'list' because 'list' is the owner of Mailman's data/aliases.db file. You can change this by `chown daemon data/aliases.db` but you need to ensure that that file is still group 'list' and group writable for Mailman to be able to update it as lists are added/deleted. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fsantiago at gjb-online.com Thu May 23 09:04:38 2019 From: fsantiago at gjb-online.com (Fabian A. Santiago) Date: Thu, 23 May 2019 09:04:38 -0400 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> Message-ID: <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> On 5/22/19 11:42 AM, Mark Sapiro wrote: > > Did you look at ? > > What was the exact complaint. It's probably easy to fix. Thank you Mark for this reference. I did not look at it. I also didn't record the error. i needed this server / list back up asap so i snapshot'ed the server 1st just in case and when i encountered the issue(s), i only had limited time to resolve or restore so i didn't spend too much time troubleshooting it. but i will try this as referenced in your link when i can afford more possible downtime and if the error reappears i will capture it. > > This is a group mismatch in the /var/lib/mailman/mail/mailman wrapper. > See for more on group mismatch. > > This may be an issue with the ppa which is maintained by @jimpop who > reads this list and may chime in. > > However, Postfix is running the wrapper as group 'list' because 'list' > is the owner of Mailman's data/aliases.db file. You can change this by > `chown daemon data/aliases.db` but you need to ensure that that file is > still group 'list' and group writable for Mailman to be able to update > it as lists are added/deleted. > Thanks for this heads up. i should also mention adding the ppa to my apt sources fails on gpg signing key import. i had to do it manually, searching for the key on ubuntu's keyserver. thanks for the help. -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 From fsantiago at gjb-online.com Thu May 23 09:40:45 2019 From: fsantiago at gjb-online.com (Fabian A. Santiago) Date: Thu, 23 May 2019 09:40:45 -0400 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> Message-ID: also, that wiki page you sent me mention an error with missing dnspython and it states to run this: "sudo apt-get install python-dnspython" but that claims it's already installed, yet the error occurs. what i found you want to install is the package "python-dev". -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 From fsantiago at gjb-online.com Thu May 23 10:02:12 2019 From: fsantiago at gjb-online.com (Fabian A. Santiago) Date: Thu, 23 May 2019 10:02:12 -0400 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> Message-ID: <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> On 5/23/19 9:04 AM, Fabian A. Santiago wrote: > On 5/22/19 11:42 AM, Mark Sapiro wrote: > >> >> Did you look at ? >> >> What was the exact complaint. It's probably easy to fix. > > Thank you Mark for this reference. I did not look at it. I also didn't > record the error. i needed this server / list back up asap so i > snapshot'ed the server 1st just in case and when i encountered the > issue(s), i only had limited time to resolve or restore so i didn't > spend too much time troubleshooting it. but i will try this as > referenced in your link when i can afford more possible downtime and if > the error reappears i will capture it. > ok, the issue occurred again with the same error. the complaint is (from systemctl status ... -l): May 23 09:44:40 mail systemd[1]: mailman.service: Can't open PID file /var/run/mailman/mailman. journalctl -xe reports: May 23 09:46:02 mail CRON[18652]: pam_unix(cron:session): session closed for user sogo May 23 09:46:10 mail systemd[1]: mailman.service: Start operation timed out. Terminating. May 23 09:46:10 mail systemd[1]: mailman.service: Failed with result 'timeout'. May 23 09:46:10 mail systemd[1]: Failed to start Mailman Master Queue Runner. -- Subject: Unit mailman.service has failed there is no pid file present in that location. am i wrong in assuming that gets created when the service starts? the dir permissions in that path are: root at mail:/var/run/mailman# ls -la total 0 drwxr-xr-x 2 list list 40 May 23 09:41 . drwxr-xr-x 38 root root 1260 May 23 09:06 .. mm seems to start if i run "mailmanctl start", but not as the service. i probably prefer to troubleshoot this issue and correct it rather than use the ppa via apt. what say you? Thanks. -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 From mark at msapiro.net Thu May 23 10:17:20 2019 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 23 May 2019 07:17:20 -0700 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> Message-ID: <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> On 5/23/19 7:02 AM, Fabian A. Santiago wrote: > > ok, the issue occurred again with the same error. the complaint is (from > systemctl status ... -l): > > May 23 09:44:40 mail systemd[1]: mailman.service: Can't open PID file > /var/run/mailman/mailman. > > journalctl -xe reports: > > May 23 09:46:02 mail CRON[18652]: pam_unix(cron:session): session closed > for user sogo > May 23 09:46:10 mail systemd[1]: mailman.service: Start operation timed > out. Terminating. > May 23 09:46:10 mail systemd[1]: mailman.service: Failed with result > 'timeout'. > May 23 09:46:10 mail systemd[1]: Failed to start Mailman Master Queue > Runner. > -- Subject: Unit mailman.service has failed > > there is no pid file present in that location. am i wrong in assuming > that gets created when the service starts? > > the dir permissions in that path are: > > root at mail:/var/run/mailman# ls -la > total 0 > drwxr-xr-x 2 list list 40 May 23 09:41 . > drwxr-xr-x 38 root root 1260 May 23 09:06 .. It's hard to say without seeing the contents of your mailman.service file (or if it is generated by systemd-sysv-generator, the /etc/init.d/mailman file), but `chmod g+w /var/run/mailman` may help. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fsantiago at gjb-online.com Thu May 23 11:07:03 2019 From: fsantiago at gjb-online.com (Fabian Aldo Santiago) Date: Thu, 23 May 2019 11:07:03 -0400 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> Message-ID: Service file: [Unit] Description=Mailman Master Queue Runner After=network.target [Service] Type=forking PIDFile=/var/run/mailman/mailman.pid ExecStart=/usr/lib/mailman/bin/mailmanctl -s start ExecStop=/usr/lib/mailman/bin/mailmanctl stop ExecReload=/usr/lib/mailman/bin/mailmanctl restart Restart=always RestartSec=3s RestartPreventExitStatus=1 [Install] WantedBy=multi-user.target Alias=mailman-qrunner.service No joy on the chmod. Same error. Thanks. On Thu, May 23, 2019, at 10:17 AM, Mark Sapiro wrote: > On 5/23/19 7:02 AM, Fabian A. Santiago wrote: > > > > ok, the issue occurred again with the same error. the complaint is (from > > systemctl status ... -l): > > > > May 23 09:44:40 mail systemd[1]: mailman.service: Can't open PID file > > /var/run/mailman/mailman. > > > > journalctl -xe reports: > > > > May 23 09:46:02 mail CRON[18652]: pam_unix(cron:session): session closed > > for user sogo > > May 23 09:46:10 mail systemd[1]: mailman.service: Start operation timed > > out. Terminating. > > May 23 09:46:10 mail systemd[1]: mailman.service: Failed with result > > 'timeout'. > > May 23 09:46:10 mail systemd[1]: Failed to start Mailman Master Queue > > Runner. > > -- Subject: Unit mailman.service has failed > > > > there is no pid file present in that location. am i wrong in assuming > > that gets created when the service starts? > > > > the dir permissions in that path are: > > > > root at mail:/var/run/mailman# ls -la > > total 0 > > drwxr-xr-x 2 list list 40 May 23 09:41 . > > drwxr-xr-x 38 root root 1260 May 23 09:06 .. > > > It's hard to say without seeing the contents of your mailman.service > file (or if it is generated by systemd-sysv-generator, the > /etc/init.d/mailman file), but `chmod g+w /var/run/mailman` may help. > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > ------------------------------------------------------ > Mailman-Users mailing list Mailman-Users at python.org > https://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://wiki.list.org/x/AgA3 > Security Policy: http://wiki.list.org/x/QIA9 > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-users/fsantiago%40gjb-online.com > -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 Quick link to shared public key: https://keys.mailvelope.com/pks/lookup?op=get&search=0xBBB5B16CFB701CD7 From mark at msapiro.net Thu May 23 11:31:01 2019 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 23 May 2019 08:31:01 -0700 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> Message-ID: <96bea343-352d-1ca8-c76f-8cee42c238a7@msapiro.net> On 5/23/19 8:07 AM, Fabian Aldo Santiago wrote: > Service file: > > [Unit] > Description=Mailman Master Queue Runner > After=network.target > > [Service] > Type=forking > PIDFile=/var/run/mailman/mailman.pid You could try PIDFile=/var/lib/mailman/data/master-qrunner.pid which is where mailmanctl actually stores the PID. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fsantiago at gjb-online.com Thu May 23 11:50:26 2019 From: fsantiago at gjb-online.com (Fabian Aldo Santiago) Date: Thu, 23 May 2019 11:50:26 -0400 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <96bea343-352d-1ca8-c76f-8cee42c238a7@msapiro.net> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> <96bea343-352d-1ca8-c76f-8cee42c238a7@msapiro.net> Message-ID: Ok making progress. Service started now but the www UI reports: Bug in Mailman version 2.1.29 We're sorry, we hit a bug! Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs or the web server logs. The web logs (nginx) contain: 2019/05/23 11:38:24 [error] 1174#1174: *2517 FastCGI sent in stderr: "PORT: 53836 REQUEST_SCHEME: https REQUEST_METHOD: GET HTTP_HOST: lists..com PATH_INFO: / CONTENT_TYPE: SERVER_PROTOCOL: HTTP/1.1 QUERY_STRING: LOGNAME: www-data REQUEST_URI: /cgi-bin/mailman/listinfo/ USER: www-data DOCUMENT_ROOT: /usr/lib" while reading response header from upstream, client: 2600:1001:b02c:d912:1501:38eb:b353:2f89, server: lists..com, request: "GET /cgi-bin/mailman/listinfo/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/fcgiwrap.socket:", host: "lists..com" Feels like another permission issue. But unsure where to go looking. Thanks. On Thu, May 23, 2019, at 11:31 AM, Mark Sapiro wrote: > On 5/23/19 8:07 AM, Fabian Aldo Santiago wrote: > > Service file: > > > > [Unit] > > Description=Mailman Master Queue Runner > > After=network.target > > > > [Service] > > Type=forking > > PIDFile=/var/run/mailman/mailman.pid > > You could try > > PIDFile=/var/lib/mailman/data/master-qrunner.pid > > which is where mailmanctl actually stores the PID. > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > ------------------------------------------------------ > Mailman-Users mailing list Mailman-Users at python.org > https://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://wiki.list.org/x/AgA3 > Security Policy: http://wiki.list.org/x/QIA9 > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-users/fsantiago%40gjb-online.com > -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 Quick link to shared public key: https://keys.mailvelope.com/pks/lookup?op=get&search=0xBBB5B16CFB701CD7 From mark at msapiro.net Thu May 23 12:11:41 2019 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 23 May 2019 09:11:41 -0700 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> <96bea343-352d-1ca8-c76f-8cee42c238a7@msapiro.net> Message-ID: <9862fe5a-043a-b358-f4fd-f98a20fc8fe2@msapiro.net> On 5/23/19 8:50 AM, Fabian Aldo Santiago wrote: > Ok making progress. Service started now but the www UI reports: > > Bug in Mailman version 2.1.29 > > > > We're sorry, we hit a bug! > > Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs or the web server logs. What's in Mailman's error log? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fsantiago at gjb-online.com Thu May 23 13:29:16 2019 From: fsantiago at gjb-online.com (Fabian A. Santiago) Date: Thu, 23 May 2019 13:29:16 -0400 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <9862fe5a-043a-b358-f4fd-f98a20fc8fe2@msapiro.net> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> <96bea343-352d-1ca8-c76f-8cee42c238a7@msapiro.net> <9862fe5a-043a-b358-f4fd-f98a20fc8fe2@msapiro.net> Message-ID: <5af5ae7b-f45f-5a61-237e-9943f0492c35@gjb-online.com> On 5/23/19 12:11 PM, Mark Sapiro wrote: > On 5/23/19 8:50 AM, Fabian Aldo Santiago wrote: >> Ok making progress. Service started now but the www UI reports: >> >> Bug in Mailman version 2.1.29 >> >> >> >> We're sorry, we hit a bug! >> >> Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs or the web server logs. > > > What's in Mailman's error log? > > nothing. -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 From fsantiago at gjb-online.com Thu May 23 13:41:05 2019 From: fsantiago at gjb-online.com (Fabian A. Santiago) Date: Thu, 23 May 2019 13:41:05 -0400 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <5af5ae7b-f45f-5a61-237e-9943f0492c35@gjb-online.com> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> <96bea343-352d-1ca8-c76f-8cee42c238a7@msapiro.net> <9862fe5a-043a-b358-f4fd-f98a20fc8fe2@msapiro.net> <5af5ae7b-f45f-5a61-237e-9943f0492c35@gjb-online.com> Message-ID: <82e8bb88-322e-6e85-2773-09fc88fed61c@gjb-online.com> On 5/23/19 1:29 PM, Fabian A. Santiago wrote: > On 5/23/19 12:11 PM, Mark Sapiro wrote: >> On 5/23/19 8:50 AM, Fabian Aldo Santiago wrote: >>> Ok making progress. Service started now but the www UI reports: >>> >>> Bug in Mailman version 2.1.29 >>> >>> >>> >>> We're sorry, we hit a bug! >>> >>> Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs or the web server logs. >> >> >> What's in Mailman's error log? >> >> > nothing. > all cgi-bin files owned by root:list. just an fyi. but private archives folder owned by www-data. i'm unsure if this is correct. -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 From mark at msapiro.net Thu May 23 13:59:04 2019 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 23 May 2019 10:59:04 -0700 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <82e8bb88-322e-6e85-2773-09fc88fed61c@gjb-online.com> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> <96bea343-352d-1ca8-c76f-8cee42c238a7@msapiro.net> <9862fe5a-043a-b358-f4fd-f98a20fc8fe2@msapiro.net> <5af5ae7b-f45f-5a61-237e-9943f0492c35@gjb-online.com> <82e8bb88-322e-6e85-2773-09fc88fed61c@gjb-online.com> Message-ID: <11a5dac6-3536-1280-a2f5-4428ef04f27c@msapiro.net> On 5/23/19 10:41 AM, Fabian A. Santiago wrote: > On 5/23/19 1:29 PM, Fabian A. Santiago wrote: >> On 5/23/19 12:11 PM, Mark Sapiro wrote: >>> >>> What's in Mailman's error log? >>> >>> >> nothing. It probably is permissions on the logs and others. Have you run check_perms. If not run `check_perms -f` as root. Don't worry about symlinks, they don't matter. If that doesn't fix it and you still don't get the error log messages, edit (not sure of path, but I think) /usr/lib/mailman/scripts/driver and at line 33 change STEALTH_MODE = 1 to STEALTH_MODE = 0 and you'll get the error info in your browser. > all cgi-bin files owned by root:list. just an fyi. but private archives > folder owned by www-data. i'm unsure if this is correct. Yes, that's correct. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fsantiago at gjb-online.com Thu May 23 14:13:44 2019 From: fsantiago at gjb-online.com (Fabian A. Santiago) Date: Thu, 23 May 2019 14:13:44 -0400 Subject: [Mailman-Users] Upgrading mm 2.1.26 on Ubuntu 18.04 to 2.1.29 In-Reply-To: <11a5dac6-3536-1280-a2f5-4428ef04f27c@msapiro.net> References: <77292f2d-1694-46e9-8502-0842b35b349e@www.fastmail.com> <26fc29a5-7e3b-bba7-317c-fda562e48377@gjb-online.com> <1df10929-a963-6133-672b-4b9ad059b983@gjb-online.com> <9e44e91d-e91c-df8a-d93f-0c47da1842e8@msapiro.net> <96bea343-352d-1ca8-c76f-8cee42c238a7@msapiro.net> <9862fe5a-043a-b358-f4fd-f98a20fc8fe2@msapiro.net> <5af5ae7b-f45f-5a61-237e-9943f0492c35@gjb-online.com> <82e8bb88-322e-6e85-2773-09fc88fed61c@gjb-online.com> <11a5dac6-3536-1280-a2f5-4428ef04f27c@msapiro.net> Message-ID: <4f133833-3378-7e1d-f4c4-ad0f04060902@gjb-online.com> > It probably is permissions on the logs and others. Have you run > check_perms. If not run `check_perms -f` as root. Don't worry about > symlinks, they don't matter. that did it, finally! check_perms -f. thanks for your help. finally on 2.1.29 now. all seems to be working now. -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 From iank at fsf.org Wed May 29 16:34:46 2019 From: iank at fsf.org (Ian Kelling) Date: Wed, 29 May 2019 16:34:46 -0400 Subject: [Mailman-Users] Migrating to a new server with upgraded mailman, should I copy qfiles ? Message-ID: <871s0h57a1.fsf@fsf.org> https://wiki.list.org/DOC/How%20do%20I%20move%20a%20list%20to%20a%20different%20server-Mailman%20installation. The linked examples don't show copying the qfiles directory. I'm migrating a very busy server (lists.gnu.org), that always has lots of queue files in the qfiles directory. Shouldn't I copy that directory over too after shutting down mailman? -- Ian Kelling | Senior Systems Administrator, Free Software Foundation GPG Key: B125 F60B 7B28 7FF6 A2B7 DF8F 170A F0E2 9542 95DF https://fsf.org | https://gnu.org From mark at msapiro.net Wed May 29 16:40:11 2019 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 29 May 2019 13:40:11 -0700 Subject: [Mailman-Users] Migrating to a new server with upgraded mailman, should I copy qfiles ? In-Reply-To: <871s0h57a1.fsf@fsf.org> References: <871s0h57a1.fsf@fsf.org> Message-ID: On 5/29/19 1:34 PM, Ian Kelling wrote: > https://wiki.list.org/DOC/How%20do%20I%20move%20a%20list%20to%20a%20different%20server-Mailman%20installation. > The linked examples don't show copying the qfiles directory. I'm > migrating a very busy server (lists.gnu.org), that always has lots of > queue files in the qfiles directory. Shouldn't I copy that directory > over too after shutting down mailman? Yes, if there are entries there, it should be copied. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fsantiago at gjb-online.com Thu May 30 10:04:59 2019 From: fsantiago at gjb-online.com (Fabian A. Santiago) Date: Thu, 30 May 2019 10:04:59 -0400 Subject: [Mailman-Users] question about email automation Message-ID: <2b62d760-6b83-5175-a1af-3b82338d7a76@gjb-online.com> Hello, this is unrelated to using mm itself but was wondering if anyone here had any insight they could share; i run a mailing list on mm 2.x. periodically i send out an event opening email to the list. this is typically followed by a conclusion email at the end of a specific period. in the interim period i would like to send out reminders of the event to the list but would like to automate this task. i want the reminders to be triggered by my opening email and end with my conclusion email. has anyone done this and how? i've looked at ifttt and zapier but wasn't sure. thanks. -- Fabian S. OpenPGP: 0xA1250C2112D2D0E53E5BD964BBB5B16CFB701CD7 From doerges at pre-sense.de Thu May 30 13:22:38 2019 From: doerges at pre-sense.de (=?UTF-8?Q?Till_D=c3=b6rges?=) Date: Thu, 30 May 2019 19:22:38 +0200 Subject: [Mailman-Users] Why are mails not being sent to all users (mails seem to disappear)? In-Reply-To: References: <30690786-37a0-4862-2b7e-bbb629218203@msapiro.net> <8f64ab34-44dc-8ece-f647-16f2194e52a6@pre-sense.de> Message-ID: <277b2ef2-ce12-ee91-3aa4-e1b8afec727f@pre-sense.de> Am 09.05.19 um 23:33 schrieb Mark Sapiro: > On 5/9/19 2:06 PM, Till D?rges wrote: >> Curiously for the message that was not delivered to the specific user I did /not/ >> find a message by Postfix like the following in /var/log/mail: [...] >> To me that looks as if Postfix did in deed not complete sending all messages. [...] >> I will monitor the situation with these reduced rate settings applied. > > OK. I figured it out. And it was neither a Mailman nor a Postfix problem. systemd-journald simply enforced some rate limiting. After removing it (setting RateLimitBurst=0 in /etc/systemd/journald.conf), delivery for all mails can be seen in the logfiles. Thanks again for your help, and sorry for the noise! Regards -- Till -- Dipl.-Inform. Till D?rges doerges at pre-sense.de PRESENSE Technologies GmbH Sachsenstr. 5, D-20097 HH Gesch?ftsf?hrer/Managing Directors AG Hamburg, HRB 107844 Till D?rges, J?rgen Sander USt-IdNr.: DE263765024 From chip at aresti.com Thu May 30 11:57:44 2019 From: chip at aresti.com (Chip Davis) Date: Thu, 30 May 2019 11:57:44 -0400 Subject: [Mailman-Users] Easy question for this crowd Message-ID: <6b8120bb-fe79-2082-0f83-195376fff789@aresti.com> I've supported a dozen Mailman listservers for over a dozen years. This doesn't represent much real effort most of the time. I've had to block specific users often and specific domains rarely, but this is the first time I've had to block an entire TLD. Recently I've been gifted with an inordinate amount of UCE from many different domains under the '.icu' TLD. Since Python RE's are _almost_ the same as the UNIX RE's I used many years ago, if I put ^@.*\.icu$ in discard_these_nonmembers, will it block all domains in that TLD? And not block anyone else? Thanks, -Chip Davis- Mailman 2.1.27 shared host linux 2.6.32-696.18.7.el6.x86_64 cPanel 80.0.10 From heller at deepsoft.com Thu May 30 19:03:19 2019 From: heller at deepsoft.com (Robert Heller) Date: Thu, 30 May 2019 19:03:19 -0400 (EDT) Subject: [Mailman-Users] Easy question for this crowd In-Reply-To: <6b8120bb-fe79-2082-0f83-195376fff789@aresti.com> References: <6b8120bb-fe79-2082-0f83-195376fff789@aresti.com> Message-ID: <20190530230320.489A026C001A@sharky3.deepsoft.com> At Thu, 30 May 2019 11:57:44 -0400 Chip Davis wrote: > > I've supported a dozen Mailman listservers for over a dozen years. > This doesn't represent much real effort most of the time. I've had to > block specific users often and specific domains rarely, but this is > the first time I've had to block an entire TLD. > > Recently I've been gifted with an inordinate amount of UCE from many > different domains under the '.icu' TLD. > > Since Python RE's are _almost_ the same as the UNIX RE's I used many > years ago, if I put > > ^@.*\.icu$ > > in discard_these_nonmembers, will it block all domains in that TLD? Yes. > > And not block anyone else? Yes. I've done this, and then I took things a step further: What *I* have done (because I can), is configure rejection of both domains AND cidrs at the Postfix level, putting REJECT's in both /etc/postfix/access and /etc/postfix/cidr.clients. (I use *REJECT* for a reason: I figure if these idiots are going to make trouble for me, I'll make trouble for them -- eg now they will will get reject messages. Also when the addresses are from legit mail servers, the admins there will get a wake up call and presumably do something -- I have discovered that there is really little point in sending anything to the [so-called] 'abuse' addresses.) I've also configured mimedefang and spamassassin to *reject* spam at the Postfix as well. Very little gets though now. > > Thanks, > > -Chip Davis- > > Mailman 2.1.27 > shared host > linux 2.6.32-696.18.7.el6.x86_64 > cPanel 80.0.10 > ------------------------------------------------------ > Mailman-Users mailing list Mailman-Users at python.org > https://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://wiki.list.org/x/AgA3 > Security Policy: http://wiki.list.org/x/QIA9 > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-users/heller%40deepsoft.com > > -- Robert Heller -- 978-544-6933 Deepwoods Software -- Custom Software Services http://www.deepsoft.com/ -- Linux Administration Services heller at deepsoft.com -- Webhosting Services From steve at pearwood.info Thu May 30 20:46:55 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 31 May 2019 10:46:55 +1000 Subject: [Mailman-Users] Why are mails not being sent to all users (mails seem to disappear)? In-Reply-To: <277b2ef2-ce12-ee91-3aa4-e1b8afec727f@pre-sense.de> References: <30690786-37a0-4862-2b7e-bbb629218203@msapiro.net> <8f64ab34-44dc-8ece-f647-16f2194e52a6@pre-sense.de> <277b2ef2-ce12-ee91-3aa4-e1b8afec727f@pre-sense.de> Message-ID: <20190531004655.GQ4221@ando.pearwood.info> On Thu, May 30, 2019 at 07:22:38PM +0200, Till D?rges wrote: > I figured it out. And it was neither a Mailman nor a Postfix problem. > > systemd-journald simply enforced some rate limiting. After removing it (setting > RateLimitBurst=0 in /etc/systemd/journald.conf), delivery for all mails can be seen > in the logfiles. Are you saying that the emails actually were sent, but systemd dropped the log messages so you wrongly thought they weren't sent? Or that systemd prevented the emails from being sent in the first place, and didn't log the fact that it had done so? I'm not sure which of those is more horrific, so I'm hoping I've misunderstood something because I will soon be migrating my mailman to a system using systemd. -- Steven From dmaziuk at bmrb.wisc.edu Thu May 30 20:59:41 2019 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Thu, 30 May 2019 19:59:41 -0500 Subject: [Mailman-Users] Why are mails not being sent to all users (mails seem to disappear)? In-Reply-To: <20190531004655.GQ4221@ando.pearwood.info> References: <30690786-37a0-4862-2b7e-bbb629218203@msapiro.net> <8f64ab34-44dc-8ece-f647-16f2194e52a6@pre-sense.de> <277b2ef2-ce12-ee91-3aa4-e1b8afec727f@pre-sense.de> <20190531004655.GQ4221@ando.pearwood.info> Message-ID: <3fbec267-c27b-3bef-dc10-ef1b17d7b33d@bmrb.wisc.edu> On 5/30/19 7:46 PM, Steven D'Aprano wrote: > Or that systemd prevented the emails from being sent in the first place, > and didn't log the fact that it had done so? I don't think systemd metastasized its own smtpd just yet, so it probably can't do that. Fear not, though, it has its own DNS resolver already, I'm sure it won't take long. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From chip at aresti.com Fri May 31 00:20:54 2019 From: chip at aresti.com (Chip Davis) Date: Fri, 31 May 2019 00:20:54 -0400 Subject: [Mailman-Users] Easy question for this crowd In-Reply-To: <20190530230320.489A026C001A@sharky3.deepsoft.com> References: <6b8120bb-fe79-2082-0f83-195376fff789@aresti.com> <20190530230320.489A026C001A@sharky3.deepsoft.com> Message-ID: <263b32d3-5a6a-9120-7fdd-5504e05513c9@aresti.com> Well, it was worth a try. :-/ About 12 hours after I put that RE in place, I got another one from a different domain in '.icu'. It was held for moderation, not automatically discarded. I have: 8 email addresses in accept_these_nonmembers 0 email addresses in hold_these_nonmembers 0 email addresses in reject_these_nonmembers ^@.*\.icu$ in discard_these_nonmembers 'Hold' for generic_nonmember_action 'Yes' for forward_auto_discards but it seemed to make no difference; the UCE was still held for moderation. I'm going to try putting "from: .*@.*\.icu" in header_filter_rules and see if that makes any difference. Any other ideas? -Chip- On 5/30/2019 7:03 PM, Robert Heller wrote: > At Thu, 30 May 2019 11:57:44 -0400 Chip Davis wrote: > >> >> I've supported a dozen Mailman listservers for over a dozen years. >> This doesn't represent much real effort most of the time. I've had to >> block specific users often and specific domains rarely, but this is >> the first time I've had to block an entire TLD. >> >> Recently I've been gifted with an inordinate amount of UCE from many >> different domains under the '.icu' TLD. >> >> Since Python RE's are _almost_ the same as the UNIX RE's I used many >> years ago, if I put >> >> ^@.*\.icu$ >> >> in discard_these_nonmembers, will it block all domains in that TLD? > > Yes. > >> >> And not block anyone else? > > Yes. > > I've done this, and then I took things a step further: > > What *I* have done (because I can), is configure rejection of both domains AND > cidrs at the Postfix level, putting REJECT's in both /etc/postfix/access and > /etc/postfix/cidr.clients. (I use *REJECT* for a reason: I figure if these > idiots are going to make trouble for me, I'll make trouble for them -- eg now > they will will get reject messages. Also when the addresses are from legit > mail servers, the admins there will get a wake up call and presumably do > something -- I have discovered that there is really little point in sending > anything to the [so-called] 'abuse' addresses.) > > I've also configured mimedefang and spamassassin to *reject* spam at the > Postfix as well. Very little gets though now. > >> >> Thanks, >> >> -Chip Davis- >> >> Mailman 2.1.27 > shared host >> linux 2.6.32-696.18.7.el6.x86_64 >> cPanel 80.0.10 >> ------------------------------------------------------ >> Mailman-Users mailing list Mailman-Users at python.org >> https://mail.python.org/mailman/listinfo/mailman-users >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Security Policy: http://wiki.list.org/x/QIA9 >> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ >> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/heller%40deepsoft.com >> >> > From doerges at pre-sense.de Fri May 31 02:31:45 2019 From: doerges at pre-sense.de (=?UTF-8?Q?Till_D=c3=b6rges?=) Date: Fri, 31 May 2019 08:31:45 +0200 Subject: [Mailman-Users] Why are mails not being sent to all users (mails seem to disappear)? In-Reply-To: <20190531004655.GQ4221@ando.pearwood.info> References: <30690786-37a0-4862-2b7e-bbb629218203@msapiro.net> <8f64ab34-44dc-8ece-f647-16f2194e52a6@pre-sense.de> <277b2ef2-ce12-ee91-3aa4-e1b8afec727f@pre-sense.de> <20190531004655.GQ4221@ando.pearwood.info> Message-ID: <8bbc7b47-5c78-a943-c592-c3138c474cd0@pre-sense.de> Am 31.05.19 um 02:46 schrieb Steven D'Aprano: > On Thu, May 30, 2019 at 07:22:38PM +0200, Till D?rges wrote: > >> I figured it out. And it was neither a Mailman nor a Postfix problem. >> >> systemd-journald simply enforced some rate limiting. After removing it (setting >> RateLimitBurst=0 in /etc/systemd/journald.conf), delivery for all mails can be seen >> in the logfiles. > > Are you saying that the emails actually were sent, but systemd dropped > the log messages so you wrongly thought they weren't sent? That was the case. Only the log messages in syslog were missing. To be clear: AFAICT everything was working as configured. Regards -- Till -- Dipl.-Inform. Till D?rges doerges at pre-sense.de PRESENSE Technologies GmbH Sachsenstr. 5, D-20097 HH Gesch?ftsf?hrer/Managing Directors AG Hamburg, HRB 107844 Till D?rges, J?rgen Sander USt-IdNr.: DE263765024 From mark at msapiro.net Fri May 31 10:11:29 2019 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 31 May 2019 07:11:29 -0700 Subject: [Mailman-Users] Easy question for this crowd In-Reply-To: <263b32d3-5a6a-9120-7fdd-5504e05513c9@aresti.com> References: <6b8120bb-fe79-2082-0f83-195376fff789@aresti.com> <20190530230320.489A026C001A@sharky3.deepsoft.com> <263b32d3-5a6a-9120-7fdd-5504e05513c9@aresti.com> Message-ID: On 5/30/19 9:20 PM, Chip Davis wrote: > > About 12 hours after I put that RE in place, I got another one from a > different domain in '.icu'.? It was held for moderation, not > automatically discarded. > > I have: > ? 8 email addresses in accept_these_nonmembers > ? 0 email addresses in hold_these_nonmembers > ? 0 email addresses in reject_these_nonmembers > ? ^@.*\.icu$ in discard_these_nonmembers > ? 'Hold' for generic_nonmember_action > ? 'Yes' for forward_auto_discards > but it seemed to make no difference; the UCE was still held for moderation. The *_these_nonmembers checks only check one address which is what Mailman considers the sender of the message. What address this is depends on a config setting. The doc says: > This can return either the From: header, the Sender: header or the > envelope header (a.k.a. the unixfrom header). The first non-empty > header value found is returned. However the search order is > determined by the following: > > - If mm_cfg.USE_ENVELOPE_SENDER is true, then the search order is > Sender:, From:, unixfrom > > - Otherwise, the search order is From:, Sender:, unixfrom So in your case, it may not be checking the From: > I'm going to try putting "from: .*@.*\.icu" in header_filter_rules and > see if that makes any difference. It probably should to be "^from: .*@.*\.icu\s" to avoid matching something like Subject: mail from: user at server.icu not discarded or From: user at sub.icure.com -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From chip at aresti.com Fri May 31 13:29:32 2019 From: chip at aresti.com (Chip Davis) Date: Fri, 31 May 2019 13:29:32 -0400 Subject: [Mailman-Users] Easy question for this crowd In-Reply-To: References: <6b8120bb-fe79-2082-0f83-195376fff789@aresti.com> <20190530230320.489A026C001A@sharky3.deepsoft.com> <263b32d3-5a6a-9120-7fdd-5504e05513c9@aresti.com> Message-ID: <00162b2d-d080-716f-a8f2-3f0bf81d90e6@aresti.com> Thanks, Mark. I hadn't thought of the "from:" being embedded in the Subject: header. And your RE correction makes perfect sense once I see it. ;-) I'm pretty sure I don't have access to 'mm.config.[anything]' so I assume it's the default value. Odds are, it was my imperfect RE that was keeping if from tripping. Thanks All, -Chip- On 5/31/2019 10:11 AM, Mark Sapiro wrote: > On 5/30/19 9:20 PM, Chip Davis wrote: >> >> About 12 hours after I put that RE in place, I got another one from a >> different domain in '.icu'.? It was held for moderation, not >> automatically discarded. >> >> I have: >> ? 8 email addresses in accept_these_nonmembers >> ? 0 email addresses in hold_these_nonmembers >> ? 0 email addresses in reject_these_nonmembers >> ? ^@.*\.icu$ in discard_these_nonmembers >> ? 'Hold' for generic_nonmember_action >> ? 'Yes' for forward_auto_discards >> but it seemed to make no difference; the UCE was still held for moderation. > > > The *_these_nonmembers checks only check one address which is what > Mailman considers the sender of the message. What address this is > depends on a config setting. The doc says: > >> This can return either the From: header, the Sender: header or the >> envelope header (a.k.a. the unixfrom header). The first non-empty >> header value found is returned. However the search order is >> determined by the following: >> >> - If mm_cfg.USE_ENVELOPE_SENDER is true, then the search order is >> Sender:, From:, unixfrom >> >> - Otherwise, the search order is From:, Sender:, unixfrom > > So in your case, it may not be checking the From: > > >> I'm going to try putting "from: .*@.*\.icu" in header_filter_rules and >> see if that makes any difference. > > > It probably should to be "^from: .*@.*\.icu\s" to avoid matching > something like > > Subject: mail from: user at server.icu not discarded > > or > > From: user at sub.icure.com > > From rmcclung at afilias.info Fri May 31 13:47:44 2019 From: rmcclung at afilias.info (Ryan McClung) Date: Fri, 31 May 2019 13:47:44 -0400 Subject: [Mailman-Users] Migrated Mailman to another server Message-ID: Greetings, Migrated mailman to another server and updated it. Site Admin can see the pending queue on all lists but the lists owners/administrators cannot. Running Centos7 and current build of mailman. (2.1.15) Lists have been migrated and are functioning properly. Permissions fix finds nothing. Any help appreciated. -- Ryan McClung Systems Administrator @ Afilias Canada A. 204-4141 Yonge Street, Toronto, ON, Canada, M2P 2A8 W. www.afilias.info T. +1.416.646.3304 x4186 From rmcclung at afilias.info Fri May 31 14:34:19 2019 From: rmcclung at afilias.info (Ryan McClung) Date: Fri, 31 May 2019 14:34:19 -0400 Subject: [Mailman-Users] Migrated Mailman to another server In-Reply-To: References: Message-ID: Close this. I figured it out. On Fri, May 31, 2019 at 1:47 PM Ryan McClung wrote: > Greetings, > > Migrated mailman to another server and updated it. > > Site Admin can see the pending queue on all lists but the lists > owners/administrators cannot. > > Running Centos7 and current build of mailman. (2.1.15) > > Lists have been migrated and are functioning properly. Permissions fix > finds nothing. > > Any help appreciated. > > -- > > Ryan McClung > Systems Administrator @ Afilias Canada > A. 204-4141 Yonge Street, Toronto, ON, Canada, M2P 2A8 > > W. www.afilias.info > T. +1.416.646.3304 x4186 > -- Ryan McClung Systems Administrator @ Afilias Canada A. 204-4141 Yonge Street, Toronto, ON, Canada, M2P 2A8 W. www.afilias.info T. +1.416.646.3304 x4186 From cpz at tuunq.com Fri May 31 14:47:42 2019 From: cpz at tuunq.com (Carl Zwanzig) Date: Fri, 31 May 2019 11:47:42 -0700 Subject: [Mailman-Users] Migrated Mailman to another server In-Reply-To: References: Message-ID: On 5/31/2019 11:34 AM, Ryan McClung wrote: > Close this. I figured it out. What was the problem? z! From geek at uniserve.com Fri May 31 15:23:19 2019 From: geek at uniserve.com (Dave Stevens) Date: Fri, 31 May 2019 12:23:19 -0700 Subject: [Mailman-Users] Migrated Mailman to another server In-Reply-To: References: Message-ID: <20190531122319.6dd8b126@dave-900X1B> On Fri, 31 May 2019 14:34:19 -0400 Ryan McClung wrote: > I figured it out. what was the fix?