From pshute at nuw.org.au  Thu Feb  1 00:00:15 2018
From: pshute at nuw.org.au (Peter Shute)
Date: Thu, 1 Feb 2018 05:00:15 +0000
Subject: [Mailman-Users] Only unmoderated messages getting to list?
In-Reply-To: <efe09ec1-d4b9-6a99-a4b8-5a64dc68aa0f@msapiro.net>
References: <555274d6f58a42ef95a34ee7f7217064@NVSM1.nuw.org.au>,
 <efe09ec1-d4b9-6a99-a4b8-5a64dc68aa0f@msapiro.net>
Message-ID: <AFD45431-C524-4E7C-8F60-46524D692C1F@nuw.org.au>

Can I get at those logs via cPanel?

Since I emailed, an approved message got through. Two of the three affected emails were from non members. We allow non members to post because some people just don?t bother subscribing.

The sender of an affected email subscribed and sent another email, and after being approved, it got through.

Sent from my iPhone

> On 1 Feb 2018, at 3:04 pm, Mark Sapiro <mark at msapiro.net> wrote:
> 
>> On 01/31/2018 07:08 PM, Peter Shute wrote:
>> Today we've noticed that messages we've approved aren't getting sent out by the list, or at least they aren't being received by list members.
> 
> 
> Are they archived?
> 
> 
>> I sent a couple of test messages to the list, and they got sent out ok. What could be happening that only allows unmoderated messages through?
> 
> 
> What's in Mailman's 'vette' and 'error' logs from the time of approval.
> 
> One possibility is the messages held and approved were subsequently
> discarded by content filtering. This can happen for example if the
> message is HTML only, content filtering doesn't accept text/html and
> filter_action is discard. This and other possibilities will be logged in
> the vette log.
> 
> -- 
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/pshute%40nuw.org.au

From mark at msapiro.net  Thu Feb  1 00:28:28 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 31 Jan 2018 21:28:28 -0800
Subject: [Mailman-Users] Only unmoderated messages getting to list?
In-Reply-To: <AFD45431-C524-4E7C-8F60-46524D692C1F@nuw.org.au>
References: <555274d6f58a42ef95a34ee7f7217064@NVSM1.nuw.org.au>
 <efe09ec1-d4b9-6a99-a4b8-5a64dc68aa0f@msapiro.net>
 <AFD45431-C524-4E7C-8F60-46524D692C1F@nuw.org.au>
Message-ID: <b2d9f972-e92b-9f0a-51ac-966f5e609da0@msapiro.net>

On 01/31/2018 09:00 PM, Peter Shute wrote:
> Can I get at those logs via cPanel?


If you can access the underlying file system, the logs are in
/usr/local/cpanel/3rdparty/mailman/logs.

You can't access them directly via the control panel.


> Since I emailed, an approved message got through. Two of the three affected emails were from non members. We allow non members to post because some people just don?t bother subscribing.
> 
> The sender of an affected email subscribed and sent another email, and after being approved, it got through.


It's not possible to know what's going on without seeing the logs. If
you don't have access to them and the issue persists, you'll have to
contact your hosting provider for help.

Also see <https://wiki.list.org/DOC/Mailman%20and%20CPanel>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From heller at deepsoft.com  Thu Feb  1 08:07:10 2018
From: heller at deepsoft.com (Robert Heller)
Date: Thu,  1 Feb 2018 08:07:10 -0500 (EST)
Subject: [Mailman-Users] Only unmoderated messages getting to list?
In-Reply-To: <555274d6f58a42ef95a34ee7f7217064@NVSM1.nuw.org.au>
References: <555274d6f58a42ef95a34ee7f7217064@NVSM1.nuw.org.au>
Message-ID: <20180201130711.62C00732444@sharky3.deepsoft.com>

At Thu, 1 Feb 2018 03:08:11 +0000 Peter Shute <pshute at nuw.org.au> wrote:

> 
> Today we've noticed that messages we've approved aren't getting sent out by
> the list, or at least they aren't being received by list members.
> 
> I sent a couple of test messages to the list, and they got sent out ok. What
> could be happening that only allows unmoderated messages through?
> 
> There haven't been a lot of messages involved yet. Maybe three moderated
> messages have gone astray, from three different members, one of whom is a
> moderator.

Does the list have archives (public or private)?  Are the messages showing up 
there?

Are the messages "not being sent out" from @gmail.com addresses?  GMail 
"tosses" messages a poster from gmail sends when it comes back to that poster. 
The message does get delivered to everyone else.

> 
> We're on v2.1.23.
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/heller%40deepsoft.com
> 
>                      

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller at deepsoft.com       -- Webhosting Services
                                                                                  

From geek at uniserve.com  Thu Feb  1 15:41:03 2018
From: geek at uniserve.com (Dave Stevens)
Date: Thu, 01 Feb 2018 12:41:03 -0800
Subject: [Mailman-Users] two problems with Mailman 2.1
Message-ID: <20180201124103.21316r5vvig3zyan@webmail.uniserve.com>

I?m having a couple of problems with a Mailman 2.1 list.

I want to add several addresses as new subscribers so as a first step  
I added one of them using the mass subscription facility in membership  
management. Mailman accepted the data but when I subsequently checked  
the subscriber list it wasn?t there. This has been the case for two  
days now.

Today I checked manually that the recipient address was in fact  
working and decided to resubscribe using the same method. I enter the  
data in the text box (I?ve done this several times) then click on the  
"submit your changes" button below and get this message, ?Error: The  
form lifetime has expired. (request forgery check)?

This happens without any special delay on my part, not more than a few  
minutes. Please advise.

Dave


-- 
"As long as politics is the shadow cast on society by big business,
the attenuation of the shadow will not change the substance."

-- John Dewey






From mark at msapiro.net  Thu Feb  1 18:12:44 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 1 Feb 2018 15:12:44 -0800
Subject: [Mailman-Users] two problems with Mailman 2.1
In-Reply-To: <20180201124103.21316r5vvig3zyan@webmail.uniserve.com>
References: <20180201124103.21316r5vvig3zyan@webmail.uniserve.com>
Message-ID: <716bb0e5-cd51-0c23-8d26-528001fa91b7@msapiro.net>

On 02/01/2018 12:41 PM, Dave Stevens wrote:
> I?m having a couple of problems with a Mailman 2.1 list.
> 
> I want to add several addresses as new subscribers so as a first step I
> added one of them using the mass subscription facility in membership
> management. Mailman accepted the data but when I subsequently checked
> the subscriber list it wasn?t there. This has been the case for two days
> now.


When you submitted the mass subscribe form the first time, did you get a
response with a message at the top saying "successfully subscribed" or
did it say something else.


> Today I checked manually that the recipient address was in fact working
> and decided to resubscribe using the same method. I enter the data in
> the text box (I?ve done this several times) then click on the "submit
> your changes" button below and get this message, ?Error: The form
> lifetime has expired. (request forgery check)?
> 
> This happens without any special delay on my part, not more than a few
> minutes. Please advise.


You need to first get the form and then submit it within whatever the
FORM_LIFETIME setting is in your installation. The default is one hour
but could be different in your installation.

I suspect the issue is something else. I don't know what the issue might
be, but one thing to check is to look at the HTML source of the
admin/LIST_NAME/members/add page in your browser. The FORM tag in that
source should be

<FORM action="../../../admin/LIST_NAME/members/add" method="POST"
enctype="multipart/form-data">

If instead it is something like

<FORM action="http://example.com/mailman/admin/gpc-talk/members/add"
method="POST" enctype="multipart/form-data">

And the URL in the address bar is different, i.e. a different domain or
a different scheme (like https vs http) that might be an issue.

Is this your Mailman installation or a hosted installation?

Do other 'admin' and 'admindb' pages work? I.e. if you change something
on the admin General Options page and submit, does it work or produce
the same form lifetime error?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From pshute at nuw.org.au  Thu Feb  1 23:26:05 2018
From: pshute at nuw.org.au (Peter Shute)
Date: Fri, 2 Feb 2018 04:26:05 +0000
Subject: [Mailman-Users] Only unmoderated messages getting to list?
In-Reply-To: <20180201130711.62C00732444@sharky3.deepsoft.com>
References: <555274d6f58a42ef95a34ee7f7217064@NVSM1.nuw.org.au>
 <20180201130711.62C00732444@sharky3.deepsoft.com>
Message-ID: <f26a6500fe464b0e8f37997478d3d53e@NVSM1.nuw.org.au>

Robert Heller wrote:
> Does the list have archives (public or private)?  Are the messages showing up
> there?

No archives.
 
> Are the messages "not being sent out" from @gmail.com addresses?  GMail
> "tosses" messages a poster from gmail sends when it comes back to that
> poster.
>
> The message does get delivered to everyone else.

I have several non gmail addresses I use to test with, and they're not receiving them either.

From Hagedorn at uni-koeln.de  Fri Feb  2 05:26:19 2018
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Fri, 02 Feb 2018 11:26:19 +0100
Subject: [Mailman-Users] Stuck OutgoingRunner
Message-ID: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>

Hi,

we've been running Mailman for many years and have never had stability 
issues, but about a month ago we moved the server from RHEL 5 to RHEL 6 and 
to the current version (2.1.25), and since then it has already happened 
twice that one of our four OutgoingRunners got "stuck" and stopped handling 
mail. When that happens a simple restart of the service does not work. 
These processes remained:

mailman   1663  0.0  0.0 233860  2204 ?        Ss   Jan16   0:00 
/usr/bin/python2.7 /usr/lib/mailman/bin/mailmanctl -s -q start
mailman   1677  0.1  0.9 295064 73284 ?        S    Jan16  35:35 
/usr/bin/python2.7 /usr/lib/mailman/bin/qrunner --runner=OutgoingRunner:3:4 
-s

root at mailman3/usr/lib/mailman/bin]$ strace -p 1677
Process 1677 attached
recvfrom(10, ^CProcess 1677 detached

[root at mailman3/usr/lib/mailman/bin]$ lsof -p 1677
COMMAND    PID    USER   FD   TYPE   DEVICE SIZE/OFF   NODE NAME
python2.7 1677 mailman  cwd    DIR    253,0     4096 173998 /usr/lib/mailman
python2.7 1677 mailman  rtd    DIR    253,0     4096      2 /
...
python2.7 1677 mailman   10u  IPv6 46441320      0t0    TCP 
mailman3.rrz.uni-koeln.de:55764->smtp-out.rrz.uni-koeln.de:smtp 
(ESTABLISHED)

In both instances the OutgoingRunner was stuck on an SMTP connection. I had 
to use "kill -9" to get rid of it.

Any ideas what might be causing that?

Cheers
Sebastian
-- 
    .:.Sebastian Hagedorn - Weyertal 121 (Geb?ude 133), Zimmer 2.02.:.
                 .:.Regionales Rechenzentrum (RRZK).:.
   .:.Universit?t zu K?ln / Cologne University - ? +49-221-470-89578.:.

From mark at msapiro.net  Sat Feb  3 00:40:25 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 2 Feb 2018 21:40:25 -0800
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
Message-ID: <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>

On 02/02/2018 02:26 AM, Sebastian Hagedorn wrote:
> Hi,
> 
> we've been running Mailman for many years and have never had stability
> issues, but about a month ago we moved the server from RHEL 5 to RHEL 6
> and to the current version (2.1.25), and since then it has already
> happened twice that one of our four OutgoingRunners got "stuck" and
> stopped handling mail. When that happens a simple restart of the service
> does not work. These processes remained:
> 
> mailman?? 1663? 0.0? 0.0 233860? 2204 ???????? Ss?? Jan16?? 0:00
> /usr/bin/python2.7 /usr/lib/mailman/bin/mailmanctl -s -q start
> mailman?? 1677? 0.1? 0.9 295064 73284 ???????? S??? Jan16? 35:35
> /usr/bin/python2.7 /usr/lib/mailman/bin/qrunner
> --runner=OutgoingRunner:3:4 -s


Because pid 1677 didn't respond to the SIGINT from the master and the
master is still waiting for it to exit.


> root at mailman3/usr/lib/mailman/bin]$ strace -p 1677
> Process 1677 attached
> recvfrom(10, ^CProcess 1677 detached
> 
> [root at mailman3/usr/lib/mailman/bin]$ lsof -p 1677
> COMMAND??? PID??? USER?? FD?? TYPE?? DEVICE SIZE/OFF?? NODE NAME
> python2.7 1677 mailman? cwd??? DIR??? 253,0???? 4096 173998
> /usr/lib/mailman
> python2.7 1677 mailman? rtd??? DIR??? 253,0???? 4096????? 2 /
> ...
> python2.7 1677 mailman?? 10u? IPv6 46441320????? 0t0??? TCP
> mailman3.rrz.uni-koeln.de:55764->smtp-out.rrz.uni-koeln.de:smtp
> (ESTABLISHED)
> 
> In both instances the OutgoingRunner was stuck on an SMTP connection. I
> had to use "kill -9" to get rid of it.
> 
> Any ideas what might be causing that?


I think I've seen this once or maybe twice, I don't recall details. I
wasn't able to determine a cause. I haven't seen it in years.

Did you look at the out queue, and if so was there a .bak file there.
This would be the entry currently being processed.

Also, the TCP connection to the MTA being ESTABLISHED says the
OutgoingRunner has called SMTPDirect.process() and it in turn is
somewhere in its delivery loop of sending SMTP transactions.

Are there any clues in the MTA logs?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From Hagedorn at uni-koeln.de  Sat Feb  3 04:03:56 2018
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Sat, 03 Feb 2018 10:03:56 +0100
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
 <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
Message-ID: <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>

Thanks for your reply!

> On 02/02/2018 02:26 AM, Sebastian Hagedorn wrote:
>> [root at mailman3/usr/lib/mailman/bin]$ lsof -p 1677
>> COMMAND??? PID??? USER?? FD?? TYPE?? DEVICE SIZE/OFF??
>> NODE NAME python2.7 1677 mailman? cwd??? DIR??? 253,0????
>> 4096 173998 /usr/lib/mailman
>> python2.7 1677 mailman? rtd??? DIR??? 253,0????
>> 4096????? 2 / ...
>> python2.7 1677 mailman?? 10u? IPv6 46441320????? 0t0??? TCP
>> mailman3.rrz.uni-koeln.de:55764->smtp-out.rrz.uni-koeln.de:smtp
>> (ESTABLISHED)
>>
>> In both instances the OutgoingRunner was stuck on an SMTP connection. I
>> had to use "kill -9" to get rid of it.
>>
>> Any ideas what might be causing that?
>
> I think I've seen this once or maybe twice, I don't recall details. I
> wasn't able to determine a cause. I haven't seen it in years.
>
> Did you look at the out queue, and if so was there a .bak file there.
> This would be the entry currently being processed.

I looked at the out queue, and there was no .bak file.

> Also, the TCP connection to the MTA being ESTABLISHED says the
> OutgoingRunner has called SMTPDirect.process() and it in turn is
> somewhere in its delivery loop of sending SMTP transactions.
>
> Are there any clues in the MTA logs?

I just found this in Mailman's smtp-failures log:

Feb 01 14:28:49 2018 (1674) Low level smtp error: [Errno 111] Connection 
refused, msgid: 
<B51BA08829F27146A07699F58B941234A27397BF at EX10DAG2.intern.xxx>
Feb 01 14:28:49 2018 (1674) delivery to xxx at uni-koeln.de failed with code 
-1: [Errno 111] Connection refused

I can't prove it, but this time stamp seems to coincide with the moment the 
OutgoingRunner got stuck, based on the age of the queue files. The 
receiving SMTP server was under heavy load at that moment, so it is 
possible that it might have refused the connection.

The message was delivered successfully after I killed the stuck runner and 
restarted the service. I wasn't able to find anything pertinent on the 
receiving server.

If this should happen again, what should we look for? Would a gdb backtrace 
be helpful?
--
Sebastian Hagedorn - Weyertal 121, Zimmer 2.02
Regionales Rechenzentrum (RRZK)
Universit?t zu K?ln / Cologne University - Tel. +49-221-470-89578

From mark at msapiro.net  Sat Feb  3 22:13:33 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 3 Feb 2018 19:13:33 -0800
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
 <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
 <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>
Message-ID: <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>

On 02/03/2018 01:03 AM, Sebastian Hagedorn wrote:
>>
>> Did you look at the out queue, and if so was there a .bak file there.
>> This would be the entry currently being processed.
> 
> I looked at the out queue, and there was no .bak file.


Interesting. That says that OutgoingRunner is not currently delivering a
message, but that is inconsistent with this:


>> Also, the TCP connection to the MTA being ESTABLISHED says the
>> OutgoingRunner has called SMTPDirect.process() and it in turn is
>> somewhere in its delivery loop of sending SMTP transactions.
>>
>> Are there any clues in the MTA logs?
> 
> I just found this in Mailman's smtp-failures log:
> 
> Feb 01 14:28:49 2018 (1674) Low level smtp error: [Errno 111] Connection
> refused, msgid:
> <B51BA08829F27146A07699F58B941234A27397BF at EX10DAG2.intern.xxx>
> Feb 01 14:28:49 2018 (1674) delivery to xxx at uni-koeln.de failed with
> code -1: [Errno 111] Connection refused
> 
> I can't prove it, but this time stamp seems to coincide with the moment
> the OutgoingRunner got stuck, based on the age of the queue files. The
> receiving SMTP server was under heavy load at that moment, so it is
> possible that it might have refused the connection.

Normally, that won't cause a problem like this. This occurs at a fairly
low level in SMTPDirect.py when Mailman is initiating a transaction with
the MTA to send to one or more recipients. The recipients will be marked
as "refused retryably" and OutgoingRunner will queue the message for
those recipients. in the retry queue to be retried

You can set SMTPLIB_DEBUG_LEVEL = 1 in mm_cfg.py to log copious smtplib
debugging info to Mailman's error log. Then the log will show the last
thing that was done before the hang.


> If this should happen again, what should we look for? Would a gdb
> backtrace be helpful?


It might be if you can find just where in the code it's hung. Also, I
didn't look carefully before, but in your OP, you show

> mailman   1663  0.0  0.0 233860  2204 ?        Ss   Jan16   0:00 
> /usr/bin/python2.7 /usr/lib/mailman/bin/mailmanctl -s -q start
> mailman   1677  0.1  0.9 295064 73284 ?        S    Jan16  35:35 
> /usr/bin/python2.7 /usr/lib/mailman/bin/qrunner --runner=OutgoingRunner:3:4 
> -s

The status of 'S' for OutgoingRunner is "uninterruptable sleep". This
means it's either called time.sleep for QRUNNER_SLEEP_TIME (default = 1
second) which is unlikely as it should wake up, or it's waiting for
response from something, most likely a response from the MTA.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From futatuki at poem.co.jp  Sat Feb  3 22:54:43 2018
From: futatuki at poem.co.jp (Yasuhito FUTATSUKI)
Date: Sun, 4 Feb 2018 12:54:43 +0900
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
 <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
 <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>
 <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>
Message-ID: <35b995d2-f45e-2e07-fbbc-aa8c69db38cc@poem.co.jp>


On 02/04/18 12:13, Mark Sapiro wrote:
> The status of 'S' for OutgoingRunner is "uninterruptable sleep". This
> means it's either called time.sleep for QRUNNER_SLEEP_TIME (default = 1
> second) which is unlikely as it should wake up, or it's waiting for
> response from something, most likely a response from the MTA.

As far as I read the code, if OutgoingRunner catch SIGINT during waiting
for response from the MTA, the signal handler for SIGINT in qrunner set
flag to exit from loop, then socket module raise socket.error for EINTR,
but SMTP module retry to read from socket and waiting for response until
receiving response or connection closing (from MTA side or by error).
Thus it cannot reach to the code to exit if the connection is kept alive
and MTA send no data.

-- 
Yasuhito FUTATSUKI <futatuki at poem.co.jp>

From mark at msapiro.net  Sun Feb  4 12:51:37 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 4 Feb 2018 09:51:37 -0800
Subject: [Mailman-Users] Mailman 2.1.26 Security release Feb 4, 2018
In-Reply-To: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>
References: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>
Message-ID: <1d05c305-c440-a07b-b992-85e8bf6c3bb0@msapiro.net>

I am pleased to announce the release of Mailman 2.1.26.

Python 2.4 is the minimum supported, but Python 2.7 is strongly recommended.

This is a security and bug fix release with a couple of new features.
See the attached README.txt for details.

For those who are concerned about the security vulnerability and can't
upgrade immediately, there is a patch at
<https://bugs.launchpad.net/mailman/+bug/1747209/+attachment/5048344/+files/options.patch>
to fix the security issue. More information on the issue itself is in
the bug report at <https://bugs.launchpad.net/mailman/+bug/1747209>.

Mailman is free software for managing email mailing lists and
e-newsletters. Mailman is used for all the python.org and
SourceForge.net mailing lists, as well as at hundreds of other sites.

For more information, please see our web site at one of:

http://www.list.org
https://www.gnu.org/software/mailman
http://mailman.sourceforge.net/
https://mirror.list.org/

Mailman 2.1.26 can be downloaded from

https://launchpad.net/mailman/2.1/
https://ftp.gnu.org/gnu/mailman/
https://sourceforge.net/projects/mailman/

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
-------------- next part --------------
2.1.26 (04-Feb-2018)

  Security

    - An XSS vulnerability in the user options CGI could allow a crafted URL
      to execute arbitrary javascript in a user's browser.  A related issue
      could expose information on a user's options page without requiring
      login.  These are fixed.  Thanks to Calum Hutton for the report.
      CVE-2018-5950  (LP: #1747209)

  New Features

    - Thanks to David Sieb?rger who adapted an existing patch by Andrea
      Veri to use Google reCAPTCHA v2 there is now the ability to add
      reCAPTCHA to the listinfo subscribe form.  There are two new mm_cfg.py
      settings for RECAPTCHA_SITE_KEY and RECAPTCHA_SECRET_KEY, the values
      for which you obtain for your domain(s) from Google at
      <https://www.google.com/recaptcha/admin>.

    - Thanks to Lindsay Haisley, there is a new bin/mailman-config command
      to display various information about this Mailman version and how it
      was configured.

  i18n

    - The Japanese message catalog has been updated for added strings by
      Yasuhito FUTATSUKI.

    - The German translation of a couple of templates has been updated by
      Thomas Hochstein.

    - The Japanese translation of Defaults.py.in has been updated by
      Yasuhito FUTATSUKI.

  Bug fixes and other patches

    - Fixed an i18n bug in the reCAPTCHA feature.  (LP: #1746189)

    - Added a few more environment variables to the list of those passed
      to CGIs to support an nginx/uwsgi configuration.  (LP #1744739)

    - Mailman 2.1.22 introduced a Python 2.7 dependency that could affect
      bin/arch processing a message without a valid Date: header.  The
      dependency has been removed.  (LP: #1740543)

    - Messages held for header_filter_rules now show the matched regexp in
      the hold reason.  (LP: #1737371)

    - When updating the group and mode of a .db file with Mailman's Postfix
      integration, a missing file is ignored.  (LP: #1734162)

    - The DELIVERY_RETRY_WAIT setting is now effective.  (LP: #1729472)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20180204/76561c18/attachment.sig>

From turnbull.stephen.fw at u.tsukuba.ac.jp  Mon Feb  5 03:07:05 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 5 Feb 2018 17:07:05 +0900
Subject: [Mailman-Users] Reply-to options not working
In-Reply-To: <76c12a6d-59bc-3b39-12ff-a466c20a2ba0@bmrb.wisc.edu>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <87d50fa6-e982-1e79-9a52-e792b5023a78@msapiro.net>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e8e71fb4-5690-b1cb-6ea5-38567aca0d14@maileater.net>
 <23150.42607.587920.347874@turnbull.sk.tsukuba.ac.jp>
 <d0929469-e75d-ce27-427c-58b3f092c201@aresti.com>
 <23152.2005.960382.794473@turnbull.sk.tsukuba.ac.jp>
 <76c12a6d-59bc-3b39-12ff-a466c20a2ba0@bmrb.wisc.edu>
Message-ID: <23160.4265.840655.825350@turnbull.sk.tsukuba.ac.jp>

Dimitri Maziuk writes:
 > On 2018-01-29 23:51, Stephen J. Turnbull wrote:
 > 
 > ... [ Reply-To ] should have a checkbox "same as my
 > >  From address."
 > 
 > Oh, great, now I'll rreecceeiivvee eevveerryytthhiinngg
 > ttwwiiccee..

No, that's not the way Reply-To works.

Anyway, the point is that Reply-To would *not* be set in that case,
because the RFC says that replies should go to From by default.  It
would simply be a UI trick to allow less sophisticated users to do the
obviously correct thing, which also involves less typing or copying/
pasting.  Thus avoiding the attractive nuisance described in the OP.

From turnbull.stephen.fw at u.tsukuba.ac.jp  Mon Feb  5 03:22:32 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 5 Feb 2018 17:22:32 +0900
Subject: [Mailman-Users] Reply-to options not working
In-Reply-To: <d1dd087e-4343-a6d8-904f-b2009fed5a33@bmrb.wisc.edu>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <87d50fa6-e982-1e79-9a52-e792b5023a78@msapiro.net>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e8e71fb4-5690-b1cb-6ea5-38567aca0d14@maileater.net>
 <d4330120-4119-06e7-0b34-f120a44210f5@spamtrap.tnetconsulting.net>
 <3e4a76f4-a68b-39e5-515c-6308654c6665@jordan.maileater.net>
 <3cf69a87-3fbc-2e5b-4fc1-38f60592b166@spamtrap.tnetconsulting.net>
 <d4713ce7-8834-2f03-07ad-4a8b978894a8@jordan.maileater.net>
 <a6511c99-f24b-b880-9247-f94a53eb53a6@spamtrap.tnetconsulting.net>
 <2531c94e-6369-c58c-81ea-000539c27a3a@bmrb.wisc.edu>
 <d32990bd-deeb-2e50-c030-e1b651f49b01@spamtrap.tnetconsulting.net>
 <d1dd087e-4343-a6d8-904f-b2009fed5a33@bmrb.wisc.edu>
Message-ID: <23160.5192.25370.470605@turnbull.sk.tsukuba.ac.jp>

Dimitri Maziuk writes:

 > Does it ave the same Message-ID though?

According to RFC, Message-ID is an originator field, and MUST be
present and MUST be unique.  The MUA or submission agent should add it
before handing off to the MTA.  As a last resort the MTA may add it.
If it gets past the MTA without it, it's non-conforming.  Mediator
software (such as mailing list managers) MAY add a Resent-Message-ID
field, which is not restricted in number.

In some cases it makes sense for a Mediator such as Mailman to change
the Message-ID, and it will always add one if not present.  Mark is
authoritative on when Mailman does it.  However, normally Mailman (and
other mailing list managers) will not change it, indicating that the
list considers the outgoing message to be the same from the author's
point of view as the incoming message.

Of course this is a judgment call.  Obviously *some* changes such as
adding Received fields to the header don't change "the message".  On
the other hand, I think it's reasonable for authors to claim that
mailing lists that go stripping attachments or HTML parts, or
translating text/html to text/plain, as Mailman can be configured to
do, have edited the message enough that it's a new message.  Stuff in
the middle (list tags and serial numbers in Subject, headers and
footers on the body) I would *never* consider to make a new message,
but some people claim they think so.

I don't recall ever seeing such a complaint from authors, even from
people who want the HTML preserved.  Authors don't question that it's
the same message, they just want the presentation preserved.  The
people who do question it generally do it in service of claims that
the mailing list "owns" the message so Reply-To munging is
RFC-conforming, etc.

The RFCs punt on "when is it a new message" in exactly the same way,
by the way.  "It's your call, just be sure to change the Message-ID if
you think it's a new message now."

From turnbull.stephen.fw at u.tsukuba.ac.jp  Mon Feb  5 03:29:36 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 5 Feb 2018 17:29:36 +0900
Subject: [Mailman-Users] Reply-to options not working
In-Reply-To: <3ca7b4ec-348d-37f7-f6bd-f0a1ea2416be@jordan.maileater.net>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <87d50fa6-e982-1e79-9a52-e792b5023a78@msapiro.net>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e523c9c5-1473-b6e6-6c22-6a68935f3092@spamtrap.tnetconsulting.net>
 <23150.42437.832863.161429@turnbull.sk.tsukuba.ac.jp>
 <734adb86-8722-c9f5-9e3f-ddc22abd99a8@jordan.maileater.net>
 <23152.2290.349851.414385@turnbull.sk.tsukuba.ac.jp>
 <3ca7b4ec-348d-37f7-f6bd-f0a1ea2416be@jordan.maileater.net>
Message-ID: <23160.5616.571593.574631@turnbull.sk.tsukuba.ac.jp>

Jordan Brown writes:

 > If you have "smart reply" as a separate function, yes.? If you have
 > the typical "Reply" and "Reply All", and the mailing list software
 > sets "Reply-To: <list>", then replying to the author is awkward and
 > error-prone.

Sure, but in this thread we all agree that Reply-To munging is
problematic.  I believe that the widespread availability of "smart
reply" would greatly reduce the incentives for lists to munge.  It
doesn't eliminate the need for reply-author and reply-all, but in my
experience it reduces them dramatically, and basically eliminates the
need for reply-list.

The question I asked, which you misinterpreted completely IMO, and
Grant partially agreed with is "Does an algorithm which 1. gives
overriding precedence to Reply-To, 2. otherwise if List-Post is
present directs it there, and 3. finally falls back to From, seem
likely to DTRT most of the time?"  I know from analyzing my own posts
that for me the answer is yes, and from experience I know that I'm
reasonably good at flexibly using different reply functions to meet
the needs of the moment even though I spend a lot of time using just
one function.

I also think that the evidence is pretty strong that the Reply-To
mungers would like this functionality.  To them, it would seem like
their MUA DTRTs even for non-munging lists.

?> RFC-compliant MUAs

... don't exist.  RFCs about mail do not apply to MUAs, they apply to
the interpretation of the messages.  An MUA can do anything it wants
to.  It's up to the user to decide whether they like the results or
not.  But the user is ultimately responsible for conformance, not the
MUA.

It's true that if a message composed by a borked MUA (eg Thunderbird
in some of its recent incarnations, or whatever the MUA is that
encourages people to add unneeded Reply-Tos) is interpreted according
to RFC 5322 et al, the result is often surprising to the user.  But
you shouldn't assume that proposed features are going to be added in a
borked way.  (Statistically, of course, some *will* be borked on
introduction.  That just means we need to be ready to fix them!)

Here my main question is whether for *many* users "smart reply" would
"DTRT" enough to streamline their UI and reduce mistaken addressing.
It should be an additional option, not replacing any of the now-
traditional features (reply to author, reply to all, reply to list).
(However, in my experience it completely replaces reply to list.)  If
it's not close enough to what a large number of users want to be the
recommended binding for the "obvious" gestures for "generic" reply, I
don't think the additional variety (complexity) in UI configuration is
worth it.

 > So for the general case where you might have gotten a message
 > directly, and through list A, and through list B, the result is
 > random unless you pay careful attention to how you got this
 > particular copy of the message.

Yes and no (I partly disagree with Mark here).  It's definitely
deterministic, and *not* random, but to users it may seem arbitrary.
This can be mitigated in many cases by list owner coordination and
subscriber setup.  So, for example, a subscriber who sorts different
lists' traffic into different folders is likely to be aware which list
it will go to (the List-Id and List-Post fields will be consistent
throughout that folder).  If it's a group of related lists with
different topics (such as mailman3-users, mailman-users, and
mailman-developers) there's likely to be strong social strictures
against cross-posting.  And with hierarchical structure, the owners
can set things up to DTRT.  Examples:

Announce list: does not set List-Post
    Users list: does set List-Post (cross-posting discouraged)
    Dev list: does set List-Post (cross-posting discouraged)

The announce list being gated to users and devs respectively
"obviously" should only get posts from approved sources.  This DTRTs.

Advisees: does not set List-Post
    Undergrads: does set List-Post (cross-posting discouraged)
    Grads: does set List-Post (cross-posting discouraged)
        Graduates 2018: does not set List-Post
        Graduates 2019: does not set List-Post
        Graduates 2020: does not set List-Post

The "all my advisees" list functions as an announce list (when I'm out
of town, all-hands meetings and university deadlines announcements,
etc).  The undergrads and grads are socially distinct and occasionally
have discussions among themselves.  Rarely the 1st, 2d, 3d year grad
students have discussions specific to that year, but mostly discussion
relates to seminar presentations and general research methods, so it's
not that inconvenient to have no List-Post for individual classes.
It's not perfect DWIM, but it's pretty close.  I don't often see this
as being a problem, and lists that set Reply-To would surely make it
worse.

So overall, I don't see this as likely to be a big problem in practice.

 > It sounds like neither of us want the list to set "Reply-To:
 > <list>".

I think that's pretty general feeling among list management software
developers.  Users and list owners, on the other hand, frequently
disagree.  I believe that is due to a deficiency of MUAs, to wit, not
offering a "smart reply" function and not encouraging its use by less
sophisticated users.

 > You want a "smart reply" button that sends to Reply-To, List-Post, or
 > From, in that order.? (Right?)

I would (and do, where available) use it, but I really want it as an
option, even default, for the crowd of users who expect Reply-To to be
set to the list with current MUAs.

 > I wouldn't use your "smart reply" button, because I think it does the
 > wrong thing for mailing lists,

I don't understand why you think that.  So far you have consistently
responded to this thread on-list AFAICS, and everybody in this thread
got here by reading it on the mailing list (all first responded to a
mailing list post, not to one where they were personally addressed).

I think that is by far the majority case.  It is *quite* unusual for
non-subscribers to be explictly addressed in my experience (see below
for an exception), and (for spam control reasons) it is very common
for lists to make it difficult for non-subscribers to post.  It is not
unreasonable in this day and age to assume that all posters are
subscribers, and that all explicitly named addressees got there
because of "reply all" rather than because somebody added a
non-subscriber.  On this list, and many like it, I personally would
rarely ever need anything other than "smart reply", and it also works
fine for most of my personal mail, which almost always should get a
reply only to author (or Reply-To, which smart reply does).

But I'm interested to hear about use cases where that is *not* the
case.

For example, I participate in one list at work where "smart reply"
would often do the wrong thing, because the subscribers to that list
are fellow faculty and staff who deal with applicants to the grad
program.  I end up splitting between smart-reply (which goes to list
for discussion of committee policy and the like) and reply-all (which
goes to the list so others know I dealt with that case, and to the
non-subscriber to whom I provide information).  However, this decision
*must* be made case-by-case, depending on the nature of the reply:
relying on reply-all (as I do on most list traffic when my MUA doesn't
have "smart reply") would quite frequently do the wrong thing and
require non-list addresses to be edited out.  Even when reply-all is
appropriate, a non-subscriber frequently needs to be added by hand to
coordinate a student-adviser pairing (mostly by the staff, not often
by faculty members).  Reply-author gets used, but quite rarely because
of the nature of committee work.  So even though I can't rely on smart
reply to DWIM, it is nevertheless useful, and fails safe (we rarely
care all that much if any given applicant falls through the cracks,
and most cases where the mail goes to the list but not the applicant
gets caught same day, but internal committee discussion should *not*
go to random non-university people!)

Of course that's just *one* use case.  That's why I ask for others.

 > but if you want to do the wrong thing with your replies, I guess
 > that's up to you.

It's not about *me*.  I have this feature in my own MUA (or will, as
soon as I re-add it to this new upgraded version).  And it does the
*right* thing for me more than 95% of the time.  I still don't
understand what makes you think it would do the wrong thing for *you*,
let alone the wrong thing for *me*, except that you object to it being
labelled "Reply" because you associate that very strongly with "Reply
to Author".

 > My only fear is that in the ongoing simplification (dumbing-down?) 
 > of this stuff, "smart reply" will become the only option.? And,
 > actually, if that happens then I *have* lost the "reply to author"
 > function.

I don't think that level of paranoia is justified.  Sure, some dev
organizations will make that kind of mistake, as we've seen with
Thunderbird.  But all of the MUAs I know that do have specialized
reply-to-list (mutt, Gnus) have very flexible interfaces for binding
UI gestures to functions, and far more available functions than
"one-click" or "one-key" gestures.  If yours doesn't, then yes, you're
at risk that a whim of the developers you could lose essential
functionality.  But that's a problem with your MUA and its dev team,
not with the suggested new functionality.

Steve

From turnbull.stephen.fw at u.tsukuba.ac.jp  Mon Feb  5 03:41:23 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 5 Feb 2018 17:41:23 +0900
Subject: [Mailman-Users] Reply-to options not working
In-Reply-To: <1f5ee0bb-5e6c-e102-623c-2e9610a418f1@spamtrap.tnetconsulting.net>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <87d50fa6-e982-1e79-9a52-e792b5023a78@msapiro.net>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e523c9c5-1473-b6e6-6c22-6a68935f3092@spamtrap.tnetconsulting.net>
 <23150.42437.832863.161429@turnbull.sk.tsukuba.ac.jp>
 <1f5ee0bb-5e6c-e102-623c-2e9610a418f1@spamtrap.tnetconsulting.net>
Message-ID: <23160.6323.693960.332675@turnbull.sk.tsukuba.ac.jp>

Grant Taylor via Mailman-Users writes:

 > Just because an MUA isn't on the Internet, does not mean that it 
 > shouldn't play by the same or very similar rules.

If it doesn't DWIM, I don't use it.  But that's not the same as
talking about conformance of clients.  The question is whether the MUA
*as used by a particular user* usually produces messages that when
interpreted according to the RFCs do what that user expects.  For
example, you could label the reply-author function "Launch Missiles"
and the reply-all function "Buy anti-Trump ad on Breitbart", which
hardly matches the idea of "conformant MUA", but once muscle memory
kicks in it would produce conformant messages that DTRT. :-)

 > I would go so far as to say that this is likely something that
 > should be a user definable configuration.  Which means that MUAs
 > should understand multiple operations and let the end user decide
 > what they want to do.

Sure, WFM.  But in practice the problematic users (remember, this is
inspired as an anti-Reply-To-munging proposal!) do not make decisions
and reconfigure their software; they bitch and moan and expect
everybody else to change.

 > > Where is List-Post a conformance issue?  You add it if you want
 > > to inform people and MUAs where to post, and you don't if you
 > > don't.
 > 
 > I don't think me adding the List-Post header to a message going into a 
 > mailing list will work out very well.

That would be non-conformant to the RFC.  List-* headers should only
be added by list managers (typically software).

 > - I expect that the MLM would munge it (if configured to add the
 >   List-Post header itself)

That's what a conformant MLM would do, yes.

 >   or remove it.

That doesn't work for me.  I have a couple of cases where I have an
umbrella list with dependent lists that are convenient for membership
management (students who get moved by class to alumni lists, see
reply to Jordan for details).  In those cases I want the umbrella
list's List-Post passed through to the dependent lists' distribution.

 > I still believe that user are the root cause of much angst.

Reply-To munging being the salient case.

 > I was saying that I think it's wrong for us to make assumptions
 > about what other people do, and to further turn those assumptions
 > into belief that they will do what we think.

I don't agree with that as a categorical statement.  Of course we
shouldn't just "assume", but rather base it on data of various kinds.
I think there are many cases where we can be sure enough what people
will do that it's worth betting the default setting on it.

 > UI/UX design can help with some, if not many, things.  But the
 > users have to have a fundamental understanding of what they are
 > doing.

Again, I disgree with your wording, at least.  UI/UX design helps if
and only if the designers have a fundamental understanding of what the
users think they are doing.  I think the Thunderbird mistake was to
think that what the users think they're doing is what the developers
think they should be doing.  And that's not the case.  We All Hate
Reply-To Munging, but some users like it.  The idea behind this
proposal is that they don't like munging for its own sake, but because
it allows them to delegate the decision about where to address a reply
to the MUA in a natural way ("natural" according to them).

 > Users may not be willing to design their own UI, but many do choose
 > the UI that they use.  Thus, there is choice involved.

Sure, but in many cases that choice is not informed.  They use what's
there, without understanding, and if the results aren't what they
expect, they want somebody else to fix it.  They don't realize that
they have alternatives that would work better for them, and they
typically are quite unwilling to change to a better UI when they're
told about one (unwilling for good reasons as well as bad ones).

 > > I believe that if this algorithm were used in all major MUAs,
 > > there would be no demand for Reply-To munging.
 > 
 > Maybe, maybe not.

That's not helpful.  I've explained why I think this would work out
well: users who like Reply-To munging like it because they have a
one-button solution to most of their reply-addressing needs.  They
want replies to personal mail to go to author, and replies to list
posts to go to the list.  My proposed algorithm does that (if the list
doesn't munge but does supply List-Post).  It also provides familiar
behavior with Reply-To munging, unlike the Thunderbird mod.  For this
group of users, I believe it could be substituted for the reply-author
function, and they would think it "works better" (because it works as
*they* expect for non-munging lists).  (For you and Jordan, feel free
to change that UI gesture back to reply-author.)

Under what circumstances would this fail for those people?  Do you
have evidence that they actually are rare, and that the demand for
Reply-To munging is based on a different psychology?  Are you really
sure "smart reply" wouldn't work for you in most cases where you
aren't confronted with a "which reply function to use" decision every
few posts on some list?


From mark at msapiro.net  Mon Feb  5 12:55:29 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 5 Feb 2018 09:55:29 -0800
Subject: [Mailman-Users] Message-ID required - was: Reply-to options not
 working
In-Reply-To: <23160.5192.25370.470605@turnbull.sk.tsukuba.ac.jp>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <87d50fa6-e982-1e79-9a52-e792b5023a78@msapiro.net>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e8e71fb4-5690-b1cb-6ea5-38567aca0d14@maileater.net>
 <d4330120-4119-06e7-0b34-f120a44210f5@spamtrap.tnetconsulting.net>
 <3e4a76f4-a68b-39e5-515c-6308654c6665@jordan.maileater.net>
 <3cf69a87-3fbc-2e5b-4fc1-38f60592b166@spamtrap.tnetconsulting.net>
 <d4713ce7-8834-2f03-07ad-4a8b978894a8@jordan.maileater.net>
 <a6511c99-f24b-b880-9247-f94a53eb53a6@spamtrap.tnetconsulting.net>
 <2531c94e-6369-c58c-81ea-000539c27a3a@bmrb.wisc.edu>
 <d32990bd-deeb-2e50-c030-e1b651f49b01@spamtrap.tnetconsulting.net>
 <d1dd087e-4343-a6d8-904f-b2009fed5a33@bmrb.wisc.edu>
 <23160.5192.25370.470605@turnbull.sk.tsukuba.ac.jp>
Message-ID: <ae30290f-955e-fa88-7061-5e9c80645370@msapiro.net>

On 02/05/2018 12:22 AM, Stephen J. Turnbull wrote:
> 
> According to RFC, Message-ID is an originator field, and MUST be
> present and MUST be unique.

Do you have a reference for this? I thought this was correct, but I
recently looked it up in RFC 5322 and predecessors (see
<https://lists.mailman3.org/archives/list/mailman-users at mailman3.org/thread/UCH46WS4RII5HKZWGQTBUDUHZMJZDFNC/>,
and those RFCs at least say it's optional and SHOULD be present.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dmaziuk at bmrb.wisc.edu  Mon Feb  5 13:19:58 2018
From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk)
Date: Mon, 5 Feb 2018 12:19:58 -0600
Subject: [Mailman-Users] Message-ID required - was: Reply-to options not
 working
In-Reply-To: <ae30290f-955e-fa88-7061-5e9c80645370@msapiro.net>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e8e71fb4-5690-b1cb-6ea5-38567aca0d14@maileater.net>
 <d4330120-4119-06e7-0b34-f120a44210f5@spamtrap.tnetconsulting.net>
 <3e4a76f4-a68b-39e5-515c-6308654c6665@jordan.maileater.net>
 <3cf69a87-3fbc-2e5b-4fc1-38f60592b166@spamtrap.tnetconsulting.net>
 <d4713ce7-8834-2f03-07ad-4a8b978894a8@jordan.maileater.net>
 <a6511c99-f24b-b880-9247-f94a53eb53a6@spamtrap.tnetconsulting.net>
 <2531c94e-6369-c58c-81ea-000539c27a3a@bmrb.wisc.edu>
 <d32990bd-deeb-2e50-c030-e1b651f49b01@spamtrap.tnetconsulting.net>
 <d1dd087e-4343-a6d8-904f-b2009fed5a33@bmrb.wisc.edu>
 <23160.5192.25370.470605@turnbull.sk.tsukuba.ac.jp>
 <ae30290f-955e-fa88-7061-5e9c80645370@msapiro.net>
Message-ID: <2f992076-d105-2b77-f8d7-35afdc462bc5@bmrb.wisc.edu>

On 02/05/2018 11:55 AM, Mark Sapiro wrote:
> On 02/05/2018 12:22 AM, Stephen J. Turnbull wrote:
>>
>> According to RFC, Message-ID is an originator field, and MUST be
>> present and MUST be unique.
> 
> Do you have a reference for this? I thought this was correct, but I
> recently looked it up in RFC 5322 and predecessors (see
> <https://lists.mailman3.org/archives/list/mailman-users at mailman3.org/thread/UCH46WS4RII5HKZWGQTBUDUHZMJZDFNC/>,
> and those RFCs at least say it's optional and SHOULD be present.

Heh. I personally believe that a message sent by a mailing list *must*
have the mailing list as the originator: dkim, id, and whatever else.
And then there *should* be a way to reply "off list". Of course then you
have to preserve the original originator all the way to the beginning, so...

And even if "Message-ID MUST be present and MUST be unique", that
doesn't make the converse true: that two copies of the same message
*must* carry the same Message-ID.

-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20180205/167cdc51/attachment.sig>

From mailman at jordan.maileater.net  Mon Feb  5 13:50:42 2018
From: mailman at jordan.maileater.net (Jordan Brown)
Date: Mon, 5 Feb 2018 10:50:42 -0800
Subject: [Mailman-Users] Reply-to options not working
In-Reply-To: <23160.5616.571593.574631@turnbull.sk.tsukuba.ac.jp>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <87d50fa6-e982-1e79-9a52-e792b5023a78@msapiro.net>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e523c9c5-1473-b6e6-6c22-6a68935f3092@spamtrap.tnetconsulting.net>
 <23150.42437.832863.161429@turnbull.sk.tsukuba.ac.jp>
 <734adb86-8722-c9f5-9e3f-ddc22abd99a8@jordan.maileater.net>
 <23152.2290.349851.414385@turnbull.sk.tsukuba.ac.jp>
 <3ca7b4ec-348d-37f7-f6bd-f0a1ea2416be@jordan.maileater.net>
 <23160.5616.571593.574631@turnbull.sk.tsukuba.ac.jp>
Message-ID: <f59297e9-0a36-371a-77ad-da4edd2382af@jordan.maileater.net>

On 2/5/2018 12:29 AM, Stephen J. Turnbull wrote:
> The question I asked, which you misinterpreted completely IMO, and
> Grant partially agreed with is "Does an algorithm which 1. gives
> overriding precedence to Reply-To, 2. otherwise if List-Post is
> present directs it there, and 3. finally falls back to From, seem
> likely to DTRT most of the time?"
You don't mention what your "smart reply" does with To and CC
addresses.? Discards them, I assume?

I suppose it depends on what "most of the time" means, and how often
cross-posting happens, and how often messages to mailing lists include
non-members.

Indeed, most of the time I want to continue the conversation in the same
fora that it's happening in.

But:? in my work contexts, it is quite common for somebody to address a
question to a different team, a team that they are not a member of.? A
"reply" that goes to the List-Post address (versus All) won't do the
right thing, because it won't include the original author.? Normal
"Reply All" does the right thing.

But:? in my work contexts, it is quite common for a discussion to span
two teams.? Again, a "reply" that goes to the List-Post address (versus
All) won't do the right thing.? Normal "Reply All" does the right thing.

But:? It's quite common for a discussion to be between an ad-hoc group
of people on the To/CC lines.? A "reply" that doesn't include To and CC
doesn't do the right thing.? Normal "Reply All" does the right thing.

But:? Even in a mailing list context, I think that "To: <author> CC:
<list>" conveys useful context; I'm replying to what *you* said, and
including everybody else in the audience.? Reply All does the right
thing.? (Yes, it's suboptimal in that the To/CC list tends to accumulate
people over time, but the MUA can't get that right because it doesn't
know who is on the mailing list, ref points above.)

And, finally, it isn't uncommon (probably 5% < x < 20%) for me to want
to reply privately, perhaps to criticize, perhaps to try to resolve a
private disagreement, or perhaps simply to pursue a side thread that
isn't of general interest.? Again, a "reply" that goes to List-Post
(versus From) won't do the right thing and may lead to significant
embarrassment, a risk that in my experience outweighs any possible
advantage.? I do *not* want my "Er, did you really mean to say <stupid
mistake>" note to go to the entire audience.? Normal "Reply" does the
right thing (assuming non-munged Reply-To).

So, net, there are many cases where "smart reply" doesn't do what I
think is the right thing, and none where I think it's appreciably better
than Reply or Reply All, as appropriate.? (If you're interested, I'll
see if I can do an analysis of my message traffic to see how often it
would do something that I would consider to be clearly wrong and how
often it would be an improvement.)


On what might be a side note, I think there might be a key difference in
attitude between different camps.? One side wants to keep discussion on
the mailing list when possible; another wants to keep discussion *off*
the mailing list if it isn't of more or less general interest.? There is
nothing quite so annoying, for instance, as a "me too" flood.? 95% of my
e-mail is work, so every message costs the company money, times the
number of people who have to pay at least enough attention to it to
delete it.? Ten seconds to scan a message, times a thousand people at
$50 to $100 or more per hour, is $140 to $280 or more per message.


>  > So for the general case where you might have gotten a message
>  > directly, and through list A, and through list B, the result is
>  > random unless you pay careful attention to how you got this
>  > particular copy of the message.
>
> Yes and no (I partly disagree with Mark here).  It's definitely
> deterministic, and *not* random, but to users it may seem arbitrary.

It is of course completely deterministic.? But note that I said "unless
you pay careful attention to how you got this particular copy of the
message".

>  > I wouldn't use your "smart reply" button, because I think it does the
>  > wrong thing for mailing lists,
>
> I don't understand why you think that.  So far you have consistently
> responded to this thread on-list AFAICS, and everybody in this thread
> got here by reading it on the mailing list (all first responded to a
> mailing list post, not to one where they were personally addressed).

You don't know about the private conversations :-)

I did have a side conversation with Grant about exactly how I manage my
e-mail addresses (distinct "From" addresses for each mailing list and
each business I deal with).? There were a couple of side comments to Mark.

You also suppose that this style of mailing list dominates my mailing
list usage... it doesn't.? It's easily beaten by my Boy Scout e-mail,
which often goes to both the "parents" and the "Scouts" lists, and at
the moment (for stupid hosting reasons and because of a mailing list
manager with ... suboptimal ... header handling) it's usually going to
two copies of each list.? And *that's* totally dominated by work e-mail.

One might say that different behaviors are appropriate for different
fora, and that wouldn't be totally wrong, but remembering that different
fora will behave differently requires effort, and since Reply/Reply-All
do the right thing in *every* fora, why would I want to spend that
effort (and take the risk of mixing it up)?

>  > My only fear is that in the ongoing simplification (dumbing-down?) 
>  > of this stuff, "smart reply" will become the only option.? And,
>  > actually, if that happens then I *have* lost the "reply to author"
>  > function.
>
> I don't think that level of paranoia is justified.  Sure, some dev
> organizations will make that kind of mistake, as we've seen with
> Thunderbird.  But all of the MUAs I know that do have specialized
> reply-to-list (mutt, Gnus) have very flexible interfaces for binding
> UI gestures to functions, and far more available functions than
> "one-click" or "one-key" gestures.  If yours doesn't, then yes, you're
> at risk that a whim of the developers you could lose essential
> functionality.  But that's a problem with your MUA and its dev team,
> not with the suggested new functionality.

I do 99%+ of my e-mail with T-bird on a Windows system, but there's
still that <1% that's done with the Mail app on my iPad, which is the
opposite end of the flexibility spectrum.? That's the end that concerns
me.? And even T-bird is not immune to the "remove features to simplify
things" disease.


From turnbull.stephen.fw at u.tsukuba.ac.jp  Tue Feb  6 05:05:28 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Tue, 6 Feb 2018 19:05:28 +0900
Subject: [Mailman-Users] Message-ID required
In-Reply-To: <ae30290f-955e-fa88-7061-5e9c80645370@msapiro.net>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <87d50fa6-e982-1e79-9a52-e792b5023a78@msapiro.net>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e8e71fb4-5690-b1cb-6ea5-38567aca0d14@maileater.net>
 <d4330120-4119-06e7-0b34-f120a44210f5@spamtrap.tnetconsulting.net>
 <3e4a76f4-a68b-39e5-515c-6308654c6665@jordan.maileater.net>
 <3cf69a87-3fbc-2e5b-4fc1-38f60592b166@spamtrap.tnetconsulting.net>
 <d4713ce7-8834-2f03-07ad-4a8b978894a8@jordan.maileater.net>
 <a6511c99-f24b-b880-9247-f94a53eb53a6@spamtrap.tnetconsulting.net>
 <2531c94e-6369-c58c-81ea-000539c27a3a@bmrb.wisc.edu>
 <d32990bd-deeb-2e50-c030-e1b651f49b01@spamtrap.tnetconsulting.net>
 <d1dd087e-4343-a6d8-904f-b2009fed5a33@bmrb.wisc.edu>
 <23160.5192.25370.470605@turnbull.sk.tsukuba.ac.jp>
 <ae30290f-955e-fa88-7061-5e9c80645370@msapiro.net>
Message-ID: <23161.32232.416992.270593@turnbull.sk.tsukuba.ac.jp>

Mark Sapiro writes:
 > On 02/05/2018 12:22 AM, Stephen J. Turnbull wrote:
 > > 
 > > According to RFC, Message-ID is an originator field, and MUST be
 > > present and MUST be unique.
 > 
 > Do you have a reference for this? I thought this was correct, but I
 > recently looked it up in RFC 5322 and predecessors and those RFCs
 > at least say it's optional and SHOULD be present.

You're right.  I knew that it was SHOULD in RFC 822, but I thought
this was updated in RFC 1123 "Host Requirements" or maybe RFC 5598
"Email Architecture".  I was wrong.

In any case, SHOULD is pretty close to MUST, especially in this case.
(What reasons based on interoperability issues can you think of for
omitting Message-ID?  SHOULD means you need one!)

I'm guessing that since the RFC authors have deprecated use of
Message-ID for anything related to security, and its semantics are a
judgment call in any case, it's simply not reliable enough to promote
to MUST.  So they never did.

Steve


From turnbull.stephen.fw at u.tsukuba.ac.jp  Tue Feb  6 05:09:42 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Tue, 6 Feb 2018 19:09:42 +0900
Subject: [Mailman-Users] Message-ID required - was: Reply-to options not
 working
In-Reply-To: <2f992076-d105-2b77-f8d7-35afdc462bc5@bmrb.wisc.edu>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e8e71fb4-5690-b1cb-6ea5-38567aca0d14@maileater.net>
 <d4330120-4119-06e7-0b34-f120a44210f5@spamtrap.tnetconsulting.net>
 <3e4a76f4-a68b-39e5-515c-6308654c6665@jordan.maileater.net>
 <3cf69a87-3fbc-2e5b-4fc1-38f60592b166@spamtrap.tnetconsulting.net>
 <d4713ce7-8834-2f03-07ad-4a8b978894a8@jordan.maileater.net>
 <a6511c99-f24b-b880-9247-f94a53eb53a6@spamtrap.tnetconsulting.net>
 <2531c94e-6369-c58c-81ea-000539c27a3a@bmrb.wisc.edu>
 <d32990bd-deeb-2e50-c030-e1b651f49b01@spamtrap.tnetconsulting.net>
 <d1dd087e-4343-a6d8-904f-b2009fed5a33@bmrb.wisc.edu>
 <23160.5192.25370.470605@turnbull.sk.tsukuba.ac.jp>
 <ae30290f-955e-fa88-7061-5e9c80645370@msapiro.net>
 <2f992076-d105-2b77-f8d7-35afdc462bc5@bmrb.wisc.edu>
Message-ID: <23161.32486.983014.578673@turnbull.sk.tsukuba.ac.jp>

Dimitri Maziuk writes:

 > Heh. I personally believe that a message sent by a mailing list
 > *must* have the mailing list as the originator: dkim, id, and
 > whatever else.

First, please be careful with terminology.  *Originator* is
well-defined (RFC 5598) as the agent of the Author that first injects
the message into the mail system.  When mailing lists distribute posts
to subscribers, they function as *Mediators* (RFC 5598).

Even if used loosely, I see no reason here to think of mailing lists
as "originators".  DKIM explicitly provides that multiple signatures
may be present, whether from the same host or different hosts.  For
message identification, Mediators are encouraged to use
Resent-Message-ID and other Resent-* fields to provide trace
information in addition to the MTA's Received fields.  Such features
are available to any agent in the mail system, not restricted to
Authors and Originators.

 > And even if "Message-ID MUST be present and MUST be unique", that
 > doesn't make the converse true: that two copies of the same message
 > *must* carry the same Message-ID.

I don't understand your point.  The RFCs make clear that in the case
of certain trivial modifications (adding trace fields to the header,
for example), the Message-ID SHOULD be preserved.  Further, Mark has
described when Mailman will alter the Message-ID, and I described some
of the cases where people disagree about whether to alter it.

However, for stock Mailman, I think that pretty much everybody who
cares about Message-ID agrees that given the kinds of changes Mailman
makes to messages, it should only change Message-ID in an overriding
case such as preserving privacy on a list that purports to anonymize
posts, or where it interferes with interoperability.  Otherwise you
interfere with features such as local duplicate suppression,
threading, and archiving that depend on stability of Message-ID.

Do you have something to add to that, or disagree with that?

Steve

From turnbull.stephen.fw at u.tsukuba.ac.jp  Tue Feb  6 05:09:56 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Tue, 6 Feb 2018 19:09:56 +0900
Subject: [Mailman-Users] Reply-to options not working
In-Reply-To: <f59297e9-0a36-371a-77ad-da4edd2382af@jordan.maileater.net>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <87d50fa6-e982-1e79-9a52-e792b5023a78@msapiro.net>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e523c9c5-1473-b6e6-6c22-6a68935f3092@spamtrap.tnetconsulting.net>
 <23150.42437.832863.161429@turnbull.sk.tsukuba.ac.jp>
 <734adb86-8722-c9f5-9e3f-ddc22abd99a8@jordan.maileater.net>
 <23152.2290.349851.414385@turnbull.sk.tsukuba.ac.jp>
 <3ca7b4ec-348d-37f7-f6bd-f0a1ea2416be@jordan.maileater.net>
 <23160.5616.571593.574631@turnbull.sk.tsukuba.ac.jp>
 <f59297e9-0a36-371a-77ad-da4edd2382af@jordan.maileater.net>
Message-ID: <23161.32500.711539.973022@turnbull.sk.tsukuba.ac.jp>

Jordan Brown writes:
 > On 2/5/2018 12:29 AM, Stephen J. Turnbull wrote:
 > [various stuff, citation line preserved to make a point below]

 > You don't mention what your "smart reply" does with To and CC
 > addresses.? Discards them, I assume?

Yes.  It's intended to do what a certain large group of naive users
expect "Reply" to do.  "Smart Reply" is not intended to do what "Reply
All" does, period.  It is intended to provide a variant on what
"Reply" does that automatically handles the common case (on the
Internet at large) of discussion lists where all participants are
subscribers.  It always picks an unique reply address (except in some
rare cases where From or Reply-To contains multiple addresses).

 > But:? in my work contexts, it is quite common for a discussion to span
 > two teams.? Again, a "reply" that goes to the List-Post address (versus
 > All) won't do the right thing.? Normal "Reply All" does the right
 > thing.

OK.  I'm assuming that each team has its own list, only one List-Post
is present, so you need Reply-All even for list posts, right?  I
almost never need to reply to such posts, and when I do, it's
invariably only to the list I received it from.  If reply-both is a
frequent use case on your lists, I can see how smart reply would
almost never DTRT for you.

 > But:? Even in a mailing list context, I think that "To: <author> CC:
 > <list>" conveys useful context; I'm replying to what *you* said, and
 > including everybody else in the audience.? Reply All does the right
 > thing.? (Yes, it's suboptimal in that the To/CC list tends to accumulate
 > people over time [...].)

I find the suboptimality aspect more important, and determine who was
replied to more from the citation line than the addressee list.  YMMV,
of course.

As for "knowing who's on the list", almost all of the lists I'm on
require membership to post, including "private" lists (address books
do get pwned), and deliberate addition of 3rd parties is *extremely*
rare, except for certain announcements.  But in that announcement
use-case, for me Reply-Author is the only mode I ever use.

 > I do *not* want my "Er, did you really mean to say <stupid
 > mistake>" note to go to the entire audience.

This turns out not to be a problem for me.  Smart reply has never
tricked me into addressing a private reply to the list (let alone
actually sending one).  I suspect it's unlikely to catch my main
audience for the feature very often -- and much less often than
Reply-To munging does.  But to find out I need to get it into a mass
MUA as default. :-/

?> Normal "Reply" does the right thing (assuming non-munged
?> Reply-To).

Reply-To munging is precisely the issue this is intended to address.
Munged lists *have* caught me (although actually sending a message
misaddressed to list is extremely rare).  I think the difference is
that when I use "smart reply" I have implicitly requested that it go
to the list.  If I really want to reply to author (which is not that
rare), I do use Reply-Author, and find it natural.  (I'm not saying
you would.)

 > than Reply or Reply All, as appropriate.? (If you're interested, I'll
 > see if I can do an analysis of my message traffic to see how often it
 > would do something that I would consider to be clearly wrong and how
 > often it would be an improvement.)

I would be interested in that.  I expect that you'll find a pretty
high ratio of wrong to right.  But if it came out anywhere near even,
it would be a pretty strong indication in favor of writing an RFC.  I
don't expect that to be enough to interest you in changing (there
would be muscle memory costs, etc).

I would appreciate it if you would *not* count "omitting the author of
a list post from the reply" as "wrong" for this purpose because I don't
think my target audience for "smart reply" would count it as wrong.

 > On what might be a side note, I think there might be a key
 > difference in attitude between different camps.? One side wants to
 > keep discussion on the mailing list when possible;

This feature is definitely aimed at that camp.  I'm not interested in
discussing whether encouraging them is a bad thing in this thread.  If
you want to talk about that (it does matter to me, it's just a
separate discussion), let's start a new thread or we can go offline.

 > It is of course completely deterministic.? But note that I said
 > "unless you pay careful attention to how you got this particular
 > copy of the message".

I disagree with that.  In your case, where you apparently get a lot of
cross-posts, yes, you'd need to pay attention.  But I think you'd
figure it out quickly because AIUI you'd normally be on your team's
list but not the other.  Your operational problem wouldn't be figuring
out which list delivered it, but simply that you frequently need to
respond to both, and the list you're not on would be omitted unless
you Reply-All.  Am I missing something?  "Smart reply" is simply not
designed to be useful in such cases.

However, in many cases, such as the two I described, there is not a
big problem unless I'm on both lists that provide List-Post, which is
relatively rare in my experience.  Even then, I don't recall wanting
to cross-post a reply more than once or twice.  This does depend on
the fact that I sort each list into a different folder on List-ID or
List-Post, so folder context means my expectations are correct.  Not
sure how this would play out in my target audience.

 > You don't know about the private conversations :-)

No, of course not.  I do know that you've been active on the thread,
and if you've been sending private replies, you had to take a
different action to do that from the one you use to reply on-thread.
So it's not obvious *why* that couldn't be Reply-Author vs Smart-Reply
instead of Reply-Author vs. Reply-All until we start breaking down the
various use cases.

 > You also suppose that this style of mailing list dominates my mailing
 > list usage... it doesn't.

I don't suppose any such thing.  I simply don't know about it, which
is why I ask.  I'm not sure your use cases are relevant to my target
audience, but they could be.  (Eg, I hadn't thought about the Boy
Scouts type of case, although I'm not sure it's really different from
my advisees case.)

 > One might say that different behaviors are appropriate for
 > different fora, and that wouldn't be totally wrong, but remembering
 > that different fora will behave differently requires effort, and
 > since Reply/Reply-All do the right thing in *every* fora, why would
 > I want to spend that effort (and take the risk of mixing it up)?

Well, I did it because I'm (intermittently) on a crusade to eliminate
Reply-To munging.  (Just so you know there is *some* method to this
madness.)  I realize that's a very specialized motivation. ;-)

I also disagree that Reply-All does the right thing in the subscribe-
to-post discussion lists I participate in.  Sure, I can go back and
edit out all but the person I'm replying to, but even I don't always
do that, and most people *never* do.  YMMV, of course.

 > I do 99%+ of my e-mail with T-bird on a Windows system, but there's
 > still that <1% that's done with the Mail app on my iPad, which is
 > the opposite end of the flexibility spectrum.? That's the end that
 > concerns me.

I don't see them adopting this in lieu of Reply-Author, though I could
be wrong.

?> And even T-bird is not immune to the "remove features to simplify
?> things" disease.

To be frank, I was a little shocked by the T-bird story.  "What *were*
they thinking?!"

Steve


From Hagedorn at uni-koeln.de  Tue Feb  6 06:48:22 2018
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Tue, 06 Feb 2018 12:48:22 +0100
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
 <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
 <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>
 <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>
Message-ID: <ACD080052B3BD0E023E309A8@tyrion.rrz.uni-koeln.de>

--On 3. Februar 2018 um 19:13:33 -0800 Mark Sapiro <mark at msapiro.net> wrote:

> On 02/03/2018 01:03 AM, Sebastian Hagedorn wrote:
>>>
>>> Did you look at the out queue, and if so was there a .bak file there.
>>> This would be the entry currently being processed.
>>
>> I looked at the out queue, and there was no .bak file.
>
>
> Interesting. That says that OutgoingRunner is not currently delivering a
> message, but that is inconsistent with this:
>
>
>>> Also, the TCP connection to the MTA being ESTABLISHED says the
>>> OutgoingRunner has called SMTPDirect.process() and it in turn is
>>> somewhere in its delivery loop of sending SMTP transactions.

Is it possible that the OutgoingRunner was done with transmitting the 
message and had already removed the queue file, but that the connection 
hadn't yet been closed?

>>> Are there any clues in the MTA logs?
>>
>> I just found this in Mailman's smtp-failures log:
>>
>> Feb 01 14:28:49 2018 (1674) Low level smtp error: [Errno 111] Connection
>> refused, msgid:
>> <B51BA08829F27146A07699F58B941234A27397BF at EX10DAG2.intern.xxx>
>> Feb 01 14:28:49 2018 (1674) delivery to xxx at uni-koeln.de failed with
>> code -1: [Errno 111] Connection refused
>>
> Normally, that won't cause a problem like this. This occurs at a fairly
> low level in SMTPDirect.py when Mailman is initiating a transaction with
> the MTA to send to one or more recipients. The recipients will be marked
> as "refused retryably" and OutgoingRunner will queue the message for
> those recipients. in the retry queue to be retried
>
> You can set SMTPLIB_DEBUG_LEVEL = 1 in mm_cfg.py to log copious smtplib
> debugging info to Mailman's error log. Then the log will show the last
> thing that was done before the hang.

The problem with that is that we run 3,200 lists on that server. Not all of 
them are high-volume, but I'm worried that our log files would explode. Ich 
just checked and yesterday we sent mails to 50,000 recipients.

>> If this should happen again, what should we look for? Would a gdb
>> backtrace be helpful?
>
> It might be if you can find just where in the code it's hung. Also, I
> didn't look carefully before, but in your OP, you show
>
>> mailman   1663  0.0  0.0 233860  2204 ?        Ss   Jan16   0:00
>> /usr/bin/python2.7 /usr/lib/mailman/bin/mailmanctl -s -q start
>> mailman   1677  0.1  0.9 295064 73284 ?        S    Jan16  35:35
>> /usr/bin/python2.7 /usr/lib/mailman/bin/qrunner
>> --runner=OutgoingRunner:3:4  -s
>
> The status of 'S' for OutgoingRunner is "uninterruptable sleep". This
> means it's either called time.sleep for QRUNNER_SLEEP_TIME (default = 1
> second) which is unlikely as it should wake up, or it's waiting for
> response from something, most likely a response from the MTA.

Now I wonder the MTA had already closed the connection and Mailman for some 
reason didn't notice. Because the Runner was stuck longer than any timeout 
on the MTA would permit. But I failed to check that. Should it hapen again 
I will have a look on the MTA end.
-- 
    .:.Sebastian Hagedorn - Weyertal 121 (Geb?ude 133), Zimmer 2.02.:.
                 .:.Regionales Rechenzentrum (RRZK).:.
   .:.Universit?t zu K?ln / Cologne University - ? +49-221-470-89578.:.

From Hagedorn at uni-koeln.de  Tue Feb  6 06:51:50 2018
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Tue, 06 Feb 2018 12:51:50 +0100
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <35b995d2-f45e-2e07-fbbc-aa8c69db38cc@poem.co.jp>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
 <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
 <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>
 <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>
 <35b995d2-f45e-2e07-fbbc-aa8c69db38cc@poem.co.jp>
Message-ID: <F7642C910711C73C4333C953@tyrion.rrz.uni-koeln.de>


--On 4. Februar 2018 um 12:54:43 +0900 Yasuhito FUTATSUKI 
<futatuki at poem.co.jp> wrote:

> On 02/04/18 12:13, Mark Sapiro wrote:
>> The status of 'S' for OutgoingRunner is "uninterruptable sleep". This
>> means it's either called time.sleep for QRUNNER_SLEEP_TIME (default = 1
>> second) which is unlikely as it should wake up, or it's waiting for
>> response from something, most likely a response from the MTA.
>
> As far as I read the code, if OutgoingRunner catch SIGINT during waiting
> for response from the MTA, the signal handler for SIGINT in qrunner set
> flag to exit from loop, then socket module raise socket.error for EINTR,
> but SMTP module retry to read from socket and waiting for response until
> receiving response or connection closing (from MTA side or by error).
> Thus it cannot reach to the code to exit if the connection is kept alive
> and MTA send no data.

Thanks. I think that might be a possible explanation, but what could cause 
a SIGINT to be sent to the OutgoingRunner?
-- 
    .:.Sebastian Hagedorn - Weyertal 121 (Geb?ude 133), Zimmer 2.02.:.
                 .:.Regionales Rechenzentrum (RRZK).:.
   .:.Universit?t zu K?ln / Cologne University - ? +49-221-470-89578.:.

From dmaziuk at bmrb.wisc.edu  Tue Feb  6 10:43:37 2018
From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk)
Date: Tue, 6 Feb 2018 09:43:37 -0600
Subject: [Mailman-Users] Message-ID required - was: Reply-to options not
 working
In-Reply-To: <23161.32486.983014.578673@turnbull.sk.tsukuba.ac.jp>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e8e71fb4-5690-b1cb-6ea5-38567aca0d14@maileater.net>
 <d4330120-4119-06e7-0b34-f120a44210f5@spamtrap.tnetconsulting.net>
 <3e4a76f4-a68b-39e5-515c-6308654c6665@jordan.maileater.net>
 <3cf69a87-3fbc-2e5b-4fc1-38f60592b166@spamtrap.tnetconsulting.net>
 <d4713ce7-8834-2f03-07ad-4a8b978894a8@jordan.maileater.net>
 <a6511c99-f24b-b880-9247-f94a53eb53a6@spamtrap.tnetconsulting.net>
 <2531c94e-6369-c58c-81ea-000539c27a3a@bmrb.wisc.edu>
 <d32990bd-deeb-2e50-c030-e1b651f49b01@spamtrap.tnetconsulting.net>
 <d1dd087e-4343-a6d8-904f-b2009fed5a33@bmrb.wisc.edu>
 <23160.5192.25370.470605@turnbull.sk.tsukuba.ac.jp>
 <ae30290f-955e-fa88-7061-5e9c80645370@msapiro.net>
 <2f992076-d105-2b77-f8d7-35afdc462bc5@bmrb.wisc.edu>
 <23161.32486.983014.578673@turnbull.sk.tsukuba.ac.jp>
Message-ID: <6bff0343-831c-892d-e456-d7bb3f6a28cb@bmrb.wisc.edu>

On 2018-02-06 04:09, Stephen J. Turnbull wrote:

> Do you have something to add to that, or disagree with that?

I said it was what *I* believe, not what IETF believes.

Dima



From mark at msapiro.net  Tue Feb  6 11:01:18 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 6 Feb 2018 08:01:18 -0800
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <F7642C910711C73C4333C953@tyrion.rrz.uni-koeln.de>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
 <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
 <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>
 <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>
 <35b995d2-f45e-2e07-fbbc-aa8c69db38cc@poem.co.jp>
 <F7642C910711C73C4333C953@tyrion.rrz.uni-koeln.de>
Message-ID: <54fb76d1-6113-68f7-7a0d-67d197ecdfbf@msapiro.net>

On 02/06/2018 03:51 AM, Sebastian Hagedorn wrote:
> 
> --On 4. Februar 2018 um 12:54:43 +0900 Yasuhito FUTATSUKI
> <futatuki at poem.co.jp> wrote:
>>
>> As far as I read the code, if OutgoingRunner catch SIGINT during waiting
>> for response from the MTA, the signal handler for SIGINT in qrunner set
>> flag to exit from loop, then socket module raise socket.error for EINTR,
>> but SMTP module retry to read from socket and waiting for response until
>> receiving response or connection closing (from MTA side or by error).
>> Thus it cannot reach to the code to exit if the connection is kept alive
>> and MTA send no data.
> 
> Thanks. I think that might be a possible explanation, but what could
> cause a SIGINT to be sent to the OutgoingRunner?


The above is an explanation of why the runner doesn't exit when it
receives a SIGINT or SIGTERM from the master when you restart or stop
Mailman and why you have to SIGKILL it. It suggests that what's
happening when it's hung is it's waiting for a response from the MTA.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From Hagedorn at uni-koeln.de  Tue Feb  6 11:09:22 2018
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Tue, 06 Feb 2018 17:09:22 +0100
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <54fb76d1-6113-68f7-7a0d-67d197ecdfbf@msapiro.net>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
 <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
 <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>
 <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>
 <35b995d2-f45e-2e07-fbbc-aa8c69db38cc@poem.co.jp>
 <F7642C910711C73C4333C953@tyrion.rrz.uni-koeln.de>
 <54fb76d1-6113-68f7-7a0d-67d197ecdfbf@msapiro.net>
Message-ID: <EB47826BAAEEC87EEB95B29D@tyrion.rrz.uni-koeln.de>


--On 6. Februar 2018 um 08:01:18 -0800 Mark Sapiro <mark at msapiro.net> wrote:

> On 02/06/2018 03:51 AM, Sebastian Hagedorn wrote:
>>
>> --On 4. Februar 2018 um 12:54:43 +0900 Yasuhito FUTATSUKI
>> <futatuki at poem.co.jp> wrote:
>>>
>>> As far as I read the code, if OutgoingRunner catch SIGINT during waiting
>>> for response from the MTA, the signal handler for SIGINT in qrunner set
>>> flag to exit from loop, then socket module raise socket.error for EINTR,
>>> but SMTP module retry to read from socket and waiting for response until
>>> receiving response or connection closing (from MTA side or by error).
>>> Thus it cannot reach to the code to exit if the connection is kept alive
>>> and MTA send no data.
>>
>> Thanks. I think that might be a possible explanation, but what could
>> cause a SIGINT to be sent to the OutgoingRunner?
>
>
> The above is an explanation of why the runner doesn't exit when it
> receives a SIGINT or SIGTERM from the master when you restart or stop
> Mailman and why you have to SIGKILL it. It suggests that what's
> happening when it's hung is it's waiting for a response from the MTA.

Ah OK, I misunderstood that part.
-- 
    .:.Sebastian Hagedorn - Weyertal 121 (Geb?ude 133), Zimmer 2.02.:.
                 .:.Regionales Rechenzentrum (RRZK).:.
   .:.Universit?t zu K?ln / Cologne University - ? +49-221-470-89578.:.

From mark at msapiro.net  Tue Feb  6 11:18:37 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 6 Feb 2018 08:18:37 -0800
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <ACD080052B3BD0E023E309A8@tyrion.rrz.uni-koeln.de>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
 <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
 <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>
 <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>
 <ACD080052B3BD0E023E309A8@tyrion.rrz.uni-koeln.de>
Message-ID: <d79719c6-2fc5-388b-1811-fd8c7e6f03ca@msapiro.net>

On 02/06/2018 03:48 AM, Sebastian Hagedorn wrote:
> 
> Is it possible that the OutgoingRunner was done with transmitting the
> message and had already removed the queue file, but that the connection
> hadn't yet been closed?


Only if something went very wrong in SMTPDirect.process() which would
have had to return to OutgoingRunner before the .bak would be removed.

As I read the code in SMTPDirect.process(), delivery is in a try: ...
finally: and the connection is closed in the finally: clause.


> Now I wonder the MTA had already closed the connection and Mailman for
> some reason didn't notice. Because the Runner was stuck longer than any
> timeout on the MTA would permit. But I failed to check that. Should it
> hapen again I will have a look on the MTA end.


You should be able to see the connect and all that follows in the MTA
logs to see if the MTA closed the connection.

Anyway, if it did do that while the runner was waiting for response,
that should raise socket.error which would be caught and handled.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mailman at jordan.maileater.net  Tue Feb  6 14:02:02 2018
From: mailman at jordan.maileater.net (Jordan Brown)
Date: Tue, 6 Feb 2018 11:02:02 -0800
Subject: [Mailman-Users] Reply-to options not working
In-Reply-To: <23161.32500.711539.973022@turnbull.sk.tsukuba.ac.jp>
References: <d2c9c0c2-90a0-9edb-9cf3-e149e6070ca9@yahoo.no>
 <87d50fa6-e982-1e79-9a52-e792b5023a78@msapiro.net>
 <6e18c204-f49b-1d5f-225c-b4b26987c857@spamtrap.tnetconsulting.net>
 <feeaccbf-efd6-4967-22b5-d226b27007f4@msapiro.net>
 <3e4be682-50d6-ee73-a0f5-164d11707e94@spamtrap.tnetconsulting.net>
 <23144.18672.491811.845571@turnbull.sk.tsukuba.ac.jp>
 <e523c9c5-1473-b6e6-6c22-6a68935f3092@spamtrap.tnetconsulting.net>
 <23150.42437.832863.161429@turnbull.sk.tsukuba.ac.jp>
 <734adb86-8722-c9f5-9e3f-ddc22abd99a8@jordan.maileater.net>
 <23152.2290.349851.414385@turnbull.sk.tsukuba.ac.jp>
 <3ca7b4ec-348d-37f7-f6bd-f0a1ea2416be@jordan.maileater.net>
 <23160.5616.571593.574631@turnbull.sk.tsukuba.ac.jp>
 <f59297e9-0a36-371a-77ad-da4edd2382af@jordan.maileater.net>
 <23161.32500.711539.973022@turnbull.sk.tsukuba.ac.jp>
Message-ID: <a2174af6-ae52-784d-e6e3-dcca8265a90d@jordan.maileater.net>

[ This was getting pretty long and a bit repetitive, so I trimmed it
brutally.? It's still pretty long, sigh. ]

On 2/6/2018 2:09 AM, Stephen J. Turnbull wrote:
>  > But:? in my work contexts, it is quite common for a discussion to span
>  > two teams.? Again, a "reply" that goes to the List-Post address (versus
>  > All) won't do the right thing.? Normal "Reply All" does the right
>  > thing.
>
> OK.  I'm assuming that each team has its own list, only one List-Post
> is present, so you need Reply-All even for list posts, right?

Our mailing list software doesn't add List-Post, so yes, no other
variation does anything like the right thing.

For discussion purposes, I'm assuming that you would consider that to be
a misconfigured mailing list, and so I'm discussing how things would
work if it *did* include List-Post.? I shudder to imagine a world where
both kinds of mailing lists (with and without List-Post) are considered
correct, and you'd have to know which kind of mailing list each was to
know how your Reply button would work.

> ?> Normal "Reply" does the right thing (assuming non-munged
> ?> Reply-To).
>
> Reply-To munging is precisely the issue this is intended to address.
> Munged lists *have* caught me (although actually sending a message
> misaddressed to list is extremely rare).  I think the difference is
> that when I use "smart reply" I have implicitly requested that it go
> to the list.  If I really want to reply to author (which is not that
> rare), I do use Reply-Author, and find it natural.  (I'm not saying
> you would.)

I think I might finally understand some of the disconnect.

When you say "smart reply", what I hear is that it's a replacement for
the Reply button.? If it's a replacement for the Reply button, the
button you use to reply just to the author in all *other* contexts, then
it will naturally lead you into sending your private message to the world.

But it seems that you're really intending it as a replacement for the
Reply All button, a multicast reply that tries to figure out what the
exactly right address is to reply to.

Do you just never have three-way conversations with specific people?? Or
do you have to mentally split replies into three kinds:? just back to
the author, to a mailing list, or to an ad-hoc group?

My mental rule is really simple:? if I want to reply to the author, I
hit Reply; if I want to reply to everybody in the conversation I hit
Reply All.? Every once in a while I need to spin off a subset or add
somebody, and then I do one of the above and edit the list.

Do you have all three buttons (Reply, Smart-Reply, Reply-All)?

If you have a message from Joe, To you, CC Sam, and you want to reply to
both Joe and Sam, what button do you use?? If you just want to reply to
Joe, what button do you use?

>  > than Reply or Reply All, as appropriate.? (If you're interested, I'll
>  > see if I can do an analysis of my message traffic to see how often it
>  > would do something that I would consider to be clearly wrong and how
>  > often it would be an improvement.)
>
> I would be interested in that.  I expect that you'll find a pretty
> high ratio of wrong to right.  But if it came out anywhere near even,
> it would be a pretty strong indication in favor of writing an RFC.  I
> don't expect that to be enough to interest you in changing (there
> would be muscle memory costs, etc).
>
> I would appreciate it if you would *not* count "omitting the author of
> a list post from the reply" as "wrong" for this purpose because I don't
> think my target audience for "smart reply" would count it as wrong.

I'll see what I can do.? The hard part will be determining whether
people on the To/CC list are on the mailing list.
[ After an experiment... ] Yeah, the SMTP server doesn't implement EXPN,
making that hard to automate.? Still, I'll see what I can do by hand.

>  > One might say that different behaviors are appropriate for
>  > different fora, and that wouldn't be totally wrong, but remembering
>  > that different fora will behave differently requires effort, and
>  > since Reply/Reply-All do the right thing in *every* fora, why would
>  > I want to spend that effort (and take the risk of mixing it up)?
>
> Well, I did it because I'm (intermittently) on a crusade to eliminate
> Reply-To munging.  (Just so you know there is *some* method to this
> madness.)  I realize that's a very specialized motivation. ;-)

Oh, I'm on a crusade to eliminate Reply-To munging too.? I'm just
nervous about doing it by pushing a UI idiom that has a very similar
effect, especially spinning it as the "does what you really want" answer.

> I also disagree that Reply-All does the right thing in the subscribe-
> to-post discussion lists I participate in.  Sure, I can go back and
> edit out all but the person I'm replying to, but even I don't always
> do that, and most people *never* do.  YMMV, of course.

And the harm is that people get duplicate copies of messages in threads
they've participated in.? Seems pretty minimal.? Nobody got dropped from
the conversation, and nobody's intended-to-be-private message got
broadcast to the world, and those seem like much more serious failures.

> ?> And even T-bird is not immune to the "remove features to simplify
> ?> things" disease.
>
> To be frank, I was a little shocked by the T-bird story.  "What *were*
> they thinking?!"

Which T-bird story are we talking about?

The "automatically ignore Reply-To if it looks like it's the result of
Reply-To munging" feature?? It's awful and it's ugly and ... it seems to
do exactly what I want, in an imperfect world where people configure
their mailing lists to try to trick me into embarrassing myself.

Here's an interesting tidbit.? I don't know why it's happening, but the
*only* copy I get of your messages is the one that went directly, so it
doesn't have List-Post.? (Maybe this is the MailMan no-dups feature in
action.)? So it seems like your "Smart Reply" wouldn't work right on
this mailing list, at least as it arrives at my mailbox.

From futatuki at poem.co.jp  Tue Feb  6 22:43:18 2018
From: futatuki at poem.co.jp (Yasuhito FUTATSUKI)
Date: Wed, 7 Feb 2018 12:43:18 +0900
Subject: [Mailman-Users] Stuck OutgoingRunner
In-Reply-To: <54fb76d1-6113-68f7-7a0d-67d197ecdfbf@msapiro.net>
References: <B049AB3977BFF7B47FF517D0@tyrion.rrz.uni-koeln.de>
 <1003ca48-6031-1c00-8a5e-f68b5ac99846@msapiro.net>
 <7D74F3692CB8DF4C4656EE54@Sebbis-iMac.local>
 <f02a8448-a249-f973-e5fc-f4394c2df122@msapiro.net>
 <35b995d2-f45e-2e07-fbbc-aa8c69db38cc@poem.co.jp>
 <F7642C910711C73C4333C953@tyrion.rrz.uni-koeln.de>
 <54fb76d1-6113-68f7-7a0d-67d197ecdfbf@msapiro.net>
Message-ID: <986cd340-53fc-f19f-96e3-3d7c76139287@poem.co.jp>

On 02/07/18 01:01, Mark Sapiro wrote:
> On 02/06/2018 03:51 AM, Sebastian Hagedorn wrote:
>>
>> --On 4. Februar 2018 um 12:54:43 +0900 Yasuhito FUTATSUKI
>> <futatuki at poem.co.jp> wrote:
>>>
>>> As far as I read the code, if OutgoingRunner catch SIGINT during waiting
>>> for response from the MTA, the signal handler for SIGINT in qrunner set
>>> flag to exit from loop, then socket module raise socket.error for EINTR,
>>> but SMTP module retry to read from socket and waiting for response until
>>> receiving response or connection closing (from MTA side or by error).
>>> Thus it cannot reach to the code to exit if the connection is kept alive
>>> and MTA send no data.

I'm sorry, above is partly wrong, it is not smtplib.SMTP object to continue
reading but socket module itself.(on Python 2.7.14, socket._fileobject.readline())
But it does not affect main subject.

>> Thanks. I think that might be a possible explanation, but what could
>> cause a SIGINT to be sent to the OutgoingRunner?
> 
> 
> The above is an explanation of why the runner doesn't exit when it
> receives a SIGINT or SIGTERM from the master when you restart or stop
> Mailman and why you have to SIGKILL it. It suggests that what's
> happening when it's hung is it's waiting for a response from the MTA.

thanks to explain for my intension.

In fact,

On 02/02/18 19:26, Sebastian Hagedorn wrote:
> root at mailman3/usr/lib/mailman/bin]$ strace -p 1677
> Process 1677 attached
> recvfrom(10, ^CProcess 1677 detached

indicates the OutGoingRunner process 1677 was still in recvfrom(2)
system call (perhaps called from recv(2)) for FD 10, and

> [root at mailman3/usr/lib/mailman/bin]$ lsof -p 1677
> COMMAND    PID    USER   FD   TYPE   DEVICE SIZE/OFF   NODE NAME
> python2.7 1677 mailman  cwd    DIR    253,0     4096 173998 /usr/lib/mailman
> python2.7 1677 mailman  rtd    DIR    253,0     4096      2 /
> ...
> python2.7 1677 mailman   10u  IPv6 46441320      0t0    TCP mailman3.rrz.uni-koeln.de:55764->smtp-out.rrz.uni-koeln.de:smtp (ESTABLISHED)

indicates its FD 10 was ESTABLISHED connection to the MTA.


If the MTA is hanging up (or very slow progress) in application layer and
keeping alive TCP connection in lower layer, client using smtplib
without specifying timeout, like current SMTPDirect handler in Mailman,
must wait for response or the MTA dying.

Unfortunately smtplib for Python 2 before 2.6 don't have way to specify
timeout. It uses a socket in blocking mode unless seting default timeout
by using socket.setdefaulttimeout() before calling smtplib.SMTP.connction().
For Python 2.6 and above, it can be specified on create smtplib.SMTP object.

-- 
Yasuhito FUTATSUKI <futatuki at poem.co.jp>


From tran at isoc.org  Thu Feb  8 16:17:17 2018
From: tran at isoc.org (Dang Tran)
Date: Thu, 8 Feb 2018 21:17:17 +0000
Subject: [Mailman-Users] Capability to export archive messages to Office 365
 groups
Message-ID: <B45CBCB6-D0B9-4F68-BFA7-AF3463EEE319@contoso.com>

Hi ? I?d like to know if at all possible to export archive message to an O365 group.  thanks

-------
Best Regards,
Dang


From ailinaini at yahoo.com  Fri Feb  9 07:01:46 2018
From: ailinaini at yahoo.com (Ert Retre)
Date: Fri, 9 Feb 2018 12:01:46 +0000 (UTC)
Subject: [Mailman-Users] MM3 book
References: <688503220.1700741.1518177706969.ref@mail.yahoo.com>
Message-ID: <688503220.1700741.1518177706969@mail.yahoo.com>

I would love to see a new book on MM3.  Anyone know of such a project
proposed or in the works?

FWIW, the new Perl 6 world (see https://perl6.org) has produced several
books in the last year and some were started via various ?fund me? websites.

Best regards,
--
Alexander Garcia
https://www.drillanddriver.com


 

From mark at msapiro.net  Fri Feb  9 14:01:16 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 9 Feb 2018 11:01:16 -0800
Subject: [Mailman-Users] Capability to export archive messages to Office
 365 groups
In-Reply-To: <B45CBCB6-D0B9-4F68-BFA7-AF3463EEE319@contoso.com>
References: <B45CBCB6-D0B9-4F68-BFA7-AF3463EEE319@contoso.com>
Message-ID: <b3de992c-28a4-cf51-db88-ef7610c5e2a0@msapiro.net>

On 02/08/2018 01:17 PM, Dang Tran wrote:
> Hi ? I?d like to know if at all possible to export archive message to an O365 group.  thanks


At least with default settings, there is a cumulative archive mailbox on
the Mailman server at archives/private/LISTNAME.mbox/LISTNAME.mbox. This
is accessible via the web at a URL like
http://example.com/mailman/private/LISTNAME.mbox/LISTNAME.mbox.

Once you have that mailbox, importing it to an Office 365 group is a
question for Microsoft.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Fri Feb  9 14:17:10 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 9 Feb 2018 11:17:10 -0800
Subject: [Mailman-Users] MM3 book
In-Reply-To: <688503220.1700741.1518177706969@mail.yahoo.com>
References: <688503220.1700741.1518177706969.ref@mail.yahoo.com>
 <688503220.1700741.1518177706969@mail.yahoo.com>
Message-ID: <a0a49a6c-5466-ba76-ed6b-b9e9a258ca35@msapiro.net>

On 02/09/2018 04:01 AM, Ert Retre via Mailman-Users wrote:
> I would love to see a new book on MM3.  Anyone know of such a project
> proposed or in the works?
> 
> FWIW, the new Perl 6 world (see https://perl6.org) has produced several
> books in the last year and some were started via various ?fund me? websites.
> 
> Best regards,


Interesting. The above text exactly duplicates that in a post to this
list a few weeks ago. See that post and replies in the archives at
<https://mail.python.org/pipermail/mailman-users/2018-January/082887.html>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From rsk at gsp.org  Fri Feb  9 14:28:50 2018
From: rsk at gsp.org (Rich Kulawiec)
Date: Fri, 9 Feb 2018 14:28:50 -0500
Subject: [Mailman-Users] MM3 book in the works
In-Reply-To: <CAFMGiz9QeWzNouzt4xHZJ6u85=X5=3zwSUtCjeFdF9Nn8WWrFA@mail.gmail.com>
References: <CAFMGiz8GdB4zWG++mVq2xdAkYbYWa6ogEVGQhK2e_Nx3NRgZHg@mail.gmail.com>
 <20180113173829.GA16176@gsp.org>
 <CAFMGiz9QeWzNouzt4xHZJ6u85=X5=3zwSUtCjeFdF9Nn8WWrFA@mail.gmail.com>
Message-ID: <20180209192850.GA15382@gsp.org>

On Sat, Jan 13, 2018 at 06:34:03PM +0000, Tom Browder wrote:
> Good deal, Rich, that book is sorely needed IMHO! Is there any place we can
> sign up to get a copy or see its status?

I'm currently shoving Markdown into my brain at an accelerated pace
while simultaneously stitching together a number of already-written
modules into something that might vaguely resemble cohesive chapters.
This is not happening in a particular order, e.g., chapters 7 and 3
may emerge before chapter 1.  But as soon as they're ready, I'll make
pieces available for review.

---rsk

From tran at isoc.org  Mon Feb 12 16:28:59 2018
From: tran at isoc.org (Dang Tran)
Date: Mon, 12 Feb 2018 21:28:59 +0000
Subject: [Mailman-Users] Retention policy for archives
In-Reply-To: <<f1e797a9-8aff-80fc-554c-d6f38a64b97f@msapiro.net>>
Message-ID: <5C9B736F-3A19-452C-800D-0C6C3F285315@contoso.com>

Hi Mark ? prior to running this prune script, do I need to stop postfix first?



On 01/15/2018 11:44 AM, Dang Tran wrote:

> Hi ? I?d like to setup a retention policy to delete all archives older than

> 1yr.  please help me to setup this.  thanks





See the script at <https://www.msapiro.net/scripts/prune_arch> (mirrored

at <https://fog.ccsf.edu/~msapiro/scripts/prune_arch>).



You could run this periodically via cron to do what you want. The down

side of this is each time you run it, the messages in the archive will

be renumbered and prior saved URLs to archived messages will no longer work.



--

Mark Sapiro <m... at msapiro.net>        The highway is for gamblers,

San Francisco Bay Area, California    better use your sense - B. Dylan

------------------------------------------------------

Mailman-Users mailing list Mailman-Users at python.org

https://mail.python.org/mailman/listinfo/mailman-users

Mailman FAQ: http://wiki.list.org/x/AgA3

Security Policy: http://wiki.list.org/x/QIA9

Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/

Unsubscribe:

https://mail.python.org/mailman/options/mailman-users/archive%40jab.org



-------
Best Regards,
Dang


From mark at msapiro.net  Mon Feb 12 16:53:23 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 12 Feb 2018 13:53:23 -0800
Subject: [Mailman-Users] Retention policy for archives
In-Reply-To: <5C9B736F-3A19-452C-800D-0C6C3F285315@contoso.com>
References: <5C9B736F-3A19-452C-800D-0C6C3F285315@contoso.com>
Message-ID: <38275cc2-3b37-a200-0d60-5a9c46e7c4dd@msapiro.net>

On 02/12/2018 01:28 PM, Dang Tran wrote:
> Hi Mark ? prior to running this prune script, do I need to stop postfix first?
...> See the script at <https://www.msapiro.net/scripts/prune_arch>
(mirrored
> at <https://fog.ccsf.edu/~msapiro/scripts/prune_arch>).


No. The script is safe. It locks the list while updating the
LISTNAME.mbox, so other processes won't be updating concurrently.

Then the list is again locked by the bin/arch process while the HTML
archive is being rebuilt.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From sarma at gwup.org  Mon Feb 12 15:34:02 2018
From: sarma at gwup.org (Amardeo Sarma)
Date: Mon, 12 Feb 2018 20:34:02 +0000
Subject: [Mailman-Users] Mailman would not start after upgrade to 2.1.26
Message-ID: <AM6PR0102MB3349D6AAECE353B18C35C4EBD7F70@AM6PR0102MB3349.eurprd01.prod.exchangelabs.com>

Hi,

After upgrading to Mailman 2.1.26 via ports in FreeBSD 11.1 and a reboot of the server, I just could not start Mailman at all. It would not indicate any error. The Mailman logs here: /usr/local/mailman/logs did not record anything.

On second looks, I found that, after the upgrade, the write permissions for all logs under /usr/local/mailman/ logs had been removed. All log files had only read permissions for user and group. After adding write permissions (chmod 660 *) to all log files, I could restart Mailman again, and everything works again.

I am not sure this has happened before but just wanted to send this note to the list for the record just in case it is a problem no one had before. And I am sorry if I am reporting a solution to a well-known problem.

Does anyone have an idea what could have changed the write permissions for the logs during an upgrade? I use portmaster under FreeBSD.

Cheers,
Amardeo
P.S.: I was subscribed before and have resubscribed now.
--
Amardeo Sarma
sarma at gwup.org
@amardeo


From mark at msapiro.net  Tue Feb 13 13:25:08 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 13 Feb 2018 10:25:08 -0800
Subject: [Mailman-Users] Mailman would not start after upgrade to 2.1.26
In-Reply-To: <AM6PR0102MB3349D6AAECE353B18C35C4EBD7F70@AM6PR0102MB3349.eurprd01.prod.exchangelabs.com>
References: <AM6PR0102MB3349D6AAECE353B18C35C4EBD7F70@AM6PR0102MB3349.eurprd01.prod.exchangelabs.com>
Message-ID: <06e42347-ae4a-6f98-bdbb-7d8a5798b846@msapiro.net>

On 02/12/2018 12:34 PM, Amardeo Sarma wrote:
> 
> Does anyone have an idea what could have changed the write permissions for the logs during an upgrade? I use portmaster under FreeBSD.


This would seem to be an issue with FreeBSD ports. You should report
this to whoever is responsible for that.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From VxMr at pge.com  Tue Feb 13 14:09:48 2018
From: VxMr at pge.com (Montanez, Victor)
Date: Tue, 13 Feb 2018 19:09:48 +0000
Subject: [Mailman-Users] Is there an option to allow open Unsubscriptions
Message-ID: <C3B68975750F324FB948A2E44C95021F9E6DD81F@EXCHMBFF124.Utility.pge.com>

Is there a way to allow users to unsubscribe themselves without having to respond to the unsubscribe confirmation email?


From cpz at tuunq.com  Tue Feb 13 15:02:19 2018
From: cpz at tuunq.com (Carl Zwanzig)
Date: Tue, 13 Feb 2018 12:02:19 -0800
Subject: [Mailman-Users] Mailman would not start after upgrade to 2.1.26
In-Reply-To: <06e42347-ae4a-6f98-bdbb-7d8a5798b846@msapiro.net>
References: <AM6PR0102MB3349D6AAECE353B18C35C4EBD7F70@AM6PR0102MB3349.eurprd01.prod.exchangelabs.com>
 <06e42347-ae4a-6f98-bdbb-7d8a5798b846@msapiro.net>
Message-ID: <d32f48cd-f0db-e644-1e74-ff5ff7a64fa3@tuunq.com>

On 2/13/2018 10:25 AM, Mark Sapiro wrote:
> This would seem to be an issue with FreeBSD ports. You should report
> this to whoever is responsible for that

FWIW, while I like FreeBSD's ports & packages, I've found that installing 
mailman from the source generally is more reliable and updating is easier. YMMV.

z!

From mark at msapiro.net  Tue Feb 13 15:08:08 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 13 Feb 2018 12:08:08 -0800
Subject: [Mailman-Users] Is there an option to allow open Unsubscriptions
In-Reply-To: <C3B68975750F324FB948A2E44C95021F9E6DD81F@EXCHMBFF124.Utility.pge.com>
References: <C3B68975750F324FB948A2E44C95021F9E6DD81F@EXCHMBFF124.Utility.pge.com>
Message-ID: <3b3111e7-e342-fb56-c927-260292b34c59@msapiro.net>

On 02/13/2018 11:09 AM, Montanez, Victor wrote:
> Is there a way to allow users to unsubscribe themselves without having to respond to the unsubscribe confirmation email?


They can go to their options page, log in and once logged in, they can
unsubscribe immediately without a confirmation. Or, they can send an
email to the listname-request address with an appropriate unsubscribe
command containing their list password, the description of which is

>     unsubscribe [password] [address=<address>]
>         Unsubscribe from the mailing list.  If given, your password must match
>         your current password.  If omitted, a confirmation email will be sent
>         to the unsubscribing address. If you wish to unsubscribe an address
>         other than the address you sent this request from, you may specify
>         `address=<address>' (no brackets around the email address, and no
>         quotes!)


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From weif at weif.net  Tue Feb 13 15:29:08 2018
From: weif at weif.net (Keith Seyffarth)
Date: Tue, 13 Feb 2018 13:29:08 -0700
Subject: [Mailman-Users] Mailman would not start after upgrade to 2.1.26
In-Reply-To: <d32f48cd-f0db-e644-1e74-ff5ff7a64fa3@tuunq.com> (message from
 Carl Zwanzig on Tue, 13 Feb 2018 12:02:19 -0800)
Message-ID: <84a7wcpraj.fsf@maxwell.cjones.org>

Carl Zwanzig <cpz at tuunq.com> writes:

> On 2/13/2018 10:25 AM, Mark Sapiro wrote:
>> This would seem to be an issue with FreeBSD ports. You should report
>> this to whoever is responsible for that
>
> FWIW, while I like FreeBSD's ports & packages, I've found that installing 
> mailman from the source generally is more reliable and updating is easier. YMMV.

If you're installing from ports, whether building from within the ports
tree or using a port management tool like portupgrade or portmaster, you
are installing from source.

package is meant to simplify the process by just installing a compiled
binary and some necessary libraries, but I do have consistent problems
with installs from pkg - not to mention the various things that aren't
in pkg yet...

Keith

-- 
----
from my mac to yours...

Keith Seyffarth
mailto:weif at weif.net
http://www.weif.net/ - Home of the First Tank Guide!
http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar
----
http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention

From cpz at tuunq.com  Tue Feb 13 17:31:00 2018
From: cpz at tuunq.com (Carl Zwanzig)
Date: Tue, 13 Feb 2018 14:31:00 -0800
Subject: [Mailman-Users] Mailman would not start after upgrade to 2.1.26
In-Reply-To: <84a7wcpraj.fsf@maxwell.cjones.org>
References: <84a7wcpraj.fsf@maxwell.cjones.org>
Message-ID: <1c81a0a1-abb0-0632-3c63-0de9216be033@tuunq.com>

On 2/13/2018 12:29 PM, Keith Seyffarth wrote:
> If you're installing from ports, whether building from within the ports
> tree or using a port management tool like portupgrade or portmaster, you
> are installing from source.

...installing using the source code of whatever version the port is 
configured for and with whatever patches the port maintainer decided to 
include. This as opposed to installing the version -you- want with the 
options that -you- want to use. For some software, the port (and pkg) can 
lag significantly behind the current released version (as it happens, the 
current pkg is .25 and the current port is .26 from about 4 days ago but 
there were some updates to that from less that 24 hours ago).

I'll stick with my preference of installing mailman from it's original code 
tarball and not from a port; it's served me well for years.

z!

From alb at pccp.com.ar  Thu Feb 15 12:56:09 2018
From: alb at pccp.com.ar (Dr. Alberto Barengols)
Date: Thu, 15 Feb 2018 14:56:09 -0300
Subject: [Mailman-Users] a silly question :-)
Message-ID: <5c675416c87f77d57ddf7e3c93866be8@pccp.com.ar>

A very silly question: In the configuration web page, where is the
variable  to configure the automatic  reminder of the subscription to a
list? 

Thanks in advance,-A 

//////////////////////////////////////////////////////////////////////////

From mark at msapiro.net  Thu Feb 15 13:17:17 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 15 Feb 2018 10:17:17 -0800
Subject: [Mailman-Users] a silly question :-)
In-Reply-To: <5c675416c87f77d57ddf7e3c93866be8@pccp.com.ar>
References: <5c675416c87f77d57ddf7e3c93866be8@pccp.com.ar>
Message-ID: <b824a037-44dc-010d-d89a-5f95e2828e56@msapiro.net>

On 02/15/2018 09:56 AM, Dr. Alberto Barengols wrote:
> A very silly question: In the configuration web page, where is the
> variable  to configure the automatic  reminder of the subscription to a
> list? 


This is complex. In order for monthly reminders to be sent at all, the
installation must be running Mailman's cron/mailpasswds. This is
normally run, at least in the recommended Mailman crontab, at 05:00 on
the first of the month.

Given that cron/mailpasswds is being run, the lists General Options ->
send_reminders setting controls whether reminders are sent for that
list. If that setting is No, no reminders are sent regardless of user's
options. If it is Yes, then each user may opt in or out of reminders via
the user's options page "Get password reminder email for this list?"
setting. This user setting usually defaults to Yes,

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From james at dorydesign.com  Fri Feb 16 14:09:38 2018
From: james at dorydesign.com (Jim Dory)
Date: Fri, 16 Feb 2018 10:09:38 -0900
Subject: [Mailman-Users] messages getting auto discarded
Message-ID: <CAKhfoy2HO_77hag=VatYtK_gwFBpBjHUDUxhu725wN0qp+PgCQ@mail.gmail.com>

I've gotten a ton of spam directed at our mailing list so I set up some
auto discard filters. Unfortunately I don't know regular expressions so may
have misconfigured - result is some seemingly valid messages getting
discarded (could be different reason than my filters).

A couple yesterday were from un-subscribed users. but usually I get a
bounce saying it is held for moderation.. though not in this case. I don't
always check the auto discard messages because of the number of them, but a
registered user sent an email that never showed up as a bounce (held for
moderation) or as accepted, so I assume it was auto discarded as well.

Here's my filter config for subject:


^Subject: .*Phentermine
^Subject: .*F\*ckbuddy
^Subject: .*H00kup
^Subject: .*InstaF\*ck
^Subject: .*Instacheat
^Subject: .*\(dating\)*
^Subject: .*Larisa
^Subject: .*Viagra
^Subject: .*on line*
^Subject: .*pills
^Subject: .*Cialis
^Subject: .*Yulia
^Subject: .*rewarded
^Subject: .*viagra
^Subject: .*Orgasm
^Subject: .*Russia
^Subject: .*my gentle sun*
^Subject: .*Good day to you*
^Subject: .*Do.not.write.me*
^Subject: .*Casino
^Subject: .*Your invoice*
^Subject: .*Re\:Hey*
^Subject: .*Ukraine
^Subject: .*Re\:You
^Subject: .*Wives
^Subject: .*PowerPack
^Subject: .*our.communication*
^Subject: .*easy.money*
^Subject: .*Live.Chat*
^Subject: .*Games.and.profits*
^Subject: .*new.message*
^Subject: .*\?\?\?\?*
^Subject: .*SuperDiscount
^Subject: .*Easily.Earn*
^Subject: .*help.you.earn*
^Subject: .*let\'s.chat*
^Subject: .*I.am.on\-line*
^Subject: .*Making.\$*
^Subject: .*\(on\-line.now\)
^Subject: .*Xenical
^Subject: .*cure.yourself
^Subject: .*celexa
^Subject: .*binary.options
^Subject: .*Re\:.don\'t
^Subject: .*effective.tabs
^Subject: .*ED\!
^Subject: .*lexapro
^Subject: .*big.deal\!
^Subject: .*your.vigor
^Subject: .*boner
^Subject: .*medications
^Subject: .*buy.meds
^Subject: .*lasting.erection
^Subject: .*RE\:.Pure
^Subject: .*pure.joy
^Subject: .*hey\!
^Subject: .*online.trade
^Subject: .*ED.remedy
^Subject: .*answer\!
^Subject: .*best.meds
^Subject: .*powerful.meds
^Subject: .*start.trading
^Subject: .*RE\:.Now
^Subject: .*RE\:.loan
^Subject: .*sex
^Subject: .*fantasies
^Subject: .*make.money
^Subject: .*your.depression
^Subject: .*antidepressant
^Subject: .*impotence
^Subject: .*anti.depression
^Subject: .*our.portal
^Subject: .*without.depression
^Subject: .*bright.life
^Subject: .*amorous

and legacy:
# Lines that *start* with a '#' are comments.
to: friend at public.com
message-id: relay.comanche.denmark.eu
from: list at listme.com
from: .*@uplinkpro.com

and email:

^[^@]+ at bcira\.com$
^[^@]+ at airablo\.com$
^[^@]+ at bfklaw\.com$
^[^@]+ at bettella\.com$
^[^@]+ at areallycool\.com$
^[^@]+ at aristo-tec\.com$
^[^@]+ at benallgood\.com$
^[^@]+ at al-meshkah\.com$
^[^@]+ at atoccs\.stream$
^[^@]+ at authors\.com$
^[^@]+ at aulson\.com$
^[^@]+ at atmyx\.bid$
^[^@]+ at airtecperforms\.com$
vmservice at nomekennelclub.com
^[^@]+ at .+\.loan$
^[^@]+ at .+\.stream$
^[^@]+ at .+\.trade$
^[^@]+ at .+\.bid$
^[^@]+ at .+\.cn$
^[^@]+ at postingmuscle\.com$
^[^@]+ at adirondack\.net$
^[^@]+ at bicycleexpertwitness\.com$
^[^@]+ at allpoetry\.com$
^[^@]+ at autecsafety\.com$
^[^@]+ at proshred\.com$
^[^@]+ at archangel-films\.com$
^[^@]+ at alansphotos\.com$
^[^@]+ at agoprofil\.com$
^[^@]+ at readytech\.com$
^[^@]+ at blakecarrington\.com$
^[^@]+ at bigcatcafe\.com$
^[^@]+ at biovectra\.com$
^[^@]+ at blueridgeknives\.com$
^[^@]+ at akarenga\.com$
^[^@]+ at appetez\.com$
^[^@]+ at angelaortiz\.com$
^[^@]+ at agridfencing\.com$
^[^@]+ at blumenstetter-bindesysteme\.com$
^[^@]+ at alienwebhost\.com$
^[^@]+ at barkingcafe\.com$
^[^@]+ at babynamegenie\.com$
^[^@]+ at bluechick\.com$
^[^@]+ at bienenstock\.com$
^[^@]+ at askwith\.com$
^[^@]+ at bespoke-fp\.com$
^[^@]+ at alcoa\.com$
^[^@]+ at fotolia\.com$
^[^@]+ at betsonenterprises\.com$
^[^@]+ at argentinosonline\.com$
^[^@]+ at adeptus\.com$
dgimmingan at gci.net
^[^@]+ at andover-healthcare\.com$
^[^@]+ at bearablemoments\.com$
^[^@]+ at avexnet\.com$
^[^@]+ at avidnano\.com$
^[^@]+ at amcarco\.com$
^[^@]+ at biyougeka-kensakuya\.com$
^[^@]+ at albawardi\.com$
^[^@]+ at barrao\.com$
^[^@]+ at affordableweddinggown\.com$
^[^@]+ at barshield\.com$
^[^@]+ at autographink\.com$
^[^@]+ at blankethealthinsurance\.com$
^[^@]+ at alfredojunior\.com$
^[^@]+ at marketingautomationtools\.org$
^[^@]+ at wexonex\.com$
^[^@]+ at bargedirect\.com$
^[^@]+ at blacktoastintolerance\.com$
^[^@]+ at bada-bing\.com$
^[^@]+ at africanews\.com$
^[^@]+ at marketplace\.amazon\.co\.uk$
^[^@]+ at blazingworld\.com$
^[^@]+ at csu\.edu$
^[^@]+ at artecollezione\.com$
^[^@]+ at billupsdesign\.com$
^[^@]+ at art4sale\.com$
^[^@]+ at bdwt\.com$
^[^@]+ at architechies\.com$
^[^@]+ at banklife\.comfe\.com$
^[^@]+ at aveek\.com$
^[^@]+ at adweek\.com$
^[^@]+ at allendistribution\.com$
^[^@]+ at 1800radiator\.com$
^[^@]+ at alarabiaco\.com$
^[^@]+ at boldconcepts\.com$
^[^@]+ at andersonvaluationgroup\.com$
^[^@]+ at armandbasi\.com$
^[^@]+ at arastra\.com$
^[^@]+ at arcticspassaskatoon\.com$
^[^@]+ at aubreynorris\.com$
^[^@]+ at bctc\-lb\.com$
^[^@]+ at alongtheway\.com$
^[^@]+ at espbs\.net$
^[^@]+ at allthingsdigital\.com$
^[^@]+ at adinfocenter\.com$
^[^@]+ at arabize\.com$
^[^@]+ at giesting\.com$
^[^@]+ at .+\.co\.nz$
^[^@]+ at morebusinesswithfacebook\.com$
^[^@]+ at .+\.de$
^[^@]+ at .+\.ru$
^[^@]+ at .+\.ca$
^parsons at nome\.net$

The subject on one discarded msg was (from unsubscribed user - just used
wrong email):
MEDITATION! RURAL BUSINESS FINANCE! Register today.

And another subject same situation:
SPARC MEETING

Message sources didn't give me any clues, but I could provide one if it
would help.

thx, JD

From luscheina at yahoo.de  Fri Feb 16 16:27:56 2018
From: luscheina at yahoo.de (Christian F Buser)
Date: Fri, 16 Feb 2018 22:27:56 +0100
Subject: [Mailman-Users] messages getting auto discarded
In-Reply-To: <CAKhfoy2HO_77hag=VatYtK_gwFBpBjHUDUxhu725wN0qp+PgCQ@mail.gmail.com>
References: <CAKhfoy2HO_77hag=VatYtK_gwFBpBjHUDUxhu725wN0qp+PgCQ@mail.gmail.com>
Message-ID: <5A874CDC.9050902@yahoo.de>

I can?t help you - but are you sure that you want to exclude all German
addresses (.de) from your list?

Christian
 
-- 
Christian F. Buser, Hohle Gasse 6, CH-5507 Mellingen (Switzerland)      
Hilfe f?r Strassenkinder in Ghana: http://www.chance-for-children.org




From james at dorydesign.com  Fri Feb 16 16:45:51 2018
From: james at dorydesign.com (Jim Dory)
Date: Fri, 16 Feb 2018 12:45:51 -0900
Subject: [Mailman-Users] messages getting auto discarded
In-Reply-To: <5A874CDC.9050902@yahoo.de>
References: <CAKhfoy2HO_77hag=VatYtK_gwFBpBjHUDUxhu725wN0qp+PgCQ@mail.gmail.com>
 <5A874CDC.9050902@yahoo.de>
Message-ID: <CAKhfoy2jCWvYvw-bp3UXrChTNc9B1kDv_jLLjPw8M_1Hb38DLw@mail.gmail.com>

Good question! Nothing against Germany, but the list is strictly local
community stuff. If someone with that domain moved here, I would remove
that block. But it was easy to block whole countries rather than individual
spammers.

On Feb 16, 2018 12:27, "Christian F Buser" <luscheina at yahoo.de> wrote:

> I can?t help you - but are you sure that you want to exclude all German
> addresses (.de) from your list?
>
> Christian
>
> --
> Christian F. Buser, Hohle Gasse 6, CH-5507 Mellingen (Switzerland)
> Hilfe f?r Strassenkinder in Ghana: http://www.chance-for-children.org
>
>
>
>

From mark at msapiro.net  Fri Feb 16 23:04:02 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 16 Feb 2018 20:04:02 -0800
Subject: [Mailman-Users] messages getting auto discarded
In-Reply-To: <CAKhfoy2HO_77hag=VatYtK_gwFBpBjHUDUxhu725wN0qp+PgCQ@mail.gmail.com>
References: <CAKhfoy2HO_77hag=VatYtK_gwFBpBjHUDUxhu725wN0qp+PgCQ@mail.gmail.com>
Message-ID: <3e598826-a8b8-007d-7c62-4bbc25da5a43@msapiro.net>

On 02/16/2018 11:09 AM, Jim Dory wrote:
> I've gotten a ton of spam directed at our mailing list so I set up some
> auto discard filters. Unfortunately I don't know regular expressions so may
> have misconfigured - result is some seemingly valid messages getting
> discarded (could be different reason than my filters).
> 
> A couple yesterday were from un-subscribed users. but usually I get a
> bounce saying it is held for moderation.. though not in this case. I don't
> always check the auto discard messages because of the number of them, but a
> registered user sent an email that never showed up as a bounce (held for
> moderation) or as accepted, so I assume it was auto discarded as well.


First, you need to look at logs. Mailman's vette log will have a message
like

Message discarded, msgid: ...'
        list: ...,
        handler: ...

for each discarded message. In the case of header_filter_rules, the
handler will be SpamDetect.

Note that if you change the action to Hold,the vette log message will be

<listname> post from <sender> held, message-id=<...>: <reason>

and if your Mailman is 2.1.26, <reason> will tell you which regexp
matched. Prior to 2.1.26 it just says "message matched a filter rule"


> Here's my filter config for subject:
> ...


You should be aware that these regexps are matched case-insensitively.
You can't make them be case-sensitive.


> and legacy:
> # Lines that *start* with a '#' are comments.

For bounce_matching_headers it appears you just have the defaults which
probably never match as they are over 15 years old, and these result in
Holds, not Discards.
> 
> and email:
> 
...

If these are in discard_these_nonmembers, the handler will be Moderate.


> The subject on one discarded msg was (from unsubscribed user - just used
> wrong email):
> MEDITATION! RURAL BUSINESS FINANCE! Register today.
> 
> And another subject same situation:
> SPARC MEETING
> 
> Message sources didn't give me any clues, but I could provide one if it
> would help.


How are you seeing these things if the messages are discarded?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From james at dorydesign.com  Sat Feb 17 02:55:06 2018
From: james at dorydesign.com (Jim Dory)
Date: Fri, 16 Feb 2018 22:55:06 -0900
Subject: [Mailman-Users] messages getting auto discarded
In-Reply-To: <3e598826-a8b8-007d-7c62-4bbc25da5a43@msapiro.net>
References: <CAKhfoy2HO_77hag=VatYtK_gwFBpBjHUDUxhu725wN0qp+PgCQ@mail.gmail.com>
 <3e598826-a8b8-007d-7c62-4bbc25da5a43@msapiro.net>
Message-ID: <CAKhfoy0T4sjr7=0DqAkCby1D0rWi2-+UNPSO3W+SEWiZ84CADw@mail.gmail.com>

We are on Mailman 2.1.23. CPanel version.

Here is a typical auto discard vette log entry from a guy who isn't
subscribed on the particular email he used, but is on other email addresses:

Feb 16 18:13:35 2018 (1341) Message discarded, msgid: <
CALKxJoSAAhuBCRZCmmbWFpsrCjdZPwiwy--N95_1BDk4kniMGw at mail.gmail.com>'
       list: Nome-announce,
       handler: Moderate

This message went to a nome-announce.bounces "Auto discard notification" I
can actually see the message in that notification.

What has changed in the last month or two is when a non-member used to try
to post, I would get a nome-announce-bounces "Nome-announce post from
someone at nome.net requires approval" and the message would be "Post to a
moderated list". In that case I could then go into the administrative
interface "Tend to pending moderator requests" and review who had tried to
post and act on it one way or another. If it was a local non-spam email I
could write to the person and tell them to subscribe and give instructions,
or to use their subscribed email account. But now it seems like if anyone
tries to post with an unsubscribed account, I don't see them unless I go
through the (up to) hundreds of "Auto discard notification"s. So I'm
thinking now it isn't to do with any of my spam filters because I haven't
touched them for several months.. except I did add that ^parsons at nome\.net$
last entry in the sender filters not too long ago.

On Fri, Feb 16, 2018 at 7:04 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 02/16/2018 11:09 AM, Jim Dory wrote:
> > I've gotten a ton of spam directed at our mailing list so I set up some
> > auto discard filters. Unfortunately I don't know regular expressions so
> may
> > have misconfigured - result is some seemingly valid messages getting
> > discarded (could be different reason than my filters).
> >
> > A couple yesterday were from un-subscribed users. but usually I get a
> > bounce saying it is held for moderation.. though not in this case. I
> don't
> > always check the auto discard messages because of the number of them,
> but a
> > registered user sent an email that never showed up as a bounce (held for
> > moderation) or as accepted, so I assume it was auto discarded as well.
>
>
> First, you need to look at logs. Mailman's vette log will have a message
> like
>
> Message discarded, msgid: ...'
>         list: ...,
>         handler: ...
>
> for each discarded message. In the case of header_filter_rules, the
> handler will be SpamDetect.
>
> Note that if you change the action to Hold,the vette log message will be
>
> <listname> post from <sender> held, message-id=<...>: <reason>
>
> and if your Mailman is 2.1.26, <reason> will tell you which regexp
> matched. Prior to 2.1.26 it just says "message matched a filter rule"
>
>
> > Here's my filter config for subject:
> > ...
>
>
> You should be aware that these regexps are matched case-insensitively.
> You can't make them be case-sensitive.
>
>
> > and legacy:
> > # Lines that *start* with a '#' are comments.
>
> For bounce_matching_headers it appears you just have the defaults which
> probably never match as they are over 15 years old, and these result in
> Holds, not Discards.
> >
> > and email:
> >
> ...
>
> If these are in discard_these_nonmembers, the handler will be Moderate.
>
>
> > The subject on one discarded msg was (from unsubscribed user - just used
> > wrong email):
> > MEDITATION! RURAL BUSINESS FINANCE! Register today.
> >
> > And another subject same situation:
> > SPARC MEETING
> >
> > Message sources didn't give me any clues, but I could provide one if it
> > would help.
>
>
> How are you seeing these things if the messages are discarded?
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/
> mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/
> james%40dorydesign.com
>

From mark at msapiro.net  Sat Feb 17 03:18:10 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 17 Feb 2018 00:18:10 -0800
Subject: [Mailman-Users] messages getting auto discarded
In-Reply-To: <CAKhfoy0T4sjr7=0DqAkCby1D0rWi2-+UNPSO3W+SEWiZ84CADw@mail.gmail.com>
References: <CAKhfoy2HO_77hag=VatYtK_gwFBpBjHUDUxhu725wN0qp+PgCQ@mail.gmail.com>
 <3e598826-a8b8-007d-7c62-4bbc25da5a43@msapiro.net>
 <CAKhfoy0T4sjr7=0DqAkCby1D0rWi2-+UNPSO3W+SEWiZ84CADw@mail.gmail.com>
Message-ID: <cd2f3cb8-96f7-5356-75d5-f87c86bbeb0f@msapiro.net>

On 02/16/2018 11:55 PM, Jim Dory wrote:
> We are on Mailman 2.1.23. CPanel version.
> 
> Here is a typical auto discard vette log entry from a guy who isn't
> subscribed on the particular email he used, but is on other email addresses:
> 
> Feb 16 18:13:35 2018 (1341) Message discarded, msgid: <
> CALKxJoSAAhuBCRZCmmbWFpsrCjdZPwiwy--N95_1BDk4kniMGw at mail.gmail.com>'
>        list: Nome-announce,
>        handler: Moderate


This is a normal non-member discard. it is either because the non-member
address is in discard_these_nonmembers or not in any *_these_nonmembers
and generic_nonmember_action is Discard.


> This message went to a nome-announce.bounces "Auto discard notification" I
> can actually see the message in that notification.


Right. I forgot about forward_auto_discards.


> What has changed in the last month or two is when a non-member used to try
> to post, I would get a nome-announce-bounces "Nome-announce post from
> someone at nome.net requires approval" and the message would be "Post to a
> moderated list".


That should only happen if the post is from a member whose mod bit is
set and member_moderation_action is Hold.


> In that case I could then go into the administrative
> interface "Tend to pending moderator requests" and review who had tried to
> post and act on it one way or another. If it was a local non-spam email I
> could write to the person and tell them to subscribe and give instructions,
> or to use their subscribed email account.


What you are describing is more like a post being held because
generic_nonmember_action is Hold, but then the reason is 'Post by
non-member to a members-only list'


> But now it seems like if anyone
> tries to post with an unsubscribed account, I don't see them unless I go
> through the (up to) hundreds of "Auto discard notification"s. So I'm
> thinking now it isn't to do with any of my spam filters because I haven't
> touched them for several months.. except I did add that ^parsons at nome\.net$
> last entry in the sender filters not too long ago.


It has nothing to do with your spam filters. I think you, perhaps
inadvertently, changed Privacy options... -> Sender filters ->
generic_nonmember_action from Hold to Discard.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From james at dorydesign.com  Sat Feb 17 03:36:19 2018
From: james at dorydesign.com (Jim Dory)
Date: Fri, 16 Feb 2018 23:36:19 -0900
Subject: [Mailman-Users] messages getting auto discarded
In-Reply-To: <cd2f3cb8-96f7-5356-75d5-f87c86bbeb0f@msapiro.net>
References: <CAKhfoy2HO_77hag=VatYtK_gwFBpBjHUDUxhu725wN0qp+PgCQ@mail.gmail.com>
 <3e598826-a8b8-007d-7c62-4bbc25da5a43@msapiro.net>
 <CAKhfoy0T4sjr7=0DqAkCby1D0rWi2-+UNPSO3W+SEWiZ84CADw@mail.gmail.com>
 <cd2f3cb8-96f7-5356-75d5-f87c86bbeb0f@msapiro.net>
Message-ID: <CAKhfoy1VwKupdbsQ1SKEs4K=Y4NpivLKu06qpariWM-+O5HbXA@mail.gmail.com>

Thanks for your help Mark.. I think your last sentence nailed it.

/jim

On Fri, Feb 16, 2018 at 11:18 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 02/16/2018 11:55 PM, Jim Dory wrote:
> > We are on Mailman 2.1.23. CPanel version.
> >
> > Here is a typical auto discard vette log entry from a guy who isn't
> > subscribed on the particular email he used, but is on other email
> addresses:
> >
> > Feb 16 18:13:35 2018 (1341) Message discarded, msgid: <
> > CALKxJoSAAhuBCRZCmmbWFpsrCjdZPwiwy--N95_1BDk4kniMGw at mail.gmail.com>'
> >        list: Nome-announce,
> >        handler: Moderate
>
>
> This is a normal non-member discard. it is either because the non-member
> address is in discard_these_nonmembers or not in any *_these_nonmembers
> and generic_nonmember_action is Discard.
>
>
> > This message went to a nome-announce.bounces "Auto discard notification"
> I
> > can actually see the message in that notification.
>
>
> Right. I forgot about forward_auto_discards.
>
>
> > What has changed in the last month or two is when a non-member used to
> try
> > to post, I would get a nome-announce-bounces "Nome-announce post from
> > someone at nome.net requires approval" and the message would be "Post to a
> > moderated list".
>
>
> That should only happen if the post is from a member whose mod bit is
> set and member_moderation_action is Hold.
>
>
> > In that case I could then go into the administrative
> > interface "Tend to pending moderator requests" and review who had tried
> to
> > post and act on it one way or another. If it was a local non-spam email I
> > could write to the person and tell them to subscribe and give
> instructions,
> > or to use their subscribed email account.
>
>
> What you are describing is more like a post being held because
> generic_nonmember_action is Hold, but then the reason is 'Post by
> non-member to a members-only list'
>
>
> > But now it seems like if anyone
> > tries to post with an unsubscribed account, I don't see them unless I go
> > through the (up to) hundreds of "Auto discard notification"s. So I'm
> > thinking now it isn't to do with any of my spam filters because I haven't
> > touched them for several months.. except I did add that ^parsons at nome
> \.net$
> > last entry in the sender filters not too long ago.
>
>
> It has nothing to do with your spam filters. I think you, perhaps
> inadvertently, changed Privacy options... -> Sender filters ->
> generic_nonmember_action from Hold to Discard.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/
> mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/
> james%40dorydesign.com
>

From alex at spottedmouse.com  Sat Feb 17 09:09:05 2018
From: alex at spottedmouse.com (Alexander)
Date: Sat, 17 Feb 2018 22:09:05 +0800
Subject: [Mailman-Users] Debugging user unknown
Message-ID: <29c9d648-9e0c-16c1-3e99-df14ef6b0d24@spottedmouse.com>

G'day all,

I am trying to setup mailman on my linux server running postfix and 
virtual host integration. So far I have updated main.cf to include:

alias_database = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases
alias_maps = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases
virtual_alias_maps = hash:/var/lib/mailman/data/virtual-mailman, 
pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf, 
hash:/var/lib/mailman/data/virtual-mailman

Ran genaliases to create /var/lib/mailman/data/aliases and 
/var/lib/mailman/data/virtual-mailman which seem to contain the correct 
settings

However emails to mailmain at domain and list-staff at domain are bounced with 
unknown user:

Feb 17 22:01:32 tinyturtle postfix/virtual[1404]: 5F4DE460402: 
to=<mailman at XXX.com>, relay=virtual, delay=0.12, delays=0.04/0/0/0.09, 
dsn=5.1.1, status=bounced (unknown user: "mailman at XXX.com")
Feb 17 22:06:18 tinyturtle postfix/virtual[1731]: 3D12D460402: 
to=<list-staff at XXX.com>, relay=virtual, delay=0.08, 
delays=0.03/0.01/0/0.04, dsn=5.1.1, status=bounced (unknown user: 
"list-staff at XXX.com")

Since postfix/virtual is doing the bouncing I am suspected that the 
mapping (virtual-mailman) was picked up, and that i cannot find the 
mapping in aliases, but I could be wrong here.

I have also run newaliases to ensure the local aliases are updated, but 
this did not resolve this issue.

Any pointers on what else to do, to help me debug my configuration.

Kind regards
Alex

From mark at msapiro.net  Sat Feb 17 13:35:02 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 17 Feb 2018 10:35:02 -0800
Subject: [Mailman-Users] Debugging user unknown
In-Reply-To: <29c9d648-9e0c-16c1-3e99-df14ef6b0d24@spottedmouse.com>
References: <29c9d648-9e0c-16c1-3e99-df14ef6b0d24@spottedmouse.com>
Message-ID: <438d2d0d-1c95-ac08-6bde-cac07563c0da@msapiro.net>

On 02/17/2018 06:09 AM, Alexander wrote:
> 
> I am trying to setup mailman on my linux server running postfix and
> virtual host integration. So far I have updated main.cf to include:
> 
> alias_database = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases

It isn't necessary to include hash:/var/lib/mailman/data/aliases in
alias_database. It only says that /var/lib/mailman/data/aliases.db will
be updated if you issue a "newaliases" or "sendmail -bi" command which
isn't needed as Mailman issues the appropriate "postalias" command
whenever it updates /var/lib/mailman/data/aliases.


> alias_maps = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases

Good.


> virtual_alias_maps = hash:/var/lib/mailman/data/virtual-mailman,
> pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf,
> hash:/var/lib/mailman/data/virtual-mailman


You have hash:/var/lib/mailman/data/virtual-mailman listed twice.

Do you have "XXX.com" in virtual_alias_domains? You probably need this
unless "XXX.com" is in virtual_mailbox_domains.


> Ran genaliases to create /var/lib/mailman/data/aliases and
> /var/lib/mailman/data/virtual-mailman which seem to contain the correct
> settings
> 
> However emails to mailmain at domain and list-staff at domain are bounced with
> unknown user:
> 
> Feb 17 22:01:32 tinyturtle postfix/virtual[1404]: 5F4DE460402:
> to=<mailman at XXX.com>, relay=virtual, delay=0.12, delays=0.04/0/0/0.09,
> dsn=5.1.1, status=bounced (unknown user: "mailman at XXX.com")
> Feb 17 22:06:18 tinyturtle postfix/virtual[1731]: 3D12D460402:
> to=<list-staff at XXX.com>, relay=virtual, delay=0.08,
> delays=0.03/0.01/0/0.04, dsn=5.1.1, status=bounced (unknown user:
> "list-staff at XXX.com")
> 
> Since postfix/virtual is doing the bouncing I am suspected that the
> mapping (virtual-mailman) was picked up, and that i cannot find the
> mapping in aliases, but I could be wrong here.


Postfix virtual is the delivery agent for virtual_mailbox_domains, not
virtual_alias_domains.

It is possible to use virtual_alias_maps to map addresses that are in a
virtual_mailbox_domain, see
<http://www.postfix.org/VIRTUAL_README.html#virtual_mailbox>, so I'm not
sure why this isn't working, but if you still have issues after reading
the above link, post the output from 'postconf -n'.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From geek at uniserve.com  Mon Feb 19 20:13:09 2018
From: geek at uniserve.com (Dave Stevens)
Date: Mon, 19 Feb 2018 17:13:09 -0800
Subject: [Mailman-Users] two problems with Mailman 2.1
In-Reply-To: <716bb0e5-cd51-0c23-8d26-528001fa91b7@msapiro.net>
References: <20180201124103.21316r5vvig3zyan@webmail.uniserve.com>
 <716bb0e5-cd51-0c23-8d26-528001fa91b7@msapiro.net>
Message-ID: <20180219171309.4e164c92@user-Satellite-A100>

On Thu, 1 Feb 2018 15:12:44 -0800
Mark Sapiro <mark at msapiro.net> wrote:

> On 02/01/2018 12:41 PM, Dave Stevens wrote:
> > I?m having a couple of problems with a Mailman 2.1 list.


fwiw this has now been deemed a bug by virtualimin whose installer put
the list in place. After a recent VM upgrade this started happening.
It's being worked on I'll post a fix if one develops

Dave

> > 
> > I want to add several addresses as new subscribers so as a first
> > step I added one of them using the mass subscription facility in
> > membership management. Mailman accepted the data but when I
> > subsequently checked the subscriber list it wasn?t there. This has
>

snip!
-- 
In modern fantasy (literary or governmental), killing people is the
usual solution to the so-called war between good and evil. My books are
not conceived in terms of such a war, and offer no simple answers to
simplistic questions.

----- Ursula Le Guin

From tran at isoc.org  Wed Feb 21 16:28:53 2018
From: tran at isoc.org (Dang Tran)
Date: Wed, 21 Feb 2018 21:28:53 +0000
Subject: [Mailman-Users] Settings for dmarc_moderation_action
Message-ID: <1A07C041-C67A-4A98-88BA-0FD2B4650FA5@isoc.org>

I'm looking for the settings location of dmarc_moderation_action.  All my lists are currently set to Accept.  I'd like to change to Munge From.   Where can I find this settings and more importantly, how do I change this?  thanks

-------
Best Regards,
Dang
 

From tran at isoc.org  Wed Feb 21 16:31:05 2018
From: tran at isoc.org (Dang Tran)
Date: Wed, 21 Feb 2018 21:31:05 +0000
Subject: [Mailman-Users] Settings for dmarc_moderation_action
In-Reply-To: <1A07C041-C67A-4A98-88BA-0FD2B4650FA5@isoc.org>
References: <1A07C041-C67A-4A98-88BA-0FD2B4650FA5@isoc.org>
Message-ID: <CF9EAA35-585C-425D-8C47-DAA743861B9D@isoc.org>

Sorry, I meant from the command line to change (so I can script to change for all lists).  Thanks very much.

-------
Best Regards,
Dang
 
?On 2/21/18, 4:28 PM, "Dang Tran" <tran at isoc.org> wrote:

    I'm looking for the settings location of dmarc_moderation_action.  All my lists are currently set to Accept.  I'd like to change to Munge From.   Where can I find this settings and more importantly, how do I change this?  thanks
    
    -------
    Best Regards,
    Dang
     
    

From mark at msapiro.net  Wed Feb 21 23:54:23 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 21 Feb 2018 20:54:23 -0800
Subject: [Mailman-Users] Settings for dmarc_moderation_action
In-Reply-To: <CF9EAA35-585C-425D-8C47-DAA743861B9D@isoc.org>
References: <1A07C041-C67A-4A98-88BA-0FD2B4650FA5@isoc.org>
 <CF9EAA35-585C-425D-8C47-DAA743861B9D@isoc.org>
Message-ID: <aeb18e68-96cb-8ff7-de08-de37ba421426@msapiro.net>

On 02/21/2018 01:31 PM, Dang Tran wrote:
> Sorry, I meant from the command line to change (so I can script to change for all lists).  Thanks very much.


Create a file in Mailman's bin/ directory named mung_from.py containing
the following 6 lines indented as shown:

def mung_from(mlist):
    if not mlist.Locked():
        mlist.Lock()
    mlist.dmarc_moderation_action = 1
    mlist.Save()
    mlist.Unlock()

Then run Mailman's

bin/withlist -a -r mung_from

The file name has a .py extension, the name in the -r option to withlist
does not

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From alex at spottedmouse.com  Thu Feb 22 08:00:06 2018
From: alex at spottedmouse.com (Alexander)
Date: Thu, 22 Feb 2018 21:00:06 +0800
Subject: [Mailman-Users] Debugging user unknown
In-Reply-To: <438d2d0d-1c95-ac08-6bde-cac07563c0da@msapiro.net>
References: <29c9d648-9e0c-16c1-3e99-df14ef6b0d24@spottedmouse.com>
 <438d2d0d-1c95-ac08-6bde-cac07563c0da@msapiro.net>
Message-ID: <5cf3d939-e901-4641-8014-9ad037c78b5e@spottedmouse.com>

Hi Mark,

Thanks for your reply.

After adding

virtual_alias_domains = pgsql:/etc/postfix/pgsql_virtual_domains_maps.cf

I received the following bouce back :

User unknown in virtual alias table

and the following warning in the log file :

warning: do not list domain XXX.com in BOTH virtual_alias_domains and 
virtual_mailbox_domains

Virtual mailboxes are working, just not the forwarding to mailman. Here 
is the output for postconf -n

alias_database = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases
alias_maps = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases
biff = no
broken_sasl_auth_clients = yes
command_directory = /usr/sbin
compatibility_level = 2
content_filter = smtp-amavis:[localhost]:10024
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd 
$daemon_directory/$process_name $process_id & sleep 5
default_destination_concurrency_limit = 20
header_checks = regexp:/etc/postfix/header_checks
header_size_limit = 26214400
home_mailbox = .maildir/
html_directory = no
inet_protocols = ipv4
local_destination_concurrency_limit = 5
mail_owner = postfix
mailq_path = /usr/bin/mailq
manpage_directory = /usr/share/man
message_size_limit = 26214400
meta_directory = /etc/postfix
mydestination = $myhostname, localhost.$mydomain, localhost
mydomain = XXX.com
myhostname = XXX.XXX.com
mynetworks = 192.168.0.0/24, 127.0.0.0/8
myorigin = $mydomain
newaliases_path = /usr/bin/newaliases
owner_request_special = no
permissive = permit
queue_directory = /var/spool/postfix
readme_directory = no
recipient_delimiter = +
restrictive = check_sender_access hash:/etc/postfix/restrictions
sample_directory = /etc/postfix
sendmail_path = /usr/sbin/sendmail
setgid_group = postdrop
shlib_directory = /usr/lib64/postfix/${mail_version}
smtp_host_lookup = dns,native
smtp_tls_loglevel = 1
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3
smtp_tls_note_starttls_offer = yes
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:/var/lib/postfix/smtp_scache
smtpd_helo_required = yes
smtpd_recipient_restrictions = check_recipient_access 
hash:/etc/postfix/recipient_blacklist, permit_mynetworks, 
permit_sasl_authenticated, reject_unauth_destination
smtpd_restriction_classes = restrictive, permissive
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = yes
smtpd_sasl_path = smtpd
smtpd_sasl_security_options = noanonymous
smtpd_tls_CAfile = /etc/ssl/SSL_ROOT/External/chain.pem
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/ssl/SSL_ROOT/Cert_www.XXX.com/certificate.pem
smtpd_tls_exclude_ciphers = aNULL, MD5 , DES, ADH, RC4, PSD, SRP, 3DES, 
eNULL
smtpd_tls_key_file = /etc/ssl/SSL_ROOT/Cert_www.XXX.com/private_key.pem
smtpd_tls_loglevel = 3
smtpd_tls_mandatory_ciphers = high
smtpd_tls_mandatory_exclude_ciphers = aNULL, MD5 , DES, ADH, RC4, PSD, 
SRP, 3DES, eNULL
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
smtpd_use_tls = yes
tls_preempt_cipherlist = yes
tls_random_source = dev:/dev/urandom
unknown_local_recipient_reject_code = 550
virtual_alias_maps = hash:/var/lib/mailman/data/virtual-mailman, 
pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf
virtual_gid_maps = static:1001
virtual_mailbox_base = /home/vmail
virtual_mailbox_domains = pgsql:/etc/postfix/pgsql_virtual_domains_maps.cf
virtual_mailbox_limit = 51200000
virtual_mailbox_maps = pgsql:/etc/postfix/pgsql_virtual_mailbox_maps.cf
virtual_transport = virtual
virtual_uid_maps = static:1001


On 18/02/2018 02:35, Mark Sapiro wrote:
> On 02/17/2018 06:09 AM, Alexander wrote:
>> I am trying to setup mailman on my linux server running postfix and
>> virtual host integration. So far I have updated main.cf to include:
>>
>> alias_database = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases
> It isn't necessary to include hash:/var/lib/mailman/data/aliases in
> alias_database. It only says that /var/lib/mailman/data/aliases.db will
> be updated if you issue a "newaliases" or "sendmail -bi" command which
> isn't needed as Mailman issues the appropriate "postalias" command
> whenever it updates /var/lib/mailman/data/aliases.
>
>
>> alias_maps = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases
> Good.
>
>
>> virtual_alias_maps = hash:/var/lib/mailman/data/virtual-mailman,
>> pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf,
>> hash:/var/lib/mailman/data/virtual-mailman
>
> You have hash:/var/lib/mailman/data/virtual-mailman listed twice.
>
> Do you have "XXX.com" in virtual_alias_domains? You probably need this
> unless "XXX.com" is in virtual_mailbox_domains.
>
>
>> Ran genaliases to create /var/lib/mailman/data/aliases and
>> /var/lib/mailman/data/virtual-mailman which seem to contain the correct
>> settings
>>
>> However emails to mailmain at domain and list-staff at domain are bounced with
>> unknown user:
>>
>> Feb 17 22:01:32 tinyturtle postfix/virtual[1404]: 5F4DE460402:
>> to=<mailman at XXX.com>, relay=virtual, delay=0.12, delays=0.04/0/0/0.09,
>> dsn=5.1.1, status=bounced (unknown user: "mailman at XXX.com")
>> Feb 17 22:06:18 tinyturtle postfix/virtual[1731]: 3D12D460402:
>> to=<list-staff at XXX.com>, relay=virtual, delay=0.08,
>> delays=0.03/0.01/0/0.04, dsn=5.1.1, status=bounced (unknown user:
>> "list-staff at XXX.com")
>>
>> Since postfix/virtual is doing the bouncing I am suspected that the
>> mapping (virtual-mailman) was picked up, and that i cannot find the
>> mapping in aliases, but I could be wrong here.
>
> Postfix virtual is the delivery agent for virtual_mailbox_domains, not
> virtual_alias_domains.
>
> It is possible to use virtual_alias_maps to map addresses that are in a
> virtual_mailbox_domain, see
> <http://www.postfix.org/VIRTUAL_README.html#virtual_mailbox>, so I'm not
> sure why this isn't working, but if you still have issues after reading
> the above link, post the output from 'postconf -n'.
>


From alex at spottedmouse.com  Thu Feb 22 09:26:33 2018
From: alex at spottedmouse.com (Alexander)
Date: Thu, 22 Feb 2018 22:26:33 +0800
Subject: [Mailman-Users] Debugging user unknown
In-Reply-To: <5cf3d939-e901-4641-8014-9ad037c78b5e@spottedmouse.com>
References: <29c9d648-9e0c-16c1-3e99-df14ef6b0d24@spottedmouse.com>
 <438d2d0d-1c95-ac08-6bde-cac07563c0da@msapiro.net>
 <5cf3d939-e901-4641-8014-9ad037c78b5e@spottedmouse.com>
Message-ID: <15125c7e-83dc-15a2-ac37-f13e50ccf17c@spottedmouse.com>

I found the problem : 
https://forums.gentoo.org/viewtopic-t-134833-start-0.html

This fixed my issue


On 22/02/2018 21:00, Alexander wrote:
> Hi Mark,
>
> Thanks for your reply.
>
> After adding
>
> virtual_alias_domains = pgsql:/etc/postfix/pgsql_virtual_domains_maps.cf
>
> I received the following bouce back :
>
> User unknown in virtual alias table
>
> and the following warning in the log file :
>
> warning: do not list domain XXX.com in BOTH virtual_alias_domains and 
> virtual_mailbox_domains
>
> Virtual mailboxes are working, just not the forwarding to mailman. 
> Here is the output for postconf -n
>
> alias_database = hash:/etc/mail/aliases, 
> hash:/var/lib/mailman/data/aliases
> alias_maps = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases
> biff = no
> broken_sasl_auth_clients = yes
> command_directory = /usr/sbin
> compatibility_level = 2
> content_filter = smtp-amavis:[localhost]:10024
> daemon_directory = /usr/libexec/postfix
> data_directory = /var/lib/postfix
> debug_peer_level = 2
> debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin 
> ddd $daemon_directory/$process_name $process_id & sleep 5
> default_destination_concurrency_limit = 20
> header_checks = regexp:/etc/postfix/header_checks
> header_size_limit = 26214400
> home_mailbox = .maildir/
> html_directory = no
> inet_protocols = ipv4
> local_destination_concurrency_limit = 5
> mail_owner = postfix
> mailq_path = /usr/bin/mailq
> manpage_directory = /usr/share/man
> message_size_limit = 26214400
> meta_directory = /etc/postfix
> mydestination = $myhostname, localhost.$mydomain, localhost
> mydomain = XXX.com
> myhostname = XXX.XXX.com
> mynetworks = 192.168.0.0/24, 127.0.0.0/8
> myorigin = $mydomain
> newaliases_path = /usr/bin/newaliases
> owner_request_special = no
> permissive = permit
> queue_directory = /var/spool/postfix
> readme_directory = no
> recipient_delimiter = +
> restrictive = check_sender_access hash:/etc/postfix/restrictions
> sample_directory = /etc/postfix
> sendmail_path = /usr/sbin/sendmail
> setgid_group = postdrop
> shlib_directory = /usr/lib64/postfix/${mail_version}
> smtp_host_lookup = dns,native
> smtp_tls_loglevel = 1
> smtp_tls_mandatory_protocols = !SSLv2, !SSLv3
> smtp_tls_note_starttls_offer = yes
> smtp_tls_security_level = may
> smtp_tls_session_cache_database = btree:/var/lib/postfix/smtp_scache
> smtpd_helo_required = yes
> smtpd_recipient_restrictions = check_recipient_access 
> hash:/etc/postfix/recipient_blacklist, permit_mynetworks, 
> permit_sasl_authenticated, reject_unauth_destination
> smtpd_restriction_classes = restrictive, permissive
> smtpd_sasl_auth_enable = yes
> smtpd_sasl_authenticated_header = yes
> smtpd_sasl_path = smtpd
> smtpd_sasl_security_options = noanonymous
> smtpd_tls_CAfile = /etc/ssl/SSL_ROOT/External/chain.pem
> smtpd_tls_auth_only = yes
> smtpd_tls_cert_file = /etc/ssl/SSL_ROOT/Cert_www.XXX.com/certificate.pem
> smtpd_tls_exclude_ciphers = aNULL, MD5 , DES, ADH, RC4, PSD, SRP, 
> 3DES, eNULL
> smtpd_tls_key_file = /etc/ssl/SSL_ROOT/Cert_www.XXX.com/private_key.pem
> smtpd_tls_loglevel = 3
> smtpd_tls_mandatory_ciphers = high
> smtpd_tls_mandatory_exclude_ciphers = aNULL, MD5 , DES, ADH, RC4, PSD, 
> SRP, 3DES, eNULL
> smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
> smtpd_tls_received_header = yes
> smtpd_tls_session_cache_timeout = 3600s
> smtpd_use_tls = yes
> tls_preempt_cipherlist = yes
> tls_random_source = dev:/dev/urandom
> unknown_local_recipient_reject_code = 550
> virtual_alias_maps = hash:/var/lib/mailman/data/virtual-mailman, 
> pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf
> virtual_gid_maps = static:1001
> virtual_mailbox_base = /home/vmail
> virtual_mailbox_domains = 
> pgsql:/etc/postfix/pgsql_virtual_domains_maps.cf
> virtual_mailbox_limit = 51200000
> virtual_mailbox_maps = pgsql:/etc/postfix/pgsql_virtual_mailbox_maps.cf
> virtual_transport = virtual
> virtual_uid_maps = static:1001
>
>
> On 18/02/2018 02:35, Mark Sapiro wrote:
>> On 02/17/2018 06:09 AM, Alexander wrote:
>>> I am trying to setup mailman on my linux server running postfix and
>>> virtual host integration. So far I have updated main.cf to include:
>>>
>>> alias_database = hash:/etc/mail/aliases, 
>>> hash:/var/lib/mailman/data/aliases
>> It isn't necessary to include hash:/var/lib/mailman/data/aliases in
>> alias_database. It only says that /var/lib/mailman/data/aliases.db will
>> be updated if you issue a "newaliases" or "sendmail -bi" command which
>> isn't needed as Mailman issues the appropriate "postalias" command
>> whenever it updates /var/lib/mailman/data/aliases.
>>
>>
>>> alias_maps = hash:/etc/mail/aliases, hash:/var/lib/mailman/data/aliases
>> Good.
>>
>>
>>> virtual_alias_maps = hash:/var/lib/mailman/data/virtual-mailman,
>>> pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf,
>>> hash:/var/lib/mailman/data/virtual-mailman
>>
>> You have hash:/var/lib/mailman/data/virtual-mailman listed twice.
>>
>> Do you have "XXX.com" in virtual_alias_domains? You probably need this
>> unless "XXX.com" is in virtual_mailbox_domains.
>>
>>
>>> Ran genaliases to create /var/lib/mailman/data/aliases and
>>> /var/lib/mailman/data/virtual-mailman which seem to contain the correct
>>> settings
>>>
>>> However emails to mailmain at domain and list-staff at domain are bounced 
>>> with
>>> unknown user:
>>>
>>> Feb 17 22:01:32 tinyturtle postfix/virtual[1404]: 5F4DE460402:
>>> to=<mailman at XXX.com>, relay=virtual, delay=0.12, delays=0.04/0/0/0.09,
>>> dsn=5.1.1, status=bounced (unknown user: "mailman at XXX.com")
>>> Feb 17 22:06:18 tinyturtle postfix/virtual[1731]: 3D12D460402:
>>> to=<list-staff at XXX.com>, relay=virtual, delay=0.08,
>>> delays=0.03/0.01/0/0.04, dsn=5.1.1, status=bounced (unknown user:
>>> "list-staff at XXX.com")
>>>
>>> Since postfix/virtual is doing the bouncing I am suspected that the
>>> mapping (virtual-mailman) was picked up, and that i cannot find the
>>> mapping in aliases, but I could be wrong here.
>>
>> Postfix virtual is the delivery agent for virtual_mailbox_domains, not
>> virtual_alias_domains.
>>
>> It is possible to use virtual_alias_maps to map addresses that are in a
>> virtual_mailbox_domain, see
>> <http://www.postfix.org/VIRTUAL_README.html#virtual_mailbox>, so I'm not
>> sure why this isn't working, but if you still have issues after reading
>> the above link, post the output from 'postconf -n'.
>>
>
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: 
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: 
> https://mail.python.org/mailman/options/mailman-users/alex%40spottedmouse.com


From dandrews at visi.com  Fri Feb 23 10:07:59 2018
From: dandrews at visi.com (David Andrews)
Date: Fri, 23 Feb 2018 09:07:59 -0600
Subject: [Mailman-Users] Spam Subscriptions
Message-ID: <auto-000099162678@mailfront2.g2host.com>

I have a mailman installation with over 300 lists. It is cPanel, but 
I am the administrator so have access to command line etc.

I have just two lists that receive a bunch of spam subscribes each 
day -- hundreds of them, in fact. For some reason -- which is good, 
they are held, so don't go through, not quite sure why.  Two 
questions -- first is there a file I can erase for each list that 
will get rid of all the held subscriptions, without breaking anything 
else.  I tried once, and my installation broke -- don't know if it is 
related, but don't want to try again unless I do it right.

Secondly, there is some commonality in the subscribe addresses, are 
there strings I can use to discard the subscribes so I never have to see them.

Below are examples, there is a common word, or a common word, a 
period ., and another common word, then a plus sign + then a 4 5 or 6 
character word, all alpha, and @gmail.com
Here are examples:


dragonommz+
jwmidnight+
nommz.naidoo+


If I could knock these out, it would be helpful.  This has happened 
several times previously, but has always stopped after a few weeks. 
This time it has been a couple months.

Finally, I know it is probably too late in the Mailman2 cycle to get 
a new feature, but in the web UI, it would be nice if you could 
delete all deferred subscriptions.  You can do so with deferred 
messages, that are held, but not subscriptions.

Thanks!

Dave




From phils at caerllewys.net  Fri Feb 23 11:12:19 2018
From: phils at caerllewys.net (Phil Stracchino)
Date: Fri, 23 Feb 2018 11:12:19 -0500
Subject: [Mailman-Users] Spam Subscriptions
In-Reply-To: <auto-000099162678@mailfront2.g2host.com>
References: <auto-000099162678@mailfront2.g2host.com>
Message-ID: <085d1b3d-2203-e7d8-235d-5cf83ec46a9f@caerllewys.net>

On 02/23/18 10:07, David Andrews wrote:
> I have just two lists that receive a bunch of spam subscribes each 
> day -- hundreds of them, in fact. For some reason -- which is good, 
> they are held, so don't go through, not quite sure why.  Two 
> questions -- first is there a file I can erase for each list that 
> will get rid of all the held subscriptions, without breaking anything 
> else.  I tried once, and my installation broke -- don't know if it is 
> related, but don't want to try again unless I do it right.

As for the held subscriptions, you should be able to go to the list's
admin interface and drop all of the pending subscription requests as a
single operation.


-- 
  Phil Stracchino
  Babylon Communications
  phils at caerllewys.net
  phil at co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

From phils at caerllewys.net  Fri Feb 23 11:10:33 2018
From: phils at caerllewys.net (Phil Stracchino)
Date: Fri, 23 Feb 2018 11:10:33 -0500
Subject: [Mailman-Users] Spam Subscriptions
In-Reply-To: <auto-000099162678@mailfront2.g2host.com>
References: <auto-000099162678@mailfront2.g2host.com>
Message-ID: <9f591979-9d3b-099d-8d2f-cdb12009bb61@caerllewys.net>

On 02/23/18 10:07, David Andrews wrote:
> Secondly, there is some commonality in the subscribe addresses, are 
> there strings I can use to discard the subscribes so I never have to see them.
> 
> Below are examples, there is a common word, or a common word, a 
> period ., and another common word, then a plus sign + then a 4 5 or 6 
> character word, all alpha, and @gmail.com
> Here are examples:
> 
> dragonommz+
> jwmidnight+
> nommz.naidoo+
> 
> If I could knock these out, it would be helpful.  This has happened 
> several times previously, but has always stopped after a few weeks. 
> This time it has been a couple months.


You can't filter based on that address format.  (At least, not and be
correct.)

This format, plus-extension, is a legitimate address structure
specifically for the purpose of generating traceable throwaway
addresses.  If I give you reddog+thislist at example.com as my email
address, which I receive at my address reddog at example.com, and I've
given that to no-one else, and a few weeks later I start getting random
spam sent to reddog+thislist at example.com, I know you have (intentionally
or otherwise) leaked my email address.

Just because an address is plus-extended does not mean it is spam.  If
you choose to refuse extended addresses, you risk refusing legitimate
subscribers.

Have you considered requiring CAPTCHAs for subscription?


-- 
  Phil Stracchino
  Babylon Communications
  phils at caerllewys.net
  phil at co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

From brian at emwd.com  Fri Feb 23 11:29:29 2018
From: brian at emwd.com (Brian Carpenter)
Date: Fri, 23 Feb 2018 11:29:29 -0500
Subject: [Mailman-Users] Spam Subscriptions
In-Reply-To: <auto-000099162678@mailfront2.g2host.com>
References: <auto-000099162678@mailfront2.g2host.com>
Message-ID: <5834801d3acc3$7ab72720$70257560$@emwd.com>

> I have a mailman installation with over 300 lists. It is cPanel, but
> I am the administrator so have access to command line etc.
> 
> I have just two lists that receive a bunch of spam subscribes each
> day -- hundreds of them, in fact. For some reason -- which is good,
> they are held, so don't go through, not quite sure why.  Two
> questions -- first is there a file I can erase for each list that
> will get rid of all the held subscriptions, without breaking anything
> else.  I tried once, and my installation broke -- don't know if it is
> related, but don't want to try again unless I do it right.
> 
> Secondly, there is some commonality in the subscribe addresses, are
> there strings I can use to discard the subscribes so I never have to see
them.
> 
> Below are examples, there is a common word, or a common word, a
> period ., and another common word, then a plus sign + then a 4 5 or 6
> character word, all alpha, and @gmail.com
> Here are examples:
> 
> 
> dragonommz+
> jwmidnight+
> nommz.naidoo+
> 
> 
> If I could knock these out, it would be helpful.  This has happened
> several times previously, but has always stopped after a few weeks.
> This time it has been a couple months.
> 
> Finally, I know it is probably too late in the Mailman2 cycle to get
> a new feature, but in the web UI, it would be nice if you could
> delete all deferred subscriptions.  You can do so with deferred
> messages, that are held, but not subscriptions.
> 
> Thanks!
> 
> Dave

Hey Dave,

If  you are using cPanel then running some Exim filters may be a better
approach to handling this subscription problem. We get tons of spam from
China via two domains and Exim is great at filtering them out.


Brian Carpenter
Owner

Providing Cloud Services and more for over 15 years.

T: 336.755.0685
E: brian at emwd.com
www.emwd.com


From mark at msapiro.net  Fri Feb 23 13:50:21 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 23 Feb 2018 10:50:21 -0800
Subject: [Mailman-Users] Spam Subscriptions
In-Reply-To: <auto-000099162678@mailfront2.g2host.com>
References: <auto-000099162678@mailfront2.g2host.com>
Message-ID: <8f5c2153-4f29-35a2-8c4d-91dfb58fd83c@msapiro.net>

On 02/23/2018 07:07 AM, David Andrews wrote:
> 
> I have just two lists that receive a bunch of spam subscribes each day
> -- hundreds of them, in fact. For some reason -- which is good, they are
> held, so don't go through, not quite sure why.? Two questions -- first
> is there a file I can erase for each list that will get rid of all the
> held subscriptions, without breaking anything else.? I tried once, and
> my installation broke -- don't know if it is related, but don't want to
> try again unless I do it right.


See the script at <https://www.msapiro.net/scripts/erase> (mirrored at
<https://fog.ccsf.edu/~msapiro/scripts/erase>). This will remove
everything for an address or addresses that match a regexp.

Also for any list you can remove the lists/LISTNAME/request.pck file,
but if there are any held messages for the list, they too will disappear
from the pending requests although the data/heldmsg-LISTNAME-nnn.pck
file will still be there. The best thing is to handle all held messages
before removing the requests.pck file, but there is a script at
<https://www.msapiro.net/scripts/hold_again> (mirrored as above) that
can reprocess the data/heldmsg-LISTNAME-nnn.pck files or they can be
removed if not wanted.


> Secondly, there is some commonality in the subscribe addresses, are
> there strings I can use to discard the subscribes so I never have to see
> them.
> 
> Below are examples, there is a common word, or a common word, a period
> ., and another common word, then a plus sign + then a 4 5 or 6 character
> word, all alpha, and @gmail.com
> Here are examples:
> 
> 
> dragonommz+
> jwmidnight+
> nommz.naidoo+


Since Mailman 2.1.21 there is a GLOBAL_BAN_LIST. See
<https://mail.python.org/pipermail/mailman-users/2018-January/082905.html>
for a bit on how to use this. You will find more in the archives from
this Google search
<https://www.google.com/search?q=site%3Amail.python.org+inurl%3Amailman-users+%22global_ban_list%22>

Also, if you haven't done so, set SUBSCRIBE_FORM_SECRET to some string
unique to your site.

Both the above are mm_cfg.py settings.

Also, I don't know when cPanel will upgrade to Mailman 2.1.26 but it
contains an ability to enable reCAPTCHA on the listinfo page subscribe form.


> Finally, I know it is probably too late in the Mailman2 cycle to get a
> new feature, but in the web UI, it would be nice if you could delete all
> deferred subscriptions.? You can do so with deferred messages, that are
> held, but not subscriptions.


If someone wants to do it, I'd accept a merge request, but I'm not
likely to do it myself

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From sarma at gwup.org  Sun Feb 25 06:02:40 2018
From: sarma at gwup.org (Amardeo Sarma)
Date: Sun, 25 Feb 2018 11:02:40 +0000
Subject: [Mailman-Users] Mailman would not start after upgrade to 2.1.26
In-Reply-To: <06e42347-ae4a-6f98-bdbb-7d8a5798b846@msapiro.net>
References: <AM6PR0102MB3349D6AAECE353B18C35C4EBD7F70@AM6PR0102MB3349.eurprd01.prod.exchangelabs.com>
 <06e42347-ae4a-6f98-bdbb-7d8a5798b846@msapiro.net>
Message-ID: <AM6PR0102MB3349DF7C5A2158B30BB88F33D7C20@AM6PR0102MB3349.eurprd01.prod.exchangelabs.com>

Hi All,

I would like to report that I have resolved the problem. It was a problem I introduced, not the ports package, showing what unintended side effects some server changes cause.

I had introduced a new log rotation scheme using newsyslog in FreeBSD at around the same time I did the upgrade. This included, as an example, the following lines:
# logfilename                           [owner:group]   mode count      size when       flags   [/pid_file] [sig_num]
/usr/local/mailman/logs/bounce          mailman:mailman 400  5          *    $W2D1      X       /usr/local/mailman/data/master-qrunner.pid
/usr/local/mailman/logs/error           mailman:mailman 400  5          *    $W1D1      X       /usr/local/mailman/data/master-qrunner.pid
/usr/local/mailman/logs/locks           mailman:mailman 400  5          *    $W1D1      X       /usr/local/mailman/data/master-qrunner.pid
[..]

The mode 400, applied on the first Tuesday morning to all mailman logs, also made all the log files unwritable for mailman. Mailman stopped, probably with the next attempt to write a log, and would not start again. I have now changed the mode to 660, which has fixed the problem. A manual chmod for all log files was the immediate fix.

Amardeo
--
Amardeo Sarma @amardeo
sarma at gwup.org

-----Original Message-----
From: Mailman-Users [mailto:mailman-users-bounces+sarma=gwup.org at python.org] On Behalf Of Mark Sapiro
Sent: Tuesday, February 13, 2018 7:25 PM
To: mailman-users at python.org
Subject: Re: [Mailman-Users] Mailman would not start after upgrade to 2.1.26

On 02/12/2018 12:34 PM, Amardeo Sarma wrote:
> 
> Does anyone have an idea what could have changed the write permissions for the logs during an upgrade? I use portmaster under FreeBSD.


This would seem to be an issue with FreeBSD ports. You should report this to whoever is responsible for that.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: https://mail.python.org/mailman/options/mailman-users/sarma%40gwup.org


From pasqualinic at fcal.uner.edu.ar  Tue Feb 27 10:40:13 2018
From: pasqualinic at fcal.uner.edu.ar (Carlos R. Pasqualini)
Date: Tue, 27 Feb 2018 12:40:13 -0300
Subject: [Mailman-Users] Mailman 2 and read notifications
Message-ID: <1519746013.32173.70.camel@fcal.uner.edu.ar>

Hi list

I see a blank response from a list's email from a user ho say he did
not sent any response. It seems like an automatic response to a read
confirmation, but this response was sent to many lists on our server
(triggering moderation alerts on the other lists, and my attention).

Is it possible that the original sender set the 'confirmation request'
on the MUA and that request propagated past mailman and got into
receiver's MUA?

If yes: Is there a way to intercept and delete these requests on the
Mailman host?


Using mailman from Debian 8: 2.1.18-2+deb8u2 


Thanks!!

-- 
Carlos Pasqualini

Administraci?n de Redes
Facultad de Ciencias de la Alimentaci?n
Universidad Nacional de Entre R?os
http://www.fcal.uner.edu.ar/


From info at ripandrevmedia.ca  Mon Feb 26 17:49:57 2018
From: info at ripandrevmedia.ca (Tammy Mitchell)
Date: Mon, 26 Feb 2018 14:49:57 -0800
Subject: [Mailman-Users] preserve html and allow attachements
Message-ID: <80672124-d440-5341-3e9e-ac135aa1a5ef@ripandrevmedia.ca>

Hi, I've been reading posts I find through google searches to try to 
solve this. I can't seem to find the correct settings.

-I'm trying to allow html email, with attachments of photos and .pdfs.
-I'd also like my archives to look like the email people get. Right now 
it looks like text with code. The html is staying raw, filled with the 
code, it is not rendering properly. My attachements are still all being 
scrubbed.
-I guess I'd like the archive messages to show as rendered html, with a 
link to any attachments that were included in the mail.
-I would like the default view of my archives to be set to Thread.

I've gotten this far:

I am allowing html email to go to my list and that is working well.
Last week, I had it working so attachments would go with the email. That 
was working. It is not now, since I made the changes described below:

Today I read instructions, on this page, and changed the settings as 
advised here.
https://wiki.list.org/DOC/How%20do%20I%20preserve%20HTML%20formatting%20while%20filtering%20other%20content%3F

Changed the settings as described in that link. I thought it would allow 
my archives to look like rendered html.
(nope).
- also now my attachments are not being sent with the email messages as 
they had been.

I'm in a plesk environment so I don't have access to the mail Mailman 
files. I might be able to ask my administrator if he'd change some files 
if needed.

My archives are at:
http://lists.cotyrone.com/pipermail/ulsterancestry/

Do you suggest the easiest thing for me to do, would be to set my own 
list up at:
https://www.mail-archive.com/search?q=preserve+html+and+allow+attachements&l=mailman-users%40python.org

Also, should I be asking my server admin to upgrade me to Mailman 3.x at 
this time, since my list is new. How hard would that be for him to 
install, and keep up to date in the Plesk environment?

I thank you in advance.

Tammy Mitchell


From mark at msapiro.net  Tue Feb 27 15:07:39 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 27 Feb 2018 12:07:39 -0800
Subject: [Mailman-Users] preserve html and allow attachements
In-Reply-To: <80672124-d440-5341-3e9e-ac135aa1a5ef@ripandrevmedia.ca>
References: <80672124-d440-5341-3e9e-ac135aa1a5ef@ripandrevmedia.ca>
Message-ID: <3c9e135e-bba5-5165-35ec-7538a509ff06@msapiro.net>

On 02/26/2018 02:49 PM, Tammy Mitchell wrote:
> Hi, I've been reading posts I find through google searches to try to
> solve this. I can't seem to find the correct settings.
> 
> -I'm trying to allow html email, with attachments of photos and .pdfs.

You have two choices. You can set Content filtering -> filter_content to
No, or you can set filter_content to Yes and set the following

pass_mime_types:
    multipart
    text/plain
    text/html
    image
    application/pdf

collapse_alternatives: No
convert_html_to_plaintext: No

You may also want to add some "rare" pdf types like application/nappdf
and application/x-pdf and possibly limit image types to, e.g.,
image/jpeg, image/png, image/tiff depending on what you want to allow.


> -I'd also like my archives to look like the email people get. Right now
> it looks like text with code. The html is staying raw, filled with the
> code, it is not rendering properly. My attachements are still all being
> scrubbed.

Pipermail doesn't do that. Non text/plain attachments will always be
scrubbed. HTML will always be HTML escaped unless you add

ARCHIVE_HTML_SANITIZER = 3

See the docs in Mailman/Defaults.py which say in part

> # 3 - Remove text/html as attachments but don't HTML-escape them. Note: this
> #     is very dangerous because it essentially means anybody can send an HTML
> #     email to your site containing evil JavaScript or web bugs, or other
> #     nasty things, and folks viewing your archives will be susceptible.  You
> #     should only consider this option if you do heavy moderation of your list
> #     postings.
> #
> # Note: given the current archiving code, it is not possible to leave
> # text/html parts inline and un-escaped.  I wouldn't think it'd be a good idea
> # to do anyway.


The alternative is to use an external archiver such as MHonArc


> -I guess I'd like the archive messages to show as rendered html, with a
> link to any attachments that were included in the mail.
> -I would like the default view of my archives to be set to Thread.
> 
> I've gotten this far:
> 
> I am allowing html email to go to my list and that is working well.
> Last week, I had it working so attachments would go with the email. That
> was working. It is not now, since I made the changes described below:
> 
> Today I read instructions, on this page, and changed the settings as
> advised here.
> https://wiki.list.org/DOC/How%20do%20I%20preserve%20HTML%20formatting%20while%20filtering%20other%20content%3F
> 
> 
> Changed the settings as described in that link. I thought it would allow
> my archives to look like rendered html.
> (nope).
> - also now my attachments are not being sent with the email messages as
> they had been.


Did you set scrub_nondigest to Yes? (The FAQ says be sure it's No), but
setting it Yes or not passing HTML at all are the only things that would
cause this.


> I'm in a plesk environment so I don't have access to the mail Mailman
> files. I might be able to ask my administrator if he'd change some files
> if needed.


He'd be the one to change ARCHIVE_HTML_SANITIZER, but it's a global
setting and doesn't do exactly what you want anyway.


> My archives are at:
> http://lists.cotyrone.com/pipermail/ulsterancestry/
> 
> Do you suggest the easiest thing for me to do, would be to set my own
> list up at:
> https://www.mail-archive.com/search?q=preserve+html+and+allow+attachements&l=mailman-users%40python.org


That search returns no results. and mail-archive.com is an archiving
service, not a list host, but you could archive your list there if you
like it better than pipermail.


> Also, should I be asking my server admin to upgrade me to Mailman 3.x at
> this time, since my list is new. How hard would that be for him to
> install, and keep up to date in the Plesk environment?


I have no idea how difficult it would be to install Mailman 3 under
Plesk. Mailman 3 installation is non-trivial. According to
<https://wiki.list.org/DOC/Mailman%20and%20Plesk>, installing Mailman 2
upgrades manually is non-trivial - "It is possible to trick Plesk into
recognizing newer versions installed manually but it's not for the faint
of heart."

The situation might be easier with Mailman 3 since Plesk knows nothing
about it.

You might want to check out
<https://lists.mailman3.org/archives/list/mailman-users at mailman3.org/>
to see what Mailman 3's HyperKitty archives look like, but that list is
plain text only. There are other lists at
<https://mail.python.org/mm3/mailman3/>, but I don't know if any have
HTML in their archives.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Tue Feb 27 23:37:48 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 27 Feb 2018 20:37:48 -0800
Subject: [Mailman-Users] preserve html and allow attachements
In-Reply-To: <3c9e135e-bba5-5165-35ec-7538a509ff06@msapiro.net>
References: <80672124-d440-5341-3e9e-ac135aa1a5ef@ripandrevmedia.ca>
 <3c9e135e-bba5-5165-35ec-7538a509ff06@msapiro.net>
Message-ID: <45e8e4a7-969d-5db6-42ac-4774b76bd63a@msapiro.net>

On 02/27/2018 12:07 PM, Mark Sapiro wrote:
> On 02/26/2018 02:49 PM, Tammy Mitchell wrote:
>>
>> I am allowing html email to go to my list and that is working well.
>> Last week, I had it working so attachments would go with the email. That
>> was working. It is not now, since I made the changes described below:
>>
>> Today I read instructions, on this page, and changed the settings as
>> advised here.
>> https://wiki.list.org/DOC/How%20do%20I%20preserve%20HTML%20formatting%20while%20filtering%20other%20content%3F
>>
>>
>> Changed the settings as described in that link. I thought it would allow
>> my archives to look like rendered html.
>> (nope).
>> - also now my attachments are not being sent with the email messages as
>> they had been.
> 
> 
> Did you set scrub_nondigest to Yes? (The FAQ says be sure it's No), but
> setting it Yes or not passing HTML at all are the only things that would
> cause this.


Actually, I think I answered that too quickly. The issue is probably
because that FAQ is only aimed at passing HTML. if you set
pass_mime_types as suggested in that FAQ you will remove all PDFs and
images and other non  - plain text/HTML attachments. Note that its title
is "How do I preserve HTML formatting while filtering other content?"

If you want to pass other things in addition to plain and HTML text you
have to add their mime types to pass_mime_types as I indicated in my
first reply.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From davekm3t at gmail.com  Wed Feb 28 14:01:32 2018
From: davekm3t at gmail.com (Dave Pascoe)
Date: Wed, 28 Feb 2018 14:01:32 -0500
Subject: [Mailman-Users] Issues w/ subscribe/reCAPTCHA v2 on Mailman 2.1.26
Message-ID: <CAH5r4X_Ha_8pxUbBheVt1kAxndizywNZYZ93dZ1dhxY9OP-V-A@mail.gmail.com>

Just upgraded to Mailman 2.1.26 on a CentOS 5.11 system (yeah, I
know....this is a legacy box and will be migrating to a newer OS soon). I'm
trying to enable reCAPTCHA v2 support. I've defined:

RECAPTCHA_SITE_KEY = '...'
RECAPTCHA_SECRET_KEY = '...'

in mm_cfg.py.

I keep getting the exception shown below when testing out the subscribe
function. The only way to get subscribe working again is to comment out the
reCAPTCHA code in subscribe.py and commenting out RECAPTCHA_SITE_KEY and
RECAPTCHA_SECRET_KEY in mm_cfg.py.

Any ideas?

TIA,
Dave

===============================================

admin(5122): [----- Mailman Version: 2.1.26 -----]
admin(5122): [----- Traceback ------]
admin(5122): Traceback (most recent call last):
admin(5122):   File "/usr/local/mailman/scripts/driver", line 102, in
run_main
admin(5122):     pkg = __import__('Mailman.Cgi', globals(), locals(),
[scriptname])
admin(5122):   File "/usr/local/mailman/Mailman/Cgi/subscribe.py", line 154
      except urllib2.URLError as e:
                               ^
  SyntaxError: invalid syntax
admin(5122): [----- Python Information -----]
admin(5122): sys.version     =   2.4.3 (#1, Jan  9 2013, 06:49:54)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)]
admin(5122): sys.executable  =   /usr/bin/python
admin(5122): sys.prefix      =   /usr
admin(5122): sys.exec_prefix =   /usr
admin(5122): sys.path        =   ['/usr/local/mailman/pythonlib',
'/usr/local/mailman', '/usr/local/mailman/scripts', '/usr/local/mailman',
'/usr/lib/python24.zip', '/usr/lib/python2.4/',
'/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib-tk',
'/usr/lib/python2.4/lib-dynload', '/usr/lib/python2.4/site-packages',
'/usr/lib/python2.4/dist-packages']
admin(5122): sys.platform    =   linux2