From jacques at lavignotte.org  Mon Jul  2 07:17:29 2018
From: jacques at lavignotte.org (Jacques Lavignotte)
Date: Mon, 2 Jul 2018 13:17:29 +0200
Subject: [Mailman-Users] Moderation impossible. Debian 9
Message-ID: <c057c4a8-95a2-a2cf-8798-218898dff50e@lavignotte.org>

Hello,

is there anything related with Debian 8 to 9 upgrade ?

now : mailman 1:2.1.23-1+deb9u2

Moderation in now impossible.

I tried to check perms :


sudo -u list /var/lib/mailman/bin/check_perms -f -v


v?rification du gid et du mode pour /var/lib/mailman/locks
Mauvais gid pour /var/lib/mailman/locks (Obtenu: root, Attendu list) 
(r?paration)
     v?rification du gid et du mode pour /var/lib/mailman/icons
Mauvais gid pour /var/lib/mailman/icons (Obtenu: root, Attendu list) 
(r?paration)
Traceback (most recent call last):
   File "/var/lib/mailman/bin/check_perms", line 391, in <module>
     checkall()
   File "/var/lib/mailman/bin/check_perms", line 204, in checkall
     os.path.walk(d, checkwalk, STATE)
   File "/usr/lib/python2.7/posixpath.py", line 231, in walk
     func(arg, top, names)
   File "/var/lib/mailman/bin/check_perms", line 126, in checkwalk
     os.chown(path, -1, MAILMAN_GID)
OSError: [Errno 1] Operation not permitted: '/var/lib/mailman/icons'



Anything I can do ?

Thanks,   Jacques


-- 
GnuPg : C8F5B1E3 WeUsePGP Because privacy matters http://weusepgp.info/

From mark at msapiro.net  Mon Jul  2 10:58:39 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 2 Jul 2018 07:58:39 -0700
Subject: [Mailman-Users] Moderation impossible. Debian 9
In-Reply-To: <c057c4a8-95a2-a2cf-8798-218898dff50e@lavignotte.org>
References: <c057c4a8-95a2-a2cf-8798-218898dff50e@lavignotte.org>
Message-ID: <8d374451-c94b-2298-59fd-2a881da84aa4@msapiro.net>

On 7/2/18 4:17 AM, Jacques Lavignotte wrote:
> Hello,
> 
> is there anything related with Debian 8 to 9 upgrade ?
> 
> now : mailman 1:2.1.23-1+deb9u2
> 
> Moderation in now impossible.


This appears to be a Debian issue. Please see
<https://wiki.list.org/x/12812344> and report this to Debian.

Also, in any report say what you mean by Moderation is now impossible.
I.e. if the admindb interface appears to work but doesn't change
anything, say that (or see <https://wiki.list.org/x/4030602>). If the
admindb interface gives the "We hit a bug" page, provide the relevant
information from Mailman's error log.

Also note the issue with

> OSError: [Errno 1] Operation not permitted: '/var/lib/mailman/icons' 

is possibly because /var/lib/mailman/icons is a symlink (another Debian
thing).

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From phils at caerllewys.net  Wed Jul  4 14:52:09 2018
From: phils at caerllewys.net (Phil Stracchino)
Date: Wed, 4 Jul 2018 14:52:09 -0400
Subject: [Mailman-Users] Unable to start mailman due to time travel
Message-ID: <2169d08d-b5f8-9054-b4d9-87be3da935fc@caerllewys.net>

I have mailman-2.1.26 installed on my Gentoo 17.0 application server,
which I just rebooted to update from kernel 4.15.15 to 4.16.13.  After
rebooting, mailman would not start, logging the following errors:


Jul 04 14:43:58 2018 mailmanctl(15743): Traceback (most recent call last):
Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
556, in <module>
Jul 04 14:43:58 2018 mailmanctl(15743):      main()
Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
391, in main
Jul 04 14:43:58 2018 mailmanctl(15743):      lock = acquire_lock(force)
Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
215, in acquire_lock
Jul 04 14:43:58 2018 mailmanctl(15743):      lock = acquire_lock_1(force)
Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
203, in acquire_lock_1
Jul 04 14:43:58 2018 mailmanctl(15743):      if not force or
qrunner_state():
Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
187, in qrunner_state
Jul 04 14:43:58 2018 mailmanctl(15743):      os.kill(pid, 0)
Jul 04 14:43:58 2018 mailmanctl(15743): OSError :  [Errno 1] Operation
not permitted


This looks like it's failing to acquire a lock file.  Investigating, I
discovered this:

minbar:root:/var/lib/mailman/logs:18 # ls -l /var/lib/mailman/locks
total 8
-rw-rw-r-- 2 mailman mailman 49 Jul  5  2018 master-qrunner
-rw-rw-r-- 2 mailman mailman 49 Jul  5  2018 master-qrunner.minbar.4623

The server itself is pretty clear that today is July 4:

minbar:root:/var/lib/mailman/logs:23 # date
Wed Jul  4 14:50:09 EDT 2018

After I deleted the errant lock files, mailman started up without
hesitation ...  and AGAIN created lock files timestamped in the future:


minbar:root:/var/lib/mailman/logs:22 # ls -l /var/lib/mailman/locks
total 8
-rw-rw-r-- 2 mailman mailman 50 Jul  5  2018 master-qrunner
-rw-rw-r-- 2 mailman mailman 50 Jul  5  2018 master-qrunner.minbar.15895



Does anyone have any idea what's going on here?


-- 
  Phil Stracchino
  Babylon Communications
  phils at caerllewys.net
  phil at co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

From mark at msapiro.net  Wed Jul  4 15:10:50 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 4 Jul 2018 12:10:50 -0700
Subject: [Mailman-Users] Unable to start mailman due to time travel
In-Reply-To: <2169d08d-b5f8-9054-b4d9-87be3da935fc@caerllewys.net>
References: <2169d08d-b5f8-9054-b4d9-87be3da935fc@caerllewys.net>
Message-ID: <48fa0fd7-3f3b-30d5-bb66-3232002aff0a@msapiro.net>

On 7/4/18 11:52 AM, Phil Stracchino wrote:
> I have mailman-2.1.26 installed on my Gentoo 17.0 application server,
> which I just rebooted to update from kernel 4.15.15 to 4.16.13.  After
> rebooting, mailman would not start, logging the following errors:
> 
> 
> Jul 04 14:43:58 2018 mailmanctl(15743): Traceback (most recent call last):
> Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
> 556, in <module>
> Jul 04 14:43:58 2018 mailmanctl(15743):      main()
> Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
> 391, in main
> Jul 04 14:43:58 2018 mailmanctl(15743):      lock = acquire_lock(force)
> Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
> 215, in acquire_lock
> Jul 04 14:43:58 2018 mailmanctl(15743):      lock = acquire_lock_1(force)
> Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
> 203, in acquire_lock_1
> Jul 04 14:43:58 2018 mailmanctl(15743):      if not force or
> qrunner_state():
> Jul 04 14:43:58 2018 mailmanctl(15743):   File "bin/mailmanctl", line
> 187, in qrunner_state
> Jul 04 14:43:58 2018 mailmanctl(15743):      os.kill(pid, 0)
> Jul 04 14:43:58 2018 mailmanctl(15743): OSError :  [Errno 1] Operation
> not permitted
> 
> 
> This looks like it's failing to acquire a lock file.  Investigating, I
> discovered this:


There was probably a stale lock due to Mailman not being cleanly
terminated upon reboot. The Operation not permitted is an attempt to
determine the status of the PID that created the lock by sending it a
signal 0. The error may be a Gentoo thing or 'mailmanctl start' may need
to run as root if it wasn't.


> minbar:root:/var/lib/mailman/logs:18 # ls -l /var/lib/mailman/locks
> total 8
> -rw-rw-r-- 2 mailman mailman 49 Jul  5  2018 master-qrunner
> -rw-rw-r-- 2 mailman mailman 49 Jul  5  2018 master-qrunner.minbar.4623
> 
> The server itself is pretty clear that today is July 4:
> 
> minbar:root:/var/lib/mailman/logs:23 # date
> Wed Jul  4 14:50:09 EDT 2018
> 
> After I deleted the errant lock files, mailman started up without
> hesitation ...  and AGAIN created lock files timestamped in the future:
> 
> 
> minbar:root:/var/lib/mailman/logs:22 # ls -l /var/lib/mailman/locks
> total 8
> -rw-rw-r-- 2 mailman mailman 50 Jul  5  2018 master-qrunner
> -rw-rw-r-- 2 mailman mailman 50 Jul  5  2018 master-qrunner.minbar.15895


This is all normal and expected. The time stamp on locks is set to that
future time when the lock expires.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From phils at caerllewys.net  Wed Jul  4 15:16:24 2018
From: phils at caerllewys.net (Phil Stracchino)
Date: Wed, 4 Jul 2018 15:16:24 -0400
Subject: [Mailman-Users] Unable to start mailman due to time travel
In-Reply-To: <48fa0fd7-3f3b-30d5-bb66-3232002aff0a@msapiro.net>
References: <2169d08d-b5f8-9054-b4d9-87be3da935fc@caerllewys.net>
 <48fa0fd7-3f3b-30d5-bb66-3232002aff0a@msapiro.net>
Message-ID: <ee602ce5-4315-eaa9-5dc7-3cb47327ac81@caerllewys.net>

On 07/04/18 15:10, Mark Sapiro wrote:
> On 7/4/18 11:52 AM, Phil Stracchino wrote:
>> This looks like it's failing to acquire a lock file.  Investigating, I
>> discovered this:
> 
> 
> There was probably a stale lock due to Mailman not being cleanly
> terminated upon reboot. The Operation not permitted is an attempt to
> determine the status of the PID that created the lock by sending it a
> signal 0. The error may be a Gentoo thing or 'mailmanctl start' may need
> to run as root if it wasn't.


Yup, actually I'm pretty sure I know how the stale lock file happened
... I had to interrupt boot a couple of times because this is a Dell
R610 server, and it has an annoying habit of disbelieving in my KVM
switch during boot.  It can be a bit of a headache getting it to
acknowledge that it actually has a keyboard connected in time to select
the kernel I want i  grub.

>> After I deleted the errant lock files, mailman started up without
>> hesitation ...  and AGAIN created lock files timestamped in the future:
>>
>>
>> minbar:root:/var/lib/mailman/logs:22 # ls -l /var/lib/mailman/locks
>> total 8
>> -rw-rw-r-- 2 mailman mailman 50 Jul  5  2018 master-qrunner
>> -rw-rw-r-- 2 mailman mailman 50 Jul  5  2018 master-qrunner.minbar.15895
> 
> 
> This is all normal and expected. The time stamp on locks is set to that
> future time when the lock expires.


AH!  I did not know that.  Mystery resolved.  Thank you.  :)



-- 
  Phil Stracchino
  Babylon Communications
  phils at caerllewys.net
  phil at co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

From christoph at grothesque.org  Fri Jul  6 04:44:46 2018
From: christoph at grothesque.org (Christoph Groth)
Date: Fri, 06 Jul 2018 10:44:46 +0200
Subject: [Mailman-Users] spam filter setting to hold makes sender filters
 useless
Message-ID: <87po00u51d.fsf@grothesque.org>

Hello,

In the mailing list that I administrate I've set a spam filter rule that
rejects all mail that matches the header

    X-Spam-Level: \*{5,}

Any messages that match

    X-Spam-Level: \*{2,}

are set on hold.  We do this in order to avoid rejecting legitimate
mail.

However this causes problems with addresses on the
discard_these_nonmembers list (that is populated by the mailman
message-on-hold page).  If a message arrives that is set on hold due to
the second spam filter rule, it won't be checked against
discard_these_nonmembers, and the list admins are annoyed by it being
put on hold.

The only solution that I see is to manually create spam filter rules for
the most notorious names on the discard_these_nonmembers list.  Is there
a better way to keep graded response to spam and at the same time use
discard_these_nonmembers?

Thanks,
Christoph

From mark at msapiro.net  Fri Jul  6 11:32:10 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 6 Jul 2018 08:32:10 -0700
Subject: [Mailman-Users] spam filter setting to hold makes sender
 filters useless
In-Reply-To: <87po00u51d.fsf@grothesque.org>
References: <87po00u51d.fsf@grothesque.org>
Message-ID: <90dc1c34-9ed2-e59c-7300-e429fde9996c@msapiro.net>

On 7/6/18 1:44 AM, Christoph Groth wrote:
> 
> The only solution that I see is to manually create spam filter rules for
> the most notorious names on the discard_these_nonmembers list.  Is there
> a better way to keep graded response to spam and at the same time use
> discard_these_nonmembers?


Maybe.

header_filter_rules are checked before list membership and
*_these_nonmembers. You could rearrange the pipeline to put Moderate
before SpamDetect, but that would have undesirable consequences.

So I think your choices are what you suggest above or remove the
'X-Spam-Level: \*{2,}' rule from header_filter_rules and implement a
custom handler to hold messages that match that and insert that handler
in the pipeline between Moderate and Hold.

See <https://wiki.list.org/x/4030615> for info on custom handlers.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From jacques at lavignotte.org  Fri Jul  6 15:22:22 2018
From: jacques at lavignotte.org (Jacques Lavignotte)
Date: Fri, 6 Jul 2018 21:22:22 +0200
Subject: [Mailman-Users] [FIXED] Moderation impossible. Debian 9
In-Reply-To: <8d374451-c94b-2298-59fd-2a881da84aa4@msapiro.net>
References: <c057c4a8-95a2-a2cf-8798-218898dff50e@lavignotte.org>
 <8d374451-c94b-2298-59fd-2a881da84aa4@msapiro.net>
Message-ID: <e151e47c-1936-039a-135b-7a5f39777888@lavignotte.org>



Le 02/07/2018 ? 16:58, Mark Sapiro a ?crit?:

> This appears to be a Debian issue. Please see
> <https://wiki.list.org/x/12812344> and report this to Debian.

4.27. Securing Mailman's web GUI by using Secure HTTP/SSL (HTTPS)

  DEFAULT_URL_PATTERN = 'https://%s/mailman/'

and

$prefix/bin/withlist -l -a -r fix_url

fixed it (as expected)

Thanks Mark

and JFB on the side.

Jacques

-- 
GnuPg : C8F5B1E3 WeUsePGP Because privacy matters http://weusepgp.info/

From mark at msapiro.net  Tue Jul 10 21:03:06 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 10 Jul 2018 18:03:06 -0700
Subject: [Mailman-Users] Mailman 2.1.18 Security fix release
Message-ID: <49581d63-1175-3802-6433-e5021d0782d2@msapiro.net>

This is a heads up. There is a recently identified minor security issue
in Mailman 2.1.27 and earlier. This is not something that allows
permanent compromise or information leak from your site, but is
something that needs to be fixed. This issue has been given CVE ID
CVE-2018-13796.

I plan to release Mailman 2.1.28 on 24 July, 2018 along with details of
the issue and a patch to apply to earlier releases.

This is just notification of that plan for those that need to plan ahead.

Also, if you are a Mailman translator and you have any updates to your
translation, please submit them in some form prior to 24 July so they
can be incorporated in the release.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20180710/a72bcba1/attachment.sig>

From mark at msapiro.net  Tue Jul 10 23:13:16 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 10 Jul 2018 20:13:16 -0700
Subject: [Mailman-Users] correction: Mailman 2.1.28 Security fix release
In-Reply-To: <49581d63-1175-3802-6433-e5021d0782d2@msapiro.net>
References: <49581d63-1175-3802-6433-e5021d0782d2@msapiro.net>
Message-ID: <106e27ee-73ba-fc83-bb22-4d6f3d34340f@msapiro.net>

Resending with correct subject (2.1.28, not 2.1.18)

On 7/10/18 6:03 PM, Mark Sapiro wrote:
> This is a heads up. There is a recently identified minor security issue
> in Mailman 2.1.27 and earlier. This is not something that allows
> permanent compromise or information leak from your site, but is
> something that needs to be fixed. This issue has been given CVE ID
> CVE-2018-13796.
> 
> I plan to release Mailman 2.1.28 on 24 July, 2018 along with details of
> the issue and a patch to apply to earlier releases.
> 
> This is just notification of that plan for those that need to plan ahead.
> 
> Also, if you are a Mailman translator and you have any updates to your
> translation, please submit them in some form prior to 24 July so they
> can be incorporated in the release.


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20180710/372e19b8/attachment.sig>

From mark at msapiro.net  Wed Jul 11 10:02:27 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 11 Jul 2018 07:02:27 -0700
Subject: [Mailman-Users] Mailman 2.1.18 Security fix release (date
 change)
In-Reply-To: <49581d63-1175-3802-6433-e5021d0782d2@msapiro.net>
References: <49581d63-1175-3802-6433-e5021d0782d2@msapiro.net>
Message-ID: <7a73d4eb-2db6-90b8-fa23-c65a930817a2@msapiro.net>

On 7/10/18 6:03 PM, Mark Sapiro wrote:
> This is a heads up. There is a recently identified minor security issue
> in Mailman 2.1.27 and earlier. This is not something that allows
> permanent compromise or information leak from your site, but is
> something that needs to be fixed. This issue has been given CVE ID
> CVE-2018-13796.
> 
> I plan to release Mailman 2.1.28 on 24 July, 2018 along with details of
> the issue and a patch to apply to earlier releases.
> 
> This is just notification of that plan for those that need to plan ahead.


Due to changed circumstances, I am changing the planned release date
from 24 July to 23 July.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20180711/c5586154/attachment.sig>

From steven.jones at vuw.ac.nz  Thu Jul 12 18:08:32 2018
From: steven.jones at vuw.ac.nz (Steven Jones)
Date: Thu, 12 Jul 2018 22:08:32 +0000
Subject: [Mailman-Users] Is there a way to determine which lists are defunct?
Message-ID: <ME1PR01MB20818007C819521EA61413B5D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>

We have 301 lists on a RHEL6 server and want to migrate to RHEL7 and change our domain name at the same time.


So as above,


Is there a way to determine which lists are defunct?



regards

Steven

From steven.jones at vuw.ac.nz  Thu Jul 12 18:16:54 2018
From: steven.jones at vuw.ac.nz (Steven Jones)
Date: Thu, 12 Jul 2018 22:16:54 +0000
Subject: [Mailman-Users] Can the following be done and as an option
 contractors available to do it?
Message-ID: <ME1PR01MB208124457ED2CBCD915707A9D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>

Hi We have a requirement to move 301 lists (minus defunct lists) on a RHEL6 server and want to migrate to RHEL7 and change our domain name at the same time.


Our users email addresses would also need changing from the old domain to the new domain.


Moving archives, we would like to move these if possible.


We also want to run on https and use something like failtoban to reduce bruteforce attacks we are suffering from.


>From what I can read in theory most of this if not all of it is quite possible to move lists?


and also this other work?


Finally assuming we dont have time or capability to do this in-house where do I go hunting for an external contractor who can do this and document the work in an "as-built" document.


regards

Steven




From cpz at tuunq.com  Thu Jul 12 19:52:25 2018
From: cpz at tuunq.com (Carl Zwanzig)
Date: Thu, 12 Jul 2018 16:52:25 -0700
Subject: [Mailman-Users] Is there a way to determine which lists are
 defunct?
In-Reply-To: <ME1PR01MB20818007C819521EA61413B5D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>
References: <ME1PR01MB20818007C819521EA61413B5D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>
Message-ID: <709000f1-780e-26d2-0d8a-44124886654d@tuunq.com>

On 7/12/2018 3:08 PM, Steven Jones wrote:
> Is there a way to determine which lists are defunct?

Depends on how you define "defunct". No members subscribed? All members set 
"no-mail"? No messages in the last year? (You'd need the archive to look at 
that.) If the lists are archived, the last is probably the most useful and 
could probably be checked with a well-crafted "find" command.

Later,

z!


From mark at msapiro.net  Thu Jul 12 20:43:53 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 12 Jul 2018 17:43:53 -0700
Subject: [Mailman-Users] Is there a way to determine which lists are
 defunct?
In-Reply-To: <709000f1-780e-26d2-0d8a-44124886654d@tuunq.com>
References: <ME1PR01MB20818007C819521EA61413B5D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>
 <709000f1-780e-26d2-0d8a-44124886654d@tuunq.com>
Message-ID: <6362f3c4-de3b-cfe6-e53f-d889982180a1@msapiro.net>

On 7/12/18 4:52 PM, Carl Zwanzig wrote:
> On 7/12/2018 3:08 PM, Steven Jones wrote:
>> Is there a way to determine which lists are defunct?
> 
> Depends on how you define "defunct". No members subscribed? All members
> set "no-mail"? No messages in the last year? (You'd need the archive to
> look at that.) If the lists are archived, the last is probably the most
> useful and could probably be checked with a well-crafted "find" command.


Or you could see <https://www.msapiro.net/scripts/last_post.py> for a
withlist script that can display the last post time for all lists.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From cpz at tuunq.com  Thu Jul 12 20:49:23 2018
From: cpz at tuunq.com (Carl Zwanzig)
Date: Thu, 12 Jul 2018 17:49:23 -0700
Subject: [Mailman-Users] Is there a way to determine which lists are
 defunct?
In-Reply-To: <6362f3c4-de3b-cfe6-e53f-d889982180a1@msapiro.net>
References: <ME1PR01MB20818007C819521EA61413B5D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>
 <709000f1-780e-26d2-0d8a-44124886654d@tuunq.com>
 <6362f3c4-de3b-cfe6-e53f-d889982180a1@msapiro.net>
Message-ID: <59651acd-ca25-f3db-5b2a-fd93ac015624@tuunq.com>

On 7/12/2018 5:43 PM, Mark Sapiro wrote:
> Or you could see<https://www.msapiro.net/scripts/last_post.py>  for a
> withlist script that can display the last post time for all lists.

Of course Mark has a script to do this :).

z!

From charles at stremen.xs4all.nl  Fri Jul 13 05:28:20 2018
From: charles at stremen.xs4all.nl (Charles Stroom)
Date: Fri, 13 Jul 2018 11:28:20 +0200
Subject: [Mailman-Users] Is there a way to determine which lists are
 defunct?
In-Reply-To: <6362f3c4-de3b-cfe6-e53f-d889982180a1@msapiro.net>
References: <ME1PR01MB20818007C819521EA61413B5D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>
 <709000f1-780e-26d2-0d8a-44124886654d@tuunq.com>
 <6362f3c4-de3b-cfe6-e53f-d889982180a1@msapiro.net>
Message-ID: <20180713112820.4bc3b6b8@fiume7.localnet>

Thanks for https://www.msapiro.net/scripts

Charles


On Thu, 12 Jul 2018 17:43:53 -0700
Mark Sapiro <mark at msapiro.net> wrote:

> 
> 
> Or you could see <https://www.msapiro.net/scripts/last_post.py> for a
> withlist script that can display the last post time for all lists.
> 
> -- 
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/charles%40stremen.xs4all.nl


-- 
Charles Stroom
email: charles at no-spam.stremen.xs4all.nl (remove the "no-spam.")

From mark at msapiro.net  Fri Jul 13 10:04:31 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 13 Jul 2018 07:04:31 -0700
Subject: [Mailman-Users] Is there a way to determine which lists are
 defunct?
In-Reply-To: <20180713112820.4bc3b6b8@fiume7.localnet>
References: <ME1PR01MB20818007C819521EA61413B5D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>
 <709000f1-780e-26d2-0d8a-44124886654d@tuunq.com>
 <6362f3c4-de3b-cfe6-e53f-d889982180a1@msapiro.net>
 <20180713112820.4bc3b6b8@fiume7.localnet>
Message-ID: <7639a082-be5f-ffe3-e40d-22d14a69ed0a@msapiro.net>

On 7/13/18 2:28 AM, Charles Stroom wrote:
> Thanks for https://www.msapiro.net/scripts


You're welcome.

They are also mirrored at <https://fog.ccsf.edu/~msapiro/scripts/> in
case of issues with <https://www.msapiro.net/scripts>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Fri Jul 13 16:11:59 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 13 Jul 2018 13:11:59 -0700
Subject: [Mailman-Users] Can the following be done and as an option
 contractors available to do it?
In-Reply-To: <ME1PR01MB208124457ED2CBCD915707A9D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>
References: <ME1PR01MB208124457ED2CBCD915707A9D4590@ME1PR01MB2081.ausprd01.prod.outlook.com>
Message-ID: <5beca229-5628-1f06-cbc7-e92e1fb733ab@msapiro.net>

On 7/12/18 3:16 PM, Steven Jones wrote:
> Hi We have a requirement to move 301 lists (minus defunct lists) on a RHEL6 server and want to migrate to RHEL7 and change our domain name at the same time.


See <https://wiki.list.org/x/4030682> and <https://wiki.list.org/x/4030592>


> Our users email addresses would also need changing from the old domain to the new domain.


See <https://www.msapiro.net/scripts/change_member_address.py>


> Moving archives, we would like to move these if possible.


the best way is to use Mailman's bin/arch to import the existing
archives/private/LISTNAME.mbox/LISTNAME.mbox file.


> We also want to run on https and use something like failtoban to reduce bruteforce attacks we are suffering from.


See <https://wiki.list.org/x/17892007>.

Configuring fail2ban is generally straightforward, but the actual logs
and regexps depend on the nature of the attack.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From minxmertzmomo at gmail.com  Wed Jul 18 21:28:47 2018
From: minxmertzmomo at gmail.com (Matt Morgan)
Date: Wed, 18 Jul 2018 21:28:47 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address in
 "Real Name"
Message-ID: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>

On one of my lists I'm seeing some spam from non-subscribers getting
through. It appears that the trick is to put a subscriber's address in the
"real name" of the sender. E.g., this got through, without being held for
moderation, on a list with generic_nonmember_action = discard (emails of
the innocent obfuscated):

*From:* "xxx at johnxxx.com <jgl at johngreenwaltlee.com>" <enrollment at ekonek.com>
*Date:* July 18, 2018 at 5:27:24 PM CDT
*To:* <listname at server.org <osg-l at cool.conservation-us.org>>
*Subject:* *[OSG-l] No. PL-01-17923 AIC Objects Specialty Group Discussion*
*Reply-To:* My List's Name <listname at server.org
<osg-l at cool.conservation-us.org>>


Account Summary:
---------------------------
Invoice No: No. PL-01-17923
Billing Date: Jul 19, 2018
Due Date: Jul 22, 2018
Amount Due: 1,047.48
Download DOC:

etc. (I'm avoiding sharing the links that follow). xxx at johnxxx.com IS a
subscriber on the list. However enrollment at ekonek.com is not. Yet this
message went straight through, as if it had been sent by a subscriber.

I've looked at the archives of mailman-users and it looks like--from a very
old discussion--that

a) this cheap trick should not be sufficient to allow the message through
b) the content of the message as delivered to the list may not reflect the
exact contents/metadata of the message as it was sent.

Still, I don't really know what else could be going on here, or how to
investigate. Suggestions?

Thanks!
Matt

From heller at deepsoft.com  Wed Jul 18 22:10:21 2018
From: heller at deepsoft.com (Robert Heller)
Date: Wed, 18 Jul 2018 22:10:21 -0400 (EDT)
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
Message-ID: <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>

At Wed, 18 Jul 2018 21:28:47 -0400 Matt Morgan <minxmertzmomo at gmail.com> wrote:

> 
> On one of my lists I'm seeing some spam from non-subscribers getting
> through. It appears that the trick is to put a subscriber's address in the
> "real name" of the sender. E.g., this got through, without being held for
> moderation, on a list with generic_nonmember_action = discard (emails of
> the innocent obfuscated):
> 
> *From:* "xxx at johnxxx.com <jgl at johngreenwaltlee.com>" <enrollment at ekonek.com>
> *Date:* July 18, 2018 at 5:27:24 PM CDT
> *To:* <listname at server.org <osg-l at cool.conservation-us.org>>
> *Subject:* *[OSG-l] No. PL-01-17923 AIC Objects Specialty Group Discussion*
> *Reply-To:* My List's Name <listname at server.org
> <osg-l at cool.conservation-us.org>>
> 
> 
> Account Summary:
> ---------------------------
> Invoice No: No. PL-01-17923
> Billing Date: Jul 19, 2018
> Due Date: Jul 22, 2018
> Amount Due: 1,047.48
> Download DOC:


Mailman only checks the From: header and it is trivial to put any random thing 
there, even if it is false information.

OTH, the contents of the Recieved: headers contain real server names and IP 
addresses.

Very often, the mail is sent directly to a SMTP server from a random PC or 
Laptop, often from a IP address without a reverse DNS.  I have a filter rule:

Received: from.*(unknown \[\d+\.\d+\.\d+\.\d+\])

Which catches this sorts of messages.  I place them on hold, since *some* 
people use E-Mail clients that directly connect to SMTP servers from ISP IP 
addresses without reverse DNS.

> 
> etc. (I'm avoiding sharing the links that follow). xxx at johnxxx.com IS a
> subscriber on the list. However enrollment at ekonek.com is not. Yet this
> message went straight through, as if it had been sent by a subscriber.
> 
> I've looked at the archives of mailman-users and it looks like--from a very
> old discussion--that
> 
> a) this cheap trick should not be sufficient to allow the message through
> b) the content of the message as delivered to the list may not reflect the
> exact contents/metadata of the message as it was sent.
> 
> Still, I don't really know what else could be going on here, or how to
> investigate. Suggestions?
> 
> Thanks!
> Matt
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/heller%40deepsoft.com
> 
>                                                                                            

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller at deepsoft.com       -- Webhosting Services
                         

From mark at msapiro.net  Wed Jul 18 22:30:07 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 18 Jul 2018 19:30:07 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
Message-ID: <29dc51e9-3d32-f0f0-0a59-1e964a29d04c@msapiro.net>

On 07/18/2018 06:28 PM, Matt Morgan wrote:
> On one of my lists I'm seeing some spam from non-subscribers getting
> through. It appears that the trick is to put a subscriber's address in the
> "real name" of the sender. E.g., this got through, without being held for
> moderation, on a list with generic_nonmember_action = discard (emails of
> the innocent obfuscated):
> 
> *From:* "xxx at johnxxx.com <jgl at johngreenwaltlee.com>" <enrollment at ekonek.com>


I'm not sure what the actual incoming From: looked like. I'm sure the
asterisks in *From:* are some MUA's bolding artifact, but that
notwithstanding, if the header was

From: "xxx at johnxxx.com <jgl at johngreenwaltlee.com>" <enrollment at ekonek.com>

Mailman will parse that as

real name: 'xxx at johnxxx.com <jgl at johngreenwaltlee.com>'
address: 'enrollment at ekonek.com'

and the only address checked for list membership will be
enrollment at ekonek.com

In any case, if you haven't changed the setting of SENDER_HEADERS in
mm_cfg.py, Mailman will consider a post to be from a list member if any
of the From: header, the envelope sender, the Reply-To: header or the
Sender: header contains the member address as an address, not as a real
name.

It is trivial to spoof a member address in one of those places.

As far as what happened in this case, I can't say without seeing the
original message as received by Mailman before various headers were
munged and the post sent to the list.

If you want to diagnose this, you can temporarily add a local file to
the alias for the list posting address to capture the incoming mail, at
least if mailman's delivery is via aliases.

I.e., if you currently have an alias like

listname:   "|/path/to/mail/mailman post virt"

add a file as in

listname:   "|/path/to/mail/mailman post listname"
      /path/to/file

Then the MTA will save the message to 'file' as well as delivering it to
mailman.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Wed Jul 18 22:33:20 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 18 Jul 2018 19:33:20 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
Message-ID: <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>

On 07/18/2018 07:10 PM, Robert Heller wrote:
> 
> Mailman only checks the From: header...


Not true. See my other reply in this thread.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From heller at deepsoft.com  Thu Jul 19 08:16:40 2018
From: heller at deepsoft.com (Robert Heller)
Date: Thu, 19 Jul 2018 08:16:40 -0400 (EDT)
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com> 
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com> 
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
Message-ID: <20180719121640.D844026C2470@sharky3.deepsoft.com>

At Wed, 18 Jul 2018 19:33:20 -0700 Mark Sapiro <mark at msapiro.net> wrote:

> 
> On 07/18/2018 07:10 PM, Robert Heller wrote:
> > 
> > Mailman only checks the From: header...
> 
> 
> Not true. See my other reply in this thread.

I mean it does not check things like the Received: headers *by default*. If
the email part of the From: header is a list member address, Mailman will
consider that the mail is from that member and pass the message on to the
list, *even if the From: header is spoofed*. I expect that this is what
happening with the OP. It is a common spammer hack: somehow get a list of
member addresses (or really hack a member's E-Mail accoung or PC and go from
there).

Yes, Mail mail can be configured to check other headers, but this requires 
some configuration settings.

> 

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller at deepsoft.com       -- Webhosting Services
                                                                                                        

From scott at qth.com  Thu Jul 19 10:25:11 2018
From: scott at qth.com (Scott Neader)
Date: Thu, 19 Jul 2018 09:25:11 -0500
Subject: [Mailman-Users] digest_send_periodic not sending digests daily when
 set to Yes
Message-ID: <CAPLm_7CLtfPoHXgTorUzfkJ8O+ECooZDUs-p896L6zrVAGZueA@mail.gmail.com>

Hi all.  Running Mailman v2.1.26 on CentOS 6.  When the list
has digest_send_periodic set to Yes, and a size of 100kb, users are seeing
digests with upwards of a week's worth of mail, if the list has been
quiet.  i.e. it seems to be respecting the 100kb limit, but not
the digest_send_periodic setting as Yes (to send the digest daily, no
matter the size.)  I've confirmed that the senddigests cron runs every day
at Noon.

Any ideas on other things to check?

- Scott

From gtaylor at tnetconsulting.net  Thu Jul 19 11:02:22 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Thu, 19 Jul 2018 09:02:22 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180719121640.D844026C2470@sharky3.deepsoft.com>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
 <20180719121640.D844026C2470@sharky3.deepsoft.com>
Message-ID: <8d82f1b4-a133-e1f6-c972-cd0470911550@spamtrap.tnetconsulting.net>

On 07/19/2018 06:16 AM, Robert Heller wrote:
> I mean it does not check things like the Received: headers*by default*. If 
> the email part of the From: header is a list member address, Mailman 
> will consider that the mail is from that member and pass the message on 
> to the list,*even if the From: header is spoofed*. I expect that this 
> is what happening with the OP. It is a common spammer hack: somehow get 
> a list of member addresses (or really hack a member's E-Mail accoung or 
> PC and go from there).
> 
> Yes, Mail mail can be configured to check other headers, but this requires 
> some configuration settings.

I have often wondered about enhancing Mailman, or augmenting it with a 
milter, to be able to test the SMTP envelope from, to, and body content 
against list parameters and be able to reject messages during the SMTP 
delivery transaction.

IMHO it's a bit more difficult to spoof SMTP envelope details and bypass 
SMTP level detections.  This does assume that the sending domain does 
publish the required info and that receiving mail servers actually 
filter based on that.



-- 
Grant. . . .
unix || die


From mark at msapiro.net  Thu Jul 19 13:16:16 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Jul 2018 10:16:16 -0700
Subject: [Mailman-Users] digest_send_periodic not sending digests daily
 when set to Yes
In-Reply-To: <CAPLm_7CLtfPoHXgTorUzfkJ8O+ECooZDUs-p896L6zrVAGZueA@mail.gmail.com>
References: <CAPLm_7CLtfPoHXgTorUzfkJ8O+ECooZDUs-p896L6zrVAGZueA@mail.gmail.com>
Message-ID: <24b04f3e-2800-0935-5e7e-d0196a4d0ce8@msapiro.net>

On 07/19/2018 07:25 AM, Scott Neader wrote:
> Hi all.  Running Mailman v2.1.26 on CentOS 6.  When the list
> has digest_send_periodic set to Yes, and a size of 100kb, users are seeing
> digests with upwards of a week's worth of mail, if the list has been
> quiet.  i.e. it seems to be respecting the 100kb limit, but not
> the digest_send_periodic setting as Yes (to send the digest daily, no
> matter the size.)  I've confirmed that the senddigests cron runs every day
> at Noon.
> 
> Any ideas on other things to check?


What exactly is the senddigests command run by cron?

Is it being run as the Mailman user? I.e. is it in Mailman's user
crontab or if in /etc/cron.d/ is the user field the Mailman user?

What happens if you run cron/senddigests manually as the Mailman user?

What's in Mailman's error log at noon or what gets posted to the site
lost at noon. I.e., if the cron is in Mailman's crontab and there's no
MAILTO= in the crontab, errors get mailed to the Mailman user and
normally this gets delivered to the site list.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jul 19 13:25:01 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Jul 2018 10:25:01 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180719121640.D844026C2470@sharky3.deepsoft.com>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
 <20180719121640.D844026C2470@sharky3.deepsoft.com>
Message-ID: <f84ed2d4-9ff6-1224-7e31-22207e2b7e84@msapiro.net>

On 07/19/2018 05:16 AM, Robert Heller wrote:
> At Wed, 18 Jul 2018 19:33:20 -0700 Mark Sapiro <mark at msapiro.net> wrote:
> 
>>
>> On 07/18/2018 07:10 PM, Robert Heller wrote:
>>>
>>> Mailman only checks the From: header...
>>
>>
>> Not true. See my other reply in this thread.
> 
> I mean it does not check things like the Received: headers *by default*. If
> the email part of the From: header is a list member address, Mailman will
> consider that the mail is from that member and pass the message on to the
> list, *even if the From: header is spoofed*. I expect that this is what
> happening with the OP. It is a common spammer hack: somehow get a list of
> member addresses (or really hack a member's E-Mail accoung or PC and go from
> there).
> 
> Yes, Mail mail can be configured to check other headers, but this requires 
> some configuration settings.


My point is that standard, default Mailman checks not only the From:
header for list member addresses, it also checks the envelope sender and
the Reply-To: and Sender: headers.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jul 19 13:28:24 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Jul 2018 10:28:24 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <8d82f1b4-a133-e1f6-c972-cd0470911550@spamtrap.tnetconsulting.net>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
 <20180719121640.D844026C2470@sharky3.deepsoft.com>
 <8d82f1b4-a133-e1f6-c972-cd0470911550@spamtrap.tnetconsulting.net>
Message-ID: <1592695e-fabf-8c73-c069-76274b4eacc5@msapiro.net>

On 07/19/2018 08:02 AM, Grant Taylor via Mailman-Users wrote:
> 
> I have often wondered about enhancing Mailman, or augmenting it with a
> milter, to be able to test the SMTP envelope from, to, and body content
> against list parameters and be able to reject messages during the SMTP
> delivery transaction.


You might be interested in
<http://ftp.tummy.com/pub/tummy/tummailmanmember/>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From heller at deepsoft.com  Thu Jul 19 13:44:03 2018
From: heller at deepsoft.com (Robert Heller)
Date: Thu, 19 Jul 2018 13:44:03 -0400 (EDT)
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <f84ed2d4-9ff6-1224-7e31-22207e2b7e84@msapiro.net>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com> 
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com> 
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net> 
 <20180719121640.D844026C2470@sharky3.deepsoft.com> 
 <f84ed2d4-9ff6-1224-7e31-22207e2b7e84@msapiro.net>
Message-ID: <20180719174403.7243326C2653@sharky3.deepsoft.com>

At Thu, 19 Jul 2018 10:25:01 -0700 Mark Sapiro <mark at msapiro.net> wrote:

> 
> On 07/19/2018 05:16 AM, Robert Heller wrote:
> > At Wed, 18 Jul 2018 19:33:20 -0700 Mark Sapiro <mark at msapiro.net> wrote:
> > 
> >>
> >> On 07/18/2018 07:10 PM, Robert Heller wrote:
> >>>
> >>> Mailman only checks the From: header...
> >>
> >>
> >> Not true. See my other reply in this thread.
> > 
> > I mean it does not check things like the Received: headers *by default*. If
> > the email part of the From: header is a list member address, Mailman will
> > consider that the mail is from that member and pass the message on to the
> > list, *even if the From: header is spoofed*. I expect that this is what
> > happening with the OP. It is a common spammer hack: somehow get a list of
> > member addresses (or really hack a member's E-Mail accoung or PC and go from
> > there).
> > 
> > Yes, Mail mail can be configured to check other headers, but this requires 
> > some configuration settings.
> 
> 
> My point is that standard, default Mailman checks not only the From:
> header for list member addresses, it also checks the envelope sender and
> the Reply-To: and Sender: headers.

All of which can be spoofed.  Mailman does not make any checks of the 
"Received:" headers (where the bogosity of the other headers can be determined 
or can flag messages as containing possibly spoofed headers).

> 

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller at deepsoft.com       -- Webhosting Services
                                      

From jimpop at domainmail.org  Thu Jul 19 14:53:52 2018
From: jimpop at domainmail.org (Jim Popovitch)
Date: Thu, 19 Jul 2018 18:53:52 +0000
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <1592695e-fabf-8c73-c069-76274b4eacc5@msapiro.net>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
 <20180719121640.D844026C2470@sharky3.deepsoft.com>
 <8d82f1b4-a133-e1f6-c972-cd0470911550@spamtrap.tnetconsulting.net>
 <1592695e-fabf-8c73-c069-76274b4eacc5@msapiro.net>
Message-ID: <A9ED84A0-815A-4539-AAB6-57215726E138@domainmail.org>

On July 19, 2018 5:28:24 PM UTC, Mark Sapiro <mark at msapiro.net> wrote:
>On 07/19/2018 08:02 AM, Grant Taylor via Mailman-Users wrote:
>> 
>> I have often wondered about enhancing Mailman, or augmenting it with
>a
>> milter, to be able to test the SMTP envelope from, to, and body
>content
>> against list parameters and be able to reject messages during the
>SMTP
>> delivery transaction.
>
>
>You might be interested in
><http://ftp.tummy.com/pub/tummy/tummailmanmember/>.

Here's an alternate take on the same thing that I wrote a couple years back.





From jimpop at domainmail.org  Thu Jul 19 14:55:12 2018
From: jimpop at domainmail.org (Jim Popovitch)
Date: Thu, 19 Jul 2018 18:55:12 +0000
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <A9ED84A0-815A-4539-AAB6-57215726E138@domainmail.org>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
 <20180719121640.D844026C2470@sharky3.deepsoft.com>
 <8d82f1b4-a133-e1f6-c972-cd0470911550@spamtrap.tnetconsulting.net>
 <1592695e-fabf-8c73-c069-76274b4eacc5@msapiro.net>
 <A9ED84A0-815A-4539-AAB6-57215726E138@domainmail.org>
Message-ID: <E051F2E4-CD6A-4F8C-B38F-9E60B616E70D@domainmail.org>

On July 19, 2018 6:53:52 PM UTC, Jim Popovitch <jimpop at domainmail.org> wrote:
>On July 19, 2018 5:28:24 PM UTC, Mark Sapiro <mark at msapiro.net> wrote:
>>On 07/19/2018 08:02 AM, Grant Taylor via Mailman-Users wrote:
>>> 
>>> I have often wondered about enhancing Mailman, or augmenting it with
>>a
>>> milter, to be able to test the SMTP envelope from, to, and body
>>content
>>> against list parameters and be able to reject messages during the
>>SMTP
>>> delivery transaction.
>>
>>
>>You might be interested in
>><http://ftp.tummy.com/pub/tummy/tummailmanmember/>.
>
>Here's an alternate take on the same thing that I wrote a couple years
>back.

With link this time!

https://code.launchpad.net/~jimpop/mailman/check_subscriber

-Jim P.



From gtaylor at tnetconsulting.net  Thu Jul 19 16:17:55 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Thu, 19 Jul 2018 14:17:55 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180719174403.7243326C2653@sharky3.deepsoft.com>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
 <20180719121640.D844026C2470@sharky3.deepsoft.com>
 <f84ed2d4-9ff6-1224-7e31-22207e2b7e84@msapiro.net>
 <20180719174403.7243326C2653@sharky3.deepsoft.com>
Message-ID: <c5d1335d-0762-8a85-3257-239d5e2e46d6@spamtrap.tnetconsulting.net>

On 07/19/2018 11:44 AM, Robert Heller wrote:
> All of which can be spoofed.

Yes.  Just about everything can be spoofed to some degree.  It really 
depends on what information the owner of the purported sending domain 
publishes and what filtering / consumption of said information the 
receiving server exercises.

I personally feel like Mailman, and many other similar things, should 
sit behind an external / edge SMTP server that does some of the heavy 
lifting and provides detection of and possibly protection against many 
spoofs.

> Mailman does not make any checks of the "Received:" headers (where the 
> bogosity of the other headers can be determined or can flag messages as 
> containing possibly spoofed headers).

I agree that there is some data in the Received: headers that may 
indicate a problem.  But such information is difficult to consistently / 
reliably / accurately extract or parse /without/ false positives.  It 
can also be difficult to correlate information across headers and 
determine what should and should not be allowed.  Let's not forget that 
it's equally easy to spoof Received: headers as it is to spoof other 
headers.  }:-)



-- 
Grant. . . .
unix || die


From johnl at taugh.com  Thu Jul 19 17:11:33 2018
From: johnl at taugh.com (John Levine)
Date: 19 Jul 2018 17:11:33 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <c5d1335d-0762-8a85-3257-239d5e2e46d6@spamtrap.tnetconsulting.net>
Message-ID: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>

In article <c5d1335d-0762-8a85-3257-239d5e2e46d6 at spamtrap.tnetconsulting.net> you write:
>Yes.  Just about everything can be spoofed to some degree.  It really 
>depends on what information the owner of the purported sending domain 
>publishes and what filtering / consumption of said information the 
>receiving server exercises.

Well, you know, this is what DMARC is intended to address.  While
DMARC checks on mail that has passed through mailing lists has all
sorts of well known problems, doing DMARC checks on mail that arrives
at a list server would be pretty benign.  It's pretty rare for the
path from a user to the mailman server to do things that would cause
DMARC fails.

If you want to reinvent DMARC, you could add an option to say that all
submissions from me must have a DKIM signature or validated SPF from
domain X, where X would usually default to the domain in your e-mail
address.

R's,
John

From gtaylor at tnetconsulting.net  Thu Jul 19 17:37:21 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Thu, 19 Jul 2018 15:37:21 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
Message-ID: <9db2d852-2bbb-5d01-3253-f350ac64b5de@spamtrap.tnetconsulting.net>

On 07/19/2018 03:11 PM, John Levine wrote:
> Well, you know, this is what DMARC is intended to address.  While DMARC 
> checks on mail that has passed through mailing lists has all sorts of 
> well known problems, doing DMARC checks on mail that arrives at a list 
> server would be pretty benign.  It's pretty rare for the path from a 
> user to the mailman server to do things that would cause DMARC fails.

Yep, that's what I was referring to.

> If you want to reinvent DMARC, you could add an option to say that all 
> submissions from me must have a DKIM signature or validated SPF from 
> domain X, where X would usually default to the domain in your e-mail 
> address.

I have no desire to reinvent DMARC (or DKIM, SPF, etc.).

I'd argue that it's best to:

1)  Do all the typical DMARC, DKIM, SPF, etc. filtering on email inbound 
to the mail server.
2)  Strip DKIM (related) headers from messages going into Mailman.
3)  ...Mailman w/ DMARC friendly settings...
4)  Apply new DKIM signatures as messages leave the mail server.



-- 
Grant. . . .
unix || die


From mark at msapiro.net  Thu Jul 19 17:40:12 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Jul 2018 14:40:12 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
Message-ID: <1ca714d0-da89-aa23-d247-4faa2133b591@msapiro.net>

On 07/19/2018 02:11 PM, John Levine wrote:
> In article <c5d1335d-0762-8a85-3257-239d5e2e46d6 at spamtrap.tnetconsulting.net> you write:
>> Yes.  Just about everything can be spoofed to some degree.  It really 
>> depends on what information the owner of the purported sending domain 
>> publishes and what filtering / consumption of said information the 
>> receiving server exercises.
> 
> Well, you know, this is what DMARC is intended to address.


Actually, DMARC is intended to address spoofing of domains and needs to
be configured by the domain owner publishing a DMARC policy.

DMARC checks won't help prevent posts that spoof a member address unless
every list member's domain publishes a DMARC policy of quarantine or
reject, and even then it only checks the From: domain and not the domain
of other addresses Mailman might use to determine list membership.

Further, a post with spoofed local part sent by someone in the same
domain might pass DMARC if sent via the domain's servers.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From heller at deepsoft.com  Thu Jul 19 18:01:30 2018
From: heller at deepsoft.com (Robert Heller)
Date: Thu, 19 Jul 2018 18:01:30 -0400 (EDT)
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <c5d1335d-0762-8a85-3257-239d5e2e46d6@spamtrap.tnetconsulting.net>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com> 
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com> 
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net> 
 <20180719121640.D844026C2470@sharky3.deepsoft.com> 
 <f84ed2d4-9ff6-1224-7e31-22207e2b7e84@msapiro.net> 
 <20180719174403.7243326C2653@sharky3.deepsoft.com> 
 <c5d1335d-0762-8a85-3257-239d5e2e46d6@spamtrap.tnetconsulting.net>
Message-ID: <20180719220130.61B0926C2B37@sharky3.deepsoft.com>

At Thu, 19 Jul 2018 14:17:55 -0600 Grant Taylor <gtaylor at tnetconsulting.net> wrote:

> 
> 
> Content-Language: en-US
> 
> On 07/19/2018 11:44 AM, Robert Heller wrote:
> > All of which can be spoofed.
> 
> Yes.  Just about everything can be spoofed to some degree.  It really 
> depends on what information the owner of the purported sending domain 
> publishes and what filtering / consumption of said information the 
> receiving server exercises.
> 
> I personally feel like Mailman, and many other similar things, should 
> sit behind an external / edge SMTP server that does some of the heavy 
> lifting and provides detection of and possibly protection against many 
> spoofs.

Yes, of course.  

> 
> > Mailman does not make any checks of the "Received:" headers (where the 
> > bogosity of the other headers can be determined or can flag messages as 
> > containing possibly spoofed headers).
> 
> I agree that there is some data in the Received: headers that may 
> indicate a problem.  But such information is difficult to consistently / 
> reliably / accurately extract or parse /without/ false positives.  It 
> can also be difficult to correlate information across headers and 
> determine what should and should not be allowed.  Let's not forget that 
> it's equally easy to spoof Received: headers as it is to spoof other 
> headers.  }:-)

I have found that just "holding" messages from an non-reversed DNS "server" 
(eg "Received: ... from ... unknown (nnn.nnn.nnn.nnn)"), results in only a 
small number of false positives.   Better a *few* false positives, than tons 
of spam.  Firewalling IP blocks, either with an actual firewall (iptables) or 
via access control, helps a great deal.

> 
> 
> 

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller at deepsoft.com       -- Webhosting Services
                                                                  

From mark at msapiro.net  Thu Jul 19 18:16:16 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Jul 2018 15:16:16 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <9db2d852-2bbb-5d01-3253-f350ac64b5de@spamtrap.tnetconsulting.net>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
 <9db2d852-2bbb-5d01-3253-f350ac64b5de@spamtrap.tnetconsulting.net>
Message-ID: <c50efe06-5e61-22e4-79bd-1c1de048703e@msapiro.net>

On 07/19/2018 02:37 PM, Grant Taylor via Mailman-Users wrote:
> 
> I'd argue that it's best to:
> 
> 1)? Do all the typical DMARC, DKIM, SPF, etc. filtering on email inbound
> to the mail server.
> 2)? Strip DKIM (related) headers from messages going into Mailman.


Mailman can be configured to remove DKIM related headers from incoming
mail before sending. When first implemented, this was done
unconditionally. There were strenuous objections, see the thread at
<https://mail.python.org/pipermail/mailman-developers/2007-February/019346.html>,
and removal was made conditional on REMOVE_DKIM_HEADERS which defaults
to No.

The bottom line is that the DKIM standard (RFC 6376) says that invalid
signatures SHOULD NOT be treated differently fro no signature, and
people feel the invalid signature may have forensic value.


> 3)? ...Mailman w/ DMARC friendly settings...
> 4)? Apply new DKIM signatures as messages leave the mail server.


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From gtaylor at tnetconsulting.net  Thu Jul 19 18:31:51 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Thu, 19 Jul 2018 16:31:51 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <c50efe06-5e61-22e4-79bd-1c1de048703e@msapiro.net>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
 <9db2d852-2bbb-5d01-3253-f350ac64b5de@spamtrap.tnetconsulting.net>
 <c50efe06-5e61-22e4-79bd-1c1de048703e@msapiro.net>
Message-ID: <89bf64b1-1edf-82f1-790b-b85e6831eaa5@spamtrap.tnetconsulting.net>

On 07/19/2018 04:16 PM, Mark Sapiro wrote:
> Mailman can be configured to remove DKIM related headers from 
> incoming mail before sending.

ACK

I'm lumping various in as well, which I'm not aware of Mailman being 
able to remove.

Authentication-Results:

I think there are others that fall into the same category, but I don't 
recall them at the moment.

> When first implemented, this was done unconditionally. There 
> were strenuous objections, see the thread at 
> <https://mail.python.org/pipermail/mailman-developers/2007-February/019346.html>, 
> and removal was made conditional on REMOVE_DKIM_HEADERS which defaults 
> to No.

ACK

> The bottom line is that the DKIM standard (RFC 6376) says that invalid 
> signatures SHOULD NOT be treated differently fro no signature, and people 
> feel the invalid signature may have forensic value.

I agree that headers should not be modified between the sender and the 
recipient.  The catch is, I believe the mailing list is the recipient 
and a subsequent (re)sender of a very similar but different message.  As 
such, I think that DKIM (and related) headers in a message going to a 
mailing list are unrelated to DKIM (and related) headers in a message 
coming from a mailing list.



-- 
Grant. . . .
unix || die


From johnl at taugh.com  Thu Jul 19 18:43:24 2018
From: johnl at taugh.com (John Levine)
Date: 19 Jul 2018 18:43:24 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <1ca714d0-da89-aa23-d247-4faa2133b591@msapiro.net>
Message-ID: <20180719224324.717D3200281177@dhcp-85af.meeting.ietf.org>

In article <1ca714d0-da89-aa23-d247-4faa2133b591 at msapiro.net> you write:
>DMARC checks won't help prevent posts that spoof a member address unless
>every list member's domain publishes a DMARC policy of quarantine or
>reject, and even then it only checks the From: domain and not the domain
>of other addresses Mailman might use to determine list membership.
>
>Further, a post with spoofed local part sent by someone in the same
>domain might pass DMARC if sent via the domain's servers.

That's all true, and if you want bullet proof spoof resistance, you'd
have to register PGP or S/MIME keys for the subscriber and require
that she sign all her mail.

On the other hand, a lot of domains do DKIM signing or publish SPF,
and the vast majority of fake From: headers I see are from botnets,
not malicious users down the hall from the victim.  So if someone is
experiencing a lot of botnet spoofage, a setting to say that a user's
mail will be authenticated by SPF or DKIM from domain X would get you
about 90% of the effect of S/MIME signing everything with 10% of the
grief.

R's,
John

From minxmertzmomo at gmail.com  Thu Jul 19 18:51:42 2018
From: minxmertzmomo at gmail.com (Matt Morgan)
Date: Thu, 19 Jul 2018 18:51:42 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180719224324.717D3200281177@dhcp-85af.meeting.ietf.org>
References: <1ca714d0-da89-aa23-d247-4faa2133b591@msapiro.net>
 <20180719224324.717D3200281177@dhcp-85af.meeting.ietf.org>
Message-ID: <CABTRqU71_LN=i9Ngsdp_eJvW8e47S6H=6ELPbyXz1LJcPVg7Ow@mail.gmail.com>

Thanks, everyone, for the thoughtful comments on my tiny little spam
problem! I've returned from my day job and will look at Mark's diagnosis
suggestions.

Best,
Matt

On Thu, Jul 19, 2018 at 6:43 PM, John Levine <johnl at taugh.com> wrote:

> In article <1ca714d0-da89-aa23-d247-4faa2133b591 at msapiro.net> you write:
> >DMARC checks won't help prevent posts that spoof a member address unless
> >every list member's domain publishes a DMARC policy of quarantine or
> >reject, and even then it only checks the From: domain and not the domain
> >of other addresses Mailman might use to determine list membership.
> >
> >Further, a post with spoofed local part sent by someone in the same
> >domain might pass DMARC if sent via the domain's servers.
>
> That's all true, and if you want bullet proof spoof resistance, you'd
> have to register PGP or S/MIME keys for the subscriber and require
> that she sign all her mail.
>
> On the other hand, a lot of domains do DKIM signing or publish SPF,
> and the vast majority of fake From: headers I see are from botnets,
> not malicious users down the hall from the victim.  So if someone is
> experiencing a lot of botnet spoofage, a setting to say that a user's
> mail will be authenticated by SPF or DKIM from domain X would get you
> about 90% of the effect of S/MIME signing everything with 10% of the
> grief.
>
> R's,
> John
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/
> mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/
> minxmertzmomo%40gmail.com
>

From minxmertzmomo at gmail.com  Thu Jul 19 18:55:03 2018
From: minxmertzmomo at gmail.com (Matt Morgan)
Date: Thu, 19 Jul 2018 18:55:03 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <CABTRqU71_LN=i9Ngsdp_eJvW8e47S6H=6ELPbyXz1LJcPVg7Ow@mail.gmail.com>
References: <1ca714d0-da89-aa23-d247-4faa2133b591@msapiro.net>
 <20180719224324.717D3200281177@dhcp-85af.meeting.ietf.org>
 <CABTRqU71_LN=i9Ngsdp_eJvW8e47S6H=6ELPbyXz1LJcPVg7Ow@mail.gmail.com>
Message-ID: <CABTRqU56gTdoBt-Yj-Jr0LNrzOdUeyXkv2FvyLTcJWGPwS=oLA@mail.gmail.com>

BTW I'm kind of flummoxed that in Mark's mail he sees the
jgl at johngreenwaltlee.com address, because that's exactly what I deleted and
replaced with the obfuscated "xxxjohnxxx.com." In what I wrote, that real
email address *did not appear*. !@#$% gmail.

On Thu, Jul 19, 2018 at 6:51 PM, Matt Morgan <minxmertzmomo at gmail.com>
wrote:

> Thanks, everyone, for the thoughtful comments on my tiny little spam
> problem! I've returned from my day job and will look at Mark's diagnosis
> suggestions.
>
> Best,
> Matt
>
> On Thu, Jul 19, 2018 at 6:43 PM, John Levine <johnl at taugh.com> wrote:
>
>> In article <1ca714d0-da89-aa23-d247-4faa2133b591 at msapiro.net> you write:
>> >DMARC checks won't help prevent posts that spoof a member address unless
>> >every list member's domain publishes a DMARC policy of quarantine or
>> >reject, and even then it only checks the From: domain and not the domain
>> >of other addresses Mailman might use to determine list membership.
>> >
>> >Further, a post with spoofed local part sent by someone in the same
>> >domain might pass DMARC if sent via the domain's servers.
>>
>> That's all true, and if you want bullet proof spoof resistance, you'd
>> have to register PGP or S/MIME keys for the subscriber and require
>> that she sign all her mail.
>>
>> On the other hand, a lot of domains do DKIM signing or publish SPF,
>> and the vast majority of fake From: headers I see are from botnets,
>> not malicious users down the hall from the victim.  So if someone is
>> experiencing a lot of botnet spoofage, a setting to say that a user's
>> mail will be authenticated by SPF or DKIM from domain X would get you
>> about 90% of the effect of S/MIME signing everything with 10% of the
>> grief.
>>
>> R's,
>> John
>> ------------------------------------------------------
>> Mailman-Users mailing list Mailman-Users at python.org
>> https://mail.python.org/mailman/listinfo/mailman-users
>> Mailman FAQ: http://wiki.list.org/x/AgA3
>> Security Policy: http://wiki.list.org/x/QIA9
>> Searchable Archives: http://www.mail-archive.com/ma
>> ilman-users%40python.org/
>> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/minxme
>> rtzmomo%40gmail.com
>>
>
>

From phils at caerllewys.net  Thu Jul 19 18:59:59 2018
From: phils at caerllewys.net (Phil Stracchino)
Date: Thu, 19 Jul 2018 18:59:59 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
Message-ID: <dedbcda0-7399-d932-c2b9-e784f73daeb1@caerllewys.net>

On 07/19/18 17:11, John Levine wrote:
> In article <c5d1335d-0762-8a85-3257-239d5e2e46d6 at spamtrap.tnetconsulting.net> you write:
>> Yes.  Just about everything can be spoofed to some degree.  It really 
>> depends on what information the owner of the purported sending domain 
>> publishes and what filtering / consumption of said information the 
>> receiving server exercises.
> 
> Well, you know, this is what DMARC is intended to address.  While
> DMARC checks on mail that has passed through mailing lists has all
> sorts of well known problems, doing DMARC checks on mail that arrives
> at a list server would be pretty benign.  It's pretty rare for the
> path from a user to the mailman server to do things that would cause
> DMARC fails.


Actually, mailing lists and other redistribution are among the places
DMARC notably breaks.  The real answer, which was created for this
purpose, is ARC (Authenticated Received Chain).  That is designed from
the start to pass through mailing lists unbroken.


(Or so I'm told.)


-- 
  Phil Stracchino
  Babylon Communications
  phils at caerllewys.net
  phil at co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

From mark at msapiro.net  Thu Jul 19 19:27:25 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Jul 2018 16:27:25 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <dedbcda0-7399-d932-c2b9-e784f73daeb1@caerllewys.net>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
 <dedbcda0-7399-d932-c2b9-e784f73daeb1@caerllewys.net>
Message-ID: <08abe617-2c99-c6e0-70e1-b098abae631f@msapiro.net>

On 07/19/2018 03:59 PM, Phil Stracchino wrote:
> 
> Actually, mailing lists and other redistribution are among the places
> DMARC notably breaks.  The real answer, which was created for this
> purpose, is ARC (Authenticated Received Chain).  That is designed from
> the start to pass through mailing lists unbroken.


Yes, ARC is designed for this and we are working on implementing ARC for
Mailman 3 but not 2.1.

ARC is a way that that I as an intermediary can say that I certify that
the message I received passed DMARC, but I transformed it in a way that
will cause DMARC to fail, but if my signature validates, downstream
should accept that DMARC passed.

The problem is downstream has to trust me. If I'm gmail.com, I'll
probably be trusted. If I'm msapiro.net, probably not. Python.org, who
knows.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From phils at caerllewys.net  Thu Jul 19 19:46:25 2018
From: phils at caerllewys.net (Phil Stracchino)
Date: Thu, 19 Jul 2018 19:46:25 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <08abe617-2c99-c6e0-70e1-b098abae631f@msapiro.net>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
 <dedbcda0-7399-d932-c2b9-e784f73daeb1@caerllewys.net>
 <08abe617-2c99-c6e0-70e1-b098abae631f@msapiro.net>
Message-ID: <20aa2a49-b66c-a4d9-4b89-2cffb8b9b1bc@caerllewys.net>

On 07/19/18 19:27, Mark Sapiro wrote:
> On 07/19/2018 03:59 PM, Phil Stracchino wrote:
>>
>> Actually, mailing lists and other redistribution are among the places
>> DMARC notably breaks.  The real answer, which was created for this
>> purpose, is ARC (Authenticated Received Chain).  That is designed from
>> the start to pass through mailing lists unbroken.
> 
> 
> Yes, ARC is designed for this and we are working on implementing ARC for
> Mailman 3 but not 2.1.

I am, by the way, eagerly waiting for Gentoo to unmask Mailman 3 ...   :)



-- 
  Phil Stracchino
  Babylon Communications
  phils at caerllewys.net
  phil at co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

From mark at msapiro.net  Thu Jul 19 20:22:28 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Jul 2018 17:22:28 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <89bf64b1-1edf-82f1-790b-b85e6831eaa5@spamtrap.tnetconsulting.net>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
 <9db2d852-2bbb-5d01-3253-f350ac64b5de@spamtrap.tnetconsulting.net>
 <c50efe06-5e61-22e4-79bd-1c1de048703e@msapiro.net>
 <89bf64b1-1edf-82f1-790b-b85e6831eaa5@spamtrap.tnetconsulting.net>
Message-ID: <1f035d2a-e278-df7b-abd9-955c6f82b2bb@msapiro.net>

On 07/19/2018 03:31 PM, Grant Taylor via Mailman-Users wrote:
> 
> I'm lumping various in as well, which I'm not aware of Mailman being
> able to remove.
> 
> Authentication-Results:
> 
> I think there are others that fall into the same category, but I don't
> recall them at the moment.


If Mailman is asked to remove or replace DKIM headers, the headers
affected are DomainKey-Signature, DKIM-Signature and Authentication-Results.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jul 19 20:40:39 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Jul 2018 17:40:39 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <CABTRqU56gTdoBt-Yj-Jr0LNrzOdUeyXkv2FvyLTcJWGPwS=oLA@mail.gmail.com>
References: <1ca714d0-da89-aa23-d247-4faa2133b591@msapiro.net>
 <20180719224324.717D3200281177@dhcp-85af.meeting.ietf.org>
 <CABTRqU71_LN=i9Ngsdp_eJvW8e47S6H=6ELPbyXz1LJcPVg7Ow@mail.gmail.com>
 <CABTRqU56gTdoBt-Yj-Jr0LNrzOdUeyXkv2FvyLTcJWGPwS=oLA@mail.gmail.com>
Message-ID: <27b5b1f4-ab8d-d810-3ebb-01007e3cfcf3@msapiro.net>

On 07/19/2018 03:55 PM, Matt Morgan wrote:
> BTW I'm kind of flummoxed that in Mark's mail he sees the
> jgl at johngreenwaltlee.com address, because that's exactly what I deleted and
> replaced with the obfuscated "xxxjohnxxx.com." In what I wrote, that real
> email address *did not appear*. !@#$% gmail.


It appeared to me that those addresses in both the From: and Reply-To:
headers were artifacts of some MUA. I didn't want to say anything about
that because I didn't want to draw attention to something you were
probably trying to not expose.

I suspect the following scenario is responsible:

You compose an email using an HTML (rich-text) editor. The editor
represents 'user at example.com' as '<a
href="mailto:user at example.com">user at example.com</a>' in the raw html it
is composing. I.e., it is a hyperlink.

You obfuscate the address to 'other at other.example.com'. This changes the
html to '<a href="mailto:user at example.com">other at other.example.com</a>'
so you see 'other at other.example.com' in your composition window, but it
is still a mailto link to 'user at example.com'

You send the message and your MUA creates a plain text alternative in
which the html hyperlink is rendered as 'other at other.example.com
<user at example.com>'.

The list's content filtering selects the plain text alternative for
delivery to the list.

I tell my users to ALWAYS compose list mail using a plain text editor.
That way you see what will actually go to the list. In this case, Gmail
thwarted your intent. I have users on my lists that use Yahoo and drag
and drop links into their posts and the links don't appear in the plain
text at all.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From gtaylor at tnetconsulting.net  Thu Jul 19 21:41:38 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Thu, 19 Jul 2018 19:41:38 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <1f035d2a-e278-df7b-abd9-955c6f82b2bb@msapiro.net>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
 <9db2d852-2bbb-5d01-3253-f350ac64b5de@spamtrap.tnetconsulting.net>
 <c50efe06-5e61-22e4-79bd-1c1de048703e@msapiro.net>
 <89bf64b1-1edf-82f1-790b-b85e6831eaa5@spamtrap.tnetconsulting.net>
 <1f035d2a-e278-df7b-abd9-955c6f82b2bb@msapiro.net>
Message-ID: <053a5e3e-28c2-5d36-8860-968d6bc9da5a@spamtrap.tnetconsulting.net>

On 07/19/2018 06:22 PM, Mark Sapiro wrote:
> If Mailman is asked to remove or replace DKIM headers, the 
> headers affected are DomainKey-Signature, DKIM-Signature and 
> Authentication-Results.

Good to know.

Thank you for clarifying Mark.



-- 
Grant. . . .
unix || die


From gtaylor at tnetconsulting.net  Thu Jul 19 21:45:19 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Thu, 19 Jul 2018 19:45:19 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <08abe617-2c99-c6e0-70e1-b098abae631f@msapiro.net>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
 <dedbcda0-7399-d932-c2b9-e784f73daeb1@caerllewys.net>
 <08abe617-2c99-c6e0-70e1-b098abae631f@msapiro.net>
Message-ID: <c74d33fb-d2e8-adec-4fb6-8f8cf8a7922d@spamtrap.tnetconsulting.net>

On 07/19/2018 05:27 PM, Mark Sapiro wrote:
> The problem is downstream has to trust me. If I'm gmail.com, I'll probably 
> be trusted. If I'm msapiro.net, probably not. Python.org, who knows.

Yep.

I've not yet seen any indication that there will be any good way to 
establish this trust relationship, save for traditional 
Business-to-Business methods.  At least I'm not aware of anything more 
automatic.

Thus I question how useful ARC will be for small operators.  :-/



-- 
Grant. . . .
unix || die


From jaybird at bluegrasspals.com  Fri Jul 20 02:40:48 2018
From: jaybird at bluegrasspals.com (Jayson Smith)
Date: Fri, 20 Jul 2018 02:40:48 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <E051F2E4-CD6A-4F8C-B38F-9E60B616E70D@domainmail.org>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
 <20180719121640.D844026C2470@sharky3.deepsoft.com>
 <8d82f1b4-a133-e1f6-c972-cd0470911550@spamtrap.tnetconsulting.net>
 <1592695e-fabf-8c73-c069-76274b4eacc5@msapiro.net>
 <A9ED84A0-815A-4539-AAB6-57215726E138@domainmail.org>
 <E051F2E4-CD6A-4F8C-B38F-9E60B616E70D@domainmail.org>
Message-ID: <eb655ff7-49bb-4d03-24be-b5e0d140379d@bluegrasspals.com>

Hi,

Both of these approaches seem to be specific to Postfix if I'm not 
mistaken. There's a similar milter for Sendmail called Mailman-Milter 
which I was using for a while. However, it worked based on Mailman's 
action E.G. it would use a Python script to determine what Mailman would 
do with a particular sender's mail for a particular list. If the answer 
was reject or discard, the incoming message got rejected at the SMTP 
data stage. This meant the list owner had to set up Mailman to reject or 
discard mail from non-subscribers. Unfortunately when I upgraded to 
Debian 9 from CentOS 6, Mailman-Milter broke somehow, and I don't 
understand the code well enough to figure out why. It also hasn't been 
updated in quite some time, and has virtually no documentation.

Could either of these milter solutions linked previously be adapted for 
use as a Sendmail milter? I'd love to find something which would query 
Mailman about the status of a particular sender address at the RCPT 
stage of the SMTP transaction so spoofed mail can be rejected right 
away, however, this might not be possible for one reason or another. Any 
thoughts would be appreciated.

Thanks,

Jayson

On 7/19/2018 2:55 PM, Jim Popovitch via Mailman-Users wrote:
> On July 19, 2018 6:53:52 PM UTC, Jim Popovitch <jimpop at domainmail.org> wrote:
>> On July 19, 2018 5:28:24 PM UTC, Mark Sapiro <mark at msapiro.net> wrote:
>>> On 07/19/2018 08:02 AM, Grant Taylor via Mailman-Users wrote:
>>>> I have often wondered about enhancing Mailman, or augmenting it with
>>> a
>>>> milter, to be able to test the SMTP envelope from, to, and body
>>> content
>>>> against list parameters and be able to reject messages during the
>>> SMTP
>>>> delivery transaction.
>>>
>>> You might be interested in
>>> <http://ftp.tummy.com/pub/tummy/tummailmanmember/>.
>> Here's an alternate take on the same thing that I wrote a couple years
>> back.
> With link this time!
>
> https://code.launchpad.net/~jimpop/mailman/check_subscriber
>
> -Jim P.
>
>
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/jaybird%40bluegrasspals.com
>
>


From turnbull.stephen.fw at u.tsukuba.ac.jp  Fri Jul 20 03:24:52 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Fri, 20 Jul 2018 16:24:52 +0900
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <08abe617-2c99-c6e0-70e1-b098abae631f@msapiro.net>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
 <dedbcda0-7399-d932-c2b9-e784f73daeb1@caerllewys.net>
 <08abe617-2c99-c6e0-70e1-b098abae631f@msapiro.net>
Message-ID: <23377.36420.423111.312750@turnbull.sk.tsukuba.ac.jp>

Mark Sapiro writes:

 > The problem is downstream has to trust me. If I'm gmail.com, I'll
 > probably be trusted. If I'm msapiro.net, probably not. Python.org, who
 > knows.

The problem is the same butt-lazy admins that caused you to implement
DKIM-stripping.[1]  Google and (AFAIK) Yahoo! and Microsoft will trust
you by default.

In fact, I snafued a couple weeks back, exposed my Mailman server to
the joe-jobbing via web subscription backscatter, and was immediately
interdicted by Microsoft.  Fixed the problem, went to Microsoft, and
immediately mail started flowing again and has ever since.

So I think ARC is in practice going to be trusted by default, at least
until the first big spammer exploit taking advantage of that trust.

Footnotes: 
[1]  In many cases, Authentication-Results should be stripped by the
domain-edge MTA, and RFC 7601 discusses when that really must be done,
and the pros and cons of doing it in general.


From gtaylor at tnetconsulting.net  Fri Jul 20 11:12:59 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Fri, 20 Jul 2018 09:12:59 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <eb655ff7-49bb-4d03-24be-b5e0d140379d@bluegrasspals.com>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
 <20180719121640.D844026C2470@sharky3.deepsoft.com>
 <8d82f1b4-a133-e1f6-c972-cd0470911550@spamtrap.tnetconsulting.net>
 <1592695e-fabf-8c73-c069-76274b4eacc5@msapiro.net>
 <A9ED84A0-815A-4539-AAB6-57215726E138@domainmail.org>
 <E051F2E4-CD6A-4F8C-B38F-9E60B616E70D@domainmail.org>
 <eb655ff7-49bb-4d03-24be-b5e0d140379d@bluegrasspals.com>
Message-ID: <38f70f73-91f0-502b-a034-1125e7a86f87@spamtrap.tnetconsulting.net>

On 07/20/2018 12:40 AM, Jayson Smith wrote:
> Could either of these milter solutions linked previously be adapted for 
> use as a Sendmail milter? I'd love to find something which would query 
> Mailman about the status of a particular sender address at the RCPT 
> stage of the SMTP transaction so spoofed mail can be rejected right 
> away, however, this might not be possible for one reason or another. Any 
> thoughts would be appreciated.

Have you tried using the alternate milters?  Postfix implemented 
Sendmail's milter protocol.  So I think they are directly compatible 
with each other.



-- 
Grant. . . .
unix || die


From mark at msapiro.net  Fri Jul 20 12:37:52 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 20 Jul 2018 09:37:52 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <38f70f73-91f0-502b-a034-1125e7a86f87@spamtrap.tnetconsulting.net>
References: <CABTRqU5Hwv05cEvF4n5GvfRxQ5iVVvKkL75Kc3wJUMy0YMM9BQ@mail.gmail.com>
 <20180719021021.A5F0026C2B24@sharky3.deepsoft.com>
 <741317be-149e-1f15-7045-16c7187e5e77@msapiro.net>
 <20180719121640.D844026C2470@sharky3.deepsoft.com>
 <8d82f1b4-a133-e1f6-c972-cd0470911550@spamtrap.tnetconsulting.net>
 <1592695e-fabf-8c73-c069-76274b4eacc5@msapiro.net>
 <A9ED84A0-815A-4539-AAB6-57215726E138@domainmail.org>
 <E051F2E4-CD6A-4F8C-B38F-9E60B616E70D@domainmail.org>
 <eb655ff7-49bb-4d03-24be-b5e0d140379d@bluegrasspals.com>
 <38f70f73-91f0-502b-a034-1125e7a86f87@spamtrap.tnetconsulting.net>
Message-ID: <31e695ea-0e88-7e77-fdc1-3f2736763dce@msapiro.net>

On 07/20/2018 08:12 AM, Grant Taylor via Mailman-Users wrote:
> 
> Have you tried using the alternate milters?? Postfix implemented
> Sendmail's milter protocol.? So I think they are directly compatible
> with each other.


Neither <http://ftp.tummy.com/pub/tummy/tummailmanmember/> nor
<https://code.launchpad.net/~jimpop/mailman/check_subscriber> are
milters in the sense of Sendmail's milter protocol.

tummailmanmember is a Postfix policy server and check_subscriber is a
script that is invoked via a Postfix table lookup.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From atann at alphasrv.net  Fri Jul 20 13:23:15 2018
From: atann at alphasrv.net (Andre Tann)
Date: Fri, 20 Jul 2018 19:23:15 +0200
Subject: [Mailman-Users] Changing the senders name in the From line in
 confirmation mails
Message-ID: <49fca567-3790-0531-2e87-2aae20054e75@alphasrv.net>

Hi folks,

I've been googling now for a while, but could not find a hint if the
following can be done:

The sender's name in the from line of a confirmation mail should be
changed/inserted. Example:

This line:
???
??? From:
newsletter-confirm+7f07b5deb0843fad9ab40ead51ac7e7541cf22d6 at example.com

should be changed to

??? From: My-Company
<newsletter-confirm+7f07b5deb0843fad9ab40ead51ac7e7541cf22d6 at example.com>

I found tons of hints how to tweak the subject line, and I already did
that BTW. But no clue how to change the name.

Any hints?

Thanks

-- 
Andre Tann


From mark at msapiro.net  Sat Jul 21 14:23:34 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 21 Jul 2018 11:23:34 -0700
Subject: [Mailman-Users] Changing the senders name in the From line in
 confirmation mails
In-Reply-To: <49fca567-3790-0531-2e87-2aae20054e75@alphasrv.net>
References: <49fca567-3790-0531-2e87-2aae20054e75@alphasrv.net>
Message-ID: <0c9fa60b-389e-d256-30f6-d0bb4c1a0732@msapiro.net>

On 7/20/18 10:23 AM, Andre Tann wrote:
> 
> The sender's name in the from line of a confirmation mail should be
> changed/inserted. Example:
> 
> This line:
> ???
> ??? From:
> newsletter-confirm+7f07b5deb0843fad9ab40ead51ac7e7541cf22d6 at example.com
> 
> should be changed to
> 
> ??? From: My-Company
> <newsletter-confirm+7f07b5deb0843fad9ab40ead51ac7e7541cf22d6 at example.com>


This can be done, but without modifying code, not exactly as you
propose. You can make it

From: (My-Company) newsletter-confirm+7... at example.com

I.e. You can include My-Company as a comment, but not as a display name.

There are two default settings:

VERP_CONFIRM_FORMAT = '%(addr)s+%(cookie)s'
VERP_CONFIRM_REGEXP =
r'(?s)^(.*<)?(?P<addr>.+)\+(?P<cookie>[0-9a-f]{40})@.*$'

You can override these in mm_cfg.py with

VERP_CONFIRM_FORMAT = '(My-Company) %(addr)s+%(cookie)s'
VERP_CONFIRM_REGEXP = r'(?s)^\(?My-Company\)?
(.*<)?(?P<addr>.+)\+(?P<cookie>[0-9a-f]{40})@.*$'

(watch out for wrapped lines).

You can't do

VERP_CONFIRM_FORMAT = 'My-Company <%(addr)s+%(cookie)s>'

because the @example.com is appended to that.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From johnl at taugh.com  Sat Jul 21 16:22:44 2018
From: johnl at taugh.com (John Levine)
Date: 21 Jul 2018 16:22:44 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <dedbcda0-7399-d932-c2b9-e784f73daeb1@caerllewys.net>
Message-ID: <20180721202245.6502620028AB39@ary.qy>

In article <dedbcda0-7399-d932-c2b9-e784f73daeb1 at caerllewys.net> you write:
>On 07/19/18 17:11, John Levine wrote:
>> In article <c5d1335d-0762-8a85-3257-239d5e2e46d6 at spamtrap.tnetconsulting.net> you write:
>>> Yes.  Just about everything can be spoofed to some degree.  It really 
>>> depends on what information the owner of the purported sending domain 
>>> publishes and what filtering / consumption of said information the 
>>> receiving server exercises.
>> 
>> Well, you know, this is what DMARC is intended to address.  While
>> DMARC checks on mail that has passed through mailing lists has all
>> sorts of well known problems, doing DMARC checks on mail that arrives
>> at a list server would be pretty benign.  It's pretty rare for the
>> path from a user to the mailman server to do things that would cause
>> DMARC fails.
>
>Actually, mailing lists and other redistribution are among the places
>DMARC notably breaks.  The real answer, which was created for this
>purpose, is ARC (Authenticated Received Chain).  That is designed from
>the start to pass through mailing lists unbroken.
>
>(Or so I'm told.)

You missed a key point.  I was suggesting DMARC-ish checks on mail *to* a
maiing list, where they should work fine.  Mail *from* a mailing list is
indeed screwed up by DMARC which is why I've been working on ARC libraries.

R's,
John


From johnl at taugh.com  Sat Jul 21 16:24:45 2018
From: johnl at taugh.com (John Levine)
Date: 21 Jul 2018 16:24:45 -0400
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <c74d33fb-d2e8-adec-4fb6-8f8cf8a7922d@spamtrap.tnetconsulting.net>
Message-ID: <20180721202445.6A57A20028AB70@ary.qy>

In article <c74d33fb-d2e8-adec-4fb6-8f8cf8a7922d at spamtrap.tnetconsulting.net> you write:
>On 07/19/2018 05:27 PM, Mark Sapiro wrote:
>> The problem is downstream has to trust me. If I'm gmail.com, I'll probably 
>> be trusted. If I'm msapiro.net, probably not. Python.org, who knows.
>
>Yep.
>
>I've not yet seen any indication that there will be any good way to 
>establish this trust relationship, save for traditional 
>Business-to-Business methods.  At least I'm not aware of anything more 
>automatic.
>
>Thus I question how useful ARC will be for small operators.  :-/

I know people working on whiteish lists to use with ARC, to say that
these domain are known to host real mailing lists so you should believe
their ARC assertions.

R's,
John

From atann at alphasrv.net  Sun Jul 22 01:26:09 2018
From: atann at alphasrv.net (Andre Tann)
Date: Sun, 22 Jul 2018 07:26:09 +0200
Subject: [Mailman-Users] Changing the senders name in the From line in
 confirmation mails
In-Reply-To: <0c9fa60b-389e-d256-30f6-d0bb4c1a0732@msapiro.net>
References: <49fca567-3790-0531-2e87-2aae20054e75@alphasrv.net>
 <0c9fa60b-389e-d256-30f6-d0bb4c1a0732@msapiro.net>
Message-ID: <e14dd5b0-d64f-5beb-1a85-64d81d46d853@alphasrv.net>

Hi Mark,

Am 21.07.2018 um 20:23 schrieb Mark Sapiro:

> This can be done, but without modifying code, not exactly as you
> propose. You can make it
[...]

Fully understood, and as always: thanks for your fast & accurate help

Andre

-- 
Andre Tann


From gtaylor at tnetconsulting.net  Sun Jul 22 15:18:39 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Sun, 22 Jul 2018 13:18:39 -0600
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <20180721202445.6A57A20028AB70@ary.qy>
References: <20180721202445.6A57A20028AB70@ary.qy>
Message-ID: <1fb88a39-0acd-f34f-c504-9eb217a75bc0@spamtrap.tnetconsulting.net>

On 07/21/2018 02:24 PM, John Levine wrote:
> I know people working on whiteish lists to use with ARC, to say that 
> these domain are known to host real mailing lists so you should believe 
> their ARC assertions.

Is there some place that I can find out more about these people and / or 
their projects?

Aside:  What does hosting mailing lists or not have to do with believing 
their ARC assertions?  -  I would hope that the ARC white lists state 
that these senders are probably trust worthy, independent of mailing 
lists or not.



-- 
Grant. . . .
unix || die


From gtaylor at tnetconsulting.net  Sun Jul 22 15:24:13 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Sun, 22 Jul 2018 13:24:13 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <dedbcda0-7399-d932-c2b9-e784f73daeb1@caerllewys.net>
References: <20180719211134.586C120027FF16@dhcp-85af.meeting.ietf.org>
 <dedbcda0-7399-d932-c2b9-e784f73daeb1@caerllewys.net>
Message-ID: <0590fe51-3f96-754d-d155-af0eb9ca48be@spamtrap.tnetconsulting.net>

On 07/19/2018 04:59 PM, Phil Stracchino wrote:
> Actually, mailing lists and other redistribution are among the places 
> DMARC notably breaks.

Does DMARC actually break or otherwise behave in a manner contrary to 
it's specification?

I personally believe that DMARC (and SPF and DKIM) are doing exactly 
what they are supposed to do.

The problem is that they are doing what they are supposed to do when 
people want them to not do that.

I don't even consider this to be a false positive.  -  If someone trains 
a dog to bark constantly (alert) when they see someone approach with a 
gun, I believe that the dog is doing exactly what they are trained to do 
when an armed police officer walks up.  The dog is doing exactly what 
they were trained to do.



-- 
Grant. . . .
unix || die


From jimpop at domainmail.org  Sun Jul 22 15:49:17 2018
From: jimpop at domainmail.org (Jim Popovitch)
Date: Sun, 22 Jul 2018 15:49:17 -0400
Subject: [Mailman-Users] enabling reCAPTCHA for older lists
Message-ID: <1532288957.5300.1.camel@domainmail.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hello,

I've crossed over into reCAPTCHA hell (voluntarily!).... currently
looking for a savior.

I have followed all the instructions in Defaults.py, and I have created
the google account and setup the site-key and secret-key in mm_cfg.py,
and restated MM. (this is v2.1.27)

Everything works fine on newly created lists.  HOWEVER, older existing
lists (from years ago) present the following browser error to the user:

   reCAPTCHA validation failed: missing-input-response?

along with the following browser console debug warning:

   A form was submitted in the windows-1252 encoding which cannot
   encode all Unicode characters, so user input may get corrupted.
   To avoid this problem, the page should be changed so that the?
   form is submitted in the UTF-8 encoding either by changing the
   encoding of the page itself to UTF-8 or by specifying accept-
   charset=utf-8 on the form element.

I've done diff's of the config-text of a new and old list, and the diff
output shows nothing more than domains, filters, privacy differences
between each list.

What could be set for an older list that would break reCAPTCHAs?

- -Jim P.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEPxwe8uYBnqxkbORSJxVetMRaJwUFAltU370ACgkQJxVetMRa
JwWWew//dYfSy1wZynpgC0wwqPlDb7utAx+C8vUd1OPsqmalaqucVxSuktu6ltjA
EC2Dv96Qh4JJojONwjQOV0mR6dmYOJJhpnJ4+FwV3kL7J8qaKzIMKA6Qjgt5hOiK
FpjIQC/+6OOLfdVlTUTMX9o0A1SuhDf2CSgnLcSBSgTJVFgBKI0wyKiZ4KKgsHyg
edbU5I4UaS+SsxWW5aN8zyizcCxi08Bh+paG+pFTCRiDKxrGXA0jIyttvX6l6LDj
t3hfWfEKmFnXtyNeL0SIA0cKf7yID+RHNF/6Fa57kZ3Co7A/h2e+T3VmDCqs7Kfk
LE+hcg5XdSep2eKwAK/kG8rpwMTvT539EN9rFHZMZsYUhQt9BziL9P5jYQZG/MC/
Ue3Xz+O5Qx9wji/3ubtn0d9/NimA7Yvu29WIbyKyKv81Zjyg/iAruJByNJU5wY+L
DC2wsk2T4/Bmfs6vvqTcmKqSxuTx3ugHuwwUM9mNAVY/w25M7vF4rfSuQuQ/5Ar3
ln2DY4FsfGx65jIdzH33OPAZA41aynWdh1fdV1AmfZB0rdntexZf0m9oCMIQjzCS
PbxFmXdIal1np2IIYW52eaD5hEgyiDi82UaI/lK/A087ZGrFUA0ffeJl30O7p5A7
+zyUqCNUc9YZb/STosMtQ1hP9r/+eCaMzbxg3KDJslW0lCcsjcs=
=kXCI
-----END PGP SIGNATURE-----


From johnl at taugh.com  Sun Jul 22 16:03:34 2018
From: johnl at taugh.com (John Levine)
Date: 22 Jul 2018 16:03:34 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <0590fe51-3f96-754d-d155-af0eb9ca48be@spamtrap.tnetconsulting.net>
Message-ID: <20180722200334.B150E20028DC36@ary.qy>

In article <0590fe51-3f96-754d-d155-af0eb9ca48be at spamtrap.tnetconsulting.net> you write:
>-=-=-=-=-=-
>-=-=-=-=-=-
>
>On 07/19/2018 04:59 PM, Phil Stracchino wrote:
>> Actually, mailing lists and other redistribution are among the places 
>> DMARC notably breaks.
>
>Does DMARC actually break or otherwise behave in a manner contrary to 
>it's specification?

No, it was specified in full knowledge that it would break pretty much
every mailing list on the planet if used on domains with human users,
instead of its intended target of notices from robot domains like
paypal.com.

That's why we have ARC, once AOL and Yahoo abused it to solve the
problem they created when they let crooks steal their users' address
books.

R's,
John

PS: This isn't conspiracy theorizing, I know the people involved.

From johnl at taugh.com  Sun Jul 22 16:05:14 2018
From: johnl at taugh.com (John Levine)
Date: 22 Jul 2018 16:05:14 -0400
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <1fb88a39-0acd-f34f-c504-9eb217a75bc0@spamtrap.tnetconsulting.net>
Message-ID: <20180722200514.EA5FA20028DC81@ary.qy>

In article <1fb88a39-0acd-f34f-c504-9eb217a75bc0 at spamtrap.tnetconsulting.net> you write:
>Is there some place that I can find out more about these people and / or 
>their projects?

See the archives of the ARC mailing lists.

>Aside:  What does hosting mailing lists or not have to do with believing 
>their ARC assertions?  -  I would hope that the ARC white lists state 
>that these senders are probably trust worthy, independent of mailing 
>lists or not.

Every domain added to a whitelist like this involves manual work.  Why
would you waste time on domains that aren't likely to send mail with
ARC headers?

R's,
John

From gtaylor at tnetconsulting.net  Sun Jul 22 17:03:51 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Sun, 22 Jul 2018 15:03:51 -0600
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <20180722200514.EA5FA20028DC81@ary.qy>
References: <20180722200514.EA5FA20028DC81@ary.qy>
Message-ID: <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>

On 07/22/2018 02:05 PM, John Levine wrote:
> Every domain added to a whitelist like this involves manual work.

Yes.

> Why would you waste time on domains that aren't likely to send mail with 
> ARC headers?

I'm not suggesting wasting time on domains that wouldn't send ARC headers.

I'm questioning why domains that do use ARC headers that don't run 
mailing lists should not be white listed.



-- 
Grant. . . .
unix || die


From gtaylor at tnetconsulting.net  Sun Jul 22 17:11:47 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Sun, 22 Jul 2018 15:11:47 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180722200334.B150E20028DC36@ary.qy>
References: <20180722200334.B150E20028DC36@ary.qy>
Message-ID: <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>

On 07/22/2018 02:03 PM, John Levine wrote:
> No, it was specified in full knowledge that it would break pretty much 
> every mailing list on the planet if used on domains with human users, 
> instead of its intended target of notices from robot domains like 
> paypal.com.

I choose to believe the mailing lists were behaving improperly.

To me, DMARC (including SPF and DKIM) is a method to determine if a 
message is coming from the original source (or authorized delegate). 
Where email is a combination of the message data and SMTP transaction 
delivering said message.

> That's why we have ARC, once AOL and Yahoo abused it to solve the problem 
> they created when they let crooks steal their users' address books.

I assume you are referring to "DMARC" when you say "?abused /it/ to solve?".

I feel like AOL's and Yahoo's actions are just additional gas on the 
fire that has been burning for a long time.  The problem of bad actors 
spoofing message senders exists independently of AOL and Yahoo.  Did 
their (in)actions make the problem worse, probably.  Did they cause the 
problem?  No.  Did they exceed critical mass?  I don't think so.  Rather 
I think it was past the critical mass long before AOL and Yahoo fueled 
the fire.



-- 
Grant. . . .
unix || die


From johnl at taugh.com  Sun Jul 22 17:37:45 2018
From: johnl at taugh.com (John Levine)
Date: 22 Jul 2018 17:37:45 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
Message-ID: <20180722213745.EB82F2002915DB@ary.qy>

In article <c2b71208-735a-398a-df87-8ae574a17f22 at spamtrap.tnetconsulting.net> you write:
>-=-=-=-=-=-
>-=-=-=-=-=-
>
>On 07/22/2018 02:03 PM, John Levine wrote:
>> No, it was specified in full knowledge that it would break pretty much 
>> every mailing list on the planet if used on domains with human users, 
>> instead of its intended target of notices from robot domains like 
>> paypal.com.
>
>I choose to believe the mailing lists were behaving improperly.

Oh, OK, sorry to disturb you.

R's,
John

From Richard at Damon-family.org  Sun Jul 22 18:25:23 2018
From: Richard at Damon-family.org (Richard Damon)
Date: Sun, 22 Jul 2018 18:25:23 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
Message-ID: <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>


> On Jul 22, 2018, at 5:11 PM, Grant Taylor via Mailman-Users <mailman-users at python.org> wrote:
> 
>> On 07/22/2018 02:03 PM, John Levine wrote:
>> No, it was specified in full knowledge that it would break pretty much every mailing list on the planet if used on domains with human users, instead of its intended target of notices from robot domains like paypal.com.
> 
> I choose to believe the mailing lists were behaving improperly.
> 
> To me, DMARC (including SPF and DKIM) is a method to determine if a message is coming from the original source (or authorized delegate). Where email is a combination of the message data and SMTP transaction delivering said message.

What actions do you think mailing lists are doing improperly?

Note, the subject modification is a long standing feature of mailing list, which is one thing that breaks DMARC, though I might be willing to give that up.

The modification of the message body to add a header or footer is also common, and in some places effectively required by law.

>> That's why we have ARC, once AOL and Yahoo abused it to solve the problem they created when they let crooks steal their users' address books.
> 
> I assume you are referring to "DMARC" when you say "?abused /it/ to solve?".
> 
> I feel like AOL's and Yahoo's actions are just additional gas on the fire that has been burning for a long time.  The problem of bad actors spoofing message senders exists independently of AOL and Yahoo.  Did their (in)actions make the problem worse, probably.  Did they cause the problem?  No.  Did they exceed critical mass?  I don't think so.  Rather I think it was past the critical mass long before AOL and Yahoo fueled the fire.
> 
> -- 
> Grant. . . .

If AOL and Yahoo just used the quarantine option for DMARC, it wouldn?t have been quite as bad. But they ABUSED DMARC by their settings. By the design of DMARC, AOL and Yahoo should have informed their users that they were changing the Terms of Service of their email systems, and now all their users are effectively prohibited to use any form of re-mailing systems, including most forms of (external) mailing lists. Instead they just told the world, we aren?t going to follow the normal rules, you deal with it.

Yes, there is a fundamental issue with email that it is easy to spoof. Fixing it is going to be a significant issue, and possible a complete recreation of the system. The issue is that to create such a new system is a major job. Such a redesign would need to look at ALL current uses and either decide that such uses were no longer valid or to accommodate them. DMARC somewhat intentionally did not consider mailing list, because they didn?t have a good solution to handle them, and their intended usage, the protection of ?valuable? mail somewhat excluded the use of such services. It basically required that any service that wanted to use DMARC needed to separate valuable protected mail from less valuable mail with different domains. AOL and YAHOO just decided to ignore that in their use of it.

From mark at msapiro.net  Sun Jul 22 19:36:58 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 22 Jul 2018 16:36:58 -0700
Subject: [Mailman-Users] enabling reCAPTCHA for older lists
In-Reply-To: <1532288957.5300.1.camel@domainmail.org>
References: <1532288957.5300.1.camel@domainmail.org>
Message-ID: <ae539294-1685-5b66-3991-ebe583c4a7e0@msapiro.net>

On 07/22/2018 12:49 PM, Jim Popovitch via Mailman-Users wrote:
> 
> What could be set for an older list that would break reCAPTCHAs?


A list specific listinfo template.

At rev 1218 (in 2.1.24), I changed the default listinfo templates. See
<https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1218#templates/en/listinfo.html>.

The change that affects reCAPTCHA is moving <MM-Subscribe-Form-Start>
from line 29 to line 68. List specific templates that have that line in
the old place don't work with the reCAPTCHA feature.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20180722/259dc96b/attachment.sig>

From jimpop at domainmail.org  Sun Jul 22 21:08:19 2018
From: jimpop at domainmail.org (Jim Popovitch)
Date: Sun, 22 Jul 2018 21:08:19 -0400
Subject: [Mailman-Users] enabling reCAPTCHA for older lists
In-Reply-To: <ae539294-1685-5b66-3991-ebe583c4a7e0@msapiro.net>
References: <1532288957.5300.1.camel@domainmail.org>
 <ae539294-1685-5b66-3991-ebe583c4a7e0@msapiro.net>
Message-ID: <1532308099.1676.1.camel@domainmail.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On Sun, 2018-07-22 at 16:36 -0700, Mark Sapiro wrote:
> On 07/22/2018 12:49 PM, Jim Popovitch via Mailman-Users wrote:
> > 
> > What could be set for an older list that would break reCAPTCHAs?
> 
> 
> A list specific listinfo template.
> 
> At rev 1218 (in 2.1.24), I changed the default listinfo templates.
> See
> <https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/12
> 18#templates/en/listinfo.html>.
> 
> The change that affects reCAPTCHA is moving <MM-Subscribe-Form-Start>
> from line 29 to line 68. List specific templates that have that line
> in the old place don't work with the reCAPTCHA feature.
> 

Thanks Mark!!  That was it.

- -Jim P.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEPxwe8uYBnqxkbORSJxVetMRaJwUFAltVKoQACgkQJxVetMRa
JwWtqQ//dSmcAJa2KzZ+0wr6NPkkONv76wDta/miQQnTAUluPXiZzjH7y6OGR7da
g73GRxgUUZCPlPdT4j3Agi+H8Dz7C5qaxlfWvdur2JpmuKWzErZNtecnXlmalDbA
gEPDA+F5iTGdcg4FAUNNmG6vN9OB2OBvCDMTEmmZJra5ioWRJDcgh1QviPdwvF1C
hRrIiZx+kEo5HpJ7Cf7btxZFFpIC2AP+JQmBu/uo8Kr3c5DpQCX6UfdXzpg/hxA6
0t/BVmGw0/fHhHRKvpiZnz/X8RGb/9XAeaOw5epO6MUmk6toGyxR/1P3B7k2TkNF
FRlAUxuYm2eKJgumW4+yEcTKvCVAuV+XLdu8zXKUHeiVH2bdn9z1IK3ls9JR5zET
4kjSjfw2lhWAHk3Yyk9AMurkKTQiZgnBA6JovwFPpx5fXknLLBLnzNmm2V6jj2/B
IING6Ua7Bu195mrMN3Udlxx6/DjHuKGpkogp8mtbYsD8rGe6jLmu+JWFcRo/MN62
40KeatuPG3LZJusKddxbMq35hLCWQx3J8nGisQAlsufvMllg8HCYsguybbyIoe5s
uH21feaWrkX4K/cUXnvLOsRNKonJfSMkbXk92Efuvu6nRhidQZUwbewwlP65j/VM
Vdxw0mJgNYGUyvPtiapmys+XCSGj27mEpW4Wv3IXfHI0BJme9BU=
=G04G
-----END PGP SIGNATURE-----


From turnbull.stephen.fw at u.tsukuba.ac.jp  Mon Jul 23 01:02:23 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 23 Jul 2018 14:02:23 +0900
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>
References: <20180722200514.EA5FA20028DC81@ary.qy>
 <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>
Message-ID: <23381.24927.74482.530338@turnbull.sk.tsukuba.ac.jp>

Grant Taylor via Mailman-Users writes:

 > I'm questioning why domains that do use ARC headers that don't run 
 > mailing lists should not be white listed.

You're misunderstanding.  The ARC community doesn't discourage
whitelisting other sites.  The work to do whitelisting does.  Mailing
lists are *known* to *frequently* (almost always) break DKIM
signatures in a way amenable to repair by ARC.[1]

The other main pain points for DMARC are third-party services that are
authorized by the owner of a mailbox to send mail "on behalf of",
without participation of the adminstrator of the mailbox's domain.  An
example is invoicing services.  These do not benefit from ARC *at all*
because they have a valid DKIM signature from the originating domain,
who can be trusted for that service, but don't get such a signature
from the mailbox's domain as required for DMARC From validation.

The other *possible* use case for ARC would be non-mailing list
forwarding.  But these almost never break the DKIM signature of the
originator.  I guess large services like GMail can eventually add a
feature where a user can configure GMail to recognize and whitelist
specific sites where they have mailboxes set to forward to GMail.  But
I doubt this will ever be a standard feature of MDAs.  It will be
complex and fragile to implement, and almost never used.



Footnotes: 
[1]  Note that I disagree somewhat with John.  I suspect that
humongous providers like GMail, Yahoo!, and Microsoft will
automatically accept ARC in the presence of a RFC 2369 List-* header,
and blacklist on bad behavior, as they do now.  That's not perfect
from a list admin's point of view---it requires a lot of resources to
do that well, so small sites probably won't---but it's not too bad.

From brennan at columbia.edu  Mon Jul 23 09:45:23 2018
From: brennan at columbia.edu (Joseph Brennan)
Date: Mon, 23 Jul 2018 09:45:23 -0400
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <1fb88a39-0acd-f34f-c504-9eb217a75bc0@spamtrap.tnetconsulting.net>
References: <20180721202445.6A57A20028AB70@ary.qy>
 <1fb88a39-0acd-f34f-c504-9eb217a75bc0@spamtrap.tnetconsulting.net>
Message-ID: <CAMSGcLBBUS2P6ZR5BmscXeFgPpcvO-rEVMasWvKznXz4oo65og@mail.gmail.com>

On Sun, Jul 22, 2018 at 3:18 PM Grant Taylor via Mailman-Users <
mailman-users at python.org> wrote:

> On 07/21/2018 02:24 PM, John Levine wrote:
> > I know people working on whiteish lists to use with ARC, to say that
> > these domain are known to host real mailing lists so you should believe
> > their ARC assertions.
>

Why not just have that list, and a X-Trust-Me: YES header? It would be much
simpler to implement than ARC.

Joseph Brennan
Columbia University I T

From johnl at taugh.com  Mon Jul 23 12:46:38 2018
From: johnl at taugh.com (John Levine)
Date: 23 Jul 2018 12:46:38 -0400
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <CAMSGcLBBUS2P6ZR5BmscXeFgPpcvO-rEVMasWvKznXz4oo65og@mail.gmail.com>
Message-ID: <20180723164639.777302002968CE@ary.qy>

In article <CAMSGcLBBUS2P6ZR5BmscXeFgPpcvO-rEVMasWvKznXz4oo65og at mail.gmail.com> you write:
>On Sun, Jul 22, 2018 at 3:18 PM Grant Taylor via Mailman-Users <
>mailman-users at python.org> wrote:
>
>> On 07/21/2018 02:24 PM, John Levine wrote:
>> > I know people working on whiteish lists to use with ARC, to say that
>> > these domain are known to host real mailing lists so you should believe
>> > their ARC assertions.
>
>Why not just have that list, and a X-Trust-Me: YES header? It would be much
>simpler to implement than ARC.

There turns out to be an actual answer to this question, which I have
asked people from Google.

When someone gets his address book stolen from his botted PC, spamware
will send spam to everyone in his address book using his address on
the From: line.  If some of those addresses are lists, those lists
will generally forward the spam even though they are otherwise legit.

Google tells me this happens often enough that they can't just
whitelist mailing lists, and ARC gives them the clues to tell
forwarded bot spam from forwarded real mail.  I've certainly seen
it both on lists I run and lists I subscribe to.

As I said a few messages ago, if lists did more stringent tests on
incoming mail, a lot of this complexity could be avoided, but they
don't so it can't.

R's,
John

From mark at msapiro.net  Mon Jul 23 13:33:12 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 23 Jul 2018 10:33:12 -0700
Subject: [Mailman-Users] Mailman 2.1.28 Security Release
Message-ID: <b4c34502-01b1-bb96-3b14-bbc2cfc6eb85@msapiro.net>

I am pleased to announce the release of Mailman 2.1.28.

Python 2.6 is the minimum supported, but Python 2.7 is strongly recommended.

This is a minor security fix release. It also has some i18n updates and
a couple of bug fixes and adds the ability to edit list specific
templates through the web admin UI in a supported language other than
the list's default. See the attached README.txt for details.

For details of the security issue, see the report at
<https://bugs.launchpad.net/mailman/+bug/1780874> which also includes a
patch for those who want to fix this issue without upgrading.

Mailman is free software for managing email mailing lists and
e-newsletters. Mailman is used for all the python.org and
SourceForge.net mailing lists, as well as at hundreds of other sites.

For more information, please see our web site at one of:

http://www.list.org
https://www.gnu.org/software/mailman
http://mailman.sourceforge.net/
https://mirror.list.org/

Mailman 2.1.28 can be downloaded from

https://launchpad.net/mailman/2.1/
https://ftp.gnu.org/gnu/mailman/
https://sourceforge.net/projects/mailman/

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: README.txt
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20180723/4c9b9087/attachment.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20180723/4c9b9087/attachment.sig>

From rmcclung at afilias.info  Mon Jul 23 10:43:52 2018
From: rmcclung at afilias.info (Ryan McClung)
Date: Mon, 23 Jul 2018 10:43:52 -0400
Subject: [Mailman-Users] Mailman 2.x to 3.1.x Migration
Message-ID: <CAH=cwAOsVLLOmn3ZPxjPjTkS3hUKd2DE7tZ=X9uwPAjjk0rG1w@mail.gmail.com>

Hey All,

I haven't read anything on gitlab about whether or not this has been
finalized. Is there a migration process available?

I also read about scripting it but unfortunately I can't find any resources
on a way to do so. Can anyone provide me with a best-practices on how to
migrate?

Ryan

From Hagedorn at uni-koeln.de  Tue Jul 24 09:18:07 2018
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Tue, 24 Jul 2018 15:18:07 +0200
Subject: [Mailman-Users] Mailman 2.x to 3.1.x Migration
In-Reply-To: <CAH=cwAOsVLLOmn3ZPxjPjTkS3hUKd2DE7tZ=X9uwPAjjk0rG1w@mail.gmail.com>
References: <CAH=cwAOsVLLOmn3ZPxjPjTkS3hUKd2DE7tZ=X9uwPAjjk0rG1w@mail.gmail.com>
Message-ID: <C33F9D3165437683DBEE217D@tyrion.rrz.uni-koeln.de>

--On 23. Juli 2018 um 10:43:52 -0400 Ryan McClung <rmcclung at afilias.info> 
wrote:

> I haven't read anything on gitlab about whether or not this has been
> finalized. Is there a migration process available?
>
> I also read about scripting it but unfortunately I can't find any
> resources on a way to do so. Can anyone provide me with a best-practices
> on how to migrate?

I have only migrated a few lists for testing purposes, but at least with 
Mailman 3.2 you just copy the .pck file and import it like this:

$ mailman import21 listname at domain /tmp/config.pck

This does not take care of list archives.
-- 
    .:.Sebastian Hagedorn - Weyertal 121 (Geb?ude 133), Zimmer 2.02.:.
                 .:.Regionales Rechenzentrum (RRZK).:.
   .:.Universit?t zu K?ln / Cologne University - ? +49-221-470-89578.:.

From andrew at hodgson.io  Tue Jul 24 09:32:20 2018
From: andrew at hodgson.io (Andrew Hodgson)
Date: Tue, 24 Jul 2018 13:32:20 +0000
Subject: [Mailman-Users] Mailman 2.x to 3.1.x Migration
In-Reply-To: <CAH=cwAOsVLLOmn3ZPxjPjTkS3hUKd2DE7tZ=X9uwPAjjk0rG1w@mail.gmail.com>
References: <CAH=cwAOsVLLOmn3ZPxjPjTkS3hUKd2DE7tZ=X9uwPAjjk0rG1w@mail.gmail.com>
Message-ID: <VI1PR08MB3358D00F449BA49EE310CF92AF550@VI1PR08MB3358.eurprd08.prod.outlook.com>

Hi,

This isn't a step by step guide or anything but just my experiences.

It is a 2 stage process:

- Import the list configuration (subscribers, list options etc);
- Import list archives.

Import list configuration using the command mailman import21.  This expects a .pck file to import and the list to which you want to import into.

Once done you can see the results by navigating to your Mailman instance and you can see the data about the list such as the creation date, number of posts and the members.  For some reason on my instance the last post date is still the last post from Mailman2, even though it has been running on Mailman3 for a month or so.

Import archives:
Use the command python manage.py hyperkitty_import to get the private mbox into your list.  I ran into 2 issues:

- If you have posts in the new archive on Mailman3 then you need to specify the earliest date you want to import into the archive as the command won't import anything older than the earliest date in the new archive.  If you run the command with the help option you can see the format required;
- There were older messages in the mbox without message-ids in the archive that failed to import.  I took the easy way out on this one and didn't import them.  In an archive with around 120,000 messages it rejected around 30 messages in all.

Hope this points you in the right direction,
Andrew.

________________________________________
From: Mailman-Users [mailman-users-bounces+andrew=hodgson.io at python.org] on behalf of Ryan McClung [rmcclung at afilias.info]
Sent: 23 July 2018 15:43
To: Mailman-Users at python.org
Subject: [Mailman-Users] Mailman 2.x to 3.1.x Migration

Hey All,

I haven't read anything on gitlab about whether or not this has been
finalized. Is there a migration process available?

I also read about scripting it but unfortunately I can't find any resources
on a way to do so. Can anyone provide me with a best-practices on how to
migrate?

Ryan
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: https://mail.python.org/mailman/options/mailman-users/andrew%40hodgson.io

From gtaylor at tnetconsulting.net  Tue Jul 24 14:47:30 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Tue, 24 Jul 2018 12:47:30 -0600
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <23381.24927.74482.530338@turnbull.sk.tsukuba.ac.jp>
References: <20180722200514.EA5FA20028DC81@ary.qy>
 <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>
 <23381.24927.74482.530338@turnbull.sk.tsukuba.ac.jp>
Message-ID: <03d440cf-7cd5-4564-2900-ffc08cd7849b@spamtrap.tnetconsulting.net>

On 07/22/2018 11:02 PM, Stephen J. Turnbull wrote:
> You're misunderstanding.  The ARC community doesn't discourage 
> whitelisting other sites.  The work to do whitelisting does.

Thank you for clarifying Stephen.  I was afraid that you were somehow 
implying that there was some sort of guideline on what sits should and 
should not implement ARC.  I didn't think that was what you were meaning 
to imply, but the doubt was there, hence the questioning.

> Mailing lists are *known* to *frequently* (almost always) break DKIM 
> signatures in a way amenable to repair by ARC.[1]

ACK

> The other main pain points for DMARC are third-party services that 
> are authorized by the owner of a mailbox to send mail "on behalf of", 
> without participation of the adminstrator of the mailbox's domain. 
> An example is invoicing services.  These do not benefit from ARC *at all* 
> because they have a valid DKIM signature from the originating domain, 
> who can be trusted for that service, but don't get such a signature from 
> the mailbox's domain as required for DMARC From validation.

I hear and acknowledge what you're saying.

I would think / hope / expect that such services would be from a 
different (sub)domain of the client that they are sending email on 
behalf of.  I would also expect the from address to reflect the 
sub-contractor's (sub)domain with a Reply-To: directing replies to 
proper main (parent) domain.  (Or some mailbox associated there in.)  I 
would also expect to see some sort of verbiage stating that "This 
message was sent on the behalf of $Parent by $3rdPartyContractor."

I would also hope / expect to see some sort of linking text / 
acknowledgement from the parent that they have (sub)contracted some 
services to a 3rd party.

But, I learn more and more every day that I have different expectations 
than most people.

> The other *possible* use case for ARC would be non-mailing list 
> forwarding.  But these almost never break the DKIM signature of the 
> originator.

They may not break DKIM.  But depending on how they operate, they may 
break SPF directly (re-sending with the original SMTP envelope From: 
thus violating SPF) -or- indirectly (re-sending with something like SRS) 
thus breaking DMARC alignment.

My understanding is that DMARC can be configured to require both SPF and 
DKIM alignment.  Maybe it's only for reporting and not for pass / fail 
tests.  I'd have to go back and re-read the specifics about DMARC again.

The point being that simple .forward(ing) may still break things.

I maintain that detecting such is one of the functional purposes of 
DMARC.  This is independent of is such benign or malicious.

> I guess large services like GMail can eventually add a feature where a 
> user can configure GMail to recognize and whitelist specific sites where 
> they have mailboxes set to forward to GMail.  But I doubt this will 
> ever be a standard feature of MDAs.  It will be complex and fragile to 
> implement, and almost never used.

Agreed to both aspects.

> Footnotes:
> [1]  Note that I disagree somewhat with John.  I suspect that humongous 
> providers like GMail, Yahoo!, and Microsoft will automatically accept 
> ARC in the presence of a RFC 2369 List-* header, and blacklist on bad 
> behavior, as they do now.  That's not perfect from a list admin's point 
> of view---it requires a lot of resources to do that well, so small sites 
> probably won't---but it's not too bad.

I question the wisdom of making processing of ARC conditional on RFC 
2369 List-* headers.  I mainly say this because there is nothing that 
prevents malicious actors from inserting (possibly bogus) List-* 
headers.  (Or lots of tiny lists of single recipients.)



-- 
Grant. . . .
unix || die


From rmcclung at afilias.info  Tue Jul 24 15:12:44 2018
From: rmcclung at afilias.info (Ryan McClung)
Date: Tue, 24 Jul 2018 15:12:44 -0400
Subject: [Mailman-Users] Mailman Migration 2.1.X to 3.2
Message-ID: <CAH=cwANXvJ8m3BNzsm8S6QO3f5jv1Yt4gqTi9TXLv1Bck8B7qQ@mail.gmail.com>

Hey All,

Looking to migrate 2.1.15 to 3.2. Right now we have a lot of lists and
pretty big archives. Is there a best practices for the migration or can
anyone offer scripts that have worked for them?

Info:

1 -- Mailman 2 server
1 -- Mailman 3 server

Both are on the same network.

From gtaylor at tnetconsulting.net  Tue Jul 24 15:20:00 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Tue, 24 Jul 2018 13:20:00 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
Message-ID: <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>

On 07/22/2018 04:25 PM, Richard Damon wrote:
> What actions do you think mailing lists are doing improperly?

I personally believe that mailing lists are their own end entity, just 
like our individual mailboxes.  (Particularly discussion mailing lists.) 
  I also believe that SPF, DKIM, and DMARC are meant to protect between 
said endpoints; message submitter and terminal mailbox.

Thus I think that DKIM and DMARC should be stripped from messages prior 
to entering the mailing list.  The mailing list does it's thing.  Then 
DKIM and DMARC are applied anew to the messages as they leave the server 
hosting the mailing list.

> Note, the subject modification is a long standing feature of mailing 
> list, which is one thing that breaks DMARC, though I might be willing 
> to give that up.

Mailing lists, as I view them, are free to mung messages to their hearts 
content in the paradigm that I use.

> The modification of the message body to add a header or footer is also 
> common, and in some places effectively required by law.

Agreed.

Such is perfectly compatible with my paradigm.

> If AOL and Yahoo just used the quarantine option for DMARC, it wouldn?t 
> have been quite as bad. But they ABUSED DMARC by their settings.

I still don't grok what you are considering "abuse" in this context?

Rather than speculating, please clarify what the abusive activity was.

> By the design of DMARC, AOL and Yahoo should have informed their users 
> that they were changing the Terms of Service of their email systems, 
> and now all their users are effectively prohibited to use any form 
> of re-mailing systems, including most forms of (external) mailing 
> lists. Instead they just told the world, we aren?t going to follow 
> the normal rules, you deal with it.

I have a different interpretation.

My understanding is that AOL and Yahoo leveraged DMARC to expressly 
identify messages that originated from AOL and Yahoo.  Or said another 
way, they leveraged DMARC to make it easy for receiving servers to 
identify messages that are not being sent from AOL or Yahoo servers 
/during/ that current SMTP transaction.

I feel the need to insert a nod towards the fact that postmasters are 
free to run their infrastructure the way that they see fit.

I also do not feel like the terms of service between AOL or Yahoo and 
their end users changed.

AOL and Yahoo simply published information to make it easier for the 
world to identify if messages in the scope of an SMTP session are coming 
from AOL or Yahoo servers.  They also published their desire for 
receiving servers to reject messages that don't pass said published 
information.

Did they do so knowing that there would likely be a problem with 
traditional .forward(ing) and mailing lists?  Quite likely.  Was an 
internal business decision made that publishing such information and 
dealing with the ramifications of .forward(ing) and mailing lists more 
important than allowing bad actors to continue pretending to be AOL or 
Yahoo?  Extremely likely.

IMHO AOL and Yahoo made a business decision.  Would you make the same 
business decision?  Maybe, maybe not.

Note:  Both AOL's and Yahoo's business decision works perfectly fine in 
my paradigm.

> Yes, there is a fundamental issue with email that it is easy to 
> spoof. Fixing it is going to be a significant issue, and possible a 
> complete recreation of the system.

I don't see a specific need to recreate the system.

> The issue is that to create such a new system is a major job. Such a 
> redesign would need to look at ALL current uses and either decide that 
> such uses were no longer valid or to accommodate them.

I am interested to see what others would propose that offers the same 
good points of our existing system (SMTP) without any (or at least 
fewer) bad points.

> DMARC somewhat intentionally did not consider mailing list, because they 
> didn?t have a good solution to handle them, and their intended usage, 
> the protection of ?valuable? mail somewhat excluded the use of such 
> services.

I think you and I have a fundamentally different view of what is being 
protected and not.

In my view, SPF, DKIM, and DMARC do a perfectly fine job of protecting 
messages between the sender and the mail recipient that they specify.

In your view (as I understand it), SPF, DKIM, and DMARC do a 
questionable job protecting messages between the sender and the ultimate 
mail recipient through an unknown number of intermediaries that may 
forward and / or expand to one or more other, different, recipients than 
the sender stated.

IMHO, much like STARTTLS protects a segment of the over all 
communications path between the sender and the ultimate recipient(s), 
SPF, DKIM, and DMARC only protect one portion.  And I happen to think 
that they do it well.

> It basically required that any service that wanted to use DMARC needed to 
> separate valuable protected mail from less valuable mail with different 
> domains. AOL and YAHOO just decided to ignore that in their use of it.

I disagree with the idea of /needing/ to apply different levels of 
security to different domains based on their (primary) use.  I have no 
objection if people /want/ to apply different levels of security to 
different domains.  IMHO they should not be required to do so.

I also object to the thought that individuals (humans) sending email 
aren't eligible to the same level of protection as other services 
(non-humans) because we as email administrators can't figure out how to 
make things work in a way that supports both.



-- 
Grant. . . .
unix || die


From johnl at taugh.com  Tue Jul 24 17:16:20 2018
From: johnl at taugh.com (John Levine)
Date: 24 Jul 2018 17:16:20 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
Message-ID: <20180724211621.0796F2002D0E92@ary.qy>

In article <78baab65-f7d3-ce56-bc36-a16a15118117 at spamtrap.tnetconsulting.net> you write:
>> If AOL and Yahoo just used the quarantine option for DMARC, it wouldn?t 
>> have been quite as bad. But they ABUSED DMARC by their settings.
>
>I still don't grok what you are considering "abuse" in this context?
>
>Rather than speculating, please clarify what the abusive activity was.

Turning it on for aol.com, yahoo.com, and other domains with user
mailboxes, to outsource the pain of the spam they were getting due
to letting user address books be stolen.

>My understanding is that AOL and Yahoo leveraged DMARC to expressly 
>identify messages that originated from AOL and Yahoo.  Or said another 
>way, they leveraged DMARC to make it easy for receiving servers to 
>identify messages that are not being sent from AOL or Yahoo servers 
>/during/ that current SMTP transaction.

Right, thereby causing a great deal of entirely legitimate mail that
DMARC cannot describe to go missing, along with a certain amount of
spam.  We've been cleaning up their mess ever since.

R's,
John

PS:

>Did they do so knowing that there would likely be a problem with 
>traditional .forward(ing) and mailing lists?  Quite likely.  Was an 
>internal business decision made that publishing such information and 
>dealing with the ramifications of .forward(ing) and mailing lists more 
>important than allowing bad actors to continue pretending to be AOL or 
>Yahoo?  Extremely likely.

Yes, they explicitly decided that the costs they imposed on
innocent bystanders were Not Their Problem.

From mark at msapiro.net  Tue Jul 24 17:29:59 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 24 Jul 2018 14:29:59 -0700
Subject: [Mailman-Users] Mailman 2.x to 3.1.x Migration
In-Reply-To: <VI1PR08MB3358D00F449BA49EE310CF92AF550@VI1PR08MB3358.eurprd08.prod.outlook.com>
References: <CAH=cwAOsVLLOmn3ZPxjPjTkS3hUKd2DE7tZ=X9uwPAjjk0rG1w@mail.gmail.com>
 <VI1PR08MB3358D00F449BA49EE310CF92AF550@VI1PR08MB3358.eurprd08.prod.outlook.com>
Message-ID: <e2fb0b62-8dba-aa10-870f-4881def80a7e@msapiro.net>

On 07/24/2018 06:32 AM, Andrew Hodgson wrote:
> 
> Once done you can see the results by navigating to your Mailman instance and you can see the data about the list such as the creation date, number of posts and the members.  For some reason on my instance the last post date is still the last post from Mailman2, even though it has been running on Mailman3 for a month or so.


This is <https://gitlab.com/mailman/mailman/issues/453>, fixed in
Mailman 3.2.


> Import archives:
> Use the command python manage.py hyperkitty_import to get the private mbox into your list.  I ran into 2 issues:
> 
> - If you have posts in the new archive on Mailman3 then you need to specify the earliest date you want to import into the archive as the command won't import anything older than the earliest date in the new archive.  If you run the command with the help option you can see the format required;


Yes. I've thought about adding a --all option to import the entire mbox
regardless of date, but for now I just specify --since='1990-01-01'.


> - There were older messages in the mbox without message-ids in the archive that failed to import.  I took the easy way out on this one and didn't import them.  In an archive with around 120,000 messages it rejected around 30 messages in all.


This is <https://gitlab.com/mailman/hyperkitty/issues/180>, fixed in
HyperKitty 1.2.0.

There are other issues that haven't yet been fixed other than by my own,
unmerged patches. See <https://gitlab.com/mailman/hyperkitty/issues/183>
and <https://gitlab.com/mailman/hyperkitty/issues/184>.

It really helps to have the import mbox as clean as possible before
starting. If it's a MM 2.1 mbox from a list created on MM 2.1.x, it's
probably pretty good, but if it's from a list created in MM 2.0 days, it
probably has issues. In any case, you need to check it for unescaped
'From ' lines with MM 2.1's bin/cleanarch. Even this isn't perfect, but
it helps.

Also, after you import archives, the Haystack search index will not
index those messages because they are not seen as new. Older versions of
hyperkitty_import would say that the Django minutely update_index job
would do it, but it won't. You can run

python manage.py update_index

to index those imported messages, but if you have a large installation
with multiple tens of thousands of archived messages, this can take
several hours. To help with this, there is now

python manage.py update_index_one_list

to do just a specified list. This too can take a long time (run it in
the background) if the imported archive was large, but it's often
preferable to doing the entire installation.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Tue Jul 24 18:59:08 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 24 Jul 2018 15:59:08 -0700
Subject: [Mailman-Users] Mailman 2.1.29 Release
Message-ID: <ee6359b9-146b-8dd6-e579-a8f9438a4d1b@msapiro.net>

I am not so pleased to announce the release of Mailman 2.1.29.

It turned out there was a bug in the security fix in 2.1.28 that broke
the web admin and listinfo overview pages. This is fixed in Mailman
2.1.29. The patch referred to below has been corrected to fix this bug.
There is also a patch attached to
<https://bugs.launchpad.net/mailman/+bug/1783417> which applies to
2.1.28 to fix this issue.

Python 2.6 is the minimum supported, but Python 2.7 is strongly recommended.

Mailman 2.1.28 was a minor security fix release. It also has some i18n
updates and a couple of bug fixes and adds the ability to edit list
specific templates through the web admin UI in a supported language
other than the list's default. See the attached README.txt for details.

For details of the security issue, see the report at
<https://bugs.launchpad.net/mailman/+bug/1780874> which also includes a
patch for those who want to fix this issue without upgrading.

Mailman is free software for managing email mailing lists and
e-newsletters. Mailman is used for all the python.org and
SourceForge.net mailing lists, as well as at hundreds of other sites.

For more information, please see our web site at one of:

http://www.list.org
https://www.gnu.org/software/mailman
http://mailman.sourceforge.net/
https://mirror.list.org/

Mailman 2.1.29 can be downloaded from

https://launchpad.net/mailman/2.1/
https://ftp.gnu.org/gnu/mailman/
https://sourceforge.net/projects/mailman/

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: README.txt
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20180724/dd9be0f4/attachment.txt>

From gtaylor at tnetconsulting.net  Tue Jul 24 20:20:59 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Tue, 24 Jul 2018 18:20:59 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180724211621.0796F2002D0E92@ary.qy>
References: <20180724211621.0796F2002D0E92@ary.qy>
Message-ID: <34e8162b-a6cb-1878-b6a6-46f68839c631@spamtrap.tnetconsulting.net>

On 07/24/2018 03:16 PM, John Levine wrote:
> Turning it on for aol.com, yahoo.com, and other domains with user 
> mailboxes,

So, are you stating that DMARC should NOT be used on domains that 
(predominantly) contain end user mailboxes?

> to outsource the pain of the spam they were getting

I'm not completely following you.  Are you referring to filtering of 
inbound email that AOL / Yahoo / etc. were having to do?  If so, I don't 
see how publishing DMARC effects that.  (I assume that they did not need 
to publish records to enhance filtering email from themselves.)  Or are 
you referring to "the pain" as being the push back / flack from the rest 
of the email industry for spoofed messages purporting to be from AOL / 
Yahoo / etc?

> due to letting user address books be stolen.

I don't know about AOL's security posture, but I do have a little idea 
about how bad Yahoo's was.  -  I still don't know that I would say that 
they allowed it, as in welcomed it.

IMHO it has been trivial to harvest email addresses for a LONG time.  As 
such, I think that address books are simply a convenient list and not 
strictly related.  Please correct me if I'm wrong.

> Right, thereby causing a great deal of entirely legitimate mail that 
> DMARC cannot describe to go missing, along with a certain amount of spam.

"legitimate mail that DMARC cannot describe"  That sounds distinctly 
like a problem with the DMARC specification, /not/ with use there of.

Aside:  The (relatively?) recent conversion from analog TV to digital TV 
broadcasting in the US comes to mind.

I feel like DMARC requires a paradigm shift in how email is forwarded 
and handled by mailing lists.  (I'm sure there are some other uses that 
I'm forgetting.)  Much like the aforementioned conversion from analog TV 
to digital TV.

Or similarly the requirement for reverse DNS for mail servers.  Personal 
opinions aside, it seems as if the email industry has decided that 
requiring reverse DNS is a mostly good thing.  Or that the good 
(slightly) outweighs the bad.

> We've been cleaning up their mess ever since.

I question if the mess is /really/ AOL's or Yahoo's doing, or if instead 
the problem was really related to (what I'm going to describe as) a 
questionable way of operating that we now must change to play well with 
DMARC.

> Yes, they explicitly decided that the costs they imposed on innocent 
> bystanders were Not Their Problem.

Please elaborate on what "the cost" is and entails.  Are you referring 
to anything more than the fallout of not being able to (easily) forward 
email in a DMARC compliant manner?

I suspect "imposed on innocent bystanders" and "not their problem" can 
also be used to describe requiring reverse DNS, SPF, and DKIM.



-- 
Grant. . . .
unix || die


From mark at msapiro.net  Tue Jul 24 20:51:58 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 24 Jul 2018 17:51:58 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <34e8162b-a6cb-1878-b6a6-46f68839c631@spamtrap.tnetconsulting.net>
References: <20180724211621.0796F2002D0E92@ary.qy>
 <34e8162b-a6cb-1878-b6a6-46f68839c631@spamtrap.tnetconsulting.net>
Message-ID: <90047fae-3fa6-3321-1a0b-ec0a9a3bb7ea@msapiro.net>

On 07/24/2018 05:20 PM, Grant Taylor via Mailman-Users wrote:
> On 07/24/2018 03:16 PM, John Levine wrote:
>> Turning it on for aol.com, yahoo.com, and other domains with user
>> mailboxes,
> 
> So, are you stating that DMARC should NOT be used on domains that
> (predominantly) contain end user mailboxes?


Many of us believe that DMARC was developed for domains such as
financial institutions and others in order to combat phishing attacks.
The developers of the DMARC standard never intended it to be used by
domains that provide email addresses for personal use.


>> to outsource the pain of the spam they were getting
> 
> I'm not completely following you.? Are you referring to filtering of
> inbound email that AOL / Yahoo / etc. were having to do?? If so, I don't
> see how publishing DMARC effects that.? (I assume that they did not need
> to publish records to enhance filtering email from themselves.)? Or are
> you referring to "the pain" as being the push back / flack from the rest
> of the email industry for spoofed messages purporting to be from AOL /
> Yahoo / etc?


The stolen address books were used to send phishing emails purportedly
from the owner of the address book the the addresses in the book.

I.e., a message From: a_known_friend at yahoo.com saying things look at
this great thing I found and a URL to evilsite.com.


> IMHO it has been trivial to harvest email addresses for a LONG time.? As
> such, I think that address books are simply a convenient list and not
> strictly related.? Please correct me if I'm wrong.


Trivial to harvest addresses, but not trivial to know a known associate
to send the mail From:.


> Please elaborate on what "the cost" is and entails.? Are you referring
> to anything more than the fallout of not being able to (easily) forward
> email in a DMARC compliant manner?
> 
> I suspect "imposed on innocent bystanders" and "not their problem" can
> also be used to describe requiring reverse DNS, SPF, and DKIM.


In this context, the innocents are subscribers to mailing lists who find
themselves unsubscribed by bounce processing because their ISPs reject
list posts From: other_users at yahoo.com and the operators of those
mailing lists.

Of course, you seem to feel that these lists were wrong from the
beginning for not claiming authorship of the posts by replacing the
From: header, but at the time, this wasn't even an option for most lists.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From Richard at Damon-family.org  Tue Jul 24 20:59:20 2018
From: Richard at Damon-family.org (Richard Damon)
Date: Tue, 24 Jul 2018 20:59:20 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
Message-ID: <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>



>> On Jul 24, 2018, at 3:20 PM, Grant Taylor via Mailman-Users <mailman-users at python.org> wrote:
>> 
>> On 07/22/2018 04:25 PM, Richard Damon wrote:
>> What actions do you think mailing lists are doing improperly?
> 
> I personally believe that mailing lists are their own end entity, just like our individual mailboxes.  (Particularly discussion mailing lists.)  I also believe that SPF, DKIM, and DMARC are meant to protect between said endpoints; message submitter and terminal mailbox.
> 
> Thus I think that DKIM and DMARC should be stripped from messages prior to entering the mailing list.  The mailing list does it's thing.  Then DKIM and DMARC are applied anew to the messages as they leave the server hosting the mailing list.

You CAN?T strip DMARC. If a domain has activated DMARC for itself (via its DNS record) then it is telling all other domains in the world that any mail that says it is from this domain MUST pass the DMARC test. This means that either it must be validly signed BY THEM or come from a server THEY have indicated is them. A setting of reject (which is what AOL and YAHOO use) indicates that other people are not to accept messages that appear to be from them that fail these tests.
> 
>> Note, the subject modification is a long standing feature of mailing list, which is one thing that breaks DMARC, though I might be willing to give that up.
> 
> Mailing lists, as I view them, are free to mung messages to their hearts content in the paradigm that I use.

See above, a re-mailer that changes a message fails DMARC.
> 
>> The modification of the message body to add a header or footer is also common, and in some places effectively required by law.
> 
> Agreed.
> 
> Such is perfectly compatible with my paradigm.

See above
> 
>> If AOL and Yahoo just used the quarantine option for DMARC, it wouldn?t have been quite as bad. But they ABUSED DMARC by their settings.
> 
> I still don't grok what you are considering "abuse" in this context?
> 
> Rather than speculating, please clarify what the abusive activity was.
> 
>> By the design of DMARC, AOL and Yahoo should have informed their users that they were changing the Terms of Service of their email systems, and now all their users are effectively prohibited to use any form of re-mailing systems, including most forms of (external) mailing lists. Instead they just told the world, we aren?t going to follow the normal rules, you deal with it.
> 
> I have a different interpretation.
> 
> My understanding is that AOL and Yahoo leveraged DMARC to expressly identify messages that originated from AOL and Yahoo.  Or said another way, they leveraged DMARC to make it easy for receiving servers to identify messages that are not being sent from AOL or Yahoo servers /during/ that current SMTP transaction.
> 
> I feel the need to insert a nod towards the fact that postmasters are free to run their infrastructure the way that they see fit.
> 
> I also do not feel like the terms of service between AOL or Yahoo and their end users changed.
> 
> AOL and Yahoo simply published information to make it easier for the world to identify if messages in the scope of an SMTP session are coming from AOL or Yahoo servers.  They also published their desire for receiving servers to reject messages that don't pass said published information.
> 
> Did they do so knowing that there would likely be a problem with traditional .forward(ing) and mailing lists?  Quite likely.  Was an internal business decision made that publishing such information and dealing with the ramifications of .forward(ing) and mailing lists more important than allowing bad actors to continue pretending to be AOL or Yahoo?  Extremely likely.
> 
> IMHO AOL and Yahoo made a business decision.  Would you make the same business decision?  Maybe, maybe not.
> 
> Note:  Both AOL's and Yahoo's business decision works perfectly fine in my paradigm.
> 

So you don?t see the problem with AOL and YAHOO changing there settings so that 99% of the discussion mailing list (guesings at percentage) are unable to deliver mail from subscribers who are AOL or YAHOO users, and if they try, they get back delivery errors that make the list software think that those users have bad email addresses and get unsubscribed for delivery errors. Those other systems now have a very tough choice, don?t honor the DMARC setting, and thus allow forgeries from banks and such to go through, or honor it to the detriment of their customers trying to use mailing lists. When they first did this and the problems started, one solution that was being proposed was to just kick all AOL and YAHOO users off all mailing lists. 

>> Yes, there is a fundamental issue with email that it is easy to spoof. Fixing it is going to be a significant issue, and possible a complete recreation of the system.
> 
> I don't see a specific need to recreate the system.
> 
>> The issue is that to create such a new system is a major job. Such a redesign would need to look at ALL current uses and either decide that such uses were no longer valid or to accommodate them.
> 
> I am interested to see what others would propose that offers the same good points of our existing system (SMTP) without any (or at least fewer) bad points.
> 
>> DMARC somewhat intentionally did not consider mailing list, because they didn?t have a good solution to handle them, and their intended usage, the protection of ?valuable? mail somewhat excluded the use of such services.
> 
> I think you and I have a fundamentally different view of what is being protected and not.
> 
> In my view, SPF, DKIM, and DMARC do a perfectly fine job of protecting messages between the sender and the mail recipient that they specify.
> 
> In your view (as I understand it), SPF, DKIM, and DMARC do a questionable job protecting messages between the sender and the ultimate mail recipient through an unknown number of intermediaries that may forward and / or expand to one or more other, different, recipients than the sender stated.
> 
> IMHO, much like STARTTLS protects a segment of the over all communications path between the sender and the ultimate recipient(s), SPF, DKIM, and DMARC only protect one portion.  And I happen to think that they do it well.

The problem with DMARC is that it DOES attempt to protect end-to-end. That works if you limit DMARC to ?valuable? emails that you can say shouldn?t go through things like mailing lists. That job it does well. The issue is when it is MISUSED  by a domain like YAHOO and used for email addresses which might be used on a mailing list.

SPF and DKIM actually work ok to as their base standards, as they accept matches to the envelope, so a message from my mailing list passes SPF and DKIM. The issue with DMARC is that it changes SPF and DKIM so that recipients need to check against the From: header in the message, so mailing list need to claim authorship of messages that pass through them to pass DMARC.
> 
>> It basically required that any service that wanted to use DMARC needed to separate valuable protected mail from less valuable mail with different domains. AOL and YAHOO just decided to ignore that in their use of it.
> 
> I disagree with the idea of /needing/ to apply different levels of security to different domains based on their (primary) use.  I have no objection if people /want/ to apply different levels of security to different domains.  IMHO they should not be required to do so.
> 
> I also object to the thought that individuals (humans) sending email aren't eligible to the same level of protection as other services (non-humans) because we as email administrators can't figure out how to make things work in a way that supports both.
> 
> -- 
> Grant. . . .
> unix || die
> 

DMARC says that if you get a message from me, it MUST have come straight from me without ANY modifications. This is a reasonable thing for things like a bank statement or a bill, that shouldn?t pass through something that changes the message. As you said above, those modification are reasonable for a mailing list. That says that DMARC wasn?t the right solution for that part of the problem. 

In some ways it is a bit like saying I want to be safer on the highway, so I am going to drive a tank, and who cares what it does to everyone else (and the road). 

The issue with different types of email using different levels of security is that some types of email can reasonably use some techniques that are just not appropriate for others due to the type and use of that email. 

From gtaylor at tnetconsulting.net  Tue Jul 24 21:09:01 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Tue, 24 Jul 2018 19:09:01 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <90047fae-3fa6-3321-1a0b-ec0a9a3bb7ea@msapiro.net>
References: <20180724211621.0796F2002D0E92@ary.qy>
 <34e8162b-a6cb-1878-b6a6-46f68839c631@spamtrap.tnetconsulting.net>
 <90047fae-3fa6-3321-1a0b-ec0a9a3bb7ea@msapiro.net>
Message-ID: <e65d2758-a421-785f-64df-cb1215964c82@spamtrap.tnetconsulting.net>

On 07/24/2018 06:51 PM, Mark Sapiro wrote:
> The stolen address books were used to send phishing emails purportedly 
> from the owner of the address book the the addresses in the book.
> 
> I.e., a message From: a_known_friend at yahoo.com saying things look at 
> this great thing I found and a URL to evilsite.com.
> 
> Trivial to harvest addresses, but not trivial to know a known associate 
> to send the mail From:.

I hadn't thought about the association of the metadata.  Thank you for 
clarifying.

I do question how much more spam was sent by stealing address books from 
large providers compared to viruses / malware doing the same with 
address books on infected machines.

> In this context, the innocents are subscribers to mailing lists who 
> find themselves unsubscribed by bounce processing because their ISPs 
> reject list posts From: other_users at yahoo.com and the operators of those 
> mailing lists.

Indeed, unfortunately "friendly fire".  :-/

> Of course, you seem to feel that these lists were wrong from the beginning 
> for not claiming authorship of the posts by replacing the From: header,

Yes, that's in line with my current view.

> but at the time, this wasn't even an option for most lists.

Lack of an option does not preclude the need for it.

Similarly, ignorance of an option does not preclude the need for it.

Admittedly, I've long struggled with how I thought discussion mailing 
lists should behave.  Originally I hadn't given any thought to munging 
the From: like is suggested for DMARC.  That being said, I did want to 
direct replies back to the discussion list.



-- 
Grant. . . .
unix || die


From gtaylor at tnetconsulting.net  Tue Jul 24 21:43:43 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Tue, 24 Jul 2018 19:43:43 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
Message-ID: <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>

On 07/24/2018 06:59 PM, Richard Damon wrote:
> You CAN?T strip DMARC.

I can most certainly strip any DKIM related headers from messages that 
are coming into my server on their way to my mailing list.

I'm not talking about altering other people's view of DNS.  (That's a 
completely different topic.)

> If a domain has activated DMARC for itself (via its DNS record) then it 
> is telling all other domains in the world that any mail that says it is 
> from this domain MUST pass the DMARC test.  This means that either it 
> must be validly signed BY THEM or come from a server THEY have indicated 
> is them. A setting of reject (which is what AOL and YAHOO use) indicates 
> that other people are not to accept messages that appear to be from them 
> that fail these tests.

Agreed.

I'm saying that the email server hosting the mailing list should strip 
DKIM (related) headers from messages before they go into the mailing 
list software.

I'm also saying that said email server should apply all the enforcement 
checks that you aptly described.


> See above, a re-mailer that changes a message fails DMARC.

I believe the SMTP path between the originating sender and the mailing 
list is distinct and completely different from the separate SMTP path 
between the mailing list and subscribers servers.

I'm free to make any and all changes to the message as it passes through 
the mailing list as long as I do the following:

1)  Remove any and all security headers from messages going into the 
mailing list.
2)  I (re)add appropriate security headers to messages exiting from the 
mailing list.  Note:  These headers should reflect the mailing list and 
NOT the original sender.

> See above

Hypothetical scenario:

Message from subscriber at aol.com comes into my email server hosting a 
Mailman mailing list.

1)  Apply all applicable filters (reverse DNS, SPF, DKIM, DMARC, Spam, 
Virus, etc) as early in the SMTP process as possible.  Reject (as 
appropriate) anything that fails respective tests.
2)  Strip all applicable security headers between the MTA and the MLM.
3)  Mailing list manager does it's thing, including munging the From: as 
it generates new messages that go out through the local MTA.
4)  Local MTA adds appropriate new security headers to the messages.

Note:  #4 absolutely MUST use data that reflects the local domain / 
sending server.

I still see nothing that prevents the MLM of doing anything and 
everything that it wants to do to the messages that pass through it.

Note:  There is ZERO association between the inbound message and the 
many outbound messages, save for body content being based on the 
original incoming message.

> So you don?t see the problem with AOL and YAHOO changing there settings 
> so that 99% of the discussion mailing list (guesings at percentage) are 
> unable to deliver mail from subscribers who are AOL or YAHOO users, and 
> if they try, they get back delivery errors that make the list software 
> think that those users have bad email addresses and get unsubscribed 
> for delivery errors.

I think your percentage is high.  I just don't know how high.  But that 
nuance doesn't really matter.

I see what was being done before (and some still do now) as a problem. 
A problem that pre-existed AOL and Yahoo or their use of DMARC.  They 
just happened to be early adopters that did a cannon ball into the 
otherwise relatively calm pool.

I believe that similar problems happened when people started using -all 
in their SPF records too.  Granted, VERP, which had been adopted by many 
mailing lists, altered that somewhat.

I view the equations as being the same, just with significantly 
different values in the variables.

> Those other systems now have a very tough choice, don?t honor the DMARC 
> setting, and thus allow forgeries from banks and such to go through, 
> or honor it to the detriment of their customers trying to use mailing 
> lists.

I believe that mailing lists (or their hosting MTAs) SHOULD do DMARC 
(and DKIM and SPF) filtering in as strict a manner as the purported 
sending domain desires.

Thus blocking the undesirable messages that you refer to.

> When they first did this and the problems started, one solution that 
> was being proposed was to just kick all AOL and YAHOO users off all 
> mailing lists.

I do not believe that the problem started then.  It may very well be 
that it became well enough known as enough people were effected that 
people that didn't know about it before suddenly learned about it.

I believe the problem was known about before AOL and Yahoo implemented 
DMARC.

I feel like kicking AOL and Yahoo users off of mailing lists is an 
unnecessary Draconian knee jerk reaction.  Something that should be 
avoided at all costs.

> The problem with DMARC is that it DOES attempt to protect end-to-end.

I obviously disagree.  Particularly on what the receiving "end" is.

If you view the message to am MLM as as separate end-to-end delivery 
process from the message from an MLM to the subscriber, DMARC can and 
does work with MLMs.

> That works if you limit DMARC to ?valuable? emails that you can 
> say shouldn?t go through things like mailing lists. That job it does 
> well.

I think that any solution should work equally well when applied to all 
subsets (types of email) of the technology it applies to (email in general).

> The issue is when it is MISUSED  by a domain like YAHOO and used for 
> email addresses which might be used on a mailing list.

I have no problem with what AOL and Yahoo did.

I think they used a tool in the way that it was meant to be used.

I do agree that there was undesired repercussions.  But I believe said 
repercussions were because MLMs didn't do what I think they should have 
done.

> SPF and DKIM actually work ok to as their base standards, as they 
> accept matches to the envelope, so a message from my mailing list passes 
> SPF and DKIM.

Messages from an MLM will NOT pass SPF checks if -all is used -and- the 
MLM is not altering the SMTP envelope's from address.

The original DKIM signature may or may not pass through an MLM, 
depending on how the MLM is configured.

> The issue with DMARC is that it changes SPF and DKIM so that recipients 
> need to check against the From: header in the message, so mailing list 
> need to claim authorship of messages that pass through them to pass DMARC.

I have no problem with that.

Look at steps 1 through 4 above and you will see that I think MLMs (or 
their hosting MTAs) SHOULD do this.

> DMARC says that if you get a message from me, it MUST have come straight 
> from me without ANY modifications.

EXACTLY.

That means that there's NO forwarding or passing through an MLM.

If the message doesn't come straight from you (or your delegates), the 
message is NOT from you.

> This is a reasonable thing for things like a bank statement or a bill, 
> that shouldn?t pass through something that changes the message.

I think it's also reasonable for things like AOL and Yahoo.

> As you said above, those modification are reasonable for a mailing list.

Yep.  I do think those modifications are reasonable for mailing lists.

> That says that DMARC wasn?t the right solution for that part of the 
> problem.

No.

There is nothing that precludes the mailing list from adding it's own 
security (SPF, DKIM, DMARC).

I also strongly suggest that the MLM remove any associated inbound 
security (after responding accordingly).

Thus the /content/ of the message can be copied from the incoming 
message to 10s / 100s / 1,000s of new completely independent outbound 
messages.  The key being the /content/ not the raw message itself.

> In some ways it is a bit like saying I want to be safer on the highway, 
> so I am going to drive a tank, and who cares what it does to everyone else 
> (and the road).

If said tank meets all the following criteria, go for it:

1)  It's licenses.  (I'm assuming that it is for the sake of this 
conversation.)
2)  It's within weight limits.
3)  It's within height, width, and length limits.
4)  It doesn't do any damage to the road.  (Read:  rubber road tracks.)
5)  You don't do anything threatening with it.

If you meet those requirements, then knock yourself out.

> The issue with different types of email using different levels of security 
> is that some types of email can reasonably use some techniques that are 
> just not appropriate for others due to the type and use of that email.

I disagree.



-- 
Grant. . . .
unix || die


From Richard at Damon-family.org  Tue Jul 24 22:11:42 2018
From: Richard at Damon-family.org (Richard Damon)
Date: Tue, 24 Jul 2018 22:11:42 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
Message-ID: <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>



> On Jul 24, 2018, at 9:43 PM, Grant Taylor via Mailman-Users <mailman-users at python.org> wrote:
> 
>> On 07/24/2018 06:59 PM, Richard Damon wrote:
>> You CAN?T strip DMARC.
> 
> I can most certainly strip any DKIM related headers from messages that are coming into my server on their way to my mailing list.
> 
> I'm not talking about altering other people's view of DNS.  (That's a completely different topic.)

Do you understand how DMARC works? Yahoo.com has an entry in their DNS that says they want DMARC protection, and if you can?t verify that the message came from them unmodified to reject it. 
> 
>> If a domain has activated DMARC for itself (via its DNS record) then it is telling all other domains in the world that any mail that says it is from this domain MUST pass the DMARC test.  This means that either it must be validly signed BY THEM or come from a server THEY have indicated is them. A setting of reject (which is what AOL and YAHOO use) indicates that other people are not to accept messages that appear to be from them that fail these tests.
> 
> Agreed.
> 
> I'm saying that the email server hosting the mailing list should strip DKIM (related) headers from messages before they go into the mailing list software.

Which doesn?t help with DMARC.

> 
> I'm also saying that said email server should apply all the enforcement checks that you aptly described.
> 
> 
>> See above, a re-mailer that changes a message fails DMARC.
> 
> I believe the SMTP path between the originating sender and the mailing list is distinct and completely different from the separate SMTP path between the mailing list and subscribers servers.
> 
> I'm free to make any and all changes to the message as it passes through the mailing list as long as I do the following:
> 
> 1)  Remove any and all security headers from messages going into the mailing list.
> 2)  I (re)add appropriate security headers to messages exiting from the mailing list.  Note:  These headers should reflect the mailing list and NOT the original sender.

Unless the mailing list claims authorship of the message by changing the From: of the message (and thus making it hard to tell who really said the words), the list relaying the message with the slightest modification of the Subject or Body will cause it to fail DMARC, as DMARC says that the From: header is the king for verification.

> 
>> See above
> 
> Hypothetical scenario:
> 
> Message from subscriber at aol.com comes into my email server hosting a Mailman mailing list.
> 
> 1)  Apply all applicable filters (reverse DNS, SPF, DKIM, DMARC, Spam, Virus, etc) as early in the SMTP process as possible.  Reject (as appropriate) anything that fails respective tests.
> 2)  Strip all applicable security headers between the MTA and the MLM.
> 3)  Mailing list manager does it's thing, including munging the From: as it generates new messages that go out through the local MTA.
> 4)  Local MTA adds appropriate new security headers to the messages.
> 
> Note:  #4 absolutely MUST use data that reflects the local domain / sending server.
> 
> I still see nothing that prevents the MLM of doing anything and everything that it wants to do to the messages that pass through it.
> 
> Note:  There is ZERO association between the inbound message and the many outbound messages, save for body content being based on the original incoming message.

Only if you think that mailman-users is the author of your message here, and that your mailing list is the proper author of every message that goes through your mailing list.
> 
>> So you don?t see the problem with AOL and YAHOO changing there settings so that 99% of the discussion mailing list (guesings at percentage) are unable to deliver mail from subscribers who are AOL or YAHOO users, and if they try, they get back delivery errors that make the list software think that those users have bad email addresses and get unsubscribed for delivery errors.
> 
> I think your percentage is high.  I just don't know how high.  But that nuance doesn't really matter.
> 
> I see what was being done before (and some still do now) as a problem. A problem that pre-existed AOL and Yahoo or their use of DMARC.  They just happened to be early adopters that did a cannon ball into the otherwise relatively calm pool.
> 
> I believe that similar problems happened when people started using -all in their SPF records too.  Granted, VERP, which had been adopted by many mailing lists, altered that somewhat.
> 
> I view the equations as being the same, just with significantly different values in the variables.

Base SPF isn?t an issue. All messages leaving my mailing list pass SPF because I publish a SPF record, and the message have an envelope From of my mailing list.
> 
>> Those other systems now have a very tough choice, don?t honor the DMARC setting, and thus allow forgeries from banks and such to go through, or honor it to the detriment of their customers trying to use mailing lists.
> 
> I believe that mailing lists (or their hosting MTAs) SHOULD do DMARC (and DKIM and SPF) filtering in as strict a manner as the purported sending domain desires.
> 
> Thus blocking the undesirable messages that you refer to.

Again, I can verify the DMARC of the incoming message, but unless I want to claim authorship by changing the From, I can not send it and have it pass DMARC.
> 
>> When they first did this and the problems started, one solution that was being proposed was to just kick all AOL and YAHOO users off all mailing lists.
> 
> I do not believe that the problem started then.  It may very well be that it became well enough known as enough people were effected that people that didn't know about it before suddenly learned about it.
> 
> I believe the problem was known about before AOL and Yahoo implemented DMARC.
> 
> I feel like kicking AOL and Yahoo users off of mailing lists is an unnecessary Draconian knee jerk reaction.  Something that should be avoided at all costs.
> 
>> The problem with DMARC is that it DOES attempt to protect end-to-end.
> 
> I obviously disagree.  Particularly on what the receiving "end" is.
> 
> If you view the message to am MLM as as separate end-to-end delivery process from the message from an MLM to the subscriber, DMARC can and does work with MLMs.

Only if you consider the mailing list the Author of every message relayed by it.
> 
>> That works if you limit DMARC to ?valuable? emails that you can say shouldn?t go through things like mailing lists. That job it does well.
> 
> I think that any solution should work equally well when applied to all subsets (types of email) of the technology it applies to (email in general).
> 
>> The issue is when it is MISUSED  by a domain like YAHOO and used for email addresses which might be used on a mailing list.
> 
> I have no problem with what AOL and Yahoo did.
> 
> I think they used a tool in the way that it was meant to be used.
> 
> I do agree that there was undesired repercussions.  But I believe said repercussions were because MLMs didn't do what I think they should have done.
> 
>> SPF and DKIM actually work ok to as their base standards, as they accept matches to the envelope, so a message from my mailing list passes SPF and DKIM.
> 
> Messages from an MLM will NOT pass SPF checks if -all is used -and- the MLM is not altering the SMTP envelope's from address.
> 

The MLM DOES change the Envelope from, it really wants to so it gets the bounces back so it can process it. That means the outgoing message can pass SPF as SPF is written. What it doesn?t pass is the modification to SPF that DMARC specifies that says that the only domain to validate in the inside From: Header, the Envelope doesn?t count.

> The original DKIM signature may or may not pass through an MLM, depending on how the MLM is configured.
> 
>> The issue with DMARC is that it changes SPF and DKIM so that recipients need to check against the From: header in the message, so mailing list need to claim authorship of messages that pass through them to pass DMARC.
> 
> I have no problem with that.
> 
> Look at steps 1 through 4 above and you will see that I think MLMs (or their hosting MTAs) SHOULD do this

So you REALY want to see your view of the mailing list as EVERY message is ?From? Mailman-users, with no indication of who wrote really wrote the message? Thus you lose the ability to easily block  
> .
> 
>> DMARC says that if you get a message from me, it MUST have come straight from me without ANY modifications.
> 
> EXACTLY.

So you don?t think mailing list should do any modifications to the message, or they need to claim authorship.
> 
> That means that there's NO forwarding or passing through an MLM.
> 
> If the message doesn't come straight from you (or your delegates), the message is NOT from you.

So you see this thread as the mailing list arguing with itself?

>> This is a reasonable thing for things like a bank statement or a bill, that shouldn?t pass through something that changes the message.
> 
> I think it's also reasonable for things like AOL and Yahoo.

Only if the TELL ALL there users that they have effectively should not use virtually any of the existing mailing lists (except of course for yahoo users using yahoo groups, as yahoo knows enough to be able to make those pass)

> 
>> As you said above, those modification are reasonable for a mailing list.
> 
> Yep.  I do think those modifications are reasonable for mailing lists.
> 
>> That says that DMARC wasn?t the right solution for that part of the problem.
> 
> No.
> 
> There is nothing that precludes the mailing list from adding it's own security (SPF, DKIM, DMARC).
> 
> I also strongly suggest that the MLM remove any associated inbound security (after responding accordingly).
> 
> Thus the /content/ of the message can be copied from the incoming message to 10s / 100s / 1,000s of new completely independent outbound messages.  The key being the /content/ not the raw message itself.

Should they also be given new message-ids (as they are new messages) and thus threaded views not work anymore?
> 
>> In some ways it is a bit like saying I want to be safer on the highway, so I am going to drive a tank, and who cares what it does to everyone else (and the road).
> 
> If said tank meets all the following criteria, go for it:
> 
> 1)  It's licenses.  (I'm assuming that it is for the sake of this conversation.)
> 2)  It's within weight limits.
> 3)  It's within height, width, and length limits.
> 4)  It doesn't do any damage to the road.  (Read:  rubber road tracks.)
But DMARC is allowed to damage the Email system 
> 5)  You don't do anything threatening with it.
> 
> If you meet those requirements, then knock yourself out.
> 
>> The issue with different types of email using different levels of security is that some types of email can reasonably use some techniques that are just not appropriate for others due to the type and use of that email.
> 
> I disagree.
> 
> 
> 
> -- 
> Grant. . . .
> unix || die
> 
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/richard%40damon-family.org


From mark at msapiro.net  Tue Jul 24 22:34:28 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 24 Jul 2018 19:34:28 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
Message-ID: <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>

On 07/24/2018 07:11 PM, Richard Damon wrote:
> 
> 
>> On Jul 24, 2018, at 9:43 PM, Grant Taylor via Mailman-Users <mailman-users at python.org> wrote:
>>
>> If you view the message to am MLM as as separate end-to-end delivery process from the message from an MLM to the subscriber, DMARC can and does work with MLMs.
> 
> Only if you consider the mailing list the Author of every message relayed by it.


To elaborate a bit. RFC 5322 says

   The "From:" field specifies the author(s) of the message,
   that is, the mailbox(es) of the person(s) or system(s) responsible
   for the writing of the message.  The "Sender:" field specifies the
   mailbox of the agent responsible for the actual transmission of the
   message.  For example, if a secretary were to send a message for
   another person, the mailbox of the secretary would appear in the
   "Sender:" field and the mailbox of the actual author would appear in
   the "From:" field.

In that sense, many of us think that the person who wrote the post is
still the author even if the list made a few simple changes that didn't
alter the basic text of the original message while the list is a Sender:

That's why we believe that Munge From is non-compliant. You may think
that the list is in fact "the person(s) or system(s) responsible for the
writing of the message", but many people don't agree.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From johnl at taugh.com  Tue Jul 24 22:41:54 2018
From: johnl at taugh.com (John Levine)
Date: 24 Jul 2018 22:41:54 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
Message-ID: <20180725024155.7B87E2002D2226@ary.qy>

In article <88902b3b-7cb3-7991-15c4-4dbc107624b8 at msapiro.net> you write:
>In that sense, many of us think that the person who wrote the post is
>still the author even if the list made a few simple changes that didn't
>alter the basic text of the original message while the list is a Sender:
>
>That's why we believe that Munge From is non-compliant. ...

Quite right.  Beyond the standards theology, there is the practical
problem that where the message list in your inbox used to tell you who
wrote the list messages, now it all seems to come from the list alias.
In my world, some people's contributions are a lot more interesting
than others, and losing the info about who wrote what makes all lists
less useful.

R's,
John

From gtaylor at tnetconsulting.net  Tue Jul 24 23:47:03 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Tue, 24 Jul 2018 21:47:03 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
Message-ID: <80d45ee1-1047-cbcc-1de2-51e5b225c8a9@spamtrap.tnetconsulting.net>

On 07/24/2018 08:11 PM, Richard Damon wrote:
> Do you understand how DMARC works?

Yes, I do believe that I do understand how DMARC works.

I have yet to have someone show me something (else) about DMARC that I'm 
not aware of.

> Yahoo.com has an entry in their DNS that says they want DMARC protection, 
> and if you can?t verify that the message came from them unmodified to 
> reject it.

Yep.

I'm doing exactly that.

> Unless the mailing list claims authorship of the message by changing the 
> From: of the message (and thus making it hard to tell who really said the 
> words), the list relaying the message with the slightest modification 
> of the Subject or Body will cause it to fail DMARC, as DMARC says that 
> the From: header is the king for verification.

I am talking about modifying the From: header such that the message no 
longer had any conflict with the original published DMARC records.

I.e.

      From: Grant Taylor <gtaylor at tnetconsulting.net>

Becomes:

      From: Grant Taylor via Mailman-Users <mailman-users at python.org>

Thus removing any conflict with any DMARC records published by 
tnetconsulting.net

Since the message is now from the Mailman-Users mailing list, it's 
perfectly possible to insert a line at the start of the message like the 
following:

      Grant Taylor <gtaylor at tnetconsulting.net> wrote the following:

> Only if you think that mailman-users is the author of your message here, 
> and that your mailing list is the proper author of every message that 
> goes through your mailing list.

I believe that the Mailman-Users mailing list is the entity responsible 
for sending the message to each and every subscriber.  I believe the 
content that the Mailman-Users mailing list is sending is strongly based 
on content provided by someone that sent a message to said mailing list.

I know that the mailing list did not generate the content.  I also know 
that it is sending content heavily based on content from someone else.

> Base SPF isn?t an issue. All messages leaving my mailing list pass 
> SPF because I publish a SPF record, and the message have an envelope 
> From of my mailing list.

What is (was) your (original) motivation for munging the envelope to be 
from the mailing list?  Are (were) you (originally) doing it because you 
want to take advantage of V.E.R.P.?  Or are (were) you (originally) 
doing it to avoid SPF issues?

I know a number of people that only started munging the envelope from 
address because of SPF issues.

You may also run into issues with SPF alignment with DMARC if you don't 
also modify the From: header.

(I can't tell what domain you are referring to.  I don't see SPF / TXT 
records for damon-family.org and I don't know if you are referring to 
some other domain.)

> Again, I can verify the DMARC of the incoming message, but unless I want 
> to claim authorship by changing the From, I can not send it and have it 
> pass DMARC.

Which, IMHO, is what DMARC is supposed to be able to enforce.

> Only if you consider the mailing list the Author of every message relayed 
> by it.

I do consider the MLM as being the author / creator / submitter of the 
SMTP message.

I view the person that sent the message as being the author / creator / 
submitter of the body content in said SMTP message.

> The MLM DOES change the Envelope from, it really wants to so it gets the 
> bounces back so it can process it. That means the outgoing message can 
> pass SPF as SPF is written. What it doesn?t pass is the modification 
> to SPF that DMARC specifies that says that the only domain to validate 
> in the inside From: Header, the Envelope doesn?t count.

Yep, VERP.

> So you REALY want to see your view of the mailing list as EVERY message 
> is ?From? Mailman-users, with no indication of who wrote really 
> wrote the message? Thus you lose the ability to easily block

Not quite.

I would much rather have the human friendly portion of the address 
remain what was originally sent.

I.e.

      From: Grant Taylor <gtaylor at tnetconsulting.net>

Becomes:

      From: Grant Taylor via Mailman-Users <mailman-users at python.org>

I would also be interested in something like the following.

      From: Grant Taylor gtaylor at tnetconsulting dot net via 
Mailman-Users <mailman-users at python.org>

I believe that retains the attribution that I believe you (and many 
others) want to retain.

Seeing as how the new outgoing message is completely new, it's perfectly 
possible to add something like the following as the first two lines of 
the message:

      Grant Taylor <gtaylor at tnetconsulting.net> wrote the following:

> So you don?t think mailing list should do any modifications to the 
> message, or they need to claim authorship.

"DMARC says that if you get a message from me, it MUST have come 
straight from me"

The key being "it MUST have come straight from me".

Thus messages that pass through a mailing list (or forwarded in any way) 
fail the "come straight from me" portion.

> So you see this thread as the mailing list arguing with itself?

I see this thread as a friendly / academic discussion from many 
different mailing list subscribers who send messages to and receive 
messages from said mailing list.

> Only if the TELL ALL there users that they have effectively should not 
> use virtually any of the existing mailing lists (except of course for 
> yahoo users using yahoo groups, as yahoo knows enough to be able to make 
> those pass)

I disagree.

> Should they also be given new message-ids (as they are new messages) 
> and thus threaded views not work anymore?  But DMARC is allowed to damage 
> the Email system

I am (currently) about 70/30 on if messages from the mailing list should 
get new Message-IDs or not.

If all messages pass through and everybody replies to the mailing list 
manager, then the new Message-ID from the MLM will work perfectly fine.

Original message:

      From: Grant Taylor <gtaylor at tnetconsulting.net>
      To: Mailman-Users <mailman-users at python.org>
      Message-ID: <68fd1dbf-eca3-4924-9531-cbf84d3f3bec at tnetconsulting.net>

Message from MLM:

      From: Grant Taylor via Mailman-Users <mailman-users at python.org>
      To: $Subscriber
      Message-ID: 
<ad933a76-faaf-41b3-bc36-2b5ac527ec2c at mailman-users.python.org>

Reply back to MLM:

      From: Grant Taylor <gtaylor at tnetconsulting.net>
      To: Mailman-Users <mailman-users at python.org>
      Message-ID: <70ef8f26-dcc2-48e7-a45e-be6fef4ea4fe
@tnetconsulting.net>
      References: 
<ad933a76-faaf-41b3-bc36-2b5ac527ec2c at mailman-users.python.org>

The only problem that I see is other people that are explicitly listed 
on the To: or CC lines.  (I'm ignoring BCCs.)

Allow me to restate:

I believe that ALL messages to / from a /discussion/ mailing list should 
go through said mailing list.



-- 
Grant. . . .
unix || die


From gtaylor at tnetconsulting.net  Tue Jul 24 23:56:26 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Tue, 24 Jul 2018 21:56:26 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <20180725024155.7B87E2002D2226@ary.qy>
References: <20180725024155.7B87E2002D2226@ary.qy>
Message-ID: <22b3046f-c6ab-37ea-b136-3714accad54c@spamtrap.tnetconsulting.net>

On 07/24/2018 08:41 PM, John Levine wrote:
> Quite right.  Beyond the standards theology, there is the practical 
> problem that where the message list in your inbox used to tell you who 
> wrote the list messages, now it all seems to come from the list alias.

That's where the human friendly portion of the From: header comes into play.

I.e.

      From: Grant Taylor <gtaylor at tnetconsulting.net>

Becomes:

      From: Grant Taylor via Mailman-Users <mailman-users at python.org>

This can show up in the index of a mailbox.

Or the two lines that I'm suggesting prefixing the body with.

      Grant Taylor <gtaylor at tnetconsulting.net> wrote the following:

Granted, that doesn't show up in the index of a mailbox.

Both methods tell you who wrote the message content.

Do you have any doubt about who wrote the first four lines of this 
message?  Does the fact that the From: header has my name (via 
Mailman-Users) cause you to believe that I wrote "Quite right.  ? list 
alias."?

Or does the fact that there is quoting "> " change that?

> In my world, some people's contributions are a lot more interesting 
> than others,

Agreed.

> and losing the info about who wrote what makes all lists less useful.

I'm not suggesting losing information about who wrote what.



-- 
Grant. . . .
unix || die


From mark at msapiro.net  Wed Jul 25 00:24:23 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 24 Jul 2018 21:24:23 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <80d45ee1-1047-cbcc-1de2-51e5b225c8a9@spamtrap.tnetconsulting.net>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
 <80d45ee1-1047-cbcc-1de2-51e5b225c8a9@spamtrap.tnetconsulting.net>
Message-ID: <9a07e5ae-3636-acd0-6d95-b6b3810fbaab@msapiro.net>

On 07/24/2018 08:47 PM, Grant Taylor via Mailman-Users wrote:
> I am talking about modifying the From: header such that the message no
> longer had any conflict with the original published DMARC records.


Consider this:

If I create a novel called _The Hexadecimal Kid and His Faithful Dog
ASCII_ [1] and submit the manuscript to a publisher and the publisher
type sets it, prints thousands of bound copies with a title page with
added information about the publisher and maybe adds a subtitle and some
info on a dust jacket, and delivers those printed books to book sellers,
has the publisher now become the author of the book or am I still the
author?

RFC 5322 is clear

   The "From:" field specifies the author(s) of the message,
   that is, the mailbox(es) of the person(s) or system(s) responsible
   for the writing of the message.

I.e. the mailbox(es) in the From: header are those of the author. It
doesn't matter if the author is in a comment or display name, if the
mailbox is that of the list, you are claiming the list is the author of
the post.

I'm not saying it is not expedient or even possibly necessary to Munge
the From: in this way. I'm only saying it is a violation of RFC 5322
unless you believe the publisher is author of my book.

[1] Bonus points for anyone who gets the reference.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From turnbull.stephen.fw at u.tsukuba.ac.jp  Wed Jul 25 05:53:40 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 25 Jul 2018 18:53:40 +0900
Subject: [Mailman-Users] ARC
In-Reply-To: <03d440cf-7cd5-4564-2900-ffc08cd7849b@spamtrap.tnetconsulting.net>
References: <20180722200514.EA5FA20028DC81@ary.qy>
 <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>
 <23381.24927.74482.530338@turnbull.sk.tsukuba.ac.jp>
 <03d440cf-7cd5-4564-2900-ffc08cd7849b@spamtrap.tnetconsulting.net>
Message-ID: <23384.18596.617417.251765@turnbull.sk.tsukuba.ac.jp>

Grant Taylor via Mailman-Users writes:

 > I would think / hope / expect that such services would be from a 
 > different (sub)domain of the client that they are sending email on 
 > behalf of.

That's not how "on behalf of" worked in practice.  What happened in
April 2014, was that a home business owner (HBO) would send a pile of
completed order notices to intuit.com, and intuit.com would send an
invoice to each customer on behalf of hbo at yahoo.com, from
hbo at yahoo.com.  HBO, of course, doesn't have a vanity domain, just a
Wordpress or SquareSite (or even Facebook) home page.  Tens of
thousands of invoices worth millions of dollars bounced off of
personal accounts and tiny business owner accounts at Yahoo! and other
receivers who take p=reject as a command.

Note that if I were intuit.com's CISO, I would fight tooth and nail
against the system you suggest, because it implies that I have DKIM
private keys for all those subdomains owned by clients.  Every spammer
in the world would be trying to hack the server that has those keys.
I could probably keep them out, but Lordy, the liability involved!

Less financially painful are the services that newspapers and tourist
destinations provided (note past tense, they're mostly dead now) where
you could sit at a terminal and send a "postcard" recommending an
article or with a picture of the attraction to family and friends
"from" yourself, including your email address, simply by typing name
and address in.  Not a huge loss, I guess, but ....

 > > The other *possible* use case for ARC would be non-mailing list 
 > > forwarding.  But these almost never break the DKIM signature of the 
 > > originator.
 > 
 > They may not break DKIM.  But depending on how they operate, they may 
 > break SPF directly (re-sending with the original SMTP envelope From: 
 > thus violating SPF) -or- indirectly (re-sending with something like SRS)

I've never seen SRS in the wild, except in the headers of some of the
crazier denizens of IETF mailing lists.  YMMV, of course.

 > thus breaking DMARC alignment.

Simple .forwarding breaks SPF of the author domain, period, unless
entirely within one administrative domain, and that domain is
well-configured so that all the forwarding is internal, and the SPF
checks are all done at the boundary.

I guess in the case where forwarding takes place entirely within an
administrative domain, DMARC From alignment could be broken without
breaking SPF itself because From alignment is validated on a different
host from SPF, but that sounds like a case of "evolution in action".

 > My understanding is that DMARC can be configured to require both SPF and 
 > DKIM alignment.

That's non-conforming to RFC 7489.  Section 4.2:

    A message satisfies the DMARC checks if AT LEAST ONE of the supported
    authentication mechanisms:

    1.  produces a "pass" result, and

    2.  produces that result based on an identifier that is in alignment,
        as defined in Section 3.

[Emphasis mine.]

 > The point being that simple .forward(ing) may still break things.
 > 
 > I maintain that detecting such is one of the functional purposes of 
 > DMARC.

Of course.  That's why it's not "DMAC" (although either would be a
great street name for a hip-hop DJ!)

 > I question the wisdom of making processing of ARC conditional on RFC 
 > 2369 List-* headers.  I mainly say this because there is nothing that 
 > prevents malicious actors from inserting (possibly bogus) List-* 
 > headers.  (Or lots of tiny lists of single recipients.)

I'm afraid I failed to make the concept explicit again.  Remember,
authenticating mail origin doesn't prevent anyone from sending
malicious mail, or even from having it delivered.  So while I suggest
that ARC processing results be taken seriously in the presence of
(List-* headers) for DMARC alignment purposes, delivery is also
conditional on (not malicious).  If a site wants to be conservative
and assume some degree of malice until it's seen "enough" benign
traffic, that's consistent with what I intended.

Also, sites like GMail and Yahoo! might very well be willing to
specially handle From addresses at sites known to have leaked a
billion address books.  Similarly, they could "throttle" on floods of
ARC email via purported mailing lists that have never been seen
before.  I acknowledge that sites managed by single admins would
likely be unwilling, and perhaps even unable, to come up with such
systems.

N.B. By "intended," I mean to admit that my wording in the previous
post strongly suggested that they would start with the assumption that
mail is benign until proven otherwise.  I can't fault you for taking
my words literally. :-)

Steve

From andrew at hodgson.io  Wed Jul 25 05:54:20 2018
From: andrew at hodgson.io (Andrew Hodgson)
Date: Wed, 25 Jul 2018 09:54:20 +0000
Subject: [Mailman-Users] Mailman 2.x to 3.1.x Migration
In-Reply-To: <e2fb0b62-8dba-aa10-870f-4881def80a7e@msapiro.net>
References: <CAH=cwAOsVLLOmn3ZPxjPjTkS3hUKd2DE7tZ=X9uwPAjjk0rG1w@mail.gmail.com>
 <VI1PR08MB3358D00F449BA49EE310CF92AF550@VI1PR08MB3358.eurprd08.prod.outlook.com>,
 <e2fb0b62-8dba-aa10-870f-4881def80a7e@msapiro.net>
Message-ID: <VI1PR08MB33587DE035D65DE6C13305A3AF540@VI1PR08MB3358.eurprd08.prod.outlook.com>

Mark Sapiro wrote:

>On 07/24/2018 06:32 AM, Andrew Hodgson wrote:

[...]

>> - There were older messages in the mbox without message-ids in the archive that failed to import.  I took the easy way out on this one and didn't import them.  In an archive with around 120,000 messages it rejected around 30 messages in all.

>This is <https://gitlab.com/mailman/hyperkitty/issues/180>, fixed in
>HyperKitty 1.2.0.

Thanks for this; I am on the Docker version which is not updated yet.  once it gets updated and I migrate to it, will I be able to re-import those missing messages by running a complete import again or will this cause more trouble now?

I did run cleanarch on the mbox before doing any of this import work and didn't get any issues.  The lists all came from Mailman 2.1 (starting from around 2.1.5).

Thanks.
Andrew.

From turnbull.stephen.fw at u.tsukuba.ac.jp  Wed Jul 25 06:03:29 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 25 Jul 2018 19:03:29 +0900
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <34e8162b-a6cb-1878-b6a6-46f68839c631@spamtrap.tnetconsulting.net>
References: <20180724211621.0796F2002D0E92@ary.qy>
 <34e8162b-a6cb-1878-b6a6-46f68839c631@spamtrap.tnetconsulting.net>
Message-ID: <23384.19185.73598.599052@turnbull.sk.tsukuba.ac.jp>

Grant Taylor via Mailman-Users writes:
 > On 07/24/2018 03:16 PM, John Levine wrote:
 > > Turning it on for aol.com, yahoo.com, and other domains with user 
 > > mailboxes,
 > 
 > So, are you stating that DMARC should NOT be used on domains that 
 > (predominantly) contain end user mailboxes?

Let's be careful to distinguish between "DMARC" and the "p=reject" and
"p=quarantine" policies.

(1) DMARC's reporting features *can* and *should* be used on all
    domains that send email, with very few exceptions.

(2) "p=reject" should never be used on domains that contain any
    non-transactional mailboxes (ie, mailboxes used as the From which
    might be reinjected into the mail system with changes).  This
    recommendation was actually in some early unofficial drafts or
    author discussions of RFC 7489, but doesn't appear to be in the
    Internet-Draft prepublication series, and it's definitely not in
    the published RFC.  (I believe that keeping such verbiage out of
    the RFC is why the RFC is informational rather than normative.)

 > "legitimate mail that DMARC cannot describe"  That sounds distinctly 
 > like a problem with the DMARC specification, /not/ with use there of.

Hey, did you really want to write that?  There's about a millennium of
mail-RFC-writing and/or large site admin experience on the DMARC-WG.

So, there's fundamental misunderstanding here somewhere.  DMARC
depends on DKIM and SPF.  Neither can ensure that the *purported
author domain* of an email can be authenticated when passed through a
mailing list.  *The whole purpose of DMARC From alignment is to
authenticate the author domain in the context of her message!*  This is
simply impossible, unless the signed portions of the message arrive at
the destination *unchanged*, or the list is an authorized mail source
of the author domain for SPF.  Much legitimate email *does* change them,
however, and few lists cater exclusively to subscribers of the same
administrative domain.  (Of course getting list host IPs authorized as
mail sources for all potential SPF-using posters would be a nightmare!)

You propose that mailing list should munge From (and in fact you've
proposed that it should do so independent of DMARC, IIRC).  AFAICS,
this has three nasty effects (at least).

(1) It ensures that validation of From alignment of the actual author
    will *fail*, because the From header *must* be signed in DMARC.
    (SPF is pretty useless in the context of mailing lists.)

(2) It breaks all the quoting mechanisms in every existing MUA, which
    now instead of, e.g.,

    >>>>> Grant Taylor writes:

    will insert

    >>>>> Grant Taylor <gtaylor at tnetconsulting.net> via Mailman-Users writes:

    Some styles would look a little better, some worse.  All, gag me.
    This is imposing work on a lot of MUA authors, and it will be
    ongoing as mailing lists keep changing their munging styles.

(3) Users may start to accept

    From: "Grant Taylor <gtaylor at tnetconsulting.net> via
          All-Passwords-Yours-Now-Ours ML" <vlad at kremlin.ru>

    (or whatever their MUA displays) as indicating authentic email
    from you, because a lot of authentic mail indeed looks like that
    in your scheme.  (This is a controversial point: a lot of security
    pundits believe that most users will click on anything anyway.)

 > I feel like DMARC requires a paradigm shift in how email is forwarded 
 > and handled by mailing lists.

Three points for your consideration here:

(1) There's a fundamental problem, though: the *only* paradigm shift
    compatible with DMARC's goal of authenticating the author's domain
    is to *pass the message through with all signed portions
    unchanged.*  (SPF doesn't have this problem: SPF can't authenticate
    author domains of mailing list posts *at all*, unless the list is
    hosted by the author's domain or its host's IP is listed as a valid
    mail source by the author's domain!)

(2) DMARC is a *private protocol*.  RFC 7489 is *Informational*, it
    has *no normative implications* for the rest of us.  We would be
    perfectly conformant to all normative RFCs if we decided that mail
    from sites that have a p=reject policy is undeliverable and
    dangerous, and rejected it.  We're not going to do that, nor even
    offer it as an option, but I'll leave that here for consideration.

(3) ARC, if the mailing list is accepted as trustworthy by recipients,
    requires *no* change to mailing list behavior (aside from
    participation in the ARC protocol, of course).  If not trusted,
    we're back in the world of (1) and (2).

 > I suspect "imposed on innocent bystanders" and "not their problem" can 
 > also be used to describe requiring reverse DNS, SPF, and DKIM.

No, it can't, not in the same way.

(1) All of the above can be implemented in the MTA without any change
    in end user or Mediator behavior.  These upgrades were frequently
    transparent to admins of virtual domains (except that their mail
    started working to sites it used to fail at).  Not so, DMARC.
    Most mailing lists munge Subject, or add footers, or both.  Either
    those modifications have to be abandoned, or the address in From
    must also be munged.  Most of us find those alternatives extremely
    distasteful.

(2) All of those requirements may result in denial of service to
    posters, but not to subscribers.  Turning on "p=reject" did DoS
    unrelated subscribers whose providers participate in DMARC.

Steve


From turnbull.stephen.fw at u.tsukuba.ac.jp  Wed Jul 25 06:11:10 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 25 Jul 2018 19:11:10 +0900
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <20180723164639.777302002968CE@ary.qy>
References: <CAMSGcLBBUS2P6ZR5BmscXeFgPpcvO-rEVMasWvKznXz4oo65og@mail.gmail.com>
 <20180723164639.777302002968CE@ary.qy>
Message-ID: <23384.19646.493495.204728@turnbull.sk.tsukuba.ac.jp>

John Levine writes:

 > As I said a few messages ago, if lists did more stringent tests on
 > incoming mail, a lot of this complexity could be avoided,

I don't understand this.  If lists got a pass, every spam would grow
RFC 2369 header fields.  No?  So ISTM the received chain needs to be
authenticated for the recipient to trust any last-hop sender, list or
not.

 > but they don't so it can't.

Origin checks are generally very useful and effective in my
experience.  But I don't see how authentication can be avoided.

From brennan at columbia.edu  Wed Jul 25 16:27:26 2018
From: brennan at columbia.edu (Joseph Brennan)
Date: Wed, 25 Jul 2018 16:27:26 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <34e8162b-a6cb-1878-b6a6-46f68839c631@spamtrap.tnetconsulting.net>
References: <20180724211621.0796F2002D0E92@ary.qy>
 <34e8162b-a6cb-1878-b6a6-46f68839c631@spamtrap.tnetconsulting.net>
Message-ID: <CAMSGcLB-sj2mYMJ+ANb54UtUGxJ-vCmWBRPRu9pJ+x3p6kaoHw@mail.gmail.com>

On Tue, Jul 24, 2018 at 8:20 PM Grant Taylor via Mailman-Users <
mailman-users at python.org> wrote:

>
> > Right, thereby causing a great deal of entirely legitimate mail that
> > DMARC cannot describe to go missing, along with a certain amount of spam.
>
> "legitimate mail that DMARC cannot describe"  That sounds distinctly
> like a problem with the DMARC specification, /not/ with use there of.


Well it is. The problem is that DMARC's notion of "alignment" contradicts
RFC 822, 2822, 5322, namely '''The "From:" field specifies the author(s) of
the message, that is, the mailbox(es) of the person(s) or system(s)
responsible for the writing of the message.''' and contradicts RFC 821,
2821, 5321 that describes the MAIL FROM address as the address used "to
report errors". Mailing lists fully comply with the standard by keeping the
writer's address in the Header From and by putting the address to report
errors in the MAIL FROM. Nothing in email standards stated or even implied
that the two addresses need to be the same.

Of course a system can reject any email message for any reason or no
reason, so all I can do is point out that lack of "alignment" is a silly
reason for rejecting list mail. For transactional mail from sources like
financial institutions, where the sender can state that the two addresses
should "align", then it makes a lot more sense.

Joseph Brennan

From mailman at jordan.maileater.net  Wed Jul 25 13:48:26 2018
From: mailman at jordan.maileater.net (Jordan Brown)
Date: Wed, 25 Jul 2018 10:48:26 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
 <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
Message-ID: <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>

Hmm.? It would take MUA changes to be fully effective, but a possibility
that comes to mind is to have mailing lists leave the original message
absolutely unmodified, but wrap it in a message that comes "from" the
mailing list.? That way everything about the message is verifiably true.

A list-aware MUA could more or less transparently unwrap such a
message.? It would probably display some indication that the message
came through the ML - the name of the list, unsubscribe mechanism,
archive pointers, ML header or footer text, et cetera, and maybe
activate alternative "reply" options[*]? - but would largely present the
message as "from" the original author, *via* the mailing list.

Perhaps the wrapper message would look like today's munged ML messages -
From: Real Person <mailinglist at example.com> / Reply-To: Real Person
<realperson at example2.com> - but a list-aware MUA would largely hide that.

Of course there are a million details and getting adoption would be hard.


From gtaylor at tnetconsulting.net  Wed Jul 25 14:55:15 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Wed, 25 Jul 2018 12:55:15 -0600
Subject: [Mailman-Users] ARC
In-Reply-To: <23384.18596.617417.251765@turnbull.sk.tsukuba.ac.jp>
References: <20180722200514.EA5FA20028DC81@ary.qy>
 <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>
 <23381.24927.74482.530338@turnbull.sk.tsukuba.ac.jp>
 <03d440cf-7cd5-4564-2900-ffc08cd7849b@spamtrap.tnetconsulting.net>
 <23384.18596.617417.251765@turnbull.sk.tsukuba.ac.jp>
Message-ID: <c040a36d-efeb-3d45-4e03-9f168ed018fc@spamtrap.tnetconsulting.net>

On 07/25/2018 03:53 AM, Stephen J. Turnbull wrote:
> That's not how "on behalf of" worked in practice.  What happened in April 
> 2014, was that a home business owner (HBO) would send a pile of completed 
> order notices to intuit.com, and intuit.com would send an invoice to each 
> customer on behalf of hbo at yahoo.com, from hbo at yahoo.com.

Will you please clarify what the SMTP envelope from and the From: header 
were in the example(s) that you're talking about?

I'm going to assume (for the sake of discussion or until corrected) that 
both the SMTP envelope from and the From: header were hbo at yahoo.com.

> HBO, of course, doesn't have a vanity domain, just a Wordpress or 
> SquareSite (or even Facebook) home page.

~twitch~

> Tens of thousands of invoices worth millions of dollars bounced off of 
> personal accounts and tiny business owner accounts at Yahoo! and other 
> receivers who take p=reject as a command.

I assume "bounced off of" means that the messages were rejected by 
receiving MTAs that honored Yahoo's (et al) p=reject DMARC configuration.

I'll argue that DMARC is designed to 1) detect such spoofed email 
forgeries and 2) enable Yahoo (et al) to publish what they would like to 
be done with such spoofed email forgeries.

Granted, all the rejected / bounced invoices are contrary to the 
behavior that many individuals wanted.

But this also enabled DMARC enabled receiving MTAs to detect and reject 
other less white hat spoofed email forgeries.

I assume that this detection and rejection of any email claiming to be 
from Yahoo that didn't actually originate from Yahoo is exactly what 
Yahoo wanted to happen.

I feel like this is saying "We want something to detect the bad guys and 
block them, but still allow the good guys to do the exact same thing as 
the bad guys."  That has never worked very well.

> Note that if I were intuit.com's CISO, I would fight tooth and nail 
> against the system you suggest, because it implies that I have DKIM 
> private keys for all those subdomains owned by clients.

What would you want to see done, as Intuit's CISO, instead of what I 
propose?

My purpose of the sub-domain of the client's domain is to transition 
(some of) the trust of the client's domain to Intuit's sub-domain 
therein.  I.e. I trust ietf.org, therefor I'll also trust intuit.ietf.org.

It is possible for Intuit to use a completely separate domain, 
intuit-ietf.org.  But there is not the same implicit relationship 
between intuit-ietf.org as there is intuit.ietf.org.

The intuit.ietf.org sub-domain is a completely different DNS zone that 
has been delegated from ietf.org to Intuit for them to manage / 
administer / operate for the purpose of hosting requisite services for 
IETF to be a client of Intuit.

What would you rather see done?  Does it also convey trust the same way 
that intuit.ietf.org does?

> Every spammer in the world would be trying to hack the server that has 
> those keys.

Why does that need to be one server?  What if it's a separate VM per 
client?  (I'd personally like that and pay more for such separation as 
Intuit's client.)

Assume for a moment that they are separate VMs.  How does the security 
of Intuit running X number of VMs differ from X number of clients 
running a VM?

I would think that it would actually worsen security posture because I 
would assume that Intuit would have a better understanding because of, 
and more data from, the larger population of VMs to administer.

Who knows more about securing things:  The thousand different entities 
that run one (or few) instances of something, or the one (or few) 
entities that run many instances of said thing?

> I could probably keep them out, but Lordy, the liability involved!

I would certainly hope that the company that I'm outsourcing financial 
transactions to could secure the systems that process money.  I'd really 
hope that they could also secure systems that only process email.  With 
the security of the former systems exceeding the security need of the 
latter systems.  One stringent security policy protects both sets of 
systems.

> Less financially painful are the services that newspapers and tourist 
> destinations provided (note past tense, they're mostly dead now) where you 
> could sit at a terminal and send a "postcard" recommending an article or 
> with a picture of the attraction to family and friends "from" yourself, 
> including your email address, simply by typing name and address in. 
> Not a huge loss, I guess, but ....

I think that all of those systems were behaving badly.  I think that 
they SHOULD have been sending with something like this:

      From:  Uncle Sam (via My Picture Kiosk) <lax at mypicturekiosk.com>
      To:  Cousin Larry <larry at mysmallisp.net>
      Reply-To:  Uncle Sam <coolunclesam at gmail.com>

      Hi Cousin Larry,

      Check out this cool picture!!!

      --
      Uncle Sam sent you this picture from the LAX airport.

I think anything that spoofs or otherwise pretends to send as someone 
they are not is a form of abuse.

I also believe this form of abuse is (at least partially) exactly what 
DMARC is designed to prevent.

> I've never seen SRS in the wild, except in the headers of some of the 
> crazier denizens of IETF mailing lists.  YMMV, of course.

I know that there is little love for SRS.  But I see little reason not 
to use it.  Yes, I am practicing what I preach.  Both of my email 
servers are configured to support SRS.

> Simple .forwarding breaks SPF of the author domain, period, unless 
> entirely within one administrative domain, and that domain is 
> well-configured so that all the forwarding is internal, and the SPF 
> checks are all done at the boundary.

I would question how proper the SPF and / or forwarding configuration is 
to even allow it within an administrative domain.  Though I suppose an 
SPF record that allows any IP within the enterprise IP space to send 
would allow that.  Even that seems overly lax to me.

> I guess in the case where forwarding takes place entirely within an 
> administrative domain, DMARC From alignment could be broken without 
> breaking SPF itself because From alignment is validated on a different 
> host from SPF, but that sounds like a case of "evolution in action".

Ew....

> That's non-conforming to RFC 7489.  Section 4.2:

Sorry, I conflated PASS / FAIL with reporting again.

> I'm afraid I failed to make the concept explicit again.  Remember, 
> authenticating mail origin doesn't prevent anyone from sending malicious 
> mail, or even from having it delivered.

Agreed.

Just like HTTPS doesn't mean that the website isn't bad.

> So while I suggest that ARC processing results be taken seriously in the 
> presence of (List-* headers) for DMARC alignment purposes, delivery is 
> also conditional on (not malicious).

I mostly agree.

> If a site wants to be conservative and assume some degree of malice until 
> it's seen "enough" benign traffic, that's consistent with what I intended.

Fair enough.

> Also, sites like GMail and Yahoo! might very well be willing to specially 
> handle From addresses at sites known to have leaked a billion address 
> books.  Similarly, they could "throttle" on floods of ARC email via 
> purported mailing lists that have never been seen before.

That sounds more like big data type analytics to determine the quality 
of a message.  Once you start applying analytics to messages, you 
completely change the ball game.  (Bring your own ice skates.)

> I acknowledge that sites managed by single admins would likely be 
> unwilling, and perhaps even unable, to come up with such systems.

I suspect that there will (eventually) be services that smaller players 
can leverage / subscribe to that allow local hosting of this type of 
spam filtering.

I'd be interested in a service that was an appliance that I could 
configure Sendmail to use as a milter.  Much like ClamAV scanning on a 
box that sites beside the MX.  Substitute BigDataFilteringService in 
place of ClamAV.

> N.B. By "intended," I mean to admit that my wording in the previous post 
> strongly suggested that they would start with the assumption that mail 
> is benign until proven otherwise.  I can't fault you for taking my words 
> literally. :-)

Thank you for clarifying.

I try not to assume incorrectly, and prefer to ask.

Thank you for the conversation Steve.



-- 
Grant. . . .
unix || die


From mailman at jordan.maileater.net  Wed Jul 25 13:04:23 2018
From: mailman at jordan.maileater.net (Jordan Brown)
Date: Wed, 25 Jul 2018 10:04:23 -0700
Subject: [Mailman-Users] ARC
In-Reply-To: <23384.18596.617417.251765@turnbull.sk.tsukuba.ac.jp>
References: <20180722200514.EA5FA20028DC81@ary.qy>
 <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>
 <23381.24927.74482.530338@turnbull.sk.tsukuba.ac.jp>
 <03d440cf-7cd5-4564-2900-ffc08cd7849b@spamtrap.tnetconsulting.net>
 <23384.18596.617417.251765@turnbull.sk.tsukuba.ac.jp>
Message-ID: <490793b6-cc8f-fed7-6005-5ae296af3622@jordan.maileater.net>

On 7/25/2018 2:53 AM, Stephen J. Turnbull wrote:
> Note that if I were intuit.com's CISO, I would fight tooth and nail
> against the system you suggest, because it implies that I have DKIM
> private keys for all those subdomains owned by clients.  Every spammer
> in the world would be trying to hack the server that has those keys.
> I could probably keep them out, but Lordy, the liability involved!

Well, yeah, but to provide such a service in a way that has any
resemblance to being secure, Intuit *must* have some secret that allows
it to send mail "from" those subdomains.? If Intuit doesn't need such a
secret, then anybody could send mail like that.

The price of the privilege of sending mail? on behalf of your clients is
that you must protect that ability so that villains cannot hijack it.


From mark at msapiro.net  Wed Jul 25 11:56:56 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 25 Jul 2018 08:56:56 -0700
Subject: [Mailman-Users] Mailman 2.x to 3.1.x Migration
In-Reply-To: <VI1PR08MB33587DE035D65DE6C13305A3AF540@VI1PR08MB3358.eurprd08.prod.outlook.com>
References: <CAH=cwAOsVLLOmn3ZPxjPjTkS3hUKd2DE7tZ=X9uwPAjjk0rG1w@mail.gmail.com>
 <VI1PR08MB3358D00F449BA49EE310CF92AF550@VI1PR08MB3358.eurprd08.prod.outlook.com>
 <e2fb0b62-8dba-aa10-870f-4881def80a7e@msapiro.net>
 <VI1PR08MB33587DE035D65DE6C13305A3AF540@VI1PR08MB3358.eurprd08.prod.outlook.com>
Message-ID: <1d7b049f-5f7b-c994-3b5b-356c3a9c9307@msapiro.net>

On 07/25/2018 02:54 AM, Andrew Hodgson wrote:
> 
> Thanks for this; I am on the Docker version which is not updated yet.  once it gets updated and I migrate to it, will I be able to re-import those missing messages by running a complete import again or will this cause more trouble now?


You can re-run the import with the complete mbox as long as you specify
--since= with a date older than any of the messages in the mbox. Without
that, it won't import anything because the messages will all be 'too old'.

Running with the full mbox will be OK as those messages with Message-ID:
headers will be detected as duplicates and won't be added a second time,
but if the mbox is large, it would be faster to run with just the
messages without Message-ID: headers, but possibly not faster overall
considering the effort to extract those messages.


> I did run cleanarch on the mbox before doing any of this import work and didn't get any issues.  The lists all came from Mailman 2.1 (starting from around 2.1.5).


That's expected. Since even somewhat before Mailman 2.1.5, archive
mboxes have been good. It's normally only those started on Mailman 2.0.x
(or older?) or mboxes imported from elsewhere that have significant issues.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From phils at caerllewys.net  Wed Jul 25 14:43:38 2018
From: phils at caerllewys.net (Phil Stracchino)
Date: Wed, 25 Jul 2018 14:43:38 -0400
Subject: [Mailman-Users] Argh. Postfix/Mailman compatibility on Gentoo
Message-ID: <3404d593-3732-b985-07d4-a8b75de2e53b@caerllewys.net>


<caerllewys at caerllewys.net>: Command died with status 2:
    "/usr/lib64/mailman/mail/mailman post caerllewys". Command output: Group
    mismatch error.  Mailman expected the mail wrapper script to be
executed as
    group "mailman", but the system's mail server executed the mail
script as
    group "nobody".  Try tweaking the mail server to run the script as group
    "mailman", or re-run configure,  providing the command line option
    `--with-mail-gid=nobody'.



I'd swear I fixed that *AT LEAST* once......


The easy solution:

1.  In /etc/postfix/main.cf:
mail_owner            = postfix
default_privs         = mailman

2.  Restart Postfix

Done.


-- 
  Phil Stracchino
  Babylon Communications
  phils at caerllewys.net
  phil at co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

From rmcclung at afilias.info  Wed Jul 25 09:55:03 2018
From: rmcclung at afilias.info (Ryan McClung)
Date: Wed, 25 Jul 2018 09:55:03 -0400
Subject: [Mailman-Users] Mailman Migrate 3.0 (beta) to 3.2
Message-ID: <CAH=cwAN20rGa87=KfaxYYFb_=bZdgpgY+n1fiHJb4WqmVRtyBA@mail.gmail.com>

The mailman 3.0 install is on the same box as 3.2. Is there a quick and
dirty way to flip over?

-- 

Ryan McClung
Systems Administrator @ Afilias Canada
A.  204-4141 Yonge Street, Toronto, ON, Canada, M2P 2A8
<https://maps.google.com/?q=4141+Yonge+Street,+Toronto,+ON,+Canada,+M2P+2A8&entry=gmail&source=g>
W. www.afilias.info
T.  +1.416.646.3304 x4186

From mark at msapiro.net  Wed Jul 25 18:52:21 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 25 Jul 2018 15:52:21 -0700
Subject: [Mailman-Users] Argh. Postfix/Mailman compatibility on Gentoo
In-Reply-To: <3404d593-3732-b985-07d4-a8b75de2e53b@caerllewys.net>
References: <3404d593-3732-b985-07d4-a8b75de2e53b@caerllewys.net>
Message-ID: <3490cefe-8375-3b56-8cea-8cdbed3a10bd@msapiro.net>

On 07/25/2018 11:43 AM, Phil Stracchino wrote:
> 
> The easy solution:
> 
> 1.  In /etc/postfix/main.cf:
> mail_owner            = postfix
> default_privs         = mailman
> 
> 2.  Restart Postfix

Or

sudo chown mailman /path/to/mailman/data/aliases.db

which doesn't even require restart/reload of Postfix. See the "DELIVERY
RIGHTS" section in 'man 8 local'

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From johnl at taugh.com  Wed Jul 25 09:52:51 2018
From: johnl at taugh.com (John R Levine)
Date: 25 Jul 2018 09:52:51 -0400
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <23384.19646.493495.204728@turnbull.sk.tsukuba.ac.jp>
References: <CAMSGcLBBUS2P6ZR5BmscXeFgPpcvO-rEVMasWvKznXz4oo65og@mail.gmail.com>
 <20180723164639.777302002968CE@ary.qy>
 <23384.19646.493495.204728@turnbull.sk.tsukuba.ac.jp>
Message-ID: <alpine.OSX.2.21.1807250948490.44596@ary.qy>

> > As I said a few messages ago, if lists did more stringent tests on
> > incoming mail, a lot of this complexity could be avoided,
>
> I don't understand this.  If lists got a pass, every spam would grow
> RFC 2369 header fields.  No?

Large mail systems already know where all the mailing lists are.  It's 
obvious from the tags in the DMARC reports I get from Gmail.  The reason 
we have kludgy ARC rather than just whitelist list servers is that lists 
don't filter inbound spam, so ARC gives the recipient systems clues to do 
filtering retroactively.

If lists filtered better, e.g., by doing DMARC checks on INBOUND mail, 
that's checking INBOUND mail as it ARRIVES at list servers*, there'd be 
much less leakage and no need for retroactive filtering.

Nobody's going to whitelist on List-ID, they'll do it with IP addresses.

Regards,
John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

* - PS to Stephen, I know you understand the difference but a lot of other 
people reading this clearly don't.

From mark at msapiro.net  Wed Jul 25 19:09:45 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 25 Jul 2018 16:09:45 -0700
Subject: [Mailman-Users] Mailman Migrate 3.0 (beta) to 3.2
In-Reply-To: <CAH=cwAN20rGa87=KfaxYYFb_=bZdgpgY+n1fiHJb4WqmVRtyBA@mail.gmail.com>
References: <CAH=cwAN20rGa87=KfaxYYFb_=bZdgpgY+n1fiHJb4WqmVRtyBA@mail.gmail.com>
Message-ID: <6faedf61-ae09-7cb8-7976-6339654db444@msapiro.net>

On 07/25/2018 06:55 AM, Ryan McClung wrote:
> The mailman 3.0 install is on the same box as 3.2. Is there a quick and
> dirty way to flip over?


Other than updating your MTA and web server to point at the 3.2 install,
you need to ensure that 3.2 accesses the old 3.0 database(s) and after
doing so, run the django management 'migrate' command. The core's
migrations should be done automatically by alembic.

Note questions like this will reach a deeper pool of knowledgeable
people if asked on the mailman-users at mailman3.org list
<https://lists.mailman3.org/mailman3/lists/mailman-users at mailman3.org/>

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From Richard at Damon-Family.org  Wed Jul 25 20:24:05 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Wed, 25 Jul 2018 20:24:05 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
 <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
 <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
Message-ID: <b8175955-c76d-2bcb-ac95-e03fa8539dca@Damon-Family.org>

On 7/25/18 1:48 PM, Jordan Brown wrote:
> Hmm.? It would take MUA changes to be fully effective, but a possibility
> that comes to mind is to have mailing lists leave the original message
> absolutely unmodified, but wrap it in a message that comes "from" the
> mailing list.? That way everything about the message is verifiably true.
>
> A list-aware MUA could more or less transparently unwrap such a
> message.? It would probably display some indication that the message
> came through the ML - the name of the list, unsubscribe mechanism,
> archive pointers, ML header or footer text, et cetera, and maybe
> activate alternative "reply" options[*]? - but would largely present the
> message as "from" the original author, *via* the mailing list.
>
> Perhaps the wrapper message would look like today's munged ML messages -
> From: Real Person <mailinglist at example.com> / Reply-To: Real Person
> <realperson at example2.com> - but a list-aware MUA would largely hide that.
>
> Of course there are a million details and getting adoption would be hard.
Yes, one set of solutions would involve defining standards of how to
compose composite messages, with standards on how to display them. A
major part of the current issue is that for anything more than a single
part plain text you can't be sure how it will be handled.

Yes, you can't be absolutely positive that every MUA will work right,
you at least have the defense that you did your best to be clear within
the standard, and perhaps the standard could even be worked that
existing non-conforming implementations are so bad that it is clear that
the user needs a new MUA.

-- 
Richard Damon


From johnl at taugh.com  Wed Jul 25 21:20:27 2018
From: johnl at taugh.com (John Levine)
Date: 25 Jul 2018 21:20:27 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
Message-ID: <20180726012028.0D5C42002D801A@ary.qy>

In article <885f93f0-36ec-d74f-7c5f-52b42f2d6d42 at jordan.maileater.net> you write:
>Hmm.? It would take MUA changes to be fully effective, but a possibility
>that comes to mind is to have mailing lists leave the original message
>absolutely unmodified, but wrap it in a message that comes "from" the
>mailing list.? That way everything about the message is verifiably true.

Yeah, we've tried that.  It would in effect make each message a
one-message digest.  Let me just say that it would take a LOT of
changes to a lot of MUAs to make that work acceptably.

R's,
John

From mark at msapiro.net  Wed Jul 25 21:27:58 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 25 Jul 2018 18:27:58 -0700
Subject: [Mailman-Users] Munge From was: non-subscribers getting
 through--email address in "Real Name"
In-Reply-To: <b8175955-c76d-2bcb-ac95-e03fa8539dca@Damon-Family.org>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
 <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
 <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
 <b8175955-c76d-2bcb-ac95-e03fa8539dca@Damon-Family.org>
Message-ID: <9c17f228-b0b7-f557-813d-4fb828894b87@msapiro.net>

On 07/25/2018 05:24 PM, Richard Damon wrote:
>>
>> Perhaps the wrapper message would look like today's munged ML messages -
>> From: Real Person <mailinglist at example.com> / Reply-To: Real Person
>> <realperson at example2.com> - but a list-aware MUA would largely hide that.
>>
>> Of course there are a million details and getting adoption would be hard.
> Yes, one set of solutions would involve defining standards of how to
> compose composite messages, with standards on how to display them. A
> major part of the current issue is that for anything more than a single
> part plain text you can't be sure how it will be handled.


First, the MLM side of this proposal is exactly what the Mailman "Wrap
Message" action does. It wraps the original message, unchanged except
for content filtering as a message/rfc822 part in an outer message. List
headers and footers if any are added as separate parts in the outer
message and all subject prefixing, From: header munging addition of
List-* headers etc is done to the outer message.

This action is almost universally ignored in favor of Munge From because
of MUA deficiencies. There are MUAs that will render such a message in a
highly readable manner, but there are others, particularly mobile device
clients, that make it difficult to impossible to view the wrapped message.

If Thunderbird can do a reasonable job of rendering such a message, and
it does, there's little if any excuse for other MUAs to not do so. Even
Yahoo web mail, which I just tried, does OK after I click "Show original
message".

But, when this was initially implemented, I tried using Wrap Message on
my bicycling club's lists, and I received many complaints so I went to
Munge From.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From jaybird at bluegrasspals.com  Thu Jul 26 07:26:12 2018
From: jaybird at bluegrasspals.com (Jayson Smith)
Date: Thu, 26 Jul 2018 07:26:12 -0400
Subject: [Mailman-Users] Mailman 2.1.X and HTTPS
Message-ID: <835bf2c0-d682-c65e-ae8f-720314227336@bluegrasspals.com>

Hi,


I'm currently using 2.1.26. Recently I had LetsEncrypt add lines to my 
Apache configuration to redirect browsers from HTTP to HTTPS URLs. This 
morning I found that this doesn't seem to work well with Mailman 
database admin pages, I couldn't get Mailman to reject a held message 
until I commented out those Apache config lines. Is there a way to make 
Mailman work using HTTPS?


My reason for the redirect is that I'd heard that browsers, in 
particular Chrome, are going to start warning about insecure sites. Does 
this mean if any URL begins with HTTP people will have to click through 
warnings, or is just the fact that HTTPS is available good enough, even 
if the user or link didn't specify HTTPS?


Thanks,


Jayson



From Richard at Damon-Family.org  Thu Jul 26 07:48:33 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Thu, 26 Jul 2018 07:48:33 -0400
Subject: [Mailman-Users] Mailman 2.1.X and HTTPS
In-Reply-To: <835bf2c0-d682-c65e-ae8f-720314227336@bluegrasspals.com>
References: <835bf2c0-d682-c65e-ae8f-720314227336@bluegrasspals.com>
Message-ID: <f963a26d-d8dd-1d94-5617-38da15502f11@Damon-Family.org>

On 7/26/18 7:26 AM, Jayson Smith wrote:
> Hi,
>
>
> I'm currently using 2.1.26. Recently I had LetsEncrypt add lines to my
> Apache configuration to redirect browsers from HTTP to HTTPS URLs.
> This morning I found that this doesn't seem to work well with Mailman
> database admin pages, I couldn't get Mailman to reject a held message
> until I commented out those Apache config lines. Is there a way to
> make Mailman work using HTTPS?
>
>
> My reason for the redirect is that I'd heard that browsers, in
> particular Chrome, are going to start warning about insecure sites.
> Does this mean if any URL begins with HTTP people will have to click
> through warnings, or is just the fact that HTTPS is available good
> enough, even if the user or link didn't specify HTTPS?
>
>
> Thanks,
>
>
> Jayson
The issue is probably that the links still point to the http version,
which is redirecting to the https and losing the form data. You need to
change the config variable that gives the base page for the admin pages
to point to the https version. (Someone more familiar with the details
will need to provide those if my rough description isn't good enough).

-- 
Richard Damon


From mark at msapiro.net  Thu Jul 26 12:24:25 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 26 Jul 2018 09:24:25 -0700
Subject: [Mailman-Users] Mailman 2.1.X and HTTPS
In-Reply-To: <f963a26d-d8dd-1d94-5617-38da15502f11@Damon-Family.org>
References: <835bf2c0-d682-c65e-ae8f-720314227336@bluegrasspals.com>
 <f963a26d-d8dd-1d94-5617-38da15502f11@Damon-Family.org>
Message-ID: <69fb4de3-d255-d23b-7f24-6f1b73c3dbc6@msapiro.net>

On 07/26/2018 04:48 AM, Richard Damon wrote:

> The issue is probably that the links still point to the http version,
> which is redirecting to the https and losing the form data. You need to
> change the config variable that gives the base page for the admin pages
> to point to the https version. (Someone more familiar with the details
> will need to provide those if my rough description isn't good enough).


Steps 2 through 4 at <https://wiki.list.org/x/17892007>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From gtaylor at tnetconsulting.net  Thu Jul 26 14:47:29 2018
From: gtaylor at tnetconsulting.net (Grant Taylor)
Date: Thu, 26 Jul 2018 12:47:29 -0600
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <23384.19185.73598.599052@turnbull.sk.tsukuba.ac.jp>
References: <20180724211621.0796F2002D0E92@ary.qy>
 <34e8162b-a6cb-1878-b6a6-46f68839c631@spamtrap.tnetconsulting.net>
 <23384.19185.73598.599052@turnbull.sk.tsukuba.ac.jp>
Message-ID: <8cf9c9da-e694-8cef-bed7-c24c7193960a@spamtrap.tnetconsulting.net>

On 07/25/2018 04:03 AM, Stephen J. Turnbull wrote:
> Let's be careful to distinguish between "DMARC" and the "p=reject" and 
> "p=quarantine" policies.

Fair point.

> (1) DMARC's reporting features *can* and *should* be used on all domains 
> that send email, with very few exceptions.

Agreed.

> (2) "p=reject" should never be used on domains that contain any 
> non-transactional mailboxes (ie, mailboxes used as the From which might 
> be reinjected into the mail system with changes).

I personally disagree.

I'll concede "SHOULD" in the typical sense that RFCs use it.  Thus my 
opinion is contrary to the recommendation.

> This recommendation was actually in some early unofficial drafts or author 
> discussions of RFC 7489, but doesn't appear to be in the Internet-Draft 
> prepublication series, and it's definitely not in the published RFC. 
> (I believe that keeping such verbiage out of the RFC is why the RFC is 
> informational rather than normative.)

Odd.

> Hey, did you really want to write that?  There's about a millennium of 
> mail-RFC-writing and/or large site admin experience on the DMARC-WG.

Yes, I did.

I'm sure that people had the best of intentions.  I'm not trying to 
impugn anybody.  Mistakes happen.  Undesirable results happen.  A recent 
example that comes to mind is the renegotiation issue with a MitM on TLS 
1.2.

> So, there's fundamental misunderstanding here somewhere.  DMARC depends 
> on DKIM and SPF.  Neither can ensure that the *purported author domain* 
> of an email can be authenticated when passed through a mailing list.

It's my understanding that one of the functions of DKIM was to allow 
messages to be forwarded, (with the body) unmodified, and still 
providing a high degree of certainty that (the signed portion of) the 
message was the same as it was submitted into SMTP.  (Assuming that the 
submission server did the DKIM signing and that it's the same domain as 
the purported sending email address.

I feel the need to pause for a moment and reflect on that assumption. 
As I type this, I realize that it's relatively easy to break that 
assumption without breaking DKIM.  ?  Thus, DKIM itself can't "?ensure 
that the *purported author domain* of an email can be authenticated?"

I'm leaving that here in case it helps others who might have been 
confused by that fact like I was.

> *The whole purpose of DMARC From alignment is to authenticate the author 
> domain in the context of her message!*

Okay.

> This is simply impossible, unless the signed portions of the message 
> arrive at the destination *unchanged*, or the list is an authorized mail 
> source of the author domain for SPF.

Agreed.

> Much legitimate email *does* change them, however, and few lists 
> cater exclusively to subscribers of the same administrative domain. 
> (Of course getting list host IPs authorized as mail sources for all 
> potential SPF-using posters would be a nightmare!)
> 
> You propose that mailing list should munge From (and in fact you've 
> proposed that it should do so independent of DMARC, IIRC).

At a minimum, yes.  You could also state that I'm proposing regenerating 
a completely new and independent message, quite a bit more than munging 
a message.

> AFAICS, this has three nasty effects (at least).
> 
> (1) It ensures that validation of From alignment of the actual author 
> will *fail*, because the From header *must* be signed in DMARC.  (SPF is 
> pretty useless in the context of mailing lists.)

Only if the From: header has the original sender's email address.  [1] 
If the From: header reflects the mailing list, there is no DMARC 
conflict with the original sender's domain.

[1]  I think it may be possible to move the email address into the human 
friendly portion of the address and replace the actual email address 
with that of the mailing list.  I.e.:

      From: Grant Taylor <gtaylor at tnetconsulting.net>

Becomes:

      From:  "Grant Taylor <gtaylor at tnetconsulting.net>" 
<mailman-users at python.org>

It is my understanding that DMARC implementations are supposed to ignore 
the human friendly portion of the email address.  However I do not feel 
comfortable trusting that.  As such, I would likely alter the text that 
looks like an address in the human friendly portion to be something like 
this:

      From:  "Grant Taylor - gtaylor (at) tnetconsulting (dot) net" 
<mailman-users at python.org>

> (2) It breaks all the quoting mechanisms in every existing MUA, which 
> now instead of, e.g.,
> 
>      >>>>> Grant Taylor writes:
> 
> will insert
> 
>      >>>>> Grant Taylor <gtaylor at tnetconsulting.net> via Mailman-Users writes:

I don't know if I'd call that "broken" per say.  But I don't object to 
the point.

> Some styles would look a little better, some worse.  All, gag me.

To each his / her own opinion.

> This is imposing work on a lot of MUA authors, and it will be ongoing 
> as mailing lists keep changing their munging styles.

I don't see how this imposes additional work on MUA authors.

Or are you suggesting that MUA authors alter what the MUA does in a 
perpetual game of whack-a-formatting-mole?

> (3) Users may start to accept
> 
>      From: "Grant Taylor <gtaylor at tnetconsulting.net> via
>            All-Passwords-Yours-Now-Ours ML" <vlad at kremlin.ru>
> 
> (or whatever their MUA displays) as indicating authentic email from you, 
> because a lot of authentic mail indeed looks like that in your scheme.

That would only apply to email from me through a mailing list.

But I take your point.

> (This is a controversial point: a lot of security pundits believe that 
> most users will click on anything anyway.)

Sadly true.

> Three points for your consideration here:
> 
> (1) There's a fundamental problem, though: the *only* paradigm shift 
> compatible with DMARC's goal of authenticating the author's domain 
> is to *pass the message through with all signed portions unchanged.*

Okay.

IMHO this speaks to how mailing lists are configured, specifically what 
they modify.

> (SPF doesn't have this problem: SPF can't authenticate author domains of 
> mailing list posts *at all*, unless the list is hosted by the author's 
> domain or its host's IP is listed as a valid mail source by the author's 
> domain!)

I disagree in so far as I believe that the MTA hosting the mailing list 
can perform SPF validation as the message comes into the mailing list.

Granted, SPF can't do anything to help subscriber's receiving MTAs 
validate the purported author's domain.  If anything it's likely to fail 
SPF if the SMTP envelope is unmodified.

> (2) DMARC is a *private protocol*.  RFC 7489 is *Informational*, it has 
> *no normative implications* for the rest of us.  We would be perfectly 
> conformant to all normative RFCs if we decided that mail from sites that 
> have a p=reject policy is undeliverable and dangerous, and rejected it. 
> We're not going to do that, nor even offer it as an option, but I'll 
> leave that here for consideration.
> 
> (3) ARC, if the mailing list is accepted as trustworthy by recipients, 
> requires *no* change to mailing list behavior (aside from participation in 
> the ARC protocol, of course).  If not trusted, we're back in the world of 
> (1) and (2).

Fair.

I think "if the mailing list is accepted as trustworthy by recipients" 
is going to be the Achilles heal of ARC.  At the very least we're going 
to end up with a priming problem.

> No, it can't, not in the same way.

Okay.

You're free to have your own opinion.

> (1) All of the above can be implemented in the MTA without any change in 
> end user or Mediator behavior.  These upgrades were frequently transparent 
> to admins of virtual domains (except that their mail started working to 
> sites it used to fail at).  Not so, DMARC.  Most mailing lists munge 
> Subject, or add footers, or both.  Either those modifications have to 
> be abandoned, or the address in From must also be munged.  Most of us 
> find those alternatives extremely distasteful.
> 
> (2) All of those requirements may result in denial of service to posters, 
> but not to subscribers.  Turning on "p=reject" did DoS unrelated 
> subscribers whose providers participate in DMARC.

Interesting points that I'll mull over.

Thank you for the constructive conversation Steve.



-- 
Grant. . . .
unix || die


From turnbull.stephen.fw at u.tsukuba.ac.jp  Fri Jul 27 00:19:16 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Fri, 27 Jul 2018 13:19:16 +0900
Subject: [Mailman-Users] ARC,
 was non-subscribers getting through--email address in "Real Name"
In-Reply-To: <alpine.OSX.2.21.1807250948490.44596@ary.qy>
References: <CAMSGcLBBUS2P6ZR5BmscXeFgPpcvO-rEVMasWvKznXz4oo65og@mail.gmail.com>
 <20180723164639.777302002968CE@ary.qy>
 <23384.19646.493495.204728@turnbull.sk.tsukuba.ac.jp>
 <alpine.OSX.2.21.1807250948490.44596@ary.qy>
Message-ID: <23386.40260.809769.867731@turnbull.sk.tsukuba.ac.jp>

John R Levine writes:

 > Large mail systems already know where all the mailing lists are.

Hm.  Well, that may be true for Google et al, but the systems at my
employer regularly mark internal business mail as "possible spam",
occasionally mark it as "almost certainly spam", and pass through
actual spam (vs. crap from my employer that I really don't want :-/)
unmarked daily.  They don't even know who their own hosts are!  (We
have two /16s, identification is trivial and there's no excuse for not
knowing.)  I get the feeling there are an awful lot of admins who need
all the help they can get.

 > * - PS to Stephen, I know you understand the difference but a lot of other 
 > people reading this clearly don't.

I'm on Twitter, I don't take anything personally anymore. ;-)

Steve

From turnbull.stephen.fw at u.tsukuba.ac.jp  Fri Jul 27 00:19:30 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Fri, 27 Jul 2018 13:19:30 +0900
Subject: [Mailman-Users] ARC
In-Reply-To: <490793b6-cc8f-fed7-6005-5ae296af3622@jordan.maileater.net>
References: <20180722200514.EA5FA20028DC81@ary.qy>
 <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>
 <23381.24927.74482.530338@turnbull.sk.tsukuba.ac.jp>
 <03d440cf-7cd5-4564-2900-ffc08cd7849b@spamtrap.tnetconsulting.net>
 <23384.18596.617417.251765@turnbull.sk.tsukuba.ac.jp>
 <490793b6-cc8f-fed7-6005-5ae296af3622@jordan.maileater.net>
Message-ID: <23386.40274.604718.454833@turnbull.sk.tsukuba.ac.jp>

Jordan Brown writes:

 > Well, yeah, but to provide such a service in a way that has any
 > resemblance to being secure, Intuit *must* have some secret that allows
 > it to send mail "from" those subdomains.? If Intuit doesn't need such a
 > secret, then anybody could send mail like that.

Sure, but (1) anyone can send mail like that anyway (and they do), (2)
the customers will (well, should) be checking invoices against their
own purchasing records before they pay, and (3) after the vendor
identifies Intuit as its billing agent, Intuit's own signature will do
the trick.

Securing a small number of own keys that get rotated on a schedule is
one thing, securing a database of others' keys that regularly gets
updated and multiple regular employees need access to is going to be
quite another.

Steve

From turnbull.stephen.fw at u.tsukuba.ac.jp  Fri Jul 27 00:19:40 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Fri, 27 Jul 2018 13:19:40 +0900
Subject: [Mailman-Users] Munge From was: non-subscribers getting
 through--email address in "Real Name"
In-Reply-To: <9c17f228-b0b7-f557-813d-4fb828894b87@msapiro.net>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
 <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
 <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
 <b8175955-c76d-2bcb-ac95-e03fa8539dca@Damon-Family.org>
 <9c17f228-b0b7-f557-813d-4fb828894b87@msapiro.net>
Message-ID: <23386.40284.300867.603713@turnbull.sk.tsukuba.ac.jp>

Mark Sapiro writes:

 > First, the MLM side of this proposal is exactly what the Mailman "Wrap
 > Message" action does.

RFC-geeks-love-Mark!-ly y'rs


From turnbull.stephen.fw at u.tsukuba.ac.jp  Fri Jul 27 02:39:53 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Fri, 27 Jul 2018 15:39:53 +0900
Subject: [Mailman-Users] ARC
In-Reply-To: <c040a36d-efeb-3d45-4e03-9f168ed018fc@spamtrap.tnetconsulting.net>
References: <20180722200514.EA5FA20028DC81@ary.qy>
 <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>
 <23381.24927.74482.530338@turnbull.sk.tsukuba.ac.jp>
 <03d440cf-7cd5-4564-2900-ffc08cd7849b@spamtrap.tnetconsulting.net>
 <23384.18596.617417.251765@turnbull.sk.tsukuba.ac.jp>
 <c040a36d-efeb-3d45-4e03-9f168ed018fc@spamtrap.tnetconsulting.net>
Message-ID: <23386.48697.546335.165178@turnbull.sk.tsukuba.ac.jp>

Grant Taylor via Mailman-Users writes:

 > On 07/25/2018 03:53 AM, Stephen J. Turnbull wrote:
 > > That's not how "on behalf of" worked in practice.  What happened in April 
 > > 2014, was that a home business owner (HBO) would send a pile of completed 
 > > order notices to intuit.com, and intuit.com would send an invoice to each 
 > > customer on behalf of hbo at yahoo.com, from hbo at yahoo.com.
 > 
 > Will you please clarify what the SMTP envelope from and the From: header 
 > were in the example(s) that you're talking about?
 > 
 > I'm going to assume (for the sake of discussion or until corrected) that 
 > both the SMTP envelope from and the From: header were
 > hbo at yahoo.com.

I don't know about the envelope of such messages, which is irrelevant
to DMARC.  I suppose it typically was "hbo at yahoo.com".  The fact that
DMARC From alignment fails for Yahoo! addresses indicates that the
typical From address was "hbo at yahoo.com".

 > > Tens of thousands of invoices worth millions of dollars bounced off of 
 > > personal accounts and tiny business owner accounts at Yahoo! and other 
 > > receivers who take p=reject as a command.
 > 
 > I assume "bounced off of" means that the messages were rejected by 
 > receiving MTAs that honored Yahoo's (et al) p=reject DMARC configuration.

Yes.

 > I'll argue that DMARC is designed to 1) detect such spoofed email 
 > forgeries

They were neither spoofed nor forged.  The authority to use that
address was delegated in conformance with RFC 5322, just as an
executive delegates signature authority to a secretary.

 > and 2) enable Yahoo (et al) to publish what they would like to 
 > be done with such spoofed email forgeries.

In both (1) and (2) remove the word "such", and you are correct.

 > Granted, all the rejected / bounced invoices are contrary to the 
 > behavior that many individuals wanted.

And there were substantial costs.  Many SMEs had income delayed for
weeks.

 > I assume that this detection and rejection of any email claiming to be 
 > from Yahoo that didn't actually originate from Yahoo is exactly what 
 > Yahoo wanted to happen.

No.  It's exactly what they expected to happen, but they acknowledge
that such mail is legitimate.  Their position was to stand on the EULA
and say "tough luck, sometimes things don't work out -- what did you
expect for free?"

 > I feel like this is saying "We want something to detect the bad
 > guys and block them, but still allow the good guys to do the exact
 > same thing as the bad guys."  That has never worked very well.

Your definition of "exact same thing" seems to be "whatever gets
caught by DMARC p=reject", which is circular for present purposes.
The original purpose of DMARC p=reject was to prevent spear-phishing
by spoofing existing relationships, typically financial, and getting
recipients to click links they shouldn't.  These are "transactional"
mail flows, ie, business mail direct from vendor to client.  It wasn't
intended to prevent all spam defined as *any* UCE, as a fair amount of
UCE originates from addresses owned by the sender.

So DMARC was not intended to prevent spam in general, at least not by
the time I joined the WG mailing list.  Although some people were
still advocating that the death of mailing lists as we know them is a
small price to pay for universal p=reject, the majority, including
senior Yahoo! admins, denied that.

The Yahoo!/AOL problem, once again, was that leaking a billion address
books to spammers made spear-spamming feasible and profitable.
("Spear-spamming" is a word I just made up, meaning use of "From: Your
Friend" to get victims to pay attention to UCE that they would
otherwise ignore with extreme prejudice.)  They were "forced" to use
p=reject in a situation it wasn't designed for.

 > What would you want to see done, as Intuit's CISO, instead of what I 
 > propose?

Have clients' customers extend limited trust to intuit.com.  I'm not
going to go into it because I haven't done and don't have time to do a
proper analysis, even a sketchy one.  For what it's worth, my
intuition is that there is no particular "trust" benefit to having
clients delegate subdomains to Intuit rather than just having their
customers trust Intuit (of course Intuit needs to sign its mail), but
there are substantial costs in DNS management and increased attack
surface.

If you want to explain why you think there *is* a trust benefit, I'd
like to hear it.  On my side it's just a feeling. :-)  I'm quite
confident about the costs, though.

 > I would certainly hope that the company that I'm outsourcing
 > financial transactions to could secure the systems that process
 > money.

As I understand Intuit's main business affected by p=reject in 2014,
financial transactions weren't outsourced to Intuit, only accounting
(including tax preparation) and invoicing.

 > I think anything that spoofs or otherwise pretends to send as someone 
 > they are not is a form of abuse.

That's inconsistent with RFC 5322.  Distinguishing the agent
responsible for the content from the agent who submits the message is
the whole purpose of the Sender field.  I see no difference between a
human executive secretary and an automated on-behalf-of service in
this respect.  But DMARC doesn't care what's in Sender -- or in the
SMTP envelope, for that matter.  It only deals with From, specifically
with the author domain.  And this is the right field for the purpose,
because it is the author that the spear-phishing victim trusts, not
the submitting agent.

I think your notion of "spoof" is too formal, it is inconsistent with
RFC 5322, and therefore not well-adapted to understanding the effects
of DMARC p=reject.  That doesn't mean your belief that mailing lists
should adapt to DMARC, rather than DMARC adapting to a world with
Mediators (see RFC 5598) in it, is "wrong", just that it's hard to
have a conversation.

 > I also believe this form of abuse is (at least partially) exactly
 > what DMARC is designed to prevent.

DMARC is designed to prevent spear-phishing, not submission of
messages by agents other than the author.  Yahoo! and AOL, and a few
other domains, also use it to prevent spear-spamming.

Steve

From mailman at jordan.maileater.net  Fri Jul 27 18:41:33 2018
From: mailman at jordan.maileater.net (Jordan Brown)
Date: Fri, 27 Jul 2018 15:41:33 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <b8175955-c76d-2bcb-ac95-e03fa8539dca@Damon-Family.org>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
 <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
 <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
 <b8175955-c76d-2bcb-ac95-e03fa8539dca@Damon-Family.org>
Message-ID: <1d18c29e-912b-1f7e-ed6f-1200c26c4b4d@jordan.maileater.net>

On 7/25/2018 5:24 PM, Richard Damon wrote:
> Yes, one set of solutions would involve defining standards of how to
> compose composite messages, with standards on how to display them. A
> major part of the current issue is that for anything more than a
> single part plain text you can't be sure how it will be handled.

I can't say that I'm a true expert, or that I've really investigated,
but I believe you can wrap an entire message (with multiple parts) into
a single part of the wrapper message.? I don't think you need any new
message-structure standards.? You might need standards for headers that
would say that that's what you've done, and you certainly would need MUA
support.

(And to others who have replied:? yes, all understood.? Like I said,
adoption would be hard.)


From mailman at jordan.maileater.net  Fri Jul 27 18:57:43 2018
From: mailman at jordan.maileater.net (Jordan Brown)
Date: Fri, 27 Jul 2018 15:57:43 -0700
Subject: [Mailman-Users] ARC
In-Reply-To: <23386.40274.604718.454833@turnbull.sk.tsukuba.ac.jp>
References: <20180722200514.EA5FA20028DC81@ary.qy>
 <155ddbec-a343-d698-ae29-05856470e8d8@spamtrap.tnetconsulting.net>
 <23381.24927.74482.530338@turnbull.sk.tsukuba.ac.jp>
 <03d440cf-7cd5-4564-2900-ffc08cd7849b@spamtrap.tnetconsulting.net>
 <23384.18596.617417.251765@turnbull.sk.tsukuba.ac.jp>
 <490793b6-cc8f-fed7-6005-5ae296af3622@jordan.maileater.net>
 <23386.40274.604718.454833@turnbull.sk.tsukuba.ac.jp>
Message-ID: <389ac70c-1c68-ceb0-6467-8f04ffef527b@jordan.maileater.net>

On 7/26/2018 9:19 PM, Stephen J. Turnbull wrote:
> Jordan Brown writes:
>
>  > Well, yeah, but to provide such a service in a way that has any
>  > resemblance to being secure, Intuit *must* have some secret that allows
>  > it to send mail "from" those subdomains.? If Intuit doesn't need such a
>  > secret, then anybody could send mail like that.
>
> Sure, but (1) anyone can send mail like that anyway (and they do),

Wasn't this in the context of signature-checking schemes that detect
forged origin metadata?

> (2) the customers will (well, should) be checking invoices against their
> own purchasing records before they pay, and (3) after the vendor
> identifies Intuit as its billing agent, Intuit's own signature will do
> the trick.

So the vendor has to notify their customers who they use to do their
billing, and every time that they change billing vendors?? Ofttimes, the
goal is that the billing vendor is completely invisible to the end
customer.? I'm buying something from FrobozzCo; I should see e-mail that
comes from FrobozzCo (in a verifiable way), web pages that say FrobozzCo
and frobozzco.com, and the entry on my credit card statement should say
FROBOZZCO.? The fact that FrobozzCo uses Intuit is none of my business
and should be totally hidden from me.

Having your billing vendor be visible is, like having your company
e-mail address be @gmail.com, a mark of a tiny company that hasn't
really figured out how to make its business work.

> Securing a small number of own keys that get rotated on a schedule is
> one thing, securing a database of others' keys that regularly gets
> updated and multiple regular employees need access to is going to be
> quite another.

Not anywhere near as hard as it is for a full-scale e-mail vendor.?
Google secures a database of millions of users' secrets, and must have
internal and external controls that keep the wrong people from sending
mail that pretends to come from those users.


From Richard at Damon-Family.org  Fri Jul 27 19:18:16 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Fri, 27 Jul 2018 19:18:16 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <1d18c29e-912b-1f7e-ed6f-1200c26c4b4d@jordan.maileater.net>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
 <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
 <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
 <b8175955-c76d-2bcb-ac95-e03fa8539dca@Damon-Family.org>
 <1d18c29e-912b-1f7e-ed6f-1200c26c4b4d@jordan.maileater.net>
Message-ID: <4d6d7aec-a243-75e7-2161-67ca8dfdea03@Damon-Family.org>

On 7/27/18 6:41 PM, Jordan Brown wrote:
> On 7/25/2018 5:24 PM, Richard Damon wrote:
>> Yes, one set of solutions would involve defining standards of how to
>> compose composite messages, with standards on how to display them. A
>> major part of the current issue is that for anything more than a
>> single part plain text you can't be sure how it will be handled.
>
> I can't say that I'm a true expert, or that I've really investigated,
> but I believe you can wrap an entire message (with multiple parts)
> into a single part of the wrapper message.? I don't think you need any
> new message-structure standards.? You might need standards for headers
> that would say that that's what you've done, and you certainly would
> need MUA support.
>
> (And to others who have replied:? yes, all understood.? Like I said,
> adoption would be hard.)
>
Yes, there are existing formats that at least mostly represent this in
the message itself, but not for display. Especially that currently the
wrapping message would say it is from the list, but you really want some
way for it to say that in the MUA's message list, it should indicate who
the author of the embedded message was, not the 'author' of the wrapping
message. This probably means that we need a new message content type to
indicate it.

Also, the MUA (or maybe their MTA) should know enough to pierce through
that wrapping message and give an indication that the wrapped message
passes or fails the appropriate tests. The current formatting doesn't
imply that that should happen.

-- 
Richard Damon


From mailman at jordan.maileater.net  Fri Jul 27 19:28:05 2018
From: mailman at jordan.maileater.net (Jordan Brown)
Date: Fri, 27 Jul 2018 16:28:05 -0700
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <4d6d7aec-a243-75e7-2161-67ca8dfdea03@Damon-Family.org>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
 <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
 <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
 <b8175955-c76d-2bcb-ac95-e03fa8539dca@Damon-Family.org>
 <1d18c29e-912b-1f7e-ed6f-1200c26c4b4d@jordan.maileater.net>
 <4d6d7aec-a243-75e7-2161-67ca8dfdea03@Damon-Family.org>
Message-ID: <5f7b7ca5-8760-b2d0-9fb8-72a684e2a103@jordan.maileater.net>

On 7/27/2018 4:18 PM, Richard Damon wrote:
> Yes, there are existing formats that at least mostly represent this in
> the message itself, but not for display. Especially that currently the
> wrapping message would say it is from the list, but you really want
> some way for it to say that in the MUA's message list, it should
> indicate who the author of the embedded message was, not the 'author'
> of the wrapping message. This probably means that we need a new
> message content type to indicate it. 

A message content type is one possibility, but other headers might also
be reasonable.

> Also, the MUA (or maybe their MTA) should know enough to pierce through
> that wrapping message and give an indication that the wrapped message
> passes or fails the appropriate tests. The current formatting doesn't
> imply that that should happen.

Shrug.? I wouldn't consider it to be silly for an MUA to apply those
tests to any message/rfc822 part, whether or not it came from a mailing
list.

If I do a forward-as-attachment to forward a message to you, it would be
good if you could independently verify that the forwarded message is
from who it says it is from.

Anyhow, it's clearly possible, probably with minimal standards for
message metadata.? The problem (after getting agreement on the metadata)
is getting an adequate number of MUAs to behave well with the wrapped
messages.


From Richard at Damon-Family.org  Fri Jul 27 20:28:36 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Fri, 27 Jul 2018 20:28:36 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <5f7b7ca5-8760-b2d0-9fb8-72a684e2a103@jordan.maileater.net>
References: <20180722200334.B150E20028DC36@ary.qy>
 <c2b71208-735a-398a-df87-8ae574a17f22@spamtrap.tnetconsulting.net>
 <BF0EFD41-D8E1-4C18-8AAE-A014240BF725@Damon-family.org>
 <78baab65-f7d3-ce56-bc36-a16a15118117@spamtrap.tnetconsulting.net>
 <7B0A9B1E-6066-4E79-844B-86532068BE1B@Damon-family.org>
 <955b6e43-1056-fdc7-e259-d2dadbcedb53@spamtrap.tnetconsulting.net>
 <C873030D-599F-4FAF-AEBB-63E82F4BE742@Damon-family.org>
 <88902b3b-7cb3-7991-15c4-4dbc107624b8@msapiro.net>
 <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
 <b8175955-c76d-2bcb-ac95-e03fa8539dca@Damon-Family.org>
 <1d18c29e-912b-1f7e-ed6f-1200c26c4b4d@jordan.maileater.net>
 <4d6d7aec-a243-75e7-2161-67ca8dfdea03@Damon-Family.org>
 <5f7b7ca5-8760-b2d0-9fb8-72a684e2a103@jordan.maileater.net>
Message-ID: <485ccc41-11fa-6a85-b233-68f665105427@Damon-Family.org>

On 7/27/18 7:28 PM, Jordan Brown wrote:
> On 7/27/2018 4:18 PM, Richard Damon wrote:
>> Yes, there are existing formats that at least mostly represent this
>> in the message itself, but not for display. Especially that currently
>> the wrapping message would say it is from the list, but you really
>> want some way for it to say that in the MUA's message list, it should
>> indicate who the author of the embedded message was, not the 'author'
>> of the wrapping message. This probably means that we need a new
>> message content type to indicate it. 
>
> A message content type is one possibility, but other headers might
> also be reasonable.
>
>> Also, the MUA (or maybe their MTA) should know enough to pierce through
>> that wrapping message and give an indication that the wrapped message
>> passes or fails the appropriate tests. The current formatting doesn't
>> imply that that should happen.
>
> Shrug.? I wouldn't consider it to be silly for an MUA to apply those
> tests to any message/rfc822 part, whether or not it came from a
> mailing list.
>
> If I do a forward-as-attachment to forward a message to you, it would
> be good if you could independently verify that the forwarded message
> is from who it says it is from.
>
> Anyhow, it's clearly possible, probably with minimal standards for
> message metadata.? The problem (after getting agreement on the
> metadata) is getting an adequate number of MUAs to behave well with
> the wrapped messages.
>
Mark, maybe, but the option to reject, no, as the message might be an
older message that is being forwarded from an archive, and perhaps keys
or other settings have changed. Also, part of DMARC (SPF) can't be
tested for, as we don't have the original source the message was sent
from, so we can only make a DKIM test.

-- 
Richard Damon


From johnl at taugh.com  Fri Jul 27 21:49:05 2018
From: johnl at taugh.com (John Levine)
Date: 27 Jul 2018 21:49:05 -0400
Subject: [Mailman-Users] non-subscribers getting through--email address
 in "Real Name"
In-Reply-To: <885f93f0-36ec-d74f-7c5f-52b42f2d6d42@jordan.maileater.net>
Message-ID: <20180728014905.34C822002E65EC@ary.qy>

In article <885f93f0-36ec-d74f-7c5f-52b42f2d6d42 at jordan.maileater.net> you write:
>Hmm.? It would take MUA changes to be fully effective, but a possibility
>that comes to mind is to have mailing lists leave the original message
>absolutely unmodified, but wrap it in a message that comes "from" the
>mailing list.? That way everything about the message is verifiably true.

Please see other messages in this thread.  Wrapping messages is an old
idea, effectively making each message into a one-message digest,
something Mailman can already do.

The problem is that MUAs vary from so-so to awful when displaying them.

 

From lists at damorris.com  Sat Jul 28 03:02:11 2018
From: lists at damorris.com (Adam Morris)
Date: Sat, 28 Jul 2018 17:02:11 +1000
Subject: [Mailman-Users] bounce notification to owner from non members?
Message-ID: <99b35c01-4a5a-efd7-897a-19236b1962a3@damorris.com>

Hi all,
I receive about 8 bounce messages a day for one of my lists from people 
trying to post to it who aren't members.
The list is set up as private so not sure how these spammers find it.
I've been to the bounce page but can't work out how to stop these 
messages but allow bounce messages from members?
TIA.


-- 

Adam Morris
Jaws certified 2016
Email, iMessage & FaceTime
adam at damorris.com


From Richard at Damon-Family.org  Sat Jul 28 07:52:57 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Sat, 28 Jul 2018 07:52:57 -0400
Subject: [Mailman-Users] bounce notification to owner from non members?
In-Reply-To: <99b35c01-4a5a-efd7-897a-19236b1962a3@damorris.com>
References: <99b35c01-4a5a-efd7-897a-19236b1962a3@damorris.com>
Message-ID: <6a9406e7-a6bd-c1b4-517f-5857fc66b53d@Damon-Family.org>

On 7/28/18 3:02 AM, Adam Morris wrote:
> Hi all,
> I receive about 8 bounce messages a day for one of my lists from
> people trying to post to it who aren't members.
> The list is set up as private so not sure how these spammers find it.
> I've been to the bounce page but can't work out how to stop these
> messages but allow bounce messages from members?
> TIA

Is it a HOLD message? If so then on the Sender Filter page you want to
make the non-member action be discard. If you make it Reject, then you
will see a bounce message if the spam message and a bad from address
(which it normally will have), also if you use reject then if the
message has a faked from address that is actually valid, you are
committing back-scatter and might get your list black-listed.

It would be nice if Mailman could tell the MTA as it was receiving the
message that it wasn't acceptable so it could be rejected then, but
since it generally can't, you need to discard spam, not reject it.

-- 
Richard Damon


From mark at msapiro.net  Sat Jul 28 10:46:59 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 28 Jul 2018 07:46:59 -0700
Subject: [Mailman-Users] bounce notification to owner from non members?
In-Reply-To: <6a9406e7-a6bd-c1b4-517f-5857fc66b53d@Damon-Family.org>
References: <99b35c01-4a5a-efd7-897a-19236b1962a3@damorris.com>
 <6a9406e7-a6bd-c1b4-517f-5857fc66b53d@Damon-Family.org>
Message-ID: <2c45ac96-145b-dc8d-7717-140760dbaefc@msapiro.net>

On 07/28/2018 04:52 AM, Richard Damon wrote:
> On 7/28/18 3:02 AM, Adam Morris wrote:
>> Hi all,
>> I receive about 8 bounce messages a day for one of my lists from
>> people trying to post to it who aren't members.
>> The list is set up as private so not sure how these spammers find it.
>> I've been to the bounce page but can't work out how to stop these
>> messages but allow bounce messages from members?
>> TIA
> 
> Is it a HOLD message? If so then on the Sender Filter page you want to
> make the non-member action be discard. If you make it Reject, then you
> will see a bounce message if the spam message and a bad from address
> (which it normally will have), also if you use reject then if the
> message has a faked from address that is actually valid, you are
> committing back-scatter and might get your list black-listed.


If the above doesn't answer the question, please be more specific about
exactly what messages you are concerned about. If you are talking about
"unrecognized bounce" notices, these are from spam sent to the
listname-bounces address. You can stop them by setting
bounce_unrecognized_goes_to_list_owner to No. The risk of a legitimate
bounce from a member not being recognized is extremely small.


> It would be nice if Mailman could tell the MTA as it was receiving the
> message that it wasn't acceptable so it could be rejected then, but
> since it generally can't, you need to discard spam, not reject it.


See both links in the post at
<https://mail.python.org/pipermail/mailman-users/2018-July/083558.html>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From bernie at fantasyfarm.com  Mon Jul 30 08:08:11 2018
From: bernie at fantasyfarm.com (Bernie Cosell)
Date: Mon, 30 Jul 2018 08:08:11 -0400
Subject: [Mailman-Users] What does mailman do with a POST?
Message-ID: <5B5EFFAB.761.4D719CC2@bernie.fantasyfarm.com>

I'm trying to submit a form or two from a program and I can't quite 
understand Mailman's form handling, this with Mailman 2.1.23.

What I've done is a little program that does something simple: it first 
authenticates [and gets the appropriate cookies], then I GET the 
mass-removal page [to get its csrf_object.  Then I do a simple POST with 
just three form fields:

csrf_object=>  <whatever it was in the GET of the page>
unsubscribees  => the email address,
'setmemberopts_btn  => 'Submit Your Changes'

The response I get back is the original page - no error and no success, just 
the mass-removal page again.  It is what would happen if you clicked on 
the submit button without giving it anything to do

I've also tried a similar POSTing with every field from the form included, 
and always the same: a 200 response and I get the mass-removal page back 
again.   I'd look at the code, but I'm a perl guy -- never messed with python.

What does Mailman do to when receiving a POST to decide if there's 
something to do or not?  THANKS!

  /Bernie\

                 Bernie Cosell
         bernie at fantasyfarm.com
-- Too many people; too few sheep --


From eggert at eehmke.de  Tue Jul 31 05:47:21 2018
From: eggert at eehmke.de (Eggert Ehmke)
Date: Tue, 31 Jul 2018 11:47:21 +0200
Subject: [Mailman-Users] Change of subdomain
Message-ID: <2319177.LPDiY5UNOF@wookie>

Hello, until recently I had my mailman web configuration in a directory in my 
www domain, like www.mydomain.de/mailman. Now I changed the structure to have 
an URL like lists.mydomain.de. I also changed the apache configuration and the 
/etc/mailman/mm_cfg.py file. This kind of works, I can access my admin panel 
fine. But some internal urls like the mailing list archive still refer to the 
old www url.  What must I do to clean that up?

From mark at msapiro.net  Tue Jul 31 16:32:38 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 31 Jul 2018 13:32:38 -0700
Subject: [Mailman-Users] Change of subdomain
In-Reply-To: <2319177.LPDiY5UNOF@wookie>
References: <2319177.LPDiY5UNOF@wookie>
Message-ID: <22233b97-f2e9-f501-fe6c-9ec44d4e2bd1@msapiro.net>

On 07/31/2018 02:47 AM, Eggert Ehmke via Mailman-Users wrote:
> Hello, until recently I had my mailman web configuration in a directory in my 
> www domain, like www.mydomain.de/mailman. Now I changed the structure to have 
> an URL like lists.mydomain.de. I also changed the apache configuration and the 
> /etc/mailman/mm_cfg.py file. This kind of works, I can access my admin panel 
> fine. But some internal urls like the mailing list archive still refer to the 
> old www url.  What must I do to clean that up?


See <https://wiki.list.org/x/4030592>. The "Changing existing archives"
section at the end has information on this.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Tue Jul 31 18:53:51 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 31 Jul 2018 15:53:51 -0700
Subject: [Mailman-Users] What does mailman do with a POST?
In-Reply-To: <5B5EFFAB.761.4D719CC2@bernie.fantasyfarm.com>
References: <5B5EFFAB.761.4D719CC2@bernie.fantasyfarm.com>
Message-ID: <239434e1-6497-55d0-6427-4840a98543c3@msapiro.net>

On 07/30/2018 05:08 AM, Bernie Cosell wrote:
> I'm trying to submit a form or two from a program and I can't quite 
> understand Mailman's form handling, this with Mailman 2.1.23.
> 
> What I've done is a little program that does something simple: it first 
> authenticates [and gets the appropriate cookies], then I GET the 
> mass-removal page [to get its csrf_object.  Then I do a simple POST with 
> just three form fields:
> 
> csrf_object=>  <whatever it was in the GET of the page>
> unsubscribees  => the email address,
> 'setmemberopts_btn  => 'Submit Your Changes'
> 
> The response I get back is the original page - no error and no success, just 
> the mass-removal page again.  It is what would happen if you clicked on 
> the submit button without giving it anything to do
> 
> I've also tried a similar POSTing with every field from the form included, 
> and always the same: a 200 response and I get the mass-removal page back 
> again.   I'd look at the code, but I'm a perl guy -- never messed with python.
> 
> What does Mailman do to when receiving a POST to decide if there's 
> something to do or not?  THANKS!


It looks at the POST data. In the case of the admin CGI just returning
the Mass Removals form with no errors and no changes, this means it got
the right path to LISTNAME/members/remove and it got a good csrf_token.

Given that, the only way it would return the form with no message is if
it got no non-blank values for unsubscribees or unsubscribees_upload.

You refer to csrf_token as csrf_object. If you are really submitting the
value as csrf_object, that wouldn't work, but you should get an error.

If you are using some kind of framework for submitting the post that
would receive a redirect response of some kind and transparently do the
redirection (eg http -> https) before returning the result, this could
be <https://wiki.list.org/x/4030602> so see that.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From bernie at fantasyfarm.com  Tue Jul 31 21:15:53 2018
From: bernie at fantasyfarm.com (Bernie Cosell)
Date: Tue, 31 Jul 2018 21:15:53 -0400
Subject: [Mailman-Users] What does mailman do with a POST?
In-Reply-To: <239434e1-6497-55d0-6427-4840a98543c3@msapiro.net>
References: <5B5EFFAB.761.4D719CC2@bernie.fantasyfarm.com>,
 <239434e1-6497-55d0-6427-4840a98543c3@msapiro.net>
Message-ID: <5B6109C9.20999.B94F0D@bernie.fantasyfarm.com>

On 31 Jul 2018 at 15:53, Mark Sapiro wrote:

> On 07/30/2018 05:08 AM, Bernie Cosell wrote:

> > I'm trying to submit a form or two from a program and I can't quite 
> > understand Mailman's form handling, this with Mailman 2.1.23.

> > What does Mailman do to when receiving a POST to decide if there's 
> > something to do or not?  THANKS!
> 
> 
> It looks at the POST data. In the case of the admin CGI just returning
> the Mass Removals form with no errors and no changes, this means it
> got
> the right path to LISTNAME/members/remove and it got a good
> csrf_token.

That's a start... :o)  

> Given that, the only way it would return the form with no message is
> if
> it got no non-blank values for unsubscribees or unsubscribees_upload.
> 
> You refer to csrf_token as csrf_object. If you are really submitting
> the
> value as csrf_object, that wouldn't work, but you should get an error.

Right , I mistyped..  I still don't understand why I'm not getting anything.  
Here's the request I just sent:

'cookie2' => '$Version="1"',
'csrf-token' => 
'2802000000694a07615b7347000000746573746c6973742d66616e74617379666
1726d2e636f6d2b61646d696e3a31393763333633326633636638316533393631
656132313039623235383137343533386232366139',
'setmemberopts-btn' => 'Submit Your Changes',
 'cookie' => 
'testlist-fantasyfarm.com+admin=2802000000694a07615b7328000000313937
633336333266336366383165333936316561323130396232353831373435333862
32366139',
 'user-agent' => 'Mozilla/5.0',
 'unsubscribees' => 'bernie'
----------------------------------------
 '_method' => 'POST',
                                        '_uri' => 
http://lists.fantasyfarm.com/admin.cgi/testlist-fantasyfarm.com/member
s/remove')}, 'URI::http' ),

And the page I got back was the remove page -- no error no success [it 
should be an error, since I'm not subscribed to the list I'm playing with..:o)]
Do I have to do something special with the data for the 'unsubscribees' ?  I 
know it is a TEXTAREA -- but I just assumed I could pass it a string and 
that'd be OK.

Thanks for the help!
  /Bernie\
            Bernie Cosell
       bernie at fantasyfarm.com
-- Too many people; too few sheep --
   




From Richard at Damon-Family.org  Tue Jul 31 21:25:32 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Tue, 31 Jul 2018 21:25:32 -0400
Subject: [Mailman-Users] What does mailman do with a POST?
In-Reply-To: <5B6109C9.20999.B94F0D@bernie.fantasyfarm.com>
References: <5B5EFFAB.761.4D719CC2@bernie.fantasyfarm.com>
 <239434e1-6497-55d0-6427-4840a98543c3@msapiro.net>
 <5B6109C9.20999.B94F0D@bernie.fantasyfarm.com>
Message-ID: <864eda80-aa64-f2ec-c166-347915774e4c@Damon-Family.org>

On 7/31/18 9:15 PM, Bernie Cosell wrote:
> On 31 Jul 2018 at 15:53, Mark Sapiro wrote:
>
>> On 07/30/2018 05:08 AM, Bernie Cosell wrote:
>>> I'm trying to submit a form or two from a program and I can't quite 
>>> understand Mailman's form handling, this with Mailman 2.1.23.
>>> What does Mailman do to when receiving a POST to decide if there's 
>>> something to do or not?  THANKS!
>>
>> It looks at the POST data. In the case of the admin CGI just returning
>> the Mass Removals form with no errors and no changes, this means it
>> got
>> the right path to LISTNAME/members/remove and it got a good
>> csrf_token.
> That's a start... :o)  
>
>> Given that, the only way it would return the form with no message is
>> if
>> it got no non-blank values for unsubscribees or unsubscribees_upload.
>>
>> You refer to csrf_token as csrf_object. If you are really submitting
>> the
>> value as csrf_object, that wouldn't work, but you should get an error.
> Right , I mistyped..  I still don't understand why I'm not getting anything.  
> Here's the request I just sent:
>
> 'cookie2' => '$Version="1"',
> 'csrf-token' => 
> '2802000000694a07615b7347000000746573746c6973742d66616e74617379666
> 1726d2e636f6d2b61646d696e3a31393763333633326633636638316533393631
> 656132313039623235383137343533386232366139',
> 'setmemberopts-btn' => 'Submit Your Changes',
>  'cookie' => 
> 'testlist-fantasyfarm.com+admin=2802000000694a07615b7328000000313937
> 633336333266336366383165333936316561323130396232353831373435333862
> 32366139',
>  'user-agent' => 'Mozilla/5.0',
>  'unsubscribees' => 'bernie'
> ----------------------------------------
>  '_method' => 'POST',
>                                         '_uri' => 
> http://lists.fantasyfarm.com/admin.cgi/testlist-fantasyfarm.com/member
> s/remove')}, 'URI::http' ),
>
> And the page I got back was the remove page -- no error no success [it 
> should be an error, since I'm not subscribed to the list I'm playing with..:o)]
> Do I have to do something special with the data for the 'unsubscribees' ?  I 
> know it is a TEXTAREA -- but I just assumed I could pass it a string and 
> that'd be OK.
>
> Thanks for the help!
>   /Bernie\
>             Bernie Cosell
>        bernie at fantasyfarm.com
> -- Too many people; too few sheep --
>    
With Mailman2, submitting a non-subscribed email address to the
unsubscribe link doesn't give an error message. If you read the text
sent back, it says something like IF you were subscribed, and email will
be set to your email address to confirm the unsubscription. I presume
this is part of Mailmans privacy policy to not let people find out if
someone is subscribed to a list (assuming the list doesn't post the
subscriber list).


-- 
Richard Damon


From bernie at fantasyfarm.com  Tue Jul 31 21:32:06 2018
From: bernie at fantasyfarm.com (Bernie Cosell)
Date: Tue, 31 Jul 2018 21:32:06 -0400
Subject: [Mailman-Users] What does mailman do with a POST?
In-Reply-To: <864eda80-aa64-f2ec-c166-347915774e4c@Damon-Family.org>
References: <5B5EFFAB.761.4D719CC2@bernie.fantasyfarm.com>,
 <5B6109C9.20999.B94F0D@bernie.fantasyfarm.com>,
 <864eda80-aa64-f2ec-c166-347915774e4c@Damon-Family.org>
Message-ID: <5B610D96.9461.C827DE@bernie.fantasyfarm.com>

On 31 Jul 2018 at 21:25, Richard Damon wrote:

> On 7/31/18 9:15 PM, Bernie Cosell wrote:

> > Here's the request I just sent:
> >
> > 'setmemberopts-btn' => 'Submit Your Changes',
> >  'cookie' => 
> >  'unsubscribees' => 'bernie'
> > ----------------------------------------
> >  '_method' => 'POST',
> >                                         '_uri' => 
> >
> > Do I have to do something special with the data for the
> 'unsubscribees' ?  I 
> > know it is a TEXTAREA -- but I just assumed I could pass it a string
> and 
> > that'd be OK.

> With Mailman2, submitting a non-subscribed email address to the
> unsubscribe link doesn't give an error message.

Hmm.  the server is running  2.1.23 and I just tried unsubscribing "bernie" 
again from my web browser [and I'm still not on the list..:o)]  and I got:
-------------------
Cannot unsubscribe non-members:

    bernie 
-------------------------
in bold red letters at the top and then it is followed by the rest of the HTML 
for the mass-removal page.

  /B\
 If you read the text
> sent back, it says something like IF you were subscribed, and email
> will
> be set to your email address to confirm the unsubscription. I presume
> this is part of Mailmans privacy policy to not let people find out if
> someone is subscribed to a list (assuming the list doesn't post the
> subscriber list).
> 
> 
> -- 
> Richard Damon
> 
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/bernie%40fantasyfa
> rm.com


            Bernie Cosell
       bernie at fantasyfarm.com
-- Too many people; too few sheep --
   




From Richard at Damon-Family.org  Tue Jul 31 21:57:18 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Tue, 31 Jul 2018 21:57:18 -0400
Subject: [Mailman-Users] What does mailman do with a POST?
In-Reply-To: <5B610D96.9461.C827DE@bernie.fantasyfarm.com>
References: <5B5EFFAB.761.4D719CC2@bernie.fantasyfarm.com>
 <5B6109C9.20999.B94F0D@bernie.fantasyfarm.com>
 <864eda80-aa64-f2ec-c166-347915774e4c@Damon-Family.org>
 <5B610D96.9461.C827DE@bernie.fantasyfarm.com>
Message-ID: <c57a1c81-4acb-3f09-8d39-3b03f6d2172d@Damon-Family.org>

On 7/31/18 9:32 PM, Bernie Cosell wrote:
> On 31 Jul 2018 at 21:25, Richard Damon wrote:
>
>> On 7/31/18 9:15 PM, Bernie Cosell wrote:
>>> Here's the request I just sent:
>>>
>>> 'setmemberopts-btn' => 'Submit Your Changes',
>>>  'cookie' => 
>>>  'unsubscribees' => 'bernie'
>>> ----------------------------------------
>>>  '_method' => 'POST',
>>>                                         '_uri' => 
>>>
>>> Do I have to do something special with the data for the
>> 'unsubscribees' ?  I 
>>> know it is a TEXTAREA -- but I just assumed I could pass it a string
>> and 
>>> that'd be OK.
>> With Mailman2, submitting a non-subscribed email address to the
>> unsubscribe link doesn't give an error message.
> Hmm.  the server is running  2.1.23 and I just tried unsubscribing "bernie" 
> again from my web browser [and I'm still not on the list..:o)]  and I got:
> -------------------
> Cannot unsubscribe non-members:
>
>     bernie 
> -------------------------
> in bold red letters at the top and then it is followed by the rest of the HTML 
> for the mass-removal page.
>
>   /B\
>  If you read the text
>> sent back, it says something like IF you were subscribed, and email
>> will
>> be set to your email address to confirm the unsubscription. I presume
>> this is part of Mailmans privacy policy to not let people find out if
>> someone is subscribed to a list (assuming the list doesn't post the
>> subscriber list).
>>
>>
>> -- 
>> Richard Damon
>>
>> ------------------------------------------------------
>> Mailman-Users mailing list Mailman-Users at python.org
>> https://mail.python.org/mailman/listinfo/mailman-users
>> Mailman FAQ: http://wiki.list.org/x/AgA3
>> Security Policy: http://wiki.list.org/x/QIA9
>> Searchable Archives:
>> http://www.mail-archive.com/mailman-users%40python.org/
>> Unsubscribe:
>> https://mail.python.org/mailman/options/mailman-users/bernie%40fantasyfa
>> rm.com
>
>             Bernie Cosell
>        bernie at fantasyfarm.com
> -- Too many people; too few sheep --
>    
Sorry, I was thinking of the member unsubscribed page, not the admin page.
The admin does know who is subscribed so no hiding is needed.

One thought is could the anti-robot code be affecting you here, and you
need to add a pause between getting the token and submitting the form to
look like a human (it really shouldn't need to be active on admin pages,
but I don't know if it makes the distinction.)

-- 
Richard Damon


From mark at msapiro.net  Tue Jul 31 22:26:18 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 31 Jul 2018 19:26:18 -0700
Subject: [Mailman-Users] What does mailman do with a POST?
In-Reply-To: <5B6109C9.20999.B94F0D@bernie.fantasyfarm.com>
References: <5B5EFFAB.761.4D719CC2@bernie.fantasyfarm.com>
 <239434e1-6497-55d0-6427-4840a98543c3@msapiro.net>
 <5B6109C9.20999.B94F0D@bernie.fantasyfarm.com>
Message-ID: <633a12e0-2c58-06eb-d569-0ab12023721a@msapiro.net>

On 07/31/2018 06:15 PM, Bernie Cosell wrote:

>  'cookie' => 
> 'testlist-fantasyfarm.com+admin=.....',


I recognize this is a test list and you may not care, but that cookie
allows anyone to access the web admin UI for the list until you change
the admin password.


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Tue Jul 31 23:08:23 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 31 Jul 2018 20:08:23 -0700
Subject: [Mailman-Users] What does mailman do with a POST?
In-Reply-To: <5B6109C9.20999.B94F0D@bernie.fantasyfarm.com>
References: <5B5EFFAB.761.4D719CC2@bernie.fantasyfarm.com>
 <239434e1-6497-55d0-6427-4840a98543c3@msapiro.net>
 <5B6109C9.20999.B94F0D@bernie.fantasyfarm.com>
Message-ID: <9fc8ade2-116d-3bbf-27e2-a32f6f95986d@msapiro.net>

On 07/31/2018 06:15 PM, Bernie Cosell wrote:
> 
> 'cookie2' => '$Version="1"',
> 'csrf-token' => 
> '...',
> 'setmemberopts-btn' => 'Submit Your Changes',
>  'cookie' => 
> 'testlist-fantasyfarm.com+admin=...',
>  'user-agent' => 'Mozilla/5.0',
>  'unsubscribees' => 'bernie'
> ----------------------------------------
>  '_method' => 'POST',
>                                         '_uri' => 
> http://lists.fantasyfarm.com/admin.cgi/testlist-fantasyfarm.com/member
> s/remove')}, 'URI::http' ),
> 
> And the page I got back was the remove page -- no error no success [it 
> should be an error, since I'm not subscribed to the list I'm playing with..:o)]
> Do I have to do something special with the data for the 'unsubscribees' ?  I 
> know it is a TEXTAREA -- but I just assumed I could pass it a string and 
> that'd be OK.


The above should be OK. The unsubscribees should be a string of email
address and/or display_name <address> lines separated by newlines, but
just "bernie" should be enough to give the error.

I note however that I went to
http://lists.fantasyfarm.com/listinfo.cgi/testlist-fantasyfarm.com just
to see it, and I can see by the form of the URL ythat this is a modified
Mailman. Primarily the fact that the listname in the URL is
'testlist-fantasyfarm.com' rather than just 'testlist' indicates that
this is a Mailman modified to support lists in multiple domains without
list name collisions. Also, the footer has no link to the listinfo overview.

Without knowing what the mods are, it's hard for me to be definitive
about anything, however if your script is posting the same data that a
web browser would post, it SHOULD work.

Further, If the CGI is comparable to the standard one, the fact that you
don't get a message about form lifetime expired means it gets your
csrf_token OK and the fact that you don't get the login page in return
means it gets your cookie OK and all you should need beyond that to at
least get an error is a non-blank unsubscribees value.

Thus, I don't know what's going on.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan