From dmaziuk at bmrb.wisc.edu Sun Oct 1 17:34:07 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Sun, 1 Oct 2017 16:34:07 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <201709301647.v8UGldlY094713@fire.js.berklix.net> References: <201709301647.v8UGldlY094713@fire.js.berklix.net> Message-ID: On 09/30/2017 11:47 AM, Julian H. Stacey wrote: >> Question: what is mailman actually doing?g? > > Look in /usr/local/mailman/logs/* /var/spool/mqueue/ > or whatever your local equivalent paths are It's /var/log/mailman/subscribe in this case, thank you, but it shows "new mail at addr, admin mass sub" for addrs from a to j up to Sep 25. (I sorted the list before upload and it goes all the way to z.) Postfix log shows welcome e-mails still going out as of an hour ago. So it looks like mailman updates its logs only a little more often then it updates its configs. Going through postfix log trying to spot an address that looks like it may be from the list is rather inconvenient. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From dmaziuk at bmrb.wisc.edu Sun Oct 1 17:44:04 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Sun, 1 Oct 2017 16:44:04 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> <71e5886c-a977-e90d-502f-e659facbc416@msapiro.net> <3a8c7444-6507-f8f4-8b71-80644fb7e0d7@bmrb.wisc.edu> Message-ID: <30bdbca9-5543-d003-6273-5d4964eea9f3@bmrb.wisc.edu> On 09/30/2017 05:19 PM, Mark Sapiro wrote: > On 09/30/2017 02:31 PM, Dimitri Maziuk wrote: .. > It's in progress as far as you're concerned, but not as far as Mailman > is concerned. As far as Mailman is concerned, some users were sent > welcome messages, but no one has been subscribed. End of story. ... >> I assume it does save state internally and the process will be resumed >> if interrupted? ... > Wrong. You have to start over. Nothing is saved from the first attempt. > The process died and the state was restored to the last known good state > which is before anyone was subscribed. So anything goes wrong, everybody got the welcome spam but nobody got subscribed. People who get the welcome messages (up to "j" now) and click on the unsubscribe link get an error. For another week or so, judging by what I can work out from the postfix logs -- because there's no other place to look. I wish I found any of it surprising. Sadly I've been dealing with software (including my own) for too long for that. Situation normal, moving along. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Sun Oct 1 17:55:24 2017 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 1 Oct 2017 14:55:24 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <201709301647.v8UGldlY094713@fire.js.berklix.net> Message-ID: <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> On 10/01/2017 02:34 PM, Dimitri Maziuk wrote: > It's /var/log/mailman/subscribe in this case, thank you, but it shows > "new mail at addr, admin mass sub" > for addrs from a to j up to Sep 25. (I sorted the list before upload and, but those subscribes were lost > it goes all the way to z.) Postfix log shows welcome e-mails still going > out as of an hour ago. > > So it looks like mailman updates its logs only a little more often then > it updates its configs. The log entries were written as Mailman did the subscribes and sent the welcomes, but those subscriptions were lost when the process aborted before finishing. No more subscriptions are currently being done. If welcome messages are still going out, they are queued either in Postfix or in Mailman's 'virgin' or 'out' queues, but new subscriptions are not happening since the list membership is still empty. The queue directories may be in /var/lib/mailman/qfiles/ or /var/spool/mailman/ or somewhere else depending on what mailman package you have. If you find a bunch of queued welcome messages in one of those queues, you can just remove them. You can view them with Mailman's bin/show_qfiles. You can also stop mailman and if the messages are queued in Mailman, Mailman will stop sending them. The real question is if your installation is so broken that it takes 5 days to send fewer than half of 7000 welcome messages, what's going to happen when you send a post to a list with 7000 members? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Sun Oct 1 18:01:33 2017 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 1 Oct 2017 15:01:33 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <30bdbca9-5543-d003-6273-5d4964eea9f3@bmrb.wisc.edu> References: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> <71e5886c-a977-e90d-502f-e659facbc416@msapiro.net> <3a8c7444-6507-f8f4-8b71-80644fb7e0d7@bmrb.wisc.edu> <30bdbca9-5543-d003-6273-5d4964eea9f3@bmrb.wisc.edu> Message-ID: <08a0ddb5-11c9-860d-8581-7652a0b9fef3@msapiro.net> On 10/01/2017 02:44 PM, Dimitri Maziuk wrote: > > For another week or so, > judging by what I can work out from the postfix logs -- because there's > no other place to look. See my other reply at -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From dmaziuk at bmrb.wisc.edu Sun Oct 1 18:34:29 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Sun, 1 Oct 2017 17:34:29 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> Message-ID: On 10/01/2017 04:55 PM, Mark Sapiro wrote: > The log entries were written as Mailman did the subscribes and sent the > welcomes, but those subscriptions were lost when the process aborted > before finishing. No more subscriptions are currently being done. So I don't get it, are you saying that is *was* aborted after the CGI timed out? Or are you following on my "if something happens to interrupt it"? -- it was a hypothetical question about checkpointing and keeping state during long long-running tasks. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Sun Oct 1 18:50:44 2017 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 01 Oct 2017 15:50:44 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> Message-ID: On October 1, 2017 3:34:29 PM PDT, Dimitri Maziuk wrote: >On 10/01/2017 04:55 PM, Mark Sapiro wrote: > >> The log entries were written as Mailman did the subscribes and sent >the >> welcomes, but those subscriptions were lost when the process aborted >> before finishing. No more subscriptions are currently being done. > >So I don't get it, are you saying that is *was* aborted after the CGI >timed out? Yes. >Or are you following on my "if something happens to interrupt it"? -- As I see it that's the same thing. The something that happened was the abort of the CGI. >it >was a hypothetical question about checkpointing and keeping state >during >long long-running tasks. Mailman 2.1 does no check pointing. There is no database in the usual sense. It does have a mechanism for backing up and recovering if delivery of a list message is interrupted, but that's it. -- Mark Sapiro Sent from my Not_an_iThing with standards compliant, open source software. From cpz at tuunq.com Sun Oct 1 23:23:40 2017 From: cpz at tuunq.com (Carl Zwanzig) Date: Sun, 1 Oct 2017 20:23:40 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> References: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> Message-ID: <1452d7d8-b694-42a9-93e6-c08f949c409a@tuunq.com> On 9/29/2017 11:34 AM, Dimitri Maziuk wrote: > (mailman 2.1.12 on centos 6.9) I don't think that Mark mentioned it, but 2.1.12 is -painfully- old (as is centos 6.9) and centos packages often lag way behind the corresponding source versions. If you need to stick with 6.9, I would consider ditching the centos package and installing the current mailman from source. Otherwise, can you move to a more modern Linux and more recent mailman? Later, z! From dmaziuk at bmrb.wisc.edu Mon Oct 2 10:47:18 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Mon, 2 Oct 2017 09:47:18 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <1452d7d8-b694-42a9-93e6-c08f949c409a@tuunq.com> References: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> <1452d7d8-b694-42a9-93e6-c08f949c409a@tuunq.com> Message-ID: On 2017-10-01 22:23, Carl Zwanzig wrote: > On 9/29/2017 11:34 AM, Dimitri Maziuk wrote: >> (mailman 2.1.12 on centos 6.9) > > I don't think that Mark mentioned it, but 2.1.12 is -painfully- old (as > is centos 6.9) and centos packages often lag way behind the > corresponding source versions. > > If you need to stick with 6.9, I would consider ditching the centos > package and installing the current mailman from source. Otherwise, can > you move to a more modern Linux and more recent mailman? Mailman's been been trouble- and maintenance-free for us since we switched from whatever-that-perl-thing-was-called all those years ago, and it never occurred to me it'd something as silly as abort the running op because my browser timed out. Now that I know, I'll consider "upgrading" -- to the university-run lyris: I might as well outsource the whole thing. Dima From dmaziuk at bmrb.wisc.edu Mon Oct 2 10:58:35 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Mon, 2 Oct 2017 09:58:35 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> Message-ID: On 2017-10-01 17:50, Mark Sapiro wrote: > On October 1, 2017 3:34:29 PM PDT, Dimitri Maziuk wrote: >> So I don't get it, are you saying that is *was* aborted after the CGI >> timed out? > > Yes. OK, thanks. Now I get to draft 3,500 apologies and then resubscribe everyone except the couple of people who replied with "please stop". PS there is nothing except postfix, spamd, mailman, and apache serving mailman's interface running on this server. It' running at load avg of 0.0 on 24 cores in 128GB of RAM. AFAICT the only reason for the software to croak on the list that size is its own coding. Dima From jhs at berklix.com Mon Oct 2 11:47:00 2017 From: jhs at berklix.com (Julian H. Stacey) Date: Mon, 02 Oct 2017 17:47:00 +0200 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: Your message "Mon, 02 Oct 2017 09:47:18 -0500." Message-ID: <201710021547.v92Fl0hu080795@fire.js.berklix.net> Dimitri Maziuk wrote: > On 2017-10-01 22:23, Carl Zwanzig wrote: > > On 9/29/2017 11:34 AM, Dimitri Maziuk wrote: > >> (mailman 2.1.12 on centos 6.9) > > > > I don't think that Mark mentioned it, but 2.1.12 is -painfully- old (as > > is centos 6.9) and centos packages often lag way behind the > > corresponding source versions. > > > > If you need to stick with 6.9, I would consider ditching the centos > > package and installing the current mailman from source. Otherwise, can > > you move to a more modern Linux and more recent mailman? > > Mailman's been been trouble- and maintenance-free for us since we > switched from whatever-that-perl-thing-was-called all those years ago, > and it never occurred to me it'd something as silly as abort the running > op because my browser timed out. Though the consequences of timeout were unfortunate, & Mailman would benefit from patching, I wouldn't blame Mailman, as you first wrote: > a few days ago I made a mistake (?) of uploading a list of ~7000 > addresses into the "bulk subscribe" box. Just delete the question mark: Yes it Was a silly mistake ! > Now that I know, I'll consider > "upgrading" -- to the university-run lyris: I might as well outsource > the whole thing. Better adopt the normal procedure for free source projects: Write & submit patches to project, to fix your problem. Cheers, Julian -- Julian H. Stacey, Computer Consultant, BSD Linux Unix Systems Engineer, Munich Reply below, Prefix '> '. Plain text, No .doc, base64, HTML, quoted-printable. http://berklix.eu/brexit/ UK stole 3,500,000 votes; 700,000 from Brits in EU. From mark at msapiro.net Mon Oct 2 14:10:06 2017 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 2 Oct 2017 11:10:06 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> Message-ID: <2f0df466-5d53-f755-ccb7-6ca7bef3bdf6@msapiro.net> On 10/02/2017 07:58 AM, Dimitri Maziuk wrote: > > OK, thanks. Now I get to draft 3,500 apologies and then resubscribe > everyone except the couple of people who replied with "please stop". If you use Mailman's mass subscribe (with smaller chunks, say 1000) you can add the apology to the new welcome message. > PS there is nothing except postfix, spamd, mailman, and apache serving > mailman's interface running on this server. It' running at load avg of > 0.0 on 24 cores in 128GB of RAM. AFAICT the only reason for the software > to croak on the list that size is its own coding. You need to look in Mailman's 'smtp' log to see how much time is being taken to deliver to postfix. One big killer in delivery from Mailman to Postfix is recipient address validation at smtpd time. I.e., you don't want reject_unknown_recipient_domain in smtpd_recipient_restrictions. Also, you don't need spamd scanning of Mailman's outbound mail as list posts will have been scanned on input. Whatever the reason, you won't be able to run a viable list if it takes days to deliver 3500 messages. Even an hour would be excessive. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From cpz at tuunq.com Mon Oct 2 14:16:33 2017 From: cpz at tuunq.com (Carl Zwanzig) Date: Mon, 2 Oct 2017 11:16:33 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> Message-ID: On 10/2/2017 7:58 AM, Dimitri Maziuk wrote: > PS there is nothing except postfix, spamd, mailman, and apache serving > mailman's interface running on this server. It' running at load avg of 0.0 > on 24 cores in 128GB of RAM. AFAICT the only reason for the software to > croak on the list that size is its own coding. While mailman routinely handles much larger lists (1), you seem to be very concerned about the size of a HTTP-based bulk add operation when there are other methods of adding that many recipients. Perhaps mailman does have a bug in the area, but there are many other potential factors that could slow things down or stop them completely (2). It also sounds like there are other issues with the system, like postfix delivery times and the length of the CGI timeout. If you're receiving "please stop" messages from users, maybe those users don't want to be on this list at all. This all seems like making a mountain out of a molehill. If you've been satisfied with mailman, why not continue to be satisfied in all other areas and just not to large bulk adds via http? (My car is a great car but won't carry 500kg of cargo, so I don't ask it to.) Later, z! (1) https://wiki.list.org/DOC/What%20is%20the%20largest%20list%20Mailman%20can%20run%3F) (2) is the mailman installed on local disk or an NFS share? what the locking scheme? timeouts? etc From mark at msapiro.net Mon Oct 2 15:19:13 2017 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 2 Oct 2017 12:19:13 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> <1452d7d8-b694-42a9-93e6-c08f949c409a@tuunq.com> Message-ID: <3ff1bc6b-0a4d-3cc4-b5ff-6806a70da526@msapiro.net> On 10/02/2017 07:47 AM, Dimitri Maziuk wrote: > > Mailman's been been trouble- and maintenance-free for us since we > switched from whatever-that-perl-thing-was-called all those years ago, > and it never occurred to me it'd something as silly as abort the running > op because my browser timed out. "all those years ago" and before, things like "real" databases, checkpointing, recovery and rollback, and adding 7000 users at once to a mailing list were not things we thought much about in a mailing list manager. The fact that Mailman has "been been trouble- and maintenance-free" for you for all those years is a tribute to the fact that we got at least most of it right. If you want a mailing list manager with a modern design, I'd recommend Mailman 3. And "abort the running op because my browser timed out" is not Mailman's doing. That's your web server and its CGI interface. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From dmaziuk at bmrb.wisc.edu Mon Oct 2 15:22:20 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Mon, 2 Oct 2017 14:22:20 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <2f0df466-5d53-f755-ccb7-6ca7bef3bdf6@msapiro.net> References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> <2f0df466-5d53-f755-ccb7-6ca7bef3bdf6@msapiro.net> Message-ID: <7637b1e8-3d02-702d-de19-c61c66339095@bmrb.wisc.edu> On 10/02/2017 01:10 PM, Mark Sapiro wrote: > You need to look in Mailman's 'smtp' log to see how much time is being > taken to deliver to postfix. Close to .1 s, with an occasional .12 on the date in question. > One big killer in delivery from Mailman to > Postfix is recipient address validation at smtpd time. I.e., you don't > want reject_unknown_recipient_domain in smtpd_recipient_restrictions. :) Well I do actually, this being our mail gateway, but I get the point. The more reason to outsource this to out university's IT instead of running it locally. > Also, you don't need spamd scanning of Mailman's outbound mail as list > posts will have been scanned on input. > > Whatever the reason, you won't be able to run a viable list if it takes > days to deliver 3500 messages. Even an hour would be excessive. Of course it's not scanning outbound mail, but it is scanning the incoming on delivery, using up cycles and i/o. /var/log/mailman/smtp is interesting, actually: > Sep 25 15:53:57 2017 (7782) smtp to XXX for 1 recips, completed in 0.081 seconds ... > Sep 25 15:59:06 2017 (7782) smtp to XXX for 1 recips, completed in 0.061 seconds > Sep 25 16:48:39 2017 (7782) smtp to XXX for 1 recips, completed in 0.065 seconds > Sep 25 19:03:16 2017 (7782) smtp to XXX for 1 recips, completed in 0.079 seconds > Sep 25 19:33:19 2017 (7782) smtp to XXX for 1 recips, completed in 0.069 seconds > Sep 26 01:14:46 2017 (7782) smtp to XXX for 1 recips, completed in 0.062 seconds > Sep 26 12:12:45 2017 (7782) smtp to XXX for 1 recips, completed in 0.060 seconds There's 3200 of these lines total, so at .1s/address it should've ran in 320 seconds. Instead it ran for 6 minutes, then hours later a message or two for a while, and nothing after Sep 26 12:12:45. /var/log/mailman/subscribe goes > Sep 25 15:51:56 2017 (29657) XXX: new aaa at ADDR, admin mass sub to > Sep 25 15:53:56 2017 (29657) XXX: new jjj at ADDR, admin mass sub and ends there. To my uneducated eye it looks like smtp transactions haven't even started until the subscription's gone about halfway through and presumably aborted. Anyway, this is getting academic. What I wanted to know was is it dead or is it still doing something behind the scenes, and I got the answer. Thanks everyone. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From dmaziuk at bmrb.wisc.edu Mon Oct 2 15:35:04 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Mon, 2 Oct 2017 14:35:04 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <3ff1bc6b-0a4d-3cc4-b5ff-6806a70da526@msapiro.net> References: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> <1452d7d8-b694-42a9-93e6-c08f949c409a@tuunq.com> <3ff1bc6b-0a4d-3cc4-b5ff-6806a70da526@msapiro.net> Message-ID: On 10/02/2017 02:19 PM, Mark Sapiro wrote: > And "abort the running op because my browser timed out" is not Mailman's > doing. That's your web server and its CGI interface. Oh, I agree: mailman worked exactly as designed. Whoever designed that particular assumed it'll take zero time to process an uploaded list of an unknown size, and that did precisely what ass-u-me always does. No surprises there, unfortunately. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From dmaziuk at bmrb.wisc.edu Mon Oct 2 15:37:15 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Mon, 2 Oct 2017 14:37:15 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> Message-ID: On 10/02/2017 01:16 PM, Carl Zwanzig wrote: > (My car is a great car > but won't carry 500kg of cargo, so I don't ask it to.) Well I bet your car comes with a manual that says what its max cargo capacity is. Could you point me at The Fine Manual for mailman where it says how many addresses I can subscribe through the web interface? -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Mon Oct 2 15:50:28 2017 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 2 Oct 2017 12:50:28 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <7637b1e8-3d02-702d-de19-c61c66339095@bmrb.wisc.edu> References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> <2f0df466-5d53-f755-ccb7-6ca7bef3bdf6@msapiro.net> <7637b1e8-3d02-702d-de19-c61c66339095@bmrb.wisc.edu> Message-ID: <8cc19b23-f790-ac2f-ab78-045e2850dcd7@msapiro.net> On 10/02/2017 12:22 PM, Dimitri Maziuk wrote: > >> One big killer in delivery from Mailman to >> Postfix is recipient address validation at smtpd time. I.e., you don't >> want reject_unknown_recipient_domain in smtpd_recipient_restrictions. > > :) Well I do actually, this being our mail gateway, but I get the point. > The more reason to outsource this to out university's IT instead of > running it locally. This is easily worked around. Define another "smtpd" service in master.cf on an alternate port with an override for smtpd_recipient_restrictions and use that port for Mailman delivery, but that may not be needed anyway. See below. > /var/log/mailman/smtp is interesting, actually: > >> Sep 25 15:53:57 2017 (7782) smtp to XXX for 1 recips, completed in 0.081 seconds > ... >> Sep 25 15:59:06 2017 (7782) smtp to XXX for 1 recips, completed in 0.061 seconds OK. So the bulk of the mail was delivered to Postfix in a bit over 5 minutes. That seem quite reasonable. I clearly misunderstood what you ment when you said mail was still going out after days. >> Sep 25 16:48:39 2017 (7782) smtp to XXX for 1 recips, completed in 0.065 seconds >> Sep 25 19:03:16 2017 (7782) smtp to XXX for 1 recips, completed in 0.079 seconds >> Sep 25 19:33:19 2017 (7782) smtp to XXX for 1 recips, completed in 0.069 seconds >> Sep 26 01:14:46 2017 (7782) smtp to XXX for 1 recips, completed in 0.062 seconds >> Sep 26 12:12:45 2017 (7782) smtp to XXX for 1 recips, completed in 0.060 seconds Some of these may be retries of temp fails from Postfix (anything in smtp-failure?) and some may be unrelated. > There's 3200 of these lines total, so at .1s/address it should've ran in > 320 seconds. Instead it ran for 6 minutes, Actually 5 minutes and 9 seconds or 309 seconds by the logs above. > then hours later a message or > two for a while, and nothing after Sep 26 12:12:45. > > /var/log/mailman/subscribe goes > >> Sep 25 15:51:56 2017 (29657) XXX: new aaa at ADDR, admin mass sub > > to > >> Sep 25 15:53:56 2017 (29657) XXX: new jjj at ADDR, admin mass sub > > and ends there. To my uneducated eye it looks like smtp transactions > haven't even started until the subscription's gone about halfway through > and presumably aborted. The welcome message is queued in Mailman's virgin queue and processed asynchronously by VirginRunner which moves it to Mailman's out queue where it is processes (again asynchronously by OutgoingRunner). > Anyway, this is getting academic. What I wanted to know was is it dead > or is it still doing something behind the scenes, and I got the answer. Yes, it is 'academic' but a deeper understanding of the processes involved can't hurt. I'm compulsive (often too much so) about answering every outstanding question, even if they are only my own questions. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From tlhackque at yahoo.com Mon Oct 2 15:54:40 2017 From: tlhackque at yahoo.com (tlhackque) Date: Mon, 2 Oct 2017 15:54:40 -0400 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> <1452d7d8-b694-42a9-93e6-c08f949c409a@tuunq.com> <3ff1bc6b-0a4d-3cc4-b5ff-6806a70da526@msapiro.net> Message-ID: <23c77dd3-f945-976a-1b48-7cf783101b03@yahoo.com> On 02-Oct-17 15:35, Dimitri Maziuk wrote: > Oh, I agree: mailman worked exactly as designed. Whoever designed that > particular assumed it'll take zero time to process an uploaded list of > an unknown size, and that did precisely what ass-u-me always does. No > surprises there, unfortunately. No, the requirement was that you'd set the timeout on your webserver to accommodate your environment. The Mailman folks can't know what that is.? As the site administrator, you are expected to. E.g. for Apache, see the TimeOut directive, which is a balance between preventing denial of service attacks, and letting long operations complete. But if you want an (apparently) perfect hands-off experience, by all means outsource to your IT department.? Then they'll be responsible for the system-level analysis, implementation, and support. If this is the case, I do not recommend Mailman V3.? Although it does have an improved design, it still requires administration.? And despite efforts by the developers, it is not yet as close to turnkey operation as Mailman V2.1.? I'm sure it will be - eventually.? But it has a long way to go.? Right now, it is more complex to setup, is missing features, has no translations, and has more bugs. From your comments on this thread, Mailman V3 will not (yet) meet your expectations. From cpz at tuunq.com Mon Oct 2 16:01:56 2017 From: cpz at tuunq.com (Carl Zwanzig) Date: Mon, 2 Oct 2017 13:01:56 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> Message-ID: On 10/2/2017 12:37 PM, Dimitri Maziuk wrote: > Could you point me at The Fine Manual for mailman where it > says how many addresses I can subscribe through the web interface? I can't (and as Mark mentioned, it's installation-dependent), but it would be helpful to others if you would add something to the wiki with your experiences and relevant config. z! From dmaziuk at bmrb.wisc.edu Mon Oct 2 16:35:31 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Mon, 2 Oct 2017 15:35:31 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <8cc19b23-f790-ac2f-ab78-045e2850dcd7@msapiro.net> References: <201709301647.v8UGldlY094713@fire.js.berklix.net> <527e8dcd-9357-5188-6f18-eb113e09e104@msapiro.net> <2f0df466-5d53-f755-ccb7-6ca7bef3bdf6@msapiro.net> <7637b1e8-3d02-702d-de19-c61c66339095@bmrb.wisc.edu> <8cc19b23-f790-ac2f-ab78-045e2850dcd7@msapiro.net> Message-ID: <0593f7dc-2a26-e77b-8794-ffbf75eea590@bmrb.wisc.edu> On 10/02/2017 02:50 PM, Mark Sapiro wrote: > On 10/02/2017 12:22 PM, Dimitri Maziuk wrote: ... >>> Sep 25 15:53:57 2017 (7782) smtp to XXX for 1 recips, completed in 0.081 seconds >> ... >>> Sep 25 15:59:06 2017 (7782) smtp to XXX for 1 recips, completed in 0.061 seconds > > OK. So the bulk of the mail was delivered to Postfix in a bit over 5 > minutes. That seem quite reasonable. I clearly misunderstood what you > ment when you said mail was still going out after days. Mea culpa, but that's why I had to ask here. Before you guys clued me in on which mailman logs meant what, my best bet was looking at postfix log. In there, however, you would see the messages deferred and being retried by postfix. There's message to at least one of these addresses still being retried (or was, back when I said "mail's still going out"), but I couldn't tell when it came out of mailman without much backtracking. > Some of these may be retries of temp fails from Postfix (anything in > smtp-failure?) and some may be unrelated. One failure logged in that time, to one of these addresses, and that's only an invalid address that got past the regexp I used to clean up the list. > Yes, it is 'academic' but a deeper understanding of the processes > involved can't hurt. I'm compulsive (often too much so) about answering > every outstanding question, even if they are only my own questions. :) I find I often don't wanna know. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From dmaziuk at bmrb.wisc.edu Mon Oct 2 16:52:47 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Mon, 2 Oct 2017 15:52:47 -0500 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <23c77dd3-f945-976a-1b48-7cf783101b03@yahoo.com> References: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> <1452d7d8-b694-42a9-93e6-c08f949c409a@tuunq.com> <3ff1bc6b-0a4d-3cc4-b5ff-6806a70da526@msapiro.net> <23c77dd3-f945-976a-1b48-7cf783101b03@yahoo.com> Message-ID: <01bed5d9-28bf-9cb6-60cb-8b5b3fbe5583@bmrb.wisc.edu> On 10/02/2017 02:54 PM, tlhackque via Mailman-Users wrote: ... > From your comments on this thread, Mailman V3 will not (yet) meet your > expectations. Mailman 3 doesn't seem to be available from my distro vendor. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Mon Oct 2 17:11:30 2017 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 2 Oct 2017 14:11:30 -0700 Subject: [Mailman-Users] bulk subscribe 7K users In-Reply-To: <01bed5d9-28bf-9cb6-60cb-8b5b3fbe5583@bmrb.wisc.edu> References: <3de25ccd-a494-bb1d-352b-34134cf48b19@bmrb.wisc.edu> <1452d7d8-b694-42a9-93e6-c08f949c409a@tuunq.com> <3ff1bc6b-0a4d-3cc4-b5ff-6806a70da526@msapiro.net> <23c77dd3-f945-976a-1b48-7cf783101b03@yahoo.com> <01bed5d9-28bf-9cb6-60cb-8b5b3fbe5583@bmrb.wisc.edu> Message-ID: <976258c7-399c-f870-1d9c-9fe6fda9ad09@msapiro.net> On 10/02/2017 01:52 PM, Dimitri Maziuk wrote: > On 10/02/2017 02:54 PM, tlhackque via Mailman-Users wrote: > ... >> From your comments on this thread, Mailman V3 will not (yet) meet your >> expectations. > > Mailman 3 doesn't seem to be available from my distro vendor. That will change. I know Debian is currently working on packaging Mailman 3, but have no info as to targets or timelines or other vendor's downstream packages. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From anon_777 at hotmail.com Mon Oct 2 20:37:04 2017 From: anon_777 at hotmail.com (Terry .) Date: Tue, 3 Oct 2017 00:37:04 +0000 Subject: [Mailman-Users] "Bounce action notification" emails for subscribes/unsubscribes In-Reply-To: <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net> References: <22975.33210.35652.332865@turnbull.sk.tsukuba.ac.jp>, <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net> Message-ID: Thanks for your replies Stephen & Mark, and sorry for the long delay in getting back to you. Re Stephen's comment: > I don't understand why you don't have this address in the first place. > Mailman uses this address as envelope sender (and sometimes From) in > order to accept failed delivery notifications (aka "bounces"), and so > automatically disable delivery from Mailman to mailboxes disabled on > the subscribed host (including non-existent addresses). This should > be configured in the MTA (mail server) along with all of the other > Mailman-specific addresses. I don?t understand why either. Maybe the address is supposed to exist in Exim, and not be visible via cPanel's interface (e.g. under "Forwarders"). But it makes me wonder whether everything will work as designed with Mailman under the current version of cPanel (11.66.0.23). If I create a new list (e.g. bugtest3 at mydomain.com, as I did today), the only forwarder that cPanel creates (and allows me to see) is: owner-bugtest3 at mydomain.com which forwards emails to bugtest3-owner at mydomain.com. If any emails actually get sent to mailman-bounces at mydomain.com, then I assume they'll end up in my catchall at mydomain.com mailbox, (whether I have the "Default Address" set to that catchall address, or I have a forwarder forwarding emails from mailman-bounces at mydomain.com to some mailbox), right??? I assume mailman isn't going to look in that catchall mailbox for anything, so do those emails still get processed because of what Mark said about Exim? I received a couple of "Bounce action notification" emails this week, due to list members' mailboxes being over quota, and they seemed to come through to me OK. They were sent from "MyListName [mailman-bounces at serverxyz.mywebhost.com] on behalf of mailman at somedomain.com" to mylistname-owner at mydomain.com. Is there anything else I can test to ensure all is working OK in regard to administrative addresses which mailman receives emails at? Re Mark's comment: > Yes. The bottom line here seems to be that things are now working as > they should be, so the issue seems to be solved. Well, before we break out the champagne... I'm not sure I'd call it "solved", Mark. I've got a couple of work-arounds (using a catch-all address or a forwarder) which I should not have had to perform, but this problem could be affecting thousands of people's lists (Jim Dory had one) which are managed via cPanel. I'm considering asking the webhost to ask cPanel to provide a proper fix, so cPanel users don't need to individually discover the cause/work-around for this problem, then remember to perform that work-around each time they create a list. Any comments on this, Mark or Stephen? Thanks again. Terry From mark at msapiro.net Mon Oct 2 22:06:01 2017 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 2 Oct 2017 19:06:01 -0700 Subject: [Mailman-Users] "Bounce action notification" emails for subscribes/unsubscribes In-Reply-To: References: <22975.33210.35652.332865@turnbull.sk.tsukuba.ac.jp> <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net> Message-ID: <0e455902-1586-e1f8-1f51-78f5c1a5ada4@msapiro.net> On 10/02/2017 05:37 PM, Terry . wrote: > > I don?t understand why either. Maybe the address is supposed to exist in Exim, and not be visible via cPanel's interface (e.g. under "Forwarders"). But it makes me wonder whether everything will work as designed with Mailman under the current version of cPanel (11.66.0.23). cPanel's Mailman is a kludge. Many things don't work as a non-cPanel Mailman user would expect. This is a result of cPanel's patches that allow lists of the same name in different domains to exist in a single Mailman installation. This in turn is necessary for them to be able to offer Mailman in more or less turnkey, multi-domain hosting environments which in the long run has been good for Mailman by allowing hosting services to offer Mailman to not highly sophisticated customers. Unfortunately it has also led to a number of situations where Mailman is available to a customer of a cPanel hosting service, whose admins have no interest in supporting Mailman. The most major change in cPanel is the list name. A list named mylist in the example.com domain is really named mylist_example.com and a list named mylist in the example.net domain is really named mylist_example.net. This leads to confusion and some minor things like Sibling lists either not working or requiring apparent list addresses like mylist_example.com at example.com. For the ordinary email case, the Exim Mailman router knows to deliver mail to mylist(-*)@example.com to the mylist_example.com list and mail to mylist(-*)@example.net to the mylist_example.net list. However, this all breaks down with 'mailman at example.com' because there is only one 'mailman' site list and its name is 'mailman', not 'mailman_example.com'. Thus, since this used to work, something has changed in cPanel's Exim config so that mail with envelope from mailman-bounces at example.com is no longer deliverable because mailman-bounces at example.com is not a valid address (mailman-bounces at the.canonical.host.domain might be). However, I'm confused because I know there are a couple of cPanel Mailman hosting services who's admins are on this list and are very conscientious, and they don't seem to see this issue. > If I create a new list (e.g. bugtest3 at mydomain.com, as I did today), the only forwarder that cPanel creates (and allows me to see) is: owner-bugtest3 at mydomain.com which forwards emails to bugtest3-owner at mydomain.com. This "forwarder" is a kludge to allow mail to owner-bugtest3 at mydomain.com to be delivered to the bugtest3 at mydomain.com owner because someone at cPanel thinks that that address should work that way even though Mailman itself only exposes bugtest3-owner at mydomain.com ad an owner address. As I said, normal mail delivery to list addresses is handled by cPanel's Mailman router, not by aliases or "forwarders" > If any emails actually get sent to mailman-bounces at mydomain.com, then I assume they'll end up in my catchall at mydomain.com mailbox, (whether I have the "Default Address" set to that catchall address, or I have a forwarder forwarding emails from mailman-bounces at mydomain.com to some mailbox), right??? I assume mailman isn't going to look in that catchall mailbox for anything, so do those emails still get processed because of what Mark said about Exim? I think all the above is correct. As far as Mailman processing those bounces is concerned, they are normally just forwarded to the site list owner. If they go to your catchall address instead, that's probably better. > I received a couple of "Bounce action notification" emails this week, due to list members' mailboxes being over quota, and they seemed to come through to me OK. They were sent from "MyListName [mailman-bounces at serverxyz.mywebhost.com] on behalf of mailman at somedomain.com" to mylistname-owner at mydomain.com. > > Is there anything else I can test to ensure all is working OK in regard to administrative addresses which mailman receives emails at? I'm sure the admin addresses for your lists work just fine. You can test by sending mail to them. -bounces is problematic, but -owner should go to the owner and -request, -confirm, -subscribe (-join), -unsubscribe (-leave) should all send some kind of reply. It's only 'mailman-bounces' (and other 'mailman' list addresses) that's problematic. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From gtaylor at tnetconsulting.net Tue Oct 3 02:24:24 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 3 Oct 2017 00:24:24 -0600 Subject: [Mailman-Users] DKIM / DMARC woes... In-Reply-To: <5e952e27-1596-8068-518e-77a3f6bbbb30@msapiro.net> References: <44c9172b-6987-c6b3-cbf3-407916280e5a@tnetconsulting.net> <5e952e27-1596-8068-518e-77a3f6bbbb30@msapiro.net> Message-ID: <2715df61-e35c-8efa-502e-3d58f9f55ddc@tnetconsulting.net> On 09/21/2017 03:23 PM, Mark Sapiro wrote: > The default behavior does nothing to DKIM related headers. This is from > Defaults.py Is the REMOVE_DKIM_HEADERS option a per mailing list setting? Or is it Mailman wide? I'm looking through the list admin interface for Mailman 2.1.20 and not finding it. -- Grant. . . . unix || die From mrbrklyn at panix.com Tue Oct 3 07:41:42 2017 From: mrbrklyn at panix.com (Ruben Safir) Date: Tue, 3 Oct 2017 07:41:42 -0400 Subject: [Mailman-Users] Django Message-ID: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> I'm not sure why you decided to add Django as a dependency of Mailman but it is a losey idea to add an additional entire operational development platform in order to just get a mailing list up and running. It is bad enough that the mailman python modules are constantly pinning the CPUs of my systems, but adding Django not only means having to completely rework my apache server configuration, which is strangely enough is not running for the sole purpose of being used as an adjunct to the mail list, but to force feeds an unwanted enormous security hole since it is a development platform which is exposed to the public. -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com From mrbrklyn at panix.com Tue Oct 3 07:41:06 2017 From: mrbrklyn at panix.com (Ruben Safir) Date: Tue, 3 Oct 2017 07:41:06 -0400 Subject: [Mailman-Users] 2.1 installation instructions Message-ID: I am porting my mailman config to a new system using openrc and, if it is possible, I would like to be pointed to installation and configuration instructions for mailman 2.1.24 and postfix thank you -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com From mrbrklyn at panix.com Tue Oct 3 07:56:56 2017 From: mrbrklyn at panix.com (Ruben Safir) Date: Tue, 3 Oct 2017 07:56:56 -0400 Subject: [Mailman-Users] 2.1 installation instructions Message-ID: <7dfff942-9be6-bf11-9b1f-635a63ac2372@panix.com> I am porting my mailman config to a new system using openrc and, if it is possible, I would like to be pointed to installation and configuration instructions for mailman 2.1.24 and postfix thank you -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com From barry at python.org Tue Oct 3 10:06:57 2017 From: barry at python.org (Barry Warsaw) Date: Tue, 3 Oct 2017 10:06:57 -0400 Subject: [Mailman-Users] Django In-Reply-To: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> References: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> Message-ID: <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> On Oct 3, 2017, at 07:41, Ruben Safir wrote: > > I'm not sure why you decided to add Django as a dependency of Mailman > but it is a losey idea to add an additional entire operational > development platform in order to just get a mailing list up and running. You must be talking about Mailman 3, where the web ui and archiver are implemented as Django applications. Please note that neither Postorius (web ui) nor HyperKitty (archiver) are *required* components so if you don?t like them, you don?t need to deploy them. Core is the only required piece and if you?re comfortable to write your own clients against Core?s REST API, you can do so using whatever technology you want, since it?s just HTTP+JSON. You can augment the functionality currently available via REST with shell access. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From phils at caerllewys.net Tue Oct 3 11:01:55 2017 From: phils at caerllewys.net (Phil Stracchino) Date: Tue, 3 Oct 2017 11:01:55 -0400 Subject: [Mailman-Users] Django In-Reply-To: <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> References: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> Message-ID: <07d7639f-b068-3bb3-cbf3-212c5530573e@caerllewys.net> On 10/03/17 10:06, Barry Warsaw wrote: > On Oct 3, 2017, at 07:41, Ruben Safir wrote: >> >> I'm not sure why you decided to add Django as a dependency of Mailman >> but it is a losey idea to add an additional entire operational >> development platform in order to just get a mailing list up and running. > > You must be talking about Mailman 3, where the web ui and archiver are implemented as Django applications. One thing to be said for Django is that at least it doesn't pull in PHP. -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 224 bytes Desc: OpenPGP digital signature URL: From dmaziuk at bmrb.wisc.edu Tue Oct 3 11:24:05 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Tue, 3 Oct 2017 10:24:05 -0500 Subject: [Mailman-Users] Django In-Reply-To: <07d7639f-b068-3bb3-cbf3-212c5530573e@caerllewys.net> References: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> <07d7639f-b068-3bb3-cbf3-212c5530573e@caerllewys.net> Message-ID: <9e37a8c2-7b92-d95e-9e05-7a8bbf94c21e@bmrb.wisc.edu> On 2017-10-03 10:01, Phil Stracchino wrote: > One thing to be said for Django is that at least it doesn't pull in PHP. You have to admit though, php scripts from 10 years ago still work. Dima From fmouse at fmp.com Tue Oct 3 11:37:48 2017 From: fmouse at fmp.com (Lindsay Haisley) Date: Tue, 03 Oct 2017 10:37:48 -0500 Subject: [Mailman-Users] Django In-Reply-To: <9e37a8c2-7b92-d95e-9e05-7a8bbf94c21e@bmrb.wisc.edu> References: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> <07d7639f-b068-3bb3-cbf3-212c5530573e@caerllewys.net> <9e37a8c2-7b92-d95e-9e05-7a8bbf94c21e@bmrb.wisc.edu> Message-ID: <1507045068.2743.12.camel@fmp.com> On Tue, 2017-10-03 at 10:24 -0500, Dimitri Maziuk wrote: > You have to admit though, php scripts from 10 years ago still work. Not always, unfortunately. PHP has implemented some show-stoppers which have required extensive editing of some of our customer scripts here. The deprecation of long array names ($HTTP_*_VARS) is an example. Yes, I can edit the php.ini file to make deprecated forms work, but the default behavior isn't always backward-compatible.? I hacked PHP support into Mailman some years ago for use in archive searches but fortunately my code was pretty simple. -- Lindsay Haisley | "The first casualty when FMP Computer Services | war comes is truth." 512-259-1190 | http://www.fmp.com | -- Hiram W Johnson From cpz at tuunq.com Tue Oct 3 11:48:53 2017 From: cpz at tuunq.com (Carl Zwanzig) Date: Tue, 3 Oct 2017 08:48:53 -0700 Subject: [Mailman-Users] 2.1 installation instructions In-Reply-To: References: Message-ID: On 10/3/2017 4:41 AM, Ruben Safir wrote: > I am porting my mailman config to a new system using openrc and, if it > is possible, I would like to be pointed to installation and > configuration instructions for mailman 2.1.24 and postfix A good place to start is the ./doc/ directory of mailman source distribution. There is also a fair bit of relevant material in the mailman wiki, which is linked in the footer of this list's messages. z! From weif at weif.net Tue Oct 3 11:56:34 2017 From: weif at weif.net (Keith Seyffarth) Date: Tue, 03 Oct 2017 09:56:34 -0600 Subject: [Mailman-Users] Django In-Reply-To: <9e37a8c2-7b92-d95e-9e05-7a8bbf94c21e@bmrb.wisc.edu> (message from Dimitri Maziuk on Tue, 3 Oct 2017 10:24:05 -0500) Message-ID: <841smkz0lp.fsf@maxwell.cjones.org> >> One thing to be said for Django is that at least it doesn't pull in PHP. > > You have to admit though, php scripts from 10 years ago still work. Well, not all of them... Some things have been deprecated, and some odd work-arounds for shortcomings no longer behave the way they did because bugs have been corrected or loopholes closed. -- ---- from my mac to yours... Keith Seyffarth mailto:weif at weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar ---- http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention From mark at msapiro.net Tue Oct 3 12:24:25 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 3 Oct 2017 09:24:25 -0700 Subject: [Mailman-Users] DKIM / DMARC woes... In-Reply-To: <2715df61-e35c-8efa-502e-3d58f9f55ddc@tnetconsulting.net> References: <44c9172b-6987-c6b3-cbf3-407916280e5a@tnetconsulting.net> <5e952e27-1596-8068-518e-77a3f6bbbb30@msapiro.net> <2715df61-e35c-8efa-502e-3d58f9f55ddc@tnetconsulting.net> Message-ID: <8619ba09-acbc-8e05-c5a4-0f9a661b0884@msapiro.net> On 10/02/2017 11:24 PM, Grant Taylor via Mailman-Users wrote: > > Is the REMOVE_DKIM_HEADERS option a per mailing list setting?? Or is it > Mailman wide? > > I'm looking through the list admin interface for Mailman 2.1.20 and not > finding it. It is not a list setting. It applies to the entire installation. It is documented in Mailman/Defaults.py and if you want to change the default, set it in Mailman/mm_cfg.py. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From gtaylor at tnetconsulting.net Tue Oct 3 12:50:02 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 3 Oct 2017 10:50:02 -0600 Subject: [Mailman-Users] DKIM / DMARC woes... In-Reply-To: <8619ba09-acbc-8e05-c5a4-0f9a661b0884@msapiro.net> References: <44c9172b-6987-c6b3-cbf3-407916280e5a@tnetconsulting.net> <5e952e27-1596-8068-518e-77a3f6bbbb30@msapiro.net> <2715df61-e35c-8efa-502e-3d58f9f55ddc@tnetconsulting.net> <8619ba09-acbc-8e05-c5a4-0f9a661b0884@msapiro.net> Message-ID: <06423fd8-3dce-cb62-856a-9b7eabecf666@tnetconsulting.net> On 10/03/2017 10:24 AM, Mark Sapiro wrote: > It is not a list setting. It applies to the entire installation. It is > documented in Mailman/Defaults.py and if you want to change the default, > set it in Mailman/mm_cfg.py. Thank you Mark. Sorry if I'm asking obvious questions. I've not admined Mailman in quite a while and I'm trying to help someone else admin Mailman remotely without access. :-/ -- Grant. . . . unix || die From mark at msapiro.net Tue Oct 3 12:56:51 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 3 Oct 2017 09:56:51 -0700 Subject: [Mailman-Users] 2.1 installation instructions In-Reply-To: <7dfff942-9be6-bf11-9b1f-635a63ac2372@panix.com> References: <7dfff942-9be6-bf11-9b1f-635a63ac2372@panix.com> Message-ID: <03b37d09-90dc-be20-7e00-405eee14f30d@msapiro.net> On 10/03/2017 04:56 AM, Ruben Safir wrote: > I am porting my mailman config to a new system using openrc and, if it > is possible, I would like to be pointed to installation and > configuration instructions for mailman 2.1.24 and postfix The installation manual is at You will also find this in the doc/ directory in the tarball. There is also an INSTALL file in the tarball which points to these docs. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From dmaziuk at bmrb.wisc.edu Tue Oct 3 13:02:42 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Tue, 3 Oct 2017 12:02:42 -0500 Subject: [Mailman-Users] Django In-Reply-To: <841smkz0lp.fsf@maxwell.cjones.org> References: <841smkz0lp.fsf@maxwell.cjones.org> Message-ID: <3a38afea-5dda-cc74-428a-a8b182508085@bmrb.wisc.edu> On 10/03/2017 10:56 AM, Keith Seyffarth wrote: >>> One thing to be said for Django is that at least it doesn't pull in PHP. >> >> You have to admit though, php scripts from 10 years ago still work. > > Well, not all of them... Some things have been deprecated, and some odd > work-arounds for shortcomings no longer behave the way they did because > bugs have been corrected or loopholes closed. Oh, come on. It was just a snark. RedHat in particular is notorious for patching their RPMs without bumping major version numbers. That's how they provide their "stable API platform". As a result your PHP may report it's at version X, but the actual bugfixes and security features are from version Z.a.svn13378337. Go figure out which fine manual you need to read to figure out what incompatible improvements you couldn't live without all these years your PHP installation has today. Film at 11. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From fmouse at fmp.com Tue Oct 3 13:26:45 2017 From: fmouse at fmp.com (Lindsay Haisley) Date: Tue, 03 Oct 2017 12:26:45 -0500 Subject: [Mailman-Users] Django In-Reply-To: <3a38afea-5dda-cc74-428a-a8b182508085@bmrb.wisc.edu> References: <841smkz0lp.fsf@maxwell.cjones.org> <3a38afea-5dda-cc74-428a-a8b182508085@bmrb.wisc.edu> Message-ID: <1507051605.2743.26.camel@fmp.com> On Tue, 2017-10-03 at 12:02 -0500, Dimitri Maziuk wrote: > Oh, come on. It was just a snark. Too early in the AM here for snark. My apologies! -- Lindsay Haisley | "The first casualty when FMP Computer Services | war comes is truth." 512-259-1190 | http://www.fmp.com | -- Hiram W Johnson From jesus at evangelizacion.org.mx Wed Oct 4 13:38:23 2017 From: jesus at evangelizacion.org.mx (Jesus Rivas) Date: Wed, 4 Oct 2017 12:38:23 -0500 Subject: [Mailman-Users] minimum recommended hardware In-Reply-To: <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> References: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> Message-ID: <933342A3-76BD-4906-B6AF-1B2C6C107291@evangelizacion.org.mx> Hi, What is the minimum recommended hardware for running and operate Mailman? If i rent a cloud service with this features: 8 Core Processor Memory: 32 GB 60 GB SSD Operating System: CentOS7 I have a mailing list of 82000, and we send daily email. With this i could operate or i need more space in the hard drive? Thanks Saludos Cualquier duda estoy a tus ordenes. Dios te bendiga. Jes?s Rivas Ayudante de Sistemas T. (01 81) 8123-1293 01 800 836 9407 website | vCard | facebook | twitter From mark at msapiro.net Wed Oct 4 21:34:15 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 4 Oct 2017 18:34:15 -0700 Subject: [Mailman-Users] minimum recommended hardware In-Reply-To: <933342A3-76BD-4906-B6AF-1B2C6C107291@evangelizacion.org.mx> References: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> <933342A3-76BD-4906-B6AF-1B2C6C107291@evangelizacion.org.mx> Message-ID: <37bb6358-5379-a1f3-793b-6909fb3c237a@msapiro.net> On 10/04/2017 10:38 AM, Jesus Rivas wrote: > What is the minimum recommended hardware for running and operate Mailman? A Raspberry Pi (only slightly facetious ;) > If i rent a cloud service with this features: > 8 Core Processor > Memory: 32 GB > 60 GB SSD > Operating System: CentOS7 > > I have a mailing list of 82000, and we send daily email. > > With this i could operate or i need more space in the hard drive? I'm assuming you're talking about Mailman 2.1 and not Mailman 3. My first comment is the RHEL/Centos Mailman 2.1 is something between 2.1.15-24.el7 and 2.1.15-24.el7 depending on which specific Centos 7 release you will have. The current Mailman release is 2.1.24 and 2.1.25 will be released probably later this month. I.e., the Centos 7 package is 9 or 10 releases old. It's not as bad as it seems because some things from later releases such as DMARC mitigations have been backported as "bug fixes", but still, I would recommend installing the latest version from source. As to your actual question, I think the configuration you describe is capable of supporting a list with 82,000 members, but see . Your actual issue will not be whether you can support a list of that size, but rather whether your MTA and whatever emailing limits might be placed on you by the cloud host can support 82,000 emails per day (or more realistically in the fraction of an hour that you expect them to be sent). You definitely need to discuss this requirement with any potential cloud hosting service. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Wed Oct 4 22:59:50 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 4 Oct 2017 19:59:50 -0700 Subject: [Mailman-Users] minimum recommended hardware In-Reply-To: <37bb6358-5379-a1f3-793b-6909fb3c237a@msapiro.net> References: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> <933342A3-76BD-4906-B6AF-1B2C6C107291@evangelizacion.org.mx> <37bb6358-5379-a1f3-793b-6909fb3c237a@msapiro.net> Message-ID: On 10/04/2017 06:34 PM, Mark Sapiro wrote: > > My > first comment is the RHEL/Centos Mailman 2.1 is something between > 2.1.15-24.el7 and 2.1.15-24.el7 ... Ooops. That should be "something between 2.1.15-17.el7 and 2.1.15-24.el7" -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From jesus at evangelizacion.org.mx Thu Oct 5 11:23:21 2017 From: jesus at evangelizacion.org.mx (Jesus Rivas) Date: Thu, 5 Oct 2017 10:23:21 -0500 Subject: [Mailman-Users] minimum recommended hardware In-Reply-To: <37bb6358-5379-a1f3-793b-6909fb3c237a@msapiro.net> References: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> <933342A3-76BD-4906-B6AF-1B2C6C107291@evangelizacion.org.mx> <37bb6358-5379-a1f3-793b-6909fb3c237a@msapiro.net> Message-ID: <9DE8F6FA-79F2-45DF-BC6E-A4DE12794293@evangelizacion.org.mx> Mark, thanks for your answer. I'm clear that cpu, ram etc... its ok. Actually i have 10mbps and mailman/postfix delivered in 1-2 hour. its greats for us. But i need to move all my servers to the cloud, so, i?m checking for service cloud, and PerfectIP offer 30mpbs (cheap server :D ). My specific question is: with this hard disk (55Gb free space on Centos 7 without graphics desktop), can i manage 100k mailing list. My main concern is the limited free space. Saludos Cualquier duda estoy a tus ordenes. Dios te bendiga. Jes?s Rivas Ayudante de Sistemas T. (01 81) 8123-1293 01 800 836 9407 website | vCard | facebook | twitter > On Oct 4, 2017, at 8:34 PM, Mark Sapiro wrote: > > On 10/04/2017 10:38 AM, Jesus Rivas wrote: > >> What is the minimum recommended hardware for running and operate Mailman? > > > A Raspberry Pi > > (only slightly facetious ;) > > >> If i rent a cloud service with this features: >> 8 Core Processor >> Memory: 32 GB >> 60 GB SSD >> Operating System: CentOS7 >> >> I have a mailing list of 82000, and we send daily email. >> >> With this i could operate or i need more space in the hard drive? > > > I'm assuming you're talking about Mailman 2.1 and not Mailman 3. My > first comment is the RHEL/Centos Mailman 2.1 is something between > 2.1.15-24.el7 and 2.1.15-24.el7 depending on which specific Centos 7 > release you will have. > > The current Mailman release is 2.1.24 and 2.1.25 will be released > probably later this month. I.e., the Centos 7 package is 9 or 10 > releases old. It's not as bad as it seems because some things from later > releases such as DMARC mitigations have been backported as "bug fixes", > but still, I would recommend installing the latest version from source. > > As to your actual question, I think the configuration you describe is > capable of supporting a list with 82,000 members, but see > . > > Your actual issue will not be whether you can support a list of that > size, but rather whether your MTA and whatever emailing limits might be > placed on you by the cloud host can support 82,000 emails per day (or > more realistically in the fraction of an hour that you expect them to be > sent). > > You definitely need to discuss this requirement with any potential cloud > hosting service. > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > ------------------------------------------------------ > Mailman-Users mailing list Mailman-Users at python.org > https://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://wiki.list.org/x/AgA3 > Security Policy: http://wiki.list.org/x/QIA9 > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-users/jesus%40evangelizacion.org.mx From Jung.Jena at gmx.de Thu Oct 5 05:24:18 2017 From: Jung.Jena at gmx.de (Sebastian Jung) Date: Thu, 5 Oct 2017 11:24:18 +0200 Subject: [Mailman-Users] Filtering of unwanted Spam-Emails Message-ID: Hi all, I administrate a Mailinglist where by default only members of the list are allowed to post messages. Lately we have Spam-Emails where the creator uses a "From"-Adress in the form of: regularListMember at somedomain.com Mailman does not block those Emails since the known and allowed Email-adress appears with in the From-Field although it is just part of the name tag. Do you know, if there is some option to deal with the problem or to set a regular-expression to filter out such unwanted mails? Thanks in advance Sebastian From m.gilliatt at live.co.uk Wed Oct 4 03:56:38 2017 From: m.gilliatt at live.co.uk (Marc Gilliatt) Date: Wed, 4 Oct 2017 07:56:38 +0000 Subject: [Mailman-Users] How to change the administrators account email? Message-ID: Hi, I?m soon to take over Mailman from a colleague who is leaving at the end of the month, I'm brand new to Mailman and I have to learn it fast. My colleague who is leaving, set up and configured Mailman, he used his work email for the administrator accounts email when he first set Mailman up. We would like to change this to my work email address. I?ve looked online, and I can?t find a way to change/reset the administrator account email. Is it possible to do this? Looking forward to hearing from you. Thanks, Marc Gilliatt From mark at msapiro.net Thu Oct 5 12:53:30 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 5 Oct 2017 09:53:30 -0700 Subject: [Mailman-Users] minimum recommended hardware In-Reply-To: <9DE8F6FA-79F2-45DF-BC6E-A4DE12794293@evangelizacion.org.mx> References: <4b0075e5-76d4-5b21-f1f9-fd8750a3f399@panix.com> <4CC3ED95-0F8B-4020-B70C-FE43A7A0F9AD@python.org> <933342A3-76BD-4906-B6AF-1B2C6C107291@evangelizacion.org.mx> <37bb6358-5379-a1f3-793b-6909fb3c237a@msapiro.net> <9DE8F6FA-79F2-45DF-BC6E-A4DE12794293@evangelizacion.org.mx> Message-ID: <36a4688a-ca9d-e353-1db3-9d9a4efa9cfc@msapiro.net> On 10/05/2017 08:23 AM, Jesus Rivas wrote: > My specific question is: with this hard disk (55Gb free space on Centos 7 without graphics desktop), can i manage 100k mailing list. > > My main concern is the limited free space. I think it should be enough as long as the list posts are not too large. I.e., if you were to send a 1 MB message to 100K users with VERP or personalization, that's 100 GB worth of messages and you might have a problem, but if the message is only 10 KB or if delivery is neither VERPed nor personalized so there is only one message copy per tens or more recipients, I don't think it would be an issue. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From heller at deepsoft.com Thu Oct 5 13:47:48 2017 From: heller at deepsoft.com (Robert Heller) Date: Thu, 5 Oct 2017 13:47:48 -0400 (EDT) Subject: [Mailman-Users] Filtering of unwanted Spam-Emails In-Reply-To: References: Message-ID: <20171005174749.A91DE73211C@sharky3.deepsoft.com> Most often these spammers are sending from Internet Cafes or from infected home PCs. This generally means that the originating IP *does not have a reverse DNS entry*. This means that the inbound MTA (or some inbound MTA) is going to add a Received: header with 'unknown' as the host it is receiving from. Putting in a spam filter like this: Received: from.*(unknown \[\d+\.\d+\.\d+\.\d+\]) with Hold action will catch these. (note: *some* E-Mail clients will also do this, so sometimes you will get a legit post from an 'unknown' SMTP server. Using "hold" allows you to pass those along.) Also: If you can install something like Spamassassin+Mimedefang and setting it to include spam scores, you can also have a spam filter for that. Also you can look at the full headers and look at the Received: headers. Sometimes the anon. IP address do have a reverse DNS entry (eg something like nnn-nnn-nnn-nnn-dsl-home-network.telecom.ru or some such nonsense -- something other than a more typical outboundmail.someprovider.com). In which case you can craft a spam filter for those as well. At Thu, 5 Oct 2017 11:24:18 +0200 "Sebastian Jung" wrote: > > Hi all, > > I administrate a Mailinglist where by default only members of the list are > allowed to post messages. Lately we have Spam-Emails where the creator > uses a "From"-Adress in the form of: > > regularListMember at somedomain.com > > Mailman does not block those Emails since the known and allowed > Email-adress appears with in the From-Field although it is just part of > the name tag. > Do you know, if there is some option to deal with the problem or to set a > regular-expression to filter out such unwanted mails? > > Thanks in advance > Sebastian > ------------------------------------------------------ > Mailman-Users mailing list Mailman-Users at python.org > https://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://wiki.list.org/x/AgA3 > Security Policy: http://wiki.list.org/x/QIA9 > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-users/heller%40deepsoft.com > > -- Robert Heller -- 978-544-6933 Deepwoods Software -- Custom Software Services http://www.deepsoft.com/ -- Linux Administration Services heller at deepsoft.com -- Webhosting Services From mark at msapiro.net Thu Oct 5 18:26:18 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 5 Oct 2017 15:26:18 -0700 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: References: Message-ID: On 10/04/2017 12:56 AM, Marc Gilliatt wrote: > > My colleague who is leaving, set up and configured Mailman, he used his work email for the administrator accounts email when he first set Mailman up. We would like to change this to my work email address. > > I?ve looked online, and I can?t find a way to change/reset the administrator account email. > > Is it possible to do this? If you are talking about the admin address for the list, go to the web admin UI for the list and on the General Options page, you will see a setting for 'owner'. Also see . -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Thu Oct 5 18:42:16 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 5 Oct 2017 15:42:16 -0700 Subject: [Mailman-Users] Filtering of unwanted Spam-Emails In-Reply-To: References: Message-ID: <3e077c22-e809-4c92-3049-3b7495d028ff@msapiro.net> On 10/05/2017 02:24 AM, Sebastian Jung wrote: > Hi all, > > I administrate a Mailinglist where by default only members of the list are > allowed to post messages. Lately we have Spam-Emails where the creator > uses a "From"-Adress in the form of: > > regularListMember at somedomain.com > > Mailman does not block those Emails since the known and allowed > Email-adress appears with in the From-Field although it is just part of > the name tag. That is not the reason why Mailman is allowing this post as a member post. Mailman understands the difference between a display name and an email address in a From: header. Mailman's membership checks look at more than just From:. By default, Mailman looks at the From: header, the envelope sender and the Reply-To: and Sender: headers. If any of those which exists contains a list member address, the post is considered to be from the member. You can reduce that list installation wide by putting a setting for SENDER_HEADERS in Mailman/mm_cfg.py - see the documentation in Mailman/Defaults.py. For example, putting SENDER_HEADERS = ('from',) in mm_cfg.py will mean only the From: header is checked for list membership. Note also that you won't see the original envelope sender or Sender: header in the delivered post or archives nor will you see the original Reply-To: if the list is configured to remove it, but the original envelope sender and Sender: if any will be in the archives/private/LISTNAME.mbox/LISTNAME.mbox file. and the envelope sender will probably be in MTA logs. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Fri Oct 6 13:00:08 2017 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 6 Oct 2017 10:00:08 -0700 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: References: Message-ID: <63869cd7-7265-263e-ead9-a4f7ecb5f3e4@msapiro.net> On 10/06/2017 06:14 AM, Marc Gilliatt wrote: > As mentioned?before, I'm brand new to Mailman. I have to learn it > quick.? The original?user who set up Mailman has several lists', I'm not > too sure on where to change the setting for the owner. I can't seem to > see the general options page.? It's at a URL something like http://example.com/mailman/admin/LISTNAME. If by chance this is a cPanel host, the listname is probably like LISTNAME_example.com, and of course example.com is your actual domain. Also note that I'm replying to the message sent directly to me. The list copy is greylisted and your retries have not yet used the same IP twice. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From m.gilliatt at live.co.uk Fri Oct 6 09:14:48 2017 From: m.gilliatt at live.co.uk (Marc Gilliatt) Date: Fri, 6 Oct 2017 13:14:48 +0000 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: References: , Message-ID: As mentioned before, I'm brand new to Mailman. I have to learn it quick. The original user who set up Mailman has several lists', I'm not too sure on where to change the setting for the owner. I can't seem to see the general options page. ________________________________ From: Mailman-Users on behalf of Mark Sapiro Sent: 05 October 2017 23:26 To: mailman-users at python.org Subject: Re: [Mailman-Users] How to change the administrators account email? On 10/04/2017 12:56 AM, Marc Gilliatt wrote: > > My colleague who is leaving, set up and configured Mailman, he used his work email for the administrator accounts email when he first set Mailman up. We would like to change this to my work email address. > > I?ve looked online, and I can?t find a way to change/reset the administrator account email. > > Is it possible to do this? If you are talking about the admin address for the list, go to the web admin UI for the list and on the General Options page, you will see a setting for 'owner'. Also see . -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan ------------------------------------------------------ Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/m.gilliatt%40live.co.uk From weif at weif.net Fri Oct 6 15:11:11 2017 From: weif at weif.net (Keith Seyffarth) Date: Fri, 06 Oct 2017 13:11:11 -0600 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: (message from Marc Gilliatt on Fri, 6 Oct 2017 13:14:48 +0000) Message-ID: <84fuawrt0w.fsf@maxwell.cjones.org> Marc Gilliatt writes: > As mentioned before, I'm brand new to Mailman. I have to learn it > quick. The original user who set up Mailman has several lists', I'm > not too sure on where to change the setting for the owner. I can't > seem to see the general options page. Marc, Are you getting logged in to the "[listname] administrative interface?" This link should be at the bottom of the mailman list pages (i.e. the subscribe page) for any given list. when you get logged in here, you should be on the General Options page people are talking about. On my lists, the field you are looking for is the second question, but customizations or changes in configuration could change the order or number of questions. The field is labeled: "The list administrator email addresses. Multiple administrator addresses, each on separate line is okay. (Details for owner)" But this could also possibly have been customized... Keith -- ---- from my mac to yours... Keith Seyffarth mailto:weif at weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar ---- http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention From fmouse at fmp.com Fri Oct 6 16:10:32 2017 From: fmouse at fmp.com (Lindsay Haisley) Date: Fri, 06 Oct 2017 15:10:32 -0500 Subject: [Mailman-Users] ZIP files in list-owner emails Message-ID: <1507320632.17785.16.camel@fmp.com> While our lists here are configured to reject ZIP file viral spam from subscribers or contributors, the same isn't true of the list-owner addresses, many of which redirect to Gmail accounts, and I get at least two or three Gmail notices a day informing me of the Google policy on this. Since they're touchy about stuff like this I'd like to reject these at the front door before they get re-mailed to list-owners. Is there a good way within Mailman to filter list-owner email for unwanted attachments? -- Lindsay Haisley | "The first casualty when FMP Computer Services | war comes is truth." 512-259-1190 | http://www.fmp.com | -- Hiram W Johnson From mark at msapiro.net Fri Oct 6 16:41:58 2017 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 6 Oct 2017 13:41:58 -0700 Subject: [Mailman-Users] ZIP files in list-owner emails In-Reply-To: <1507320632.17785.16.camel@fmp.com> References: <1507320632.17785.16.camel@fmp.com> Message-ID: On 10/06/2017 01:10 PM, Lindsay Haisley wrote: > > Is there a good way within Mailman to filter list-owner email for > unwanted attachments? You can put OWNER_PIPELINE.insert(OWNER_PIPELINE.index('SpamDetect')+1, 'MimeDel') in mm_cfg.py. This will apply each list's content filtering settings to -owner messages. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mrbrklyn at panix.com Fri Oct 6 16:59:01 2017 From: mrbrklyn at panix.com (Ruben Safir) Date: Fri, 6 Oct 2017 16:59:01 -0400 Subject: [Mailman-Users] openrc init.d script Message-ID: <04939d2e-c0ca-dfb1-6585-dfc7e41c5540@panix.com> I wrote an init.d script for mailman for openrc and I have bug I can't track down in mailmanctl when I run the script it feeds me this: www3 ~]# /etc/init.d/mailman start Go find the config Yo - go find /usr/lib/mailman/Mailman/mm_cfg.py * Starting mailman ... PIDFILE /var/lib/mailman/data/master-qrunner.pid PIDFILE /usr/lib/mailman/bin/mailmanctl * start-stop-daemon: fopen `/var/lib/mailman/data/master-qrunner.pid': No such file or directory * Detaching to start `/usr/lib/mailman/bin/mailmanctl' ... Starting Mailman's master qrunner. [ ok ] [www3 ~]# Traceback (most recent call last): File "/usr/lib/mailman/bin/mailmanctl", line 556, in main() File "/usr/lib/mailman/bin/mailmanctl", line 412, in main fp = open(mm_cfg.PIDFILE, 'w') IOError: [Errno 13] Permission denied: '/var/lib/mailman/data/master-qrunner.pid' the file permissions look right: [www3 ~]# ls -al /var/lib/mailman/ total 48 drwxrwsr-x 10 mailman mailman 4096 Oct 25 2015 . drwxr-xr-x 28 root root 4096 Oct 2 16:03 .. drwxrwsr-x 4 root mailman 4096 Oct 8 2014 archives -rw------- 1 mailman mailman 27 Oct 25 2015 .bash_history drwxrwsr-x 2 root mailman 4096 Oct 6 16:28 data drwxrwsr-x 5 root mailman 4096 Apr 2 2017 lists drwxrwsr-x 3 mailman mailman 4096 Oct 20 2015 .local drwxrwsr-x 2 root mailman 4096 Oct 6 16:30 locks drwxrwsr-x 2 root mailman 4096 Mar 5 2016 logs drwxrwsr-x 12 root mailman 4096 Jan 1 2016 qfiles drwxrwsr-x 2 root mailman 4096 Oct 8 2014 spam -rw------- 1 mailman mailman 601 Oct 25 2015 .viminfo [www3 ~]# ls -al /var/lib/mailman/data/ total 92 drwxrwsr-x 2 root mailman 4096 Oct 6 16:28 . drwxrwsr-x 10 mailman mailman 4096 Oct 25 2015 .. -rw-r----- 1 root mailman 41 Jan 1 2016 adm.pw -rw-rw---- 1 mailman mailman 2642 Apr 15 12:47 aliases -rw-r----- 1 mailman mailman 12288 Apr 15 12:52 aliases.db -rw-rw---- 1 mailman mailman 34742 Mar 16 2017 bounce-events-09990.pck -rw-rw-r-- 1 mailman mailman 2390 Jun 11 03:13 heldmsg-hangout-122.pck -rw-r--r-- 1 root mailman 10 Jan 1 2016 last_mailman_version -rw-r--r-- 1 root mailman 5 Oct 6 16:28 master-qrunner.pid -rw-r--r-- 1 root mailman 14100 Oct 8 2014 sitelist.cfg And what really puzzles me is the error line on the trace /usr/lib/mailman/bin/mailmanctl omask = os.umask(6) try: fp = open(mm_cfg.PIDFILE, 'w') print >> fp, os.getpid() fp.close() finally: os.umask(omask) That file, mm_cfg.PIDFILE doesn't exist anywhere on the system. When you run it from the command line, it works without complaining. I think there is a problem with the permissions being created for the pid file. I would really prefer the pid file to be put in /var/run -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www2.mrbrklyn.com/resources - Unpublished Archive http://www.coinhangout.com - coins! http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013 From mark at msapiro.net Fri Oct 6 18:07:14 2017 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 6 Oct 2017 15:07:14 -0700 Subject: [Mailman-Users] openrc init.d script In-Reply-To: <04939d2e-c0ca-dfb1-6585-dfc7e41c5540@panix.com> References: <04939d2e-c0ca-dfb1-6585-dfc7e41c5540@panix.com> Message-ID: <9887ab6e-9b3e-cb39-2052-eb3c8a657322@msapiro.net> On 10/06/2017 01:59 PM, Ruben Safir wrote: > > And what really puzzles me is the error line on the trace > > /usr/lib/mailman/bin/mailmanctl > omask = os.umask(6) > try: > fp = open(mm_cfg.PIDFILE, 'w') > print >> fp, os.getpid() > fp.close() > finally: > os.umask(omask) > > That file, mm_cfg.PIDFILE doesn't exist anywhere on the system. mm_cfg.PIDFILE is the setting of PIDFILE in Mailman/Defaults.py or as overriden in Mailman/mm_cfg.py. The Default setting from Mailman/Defaults.py is PIDFILE = os.path.join(DATA_DIR, 'master-qrunner.pid') and DATA_DIR is DATA_DIR = os.path.join(VAR_PREFIX, 'data') and VAR_PREFIX is set by configure, and is also defined in Defaults.py and in your case is VAR_PREFIX = '/var/lib/mailman/' > When you run it from the command line, it works without complaining. > > I think there is a problem with the permissions being created for the pid file. The permissions look OK. Could this be a SELinux or apparmor issue? > I would really prefer the pid file to be put in /var/run You can always put PIDFILE = '/var/run/mailman.pid' or whatever you want in mm_cfg.py. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From m.gilliatt at live.co.uk Mon Oct 9 04:03:23 2017 From: m.gilliatt at live.co.uk (Marc Gilliatt) Date: Mon, 9 Oct 2017 08:03:23 +0000 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: <84fuawrt0w.fsf@maxwell.cjones.org> References: (message from Marc Gilliatt on Fri, 6 Oct 2017 13:14:48 +0000),<84fuawrt0w.fsf@maxwell.cjones.org> Message-ID: I do see the subscribe icon - do I have to subscribe first to a list to then get to the general options page? Is this for all lists? ________________________________ From: Keith Seyffarth Sent: 06 October 2017 20:11 To: Marc Gilliatt Cc: mark at msapiro.net; mailman-users at python.org Subject: Re: [Mailman-Users] How to change the administrators account email? Marc Gilliatt writes: > As mentioned before, I'm brand new to Mailman. I have to learn it > quick. The original user who set up Mailman has several lists', I'm > not too sure on where to change the setting for the owner. I can't > seem to see the general options page. Marc, Are you getting logged in to the "[listname] administrative interface?" This link should be at the bottom of the mailman list pages (i.e. the subscribe page) for any given list. when you get logged in here, you should be on the General Options page people are talking about. On my lists, the field you are looking for is the second question, but customizations or changes in configuration could change the order or number of questions. The field is labeled: "The list administrator email addresses. Multiple administrator addresses, each on separate line is okay. (Details for owner)" But this could also possibly have been customized... Keith -- ---- from my mac to yours... Keith Seyffarth mailto:weif at weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar ---- http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention From mark at msapiro.net Mon Oct 9 08:53:53 2017 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 9 Oct 2017 05:53:53 -0700 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: References: <84fuawrt0w.fsf@maxwell.cjones.org> Message-ID: On 10/09/2017 01:03 AM, Marc Gilliatt wrote: > I do see the subscribe icon - do I have to subscribe first to a list to then get to the general options page? Is this for all lists? We have no idea what you are seeing. If you can post the actual URLs that you are going to, we might be able to be more helpful. Where do you see this "subscribe" icon? If you click it, does it take you to a page that looks something like the one at ? If so, where you ultimately want to go is to the link at the bottom of that page that looks like LISTNAME administrative interface (requires authorization) Once there, you need to authenticate with either the list's admin password or the site password if there is one. See . Perhaps you should be talking to your "colleague who is leaving at the end of the month". Also note that what we are talking about here is the 'owner' address for a particular list. Standard GNU Mailman has no concept of a sitewide admin email address other than perhaps the 'owner' of the 'mailman' site list. If what you are dealing with is some non-standard Mailman, e.g., Mailman on a cPanel host, there may be others on this list who can help, but I can't. But you need to tell us exactly what you are talking about. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From m.gilliatt at live.co.uk Mon Oct 9 09:20:39 2017 From: m.gilliatt at live.co.uk (Marc Gilliatt) Date: Mon, 9 Oct 2017 13:20:39 +0000 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: References: <84fuawrt0w.fsf@maxwell.cjones.org> , Message-ID: I appreciate your help. I've spoken to my colleague, he is not sure on how to change the email either, which helps a lot I suppose. What he has told me, is that when he first set up and configured Mailman, he used his work email instead of IT's. I'm not too sure if this has shed some light on the issue I'm having? ________________________________ From: Mailman-Users on behalf of Mark Sapiro Sent: 09 October 2017 13:53 To: mailman-users at python.org Subject: Re: [Mailman-Users] How to change the administrators account email? On 10/09/2017 01:03 AM, Marc Gilliatt wrote: > I do see the subscribe icon - do I have to subscribe first to a list to then get to the general options page? Is this for all lists? We have no idea what you are seeing. If you can post the actual URLs that you are going to, we might be able to be more helpful. Where do you see this "subscribe" icon? If you click it, does it take you to a page that looks something like the one at ? If so, where you ultimately want to go is to the link at the bottom of that page that looks like LISTNAME administrative interface (requires authorization) Once there, you need to authenticate with either the list's admin password or the site password if there is one. See . Perhaps you should be talking to your "colleague who is leaving at the end of the month". Also note that what we are talking about here is the 'owner' address for a particular list. Standard GNU Mailman has no concept of a sitewide admin email address other than perhaps the 'owner' of the 'mailman' site list. If what you are dealing with is some non-standard Mailman, e.g., Mailman on a cPanel host, there may be others on this list who can help, but I can't. But you need to tell us exactly what you are talking about. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan ------------------------------------------------------ Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/m.gilliatt%40live.co.uk From mark at msapiro.net Mon Oct 9 12:02:06 2017 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 9 Oct 2017 09:02:06 -0700 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: References: <84fuawrt0w.fsf@maxwell.cjones.org> Message-ID: <508b52b8-c426-e2fb-fd36-0cb4f23fbcaf@msapiro.net> On 10/09/2017 06:20 AM, Marc Gilliatt wrote: > I appreciate your help. I've spoken to my colleague, he is not sure on how to change the email either, which helps a lot I suppose. What he has told me, is that when he first set up and configured Mailman, he used his work email instead of IT's. I'm not too sure if this has shed some light on the issue I'm having? It does not. Perhaps if we knew exactly what he did to install/configure Mailman, that would help. Please provide the information previously asked for. > We have no idea what you are seeing. If you can post the actual URLs > that you are going to, we might be able to be more helpful. > > Where do you see this "subscribe" icon? If you click it, does it take > you to a page that looks something like the one at > ? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From weif at weif.net Mon Oct 9 12:21:19 2017 From: weif at weif.net (Keith Seyffarth) Date: Mon, 09 Oct 2017 10:21:19 -0600 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: (message from Marc Gilliatt on Mon, 9 Oct 2017 13:20:39 +0000) Message-ID: <84shess35s.fsf@maxwell.cjones.org> Marc, > I appreciate your help. I've spoken to my colleague, he is not sure on > how to change the email either, which helps a lot I suppose. What he > has told me, is that when he first set up and configured Mailman, he > used his work email instead of IT's. I'm not too sure if this has shed > some light on the issue I'm having? It would really help if you could answer some of Mark's questions. I see the instruction I tried to give was confusing to you. You do not need to be subscribed to have administrative access and to access the General Options page. Here are Mark's questions and instructions again: > Where do you see this "subscribe" icon? If you click it, does it take > you to a page that looks something like the one at > ? > > If so, where you ultimately want to go is to the link at the bottom of > that page that looks like > > LISTNAME administrative interface (requires authorization) > > Once there, you need to authenticate with either the list's admin > password or the site password if there is one. See > . Does your subscribe page look like the one that Mark linked to above? Are you able to find the "LISTNAME administrative interface" link at the bottom of that page? Keith -- ---- from my mac to yours... Keith Seyffarth mailto:weif at weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar ---- http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention From m.gilliatt at live.co.uk Tue Oct 10 05:53:29 2017 From: m.gilliatt at live.co.uk (Marc Gilliatt) Date: Tue, 10 Oct 2017 09:53:29 +0000 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: <84shess35s.fsf@maxwell.cjones.org> References: (message from Marc Gilliatt on Mon, 9 Oct 2017 13:20:39 +0000),<84shess35s.fsf@maxwell.cjones.org> Message-ID: Hi, I've attached screenshots of the lists, and when I go into one of those lists. That's where I'm seeing the subscribe icon? Thanks, Marc ________________________________ From: Keith Seyffarth Sent: 09 October 2017 17:21 To: Marc Gilliatt Cc: mark at msapiro.net; mailman-users at python.org Subject: Re: [Mailman-Users] How to change the administrators account email? Marc, > I appreciate your help. I've spoken to my colleague, he is not sure on > how to change the email either, which helps a lot I suppose. What he > has told me, is that when he first set up and configured Mailman, he > used his work email instead of IT's. I'm not too sure if this has shed > some light on the issue I'm having? It would really help if you could answer some of Mark's questions. I see the instruction I tried to give was confusing to you. You do not need to be subscribed to have administrative access and to access the General Options page. Here are Mark's questions and instructions again: > Where do you see this "subscribe" icon? If you click it, does it take > you to a page that looks something like the one at > ? > > If so, where you ultimately want to go is to the link at the bottom of > that page that looks like > > LISTNAME administrative interface (requires authorization) > > Once there, you need to authenticate with either the list's admin > password or the site password if there is one. See > . Does your subscribe page look like the one that Mark linked to above? Are you able to find the "LISTNAME administrative interface" link at the bottom of that page? Keith -- ---- from my mac to yours... Keith Seyffarth mailto:weif at weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar ---- http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention From paul at tokyoprogressive.org Tue Oct 10 02:56:02 2017 From: paul at tokyoprogressive.org (paul at tokyoprogressive.org) Date: Tue, 10 Oct 2017 15:56:02 +0900 Subject: [Mailman-Users] cause of bounces Message-ID: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> Hi and hope the answer(s) to my question are relatively simple. On one of two lists I manage, some people are getting deleted due to too many bounces. And the bounces seem to be related to their mail provider not allowing the messages. As far as I can tell, the main culprits are gmail, yahoo, and hotmail. Of course, those people blame Mailman. From what i have read, it is not necessarily that. But complicating things is that people were complaining that the default REPLY to SENDER was not appropriate for a discussion list, so I just switched it over to REPLY TO GROUP. I do not THINK that is the reason for the trouble, but here in the space of a few minutes are some of the messages I have gotten with my questions in caps. PROBLEM ONE UNABLE TO SEND? RECEIVE? a at yahoo.com host mta7.am0.yahoodns.net [66.196.118.37] SMTP error from remote mail server after end of data: 554 5.7.9 Message not accepted for policy reasons. See https://help.yahoo.com/kb/postmaster/SLN7253.html SEEMS TO BE YAHOO DOE SNOT LIKE MAILMAN AND ALSO SHE CANNOT ADD MAILMAN TO A WHITE LIST (THEY DO NOT HAVE).?> IS IT POSSIBLE SHE IS USING A MAIL CLENT THAT DOES NOT SEND CORRECTLY? OR IS THIS THE YAHOO INTERACE? IS THIS HER MESSAGE BEING SENT OR HER MAIL PROGRAM NOT ACCEPTING MESSAGES? RELATED TO THIS: Some people are getting unsubscribed as a result: List: Galeexec Member ie at hotmail.com Action: Subscription disabled. Reason: Excessive or fatal bounces. CAUSE OF ALL THE ABOVE? SOLUTIONS?BY LIST OWNER? BY MEMBER? NEXT hn at hotmail.com host hotmail-com.olc.protection.outlook.com [104.44.194.233] SMTP error from remote mail server after end of data: 550 5.7.0 (SNT004-MC7F11) Unfortunately, messages from (199.223.209.221) on behalf of (yahoo.com ) could not be delivered due to domain owner policy restrictions. SAME THING? I note the user has a hotmail address. CAUSE OF THE ABOVE? SOLUTIONS?BY LIST OWNER? BY MEMBER? BELOW THAT IS MORE INFO (RELATED TO THE ABOVE?) RESPECTIVELY Action: failed Final-Recipient: rfc822;ya at yahoo.com Status: 5.0.0 Remote-MTA: dns; mta7.am0.yahoodns.net Diagnostic-Code: smtp; 554 5.7.9 Message not accepted for policy reasons. See https://help.yahoo.com/kb/postmaster/SLN7253.html Action: failed Final-Recipient: rfc822; hn at hotmail.com Status: 5.0.0 Remote-MTA: dns; hotmail-com.olc.protection.outlook.com Diagnostic-Code: smtp; 550 5.7.0 (SNT004-MC7F11) Unfortunately, messages from (199.223.209.221) on behalf of (yahoo.com ) could not be delivered due to domain owner policy restrictions. There are others like this and they all share one of the two providers. PROBLEM TWO Though GMAIL USERS have reported messages in SPAM. No errors messages to the list. CAUSE OF THE ABOVE? SOLUTIONS?BY LIST OWNER? BY MEMBER? Thank you very much. Paul Arenson Japan EMAIL tokyoprogressive at mailbox.org paul at tokyoprogressive.org - - - - - - - - - - - - - - - - - - - - - - NEWS AND ACTIVISM http://tokyoprogressive.org MUSIC http://paularenson.org - - - - - - - - - - - - - - - - - - - - - - Phone/Voice Mail 050-5308-5394 From abroad 81-50-5308-5394 Phone/SMS 090-4173-3873 From abroad 81-90-4173-3873 Contact via LINE is also possible. - - - - - - - - - - - - - - - - - - - - - - EMAIL tokyoprogressive at mailbox.org paul at tokyoprogressive.org - - - - - - - - - - - - - - - - - - - - - - NEWS AND ACTIVISM http://tokyoprogressive.org MUSIC http://paularenson.org - - - - - - - - - - - - - - - - - - - - - - Phone/Voice Mail 050-5308-5394 From abroad 81-50-5308-5394 Phone/SMS 090-4173-3873 From abroad 81-90-4173-3873 Contact via LINE is also possible. - - - - - - - - - - - - - - - - - - - - - - From paul at tokyoprogressive.org Tue Oct 10 03:34:20 2017 From: paul at tokyoprogressive.org (paul at tokyoprogressive.org) Date: Tue, 10 Oct 2017 16:34:20 +0900 Subject: [Mailman-Users] Mail delivery or sending issues and list reply settings Message-ID: <700E73BC-B089-491D-A07E-3ABCE0D43E41@tokyoprogressive.org> Hi and hope the answer(s) to my question are relatively simple. On one of two lists I manage, some people are getting deleted due to too many bounces. And the bounces seem to be related to their mail provider not allowing the messages. As far as I can tell, the main culprits are gmail, yahoo, and hotmail. Of course, those people blame Mailman. From what i have read, it is not necessarily that. But complicating things is that people were complaining that the default REPLY to SENDER was not appropriate for a discussion list, so I just switched it over to REPLY TO GROUP. I do not THINK that is the reason for the trouble, but here in the space of a few minutes are some of the messages I have gotten with my questions in caps. (On the other hand, we didn?t have this issue UNTIL I switched to reply to the list. So I am wondering if there IS indeed a connection. Also unsure if this is a problem with the user not being able to receive mail, send mail, or both so far). PROBLEM ONE: UNABLE TO SEND AND/OR RECEIVE EXAMPLES FROM YAHOO AND HOTMAIL a at yahoo.com host mta7.am0.yahoodns.net [66.196.118.37] SMTP error from remote mail server after end of data: 554 5.7.9 Message not accepted for policy reasons. See https://help.yahoo.com/kb/postmaster/SLN7253.html SEEMS TO BE YAHOO DOES NOT LIKE MAILMAN AND ALSO SHE CANNOT ADD MAILMAN TO A WHITE LIST (YAHOO DO NOT HAVE!!!!).?> IS IT POSSIBLE SHE IS USING A MAIL CLENT THAT DOES NOT SEND CORRECTLY? OR IS THIS THE YAHOO INTERFACE? IS THIS HER MESSAGE BEING SENT OR HER MAIL PROGRAM NOT ACCEPTING MESSAGES? RELATED TO THIS: Some people are getting unsubscribed as a result: List: Galeexec Member ie at hotmail.com Action: Subscription disabled. Reason: Excessive or fatal bounces. CAUSE OF ALL THE ABOVE? SOLUTIONS?BY LIST OWNER? BY MEMBER? NEXT EXAMPLE OF SAME hn at hotmail.com host hotmail-com.olc.protection.outlook.com [104.44.194.233] SMTP error from remote mail server after end of data: 550 5.7.0 (SNT004-MC7F11) Unfortunately, messages from (199.223.209.221) on behalf of (yahoo.com ) could not be delivered due to domain owner policy restrictions. SAME THING? I note the user has a hotmail address. CAUSE OF THE ABOVE? SOLUTIONS?BY LIST OWNER? BY MEMBER? BELOW THAT IS MORE INFO (RELATED TO THE ABOVE?) RESPECTIVELY Action: failed Final-Recipient: rfc822;ya at yahoo.com Status: 5.0.0 Remote-MTA: dns; mta7.am0.yahoodns.net Diagnostic-Code: smtp; 554 5.7.9 Message not accepted for policy reasons. See https://help.yahoo.com/kb/postmaster/SLN7253.html Action: failed Final-Recipient: rfc822; hn at hotmail.com Status: 5.0.0 Remote-MTA: dns; hotmail-com.olc.protection.outlook.com Diagnostic-Code: smtp; 550 5.7.0 (SNT004-MC7F11) Unfortunately, messages from (199.223.209.221) on behalf of (yahoo.com ) could not be delivered due to domain owner policy restrictions. There are others like this and they all share one of the two providers. PROBLEM TWO Though GMAIL USERS have reported messages in SPAM. No errors messages to the list. CAUSE OF THE ABOVE? SOLUTIONS?BY LIST OWNER? BY MEMBER? Thank you very much. Paul Arenson Japan From mark at msapiro.net Tue Oct 10 08:58:36 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 10 Oct 2017 05:58:36 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> Message-ID: <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> On October 9, 2017 11:56:02 PM PDT, paul at tokyoprogressive.org wrote: >Hi and hope the answer(s) to my question are relatively simple. On one >of two lists I manage, some people are getting deleted due to too many >bounces. And the bounces seem to be related to their mail provider not >allowing the messages. As far as I can tell, the main culprits are >gmail, yahoo, and hotmail. I think this is a DMARC issue. See . -- Mark Sapiro Sent from my Not_an_iThing with standards compliant, open source software. From weif at weif.net Tue Oct 10 11:36:37 2017 From: weif at weif.net (Keith Seyffarth) Date: Tue, 10 Oct 2017 09:36:37 -0600 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: (message from Marc Gilliatt on Tue, 10 Oct 2017 09:53:29 +0000) Message-ID: <84k203581m.fsf@maxwell.cjones.org> Marc, > I've attached screenshots of the lists, and when I go into one of those lists. > That's where I'm seeing the subscribe icon? On the screen with the tan or beige blocks and the "Using Assets-budget-test" and "Subscribing to Assets-budget-test" headings (this is the main page for this mailing list), if you scroll to the bottom of the page, do you see the link Mark described: >> LISTNAME administrative interface (requires authorization) Keith -- ---- from my mac to yours... Keith Seyffarth mailto:weif at weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar ---- http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention From rclemings at gmail.com Tue Oct 10 14:54:38 2017 From: rclemings at gmail.com (Russell Clemings) Date: Tue, 10 Oct 2017 11:54:38 -0700 Subject: [Mailman-Users] ModSecurity false positive on list options pages Message-ID: Using OWASP ModSecurity Core Rule Set ver.3.0.2 on cPanel v66.0.23; CentOS 7.3, Mailman 2.1.23: ModSecurity with the OWASP rules (which come with cPanel nowadays) doesn't like Mailman's list options url for some email addresses. Specifically a URL in the form http://lists.xxx.xxx/mailman/options/listname/localpart--at--domain.com is blocked with a 403 error. Only '.com" addresses are affected, as far as I can tell, and the reason is rule 920440: SecRule REQUEST_BASENAME "\.(.*)$" "chain, capture, phase:request, t:none,t:urlDecodeUni,t:lowercase, block, msg:'URL file extension is restricted by policy', severity:'CRITICAL', rev:'2', ver:'OWASP_CRS/3.0.0', maturity:'9', accuracy:'9', id:920440, logdata:'%{TX.0}', tag:'application-multi', tag:'language-multi', tag:'platform-multi', tag:'attack-protocol', tag:'OWASP_CRS/POLICY/EXT_RESTRICTED', tag:'WASCTC/WASC-15', tag:'OWASP_TOP_10/A7', tag:'PCI/6.5.10',logdata:'%{TX.0}', setvar:tx.extension=.%{tx.1}/" SecRule TX:EXTENSION "@within %{tx.restricted_extensions}" "t:none, setvar:'tx.msg=%{rule.msg}', setvar:tx.anomaly_score=+%{tx.critical_anomaly_score}, setvar:tx.%{rule.id }-OWASP_CRS/POLICY/EXT_RESTRICTED-%{matched_var_name}=%{matched_var}" tx.restricted_extensions, defined elsewhere, lists a lot of file extensions, .com being among them: 'tx.restricted_extensions=.asa/ .asax/ .ascx/ .axd/ .backup/ .bak/ .bat/ .cdx/ .cer/ .cfg/ .cmd/ .com/ .config/ .conf/ .cs/ .csproj/ .csr/ .dat/ .db/ .dbf/ .dll/ .dos/ .htr/ .htw/ .ida/ .idc/ .idq/ .inc/ .ini/ .key/ .licx/ .lnk/ .log/ .mdb/ .old/ .pass/ .pdb/ .pol/ .printer/ .pwd/ .resources/ .resx/ .sql/ .sys/ .vb/ .vbs/ .vbproj/ .vsdisco/ .webinfo/ .xsd/ .xsx/' So basically ModSecurity sees .com at the end of the URL and thinks a risky file is being requested and blocks the request. With some help from the OWASP list I wrote a new rule that works around this problem. It is entered in the "ModSecurity Tools" section of cPanel's WHM. I don't know if there's a way to do it if you don't have access to WHM. I couldn't find one. SecRule REQUEST_URI "^\/mailman\/options\/.*" "id:900240,phase:1,nolog,pass,t:none,setvar:'tx.restricted_extensions=.asa/ .asax/ .ascx/ .axd/ .backup/ .bak/ .bat/ .cdx/ .cer/ .cfg/ .cmd/ .config/ .conf/ .cs/ .csproj/ .csr/ .dat/ .db/ .dbf/ .dll/ .dos/ .htr/ .htw/ .ida/ .idc/ .idq/ .inc/ .ini/ .key/ .licx/ .lnk/ .log/ .mdb/ .old/ .pass/ .pdb/ .pol/ .printer/ .pwd/ .resources/ .resx/ .sql/ .sys/ .vb/ .vbs/ .vbproj/ .vsdisco/ .webinfo/ .xsd/ .xsx/'" That substitutes a new list of restricted extensions for requests to a page containing the "/mailman/options/" string. Now: http://lists.xxx.xxx/mailman/options/listname/localpart--at--domain.com is admitted http://lists.xxx.xxx/mailman/Xoptions/listname/localpart--at--domain.com is blocked I'm posting this mostly for the sake of anyone else who runs into this problem. If there's a simple fix on the Mailman side, though, so much the better. Maybe substitute another character for the dot? From paul at tokyoprogressive.org Tue Oct 10 18:10:27 2017 From: paul at tokyoprogressive.org (paul at tokyoprogressive.org) Date: Wed, 11 Oct 2017 07:10:27 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> Message-ID: EMAIL tokyoprogressive at mailbox.org paul at tokyoprogressive.org - - - - - - - - - - - - - - - - - - - - - - NEWS AND ACTIVISM http://tokyoprogressive.org MUSIC http://paularenson.org - - - - - - - - - - - - - - - - - - - - - - Phone/Voice Mail 050-5308-5394 From abroad 81-50-5308-5394 Phone/SMS 090-4173-3873 From abroad 81-90-4173-3873 Contact via LINE is also possible. - - - - - - - - - - - - - - - - - - - - - - > On Oct 10, 2017, at 23:57, paul at tokyoprogressive.org wrote: > > Thank you, Mark. All the options look bad (except telling people to use another email provider). > > > OPTIONS > A few are unclear, such as RESTARTING Mailman. How does one restart it? I use Cpanel and do not know the inner workings. > > Which do you think it the best of the suggestions? I already have content filtering set to off to allow attachments. And there is currently only a footer that says: > > _______________________________________________ > Galeexec mailing list > Galeexec at gale-sig.org > http://mail.gale-sig.org/mailman/listinfo/galeexec_gale-sig.org > > > Again, the list is REPLY TO LIST. > > > IS IT ONLY YAHOO ADDRESSES? > > I found another article that makes me wonder? https://www.pcworld.com/article/2141120/yahoo-email-antispoofing-policy-breaks-mailing-lists.html > > it says "List subscribers with email accounts on servers that perform DMARC checks, such as Gmail, Hotmail (Outlook.com ), Comcast or Yahoo itself, will reject the original message and respond back to the list with automated DMARC error messages"??. making it seem that all of these providers are no-nos. > > But later it says "So users of Gmail, Hotmail and other DMARC-enabled providers will not only fail to receive messages sent to the mailing list by Yahoo users, but will flood the list with bounce messages, risking to be bounced off the list themselves?. > > > This sentence seems to imply that it is YAHOO users who should switch. But the previous quote implies people with all of those providers should switch. > > > Can you give me your opinion. Is it Yahoo that is breaking mailing lists, or is it Yahoo, Gmail and Hotmail? > > > Thanls > > > > > > >> On Oct 10, 2017, at 21:58, Mark Sapiro > wrote: >> >> On October 9, 2017 11:56:02 PM PDT, paul at tokyoprogressive.org wrote: >>> Hi and hope the answer(s) to my question are relatively simple. On one >>> of two lists I manage, some people are getting deleted due to too many >>> bounces. And the bounces seem to be related to their mail provider not >>> allowing the messages. As far as I can tell, the main culprits are >>> gmail, yahoo, and hotmail. >> >> >> I think this is a DMARC issue. See >. >> >> >> >> -- >> Mark Sapiro > >> Sent from my Not_an_iThing with standards compliant, open source software. > From paul at tokyoprogressive.org Tue Oct 10 18:20:19 2017 From: paul at tokyoprogressive.org (paul at tokyoprogressive.org) Date: Wed, 11 Oct 2017 07:20:19 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> Message-ID: Thank you, Mark. Had to resend this as I forgot to remove the quotes in the first attempt. Re it being a DMARC issue, all the options look bad (except telling people to use another email provider). OPTIONS A few are unclear, such as RESTARTING Mailman. How does one restart it? I use Cpanel and do not know the inner workings. Which do you think it the best of the suggestions? I already have content filtering set to off to allow attachments. And there is currently only a footer that says: _______________________________________________ Galeexec mailing list Galeexec at gale-sig.org http://mail.gale-sig.org/mailman/listinfo/galeexec_gale-sig.org Again, the list is REPLY TO LIST. IS IT ONLY YAHOO ADDRESSES? I found another article that makes me wonder? https://www.pcworld.com/article/2141120/yahoo-email-antispoofing-policy-breaks-mailing-lists.html it says "List subscribers with email accounts on servers that perform DMARC checks, such as Gmail, Hotmail (Outlook.com ), Comcast or Yahoo itself, will reject the original message and respond back to the list with automated DMARC error messages"??. making it seem that all of these providers are no-nos. But later it says "So users of Gmail, Hotmail and other DMARC-enabled providers will not only fail to receive messages sent to the mailing list by Yahoo users, but will flood the list with bounce messages, risking to be bounced off the list themselves?. This sentence seems to imply that it is YAHOO users who should switch. But the previous quote implies people with all of those providers should switch. Can you give me your opinion. Is it Yahoo that is breaking mailing lists, or is it Yahoo, Gmail and Hotmail? Thanks Paul Arenson > On Oct 10, 2017, at 21:58, Mark Sapiro > wrote: > > On October 9, 2017 11:56:02 PM PDT, paul at tokyoprogressive.org wrote: >> Hi and hope the answer(s) to my question are relatively simple. On one >> of two lists I manage, some people are getting deleted due to too many >> bounces. And the bounces seem to be related to their mail provider not >> allowing the messages. As far as I can tell, the main culprits are >> gmail, yahoo, and hotmail. > > > I think this is a DMARC issue. See >. > > > > -- > Mark Sapiro > > Sent from my Not_an_iThing with standards compliant, open source software. From dmaziuk at bmrb.wisc.edu Tue Oct 10 18:24:16 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Tue, 10 Oct 2017 17:24:16 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> Message-ID: On 10/10/2017 05:10 PM, paul at tokyoprogressive.org wrote: >> Can you give me your opinion. Is it Yahoo that is breaking mailing lists, or is it Yahoo, Gmail and Hotmail? All of the above. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From chip at aresti.com Tue Oct 10 19:28:07 2017 From: chip at aresti.com (Chip Davis) Date: Tue, 10 Oct 2017 19:28:07 -0400 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> Message-ID: <30143051-0ce2-b40d-6602-211f38766bba@aresti.com> On 10/10/2017 6:20 PM, paul at tokyoprogressive.org wrote: > > Thank you, Mark. Had to resend this as I forgot to remove the quotes in the first attempt. Re it being a DMARC issue, all the options look bad (except telling people to use another email provider). Which is exactly what I do with the dozen-ish lists that I host/admin. Friends don't let friends use Yahoo... :-( -Chip- From mark at msapiro.net Tue Oct 10 21:52:52 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 10 Oct 2017 18:52:52 -0700 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: References: <84shess35s.fsf@maxwell.cjones.org> Message-ID: <942c4786-2c54-295f-7d39-3f191805a820@msapiro.net> On 10/10/2017 02:53 AM, Marc Gilliatt wrote: > Hi, > > I've attached screenshots of the lists, and when I go into one of those lists. That's where I'm seeing the subscribe icon? Note: the screenshots did not go to the list because the list's content filtering removed them, but Keith and I got them by direct copy and they were what appeared to be a portion of the listinfo/ overview page and a portion of the listinfo page for a particular list. As Keith said, if you scroll to the bottom of the page that begins "About Assets-budget-test" you should see a footer like Assets-budget-test list run by user at example.com Assets-budget-test administrative interface (requires authorization) Overview of all example.com mailing lists The middle link is a link to the admin interface for the list where you can change things for this list including the owner address which is the user at example.com address on the first line. Even if you don't see the footer, you can get to the admin interface by changing '/listinfo/' and only that in the URL of that page to '/admin/'. If you have command line access to the server, you can do this more easily from the command line than by visiting each lists admin pages in turn. To visit all the admin pages on the web, it is easier to start with a URL like the one for a single list but ending with '/admin/' which will give a page similar to the listinfo overview but with links to the admin pages. To set the owner for all lists from the command line run this script: #!/bin/sh cd /path/to/mailman/bin f=`mktemp` echo "owner = 'user at example.com'" > $f for l in `./list_lists --bare`; do ./config_list -i $f $l done rm $f where /path/to/mailman/bin is the path to Mailman's bin/ directory and user at example.com is the address you want to set the owner to. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Tue Oct 10 22:23:30 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 10 Oct 2017 19:23:30 -0700 Subject: [Mailman-Users] ModSecurity false positive on list options pages In-Reply-To: References: Message-ID: <85952b36-4422-3107-45c9-9f5065350829@msapiro.net> On 10/10/2017 11:54 AM, Russell Clemings wrote: > > I'm posting this mostly for the sake of anyone else who runs into this > problem. If there's a simple fix on the Mailman side, though, so much the > better. Maybe substitute another character for the dot? It would be fairly simple to modify Mailman.Utils.ObscureEmail and Mailman.Utils.UnobscureEmail to in addition to replacing @ with --at-- replace the final dot with say an = and vice versa. It could even be done in a way that would allow a URL with --at-- and no = in the domain to still work. However, it's more complex than that because a list can be configured with Privacy options... -> Subscription rules -> obscure_addresses = No and Mailman.Utils.ObscureEmail won't be called to munge the address(es). For this and other reasons having to do with simply wanting to dial way back on MM 2.1 in favor of MM 3, I'm not inclined to do this, at least for 2.1.25. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Tue Oct 10 22:55:06 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 10 Oct 2017 19:55:06 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> Message-ID: <4720404f-3503-65ed-2618-2bc01767875f@msapiro.net> On 10/10/2017 03:20 PM, paul at tokyoprogressive.org wrote: > > OPTIONS > A few are unclear, such as RESTARTING Mailman. How does one restart it? I use Cpanel and do not know the inner workings. > > Which do you think it the best of the suggestions? I already have content filtering set to off to allow attachments. And there is currently only a footer that says: If your cPanel is reasonably up to date, you have Mailman 2.1.23 or 2.1.14. This what you want is in the list admin UI on the Privacy options... -> Sender filters page set dmarc_moderation_action = Munge From dmarc_quarantine_moderation_action = Yes and if it exists dmarc_none_moderation_action = No These are the settings referred to in the last paragraph of item 1) at and described in more detal it the linked DMARC page at . > it says "List subscribers with email accounts on servers that perform DMARC checks, such as Gmail, Hotmail (Outlook.com ), Comcast or Yahoo itself, will reject the original message and respond back to the list with automated DMARC error messages"??. making it seem that all of these providers are no-nos. > > But later it says "So users of Gmail, Hotmail and other DMARC-enabled providers will not only fail to receive messages sent to the mailing list by Yahoo users, but will flood the list with bounce messages, risking to be bounced off the list themselves?. > > > This sentence seems to imply that it is YAHOO users who should switch. But the previous quote implies people with all of those providers should switch. > > > Can you give me your opinion. Is it Yahoo that is breaking mailing lists, or is it Yahoo, Gmail and Hotmail? The issue is twofold. Mail which will be bounced is mail From: yahoo.com, aol.com and any other domain that publishes a DMARC policy of reject. Initially, the only freemail provider to do this was Yahoo, but AOL soon followed. Currently both Gmail and Hotmail and also Comcast publish DMARC p=none, so mail From: those domains should not be bounced for DMARC policy reasons, BUT all 5 of those ISPs and many others honor DMARC which means they will all reject mail that fails DMARC from Yahoo and AOL and any other domain that publishes DMARC p=reject. So no, neither quote implies Gmail or Hotmail user's need to switch. It only says that those users won't receive unmitigated posts sent by Yahoo users. and that those ISP's and others will bounce the Yahoo mail. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Tue Oct 10 23:00:11 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 10 Oct 2017 20:00:11 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <30143051-0ce2-b40d-6602-211f38766bba@aresti.com> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> <30143051-0ce2-b40d-6602-211f38766bba@aresti.com> Message-ID: <5cbefb08-a48a-e487-da2f-7f70b768a115@msapiro.net> On 10/10/2017 04:28 PM, Chip Davis wrote: > On 10/10/2017 6:20 PM, paul at tokyoprogressive.org wrote: >> >> Thank you, Mark.? Had to resend this as I forgot to remove the quotes >> in the first attempt. Re it being a DMARC issue, all the options look >> bad (except telling people to use another email provider). > > Which is exactly what I do with the dozen-ish lists that I host/admin. > ?Friends don't let friends use Yahoo... :-( I agree, but unfortunately many list owners are in situations where it is at least politically infeasible to do that. One would think with all the security breaches that everyone would have voluntarily quit using Yahoo mail by now, but it seems not. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From turnbull.stephen.fw at u.tsukuba.ac.jp Tue Oct 10 23:33:18 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Wed, 11 Oct 2017 12:33:18 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> Message-ID: <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> @mark: The Hotmail error message does make it sufficiently clear that these bounces are due to DMARC. I will probably file an RFE to catch these against Mailman 3. Would you like me to do that for Mailman 2, or is this "obvously not worth it" in your opinion? (I intend to supply code "eventually". ;-) I should also fix the FAQ to mention the names of the new options, no? @paul: Mark Sapiro writes: > On October 9, 2017 11:56:02 PM PDT, paul at tokyoprogressive.org wrote: > >Hi and hope the answer(s) to my question are relatively simple. On one > >of two lists I manage, some people are getting deleted due to too many > >bounces. And the bounces seem to be related to their mail provider not > >allowing the messages. As far as I can tell, the main culprits are > >gmail, yahoo, and hotmail. The messages that *cause* the problem are from Yahoo! and Hotmail users (I just ban posting from those addresses, but I can do that because using them for university business is against Monkeyshow = Japanese government policy ;-). Many large providers will bounce these messages when resent via Mailman (and most featureful list managers). Gmail does "quarantine" these messages by putting them in "spam". (I think this is the appropriate action, both according to the DMARC standard and according the the "finger in eye" theory of corporate rivalry :-). > I think this is a DMARC issue. See > . The Hotmail errors explicitly state that "on behalf of" messages violate Yahoo!'s policy, so yes, this is a DMARC issue in at least some cases. I suspect the reason that this "suddenly" came up after changing the list's Reply-To policy is that some Hotmail and Yahoo! users started sending to list when they hadn't been doing so before. Note that the relevant option name has changed in recent Mailman; it is now Privacy Options -> Sender Filters -> DMARC Moderation Action. You almost certainly want that set to "Munge From", and the following option DMARC Quarantine Moderation Action set to "Yes". (Note that "quarantine" is a setting on the mail sender's side; Mailman neither knows nor cares about Gmail's quarantine policy.) If subscribers complain that Yahoo! and Hotmail users' mail now comes "From: LIST on behalf of USER (EMAIL) " but others come "From: USER ", you may also want to set the third option DMARC None Moderation Action to "Yes". From mark at msapiro.net Wed Oct 11 00:08:11 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 10 Oct 2017 21:08:11 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> Message-ID: <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> On 10/10/2017 08:33 PM, Stephen J. Turnbull wrote: > @mark: > > The Hotmail error message does make it sufficiently clear that these > bounces are due to DMARC. I will probably file an RFE to catch these > against Mailman 3. Would you like me to do that for Mailman 2, or is > this "obvously not worth it" in your opinion? (I intend to supply > code "eventually". ;-) Are you suggesting that we ignore bounces that can be determined to be due to DMARC policy. This is an interesting idea and would help prevent "innocent" list members from being "bounced" off the list, but the cost would be that these innocent members don't receive some posts and this happens "silently" so the list admins may not even be aware that there is a problem. Of course, we could inundate them with notices ;) I'd be interested in what you come up with for MM 3, and then maybe consider a backport. > I should also fix the FAQ to mention the names of the new options, no? Both FAQs could use tweaking or more. needs more about current options and the MM 3 section at needs a significant update. I suspect one of us will get to it. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From m.gilliatt at live.co.uk Wed Oct 11 03:51:37 2017 From: m.gilliatt at live.co.uk (Marc Gilliatt) Date: Wed, 11 Oct 2017 07:51:37 +0000 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: <942c4786-2c54-295f-7d39-3f191805a820@msapiro.net> References: <84shess35s.fsf@maxwell.cjones.org> , <942c4786-2c54-295f-7d39-3f191805a820@msapiro.net> Message-ID: I did click on the middle link, and it took me to the "List Administrator Password" page. Is it a simple process of resetting that password? Also, that script you have given me, can that reset the owner's email for all the lists? And thank you for the help you have shown me, I truly appreciate it. ________________________________ From: Mailman-Users on behalf of Mark Sapiro Sent: 11 October 2017 02:52 To: mailman-users at python.org Subject: Re: [Mailman-Users] How to change the administrators account email? On 10/10/2017 02:53 AM, Marc Gilliatt wrote: > Hi, > > I've attached screenshots of the lists, and when I go into one of those lists. That's where I'm seeing the subscribe icon? Note: the screenshots did not go to the list because the list's content filtering removed them, but Keith and I got them by direct copy and they were what appeared to be a portion of the listinfo/ overview page and a portion of the listinfo page for a particular list. As Keith said, if you scroll to the bottom of the page that begins "About Assets-budget-test" you should see a footer like Assets-budget-test list run by user at example.com Assets-budget-test administrative interface (requires authorization) Overview of all example.com mailing lists The middle link is a link to the admin interface for the list where you can change things for this list including the owner address which is the user at example.com address on the first line. Even if you don't see the footer, you can get to the admin interface by changing '/listinfo/' and only that in the URL of that page to '/admin/'. If you have command line access to the server, you can do this more easily from the command line than by visiting each lists admin pages in turn. To visit all the admin pages on the web, it is easier to start with a URL like the one for a single list but ending with '/admin/' which will give a page similar to the listinfo overview but with links to the admin pages. To set the owner for all lists from the command line run this script: #!/bin/sh cd /path/to/mailman/bin f=`mktemp` echo "owner = 'user at example.com'" > $f for l in `./list_lists --bare`; do ./config_list -i $f $l done rm $f where /path/to/mailman/bin is the path to Mailman's bin/ directory and user at example.com is the address you want to set the owner to. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan ------------------------------------------------------ Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/m.gilliatt%40live.co.uk From dlugasny at protonmail.com Wed Oct 11 04:23:31 2017 From: dlugasny at protonmail.com (Dlugasny) Date: Wed, 11 Oct 2017 04:23:31 -0400 Subject: [Mailman-Users] DKIM signing issue - relaying mailman e-mails from third party sources Message-ID: Hallo Mark, Kolleagues, maybe somebody will be able to help me here. I would like to relay (also check with Antivir and sign them with DKIM) all Mailman e-mails from our external partners to our final customers. Relaying seems to work nice but we have a problem with DKIM authentication and Return-Path. Mailman server needs to stay by external partners with old domains and needs to receive bounces. My SMTP gateway is only responsible for receiving and sending e-mails out from the all mailman instances. The problem is that we are sending an E-mail which looks as follow: From: campaign at myserver.com Return-Path: mailman-bounces at external-company.com To: @gmail.com The problem is that DKIM check on the gmail server server (and all others) returning error: [...mailman-bounces at external-company.com](mailto:mailman-bounces at external-company.com) does not designate xx.xx.xx.xx as permitted sender How to solve that issue ? At the moment we have a DKIM key only for myserver.com. Why DKIM check checking Return-Path and not From address ? Could You please help here how to manage that issue ? We simply would like to forward all messages from the external mailman instances installed on the different domains to the final customers using our sender domain [myserver.com.](mailto:campaign at myserver.com) I will appreciate any feedback from Your side. Cheers Dlugasny From weif at weif.net Wed Oct 11 10:22:52 2017 From: weif at weif.net (Keith Seyffarth) Date: Wed, 11 Oct 2017 08:22:52 -0600 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: (message from Marc Gilliatt on Wed, 11 Oct 2017 07:51:37 +0000) Message-ID: <84y3ohlq6b.fsf@maxwell.cjones.org> Marc Gilliatt writes: > I did click on the middle link, and it took me to the "List > Administrator Password" page. Is it a simple process of resetting that > password? Also, that script you have given me, can that reset the > owner's email for all the lists? And thank you for the help you have > shown me, I truly appreciate it. Marc, Now that you have gotten to the "list administrator password" page, you need to enter the administrator password for the list. Once you enter this, you will be taken to the General Options page where you can change the email address of the list owner. Hopefully Mark will weigh in on the commands he sent. I don't recall all of them currently, but the discussion so far had been about changing the list owner email address. Keith -- ---- from my mac to yours... Keith Seyffarth mailto:weif at weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar ---- http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention From mailman at jordan.maileater.net Wed Oct 11 10:46:57 2017 From: mailman at jordan.maileater.net (Jordan Brown) Date: Wed, 11 Oct 2017 07:46:57 -0700 Subject: [Mailman-Users] Wish list: bounce notifications Message-ID: My mailing list is hosted on a low-cost service with shared servers and so a couple of times a year some other customer gets them onto a spam blacklist and mail starts bouncing.? I need to know about those situations ASAP so that I can prod the provider into fixing the problem and requesting delisting, but Mailman doesn't seem to have a way to tell me when it gets a bounce... only when it's gotten several bounces and disabled a membership.? Since I don't want to wait for several bounces, I have it set on a hair trigger; it disables people on the first bounce and so I immediately get a notification.? That works, kind of, but it means that I have an extra re-enable step to repair the damage and that if I somehow drop a notification on the floor I can leave somebody disabled. What I'd like would be a way to ask Mailman to notify me on *every* bounce, without disabling the user. (Or, of course, if there's just something I've missed in the config pages, please educate me.) From luscheina at yahoo.de Wed Oct 11 13:29:45 2017 From: luscheina at yahoo.de (Christian F Buser) Date: Wed, 11 Oct 2017 19:29:45 +0200 Subject: [Mailman-Users] Wish list: bounce notifications In-Reply-To: References: Message-ID: <20171011192945193619.88d2e3aa@yahoo.de> Hello Jordan Brown. On Wed, 11 Oct 2017 07:46:57 -0700, you wrote: > My mailing list is hosted on a low-cost service with shared servers and > so a couple of times a year some other customer gets them onto a spam > blacklist and mail starts bouncing. I need to know about those > situations ASAP so that I can prod the provider into fixing the problem > and requesting delisting, but Mailman doesn't seem to have a way to tell > me when it gets a bounce... only when it's gotten several bounces and > disabled a membership. Since I don't want to wait for several bounces, > I have it set on a hair trigger; it disables people on the first bounce > and so I immediately get a notification. That works, kind of, but it > means that I have an extra re-enable step to repair the damage and that > if I somehow drop a notification on the floor I can leave somebody disabled. > > What I'd like would be a way to ask Mailman to notify me on *every* > bounce, without disabling the user. I think the last 4 settings in the cPanel installation "Bounce processing Section" should all be set to YES Should Mailman send you, the list owner, any bounce messages that failed to be detected by the bounce processor? Yes is recommended. (Details for bounce_unrecognized_goes_to_list_owner) Should Mailman notify you, the list owner, when bounces cause a member's bounce score to be incremented? (Details for bounce_notify_owner_on_bounce_increment) Should Mailman notify you, the list owner, when bounces cause a member's subscription to be disabled? (Details for bounce_notify_owner_on_disable) Should Mailman notify you, the list owner, when bounces cause a member to be unsubscribed? (Details for bounce_notify_owner_on_removal) Depending on the possibilities your provider offers, I would imagine that it could be possible that you receive a copy of the bounce messages into your personal mailbox. -- Christian F. Buser, Hohle Gasse 6, CH-5507 Mellingen (Switzerland) Hilfe fuer Strassenkinder in Ghana: http://www.chance-for-children.org From mark at msapiro.net Wed Oct 11 14:12:00 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 11 Oct 2017 11:12:00 -0700 Subject: [Mailman-Users] DKIM signing issue - relaying mailman e-mails from third party sources In-Reply-To: References: Message-ID: <5e8e0f7a-24fa-e3fb-30c7-c9e66c296611@msapiro.net> On 10/11/2017 01:23 AM, Dlugasny via Mailman-Users wrote: > > The problem is that we are sending an E-mail which looks as follow: > > From: campaign at myserver.com > Return-Path: mailman-bounces at external-company.com > To: @gmail.com > > The problem is that DKIM check on the gmail server server (and all others) returning error: > [...mailman-bounces at external-company.com](mailto:mailman-bounces at external-company.com) does not designate xx.xx.xx.xx as permitted sender This is not DKIM. it is SPF. external-company.com publishes an SPF record that doesn't allow myserver.com as a sender. Start at to learn more about SPF. There are two solutions to this. The Return-Path: mailman-bounces at external-company.com header indicates that mailman-bounces at external-company.com is the envelope sender of the message and SPF is based on the domain of the envelope sender. solution 1). external-company.com can augment its published SPF record to designate your myserver.com server as a permitted sender. solution 2). Your mail relaying process can rewrite the envelope sender to your domain, e.g., campaign at myserver.com or some other appropriate @myserver.com address. This will break mailman's automated bounce processing for mail from mailman-bounces at external-company.com that is relayed by you, but if you can verify the deliverability of that mail before relaying it and if it's not deliverable, reject it before rewriting the envelope sender, that won't be an issue. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Wed Oct 11 14:39:15 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 11 Oct 2017 11:39:15 -0700 Subject: [Mailman-Users] How to change the administrators account email? In-Reply-To: References: <84shess35s.fsf@maxwell.cjones.org> <942c4786-2c54-295f-7d39-3f191805a820@msapiro.net> Message-ID: <70e6dc89-643d-8b8c-9920-6a5a881d157d@msapiro.net> On 10/11/2017 12:51 AM, Marc Gilliatt wrote: > I did click on the middle link, and it took me to the "List Administrator Password" page. Is it a simple process of resetting that password? As I said at , > Once there, you need to authenticate with either the list's admin > password or the site password if there is one. See > . which has info on resetting the password and setting a site password. > Also, that script you have given me, can that reset the owner's email for all the lists? And thank you for the help you have shown me, I truly appreciate it. ... > > To set the owner for all lists from the command line run this script: ^^^ The for loop does all lists. > > #!/bin/sh > cd /path/to/mailman/bin > f=`mktemp` > echo "owner = 'user at example.com'" > $f > for l in `./list_lists --bare`; do > ./config_list -i $f $l > done > rm $f > > where /path/to/mailman/bin is the path to Mailman's bin/ directory and > user at example.com is the address you want to set the owner to. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Wed Oct 11 14:53:13 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 11 Oct 2017 11:53:13 -0700 Subject: [Mailman-Users] Wish list: bounce notifications In-Reply-To: References: Message-ID: On 10/11/2017 07:46 AM, Jordan Brown wrote: > > What I'd like would be a way to ask Mailman to notify me on *every* > bounce, without disabling the user. > > (Or, of course, if there's just something I've missed in the config > pages, please educate me.) As mentioned by Christian F Buser in his reply, Since Mailman 2.1.19 there has been a bounce_notify_owner_on_bounce_increment setting that does what you want. Also see . -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tokyoprogressive at mailbox.org Tue Oct 10 22:44:02 2017 From: tokyoprogressive at mailbox.org (Paul Arenson) Date: Wed, 11 Oct 2017 02:44:02 +0000 (UTC) Subject: [Mailman-Users] cause of bounces In-Reply-To: <3044677CDF15CF0B.C9CC83F1-580C-4B3B-A31A-BA0748DD7253@mail.outlook.com> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> <30143051-0ce2-b40d-6602-211f38766bba@aresti.com> <3044677CDF15CF0B.C9CC83F1-580C-4B3B-A31A-BA0748DD7253@mail.outlook.com> Message-ID: <3044677CDF15CF0B.F64842F2-65E5-41E3-961F-7243DB05D9BE@mail.outlook.com> Dmitri, Chip, Mark Thank you. So your judgment would be for Yahoo users in particular to get a new address? How about hotmail, outlook, or gmail users? Of course they could, ?I assume, keep those addresses for nob mailing list stuff and open a new account for mailing lists with protonmail, tutanova etc if they do not have a company or school address (and forward to a mail client if they know how to do that-which many of my users being language teachers seem to be resistant to understanding-grin). Any recommendations on big email companies that are safe? And would you limit it to Yahoo or tell others on hotmail and gmail having trouble the same? Thanks ------------------Paul Arensonpaul at tokyoprogressive.org From tokyoprogressive at mailbox.org Wed Oct 11 06:31:00 2017 From: tokyoprogressive at mailbox.org (mailbox.org) Date: Wed, 11 Oct 2017 19:31:00 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> Message-ID: <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> Let me repost that. Somehow my signature appeared mid post and made it look like that was my entire message. I should not post from my phone. Easier to edit at the computer as I am now. Thanks, Mark and Stephen, Anyway, to confirm my version is 2.1.23 It seems that the names that both of you, Mark and Stephen, are referring to the same functions with slightly different names because of the version? I want to make sure that I don?t make the wrong setting. Mark mentioned list admin UI on the Privacy > options... -> Sender filters page set > > dmarc_moderation_action = Munge From > dmarc_quarantine_moderation_action = Yes > and if it exists > dmarc_none_moderation_action = No WELL I HAVE ONE THAT SAYS Shall the above dmarc_moderation_action apply to messages From: domains with DMARC p=quarantine as well as p=reject and Shall the above dmarc_moderation_action apply to messages From: domains with DMARC p=none as well as p=quarantine and p=reject Are these the same, and is that a YES and NO? On the other hand Stephen said: > > Note that the relevant option name has changed in recent Mailman; it > is now Privacy Options -> Sender Filters -> DMARC Moderation Action. > You almost certainly want that set to "Munge From", and the following > option DMARC Quarantine Moderation Action set to "Yes". (Note that > "quarantine" is a setting on the mail sender's side; Mailman neither > knows nor cares about Gmail's quarantine policy.) and > If subscribers complain that Yahoo! and Hotmail users' mail now comes > "From: LIST on behalf of USER (EMAIL) " but > others come "From: USER ", you may also want to set the third > option DMARC None Moderation Action to "Yes?. OK, there are three: Action to take when anyone posts to the list from a domain with a DMARC Reject/Quarantine Policy. MUNGE FROM Shall the above dmarc_moderation_action apply to messages From: domains with DMARC p=quarantine as well as p=reject YES? Shall the above dmarc_moderation_action apply to messages From: domains with DMARC p=none as well as p=quarantine and p=rejectn NO?? Have I got the gist of what you both are saying? Are these the agreed upon settings? Finally, I am understanding (hopefully correctly) that Yahoo and Hotmail are the trouble makers. And it especially makes trouble for the others (AOL and GMAIL) Good reason to suggest at least my YAHOO and HOTMAIL users switch to another provider. I found the free and secure provider disroot that offers a large amount of space. Maybe I will suggest that one. Thanks, Paul From mark at msapiro.net Wed Oct 11 15:54:39 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 11 Oct 2017 12:54:39 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <3044677CDF15CF0B.F64842F2-65E5-41E3-961F-7243DB05D9BE@mail.outlook.com> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <0B4DE886-C540-4CEF-9C04-A87ED0217E42@tokyoprogressive.org> <30143051-0ce2-b40d-6602-211f38766bba@aresti.com> <3044677CDF15CF0B.C9CC83F1-580C-4B3B-A31A-BA0748DD7253@mail.outlook.com> <3044677CDF15CF0B.F64842F2-65E5-41E3-961F-7243DB05D9BE@mail.outlook.com> Message-ID: <222e6fa1-2c5d-1b9a-619e-09e908d4c9b6@msapiro.net> On 10/10/2017 07:44 PM, Paul Arenson wrote: > So your judgment would be for Yahoo users in particular to get a new address? How about hotmail, outlook, or gmail users? ; <<>> DiG 9.10.3-P4-Ubuntu <<>> txt _dmarc.hotmail.com ... ;; ANSWER SECTION: _dmarc.hotmail.com. 3600 IN TXT "v=DMARC1; p=none; sp=quarantine; pct=100; rua=mailto:d at rua.agari.com; ruf=mailto:d at ruf.agari.com; fo=1" ; <<>> DiG 9.10.3-P4-Ubuntu <<>> txt _dmarc.gmail.com ... ;; ANSWER SECTION: _dmarc.gmail.com. 274 IN TXT "v=DMARC1; p=none; rua=mailto:mailauth-reports at google.com" ; <<>> DiG 9.10.3-P4-Ubuntu <<>> txt _dmarc.outlook.com ;; ANSWER SECTION: _dmarc.outlook.com. 3600 IN TXT "v=DMARC1; p=none; pct=100; rua=mailto:d at rua.agari.com; ruf=mailto:d at ruf.agari.com; fo=1" I.e., all of those domains CURRENTLY publish DMARC p=none. Who knows what they might do tomorrow. I personally don't think it's likely that they will change for various reasons including the fact that DMARC p=reject policies are not designed for and not appropriate for freemail providers, but I have no crystal ball. Yahoo began publishing DMARC p=reject (inappropriately IMO) in response to various Yahoo security breaches. Smart people at Yahoo believe it was worth it for the good it did them in spite of the disruption it causes to mailing lists. AOL quickly followed - who knows what motivates AOL. > Of course they could, ?I assume, keep those addresses for nob mailing list stuff and open a new account for mailing lists with protonmail, tutanova etc if they do not have a company or school address (and forward to a mail client if they know how to do that-which many of my users being language teachers seem to be resistant to understanding-grin). > Any recommendations on big email companies that are safe? > And would you limit it to Yahoo or tell others on hotmail and gmail having trouble the same? If you want to go that route, set dmarc_moderation_action (or if you're confused by that) Action to take when anyone posts to the list from a domain with a DMARC Reject/Quarantine Policy. (Details for dmarc_moderation_action) to "Reject" and set Text to include in any rejection notice to be sent to anyone who posts to this list from a domain with a DMARC Reject/Quarantine Policy. (Edit dmarc_moderation_notice) to something like: "You are not allowed to post to this list From: a domain that publishes a DMARC policy of reject or quarantine. For more information see ." or some other message. If you don't want to take this step, just set Action to take when anyone posts to the list from a domain with a DMARC Reject/Quarantine Policy. (Details for dmarc_moderation_action) to "Munge From" and avoid the problem that way. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Wed Oct 11 16:13:30 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 11 Oct 2017 13:13:30 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> Message-ID: <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> On 10/11/2017 03:31 AM, mailbox.org wrote: > > OK, there are three: > Action to take when anyone posts to the list from a domain with a DMARC Reject/Quarantine Policy. MUNGE FROM > Shall the above dmarc_moderation_action apply to messages From: domains with DMARC p=quarantine as well as p=reject YES? > Shall the above dmarc_moderation_action apply to messages From: domains with DMARC p=none as well as p=quarantine and p=rejectn NO?? > > > > Have I got the gist of what you both are saying? Are these the agreed upon settings? Yes. > Finally, I am understanding (hopefully correctly) that Yahoo and Hotmail are the trouble makers. And it especially makes trouble for the others (AOL and GMAIL) Good reason to suggest at least my YAHOO and HOTMAIL users switch to another provider. I found the free and secure provider disroot that offers a large amount of space. Maybe I will suggest that one. The "trouble makers" are those freemail providers that publish DMARC policies of "reject" or "quarantine" These currently include Yahoo and AOL but not Hotmail or Gmail. All of those ISPs are part of the chain of events that leads to trouble because they reject or quarantine mail From: domains that publish DMARC policies of "reject" or "quarantine" and which fails DMARC validation. disroot.org seems like a viable freemail provider, but getting people to switch is problematic. Unless they *need* to post to your list and can't otherwise do it, they won't switch providers or use an alternate just to be able to. In any case, with the recommended settings above, you will have avoided the problem, so they don't need to switch. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mailman at jordan.maileater.net Wed Oct 11 16:15:21 2017 From: mailman at jordan.maileater.net (Jordan Brown) Date: Wed, 11 Oct 2017 13:15:21 -0700 Subject: [Mailman-Users] Wish list: bounce notifications In-Reply-To: References: Message-ID: <2762ae4e-e388-6ddf-cbd9-b0f592b81853@maileater.net> On 10/11/2017 11:53 AM, Mark Sapiro wrote: > On 10/11/2017 07:46 AM, Jordan Brown wrote: >> What I'd like would be a way to ask Mailman to notify me on *every* >> bounce, without disabling the user. >> >> (Or, of course, if there's just something I've missed in the config >> pages, please educate me.) > As mentioned by Christian F Buser in his reply, Since Mailman 2.1.19 > there has been a bounce_notify_owner_on_bounce_increment setting that > does what you want. Also see > . Yep, sorry, duh.? It's new since the last time I did a full sweep through the options, and I didn't look before writing. From tokyoprogressive at mailbox.org Thu Oct 12 09:39:48 2017 From: tokyoprogressive at mailbox.org (mailbox.org) Date: Thu, 12 Oct 2017 22:39:48 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> Message-ID: <58A1B943-CC6E-46E6-9D80-4E564680244C@mailbox.org> Thanks Mark. Have made the changes (and told people they can refrain from changing providers for the time being. I appreciate your help as well as that of the other contributors. > On Oct 12, 2017, at 5:13, Mark Sapiro wrote: > > On 10/11/2017 03:31 AM, mailbox.org wrote: >> >> OK, there are three: >> Action to take when anyone posts to the list from a domain with a DMARC Reject/Quarantine Policy. MUNGE FROM >> Shall the above dmarc_moderation_action apply to messages From: domains with DMARC p=quarantine as well as p=reject YES? >> Shall the above dmarc_moderation_action apply to messages From: domains with DMARC p=none as well as p=quarantine and p=rejectn NO?? >> >> >> >> Have I got the gist of what you both are saying? Are these the agreed upon settings? > > > Yes. > From gtaylor at tnetconsulting.net Thu Oct 12 17:15:34 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Thu, 12 Oct 2017 15:15:34 -0600 Subject: [Mailman-Users] DKIM signing issue - relaying mailman e-mails from third party sources In-Reply-To: <5e8e0f7a-24fa-e3fb-30c7-c9e66c296611@msapiro.net> References: <5e8e0f7a-24fa-e3fb-30c7-c9e66c296611@msapiro.net> Message-ID: On 10/11/2017 12:12 PM, Mark Sapiro wrote: > solution 2). Your mail relaying process can rewrite the envelope sender > to your domain, e.g., campaign at myserver.com or some other appropriate > @myserver.com address. This will break mailman's automated bounce > processing for mail from mailman-bounces at external-company.com that is > relayed by you, but if you can verify the deliverability of that mail > before relaying it and if it's not deliverable, reject it before > rewriting the envelope sender, that won't be an issue. Would something like configuring the MTA to use Sender Rewrite Scheme help avoid this issue? SRS would mean that the MTA would rewrite the SMTP envelope from address to be a local domain that is permitted by SPF. SRS would also decode any bounces and send the original address into Mailman. - I think. -- Grant. . . . unix || die From mark at msapiro.net Thu Oct 12 19:25:20 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 12 Oct 2017 16:25:20 -0700 Subject: [Mailman-Users] DKIM signing issue - relaying mailman e-mails from third party sources In-Reply-To: References: <5e8e0f7a-24fa-e3fb-30c7-c9e66c296611@msapiro.net> Message-ID: <0cf91fda-36d0-01ba-4c05-919341dd56bb@msapiro.net> On 10/12/2017 02:15 PM, Grant Taylor via Mailman-Users wrote: > On 10/11/2017 12:12 PM, Mark Sapiro wrote: >> solution 2). Your mail relaying process can rewrite the envelope >> sender to your domain, e.g., campaign at myserver.com or some other >> appropriate @myserver.com address. This will break mailman's automated >> bounce processing for mail from mailman-bounces at external-company.com >> that is relayed by you, but if you can verify the deliverability of >> that mail before relaying it and if it's not deliverable, reject it >> before rewriting the envelope sender, that won't be an issue. > > Would something like configuring the MTA to use Sender Rewrite Scheme > help avoid this issue? Yes. SRS, as I understand it from , would solve the whole problem. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From turnbull.stephen.fw at u.tsukuba.ac.jp Sat Oct 14 16:07:33 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Sun, 15 Oct 2017 05:07:33 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> Message-ID: <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> tl;dr I miswrote when I wrote that Hotmail is a problem sending domain. It never was and currently is not. I was thinking of AOL which was and is a problem. The rest of this post explains why DMARC is a mostly good thing, including a *very* high-level view of what it is good *for*. Mark Sapiro writes: > On 10/11/2017 03:31 AM, mailbox.org wrote: > > Finally, I am understanding (hopefully correctly) that Yahoo and > > Hotmail are the trouble makers. I miswrote here. Hotmail doesn't publish a DMARC p=reject policy; it's AOL that does. There are many other domains that do publish p=reject but they also have internal policies against posting to mailing lists. Yahoo! and AOL are the only posting addresses you're likely to run into that cause problems. > > And it especially makes trouble for the others (AOL and GMAIL) > > Good reason to suggest at least my YAHOO and HOTMAIL users switch > > to another provider. I found the free and secure provider > > disroot that offers a large amount of space. Maybe I will > > suggest that one. I'm with Mark here. Unless you "own" your users in some way (most of mine are my students, for example), it's way more trouble than it's worth to ask people to change. They'll need to move archived mail, for example. Also, AFAIK it's not possible to disable a Yahoo! address unless you delete it. BUT IT MIGHT NOT STAY DELETED: Yahoo! recycles unused addresses after a few months. It turns out that such reused account names will have access to any resources that authenticate using that Yahoo! address. So in practice users probably should keep their Yahoo! accounts anyway, even if they don't use them. DMARC itself is a good thing, and you should encourage users to use email providers who participate in the protocol. Specifically, Gmail has an excellent policy: if it believes that a message that violates a sending provider's p=reject policy is a mailing list post, it will "quarantine" the mail in the "spam" folder. This means that there are no bounces for the *receiver* (which is why your subscribers were getting disabled or unsubscribed), and the receiver can easily find the mail at minor inconvenience (if they know to look, which is something of a problem, of course). I don't know of other providers with this policy but I expect it is in use at others and will probably spread. How is DMARC a good thing? DMARC does the following (1) Provide a way for email providers to get reports about usage of their mailboxes by third parties from recipients of such mail. This helps them to learn about spam campaigns, especially "spear-spamming" where the bad guys know your correspondents and send you spam (or phishing) email that appears to be from an acquaintance. The DMARC consortium claimed in late 2015 that over 80% of all email was covered by DMARC reporting, so providers who have the skills to take advantage of this data have extremely precise knowledge of usage. (2) Provide a way to notify recipients that all mail from the provider's domain is authenticated, and mail whose credentials do not verify must be presumed to be malicious. This is the "p=reject" policy that several of us have mentioned. For (2) to make sense, the email provider should have a policy that prohibits use of its mailboxes to post to mailing lists, and it must not provide "on behalf of" services such as sending photographs or newspaper articles using your address in From. This makes sense for banks and other financial institutions, and use of DMARC "p=reject" has pretty much eliminated phishing using mail with real bank addresses in From. This is how Yahoo! and AOL met trouble. Both leaked N x 100,000,000 contact lists to spammers, who used them for spear-spamming, much of it phishing of various sorts. Turning on p=reject is said to have reduced those spam campaigns from MILLIONS OF SPAMS PER MINUTE (!!) to a trickle. The business argument for doing this despite collateral damage to lists and various on-behalf-of businesses and their clients is obvious, and given how dangerous spear-phishing is, there's even a plausible ethical argument for it. (You can say "they shouldn't have leaked", but they did -- now what?) Note that there is a new protocol in the works called ARC which will mitigate the problem for mailing lists as it's adopted. Unfortunately it is no help for "on behalf of" services, but as a mailing list developer and admin, I'll take it! Gmail and I think Yahoo! are already using it experimentally, although I don't know how they evaluate the "transitive trust" that is involved. (Ie, ARC involves the mailing list testifying that they verified the credentials of the poster.) HTH Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Sat Oct 14 16:09:06 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Sun, 15 Oct 2017 05:09:06 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> Message-ID: <23010.28386.514692.181431@turnbull.sk.tsukuba.ac.jp> Mark Sapiro writes: > Are you suggesting that we ignore bounces that can be determined to be > due to DMARC policy. Not *completely* ignore. There are several independent actions we can take based on bounces, depending on list option settings. I'm suggesting only that we not increment the bounce count for members. Notifications are another matter. > This is an interesting idea and would help prevent "innocent" list > members from being "bounced" off the list, but the cost would be > that these innocent members don't receive some posts I prefer to think of it as "guilty" posters don't get full distribution of their posts. ;-) > and this happens "silently" so the list admins may not even be > aware that there is a problem. Of course, we could inundate them > with notices > ;) Well, I'd prefer to avoid both fire and flood.[1] :-/ But this is pretty silent already, and from the point of view of many admins, bounced users who don't resubscribe are a cost. > I'd be interested in what you come up with for MM 3, and then maybe > consider a backport. Sounds good to me! > > I should also fix the FAQ to mention the names of the new options, no? > > > Both FAQs could use tweaking or more. > needs more about current options and the MM 3 section at > needs a significant update. I suspect > one of us will get to it. Yep! Don't wait for me, though. :-P Steve Footnotes: [1] I hope you and your circle are OK! From tokyoprogressive at mailbox.org Tue Oct 17 00:27:13 2017 From: tokyoprogressive at mailbox.org (mailbox.org) Date: Tue, 17 Oct 2017 13:27:13 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> Message-ID: Thank you Steve! Now I understand it is not all bad. Just the way that AOIL and YAHOO went about it (or something like that). paul - - - - - - - - - - - - - - - - - - - - - - > On Oct 15, 2017, at 5:07, Stephen J. Turnbull wrote: > > tl;dr I miswrote when I wrote that Hotmail is a problem sending > domain. It never was and currently is not. I was thinking of AOL > which was and is a problem. > > The rest of this post explains why DMARC is a mostly good thing, > including a *very* high-level view of what it is good *for*. > > Mark Sapiro writes: >> On 10/11/2017 03:31 AM, mailbox.org wrote: > >>> Finally, I am understanding (hopefully correctly) that Yahoo and >>> Hotmail are the trouble makers. > > I miswrote here. Hotmail doesn't publish a DMARC p=reject policy; > it's AOL that does. There are many other domains that do publish > p=reject but they also have internal policies against posting to > mailing lists. Yahoo! and AOL are the only posting addresses you're > likely to run into that cause problems. > >>> And it especially makes trouble for the others (AOL and GMAIL) >>> Good reason to suggest at least my YAHOO and HOTMAIL users switch >>> to another provider. I found the free and secure provider >>> disroot that offers a large amount of space. Maybe I will >>> suggest that one. > > I'm with Mark here. Unless you "own" your users in some way (most of > mine are my students, for example), it's way more trouble than it's > worth to ask people to change. They'll need to move archived mail, > for example. Also, AFAIK it's not possible to disable a Yahoo! > address unless you delete it. BUT IT MIGHT NOT STAY DELETED: Yahoo! > recycles unused addresses after a few months. It turns out that such > reused account names will have access to any resources that > authenticate using that Yahoo! address. So in practice users probably > should keep their Yahoo! accounts anyway, even if they don't use them. > > DMARC itself is a good thing, and you should encourage users to use > email providers who participate in the protocol. Specifically, Gmail > has an excellent policy: if it believes that a message that violates a > sending provider's p=reject policy is a mailing list post, it will > "quarantine" the mail in the "spam" folder. This means that there are > no bounces for the *receiver* (which is why your subscribers were > getting disabled or unsubscribed), and the receiver can easily find > the mail at minor inconvenience (if they know to look, which is > something of a problem, of course). I don't know of other providers > with this policy but I expect it is in use at others and will probably > spread. > > How is DMARC a good thing? DMARC does the following > > (1) Provide a way for email providers to get reports about usage of > their mailboxes by third parties from recipients of such mail. > > This helps them to learn about spam campaigns, especially > "spear-spamming" where the bad guys know your correspondents and send > you spam (or phishing) email that appears to be from an acquaintance. > The DMARC consortium claimed in late 2015 that over 80% of all email > was covered by DMARC reporting, so providers who have the skills to > take advantage of this data have extremely precise knowledge of usage. > > (2) Provide a way to notify recipients that all mail from the > provider's domain is authenticated, and mail whose credentials do > not verify must be presumed to be malicious. This is the > "p=reject" policy that several of us have mentioned. > > For (2) to make sense, the email provider should have a policy that > prohibits use of its mailboxes to post to mailing lists, and it must > not provide "on behalf of" services such as sending photographs or > newspaper articles using your address in From. This makes sense for > banks and other financial institutions, and use of DMARC "p=reject" > has pretty much eliminated phishing using mail with real bank > addresses in From. > > This is how Yahoo! and AOL met trouble. Both leaked N x 100,000,000 > contact lists to spammers, who used them for spear-spamming, much of > it phishing of various sorts. Turning on p=reject is said to have > reduced those spam campaigns from MILLIONS OF SPAMS PER MINUTE (!!) to > a trickle. The business argument for doing this despite collateral > damage to lists and various on-behalf-of businesses and their clients > is obvious, and given how dangerous spear-phishing is, there's even a > plausible ethical argument for it. (You can say "they shouldn't have > leaked", but they did -- now what?) > > Note that there is a new protocol in the works called ARC which will > mitigate the problem for mailing lists as it's adopted. Unfortunately > it is no help for "on behalf of" services, but as a mailing list > developer and admin, I'll take it! Gmail and I think Yahoo! are > already using it experimentally, although I don't know how they > evaluate the "transitive trust" that is involved. (Ie, ARC involves > the mailing list testifying that they verified the credentials of the > poster.) > > HTH > > Steve > ------------------------------------------------------ > Mailman-Users mailing list Mailman-Users at python.org > https://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://wiki.list.org/x/AgA3 > Security Policy: http://wiki.list.org/x/QIA9 > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-users/paul%40tokyoprogressive.org From dmaziuk at bmrb.wisc.edu Tue Oct 17 11:06:03 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Tue, 17 Oct 2017 10:06:03 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> Message-ID: <7af6551a-8805-ecbd-76d1-b48a103dc140@bmrb.wisc.edu> On 2017-10-16 23:27, mailbox.org wrote: > Thank you Steve! > > Now I understand it is not all bad. Just the way that AOIL and YAHOO went about it (or something like that). It's not bad, only it's mostly useless for human people like you and I. What good it does is mostly for google-person and yahoo-person and their kind. Dima From gtaylor at tnetconsulting.net Tue Oct 17 12:10:56 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 17 Oct 2017 10:10:56 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> Message-ID: <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> On 10/14/2017 02:07 PM, Stephen J. Turnbull wrote: > For (2) to make sense, the email provider should have a policy that > prohibits use of its mailboxes to post to mailing lists, and it must > not provide "on behalf of" services such as sending photographs or > newspaper articles using your address in From. This makes sense for > banks and other financial institutions, and use of DMARC "p=reject" > has pretty much eliminated phishing using mail with real bank > addresses in From. Some drive by comments: - IMHO, "on behalf of" services (I like that description) should be sent with a From: address that reflects the service -and- utilize a Reply-To: that reflects the email address of the purported sender. (Further, the service's From: address /should/ be legitimate and not bounce. But that's more pedantic.) - I feel like DMARC is perfectly compatible with mailing lists as long as the mailing list is set up to modify the message as it passes through the list: 1) Change the From: header to reflect the mailing list. 2) Send the message with an SMTP from reflecting the mailing list. (VERP is suggested.) 3) Remove any / all DKIM headers. - I *STRONGLY* feel that mailing lists / forwarders / etc are email endpoints. Many of them generate new messages with content based on the incoming content. - Thus it is perfectly acceptable to do all of the above /because/ it is a /new/ and /different/ message. I know that I am not personally sending this message to anyone other than the single address that is the mailman-users mailing list. - The mailman-users mailing list is what is sending message to all the subscribers, *NOT* me. Both my mail server and the mail list server's MTA logs will corroborate this. - I think pretending that I am /personally/ (thus my MTA is) sending messages to all the subscribers is a farce. Further I believe that said farce is part of (if not the crux of) the perceived problems with SPF / DKIM / DMARC on conjunction with mailing lists. Think about it this way. If Alice sends a message to Bob, who has his email configured to forward to Charlie who also forwards to Dave, and so on until we reach Mike, I will *STRONGLY* argue that I never sent a message to Mike if asked. Sure, /someone's/ server sent a message to Mike, possibly claiming to be from me. But it was *NOT* /from/ me or my server. Thus, the message is bogus and /should/ be treated as such. - I recently compared forwarders / mailing lists to be like phone messages. The person taking the phone message does not pretend to be the caller when passing the message along. Instead the message taker typically says something to the effect of "$SoandSo called and left a message for you." The phone message is a /new/ message based on the contents of the original call, *NOT* a (replay) of the original call. -- Grant. . . . unix || die From luscheina at yahoo.de Tue Oct 17 12:55:42 2017 From: luscheina at yahoo.de (Christian F Buser) Date: Tue, 17 Oct 2017 18:55:42 +0200 Subject: [Mailman-Users] cause of bounces In-Reply-To: <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> Message-ID: <20171017185542995737.b7e31827@yahoo.de> Hello Grant Taylor via Mailman-Users. On Tue, 17 Oct 2017 10:10:56 -0600, you wrote: > Some drive by comments: > ... I can perfectly follow your thoughts and arguments, they appear to be justified and reasonable. However, could you please elaborate whether Mailman (version 2.x or 3.x) or any other mailing list software really follows your ideas? If it is just a question of the setup for the mailing list, I would expect your instructions on how to set it up properly. Thank you, Christian -- Christian F. Buser, Hohle Gasse 6, CH-5507 Mellingen (Switzerland) Hilfe fuer Strassenkinder in Ghana: http://www.chance-for-children.org From gtaylor at tnetconsulting.net Tue Oct 17 13:38:48 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 17 Oct 2017 11:38:48 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <20171017185542995737.b7e31827@yahoo.de> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> Message-ID: <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> On 10/17/2017 10:55 AM, Christian F Buser via Mailman-Users wrote: > I can perfectly follow your thoughts and arguments, they appear to be > justified and reasonable. Thank you. I tried to make them so that people could understand, even if they choose to disagree. > However, could you please elaborate whether Mailman (version 2.x or > 3.x) or any other mailing list software really follows your ideas? Yes!!! Mailman (and other MLMs) /can/ be configured to be SPF / DKIM / DMARC compliant! > If it is just a question of the setup for the mailing list, I would > expect your instructions on how to set it up properly. I don't have the exact step by step details. - I'm sure others (Mark...) on this list can give specifics on /how/ to configure Mailman. The high level as I understand it is to do the following: 1) Set dmarc_moderation_action to munge From header. 2) Set REMOVE_DKIM_HEADERS to Yes (1) or 2 or 3. 3) Send messages from the list address. I recommend VERP. Doing those three things ensures that messages leaving the mailing list do not violate the original sending domain's SPF / DKIM / DMARC security settings. I would suggest that you also consider adding SPF / DKIM / DMARC for the domain of the mailing list to apply similar protections to outgoing messages. However that is not necessary to avoid undesired bounces. > Thank you, Christian You're welcome. -- Grant. . . . unix || die From dmaziuk at bmrb.wisc.edu Tue Oct 17 13:45:05 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Tue, 17 Oct 2017 12:45:05 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> Message-ID: <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> On 10/17/2017 11:10 AM, Grant Taylor via Mailman-Users wrote: > ?- I *STRONGLY* feel that mailing lists / forwarders / etc are email > endpoints.? Many of them generate new messages with content based on the > incoming content.? -? Thus it is perfectly acceptable to do all of the > above /because/ it is a /new/ and /different/ message. +1 > Sure, /someone's/ server sent a message to Mike, possibly claiming to be > from me.? But it was *NOT* /from/ me or my server.? Thus, the message is > bogus and /should/ be treated as such. If these actually exist, my spamassassin has been delivering to /dev/null for quite some time now. My impression is they largely died off, possibly thanks to adoption of SPF. Now it is much easier and cheaper to send spam from botnets of perfectly legitimate pwn3d peecees. Or to anonymously register a perfectly valid domain (e.g. tn?t??nsulting.net -- there's 3 "language-specific script" chars in there), add all the DMARC embellishments, and send perfectly compliant spam as gtaylor from there. For bonus points, pay with stolen credit card number and have your spam campaign all done by the time visa fraud department calls you domain registar. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Tue Oct 17 16:38:40 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 13:38:40 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> Message-ID: <5367c96d-2697-b70b-1834-601e92604e18@msapiro.net> On 10/17/2017 09:10 AM, Grant Taylor via Mailman-Users wrote: > > I know that I am not personally sending this message to anyone other > than the single address that is the mailman-users mailing list.? -? The > mailman-users mailing list is what is sending message to all the > subscribers, *NOT* me. That is quite true, but in this example, the mailing list is the 'sender' of the message I receive. It is not the 'author' of the message. You are still the 'author'. RFC 5322, sec 3.6.2 and predecessors are clear: The "From:" field specifies the author(s) of the message, that is, the mailbox(es) of the person(s) or system(s) responsible for the writing of the message. The "Sender:" field specifies the mailbox of the agent responsible for the actual transmission of the message. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Tue Oct 17 17:22:33 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 14:22:33 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> Message-ID: On 10/17/2017 10:38 AM, Grant Taylor via Mailman-Users wrote: > On 10/17/2017 10:55 AM, Christian F Buser via Mailman-Users wrote: > >> However, could you please elaborate whether Mailman (version 2.x or >> 3.x) or any other mailing list software really follows your ideas? > > Yes!!!? Mailman (and other MLMs) /can/ be configured to be SPF / DKIM / > DMARC compliant! Agreed, but the above imply NOT RFC 5322 compliant. > I don't have the exact step by step details.? -? I'm sure others > (Mark...) on this list can give specifics on /how/ to configure Mailman. > > The high level as I understand it is to do the following: > > 1) Set dmarc_moderation_action to munge From header. This is available in both MM 2.1 and 3.1 > 2) Set REMOVE_DKIM_HEADERS to Yes (1) or 2 or 3. In MM 3, The only options are always remove or never remove. The "remove only if munging From:" and "rename" options are not in MM 3 However, it SHOULD not be necessary. Section 6.3 of RFC 4871 says in part: If the email cannot be verified, then it SHOULD be rendered the same as all unverified email regardless of whether or not it looks like it was signed. In other words, an invalid DKIM signature SHOULD be treated no differently from no signature. > 3) Send messages from the list address.? I recommend VERP. Mailman sends (SMTP envelope) all messages from the list-bounces address. Both MM 2.1 and MM 3 can be configured to VERP some or all deliveries. > I would suggest that you also consider adding SPF / DKIM / DMARC for the > domain of the mailing list to apply similar protections to outgoing > messages.? However that is not necessary to avoid undesired bounces. Publishing SPF and DKIM signing outgoing mail are good things. Publishing a DMARC policy and what policy to publish depend on how your server is used and what classes of mail it sends. In particular, if individuals send personal email, possibly to mailing lists From: addresses in the server's domain, I think publishing a DMARC policy other than "none" is not a good idea. On the other hand, if you are a financial institution and all mail From: your domain is official correspondence between you and clients, you are who DMARC was designed for. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From gtaylor at tnetconsulting.net Tue Oct 17 17:40:45 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 17 Oct 2017 15:40:45 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> Message-ID: <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> On 10/17/2017 03:22 PM, Mark Sapiro wrote: > Agreed, but the above imply NOT RFC 5322 compliant. Please elaborate, if you're referring to more than From: vs Sent-By:. > In other words, an invalid DKIM signature SHOULD be treated no > differently from no signature. Fair enough. - I suspect DKIM by itself can tolerate that, like you are referencing. I believe the problem is when DMARC is added to the mix, particularly with a policy of reject. -- Grant. . . . unix || die From mark at msapiro.net Tue Oct 17 17:54:55 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 14:54:55 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> Message-ID: On 10/17/2017 02:40 PM, Grant Taylor via Mailman-Users wrote: > On 10/17/2017 03:22 PM, Mark Sapiro wrote: >> Agreed, but the above imply NOT RFC 5322 compliant. > > Please elaborate, if you're referring to more than From: vs Sent-By:. What I mean is as I posted previously , RFC 5322 says the From: contains the "the mailbox(es) of the person(s) or system(s) responsible for the writing of the message." and munging the From: to the list address is not compliant with this requirement. In the spirit of DMARC mitigation, we all agree that it is a necessary evil, at least in some cases, but that doesn't change the fact that it is an 'evil'. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fmouse at fmp.com Tue Oct 17 18:15:00 2017 From: fmouse at fmp.com (Lindsay Haisley) Date: Tue, 17 Oct 2017 17:15:00 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> Message-ID: <1508278500.47036.64.camel@fmp.com> On Tue, 2017-10-17 at 14:54 -0700, Mark Sapiro wrote: > In the spirit of DMARC mitigation, we all agree that it is a necessary > evil, at least in some cases, but that doesn't change the fact that it > is an 'evil'. Just as an aside here, my understanding is that validation of an email by DMARC requires ONE of two things: EITHER the DKIM signature in the email must validate, OR the domain of the From body header must resolve to the IP address of the Sender system (list server or mail reflector). Is this correct? Where's a reference on this? -- Lindsay Haisley | "The first casualty when FMP Computer Services | war comes is truth." 512-259-1190 | http://www.fmp.com | -- Hiram W Johnson From gtaylor at tnetconsulting.net Tue Oct 17 18:28:01 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 17 Oct 2017 16:28:01 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <1508278500.47036.64.camel@fmp.com> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> Message-ID: <4f7b8bfb-0e1b-7579-ddbb-f519ae532b0b@tnetconsulting.net> On 10/17/2017 04:15 PM, Lindsay Haisley wrote: > Just as an aside here, my understanding is that validation of an email > by DMARC requires ONE of two things: EITHER the DKIM signature in the > email must validate, OR the domain of the From body header must resolve > to the IP address of the Sender system (list server or mail reflector). > Is this correct? Where's a reference on this? That is a per domain setting left up to the DMARC publisher. At least my understanding is that you can specify any of the following conditions to cause DMARC to pass / fail. 1) /Only/ SPF 2) /Only/ DKIM 3) SPF /or/ DKIM 4) SPF /and/ DKIM -- Grant. . . . unix || die From dmaziuk at bmrb.wisc.edu Tue Oct 17 18:28:08 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Tue, 17 Oct 2017 17:28:08 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> Message-ID: On 10/17/2017 04:40 PM, Grant Taylor via Mailman-Users wrote: > On 10/17/2017 03:22 PM, Mark Sapiro wrote: >> Agreed, but the above imply NOT RFC 5322 compliant. > > Please elaborate, if you're referring to more than From: vs Sent-By:. > >> In other words, an invalid DKIM signature SHOULD be treated no >> differently from no signature. > > Fair enough. Why? If this message doesn't match its signature, then it has been altered in transit for sure. If were not signed, like when I post from home (because I can't be arsed to set gpg up on winderz), then there's no telling if it was or wasn't. One of those things is quite a bit not like the other. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From gtaylor at tnetconsulting.net Tue Oct 17 18:35:03 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 17 Oct 2017 16:35:03 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> Message-ID: <1c0c8719-fef3-8acb-d761-f2f5fea192ab@tnetconsulting.net> On 10/17/2017 04:28 PM, Dimitri Maziuk wrote: > Why? If this message doesn't match its signature, then it has been > altered in transit for sure. If were not signed, like when I post from > home (because I can't be arsed to set gpg up on winderz), then there's > no telling if it was or wasn't. One of those things is quite a bit not > like the other. If I understand your question correctly.... DKIM is meant to cryptographically prove that a message is unaltered (*). I think that DKIM is avoiding the possibility that a message could be incidentally modified in transit, i.e. encoding conversion, thus not maliciously modified. As such, DKIM does not penalize for broken signatures. Instead, DKIM rewards valid signatures. I know it's a small nuanced distinction, but it is there. * ROPEMAKER further complicates this throwing lots of wrenches in the works. -- Grant. . . . unix || die From gtaylor at tnetconsulting.net Tue Oct 17 18:36:22 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 17 Oct 2017 16:36:22 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> Message-ID: On 10/17/2017 11:45 AM, Dimitri Maziuk wrote: > If these actually exist, my spamassassin has been delivering to > /dev/null for quite some time now. My impression is they largely died > off, possibly thanks to adoption of SPF. If these actually exist? - I'm talking about someone configuring their old email address to forward to their new email address. - I just happened to extrapolate out further. I.e. old college email forwards to Yahoo, which forwards to Gmail, etc. - I suspect the single level forwarding is quite common. Are we talking about the same thing? I.e. .forward files? Or are you thinking something more nefarious? > Now it is much easier and cheaper to send spam from botnets of perfectly > legitimate pwn3d peecees. Or to anonymously register a perfectly valid > domain (e.g. tn?t??nsulting.net -- there's 3 "language-specific script" > chars in there), add all the DMARC embellishments, and send perfectly > compliant spam as gtaylor from there. I scowl at you sir. I dislike being the example. But I think what you did is quite neat and perfectly valid example. Nicely played sir. I actually have no idea how to defend against such attacks, save for registering all such permutations. I wonder how some such language-specific script characters would show up in logs. Especially ASCII without UTF support. > For bonus points, pay with stolen credit card number and have your spam > campaign all done by the time visa fraud department calls you domain > registar. /me wonders what color Dimitri's hat is. ;-) #knowtheyenemy -- Grant. . . . unix || die From fmouse at fmp.com Tue Oct 17 18:38:20 2017 From: fmouse at fmp.com (Lindsay Haisley) Date: Tue, 17 Oct 2017 17:38:20 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: <4f7b8bfb-0e1b-7579-ddbb-f519ae532b0b@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <4f7b8bfb-0e1b-7579-ddbb-f519ae532b0b@tnetconsulting.net> Message-ID: <1508279900.47036.70.camel@fmp.com> On Tue, 2017-10-17 at 16:28 -0600, Grant Taylor via Mailman-Users wrote: > That is a per domain setting left up to the DMARC publisher. The DMARC publisher is not the system refusing delivery. The publisher advertises a policy. The receiving system honors it, or not. > At least my understanding is that you can specify any of the > following=20 > conditions to cause DMARC to pass / fail. > > 1) /Only/ SPF > 2) /Only/ DKIM > 3) SPF /or/ DKIM > 4) SPF /and/ DKIM Any system which REQUIRES DKIM validation to pass is out of compliance with RFCs, as I understand it. A DKIM signature which doesn't validate MUST be treated the same as no DKIM signature at all. And I don't believe we're dealing with SPF here, just alignment between the domain of an email's author (From) and the IP address of the system communicating SMTP server from which the recipient's SMTP server received the mail. Correct me if I'm wrong on this, but I don't believe a SPF record in DNS is required. -- Lindsay Haisley | "The first casualty when FMP Computer Services | war comes is truth." 512-259-1190 | http://www.fmp.com | -- Hiram W Johnson From mark at msapiro.net Tue Oct 17 19:07:09 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 16:07:09 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <1508278500.47036.64.camel@fmp.com> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> Message-ID: <2121e207-6500-be24-b577-b25ac12a06de@msapiro.net> On 10/17/2017 03:15 PM, Lindsay Haisley wrote: > > Just as an aside here, my understanding is that validation of an email > by DMARC requires ONE of two things: EITHER the DKIM signature in the > email must validate, OR the domain of the From body header must resolve > to the IP address of the Sender system (list server or mail reflector). > Is this correct? Where's a reference on this? The reference is the DMARC standard RFC 7489 . It's more complicated than the above. There is a concept of domain alignment. Alignment is satisfied in either "strict" or relaxed "mode". A dmarc policy record may optionally specify either mode for DKIM alignment or SPF alignment or both with the default being "relaxed. For a message to pass DMARC it must meet 1 of 2 requirements. 1) It must possess a valid DKIM signature from a domain aligned with the From: domain. In strict mode aligned means equal. In relaxed mode aligned means the corresponding organizational domains are equal. or 2) It must pass SPF. SPF works on the domain of the SMTP envelope from. Thus for SPF to pass, that domain must publish an SPF record specifying the IP of the sending server as a permitted sender. Further, for DMARC the envelope from (SPF) domain must align with the From: domain. Again, in strict mode aligned means equal. In relaxed mode aligned means the corresponding organizational domains are equal. Note that if you are relaying mail, SPF probably will pass for your server if the envelope from domain is your server, but it won't align with an unmunged From: domain and if it does align because you didn't rewrite it, SPF will fail unless the original sending domain publishes SPF that permits your server as a sender. So the bottom line is as an "unaffiliated" relay without munging From:, SPF will never pass for DMARC and DKIM will only pass if you don't transform the message in ways that break the From: domain's DKIM signature. There is a remote possibility that the originating domain that publishes a DMARC policy relies on SPF and doesn't DKIM sign the message in which case, unmumged, relayed mail will almost certainly fail DMARC. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Tue Oct 17 19:16:00 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 16:16:00 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> Message-ID: On 10/17/2017 03:28 PM, Dimitri Maziuk wrote: > On 10/17/2017 04:40 PM, Grant Taylor via Mailman-Users wrote: >> On 10/17/2017 03:22 PM, Mark Sapiro wrote: >> >>> In other words, an invalid DKIM signature SHOULD be treated no >>> differently from no signature. >> >> Fair enough. > > Why? If this message doesn't match its signature, then it has been > altered in transit for sure. If were not signed, like when I post from > home (because I can't be arsed to set gpg up on winderz), then there's > no telling if it was or wasn't. One of those things is quite a bit not > like the other. Why? Because that's what the DKIM standard, RFC 4871, says. You have a point, but to be safe you should assume that unsigned mail has been altered and if it's important, insist on some kind of cryptographic verification. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Tue Oct 17 19:20:45 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 16:20:45 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <4f7b8bfb-0e1b-7579-ddbb-f519ae532b0b@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <4f7b8bfb-0e1b-7579-ddbb-f519ae532b0b@tnetconsulting.net> Message-ID: <8d9a4a1c-e632-da8a-c0e3-2e10456d5ab4@msapiro.net> On 10/17/2017 03:28 PM, Grant Taylor via Mailman-Users wrote: > On 10/17/2017 04:15 PM, Lindsay Haisley wrote: >> Just as an aside here, my understanding is that validation of an email >> by DMARC requires ONE of two things: EITHER the DKIM signature in the >> email must validate, OR the domain of the From body header must >> resolve to the IP address of the Sender system (list server or mail >> reflector). ?Is this correct? Where's a reference on this? > > That is a per domain setting left up to the DMARC publisher. Not quite. > At least my understanding is that you can specify any of the following > conditions to cause DMARC to pass / fail. > > 1) /Only/ SPF > 2) /Only/ DKIM > 3) SPF /or/ DKIM > 4) SPF /and/ DKIM No. The standard says DMARC passes if either SPF or DKIM passes with an aligned domain. The only thing the DMARC publisher controls for one or the other or both is whether alignment is strict or relaxed. See my post that I was still typing when this was sent . -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Tue Oct 17 19:26:41 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 16:26:41 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <1508279900.47036.70.camel@fmp.com> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <4f7b8bfb-0e1b-7579-ddbb-f519ae532b0b@tnetconsulting.net> <1508279900.47036.70.camel@fmp.com> Message-ID: <73a074d0-c3d7-150d-8cf8-56fed8395960@msapiro.net> On 10/17/2017 03:38 PM, Lindsay Haisley wrote: > > Any system which REQUIRES DKIM validation to pass is out of compliance > with RFCs, as I understand it. A DKIM signature which doesn't validate > MUST be treated the same as no DKIM signature at all. Actually, it's SHOULD, not MUST in the latest RFC. And, DMARC doesn't exactly REQUIRE DKIM validation to pass. DMARC treats a message with no DKIM signature the same as one with an invalid signature so it is compliant in that sense. > And I don't believe we're dealing with SPF here, just alignment between > the domain of an email's author (From) and the IP address of the system > communicating SMTP server from which the recipient's SMTP server > received the mail. Correct me if I'm wrong on this, but I don't believe > a SPF record in DNS is required. It's not required, but it can enable DMARC to pass IF it passes and the envelope from domain aligns with the From: domain. See . -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fmouse at fmp.com Tue Oct 17 19:38:07 2017 From: fmouse at fmp.com (Lindsay Haisley) Date: Tue, 17 Oct 2017 18:38:07 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: <8d9a4a1c-e632-da8a-c0e3-2e10456d5ab4@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <4f7b8bfb-0e1b-7579-ddbb-f519ae532b0b@tnetconsulting.net> <8d9a4a1c-e632-da8a-c0e3-2e10456d5ab4@msapiro.net> Message-ID: <1508283487.47036.77.camel@fmp.com> On Tue, 2017-10-17 at 16:20 -0700, Mark Sapiro wrote: > See my post that I was still typing when this was sent > html>. > 2) It must pass SPF. SPF works on the domain of the SMTP envelope from. > Thus for SPF to pass, that domain must publish an SPF record specifying > the IP of the sending server as a permitted sender. Further, for DMARC > the envelope from (SPF) domain must align with the From: domain. Again, > in strict mode aligned means equal. In relaxed mode aligned means the > corresponding organizational domains are equal. OK, thanks. This is clear, and useful information. fmp.com publishes a proper SPF record, and with regard to the mail server DMARC mitigation program I wrote for Courier, the envelope sender is "alias at fmp.com", which can possibly be adjusted, but which matches "postmaster at fmp.com" which I'm using for the body From header on munged emails, and on top of this FMP publishes "a mx ptr ip4:198.58.125.221 mx:linode.fmp.com -all" for SPF, which grabs just about everything and should be OK. -- Lindsay Haisley | "The first casualty when FMP Computer Services | war comes is truth." 512-259-1190 | http://www.fmp.com | -- Hiram W Johnson From gtaylor at tnetconsulting.net Tue Oct 17 19:46:14 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 17 Oct 2017 17:46:14 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> Message-ID: <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> On 10/17/2017 03:54 PM, Mark Sapiro wrote: > What I mean is as I posted previously > , > RFC 5322 says the From: contains the "the mailbox(es) of the person(s) > or system(s) responsible for the writing of the message." and munging > the From: to the list address is not compliant with this requirement. ACK That's what I figured you were talking about, but figured I ask instead of assuming. (Long pause to pontificate and research.) I decided to see if there was an update to RFC 5322, and lo and behold there is. RFC 6854, which specifically updates RFC 5322 section 3.6.2 and allows group address syntax exists. TL;DR: From: can now contain a Group address / name, which can zero or one or more mailbox addresses. I feel like RFC 6854 provides some light at the end of the tunnel and allows mailing list managers to modify the From: to be the group, including the Group's address. Sender: is not needed because it would be the same as the Group's from address. Similarly, I found wording in RFC 5322 that indicates that a user agent forwarding a message, is actually a new message. Section 3.6.6 has the following copy: > Note: Reintroducing a message into the transport system and using > resent fields is a different operation from "forwarding". > "Forwarding" has two meanings: One sense of forwarding is that a > mail reading program can be told by a user to forward a copy of a > message to another person, making the forwarded message the body > of the new message. A forwarded message in this sense does not > appear to have come from the original sender, but is an entirely > new message from the forwarder of the message. Forwarding may > also mean that a mail transport program gets a message and > forwards it on to a different destination for final delivery. > Resent header fields are not intended for use with either type of > forwarding. I consider a mailing list manager to be a fancy MUA that automatically forwards in this context. I know that this copy is addressing the Recent-* headers, but I think that it clearly describes that a forwarded message (like an MLM generates) is a new message, and as such should reflect the person ~> entity (read: MLM) that is sending the new message. > In the spirit of DMARC mitigation, we all agree that it is a necessary > evil, at least in some cases, but that doesn't change the fact that it > is an 'evil'. I will concede that modifying the From: header is a questionable technique. - However I think it is done with a white hat spirit. Further, I feel like RFC 6854 helps enable (if not indirectly condone) use of said technique. -- Grant. . . . unix || die From dmaziuk at bmrb.wisc.edu Tue Oct 17 20:00:02 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Tue, 17 Oct 2017 19:00:02 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> Message-ID: <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> On 10/17/2017 05:36 PM, Grant Taylor via Mailman-Users wrote: > /me wonders what color Dimitri's hat is.? ;-)? #knowtheyenemy I've a "tactical foliage green" kufiah, best five bucks I ever spent on an article of clothing. The point was that SPF will flag messages with ineptly spoofed From addresses, and I don't seem to see any of those anymore. As for DKIM, say you proved that the message was altered after the postmaster at yourdomain was done with it. Now what? Depending on how you look at it, the standard says either - now pretend you don't know if it was altered (in your interpretation: "maliciously") or not, or - (in Mark's version) assume anything not signed is malicious and invalid. I strongly dislike either alternative. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From gtaylor at tnetconsulting.net Tue Oct 17 20:04:35 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 17 Oct 2017 18:04:35 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <2121e207-6500-be24-b577-b25ac12a06de@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <2121e207-6500-be24-b577-b25ac12a06de@msapiro.net> Message-ID: <27e3d303-1bd8-cee2-a8d6-de0f60b7b12a@tnetconsulting.net> On 10/17/2017 05:07 PM, Mark Sapiro wrote: > The reference is the DMARC standard RFC 7489 > . I need to go back and re-read that again. > It's more complicated than the above. There is a concept of domain > alignment. Alignment is satisfied in either "strict" or relaxed "mode". > A dmarc policy record may optionally specify either mode for DKIM > alignment or SPF alignment or both with the default being "relaxed. My brain is failing to translate "corresponding organizational domains" to "sub-domains" properly and what that means for strict vs relaxed. > For a message to pass DMARC it must meet 1 of 2 requirements. > > 1) It must possess a valid DKIM signature from a domain aligned with the > From: domain. In strict mode aligned means equal. In relaxed mode > aligned means the corresponding organizational domains are equal. > > or > > 2) It must pass SPF. SPF works on the domain of the SMTP envelope from. > Thus for SPF to pass, that domain must publish an SPF record specifying > the IP of the sending server as a permitted sender. Further, for DMARC > the envelope from (SPF) domain must align with the From: domain. Again, > in strict mode aligned means equal. In relaxed mode aligned means the > corresponding organizational domains are equal. As I was reading this, I realized that I may have conflated DMARC reporting with DMARC pass / fail. > Note that if you are relaying mail, SPF probably will pass for your > server if the envelope from domain is your server, but it won't align > with an unmunged From: domain and if it does align because you didn't > rewrite it, SPF will fail unless the original sending domain publishes > SPF that permits your server as a sender. *nod* > So the bottom line is as an "unaffiliated" relay without munging From:, > SPF will never pass for DMARC and DKIM will only pass if you don't > transform the message in ways that break the From: domain's DKIM signature. I assume that you're talking about the SMTP envelope from and not the From: header. > There is a remote possibility that the originating domain that publishes > a DMARC policy relies on SPF and doesn't DKIM sign the message in which > case, unmumged, relayed mail will almost certainly fail DMARC. I know someone who is doing exactly that, purely for the purpose of receiving the feedback reports. -- Grant. . . . unix || die From gtaylor at tnetconsulting.net Tue Oct 17 20:09:39 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 17 Oct 2017 18:09:39 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> Message-ID: <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> On 10/17/2017 06:00 PM, Dimitri Maziuk wrote: > I've a "tactical foliage green" kufiah, best five bucks I ever spent on > an article of clothing. I like it. > The point was that SPF will flag messages with ineptly spoofed From > addresses, and I don't seem to see any of those anymore. ;-) > As for DKIM, say you proved that the message was altered after the > postmaster at yourdomain was done with it. Now what? Depending on how you > look at it, the standard says either > - now pretend you don't know if it was altered (in your interpretation: > "maliciously") or not, or > - (in Mark's version) assume anything not signed is malicious and invalid. > I strongly dislike either alternative. I personally work under the assumption that: If DKIM signature validates, then I consider the message good. If DKIM signature fails, then there is something wrong with the message, and treat it suspiciously. Read: I increment the spam score. (If the spam score is high enough I reject the message at SMTP time.) If there is no DKIM signature, I continue processing normally. -- Grant. . . . unix || die From mark at msapiro.net Tue Oct 17 20:33:13 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 17:33:13 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <27e3d303-1bd8-cee2-a8d6-de0f60b7b12a@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <2121e207-6500-be24-b577-b25ac12a06de@msapiro.net> <27e3d303-1bd8-cee2-a8d6-de0f60b7b12a@tnetconsulting.net> Message-ID: On 10/17/2017 05:04 PM, Grant Taylor via Mailman-Users wrote: > On 10/17/2017 05:07 PM, Mark Sapiro wrote: > > My brain is failing to translate "corresponding organizational domains" > to "sub-domains" properly and what that means for strict vs relaxed. In another thread on mailman-developers, I discussed organizational domains with Lindsay, so I assumed he knew. In summary, every domain has a corresponding organizational domains which may be the same or a "super" domain. In short, the organizational domain is the domain that might be found in whois. For "common" tlds like .com, .org, net, .edu, etc. the organizational domain is the top two levels. E.g. the organizational domain for some.sub.domain.example.com is example.com, but it's much more complicated than that. See if you want to know more. >> So the bottom line is as an "unaffiliated" relay without munging From:, >> SPF will never pass for DMARC and DKIM will only pass if you don't >> transform the message in ways that break the From: domain's DKIM >> signature. > > I assume that you're talking about the SMTP envelope from and not the > From: header. No. I could have slipped, but when I write From: domain, I mean the domain of the address in the From: header (That's what DMARC is all about - verifying that the message actually came from the domain of the address in the From: header). If I mean the domain of the envelope from, I try to use that phrase, but in the context of DMARC that domain is only relevant for SPF and only if it "aligns" with the From: domain. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From dandrews at visi.com Tue Oct 17 20:32:48 2017 From: dandrews at visi.com (David Andrews) Date: Tue, 17 Oct 2017 19:32:48 -0500 Subject: [Mailman-Users] Mail Delivery Message-ID: At one time I set Reply-To: header munging under general settings to Yes. Some of my users used a screen reader that balked unless the header was munged, for some reason. Well that software has gone away, and ISP's are much pickier these days, with MARC and dkim and SPF etc. Would this setting cause me delivery problems ?? Should I go back and change it on older lists. I no longer set it to yes, leave it at no, its default. Dave --- This email has been checked for viruses by AVG. http://www.avg.com From mark at msapiro.net Tue Oct 17 20:56:02 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 17:56:02 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: On 10/17/2017 04:46 PM, Grant Taylor via Mailman-Users wrote: > > I decided to see if there was an update to RFC 5322, and lo and behold > there is.? RFC 6854, which specifically updates RFC 5322 section 3.6.2 > and allows group address syntax exists. > > TL;DR:? From: can now contain a Group address / name, which can zero or > one or more mailbox addresses. > > I feel like RFC 6854 provides some light at the end of the tunnel and > allows mailing list managers to modify the From: to be the group, > including the Group's address. Group address syntax is something else. it is a specific syntax which is a name followed literally by a colon followed by a list of zero or more mailboxes (email addresses) and terminated by a semicolon. E.g., from the RFC Second, consider an email message that is meant to be "from" the two managing partners of a business, Ben and Carol, and that is sent by their assistant, Dave. This message could always have been presented this way: From: ben at example.com,carol at example.com Sender: dave at example.com This change allows it to be represented this way: From: Managing Partners:ben at example.com,carol at example.com; Sender: dave at example.com The group syntax has always been allowed in To: and some other headers. RFC 6854 just extends it to From: This is most commonly seen with some MUAs when all the recipients are Bccs, the message is To: undisclosed recipients:; > Sender: is not needed because it would be the same as the Group's from > address. There's no such thing as a group's address unless the addresses are listed along with the group name. Anyway, using a group name alone as From: avoids DMARC as there is no From: address domain for a DMARC lookup. > Similarly, I found wording in RFC 5322 that indicates that a user agent > forwarding a message, is actually a new message.? Section 3.6.6 has the > following copy: > >> ????Note: Reintroducing a message into the transport system and using >> resent fields is a different operation from "forwarding". >> ?"Forwarding" has two meanings: One sense of forwarding is that a mail >> reading program can be told by a user to forward a copy of a message >> to another person, making the forwarded message the body of the new >> message.? A forwarded message in this sense does not appear to have >> come from the original sender, but is an entirely new message from the >> forwarder of the message.? Forwarding may also mean that a mail >> transport program gets a message and forwards it on to a different >> destination for final delivery. ?Resent header fields are not intended >> for use with either type of forwarding. That type of forwarding is exactly what is done by Mailman's DMARC Wrap Message action and that is the reason that action exists. Because in that case the list message is RFC 5322 compliant. However many MUAs, particularly mobile apps, have difficulty rendering such a message in a good way, so Wrap Message isn't always the best option. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Tue Oct 17 21:23:53 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 18:23:53 -0700 Subject: [Mailman-Users] Mail Delivery In-Reply-To: References: Message-ID: <74d61f0d-276d-d7f5-8204-6cb54897a9ec@msapiro.net> On 10/17/2017 05:32 PM, David Andrews wrote: > At one time I set > Reply-To: header munging > under general settings to Yes. Some of my users used a screen reader > that balked unless the header was munged, for some reason.? Well that > software has gone away, and ISP's are much pickier these days, with MARC > and dkim and SPF etc. Would this setting cause me delivery problems ?? > Should I go back and change it on older lists. I no longer set it to > yes, leave it at no, its default. By "set Reply-To: header munging under general settings to Yes" I assume you mean reply_goes_to_list = This List, but I'm not sure what you're asking. Reply-To: header munging is controversial and is a religious war. Mailman developers think it shouldn't be done, but many think it should be which is why the option exists. I am not aware of message delivery issues one way or the other, but there is an issue with Thunderbird and possibly other MUAs. Recent T'bird has changed so that if you are looking at a list post with a Reply-To: to the list and you do a simple reply or control-R, the reply will be addressed to the From: and not the list. More recent T'bird has a config editor option to restore the old behavior, but it's not the default. See these threads for all the gory details: -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fmouse at fmp.com Tue Oct 17 21:28:08 2017 From: fmouse at fmp.com (Lindsay Haisley) Date: Tue, 17 Oct 2017 20:28:08 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <2121e207-6500-be24-b577-b25ac12a06de@msapiro.net> <27e3d303-1bd8-cee2-a8d6-de0f60b7b12a@tnetconsulting.net> Message-ID: <1508290088.47036.105.camel@fmp.com> On Tue, 2017-10-17 at 17:33 -0700, Mark Sapiro wrote: > In another thread on mailman-developers, I discussed organizational > domains with Lindsay, so I assumed he knew. Yes, technically I know, but this kind of stuff makes my head hurt and my hats to change colors, so I fall back on "If it works, don't fix it". The pieces I pulled out of MM code work, and I've set up a cron job to pull the org domains db to a local server where it comes up fast, but with everything I'm doing, learning how the cow eats the cabbages in this kind of thing is pretty much on a need-to-know basis. -- Lindsay Haisley | "The first casualty when FMP Computer Services | war comes is truth." 512-259-1190 | http://www.fmp.com | -- Hiram W Johnson From mark at msapiro.net Tue Oct 17 21:35:41 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Oct 2017 18:35:41 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <1508290088.47036.105.camel@fmp.com> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <2121e207-6500-be24-b577-b25ac12a06de@msapiro.net> <27e3d303-1bd8-cee2-a8d6-de0f60b7b12a@tnetconsulting.net> <1508290088.47036.105.camel@fmp.com> Message-ID: On 10/17/2017 06:28 PM, Lindsay Haisley wrote: > > Yes, technically I know, but this kind of stuff makes my head hurt and > my hats to change colors, so I fall back on "If it works, don't fix > it". I hear that and I feel your pain. Somehow it was all simpler when I was younger, and I don't think it was just because I was younger. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From dandrews at visi.com Tue Oct 17 22:18:00 2017 From: dandrews at visi.com (David Andrews) Date: Tue, 17 Oct 2017 21:18:00 -0500 Subject: [Mailman-Users] Mail Delivery In-Reply-To: <74d61f0d-276d-d7f5-8204-6cb54897a9ec@msapiro.net> References: <74d61f0d-276d-d7f5-8204-6cb54897a9ec@msapiro.net> Message-ID: At 08:23 PM 10/17/2017, Mark Sapiro wrote: >On 10/17/2017 05:32 PM, David Andrews wrote: > >At one time I set > Reply-To: header munging > >under general settings to Yes. Some of my users >used a screen reader > that balked unless the >header was munged, for some reason.? Well >that > software has gone away, and ISP's are >much pickier these days, with MARC > and dkim >and SPF etc. Would this setting cause me >delivery problems ?? > Should I go back and >change it on older lists. I no longer set it >to > yes, leave it at no, its default. By "set >Reply-To: header munging under general settings >to Yes" I assume you mean reply_goes_to_list = >This List, but I'm not sure what you're asking. >Reply-To: header munging is controversial and is >a religious war. Mailman developers think it >shouldn't be done, but many think it should be >which is why the option exists. I am not aware >of message delivery issues one way or the other, >but there is an issue with Thunderbird and >possibly other MUAs. Recent T'bird has changed >so that if you are looking at a list post with a >Reply-To: to the list and you do a simple reply >or control-R, the reply will be addressed to the >From: and not the list. More recent T'bird has a >config editor option to restore the old >behavior, but it's not the default. See these >threads for all the gory details: > > > >-- Mark Sapiro The >highway is for gamblers, San Francisco Bay Area, California I didn't mean "reply goes to list" but you answered my question, as usual! Thanks! Dave --- This email has been checked for viruses by AVG. http://www.avg.com From turnbull.stephen.fw at u.tsukuba.ac.jp Tue Oct 17 23:38:59 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Wed, 18 Oct 2017 12:38:59 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <2121e207-6500-be24-b577-b25ac12a06de@msapiro.net> <27e3d303-1bd8-cee2-a8d6-de0f60b7b12a@tnetconsulting.net> <1508290088.47036.105.camel@fmp.com> Message-ID: <23014.52435.168479.518346@turnbull.sk.tsukuba.ac.jp> This whole thread reminds me of an evangelical arguing with a Jesuit. 2000 years of Bible study does make for strong debating! Please note that the Sender/From distinction *and* the semantic interpretations of those fields go back to RFC 733 (1977!) at least, and the Society of Jesus, er, IETF has refused to change those semantics on three occasions separated by about a decade apiece (RFCs 822, 2822 = STD 11 IIRC, and 5322). There are strong reasons, founded in human behavior, for this standard. Mark Sapiro writes: > On 10/17/2017 06:28 PM, Lindsay Haisley wrote: > > > > Yes, technically I know, but this kind of stuff makes my head hurt and > > my hats to change colors, so I fall back on "If it works, don't fix > > it". > > > I hear that and I feel your pain. Somehow it was all simpler when I was > younger, and I don't think it was just because I was younger. A big 10-4 and PLOS 1 to everything you say, Mark! Steve From fmouse at fmp.com Tue Oct 17 23:47:45 2017 From: fmouse at fmp.com (Lindsay Haisley) Date: Tue, 17 Oct 2017 22:47:45 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: <23014.52435.168479.518346@turnbull.sk.tsukuba.ac.jp> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <1508278500.47036.64.camel@fmp.com> <2121e207-6500-be24-b577-b25ac12a06de@msapiro.net> <27e3d303-1bd8-cee2-a8d6-de0f60b7b12a@tnetconsulting.net> <1508290088.47036.105.camel@fmp.com> <23014.52435.168479.518346@turnbull.sk.tsukuba.ac.jp> Message-ID: <1508298465.47036.136.camel@fmp.com> On Wed, 2017-10-18 at 12:38 +0900, Stephen J. Turnbull wrote: > This whole thread reminds me of an evangelical arguing with a Jesuit. > 2000 years of Bible study does make for strong debating! Welll.... ?Spending much time reading RFCs can certainly put one in a biblical frame of mind ;) ?Lots of SHOULD, MUST and MAY therein. -- Lindsay Haisley | "The first casualty when FMP Computer Services | war comes is truth." 512-259-1190 | http://www.fmp.com | -- Hiram W Johnson From dmaziuk at bmrb.wisc.edu Wed Oct 18 11:18:28 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Wed, 18 Oct 2017 10:18:28 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> Message-ID: On 2017-10-17 19:09, Grant Taylor via Mailman-Users wrote: > If DKIM signature fails, then there is something wrong with the message, > and treat it suspiciously.? Read:? I increment the spam score.? (If the > spam score is high enough I reject the message at SMTP time.) > > If there is no DKIM signature, I continue processing normally. Then you seem to misunderstand what crypto signatures actually do. If signature check fails, then the message is not what its author actually wrote. IRL it's mainly SorceForge and the like injecting its ads into signed parts, (and the real reason google is pushing https and dkim so hard is it's messing with their ad revenue,) but in principle if the check fails the message *content* is *invalid*. Whoever the author and whatever the content. Dimitri From gtaylor at tnetconsulting.net Wed Oct 18 12:31:05 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 18 Oct 2017 10:31:05 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: I didn't completely follow all of your message. I think we may have been talking past each other. On 10/17/2017 06:56 PM, Mark Sapiro wrote: > There's no such thing as a group's address unless the addresses are > listed along with the group name. Um.... My interpretation of 6854 ? 1 and ? 4 makes me think that an empty group list is perfectly acceptable. Further, the group list can be non-empty and contain the lists posting address. > Anyway, using a group name alone as From: avoids DMARC as there is no > From: address domain for a DMARC lookup. Agreed. I would rather do something like the following so that users could reply to the message. (It would also avoid potential MUA issues as indicated by RFC 6854.) I would think that it would be acceptable to use a From "group address" that is the mailing list. I.e. From: Mailman Users:mailman-users at python.org; Possibly even something like the following: From: Grant via Mailman Users:mailman-users at python.org; Arguably, this is conceptually very similar to what has become the defacto method to deal with DMARC today by munging the From: From: Grant via Mailman Users The difference is that RFC 6854 codifies that there are times to alter the from. - At least that's how I'm interpreting this. Further, if you believe the fact that the outbound message is indeed a completely new message (as I do) then it's completely legit to set the from to what ever you want. ():-) > That type of forwarding is exactly what is done by Mailman's DMARC Wrap > Message action and that is the reason that action exists. Because in > that case the list message is RFC 5322 compliant. However many MUAs, > particularly mobile apps, have difficulty rendering such a message in a > good way, so Wrap Message isn't always the best option. It sounds like you're talking about message/rfc822 message attachments. - That is a viable option. However I see no reason that you can't take the body copy from the incoming email and use it directly in the new outgoing email. No need to message/rfc822 wrap (or other digest like raping) the outgoing message. -- Grant. . . . unix || die From gtaylor at tnetconsulting.net Wed Oct 18 12:37:08 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 18 Oct 2017 10:37:08 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> Message-ID: On 10/18/2017 09:18 AM, Dimitri Maziuk wrote: > Then you seem to misunderstand what crypto signatures actually do. I believe I understand what the crypto signatures actually do. We are each entitled to decide what to actually do based on the result of the crypto signature (in)validity. > If signature check fails, then the message is not what its author > actually wrote. IRL it's mainly SorceForge and the like injecting its > ads into signed parts, (and the real reason google is pushing https and > dkim so hard is it's messing with their ad revenue,) but in principle if > the check fails the message *content* is *invalid*. Whoever the author > and whatever the content. I believe I remember (but can't point to) something in the DKIM spec that referenced the possibility that the DKIM signature could be broken by things as benign as an MTA doing a content transfer encoding conversion. - I have personally seen this. As such, you can't be 100% positive that the message content's meaning / copy has actually changed, just that something about the message has changed. - Thus it is advised to only treat valid signatures as a good thing and be cautious of treating invalid signatures as a bad thing. I use DKIM validity as a signal that I then make decisions based on. - Hence why I have chosen to alter spam score on my mail server based on the DKIM result. -- Grant. . . . unix || die From mark at msapiro.net Wed Oct 18 13:50:24 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Oct 2017 10:50:24 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: On 10/18/2017 09:31 AM, Grant Taylor via Mailman-Users wrote: > > Um....? My interpretation of 6854 ? 1 and ? 4 makes me think that an > empty group list is perfectly acceptable.? Further, the group list can > be non-empty and contain the lists posting address. True, but in either case it still does not represent the "author" of the message. > I would rather do something like the following so that users could reply > to the message.? (It would also avoid potential MUA issues as indicated > by RFC 6854.) > > I would think that it would be acceptable to use a From "group address" > that is the mailing list.? I.e. > > ?? From: Mailman Users:mailman-users at python.org; > > Possibly even something like the following: > > ?? From:? Grant via Mailman Users:mailman-users at python.org; > > Arguably, this is conceptually very similar to what has become the > defacto method to deal with DMARC today by munging the From: > > ?? From:? Grant via Mailman Users > > The difference is that RFC 6854 codifies that there are times to alter > the from.? -? At least that's how I'm interpreting this. That's where you are wrong. All RFC 6854 does is allow the "group" syntax to be used as the content of the From: header. It does not change the RFC 5322 at al requirement that the From: header represent the author(s) of the message. > Further, if you believe the fact that the outbound message is indeed a > completely new message (as I do) then it's completely legit to set the > from to what ever you want.? ():-) This is the crux of our disagreement. The outbound message is still the original author's message, albeit slightly altered by subject prefixing, content filtering and/or other transformations to conform with list policies. I don't agree that it is a completely new message. I think it is still the original message with only technical and formatting changes. >> That type of forwarding is exactly what is done by Mailman's DMARC Wrap >> Message action and that is the reason that action exists. Because in >> that case the list message is RFC 5322 compliant. However many MUAs, >> particularly mobile apps, have difficulty rendering such a message in a >> good way, so Wrap Message isn't always the best option. > It sounds like you're talking about message/rfc822 message attachments. > -? That is a viable option. > > However I see no reason that you can't take the body copy from the > incoming email and use it directly in the new outgoing email.? No need > to message/rfc822 wrap (or other digest like raping) the outgoing message. The difference is wrapping the message preserves the original message's headers (particularly From:) and makes it the content of another message which says essentially "here's the message the list received". That outer message can be From: the list and still be standards compliant. However, if you are just sending the body of the original message From: the list, according to RFC 5322 et al, you are saying the list is the author of that message body. This is not true and is why I say the message is not compliant with RFC 5322 et al. Granted, all things considered, this is what most of us choose to do. I'm not saying this shouldn't be done. It is something we are forced to do because certain freemail providers choose to publish DMARC p=reject policies contrary to the original intent of DMARC, but all I'm saying is we should not forget that when we do this, we are sending messages that are not strictly standards compliant. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From dmaziuk at bmrb.wisc.edu Wed Oct 18 13:51:35 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Wed, 18 Oct 2017 12:51:35 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> Message-ID: <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> On 10/18/2017 11:37 AM, Grant Taylor via Mailman-Users wrote: > I believe I remember (but can't point to) something in the DKIM spec > that referenced the possibility that the DKIM signature could be broken > by things as benign as an MTA doing a content transfer encoding > conversion.? -? I have personally seen this. Like tn?t??nsulting.n?t being a benign minor encoding change in a couple of characters? Just because the authors of the RFC have also chosen to stick the square peg in the round hole doesn't make the hole any less round, nor the peg any less square. Somewhere I've a 10-year old e-mail from Whit Diffie explaining how SSL was a PR solution to a marketing problem. So this kind of problem-finding and problem-solving has made to SMTP RFCs now, colour me shocked. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From gtaylor at tnetconsulting.net Wed Oct 18 14:14:35 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 18 Oct 2017 12:14:35 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: On 10/18/2017 11:50 AM, Mark Sapiro wrote: ... > This is the crux of our disagreement. The outbound message is still the > original author's message, albeit slightly altered by subject prefixing, > content filtering and/or other transformations to conform with list > policies. I don't agree that it is a completely new message. I think it > is still the original message with only technical and formatting changes. I feel we have reached an impasse and we must agree to disagree. > The difference is wrapping the message preserves the original message's > headers (particularly From:) and makes it the content of another message > which says essentially "here's the message the list received". That > outer message can be From: the list and still be standards compliant. Agreed. > However, if you are just sending the body of the original message From: > the list, according to RFC 5322 et al, you are saying the list is the > author of that message body. This is not true and is why I say the > message is not compliant with RFC 5322 et al. I believe we are each entitled to our own opinions. ;-) > Granted, all things considered, this is what most of us choose to do. > I'm not saying this shouldn't be done. It is something we are forced to > do because certain freemail providers choose to publish DMARC p=reject > policies contrary to the original intent of DMARC, but all I'm saying is > we should not forget that when we do this, we are sending messages that > are not strictly standards compliant. I think it will be interesting to see what happens as more and more domains adopt DMARC, including those that use p=reject. Especially with some of governmental institutions purportedly being mandated to use DMARC. - IMHO, DMARC is going to eventually become the new norm. I also wonder what ARC is going to do to this paradigm. -- Grant. . . . unix || die From gtaylor at tnetconsulting.net Wed Oct 18 14:30:10 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 18 Oct 2017 12:30:10 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> Message-ID: On 10/18/2017 11:51 AM, Dimitri Maziuk wrote: > Like tn?t??nsulting.n?t being a benign minor encoding change in a couple > of characters? No. That is not a simple content encoding change. Content (re)encoding changes the representation of the same encoded data. 1077, Hex 0435, Octal 2065 != 101, Hex 65, Octal 145 1089, Hex 0441, Octal 2101 != 99, Hex 63, Octal 143 1086, Hex 043e, Octal 2076 != 111, Hex 6f, Octal 157 An MTA changing the encoding method of data to / from: base 64 / quoted-printable / 8-bit, is distinctly different than what you have done, which is changing actual encoded data. The (decimal) number 17 can be encoded multiple ways: 10001 = binary base 2 25 = hex base 6 21 = octal base 8 17 = decimal base 10 11 = hexadecimal base 16 All five encoded numbers represent the same value (decimal) 17. What you have done (in the spirit of a white hat) is actually a homograph attack. Something quite different from simple encoding differences. Quite similar to a computer seeing a the following three characters as quite distinctly different things, each with different computational meanings. 0 O o > Just because the authors of the RFC have also chosen to stick the square > peg in the round hole doesn't make the hole any less round, nor the peg > any less square. Fair. > Somewhere I've a 10-year old e-mail from Whit Diffie explaining how SSL > was a PR solution to a marketing problem. So this kind of > problem-finding and problem-solving has made to SMTP RFCs now, colour me > shocked. I'd be curious to read said email, if it's convenient to dig up. -- Grant. . . . unix || die From mark at msapiro.net Wed Oct 18 14:35:04 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Oct 2017 11:35:04 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: <68797a95-beff-3c06-cd07-3f589adcd193@msapiro.net> On 10/18/2017 11:14 AM, Grant Taylor via Mailman-Users wrote: > > I think it will be interesting to see what happens as more and more > domains adopt DMARC, including those that use p=reject.? Especially with > some of governmental institutions purportedly being mandated to use > DMARC.? -? IMHO, DMARC is going to eventually become the new norm. DMARC is not the problem. It is perfectly reasonable for say, irs.gov to publish DMARC p=reject as long as mail From: irs.gov is not an employees personal post to an email list. Presumably the IRS would have rules against that. The problem is when general ESPs that provide addresses in their domain for anyone to use for any personal purpose publish DMARC p=reject. > I also wonder what ARC is going to do to this paradigm. ARC has the potential to help. When say a yahoo.com user posts to a list on my server and the list sends the post to a hotmail.com user, ARC allows me to certify that Yahoo's DKIM signature was valid when I received the mail, then I broke the sig but resigned the mail with my domain's sig and sent it on to Hotmail. Now there is a chain by which Hotmail can verify my sig and the fact that I certify Yahoo's sig. The crux however is Hotmail has to trust me. Now if I'm GoogleGroups, Hotmail will probably trust me but if I'm mail.python.org there might be a mechanism by which I can ask Hotmail and every other ISP to trust me, but is that going to work in practice. I think that remains to be seen. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From gtaylor at tnetconsulting.net Wed Oct 18 14:57:55 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 18 Oct 2017 12:57:55 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <68797a95-beff-3c06-cd07-3f589adcd193@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> <68797a95-beff-3c06-cd07-3f589adcd193@msapiro.net> Message-ID: On 10/18/2017 12:35 PM, Mark Sapiro wrote: > DMARC is not the problem. It is perfectly reasonable for say, irs.gov to > publish DMARC p=reject as long as mail From: irs.gov is not an > employees personal post to an email list. Presumably the IRS would have > rules against that. > > The problem is when general ESPs that provide addresses in their domain > for anyone to use for any personal purpose publish DMARC p=reject. I question what the fine line distinction will be for what domains can / should use DMARC (or the next disrupting technology). Further, I question why domains that don't qualify, should be excluded from using said technology. I suspect that we should also agree to disagree on this. > ARC has the potential to help. When say a yahoo.com user posts to a list > on my server and the list sends the post to a hotmail.com user, ARC > allows me to certify that Yahoo's DKIM signature was valid when I > received the mail, then I broke the sig but resigned the mail with my > domain's sig and sent it on to Hotmail. Now there is a chain by which > Hotmail can verify my sig and the fact that I certify Yahoo's sig. The > crux however is Hotmail has to trust me. Now if I'm GoogleGroups, > Hotmail will probably trust me but if I'm mail.python.org there might be > a mechanism by which I can ask Hotmail and every other ISP to trust me, > but is that going to work in practice. I think that remains to be seen. It sounds like you have the same concern / unknown that I do. What do I need to do to get to trust my ARC signature. - Is ARC overloading my published DKIM key without clearly stating that it's using it? Or is there something else that I'm not aware of? Or is it simply a white list, or trust list, type issue. If it's the latter, I feel like ARC has a design flaw before it even gets out of the gate. I hope that's not the case. -- Grant. . . . unix || die From dmaziuk at bmrb.wisc.edu Wed Oct 18 15:07:56 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Wed, 18 Oct 2017 14:07:56 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> Message-ID: <9f7b28e3-6a5d-57ab-4390-242b4367f453@bmrb.wisc.edu> On 10/18/2017 01:30 PM, Grant Taylor via Mailman-Users wrote: > The (decimal) number 17 can be encoded multiple ways: > > 10001 = binary????? base? 2 > ?? 25 = hex???????? base? 6 > ?? 21 = octal?????? base? 8 > ?? 17 = decimal???? base 10 > ?? 11 = hexadecimal base 16 > > All five encoded numbers represent the same value (decimal) 17. 17 == 0x11. "17" != "0x11". Which was precisely the point: if your MTA, say, does unicodedata.normalize( 'NFKD' ... ), and turns u-umlaut into a regular "u", you may consider it benign. Many won't. Most importantly, crypto signature will change, and DKIM check will fail. Benign is in the eye of the beholder. We're inserting this stuff into a database where a search for "Wutrich" will find neither "W?trich" nor "W\u0308trich" so I wouldn't consider it benign at all. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From gtaylor at tnetconsulting.net Wed Oct 18 15:32:48 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 18 Oct 2017 13:32:48 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <9f7b28e3-6a5d-57ab-4390-242b4367f453@bmrb.wisc.edu> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> <9f7b28e3-6a5d-57ab-4390-242b4367f453@bmrb.wisc.edu> Message-ID: On 10/18/2017 01:07 PM, Dimitri Maziuk wrote: > 17 == 0x11. "17" != "0x11". Which was precisely the point: if your MTA, > say, does unicodedata.normalize( 'NFKD' ... ), and turns u-umlaut into a > regular "u", you may consider it benign. Many won't. I would not consider that benign at all. I'm referring to the difference between: - ? - ASCII (?) - =C3=BC - quoted-printable - w7w= - base 64 - ü - HTML All four representations are for the *same* letter / character / glyph / byte(s). I consider those to be (effectively) benign content encoding changes. - Note the content is the same, with the only difference being how it's encoded. > Most importantly, crypto signature will change, and DKIM check will fail. DKIM, by design will fail if anything that is signed changes. (See the ROPEMAKER attack for a better explanation about anything signed.) > Benign is in the eye of the beholder. ~eh~ ... Okay. > We're inserting this stuff into a > database where a search for "Wutrich" will find neither "W?trich" nor > "W\u0308trich" so I wouldn't consider it benign at all. I do not consider "Wutrich" and "W?trich" to be the same string. The former may be considered a poor representation of the latter. I'm not sure which Unicode code point 308 is, but I doubt that it is the same as 252, Hex 00fc, Octal 374. (I would have to look it up to know for sure.) I would hope that data would be normalized to the same encoding in the database. I.e. "=C3=BC" (quoted-printable) would be normalized to "?" and stored in the database as such. I would further hope that any search of the database would be able to do something like a character class (type) search so that it could match on "W[?u]trich". (Adjust as necessary.) -- Grant. . . . unix || die From dmaziuk at bmrb.wisc.edu Wed Oct 18 16:10:19 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Wed, 18 Oct 2017 15:10:19 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> <9f7b28e3-6a5d-57ab-4390-242b4367f453@bmrb.wisc.edu> Message-ID: <2d063de0-5bd3-f4d3-0904-f42a1319444f@bmrb.wisc.edu> On 10/18/2017 02:32 PM, Grant Taylor via Mailman-Users wrote: > I'm referring to the difference between: > > ?- ?????? - ASCII (?) > ?- =C3=BC - quoted-printable > ?- w7w=?? - base 64 > ?- ü - HTML > > All four representations are for the *same* letter / character / glyph / > byte(s). They are different ASCII representations of the same byte, yes. They are not the same text. Sign the text, re-encode text and signature together, anyone who cares about it can decode it back to where the signature will match. Only, you can't do that on the MX, it has to be done on the client. > DKIM, by design will fail if anything that is signed changes. DKIM is designed to produce false positives. Which means DKIM-based tests will have low specificity (https://en.wikipedia.org/wiki/Sensitivity_and_specificity). Which makes them bad for detecting spam. But that's OK, DMARC in general is for *fraudulent* e-mail, not *unsolicited* e-mail. I'm sure once I'm plagued by *fraudulent* e-mail, I'll start caring about RFC 7489 and the rest of them. When those e-mail are from mailman I'll start caring about what mailman does with DMARC headers. But at this point I'd just strip them all off. (And since I'm tripping down the memory lane: https://catless.ncl.ac.uk/Risks/23/21#subj9.1) -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From gtaylor at tnetconsulting.net Wed Oct 18 17:26:13 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 18 Oct 2017 15:26:13 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <2d063de0-5bd3-f4d3-0904-f42a1319444f@bmrb.wisc.edu> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> <9f7b28e3-6a5d-57ab-4390-242b4367f453@bmrb.wisc.edu> <2d063de0-5bd3-f4d3-0904-f42a1319444f@bmrb.wisc.edu> Message-ID: <7e323654-83b1-0940-48b0-7adc6bb992a7@tnetconsulting.net> On 10/18/2017 02:10 PM, Dimitri Maziuk wrote: > They are different ASCII representations of the same byte, yes. They are > not the same text. Hum. I wonder if we have been talking about slightly different things. I've been referring to "?" being displayed the same in MUAs which is interpreting the different underlying text in the various content transfer encodings. > Sign the text, re-encode text and signature together, > anyone who cares about it can decode it back to where the signature will > match. Do I understand you correctly to mean to create the signature before applying transport encoding? > Only, you can't do that on the MX, it has to be done on the client. Why can't you do it at the MX? Or do you mean that it's inefficient to do so at the MX? > DKIM is designed to produce false positives. Which means DKIM-based > tests will have low specificity > (https://en.wikipedia.org/wiki/Sensitivity_and_specificity). My experience ~> opinion, save for mailing lists, differs. In fact, most of the email that I receive passes DKIM. > Which makes > them bad for detecting spam. But that's OK, DMARC in general is for > *fraudulent* e-mail, not *unsolicited* e-mail. I don't think DKIM (or SPF or DMARC) have /anything/ to do with spam detection. SPF is for envelope sender authorization. DKIM is for message integrity. DMARC is for policy and reporting. None of that has anything to do with spam detection / filtering. In fact, I've found that spammers (worth their salt) tend to be early adopters of email technology. Thus they are quite likely to send spam that passes SPF and DKIM and DMARC. > I'm sure once I'm plagued by *fraudulent* e-mail, I'll start caring > about RFC 7489 and the rest of them. I started caring about SPF / DKIM / DMARC for a couple of reasons: 1) I'm pedantic and want to have the best filtering / security that I possibly can on my personal domain. 2) I was seeing blow back from mailing lists about DKIM and / or DMARC. Thus I dug in more and learned more. To each his / her own motivation (or lack there of.) > When those e-mail are from mailman > I'll start caring about what mailman does with DMARC headers. But at > this point I'd just strip them all off. I suspect that when (if) you care will be after you implement filtering (Chicken / Egg?) that possibly rejects messages from mailing lists. Or possibly if your messages with enhanced security cause others to have a problem. (Again with the chicken & egg.) > (And since I'm tripping down the memory lane: > https://catless.ncl.ac.uk/Risks/23/21#subj9.1) :-P -- Grant. . . . unix || die From dmaziuk at bmrb.wisc.edu Wed Oct 18 17:42:40 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Wed, 18 Oct 2017 16:42:40 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: <7e323654-83b1-0940-48b0-7adc6bb992a7@tnetconsulting.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> <9f7b28e3-6a5d-57ab-4390-242b4367f453@bmrb.wisc.edu> <2d063de0-5bd3-f4d3-0904-f42a1319444f@bmrb.wisc.edu> <7e323654-83b1-0940-48b0-7adc6bb992a7@tnetconsulting.net> Message-ID: <117e3a93-8ec1-e421-1110-86ebf7b626e2@bmrb.wisc.edu> On 10/18/2017 04:26 PM, Grant Taylor via Mailman-Users wrote: > On 10/18/2017 02:10 PM, Dimitri Maziuk wrote: > > Do I understand you correctly to mean to create the signature before > applying transport encoding? > >> Only, you can't do that on the MX, it has to be done on the client. > > Why can't you do it at the MX? Because the very first $relayhost may apply transport encoding. You have to compute the hash before that happens. >> DKIM is designed to produce false positives. Which means DKIM-based >> tests will have low specificity >> (https://en.wikipedia.org/wiki/Sensitivity_and_specificity). > > My experience ~> opinion, save for mailing lists, differs.? In fact, > most of the email that I receive passes DKIM. That does not contradict what I said. Low specificity means low probability of detection of "bad stuff". I.e. it doesn't mean much that most of it passes. > I don't think DKIM (or SPF or DMARC) have /anything/ to do with spam > detection.? SPF is for envelope sender authorization.? DKIM is for > message integrity.? DMARC is for policy and reporting.? None of that has > anything to do with spam detection / filtering. Ohkay, so what exactly am I the end user is supposed to need it for? -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: OpenPGP digital signature URL: From gtaylor at tnetconsulting.net Wed Oct 18 18:38:52 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 18 Oct 2017 16:38:52 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <117e3a93-8ec1-e421-1110-86ebf7b626e2@bmrb.wisc.edu> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> <9f7b28e3-6a5d-57ab-4390-242b4367f453@bmrb.wisc.edu> <2d063de0-5bd3-f4d3-0904-f42a1319444f@bmrb.wisc.edu> <7e323654-83b1-0940-48b0-7adc6bb992a7@tnetconsulting.net> <117e3a93-8ec1-e421-1110-86ebf7b626e2@bmrb.wisc.edu> Message-ID: <90063269-47d8-123e-8917-3c204a3acc5f@tnetconsulting.net> On 10/18/2017 03:42 PM, Dimitri Maziuk wrote: > Because the very first $relayhost may apply transport encoding. You have > to compute the hash before that happens. It's my understanding that DKIM is usually applied by the egress MSA / MTA. I guess an MSA could apply DKIM itself. It would need to publish it's public key / selector in DNS. So that's probably a reason not to have every MUA apply DKIM itself. It is probably much more economical to apply DKIM at the MSA / 1st MTA. Ideally intermediary MTAs / receiving MTA would not need to apply content transfer encoding. It's my understanding that MTAs prefer to avoid changing the message unless there is a requirement to do so. I.e. downstream MTA won't accept the message as it currently is. My "why can't you..." question was more why can't an MX do an operation that an MUA can do. - I was thinking you were saying that a receiving MTA couldn't validate before accepting a message. > That does not contradict what I said. Low specificity means low > probability of detection of "bad stuff". I.e. it doesn't mean much that > most of it passes. > Ohkay, so what exactly am I the end user is supposed to need it for? I don't know that DKIM is really targeting end users. I think DKIM is more targeting postmasters to configure on their MTAs. I'm using a Thunderbird add-on that allows me to see / validate DKIM in my receiving MUA. (My MSA applies DKIM for me.) I, as a postmaster, want DKIM for a couple of reasons, 1) I want to be able to filter incoming messages based on DKIM (for better or worse) and 2) outgoing DKIM signing for use in conjunction with DMARC. You (/me waves hands around the room) may not care enough to bother with DKIM. That's your prerogative. Just like we are all free to run our mail servers that way that we want to. -- Grant. . . . unix || die From mailman at jordan.maileater.net Wed Oct 18 18:41:29 2017 From: mailman at jordan.maileater.net (Jordan Brown) Date: Wed, 18 Oct 2017 15:41:29 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <68797a95-beff-3c06-cd07-3f589adcd193@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> <68797a95-beff-3c06-cd07-3f589adcd193@msapiro.net> Message-ID: <8872d3d1-6671-f164-da85-90cc9932a014@maileater.net> On 10/18/2017 11:35 AM, Mark Sapiro wrote: > DMARC is not the problem. It is perfectly reasonable for say, irs.gov > to publish DMARC p=reject as long as mail From: irs.gov is not an > employees personal post to an email list. Presumably the IRS would > have rules against that. Would they?? Shouldn't IRS sysadmins who use Mailman in the course of their jobs send messages to this mailing list using their @irs.gov addresses? Not all submissions to public mailing lists are personal use. From mark at msapiro.net Wed Oct 18 19:10:02 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Oct 2017 16:10:02 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <8872d3d1-6671-f164-da85-90cc9932a014@maileater.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> <68797a95-beff-3c06-cd07-3f589adcd193@msapiro.net> <8872d3d1-6671-f164-da85-90cc9932a014@maileater.net> Message-ID: <7274c22c-3a11-28c6-9b79-90b214990277@msapiro.net> On 10/18/2017 03:41 PM, Jordan Brown wrote: > On 10/18/2017 11:35 AM, Mark Sapiro wrote: >> DMARC is not the problem. It is perfectly reasonable for say, irs.gov >> to publish DMARC p=reject as long as mail From: irs.gov is not an >> employees personal post to an email list. Presumably the IRS would >> have rules against that. > > Would they?? Shouldn't IRS sysadmins who use Mailman in the course of > their jobs send messages to this mailing list using their @irs.gov > addresses? > > Not all submissions to public mailing lists are personal use. Agreed, but for DMARC to work seamlessly with pre-existing accepted norms, DMARC policies of reject or quarantine should only be published for domains that send "official" mail directly to end recipients. If irs.gov published such a policy (currently it publishes p=none) and IRS employees needed to post From: some irs.gov address, they could easily post From: @subdomain.irs.gov and publish p=none for that subdomain. However, all this is really moot because whatever any of us thinks, DMARC is already being used in ways that disrupt pre-existing accepted norms so for mailing lists to remain viable, they have to mitigate the effects in some way. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Oct 19 02:36:31 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 19 Oct 2017 15:36:31 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> Message-ID: <23016.18415.258899.171098@turnbull.sk.tsukuba.ac.jp> Grant Taylor via Mailman-Users writes: > I use DKIM validity as a signal that I then make decisions based on. - > Hence why I have chosen to alter spam score on my mail server based on > the DKIM result. You can do that. But call it what it is: a deliberate decision NOT to conform to a standards-track RFC. The fact of the matter is that the spammers are laughing at you. THEY have perfectly valid DKIM signatures, or if they're going to try a replay attack, they remove the DKIM signature they're about to break. Broken DKIM signatures principally mean somebody added a footer to the body, a DMARC mitigation in From, or a tag to the Subject. So this rule primarily targets perfectly legitimate mail posted to mailing lists. (I don't understand Dimitri's claim about SourceForge ads; all the mail I get from SourceForge is originated there and AFAIK the DKIM validates. If it doesn't, their system is pretty brain-damaged.) Steve -- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull at sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Oct 19 02:37:02 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 19 Oct 2017 15:37:02 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: <23016.18446.717809.67680@turnbull.sk.tsukuba.ac.jp> Mark Sapiro writes: > I don't agree that it is a completely new message. I think it is > still the original message with only technical and formatting > changes. The IETF's position is that this decision is up to the forwarding agent. If they change the Message-ID, that means they consider it a new message, and are taking authorship (perhaps with substantial quoting, but it's quoting, not forwarding). If they don't, it's not new, and From MUST contain the address placed there by the original author. (That's an RFC-2119 "must". This is why Mark is correct to say that Munge From is non-conforming.) The IETF has NO position on WHEN this should be done because it's not relevant to interoperability. My personal reasoning with respect to mailing list managers like Mailman which normally pass through all text/plain, and perhaps add some tags to Subject and prefix or suffix the body, is that users (including posters) would be quite annoyed if de-duping didn't work. And those of us who deal with mail in sophisticated ways would be quite upset if the Message-ID we give it doesn't correspond to the Message-ID distributed by the list and in the archive. > However, if you are just sending the body of the original message From: > the list, according to RFC 5322 et al, you are saying the list is the > author of that message body. This is not true and is why I say the > message is not compliant with RFC 5322 et al. This isn't quite accurate. We do make an effort to identify the author, so I wouldn't say we're "claiming authorship". The problems are that we make it impossible to identify the author by the usual methods (filtering on email address), and it's ugly, especially for folks with MUAs that display only the display name (and of course we had a lot of people rather confused by this through most of 2014!) Steve -- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull at sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Oct 19 02:37:24 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 19 Oct 2017 15:37:24 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <117e3a93-8ec1-e421-1110-86ebf7b626e2@bmrb.wisc.edu> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> <5c63defa-db2f-02c0-94a8-9d7f4ff92283@bmrb.wisc.edu> <9f7b28e3-6a5d-57ab-4390-242b4367f453@bmrb.wisc.edu> <2d063de0-5bd3-f4d3-0904-f42a1319444f@bmrb.wisc.edu> <7e323654-83b1-0940-48b0-7adc6bb992a7@tnetconsulting.net> <117e3a93-8ec1-e421-1110-86ebf7b626e2@bmrb.wisc.edu> Message-ID: <23016.18468.747430.445982@turnbull.sk.tsukuba.ac.jp> Dimitri Maziuk writes: > That does not contradict what I said. Low specificity means low > probability of detection of "bad stuff". I.e. it doesn't mean much that > most of it passes. That may be true for you, but for most of us having most of our mail have a valid DKIM signature, plus a DMARC PASS, means that most of our mail is authentic. I care a *lot* about having my filters throw away, or even quarantine, mail from a known correspondent using a known address. This almost never happens any more. > Ohkay, so what exactly am I the end user is supposed to need it > for? That depends on how much mail you get, how much of it is unwanted, how much you care about the time you spend dealing with unwanted mail, and how much you care about losing wanted mail. Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Oct 19 02:46:29 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 19 Oct 2017 15:46:29 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: <23016.19013.547017.523388@turnbull.sk.tsukuba.ac.jp> Grant Taylor via Mailman-Users writes: > IMHO, DMARC is going to eventually become the new norm. It has been so since late 2015, according to the DMARC Consortium. At that time they claimed that 80% of legitimate email was originated at domains that participate in DMARC reporting protocols. I don't think p=reject will ever be the norm for freemail providers. > I also wonder what ARC is going to do to this paradigm. It may or may not help mailing lists. It depends on whether the spammers successfully jump on it to obfuscate themselves, which they could do, in which case you might end up in the current situation where you need to apply for whitelisting at some of the large providers. On the other hand, the large providers are getting better at identifying responsible lists for themselves, and ARC would definitely make authenticating those lists easier. Steve -- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull at sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Oct 19 02:48:37 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 19 Oct 2017 15:48:37 +0900 Subject: [Mailman-Users] cause of bounces In-Reply-To: <8872d3d1-6671-f164-da85-90cc9932a014@maileater.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> <68797a95-beff-3c06-cd07-3f589adcd193@msapiro.net> <8872d3d1-6671-f164-da85-90cc9932a014@maileater.net> Message-ID: <23016.19141.407009.996993@turnbull.sk.tsukuba.ac.jp> Jordan Brown writes: > Would they?? Shouldn't IRS sysadmins who use Mailman in the course of > their jobs send messages to this mailing list using their @irs.gov > addresses? As Mark says, they should use an @sysadmins.irs.gov address or something like that, which would have its own p=none policy. Note that this has been already standard practice at Yahoo! (!), AOL (!!), LinkedIn, and several banks that participate in IETF discussions. Since 2013 for Yahoo! and LinkedIn IIRC. This need not be burdensome. Many MUAs support automatic "personality" switching based on addressee. Steve From dmaziuk at bmrb.wisc.edu Thu Oct 19 11:02:37 2017 From: dmaziuk at bmrb.wisc.edu (Dimitri Maziuk) Date: Thu, 19 Oct 2017 10:02:37 -0500 Subject: [Mailman-Users] cause of bounces In-Reply-To: <23016.18415.258899.171098@turnbull.sk.tsukuba.ac.jp> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <8d54374c-9473-a361-d542-d6346e3bed69@bmrb.wisc.edu> <9a129671-da65-bee0-55d0-93623f6e4859@bmrb.wisc.edu> <0171ebb5-7620-51e9-b33b-3b8256f37835@tnetconsulting.net> <23016.18415.258899.171098@turnbull.sk.tsukuba.ac.jp> Message-ID: <361e4c7f-0bb5-bfc5-e298-4f70f8b28b28@bmrb.wisc.edu> On 2017-10-19 01:36, Stephen J. Turnbull wrote: > (I don't understand Dimitri's claim about SourceForge ads; all the > mail I get from SourceForge is originated there and AFAIK the DKIM > validates. If it doesn't, their system is pretty brain-damaged.) It is, but not DKIM-drain-bramaged. I PGP-sign when sending from my linux PCs and SF injects their ads into the signed part. Which is part of the reason why they don't want you to sign your messages on the client, before they got their ads in. > That depends on how much mail you get, how much of it is unwanted, > how much you care about the time you spend dealing with unwanted mail, > and how much you care about losing wanted mail. :) How would I know: it got thrown away, I never knew it existed. Seriously, though, for me gmail is the only one that doesn't deliver wanted mail and sticks it into their "all mail" -- despite the blanket .forward I have in there. On my work MTA I pretend DMARC doesn't exist and I don't spend any more time on spam now than I did in 2007. Dimitri From rb211 at tds.net Thu Oct 19 07:07:08 2017 From: rb211 at tds.net (William Bagwell) Date: Thu, 19 Oct 2017 07:07:08 -0400 Subject: [Mailman-Users] cause of bounces In-Reply-To: <23016.19141.407009.996993@turnbull.sk.tsukuba.ac.jp> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8872d3d1-6671-f164-da85-90cc9932a014@maileater.net> <23016.19141.407009.996993@turnbull.sk.tsukuba.ac.jp> Message-ID: <201710190707.08859.rb211@tds.net> On Thursday 19 October 2017, Stephen J. Turnbull wrote: > As Mark says, they should use an @sysadmins.irs.gov address or > something like that, which would have its own p=none policy. ?Note > that this has been already standard practice at Yahoo! (!), AOL (!!), > LinkedIn, and several banks that participate in IETF discussions. > Since 2013 for Yahoo! and LinkedIn IIRC. So if enough users of Yahoo and AOL requested something such as user at list.aol.com to not be DMARC p=reject they /might/ listen? Only list I help administer the owner simply moderates the few remaining hold outs who can not switch and manually re-posts their messages. Would not have worked back when the list was busy... Think I now understand the correct fix but did not a few years ago when this mess started and that was the solution we came up with. -- William From mark at msapiro.net Thu Oct 19 23:15:37 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 19 Oct 2017 20:15:37 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: <201710190707.08859.rb211@tds.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8872d3d1-6671-f164-da85-90cc9932a014@maileater.net> <23016.19141.407009.996993@turnbull.sk.tsukuba.ac.jp> <201710190707.08859.rb211@tds.net> Message-ID: <4c0772f5-5df0-312a-1bf8-5f29544ff783@msapiro.net> On 10/19/2017 04:07 AM, William Bagwell wrote: > > So if enough users of Yahoo and AOL requested something such as > user at list.aol.com to not be DMARC p=reject they /might/ listen? I think that won't happen. The use of p=none subdomains by various entities that publish p=reject for their primary domain is intended for addresses for their own staff to use in communicating via mailing lists and perhaps other channels. If a freemail provider such as Yahoo would be willing to create a lists.yahoo.com domain with p=none for use by their freemail users, that domain would be subject to the same abuses that caused them to publish p=reject in the first place. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From gtaylor at tnetconsulting.net Fri Oct 20 00:14:42 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Thu, 19 Oct 2017 22:14:42 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: On 10/18/2017 11:50 AM, Mark Sapiro wrote: > This is the crux of our disagreement. The outbound message is still the > original author's message, albeit slightly altered by subject prefixing, > content filtering and/or other transformations to conform with list > policies. I don't agree that it is a completely new message. I think it > is still the original message with only technical and formatting changes. RFC 6377 - DomainKeys Identified Mail (DKIM) and Mailing Lists, disagrees with you. (RFC 6377 is also currently known as BCP 167.) ? 3.2 calls this out specifically: > resending: A resending MLM (see Sections 5.2 and 5.3 of [EMAIL-ARCH]) > is one that may make changes to a message. The output of such an MLM > is considered to be a *new message*; > *delivery of the original has been completed* prior to distribution of > the reposted message. Such messages > are often reformatted, such as with list-specific header fields or other > properties, to facilitate discussion among list subscribers. /The output of a resending MLM is/ *a new message*. > MLM Output: *MLM* (sending its reconstructed copy of the originating user's > message) *is Author*; MLM's ADMD is Originator and Signer; the ADMD of each > subscriber of the list is a Verifier; each subscriber is a Receiver. *The resending MLM is the author* /of the new message/. > The dissection of the overall MLM operation into these two distinct phases > allows the DKIM-specific issues with respect to MLMs to be isolated and > handled in a logical way. The main issue is that the repackaging and > reposting of a message by an MLM is actually the construction of a > completely new message, and as such, the MLM is introducing new content > into the email ecosystem, consuming the Author's copy of the message, > and creating its own. When considered in this way, the dual role of the > MLM and its ADMD becomes clear. Since we have been talking about modifying more than /just/ the SMTP envelope, we are indeed talking about a resending MLM and not an alias MLM. -- Grant. . . . unix || die From gtaylor at tnetconsulting.net Fri Oct 20 00:26:25 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Thu, 19 Oct 2017 22:26:25 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <23016.18446.717809.67680@turnbull.sk.tsukuba.ac.jp> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> <23016.18446.717809.67680@turnbull.sk.tsukuba.ac.jp> Message-ID: On 10/19/2017 12:37 AM, Stephen J. Turnbull wrote: > The IETF has NO position on WHEN this should be done because it's not > relevant to interoperability. My personal reasoning with respect to > mailing list managers like Mailman which normally pass through all > text/plain, and perhaps add some tags to Subject and prefix or suffix > the body, is that users (including posters) would be quite annoyed if > de-duping didn't work. And those of us who deal with mail in > sophisticated ways would be quite upset if the Message-ID we give it > doesn't correspond to the Message-ID distributed by the list and in > the archive. I believe RFC 6377 makes it fairly clear if a message is new or not. TL;DR: If anything other than the SMTP envelope is modified, then the MLM is a resending MLM, which necessitates a new message with a new author and Message-ID. I can respect your concern about the Message-ID changing, especially with deduplication. However, I counter that the new message from the resending MLM is in fact a different message than the one that the original author sent to the resending MLM. So, if you were in the To / CC / BCC of the message from the original author, you /should/ receive two copies of the message. Fortunately nicer MLMs, like Mailman, can detect that a list subscriber was included in the To or CC and act on the subscriber's configured option if they want to receive a copy of the message from the MLM that they received directly. -- Grant. . . . unix || die From gtaylor at tnetconsulting.net Fri Oct 20 00:28:54 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Thu, 19 Oct 2017 22:28:54 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: <4c0772f5-5df0-312a-1bf8-5f29544ff783@msapiro.net> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8872d3d1-6671-f164-da85-90cc9932a014@maileater.net> <23016.19141.407009.996993@turnbull.sk.tsukuba.ac.jp> <201710190707.08859.rb211@tds.net> <4c0772f5-5df0-312a-1bf8-5f29544ff783@msapiro.net> Message-ID: On 10/19/2017 09:15 PM, Mark Sapiro wrote: > I think that won't happen. The use of p=none subdomains by various > entities that publish p=reject for their primary domain is intended for > addresses for their own staff to use in communicating via mailing lists > and perhaps other channels. If a freemail provider such as Yahoo would > be willing to create a lists.yahoo.com domain with p=none for use by > their freemail users, that domain would be subject to the same abuses > that caused them to publish p=reject in the first place. > Agreed. Further, end users would either need to make a choice of which sending domain to use, or Yahoo (et al) would need to have a list of domains to send from the list subdomain. -- Grant. . . . unix || die From gtaylor at tnetconsulting.net Fri Oct 20 00:38:39 2017 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Thu, 19 Oct 2017 22:38:39 -0600 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: <4d697520-5d99-92f4-0f30-77cde1f0f01f@tnetconsulting.net> On 10/19/2017 10:14 PM, Grant Taylor via Mailman-Users wrote: > /The output of a resending MLM is/ *a new message*. ... > *The resending MLM is the author* /of the new message/. Since the MLM is the author of the new message, I think it would be prudent to use either of the following as the RFC5322.From address: From: Grant Taylor via Mailman-Users Or, optionally use the Group syntax to help indicate that a group (read: mailing list) was the source. From: Mailman-Users:mailman-users at python.org; I might be inclined to prefix body copy with something like the following: Message posted to Mailman-Users by: Grant Taylor -- Grant. . . . unix || die From mark at msapiro.net Fri Oct 20 01:55:38 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 19 Oct 2017 22:55:38 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: On 10/19/2017 09:14 PM, Grant Taylor via Mailman-Users wrote: > > RFC 6377 - DomainKeys Identified Mail (DKIM) and Mailing Lists, > disagrees with you.? (RFC 6377 is also currently known as BCP 167.) I am too tired at the moment to respond to your posts more completely. I may do so tomorrow. But I suggest that if you are going to quote RFCs that you understand the differences between Best Current Practice and Standards Track categories. Also, I don't disagree that there are issues between DKIM, DMARC and Mailing Lists that make seamless integration of these impossible without changing long standing norms and expectations for Mailing Lists. I also think Mailman (both 2.1 and 3) give you tools to do pretty much whatever you want in this vein except for changing the Message-ID: of the original post. Note that one of the biggest reasons for that is if the list copy has a different Message-ID: and some people receive and reply to a list copy and some receive a direct To: or Cc: and reply to that and people use MUAs that produce threaded views based on Message-ID:, References: and In-Reply-To: headers, threading can get pretty messed up. Finally, I think all we disagree on (as Steve implied in a post a day or two ago) is very arcane, small technical details, and while we may never come to agreement on these, I think we do agree that Mailman can operate in this environment in ways we think are satisfactory. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Fri Oct 20 20:49:43 2017 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 20 Oct 2017 17:49:43 -0700 Subject: [Mailman-Users] cause of bounces In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> <23016.18446.717809.67680@turnbull.sk.tsukuba.ac.jp> Message-ID: <787487a8-0fa3-5191-0e67-78cf06b2b347@msapiro.net> On 10/19/2017 09:26 PM, Grant Taylor via Mailman-Users wrote: > > I can respect your concern about the Message-ID changing, especially > with deduplication.? However, I counter that the new message from the > resending MLM is in fact a different message than the one that the > original author sent to the resending MLM.? So, if you were in the To / > CC / BCC of the message from the original author, you /should/ receive > two copies of the message. And then as I suggested in an earlier reply, threading is bifurcated and then the sub-threads are again bifurcated and so on as people reply to one or the other. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From turnbull.stephen.fw at u.tsukuba.ac.jp Mon Oct 23 00:57:15 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Mon, 23 Oct 2017 13:57:15 +0900 Subject: [Mailman-Users] A rant on parsing RFCs In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <8CFAF91E-3E47-4C0A-95DE-D5FE5370FC1A@msapiro.net> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> Message-ID: <23021.30379.230432.634485@turnbull.sk.tsukuba.ac.jp> Grant Taylor via Mailman-Users writes: > RFC 6377 - DomainKeys Identified Mail (DKIM) and Mailing Lists, > disagrees with you. (RFC 6377 is also currently known as BCP 167.) tl;dr version: RFC 5598 (non-normative but authoritative) disagrees with you. In practice, the mailing list *decides* whether it is producing new messages or not, and adjusts Message-ID if "new". The RFC recommends that a message undergoing only Mailman-style changes should be considered the *same* message. I appreciate you going to the source here, but you shouldn't read RFCs the way you read Wikipedia. As I joked before, you need an education comparable to a Jesuit's Bible study to parse RFCs with facility. Anybody can read RFCs, of course (I have no training in this!), but be careful: context matters. Each document has a class, standards-track or informational or best current practice, and these classes are written and vetted to different standards of precision. Only standards-track RFCs are normative. Some authors are more authoritative than others. Some terminology is standardized, others are local to a particular document, and sometimes these usages overlap. RFC-defined protocols are layered, and it's not always clear what level is referred to in a particular document without careful analysis. So complicated! In this case, RFC 6377 doesn't really matter. The whole RFC is non-normative, and it is very unlikely that Murray was being precise in the section you quote. The purpose of the RFC is not to answer the question at hand, and the terminology was defined for the convenience of the actual purpose, which is to *describe* (not "define") practices that seem relatively successful in dealing with problems induced by introducing DKIM into an environment not intended for authenticated mail. Furthermore, except for the rather strange use of the term "Author" in the context, he seems to be referring to the SMTP (RFC 5321) transport level when he writes "delivery is completed", in which context everybody agrees that when transmitted by Mailman it's a new message. (The contrasting case is relays among MXs, which are *not* new messages although the content is altered by addition of trace fields in the header. Hairsplitting arises when you deal with milters, and DKIM lives in a grey area between RFC 5321 and RFC 5322.) In any case, RFC 6377 doesn't mention changing the Message-ID, which is the standard indication of the semantics of "new message", nor does it mentioning changing From, which is the standard indication of an RFC 5322 Author. I can only guess that Murray is (mis-)appropriating RFC 5322 language denoting various actors in the mail system for his own purposes (although it might be from RFC 5321, with which I'm not as familiar). Here is most of the discussion from RFC 5598. Glosses on acronyms in [square brackets] were added by me, those in round parentheses are from the original. Square brackets are also used by the author for references to the bibliography. 3.4.1. Message-ID IMF [Internet Message Format, ie RFC 5322, MIME, etc.] provides for, at most, a single Message-ID:. The Message-ID: for a single message, which is a user-level IMF tag, has a variety of uses including threading, aiding identification of duplicates, and DSN (Delivery Status Notification) tracking. The Originator assigns the Message-ID:. The Recipient's ADMD [Administrative Domain] is the intended consumer of the Message-ID:, although any Actor along the transfer path can use it. Message-ID: is globally unique. Its format is similar to that of a mailbox, with two distinct parts separated by an at-sign (@). Typically, the right side specifies the ADMD or host that assigns the identifier, and the left side contains a string that is globally opaque and serves to uniquely identify the message within the domain referenced on the right side. The duration of uniqueness for the message identifier is undefined. When a message is revised in any way, the decision whether to assign a new Message-ID: requires a subjective assessment to determine whether the editorial content has been changed enough to constitute a new message. [RFC5322] states that "a message identifier pertains to exactly one version of a particular message; subsequent revisions to the message each receive new message identifiers." Yet experience suggests that some flexibility is needed. An impossible test is whether the Recipient will consider the new message to be equivalent to the old one. For most components of Internet Mail, there is no way to predict a specific Recipient's preferences on this matter. Both creating and failing to create a new Message-ID: have their downsides. Here are some guidelines and examples: o If a message is changed only in form, such as character encoding, it is still the same message. o If a message has minor additions to the content, such as a Mailing List tag at the beginning of the RFC5322.Subject header field, or some Mailing List administrative information added to the end of the primary body part text, it is probably the same message. [further guidelines elided] As Mark has pointed out, there are practical reasons that are important to authors and recipients for considering the Mailman- altered message to still be the same message for this purpose. Of course, there are also pragmatic reasons for altering From: in our context, but these *are* pragmatic. I can find no support for altering From: in normative RFCs, and a lot of contradictory discussion in informative RFCs, with the most authoritative RFCs concluding that affixing new information while preserving all existing information does not create a "new version of the message" requiring a new Message-ID. I will add that in discussions of this kind of thing, Murray (author of RFC 6377) normally agrees with Dave (author of RFC 5598), and when they reach the agree-to-disagree stage, Murray shuts up and Dave gets his way (Dave is much higher ranked in the IETF). :-) Regards, Steve From mrbrklyn at panix.com Mon Oct 23 18:51:54 2017 From: mrbrklyn at panix.com (Ruben Safir) Date: Mon, 23 Oct 2017 18:51:54 -0400 Subject: [Mailman-Users] A rant on parsing RFCs In-Reply-To: <23021.30379.230432.634485@turnbull.sk.tsukuba.ac.jp> References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> <23021.30379.230432.634485@turnbull.sk.tsukuba.ac.jp> Message-ID: On 10/23/2017 12:57 AM, Stephen J. Turnbull wrote: > Grant Taylor via Mailman-Users writes: > > > RFC 6377 - DomainKeys Identified Mail (DKIM) and Mailing Lists, > > disagrees with you. (RFC 6377 is also currently known as BCP 167.) > > tl;dr version: RFC 5598 (non-normative but authoritative) disagrees > with you. In practice, the mailing list *decides* whether it is > producing new messages or not, and adjusts Message-ID if "new". The > RFC recommends that a message undergoing only Mailman-style changes > should be considered the *same* message. > > I appreciate you going to the source here, but you shouldn't read RFCs > the way you read Wikipedia. As I joked before, you need an education > comparable to a Jesuit's Bible study to parse RFCs with facility. > Anybody can read RFCs, of course (I have no training in this!), but be > careful: context matters. RFCs are a record of a process. Unless you were directly involved that that process, RFCs are about as useless as garbage. They are not only without clear explanation, but they are often just plain wrong and contradictory. People who suggest reading them need to have their meds adjusted. -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www2.mrbrklyn.com/resources - Unpublished Archive http://www.coinhangout.com - coins! http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013 From anon_777 at hotmail.com Tue Oct 24 01:21:43 2017 From: anon_777 at hotmail.com (Terry .) Date: Tue, 24 Oct 2017 05:21:43 +0000 Subject: [Mailman-Users] "Bounce action notification" emails for subscribes/unsubscribes In-Reply-To: References: <22975.33210.35652.332865@turnbull.sk.tsukuba.ac.jp>, <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net>, Message-ID: Hi again Mark, etc. Sorry for the delay in responding. I passed this back to my webhost, and they passed it on to cPanel for me: ========================================== I?ve been thinking about the solution that cPanel have provided for the previously mentioned 3 subscribe/unsubscribe notification issues, and to me it just sounds like a not-so-good work-around for a problem which has started occurring to my lists (which I've had for years) this year, and according to the mailman users mailing list, WHB is not the only webhost which has been affected. It?s a relatively minor issue though, so cPanel may not have received many complaints...yet. The reason I say it?s a not-so-good work-around are: a) Creating a Default Address for a domain has side affects which may not be wanted. (I discovered that I can avoid those side affects by creating a forwarder (i.e. alias) which points from mailman-bounces at mydomain.com to any mailbox, but that is still just a work-around.) b) Requiring list creators to always work around these cPanel problems by creating a default address or forwarder is not a proper fix. c) How are list creators supposed to discover that they need to perform this work-around to avoid these problems? Is cPanel going to have something pop up on their Mailing Lists page to say this? Better to fix the problem! d) How are owners of existing lists expected to discover it? A pop up may not be possible there. Do cPanel expect every customer to have the time, energy and skills to prove the problems to their webhost, and hope their webhost passes them on to cPanel, so they can be given the work-around? This is a waste of time for everyone including webhost and cPanel staff. e) These Mailman problems seem to be happening in cPanel environments only, so cPanel should fix them properly, and save their Mailman users from wasting huge amounts of time and energy to discover a work-around which should not even be necessary. The server I?m on (server103) is currently running Mailman 2.1.23 and cPanel 11.66.0.25, and today I confirmed the problem still occurs, unless I work-around it. What is cPanel going to do about this? ========================================== cPanel responded with the following 2 emails: Response #1: ==================================== Hello, Unfortunately though, as I mentioned previously this is a result of an upstream design choice from Mailman not from cPanel, we had an open inquiry for our development team as I mentioned previously as well which addressed this. This isn't a modification we made in the behavior. In response to point a) A default address is created for every account on the server automatically, I did suggest in my previous response that it is a bad idea to utilize this as a resolution. In regard to the rest of the points I think it is useful to stress the following: This isn't an issue that cPanel created, this is an issue that is present due to the utilization of two incompatible configurations in conjunction with each other, sender-verify cannot verify a sender that does not exist, mailman does not create an alias for mailman-bounces, so essentially unless one of the two is changed (which sender verify cannot) the solution is either to disable sender verify or create an alias/forwarder/account so the user does exist. My suggestion to prevent this from occurring would be to disable sender-verify in this instance since the way that mailman works is incompatible with the configuration. The sole purpose of providing the workarounds was to provide a way for this to work with sender-verify enabled, if these are not suitable you can disable sender-verify and the mailman mailing list would work as intended. Thank you, -- Lauren N Linux Technical Analyst II cPanel, Inc. ==================================== Response #2: ==================================== Hello, I just wanted to follow up on this and let you know that I did open another internal case CPANEL-16468 to inform our development of the specific behavior that's occurring. Should our developers choose to address this issue updates to this case will be added to our changelogs when they're available. You can check them here: https://documentation.cpanel.net/display/CL/Change+Logs I'll do my best to respond here if any response is made by the development team to the internal case as well. Thank you, -- Lauren N Linux Technical Analyst II cPanel, Inc. ==================================== So response #1 makes it look as if it's Mailman's fault, not cPanel's, so cPanel don't need to fix it, which begs the question: how was all this working fine until earlier this year, and what's changed that broke it? I don't know if a new version of Mailman was installed on the server in that time, but even if it were, it doesn't sound to me as if it would have changed in this department. And I think the webhost probably has had sender_verify turned on for years now, but I'm checking that with them. Then response #2 makes it look as if cPanel *may* be willing to deal with the issue, despite the above. Any comments on any of this, Mark or anyone else, especially re this claim: "...this is a result of an upstream design choice from Mailman not from cPanel, we had an open inquiry for our development team as I mentioned previously as well which addressed this. This isn't a modification we made in the behavior." Thanks. Terry From turnbull.stephen.fw at u.tsukuba.ac.jp Tue Oct 24 04:14:07 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 24 Oct 2017 17:14:07 +0900 Subject: [Mailman-Users] A rant on parsing RFCs In-Reply-To: References: <6017DADC-C5E3-4165-A163-2963C0B44047@tokyoprogressive.org> <23005.37118.127328.245124@turnbull.sk.tsukuba.ac.jp> <2365da49-a5d1-56fd-5c18-e146e834d1af@msapiro.net> <60CBA5A9-7C2E-4583-8213-F63DE3E5B8D4@mailbox.org> <2782ddf3-008b-5fb8-15c5-4f402dc7ac09@msapiro.net> <23010.28293.631629.541809@turnbull.sk.tsukuba.ac.jp> <4a92a4ba-f349-33f9-07c7-b8a9b1dda6ff@tnetconsulting.net> <20171017185542995737.b7e31827@yahoo.de> <526b993d-f862-c0bd-b8cf-8394f802b1ef@tnetconsulting.net> <74a282f6-0386-0050-92c1-b496f9a7b73c@tnetconsulting.net> <31f60087-9f2c-150f-7ce2-69a4c0cc66c9@tnetconsulting.net> <23021.30379.230432.634485@turnbull.sk.tsukuba.ac.jp> Message-ID: <23022.63055.798538.140778@turnbull.sk.tsukuba.ac.jp> Ruben Safir writes: > RFCs are a record of a process. Partially true. The process almost invariably leaves its trace in the text, and (as in any committee work) many compromises are inexplicable without reference to the process. But the text of an RFC is a specification, not a narrative. > Unless you were directly involved that that process, RFCs are about > as useless as garbage. False. True, RFCs are difficult for most who haven't participated in the process to read and understand correctly. But I can say from experience that it's quite possible to understand them without participating in the drafting process. What's needed to understand RFCs is first, a formal mindset, and second, firm (indeed, I might say "desperate") grasp of the principle that these are *wire protocols*, and that therefore behavior of systems at either end of the channel is a pretty poor foundation for understanding them. Endpoint behaviors can be perfectly conformant no matter how they look to users, as far as the RFCs are concerned. RFCs are only concerned with defining and serializing data structures at the channel's transmitting end, conveying the data to the receiving end, and reconstructing the data there. > They are not only without clear explanation, You're looking in the wrong place if you look in standards-track RFCs for explanation useful to non-specialists. That's why Grant quoted a BCP, and I, an informational RFC. > but they are often just plain wrong and contradictory. In other words, RFCs are a human endeavor. :-) > People who suggest reading them need to have their meds adjusted. Nobody suggested reading them. People who already have experience reading them did read them because reading RFCs is necessary to understanding *how* any Internet function is intended to work[1], quoted them, and complimented each other for doing so. Anybody who doesn't want to read RFCs is still welcome to comment, but it's most profitable for them to stick to what they *want* to happen. They'll have to rely on those of us who know what the RFCs say for judgments of feasibility and cost, and for design, in implementing those requested behaviors. Footnotes: [1] Alternative behavior is permitted, but is unlikely to work as desired without explicit private agreement. Taking existing RFCs as a baseline and agreeing on slightly variant protocols is often a very productive way to implement new Internet features, as well as private protocols. From turnbull.stephen.fw at u.tsukuba.ac.jp Tue Oct 24 04:16:24 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 24 Oct 2017 17:16:24 +0900 Subject: [Mailman-Users] "Bounce action notification" emails for subscribes/unsubscribes In-Reply-To: References: <22975.33210.35652.332865@turnbull.sk.tsukuba.ac.jp> <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net> Message-ID: <23022.63192.83194.820406@turnbull.sk.tsukuba.ac.jp> Terry . writes: > Any comments on any of this, Mark or anyone else, especially re > this claim: "...this is a result of an upstream design choice from > Mailman not from cPanel, As I understand it, the "design choice" meant is to have a sitewide address "mailman at site.tld". This isn't so much a "design choice" as a long-established Internet mail tradition that there needs to be a contact address that reaches humans for every automatic installation. For the mail system itself, this is formalized in RFC 2142, which defines addresses like "postmaster" and "hostmaster", as well as the "LIST-request" address for mailing lists. Since Mailman has an additional layer of "site" administration above the lists themselves, we added *one address per mailman instance*, the "mailman" address. Mailman was designed for "real" sites with a single domain hosting lists, not for virtual hosting. This is unfortunate for cPanel, we admit, but handling of the "mailman" list and its associated aliases in Mailman 2 (which is a 15-year-old architecture IIRC) are well- adapted to that use case. (Making it a list is a natural choice, and I don't see how that causes additional difficulties for cPanel.) It is certainly true that "Mailman does not create an alias for mailman-bounces." Mailman doesn't create *any* aliases, because alias management is done by MTAs. Mailman is agnostic about MTAs, and each MTA has its own system for setting up aliases. Furthermore, many hosts have unique needs for their systems, so there is no "one size fits all" configuration for targets of aliases. We do provide sample configurations for simple cases (single host, single instance, site owner manages most lists too) for the MTAs we are most familiar with, but the responsibility for setting up aliases is with the system administrator who installs Mailman. (This division of responsibilities remains in Mailman 3.) That "system administrator" might be a person, or it might be a distribution script. I don't know how cPanel is architected, so I don't know where this reponsibility might best be handled in a cPanel installation. But I can tell you that Mailman has never assumed it, while all of the usual distros (Debian, Ubuntu, Red Hat, Centos, FreeBSD, etc) do, each in its own way. I get the feeling that this doesn't fully address the conversation among you, your host, and cPanel. It should give you some idea of how we view the system administration responsibilities, though. Hope this helps. Steve From jimpop at gmail.com Tue Oct 24 09:06:33 2017 From: jimpop at gmail.com (Jim Popovitch) Date: Tue, 24 Oct 2017 09:06:33 -0400 Subject: [Mailman-Users] "Bounce action notification" emails for subscribes/unsubscribes In-Reply-To: <23022.63192.83194.820406@turnbull.sk.tsukuba.ac.jp> References: <22975.33210.35652.332865@turnbull.sk.tsukuba.ac.jp> <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net> <23022.63192.83194.820406@turnbull.sk.tsukuba.ac.jp> Message-ID: On Oct 24, 2017 04:20, "Stephen J. Turnbull" wrote: (Making it a list is a natural choice, and I don't see how that causes additional difficulties for cPanel.) I'm jumping in late here.. Is the problem possibly DMARC alignment failures? If so, I've been working on a patch for this: https://code.launchpad.net/~jimpop/mailman/virtual-notices -Jim P. From mark at msapiro.net Tue Oct 24 13:52:10 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 24 Oct 2017 10:52:10 -0700 Subject: [Mailman-Users] "Bounce action notification" emails for subscribes/unsubscribes In-Reply-To: References: <22975.33210.35652.332865@turnbull.sk.tsukuba.ac.jp> <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net> Message-ID: On 10/23/2017 10:21 PM, Terry . wrote: > > cPanel responded with the following 2 emails: > > > Response #1: > > ==================================== > > Hello, > > Unfortunately though, as I mentioned previously this is a result of an upstream design choice from Mailman not from cPanel, we had an open inquiry for our development team as I mentioned previously as well which addressed this. This isn't a modification we made in the behavior. In fact it is a modification that cPanel made to require sender-verify. > In response to point a) A default address is created for every account on the server automatically, I did suggest in my previous response that it is a bad idea to utilize this as a resolution. > > In regard to the rest of the points I think it is useful to stress the following: > > This isn't an issue that cPanel created, this is an issue that is present due to the utilization of two incompatible configurations in conjunction with each other, sender-verify cannot verify a sender that does not exist, mailman does not create an alias for mailman-bounces, so essentially unless one of the two is changed (which sender verify cannot) the solution is either to disable sender verify or create an alias/forwarder/account so the user does exist. My suggestion to prevent this from occurring would be to disable sender-verify in this instance since the way that mailman works is incompatible with the configuration. This is simply misleading at best. As Steve points out, Mailman doesn't create aliases period except for the special case of MTA = 'Postfix' which does not apply to cPanel because their installations use Exim. The issue is in cPanel's exim router for Mailman addresses. There is good documentation for creating an Exim configuration for Mailman at and in particular, a Mailman router at and transport at . The problem is cPanel's Exim router and transport is different because in a 'normal' Mailman install, mail to list at example.com is piped to 'the_mail_wrapper post list' and mail to, e.g., list-bounces at example.com is piped to 'the_mail_wrapper bounces list', but in cPanel, mail to list at example.com is piped to 'the_mail_wrapper post list_example.com' and mail to, e.g., list-bounces at example.com is piped to 'the_mail_wrapper bounces list_example.com' Now in cPanel as in all Mailman. there is a 'mailman' list so mail to all the mailman(-*) addresses should be delivered to that list, but cPanel has not programmed their Exim router/transport to know that 'mailman(-*)@example.com is a special case for which the list name is (probably I think) 'mailman' and not mailman_example.com. If spmeone from cPanel (Lauren N or anyone) would contact me about this, I would be willing to work with them to figure out a solution, but it is not an issue in upstream Mailman, it is an issue in cPanel's Exim configuration for Mailman lists that doesn't take into account the fact the due to their changes, the 'mailman' site list is different from other lists. ... > So response #1 makes it look as if it's Mailman's fault, not cPanel's, so cPanel don't need to fix it, which begs the question: how was all this working fine until earlier this year, and what's changed that broke it? Requiring sender-verify in Exim. > I don't know if a new version of Mailman was installed on the server in that time, but even if it were, it doesn't sound to me as if it would have changed in this department. > And I think the webhost probably has had sender_verify turned on for years now, but I'm checking that with them. If so, then I don't know. > Then response #2 makes it look as if cPanel *may* be willing to deal with the issue, despite the above. If they would open a dialog with me, we could fix this. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Tue Oct 24 20:25:30 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 24 Oct 2017 17:25:30 -0700 Subject: [Mailman-Users] "Bounce action notification" emails for subscribes/unsubscribes In-Reply-To: References: <22975.33210.35652.332865@turnbull.sk.tsukuba.ac.jp> <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net> <23022.63192.83194.820406@turnbull.sk.tsukuba.ac.jp> Message-ID: <68519d4b-47ba-7bce-f090-9580e63c7736@msapiro.net> On 10/24/2017 06:06 AM, Jim Popovitch wrote: > > I'm jumping in late here.. Is the problem possibly DMARC alignment > failures? If so, I've been working on a patch for this: No. This has nothing to do with DMARC. However your branch might be relevant. The issue is that owner notifications from Mailman are sent with envelope from the sitelist-bounces address in the list's domain. In cPanel this results in the envelope being from mailman-bounces at list.domain. cPanel's Exim configuration apparently only sees this as a valid address if a list named mailman_list.domain exists which it doesn't. There is only one 'mailman' list and its name is maybe just 'mailman' or maybe 'mailman_some_canonical_host_name', I'm not sure which but probably the former, but there isn't a separate 'mailman_list.domain' list for every list domain. So then Exim is configured to do sender verification and mailman-bounces at list.domain is not a valid address as far as Exim is concerned so these owner notifications are bounced by Exim and never sent. The workaround that Terry and others have implemented is to make an alias for mailman-bounces at list.domain which, cPanel's opinion to the contrary notwithstanding, I believe is the correct solution as it is the only way the domain admin is going to see any real bounces. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fsantiago at garbage-juice.com Wed Oct 25 16:10:25 2017 From: fsantiago at garbage-juice.com (Fabian A. Santiago) Date: Wed, 25 Oct 2017 20:10:25 +0000 Subject: [Mailman-Users] integrating mm3 with postfix / lmtp Message-ID: <36fa2bd027144eaabce1cf70dbce8d1f@garbage-juice.com> Hello, I have mm3 up and running via docker (courtesy of maxking/docker-mailman) but when i send an email to my test list, it bounces back claiming user unknown. it states: <"lmtp:[172.19.199.2]:8024"@>: unknown user: "lmtp:[172.19.199.2]:8024" looking at the email source, it seems as though the message is trying to be delivered to: Final-Recipient: rfc822; "lmtp:[172.19.199.2]:8024"@ Original-Recipient: rfc822;test123@ all of various alias transport maps are defined in postfix's main.cf. not sure where to go from here. does anyone have any clues? -- Thanks, Fabian S. OpenPGP: 3C3FA072ACCB7AC5DB0F723455502B0EEB9070FC From mark at msapiro.net Wed Oct 25 18:30:15 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 25 Oct 2017 15:30:15 -0700 Subject: [Mailman-Users] integrating mm3 with postfix / lmtp In-Reply-To: <36fa2bd027144eaabce1cf70dbce8d1f@garbage-juice.com> References: <36fa2bd027144eaabce1cf70dbce8d1f@garbage-juice.com> Message-ID: On 10/25/2017 01:10 PM, Fabian A. Santiago wrote: > Hello, > > I have mm3 up and running via docker (courtesy of maxking/docker-mailman) ... You also posted this to mailman-users at mailman3.org which is the proper place. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fsantiago at garbage-juice.com Wed Oct 25 18:32:23 2017 From: fsantiago at garbage-juice.com (Fabian A. Santiago) Date: Wed, 25 Oct 2017 18:32:23 -0400 Subject: [Mailman-Users] integrating mm3 with postfix / lmtp In-Reply-To: References: <36fa2bd027144eaabce1cf70dbce8d1f@garbage-juice.com> Message-ID: <7EDC8D6E-547E-4FEA-97C2-EFC47C71AD6D@garbage-juice.com> On October 25, 2017 6:30:15 PM EDT, Mark Sapiro wrote: >On 10/25/2017 01:10 PM, Fabian A. Santiago wrote: >> Hello, >> >> I have mm3 up and running via docker (courtesy of >maxking/docker-mailman) ... > > >You also posted this to mailman-users at mailman3.org which is the proper >place. Ok wasn't sure thanks. Just covering my bases. -- Thanks, Fabian S. OpenPGP: 3C3FA072ACCB7AC5DB0F723455502B0EEB9070FC From anon_777 at hotmail.com Tue Oct 24 18:53:01 2017 From: anon_777 at hotmail.com (Terry .) Date: Tue, 24 Oct 2017 22:53:01 +0000 Subject: [Mailman-Users] "Bounce action notification" emails for subscribes/unsubscribes In-Reply-To: References: <22975.33210.35652.332865@turnbull.sk.tsukuba.ac.jp>, <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net>, , Message-ID: Thanks for your excellent answers, Steve and Mark. Mark, I might pass your generous offer to work with cPanel, on to the webhost (again), to be passed on to cPanel, once I've heard back from the webhost re the timing of when sender_verify was turned on (presumably through a cPanel update). When I asked the webhost yesterday how long it's been turned on, the response was: 'Actually, the "Sender Verify" option is the default and recommended by cPanel, so I don't think it was touched by our support staff. I'm attaching the screenshot of this setting for your convenience.' I won't attach the screen shot, but it describes the "Sender Verification" option as "Verify that the domain mail reports as it origin actually exists". (The grammar looks strange to my uneducated eyes - maybe that should read "Verify that the domain origin email address actually exists" or "Verify that the domain origin actually exists".) The options are "On (default)" and "Off". "On" is selected. Jim, I don't know the answer to your question, but thanks for commenting. Terry From mark at msapiro.net Thu Oct 26 00:11:03 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 25 Oct 2017 21:11:03 -0700 Subject: [Mailman-Users] "Bounce action notification" emails for subscribes/unsubscribes In-Reply-To: References: <22975.33210.35652.332865@turnbull.sk.tsukuba.ac.jp> <81c86029-b5b5-225d-db0b-0042c445cb13@msapiro.net> Message-ID: On 10/24/2017 03:53 PM, Terry . wrote: > > Mark, I might pass your generous offer to work with cPanel, on to the webhost (again), to be passed on to cPanel ... That would be good. Emphasize that I am the person who does all the upstream maintenance of Mailman 2.1 and that I want to work with them to resolve this issue. Of course, the resolution is problematic, because I really think your workaround is the correct solution. Here's why. Notifications to list -owner addresses are sent with envelope from the site list -bounces address to avoid bounce loops that would occur if they were sent with envelope from the list -bounces address if the -owner address is actually bouncing (because it's not deliverable, not because of sender verify). In a normal (non-cPanel) installation this will result in the bounce going to the site list owner which is reasonable. If we fix cPanel's Exim config to understand that mailman-bounces at your.domain is really the -bounces address for the site list, those bounces will go to the site list owner which in the cPanel case is often the web host who (with a few notable exceptions of which I am aware) knows nothing about Mailman or your lists. With your workaround, they go to you and you are in a better position to deal with undeliverable owner addresses for lists in your domain. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From DonK at dbscompany.com Thu Oct 26 06:11:17 2017 From: DonK at dbscompany.com (Don Keating) Date: Thu, 26 Oct 2017 06:11:17 -0400 Subject: [Mailman-Users] Change List Name Message-ID: <003701d34e42$c3bb81d0$4b328570$@dbscompany.com> Can a list name be changed? If so, how? Thanks. >don< Donald R Keating DBS Company www.dbscompany.com 803-312-4246 Phone 800-513-2321 Fax From james.dore at new.ox.ac.uk Thu Oct 26 10:04:16 2017 From: james.dore at new.ox.ac.uk (James Dore) Date: Thu, 26 Oct 2017 14:04:16 +0000 Subject: [Mailman-Users] Postfix delivers to mailman, but not aliases Message-ID: <25DEBC0A-61B3-44B3-93E3-EA759F1F112A@new.ox.ac.uk> Hi list, I recently migrated our mailman server from an old SLES 11 box to Ubuntu 16.04.3 LTS, and installed Mailman from the Ubuntu repositories along with Postfix and other prerequisites. Mailman itself is working fine, but I have a handful of regular email aliases in /etc/aliases which do not receive mail, and when examining the logs, get bounced with a ?User unknown? error. What did I screw up? (I?ve checked my aliases and they?re good, and I?ve run the newaliases command numerous times). Cheers, James -- James Dore IT Officer New College, Oxford, OX1 3BN 01865 279252 (Mon-Fri 0830-1600) - 01865 612345 All other www.new.ox.ac.uk [cid:481BAAE5-F170-469A-B15F-59C56158337F at new.ox.ac.uk] New College is registered with the Charity Commissioners. New College Oxford? is a registered trade mark - No. 2588652 From mark at msapiro.net Thu Oct 26 12:08:08 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 26 Oct 2017 09:08:08 -0700 Subject: [Mailman-Users] Change List Name In-Reply-To: <003701d34e42$c3bb81d0$4b328570$@dbscompany.com> References: <003701d34e42$c3bb81d0$4b328570$@dbscompany.com> Message-ID: <2e5d9889-8e17-5779-be74-dca8e6d6c5df@msapiro.net> On 10/26/2017 03:11 AM, Don Keating wrote: > Can a list name be changed? Yes > If so, how? See -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Thu Oct 26 12:29:40 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 26 Oct 2017 09:29:40 -0700 Subject: [Mailman-Users] Postfix delivers to mailman, but not aliases In-Reply-To: <25DEBC0A-61B3-44B3-93E3-EA759F1F112A@new.ox.ac.uk> References: <25DEBC0A-61B3-44B3-93E3-EA759F1F112A@new.ox.ac.uk> Message-ID: <96fdbf63-34da-aa82-d206-0c3af6287f7c@msapiro.net> On 10/26/2017 07:04 AM, James Dore wrote: > > I recently migrated our mailman server from an old SLES 11 box to Ubuntu 16.04.3 LTS, and installed Mailman from the Ubuntu repositories along with Postfix and other prerequisites. Mailman itself is working fine, but I have a handful of regular email aliases in /etc/aliases which do not receive mail, and when examining the logs, get bounced with a ?User unknown? error. What did I screw up? I don't think this is a Mailman question, but What is the output from postconf alias_maps and ls -l /etc/aliases* What is the full "User unknown" log message? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Thu Oct 26 18:09:28 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 26 Oct 2017 15:09:28 -0700 Subject: [Mailman-Users] Mailman 2.1.25 released Message-ID: <102d4978-61d0-67ac-d5ed-7c1603c86fd8@msapiro.net> I am pleased to announce the release of Mailman 2.1.25. Python 2.4 is the minimum supported, but Python 2.7 is strongly recommended. This is a routine bug fix release with a minor new feature and some accessibility improvements for screen readers. See the attached README.txt for details. Mailman is free software for managing email mailing lists and e-newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites. For more information, please see our web site at one of: http://www.list.org https://www.gnu.org/software/mailman http://mailman.sourceforge.net/ https://mirror.list.org/ Mailman 2.1.25 can be downloaded from https://launchpad.net/mailman/2.1/ https://ftp.gnu.org/gnu/mailman/ https://sourceforge.net/projects/mailman/ -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- 2.1.25 (26-Oct-2017) New Features - The admindb held subscriptions listing now includes the date of the most recent request from the address. (LP: #1697097) Accessibility - The admin Membership List now includes text for screen readers which identifies the function of each checkbox. CSS is added to the page to visually hide the text but still allow screen readers to read it. Similar text has been added to some radio buttons on the admindb pages. i18n - The Russian translation has been updated by Sergey Matveev. (LP:#1708016) Bug fixes and other patches - Thanks to Jim Popovitch, certain failures in DNS lookups of DMARC policy will now result in mitigations being applied. (LP: #1722013) - The default DMARC reject reason now properly replaces %(listowner)s. (LP: #1718962) - The web roster page now shows case preserved email addresses. (LP: #1707447) - Changed the SETGID wrappers to only pass those items in the environment that are needed by the called scripts. (LP: #1705736) - Fixed MTA/Postfix.py to ensure that created aliases(.db) and virtual-mailman(.db) files are readable by Postfix and the .db files are owned by the Mailman user. (LP: #1696066) - Defended against certain web attacks that cause exceptions and "we hit a bug" responses when POST data or query fragments contain multiple values for the same parameter. (LP: #1695667) - The fix for LP: #1614841 caused a regression in the options CGI. This has been fixed. (LP: #1602608) - Added a -a option to the (e)grep commands in contrib/mmdsr to account for logs that may have non-ascii and be seen as binary. - Fixed the -V option to bin/list_lists to not show lists whose host is a subdomain of the given domain. (LP: #1695610) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: