From p at sys4.de Wed May 10 08:07:23 2017 From: p at sys4.de (Patrick Ben Koetter) Date: Wed, 10 May 2017 14:07:23 +0200 Subject: [Mailman-Developers] Signaling One-Click Functionality for List Email Headers Message-ID: <20170510120723.xgg3apmj65cmvy3z@sys4.de> Greetings, I'm not sure if anyone has followed development of RFC 8058 "Signaling One-Click Functionality for List Email Headers" located at and brought this topic up on this list. Is that something mailman(2|3) should support? To me it looks useful. p at rick -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schlei?heimer Stra?e 26/MG,80333 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer, Wolfgang Stief Aufsichtsratsvorsitzender: Florian Kirstein From johnl at taugh.com Wed May 10 13:05:36 2017 From: johnl at taugh.com (John Levine) Date: 10 May 2017 17:05:36 -0000 Subject: [Mailman-Developers] Signaling One-Click Functionality for List Email Headers In-Reply-To: <20170510120723.xgg3apmj65cmvy3z@sys4.de> Message-ID: <20170510170536.29935.qmail@ary.lan> In article <20170510120723.xgg3apmj65cmvy3z at sys4.de> you write: >Greetings, > >I'm not sure if anyone has followed development of RFC 8058 "Signaling >One-Click Functionality for List Email Headers" located at > and brought this topic up on this >list. > >Is that something mailman(2|3) should support? To me it looks useful. It would certainly make it easier to deal with grumpy gmail users, since gmail does not provide junk button feedback. The disadvantage is that every recipient needs to get a separate copy of each message, because the list and user info has to be encoded in the list-unsubscribe URL. That's been standard in commercial e-mail for a decade, but a lot of discussion list operators still imagine that it's too slow. R's, John From barry at list.org Wed May 10 13:36:09 2017 From: barry at list.org (Barry Warsaw) Date: Wed, 10 May 2017 13:36:09 -0400 Subject: [Mailman-Developers] Signaling One-Click Functionality for List Email Headers In-Reply-To: <20170510170536.29935.qmail@ary.lan> References: <20170510120723.xgg3apmj65cmvy3z@sys4.de> <20170510170536.29935.qmail@ary.lan> Message-ID: <20170510133609.61fba76f@subdivisions.wooz.org> On May 10, 2017, at 05:05 PM, John Levine wrote: >>I'm not sure if anyone has followed development of RFC 8058 "Signaling >>One-Click Functionality for List Email Headers" located at >> and brought this topic up on this >>list. >> >>Is that something mailman(2|3) should support? To me it looks useful. I probably need more convincing that it would actually be used out in the field, since there are a lot of email standards that have been ignored (by some tools) for decades. But OTOH, if it's of some utility it doesn't look like it would be difficult in core to support the extra header. We'd need a small bit of REST and db schema/style setting work so that the list itself could be configured for one-click or not, depending on the web u/i being used. (E.g. maybe one-click unsub is supported in Postorius, but other sites might not support it.) >It would certainly make it easier to deal with grumpy gmail users, >since gmail does not provide junk button feedback. Let's call that the Grumpy 800lb Gorilla principle. :) >The disadvantage is that every recipient needs to get a separate copy >of each message, because the list and user info has to be encoded in >the list-unsubscribe URL. That's been standard in commercial e-mail >for a decade, but a lot of discussion list operators still imagine >that it's too slow. For a while now I've thought about changing the defaults to individual personalization (i.e. everyone gets a unique copy, but we don't modify the headers). I think the constraints leading us to no personalization may not be all that prevalent any more, and there's no question that personalization improves the user experience. -Barry From johnl at taugh.com Wed May 10 15:18:18 2017 From: johnl at taugh.com (John Levine) Date: 10 May 2017 19:18:18 -0000 Subject: [Mailman-Developers] Signaling One-Click Functionality for List Email Headers In-Reply-To: <20170510133609.61fba76f@subdivisions.wooz.org> Message-ID: <20170510191818.30228.qmail@ary.lan> In article <20170510133609.61fba76f at subdivisions.wooz.org> you write: >I probably need more convincing that it would actually be used out in the >field, ... Gmail's already implemented it. I'm pretty sure Yahoo is also planning to. > But OTOH, if it's of some utility it doesn't look >like it would be difficult in core to support the extra header. We'd need a >small bit of REST and db schema/style setting work so that the list itself >could be configured for one-click or not, depending on the web u/i being >used. (E.g. maybe one-click unsub is supported in Postorius, but other sites >might not support it.) Keep in mind that the list and user info have to be encoded in the existing List-Unsubscribe header, and one-click just adds a fixed List-Unsubscribe-Post header to tell the recipient that it can do a POST for one click. The encoded header makes regular unsub work better too, since it knows what address to remove and needn't ask the user. R's, John >>It would certainly make it easier to deal with grumpy gmail users, >>since gmail does not provide junk button feedback. > >Let's call that the Grumpy 800lb Gorilla principle. :) Yes indeed. From johny at neuromancer.sk Wed May 10 20:07:45 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Thu, 11 May 2017 02:07:45 +0200 Subject: [Mailman-Developers] [GSoC] Encrypted mailing lists Message-ID: Hi Mailman Developers. I am sending this mail as my proposal of encrypted mailing lists for GNU Mailman got accepted and I will be working on it this summer. Sorry about not contacting you earlier, I had some issues where my site and mail server were down. If any of you tried to reach me and failed in the last ~week you can try next time on my backup mail jancar.jj at gmail.com which should always work. You can find the original (accepted) version of my proposal on: https://neuromancer.sk/static/mailman.pdf # Status report so far into the Community bonding period: - As it was proposed on this list a plugin-like implementation of encrypted mailing lists is really the only way to go forward here, as just pushing in what might end up being a rather niche feature into Mailman Core is not maintainable / wanted. - I started evaluating how much of my current proposal can be implemented without touching Mailman Core at all (as a plugin), what would require general changes and what might require specific changes (that means it needs a better solution). - So far it seems most functionality of encrypted mailing lists can be easily implemented out-of-tree with only minor general changes to Mailman Core with the following exceptions that I'm currently solving: * Making all commands require a confirmation (as subscribe / unsubscribe has). This is necessary to stop replay attacks. * Subscription command needs to contain users public key, either as an argument or attachment or any other way the plugin might get it. * List key fingerprint and per-address/user key fingerprints need to be stored somehow, directly in the mailing list model would make the most sense, but that's very specific so the plugin should store this itself. Although that means data duplication. * Plugins don't seem to have a way to add features to the core REST API, so exposing key administration for Postorious that way is out. + Some questions that I had in my original proposal: + Is exposing key management through the REST api and Postorius a good idea at all? Those have very different level of access control, changing a key on a list requires a signed request + signed confirmation token whereas doing it in Postorius might only require a password. + A way of sharing the lists public key that makes the user trust it the most. # What I would like to definitely finish in the Community bonding period: - Finish SMTPS/STARTTLS support for Mailman Core (really only needs tests now): https://gitlab.com/J08nY/mailman/tree/mta-smtps-starttls - Establish real-time communication channels with mentors (text/voice?) and have a meeting to discuss the proposal. - Add proper objective milestones to the proposal. - Change the proposal to reflect movement towards a more plugin-like implementation. Cheers, -- Jan ______________________________________________________ /\ # PGP: 362056ADA8F2F4E421565EF87F4A448FE68F329D /__\ # https://neuromancer.sk /\ /\ # Eastern Seaboard Phishing Authority /__\/__\ # -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Thu May 11 14:46:28 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 11 May 2017 11:46:28 -0700 Subject: [Mailman-Developers] Signaling One-Click Functionality for List Email Headers In-Reply-To: <20170510191818.30228.qmail@ary.lan> References: <20170510191818.30228.qmail@ary.lan> Message-ID: On 05/10/2017 12:18 PM, John Levine wrote: > In article <20170510133609.61fba76f at subdivisions.wooz.org> you write: >> I probably need more convincing that it would actually be used out in the >> field, ... > > Gmail's already implemented it. I'm pretty sure Yahoo is also planning to. Are you saying Google Groups and maybe Yahoo Groups are adding the headers or their web mail clients are/will be supporting it? Given that Gmail's web mail client doesn't seem to offer "reply to list" when there's a List-Post: header, the latter seems unlikely. I have tried one-click unsubscribe in message footers, and this is generally not a good idea because of people unwittingly replying/forwarding a post with their personal link to others and then others unsubscribing the first person thinking they are unsubscribing themselves. This is a header, so it normally won't be included in a forward or reply, so that issue is moot and makes the RFC 8058 method attractive. I agree with Barry that personalization is not the big performance hit it once was, but there are admins that don't want to enable Mailman's VERP or allow personalization because they think it is. In MM 2.1 at least, we could avoid that issue by only adding the personalized unsubscribe link if the delivery were already personalized or VERPed. I'm still concerned that mainstream MUAs including web mail clients won't support it for some time if ever. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From johnl at taugh.com Thu May 11 21:59:41 2017 From: johnl at taugh.com (John Levine) Date: 12 May 2017 01:59:41 -0000 Subject: [Mailman-Developers] Signaling One-Click Functionality for List Email Headers In-Reply-To: Message-ID: <20170512015941.35511.qmail@ary.lan> In article you write: >On 05/10/2017 12:18 PM, John Levine wrote: >> In article <20170510133609.61fba76f at subdivisions.wooz.org> you write: >>> I probably need more convincing that it would actually be used out in the >>> field, ... >> >> Gmail's already implemented it. I'm pretty sure Yahoo is also planning to. > >Are you saying Google Groups and maybe Yahoo Groups are adding the >headers or their web mail clients are/will be supporting it? The latter. If you mark something as junk in the gmail client and it has the appropriate headers, Gmail will offer the option to unsubscribe, and that'll be one-click. Remember that this is mostly intended for bulk advertising mail which gets reported as spam all the time. >I agree with Barry that personalization is not the big performance hit >it once was, but there are admins that don't want to enable Mailman's >VERP or allow personalization because they think it is. Then they can't use one-click, either. Their loss. Speaking of Gmail, they tell me they're now adding ARC headers on mail forwarded through Google Groups and looking at ARC when making spam filtering decisions. >I'm still concerned that mainstream MUAs including web mail clients >won't support it for some time if ever. Like I said, Gmail supports it now. R's, John From j.knight at keele.ac.uk Fri May 12 08:13:50 2017 From: j.knight at keele.ac.uk (Jonathan Knight) Date: Fri, 12 May 2017 13:13:50 +0100 Subject: [Mailman-Developers] Mailing lists exploited Message-ID: Hi Our mailman lists were attacked this morning successfully sending spam to a large number of our users. The method was to use the list administrator address found on the public facing web interface (see here https://mail.python.org/mailman/listinfo/mailman-developers for an example). The X at Y form doesn't pose much of a challenge. They then crafted email addresses in the envelope sender which matched the sending IP numbers so our SPF checks passed, but used the list administrator address in the From: field which avoided moderation in a number of our lists. Many of our list administrators either didn't use moderation, or explicitly allowed their own address to post without moderation. I've removed the administrator address display on our lists (thus cleverly bolting the stable door) and I'm turning on moderation for all administrator addresses and also checking the sender filters for addresses that bypass moderation. So far it's just caused a bit of a flap and made list administrators wonder if their email account was hacked. Maybe listing administrator email addresses needs the be a thing of the past. -- Jonathan Knight IT Services Keele University From raj.abhilash1 at gmail.com Sun May 14 02:18:16 2017 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Sun, 14 May 2017 06:18:16 +0000 Subject: [Mailman-Developers] [GSoC] Encrypted mailing lists In-Reply-To: References: Message-ID: <1494741395.c8x8nrccpo.maxking@angel> Hi Jan, Congratulations and welcome to Mailman! Excerpts from Jan Jancar's message of May 10, 2017 5:07 pm: > Hi Mailman Developers. > > I am sending this mail as my proposal of encrypted mailing lists for > GNU Mailman got accepted and I will be working on it this summer. Awesome! > Sorry about not contacting you earlier, I had some issues where > my site and mail server were down. If any of you tried to reach me > and failed in the last ~week you can try next time on my backup mail > jancar.jj at gmail.com which should always work. > > You can find the original (accepted) version of my proposal on: > https://neuromancer.sk/static/mailman.pdf > > # Status report so far into the Community bonding period: > > - As it was proposed on this list a plugin-like implementation of > encrypted mailing lists is really the only way to go forward here, > as just pushing in what might end up being a rather niche feature > into Mailman Core is not maintainable / wanted. I feel like core already has the architecture(interfaces everywhere! :) that will make it pretty easy to write plugins. If you feel you need some changes in core to support your plugin better or plugins in general, I should be able to help you with that part. > - I started evaluating how much of my current proposal can be > implemented without touching Mailman Core at all (as a plugin), > what would require general changes and what might require specific > changes (that means it needs a better solution). > - So far it seems most functionality of encrypted mailing lists > can be easily implemented out-of-tree with only minor general changes > to Mailman Core with the following exceptions that I'm currently solving: > > * Making all commands require a confirmation (as subscribe / unsubscribe > has). This is necessary to stop replay attacks. > > * Subscription command needs to contain users public key, either as an > argument or attachment or any other way the plugin might get it. > > * List key fingerprint and per-address/user key fingerprints need > to be stored somehow, directly in the mailing list model would make > the most sense, but that's very specific so the plugin should store > this itself. Although that means data duplication. > > * Plugins don't seem to have a way to add features to the core REST > API, so exposing key administration for Postorious that way is out. > > + Some questions that I had in my original proposal: > + Is exposing key management through the REST api and Postorius a > good idea at all? Those have very different level of access control, > changing a key on a list requires a signed request + signed confirmation > token whereas doing it in Postorius might only require a password. True, but there is a lot of trust already there on the password for postorius. What if someone un-subscribes from the Postorius and then re-subscribes sending along a key not owned by the user? I don't know if that did make any sense, because as I understand the subscription would be moderated and it would be up to List Admin to not allow keys he doesn't recognize to be subscribed? Is there anything else except the admin stopping some attacker from doing that? > + A way of sharing the lists public key that makes the user trust it > the most. I feel the key signed by the List Owner would be the best way to indentify the lists public key. Maybe mandate signing by the Site Owner and List Owner/List Owners? > # What I would like to definitely finish in the Community bonding period: > > - Finish SMTPS/STARTTLS support for Mailman Core (really only needs > tests now): https://gitlab.com/J08nY/mailman/tree/mta-smtps-starttls > - Establish real-time communication channels with mentors (text/voice?) > and have a meeting to discuss the proposal. I am available as maxking on IRC(#mailman). I am a little busy for next week and then we have Pycon, but I should be able to meet you anytime after Friday 26th. I am not very sure, but I will have some time on 17th too, I will let you know? > - Add proper objective milestones to the proposal. > - Change the proposal to reflect movement towards a more plugin-like > implementation. > > I hope this summer is fun for you! thanks, Abhilash -- thanks, Abhilash Raj -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: not available URL: From mark at msapiro.net Mon May 15 14:03:52 2017 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 15 May 2017 11:03:52 -0700 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: References: Message-ID: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> On 05/12/2017 05:13 AM, Jonathan Knight wrote: > > Maybe listing administrator email addresses needs the be a thing of the > past. It's not done in Mailman 3. For mailman 2.1, the administrator email addresses are a mailto: link the goes to the LISTNAME-owner address, but the email addresses are exposed and only mildly obfuscated ('@' -> ' at '). I would consider adding a configuration option to either obfuscate the addresses further (e.g. drop the domain entirely) or replace the text with something like "Listname list run by listname-owner at example.com". WDOT? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Mon May 15 18:19:09 2017 From: barry at list.org (Barry Warsaw) Date: Mon, 15 May 2017 18:19:09 -0400 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> Message-ID: <20170515181909.22f56dc2@subdivisions.wooz.org> On May 15, 2017, at 11:03 AM, Mark Sapiro wrote: >It's not done in Mailman 3. > >For mailman 2.1, the administrator email addresses are a mailto: link the >goes to the LISTNAME-owner address, but the email addresses are exposed and >only mildly obfuscated ('@' -> ' at '). > >I would consider adding a configuration option to either obfuscate the >addresses further (e.g. drop the domain entirely) or replace the text with >something like "Listname list run by listname-owner at example.com". I'm a little confused by the OP. Is it: 1) A message to the posting address From: LISTNAME-owner at example.com is not being moderated? I would expect it to be since that address is not a member of the list. 2) Emailing To: LISTNAME-owner at example.com directly which would end up spamming the list owners? MM3 doesn't currently moderate messages sent to the list owners, but it could. Messages to -owners flows through a different, shorter chain of rules and pipeline, but I've always thought that that would be configurable. -Barry From mark at msapiro.net Mon May 15 18:41:30 2017 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 15 May 2017 15:41:30 -0700 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: <20170515181909.22f56dc2@subdivisions.wooz.org> References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> Message-ID: <6b78f112-3d46-8670-16da-d9e74373aabf@msapiro.net> On 05/15/2017 03:19 PM, Barry Warsaw wrote: > > I'm a little confused by the OP. Is it: > > 1) A message to the posting address From: LISTNAME-owner at example.com is not > being moderated? I would expect it to be since that address is not a member > of the list. > > 2) Emailing To: LISTNAME-owner at example.com directly which would end up > spamming the list owners? I don't think it's either. I think it is scraping the list owner addresses from the LISTNAME run by joe at example.com line on the web UI pages, s/ at /@/ and spoofing that address as the sender of a spam post to the list. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From j.knight at keele.ac.uk Tue May 16 04:29:12 2017 From: j.knight at keele.ac.uk (Jonathan Knight) Date: Tue, 16 May 2017 09:29:12 +0100 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: <20170515181909.22f56dc2@subdivisions.wooz.org> References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> Message-ID: Mark is right. The spamming process was to scrape the listinfo page and locate the "list is run by" line and then de-obfuscate the "j.knight at keele.ac.uk" into " j.knight at keele.ac.uk". Then an email was faked using j.knight at keele.ac.uk as the sender to see if the list is either unmoderated or whether the administrator had set their own email address as unmoderated on a moderated list. There's not a lot that can be done to protect against that other than changing the "list is run by" so that the administrators real email address isn't obvious. Jon. On 15 May 2017 at 23:19, Barry Warsaw wrote: > On May 15, 2017, at 11:03 AM, Mark Sapiro wrote: > > >It's not done in Mailman 3. > > > >For mailman 2.1, the administrator email addresses are a mailto: link the > >goes to the LISTNAME-owner address, but the email addresses are exposed > and > >only mildly obfuscated ('@' -> ' at '). > > > >I would consider adding a configuration option to either obfuscate the > >addresses further (e.g. drop the domain entirely) or replace the text with > >something like "Listname list run by listname-owner at example.com". > > I'm a little confused by the OP. Is it: > > 1) A message to the posting address From: LISTNAME-owner at example.com is > not > being moderated? I would expect it to be since that address is not a > member > of the list. > > 2) Emailing To: LISTNAME-owner at example.com directly which would end up > spamming the list owners? > > MM3 doesn't currently moderate messages sent to the list owners, but it > could. Messages to -owners flows through a different, shorter chain of > rules > and pipeline, but I've always thought that that would be configurable. > > -Barry > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/ > mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman- > developers/j.knight%40keele.ac.uk > > Security Policy: http://wiki.list.org/x/QIA9 > -- Jonathan Knight IT Services Keele University From barry at list.org Tue May 16 07:58:03 2017 From: barry at list.org (Barry Warsaw) Date: Tue, 16 May 2017 07:58:03 -0400 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> Message-ID: <20170516075803.11bc50dd@subdivisions.wooz.org> On May 16, 2017, at 09:29 AM, Jonathan Knight wrote: >There's not a lot that can be done to protect against that other than >changing the "list is run by" so that the administrators real email address >isn't obvious. I suppose we should either use the moderator's real name, or just the local part of their address. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From j.knight at keele.ac.uk Tue May 16 08:29:21 2017 From: j.knight at keele.ac.uk (Jonathan Knight) Date: Tue, 16 May 2017 13:29:21 +0100 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: <20170516075803.11bc50dd@subdivisions.wooz.org> References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> <20170516075803.11bc50dd@subdivisions.wooz.org> Message-ID: Hi Barry I think the real name if its available and the list owner address if not. If you use the local part (e.g. j.knight) would still make it possible to guess the @keele.ac.uk if the mailing lists are all hosted on maillists.keele.ac.uk. I can't think of a better solution. Jon. On 16 May 2017 at 12:58, Barry Warsaw wrote: > On May 16, 2017, at 09:29 AM, Jonathan Knight wrote: > > >There's not a lot that can be done to protect against that other than > >changing the "list is run by" so that the administrators real email > address > >isn't obvious. > > I suppose we should either use the moderator's real name, or just the local > part of their address. > > -Barry > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/ > mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman- > developers/j.knight%40keele.ac.uk > > Security Policy: http://wiki.list.org/x/QIA9 > > -- Jonathan Knight IT Services Keele University From david.terni at gmail.com Tue May 16 04:30:38 2017 From: david.terni at gmail.com (David Terni) Date: Tue, 16 May 2017 10:30:38 +0200 Subject: [Mailman-Developers] Add information on first line of email Message-ID: Hi, I need to use the text of "description or information filed" used to describe the mailinglist and put this information in the firsts lines e-mail. Like the credits but on head of email. How I can do it? Thank's a lot From johny at neuromancer.sk Tue May 16 09:10:14 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Tue, 16 May 2017 15:10:14 +0200 Subject: [Mailman-Developers] [GSoC] Encrypted mailing lists In-Reply-To: <1494741395.c8x8nrccpo.maxking@angel> References: <1494741395.c8x8nrccpo.maxking@angel> Message-ID: <62d5770e-63f3-9113-8eb2-93d90d9634c8@neuromancer.sk> Hey Abhilash! On 05/14/2017 08:18 AM, Abhilash Raj wrote: >> - As it was proposed on this list a plugin-like implementation of >> encrypted mailing lists is really the only way to go forward here, >> as just pushing in what might end up being a rather niche feature >> into Mailman Core is not maintainable / wanted. > > I feel like core already has the architecture(interfaces everywhere! :) that > will make it pretty easy to write plugins. If you feel you need some changes in > core to support your plugin better or plugins in general, I should be able to > help you with that part. Yes, so far it has everything necessary. Some things I noted: - A plugin cannot create a Pipeline the same way it creates Handlers or Rules, it can only do so in a post_hook. Since the Pipeline classes are enumerated when initializing them. https://gitlab.com/mailman/mailman/blob/master/src/mailman/core/pipelines.py#L150 - And then the issues I outlined in my previous email, which mainly stem from the encrypted lists plugin having some pretty strong requirements on current Mailman features. >> + Some questions that I had in my original proposal: >> + Is exposing key management through the REST api and Postorius a good idea at all? Those have very different level of access control, >> changing a key on a list requires a signed request + signed confirmation >> token whereas doing it in Postorius might only require a password. > > True, but there is a lot of trust already there on the password for > postorius. What if someone un-subscribes from the Postorius and then > re-subscribes sending along a key not owned by the user? > > I don't know if that did make any sense, because as I understand the > subscription would be moderated and it would be up to List Admin to not allow > keys he doesn't recognize to be subscribed? Is there anything else except the > admin stopping some attacker from doing that? Sure, subscription will be moderated and the List Admin will have to trust both the address he is accepting and the key provided. However, this is something we would like to stop, someone unsubscribing a user from an encrypted mailing list with just a password and not access to his private key. This is something that definitely needs comments. What actions should a subscriber / unsubscribed user be able to do on an encrypted mailing list with: - just a password - his private key and what parts of this should be configurable and in what way / granularity? I propose the following: - all email commands should require a signed confirmation (the same workflow as subscribe / unsubscribe), to stop replay of commands. - posts should require being encrypted and signed, and this should be configurable per list. [require/allow encrypted, require/allow signed] - Postorius user operations should also somehow require signed confirmation. > >> + A way of sharing the lists public key that makes the user trust it >> the most. > > I feel the key signed by the List Owner would be the best way to indentify the > lists public key. Maybe mandate signing by the Site Owner and List Owner/List > Owners? Right, I also think that utilizing PGP web-of-trust the most, is the best we can do here. Also just recently found out, schleuder (an encrypted mailing list server) supports uploading signed list keys to it a will serve them with the signatures from then on. >> # What I would like to definitely finish in the Community bonding period: >> >> - Finish SMTPS/STARTTLS support for Mailman Core (really only needs tests now): https://gitlab.com/J08nY/mailman/tree/mta-smtps-starttls >> - Establish real-time communication channels with mentors (text/voice?) >> and have a meeting to discuss the proposal. > > I am available as maxking on IRC(#mailman). I am a little busy for next week and > then we have Pycon, but I should be able to meet you anytime after Friday > 26th. I am not very sure, but I will have some time on 17th too, I will let you > know? Sure, I'm in #mailman and I have some questions better suited for irc, so you'll definitely hear me there. > >> - Add proper objective milestones to the proposal. >> - Change the proposal to reflect movement towards a more plugin-like >> implementation. >> >> > > I hope this summer is fun for you! I think it will! Cheers, -- Jan ______________________________________________________ /\ # PGP: 362056ADA8F2F4E421565EF87F4A448FE68F329D /__\ # https://neuromancer.sk /\ /\ # Eastern Seaboard Phishing Authority /__\/__\ # -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From dkg at fifthhorseman.net Tue May 16 23:17:16 2017 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Tue, 16 May 2017 23:17:16 -0400 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> <20170516075803.11bc50dd@subdivisions.wooz.org> Message-ID: <87h90kdver.fsf@fifthhorseman.net> On Tue 2017-05-16 13:29:21 +0100, Jonathan Knight wrote: > I think the real name if its available and the list owner address if not. > If you use the local part (e.g. j.knight) would still make it possible to > guess the @keele.ac.uk if the mailing lists are all hosted on > maillists.keele.ac.uk. surely it's easy for an attacker to guess moderation-free sender addresses by a quick scan of the list archives as well. what attackers are we really trying to defend against here? --dkg From mark at msapiro.net Tue May 16 23:52:01 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 16 May 2017 20:52:01 -0700 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: <87h90kdver.fsf@fifthhorseman.net> References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> <20170516075803.11bc50dd@subdivisions.wooz.org> <87h90kdver.fsf@fifthhorseman.net> Message-ID: <290afc60-653a-c25e-f283-4435ae19ce4a@msapiro.net> On 05/16/2017 08:17 PM, Daniel Kahn Gillmor wrote: > > surely it's easy for an attacker to guess moderation-free sender > addresses by a quick scan of the list archives as well. Only if there are public archives. I realized I am more or less immune from this attack for my several production lists. The lists are all @example.org (obviously not the real domain) and the list owner is listmanager at example.org which is a forwarder to the real list admins and is not a member or authorized poster of any of the lists. It was set up this way because we have a number of such forwarders for various functions and having a generic address for a function is a convenience that avoids people mailing the wrong people when responsibilities change, but a side benefit is the address exposed on web pages can't post without moderation, plus one could add it to discard_these_nonmembers and never see posts From: that address. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From j.knight at keele.ac.uk Wed May 17 04:20:21 2017 From: j.knight at keele.ac.uk (Jonathan Knight) Date: Wed, 17 May 2017 09:20:21 +0100 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: <87h90kdver.fsf@fifthhorseman.net> References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> <20170516075803.11bc50dd@subdivisions.wooz.org> <87h90kdver.fsf@fifthhorseman.net> Message-ID: The attack we're trying to defend against is a scripted one which grabs a list of all the mailing lists, then harvests the administrator email and then tries to spam each list using the administrator as a sender address. If the archives are public then I guess you could write a reasonable algorithm to try and guess an unmoderated address but I don't think its as easy to hit thousands of mailing lists using that approach. Jon On 17 May 2017 at 04:17, Daniel Kahn Gillmor wrote: > On Tue 2017-05-16 13:29:21 +0100, Jonathan Knight wrote: > > > I think the real name if its available and the list owner address if not. > > If you use the local part (e.g. j.knight) would still make it possible to > > guess the @keele.ac.uk if the mailing lists are all hosted on > > maillists.keele.ac.uk. > > surely it's easy for an attacker to guess moderation-free sender > addresses by a quick scan of the list archives as well. what attackers > are we really trying to defend against here? > > --dkg > -- Jonathan Knight IT Services Keele University From dkg at fifthhorseman.net Wed May 17 10:57:57 2017 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Wed, 17 May 2017 10:57:57 -0400 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> <20170516075803.11bc50dd@subdivisions.wooz.org> <87h90kdver.fsf@fifthhorseman.net> Message-ID: <87d1b7cyyy.fsf@fifthhorseman.net> On Wed 2017-05-17 09:20:21 +0100, Jonathan Knight wrote: > The attack we're trying to defend against is a scripted one which grabs a > list of all the mailing lists, then harvests the administrator email and > then tries to spam each list using the administrator as a sender address. > > If the archives are public then I guess you could write a reasonable > algorithm to try and guess an unmoderated address but I don't think its as > easy to hit thousands of mailing lists using that approach. i'm not convinced that these two scripts are significantly different in difficulty, though i acknowledge that the former is marginally easier. it sounds to me like the real underlying concern is about allowing submissions to bypass moderation based on forgeable data like the From: header. fixing it in the display side seems likely to trigger a game of whack-a-mole. --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From j.knight at keele.ac.uk Wed May 17 11:08:50 2017 From: j.knight at keele.ac.uk (Jonathan Knight) Date: Wed, 17 May 2017 16:08:50 +0100 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: <87d1b7cyyy.fsf@fifthhorseman.net> References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> <20170516075803.11bc50dd@subdivisions.wooz.org> <87h90kdver.fsf@fifthhorseman.net> <87d1b7cyyy.fsf@fifthhorseman.net> Message-ID: Hi Daniel Our use case is that most (but not all) of our lists are internal and so the archives are not public. However the listinfo pages are public for the few public lists that we run and to allow of campus staff and students to access the list management screens. So for us, hiding the list administrator email on the list info pages effectively cuts off the ability to get a prospective list of possible administrators. But I agree that for public lists with public archives the benefit is minimal, but I don't think it does much harm Jon On 17 May 2017 at 15:57, Daniel Kahn Gillmor wrote: > On Wed 2017-05-17 09:20:21 +0100, Jonathan Knight wrote: > > The attack we're trying to defend against is a scripted one which grabs a > > list of all the mailing lists, then harvests the administrator email and > > then tries to spam each list using the administrator as a sender address. > > > > If the archives are public then I guess you could write a reasonable > > algorithm to try and guess an unmoderated address but I don't think its > as > > easy to hit thousands of mailing lists using that approach. > > i'm not convinced that these two scripts are significantly different in > difficulty, though i acknowledge that the former is marginally easier. > > it sounds to me like the real underlying concern is about allowing > submissions to bypass moderation based on forgeable data like the From: > header. fixing it in the display side seems likely to trigger a game of > whack-a-mole. > > --dkg > -- Jonathan Knight IT Services Keele University From aurelien at bompard.org Thu May 18 13:01:27 2017 From: aurelien at bompard.org (Aurelien Bompard) Date: Thu, 18 May 2017 19:01:27 +0200 Subject: [Mailman-Developers] Multiple REST servers Message-ID: Hey people! We now have enough lists and activity that Mailman's REST API is becoming a bottleneck, since it only answers one request at a time. Is there a way to start mutiple REST runners? Or to make the REST runner multithreaded? Thanks! Aur?lien From barry at list.org Fri May 19 13:50:17 2017 From: barry at list.org (Barry Warsaw) Date: Fri, 19 May 2017 10:50:17 -0700 Subject: [Mailman-Developers] Multiple REST servers In-Reply-To: References: Message-ID: <20170519105017.73d884ad@presto> On May 18, 2017, at 07:01 PM, Aurelien Bompard wrote: >We now have enough lists and activity that Mailman's REST API is >becoming a bottleneck, since it only answers one request at a time. >Is there a way to start mutiple REST runners? Or to make the REST >runner multithreaded? Thanks for identifying this Aurelien. Could you open an issue on Core for this? Have you done any other analysis on where the bottlenecks are? Is it CPU, db, I/O? Currently the REST server uses Python's stdlib wsgiref WSGIServer which I'm positive is not performant enough for your lists. There are a couple of options that would be worth investigating. I played with trying to use gunicorn, but deleted the branch a while ago (IIRC, there were problems and I didn't have time to investigate further). At the very least, I want to look into asyncio-based REST server (e.g. aiohttp, uvloop), but maybe there are other parts of the stack we need to look at too. -Barry From aurelien at bompard.org Fri May 19 13:51:52 2017 From: aurelien at bompard.org (Aurelien Bompard) Date: Fri, 19 May 2017 19:51:52 +0200 Subject: [Mailman-Developers] Recent changes in HyperKitty Message-ID: Hey folks! I thought I'd keep you posted on a rather big change that landed in HyperKitty a couple weeks ago. It will be useful for those of you who run the master branch. If you updated and HyperKitty started requiring Redis or not updating some parts of the UI, that may be the reason. I have added support for asychronous tasks. They bring a huge performance boost but they need an additional step in the setup. The relevant part of the documentation is: http://hyperkitty.readthedocs.io/en/latest/install.html#asynchronous-tasks There are two changes to the setup: First, the configuration must be updated, there's one more line in the INSTALLED_APPS and there an additional variable that must be defined if you don't want to depend on Redis. Here's the diff proposed in the example project directory: https://gitlab.com/mailman/hyperkitty/commit/681170c6986b7bf3a2731845196f9393ea0c8e84#8e136c7b1cda5d4f447a88940dbacf003e1aa32f. If you do this change (and run the django migrate command), you don't need to install Redis. The task queue will be in Django's own database. Secondly, there is an additional command to run in the background: django-admin qcluster --pythonpath /path/to/project --settings settings or: /path/to/project/manage.py qcluster depending on how you choose to run Django commands This command will start the workers that will process the work queue. The work queue is (for now) mainly about updating the cache, that's why you may see those weird inconstitancies in the UI. Sorry for not communicating about this earlier. If you have any questions, feel free to ask me. Aur?lien From aurelien at bompard.org Fri May 19 13:59:54 2017 From: aurelien at bompard.org (Aurelien Bompard) Date: Fri, 19 May 2017 19:59:54 +0200 Subject: [Mailman-Developers] Multiple REST servers In-Reply-To: <20170519105017.73d884ad@presto> References: <20170519105017.73d884ad@presto> Message-ID: > Could you open an issue on Core for this? There you go: https://gitlab.com/mailman/mailman/issues/327 > Have you done any other analysis on where the bottlenecks are? Is it CPU, db, > I/O? I haven't investigated properly but it seems to be the database (which is on another host). > Currently the REST server uses Python's stdlib wsgiref WSGIServer which I'm > positive is not performant enough for your lists. There are a couple of > options that would be worth investigating. I played with trying to use > gunicorn, but deleted the branch a while ago (IIRC, there were problems and I > didn't have time to investigate further). At the very least, I want to look > into asyncio-based REST server (e.g. aiohttp, uvloop), but maybe there are > other parts of the stack we need to look at too. Interesting. I wonder if we can have SQLAlchemy play well with asyncio, I haven't tried that before. There is still a gunicorn.py file in the contrib directory, but I haven't tested it either. Aur?lien From turnbull.stephen.fw at u.tsukuba.ac.jp Sat May 20 12:42:52 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Sun, 21 May 2017 01:42:52 +0900 Subject: [Mailman-Developers] Signaling One-Click Functionality for List Email Headers In-Reply-To: <20170510170536.29935.qmail@ary.lan> References: <20170510120723.xgg3apmj65cmvy3z@sys4.de> <20170510170536.29935.qmail@ary.lan> Message-ID: <22816.29196.804602.506686@turnbull.sk.tsukuba.ac.jp> John Levine writes: >>>>> Patrick wrote: > >Is that something mailman(2|3) should support? To me it looks > >useful. Personally I am +1 on "patches welcome" until users start asking for it (including via list and site managers). Until I see evidence of intent to use, I'm not excited about putting the effort into developing and maintaining it myself. > It would certainly make it easier to deal with grumpy gmail users, > since gmail does not provide junk button feedback. > > The disadvantage is that every recipient needs to get a separate copy > of each message, because the list and user info has to be encoded in > the list-unsubscribe URL. That's been standard in commercial e-mail > for a decade, but a lot of discussion list operators still imagine > that it's too slow. I wouldn't have a problem with experimenting with enabling personalization by default in Mailman 3 to get experience with it. I would oppose it in Mailman 2 at this point in its lifecycle because throttling of hosted lists is still a FAQ, and (at least theoretically) multiple recipient transactions can alleviate those limits. Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Sat May 20 12:45:38 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Sun, 21 May 2017 01:45:38 +0900 Subject: [Mailman-Developers] Mailing lists exploited In-Reply-To: References: <0376b710-1007-64f2-aa69-4de9bb98c8db@msapiro.net> <20170515181909.22f56dc2@subdivisions.wooz.org> <20170516075803.11bc50dd@subdivisions.wooz.org> Message-ID: <22816.29362.336811.545091@turnbull.sk.tsukuba.ac.jp> Jonathan Knight writes: > I think the real name if its available and the list owner address if not. > If you use the local part (e.g. j.knight) would still make it possible to > guess the @keele.ac.uk if the mailing lists are all hosted on > maillists.keele.ac.uk. I agree with Barry. More precisely, I think we should more or less hard-code the $LIST-owner address in the mail-to URL, allow the display name (presented in the HTML) to be specified (defaulting to "$LIST-owner", maybe), and document that the list-owner address should NOT be given any special permissions (specifically, should not be subscribed to the list), and that a subscribed address should NOT be mentioned in that text. I believe the $LIST-owner address is handled by Mailman, so we can require that be configured when setting up the list. This setup is just a BCP anyway in the "modern" Internet. Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Sat May 20 12:47:31 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Sun, 21 May 2017 01:47:31 +0900 Subject: [Mailman-Developers] Add information on first line of email In-Reply-To: References: Message-ID: <22816.29475.717774.629478@turnbull.sk.tsukuba.ac.jp> David Terni writes: > Hi, > I need to use the text of "description or information filed" used to > describe the mailinglist and put this information in the firsts lines > e-mail. Like the credits but on head of email. > > How I can do it? You'll have to manually copy it to the "Header" options. There are two of them, one each for the "Digest options" and the other for "Non-digest options". If you want to do this programmatically across your site without the manual copy, you'll need to write a script. It would also be possible to patch the code to automatically pull in the "description" string, but I doubt we want to support that in Mailman 2. If you want help with those, just ask! (on list, there are several of us who can help -- note we're mostly at PyCon so responses may be delayed). Steve From johnl at taugh.com Sat May 20 13:18:34 2017 From: johnl at taugh.com (John R Levine) Date: 20 May 2017 13:18:34 -0400 Subject: [Mailman-Developers] Signaling One-Click Functionality for List Email Headers In-Reply-To: <22816.29196.804602.506686@turnbull.sk.tsukuba.ac.jp> References: <20170510120723.xgg3apmj65cmvy3z@sys4.de> <20170510170536.29935.qmail@ary.lan> <22816.29196.804602.506686@turnbull.sk.tsukuba.ac.jp> Message-ID: On Sun, 21 May 2017, Stephen J. Turnbull wrote: > > >Is that something mailman(2|3) should support? To me it looks > > >useful. > > Personally I am +1 on "patches welcome" until users start asking for > it (including via list and site managers). Until I see evidence of > intent to use, I'm not excited about putting the effort into > developing and maintaining it myself. I wouldn't disagree. It's nice, but I wouldn't put it at the top of the list of things to add. One-click is much more important for commercial mailers who tend to add people to lists who don't want to be there (e.g., everyone who ever provides an e-mail address with an order) and need to make unsubscribes as easy as possible. If a recipient ends up on a Mailman list without wanting to be there, something's wrong. Regards, John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail. https://jl.ly From johny at neuromancer.sk Sat May 20 17:00:18 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Sat, 20 May 2017 23:00:18 +0200 Subject: [Mailman-Developers] [GSoC] Encrypted mailing lists In-Reply-To: References: Message-ID: <4df86de1-9abb-2c72-b28a-7cb9ecf6d940@neuromancer.sk> Hi all. Currently still looking into what changes will be necessary in core in order to cleanly implement encrypted mailing lists as a plugin, so this is my second status report. I will start doing these pretty much weekly now, and definitely weekly/ more frequent when the coding period starts. All of my reports (currently only this one) can be found on: https://neuromancer.sk/ https://neuromancer.sk/recent.atom (https://neuromancer.sk/article/2) Also pasting it here for clarity and replies: ## Plugin API enhancements in Core To cleanly implement encrypted mailing lists as a plugin to Mailman Core I propose several general changes to the plugin api, to allow for cleaner integration of plugins, more flexibility and easier plugin deployment. First I present the current state of pluggability in Mailman core and then present the proposed changes. ### Current state Relevant mailman-developers thread from GSoC 2015: https://mail.python.org/pipermail/mailman-developers/2015-March/024477.html Example plugin: http://threebean.org/blog/plugins-for-mailman3/ * A plugin creator has many ways of "injecting" his code to run at certain phases of Mailman's operation, since Mailman looks for its classes and components dynamically, it doesn't care whether they are from a plugin or originally from core. - Implementing IHandler, IChain, IRule, IEmailCommand, ICLISubCommand or IStyle and placing modules containing the classes in the appropriate directories where Mailman finds them and instantiates them. - Implementing IRunner and adding the custom runner to mailman.cfg. - Implementing IArchiver and adding the custom archiver to mailman.cfg. - Implementing IMailTransportAgentLifecycle and setting the custom MTA in mailman.cfg. - Setting a custom callable in pre_hook or post_hook (only one callable per hook). * Core config variable ext_dir is unused. * There is not much documentation / examples of plugin creation, however core is very well documented so it's just a matter of figuring out what's pluggable or not. ### Proposed changes * Add configuration option similar to config.styles.paths to Handlers, Rules, Chains etc.. Which Mailman will use to look for components, so that plugins can just be accessible on the python path and not necessarily inside the Mailman package. Or use just one path per plugin and a standardized plugin structure: - plugin package - handlers - rules - chains - commands - pipelines - styles * Let plugins add Pipelines the same way they can add Handlers etc... - This means refactoring BasePipeline, OwnerPipeline, PostingPipeline, VirginPipeline from mailman.core.pipelines.py into a package mailman.pipelines - Use find_components * Let plugins subscribe to receive events. * Let List creator specify List Style when creating it through Postorius. * Allow multiple callables in pre_hook and post_hook run in order specified. * Drop ext_dir. Cheers, -- Jan ______________________________________________________ /\ # PGP: 362056ADA8F2F4E421565EF87F4A448FE68F329D /__\ # https://neuromancer.sk /\ /\ # Eastern Seaboard Phishing Authority /__\/__\ # -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From johny at neuromancer.sk Mon May 22 13:14:45 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Mon, 22 May 2017 19:14:45 +0200 Subject: [Mailman-Developers] [GSoC] Encrypted mailing lists - update v2 In-Reply-To: References: Message-ID: Hi all. This is my second update, sketching out the plugin details and issues that came up while working. https://neuromancer.sk/article/3 # PGPMailman plugin ## Structure * pgpmailman - A Core plugin. - styles - Both styles generate a list keypair based on plugin settings on list creation as well as set other attributes for an encrypted mailing list. Such as the custom encrypted chain. - EncryptedDefaultStyle - EncryptedAnnounceStyle - pgp - rules - EncryptionRule - Decrypts message and enforces per-list encryption requirements. - SignatureRule - Checks message signature and enforces per-list signature requirements. Strips signature to msgdata. - chains - EncryptedOwnerChain - Passes message through EncryptionRule before letting it continue default-owner-chain. - EncryptedPostingChain - Passes message through EncryptionRule and SignatureRule before letting it continue default-posting-chain. - commands - KeyEmailCommand - Handles user key management through the key command. - KeyCLICommand - runners - EncryptedOutgoingRunner - Encrypts and optionally signs for configured lists. This is a runner and not a Pipeline since we need to encrypt all outgoing messages, so digests, virgin messages, posts... - archivers - EncryptedHyperKittyArchiver - Fetches list archive public keys from pgphyperkitty, uses them to send messages to archive encrypted, for encrypted lists. * pgphyperkitty- Django app, receives messages encrypted with list archive keys, decrypts them and passes them to HyperKitty. Also generates, holds and provides(public part) list archive keypairs. ## Issues - Rules shouldn't change the message as they process it, since, well they are "rules" not "handlers". However other Mailman's rules need the message plaintext after decrypting for processing, so they cannot receive the message as it arrived encrypted. One solution would be to use a custom Incoming runner to sort-of unwrap the PGP/MIME message from encryption and signature (storing it in msgdata) and then pass it to the default Incoming runner to run chains. This would also make sense from a PGP/MIME standpoint as the "real" message is really the encrypted content so for that's what other Mailman rules should get in the msg object. - Storing of user and list related metadata is unsolved. This means storing the user key fingerprint for each user / address, as well as storing per-list configuration. The list key fingerprint might not need to be stored as the key can be looked up by gnupg.user_id == list_fqdn. The plugin can have it's own db although this means data duplication. A solution I see would be to allow plugins to store arbitrary metadata in the core db attached to certain models. So that the mailinglist model would have another attribute plugin_data of PickleType which would really be a dict with the plugin name as keys, with additional api allowing a plugin to access it's metadata stored in this object. - Extending the Postorious and Hyperkitty interface is unsolved. Since plugins cannot add routes to the REST api. A solution I see that would be general and also require little changes to core would be to let plugins add custom routes to /plugins//... which could be then accessed either from a Postorious plugin or a custom app. Maybe with falcon.API.add_sink()? - Even when extending the REST api at core side is possible as just described, that still leaves integrating into HyperKitty and Postorius somehow, out-of-tree, as a django app. Looking at how HyperKitty and Postorius plug into each other currently, we see direct references at each other in the templates: https://gitlab.com/mailman/postorius/blob/master/src/postorius/templates/postorius/base.html#L48 {% if 'hyperkitty' in INSTALLED_APPS %} {# insert hyperkitty link #} {% endif %} - While that might be okay for integrating HyperKitty and Postorius together, and it's even used at very few places and seems to not be the best solution, it's not practically usable for integrating this plugin. Ideal integration will encrypt messages when sending them to HyperKitty while showing them decrypted to users. I am thinking of having a custom encrypted HyperKitty archiver that will handle encrypting with proper archive key at the core side and a custom django app at the HyperKitty side that will receive the message, decrypt it and pass it to HyperKitty decrypted. That however leaves the messages decrypted in HyperKitty's DB so doesn't really fulfill project requirements. In addition a custom DB engine can be used in HyperKitty's settings that wraps whatever DB engine and encrypts/decrypts objects on the fly, although I am not sure of the doability of this. ## Installation - Just installing the plugin into the same virtualenv as the Mailman instance, and setting up the proper options in mailman.cfg should work. Cheers, -- Jan ______________________________________________________ /\ # PGP: 362056ADA8F2F4E421565EF87F4A448FE68F329D /__\ # https://neuromancer.sk /\ /\ # Eastern Seaboard Phishing Authority /__\/__\ # -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From turnbull.stephen.fw at u.tsukuba.ac.jp Mon May 22 13:53:05 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 23 May 2017 02:53:05 +0900 Subject: [Mailman-Developers] [GSoC] Encrypted mailing lists In-Reply-To: <4df86de1-9abb-2c72-b28a-7cb9ecf6d940@neuromancer.sk> References: <4df86de1-9abb-2c72-b28a-7cb9ecf6d940@neuromancer.sk> Message-ID: <22819.9601.24634.758477@turnbull.sk.tsukuba.ac.jp> Hi Jan! Congratulations on being selected! Note that Mailman is a PSF suborg, so you have to comply with PSF reporting standards. I'll get back to you if you need to do anything more. The only thing I can think of offhand is to register with the PSF "planet" (or whatever they're using to aggregate blog posts). I *think* that's not even ready yet (or maybe they set it up today). My apologies for disappearing on you after the application period, it's been very busy for me (most of my teaching responsibilities are in spring term, April-June, and startup is alway demanding). A few comments on the content. Jan Jancar writes: > ### Current state > > Relevant mailman-developers thread from GSoC 2015: > https://mail.python.org/pipermail/mailman-developers/2015-March/024477.html > > Example plugin: Thanks for the summary. I was wondering what people meant by "plugin", now I know! (I wouldn't call this a "plug-in architecture" to be honest; I'll be interested to study your improvements.) > ### Proposed changes > > * Add configuration option similar to config.styles.paths to > Handlers, Rules, Chains etc.. Which Mailman will use to look > for components, so that plugins can just be accessible on the > python path and not necessarily inside the Mailman package. Not enough, IMO. > Or use just one path per plugin and a standardized plugin > structure: Yes, more structure, please. > * Let plugins add Pipelines the same way they can add Handlers etc... OK. > * Let plugins subscribe to receive events. OK. > * Let List creator specify List Style when creating it through > Postorius. AFAIK this is already an RFE. > * Allow multiple callables in pre_hook and post_hook run > in order specified. Seems reasonable, you'll need to talk to Barry about that. (Of course you can always implement this with a hook-managing hook function. So it's a matter of abackward compatibility. Steve From barry at list.org Mon May 22 15:40:01 2017 From: barry at list.org (Barry Warsaw) Date: Mon, 22 May 2017 12:40:01 -0700 Subject: [Mailman-Developers] HyperKitty issues and MRs for 3.1 Message-ID: <20170522124001.11d4ec7b@presto> Hi Aurelien, We're missing you and Florian at the Pycon sprints, but we have a good group of folks, and we're psyched to release Mailman 3.1 this week. We're noticing that there are no bugs or MRs assigned to milestone 3.1 for HyperKitty so our question is whether that's accurate and we can do a release, or whether we need to assign some to 3.1 and fix them before the release. Here are some issues we have questions about. #104 - Server Error (500) upon login with duplicate email addresses (we're thinking maybe just turning that into a non-500 error?) #127 - Server Error (500) upon signup #128 - hyperkitty/setup.py should require Django<1.11 (Assigned to a sprinter, should be easy) There are also a bunch of merge requests which we're unsure about. Can you do a quick triage and assign anything needed to 3.1 for HyperKitty? Thanks! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From johny at neuromancer.sk Mon May 22 19:21:07 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Tue, 23 May 2017 01:21:07 +0200 Subject: [Mailman-Developers] [GSoC] Encrypted mailing lists In-Reply-To: <22819.9601.24634.758477@turnbull.sk.tsukuba.ac.jp> References: <4df86de1-9abb-2c72-b28a-7cb9ecf6d940@neuromancer.sk> <22819.9601.24634.758477@turnbull.sk.tsukuba.ac.jp> Message-ID: <2908f7b8-651a-2400-bcb2-eaed4bc20090@neuromancer.sk> On 05/22/2017 07:53 PM, Stephen J. Turnbull wrote: > Hi Jan! > > Congratulations on being selected! Thanks! Very happy to be selected. > > Note that Mailman is a PSF suborg, so you have to comply with PSF > reporting standards. I'll get back to you if you need to do anything > more. The only thing I can think of offhand is to register with the > PSF "planet" (or whatever they're using to aggregate blog posts). I > *think* that's not even ready yet (or maybe they set it up today). I only found: https://wiki.python.org/moin/SummerOfCode/Expectations although having a look at planetpython it seems my site can be integrated into it, so no problem there. Should I send an integration request? > My apologies for disappearing on you after the application period, > it's been very busy for me (most of my teaching responsibilities are > in spring term, April-June, and startup is alway demanding). Know that very well myself, as I struggle to finish all school / research related tasks before the coding period starts. > > * Let List creator specify List Style when creating it through > > Postorius. > > AFAIK this is already an RFE. Now that I looked at list styles more closely the way they apply all of their parent classes in their apply method makes using them for plugins kind of heavy handed. Let's say that my encrypted lists plugin wants to work through a list style, which I proposed in my second update, as it's a really nice mechanism for this. Now there are two built-in styles "legacy-default" and "legacy-announce", because of the way list styles compose I will have to create two list styles myself, for "legacy-default" and "legacy-announce". If more styles are added, each of them will have to have a corresponding "encrypted" variant, and this applies to any plugin, or even to core styles and their combination. It might make more sense to have a list of styles associated with a mailing list, which are linearly applied on list creation. The compound styles("legacy-default" and "legacy-announce") would be possible the same way they are now, so something like setting "legacy-default" and "encrypted-mixin" styles on a list would make it an encrypted mailing list. With the current way, the amount of different list-styles would get rather large very quick. Might not be a problem since the amount of list styles I can imagine that would make sense combining is rather low. But just by adding another style, something like "AnonymousListStyle" would require creating 4 styles: anonymous-default anonymous-encrypted-default anonymous-announce anonymous-encrypted-announce and so on. The next style if combinable with all the other styles would need 8 styles. > > * Allow multiple callables in pre_hook and post_hook run > > in order specified. > > Seems reasonable, you'll need to talk to Barry about that. (Of course > you can always implement this with a hook-managing hook function. So > it's a matter of abackward compatibility. As you pointed out out-of-list, there is a MR #264 already for this feature, along with a signal passing between core and plugins, so sending a link for the list: https://gitlab.com/mailman/mailman/merge_requests/264 Cheers, -- Jan ______________________________________________________ /\ # PGP: 362056ADA8F2F4E421565EF87F4A448FE68F329D /__\ # https://neuromancer.sk /\ /\ # Eastern Seaboard Phishing Authority /__\/__\ # -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From aurelien at bompard.org Tue May 23 05:29:06 2017 From: aurelien at bompard.org (Aurelien Bompard) Date: Tue, 23 May 2017 11:29:06 +0200 Subject: [Mailman-Developers] HyperKitty issues and MRs for 3.1 In-Reply-To: <20170522124001.11d4ec7b@presto> References: <20170522124001.11d4ec7b@presto> Message-ID: Hey! :-) I hope you're having a great time at PyCon :-) #104 - Server Error (500) upon login with duplicate email addresses > (we're thinking maybe just turning that into a non-500 error?) > Not sure I can do much to prevent it, it doesn't seem to go through my code at all. And since it's a database integrity problem, a 500 error seems more appropriate. Maybe add a FAQ in the docs? > #127 - Server Error (500) upon signup > This is more of a documentation problem, there's one more command that needs to be run after installing HyperKitty > #128 - hyperkitty/setup.py should require Django<1.11 > (Assigned to a sprinter, should be easy) > Fixed already, thanks to the sprinter for closing the bug. I'll work on the documentation issues, but it shouldn't block you from releasing 3.1. A. From barry at list.org Tue May 23 18:08:16 2017 From: barry at list.org (Barry Warsaw) Date: Tue, 23 May 2017 15:08:16 -0700 Subject: [Mailman-Developers] HyperKitty issues and MRs for 3.1 In-Reply-To: References: <20170522124001.11d4ec7b@presto> Message-ID: <20170523150816.307272ea@presto> Hi Aurelien! On May 23, 2017, at 11:29 AM, Aurelien Bompard wrote: >I hope you're having a great time at PyCon :-) We are! >Fixed already, thanks to the sprinter for closing the bug. > >I'll work on the documentation issues, but it shouldn't block you from >releasing 3.1. Great! And thanks for taking a look at the milestoned issues. It looks like HyperKitty has no more open issues or merge requests milestoned to 3.1, and you're confirming that all HyperKitty systems are "go". Cheers, -Barry From terri at toybox.ca Tue May 23 21:09:49 2017 From: terri at toybox.ca (Terri Oda) Date: Tue, 23 May 2017 18:09:49 -0700 Subject: [Mailman-Developers] Postorius and MRs for 3.1 Message-ID: Hi Florian, Your turn for the "last chance to tag things for 3.1" email now that Aurelien's weighed in. ;) (I'm cc'ing the rest of mailman-developers in case anyone else wants to advocate for a bug too...) I *think* we've got 3.1 tagged appropriately, which means that the unicode bug is the last big one, and I'd like to get the default preferences fixed for good (currently they display correctly but can't save). Do you have anything else that we should look at before we push out RC1? Also, if you have a chance, can you look at https://gitlab.com/mailman/postorius/merge_requests/159/ I put a question about address-based preferences and mailmanclient in there for you, but I could also use a second look for "what's the best way to fix it so these save properly?" if you have a chance. I've broken the django magic and need to override the save method I think. Terri From barry at list.org Wed May 24 00:32:21 2017 From: barry at list.org (Barry Warsaw) Date: Tue, 23 May 2017 21:32:21 -0700 Subject: [Mailman-Developers] Postorius and MRs for 3.1 In-Reply-To: References: Message-ID: <20170523213221.42002017@presto> On May 23, 2017, at 06:09 PM, Terri Oda wrote: >Your turn for the "last chance to tag things for 3.1" email now that >Aurelien's weighed in. ;) (I'm cc'ing the rest of mailman-developers in case >anyone else wants to advocate for a bug too...) I uploaded Core 3.1rc1 to PyPI this afternoon, so barring any unforeseen last minute problems, I plan on just bumping the version number and releasing it. I'm happy for others to do the tagging and releasing the other components, or I can help with that if you prefer. The only project I can't release to PyPI is HyperKitty. Aurelien, we can leave that to you or you can defer to me, but you have to give me upload rights on the HyperKitty (and Core plugin) PyPI project for the latter. I'm hoping we can work on some release notes and an announcement tomorrow while we put the finishing touches on it. 3.0 was released just over 2 years ago. It's exciting to see all the new and fixed stuff in 3.1; I hope you are too! My deep thanks to the core developers, and to all of you in the community. As has been said at Pycon many times this week: come for the code, stay for the community. Cheers, -Barry From barry at list.org Wed May 24 00:47:45 2017 From: barry at list.org (Barry Warsaw) Date: Tue, 23 May 2017 21:47:45 -0700 Subject: [Mailman-Developers] Multiple REST servers In-Reply-To: References: <20170519105017.73d884ad@presto> Message-ID: <20170523214745.58e9bf63@presto> On May 19, 2017, at 07:59 PM, Aurelien Bompard wrote: >> Have you done any other analysis on where the bottlenecks are? Is it CPU, >> db, I/O? > >I haven't investigated properly but it seems to be the database (which >is on another host). That's interesting because most of that runs through SQLAlchemy. I wonder if it's a general problem with the way we integrate with that, or if there are specific queries that are inefficient. I have no doubt there's room for improvement on the latter, but we'd have to do the instrumentation to find out. >Interesting. I wonder if we can have SQLAlchemy play well with >asyncio, I haven't tried that before. Me neither! asyncio is pretty new, so we're seeing lots of efforts to async-ify more parts of the ecosystem. I don't know whether the SA folks are looking into that at all. A general search shows some interesting projects and discussions on the topic. >There is still a gunicorn.py file in the contrib directory, but I >haven't tested it either. I tried to get it to work but had problems. I'm really interested in seeing if we can leverage uvloop for improved performance. -Barry From aurelien at bompard.org Wed May 24 05:33:23 2017 From: aurelien at bompard.org (Aurelien Bompard) Date: Wed, 24 May 2017 11:33:23 +0200 Subject: [Mailman-Developers] Postorius and MRs for 3.1 In-Reply-To: <20170523213221.42002017@presto> References: <20170523213221.42002017@presto> Message-ID: Thanks Barry! Don't worry I'll do the releasing of HyperKitty when Mailman is up. However, I have just found two rather simple bugs: - https://gitlab.com/mailman/mailman/issues/336 - https://gitlab.com/mailman/mailman/issues/337 I have set the 3.1 milestone on them, and I think the 1st one should really be fixed (I sent a merge request) before we release. What do you think? A. 2017-05-24 6:32 GMT+02:00 Barry Warsaw : > On May 23, 2017, at 06:09 PM, Terri Oda wrote: > > >Your turn for the "last chance to tag things for 3.1" email now that > >Aurelien's weighed in. ;) (I'm cc'ing the rest of mailman-developers in > case > >anyone else wants to advocate for a bug too...) > > I uploaded Core 3.1rc1 to PyPI this afternoon, so barring any unforeseen > last > minute problems, I plan on just bumping the version number and releasing > it. > > I'm happy for others to do the tagging and releasing the other components, > or > I can help with that if you prefer. The only project I can't release to > PyPI > is HyperKitty. Aurelien, we can leave that to you or you can defer to me, > but > you have to give me upload rights on the HyperKitty (and Core plugin) PyPI > project for the latter. > > I'm hoping we can work on some release notes and an announcement tomorrow > while we put the finishing touches on it. > > 3.0 was released just over 2 years ago. It's exciting to see all the new > and > fixed stuff in 3.1; I hope you are too! My deep thanks to the core > developers, and to all of you in the community. As has been said at Pycon > many times this week: come for the code, stay for the community. > > Cheers, > -Barry > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/ > mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman- > developers/aurelien%40bompard.org > > Security Policy: http://wiki.list.org/x/QIA9 > From barry at list.org Wed May 24 13:45:34 2017 From: barry at list.org (Barry Warsaw) Date: Wed, 24 May 2017 10:45:34 -0700 Subject: [Mailman-Developers] Postorius and MRs for 3.1 In-Reply-To: References: <20170523213221.42002017@presto> Message-ID: <20170524104534.2ac6bdd7@presto> On May 24, 2017, at 11:33 AM, Aurelien Bompard wrote: >Don't worry I'll do the releasing of HyperKitty when Mailman is up. Thanks! >However, I have just found two rather simple bugs: >- https://gitlab.com/mailman/mailman/issues/336 >- https://gitlab.com/mailman/mailman/issues/337 > >I have set the 3.1 milestone on them, and I think the 1st one should really >be fixed (I sent a merge request) before we release. > >What do you think? I'm reviewing these now. Follow ups on the issues. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From barry at list.org Thu May 25 19:28:00 2017 From: barry at list.org (Barry Warsaw) Date: Thu, 25 May 2017 16:28:00 -0700 Subject: [Mailman-Developers] ANNOUNCE: Mailman 3.1.0 final! Message-ID: <20170525162800.5d2c681b@presto> Hello Mailpeople! On behalf of the entire team and all our wonderful contributors, I'm happy to announce the release of GNU Mailman 3.1 final. My deep thanks go to all the Mailman project sprinters at Pycon 2017 for getting us over the line! Two years after the original release of Mailman 3.0, this version contains a huge number of improvements across the entire stack. Many bugs have been fixed and new features added in the Core, Postorius (web u/i), and HyperKitty (archiver). Upgrading from Mailman 2.1 should be better too. We are seeing more production sites adopt Mailman 3, and we've been getting great feedback as these have rolled out. Important: mailman-bundler, our previous recommended way of deploying Mailman 3, has been deprecated. Abhilash Raj is putting the finishing touches on Docker images to deploy everything, and he'll have a further announcement in a week or two. Feedback is welcome: https://github.com/maxking/docker-mailman What is GNU Mailman? GNU Mailman is free software for managing electronic mail discussion and e-newsletter lists. Mailman is integrated with the web, making it easy for users to manage their accounts and for list owners to administer their lists. Mailman supports built-in archiving, automatic bounce processing, content filtering, digest delivery, and more. Mailman 3 is released under the terms of the GNU General Public License, version 3. The best places to start for all things related to this release: http://docs.mailman3.org/ http://www.list.org/ https://gitlab.com/mailman (Note: due to timezone skew, some of the tarballs may not be available on PyPI until tomorrow.) Happy Mailman Day, -Your friendly neighborhood cabal An overview of what's new in Mailman 3.1 ======================================== Feature parity with Mailman 2.1 ------------------------------- * You should be able to do just about everything that you could do in Mailman 2.1 *except* for topics and sibling/umbrella lists. Core ---- * Added support for Python 3.5 and 3.6 * MySQL is now an officially supported database * Many improvements with importing Mailman 2.1 lists * DMARC mitigations have been added, based on, but different than the same feature in Mailman 2.1 * The REST API requires HTTP/1.1 * A new REST API version (3.1) has been added which changes how UUIDs are interpreted, fixing the problem for some JavaScript libraries * Many new REST resources and methods have been added * Individual mailing lists can augment the system's header matching rules * `mailman create` now creates missing domains by default * `mailman digests` now has `--verbose` and `--dry-run` options * `mailman shell` now supports readline history * `mailman members` can filter members based on their subscription roles * A new template system has been added for all messages originating from inside Mailman. * The Message-ID-Hash header replaces X-Message-ID-Hash * New placeholders have been added for headers and footers * Unsubscriptions can now be confirmed and/or moderated Postorius/HyperKitty -------------------- * General U/I and U/X improvements * Many more features from the Core's have been plumbed through * We've adopted Django social auth logins and dropped Persona (since it's no longer supported upstream). You can now log in via Facebook, Google, GitHub, and GitLab. Backward incompatibilities -------------------------- * Core/REST: Held message resources now have an `original_subject` key that is not RFC 2047 decoded. `subject` is now RFC 2047 decoded. * Core/REST: If you've run pre-release versions from git head, and stored welcome and goodbye templates via REST, the template key names have changed backward incompatibility. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Fri May 26 17:29:17 2017 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 26 May 2017 14:29:17 -0700 Subject: [Mailman-Developers] ANNOUNCE: Mailman 3.1.0 final! In-Reply-To: <20170525162800.5d2c681b@presto> References: <20170525162800.5d2c681b@presto> Message-ID: <56c1dc86-e7f6-7c89-d03d-f2f3e1f3f99b@msapiro.net> On 05/25/2017 04:28 PM, Barry Warsaw wrote: > Hello Mailpeople! > > On behalf of the entire team and all our wonderful contributors, I'm happy to > announce the release of GNU Mailman 3.1 final. My deep thanks go to all the > Mailman project sprinters at Pycon 2017 for getting us over the line! And I am happy to report that the Mailman 3 installations at and are now happily running this release. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From fsantiago at garbage-juice.com Fri May 26 17:38:50 2017 From: fsantiago at garbage-juice.com (Fabian A. Santiago) Date: Fri, 26 May 2017 21:38:50 +0000 Subject: [Mailman-Developers] ANNOUNCE: Mailman 3.1.0 final! In-Reply-To: <20170525162800.5d2c681b@presto> References: <20170525162800.5d2c681b@presto> Message-ID: On May 25, 2017 7:28:00 PM EDT, Barry Warsaw wrote: >Hello Mailpeople! > >On behalf of the entire team and all our wonderful contributors, I'm >happy to >announce the release of GNU Mailman 3.1 final. My deep thanks go to >all the >Mailman project sprinters at Pycon 2017 for getting us over the line! > >Two years after the original release of Mailman 3.0, this version >contains a >huge number of improvements across the entire stack. Many bugs have >been >fixed and new features added in the Core, Postorius (web u/i), and >HyperKitty >(archiver). Upgrading from Mailman 2.1 should be better too. We are >seeing >more production sites adopt Mailman 3, and we've been getting great >feedback >as these have rolled out. > >Important: mailman-bundler, our previous recommended way of deploying >Mailman >3, has been deprecated. Abhilash Raj is putting the finishing touches >on >Docker images to deploy everything, and he'll have a further >announcement in a >week or two. > >Feedback is welcome: >https://github.com/maxking/docker-mailman > >What is GNU Mailman? > >GNU Mailman is free software for managing electronic mail discussion >and >e-newsletter lists. Mailman is integrated with the web, making it easy >for >users to manage their accounts and for list owners to administer their >lists. >Mailman supports built-in archiving, automatic bounce processing, >content >filtering, digest delivery, and more. Mailman 3 is released under the >terms >of the GNU General Public License, version 3. > >The best places to start for all things related to this release: > > http://docs.mailman3.org/ > http://www.list.org/ > https://gitlab.com/mailman > >(Note: due to timezone skew, some of the tarballs may not be available >on PyPI >until tomorrow.) > >Happy Mailman Day, >-Your friendly neighborhood cabal > >An overview of what's new in Mailman 3.1 >======================================== > >Feature parity with Mailman 2.1 >------------------------------- >* You should be able to do just about everything that you could do in >Mailman > 2.1 *except* for topics and sibling/umbrella lists. > >Core >---- >* Added support for Python 3.5 and 3.6 >* MySQL is now an officially supported database >* Many improvements with importing Mailman 2.1 lists >* DMARC mitigations have been added, based on, but different than the >same > feature in Mailman 2.1 >* The REST API requires HTTP/1.1 >* A new REST API version (3.1) has been added which changes how UUIDs >are > interpreted, fixing the problem for some JavaScript libraries >* Many new REST resources and methods have been added >* Individual mailing lists can augment the system's header matching >rules >* `mailman create` now creates missing domains by default >* `mailman digests` now has `--verbose` and `--dry-run` options >* `mailman shell` now supports readline history >* `mailman members` can filter members based on their subscription >roles >* A new template system has been added for all messages originating >from > inside Mailman. >* The Message-ID-Hash header replaces X-Message-ID-Hash >* New placeholders have been added for headers and footers >* Unsubscriptions can now be confirmed and/or moderated > >Postorius/HyperKitty >-------------------- >* General U/I and U/X improvements >* Many more features from the Core's have been plumbed through >* We've adopted Django social auth logins and dropped Persona (since >it's no > longer supported upstream). You can now log in via Facebook, Google, > GitHub, and GitLab. > >Backward incompatibilities >-------------------------- >* Core/REST: Held message resources now have an `original_subject` key >that is > not RFC 2047 decoded. `subject` is now RFC 2047 decoded. >* Core/REST: If you've run pre-release versions from git head, and >stored >welcome and goodbye templates via REST, the template key names have >changed > backward incompatibility. I see in the docker readme that it still hasn't been determined how to use postfix with mm3. Is this in fact the case? Does anyone know how? -- Thanks. Fabian S. OpenPGP: 3c3fa072accb7ac5db0f723455502b0eeb9070fc From mark at msapiro.net Fri May 26 18:27:07 2017 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 26 May 2017 15:27:07 -0700 Subject: [Mailman-Developers] ANNOUNCE: Mailman 3.1.0 final! In-Reply-To: References: <20170525162800.5d2c681b@presto> Message-ID: <0275b92d-3076-5adf-23be-7dffa1c494b3@msapiro.net> On 05/26/2017 02:38 PM, Fabian A. Santiago wrote: > > I see in the docker readme that it still hasn't been determined how to use postfix with mm3. Is this in fact the case? Does anyone know how? The comment in the README.md only refers to the Mailman instance in the docker container communicating with Postfix outside the docker container for purposes of running 'postalias' and 'postmap' commands. This has actually been addressed even in the docker container by allowing Postfix 'regex' tables to replace the default 'hash' tables. There has never been an issue running Mailman 3 with Postfix in general. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From johny at neuromancer.sk Sun May 28 19:09:07 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Mon, 29 May 2017 01:09:07 +0200 Subject: [Mailman-Developers] [GSoC] Encrypted mailing lists - update v3 In-Reply-To: References: Message-ID: Hi all. I have created a repository to keep the original proposal for this project, as well as the current working version and similar documentation / specs at: https://gitlab.com/J08nY/pgpmailman-proposal The repo contains up-to-date changes I propose to Mailman core to accommodate an encrypted lists plugin as well as other possible future plugins. (in core_changes.md) It also contains a current working sketch of the plugin (in plugin.md). Comments are very appreciated. I wrote a post on the current issues I am facing on integrating the encrypted lists plugin with Postorius and HyperKitty: https://neuromancer.sk/article/4 Integrating with Postorius and HyperKitty ========================================= Since a plugin-like out-of-tree approach is required for implementing encrypted lists into Mailman, a straight forward integration into Postorius and HyperKitty (as first proposed) by making them "aware" of the encrypted lists plugin is not possible. Thus a new approach for providing their functionality and conforming to the project requirements is necessary. I see three possible pathways forward and a middle-ground between them. ## Standalone django app A new django app will be created, using django-mailman3 as a base, that will implement all the web based functionality for encrypted mailing lists, such as: - Displaying the List key for all public encrypted mailing lists. - List key management for list admins - User key management - Encrypted archives, that are server unencrypted (effectively replaces HyperKitty for encrypted lists) This app will then be run besides Postorius. ## A fork/patchset approach This approach will create a fork of Postorius and HyperKitty that will integrate changes necessary for the encrypted lists plugin seamlessly. Thus users wanting to use encrypted mailing lists will have to setup Postorius and HyperKitty from this fork. ## Wrenching it in This approach tries to integrate all of the functionality using configurable options of Postorius and HyperKitty. For example storing messages encrypted could be done via a custom django.db.backend. Receiving messages encrypted could be done via a small custom django app that will receive them, decrypt and pass to HyperKitty decrypted. ## A middle-ground Somewhat of a middle ground seems to be most sensible. A standalone app will be necessary to provide functionality that is simply not possible to be integrated into Postorius and HyperKitty sensibly. This app will mostly provide key management (user and list), receive the messages encrypted and so on. However Postorius and HyperKitty will work with the least amount of "wrenching it in" as possible. Cheers, -- Jan ______________________________________________________ /\ # PGP: 362056ADA8F2F4E421565EF87F4A448FE68F329D /__\ # https://neuromancer.sk /\ /\ # Eastern Seaboard Phishing Authority /__\/__\ # -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: