From stephanwehner at gmail.com Fri Mar 3 21:44:48 2017 From: stephanwehner at gmail.com (Stephan Wehner) Date: Fri, 3 Mar 2017 18:44:48 -0800 Subject: [Mailman-Developers] Hyperkitty: full-text search Message-ID: Hello, I'm installing mailman3 from the current source, including Hyperkitty Making progress, the archiving part of Hyperkitty works, but not the full-text search part (URL's like /hyperkitty/search?mlist=announce%40testmail.testing.net&q=testing) Is there a trick? I don't see much being logged. Stephan From stephanwehner at gmail.com Sat Mar 4 15:30:06 2017 From: stephanwehner at gmail.com (Stephan Wehner) Date: Sat, 4 Mar 2017 12:30:06 -0800 Subject: [Mailman-Developers] Hyperkitty: full-text search In-Reply-To: References: Message-ID: Received message from Mark Sapiro to run the command django-admin update_index. That did it! File is hyperkitty/jobs/update_index.py This is invoked with the "minutely" job, see file example_project/crontab Thanks Mark! Stephan On Fri, Mar 3, 2017 at 6:44 PM, Stephan Wehner wrote: > Hello, > > I'm installing mailman3 from the current source, including Hyperkitty > > Making progress, the archiving part of Hyperkitty works, but not the > full-text search part (URL's like > /hyperkitty/search?mlist=announce%40testmail.testing.net&q=testing) > > Is there a trick? I don't see much being logged. > > Stephan -- Stephan Wehner -> http://stephan.sugarmotor.org (blog and homepage) -> http://twitter.com/stephanwehner / @stephanwehner -> http://www.thrackle.org Through http://www.buckmaster.ca, my company: -> http://whatzilla.com -> http://loggingit.com -> http://www.trafficlife.com -> http://clocklist.com -> http://stephansmap.org From bhavishyagopesh at gmail.com Sun Mar 5 14:24:33 2017 From: bhavishyagopesh at gmail.com (Bhavishya) Date: Mon, 6 Mar 2017 00:54:33 +0530 Subject: [Mailman-Developers] In regard to GSoc-17 Message-ID: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> Hi, I saw the idea to create various kinds of encrypted lists, Could you elaborate the following: 1)The amount of knowledge of security (and if possible the sources to achieve the same) 2)The development environment (what else is required apart from linux) 3)Any other task for me to strengthen my application(I would try fixing bugs on my level though) Thank-you. From raj.abhilash1 at gmail.com Sun Mar 5 23:18:02 2017 From: raj.abhilash1 at gmail.com (Abhilash) Date: Sun, 5 Mar 2017 20:18:02 -0800 Subject: [Mailman-Developers] In regard to GSoc-17 In-Reply-To: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> References: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> Message-ID: <644da968-e41b-c771-9993-8a3cb6a6b18a@gmail.com> Hi Bhavishya, On 03/05/2017 11:24 AM, Bhavishya wrote: > Hi, I saw the idea to create various kinds of encrypted lists, > Could you elaborate the following: > 1)The amount of knowledge of security (and if possible the sources to achieve the same) Since we are going to work on the idea of encrypted lists, you are expected to be aware about the basic types of encryption, their purpose and which would be suitable for this case. Usually, we use public-key cryptography for signing/encrypting emails. You may want to read up a bit on how and what would "encrypted lists" actually mean. There was a pretty good discussion for Google Summer of Code 2013 on the same topic and I believe it might be helpful for you get started (You can search the archives for this list and if possible filter with my name as sender to find the threads). > 2)The development environment (what else is required apart from linux) Mostly just Linux and python are what we use for development. > 3)Any other task for me to strengthen my application(I would try fixing bugs on my level though) A bug fix is needed for you to complete your application, so that is a must. Apart from that, we want you to have some idea about what this project is actually about. We (the mentors) would then chime in with the details and reviews and all our help for your application. > Thank-you. Thank you! > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- thanks, Abhilash Raj -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 837 bytes Desc: OpenPGP digital signature URL: From turnbull.stephen.fw at u.tsukuba.ac.jp Mon Mar 6 05:11:51 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Mon, 6 Mar 2017 19:11:51 +0900 Subject: [Mailman-Developers] In regard to GSoc-17 In-Reply-To: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> References: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> Message-ID: <22717.13799.287192.159296@turnbull.sk.tsukuba.ac.jp> Hi, Apart from what Abhilash wrote, I have a few ;-) additional comments. Bhavishya writes: > Hi, I saw the idea to create various kinds of encrypted lists, > Could you elaborate the following: > 1)The amount of knowledge of security (and if possible the sources > to achieve the same) You need to understand what a "threat model" is, and the structure of a mailing list as a system to know what threats can be defended against, and to decide which threats to defend against. You need to understand how mailing lists and the mail system work in some detail, and what the use cases for encrypted lists might be. For self-study, you could start with Bruce Schneier's blog, especially his famous post on "the security mindset", and with Steve Bellovin's book, _Thinking Security_. Bellovin's book has many references. The core mail security RFCs are enumerated below. The code for various encryption algorithms is already available in the standard library (OpenSSL, for example, although a lot of people deprecate it) and in 3rd-party libraries on PyPI. Writing encryption modules is not part of this task. > 2)The development environment (what else is required apart from > linux) Python 2.7, Python 3.5 (both 2.7 and 3.5 are currently *required*), plus Python 3.6 if you're adventurous (GNU Mailman 3 doesn't officially support Python 3.6 yet), modules from PyPI as-needed. Your Python(s) must be built to support OpenSSL, or some other source of implementations for encryption algorithms. git. The Mailman Suite (the subprojects mailman, mailmanclient, django-mailman3, postorius, hyperkitty, mailman-hyperkitty) from http://gitlab.com/mailman. (Dependencies for the suite will be installed automatically by the setup.py for each component.) An MTA, either Postfix (most popular among Mailman core developers) or Exim4 (supported). Sendmail and Qmail may be usable but are not advised unless you can provide support for them yourself -- there is ZERO support in Mailman 3 itself. It may not be very hard to support a new MTA (it took me 10 minutes to configure Exim4 and 30 to write the docs), but you won't get much help from us. Why risk it? It would be nice if you have a test domain where you can install Mailman on the standard SMTP port 25 or submission port 587, but testing on localhost is acceptable. > 3)Any other task for me to strengthen my application(I would try > fixing bugs on my level though) Get an account on gitlab. Read the FAQ for Mailman 2, the archives for mailman-users, mailman-developers, and mailman3-users to get some idea of the level and needs of our users. Subscribe to those lists. Take a look at RFCs 5321 (SMTP), 5322 (Internet Message Format), 4949 (security glossary), and 5598 (email architecture). Bookmark them and RFCs 2045 (MIME), 2046 (MIME), 2387 (multipart/related), 2015 (MIME/PGP), 3156 (MIME/OpenPGP), 5751 (S/MIME), and 5752 (multiple signatures). I recommend reading all the way through RFC 4949, as a complement to Schneier's blog and Bellovin's book (or similar). RFC 5598 is very important, as it is fundamental to understanding the threat models involved in email and indirect flows including mailing lists. Read the abstracts and introductions to RFCs 5321 and 5322, as understanding the basic concepts of email are going to be very important. For the rest just bookmarking is fine. We would eventually be referring to them in the implementation most likely, but you don't need to be totally familiar for the application. Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Mon Mar 6 05:12:05 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Mon, 6 Mar 2017 19:12:05 +0900 Subject: [Mailman-Developers] Discourse Integration In-Reply-To: References: Message-ID: <22717.13813.915523.526404@turnbull.sk.tsukuba.ac.jp> Vaibhav Lohani writes: > There was a mention of integrating mailman 3 with discourse some > times back. I am interested in working on it. Is someone else > interested in it or already working on it ? I don't know of anyone who is currently working on it, nor do we have specific plans at the moment. What sort of integration do you have in mind? Steve From barry at list.org Mon Mar 6 12:10:44 2017 From: barry at list.org (Barry Warsaw) Date: Mon, 6 Mar 2017 12:10:44 -0500 Subject: [Mailman-Developers] In regard to GSoc-17 In-Reply-To: <22717.13799.287192.159296@turnbull.sk.tsukuba.ac.jp> References: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> <22717.13799.287192.159296@turnbull.sk.tsukuba.ac.jp> Message-ID: <20170306121044.7a28039a@subdivisions.wooz.org> On Mar 06, 2017, at 07:11 PM, Stephen J. Turnbull wrote: >Python 2.7, Python 3.5 (both 2.7 and 3.5 are currently *required*), >plus Python 3.6 if you're adventurous (GNU Mailman 3 doesn't >officially support Python 3.6 yet) Note that 3.4 is also an officially supported version for core. Mailman 3.1 will -and current core git HEAD does- officially support 3.6. Cheers, -Barry From manpreetsingh_er at yahoo.com Mon Mar 6 05:36:00 2017 From: manpreetsingh_er at yahoo.com (Manpreet Singh) Date: Mon, 6 Mar 2017 10:36:00 +0000 (UTC) Subject: [Mailman-Developers] Mailman 3 Project discussion References: <347814594.1809193.1488796560140.ref@mail.yahoo.com> Message-ID: <347814594.1809193.1488796560140@mail.yahoo.com> Hello, Hope you are doing well. My name is Manpreet Singh. I found the Mailman project quite interesting as I have been using linux from 2-3 years and I have quite good knowledge of python3 as I am doing projects on python since last year. I have started viewing issues on the official git lab repository of Mailman and I am trying to be familiar with code base and its work flow.?My question is apart from this, on what areas of coding, team will be working on specifically which python library, framework etc for this project ? And do you want us to learn specifically new technology apart from python to successfully meet the requirements of this project ? ThanksManpreet SinghB.S. Software EngineeringSan Jose State University From andrew at mcnaughty.com Tue Mar 7 01:40:18 2017 From: andrew at mcnaughty.com (Andrew McN) Date: Tue, 7 Mar 2017 17:40:18 +1100 Subject: [Mailman-Developers] Trying out mailman3 (docker recipe fails) Message-ID: <208c8629-a883-73bc-dd53-a168cee5469d@mcnaughty.com> A community I'm part of is looking for a new mailing list solution, and I thought I'd look at where mailman 3 is up to these days. In the process I found that the instructions for firing up mailman 3.0 in docker aren't currently working. I followed the recipe at: https://wiki.list.org/DEV/Mailman%203.0/Mailman%203.0%20Suite%20Dockerfile But it seems that it doesn't currently work. Error messages are as follow. There were earlier warnings about missing files, but the build seemed happy to ignore those. > Getting distribution for 'postorius==1.0.3'. > no previously-included directories found matching 'src/postorius/doc/_build' > zip_safe flag not set; analyzing archive contents... > postorius.doc.conf: module references __file__ > postorius.tests.__init__: module references __file__ > While: > Installing mailman-web. > Getting distribution for 'postorius==1.0.3'. > > An internal error occurred due to a bug in either zc.buildout or in a > recipe being used: > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/zc/buildout/buildout.py", > line 1982, in main > getattr(buildout, command)(args) > File "/usr/local/lib/python2.7/dist-packages/zc/buildout/buildout.py", > line 668, in install > installed_files = self[part]._call(recipe.install) > File "/usr/local/lib/python2.7/dist-packages/zc/buildout/buildout.py", > line 1419, in _call > return f() > File > "/mailman3/mailman-bundler/eggs/djangorecipe-2.2.1-py2.7.egg/djangorecipe/recipe.py", > line 82, in install > ws = self.egg.working_set(['djangorecipe'])[1] > File > "/mailman3/mailman-bundler/eggs/zc.recipe.egg-2.0.3-py2.7.egg/zc/recipe/egg/egg.py", > line 84, in working_set > allow_hosts=self.allow_hosts) > File > "/usr/local/lib/python2.7/dist-packages/zc/buildout/easy_install.py", > line 913, in install > return installer.install(specs, working_set) > File > "/usr/local/lib/python2.7/dist-packages/zc/buildout/easy_install.py", > line 665, in install > for dist in self._get_dist(requirement, ws): > File > "/usr/local/lib/python2.7/dist-packages/zc/buildout/easy_install.py", > line 563, in _get_dist > dists = [_move_to_eggs_dir_and_compile(dist, self._dest)] > File > "/usr/local/lib/python2.7/dist-packages/zc/buildout/easy_install.py", > line 1730, in _move_to_eggs_dir_and_compile > assert newdist is not None # newloc above is missing our dist?! > AssertionError Can someone help me identify the problem here? Alternatively, is there a better way for me to review where things are up to, and what the interface is like? I'll probably be approaching this as a sysadmin before long, but just now I'm looking for a list-admin and user perspective. When I get to hosting seriously, Docker is preferred. The environments I'm working in are moving towards Docker/kubernetes. Regards, Andrew From turnbull.stephen.fw at u.tsukuba.ac.jp Tue Mar 7 04:29:37 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 7 Mar 2017 18:29:37 +0900 Subject: [Mailman-Developers] In regard to GSoc-17 In-Reply-To: <20170306121044.7a28039a@subdivisions.wooz.org> References: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> <22717.13799.287192.159296@turnbull.sk.tsukuba.ac.jp> <20170306121044.7a28039a@subdivisions.wooz.org> Message-ID: <22718.32129.862438.348283@turnbull.sk.tsukuba.ac.jp> Barry Warsaw writes: > Note that 3.4 is also an officially supported version for core. Yes. Let me clarify that I was referring to what the mentor (me ;-) is willing to support for this GSoC project (in the interest of minimizing the motion of the target platform for the intern; that's also why I specify OpenSSL[1] for algorithm support). Footnotes: [1] Which Cory and Christian and Donald use as a punching bag on python-dev and Twitter. From turnbull.stephen.fw at u.tsukuba.ac.jp Tue Mar 7 04:32:34 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 7 Mar 2017 18:32:34 +0900 Subject: [Mailman-Developers] Mailman 3 Project discussion In-Reply-To: <347814594.1809193.1488796560140@mail.yahoo.com> References: <347814594.1809193.1488796560140.ref@mail.yahoo.com> <347814594.1809193.1488796560140@mail.yahoo.com> Message-ID: <22718.32306.227782.152968@turnbull.sk.tsukuba.ac.jp> Manpreet Singh via Mailman-Developers writes: >?My question is apart from this, on what areas of coding, team will > be working on specifically which python library, framework etc for > this project ? And do you want us to learn specifically new > technology apart from python to successfully meet the requirements > of this project ? I don't understand your question. Are you referring to the GSoC project on encrypted lists under the Python Software Foundation umbrella? If so, I just answered that question on this list (probably crossed with your post). See also Barry's and my followups. Otherwise, please be more specific. Note that the encrypted lists project is probably (95%) the only Mailman project this summer, as we have a release upcoming and are short on mentors. Steve From barry at list.org Tue Mar 7 09:56:03 2017 From: barry at list.org (Barry Warsaw) Date: Tue, 7 Mar 2017 09:56:03 -0500 Subject: [Mailman-Developers] In regard to GSoc-17 In-Reply-To: <22718.32129.862438.348283@turnbull.sk.tsukuba.ac.jp> References: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> <22717.13799.287192.159296@turnbull.sk.tsukuba.ac.jp> <20170306121044.7a28039a@subdivisions.wooz.org> <22718.32129.862438.348283@turnbull.sk.tsukuba.ac.jp> Message-ID: <20170307095603.5d20e3c3@subdivisions.wooz.org> On Mar 07, 2017, at 06:29 PM, Stephen J. Turnbull wrote: >Barry Warsaw writes: > > > Note that 3.4 is also an officially supported version for core. > >Yes. Let me clarify that I was referring to what the mentor (me ;-) >is willing to support for this GSoC project (in the interest of >minimizing the motion of the target platform for the intern; that's >also why I specify OpenSSL[1] for algorithm support). Fair enough! :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From raditya at iitk.ac.in Tue Mar 7 08:10:17 2017 From: raditya at iitk.ac.in (raditya) Date: Tue, 07 Mar 2017 18:40:17 +0530 Subject: [Mailman-Developers] Gsoc idea explanation needed Message-ID: <90e7f79f8f63b26275b489100549894d@iitk.ac.in> Dear mentors, could you give me a rough idea about the level of knowledge required in crypto libraries of python, and also possible sources of learning. From turnbull.stephen.fw at u.tsukuba.ac.jp Wed Mar 8 04:35:47 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Wed, 8 Mar 2017 18:35:47 +0900 Subject: [Mailman-Developers] Gsoc idea explanation needed In-Reply-To: <90e7f79f8f63b26275b489100549894d@iitk.ac.in> References: <90e7f79f8f63b26275b489100549894d@iitk.ac.in> Message-ID: <22719.53363.863312.321654@turnbull.sk.tsukuba.ac.jp> Posted to this list yesterday. Probably crossed in the mail. Check the archives. > Dear mentors, could you give me a rough idea about the level of > knowledge required in crypto libraries of python, and also possible > sources of learning. From raditya at iitk.ac.in Wed Mar 8 16:32:20 2017 From: raditya at iitk.ac.in (raditya) Date: Thu, 09 Mar 2017 03:02:20 +0530 Subject: [Mailman-Developers] Dev-env setup query Message-ID: <3a04a2206a57d8adc3e9366bf527f97c@iitk.ac.in> Dear mentors, for some reason i'm not able to setup the dev-env on my machine using the SSH key method so i figured that i can do it using HTTPS as well. Is there any drawback of doing so other than the fact that i will have to input my username and password everytime. From barry at list.org Wed Mar 8 16:43:21 2017 From: barry at list.org (Barry Warsaw) Date: Wed, 8 Mar 2017 16:43:21 -0500 Subject: [Mailman-Developers] Dev-env setup query In-Reply-To: <3a04a2206a57d8adc3e9366bf527f97c@iitk.ac.in> References: <3a04a2206a57d8adc3e9366bf527f97c@iitk.ac.in> Message-ID: <20170308164321.072042f6@subdivisions.wooz.org> On Mar 09, 2017, at 03:02 AM, raditya wrote: >Dear mentors, for some reason i'm not able to setup the dev-env on my >machine using the SSH key method so i figured that i can do it using >HTTPS as well. Is there any drawback of doing so other than the fact >that i will have to input my username and password everytime. If you're only pulling from the GitLab git repo, there shouldn't be any difference in content. -Barry From kohli.sagar2 at gmail.com Wed Mar 8 15:19:37 2017 From: kohli.sagar2 at gmail.com (sagar kohli) Date: Thu, 9 Mar 2017 01:49:37 +0530 Subject: [Mailman-Developers] In regard to GSoc-17 In-Reply-To: <20170307095603.5d20e3c3@subdivisions.wooz.org> References: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> <22717.13799.287192.159296@turnbull.sk.tsukuba.ac.jp> <20170306121044.7a28039a@subdivisions.wooz.org> <22718.32129.862438.348283@turnbull.sk.tsukuba.ac.jp> <20170307095603.5d20e3c3@subdivisions.wooz.org> Message-ID: Hi everyone, My name is Sagar Kohli. I am a sophomore from IIT Guwahati. I have recently gotten involved with the mailman project, and gained familiarity with postorius and mailman core. I want to work on the project "Encrypted lists". After going through the previous posts, I have begun reading about encryption and RFC a bit. After getting familiar with these topics i would like to discuss the project with the mentors. Thanks. Sagar Kohli. _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/ma > ilman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/k > ohli.sagar2%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From mark at msapiro.net Thu Mar 9 03:01:06 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 9 Mar 2017 00:01:06 -0800 Subject: [Mailman-Developers] Dev-env setup query In-Reply-To: <20170308164321.072042f6@subdivisions.wooz.org> References: <3a04a2206a57d8adc3e9366bf527f97c@iitk.ac.in> <20170308164321.072042f6@subdivisions.wooz.org> Message-ID: On 03/08/2017 01:43 PM, Barry Warsaw wrote: > On Mar 09, 2017, at 03:02 AM, raditya wrote: > >> Dear mentors, for some reason i'm not able to setup the dev-env on my >> machine using the SSH key method so i figured that i can do it using >> HTTPS as well. Is there any drawback of doing so other than the fact >> that i will have to input my username and password everytime. > > If you're only pulling from the GitLab git repo, there shouldn't be any > difference in content. And it's a public repo so you shouldn't need a user/password authentication for https either. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From raditya at iitk.ac.in Thu Mar 9 03:24:50 2017 From: raditya at iitk.ac.in (raditya) Date: Thu, 09 Mar 2017 13:54:50 +0530 Subject: [Mailman-Developers] git or bazaar Message-ID: <297258aafd060ab15e39a475f9f32570@iitk.ac.in> Dear mentors, i already know git so what are the advantages of learning bazaar, cause the wiki mentions it even though all the installation steps are in git. From mark at msapiro.net Thu Mar 9 04:01:29 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 9 Mar 2017 01:01:29 -0800 Subject: [Mailman-Developers] git or bazaar In-Reply-To: <297258aafd060ab15e39a475f9f32570@iitk.ac.in> References: <297258aafd060ab15e39a475f9f32570@iitk.ac.in> Message-ID: <465d77ad-06c2-e6f4-f708-7463a1d684fc@msapiro.net> On 03/09/2017 12:24 AM, raditya wrote: > Dear mentors, i already know git so what are the advantages of learning > bazaar, cause the wiki mentions it even though all the installation > steps are in git. Mailman 3 repos are all git at gitlab.com. There is no need to use bazaar for anything to do with Mailman 3. Mailman 2.1 repos are bazaar at launchpad.net. If you are going to work with Mailman 2.1 development/maintenance, bazaar knowledge is useful. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Mar 9 11:01:14 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 10 Mar 2017 01:01:14 +0900 Subject: [Mailman-Developers] In regard to GSoc-17 In-Reply-To: References: <7D61EAA5-31E6-4615-9FAC-0CDC29A3B37B@gmail.com> <22717.13799.287192.159296@turnbull.sk.tsukuba.ac.jp> <20170306121044.7a28039a@subdivisions.wooz.org> <22718.32129.862438.348283@turnbull.sk.tsukuba.ac.jp> <20170307095603.5d20e3c3@subdivisions.wooz.org> Message-ID: <22721.31818.969776.891148@turnbull.sk.tsukuba.ac.jp> A few days ago I posted more details about preparing for this project. Please check the archives. sagar kohli writes: > Hi everyone, > My name is Sagar Kohli. I am a sophomore from IIT Guwahati. > I have recently gotten involved with the mailman project, and gained > familiarity with postorius and mailman core. > I want to work on the project "Encrypted lists". > After going through the previous posts, I have begun reading about > encryption and RFC a bit. > After getting familiar with these topics > i would like to discuss the project with the mentors. > Thanks. > > Sagar Kohli. > > > _______________________________________________ > > Mailman-Developers mailing list > > Mailman-Developers at python.org > > https://mail.python.org/mailman/listinfo/mailman-developers > > Mailman FAQ: http://wiki.list.org/x/AgA3 > > Searchable Archives: http://www.mail-archive.com/ma > > ilman-developers%40python.org/ > > Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/k > > ohli.sagar2%40gmail.com > > > > Security Policy: http://wiki.list.org/x/QIA9 > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/turnbull.stephen.fw%40u.tsukuba.ac.jp > > Security Policy: http://wiki.list.org/x/QIA9 > > From raditya at iitk.ac.in Thu Mar 9 20:08:22 2017 From: raditya at iitk.ac.in (raditya) Date: Fri, 10 Mar 2017 06:38:22 +0530 Subject: [Mailman-Developers] Project idea understanding Message-ID: <773fbb2a6f0d8938a75e0dfda2c2559b@iitk.ac.in> Dear Mentors, After going through a lot of sources mentioned in recent replies to other subscribers and some online sources i have got some understanding about the idea, which i would like to share with you guys and get some feedback and suggestions . I think we can encrypt the list in the following ways (broadly speaking): 1) Each message will be encrypted with all users as recipients. The mailing-list maintainer will have to regularly send the list of recipients over the mailing-list itself, so that every user comes to know the public keys of all recipients. But this way the maintainer will have to send recipients list to a large number of users. 2) Another way could be relaying each message through the maintainer. The maintainer then encrypts it for single recipient. This might solve the problem of sending messageto large number of users but it increases latency (each message must wait for the maintainer to process it) the maintainer can censor each message as they all are relayed through him. 3) Or instead of using asymmetric keys we use a password given to all users. I have attached an image in hopes of conveying my idea more clearly for points 1 & 2. Also I'm still not finished with the book "Thinking Security - Steven M. Bellovin" mentioned by Stephen in one of the replies, so i hope to refine this crude idea in future. The Python GSoC page says that every student must maintain a blog which i have been maintaining at : Parseltongue.pythonanywhere.com -------------- next part -------------- A non-text attachment was scrubbed... Name: mailman.jpg Type: image/jpeg Size: 2095546 bytes Desc: not available URL: From simon.hanna at serve-me.info Fri Mar 10 06:15:22 2017 From: simon.hanna at serve-me.info (Simon Hanna) Date: Fri, 10 Mar 2017 12:15:22 +0100 Subject: [Mailman-Developers] Project idea understanding In-Reply-To: <773fbb2a6f0d8938a75e0dfda2c2559b@iitk.ac.in> References: <773fbb2a6f0d8938a75e0dfda2c2559b@iitk.ac.in> Message-ID: <81d6a95e-6e6d-7425-73f6-ba4b4695f2cd@serve-me.info> On 03/10/2017 02:08 AM, raditya wrote: > Dear Mentors, > > After going through a lot of sources mentioned in recent replies to > other subscribers and some online sources i have got some understanding > about the idea, which i would like to share with you guys and get some > feedback and suggestions . > > I think we can encrypt the list in the following ways (broadly > speaking): > > 1) Each message will be encrypted with all users as recipients. > The mailing-list maintainer will have to regularly send the list of > recipients over the mailing-list itself, > so that every user comes to know the public keys of all recipients. > But this way the maintainer will have to send recipients list to a large > number of users. If you as sender encrypt the message for every receiver, you are literally doing all the work manually and don't need mailman > 2) Another way could be relaying each message through the maintainer. > The maintainer then encrypts it for single recipient. > This might solve the problem of sending messageto large number of users > but it increases latency (each message must wait for the maintainer to > process it) > the maintainer can censor each message as they all are relayed through > him. This approach makes "maintainers" (you probably mean moderators) do all the work which also eliminates the need for mailman > 3) Or instead of using asymmetric keys we use a password given to all > users. This can already be done right now. It's as if you send an encrypted attachment that contains the actual message There has been a GSoC project like this in the past and there was quite some discussion on the mailing list. https://www.mail-archive.com/search?q=encrypt&l=mailman-developers%40python.org From turnbull.stephen.fw at u.tsukuba.ac.jp Sat Mar 11 02:58:33 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Sat, 11 Mar 2017 16:58:33 +0900 Subject: [Mailman-Developers] Project idea understanding In-Reply-To: <773fbb2a6f0d8938a75e0dfda2c2559b@iitk.ac.in> References: <773fbb2a6f0d8938a75e0dfda2c2559b@iitk.ac.in> Message-ID: <22723.44585.967029.696464@turnbull.sk.tsukuba.ac.jp> raditya writes: > 1) Each message will be encrypted with all users as recipients. You are way ahead of yourself. Before thinking about *how* to manage an encrypted list, both operationally and socially, we need to decide (1) *who* is going to be posting (2) *why* are they posting to an encrypted list (3) *who* is going to be subscribing to such a list (4) *why* are they subscribing (5) *who* the traffic is being hidden from (6) *what capabilities* the adversary may be expected to have (7) *who* will be managing the list and host (8) *the trustworthiness* required of list and host managers (9) *what other features* are required in view of (1) -- (8), such as anonymity and there may be some other basic features to consider. I strongly suggest a close look at RFC 5598 "Internet Mail Architecture" to get a good understanding of the "players" in the game of "mailing list". Then we need to talk about what threats encryption can protect against, what threats other parts of Mailman can protect against, and what can't be protected against. Documentation will eventually be the hardest part of this task. Although the GSoC intern won't be responsible for perfecting it, documenting the threat model is going to be important both for the users (subscribers and list owners) and for the design and implementation. Don't worry about starting with a narrow use case, limited to some "user story" you personally are familiar with. We can think about generalizing as we go along, or as a follow-on after the project is done. Original design and maximum generality are not required of a GSoC project (although working on the design yourself does help with planning implementation). Also, I have a lot of ideas about "generic" threats, and will reveal them gradually as we go along. But it's important for you to get over the idea that encryption itself is a big deal. It's not, and the basic mechanisms are already in place (for signing, but it's quite similar) from a previous project. Regarding your suggestions so far, I can tell you already that there are technical or "audience" problems with your initial ideas. > The mailing-list maintainer will have to regularly send the list of > recipients over the mailing-list itself, so that every user comes > to know the public keys of all recipients. As you seem to recognize, and Simon also points out, this is not an attractive UI for many scenarios, including "large" lists and non-technical posters. And it's already possible in any case for users competent in such operations. > 2) Another way could be relaying each message through the maintainer. > The maintainer then encrypts it for single recipient. > This might solve the problem of sending messageto large number of users > but it increases latency (each message must wait for the maintainer to > process it) > the maintainer can censor each message as they all are relayed through > him. This is not going to be done by a human, there will be some kind of Handler (and probably a Rule) to deal with it. "Censoring" (eg, anonymization, as well as moderation) is a *good point*, whose general application I missed above. This is non-trivial for secret traffic in some cases. Some of it can be automated, and in some applications all of it should be. "Hold that thought." :-) > 3) Or instead of using asymmetric keys we use a password given to all > users. Again, not an attractive solution. Consider the "secret" distribution problem in general, and especially the revocation problem if you decide to kick somebody off the list. The primary goal here is to avoid forcing subscribers and posters to learn about encryption and security. We probably can't avoid some burden on the users, but the less the better -- we know very well from the security problems that become public every day that users are not very good are secure usage. I hope this helps you get a better handle on the scope of this project. Regards, Steve From kohli.sagar2 at gmail.com Sat Mar 11 14:12:15 2017 From: kohli.sagar2 at gmail.com (sagar kohli) Date: Sun, 12 Mar 2017 00:42:15 +0530 Subject: [Mailman-Developers] GSOC Project Idea discussion : Encrypted Lists Message-ID: Dear Mentors. Alter getting some idea of how encryption works, and reading about the architecture of mailman. I would like to share the possible idea that im thinking of which is as follows: So basically what i was thinking is 1. Every subscriber will share there public key to the mailman server 2. Subscribers can get mailing list public key. 3. Poster will then encrypt the message with the mailing list public key and send the message to the mailman server. 4. The message will pass through a handler to check whether the message is encrypted or not. if the massage is not encrypted, it will get bounce back to the the sender. Else mailman will decrypt the message, do its other processes and then send it to all the subscriber after encrypting it with respective public key of the subscribers. 5. The other users decrypt the message with there private key. So the main question here is who will be allowed to get into the mailing list and how will they subscribe to it. One idea is the owner of the list will decide whether to accept the users request to join the list or not. And then add his public key in the database and send him the lists public key. Is this the right direction im thinking in ? I really need your feedback and suggestion on this. regards Sagar Kohli From bhavishyagopesh at gmail.com Sun Mar 12 12:11:52 2017 From: bhavishyagopesh at gmail.com (Bhavishya Desai) Date: Sun, 12 Mar 2017 21:41:52 +0530 Subject: [Mailman-Developers] Mailman-Developers Digest, Vol 335, Issue 9 In-Reply-To: References: Message-ID: Hello, Here's what I propose(of course deeply inspired from abhilash and schneuder) :- 1)A list owner creates a key-pair, and also a passphrase(to protect LIST's secret key,may be using gnupg, more on that later ).Here ,if we want, can raise the security standards by using gpg-agent(to manage security-key and its passphrase,it leads to some hardware token like cryptostick). 2)Now when a user subscribes, he will upload his public-key(may be through some interface on postorious),now the owner can decide his subscription(may be based on some write-up).Then he sends the LIST's public key to user. 3)Now each time, when user needs to post something, he has to sign it using his private-key and encrpyt using List's public key(signing would reduce spam). 4)Now mailman checks the signature with public-key(if failure occurs, can bounce it back),now first it signs it using LIST' secret key (here gpg-agent comes to play,only the owner can do this using the passphrase) and can be sent to the users after encrypting using their respective public key.Here we need to ensure one thing that the message never remains in decrypted form on Mailman server(which is done by that whole secret key-passphrase thing). 5)Now the user can verify using list's public key,and can henceforth decrypt the message using his private key. I would like to point out that the message would never travel decrypted and while the owner accesses it appropriate censorship could be applied(as pointed by raditya). Now I would like to know(specifically) what are some other threats,which could effect this and any difficulties with implementation. On Sat, Mar 11, 2017 at 10:30 PM, wrote: > Send Mailman-Developers mailing list submissions to > mailman-developers at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/mailman-developers > or, via email, send a message with subject or body 'help' to > mailman-developers-request at python.org > > You can reach the person managing the list at > mailman-developers-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Mailman-Developers digest..." > > > Today's Topics: > > 1. Project idea understanding (Stephen J. Turnbull) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 11 Mar 2017 16:58:33 +0900 > From: "Stephen J. Turnbull" > To: raditya > Cc: Mailman-Developers at python.org > Subject: [Mailman-Developers] Project idea understanding > Message-ID: <22723.44585.967029.696464 at turnbull.sk.tsukuba.ac.jp> > Content-Type: text/plain; charset=us-ascii > > raditya writes: > > > 1) Each message will be encrypted with all users as recipients. > > You are way ahead of yourself. Before thinking about *how* to manage > an encrypted list, both operationally and socially, we need to decide > > (1) *who* is going to be posting > (2) *why* are they posting to an encrypted list > (3) *who* is going to be subscribing to such a list > (4) *why* are they subscribing > (5) *who* the traffic is being hidden from > (6) *what capabilities* the adversary may be expected to have > (7) *who* will be managing the list and host > (8) *the trustworthiness* required of list and host managers > (9) *what other features* are required in view of (1) -- (8), such as > anonymity > > and there may be some other basic features to consider. I strongly > suggest a close look at RFC 5598 "Internet Mail Architecture" to get a > good understanding of the "players" in the game of "mailing list". > Then we need to talk about what threats encryption can protect > against, what threats other parts of Mailman can protect against, and > what can't be protected against. Documentation will eventually be the > hardest part of this task. Although the GSoC intern won't be > responsible for perfecting it, documenting the threat model is going > to be important both for the users (subscribers and list owners) and > for the design and implementation. > > Don't worry about starting with a narrow use case, limited to some > "user story" you personally are familiar with. We can think about > generalizing as we go along, or as a follow-on after the project is > done. Original design and maximum generality are not required of a > GSoC project (although working on the design yourself does help with > planning implementation). Also, I have a lot of ideas about "generic" > threats, and will reveal them gradually as we go along. But it's > important for you to get over the idea that encryption itself is a big > deal. It's not, and the basic mechanisms are already in place (for > signing, but it's quite similar) from a previous project. > > Regarding your suggestions so far, I can tell you already that there > are technical or "audience" problems with your initial ideas. > > > The mailing-list maintainer will have to regularly send the list of > > recipients over the mailing-list itself, so that every user comes > > to know the public keys of all recipients. > > As you seem to recognize, and Simon also points out, this is not > an attractive UI for many scenarios, including "large" lists and > non-technical posters. And it's already possible in any case for > users competent in such operations. > > > 2) Another way could be relaying each message through the maintainer. > > The maintainer then encrypts it for single recipient. > > This might solve the problem of sending messageto large number of users > > but it increases latency (each message must wait for the maintainer to > > process it) > > the maintainer can censor each message as they all are relayed through > > him. > > This is not going to be done by a human, there will be some kind of > Handler (and probably a Rule) to deal with it. > > "Censoring" (eg, anonymization, as well as moderation) is a *good > point*, whose general application I missed above. This is non-trivial > for secret traffic in some cases. Some of it can be automated, and in > some applications all of it should be. "Hold that thought." :-) > > > 3) Or instead of using asymmetric keys we use a password given to all > > users. > > Again, not an attractive solution. Consider the "secret" distribution > problem in general, and especially the revocation problem if you > decide to kick somebody off the list. The primary goal here is to > avoid forcing subscribers and posters to learn about encryption and > security. We probably can't avoid some burden on the users, but the > less the better -- we know very well from the security problems that > become public every day that users are not very good are secure usage. > > I hope this helps you get a better handle on the scope of this > project. > > Regards, > > Steve > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > > > ------------------------------ > > End of Mailman-Developers Digest, Vol 335, Issue 9 > ************************************************** > From stephanwehner at gmail.com Sun Mar 12 12:36:31 2017 From: stephanwehner at gmail.com (Stephan Wehner) Date: Sun, 12 Mar 2017 09:36:31 -0700 Subject: [Mailman-Developers] GSOC Project Idea discussion : Encrypted Lists (sagar kohli) Message-ID: > sagar kohli wrote: > > Alter getting some idea of how encryption works, and reading about the > architecture of mailman. I would like to share the possible idea that im > thinking of which is as follows: > [snip] The way I understand your plan, it is a bit weaker compared to what is described in this 2006 paper, "From Proxy Encryption Primitives to a Deployable Secure-Mailing-List Solution," available at http://www.ncsa.illinois.edu/People/hkhurana/ICICS.pdf When I looked up the first author, I found this site: http://sels.ncsa.illinois.edu/ "Secure Email List Services (SELS) is an open source software for creating and developing secure email list services among user communities. SELS provides signature and encryption capabilities while ensuring that ***the List Server does not have access to email plain text***. SELS has been developed with available open-source components and is compatible with many commonly used email clients." (I added the stars) The last link there goes to https://sourceforge.net/projects/sels/ Source code is at https://sourceforge.net/p/sels/code/HEAD/tree/ Did you could ask for advice on encryption mailing lists? To me it looks like a topic which cryptologists would be interested in. Stephan > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > > > ------------------------------ > > End of Mailman-Developers Digest, Vol 335, Issue 10 > *************************************************** -- Stephan Wehner -> http://stephan.sugarmotor.org (blog and homepage) -> http://twitter.com/stephanwehner / @stephanwehner -> http://www.thrackle.org Through http://www.buckmaster.ca, my company: -> http://whatzilla.com -> http://loggingit.com -> http://www.trafficlife.com -> http://clocklist.com -> http://stephansmap.org From jbn at forestfield.org Sun Mar 12 20:34:17 2017 From: jbn at forestfield.org (J.B. Nicholson) Date: Sun, 12 Mar 2017 19:34:17 -0500 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: References: Message-ID: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> Bhavishya Desai wrote: > Now I would like to know(specifically) what are some other threats,which > could effect this and any difficulties with implementation. I imagine that the encryption and/or hash algorithms will change over time as encryption is broken and people figure out ways to create hash collisions. Therefore I'd imagine keys will change over time. I'm guessing this carries some consequences: - the list key will change over time, therefore: - old subscribers need to be alerted to the new list key. - MUAs need to be able to make use of the current list key (perhaps the list accepts posts under the old key for a limited time to give subscribers a chance to switch to using the new key). - subscribers' keys will change over time, therefore: - there needs to be an easy way for a subscriber to get the list to accept posts under the subscriber's new key. - list archives raise interesting questions: - if the goal is to never let a list post travel unencrypted, I guess the list archives will be encrypted too? - each archive post will be signed+encrypted with the current list key. Therefore each archive message will be decrypted and re-encrypted at each list key change? - yes: this implies a bigger and bigger decryption/re-encryption job at each list key change as a list archive grows. Presumably this task will become computationally intensive at some point, possibly beyond the scale of being done at all for very large mailing lists carrying large list posts. - no: each archive post will be untouched after sending. Therefore archives will feature a set of list posts signed+encrypted with whatever list key was current at the time that post was sent out. - thus old list public keys must be kept around and published forever so archive readers can decrypt/verify signatures of archived posts? - yes: this carries some questions about GPG policy (below). - no: how will this work? - GPG policy issues: the above raises questions about GPG: - will GPG keep old encryption algorithms and hash algorithms around (perhaps warning not to use them for new keys, and only using them for decryption as needed)? - or users going to need to retain old versions of GPG to handle verifying archive list posts old encrypted & signed with list keys (I don't see this working out well)? It's possible there is something fundamental about this entire process I do not understand and therefore I've completely missed something about this process which led me down a path I should not have gone down. If that's the case, please do let me know where I went wrong. Thanks. From johny at neuromancer.sk Mon Mar 13 13:26:23 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Mon, 13 Mar 2017 18:26:23 +0100 Subject: [Mailman-Developers] GSoC 2017 - Encrypted lists draft Message-ID: <33125227-d1ed-a102-763d-d3e6d7ae33e1@neuromancer.sk> Hi Mailman developers. My name is Jan Jancar and I study IT Security at the Masaryk university in Brno, Czech republic. I'm interested in this years Mailman GSoCproject, as encryption/security is a topic I'm familiar with and interested in. I'm currently working on an Elliptic Curve domain parameter generator , as well as some other crypto/security related projects, and my site runs on Python. I have started working on SMTPS/STARTTLS support for Mailman Core as my GSoC micro-project and currently it's almost done: https://gitlab.com/J08nY/mailman/commits/mta-smtps-starttls Tests not yet done, as the SMTP test layer doesn't currently support SMTPS, more changes to it will be required. I've read through some past discussions on this list about PGP integration, went through Mailman Core code and started writing a draft of a project proposal (appended). An up to date version is also on: https://neuromancer.sk/page/gsoc/mailman I also have a question regarding project requirements: 1) What priority does accommodating naive users (without PGP setup) have for this project? As I haven't found a solution that wouldn't have a significant security impact, other than: install gnupg and setup a keypair. Please let me know what you think about my proposal so far, thanks! Jan ####################################################################### # Mailman - encrypted lists ## Threat model ### Assets - Message body - Message metadata (headers + existence) - Subscriber's identity (list of subscribers of a given list + their public keys as described in this proposal) - Existence of subscription for a given address - Existence of subscription for a given key - Keyrings (and keypairs in them, as described in this proposal) ### Adversary We assume that an adversary: - Can read, write, alter, drop any data passed between: - Mailman Core and it's outgoing and incoming MTA - outgoing and incoming MTA and lists subscribers - HyperKitty and Mailman Core (mailman-hyperkitty) - Is not a subscriber of the list / can not subscribe to the list, as it it's moderated. - Can not gain access into a subscriber's mailbox as well as his private part of a PGP key - Can not gain physical access to the machine running Mailman Core or HyperKitty. (RAM access, Coldboot mitigation out of scope) - Can not get access to the machine running Mailman Core as the Mailman core user or root - Can gain filesystem access to HyperKitty's archive Optional assumptions, these can be somewhat protected against and thus become real assumptions, see Attacks and Mitigations: - Can get filesystem access with enough permissions to access (read/write) Mailman Core queues with Mailman offline - Can get filesystem access to keyrings described in this proposal ## Design - On top of PGP/MIME ### List - has a list keypair stored in *core keyring* - has a list archive keypair, which is stored in *archive keyring* - public key from lists archive keypair also stored in *core keyring* ### User workflow #### List key - User gains knowledge of lists public key: - Through Postorious - Sends mail to `list_id`-key at domain.tld, receives the lists public key, signed by users that chose to publicly sign it. _TODO:_ not the best solution, the problem of binding the list key to a list, and in general key management, is key for this project (see what I did there? :) #### Subscription - Public key a required argument on list subscription, confirmation token sent encrypted with given key to subscribed address, signed confirmation token required (binds users public key with email address used for subscription) #### List moderation - Subscription moderation required for an encrypted list, otherwise, what's the point? #### Commands - Required to be signed by user's private key, otherwise discard/bounce (configurable) - New command for key management - `key` - `change` - would require signature with old key, also a new key as an argument - `sign` - would require one argument, list's public key signed with users private key, this list key with users signature will be distributed as the list public key (if the signature is valid and from the correct subscriber of the list). Users who do this have to understand that their signature of the lists public key will be public, thus their subscription will also be public. _TODO:_ It may be possible to allow non-subscribers to sign the lists public key, thus subscibers get some deniability of being a subscriber. #### Posting - Based on list configuration, posting should be encrypted with list pubkey and signed with users privkey #### Unsubscription - Same as now, signature required as for other commands ### Technical details - Will use OpenPGP at a low-level(OpenPGP packet manipulation) to: - keep the original user's signature - re-encrypt the message to the list's recipients (in configurable size batches) - re-encrypt in a way that strips key-ids (not to compromise privacy) - additionally sign with the list's private key (configurable) - currently most promising python lib -> https://github.com/mitchellrj/python-pgp - All possible, see rfc4880 - Considered SELS http://sels.ncsa.illinois.edu/ or similiar proxy encryption scheme (see references), however: - In SELS the list server has no access to the message plaintext, which is great for confidentiality but not for list archiving/moderation/many Mailman features. - In SELS the list server generates the users private key or at least has to have some information derived from subscribers private keys to work. - In SELS, many tasks are offset from the list server to the list manager, which actually enables the level of confidentiality the list server has in that model, however seems very impractical. - General conclusion, SELS doesn't satisfy Mailman's needs for this project idea ### What and where? - Mailman Core - *core keyring* - (gnupg keyring) contains list keypairs, and list archive public keys - *users keyring* - (gnupg keyring) contains users public keys (for all lists in a given Mailman instance) - rules in chain, will enforce list configuration such as: - discard/bounce non-encrypted - discard/bounce non-signed - handlers - to decrypt PGP/MIME message content - to strip additional header data leaks - to re-rencrypt and sign PGP/MIME message content for subscribers, in batches - runners - possibly a new runner to send dummy messages to subscribers to mitigate traffic analysis (anonymity is not a requirement here/nor a goal, but message existence and list traffic pattern is data that can not be "encrypted" yet can leak) - REST api - list key management - user key management - _TODO:_ this gives the BASIC AUTH required to access the REST api pretty huge permissions, a more granular access control would be beneficial - mta - SMTPS/STARTLS support - models/db - address <-> key fingerprint (to find key in *users keyring*) - list <-> list key fingerprint - list <-> list archive key fingerprint - commands - `key` command for key management - new module -> security - provides interface to manage the *core* and *users keyrings* to rest of Mailman core - Mailman client - binding of - list key management REST api - user key management REST api - HyperKitty - *archive keyring* - (gnupg keyring) contains list archive keypairs - decrypts messages received from Mailman-HyperKitty using list archive private keys - Mailman-HyperKitty - has access to the *core keyring* with list archive public keys, uses them to encrypt before sending to HyperKitty - Postorius - list configuration - list key management (possibly too dangerous if not run behind HTTPS, and even then), only accessible to list owners - list subscription - public key a required argument - user key management - _TODO:_ doesn't have the same level of security when done through Postorious, with just mail + password, as when done through list (signed), what do? ## Performance - Message encryption an obvious bottle-neck - Working with OpenPGP packets brings a speedup since the message itself is encrypted only once for a batch, but many PKESK packets will need to be created (this is unavoidable). - Current Mailman Core architecture with runners being separate processes adapts to this nicely ## Attacks and mitigations - User -> MTA - sniffing, altering: - Messages encrypted with list pubkey - dropping: - No mitigation - writing: - List configured to require subscribers signature - MTA -> Mailman - sniffing, altering, dropping, writing: - same as above - Mailman -> MTA - sniffing, altering: - Messages encrypted with users public keys - dropping: - No mitigation - writing: - Messages signed with list private key (in addition to any user's original signature) - MTA -> User - sniffing, altering: - Messages encrypted with users pubkey - dropping: - No mitigation - writing: - Messages signed with list private key (in addition to any user's original signature) - Replay attacks - Since user signature is kept, when the list is set to discard non-signed messages a replay attack without list subscribers noticing is not possible (as the signature couldn't be stripped). The signature of the original and replayed message would be the same, which would alert the subscribers that the message was replayed. - list <-> list pubkey binding - _TODO:_ very important, have some ideas - filesystem access: - Mailman Core queues - Put Mailman queue_dir or possibly the whole var_dir into an encrypted fs, mount on start (admin enters passphrase), unmount on quit - Mailman core keyring/HyperKitty archive keyring - Use a passphrase encrypted keyring, enter passphrase manually on Mailman start, use gpg-agent. _TODO:_ gpg-agent doesn't have infinite ttl support, try merge upstream? ## References - SELS: A Secure E-mail List Service - Khurana, Slagell, Bonilla: https://pdfs.semanticscholar.org/fbc2/f88ccc19eaf864171554e52af66b31bb1e91.pdf - SELS slides - Khurana: http://www.ncsa.illinois.edu/People/hkhurana/UIUCSecurity.pdf - From Proxy Encryption Primitives to a Deployable Secure-Mailing-List Solution - Khurana, Heo, Pant: http://www.ncsa.illinois.edu/People/hkhurana/ICICS.pdf - Multiplex Encryption: A Practical Approach to Encrypting Multi-Recipient Emails - Wei, Ding, Chen: http://www.mysmu.edu/faculty/xhding/publications/m-enc.pdf - On the Security of a Multi-party Certified Email Protocol - Zhou: https://pdfs.semanticscholar.org/faa0/2c9e25afcee3e357a321ca323bfbeddefd9c.pdf) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From i.fothergill at lancaster.ac.uk Wed Mar 15 12:37:39 2017 From: i.fothergill at lancaster.ac.uk (Fothergill, Iain) Date: Wed, 15 Mar 2017 16:37:39 +0000 Subject: [Mailman-Developers] Mailman 2.1.23 patch for MIME changes with headers or footers Message-ID: Hi, I hope this hits the right people (looking at some archives it seems to be, despite the list being mainly for MM3). Apologies if you consider it just noise. I have a patch for Mailman 2.1.23 for MIME munging when there are no headers or footers. There is code to prevent this from happening but it wasn't happening for us. The problem we saw occurred when a meeting request was created in Outlook and a mailing list was added as an invited recipient. Mailman would place MIME sections in a multipart which Outlook didn't recognice as a meeting request. This patch simply allows whitespace-only headers and footers to be considered 'empty' so that the MIME structure is not altered in such cases. I suspect just testing for a single newline character would be enough, but I like to wear a belt and braces. Iain -------------- next part -------------- A non-text attachment was scrubbed... Name: Mailman--Handlers--Decorate.py.patch Type: text/x-patch Size: 1064 bytes Desc: Mailman--Handlers--Decorate.py.patch URL: From rsk at gsp.org Wed Mar 15 21:47:46 2017 From: rsk at gsp.org (Rich Kulawiec) Date: Wed, 15 Mar 2017 21:47:46 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> Message-ID: <20170316014746.GA19047@gsp.org> All of these proposals overlook significant known, current threats -- none of which they're capable of addressing, but some of which badly undercut the suggested approaches. To list just one of those -- albeit a rather prominent one -- the Internet's population of hijacked systems (aka bots or zombies) continues to grow. This has been a growing problem for the last 15 years, e.g.: Vint Cerf: one quarter of all computers part of a botnet http://arstechnica.com/news.ars/post/20070125-8707.html I have studied this issue extensively since 2002 and while I initially thought Cerf's estimate a bit high, further study and retropection suggests that it was probably about right. Extrapolating to the present day, one-quarter is probably still about right -- but of course the system population has grown massively in the interim. The problem has recently been badly exacerbated by the rapid deployment of IoT devices whose security ranges between "laughable" and "non-existent". These in turn are quickly being utilized to compromise other systems. The problem is also being badly exacerbated by various governments and organized criminal operations which are developing, acquiring, and deploying zero-days as fast as they possibly can. And it's being further exacerbated by the increasingly sophisticated attacks conducted by less prominent and well-resourced adversaries; to put it another way, the average attacker now has access to means and methods far beyond what they had a decade ago. I rather suspect that "one quarter" will become "one third" in the next few years. What all of this means is that once a list passes N members, where we can debate about N, the probability that at least one of those members has already been compromised even before they've joined the list starts rapidly increasing. Of course other factors may mitigate this: if all N members use exclusively open-source software, do not use freemail providers, do not use smartphones or IoT devices, etc., then the probability that one of them is compromised diminishes. (Worth noting that in a list constituted like this, encryption offers little additional security value, since its members are already doing the things most likely to avoid being compromised.) If on the other hand, some of the list members are using worst practices, then the probability that at least one is compromised will increase. As I said, we can debate N -- and we can debate the probability. What is not open to debate is that this is real and significant. Very long experience running mailing lists and observing partial bot-generated activity from members strongly suggests, to give just one data point, that once N reaches "a few hundred" the probability approaches unity. However, I must emphasize that the word "partial" means that this is a significant UNDER-observation -- it's very clear that there is bot-generated activity I'm missing. Rather a lot of it, actually. So "a few hundred" is probably a highly optimistic estimate for N and its true value is probably much lower. So even if the encryption works perfectly (which it won't) and it's deployed perfectly (which it won't be) and it's usable by everyone (which it won't be) and it plays nice with policies like attachment removal, signature removal, boilerplate addition, etc. (which it won't) and the encryption algorithm is perfect (which it won't be) and the encryption implementation is perfect (which it won't be) and all of this rather complex machinery works perfectly...it will all be rendered moot the moment one list member's system is compromised. In other words, what you propose to build here is an extremely brittle system that's subject to total failure if even just a single endpoint fails. And there are *hundreds of millions* of endpoints that have already failed. Thus, even assuming that the systems of encrypted-list members aren't specifically targeted, there is an uncomfortably high probability that the messages traversing it will be pre-compromised from the start. And of course if those systems *are* specifically targeted, which of course is likely for people with use cases that suggest encrypted mailing lists, then the threat models changes and no longer consists of the normal level of attacks that all systems are subject to, but includes an elevated level of attacks that will target them in particular. I think that this is an instance where a huge amount of well-intended design and development effort will result in a "solution" that cannot provide what it intends to because underlying circumstances prevent it. And -- having studied those underlying circumstances for a long time -- I can sadly report that the problem is getting worse and will continue to get worse, because (a) all of the various factors contributing to it are also getting worse and (b) there are no reasons for anyone to significantly invest in making it better. ---rsk From mark at msapiro.net Wed Mar 15 22:51:15 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 15 Mar 2017 19:51:15 -0700 Subject: [Mailman-Developers] Mailman 2.1.23 patch for MIME changes with headers or footers In-Reply-To: References: Message-ID: <5f741036-51ba-d745-35f0-fec8ae1dfc92@msapiro.net> On 03/15/2017 09:37 AM, Fothergill, Iain wrote: > > I hope this hits the right people (looking at some archives it seems to be, despite the list being mainly for MM3). Apologies if you consider it just noise. I'm here. It probably would have been better to report this at , but I have now done so > I have a patch for Mailman 2.1.23 for MIME munging when there are no headers or footers. There is code to prevent this from happening but it wasn't happening for us. > > The problem we saw occurred when a meeting request was created in Outlook and a mailing list was added as an invited recipient. Mailman would place MIME sections in a multipart which Outlook didn't recognice as a meeting request. You can avoid this issue by ensuring that msg_header and msg_footer are truly empty, but since this seems to come up occasionally, I'll fix it. I probably won't patch Decorate.py exactly as you have, but thanks for the hint. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From jbn at forestfield.org Thu Mar 16 00:31:44 2017 From: jbn at forestfield.org (J.B. Nicholson) Date: Wed, 15 Mar 2017 23:31:44 -0500 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170316014746.GA19047@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> Message-ID: <20836e20-3a1f-57bc-d520-8fa722b3fbc0@forestfield.org> Rich Kulawiec wrote: > What all of this means is that once a list passes N members, where > we can debate about N, the probability that at least one of those > members has already been compromised even before they've joined the > list starts rapidly increasing. I understand there are more insecure devices on the Internet all the time and that's unfortunate, but I don't think it's avoidable. What do you suggest we do about this using Mailman (since this is Mailman-developers)? Perhaps this means I don't understand what the goals of combining a mailing list and public key cryptography are (could someone please state what those goals are?). I took the goals to be the following: - make changes in messages easier to identify at the endpoints. So long as posters use strong cryptography methods and sign+encrypt their posts. Sure, a compromised device could change the message between the time someone writes their message and the time they sign+encrypt it, thus signing+encrypting an altered message. But we have that problem now and I don't see anyone calling for all research work to stop on any number of other things because of it. Also, for those without compromised devices who know what they're doing (a smaller set of people, as you point out) posts to mailing lists are likely easily changeable without most people being the wiser or having any ability to verify short of constantly asking others "Did you really post this?". Given how much en route data alteration is going on, it seems we ought to do something to at least let the user know the message they're looking at has a high likelihood of not being what was sent. - provide a practical means of using extant services (along with most of the UI expectations and technical advantages we've come to expect) to convey encrypted data and store encrypted data such that the plaintext of a message is not often exposed to any program server-side. - allow users to do some degree of identity confirmation. With what I've seen in this thread so far, poster identities are as verifiable as public key encryption and web of trust allow. If I see a post from someone I trust whom I know knows how to use, say, GPG correctly I then have increased confidence their post was signed by them. Currently, where lists are typically entirely plaintext, I understand it's quite easy for someone to post in someone else's name and email address and for any network operator (such as one's ISP) to alter the data en route. But I could have the goals of this entire endeavor completely wrong, in which case I await correction. From rsk at gsp.org Thu Mar 16 10:46:27 2017 From: rsk at gsp.org (Rich Kulawiec) Date: Thu, 16 Mar 2017 10:46:27 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20836e20-3a1f-57bc-d520-8fa722b3fbc0@forestfield.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20836e20-3a1f-57bc-d520-8fa722b3fbc0@forestfield.org> Message-ID: <20170316144626.GC31856@gsp.org> On Wed, Mar 15, 2017 at 11:31:44PM -0500, J.B. Nicholson wrote: > I understand there are more insecure devices on the Internet all the time > and that's unfortunate, but I don't think it's avoidable. What do you > suggest we do about this using Mailman (since this is Mailman-developers)? I suggest that Mailman do nothing, because even if it solves all the problems that it can solve, all it will do is provide a thin veneer of security/privacy on top of a thoroughly rotten foundation. Yes, there will be small, limited cases where it'll be able to deliver on its promises -- because every person involved is diligent and every device involved is secure -- but that's clearly not the way to bet. Moreover, none of this comes for free: there is opportunity cost, complexity cost, maintenance cost, interoperability cost, etc. In my view, it's not worth incurring all these costs to implement something that we already know, today, right now, is not going to work in the contemporary Internet environment -- because it relies on underlying assumptions about endpoint security that almost certainly won't be true as soon as the deployment scale reaches modest numbers. I think a better course of action is to recommend that those with the sort of requirements being articulated here not use mailing lists at all. ---rsk From nb at bollow.ch Thu Mar 16 15:10:03 2017 From: nb at bollow.ch (Norbert Bollow) Date: Thu, 16 Mar 2017 20:10:03 +0100 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170316144626.GC31856@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20836e20-3a1f-57bc-d520-8fa722b3fbc0@forestfield.org> <20170316144626.GC31856@gsp.org> Message-ID: <20170316201003.3dfa529d@quill> On Thu, 16 Mar 2017 10:46:27 -0400 Rich Kulawiec wrote: > I suggest that Mailman do nothing, because even if it solves all the > problems that it can solve, all it will do is provide a thin veneer of > security/privacy on top of a thoroughly rotten foundation. Yes, there > will be small, limited cases where it'll be able to deliver on its > promises -- because every person involved is diligent and every device > involved is secure -- but that's clearly not the way to bet. Even if not every device is secure, the difficulty, and likely cost, for an attacker to snoop on the communications is much greater for an encrypted mailing list is than for a non-encrypted one. FWIW, I'm part of an NGO (Digital Society Switzerland, https://digitale-gesellschaft,ch ) which uses encrypted mailing lists for its internal communications. We use Schleuder ( https://schleuder.nadir.org/ ), which isn't perfect, but works fine for us. Greetings, Norbert > > Moreover, none of this comes for free: there is opportunity cost, > complexity cost, maintenance cost, interoperability cost, etc. > In my view, it's not worth incurring all these costs to implement > something that we already know, today, right now, is not going to > work in the contemporary Internet environment -- because it relies > on underlying assumptions about endpoint security that almost > certainly won't be true as soon as the deployment scale reaches > modest numbers. > > I think a better course of action is to recommend that those with the > sort of requirements being articulated here not use mailing lists at > all. > > ---rsk > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > https://mail.python.org/mailman/options/mailman-developers/nb%40bollow.ch > > Security Policy: http://wiki.list.org/x/QIA9 From barry at list.org Thu Mar 16 17:30:36 2017 From: barry at list.org (Barry Warsaw) Date: Thu, 16 Mar 2017 17:30:36 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170316014746.GA19047@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> Message-ID: <20170316173036.0541e65b@subdivisions.wooz.org> On Mar 15, 2017, at 09:47 PM, Rich Kulawiec wrote: >What all of this means is that once a list passes N members, where >we can debate about N, the probability that at least one of those >members has already been compromised even before they've joined the >list starts rapidly increasing. That assumes an open membership policy. Wouldn't much of this be mitigated with a closed subscription policy? I agree that the security of an encrypted remailer such as we're discussing is only as secure as its recipients. Yet there still may be value in encrypting the communication channels into and out of Mailman, even if that can be compromised at the end-points. Does that make it worth it to add as a supported feature in the core? It depends. What I would be very interested in -at least as a first step- are ways to enable experimentation into such features by the addition of hooks and APIs that allow third-party plugins that could support this feature. Presumably such plugins would have utility for other use cases too. >I can sadly report that the problem is getting worse and will continue >to get worse, because (a) all of the various factors contributing to it >are also getting worse and (b) there are no reasons for anyone to >significantly invest in making it better. (b) is not necessarily true. There is lots of work going on to provide secure base platforms on which to implement IoT devices. Since that's mostly off topic for this list, I'll avoid plugging technologies here, but if you know me and my $dayjob, you can probably guess. Feel free to email me off-list if you want more information. Cheers, -Barry From morgan.s.reed at gmail.com Thu Mar 16 18:54:48 2017 From: morgan.s.reed at gmail.com (Morgan Reed) Date: Fri, 17 Mar 2017 09:54:48 +1100 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170316014746.GA19047@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> Message-ID: On Thu, Mar 16, 2017 at 12:47 PM, Rich Kulawiec wrote: > I think that this is an instance where a huge amount of well-intended > design and development effort will result in a "solution" that cannot > provide what it intends to because underlying circumstances prevent it. > And -- having studied those underlying circumstances for a long time -- > I can sadly report that the problem is getting worse and will continue > to get worse, because (a) all of the various factors contributing to it > are also getting worse and (b) there are no reasons for anyone to > significantly invest in making it better. > I'd submit that this is tantamount to saying "it's impossible to make a 100% secure system so why bother even trying". Yes, a straight forward encrypted remailer such as is being discussed here has limited applicability, and will require careful implementation to function "properly" in the "real world". However I'm certainly aware of several organisations who would happily use such a platform, in a limited capacity for endpoints which have already been hardened. Nobody expects that when this work is completed that every mailman list in the world will suddenly become encrypted, because it won't and the VAST majority of them never will. Anything that raises the barrier to entry for surveillance is a good thing IMO, if you go from being able to "passively sniffing plaintext off the wire" to having to "actively compromising an endpoint" that is still an improvement in your security posture and that is the way things change, incrementally. Somebody else on the thread has raised the point of whether this should actually be implemented in core or if we should merely expose API to enable this functionality to be added plugin-wise, I think that's a debate that's worth having, but suggesting we write the whole thing off because "all these nodes are already compromised" is not remotely useful. From jbn at forestfield.org Thu Mar 16 23:31:33 2017 From: jbn at forestfield.org (J.B. Nicholson) Date: Thu, 16 Mar 2017 22:31:33 -0500 Subject: [Mailman-Developers] Public-key signing+encryption plugin API foreseeable needs In-Reply-To: <20170316173036.0541e65b@subdivisions.wooz.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> Message-ID: <25897f51-0e81-9145-6a4f-08a32be93c99@forestfield.org> Barry Warsaw wrote: > Does that make it worth it to add as a supported feature in the core? It > depends. What I would be very interested in -at least as a first step- are > ways to enable experimentation into such features by the addition of hooks and > APIs that allow third-party plugins that could support this feature. > Presumably such plugins would have utility for other use cases too. I was thinking this same thing; if Mailman had such hooks and an API allowing people to write plugins, everything discussed in this thread so far could be done with a plugin. The overall goal for this plugin's API needs is to allow a Mailman admin or list owner to install this plugin and see the normal Mailman UI change slightly in relevant places to accommodate the needs of public keys. Such a plugin for this task would need to: - have read access to each inbound message sent to a list where this plugin is supposed to be active. This is used to check a signature and possibly decrypt a message intended for list consumption when people send commands to the list encrypted with the list's public key. - somehow signal acceptance/rejection (to the next stage in Mailman's normal processing queue) of a message as coming from a subscriber (for lists where only subscribers may post) or a suitable other user (for lists where non-subscribers may also post). - get read/write access to each outbound message from a list before it is mailed to subscribers. This is where the list-signing + encryption would be done. - make a list command (even one that overrides the normal Mailman list commands) so one could do things with public keys relevant to this plugin -- subscribe to a list and supply either a pointer to or a copy of a public key at the same time in the same subscribe command, for example. I imagine there would be other commands that need similar public key additions to syntax. - be able to add fields, table columns, and form elements to an extensible web UI so key-related functions and displays can be integrated into extant displays -- subscribing on the web should include a prompt where one uploads one's public key or provides a URL where Mailman can fetch said public key. Getting a list of subscribers (if that's allowed per list policy) should include perhaps the key signature of each subscriber (if that subscriber is okay with publishing that info), the subscriber info for each list should include a setting where each user can decide if they want the list to publish the key signature (I think this is akin to current Mailman asking about hiding the user's identity in the subscriber's list), etc. - a way to place the list's public key in a file everyone could read at a predictable static URL. MUAs would use this URL to automatically fetch the list's public key. - a way to add to the documentation describing the plugin's added functionality; best would be to not have to replace extant documentation pages, but add pointers in current documentation pointing to new pages (which are installed in a place along side extant docs). The plugin might also need an acceptable place on the server to store state data: a GPG keyring holding a copy of each subscriber & moderator's public key, the list's public and private keys, and some temporary space where the list can do work signing+encrypting messages (or perhaps GPG has a better way to handle this and the plugin should use whatever GPG provides). I'm not familiar with Mailman's current hooks/API to know if any of this already exists, or is in-line with how Mailman generally does things. Functionally, the list needs to inform all subscribers of the new list public key each time it changes. Maybe that is done with a header in a post with the list's public key's signature (saying the key with this signature was used to sign+encrypt this message) and it would be up to the MUA to get (and cache) the appropriate public key to verify this post? There are also some policy issues I raised in another post on this thread regarding GPG's support for: obsolete hash/encryption algorithms, how this project wants to handle list archives (particularly re-signing+encrypting list archives). I think those issues should be considered. They seem to me to be important factors in how everyone interested in the list would use the list. Finally, I'm not sure what the goals for the project were in the first place and the goals I posted are really only things I think are possible to implement. Here's another of my guesses aimed at succinctly describing what this project will do: replace the subscription and posting filtering mechanism (currently based on email address string comparisons) with public key cryptography; if one sends in a post from any email address signed (or signed+encrypted, as per list policy?) with the right public key, that post is deemed to come from a subscriber. Other posts are handled in accordance with list policy. If the project leader could speak to the project's goals, it might help us all understand what's in and out of scope for the project. Also, thanks for letting someone completely unconnected with this project chime in out of nowhere. From rsk at gsp.org Sat Mar 18 13:54:05 2017 From: rsk at gsp.org (Rich Kulawiec) Date: Sat, 18 Mar 2017 13:54:05 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170316201003.3dfa529d@quill> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20836e20-3a1f-57bc-d520-8fa722b3fbc0@forestfield.org> <20170316144626.GC31856@gsp.org> <20170316201003.3dfa529d@quill> Message-ID: <20170318175405.GA32096@gsp.org> On Thu, Mar 16, 2017 at 08:10:03PM +0100, Norbert Bollow wrote: > Even if not every device is secure, the difficulty, and likely cost, > for an attacker to snoop on the communications is much greater for an > encrypted mailing list is than for a non-encrypted one. The difficulty is greater -- but not by much. Attackers have long since become extremely proficient at installing keystroke loggers and extracting credentials in order to compromise many other forms of communication. It's only an incremental, low-cost step for them to extend those techniques to encrypted mailing lists. Now I'll grant that this is unlikely to happen immediately (except for intelligence agencies, who will be ready for this before it's deployed in the field). But one of the things that we've seen over and over again is that once attackers decide that a particular target (or kind of target) has value, they'll focus on it with surprisingly rapidity. ---rsk From rsk at gsp.org Sat Mar 18 16:04:09 2017 From: rsk at gsp.org (Rich Kulawiec) Date: Sat, 18 Mar 2017 16:04:09 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> Message-ID: <20170318200409.GA9534@gsp.org> On Fri, Mar 17, 2017 at 09:54:48AM +1100, Morgan Reed wrote: > I'd submit that this is tantamount to saying "it's impossible to make a > 100% secure system so why bother even trying". Then you're not grasping my point. Let me try again. I suggest that you re-read what I've written *and* consider as well the disclosures of the past week vis-a-vis smartphones and their encrypted communications applications. In particular, note that entities like Whisper and Signal have been, as I've said for years, peddling snake-oil. They cannot possibly deliver on their promises *even if they do everything they say they can do* because all of it is immediately and completely undercut if the underlying system is compromised. Which is exactly what the disclosures of Vault 7 show everyone, although it's not really news to anyone who's been paying attention. Intelligence agencies, vulnerability brokers, organized cybercrime, and others have been knocking themselves out to hack everything for years -- and whaddaya know, they've succeeded. This set of disclosures is merely the latest, and it and all the other ones to date are merely the tip of the iceberg. So what I am saying, and what I hope is obvious, is that you cannot build a secure system on top of an insecure one. This isn't about not being able to build a 100% secure system: as a long-time security professional, I'm fully aware that's impossible and that the best we can do is to stack the deck in our favor. This is about building a system that is known 0% secure from the start. I think, in the end, this will serve the community poorly -- because people who don't grasp the contemporary security landscape will deploy it, will rely on it, and will not understand that they lost the game before they even started to play it. This will have consequences. ---rsk From rsk at gsp.org Sat Mar 18 16:37:40 2017 From: rsk at gsp.org (Rich Kulawiec) Date: Sat, 18 Mar 2017 16:37:40 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170316173036.0541e65b@subdivisions.wooz.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> Message-ID: <20170318203740.GB9534@gsp.org> On Thu, Mar 16, 2017 at 05:30:36PM -0400, Barry Warsaw wrote: > On Mar 15, 2017, at 09:47 PM, Rich Kulawiec wrote: > > >What all of this means is that once a list passes N members, where > >we can debate about N, the probability that at least one of those > >members has already been compromised even before they've joined the > >list starts rapidly increasing. > > That assumes an open membership policy. Wouldn't much of this be mitigated > with a closed subscription policy? It *might* be. The problem is that the list owner and other list members have no way to know. From their point of view, there is no way to know that whether the latest list member -- whether that's list member #8 or #7,221 -- is using a reasonably secure mail client on a reasonably secure operating system in a reasonably secure environment -- or whether they're reading list traffic on an iPhone that was fully compromised eight months ago. Morever, even if that newest list member is doing the former today, nothing from prevents them from doing the latter tomorrow. (Yes, one could ask them not to, even make not doing so a condition of membership. That won't work. Somebody is going read email on their fridge or their car or their Android phone because they can, because they're lazy, because it's convenient, because they feel like it.) It's thus impossible to (a) estimate the risk or (b) control the risk or (c) know when a full compromise has taken place, absent outside indicators. That's a really bad combination to have in anything that's trying to be secure. > Yet there still may be value in encrypting the communication channels > into and out of Mailman, even if that can be compromised at the end-points. I agree. > >I can sadly report that the problem is getting worse and will continue > >to get worse, because (a) all of the various factors contributing to it > >are also getting worse and (b) there are no reasons for anyone to > >significantly invest in making it better. > > (b) is not necessarily true. There is lots of work going on to provide secure > base platforms on which to implement IoT devices. I'm aware of at least some of that, and I'd like to hope for the best. But economic incentives being what they are, there is little motivation for vendors to bother. Moreover, many vendors are deliberately compromising end-user privacy and security (e.g., Vizio) because it's profitable to do so and the penalties, if any, are a mere slap-on-the-wrist. (I know you see a lot of this because of what you do; other folks might want to browse through TechDirt's ongoing partial catalog of IoT failures.) My view -- at the moment, ask again tomorrow ;) -- is that so many IoT devices have been rushed to market with no consideration for security and privacy issues that the present situation is untenable. The best thing would be to recall *all* of them: all the smartphones, all the watches, all the TVs, everything...and start over. That's of course ludicrous and won't happen. Which means all those devices will persist in the field, joined by new ones in large numbers every day. And the slow backfill of fixes which *might*, in a vacuum, actually suffice, aren't going to be enough because so much of the rest of the IoT ecosystem is a mess. In a relatively short time we've taken a system built to resist destruction by nuclear weapons and made it vulnerable to toasters. --- Jeff Jarmoc ---rsk From Richard at Damon-Family.org Sun Mar 19 07:33:24 2017 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 19 Mar 2017 07:33:24 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170318203740.GB9534@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> Message-ID: <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> On 3/18/17 4:37 PM, Rich Kulawiec wrote: > On Thu, Mar 16, 2017 at 05:30:36PM -0400, Barry Warsaw wrote: > ... > It *might* be. > > The problem is that the list owner and other list members have no way to > know. From their point of view, there is no way to know that whether the > latest list member -- whether that's list member #8 or #7,221 -- is using > a reasonably secure mail client on a reasonably secure operating system in > a reasonably secure environment -- or whether they're reading list traffic > on an iPhone that was fully compromised eight months ago. Morever, even if > that newest list member is doing the former today, nothing from prevents > them from doing the latter tomorrow. > > (Yes, one could ask them not to, even make not doing so a condition of > membership. That won't work. Somebody is going read email on their > fridge or their car or their Android phone because they can, because > they're lazy, because it's convenient, because they feel like it.) > > It's thus impossible to (a) estimate the risk or (b) control the risk or > (c) know when a full compromise has taken place, absent outside indicators. > > That's a really bad combination to have in anything that's trying to be secure. > Barry, I would say that the problem that is being attempted to solve is fundamentally impossible to do perfectly. It is impossible to distribute messages in a secure manner to a number of recipients that you don't have total control over their enviroment and KNOW that security is being maintained. Communication always has that sort of issue, if you tell someone something private, you need to be able to trust that they will keep it private, and their is always a risk that they will reveal the information intentionally or accidentally. The question comes, is it better to provide a method that gets you part way to the goal, and risk a false sense of security, or to not provide any method at all. The is comparable to the fact that we lock our homes and cars to keep them 'secure', even though we know that security isn't perfect. Doing so reduces that attack surface, but it is sometimes hard to estimate by how much. Yes, if such a feature was added, adding a notice to remind people that the security provided is only as good as the weakest link among all the members of the list would make sense. -- Richard Damo From nb at bollow.ch Sun Mar 19 13:14:22 2017 From: nb at bollow.ch (Norbert Bollow) Date: Sun, 19 Mar 2017 18:14:22 +0100 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170318175405.GA32096@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20836e20-3a1f-57bc-d520-8fa722b3fbc0@forestfield.org> <20170316144626.GC31856@gsp.org> <20170316201003.3dfa529d@quill> <20170318175405.GA32096@gsp.org> Message-ID: <20170319181422.5febb8d9@quill> On Sat, 18 Mar 2017 13:54:05 -0400 Rich Kulawiec wrote: > On Thu, Mar 16, 2017 at 08:10:03PM +0100, Norbert Bollow wrote: > > Even if not every device is secure, the difficulty, and likely cost, > > for an attacker to snoop on the communications is much greater for > > an encrypted mailing list is than for a non-encrypted one. > > The difficulty is greater -- but not by much. Attackers have long > since become extremely proficient at installing keystroke loggers > and extracting credentials in order to compromise many other forms > of communication. It's only an incremental, low-cost step for them > to extend those techniques to encrypted mailing lists. > > Now I'll grant that this is unlikely to happen immediately (except > for intelligence agencies, who will be ready for this before it's > deployed in the field). But one of the things that we've seen over > and over again is that once attackers decide that a particular > target (or kind of target) has value, they'll focus on it with > surprisingly rapidity. That is true, if the attacker already knows whose communications they want to snoop on. However one of the main benefit of using encrypted communications is in the area of making it much more expensive and politically risky for the attacker to determine which targets have value. In the absence of encryption, that can be achieved by means of mass surveillance anywhere between the communications endpoints followed by (possibly AI-based) pattern analysis, at near-zero incremental cost and near-zero incremental risk per additional group that is subjected to such surveillance for reasons of its communications being possibly of interest to the attacker. Greetings, Norbert From turnbull.stephen.fw at u.tsukuba.ac.jp Tue Mar 21 06:24:43 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 21 Mar 2017 19:24:43 +0900 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170316014746.GA19047@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> Message-ID: <22736.65387.760048.810453@turnbull.sk.tsukuba.ac.jp> Rich Kulawiec writes: > What all of this means is that once a list passes N members, where > we can debate about N, the probability that at least one of those > members has already been compromised even before they've joined the > list starts rapidly increasing. This is true, but you've omitted (well, hidden in the gloss about "correct usage") the most important source of compromise: the subscribers themselves. The most important use case I have in mind is not actually encrypted lists per se, but rather anonymized and encrypted lists. If done properly (I have not even attempted the analysis yet, and may not be competent to conduct it), it may be useful against garden-variety stalkers, requiring them to access server logs or the particular user's client (in which case they're presumably already done for) to de-anonymize. Of course, in most cases a competent (I'm not even going so far as "advanced") and persistent attacker will be able to compromise a server as well (thus "garden-variety" = script kiddie). If nothing else, I hope to educate a half-dozen GSoC applicants that "encryption is at best a 10% solution, and more likely 7%". :-/ And then Rich Kulawiec writes: > In particular, note that entities like Whisper and Signal have been, as > I've said for years, peddling snake-oil. They cannot possibly deliver > on their promises *even if they do everything they say they can do* > because all of it is immediately and completely undercut if the > underlying system is compromised. Compromise of the underlying systems still typically requires cooperation from the user. Agreed, most people who casually think, "oh, an encrypted list! that's useful", are already busted. "Caveat: that word ('encrypted') doesn't mean what you think it means" should be warning enough (for our CYA, anyway). > This is about building a system that is known 0% secure from the > start. Is that 0% Kelvin or 0% Celsius? ;-) > I think, in the end, this will serve the community poorly -- because > people who don't grasp the contemporary security landscape will deploy it, > will rely on it, and will not understand that they lost the game > before they even started to play it. This will have consequences. People are already affronted that pretty much anybody who wants to can read their email, but that's the fact and it has consequences. I'm not sure we need to take responsibility for that. I'm willing to hear more about that, but not on the basis of strawmen like "0% secure". As Zeynep Tufekci has been at pains to point out since the Guardian WhatsApp fiasco, that doesn't mean we shouldn't provide tools imposing additional effort on the bad guys for use by those who do understand the risks in this environment. If you want to point out what use cases are broken from the word "go", even if that includes my preferred applications, fine. But I think it's reasonable to expect that a number of user groups capable of taking advantage do exist. And he persists: > Moreover, none of this comes for free: there is opportunity cost, > complexity cost, maintenance cost, interoperability cost, etc. It's nearly free, because there are a lot of GSoC wannabes out there who think this is "way cool". Disabusing them of that notion may be the most important contribution of this project. > In my view, it's not worth incurring all these costs to implement > something that we already know, today, right now, is not going to > work in the contemporary Internet environment -- because it relies > on underlying assumptions about endpoint security that almost > certainly won't be true as soon as the deployment scale reaches > modest numbers. I thought that it already wasn't true even before we thought up this project, let alone when deployment can be expected to reach "modest scale"? Rich, calm down -- inconsistency is unbecoming of a security professional. ;-) Note my reply to Barry: it's not unreasonable to expect that the odds turn against you as soon as you *pass three subscribers*. So, yes, we are going to have to document that to the users, and tell them they are going to have to make serious effort to turn the odds in their favor for *any* use case they may have in mind. From turnbull.stephen.fw at u.tsukuba.ac.jp Tue Mar 21 06:27:17 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 21 Mar 2017 19:27:17 +0900 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170316173036.0541e65b@subdivisions.wooz.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> Message-ID: <22737.5.528094.307134@turnbull.sk.tsukuba.ac.jp> I hope it doesn't surprise anybody that despite being proponent of this project I'm quite sympathetic to Rich. Barry Warsaw writes: > That assumes an open membership policy. Wouldn't much of this be > mitigated with a closed subscription policy? Not if the target membership isn't already paranoid. Remember, 20%-40% of devices are already compromised. Even at the low end, assuming uniform draws, with *three* members odds are *even* that one is compromised. Sure, your assumption is non-uniform, but it's not clear it's more optimistic -- suppose feeling paranoid enough to consider an encrypted list means the probability they're out to get you is *higher* than uniform? And do you really think the proportion of truly tight-lipped potential subscribers is better than 80%? I'm not saying there's nothing useful here, but there's no longer any such thing as "paranoia" when it comes to IoT (where "thing" includes anything connected, not just embedded devices). > I agree that the security of an encrypted remailer such as we're > discussing is only as secure as its recipients. Yet there still > may be value in encrypting the communication channels into and out > of Mailman, even if that can be compromised at the end-points. Unless you're talking about a resistence cell in a society that has been authoritarian for a few decades, I think we should assume that content is freely available to anybody who really wants it. It's not just John Podesta "who should know better", I've seen testimony recently from a security professional saying they'd clicked on a spearphish. They were in an isolated environment and they're pretty sure no harm was done, but they did click unintentionally. Jus' plain folks have no chance. As I've said elsewhere, the only use case I'm seriously considering is encrypted + anonymized, so that you need to compromise (or subpoena) the server (or the exact sender) to identify senders of particular content. People smarter than me might be able to extend that area of applicability. > (b) is not necessarily true. There is lots of work going on to > provide secure base platforms on which to implement IoT devices. There's also active avoidance of the whole concept of security by major device (vs. platform) vendors. C'mon, guys, open telnet port on a router? Plus the reality that many devices produced by Chinese companies are almost certainly backdoored. It will be many years, maybe decades, before IoT means anything but "Internet of Threats". I still think this is worth doing, both for the occasional use case, and for many of the reasons you give, but the applications are far more restricted than the GSoC applicants seem to think. :-/ Steve From johny at neuromancer.sk Tue Mar 21 11:04:20 2017 From: johny at neuromancer.sk (johny) Date: Tue, 21 Mar 2017 16:04:20 +0100 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170318200409.GA9534@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170318200409.GA9534@gsp.org> Message-ID: <711c0682-9d77-8169-e613-bcb563e029f8@neuromancer.sk> On 03/18/2017 09:04 PM, Rich Kulawiec wrote: > On Fri, Mar 17, 2017 at 09:54:48AM +1100, Morgan Reed wrote: >> I'd submit that this is tantamount to saying "it's impossible to make a >> 100% secure system so why bother even trying". > > Then you're not grasping my point. Let me try again. > > I suggest that you re-read what I've written *and* consider as well the > disclosures of the past week vis-a-vis smartphones and their encrypted > communications applications. > > In particular, note that entities like Whisper and Signal have been, as > I've said for years, peddling snake-oil. They cannot possibly deliver > on their promises *even if they do everything they say they can do* > because all of it is immediately and completely undercut if the > underlying system is compromised. Open Whisper Systems and Signal provide what they state, End-to-End encryption. Applications and technologies like these make mass surveillance harder, as passively sniffing traffic is no longer viable. Shifting the attacker to actively compromise devices is an overall improvement. > > Which is exactly what the disclosures of Vault 7 show everyone, > although it's not really news to anyone who's been paying attention. > Intelligence agencies, vulnerability brokers, organized cybercrime, > and others have been knocking themselves out to hack everything > for years -- and whaddaya know, they've succeeded. This set of > disclosures is merely the latest, and it and all the other ones > to date are merely the tip of the iceberg. Obviously protection against state actors is hard [1]. However thats not the only threat source that there are reasons to protect against. There are plenty of threat actors for which sniffing traffic to a plaintext mailing list might be easy, however overcoming a well setup encrypted mailing list system would be quite hard. > > So what I am saying, and what I hope is obvious, is that you cannot > build a secure system on top of an insecure one. > > This isn't about not being able to build a 100% secure system: > as a long-time security professional, I'm fully aware that's impossible > and that the best we can do is to stack the deck in our favor. > This is about building a system that is known 0% secure from the start. The system security only increases in this case. It's security is obviously debatable against state actors/equivalent threats, there it might not improve much, but improves significantly against other threats. > > I think, in the end, this will serve the community poorly -- because > people who don't grasp the contemporary security landscape will deploy it, > will rely on it, and will not understand that they lost the game > before they even started to play it. This will have consequences. This assumes that those people are not currently relying on plaintext mailing lists or any other insecure messaging technology. I think it is quite obvious, from the nature of a mailing list, that every subscriber can read all messages. With proper documentation about security of endpoint devices and security of mailing lists, I think this feature has viable use-cases. -Jan [1]: https://www.usenix.org/system/files/1401_08-12_mickens.pdf -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From rsk at gsp.org Tue Mar 21 18:16:20 2017 From: rsk at gsp.org (Rich Kulawiec) Date: Tue, 21 Mar 2017 18:16:20 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <711c0682-9d77-8169-e613-bcb563e029f8@neuromancer.sk> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170318200409.GA9534@gsp.org> <711c0682-9d77-8169-e613-bcb563e029f8@neuromancer.sk> Message-ID: <20170321221620.GA3537@gsp.org> On Tue, Mar 21, 2017 at 04:04:20PM +0100, johny wrote: > Shifting the attacker to actively compromise devices is an overall > improvement. If "compromising devices" was difficult, I might agree. But it's not. Devices of all descriptions have been and are being compromised in enormous numbers on an ongoing basis even by relatively unskilled attackers -- since they can buy/lease the requisite tools and infrastructure and use them without needing to understand how they work. I think you (and others) are continuing to badly underestimate the scale at which this is taking place. We're not talking about an ecosystem in which 2% or 6% of devices are compromised; we're talking about one in which any estimate under 25% should be laughed out of the room and an estimate of 50% should be given serious consideration. (I think the latter may be still be too high. But it's certainly within the realm of plausibility.) We're also talking about an ecosystem in which, increasingly, vendors are shipping devices that are essentially pre-compromised at the factory due to very poor and entirely self-serving design and implementation decisions. > There are plenty of threat actors for which sniffing traffic to a > plaintext mailing list might be easy, however overcoming a well setup > encrypted mailing list system would be quite hard. I don't think so, if the mailing list is of sufficient size. (Certainly one with only a handful of people might be hard to crack, but that would be fairly hard today even without encryption.) > The system security only increases in this case. It's security is > obviously debatable against state actors/equivalent threats, there it > might not improve much, but improves significantly against other threats. State actors are not necessarily the biggest threat. They might have the most resources, and they might have the best cryptographers, and they certainly have the most political power (e.g., NSLs in the US), but they also have their own areas of focus and this may not be one of them. If there's money to be made by trafficking in information, then others will take an interest. They may not have the resources, cryptographers, power, etc., but they do have some very talented personnel, stockpiled exploits, and rather a lot of experience with mass compromise of end user systems. These are not script kiddies in mom's basement, these are professionals with the discipline to identify and attack targets successfully on a large scale. Don't underestimate them. *That* particular mistake was already made by every ISP on this planet ~15 years ago, which is one of the major reasons the problem has the scope that it has today. ---rsk From rsk at gsp.org Tue Mar 21 18:30:17 2017 From: rsk at gsp.org (Rich Kulawiec) Date: Tue, 21 Mar 2017 18:30:17 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> Message-ID: <20170321223017.GA4571@gsp.org> On Sun, Mar 19, 2017 at 07:33:24AM -0400, Richard Damon wrote: > I would say that the problem that is being attempted to solve is > fundamentally impossible to do perfectly. It is impossible to distribute > messages in a secure manner to a number of recipients that you don't have > total control over their enviroment and KNOW that security is being > maintained. Communication always has that sort of issue, if you tell someone > something private, you need to be able to trust that they will keep it > private, and their is always a risk that they will reveal the information > intentionally or accidentally. [snip] I think this (and the rest, which I've elided for brevity) is a very good statement of the problem. I'll just add that -- in the general case, and quoting from the above, we already KNOW that security is *not* being maintained. It's not an open question, it's been answered very clearly for well over a decade. (In the specific case, e.g., the right people using the right devices with the right knowledge and self-discipline: maybe. But there are not many of those cases and any of them can revert to the general case in seconds with one poor decision or perhaps even without one.) ---rsk From Richard at Damon-Family.org Wed Mar 22 08:23:02 2017 From: Richard at Damon-Family.org (Richard Damon) Date: Wed, 22 Mar 2017 08:23:02 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170321223017.GA4571@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> <20170321223017.GA4571@gsp.org> Message-ID: <0413b78b-31b4-7f74-b8c4-8aafb4fb0937@Damon-Family.org> On 3/21/17 6:30 PM, Rich Kulawiec wrote: > On Sun, Mar 19, 2017 at 07:33:24AM -0400, Richard Damon wrote: >> I would say that the problem that is being attempted to solve is >> fundamentally impossible to do perfectly. It is impossible to distribute >> messages in a secure manner to a number of recipients that you don't have >> total control over their enviroment and KNOW that security is being >> maintained. Communication always has that sort of issue, if you tell someone >> something private, you need to be able to trust that they will keep it >> private, and their is always a risk that they will reveal the information >> intentionally or accidentally. > [snip] > > I think this (and the rest, which I've elided for brevity) is a very good > statement of the problem. > > I'll just add that -- in the general case, and quoting from the above, > we already KNOW that security is *not* being maintained. It's not an > open question, it's been answered very clearly for well over a decade. > > (In the specific case, e.g., the right people using the right devices > with the right knowledge and self-discipline: maybe. But there are > not many of those cases and any of them can revert to the general case > in seconds with one poor decision or perhaps even without one.) > > ---rsk > The only way to keep a secret is not to tell it, as once you have told it, there is no way to keep the person you have told it from repeating it (intentionally, accidentally, or unknowingly). There are times (many of them) where it still makes sense to tell the secret and do your best to keep security. It is similar to the fact that I know my house is not totally burglar proof. A determined person will be able to break into my home to take/place things, and if they were very determined, maybe even do so undetected. This doesn't mean I give up on security, I still lock my door, because it make me more secure than otherwise. In the same way, an encrypted mailing list is not perfect, but it is a help, for the transmission of sensitive information that I wish to keep secret. It makes the transmission phase much more secure, and maybe helps a tiny bit on keeping the data at the end point secure. It should be know that, and prominently displayed in the documentation, that encrypted transmission doesn't help significantly with the security at the end points, and you need to evaluate your trust of the recipients to keep the information secure, One big thing that I haven't seen in the discussion of this problem is exactly WHAT issue/problem this feature is intended to solve, There are several different problems that encryption can help with, each needing different sort of support from the software. -- Richard Damon From turnbull.stephen.fw at u.tsukuba.ac.jp Wed Mar 22 11:02:40 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 23 Mar 2017 00:02:40 +0900 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <0413b78b-31b4-7f74-b8c4-8aafb4fb0937@Damon-Family.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> <20170321223017.GA4571@gsp.org> <0413b78b-31b4-7f74-b8c4-8aafb4fb0937@Damon-Family.org> Message-ID: <22738.37392.894338.996420@turnbull.sk.tsukuba.ac.jp> Richard Damon writes: > One big thing that I haven't seen in the discussion of this problem is > exactly WHAT issue/problem this feature is intended to solve, There are > several different problems that encryption can help with, each needing > different sort of support from the software. Yup, and I've been telling the prospective interns that throughout. But Rich Kulewiec is right that many or most of them can be eliminated right off. For example, I don't see any point in actual end-to-end encryption, as that would require everybody to know everybody's keys. OK, so we could create a PKI for each list, but that's effort over and above the encryption module, probably not appropriate for this GSoC. (It's been mentioned that algorithms are not forever, but similarly I think that's out of scope.) AFAICS, this means that root on the Mailman host is trusted, and needs to know the session key for each message. Perhaps you can avoid having to trust list owners, but when does that scenario actually make sense? Note that GSoC is Google Summer of CODE - the reason for being cagey about what I'm thinking about as specs and use cases is not that the intern will be responsible for design. It's that the intern needs to understand that design and the use cases it serves in order to determine whether the implementation is correct, write tests, and so on, and I prefer to mentor Socratically. That doesn't mean anybody else needs to be coy! Feel free to put your ideas about use cases out there. Also references to existing knowledge would be appreciated, such as "zero knowledge" schemes that might allow untrusted root on Mailman host, and the various implementations like SELS that have been mentioned. Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Wed Mar 22 11:06:23 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 23 Mar 2017 00:06:23 +0900 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170321223017.GA4571@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> <20170321223017.GA4571@gsp.org> Message-ID: <22738.37615.934880.875272@turnbull.sk.tsukuba.ac.jp> Rich Kulawiec writes: > (In the specific case, e.g., the right people using the right > devices with the right knowledge and self-discipline: maybe. But > there are not many of those cases and any of them can revert to the > general case in seconds with one poor decision or perhaps even > without one.) I'm with Richard Damon on this. FYI: Encrypted lists *are* occasionally requested. Even if we are forced to give up, we need to investigate this, and convince ourselves that there really are NO valid use cases so we can make the case that it's a bad idea to those users. I note that several other projects have created variations on encrypted lists. It's reasonable for us to want to learn what they are and are not good for in order to converse with users about their requests for encrypted lists. You have my permission to say "I told you so" if we're forced to abandon this as a silly idea. Until then, I think you're wasting bandwidth in opposing it from the get-go. Once again, I'd be happy to hear where our threat models are deficient once we start to talk about them. But none of the proposals so far have really identified a threat model let alone a corresponding use case! So there's nothing to criticize yet. Regards, Steve From johny at neuromancer.sk Wed Mar 22 20:36:49 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Thu, 23 Mar 2017 01:36:49 +0100 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170321221620.GA3537@gsp.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170318200409.GA9534@gsp.org> <711c0682-9d77-8169-e613-bcb563e029f8@neuromancer.sk> <20170321221620.GA3537@gsp.org> Message-ID: On 03/21/2017 11:16 PM, Rich Kulawiec wrote: > On Tue, Mar 21, 2017 at 04:04:20PM +0100, johny wrote: >> Shifting the attacker to actively compromise devices is an overall >> improvement. > > If "compromising devices" was difficult, I might agree. But it's not. > Devices of all descriptions have been and are being compromised in > enormous numbers on an ongoing basis even by relatively unskilled > attackers -- since they can buy/lease the requisite tools and infrastructure > and use them without needing to understand how they work. > > I think you (and others) are continuing to badly underestimate the > scale at which this is taking place. We're not talking about an > ecosystem in which 2% or 6% of devices are compromised; we're talking > about one in which any estimate under 25% should be laughed out of > the room and an estimate of 50% should be given serious consideration. > (I think the latter may be still be too high. But it's certainly > within the realm of plausibility.) We're also talking about an > ecosystem in which, increasingly, vendors are shipping devices that > are essentially pre-compromised at the factory due to very poor > and entirely self-serving design and implementation decisions. Before introducing encryption during transport an attacker had a choice to either sniff traffic or compromise a device. Plugging one hole, encrypting the transport, *MUST* be an overall security improvement, it will not be only in some edge circumstances. I think that the percentages you are presenting are not the right ones for what you could expect from users of a GPG encrypted mailing list. a) The fact that they are even considering to subscribe to an encrypted mailing list, that they have a GPG keypair moves the probability that they are doing this on a compromised device way lower. (at least I think so, I don't have any concrete data) b) You are assuming that the percentages you give are actually all one homogenic attacker, aimed at gaining access to an encrypted mailing list, having the capabilities to do so and having control of X% of devices. No such one attacker actually exists. >> There are plenty of threat actors for which sniffing traffic to a >> plaintext mailing list might be easy, however overcoming a well setup >> encrypted mailing list system would be quite hard. > > I don't think so, if the mailing list is of sufficient size. (Certainly > one with only a handful of people might be hard to crack, but that > would be fairly hard today even without encryption.) > >> The system security only increases in this case. It's security is >> obviously debatable against state actors/equivalent threats, there it >> might not improve much, but improves significantly against other threats. > > State actors are not necessarily the biggest threat. They might > have the most resources, and they might have the best cryptographers, > and they certainly have the most political power (e.g., NSLs in the US), > but they also have their own areas of focus and this may not be one of them. > > If there's money to be made by trafficking in information, then others > will take an interest. They may not have the resources, cryptographers, > power, etc., but they do have some very talented personnel, stockpiled > exploits, and rather a lot of experience with mass compromise of end > user systems. These are not script kiddies in mom's basement, these > are professionals with the discipline to identify and attack targets > successfully on a large scale. Don't underestimate them. *That* > particular mistake was already made by every ISP on this planet ~15 > years ago, which is one of the major reasons the problem has the > scope that it has today. I still think that there is a security improvement that outweights the drawbacks. Of which two were noted: a) Mailman's encrypted mailing list security will be over-estimated by naive users that will get bitten by this. This is valid for cases where the users aren't already communicating in an insecure way (which is likely), so only covers a small userbase and can be tried to be mitigated by proper warnings, documentation, common sense. (As many E2E and similar apps do, or should do, warn that endpoint security is important and that endpoint compromise is catastrophic) b) Added complexity, maintenance cost to Mailman's infrastructure. This can be mitigated by implementing encrypted mailing lists either as a plugin as was proposed here before, or just by making the feature as non-intrusive into Mailman's infrastructure as possible . (which I think is doable if the proposal aims to do so) -Jan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From johny at neuromancer.sk Wed Mar 22 20:38:18 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Thu, 23 Mar 2017 01:38:18 +0100 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <22738.37615.934880.875272@turnbull.sk.tsukuba.ac.jp> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> <20170321223017.GA4571@gsp.org> <22738.37615.934880.875272@turnbull.sk.tsukuba.ac.jp> Message-ID: <28adc328-8903-61dc-854a-b70538d5f04d@neuromancer.sk> On 03/22/2017 04:06 PM, Stephen J. Turnbull wrote: > Rich Kulawiec writes: > > > (In the specific case, e.g., the right people using the right > > devices with the right knowledge and self-discipline: maybe. But > > there are not many of those cases and any of them can revert to the > > general case in seconds with one poor decision or perhaps even > > without one.) > > I'm with Richard Damon on this. > > FYI: Encrypted lists *are* occasionally requested. Even if we are > forced to give up, we need to investigate this, and convince ourselves > that there really are NO valid use cases so we can make the case that > it's a bad idea to those users. I note that several other projects > have created variations on encrypted lists. It's reasonable for us to > want to learn what they are and are not good for in order to converse > with users about their requests for encrypted lists. > > You have my permission to say "I told you so" if we're forced to > abandon this as a silly idea. Until then, I think you're wasting > bandwidth in opposing it from the get-go. Once again, I'd be happy to > hear where our threat models are deficient once we start to talk about > them. But none of the proposals so far have really identified a > threat model let alone a corresponding use case! So there's nothing > to criticize yet. A use case I have in mind is for mailing lists that: a) have a relatively low number of subscribers. b) have or can establish some sort of a PGP web-of-trust, in a sense that a subscriber has to trust the list owner's key or the list key, and that the list owner has to trust the subscriber's key when accepting his subscription. (this is due to fairly strong assumptions about attacker's abilities) c) can be anonymous (apart from obvious information stemming from b) and no other info about subscriber's or sender's identity has to be disclosed to other subscribers than what is now with anonymous lists. (see Technical details in my proposal) This is what my proposal is aiming for and what I think is a realistic application of encrypted mailing lists. -Jan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From johny at neuromancer.sk Wed Mar 22 20:38:39 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Thu, 23 Mar 2017 01:38:39 +0100 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <22738.37392.894338.996420@turnbull.sk.tsukuba.ac.jp> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> <20170321223017.GA4571@gsp.org> <0413b78b-31b4-7f74-b8c4-8aafb4fb0937@Damon-Family.org> <22738.37392.894338.996420@turnbull.sk.tsukuba.ac.jp> Message-ID: <02b29eec-007e-b3dc-d8ef-d92c51ea61dc@neuromancer.sk> On 03/22/2017 04:02 PM, Stephen J. Turnbull wrote: > Also references to existing knowledge would be appreciated, such as > "zero knowledge" schemes that might allow untrusted root on Mailman > host, and the various implementations like SELS that have been > mentioned. In my proposal [1 or 2], I concluded that a SELS like proxy encryption scheme doesn't apply well to Mailman's existing infrastructure as well as the stated requirements in the project idea. -Jan (Just to note, I have recently updated my proposal, make sure to have the latest version) [1]: https://neuromancer.sk/page/gsoc/mailman#technical-details [2]: https://neuromancer.sk/static/mailman.pdf -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From barry at list.org Wed Mar 22 21:15:46 2017 From: barry at list.org (Barry Warsaw) Date: Wed, 22 Mar 2017 21:15:46 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <22736.65387.760048.810453@turnbull.sk.tsukuba.ac.jp> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <22736.65387.760048.810453@turnbull.sk.tsukuba.ac.jp> Message-ID: <20170322211546.7d6fc308@subdivisions.wooz.org> On Mar 21, 2017, at 07:27 PM, Stephen J. Turnbull wrote: >Not if the target membership isn't already paranoid. Remember, >20%-40% of devices are already compromised. Even at the low end, >assuming uniform draws, with *three* members odds are *even* that one >is compromised. Is anybody even aware of any mainstream mobile email readers that support encryption? Or webmail interfaces? I seem to remember a recent announcement that Gmail will soon be supported plugins that could be used to read and send GPG/PGP encrypted messages. An encrypted mailing list won't help you much regardless of the compromised nature of your device if you can't even read the encrypted messages. ;) -Barry From barry at list.org Wed Mar 22 21:27:11 2017 From: barry at list.org (Barry Warsaw) Date: Wed, 22 Mar 2017 21:27:11 -0400 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <22738.37615.934880.875272@turnbull.sk.tsukuba.ac.jp> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> <20170321223017.GA4571@gsp.org> <22738.37615.934880.875272@turnbull.sk.tsukuba.ac.jp> Message-ID: <20170322212711.57af16f6@subdivisions.wooz.org> On Mar 23, 2017, at 12:06 AM, Stephen J. Turnbull wrote: >FYI: Encrypted lists *are* occasionally requested. Another possible use case would be attempting to prevent the wholesale compromise of email storage. Meaning, if you keep your email on some external server, and that server is compromised, if those messages are encrypted, then at least they likely will be very difficult for the attacker to decrypt since the keys won't likely be colocated with the emails. Sure you can probably phish specific individuals, but it won't be "crack the server and now you have a million secret messages". It's the same as with encrypted person-to-person messages (which almost no one uses because Reasons). >You have my permission to say "I told you so" if we're forced to >abandon this as a silly idea. Until then, I think you're wasting >bandwidth in opposing it from the get-go. Once again, I'd be happy to >hear where our threat models are deficient once we start to talk about >them. But none of the proposals so far have really identified a >threat model let alone a corresponding use case! So there's nothing >to criticize yet. I should state for the record that my personal interest in this feature isn't so much encrypted mailing lists per se, but the architectural and design pressure it will put on Mailman 3, and our responses to that. Encrypted lists are the kinds of things I want to make possible with Mailman 3, so the APIs, hooks, configurations, and plugins that would be needed to implement encrypted lists (assuming, IMHO correctly that they won't be integrated into the core) will be of use to others who want to do Interesting Things with mailing lists. Cheers, -Barry From nb at bollow.ch Thu Mar 23 04:16:16 2017 From: nb at bollow.ch (Norbert Bollow) Date: Thu, 23 Mar 2017 09:16:16 +0100 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: <20170322211546.7d6fc308@subdivisions.wooz.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <22736.65387.760048.810453@turnbull.sk.tsukuba.ac.jp> <20170322211546.7d6fc308@subdivisions.wooz.org> Message-ID: <20170323091616.7b40862d@quill> On Wed, 22 Mar 2017 21:15:46 -0400 Barry Warsaw wrote: > Is anybody even aware of any mainstream mobile email readers that > support encryption? One of my friends uses K9 on his Samsung mobile phone; it works fine for him, allowing him to exchange GPG-encrypted emails with me. If you need more examples, I could ask on the (encrypted, Schleuder-based) internal mailing list of Digital Society Switzerland. > An encrypted mailing list won't help you much regardless of the > compromised nature of your device if you can't even read the > encrypted messages. ;) Personally I prefer to use my mobile phone for plain old telephony and SMS only; I lovingly refer to it as "my dumbphone". I believe that the risk of compromise is much, much lower than for Internet-connected smartphones, even without taking any special security precautions in relation to the dumbphone. By contrast, a laptop running Debian GNU/Linux can be kept secure enough for my needs without too much trouble even while communicating via the Internet with somewhat-trusted as well as with totally untrusted third parties. That is good enough for me, because I don't have a need to be online continually while awake; it suffices for me to be able to ensure that I'll be able to live up to a commitment of having promised 24hrs turn-around reaction time for anything that may be relatively urgent. A laptop computer is sufficiently mobile for ensuring that, as long as I don't travel to really remote areas. Greetings, Norbert From hasan110.mr at gmail.com Thu Mar 23 12:47:13 2017 From: hasan110.mr at gmail.com (Muhammad Rizvi) Date: Thu, 23 Mar 2017 09:47:13 -0700 Subject: [Mailman-Developers] GSoC Help Wanted In-Reply-To: References: Message-ID: Hello! I want to let you that I want to take part in Python Organization but don?t exactly know what to do. The thing that I came to know so far is that, first I have to find a mentor but don?t know from where I can contact a mentor, secondly I have to submit a proposal but what proposal should I give? Please, help me out as I am much curious to take part in GSoC and I don?t want to lose this opportunity. Best, Muhammad Hasan From hasan110.mr at gmail.com Fri Mar 24 05:55:09 2017 From: hasan110.mr at gmail.com (Muhammad Rizvi) Date: Fri, 24 Mar 2017 02:55:09 -0700 Subject: [Mailman-Developers] GSoC Help Wanted In-Reply-To: References: Message-ID: Hello! I want to let you that I want to take part in Python Organization but don?t exactly know what to do. The thing that I came to know so far is that, first I have to find a mentor but don?t know from where I can contact a mentor, secondly I have to submit a proposal but what proposal should I give? Please, help me out as I am much curious to take part in GSoC and I don?t want to lose this opportunity. Best, Muhammad Hasan From terri at toybox.ca Mon Mar 27 13:47:23 2017 From: terri at toybox.ca (Terri Oda) Date: Mon, 27 Mar 2017 10:47:23 -0700 Subject: [Mailman-Developers] GSoC Help Wanted In-Reply-To: References: Message-ID: <1a365158-e1d6-b055-a6f0-caf17d184206@toybox.ca> On 2017-03-24 2:55 AM, Muhammad Rizvi wrote: > I want to let you that I want to take part in Python Organization but don?t > exactly know what to do. The thing that I came to know so far is that, > first I have to find a mentor but don?t know from where I can contact a > mentor, secondly I have to submit a proposal but what proposal should I > give? You've actually done it! Posting to the public mailing list for a Python sub-org is exactly how you contact a mentor. This year, Mailman's only running one project, the encrypted lists one, and the primary person for that is Stephen, but in general we prefer if students ask questions in public so that the community and other students get the benefits of answers. :) Stephen's given some pretty precise instructions on what he wants in a proposal on the mailman ideas page, and as I said we're only planning one project this year so it needs to be related to encrypted lists unless you can get someone excited enough about another idea that you have that they're willing to offer to be a primary mentor for that. Anything other than encrypted lists will be a hard sell this year, though, since most of us need the time off for regular mailman development or other projects! Terri From terri at toybox.ca Mon Mar 27 14:04:38 2017 From: terri at toybox.ca (Terri Oda) Date: Mon, 27 Mar 2017 11:04:38 -0700 Subject: [Mailman-Developers] Encrypted lists are still a valid GSoC project, in case you were wondering. In-Reply-To: <20170322212711.57af16f6@subdivisions.wooz.org> References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> <20170321223017.GA4571@gsp.org> <22738.37615.934880.875272@turnbull.sk.tsukuba.ac.jp> <20170322212711.57af16f6@subdivisions.wooz.org> Message-ID: On 2017-03-22 6:27 PM, Barry Warsaw wrote: > I should state for the record that my personal interest in this feature isn't > so much encrypted mailing lists per se, but the architectural and design > pressure it will put on Mailman 3, and our responses to that. Encrypted lists > are the kinds of things I want to make possible with Mailman 3, so the APIs, > hooks, configurations, and plugins that would be needed to implement encrypted > lists (assuming, IMHO correctly that they won't be integrated into the core) > will be of use to others who want to do Interesting Things with mailing lists. I just want to pull this out and make sure students have seen it, because I know a lot of folk will see a discussion like the one we've had on the challenges and assume that "this is hard to do" means "this project is a waste of time and I should find another org to work with." And that's not true at all! As Barry says, this is an interesting project for Mailman for many reasons that have nothing to do with encryption and everything to do with how to build a moderately complex system that hooks into Mailman. Those reasons are still valid regardless of how you feel about encryption. :) More than that, GSoC's goal is generally *not* that students produce perfect workable code (you can sort of tell this by the fact that the code doesn't even have to be used by the organization providing mentors!) but rather to get students experience working in open source communities, learning new architectures, and working on real-world problems. Again, even if everyone everywhere is compromised, this is an interesting enough problem that meets all those other needs. When Stephen floated this idea, I thought it was great because on top of learning about mailman, it gives students a chance to work in a challenging security problem as well. Getting encryption even partially right is a thing that developers struggle with (my day job involves helping open source dev teams understand security) and a little experience tends to go a long way when it comes to future understanding of threat models and other key concepts in defining security. It's also worth noting that one of the reasons this was chosen was also that this *isnt* an urgently-needed release-blocking feature for Mailman, but rather a "nice to have" that someone can work on without quite as much pressure. Again, this is great for a student project in ways that it might not be ideal for a core developer. Basically, don't just read "Why Johnny Can't Encrypt" [1] and assume the problem of encrypted is dead and never will be solved. As my PhD supervisor used to say "you should look at impossible, insolvable problems as research opportunities rather than dead ends. That's science." :) [1] https://www.usenix.org/conference/8th-usenix-security-symposium/why-johnny-cant-encrypt-usability-evaluation-pgp-50 From turnbull.stephen.fw at u.tsukuba.ac.jp Mon Mar 27 15:48:04 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 28 Mar 2017 04:48:04 +0900 Subject: [Mailman-Developers] Encrypted lists are still a valid GSoC project, in case you were wondering. In-Reply-To: References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170316173036.0541e65b@subdivisions.wooz.org> <20170318203740.GB9534@gsp.org> <8e6ecc98-b312-e4fb-02d0-07901fc14fab@Damon-Family.org> <20170321223017.GA4571@gsp.org> <22738.37615.934880.875272@turnbull.sk.tsukuba.ac.jp> <20170322212711.57af16f6@subdivisions.wooz.org> Message-ID: <22745.27764.143936.65231@turnbull.sk.tsukuba.ac.jp> Terri Oda writes: > Basically, don't just read "Why Johnny Can't Encrypt" [1] and assume the > problem of encrypted is dead and never will be solved. But you might want also to read JWZ's blog on Signal[2] *and all the comments* to see why threat models matter, and how subtle it can be. (If you're not going to read a large fraction of the comments, don't bother, nothing to see here.) It's the disagreement among smart, well-intentioned -- if a bit mouthy in JWZ's case ;-) -- people that's of interest here. AFAICT, in the whole thread there are no two individuals who agree on what threat model this particular encrypted messaging system should try to address! [1] https://www.usenix.org/conference/8th-usenix-security-symposium/why-johnny-cant-encrypt-usability-evaluation-pgp-50 [2] https://www.jwz.org/blog/2017/03/signal-leaks-your-phone-number-to-everyone-in-your-contacts/ From turnbull.stephen.fw at u.tsukuba.ac.jp Mon Mar 27 15:54:44 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 28 Mar 2017 04:54:44 +0900 Subject: [Mailman-Developers] GSoC Help Wanted In-Reply-To: References: Message-ID: <22745.28164.606872.217843@turnbull.sk.tsukuba.ac.jp> Muhammad Rizvi writes: > I want to let you that I want to take part in Python Organization but don?t > exactly know what to do. The thing that I came to know so far is that, > first I have to find a mentor but don?t know from where I can contact a > mentor, secondly I have to submit a proposal but what proposal should I > give? In a little more detail than what Terri wrote: First, get signed up on the Google site at https://summerofcode.withgoogle.com/. Become familiar with the guide for students, there is a lot of help there. Sign up for the students' mailing list, and read it every day to see what kinds of things people talk about. Some kinds of help will be much more available there (more on that below). Next, go to the PSF GSoC page at http://python-gsoc.github.io. As the PSF is an "umbrella" organization, it does not actually provide any mentoring itself, it handles financial matters and Google relations for over a dozen suborganizations. Browse the suborganizations, and find one or more whose project is interesting. Then check the individual "Ideas" pages for each project for lists of tasks that may interest you, and write one or more proposals to implement the ideas. (I believe that GSoC allows you to submit up to five applications, spread across 1 to 5 organizations. In terms of return for effort, I recommend two or three.) You should submit something as early as possible. You will be allowed to revise and improve your proposal up until the submission deadline. As a new student to GSoC, you should focus on organizations that give you feedback promptly (a day or two). Some put more effort into that than others, so you will need to balance the intrisic interest in the project against the help you get improving your proposals. Sign up for the developers' lists for the projects you are interested in. *Browse their archives for the last month or two* to see how the members tend to interact with each other. Normally you should lurk for at least that long before speaking up, but you're up against the submission deadline so reading the recent threads is a reasonable substitute. Some organizations have specific lists for GSoC project discussion, but most do it on their developer lists. When you have made a complete first draft, and after major revisions, you may request review where the mentors are likely to see it. Expect it to take at least a couple of days, the best mentors are generally busy people. Of course if it takes longer than that, you won't get very many chances to revise. So you'll have to find an appropriate balance for yourself between prompt responses and more detailed reviews and more interesting projects. Be very careful about email protocol. Some organizations strongly prefer that discussions of projects be kept on the mailing lists, so that all applicants can benefit, and so that other mentors and developers on the project can participate in the conversation. In many cases, private mail to mentors is frowned on. Figure out how the project you're interested in handles communications before going private. Be careful to check that the mailing list is among the addressees of mail that you send. You may need to use "reply to all", depending on the list configuration. PSF organizations generally require that student applicants submit contributions to the organization during the application period, so you should also find those organizations' issue trackers and look for "easy" tasks you can work on right away. (The point of the requirement is more to ensure that you don't waste time during the project period becoming familiar with infrastructure like repositories and trackers, not very much to prove your coding ability.) Be aware that you're already late to the party. Almost all organizations prefer to award internships to students who have already been productive in their projects for months (and for some applicants, years). Remember that mentors are unpaid volunteers; the attraction to mentors is working with motivated and productive junior developers who make major contributions to their projects. We appreciate your interest and that you are inexperienced with GSoC, but realistically the more you do for yourself and the less handholding you need, the better your chances are. It's worth the effort on your part. Many of the students you are competing with will treat the GSoC application as a fulltime job. As much as possible, investigate the process yourself, or on the students' mailing list. When asking questions of prospective mentors, do your homework and make them as clear and specific as possible. Again, this is a question of balance. If it's something you can do for yourself, an hour of your time may be a worthwhile trade to save 5 minutes for a prospective mentor. If it's something only the mentor can answer, you have to ask, of course. And don't worry about screwing up by asking; getting the right balance is something you only learn with experience. Just keep your antenna out and learn what excites the mentors and what drags them down as you go along. Good luck! Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Mon Mar 27 15:57:09 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 28 Mar 2017 04:57:09 +0900 Subject: [Mailman-Developers] Encrypted lists predictable difficulties and implementation needs In-Reply-To: References: <78383d49-08a8-320a-27d7-cc573cbd28da@forestfield.org> <20170316014746.GA19047@gsp.org> <20170318200409.GA9534@gsp.org> <711c0682-9d77-8169-e613-bcb563e029f8@neuromancer.sk> <20170321221620.GA3537@gsp.org> Message-ID: <22745.28309.906905.402165@turnbull.sk.tsukuba.ac.jp> Jan Jancar writes: > b) Added complexity, maintenance cost to Mailman's infrastructure. > This can be mitigated by implementing encrypted mailing lists > either as a plugin as was proposed here before, In one sense, a plugin is the ONLY way this feature can be reasonably implemented in Mailman 3, as all the relevant work will be done in Rules and Handlers. However, as Mailman core tends to provide information to companion applications (Postorius, HyperKitty) without much concern for authorization, there may be issues that require either invasive changes or can only be addressed in current Mailman 3 architecture by host security. The latter is the current answer to all questions of security, in fact. This will indeed be an ongoing security concern in maintenance, unless the information that needs to be secured is by design carefully partitioned away from "low security" operations used by Postorius, HyperKitty, and the REST interface generally. Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Wed Mar 29 01:07:06 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Wed, 29 Mar 2017 14:07:06 +0900 Subject: [Mailman-Developers] How to set up mailman 3 In-Reply-To: <2a2a36d2-6498-19b0-20fc-1b60c0710888@msapiro.net> References: <20170325173552.1f2eb935.barry@wooz.org> <0faa8e3b-db75-43ef-4247-49a20abab2b6@msapiro.net> <2a2a36d2-6498-19b0-20fc-1b60c0710888@msapiro.net> Message-ID: <22747.16634.138811.682766@turnbull.sk.tsukuba.ac.jp> On [MM3-users] Mark Sapiro writes: > On 03/25/2017 04:55 PM, Mark Sapiro wrote: > > > > A list's message_size can't currently be set in Postorius. File an issue > > at . > > Actually the attribute name is max_message_size, not message_size. > > It exists in core and can be set, e.g., from the command line via > 'mailman shell', but it needs to be exposed in Postorius. Would it be possible to automate this? I haven't thought carefully about it, specifically security issues. However, there could (in theory) be an automated export of "all parameters" in "expert mode" or "site/list initialization mode". Steve From barry at list.org Wed Mar 29 18:16:11 2017 From: barry at list.org (Barry Warsaw) Date: Wed, 29 Mar 2017 18:16:11 -0400 Subject: [Mailman-Developers] How to set up mailman 3 In-Reply-To: <22747.16634.138811.682766@turnbull.sk.tsukuba.ac.jp> References: <20170325173552.1f2eb935.barry@wooz.org> <0faa8e3b-db75-43ef-4247-49a20abab2b6@msapiro.net> <2a2a36d2-6498-19b0-20fc-1b60c0710888@msapiro.net> <22747.16634.138811.682766@turnbull.sk.tsukuba.ac.jp> Message-ID: <20170329181611.13c7a423@subdivisions.wooz.org> On Mar 29, 2017, at 02:07 PM, Stephen J. Turnbull wrote: >Would it be possible to automate this? I haven't thought carefully >about it, specifically security issues. However, there could (in >theory) be an automated export of "all parameters" in "expert mode" or >"site/list initialization mode". It's probably near impossible to automate, because the Core's REST API isn't programmatically discoverable. As a counter-example, Launchpad's REST API provides a WADL[1] description extracted from the code, but this makes it way too heavyweight for my tastes. And because the Core uses a dynamic object-based traversal system (inherited from restish and ported to falcon) it would make any kind of static REST API description pretty difficult to generate. Cheers, -Barry [1] https://en.wikipedia.org/wiki/Web_Application_Description_Language From fsantiago at garbage-juice.com Wed Mar 29 18:22:09 2017 From: fsantiago at garbage-juice.com (Fabian A. Santiago) Date: Wed, 29 Mar 2017 22:22:09 +0000 Subject: [Mailman-Developers] Mailman 3 deployment / packages available Message-ID: <3BA69DBC-5894-4648-8FA6-F14D414A4036@garbage-juice.com> Here to ask again, I haven't really been following the progress on mm3; is there such a thing yet as a yum repo with packages to deploy mailman 3 on a readily configurable, stable, production ready system? Deploying old mailman is pretty easy by comparison, in my mind, but I'm no pro either; just a hobbyist. -- Thanks. Fabian S. From turnbull.stephen.fw at u.tsukuba.ac.jp Wed Mar 29 23:57:22 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 30 Mar 2017 12:57:22 +0900 Subject: [Mailman-Developers] How to set up mailman 3 In-Reply-To: <20170329181611.13c7a423@subdivisions.wooz.org> References: <20170325173552.1f2eb935.barry@wooz.org> <0faa8e3b-db75-43ef-4247-49a20abab2b6@msapiro.net> <2a2a36d2-6498-19b0-20fc-1b60c0710888@msapiro.net> <22747.16634.138811.682766@turnbull.sk.tsukuba.ac.jp> <20170329181611.13c7a423@subdivisions.wooz.org> Message-ID: <22748.33314.134366.157843@turnbull.sk.tsukuba.ac.jp> Barry Warsaw writes: > On Mar 29, 2017, at 02:07 PM, Stephen J. Turnbull wrote: > > >Would it be possible to automate this? I haven't thought carefully > >about it, specifically security issues. However, there could (in > >theory) be an automated export of "all parameters" in "expert mode" or > >"site/list initialization mode". > > It's probably near impossible to automate, because the Core's REST > API isn't programmatically discoverable. Right (I did know that). I was thinking "decorator". > As a counter-example, Launchpqad's REST API provides a WADL[1] description > extracted from the code, but this makes it way too heavyweight for my tastes. You have good taste, I'll grant that. ;-) > And because the Core uses a dynamic object-based traversal system > (inherited from restish and ported to falcon) it would make any > kind of static REST API description pretty difficult to generate. OK. Would you recommend against even exploring for a lightweight approach for some reason? Ie, the idea is bad according to some principle? From barry at list.org Thu Mar 30 09:37:08 2017 From: barry at list.org (Barry Warsaw) Date: Thu, 30 Mar 2017 09:37:08 -0400 Subject: [Mailman-Developers] How to set up mailman 3 In-Reply-To: <22748.33314.134366.157843@turnbull.sk.tsukuba.ac.jp> References: <20170325173552.1f2eb935.barry@wooz.org> <0faa8e3b-db75-43ef-4247-49a20abab2b6@msapiro.net> <2a2a36d2-6498-19b0-20fc-1b60c0710888@msapiro.net> <22747.16634.138811.682766@turnbull.sk.tsukuba.ac.jp> <20170329181611.13c7a423@subdivisions.wooz.org> <22748.33314.134366.157843@turnbull.sk.tsukuba.ac.jp> Message-ID: <20170330093708.0b40f9bd@subdivisions.wooz.org> On Mar 30, 2017, at 12:57 PM, Stephen J. Turnbull wrote: >Would you recommend against even exploring for a lightweight approach >for some reason? Ie, the idea is bad according to some principle? To the contrary, I think if it were possible it would be pretty nice. In fact, you could conceivably eliminate mailman-client if you could do it, and other languages could auto-generate bindings to the REST API. I haven't kept up on the state of the art, but IIRC, there were Python libraries that could generate bindings based on WADL. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Mar 30 14:01:07 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 31 Mar 2017 03:01:07 +0900 Subject: [Mailman-Developers] How to set up mailman 3 In-Reply-To: <20170330093708.0b40f9bd@subdivisions.wooz.org> References: <20170325173552.1f2eb935.barry@wooz.org> <0faa8e3b-db75-43ef-4247-49a20abab2b6@msapiro.net> <2a2a36d2-6498-19b0-20fc-1b60c0710888@msapiro.net> <22747.16634.138811.682766@turnbull.sk.tsukuba.ac.jp> <20170329181611.13c7a423@subdivisions.wooz.org> <22748.33314.134366.157843@turnbull.sk.tsukuba.ac.jp> <20170330093708.0b40f9bd@subdivisions.wooz.org> Message-ID: <22749.18403.505849.557796@turnbull.sk.tsukuba.ac.jp> Barry Warsaw writes: > up on the state of the art, but IIRC, there were Python libraries that could > generate bindings based on WADL. That sounds cool, but I tend to feel that WADL gets pretty heavy. If we can't just mark things with something like "@exported_REST", it's not clear that the maintenance burden is worth it. Steve From johny at neuromancer.sk Thu Mar 30 15:45:05 2017 From: johny at neuromancer.sk (Jan Jancar) Date: Thu, 30 Mar 2017 21:45:05 +0200 Subject: [Mailman-Developers] How to set up mailman 3 In-Reply-To: <22749.18403.505849.557796@turnbull.sk.tsukuba.ac.jp> References: <20170325173552.1f2eb935.barry@wooz.org> <0faa8e3b-db75-43ef-4247-49a20abab2b6@msapiro.net> <2a2a36d2-6498-19b0-20fc-1b60c0710888@msapiro.net> <22747.16634.138811.682766@turnbull.sk.tsukuba.ac.jp> <20170329181611.13c7a423@subdivisions.wooz.org> <22748.33314.134366.157843@turnbull.sk.tsukuba.ac.jp> <20170330093708.0b40f9bd@subdivisions.wooz.org> <22749.18403.505849.557796@turnbull.sk.tsukuba.ac.jp> Message-ID: <7fe9a3bb-8b6a-a845-f2fa-df5986d3a6a6@neuromancer.sk> On 03/30/2017 08:01 PM, Stephen J. Turnbull wrote: > Barry Warsaw writes: > > > up on the state of the art, but IIRC, there were Python libraries that could > > generate bindings based on WADL. > > That sounds cool, but I tend to feel that WADL gets pretty heavy. If > we can't just mark things with something like "@exported_REST", it's > not clear that the maintenance burden is worth it. This would also go well with an idea I had about the current state of the REST API and encrypted lists. With having decorators like "@exported_REST", another one could get introduced, something like "@requires_permission("some.perm.name")" which would introduce permission-based granularity to the REST API. Then multiple user:password pairs could be specified in a config with different permissions and so Mailman could provide different levels of API access to different apps. Although, reading Barry's message again, this may not be possible / maintainable. -Jan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From barry at list.org Thu Mar 30 16:13:06 2017 From: barry at list.org (Barry Warsaw) Date: Thu, 30 Mar 2017 16:13:06 -0400 Subject: [Mailman-Developers] How to set up mailman 3 In-Reply-To: <22749.18403.505849.557796@turnbull.sk.tsukuba.ac.jp> References: <20170325173552.1f2eb935.barry@wooz.org> <0faa8e3b-db75-43ef-4247-49a20abab2b6@msapiro.net> <2a2a36d2-6498-19b0-20fc-1b60c0710888@msapiro.net> <22747.16634.138811.682766@turnbull.sk.tsukuba.ac.jp> <20170329181611.13c7a423@subdivisions.wooz.org> <22748.33314.134366.157843@turnbull.sk.tsukuba.ac.jp> <20170330093708.0b40f9bd@subdivisions.wooz.org> <22749.18403.505849.557796@turnbull.sk.tsukuba.ac.jp> Message-ID: <20170330161306.6e2d8339@subdivisions.wooz.org> On Mar 31, 2017, at 03:01 AM, Stephen J. Turnbull wrote: >That sounds cool, but I tend to feel that WADL gets pretty heavy. If >we can't just mark things with something like "@exported_REST", it's >not clear that the maintenance burden is worth it. Agreed, but note this is only one half of the equation. You also need to publish the static definition of the REST API so that HTTP clients could discover it. That's my understanding of the role that WADL serves. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From barry at list.org Thu Mar 30 16:15:09 2017 From: barry at list.org (Barry Warsaw) Date: Thu, 30 Mar 2017 16:15:09 -0400 Subject: [Mailman-Developers] How to set up mailman 3 In-Reply-To: <7fe9a3bb-8b6a-a845-f2fa-df5986d3a6a6@neuromancer.sk> References: <20170325173552.1f2eb935.barry@wooz.org> <0faa8e3b-db75-43ef-4247-49a20abab2b6@msapiro.net> <2a2a36d2-6498-19b0-20fc-1b60c0710888@msapiro.net> <22747.16634.138811.682766@turnbull.sk.tsukuba.ac.jp> <20170329181611.13c7a423@subdivisions.wooz.org> <22748.33314.134366.157843@turnbull.sk.tsukuba.ac.jp> <20170330093708.0b40f9bd@subdivisions.wooz.org> <22749.18403.505849.557796@turnbull.sk.tsukuba.ac.jp> <7fe9a3bb-8b6a-a845-f2fa-df5986d3a6a6@neuromancer.sk> Message-ID: <20170330161509.1d6b8b23@subdivisions.wooz.org> On Mar 30, 2017, at 09:45 PM, Jan Jancar wrote: >This would also go well with an idea I had about the current state of >the REST API and encrypted lists. With having decorators like >"@exported_REST", another one could get introduced, something like >"@requires_permission("some.perm.name")" which would introduce >permission-based granularity to the REST API. Then multiple >user:password pairs could be specified in a config with different >permissions and so Mailman could provide different levels of API access >to different apps. Our intention is to support permission based access to the REST API via an "authenticating proxy", which we call lemme: https://gitlab.com/mailman/lemme/tree/master and for an outline on how this might work: https://gitlab.com/mailman/lemme/blob/master/OUTLINE.rst We had good discussions about this at Pycon 2016, but haven't gotten very far in implementation details. I'm hoping we can spend a little bit of time on that this year. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: