From ashesh.vidyut at gmail.com Mon Apr 1 13:03:13 2013 From: ashesh.vidyut at gmail.com (Ashesh Vidyut) Date: Mon, 1 Apr 2013 07:03:13 -0400 Subject: [Mailman-Developers] web posting interface gsoc project Message-ID: i really want to get selected in gsoc 2013 http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013 i wanna choose the first project that is mentioned above ad "Web posting interface" its basically like phpbb in italic its mentioned to add more features to this project for selection kindly propose the features that i can add in this thank u Ashesh Vidyut From ashesh.vidyut at gmail.com Mon Apr 1 18:15:00 2013 From: ashesh.vidyut at gmail.com (Ashesh Vidyut) Date: Mon, 1 Apr 2013 21:45:00 +0530 Subject: [Mailman-Developers] error while setting up mailman on ubuntu please reply Message-ID: i m following http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+running i had brached mailman after that i excecuted python bootstrap.py in bin/buildout i get An internal error occured due to a bug in either zc.buildout or in a recipe being used: Traceback (most recent call last): File "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", line 1923, in main getattr(buildout, command)(args) File "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", line 466, in install installed_develop_eggs = self._develop() File "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", line 707, in _develop zc.buildout.easy_install.develop(setup, dest) File "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", line 871, in develop call_subprocess(args) File "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", line 129, in call_subprocess % repr(args)[1:-1]) Exception: Failed to run command: '/usr/bin/python', '/tmp/tmpu69D_h', '-q', 'develop', '-mxN', '-d', '/home/ashesh/mailmanc/develop-eggs/tmpNyd439build' root at av:/home/ashesh/mailmanc# how to solve this ? please reply From suryak at ieee.org Mon Apr 1 17:00:21 2013 From: suryak at ieee.org (Surya Kasturi) Date: Mon, 1 Apr 2013 20:30:21 +0530 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) Message-ID: Hi, I am Surya, Electronics & Communication Engineering student from India. I have gone through the proposed ideas http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013 and found some of them to be interesting.. 1. RSS and NNTP access to mailman services 2. Authentication of REST API in Postorius/Django 3. Fully Anonymization 4. Currently the above, but even looking in other possibilities My skills are in Python, Django little HTML, CSS. Frankly, I don't grade myself to be "expert" but I am in "just - intermediate" stage, where people are not afraid of taking new challenges, roles. I recently started contributing in SciPy, specifically in SciPyCentral(a file management system) and my current contributions include 1. A new design from scratch using Twitter Bootstrap (merged in master & yet to merge in server). a) The preview pages are here, here , here (Github repo ). Further the design has been improved directly on the cloned branch. However, these pages should give a decent idea of the design. b) ScipyCentral Master 2. Working on implementing RSS/ Atom feeds for the site (repo's exact directory). Considering the level of Feed Syndication Framework, its quite very simple to write this.. however, I am planning to lift of this feeds as "read-only" API.. don't know how far it goes 5. Will admin production server (migration under process) Also, I even wrote some small, cool apps (weekend work) using Django.. Here are : 1. Pingmee (Facebook App, lets people ping their friends using photos) 2. DJerry (social reader Framework for blogger on Facebook -- currently running my photography blog) I appreciate if you could comment on my above recent work (good or bad). Actually, I just want to introduce myself to the community at first.. So, regarding the projects.. It would be great if you could direct me in the right direction.. Many thanks for reading such a long email. Looking forward for your replies. -- Surya From flo.fuchs at gmail.com Mon Apr 1 23:42:36 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Mon, 1 Apr 2013 23:42:36 +0200 Subject: [Mailman-Developers] error while setting up mailman on ubuntu please reply In-Reply-To: References: Message-ID: Hi Ashesh, it looks like installation of one of the dependencies is failing, but it's a little hard to say which one and why (at least to me). Maybe there's a buildout expert on the list who has seen this kind of traceback before and can provide help right away... If not, could you provide us with a little more information: - The revision number your mailman branch is at (type "bzr log -l1") - Your default Python version (type: "python --version") - If we're lucky, the tempfile that failed to be executed still exisits ('/tmp/tmpu69D_h', see the second to last line of your traceback) -- maybe you can post the contents of that file to some paster service and send the link. If it isn't there any more (most definitely after a reboot), you could run bin/buildout again and use the new file name from the bottom of the traceback. Another idea: Have you tried installing mailman inside a virtualenv? Also: It looks like you're running the installation as root. It should work without root privileges. One last thing: We're currently working on simplifying those complicated installation instructions (aka "create non-homicidal install docs" ;). So don't give up now! Cheers, Florian 2013/4/1 Ashesh Vidyut > i m following > > > http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+running > > i had brached mailman > > after that i excecuted python bootstrap.py > > in bin/buildout > > i get > > An internal error occured due to a bug in either zc.buildout or in a > recipe being used: > Traceback (most recent call last): > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > line 1923, in main > getattr(buildout, command)(args) > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > line 466, in install > installed_develop_eggs = self._develop() > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > line 707, in _develop > zc.buildout.easy_install.develop(setup, dest) > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > line 871, in develop > call_subprocess(args) > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > line 129, in call_subprocess > % repr(args)[1:-1]) > Exception: Failed to run command: > '/usr/bin/python', '/tmp/tmpu69D_h', '-q', 'develop', '-mxN', '-d', > '/home/ashesh/mailmanc/develop-eggs/tmpNyd439build' > root at av:/home/ashesh/mailmanc# > > > how to solve this ? > please reply > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/flo.fuchs%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From barry at list.org Tue Apr 2 00:52:09 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 1 Apr 2013 18:52:09 -0400 Subject: [Mailman-Developers] error while setting up mailman on ubuntu please reply In-Reply-To: References: Message-ID: <20130401185209.731d1cb7@anarchist> On Apr 01, 2013, at 11:42 PM, Florian Fuchs wrote: >it looks like installation of one of the dependencies is failing, but it's >a little hard to say which one and why (at least to me). >Maybe there's a buildout expert on the list who has seen this kind of >traceback before and can provide help right away... Hmm, no idea here. It works for me with current trunk, Python 2.7 on Ubuntu 13.04 with a fresh checkout. -Barry From macobo at ut.ee Thu Apr 4 18:59:01 2013 From: macobo at ut.ee (Karl-Aksel Puulmann) Date: Thu, 4 Apr 2013 19:59:01 +0300 Subject: [Mailman-Developers] GSoC 2013 (GNU Mailman) - introductions Message-ID: Hello I'm Karl, 2nd year bachelors CS student from University of Tartu in Estonia. I went through the ideas list and noted down some that I found interesting: 1. Full anonymization 2. Log monitor 3. RSS, NNTP access to archives I got the dev enviroment set up and running and will go through the bug tracker and try to fix a few bugs. If I should run into trouble, I'll let it be known. A little about myself (you may want to skip this). I have about 5 years of programming experience, including programming in python. Last semester I gave lab sessions to first year students (python), and also have some work experience with flask. Other strengths include algorithmic skills (I participated in International Olympiad in Informatics 2 years ago and my team was in semi-finals of the ACM-ICPC competition this year). I'm hoping to participate in GSoC since while I'm an avid open-source consumer, I haven't had any success in contributing to something this big before and it's about time to fix that. You can find some of my toy projects at https://github.com/macobo. I will be accessible by email, skype (macob0) and through IRC (nick macobo at #mailman), but since my time zone is UTC+2, I might not be accessible at all times. Looking forward to (hopefully) working with you Karl-Aksel Puulmann From barry at list.org Thu Apr 4 19:56:12 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 4 Apr 2013 13:56:12 -0400 Subject: [Mailman-Developers] GSoC 2013 (GNU Mailman) - introductions In-Reply-To: References: Message-ID: <20130404135612.4a401900@anarchist> On Apr 04, 2013, at 07:59 PM, Karl-Aksel Puulmann wrote: >I'm Karl, 2nd year bachelors CS student from University of Tartu in >Estonia. Welcome Karl! FWIW, I'm utc-4 but feel free to ping me on #mailman during any overlapping time. -Barry From terri at zone12.com Thu Apr 4 20:38:12 2013 From: terri at zone12.com (Terri Oda) Date: Thu, 04 Apr 2013 12:38:12 -0600 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: References: Message-ID: <515DC894.4030605@zone12.com> Hi Surya! On 04/01/2013 09:00 AM, Surya Kasturi wrote: > I have gone through the proposed ideas > http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013 and found some > of them to be interesting.. > > 1. RSS and NNTP access to mailman services > 2. Authentication of REST API in Postorius/Django > 3. Fully Anonymization > 4. Currently the above, but even looking in other possibilities All of these are things we have need of. If the conversations at PyCon were anything to go by, I think more people are excited by #1, but it is perfectly reasonable if you simply choose whichever one of these appeals to you most. > My skills are in Python, Django little HTML, CSS. > Frankly, I don't grade myself to be "expert" but I am in "just - > intermediate" stage, where people are not afraid of taking new challenges, > roles. No worries -- we don't have any tasks this year that I would say require an expert, and all of them should be good for getting you more practice in a skill. Have you had a chance to set up a Mailman 3 development server and play around yet? With your experience with django, you could easily handle some of the bugs in postorius to get some practice working with our code base. Terri From macobo at ut.ee Thu Apr 4 21:48:50 2013 From: macobo at ut.ee (Karl-Aksel Puulmann) Date: Thu, 4 Apr 2013 22:48:50 +0300 Subject: [Mailman-Developers] error while setting up mailman on ubuntu please reply Message-ID: Hello When I ran into similar problems, running bin/buildout with root premissions helped (sudo bin/buildout). This installed some missing packages (zope.interface). Hopefully this helps. :) All the best Karl-Aksel Puulmann 2013/4/1 Ashesh Vidyut > i m following > > > http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+running > > i had brached mailman > > after that i excecuted python bootstrap.py > > in bin/buildout > > i get > > An internal error occured due to a bug in either zc.buildout or in a > recipe being used: > Traceback (most recent call last): > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > line 1923, in main > getattr(buildout, command)(args) > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > line 466, in install > installed_develop_eggs = self._develop() > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > line 707, in _develop > zc.buildout.easy_install.develop(setup, dest) > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > line 871, in develop > call_subprocess(args) > File > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > line 129, in call_subprocess > % repr(args)[1:-1]) > Exception: Failed to run command: > '/usr/bin/python', '/tmp/tmpu69D_h', '-q', 'develop', '-mxN', '-d', > '/home/ashesh/mailmanc/develop-eggs/tmpNyd439build' > root at av:/home/ashesh/mailmanc# > > > how to solve this ? > please reply > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/flo.fuchs%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 From xuwang at gmail.com Thu Apr 4 23:54:19 2013 From: xuwang at gmail.com (Xu Wang) Date: Thu, 4 Apr 2013 14:54:19 -0700 Subject: [Mailman-Developers] error while setting up mailman on ubuntu please reply In-Reply-To: References: Message-ID: It depends on the version of zope on your system. I'd update the zope.interface before mailman buildout. Here is an example chef recipe for mm3: nclude_recipe "python" # Install base packages for mailman3 %w{ bzr postfix openssl-blacklist ssl-cert }.each do |pkg| package pkg end python_pip "zope.interface" do version "4.0.5" action :install end bash "Download mailman" do user "root" code <<-EOH cd "#{node['mm3']['install_dir']}" bzr branch lp:mailman EOH creates node['mm3']['home'] end bash "Install mailman3" do user "root" code <<-EOH cd "#{node['mm3']['home']}" python setup.py install EOH end # Install mailman conf file service "mailman" do supports :restart => true, :start => true, :stop => true, :status => true action :nothing end template "mailman" do path "/etc/init.d/mailman" source "mailman.d.erb" owner "root" group "root" mode "0755" notifies :enable, "service[mailman]" notifies :reload, "service[mailman]" end template "#{node['mm3']['config_dir']}/mailman.cfg" do mode 0440 owner node['mm3']['owner'] group node['mm3']['group'] variables( 'mm3_config' => node['mm3'] ) notifies :reload, "service[mailman]" end # Install Postfix hook for Mailman service "postfix" do supports :restart => true, :start => true, :stop => true, :reload => true action :nothing end postfixHook = < wrote: > Hello > > When I ran into similar problems, running bin/buildout with root > premissions helped (sudo bin/buildout). This installed some missing > packages (zope.interface). Hopefully this helps. :) > > All the best > Karl-Aksel Puulmann > > 2013/4/1 Ashesh Vidyut > > > i m following > > > > > > > > http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+running > > > > i had brached mailman > > > > after that i excecuted python bootstrap.py > > > > in bin/buildout > > > > i get > > > > An internal error occured due to a bug in either zc.buildout or in a > > recipe being used: > > Traceback (most recent call last): > > File > > > > > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > > line 1923, in main > > getattr(buildout, command)(args) > > File > > > > > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > > line 466, in install > > installed_develop_eggs = self._develop() > > File > > > > > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > > line 707, in _develop > > zc.buildout.easy_install.develop(setup, dest) > > File > > > > > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > > line 871, in develop > > call_subprocess(args) > > File > > > > > > "/home/ashesh/mailmanc/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > > line 129, in call_subprocess > > % repr(args)[1:-1]) > > Exception: Failed to run command: > > '/usr/bin/python', '/tmp/tmpu69D_h', '-q', 'develop', '-mxN', '-d', > > '/home/ashesh/mailmanc/develop-eggs/tmpNyd439build' > > root at av:/home/ashesh/mailmanc# > > > > > > how to solve this ? > > please reply > > _______________________________________________ > > Mailman-Developers mailing list > > Mailman-Developers at python.org > > http://mail.python.org/mailman/listinfo/mailman-developers > > Mailman FAQ: http://wiki.list.org/x/AgA3 > > Searchable Archives: > > http://www.mail-archive.com/mailman-developers%40python.org/ > > Unsubscribe: > > > > http://mail.python.org/mailman/options/mailman-developers/flo.fuchs%40gmail.com > > > > Security Policy: http://wiki.list.org/x/QIA9 > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/xuwang%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From stephen at xemacs.org Fri Apr 5 05:02:52 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 05 Apr 2013 12:02:52 +0900 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: <515DC894.4030605@zone12.com> References: <515DC894.4030605@zone12.com> Message-ID: <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> Terri Oda writes: > Hi Surya! > > On 04/01/2013 09:00 AM, Surya Kasturi wrote: > > I have gone through the proposed ideas > > http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013 and found some > > of them to be interesting.. > > > > 1. RSS and NNTP access to mailman services > > 2. Authentication of REST API in Postorius/Django > > 3. Fully Anonymization > > 4. Currently the above, but even looking in other possibilities > > All of these are things we have need of. If the conversations at PyCon > were anything to go by, I think more people are excited by #1, but it is > perfectly reasonable if you simply choose whichever one of these appeals > to you most. I'd be a little careful about "full anonymization". I would guess it's not a big enough project for GSoC. From sreyanth at gmail.com Fri Apr 5 06:28:11 2013 From: sreyanth at gmail.com (Sreyanth) Date: Fri, 5 Apr 2013 09:58:11 +0530 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion Message-ID: Hi all! I am interested to participate in GSoC this year and would like to choose GNU Mailman for it. I have gone through the proposed ideas and I would like people to tell if they are feasible for one summer! 1. Boilerplate stripper AND Better content-filtering / handling error messages. 2. No-logging mode. 3. Anti-spam / anti-abuse in Mailman. 4. My own project idea: Mining the list logs and recognize interesting patterns for better enhancements (the admin need not have data mining experience) By far, I am looking to duplicate bugs on my comp and am aiming at writing a patch or two before the student deadline, so that I have much more time to work on my application. # Bragging mode ON. You may want to skip this. And, let me introduce myself. I am Sreyantha Chary (lets make it Sreyanth), a 3rd year undergrad majoring in Computer Engineering at the National Institute of Technology Karnataka. I wont consider myself an expert in python programming, but yeah, I am good enough to work on intermediate projects. I have used Django for my university projects and loved it. Research wise, I am into Machine Learning, Information Retrieval and Data Mining. And coding wise, I try crazy stuff, sometimes just to check if I remember anything from the documentation! ?# Bragging mode OFF.? People, please let me know your take on the projects I am interested in. How far is my proposed idea feasible? Thanks in advance. Thanks for your time! * ?Thanks and regards * * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From rahul.nbg at gmail.com Sat Apr 6 04:55:03 2013 From: rahul.nbg at gmail.com (Rahul Gaur) Date: Sat, 6 Apr 2013 08:25:03 +0530 Subject: [Mailman-Developers] GSOC 2013 : Authenticated REST-API in Postorius/Django In-Reply-To: <05BC1D67-31B0-41AA-82C2-8A47AB7AF7BD@NFSNet.org> References: <05BC1D67-31B0-41AA-82C2-8A47AB7AF7BD@NFSNet.org> Message-ID: Hi , Sorry for the late reply , I have been out of the city and I came back to the college yesterday only . Welcome to our community, > Glad to be a part of the community , I hope to contribute towards the development of Gnu Mailman :) > > I'm happy to see that you are interested in the REST interfaces. > In addition to TastyPie, I suggest that you also look at > django-rest-framework. > Please compare the two and let us know what you see as the advantages and > disadvantages of each. > > Richard > > So , last night I tried my hand on django-rest-framework [0] and TastyPie [1] as well. What I could figure out with my two quick and dirty hands on applications of both the frameworks , the django-rest-framework is a bit more lengthy ( but those few extra lines are for the best) when stacked up against Tastypie. I wrote this quick django app and then I tried to write API's for that app in both the framework.Both the frameworks have there own advantages lets talk about that. Lets talk about the django-rest-framework, this was the first framework I tried and so I started from the scratch.So for my toy app , I referred the tutorial(I felt it's better documented than TastyPie ) and started writing the code.By the time I was done ,I could do lot of things with it.Biggest advantage it has over the TastyPie is this one : *$ curl http://127.0.0.1:8080/formpost/ -H 'Accept:text/html' > * The Django-rest-frameworks API's returns web browsable representation of the data by default :) I find this to be a lot more useful in the scenario where I am developing a new app from the scratch. Plus the default frontend is done with bootstrap , which makes it easier to customize as per the requirements as well. Now , I would use TastyPie for an existing django project/app which has a web interface and I just need API's so I can provide third party support for my Project. TastyPie is kind of more easy to get started with as compared to Django-rest-framework. I highly recommend Leonard Richardson's and Sam Ruby's O'Reilly book on > RESTful Web Services: > http://shop.oreilly.com/product/9780596529260.do > Although the book's been out for many years now, I still think it holds up > well, and will give you a great understanding of the basic concepts and > design > principles behind REST. It also gives lots of great tips for organizing > your > resources, etc. Thanks for recommending this book, I have started going through it . Of course, it makes the most sense for the authenticated REST API to as > closely mimic the core admin API as possible. Where there are conflicts, we > should discuss on this mailing list. > Cheers, > -Barry So , the confusion I am facing here is that the Postorius is already mature and stable app and while playing with the two frameworks for API's what I figured out is that if I opt for django-rest-framework , I would need to write new Views ? But if I go for TastyPie , I won't have to touch the existing Views and write authenticated API's. Hence to implement authenticated API's , which one would be better ? [0] - https://github.com/aregee/restful [1] - https://github.com/aregee/restpie Cheers ! ------------------------------------------------------------------------------------------------------- *Rahul Gaur* *irc : iamaregee2* *web: *http://www.rahulgaur.info* * *blogs : *aregee.wordpress.com , http://sanencynicalwriter.wordpress.com/ *fb:* http://facebook.com/iamaregee *github: *https://github.com/aregee From saxena.udit at gmail.com Sat Apr 6 11:33:10 2013 From: saxena.udit at gmail.com (Udit Saxena) Date: Sat, 6 Apr 2013 15:03:10 +0530 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion Message-ID: Hey. I'm interested in participating in GSoC this year and I thought that GNU Mailman would be a good choice because I really wanted to get more involved in python-devs. After going through the list of proposed projects I have listed the projects I think I will be able to complete over the summer. Any inputs from your side ? 1. Authenticated REST-API in Postorius/Django 2. Web Posting Interface. And OpenPGP integration is something I would really interested in learning about but have no prior experience or knowledge about how to go about it. Can you guys guide me to someone who is overlooking these projects ? I would like to ask a few questions. I am Udit, a student in BITS Pilani, India doing my bachelors in Computer Science Engineering. I have experience in coding in python, php, C/C++ and web dev. Currently I am focusing on getting acquainted with Django and trying to fix the easy bugs I find. Thanks. -- --------------------------------------- Udit Saxena Student, BITS Pilani From raj.abhilash1 at gmail.com Sat Apr 6 23:19:31 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Sun, 7 Apr 2013 02:49:31 +0530 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration Message-ID: Hello all, I am a undergrad student interested in OpenPGP integration in mailman as a GSOC project this summer. I haven't worked around with security and encryption but want to attempt this project for i find this interesting. I have prior experience in python and have worked in frameworks like django, tornado. I have installed mailman(v3.0.0b3) and trying to play around with it. It would be really nice if i could get some input from you all to go along with this project as this is a new topic for me. Thanks -- Abhilash Raj From pabs3 at bonedaddy.net Sun Apr 7 00:53:10 2013 From: pabs3 at bonedaddy.net (Paul Wise) Date: Sun, 7 Apr 2013 06:53:10 +0800 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: Message-ID: On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: > I am a undergrad student interested in OpenPGP integration in mailman as a > GSOC project this summer. Cool! I'm not sure about the scope of your project but you may want to review some prior efforts: http://schleuder2.nadir.org/ http://www.synacklabs.net/projects/crypt-ml/ My pet favourite feature from the lurker mail archiver is showing photos from OpenPGP keys in the archive pages. -- bye, pabs http://bonedaddy.net/pabs3/ From dkg at fifthhorseman.net Sun Apr 7 01:17:12 2013 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Sat, 06 Apr 2013 19:17:12 -0400 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: Message-ID: <5160ACF8.1050108@fifthhorseman.net> On 04/06/2013 06:53 PM, Paul Wise wrote: > On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: > >> I am a undergrad student interested in OpenPGP integration in mailman as a >> GSOC project this summer. neat, i'm glad to hear it! > I'm not sure about the scope of your project but you may want to > review some prior efforts: > > http://schleuder2.nadir.org/ > http://www.synacklabs.net/projects/crypt-ml/ see also: http://non-gnu.uvt.nl/mailman-pgp-smime/ http://sels.ncsa.illinois.edu/ > My pet favourite feature from the lurker mail archiver is showing > photos from OpenPGP keys in the archive pages. :) there are a lot of different ways that you might try to integrate message encryption, message signing, etc into a mailing list. There are also a lot of ways to make it easy for users and administrators to shoot themselves in the foot with this stuff; and even seasoned system administrators with years of crypto background can get wrong. :( If i were you, Abhilash, i would start by trying to write up a concise statement about what specific enhancement you want to make from an end-user perspective, and what threat model your enhancement addresses. here are three (very different) starting points as examples: A) I want to make it so that only correctly-signed messages will be redistributed to the list. B) I want to make it so that no one but the list subscribers will be able to be able to view the content of messages sent to the list. C) I don't want the identities of anyone subscribed to the mailing list to be known to anyone but the other subscribers. There are layers of nuance to resolve with each of those goals. i had a hard time keeping them that short because of all the exceptions and questions they raised in my head when i wrote them (Hint: i'm not convinced that either of them is actually well-defined enough to even be considered possible), but some form of either of them might be possible if you make them more precise. Can you try defining what sort of feature you'd like to see implemented? Also, key management is likely to be a large part of any project like this. Have you thought about how a keyring for a mailing list should be handled? Regards, --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1027 bytes Desc: OpenPGP digital signature URL: From raj.abhilash1 at gmail.com Sun Apr 7 13:01:24 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Sun, 7 Apr 2013 16:31:24 +0530 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <5160ACF8.1050108@fifthhorseman.net> References: <5160ACF8.1050108@fifthhorseman.net> Message-ID: Thanks all for replying. On Sun, Apr 7, 2013 at 4:47 AM, Daniel Kahn Gillmor wrote: > On 04/06/2013 06:53 PM, Paul Wise wrote: > > On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: > > > >> I am a undergrad student interested in OpenPGP integration in mailman > as a > >> GSOC project this summer. > > neat, i'm glad to hear it! > > > I'm not sure about the scope of your project but you may want to > > review some prior efforts: > > > > http://schleuder2.nadir.org/ > > http://www.synacklabs.net/projects/crypt-ml/ > > see also: > > http://non-gnu.uvt.nl/mailman-pgp-smime/ > http://sels.ncsa.illinois.edu/ > > > My pet favourite feature from the lurker mail archiver is showing > > photos from OpenPGP keys in the archive pages. > > Thanks for these links. I am currently going through these projects to figure out the implementation part of the OpenPGP into mailman. Also trying to use the mailman-php-smime patch to figure out how it is implemented. > :) > > there are a lot of different ways that you might try to integrate > message encryption, message signing, etc into a mailing list. There are > also a lot of ways to make it easy for users and administrators to shoot > themselves in the foot with this stuff; and even seasoned system > administrators with years of crypto background can get wrong. :( > > If i were you, Abhilash, i would start by trying to write up a concise > statement about what specific enhancement you want to make from an > end-user perspective, and what threat model your enhancement addresses. > > here are three (very different) starting points as examples: > > A) I want to make it so that only correctly-signed messages will be > redistributed to the list. > > B) I want to make it so that no one but the list subscribers will be > able to be able to view the content of messages sent to the list. > > C) I don't want the identities of anyone subscribed to the mailing list > to be known to anyone but the other subscribers. > > There are layers of nuance to resolve with each of those goals. i had a > hard time keeping them that short because of all the exceptions and > questions they raised in my head when i wrote them (Hint: i'm not > convinced that either of them is actually well-defined enough to even be > considered possible), but some form of either of them might be possible > if you make them more precise. > > Can you try defining what sort of feature you'd like to see implemented? > > Well what i want to make it is that whenever a user sends a mail to the list it should be singed with his private key so that it can be verified against his public that he uploads if he wants permissions to post in the list. As the message is received by mailman its signature is verified and then its encrypted and sent to each person, wherein those who haven't uploaded their key will also receive an unencrypted copy(with a probability that it may not be intended for them or not authentic mail). I also agree that I am new to cryptography so I cannot comment/assure about the implementation of this idea. But with your help I think I think I would be able to implement the best possible version of this idea. > Also, key management is likely to be a large part of any project like > this. Have you thought about how a keyring for a mailing list should be > handled? > > Yes, this was on the top of my mind while trying to attempt this project. I learned about key-servers. I think we could setup one wherein all the public key would be stored that are uploaded by users and retrieved when needed. > Regards, > > --dkg > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > Thanks! -- Abhilash Raj From macobo at ut.ee Sun Apr 7 14:24:33 2013 From: macobo at ut.ee (Karl-Aksel Puulmann) Date: Sun, 7 Apr 2013 15:24:33 +0300 Subject: [Mailman-Developers] GSoC 2013 (GNU Mailman) - introductions In-Reply-To: <20130404135612.4a401900@anarchist> References: <20130404135612.4a401900@anarchist> Message-ID: On Thu, Apr 4, 2013 at 8:56 PM, Barry Warsaw wrote: > On Apr 04, 2013, at 07:59 PM, Karl-Aksel Puulmann wrote: > > >I'm Karl, 2nd year bachelors CS student from University of Tartu in > >Estonia. > > Welcome Karl! FWIW, I'm utc-4 but feel free to ping me on #mailman during > any > overlapping time. > > -Barry > > Will do, as I have some questions about a few issues on the tracker. But as I will be away in Germany next week, that'll have to be saved for next weekend or later. I tried to fix two of the bugs on the tracker, and am waiting for the inevitable feedback on code style etc. Please review. :) About GSoC project ideas - right now working on mailman core seems to be more fun. Which brings me to the following question - which of the projects and bug fixes would benefit Mailman the most, making MM3 more stable and increasing adoption by mailing lists? Thanks to all the helpful people on IRC (florianf, terri, mspiro and others) who have helped me with various issues I ran into. All the best Karl [1]: https://code.launchpad.net/~macobo/mailman/macobo-rcpt-stage-reject/+merge/157516 [2]: https://code.launchpad.net/~macobo/mailman/macobo-conf_sort_option/+merge/157469 From avikpal.me at gmail.com Sun Apr 7 09:46:49 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Sun, 7 Apr 2013 13:16:49 +0530 Subject: [Mailman-Developers] Google Summer of Code'13 introduction Message-ID: Hello, I'm Avik, 3rd yr (junior) bachelor CS student from Bengal Engineering & Science University,Shibpur. I went through the GSOC '13 ideas list and found the project *Anti-spam/anti-abuse in mailman* very interesting. I also would like to say that I also fulfill all the necessary prerequisites given in the project ideas page. A little about myself I have about 4 years of programming experience, I am the coordinator of the Software Club in my University and help first and second year students with Algorithms, I sometimes also take coding sessions here. My team qualified for the ACM-ICPC 2012 Regionals(only team selected from my university in that region). I also have over two years experience in machine learning, NLP (just developed a very basic rudimentary twitter classifier) and have about 5 years experience in statistics. I'm hoping to participate in GSoC since while I'm an avid open-source enthusiast, I haven't had any chance in contributing to something this big before and it's about time to fix that.You can find some of my toy projects at https://github.com/avikpal I have been going through the links given in the mailman developers' page and documentations here and there, maybe you can help me with some useful resources so that right away I can start contributing by fixing small bugs and in this way sharpen my skills before GSOC actually starts. I will be accessible by email and through IRC (nick avikp at freenode), but since my time zone is UTC+5:30, I might not be there at all times just drop me a mail and I will get in touch as soon as possible. Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme From stephen at xemacs.org Sun Apr 7 16:16:56 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 07 Apr 2013 23:16:56 +0900 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <5160ACF8.1050108@fifthhorseman.net> Message-ID: <87fvz28lyf.fsf@uwakimon.sk.tsukuba.ac.jp> Abhilash Raj writes: > Well what i want to make it is that whenever a user sends a mail to the > list it should be singed with his private key so that it can be verified > against his public that he uploads if he wants permissions to post in the > list. You mean that the user should sign it himself (or with the help of his mail client), is that correct? > As the message is received by mailman its signature is verified and > then its encrypted and sent to each person, wherein those who > haven't uploaded their key will also receive an unencrypted > copy(with a probability that it may not be intended for them or not > authentic mail). I don't understand the use case for having both encrypted and unencrypted copies distributed. Is the encryption intended to be merely authentication? But what Mailman has is by definition the subscriber's public key; anybody might have that. It *could* be kept secret, but I think that's not so easy to prove. I would have imagined that maybe Mailman would resign using its own private key, to authenticate the list, and testify that it had authenticated the sender. I also don't understand what you mean by "not authentic mail". The original signature proves it authentic. The subscribers may not have the appropriate to key to verify, but in that case I don't see why they would want to delegate it to Mailman. I think you have a difficult task in merely specifying what you want this system to do. That's likely to be a couple orders of magnitude harder than the implementation! > Yes, this was on the top of my mind while trying to attempt this > project. I learned about key-servers. I think we could setup one > wherein all the public key would be stored that are uploaded by > users and retrieved when needed. But who watches the watcher? That is, what does the keyserver need to know about the key's owner, and how does the candidate subscriber prove it to the keyserver? I think there are lots of use cases for integrating mailing list managers into the public key infrastructure, but you need to be careful to specify them. I think you probably should start with simple use cases, like proving subscriber identity to the mailing list manager, eg for anti-spam purposes.[1] Footnotes: [1] Even that is not a sure winner, since most users will not know how to do this for themselves. So it will have to be integrated into clients, which themselves might be infected by a virus. From viswanathkarthikeya at gmail.com Sun Apr 7 11:47:24 2013 From: viswanathkarthikeya at gmail.com (Karthikeya Viswanath) Date: Sun, 7 Apr 2013 10:47:24 +0100 Subject: [Mailman-Developers] [GSOC 2013] Introduction and Ideas Message-ID: Hi, I'm Karthikeya , a 2nd year undergrad from IIT Madras in Chennai, India. I'd like to take up a Mailman project as a part of GSOC 2013. I set up Mailman 3 and started tinkering with a couple of bugs . Hope to come up with something useful soon. I went through the ideas list, and some ideas seemed interesting. In no particular order, 1. RSS , NNTP access to the archives The ideas page mentions a previous GSOC project on this. I've not been able to find it , so could someone please direct me to it? 2. Full anonymization This seems straightforward, so is there something else that can be added to it? Like enabling the hashing option for other lists in which anonymization is allowed? 3. No logging This seems a very interesting project, and something a mail archive should have. But I'm not very familiar with the laws that concern subpoena with respect to archived communication. Can someone please elaborate on this? 4. Web Posting Interface I've had experience in designing a forum structure before, and this seems a really great addition to Mailman. I also have a couple of ideas of my own, and I'd like to see how feasible they are. 1. Tags: Could we have tags for mails so we could get an idea for what the mail is about at a glance? This could also help in the metrics and also in spam reduction . ( As in undesired mails ) 2. Better filters for archive download: Give users the option to download specific threads of an archive , or all contributions by a specific user or all threads with a keyword. Last time I had to do it, I had to iterate over the entire list and save the pages manually. I have experience at programming in a wide variety of languages . I've had to set up mailman 2 for our department lists and manage them. I've worked primarily as a web designer using HTML, PHP and Django. I've longed to work on an open source project and these are my first steps at contributing to a large one. Tl;Dr :- Just another mail by a GSOC hopeful. Bit long, yes. Thank you. Karthikeya Viswanath From suryak at ieee.org Mon Apr 8 03:20:30 2013 From: suryak at ieee.org (Surya Kasturi) Date: Mon, 8 Apr 2013 06:50:30 +0530 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Fri, Apr 5, 2013 at 8:32 AM, Stephen J. Turnbull wrote: > Terri Oda writes: > > Hi Surya! > > > > On 04/01/2013 09:00 AM, Surya Kasturi wrote: > > > I have gone through the proposed ideas > > > http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013 and > found some > > > of them to be interesting.. > > > > > > 1. RSS and NNTP access to mailman services > > > 2. Authentication of REST API in Postorius/Django > > > 3. Fully Anonymization > > > 4. Currently the above, but even looking in other possibilities > > > > All of these are things we have need of. If the conversations at PyCon > > were anything to go by, I think more people are excited by #1, but it is > > perfectly reasonable if you simply choose whichever one of these appeals > > to you most. > > I'd be a little careful about "full anonymization". I would guess > it's not a big enough project for GSoC. > > Actually, I am on Windows. setting up Mailman is a problem for me.. need time for that. From stephen at xemacs.org Mon Apr 8 03:56:15 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 08 Apr 2013 10:56:15 +0900 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> Surya Kasturi writes: > Actually, I am on Windows. setting up Mailman is a problem for me.. need > time for that. Ouch. Do you have access to a VM hypervisor so you can run Linux inside of Windows? If so, the shortest path home is likely to be installing Ubuntu there. Otherwise, we'll help as we can, but I doubt any of the mentors are using that environment. It's likely to be somewhat painful in the long run as well, as Barry has indicated that he really wants Mailman 3 to be a Python 3 application. I don't want to discourage you. But please be realistic about your choice of environment. From stephen at xemacs.org Mon Apr 8 04:00:45 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 08 Apr 2013 11:00:45 +0900 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> Here's a brain bubble for you: Stephen J. Turnbull writes: > I don't want to discourage you. But please be realistic about your > choice of environment. Ooh, tasty! How about a Mailman Raspberry Pi? Seriously, Barry, ya think it's possible? This would solve the environment problem for students! Dunno if it's strictly better than an AWS instance, but I bet many would prefer it. Steve From terri at zone12.com Mon Apr 8 04:14:59 2013 From: terri at zone12.com (Terri Oda) Date: Sun, 07 Apr 2013 20:14:59 -0600 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <51622823.1040003@zone12.com> On 13-04-07 8:00 PM, Stephen J. Turnbull wrote: > This would solve the environment problem for students! Dunno if it's > strictly better than an AWS instance, but I bet many would prefer it. > I *do* have an older fedora vm set up for postorius work, which can be run from vmware on windows. It hasn't been updated since the GHC13 codeathon (fall 2012), but if anyone needs it to play with, I can put it somewhere accessible. It isn't set up to deliver email, but it shouldn't be impossible to set it up if someone needs it. Terri From terri at zone12.com Mon Apr 8 04:18:29 2013 From: terri at zone12.com (Terri Oda) Date: Sun, 07 Apr 2013 20:18:29 -0600 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: References: Message-ID: <516228F5.9060600@zone12.com> On 13-04-06 3:33 AM, Udit Saxena wrote: > 1. Authenticated REST-API in Postorius/Django > 2. Web Posting Interface. > > And OpenPGP integration is something I would really interested in learning > about but have no prior experience or knowledge about how to go about it. > > Can you guys guide me to someone who is overlooking these projects ? I > would like to ask a few questions. We haven't actually parcelled out the projects to specific mentors at this stage, and we prefer to keep the whole community involved in discussions as much as possible, so please ask all questions directly on the list! Terri From terri at zone12.com Mon Apr 8 04:35:25 2013 From: terri at zone12.com (Terri Oda) Date: Sun, 07 Apr 2013 20:35:25 -0600 Subject: [Mailman-Developers] Google Summer of Code'13 introduction In-Reply-To: References: Message-ID: <51622CED.6010500@zone12.com> On 13-04-07 1:46 AM, Avik Pal wrote: > I have been going through the links given in the mailman developers' page > and documentations here and there, maybe you can help me with some useful > resources so that right away I can start contributing by fixing small bugs > and in this way sharpen my skills before GSOC actually starts. Since you're looking at something in Mailman core, I recommend you take a look at the "easy" bugs there: https://bugs.launchpad.net/mailman/+bugs?field.tag=easy You might also find it helpful to read the .rst docs for Mailman, which can be found in html format here: http://pythonhosted.org/mailman/ You might also want to do a bit of searching through the wiki/list archives to see some of the thoughts people have had on what anti-spam in Mailman should look like. You don't have to take any of it as The One True Way, or even pay attention to it all in the end, but if you could set up a wiki page summarizing the ideas and arguments, it'd get you up to speed and provide a resource for later. Other than that... we've pretty much tried to put everything we could think of into the wiki docs, so if you've read those, you've got a good idea of where to start! Terri From terri at zone12.com Mon Apr 8 05:14:17 2013 From: terri at zone12.com (Terri Oda) Date: Sun, 07 Apr 2013 21:14:17 -0600 Subject: [Mailman-Developers] [GSOC 2013] Introduction and Ideas In-Reply-To: References: Message-ID: <51623609.6050909@zone12.com> On 13-04-07 3:47 AM, Karthikeya Viswanath wrote: > I also have a couple of ideas of my own, and I'd like to see how feasible > they are. > 1. Tags: Could we have tags for mails so we could get an idea for what the > mail is about at a glance? This could also help in the metrics and also in > spam reduction . ( As in undesired mails ) > 2. Better filters for archive download: Give users the option to download > specific threads of an archive , or all contributions by a specific user or > all threads with a keyword. Last time I had to do it, I had to iterate over > the entire list and save the pages manually. Getting people to use tags would be challenging for a lot of mailing lists, given how hard it is to get them to use topics correctly (or even dlists, although I've seen a lot more success with those). You'd probably have to work on tagging automatically as well if you wanted it to ever be useful. It might be cool, but to be completely honest, we're looking for more sure-thing desired projects this summer so that that we can showcase them in our next release, so you might have a bit of an uphill battle convincing us that this particular feature would fit the bill. Filters for archive download might be more interesting, although again, I'm unsure how many people would use it. Still, I can see that lots of people would potentially use filters for views on archives. Have you taken a look at what HyperKitty is doing now to see how your idea might fit in with what's already there? Both of these might actually be interesting in conjunction with the work that's already happening on dlists, so you might want to read up on that (check the Mailman or Systers wikis or just go through the archives of the list) and see if it spawns any new project ideas. Terri From saxena.udit at gmail.com Mon Apr 8 10:13:10 2013 From: saxena.udit at gmail.com (Udit Saxena) Date: Mon, 8 Apr 2013 13:43:10 +0530 Subject: [Mailman-Developers] Fwd: GSOC 2013 - Introduction and Project Discussion In-Reply-To: References: <516228F5.9060600@zone12.com> Message-ID: ---------- Forwarded message ---------- From: Udit Saxena Date: Mon, Apr 8, 2013 at 1:42 PM Subject: Re: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion To: I'm having problems installing the environment. From the 5 minute guide installation page: http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+running my "bin/buildout" from the mailman folder is not able to run. bootstrap.py runs fine. But I don't know how to go further. The error says: Exception: failed to run command : "usr/bin/python3.3", "/tmp/tmpoe01dx", "-q","develop", "-mxN".... The same error occurs with python3 as well. Please help. On Mon, Apr 8, 2013 at 7:48 AM, Terri Oda wrote: > > On 13-04-06 3:33 AM, Udit Saxena wrote: > >> 1. Authenticated REST-API in Postorius/Django >> 2. Web Posting Interface. >> >> And OpenPGP integration is something I would really interested in learning >> about but have no prior experience or knowledge about how to go about it. >> >> Can you guys guide me to someone who is overlooking these projects ? I >> would like to ask a few questions. >> > > We haven't actually parcelled out the projects to specific mentors at this > stage, and we prefer to keep the whole community involved in discussions as > much as possible, so please ask all questions directly on the list! > > Terri > > -- --------------------------------------- Udit Saxena Student, BITS Pilani, +917891400270 -- --------------------------------------- Udit Saxena Student, BITS Pilani, +917891400270 From suryak at ieee.org Mon Apr 8 12:37:28 2013 From: suryak at ieee.org (Surya Kasturi) Date: Mon, 8 Apr 2013 16:07:28 +0530 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: <51622823.1040003@zone12.com> References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> <51622823.1040003@zone12.com> Message-ID: On Mon, Apr 8, 2013 at 7:44 AM, Terri Oda wrote: > > On 13-04-07 8:00 PM, Stephen J. Turnbull wrote: > >> This would solve the environment problem for students! Dunno if it's >> strictly better than an AWS instance, but I bet many would prefer it. >> >> I *do* have an older fedora vm set up for postorius work, which can be > run from vmware on windows. It hasn't been updated since the GHC13 > codeathon (fall 2012), but if anyone needs it to play with, I can put it > somewhere accessible. It isn't set up to deliver email, but it shouldn't > be impossible to set it up if someone needs it. I guess setting up a VM works for now..(will do it).. I am quite familiar with Ubuntu (had used ver 8 for an year).. These days, the linux kernel is showing some power regression issues which is making me to use windows.. Since no one had so far configured mailman on windows, I guess, I can try it (not now but.. later on). Why not we build an executable for windows (.exe) for users to install? This could take sometime to port and solve conflicts.. but at the end, it would be superb. > > Terri > > > ______________________________**_________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/**mailman/listinfo/mailman-**developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/** > mailman-developers%40python.**org/ > Unsubscribe: http://mail.python.org/**mailman/options/mailman-** > developers/suryak%40ieee.org > > Security Policy: http://wiki.list.org/x/QIA9 > From follybeachris at gmail.com Mon Apr 8 15:46:03 2013 From: follybeachris at gmail.com (Chris Cargile) Date: Mon, 8 Apr 2013 09:46:03 -0400 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> <51622823.1040003@zone12.com> Message-ID: Stephen wrote: >>This would solve the environment problem for students! Dunno if it's >> strictly better than an AWS instance, but I bet many would prefer it. I had mentioned sharing an AMI instance with mailman 'working' on it but for some reason had lagged. I did not configure it to send mail yet, but setting up postfix on Ubuntu can be done for that relatively easily, if configuring MM3 is about as tough/easy as for mm2 (i'd guess it is ~the same) - the issue with making AMI's available is that the images have to be backed up to an ElasticBlockStore(EBS), which entails a cost (considered as: live-,running image==free; static,shareable image==fee associated) The last status discussed was to maybe upload the image to somewhere else that would be free, I think (Terri?) but it seems that approach would negate any advantage offered by having the image (the download would be on the order of several Gigabytes...do we want to keep tossing ideas about the VM / (rasbPi???) approach? I'd be interested On Mon, Apr 8, 2013 at 6:37 AM, Surya Kasturi wrote: > On Mon, Apr 8, 2013 at 7:44 AM, Terri Oda wrote: > > > > > On 13-04-07 8:00 PM, Stephen J. Turnbull wrote: > > > >> This would solve the environment problem for students! Dunno if it's > >> strictly better than an AWS instance, but I bet many would prefer it. > >> > >> I *do* have an older fedora vm set up for postorius work, which can be > > run from vmware on windows. It hasn't been updated since the GHC13 > > codeathon (fall 2012), but if anyone needs it to play with, I can put it > > somewhere accessible. It isn't set up to deliver email, but it shouldn't > > be impossible to set it up if someone needs it. > > > I guess setting up a VM works for now..(will do it).. I am quite familiar > with Ubuntu (had used ver 8 for an year).. These days, the linux kernel is > showing some power regression issues which is making me to use windows.. > > Since no one had so far configured mailman on windows, I guess, I can try > it (not now but.. later on). Why not we build an executable for windows > (.exe) for users to install? This could take sometime to port and solve > conflicts.. but at the end, it would be superb. > > > > > > > Terri > > > > > > ______________________________**_________________ > > Mailman-Developers mailing list > > Mailman-Developers at python.org > > http://mail.python.org/**mailman/listinfo/mailman-**developers< > http://mail.python.org/mailman/listinfo/mailman-developers> > > Mailman FAQ: http://wiki.list.org/x/AgA3 > > Searchable Archives: http://www.mail-archive.com/** > > mailman-developers%40python.**org/< > http://www.mail-archive.com/mailman-developers%40python.org/> > > Unsubscribe: http://mail.python.org/**mailman/options/mailman-** > > developers/suryak%40ieee.org< > http://mail.python.org/mailman/options/mailman-developers/suryak%40ieee.org > > > > > > Security Policy: http://wiki.list.org/x/QIA9 > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/follybeachris%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From follybeachris at gmail.com Mon Apr 8 15:57:53 2013 From: follybeachris at gmail.com (Chris Cargile) Date: Mon, 8 Apr 2013 09:57:53 -0400 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: References: Message-ID: A while back, I proposed contributing to MM3 in the area of archive searching and MM2 backwards -compatibility and am considering further, still: Some research I wanted to share is: When navigating some Apache projects, I discovered many are hosted on one of four platforms [per here] but since we welcome the idea of ideas and frown at lock-in, what if we could invite/reach out to administrators who've chosen apache lists to use (on mod_mbox approach) to consider MM3 as a replacement solution? This would allow us to grow the base of users by attracting an audience potentially with the resources to provide some nice observations,contributions to MM3's project I'd guess, maybe? MM3 marketing/capture strategies to rest - to return to researching MM3 alternatives, I evaluate Apache's alternatives as: - mail-archive.com, markmail (good but kinda proprietary, right?); - marc (monolithic?); - and Apache's mod_mbox (GSOC project - 2005) To expand more, maybe it is required for apache projects to host/use mod_mbox and others, but I am wondering, then, is mail-archive.com really the likely system of choice used in *lots* projects due to searchable-archives function? Can I say 'yay' for up-voting (again) the search function and express a personal appeal towards the ideas @ GSOC ideas (#1,#2 ) I'm quite glad the activity is continuing here! Chris From follybeachris at gmail.com Mon Apr 8 18:10:05 2013 From: follybeachris at gmail.com (Chris Cargile) Date: Mon, 8 Apr 2013 12:10:05 -0400 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: References: Message-ID: I promise to try and do better proof-reading for the emails I post. To correct how my earlier email reads, please interpret the question I asked as: ..what if we could invite/reach out to administrators who've chosen to host their apache project-lists using 'mod_mbox approach' to consider MM3 as a replacement solution? With any list-admins among those projects tying the projects' mbox files into a mailman page, we'd achieve more users trying mailman, was my only take-away point for that, I guess. To move into GSOC, I'd like some information on "RSS and/or NNTP access to Mailman archives." - and, more specifically for RSS, is displaying an mbox file being achieved anywhere on-line using mm3 facilities (ie: hyperkitty + django templates?) - I assume yes per mention of the live MM3 instance with 2500 users' existence, but perhaps that only runs the email part and its archives are not web-exposed via archive-interface- was the URL for it mentioned when it was learned of during pycon? Thanks for your help and sharing more about this functionality! Chris On Mon, Apr 8, 2013 at 9:57 AM, Chris Cargile wrote: > A while back, I proposed contributing to MM3 in the area of archive > searching and MM2 backwards -compatibility and am considering further, > still: > > change: ... what if we could invite/reach out to administrators who've > chosen to use (on on mod_mbox approach) to consider MM3 as a replacement > solution? This would allow us to grow the base of users by attracting an > audience potentially with the resources to provide some nice > observations,contributions to MM3's project I'd guess, maybe? > > MM3 marketing/capture strategies to rest - to return to researching MM3 > alternatives, I evaluate Apache's alternatives as: > - mail-archive.com, markmail (good but kinda proprietary, right?); > - marc (monolithic?); > - and Apache's mod_mbox (GSOC project - 2005) > > To expand more, maybe it is required for apache projects to host/use > mod_mbox and others, but I am wondering, then, is mail-archive.com really > the likely system of choice used in *lots* projects due to > searchable-archives function? Can I say 'yay' for up-voting (again) the > search function and express a personal appeal towards the ideas @ GSOC > ideas (#1,#2 > ) > > I'm quite glad the activity is continuing here! > Chris > From barry at list.org Mon Apr 8 18:15:55 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 8 Apr 2013 12:15:55 -0400 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: References: Message-ID: <20130408121555.2bdde3e3@anarchist> On Apr 08, 2013, at 12:10 PM, Chris Cargile wrote: >..what if we could invite/reach out to administrators who've chosen to host >their apache project-lists using 'mod_mbox approach' to consider MM3 as a >replacement solution? Please note too that MM3 supports an IArchiver interface, for which multiple implementations can and do exist. The core has several archiver interfaces, and HyperKitty has an add on that can be enabled to add HK support too. I'm not sure what it would take to add mod_mbox support, but I can't imagine it would be that difficult. One thing we are lacking though is the ability for individual list owners to opt-in to or -out of system-wide available archivers. That's probably not a lot of work to enable, but it does involve a schema migration most likely. (Hmm, I just had an interesting thought about that...) -Barry From barry at list.org Mon Apr 8 18:16:58 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 8 Apr 2013 12:16:58 -0400 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130408121658.63e7fd84@anarchist> On Apr 08, 2013, at 11:00 AM, Stephen J. Turnbull wrote: >Ooh, tasty! How about a Mailman Raspberry Pi? > >Seriously, Barry, ya think it's possible? > >This would solve the environment problem for students! Dunno if it's >strictly better than an AWS instance, but I bet many would prefer it. It's a very cool idea! I'll have to steal mine back away from my son to give that a try. :) -Barry From barry at list.org Mon Apr 8 18:19:27 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 8 Apr 2013 12:19:27 -0400 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> <51622823.1040003@zone12.com> Message-ID: <20130408121927.1b9a1185@anarchist> On Apr 08, 2013, at 04:07 PM, Surya Kasturi wrote: >Since no one had so far configured mailman on windows, I guess, I can try >it (not now but.. later on). Why not we build an executable for windows >(.exe) for users to install? This could take sometime to port and solve >conflicts.. but at the end, it would be superb. IIRC, Mark was using Cygwin at one point. The thing about Windows is that we're using several *nix-isms in process management and possibly in other places. I have no burning desire to try to make these Windows compatible. -Barry From terri at zone12.com Mon Apr 8 18:53:38 2013 From: terri at zone12.com (Terri Oda) Date: Mon, 08 Apr 2013 10:53:38 -0600 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: <20130408121927.1b9a1185@anarchist> References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> <51622823.1040003@zone12.com> <20130408121927.1b9a1185@anarchist> Message-ID: <5162F612.7020600@zone12.com> On 13-04-08 10:19 AM, Barry Warsaw wrote: > The thing about Windows is that we're using several *nix-isms in > process management and possibly in other places. I have no burning > desire to try to make these Windows compatible. I'll bet the documentation would bite us pretty badly, too. While I don't think it's impossible to run mailman on windows, I'd rather the effort went to making this release awesome for the platforms we already support rather than making it run on other platforms. Maybe in a year or two when we're a bit more stable so porting won't be a continual drain on resources? Terri From stephen at xemacs.org Mon Apr 8 18:59:55 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 09 Apr 2013 01:59:55 +0900 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: References: <515DC894.4030605@zone12.com> <87obdt8ysj.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqsd7pkw.fsf@uwakimon.sk.tsukuba.ac.jp> <87vc7x7pde.fsf@uwakimon.sk.tsukuba.ac.jp> <51622823.1040003@zone12.com> Message-ID: <87r4il6jqs.fsf@uwakimon.sk.tsukuba.ac.jp> Surya Kasturi writes: > Since no one had so far configured mailman on windows, I guess, I > can try it (not now but.. later on). Why not we build an executable > for windows (.exe) for users to install? This could take sometime > to port and solve conflicts.. but at the end, it would be superb. Realistically? On the build side, we have few to no Windows experts. I also have my doubts about "superb" in principle: Microsoft honors the POSIX functions and RFC protocols that Mailman depends on more in the breach than the observance. This could be very painful.... On the user side, are there really people out there knocking down doors to install Mailman on Windows? From rahul.nbg at gmail.com Mon Apr 8 19:30:51 2013 From: rahul.nbg at gmail.com (Rahul Gaur) Date: Mon, 8 Apr 2013 23:00:51 +0530 Subject: [Mailman-Developers] GSOC 2013 : Authenticated REST-API in Postorius/Django In-Reply-To: References: <05BC1D67-31B0-41AA-82C2-8A47AB7AF7BD@NFSNet.org> Message-ID: Hi , Sorry for the late reply , I have been out of the city and I came back to the college yesterday only . > Welcome to our community, Glad to be a part of the community , I hope to contribute towards the development of Gnu Mailman :) > > > I'm happy to see that you are interested in the REST interfaces. > In addition to TastyPie, I suggest that you also look at django-rest-framework. > Please compare the two and let us know what you see as the advantages and disadvantages of each. > > Richard > So , last night I tried my hand on django-rest-framework [0] and TastyPie [1] as well. What I could figure out with my two quick and dirty hands on applications of both the frameworks , the django-rest-framework is a bit more lengthy ( but those few extra lines are for the best) when stacked up against Tastypie. I wrote this quick django app and then I tried to write API's for that app in both the framework.Both the frameworks have there own advantages lets talk about that. Lets talk about the django-rest-framework, this was the first framework I tried and so I started from the scratch.So for my toy app , I referred the tutorial(I felt it's better documented than TastyPie ) and started writing the code.By the time I was done ,I could do lot of things with it.Biggest advantage it has over the TastyPie is this one : >> >> $ curl http://127.0.0.1:8080/formpost/ -H 'Accept:text/html' > > > The Django-rest-frameworks API's returns web browsable representation of the data by default :) I find this to be a lot more useful in the scenario where I am developing a new app from the scratch. Plus the default frontend is done with bootstrap , which makes it easier to customize as per the requirements as well. Now , I would use TastyPie for an existing django project/app which has a web interface and I just need API's so I can provide third party support for my Project. TastyPie is kind of more easy to get started with as compared to Django-rest-framework. > I highly recommend Leonard Richardson's and Sam Ruby's O'Reilly book on > RESTful Web Services: > http://shop.oreilly.com/product/9780596529260.do > Although the book's been out for many years now, I still think it holds up > well, and will give you a great understanding of the basic concepts and design > principles behind REST. It also gives lots of great tips for organizing your > resources, etc. Thanks for recommending this book, I have started going through it . > Of course, it makes the most sense for the authenticated REST API to as > closely mimic the core admin API as possible. > > Where there are conflicts, we > should discuss on this mailing list. > Cheers, > -Barry So , the confusion I am facing here is that the Postorius is already mature and stable app and while playing with the two frameworks for API's what I figured out is that if I opt for django-rest-framework , I would need to write new Views ? But if I go for TastyPie , I won't have to touch the existing Views and write authenticated API's. Hence to implement authenticated API's , which one would be better ? [0] - https://github.com/aregee/restful [1] - https://github.com/aregee/restpie Cheers ! ------------------------------------------------------------------------------------------------------- Rahul Gaur irc : iamaregee2 web: http://www.rahulgaur.info blogs : aregee.wordpress.com , http://sanencynicalwriter.wordpress.com/ fb: http://facebook.com/iamaregee github: https://github.com/aregee From rkw at DATAPLEX.NET Mon Apr 8 14:02:57 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Mon, 8 Apr 2013 07:02:57 -0500 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: References: <516228F5.9060600@zone12.com> Message-ID: <46D2A52B-6726-4165-8BA9-E1A0B6EAAF19@DATAPLEX.NET> Although we would like for MM3 to run on Python 3, I don't think that all of our dependency libraries are ready for that yet. I am running MM under Python 2.7 Richard On Apr 8, 2013, at 3:13 AM, Udit Saxena wrote: > ---------- Forwarded message ---------- > From: Udit Saxena > Date: Mon, Apr 8, 2013 at 1:42 PM > Subject: Re: [Mailman-Developers] GSOC 2013 - Introduction and Project > Discussion > To: > > > I'm having problems installing the environment. From the 5 minute guide > installation page: > http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+running > > my "bin/buildout" from the mailman folder is not able to run. bootstrap.py > runs fine. But I don't know how to go further. > The error says: Exception: failed to run command : "usr/bin/python3.3", > "/tmp/tmpoe01dx", "-q","develop", "-mxN".... > > The same error occurs with python3 as well. > > Please help. > > > On Mon, Apr 8, 2013 at 7:48 AM, Terri Oda wrote: > >> >> On 13-04-06 3:33 AM, Udit Saxena wrote: >> >>> 1. Authenticated REST-API in Postorius/Django >>> 2. Web Posting Interface. >>> >>> And OpenPGP integration is something I would really interested in learning >>> about but have no prior experience or knowledge about how to go about it. >>> >>> Can you guys guide me to someone who is overlooking these projects ? I >>> would like to ask a few questions. >>> >> >> We haven't actually parcelled out the projects to specific mentors at this >> stage, and we prefer to keep the whole community involved in discussions as >> much as possible, so please ask all questions directly on the list! >> >> Terri >> >> > > > -- > --------------------------------------- > Udit Saxena > Student, BITS Pilani, > +917891400270 > > > > > -- > --------------------------------------- > Udit Saxena > Student, BITS Pilani, > +917891400270 > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/richard%40nfsnet.org > > Security Policy: http://wiki.list.org/x/QIA9 From barry at list.org Mon Apr 8 20:34:01 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 8 Apr 2013 14:34:01 -0400 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: <46D2A52B-6726-4165-8BA9-E1A0B6EAAF19@DATAPLEX.NET> References: <516228F5.9060600@zone12.com> <46D2A52B-6726-4165-8BA9-E1A0B6EAAF19@DATAPLEX.NET> Message-ID: <20130408143401.4088c3b2@anarchist> On Apr 08, 2013, at 07:02 AM, Richard Wackerbarth wrote: >Although we would like for MM3 to run on Python 3, I don't think that all of >our dependency libraries are ready for that yet. I am running MM under >Python 2.7 We're down to two dependencies: restish and storm. I took a crack at restish, but got stuck. At Pycon, Thomi and I talked about another approach, and he was taking another stab at it, but I haven't talked with him about since then. I suspect storm will never get ported. I've said before that if that's the last dependency, we'll switch, probably back to SQLAlchemy as that's the only Python 3 compatible ORM I'm aware of. Help certainly wouldn't be turned down. :) -Barry From varunsharmalive at gmail.com Mon Apr 8 18:18:00 2013 From: varunsharmalive at gmail.com (varun sharma) Date: Mon, 8 Apr 2013 21:48:00 +0530 Subject: [Mailman-Developers] gsoc 2013 introduction and postorius merge request Message-ID: Hi, I am an undergrad computer science student at manipal institute of technology, india and a prospective gsoc 2013 participant. I have a nice experience in django,bootstrap, jquery and json which i got during my previous internship. I really liked most of the mailman project ideas and started working on one of them today. As i know, there is a tradition of fixing a bug to get into any open source organization. So i worked for few hours on https://bugs.launchpad.net/postorius/+bug/1066184 today. Now user can edit his first_name and last_name. There are few problems that i faced and want to discuss with core developers: 1. There is no model class which derives the base User class, to store values like twitter_id, irc_nick etc. (Should i create one ?) 2. Can username be changed ? The changes i have done today: 1. first_name and last_name can be edited and saved using jquery Changes that i am planning to do tomorrow: 1. Add bootstrap timeout alert on change() and save 2. add model entry for irc_nick,website and twitter_id I have already submitted the merge request. Hope you guys will look into my code and correct my mistakes. Thanks Varun Sharma From richard at NFSNet.org Mon Apr 8 20:59:07 2013 From: richard at NFSNet.org (Richard Wackerbarth) Date: Mon, 8 Apr 2013 13:59:07 -0500 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: <20130408143401.4088c3b2@anarchist> References: <516228F5.9060600@zone12.com> <46D2A52B-6726-4165-8BA9-E1A0B6EAAF19@DATAPLEX.NET> <20130408143401.4088c3b2@anarchist> Message-ID: <61C5133E-A5E8-4243-A27A-8D42F52C5F84@NFSNet.org> Well, the Django ORM does run on python 3. So, even if it isn't "the best", it is a possibility. Richard On Apr 8, 2013, at 1:34 PM, Barry Warsaw wrote: > On Apr 08, 2013, at 07:02 AM, Richard Wackerbarth wrote: > >> Although we would like for MM3 to run on Python 3, I don't think that all of >> our dependency libraries are ready for that yet. I am running MM under >> Python 2.7 > > We're down to two dependencies: restish and storm. I took a crack at restish, > but got stuck. At Pycon, Thomi and I talked about another approach, and he > was taking another stab at it, but I haven't talked with him about since then. > > I suspect storm will never get ported. I've said before that if that's the > last dependency, we'll switch, probably back to SQLAlchemy as that's the only > Python 3 compatible ORM I'm aware of. > > Help certainly wouldn't be turned down. :) > > -Barry > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/richard%40nfsnet.org > > Security Policy: http://wiki.list.org/x/QIA9 From barry at list.org Mon Apr 8 21:02:21 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 8 Apr 2013 15:02:21 -0400 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: <61C5133E-A5E8-4243-A27A-8D42F52C5F84@NFSNet.org> References: <516228F5.9060600@zone12.com> <46D2A52B-6726-4165-8BA9-E1A0B6EAAF19@DATAPLEX.NET> <20130408143401.4088c3b2@anarchist> <61C5133E-A5E8-4243-A27A-8D42F52C5F84@NFSNet.org> Message-ID: <20130408150221.3f0875ae@anarchist> On Apr 08, 2013, at 01:59 PM, Richard Wackerbarth wrote: >Well, the Django ORM does run on python 3. So, even if it isn't "the best", >it is a possibility. Is it usable outside of Django? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From flo.fuchs at gmail.com Mon Apr 8 21:38:06 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Mon, 8 Apr 2013 21:38:06 +0200 Subject: [Mailman-Developers] Mailman participating in GSoC 2013 under the umbrella of the PSF Message-ID: Hi everyone, the list of organizations accepted for GSoC this year has been announced, but apparently google-melange.com suffers from a little "end of countdown" stress and only displays a small number of them. So a quick announcement for those of you currently watching that page: Like last year and the year before, Mailman will participate in GSoC under the umbrella of the Python Software Foundation! Yay...! (Of course it would have been even better had Mailman been accepted as a separate organization, but unfortunately that's not the case. Maybe next year!) So let's have another great three to four months of learning and coding and doing beautiful things! Cheers Florian From mark at msapiro.net Mon Apr 8 22:16:58 2013 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 8 Apr 2013 13:16:58 -0700 Subject: [Mailman-Developers] Like to participate in GSOC 2013 (GNUMailman) - Like some ideas proposed in the list (Django, Python) In-Reply-To: <20130408121927.1b9a1185@anarchist> Message-ID: Barry Warsaw wrote: > >IIRC, Mark was using Cygwin at one point. Yes. Actually, my main Mailman 2.1 development platform is Cygwin, but I gave up on trying to run MM 3 under Cygwin. I don't recall all the issues, but there were several around file names with unacceptable characters for Windows (there's one in MM 2.1, but it's easily worked around). >The thing about Windows is that we're using several *nix-isms in process >management and possibly in other places. I have no burning desire to try to >make these Windows compatible. The real problem with running Mailman on Windows, at least for real lists, is you need a viable mail server. It seems to me that most people who might consider running Mailman on Windows do not have a machine with a fixed IP with full circle DNS and a non-generic host name and preferably an IP not in a 'home use' block. These things are needed if you want large ISPs to accept your mail. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From pingou at pingoured.fr Mon Apr 8 19:39:43 2013 From: pingou at pingoured.fr (Pierre-Yves Chibon) Date: Mon, 08 Apr 2013 19:39:43 +0200 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: References: Message-ID: <1365442783.23084.8.camel@ambre.pingoured.fr> On Sat, 2013-04-06 at 15:03 +0530, Udit Saxena wrote: > 2. Web Posting Interface. I haven't really followed the GSoC ideas for MM3 this year but that line poped-up on my radar. Isn't this similar/overlapping to what HyperKitty already does? I don't think one would want to embed posting messages from the admin interface of mailman (postorious), so posting from an interface would have to deal with archives as well (since one might want to reply to an existing thread). That's the reasoning that brought me to the question above :) Did I miss something? Pierre From flo.fuchs at gmail.com Tue Apr 9 00:17:54 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Tue, 9 Apr 2013 00:17:54 +0200 Subject: [Mailman-Developers] GSOC 2013 : Authenticated REST-API in Postorius/Django In-Reply-To: References: <05BC1D67-31B0-41AA-82C2-8A47AB7AF7BD@NFSNet.org> Message-ID: Hi Rahul, 2013/4/6 Rahul Gaur > So , last night I tried my hand on django-rest-framework [0] and TastyPie > [1] as well. > > What I could figure out with my two quick and dirty hands on applications > of both the frameworks , the django-rest-framework is a bit more lengthy ( > but those few extra lines are for the best) when stacked up against > Tastypie. > > I wrote this quick django app and then I tried to write API's for that app > in both the framework.Both the frameworks have there own advantages lets > talk about that. Thanks for creating the two examples! I took a quick look at them and it looks like both frameworks are pretty easy to use if a resource is based on a typical Django model. The Mailman resources OTOH are not retrieved from a database but from another API -- Mailman's core REST API that is only served locally (postorius uses the mailman.client library to access that data). Which of the two frameworks would you consider more flexible if you want to get your data from anything else but a Django DB model (I *think* Tastypie has different classes for model resources and everythingelse-resources)? > *$ curl http://127.0.0.1:8080/formpost/ -H 'Accept:text/html' > > The Django-rest-frameworks API's returns web browsable representation of > the data by default :) Sounds nice! :-) > So , the confusion I am facing here is that the Postorius is already mature > and stable app and while playing with the two frameworks for API's what I > figured out is that if I opt for django-rest-framework , I would need to > write new Views ? > But if I go for TastyPie , I won't have to touch the existing Views and > write authenticated API's. > > Hence to implement authenticated API's , which one would be better ? Hard to say - judging by how they're used in the examples, both look good. But there are some more things you might want to consider before choosing one: Authentication methods Which auth methods are provided out of the box by the frameworks? And which one(s) would you like to support (OAuth(2), BasicAuth over SSL, ...)? I guess choosing and implementing authentication could become one of the major sub-tasks of that project. Support for Django 1.5 and Python3: At some point in the not so fare future mailman and postorius will be ported to Python3. Django 1.5 already supports Python3, so it would be a bonus point if the rest framework supports it, too (or if their developers are making efforts to do so...). Another idea (not sure if it's a good one): Maybe also take a closer look at Django's class-based views. They are capable of routing requests to the same URI to different dispatch methods, depending on the HTTP method. So that might be also be an interesting option. Cheers Florian From stephen at xemacs.org Tue Apr 9 01:27:43 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 09 Apr 2013 08:27:43 +0900 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: <1365442783.23084.8.camel@ambre.pingoured.fr> References: <1365442783.23084.8.camel@ambre.pingoured.fr> Message-ID: <87obdo7gcw.fsf@uwakimon.sk.tsukuba.ac.jp> Pierre-Yves Chibon writes: > On Sat, 2013-04-06 at 15:03 +0530, Udit Saxena wrote: > > 2. Web Posting Interface. > > Isn't this similar/overlapping to what HyperKitty already does? No. There's no reason why a web posting interface needs to interact with the archives; it can talk directly to Mailman core, and will need to do so for other features such as authentication, sister lists, and the like. You get archiving for free. > I don't think one would want to embed posting messages from the admin > interface of mailman (postorious), so posting from an interface would > have to deal with archives as well (since one might want to reply to an > existing thread). It could be as simple as embedding a button with an appropriate URL communicating the information needed to compose a message in the archive display interface. On the other hand, the posting interface to the archives could be a full separate subsystem, with a special- purpose browsing system oriented to rapidly selecting and yanking content from related messages. It might make sense to embed the interface in HyperKitty, which also has to deal with authentication, at least. But since posting and browsing are separate features I tend to favor creating an appropriate protocol for web posting, independent of the archive protocol and implementation. From terri at zone12.com Tue Apr 9 02:42:12 2013 From: terri at zone12.com (Terri Oda) Date: Mon, 08 Apr 2013 18:42:12 -0600 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: <87obdo7gcw.fsf@uwakimon.sk.tsukuba.ac.jp> References: <1365442783.23084.8.camel@ambre.pingoured.fr> <87obdo7gcw.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <516363E4.50108@zone12.com> On 13-04-08 5:27 PM, Stephen J. Turnbull wrote: > Pierre-Yves Chibon writes: > > On Sat, 2013-04-06 at 15:03 +0530, Udit Saxena wrote: > > > 2. Web Posting Interface. > > > > Isn't this similar/overlapping to what HyperKitty already does? > > No. There's no reason why a web posting interface needs to interact > with the archives; it can talk directly to Mailman core, and will need > to do so for other features such as authentication, sister lists, and > the like. You get archiving for free. I think Stephen covered most of it, but I just want to add that I had in mind that if we do the separate posting interface right, we might be able to use this as the beginnings of a way to integrate any existing forum software into Mailman, much like we want to gateway from newsgroups. That said... I'm totally ok with this particular project being entirely part of hyperkitty in the short-term, though, since it'll clearly be a nice fit there right now and goodness knows I don't want to deal with yet another django-standalone instance running just to post messages. ;) The only reason I didn't suggest this be part of hyperkitty in the project description is that when I was writing up project information, I didn't know if any of the hyperkitty team would have time to mentor. If one of the hyperkitty devs wants to take the lead on mentoring this project, that would work for me! Terri From rkw at DATAPLEX.NET Tue Apr 9 03:05:03 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Mon, 8 Apr 2013 20:05:03 -0500 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: <516363E4.50108@zone12.com> References: <1365442783.23084.8.camel@ambre.pingoured.fr> <87obdo7gcw.fsf@uwakimon.sk.tsukuba.ac.jp> <516363E4.50108@zone12.com> Message-ID: <3FEF54A8-21FB-4118-A3D8-3B0C6136CD19@DATAPLEX.NET> Let me suggest that it would be useful to have the student develop something which could act as a plug-in module for any website. If we assume that there is an optional archiver such as hyperkitty, there should be a mechanism to "seed" the submission (in-reply-to, quoted text, etc.). But, if the archiver is not present, the same submission mechanism could handle submissions for any website, with or without postorius present. The interface should address user authentication, using, when available, credentials that have already been provided. But it should also accept un-authenticated submissions and interact with the system to implement the list policies. It should be interesting to see just what the students propose. -- Richard On Apr 8, 2013, at 7:42 PM, Terri Oda wrote: > On 13-04-08 5:27 PM, Stephen J. Turnbull wrote: >> Pierre-Yves Chibon writes: >> > On Sat, 2013-04-06 at 15:03 +0530, Udit Saxena wrote: >> > > 2. Web Posting Interface. >> > >> > Isn't this similar/overlapping to what HyperKitty already does? >> >> No. There's no reason why a web posting interface needs to interact >> with the archives; it can talk directly to Mailman core, and will need >> to do so for other features such as authentication, sister lists, and >> the like. You get archiving for free. > > I think Stephen covered most of it, but I just want to add that I had in mind that if we do the separate posting interface right, we might be able to use this as the beginnings of a way to integrate any existing forum software into Mailman, much like we want to gateway from newsgroups. > > That said... > > I'm totally ok with this particular project being entirely part of hyperkitty in the short-term, though, since it'll clearly be a nice fit there right now and goodness knows I don't want to deal with yet another django-standalone instance running just to post messages. ;) The only reason I didn't suggest this be part of hyperkitty in the project description is that when I was writing up project information, I didn't know if any of the hyperkitty team would have time to mentor. If one of the hyperkitty devs wants to take the lead on mentoring this project, that would work for me! > > Terri > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/richard%40nfsnet.org > > Security Policy: http://wiki.list.org/x/QIA9 From markoupetr at gmail.com Tue Apr 9 08:49:26 2013 From: markoupetr at gmail.com (Peter Markou) Date: Tue, 9 Apr 2013 09:49:26 +0300 Subject: [Mailman-Developers] GSOC 2013 : Web Posting Interface Message-ID: Hello to all the members of the community. My name is Peter Markou and I'm currently running the last year of my university course. I'm deeply interested in implementing the "Web Posting Interface". Since I've maintained and integrated phpBB forum in a european project that I participated about 8 months ago(MUTW - Multinational Undergraduate Team Work http://mutw.praxisnetwork.eu/ ) I think I will be able to offer some additional features to make it a replacement for phpBB. Thanks in advance for your time and looking forward to discuss any further details. From pingou at pingoured.fr Tue Apr 9 17:23:24 2013 From: pingou at pingoured.fr (Pierre-Yves Chibon) Date: Tue, 09 Apr 2013 17:23:24 +0200 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: <87obdo7gcw.fsf@uwakimon.sk.tsukuba.ac.jp> References: <1365442783.23084.8.camel@ambre.pingoured.fr> <87obdo7gcw.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1365521004.23084.77.camel@ambre.pingoured.fr> On Tue, 2013-04-09 at 08:27 +0900, Stephen J. Turnbull wrote: > Pierre-Yves Chibon writes: > > On Sat, 2013-04-06 at 15:03 +0530, Udit Saxena wrote: > > > 2. Web Posting Interface. > > > > Isn't this similar/overlapping to what HyperKitty already does? > > No. There's no reason why a web posting interface needs to interact > with the archives; it can talk directly to Mailman core, and will need > to do so for other features such as authentication, sister lists, and > the like. You get archiving for free. How do you reply to a thread if you don't have access to the archives? > > I don't think one would want to embed posting messages from the admin > > interface of mailman (postorious), so posting from an interface would > > have to deal with archives as well (since one might want to reply to an > > existing thread). > > It could be as simple as embedding a button with an appropriate URL > communicating the information needed to compose a message in the > archive display interface. On the other hand, the posting interface > to the archives could be a full separate subsystem, with a special- > purpose browsing system oriented to rapidly selecting and yanking > content from related messages. > > It might make sense to embed the interface in HyperKitty, which also > has to deal with authentication, at least. But since posting and > browsing are separate features I tend to favor creating an appropriate > protocol for web posting, independent of the archive protocol and > implementation. If it is thought as a different system then I understand. I was more confused if that was thought as part of HyperKitty as I believe it already does that to some extend (I'll let Aur?lien specify to which extend). Thanks for clarifying, Pierre From avikpal.me at gmail.com Tue Apr 9 17:44:49 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Tue, 9 Apr 2013 21:14:49 +0530 Subject: [Mailman-Developers] Mailman not in GSOC'13 accepted organization Message-ID: Hello, can not see mailman in GSOC'13 accepted organization list, I am a bit baffled. did it make through this year? Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme From apoorvupreti at gmail.com Tue Apr 9 17:47:32 2013 From: apoorvupreti at gmail.com (Apoorv Upreti) Date: Tue, 9 Apr 2013 21:17:32 +0530 Subject: [Mailman-Developers] Mailman not in GSOC'13 accepted organization In-Reply-To: References: Message-ID: It's participating with Python as its parent organization. Check the gsoc13 page for Python. On Tue, Apr 9, 2013 at 9:14 PM, Avik Pal wrote: > Hello, > can not see mailman in GSOC'13 accepted organization list, I am a > bit baffled. did it make through this year? > > > > > Avik Pal > Bengal Engineering & Scieence University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/apoorvupreti%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- -Apoorv From terri at zone12.com Tue Apr 9 17:51:21 2013 From: terri at zone12.com (Terri Oda) Date: Tue, 09 Apr 2013 09:51:21 -0600 Subject: [Mailman-Developers] Mailman not in GSOC'13 accepted organization In-Reply-To: References: Message-ID: <516438F9.1030803@zone12.com> Just like last year, Mailman's going to be participating as a sub-organization under the Python Software Foundation. Unfortunately, this means that our name doesn't show up directly on the org list -- you'll need to click through from python to find us. I'm going to set up some tags so the sub-orgs will be more visible to those searching that list, but it probably won't get done 'till I'm finished work for the day. Terri On 04/09/2013 09:44 AM, Avik Pal wrote: > Hello, > can not see mailman in GSOC'13 accepted organization list, I am > a bit baffled. did it make through this year? > > > > > Avik Pal > Bengal Engineering & Scieence University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > From flo.fuchs at gmail.com Tue Apr 9 23:04:06 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Tue, 9 Apr 2013 23:04:06 +0200 Subject: [Mailman-Developers] Mailman/PSF GSoC Students: Next steps Message-ID: Hello prospective GSoC students, there are roughly four weeks left until the application deadline on May 03 2013. Sounds like a lot, but it time's running fast. ;-) The application period opens on April 22. Ideally, you should already have a pretty good understanding of the boundaries of your project proposal(s) around that time. So use this week and the next for further discussions, ask questions, do some reading. This will give you a better feel for what the overall work load might be (which is crucial for a healthy project timeline). Most important, don't worry about sounding stupid or not knowing something! There are several moving parts inside the Mailman software ecosystem and no-one expects you to grok everything right from the start. Once the application period has started, it's better to apply sooner than later. Don't worry if your application is unfinished -- it will be editable until the deadline (at which point it must, of course, be complete). But it's better to have it in the system early than to wait until the last minute. Just leave a note saying it's still in draft status. We will put an application template on the wiki to give you a rough outline of what we expect the applications to contain. Lastly, to avoid possible confusion: The mentoring organisation you have to apply with is the Python Software Foundation. Since the PSF handles a lot of sub projects like Mailman, it'll be a good idea to put "Mailman" in the title of your application so we can find it and nothing gets missed. Cheers Florian From chavarria1991 at gmail.com Wed Apr 10 01:55:46 2013 From: chavarria1991 at gmail.com (=?ISO-8859-1?Q?Marcos_Chavarr=EDa_Teijeiro?=) Date: Wed, 10 Apr 2013 01:55:46 +0200 Subject: [Mailman-Developers] OpenPGP Integration on GSoC Message-ID: Hi all, My name is Marcos Chavarr?a and I'm a fourth year computer science student from Galicia, Spain. I'm interested in the OpenPGP integration project for this year SoC. I have experience working with python in several university projects and I have some knowledge about crypto primitives (I made Coursera Cryptography course[1]). The problem is that I'm not sure if I understand the idea. This is how I see it: 1) Users summit their public key to MailMan server when they register to mail list. 2) The user can get MailMan Server public key 3) When an user want to post a message they both sign and encrypt this message. They encrypt the message using MailMan public key. Then the message is sent to MailMan Server. 4) MailMan decrypt the received message and check if the sign is correct (with the stored public user public key). If the sign is correct, it sends a message to every mail-list subscripter encrypted with each user public key. 5) The other user receive the email and decrypt it. Is this correct? Best Regards, Marcos Chavarr?a [1] https://www.coursera.org/course/crypto From raj.abhilash1 at gmail.com Wed Apr 10 03:30:00 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Wed, 10 Apr 2013 07:00:00 +0530 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <87fvz28lyf.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5160ACF8.1050108@fifthhorseman.net> <87fvz28lyf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sun, Apr 7, 2013 at 7:46 PM, Stephen J. Turnbull wrote: > Abhilash Raj writes: > > > Well what i want to make it is that whenever a user sends a mail to the > > list it should be singed with his private key so that it can be verified > > against his public that he uploads if he wants permissions to post in > the > > list. > > You mean that the user should sign it himself (or with the help of his > mail client), is that correct? Yes, the user should sign it himself. I am not sure about how it would be done though. > > > As the message is received by mailman its signature is verified and > > then its encrypted and sent to each person, wherein those who > > haven't uploaded their key will also receive an unencrypted > > copy(with a probability that it may not be intended for them or not > > authentic mail). > > I don't understand the use case for having both encrypted and > unencrypted copies distributed. Is the encryption intended to be > merely authentication? But what Mailman has is by definition the > subscriber's public key; anybody might have that. It *could* be kept > secret, but I think that's not so easy to prove. > > I would have imagined that maybe Mailman would resign using its own > private key, to authenticate the list, and testify that it had > authenticated the sender. > > I also don't understand what you mean by "not authentic mail". The > original signature proves it authentic. The subscribers may > not have the appropriate to key to verify, but in that case I don't > see why they would want to delegate it to Mailman. > > I think you have a difficult task in merely specifying what you want > this system to do. That's likely to be a couple orders of magnitude > harder than the implementation! > > > Yes, this was on the top of my mind while trying to attempt this > > project. I learned about key-servers. I think we could setup one > > wherein all the public key would be stored that are uploaded by > > users and retrieved when needed. > > But who watches the watcher? That is, what does the keyserver need to > know about the key's owner, and how does the candidate subscriber > prove it to the keyserver? > > I think there are lots of use cases for integrating mailing list > managers into the public key infrastructure, but you need to be > careful to specify them. I think you probably should start with > simple use cases, like proving subscriber identity to the mailing list > manager, eg for anti-spam purposes.[1] > > I gave a thought and yes some parts of it doesn't actually makes sense. Instead for proving a subscribers identity to a list manager we could add add a setting to accept messages only from registered signatures. Each subscriber add his public key when he subscribes to the list( or when settings are changed to accept mails with only registered signature). This could also help in spam reduction as only mails with registered users(with registered keys) would be distributed among the list subscribers. Can you please point me in some direction to learn about the various possible ways to sign a mail and/or encrypt it. Also i think adding the key as a new column against the email in the list of subscriber would do the work. I haven't worked with postorius but i have experience with django so i think some ui can also be added in postorius to manage this although this is just and idea which i think i can expand in a few days as I am working on postorius. > Footnotes: > [1] Even that is not a sure winner, since most users will not know > how to do this for themselves. So it will have to be integrated into > clients, which themselves might be infected by a virus. > > -- Abhilash Raj From stephen at xemacs.org Wed Apr 10 07:22:18 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 10 Apr 2013 14:22:18 +0900 Subject: [Mailman-Developers] GSOC 2013 - Introduction and Project Discussion In-Reply-To: <1365521004.23084.77.camel@ambre.pingoured.fr> References: <1365442783.23084.8.camel@ambre.pingoured.fr> <87obdo7gcw.fsf@uwakimon.sk.tsukuba.ac.jp> <1365521004.23084.77.camel@ambre.pingoured.fr> Message-ID: <87bo9n6jud.fsf@uwakimon.sk.tsukuba.ac.jp> Pierre-Yves Chibon writes: > How do you reply to a thread if you don't have access to the > archives? The archives provide a URL to the web interface instead of a mailto URL. > If it is thought as a different system then I understand. I was more > confused if that was thought as part of HyperKitty as I believe it > already does that to some extend (I'll let Aur?lien specify to which > extend). Sure. From stephen at xemacs.org Wed Apr 10 07:41:21 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 10 Apr 2013 14:41:21 +0900 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <5160ACF8.1050108@fifthhorseman.net> <87fvz28lyf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87a9p76iym.fsf@uwakimon.sk.tsukuba.ac.jp> Abhilash Raj writes: > Can you please point me in some direction to learn about the various > possible ways to sign a mail and/or encrypt it. Basically that's going to be MUA-dependent. There are standards for this (prominently S/MIME aka RFC 5751), but whether MUAs implement it is MUA-specific. Also, S/MIME is not the same as using OpenPGP (I guess that OpenPGP can be used to implement it, but I doubt that most systems using OpenPGP actually conform to S/MIME). I suspect that many webmail programs and Windows MUAs do not support OpenPGP (webmail programs generally don't support any form of secure mail AFAIK). Other important RFCs include PKCS (RFC 2315) and Security Multiparts for MIME (RFC 1847). (Do check those references before implementing them: I haven't followed this field that closely for several years, and several of them are probably superseded by now.) > Also i think adding the key as a new column against the email in > the list of subscriber would do the work. I still think you're getting ahead of yourself. What work are you talking about? Just getting keys stored in the subscriber database isn't much help if we haven't decided how we are going to use them. From mgill25 at outlook.com Wed Apr 10 10:00:37 2013 From: mgill25 at outlook.com (Manish Gill) Date: Wed, 10 Apr 2013 13:30:37 +0530 Subject: [Mailman-Developers] Regarding Authentication of REST API Message-ID: For the GSoC REST API project, I've been wondering about how authentication would work. OAuth is a way to go if we want authenticated/signed requests. I have a few questions regarding that. - Will Mailman core become an OAuth provider, with Postorius/API being the consumers? - If the answer to the above is no, is the plan to support populer OAuth providers like Facebook/Twitter ? (If not, can you guys please explain how would the authentication protocol really work?) - Since Postorius is already using Mozilla Persona, can that also be used to provide authentication to API clients? - Am I over-thinking this? :) Thanks! From joostvb-mailman-developers at mdcc.cx Wed Apr 10 11:39:12 2013 From: joostvb-mailman-developers at mdcc.cx (Joost van =?utf-8?Q?Baal-Ili=C4=87?=) Date: Wed, 10 Apr 2013 11:39:12 +0200 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration Message-ID: <20130410093912.GY16254@beskar.mdcc.cx> Hi Abhilash Raj, Abhilash Raj raj.abhilash1 at gmail.com schreef: >On Sun, Apr 7, 2013 at 4:47 AM, Daniel Kahn Gillmor >wrote: >> On 04/06/2013 06:53 PM, Paul Wise wrote: >> > On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: >> > >> >> I am a undergrad student interested in OpenPGP integration in mailman >> >> as a GSOC project this summer. >> neat, i'm glad to hear it! Be aware however: it's not an easy task, as Daniel Kahn Gillmor pointed out. >> > I'm not sure about the scope of your project but you may want to >> > review some prior efforts: >> > >> > http://schleuder2.nadir.org/ >> > http://www.synacklabs.net/projects/crypt-ml/ >> >> see also: >> >> http://non-gnu.uvt.nl/mailman-pgp-smime/ >> http://sels.ncsa.illinois.edu/ >> >> > My pet favourite feature from the lurker mail archiver is showing >> > photos from OpenPGP keys in the archive pages. >> >Thanks for these links. I am currently going through these projects to >figure out the implementation part of the OpenPGP into mailman. Also trying >to use the mailman-php-smime patch to figure out how it is implemented. The Mailman Secure List Server Patch hasn't been touched since 2010-09. It's a patch for mailman-2.1.15, not for the development branch. However, studying it will surely give you some inspiration. Some code might be reusable too. If you'd like to discuss details of this patch, you're invited to join the list at ssls-dev at ulm.ccc.de. I'd be glad to help you dealing with the work. Bye, Joost -- http://mdcc.cx/ x http://ad1810.com/ From fte08eas at student.lu.se Wed Apr 10 11:05:34 2013 From: fte08eas at student.lu.se (Elias Assarsson) Date: Wed, 10 Apr 2013 11:05:34 +0200 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project Message-ID: <51652B5E.60200@student.lu.se> Hi, My name is Elias Assarsson and I am interested in doing a GSoC project for Mailman. The idea I found most interesting is Scripts for migrating from Mailman 2.1 to Mailman 3 http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013#GoogleSummerofCode2013-ScriptsformigratingfromMailman21toMailman3 . Where can I more information about this idea? I do not know if it is a good idea but I am thinking that using Augeas http://augeas.net might be a way to handle the migration of configuration from one format to another. For instance I found the following https://www.redhat.com/archives/augeas-devel/2012-January/msg00050.html when researching the idea. I have no experience with Augeas but this might be a good opportunity to learn it. I do however have experience with Python (and other scripting languages such as Ruby). Any input or help in developing this idea would be welcome. Some things about me. I am a GNU/Linux user for almost ten year and a student in computer science courses in Sweden. If you want to know more, please ask. Regards, Elias Assarsson From iampratiksarkar at gmail.com Wed Apr 10 16:25:32 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Wed, 10 Apr 2013 07:25:32 -0700 Subject: [Mailman-Developers] Fwd: Google summer of code 2013 In-Reply-To: References: Message-ID: I am interested to work in the "Anti-spam/anti-abuse" in the Mailman project for this year's Google summer of code.How can I contact the respective mentors? Regards, Pratik From pabs at debian.org Wed Apr 10 16:46:50 2013 From: pabs at debian.org (Paul Wise) Date: Wed, 10 Apr 2013 22:46:50 +0800 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: <51652B5E.60200@student.lu.se> References: <51652B5E.60200@student.lu.se> Message-ID: In addition to Augeas, I would encourage you to look at Config::Model. I think you will need the pair of them to do upgrades. There are a number of blog posts on Planet Debian about it and I'm sure the author would be willing to help out. http://planet-search.debian.org/cgi-bin/search.cgi?terms=Config::Model -- bye, pabs http://wiki.debian.org/PaulWise From fte08eas at student.lu.se Wed Apr 10 19:37:23 2013 From: fte08eas at student.lu.se (Elias Assarsson) Date: Wed, 10 Apr 2013 19:37:23 +0200 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: References: <51652B5E.60200@student.lu.se> Message-ID: <5165A353.8090208@student.lu.se> Thanks. I will have a look at Config::Model too. Any pointers to good sample configuration files to use in trying to evaluate the mentioned tools? 2013-04-10 16:46, Paul Wise skrev: > In addition to Augeas, I would encourage you to look at Config::Model. > I think you will need the pair of them to do upgrades. There are a > number of blog posts on Planet Debian about it and I'm sure the author > would be willing to help out. > > http://planet-search.debian.org/cgi-bin/search.cgi?terms=Config::Model From barry at list.org Wed Apr 10 20:14:18 2013 From: barry at list.org (Barry Warsaw) Date: Wed, 10 Apr 2013 14:14:18 -0400 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: <51652B5E.60200@student.lu.se> References: <51652B5E.60200@student.lu.se> Message-ID: <20130410141418.1e614d12@anarchist> On Apr 10, 2013, at 11:05 AM, Elias Assarsson wrote: >I do not know if it is a good idea but I am thinking that using Augeas >http://augeas.net might be a way to handle the migration of configuration >from one format to another. Definitely investigate these tools, although my suspicion is that they won't help, or at least won't help enough to make accepting a new dependency worth it. There are a few problems to consider: * MM2's configuration file is a Python file which really must be imported in order to get a valid set of values. MM3's configuration file is a stack of .ini-style files. * There is not always a 1-to-1 correspondence between MM2 values and MM3 values. Some configurations have been merged, some removed, etc. You will pretty much have to go through each set of MM2 variables and decide if and how to transform them into something meaningful for MM3. * The configuration files are only for system-wide settings. We really also want to be able to upgrade MM2 lists to MM3 lists, and that involves unpickling the config.db state and again, mapping the MM2 variables to MM3 variables, which are stored in a database. * What about importing archives? There is some moderate beginnings in the 3.0 tree, but none of it is functional in all likelihood. Take a look at src/mailman/bin/export.py and src/mailman/commands/cli_import.py. Cheers, -Barry From rkw at DATAPLEX.NET Wed Apr 10 19:58:04 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Wed, 10 Apr 2013 12:58:04 -0500 Subject: [Mailman-Developers] Google summer of code 2013 In-Reply-To: References: Message-ID: <9B0E9DBC-73D4-4001-8638-0834FABE5D54@DATAPLEX.NET> As has been suggested previously, at this point it is best that you simply start the discussion here on the mailing list. All of the mentors read the list and technical discussions should involve the entire community. On Apr 10, 2013, at 9:25 AM, Pratik Sarkar wrote: > I am interested to work in the "Anti-spam/anti-abuse" in the Mailman > project for this year's Google summer of code.How can I contact the > respective mentors? > Regards, > Pratik > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/richard%40nfsnet.org > > Security Policy: http://wiki.list.org/x/QIA9 From iampratiksarkar at gmail.com Wed Apr 10 21:44:08 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Wed, 10 Apr 2013 12:44:08 -0700 Subject: [Mailman-Developers] Fwd: Eta patha In-Reply-To: References: Message-ID: Hi all, I am a 2nd yr undergrad student studying CS at Bengal Engineering and Science University, India. This is the first time I am participating in Google Summer of Code. And I am very interested to contribute to machine learning or NLP related projects. I went through the Python projects of gsoc 2013 and I found the anti-spam/anti-abuse filter of Mailman very interesting. I had some own ideas regarding the project.I want to integrate/use external NLP toolkits like Lingpipe and LIBSVM. Classification of the text will be done on an N-gram Language Model and a Support Vector Machine (I would prefer LIBSVM), with the "localized knowledge" (which would be used as the training set for the classifier) to help in identifying the abuse/spam. Adding a feedback procedure to the classifier might also be helpful as it will help us to improve classification and the classifier can update itself with previously classified spams/abuse and hence the program can filter latest spams/abuses without any update, on programmer's part. About me: 1. About 5 years of programming experience mainly in Java,python, C and C++ and a bit of PHP and LISP. I am a part of our college Software and programming club. 2. I qualified the Zonal Informatics olympiad 2010-11. I regularly participate in online coding competitions and hackathons. 3. Developed a email client in python. 4. I have some prior experience in machine learning and natural language processing. I participated in Twitminer 2013 (IISC Bangalore) where our team (BEing) came 6th. In the last few months, I have been working on the sentiment analysis of tweets, based on Language Models of Lingpipe Toolkit of Java.and LIBSVM to analyze the efficiency of classification. Looking forward to hear back from all of you! Pratik Sarkar From iampratiksarkar at gmail.com Wed Apr 10 22:36:31 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Wed, 10 Apr 2013 13:36:31 -0700 Subject: [Mailman-Developers] Introduction and some ideas Message-ID: Hi all, I am a 2nd yr undergrad student studying CS at Bengal Engineering and Science University, India. This is the first time I am participating in Google Summer of Code. And I am very interested to contribute to machine learning or NLP related projects. I went through the Python projects of gsoc 2013 and I found the anti-spam/anti-abuse filter of Mailman very interesting. I had some own ideas regarding the project. I want to integrate/use external NLP toolkits like Lingpipe and LIBSVM. Classification of the text will be done on an N-gram Language Model and a Support Vector Machine (I would prefer LIBSVM), with the "localized knowledge" (which would be used as the training set for the classifier) to help in identifying the abuse/spam. Adding a feedback procedure to the classifier might also be helpful as it will help us to improve classification and the classifier can update itself with previously classified spams/abuse and hence the program can filter latest spams/abuses without any update, on programmer's part. About me: 1. About 5 years of programming experience mainly in Java,python, C and C++ and a bit of PHP and LISP. I am a part of our college Software and programming club. 2. I qualified the Zonal Informatics olympiad 2010-11. I regularly participate in online coding competitions and hackathons. 3. Developed a email client in python. 4. I have some prior experience in machine learning and natural language processing. I participated in Twitminer 2013 (IISC Bangalore) where our team (BEing) came 6th. In the last few months, I have been working on the sentiment analysis of tweets, based on Language Models of Lingpipe Toolkit of Java.and LIBSVM to analyze the efficiency of classification. Looking forward to hear back from all of you! Pratik Sarkar From dkg at fifthhorseman.net Thu Apr 11 04:04:04 2013 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Wed, 10 Apr 2013 22:04:04 -0400 Subject: [Mailman-Developers] OpenPGP Integration on GSoC In-Reply-To: References: Message-ID: <51661A14.1050806@fifthhorseman.net> On 04/09/2013 07:55 PM, Marcos Chavarr?a Teijeiro wrote: > The problem is that I'm not sure if I understand the idea. This is how I > see it: > 1) Users summit their public key to MailMan server when they register to > mail list. > 2) The user can get MailMan Server public key > 3) When an user want to post a message they both sign and encrypt this > message. They encrypt the message using MailMan public key. Then the > message is sent to MailMan Server. > 4) MailMan decrypt the received message and check if the sign is correct > (with the stored public user public key). If the sign is correct, it sends > a message to every mail-list subscripter encrypted with each user public > key. > 5) The other user receive the email and decrypt it. > > Is this correct? This sounds like a reasonable proposal, though there are potentially a lot of gotchas in such an implementation (in particular, keyring management, and dealing sensibly with cryptographic failures are two rough spots that you probably need to tihnk more about). Have you looked at schleuder? In my experience, schleuder is the most widely-used mailing list software that maps to the model you describe, so learning from their experiences and figuring out why they made the implementation decisions they did would probably be helpful: http://schleuder2.nadir.org/ You might also want to compare notes with Abhilash Raj (who has been posting to this list), since y'all seem to be interested in similar topics. all the best, --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1027 bytes Desc: OpenPGP digital signature URL: From joostvb-mailman-developers at mdcc.cx Thu Apr 11 06:19:22 2013 From: joostvb-mailman-developers at mdcc.cx (Joost van =?utf-8?Q?Baal-Ili=C4=87?=) Date: Thu, 11 Apr 2013 06:19:22 +0200 Subject: [Mailman-Developers] OpenPGP Integration on GSoC In-Reply-To: <51661A14.1050806@fifthhorseman.net> References: <51661A14.1050806@fifthhorseman.net> Message-ID: <20130411041922.GI25275@beskar.mdcc.cx> Hi Marcos, On Wed, Apr 10, 2013 at 10:04:04PM -0400, Daniel Kahn Gillmor wrote: > On 04/09/2013 07:55 PM, Marcos Chavarr?a Teijeiro wrote: > > > The problem is that I'm not sure if I understand the idea. This is how I > > see it: > > 1) Users summit their public key to MailMan server when they register to > > mail list. > > 2) The user can get MailMan Server public key > > 3) When an user want to post a message they both sign and encrypt this > > message. They encrypt the message using MailMan public key. Then the > > message is sent to MailMan Server. > > 4) MailMan decrypt the received message and check if the sign is correct > > (with the stored public user public key). If the sign is correct, it sends > > a message to every mail-list subscripter encrypted with each user public > > key. > > 5) The other user receive the email and decrypt it. > > > > Is this correct? > > This sounds like a reasonable proposal, though there are potentially a > lot of gotchas in such an implementation (in particular, keyring > management, and dealing sensibly with cryptographic failures are two > rough spots that you probably need to tihnk more about). > > Have you looked at schleuder? One of the issues you'd have to think about is how to deal with this: I am Joost van Baal-Ili?. I create a PGP keypair with ID Barry Warsaw. I sent the public key to the list server. I sent a mail, signed with the Barry-key, encrtypted to the listkey, with From: Barry's email address, to the list. The listserver now distributes it to the lists subscribers, yes? The list subscribers will believe the message is from Barry. There's more than 1 way to solve this problem. You'd have to pick one solution. Bye, Joost -- Perfection in design is achieved not when there is nothing left to add, but rather when there is nothing left to take away. --Antoine de Saint-Exupery From sreyanth at gmail.com Thu Apr 11 06:45:16 2013 From: sreyanth at gmail.com (Sreyanth) Date: Thu, 11 Apr 2013 10:15:16 +0530 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion Message-ID: Hi all! I am interested to participate in GSoC this year and would like to choose GNU Mailman ?under PSF ? for it. I have gone through the proposed ideas and I would like people to tell if they are feasible for one summer! 1. Boilerplate stripper AND Better content-filtering / handling error messages. 2. No-logging mode. 3. Anti-spam / anti-abuse in Mailman. 4. My own project idea: Mining the list logs and recognize interesting patterns for better enhancements (the admin need not have data mining experience) By far, I am looking to duplicate bugs on my comp and am aiming at writing a patch or two before the student deadline, so that I have much more time to work on my application. # Bragging mode ON. You may want to skip this. And, let me introduce myself. I am Sreyantha Chary (lets make it Sreyanth), a 3rd year undergrad majoring in Computer Engineering at the National Institute of Technology Karnataka. I wont consider myself an expert in python programming, but yeah, I am good enough to work on intermediate projects. I have used Django for my university projects and loved it. Research wise, I am into Machine Learning, Information Retrieval and Data Mining. And coding wise, I try crazy stuff, sometimes just to check if I remember anything from the documentation! ?# Bragging mode OFF.? People, please let me know your take on the projects I am interested in. How far is my proposed idea feasible? Thanks in advance. Thanks for your time! * ?Thanks and regards * * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From mark at msapiro.net Thu Apr 11 07:45:10 2013 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 10 Apr 2013 22:45:10 -0700 Subject: [Mailman-Developers] OpenPGP Integration on GSoC In-Reply-To: References: Message-ID: <51664DE6.5050702@msapiro.net> Marcos Chavarr?a Teijeiro wrote: > 4) MailMan decrypt the received message and check if the sign is correct > (with the stored public user public key). If the sign is correct, it sends > a message to every mail-list subscripter encrypted with each user public > key. As Stephen suggests in another thread at , Mailman should resign with its own private key to verify that it has accepted the incoming post and that the post was signed by a list member -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From stefan.schlott at ulm.ccc.de Thu Apr 11 09:23:35 2013 From: stefan.schlott at ulm.ccc.de (Stefan Schlott) Date: Thu, 11 Apr 2013 09:23:35 +0200 Subject: [Mailman-Developers] OpenPGP Integration on GSoC In-Reply-To: <20130411041922.GI25275@beskar.mdcc.cx> References: <51661A14.1050806@fifthhorseman.net> <20130411041922.GI25275@beskar.mdcc.cx> Message-ID: <516664F7.8070407@ulm.ccc.de> On 11.04.2013 06:19, Joost van Baal-Ili? wrote: > I am Joost van Baal-Ili?. I create a PGP keypair with ID Barry Warsaw. I sent > the public key to the list server. I sent a mail, signed with the Barry-key, > encrtypted to the listkey, with From: Barry's email address, to the list. > The listserver now distributes it to the lists subscribers, yes? The list > subscribers will believe the message is from Barry. You would have to do some key confirmation, just like you have to click a mail confirmation link upon subscription. Next problem: Mailman will have to decrypt the message and re-encrypt it for each recipient. This also strips the signature of the original sender. How do you show to the recipients that the original message was signed (in a way which cannot be forged by any other sender)? Generally speaking PGP support would be great, the efforts Joost and I made about 10 years ago never made it beyond alpha (or beta at best) stadium... Stefan. From joostvb-mailman-developers at mdcc.cx Thu Apr 11 09:50:49 2013 From: joostvb-mailman-developers at mdcc.cx (Joost van =?utf-8?Q?Baal-Ili=C4=87?=) Date: Thu, 11 Apr 2013 09:50:49 +0200 Subject: [Mailman-Developers] OpenPGP Integration on GSoC In-Reply-To: <516664F7.8070407@ulm.ccc.de> References: <51661A14.1050806@fifthhorseman.net> <20130411041922.GI25275@beskar.mdcc.cx> <516664F7.8070407@ulm.ccc.de> Message-ID: <20130411075049.GG16254@beskar.mdcc.cx> Hi (and hi Stefan!), On Thu, Apr 11, 2013 at 09:23:35AM +0200, Stefan Schlott wrote: > On 11.04.2013 06:19, Joost van Baal-Ili? wrote: > > > I am Joost van Baal-Ili?. I create a PGP keypair with ID Barry Warsaw. I sent > > the public key to the list server. I sent a mail, signed with the Barry-key, > > encrtypted to the listkey, with From: Barry's email address, to the list. > > The listserver now distributes it to the lists subscribers, yes? The list > > subscribers will believe the message is from Barry. > > You would have to do some key confirmation, just like you have to click > a mail confirmation link upon subscription. > > Next problem: Mailman will have to decrypt the message and re-encrypt it > for each recipient. This also strips the signature of the original > sender. Not necessarily, iirc. > How do you show to the recipients that the original message was > signed (in a way which cannot be forged by any other sender)? > > Generally speaking PGP support would be great, the efforts Joost and I > made about 10 years ago never made it beyond alpha (or beta at best) > stadium... ACK. Bye, Joost From markoupetr at gmail.com Thu Apr 11 11:23:04 2013 From: markoupetr at gmail.com (Peter Markou) Date: Thu, 11 Apr 2013 12:23:04 +0300 Subject: [Mailman-Developers] Setting up a VM. Message-ID: Hello again community. Two days ago I made a sort introduction of my self and stated that I'm interested in implementing the "Web Posting Interface" project(through this e-mail - http://www.mail-archive.com/mailman-developers%40python.org/msg13451.html). In order to getting started with Mailman development I went through the equivalent quote(here - http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013). Since I'm able to consume the services described here : https://okeanos.grnet.gr/services/cyclades/ , I would like to setup a VM and also make it available for future developers. If you're interested in this, poke me to arrange a meeting in IRC and discuss any further details. Best regards, Peter Markou. From fte08eas at student.lu.se Thu Apr 11 14:22:15 2013 From: fte08eas at student.lu.se (Elias Assarsson) Date: Thu, 11 Apr 2013 14:22:15 +0200 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: <20130410141418.1e614d12@anarchist> References: <51652B5E.60200@student.lu.se> <20130410141418.1e614d12@anarchist> Message-ID: <5166AAF7.2060805@student.lu.se> Thanks for an informative answer! 2013-04-10 20:14, Barry Warsaw skrev: > Definitely investigate these tools, although my suspicion is that they won't > help, or at least won't help enough to make accepting a new dependency worth > it. > > There are a few problems to consider: > > * MM2's configuration file is a Python file which really must be imported in > order to get a valid set of values. MM3's configuration file is a stack of > .ini-style files. I am trying to find and understand the configuration files so that I know what that that needs to be migrated and to what form. Is the MM2 configuration you refer to mainly Mailman/mm_cfg.py and MM3 configuration files src/mailman/config/* and /src/mailman//*? > * There is not always a 1-to-1 correspondence between MM2 values and MM3 > values. Some configurations have been merged, some removed, etc. You will > pretty much have to go through each set of MM2 variables and decide if and > how to transform them into something meaningful for MM3. > > * The configuration files are only for system-wide settings. We really also > want to be able to upgrade MM2 lists to MM3 lists, and that involves > unpickling the config.db state and again, mapping the MM2 variables to MM3 > variables, which are stored in a database. Given the two points above it seems that handling the migration from within Python is the best choice (rather than using Augeas which is C based or Config::Model which is Perl based). Obviously one wants to use whatever feature of Python that can ease the process. You seem to be using configparser in MM3. I dunno if there are any other Python tool one should look at in helping migration. Maybe one should investigate this further. > * What about importing archives? I have tried to find information on how archives are stored in MM2 and MM3 but failed to find any. What is a good source to learn about this? > There is some moderate beginnings in the 3.0 tree, but none of it is > functional in all likelihood. Take a look at src/mailman/bin/export.py > and src/mailman/commands/cli_import.py. So those are the files that are supposed to handle migration for which the project is to make them handle a complete migration from MM2 to MM3? On another note I have been able to setup a web UI although it took far longer than 5 minutes as I struggled with problems due to having both Python 2.7 and 3.2 on my system. From Richard at Damon-Family.org Thu Apr 11 14:35:17 2013 From: Richard at Damon-Family.org (Richard Damon) Date: Thu, 11 Apr 2013 08:35:17 -0400 Subject: [Mailman-Developers] OpenPGP Integration on GSoC In-Reply-To: <516664F7.8070407@ulm.ccc.de> References: <51661A14.1050806@fifthhorseman.net> <20130411041922.GI25275@beskar.mdcc.cx> <516664F7.8070407@ulm.ccc.de> Message-ID: <5166AE05.9060000@Damon-Family.org> On 4/11/13 3:23 AM, Stefan Schlott wrote: > On 11.04.2013 06:19, Joost van Baal-Ili? wrote: > >> I am Joost van Baal-Ili?. I create a PGP keypair with ID Barry Warsaw. I sent >> the public key to the list server. I sent a mail, signed with the Barry-key, >> encrtypted to the listkey, with From: Barry's email address, to the list. >> The listserver now distributes it to the lists subscribers, yes? The list >> subscribers will believe the message is from Barry. > You would have to do some key confirmation, just like you have to click > a mail confirmation link upon subscription. > > Next problem: Mailman will have to decrypt the message and re-encrypt it > for each recipient. This also strips the signature of the original > sender. How do you show to the recipients that the original message was > signed (in a way which cannot be forged by any other sender)? > > > Generally speaking PGP support would be great, the efforts Joost and I > made about 10 years ago never made it beyond alpha (or beta at best) > stadium... > > > Stefan. > Decrypting and re-encrypting shouldn't break signatures as the sender should First sign the unencrypted message, and then encrypt it. The signature can then be passed on in the re-encrypted message, and people can do their verification of the signature. It is up to each recipient to decide how well they trust the identity of the sender. Digital keys do NOT naturally verify the identity of the sender, the verify that the sender is a possessor of the signing key, and it is the web of trust on the key management side that connects that with an individual identity. Also, re-encrypting to each recipient isn't as big of a job as it might seem, as actually what happens is a session key is made, and this is used to encrypt the message, the the session key is encrypted with the recipients public-key, so only this last piece needs to be done per recipient. You probably need to send copies individually, or every message will have information about who is subscribed to the list. -- Richard Damon From stefan.schlott at ulm.ccc.de Thu Apr 11 15:13:34 2013 From: stefan.schlott at ulm.ccc.de (Stefan Schlott) Date: Thu, 11 Apr 2013 15:13:34 +0200 Subject: [Mailman-Developers] OpenPGP Integration on GSoC In-Reply-To: <5166AE05.9060000@Damon-Family.org> References: <51661A14.1050806@fifthhorseman.net> <20130411041922.GI25275@beskar.mdcc.cx> <516664F7.8070407@ulm.ccc.de> <5166AE05.9060000@Damon-Family.org> Message-ID: <5166B6FE.4080107@ulm.ccc.de> On 11.04.2013 14:35, Richard Damon wrote: >> Next problem: Mailman will have to decrypt the message and re-encrypt it >> for each recipient. This also strips the signature of the original >> sender. How do you show to the recipients that the original message was >> signed (in a way which cannot be forged by any other sender)? > Decrypting and re-encrypting shouldn't break signatures as the sender > should First sign the unencrypted message, and then encrypt it. The > signature can then be passed on in the re-encrypted message, and people > can do their verification of the signature. True, the PGP file structure encapsulates the signature within the encryption (in contrast to S/MIME, which does it vice versa). But the standard PGP binary will strip both in one step, so keeping the signature won't work out of the box (at least I didn't manage to do that, I'd be really interested how to do that - would be useful for searchable mail archives). Stefan. From stephen at xemacs.org Thu Apr 11 17:58:50 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 12 Apr 2013 00:58:50 +0900 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: <5166AAF7.2060805@student.lu.se> References: <51652B5E.60200@student.lu.se> <20130410141418.1e614d12@anarchist> <5166AAF7.2060805@student.lu.se> Message-ID: <87ppy15a9x.fsf@uwakimon.sk.tsukuba.ac.jp> Elias Assarsson writes: > On another note I have been able to setup a web UI although it took far > longer than 5 minutes as I struggled with problems due to having both > Python 2.7 and 3.2 on my system. Did you try a virtualenv? That usually helps with such problems. From stephen at xemacs.org Thu Apr 11 18:44:14 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 12 Apr 2013 01:44:14 +0900 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: References: Message-ID: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> Sreyanth writes: > 3. Anti-spam / anti-abuse in Mailman. A couple of people have mentioned anti-spam, and it's a frequently requested feature. Nevertheless, I don't think we should spend Google money and mentor time on it. 1. Mailman is the wrong place to do filtering. It's equally effective, normally covers more messages, and is somewhat more efficient in resource usage to do it at the MTA. 2. Any new algorithms *should* be made available at the MTA level where they can be best put to use by more people. This implies something that either plugs into existing filters (such as spamassassin) or MTAs (ie, milters) rather than a Handler. 3. Adapting existing filters is generally pretty trivial: you write a 10-line custom Handler that pipes it to an external process. This isn't big enough for a GSoC project. 4. To the extent that new algorithms are involved, I have doubts that Mailman mentors have the kind of expertise needed to really help with such a project (I could be wrong, but I certainly don't know much about that kind of text processing, and I don't know that anybody else in Mailman has expertise in it). On the other hand, I don't know which project in GSoC would be a better place for it. It's possible to argue that Mailman is a reasonable place for it, but IMHO we probably shouldn't. Regarding anti-abuse, we would like to do something about problems like backscatter. However, I have to wonder how much *code* (vs *specification* and *design*) is needed for those problems. If the project is really spec-heavy, it's probably not really what Google has in mind (based on comments on the mentors' list, not on any official Google pronouncements, though). From terri at zone12.com Thu Apr 11 21:52:44 2013 From: terri at zone12.com (Terri Oda) Date: Thu, 11 Apr 2013 13:52:44 -0600 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <5167148C.4000701@zone12.com> On 13-04-11 10:44 AM, Stephen J. Turnbull wrote: > 1. Mailman is the wrong place to do filtering. It's equally > effective, normally covers more messages, and is somewhat more > efficient in resource usage to do it at the MTA. > 2. Any new algorithms*should* be made available at the MTA level > where they can be best put to use by more people. This implies > something that either plugs into existing filters (such as > spamassassin) or MTAs (ie, milters) rather than a Handler. > 3. Adapting existing filters is generally pretty trivial: you write a > 10-line custom Handler that pipes it to an external process. This > isn't big enough for a GSoC project. > 4. To the extent that new algorithms are involved, I have doubts that > Mailman mentors have the kind of expertise needed to really help > with such a project (I could be wrong, but I certainly don't know > much about that kind of text processing, and I don't know that > anybody else in Mailman has expertise in it). Writing individual pipelines may be trivial, but making a user interface for managing said pipelines is non-trivial. Right now, our pipeline management interface is "there's a text box in postorius that lets you choose a pipeline. It's not even a dropdown, and you may be screwed if you make a typo" which is obviously not how I want it when we release. ;) I see a potential project timeline going something like this: A. make a set of custom Mailman 3 Handlers for some well-known existing anti-spam/anti-malware software. (Maybe 2-3 weeks of work here, finding 2-4 reasonable pieces of software, setting them up, writing the handlers, and testing them) B. make an interface in Postorius so list admins can enable/disable/reorder these and any whitelisting happening within mailman. This should involve making an interface in Postorius that gives admins the ability to change the Pipeline being used, and will likely involve a small amount of user testing to make sure said interface doesn't have risk of disastrous results if the administrator does the wrong thing. (Another 3-4 weeks of work including user testing, unit tests, and documentation) C. Figure out how to set up some sort of packager that can install handlers + antispam software so that the site admin has an easy way to set these up if requested. (Another 3-4 weeks of work, including testing any scripts on a few different OSes and extensive documentation) D. If there's any time leftover, implement some clever new filter (and appropriate Handler) that makes use of the list information itself (e.g. subscriber list, archives, etc.) to make better spam decisions. (at this point, you've got maybe 2 weeks left in the GSoC timeline) I think that constitutes enough useful-to-mailman work to justify the google funds, gets us some customizable spam filtering (which as you say, is a frequently requested feature), but doesn't turn us into something we're not. That's why anti-spam made this year's gsoc list even though we've always said "do it in the MTA" and I'm not about to change that policy in general. Do feel free to disagree with me, of course, Stephen. Or complain that I'm using the lure of antispam to get someone solve my user interface for pipelines problem, which I totally am. ;) Terri From barry at list.org Thu Apr 11 22:11:55 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 11 Apr 2013 16:11:55 -0400 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: <5166AAF7.2060805@student.lu.se> References: <51652B5E.60200@student.lu.se> <20130410141418.1e614d12@anarchist> <5166AAF7.2060805@student.lu.se> Message-ID: <20130411161155.6a90c2e5@anarchist> On Apr 11, 2013, at 02:22 PM, Elias Assarsson wrote: >> * MM2's configuration file is a Python file which really must be imported >> in order to get a valid set of values. MM3's configuration file is a stack >> of .ini-style files. >I am trying to find and understand the configuration files so that I know >what that that needs to be migrated and to what form. Is the MM2 >configuration you refer to mainly Mailman/mm_cfg.py and MM3 configuration >files src/mailman/config/* and /src/mailman//*? Yes, in a deployed Mailman 2, mm_cfg.py will contain the system configuration settings. These override the settings from Defaults.py so a good way to explore is to use `bin/withlist` and `import mm_cfg` at the Python prompt. In MM3, we use lazr.config, which is essentially a configparser type package that allows for stacks of configurations, with pushing and popping values on this stack. http://pythonhosted.org/lazr.config/ The src/mailman/config/schema.cfg file is at the bottom of the stack and defines the schema, obviously :). From there, src/mailman/config/mailman.cfg essentially instantiates this schema, providing all the defaults. In the testing environment, src/mailman/testing/testing.cfg gets pushed on top of that (and many tests push and pop micro-overrides). In a deployed system, the site's mailman.cfg is on the top of the stack and so can override anything. There are various places this mailman.cfg file is searched; see src/mailman/core/initialize.py for all the gory details. List-specific configurations live in the various config.db files for MM2. These are pickles. In MM3, everything's in the database. >Given the two points above it seems that handling the migration from within >Python is the best choice (rather than using Augeas which is C based or >Config::Model which is Perl based). I think so. Pickles in particular are Python-specific. You could generate an intermediate format, but I'm not sure it's worth it. >Obviously one wants to use whatever feature of Python that can ease the >process. You seem to be using configparser in MM3. I dunno if there are any >other Python tool one should look at in helping migration. Maybe one should >investigate this further. >> * What about importing archives? >I have tried to find information on how archives are stored in MM2 and MM3 >but failed to find any. What is a good source to learn about this? Importing archives will either be trivial or impossible :). At the lowest level, parsing the MM2 .mbox file and generating maildirs would work, but there are existing tools to do that, so probably little for GSoC to worry about. Much more interesting would be to parse the Pipermail database files and try to build a reverse mapping from URL to Message-ID so that these could be preserved when the archive is regenerated for HyperKitty or whatever. >> There is some moderate beginnings in the 3.0 tree, but none of it is >> functional in all likelihood. Take a look at src/mailman/bin/export.py >> and src/mailman/commands/cli_import.py. >So those are the files that are supposed to handle migration for which the >project is to make them handle a complete migration from MM2 to MM3? Well, I'd say they were more some unfinished work in that direction. -Barry From rkw at DATAPLEX.NET Fri Apr 12 02:02:53 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Thu, 11 Apr 2013 19:02:53 -0500 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: <20130411161155.6a90c2e5@anarchist> References: <51652B5E.60200@student.lu.se> <20130410141418.1e614d12@anarchist> <5166AAF7.2060805@student.lu.se> <20130411161155.6a90c2e5@anarchist> Message-ID: I encourage any GSoC candidates to actively discuss design issues on this list. Many aspects of MM3 remain only partially defined and still require design in addition to the coding that will follow. Although some might expect the mentors might "spoon feed" coding tasks, as a mentor, I would prefer to work with someone who is actively involved in the design as well as the implementation. Having looked at MM2->MM3 migration in the past (and deferred implementation because critical infrastructure was not available at that time) let me present a different perspective. First, there are some parameters that do not relate to any particular mailing list. IMHO, these aspects need not even be addressed in a conversion. If I wish to set up a MM3 installation, I should first, manually, set up a sample list. After I do so, the configuration of any "real" lists needs to be COMPLETELY configurable using the REST interface. If that interface is not presently adequate, it needs to be revised rather than "working around" any deficiency. Therefore, I should be able to set up my "sample list" and, thereafter, add/edit all of the real lists utilizing the same interface (using mailman.cliient, etc.) that is available to the Postorius interface for list creation/editing. A migration tool would, therefore, need only "simulate" those manual steps that the installer would execute on a web-based interface to create a new list and adjust its settings. Similarly, handling the subscriptions must be something that can be done using just the access exposed via the REST interface. The big distinction between MM2 and MM3 lies in the conceptual model of entities. In MM2, each subscription is a separate entity. In MM3, subscriptions belong to "persons" and management functions are made available for the person to affect multiple subscriptions at the same time. In translating from MM2 to MM3, the aggregation of subscriptions under a common "person" becomes a non-trivial task. However the mechanisms required to handle reassignment are needed within MM3 implementations because there is an alternate access mechanism (admin by email) which cannot directly identify these "persons". Therefore, I would suggest that a migration be broken into some components, 1) Migrate individual list parameters 2) Aggregate groups of lists 3) Migrate individual subscriptions 4) Aggregate subscriptions by "person". Note that the aggregation functions for both lists and persons require similar mechanisms and that having the ability to edit those configurations within Postorius will be beneficial to both migration and routine system operation. Richard On Apr 11, 2013, at 3:11 PM, Barry Warsaw wrote: > On Apr 11, 2013, at 02:22 PM, Elias Assarsson wrote: > >>> * MM2's configuration file is a Python file which really must be imported >>> in order to get a valid set of values. MM3's configuration file is a stack >>> of .ini-style files. > >> I am trying to find and understand the configuration files so that I know >> what that that needs to be migrated and to what form. Is the MM2 >> configuration you refer to mainly Mailman/mm_cfg.py and MM3 configuration >> files src/mailman/config/* and /src/mailman//*? > > Yes, in a deployed Mailman 2, mm_cfg.py will contain the system configuration > settings. These override the settings from Defaults.py so a good way to > explore is to use `bin/withlist` and `import mm_cfg` at the Python prompt. > > In MM3, we use lazr.config, which is essentially a configparser type package > that allows for stacks of configurations, with pushing and popping values on > this stack. > > http://pythonhosted.org/lazr.config/ > > The src/mailman/config/schema.cfg file is at the bottom of the stack and > defines the schema, obviously :). From there, src/mailman/config/mailman.cfg > essentially instantiates this schema, providing all the defaults. In the > testing environment, src/mailman/testing/testing.cfg gets pushed on top of > that (and many tests push and pop micro-overrides). In a deployed system, the > site's mailman.cfg is on the top of the stack and so can override anything. > There are various places this mailman.cfg file is searched; see > src/mailman/core/initialize.py for all the gory details. > > List-specific configurations live in the various config.db files for MM2. > These are pickles. In MM3, everything's in the database. From barry at list.org Fri Apr 12 02:44:50 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 11 Apr 2013 20:44:50 -0400 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: References: <51652B5E.60200@student.lu.se> <20130410141418.1e614d12@anarchist> <5166AAF7.2060805@student.lu.se> <20130411161155.6a90c2e5@anarchist> Message-ID: <20130411204450.0972ece2@anarchist> On Apr 11, 2013, at 07:02 PM, Richard Wackerbarth wrote: >I encourage any GSoC candidates to actively discuss design issues on this >list. Many aspects of MM3 remain only partially defined and still require >design in addition to the coding that will follow. Although some might expect >the mentors might "spoon feed" coding tasks, as a mentor, I would prefer to >work with someone who is actively involved in the design as well as the >implementation. Totally agree. >First, there are some parameters that do not relate to any particular mailing >list. IMHO, these aspects need not even be addressed in a conversion. If I >wish to set up a MM3 installation, I should first, manually, set up a sample >list. After I do so, the configuration of any "real" lists needs to be >COMPLETELY configurable using the REST interface. If that interface is not >presently adequate, it needs to be revised rather than "working around" any >deficiency. The REST API is pretty easy to extend. By far the most time consuming bit (assuming you know where you want your resources to live) is writing docs and tests. For many of the GSoC tasks, extending the REST API should be considered part of the work. This of course includes both in the core and in the mailman-client wrapper. >Therefore, I should be able to set up my "sample list" and, thereafter, >add/edit all of the real lists utilizing the same interface (using >mailman.cliient, etc.) that is available to the Postorius interface for list >creation/editing. A migration tool would, therefore, need only "simulate" >those manual steps that the installer would execute on a web-based interface >to create a new list and adjust its settings. Perhaps. I think migrations could be done this way, but it's probably more efficient to do it against the internal API. >Similarly, handling the subscriptions must be something that can be done >using just the access exposed via the REST interface. > >The big distinction between MM2 and MM3 lies in the conceptual model of >entities. In MM2, each subscription is a separate entity. In MM3, >subscriptions belong to "persons" and management functions are made available >for the person to affect multiple subscriptions at the same time. > >In translating from MM2 to MM3, the aggregation of subscriptions under a >common "person" becomes a non-trivial task. However the mechanisms required >to handle reassignment are needed within MM3 implementations because there is >an alternate access mechanism (admin by email) which cannot directly identify >these "persons". Absolutely right. It's an open question as to how to merge user records. Because the user "databases" in a multi-list MM2 site are silos, there's no connection between my settings in listA and my settings in listB. How would you combine those into a single user record for MM3? Additionally, in MM2 you don't now that me1 at example.com and me2 at example.com are both owned by me. While it's probably impossible to merge these two records when migrating them to MM3, we would like to have *some* way to merge the two records once they're migrated to MM3. We'd need to work out the workflow for that, including any security guarantees, so that a user could merge those records after a migration. >Therefore, I would suggest that a migration be broken into some components, >1) Migrate individual list parameters >2) Aggregate groups of lists >3) Migrate individual subscriptions >4) Aggregate subscriptions by "person". -Barry From stephen at xemacs.org Fri Apr 12 05:50:06 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 12 Apr 2013 12:50:06 +0900 Subject: [Mailman-Developers] A feature I'd like to emphasize for GSoC ... Message-ID: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> ... is *convenient access to logs via the admin interface*. Steve From p at sys4.de Fri Apr 12 08:28:26 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Fri, 12 Apr 2013 08:28:26 +0200 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130412062826.GA12030@sys4.de> * Stephen J. Turnbull : > Sreyanth writes: > > > 3. Anti-spam / anti-abuse in Mailman. > > A couple of people have mentioned anti-spam, and it's a frequently > requested feature. Nevertheless, I don't think we should spend Google > money and mentor time on it. I concur. > 1. Mailman is the wrong place to do filtering. It's equally > effective, normally covers more messages, and is somewhat more > efficient in resource usage to do it at the MTA. Spam-filtering is expensive. It should be done only once - at sender level and not for each recipient of a mailing list. We could let Mailman do it when the mail enters, but what would be the gain? There's plenty of software out there that already knows how to battle spam. Even worse! In some countries - take Germany for example - you either reject spam at SMTP session level while the sending client is still there and will take notice or you MUST deliver it - else you break the law because you took reponsibility for transport, but supressed the message. Mailman is part of a mail system, but it I don't expect it will ever become the component that will communicate directly with a remote (spam sending) client. All the work to add an anti-spam feature in Mailman would be 'useless' to countries with laws as I described above. BUT ... I think it would be real nice to have a MILTER interface at LMTP server level to allow mail modification as required. Mailman runs in large environments and all the 'large organizations' I have worked asked my team and me to customize how mail is processed. MILTER is a great interface to modify mail. > 2. Any new algorithms *should* be made available at the MTA level > where they can be best put to use by more people. This implies > something that either plugs into existing filters (such as > spamassassin) or MTAs (ie, milters) rather than a Handler. > 3. Adapting existing filters is generally pretty trivial: you write a > 10-line custom Handler that pipes it to an external process. This > isn't big enough for a GSoC project. > 4. To the extent that new algorithms are involved, I have doubts that > Mailman mentors have the kind of expertise needed to really help > with such a project (I could be wrong, but I certainly don't know > much about that kind of text processing, and I don't know that > anybody else in Mailman has expertise in it). > > On the other hand, I don't know which project in GSoC would be a > better place for it. It's possible to argue that Mailman is a > reasonable place for it, but IMHO we probably shouldn't. I hate to stand in the way of someone, who wants to contribute to OSS, but IMHO we shouldn't either. > Regarding anti-abuse, we would like to do something about problems > like backscatter. However, I have to wonder how much *code* (vs > *specification* and *design*) is needed for those problems. If the > project is really spec-heavy, it's probably not really what Google has > in mind (based on comments on the mentors' list, not on any official > Google pronouncements, though). Has anyone ever mentioned SNMP as a feature for Mailman? p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stephen at xemacs.org Fri Apr 12 08:44:58 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 12 Apr 2013 15:44:58 +0900 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: <5167148C.4000701@zone12.com> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> Message-ID: <87eheg5jth.fsf@uwakimon.sk.tsukuba.ac.jp> Terri Oda writes: > Writing individual pipelines may be trivial, but making a user interface > for managing said pipelines is non-trivial. Right now, our pipeline > management interface is "there's a text box in postorius that lets you > choose a pipeline. It's not even a dropdown, and you may be screwed if > you make a typo" which is obviously not how I want it when we > release. ;) That's a more general issue (oh, I see you noticed that! :-), and I have no problem with doing something about it -- indeed, I'd be more than happy to (co-)mentor it because I just *love* custom Handlers. Here's what I would do: 1. Get the list of handlers active to the list. 2. Append the list of inactive handlers from Mailman/Handlers (the site's list, not the distributed handlers). 3. The UI is table with rows containing a checkbox for "active handler" (the row should be greyed out if it's inactive), an ordinal (numerical), and the handler name (gold star for popping up a tooltip with a detailed description/docstring on mouseover). 4. Users can either change the numbers (error checked for uniqueness), with a partial order on standard handlers -- if the partial order is violated (including a missing handler like "ToOutgoing") the user is warned; or (platinum star) drag the handlers into the order they like (with same checks on the partial order). > I see a potential project timeline going something like this: > > A. make a set of custom Mailman 3 Handlers for some well-known existing > anti-spam/anti-malware software. (Maybe 2-3 weeks of work here, finding > 2-4 reasonable pieces of software, setting them up, writing the > handlers, and testing them) One week for that work, it's all in the FAQ already I suspect. > B. make an interface in Postorius so list admins can > enable/disable/reorder these and any whitelisting happening within > mailman. This should involve making an interface in Postorius that > gives admins the ability to change the Pipeline being used, and will > likely involve a small amount of user testing to make sure said > interface doesn't have risk of disastrous results if the administrator > does the wrong thing. (Another 3-4 weeks of work including user > testing, unit tests, and documentation) You think the design above will take more than two days (one to learn how to do D&D to reorder a list) to code, and 4 to document and test? (I'm assuming Mailman2 kinds of pipeline APIs are already available. If new REST API is needed, OK, 3 weeks total.) > C. Figure out how to set up some sort of packager that can install > handlers + antispam software so that the site admin has an easy way to > set these up if requested. (Another 3-4 weeks of work, including testing > any scripts on a few different OSes and extensive documentation) OK, yes, getting PyPI down for the Handlers themselves (while these *could* be delivered with Mailman, I think it would be more valuable to have a standard PyPI delivery protocol for 3rd party Handlers) will likely take that much time, and indeed one needs to deal with OS PMS. > Do feel free to disagree with me, of course, Stephen. I am indeed a curmudgeon about the antispam stuff. I don't think the first release of Mailman 3 should contain an attractive nuisance like serious antispam in Mailman (vs antispam in the MTA). I'll try to keep such negative thinking to one paragraph per post, though. :-) > Or complain that I'm using the lure of antispam to get someone > solve my user interface for pipelines problem, which I totally > am. ;) While I do think that an initial implementation is probably a total of about 2 weeks worth of work, I suspect that one could riff on the theme (hi, Barry, like that metaphor?) for a couple more weeks, and robust disaster recovery (saving off the old pipeline and restoring looks simple enough, but Mr. Murphy is lurking, I'm sure -- in particular, if we're going to allow through-the-web pipelines, we need to guarantee that received mail will not get lost if the pipeline is horked) could account for a couple more weeks. From stephen at xemacs.org Fri Apr 12 08:47:00 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 12 Apr 2013 15:47:00 +0900 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: References: <51652B5E.60200@student.lu.se> <20130410141418.1e614d12@anarchist> <5166AAF7.2060805@student.lu.se> <20130411161155.6a90c2e5@anarchist> Message-ID: <87d2u05jq3.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > Therefore, I would suggest that a migration be broken into some components, > 1) Migrate individual list parameters > 2) Aggregate groups of lists > 3) Migrate individual subscriptions > 4) Aggregate subscriptions by "person". +1, and perhaps some of these are big/error-prone enough to constitute whole GSoC projects themselves. (I don't say that they are, but we should think about it during design of an intern's project.) From dkg at fifthhorseman.net Fri Apr 12 09:12:29 2013 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Fri, 12 Apr 2013 03:12:29 -0400 Subject: [Mailman-Developers] OpenPGP Integration on GSoC In-Reply-To: <5166B6FE.4080107@ulm.ccc.de> References: <51661A14.1050806@fifthhorseman.net> <20130411041922.GI25275@beskar.mdcc.cx> <516664F7.8070407@ulm.ccc.de> <5166AE05.9060000@Damon-Family.org> <5166B6FE.4080107@ulm.ccc.de> Message-ID: <5167B3DD.4040403@fifthhorseman.net> On 04/11/2013 09:13 AM, Stefan Schlott wrote: > True, the PGP file structure encapsulates the signature within the > encryption (in contrast to S/MIME, which does it vice versa). But the > standard PGP binary will strip both in one step, so keeping the > signature won't work out of the box (at least I didn't manage to do > that, I'd be really interested how to do that - would be useful for > searchable mail archives). It's certainly possible within the OpenPGP spec to have the mailing list software decrypt its Encrypted Session Key (ESK) OpenPGP packet from an encrypted message, and then add a new ESK packet (or replace the old one) for each list subscriber. IIUC, this should leave the original message's signature intact. Whether any of the various OpenPGP-related toolkits that are readily available for python are capable of doing these operations is another matter. If you're playing with this stuff, i recommend reading the OpenPGP RFC, which actually describes how all the data fits together: https://tools.ietf.org/html/rfc4880 you may also be interested in the PGP/MIME spec, which concerns how to to format OpenPGP within an e-mail: https://tools.ietf.org/html/rfc3156 Note that the design proposed in this thread is similar to the schleuder2 design, though schleuder doesn't preserve the original signer's signature either, but substitutes it with a message signature from the mailing list itself. This design also exposes the content of each message to the mailing list software itself. There are other architectures that make it so the mailing list software never actually gets to see the content of the message (see PSELS for an example). --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1027 bytes Desc: OpenPGP digital signature URL: From barry at list.org Fri Apr 12 15:54:21 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 12 Apr 2013 09:54:21 -0400 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130412095421.22083a33@anarchist> On Apr 12, 2013, at 01:44 AM, Stephen J. Turnbull wrote: >A couple of people have mentioned anti-spam, and it's a frequently >requested feature. Nevertheless, I don't think we should spend Google >money and mentor time on it. From the core's perspective, I tend to agree that there is some interesting things we'd like to add here, but it's probably not enough work to justify a GSoC slot. I'm not sure if additional ui work can pad that out. I also agree that in general, we want to encourage sites to push anti-spam defenses into the MTA as much as possible. The counter argument is that we get plenty of requests from folks who have no control over their MTA and want to be able to configure Mailman to help reduce spam. I think the following avenues would be interesting to pursue. * Assume the MTA is doing filtering, and that messages will fall into three categories: known bad (these get dropped at the MTA), known good (these flow through), unsure. For the latter, the message will probably be marked in some way, e.g. a header with a spam score, and it would be good if Mailman has some facility (e.g. a rule) to parse that header and make disposition decisions based on that value. One thing Mailman can do that the MTA cannot is allow for human intervention for disposition. * Provide an option for messages to detour into spam filters like spamassassin during Mailman message processing. This probably means a rule which calls out to SA or equivalent, and stores the score in some metadata. A rule hit might mean that the message has a spam score higher than a threshold, in which case processing jumps to a chain which can discard, reject, or hold th message. >Regarding anti-abuse, we would like to do something about problems >like backscatter. However, I have to wonder how much *code* (vs >*specification* and *design*) is needed for those problems. If the >project is really spec-heavy, it's probably not really what Google has >in mind (based on comments on the mentors' list, not on any official >Google pronouncements, though). Agreed. -Barry From barry at list.org Fri Apr 12 15:59:39 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 12 Apr 2013 09:59:39 -0400 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: <20130412062826.GA12030@sys4.de> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <20130412062826.GA12030@sys4.de> Message-ID: <20130412095939.66420805@anarchist> On Apr 12, 2013, at 08:28 AM, Patrick Ben Koetter wrote: >I think it would be real nice to have a MILTER interface at LMTP server level >to allow mail modification as required. Mailman runs in large environments and >all the 'large organizations' I have worked asked my team and me to customize >how mail is processed. MILTER is a great interface to modify mail. Do you mean a hook in Mailman's LMTP server process? I thought about that in my previous message but decided not to mention it because it's not clear to me how performant Mailman's current smtpd-based (read: async) LMTP server is. What I mean is, I'm not sure how much additional work we want the LMTP server to do. It would be cool if someone did some performance testing of the LMTP implementation, and it would be cool if someone tried to add some hooks into that server. It might also be interesting to look into alternative implementations. Another reason to push for getting Mailman 3 onto Python 3 would be the ability to leverage Guido's Tulip work for better async IO performance. >Has anyone ever mentioned SNMP as a feature for Mailman? Nope, but that would be interesting too. -Barry From barry at list.org Fri Apr 12 16:15:05 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 12 Apr 2013 10:15:05 -0400 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: <5167148C.4000701@zone12.com> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> Message-ID: <20130412101505.2f6961db@anarchist> On Apr 11, 2013, at 01:52 PM, Terri Oda wrote: >Writing individual pipelines may be trivial, but making a user interface for >managing said pipelines is non-trivial. Right now, our pipeline management >interface is "there's a text box in postorius that lets you choose a >pipeline. It's not even a dropdown, and you may be screwed if you make a >typo" which is obviously not how I want it when we release. ;) Remember that in MM3 you have two processing queues. Think of the first as the "moderation" process and the second as the "modification" process. http://pythonhosted.org/mailman/src/mailman/docs/8-miles-high.html#basic-messaging-handling-workflow When the message is first accepted by the LMTP server, it is dumped into the incoming queue. The incoming runner will send the message through the rule chain for target mailing list. (There are actually two of these chains for every mailing list - one for messages to the owner/moderator and one for messages posted to the list.) Rule chains are made up of individual links where each link usually contains a rule and an action. Rules only inspect the message, never modify it (though they can add metadata to the associated dictionary), and rules produce a binary decision. If the rule "hits", the action is take. If the rule "misses", the next link in the chain is executed. At the end of the rule chain is usually a "truth" rule which always hits, and its action is to jump to the "accept" chain, which essentially just dumps the message into the pipeline queue. The pipeline runner only handles messages that have been approved for posting, so it doesn't have to make any moderation decisions. All it has to do is modify the message for posting. Again, there are two pipelines, one for owner messages and one for list posts, but pipelines are much simpler. They are just sequences of handlers. Each handler does whatever it wants to the message, and it can use, add, or delete information from the metadata dictionary to do this work. E.g. handlers can remove or add RFC 822 headers, filter content, add those informative footers, etc. Handlers also copy the messages to other queues, so that other runners can send the message to the archives, NNTP, and the outgoing SMTPd. I mention all this because to show that there are *lots* of ways the system can be configured, and I'm not sure all should be exposed via Postorius. Certainly the full power isn't something a list owner will usually want to be confronted with ;). Each chain and pipeline (as well as rules and handlers) have a unique name, and the choice of chain and pipeline is part of a list's style (i.e. initial setting), so that may be a better way to allow list owners some flexibility in setting up their lists. Anyway, some things to keep in mind. :) -Barry From barry at list.org Fri Apr 12 16:59:53 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 12 Apr 2013 10:59:53 -0400 Subject: [Mailman-Developers] A feature I'd like to emphasize for GSoC ... In-Reply-To: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130412105953.5110fa0d@anarchist> On Apr 12, 2013, at 12:50 PM, Stephen J. Turnbull wrote: >... is *convenient access to logs via the admin interface*. +1. Can you add that to the wiki page? -Barry From sreyanth at gmail.com Fri Apr 12 17:17:39 2013 From: sreyanth at gmail.com (Sreyanth) Date: Fri, 12 Apr 2013 20:47:39 +0530 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: <5167148C.4000701@zone12.com> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> Message-ID: Hi all! Thank you very much for awesome discussion here! On Fri, Apr 12, 2013 at 1:22 AM, Terri Oda wrote: > On 13-04-11 10:44 AM, Stephen J. Turnbull wrote: > > 1. Mailman is the wrong place to do filtering. It's equally > effective, normally covers more messages, and is somewhat more > efficient in resource usage to do it at the MTA. > 2. Any new algorithms **should** be made available at the MTA level > where they can be best put to use by more people. This implies > something that either plugs into existing filters (such as > spamassassin) or MTAs (ie, milters) rather than a Handler. > 3. Adapting existing filters is generally pretty trivial: you write a > 10-line custom Handler that pipes it to an external process. This > isn't big enough for a GSoC project. > 4. To the extent that new algorithms are involved, I have doubts that > Mailman mentors have the kind of expertise needed to really help > with such a project (I could be wrong, but I certainly don't know > much about that kind of text processing, and I don't know that > anybody else in Mailman has expertise in it). > > I agree.?? > > Writing individual pipelines may be trivial, but making a user interface > for managing said pipelines is non-trivial. Right now, our pipeline > management interface is "there's a text box in postorius that lets you > choose a pipeline. It's not even a dropdown, and you may be screwed if you > make a typo" which is obviously not how I want it when we release. ;) > > I see a potential project timeline going something like this: > > A. make a set of custom Mailman 3 Handlers for some well-known existing > anti-spam/anti-malware software. (Maybe 2-3 weeks of work here, finding > 2-4 reasonable pieces of software, setting them up, writing the handlers, > and testing them) > > B. make an interface in Postorius so list admins can > enable/disable/reorder these and any whitelisting happening within > mailman. This should involve making an interface in Postorius that gives > admins the ability to change the Pipeline being used, and will likely > involve a small amount of user testing to make sure said interface doesn't > have risk of disastrous results if the administrator does the wrong thing. > (Another 3-4 weeks of work including user testing, unit tests, and > documentation) > > C. Figure out how to set up some sort of packager that can install > handlers + antispam software so that the site admin has an easy way to set > these up if requested. (Another 3-4 weeks of work, including testing any > scripts on a few different OSes and extensive documentation) > > D. If there's any time leftover, implement some clever new filter (and > appropriate Handler) that makes use of the list information itself (e.g. > subscriber list, archives, etc.) to make better spam decisions. (at this > point, you've got maybe 2 weeks left in the GSoC timeline) > > This really looks great! Almost what I actually expected from a project like this. But, like Stephen and Barry pointed out, I am unsure as to how far this comes under GSoC's purview. ?? > > I think that constitutes enough useful-to-mailman work to justify the > google funds, gets us some customizable spam filtering (which as you say, > is a frequently requested feature), but doesn't turn us into something > we're not. That's why anti-spam made this year's gsoc list even though > we've always said "do it in the MTA" and I'm not about to change that > policy in general. > > Do feel free to disagree with me, of course, Stephen. Or complain that I'm > using the lure of antispam to get someone solve my user interface for > pipelines problem, which I totally am. ;) > > Terri > > Thanks for such a great timeline Terri. I dont have issues with this. As Stephen and Barry said, I even liked the idea of having a MILTER interfaced at LMTP level. On a overall positive note, I am quite convinced that giving the admin of the list with great flexible options to choose from (and as Barry pointed out, why should everything be exposed to the admin via Postorius?, which may not be of the admin's interest! ). I believe this could be make a nice GSoC project, but with many spam filters which people are already acquainted with, I am not sure how far people tend to use this feature. Also, I would like to hear more about : Boilerplate stripper AND Better content-filtering / handling error messages. ?Boilerplate stripping is trivial to understand. But, can anyone elaborate on Better content-filtering / handling error messages? I strongly believe that Boilerplate stripping will be a cool thing to have in Mailman and obviously, who would not want to welcome better content-filtering / error handling techniques on board?? -- *Yours Sincerely* * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From sreyanth at gmail.com Fri Apr 12 17:22:23 2013 From: sreyanth at gmail.com (Sreyanth) Date: Fri, 12 Apr 2013 20:52:23 +0530 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> Message-ID: And this is my another idea, which I am interested to work on 4. My own project idea: Mining the list logs and recognize interesting patterns for better enhancements (the admin need not have data mining experience) ?We can actually have this integrated to the admin console where the logs can be accessed, at the same time, some interesting patterns can shown, along with stats and all (just a basic idea, need to work on this more). Depending on the detected patterns, the admin may want to change some settings! Given my experience with IR and Django, I feel this is a potential GSoC project! Any suggestions?? On Fri, Apr 12, 2013 at 8:47 PM, Sreyanth wrote: > Hi all! Thank you very much for awesome discussion here! > > On Fri, Apr 12, 2013 at 1:22 AM, Terri Oda wrote: > >> On 13-04-11 10:44 AM, Stephen J. Turnbull wrote: >> >> 1. Mailman is the wrong place to do filtering. It's equally >> effective, normally covers more messages, and is somewhat more >> efficient in resource usage to do it at the MTA. >> 2. Any new algorithms **should** be made available at the MTA level >> where they can be best put to use by more people. This implies >> something that either plugs into existing filters (such as >> spamassassin) or MTAs (ie, milters) rather than a Handler. >> 3. Adapting existing filters is generally pretty trivial: you write a >> 10-line custom Handler that pipes it to an external process. This >> isn't big enough for a GSoC project. >> 4. To the extent that new algorithms are involved, I have doubts that >> Mailman mentors have the kind of expertise needed to really help >> with such a project (I could be wrong, but I certainly don't know >> much about that kind of text processing, and I don't know that >> anybody else in Mailman has expertise in it). >> >> I agree.?? > >> >> Writing individual pipelines may be trivial, but making a user interface >> for managing said pipelines is non-trivial. Right now, our pipeline >> management interface is "there's a text box in postorius that lets you >> choose a pipeline. It's not even a dropdown, and you may be screwed if you >> make a typo" which is obviously not how I want it when we release. ;) >> >> I see a potential project timeline going something like this: >> >> A. make a set of custom Mailman 3 Handlers for some well-known existing >> anti-spam/anti-malware software. (Maybe 2-3 weeks of work here, finding >> 2-4 reasonable pieces of software, setting them up, writing the handlers, >> and testing them) >> >> B. make an interface in Postorius so list admins can >> enable/disable/reorder these and any whitelisting happening within >> mailman. This should involve making an interface in Postorius that gives >> admins the ability to change the Pipeline being used, and will likely >> involve a small amount of user testing to make sure said interface doesn't >> have risk of disastrous results if the administrator does the wrong thing. >> (Another 3-4 weeks of work including user testing, unit tests, and >> documentation) >> >> C. Figure out how to set up some sort of packager that can install >> handlers + antispam software so that the site admin has an easy way to set >> these up if requested. (Another 3-4 weeks of work, including testing any >> scripts on a few different OSes and extensive documentation) >> >> D. If there's any time leftover, implement some clever new filter (and >> appropriate Handler) that makes use of the list information itself (e.g. >> subscriber list, archives, etc.) to make better spam decisions. (at this >> point, you've got maybe 2 weeks left in the GSoC timeline) >> >> This really looks great! Almost what I actually expected from a project > like this. > But, like Stephen and Barry pointed out, I am unsure as to how far this > comes under GSoC's purview. > ?? > >> >> I think that constitutes enough useful-to-mailman work to justify the >> google funds, gets us some customizable spam filtering (which as you say, >> is a frequently requested feature), but doesn't turn us into something >> we're not. That's why anti-spam made this year's gsoc list even though >> we've always said "do it in the MTA" and I'm not about to change that >> policy in general. >> >> Do feel free to disagree with me, of course, Stephen. Or complain that >> I'm using the lure of antispam to get someone solve my user interface for >> pipelines problem, which I totally am. ;) >> >> Terri >> >> Thanks for such a great timeline Terri. I dont have issues with this. As > Stephen and Barry said, I even liked the idea of having a MILTER interfaced > at LMTP level. > > On a overall positive note, I am quite convinced that giving the admin of > the list with great flexible options to choose from (and as Barry pointed > out, why should everything be exposed to the admin via Postorius?, which > may not be of the admin's interest! ). I believe this could be make a nice > GSoC project, but with many spam filters which people are already > acquainted with, I am not sure how far people tend to use this feature. > > Also, I would like to hear more about : Boilerplate stripper AND Better > content-filtering / handling error messages. > ?Boilerplate stripping is trivial to understand. But, can anyone elaborate > on Better content-filtering / handling error messages? > I strongly believe that Boilerplate stripping will be a cool thing to have > in Mailman and obviously, who would not want to welcome better > content-filtering / error handling techniques on board?? > > > > -- > *Yours Sincerely* > * > * > *Mora Sreyantha Chary* > *Computer Engineering '14* > *National Institute of Technology Karnataka* > *Surathkal, India 575 025* > -- *Yours Sincerely* * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From kushal124 at gmail.com Fri Apr 12 19:55:39 2013 From: kushal124 at gmail.com (Kushal Khandelwal) Date: Fri, 12 Apr 2013 23:25:39 +0530 Subject: [Mailman-Developers] GSOC Message-ID: Hey , My name is Kushal Khandelwal and I am a third year undergraduate in engineering.I am interested in developing for Mailman. I have experience in programming with Python , and I am also into web development using PHP HTML CSS and javascript. Also I am good at data mining and vizualization and currently working on Natural Language processing. I would like to work on the following projects : 1)Design interface "themes" for specific types of list 2) Log Monitor 3) Web Posting Interface Please guide me how should go about developing the projects ? I am currently setting up my local mailman development repo. -- Kushal Khandelwal Birla Institute of Technology & Science, Pilani K K Birla Goa Campus From mark at msapiro.net Fri Apr 12 22:28:32 2013 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 12 Apr 2013 13:28:32 -0700 Subject: [Mailman-Developers] MM 2.1 - Relative vs. absolute URLs in admin and admindb interfaces. Message-ID: <51686E70.3020404@msapiro.net> There is a thread in mailman-users at . In short the user's difficulty is her world facing host name doesn't work inside her LAN so from inside, she accesses the web interface via IP address or some other host name. This leads to the issue that some links and form actions in the web admin and admindb interfaces are absolute using the list's web_page_url which has the host name that doesn't work inside the LAN. Granted, this can be addressed in the network or the work station's /etc/hosts file, but the question arises as to whether these URLs need to be absolute, particularly because they're already in the minority. I was going to call this a MM 2.1 bug and fix it but then I came across this 10 year old comment from Barry at . We had lots of problems when non-absolute urls where used, although I don't remember the exact details of the problems. Eventually we'll probably make all urls absolute and get rid of this argument to GetScriptURL(). So my question is would there be issues if the URL in the form action and the leter/digit links on the web admin Membership List page or those in the admindb interface were relative instead of absolute and what might those issues be? Does anyone know? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Fri Apr 12 22:17:15 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 12 Apr 2013 16:17:15 -0400 Subject: [Mailman-Developers] MM 2.1 - Relative vs. absolute URLs in admin and admindb interfaces. In-Reply-To: <51686E70.3020404@msapiro.net> References: <51686E70.3020404@msapiro.net> Message-ID: <20130412161715.7823ea6d@anarchist> On Apr 12, 2013, at 01:28 PM, Mark Sapiro wrote: >Does anyone know? Probably not . -Barry From p at sys4.de Fri Apr 12 22:39:27 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Fri, 12 Apr 2013 22:39:27 +0200 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: <20130412095939.66420805@anarchist> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <20130412062826.GA12030@sys4.de> <20130412095939.66420805@anarchist> Message-ID: <20130412203927.GA3070@sys4.de> * Barry Warsaw : > On Apr 12, 2013, at 08:28 AM, Patrick Ben Koetter wrote: > > >I think it would be real nice to have a MILTER interface at LMTP server level > >to allow mail modification as required. Mailman runs in large environments and > >all the 'large organizations' I have worked asked my team and me to customize > >how mail is processed. MILTER is a great interface to modify mail. > > Do you mean a hook in Mailman's LMTP server process? I thought about that in Yes, I mean to hook MILTER capability into Mailman's LMTP server process. > my previous message but decided not to mention it because it's not clear to me > how performant Mailman's current smtpd-based (read: async) LMTP server is. It's not clear to me either, but now that you made me think about it I begin to ask myself how fast is fast enough and I also ask myself are we dealing with a bogey (had to look this up. hope it fits) or are trying to address a reasonable bottleneck. (I've experienced quite a few "problematic" situations in mail transport which turned out to be more driven by myth and oral history than by vested knowledge). I agree we should measure, just in order not to speculate, but let me send some thoughts ahead before we take out to test performance: - Input/output ratio on a mailing list system is 1:n. Performance requirements on the receiving side should be the least to worry about. - In most usage scenarios that come to my mind companies run an MLM as a supplement to their 'regular' mail system. Only a minor ratio of mail that enters the mail system is routed forward to the MLM (here: MM3 LMTP server). - At the moment (MM2) mail enters Mailman via a script that is called. Scripts are _a lot_ slower than a server process. My understanding is MM3 will have an LMTP server process. Any site that switches to MM3 should experience a performance boost on the receiving side. It seems to me most people will be off fine. Unfortunately I think "most people" will not need to use a MILTER, too. What characterizes the remaining group: - They run sites dedicated solely to mailing lists. - They need special filtering (read: MILTER and other methods). - They split load via clusters. - They have their own development teams to customize and optimize software as required > What I mean is, I'm not sure how much additional work we want the LMTP server > to do. How much should it be able to do at all? Do you collect and log statistics at the moment? Personally I like the "delays=0.04/0.01/0.05/0.1" entry in Postfix's log. Quote from postconf(5): The format of the "delays=a/b/c/d" logging is as follows: ? a = time from message arrival to last active queue entry ? b = time from last active queue entry to connection setup ? c = time in connection setup, including DNS, EHLO and STARTTLS ? d = time in message transmission -- $ man 5 postconf | less +/delay_logging_resolution_limit > It would be cool if someone did some performance testing of the LMTP > implementation, and it would be cool if someone tried to add some hooks into > that server. It might also be interesting to look into alternative > implementations. Another reason to push for getting Mailman 3 onto Python 3 > would be the ability to leverage Guido's Tulip work for better async IO > performance. I'm short on time to do performance testing myself, but I'll forward the request to my team members since we are doing tests at the moment anyway. Maybe someone finds time to squeeze LMTP server testing in. My first idea would be to use either Postfix smtp-source (multi-threaded SMTP/LMTP test generator) or swaks (Swiss Army Knife for SMTP) and create a wrapper around it that produces the load. > >Has anyone ever mentioned SNMP as a feature for Mailman? > > Nope, but that would be interesting too. We (sys4) will contribute the MIB and monitoring server during development, if someone takes onto the programming. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From 24.sneha at gmail.com Sat Apr 13 06:39:58 2013 From: 24.sneha at gmail.com (Sneha Priscilla) Date: Sat, 13 Apr 2013 10:09:58 +0530 Subject: [Mailman-Developers] GSoC 2013 (GNU Mailman) - introductions Message-ID: Hello! I'm Sneha Priscilla, a final year undergraduate CS student from Bangalore, India. I was a previous Summer of Code student for Systers last year and I have in the process, worked with a lot of Mailman (mostly the stable 2.1 version). I am especially interested in Postorius and would like to implement one of the following ideas this summer: 1.Better User Settings Management 2.Design interface "themes" for specific types of list 3.Enhance list style capabilities If any of the above are not sufficient to serve as an independent project, I thought I could team it up with any others above or with the project idea: Convenient access to logs via admin interface I would love to have some guidance regarding the above projects and their feasibility , maybe how to go about it. A little about myself: I have 4 years of programming experience, 1 year in Python, coding for Systers for GSoC 2012. After that summer, I've worked with Mailman 3 and Postorius and have found adding features to Postorius a lot of fun. I've also worked on a bug in the month of January which was merged during PyCon: https://code.launchpad.net/~stylistica/postorius/inline_helptext Apart from this, I've also filed 2 bugs related to Postorius and which I am currently working on. I love algorithms and clean UI design - I have some experience with Django and my current final year project uses Bootstrap for CSS. I'm hoping to work with Mailman in this year's GSoC as I already have a tiny bit of experience with it and this would serve as an ideal opportunity to make a big contribution and also give me a chance to work with Mailman formally and prove my mettle. I am accessible via email and also available on Launchpad and IRC by the nick: stylistica . Looking forward to (Hopefully!) working with the Mailman team. Thanks and Regards, Priscilla From terri at zone12.com Sat Apr 13 18:09:09 2013 From: terri at zone12.com (Terri Oda) Date: Sat, 13 Apr 2013 10:09:09 -0600 Subject: [Mailman-Developers] Setting up a VM. In-Reply-To: References: Message-ID: <51698325.5090808@zone12.com> Hi Peter, Sorry about not getting back to you earlier; I'm the org admin for the PSF for the first time this year, and in case that wasn't enough, we've grown considerably and I've got a huge amount of email. Thanks for pinging me off-list to let me know I missed this one once you figured out i was the one who needed to see it. The short answer is just "+1" (Which, if you're not used to this particular terminology, means "that's a good idea; go ahead and do it") So yeah, go ahead and get a VM set up; you don't need to ask. You'll probably want to follow the mailman install instructions here: http://pythonhosted.org/mailman/ Chris, who was working on such a thing before, seems to have gotten stuck somewhere. You might want to ping him to see if the two of you can work together to sort out any problems (I'm cc'ing him so he sees this mail) or if he can offer any advice. You might also want to do a quick search through the archives to see what we told him when he asked about setting things up. You probably also want to talk to other incoming students who've already set up their dev environment since they'll have any issues they encountered fresh in their minds! If you're looking for some help via IRC, most of our mentors and a decent number of students are on IRC when they have some time. I think wacky is our most stalwart IRC'er, so you may want to ping him if you're having trouble getting someone's attention. Many of us are are on IRC while busy doing other things, so if you want someone's attention the best way is to say their irc nick name in channel: most IRC clients are set up to help bring anything directed at us to our attention. When you get to the point of a final working VM, either post a link here & to the wiki, or if you need hosting, ask on the mailing list. I know I have access to some machines that can do it, but I'm sure I'm not the only one. FYI: I've got a friend flying in for a visit tonight, so I'm pretty much going to be offline as much as possible for a couple of days, but hopefully that should get you pointers to other people who be more available to help this weekend. Terri On 13-04-11 3:23 AM, Peter Markou wrote: > Hello again community. > > Two days ago I made a sort introduction of my self and stated that I'm > interested in implementing the "Web Posting Interface" project(through this > e-mail - > http://www.mail-archive.com/mailman-developers%40python.org/msg13451.html). > > In order to getting started with Mailman development I went through the > equivalent quote(here - > http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013). > Since I'm able to consume the services described here : > https://okeanos.grnet.gr/services/cyclades/ , I would like to setup a VM > and also make it available for future developers. > > If you're interested in this, poke me to arrange a meeting in IRC and > discuss any further details. > > Best regards, > Peter Markou. > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/terri%40zone12.com > > Security Policy: http://wiki.list.org/x/QIA9 > From avikpal.me at gmail.com Sat Apr 13 20:56:52 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Sun, 14 Apr 2013 00:26:52 +0530 Subject: [Mailman-Developers] error installing the mailman web UI Message-ID: Hello, I was following the steps given at http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut I am getting the following error every time. please help. An internal error occured due to a bug in either zc.buildout or in a recipe being used: Traceback (most recent call last): File "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", line 1923, in main getattr(buildout, command)(args) File "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", line 604, in install installed_files = self[part]._call(recipe.install) File "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", line 1358, in _call return f() File "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", line 126, in install reqs, ws = self.working_set() File "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", line 84, in working_set allow_hosts=self.allow_hosts) File "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", line 782, in install return installer.install(specs, working_set) File "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", line 626, in install for dist in self._get_dist(requirement, ws): File "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", line 448, in _get_dist dist, avail = self._satisfied(requirement) File "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", line 204, in _satisfied return None, self._obtain(req, source) File "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", line 372, in _obtain if index.obtain(requirement) is None: File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 341, in obtain self.prescan(); self.find_packages(requirement) File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 326, in find_packages self.scan_url(self.index_url + requirement.unsafe_name+'/') File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 673, in scan_url self.process_url(url, True) File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 224, in process_url page = self.process_index(url, page) File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 301, in process_index self.scan_url(new_url) File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 673, in scan_url self.process_url(url, True) File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 202, in process_url f = self.open_url(url, "Download error on %s: %%s -- Some packages may not be found!" % url) File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 611, in open_url return open_with_auth(url) File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 807, in _socket_timeout return func(*args, **kwargs) File "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", line 854, in open_with_auth fp = urllib2.urlopen(request) File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 400, in open response = self._open(req, data) File "/usr/lib/python2.7/urllib2.py", line 418, in _open '_open', req) File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain result = func(*args) File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open r = h.getresponse(buffering=True) File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse response.begin() File "/usr/lib/python2.7/httplib.py", line 407, in begin version, status, reason = self._read_status() File "/usr/lib/python2.7/httplib.py", line 365, in _read_status line = self.fp.readline() File "/usr/lib/python2.7/socket.py", line 447, in readline data = self._sock.recv(self._rbufsize) timeout: timed out Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme From 24.sneha at gmail.com Sat Apr 13 21:19:59 2013 From: 24.sneha at gmail.com (Sneha Priscilla) Date: Sun, 14 Apr 2013 00:49:59 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: Hey Avik, Why don't you try re-branching the repository again and start over? Plus buildout instruction needs sudo access as far as I know, so make sure you're doing that. Priscilla On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal wrote: > Hello, > > I was following the steps given at > > http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut > I am getting the following error every time. please help. > > An internal error occured due to a bug in either zc.buildout or in a > recipe being used: > Traceback (most recent call last): > File > > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > line 1923, in main > getattr(buildout, command)(args) > File > > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > line 604, in install > installed_files = self[part]._call(recipe.install) > File > > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > line 1358, in _call > return f() > File > > "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", > line 126, in install > reqs, ws = self.working_set() > File > > "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", > line 84, in working_set > allow_hosts=self.allow_hosts) > File > > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > line 782, in install > return installer.install(specs, working_set) > File > > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > line 626, in install > for dist in self._get_dist(requirement, ws): > File > > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > line 448, in _get_dist > dist, avail = self._satisfied(requirement) > File > > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > line 204, in _satisfied > return None, self._obtain(req, source) > File > > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > line 372, in _obtain > if index.obtain(requirement) is None: > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 341, in obtain > self.prescan(); self.find_packages(requirement) > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 326, in find_packages > self.scan_url(self.index_url + requirement.unsafe_name+'/') > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 673, in scan_url > self.process_url(url, True) > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 224, in process_url > page = self.process_index(url, page) > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 301, in process_index > self.scan_url(new_url) > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 673, in scan_url > self.process_url(url, True) > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 202, in process_url > f = self.open_url(url, "Download error on %s: %%s -- Some packages may > not be found!" % url) > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 611, in open_url > return open_with_auth(url) > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 807, in _socket_timeout > return func(*args, **kwargs) > File > > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > line 854, in open_with_auth > fp = urllib2.urlopen(request) > File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen > return _opener.open(url, data, timeout) > File "/usr/lib/python2.7/urllib2.py", line 400, in open > response = self._open(req, data) > File "/usr/lib/python2.7/urllib2.py", line 418, in _open > '_open', req) > File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain > result = func(*args) > File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open > return self.do_open(httplib.HTTPConnection, req) > File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open > r = h.getresponse(buffering=True) > File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse > response.begin() > File "/usr/lib/python2.7/httplib.py", line 407, in begin > version, status, reason = self._read_status() > File "/usr/lib/python2.7/httplib.py", line 365, in _read_status > line = self.fp.readline() > File "/usr/lib/python2.7/socket.py", line 447, in readline > data = self._sock.recv(self._rbufsize) > timeout: timed out > > > > Avik Pal > Bengal Engineering & Scieence University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From avikpal.me at gmail.com Sat Apr 13 21:36:39 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Sun, 14 Apr 2013 01:06:39 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: yep, tried all those things. can you tell me which os you are using and the version of python that you got? Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 14 April 2013 00:49, Sneha Priscilla <24.sneha at gmail.com> wrote: > Hey Avik, > > Why don't you try re-branching the repository again and start over? Plus > buildout instruction needs sudo access as far as I know, so make sure > you're doing that. > > Priscilla > > > On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal wrote: > >> Hello, >> >> I was following the steps given at >> >> http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut >> I am getting the following error every time. please help. >> >> An internal error occured due to a bug in either zc.buildout or in a >> recipe being used: >> Traceback (most recent call last): >> File >> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >> line 1923, in main >> getattr(buildout, command)(args) >> File >> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >> line 604, in install >> installed_files = self[part]._call(recipe.install) >> File >> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >> line 1358, in _call >> return f() >> File >> >> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >> line 126, in install >> reqs, ws = self.working_set() >> File >> >> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >> line 84, in working_set >> allow_hosts=self.allow_hosts) >> File >> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> line 782, in install >> return installer.install(specs, working_set) >> File >> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> line 626, in install >> for dist in self._get_dist(requirement, ws): >> File >> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> line 448, in _get_dist >> dist, avail = self._satisfied(requirement) >> File >> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> line 204, in _satisfied >> return None, self._obtain(req, source) >> File >> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> line 372, in _obtain >> if index.obtain(requirement) is None: >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 341, in obtain >> self.prescan(); self.find_packages(requirement) >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 326, in find_packages >> self.scan_url(self.index_url + requirement.unsafe_name+'/') >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 673, in scan_url >> self.process_url(url, True) >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 224, in process_url >> page = self.process_index(url, page) >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 301, in process_index >> self.scan_url(new_url) >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 673, in scan_url >> self.process_url(url, True) >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 202, in process_url >> f = self.open_url(url, "Download error on %s: %%s -- Some packages may >> not be found!" % url) >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 611, in open_url >> return open_with_auth(url) >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 807, in _socket_timeout >> return func(*args, **kwargs) >> File >> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> line 854, in open_with_auth >> fp = urllib2.urlopen(request) >> File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen >> return _opener.open(url, data, timeout) >> File "/usr/lib/python2.7/urllib2.py", line 400, in open >> response = self._open(req, data) >> File "/usr/lib/python2.7/urllib2.py", line 418, in _open >> '_open', req) >> File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain >> result = func(*args) >> File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open >> return self.do_open(httplib.HTTPConnection, req) >> File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open >> r = h.getresponse(buffering=True) >> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse >> response.begin() >> File "/usr/lib/python2.7/httplib.py", line 407, in begin >> version, status, reason = self._read_status() >> File "/usr/lib/python2.7/httplib.py", line 365, in _read_status >> line = self.fp.readline() >> File "/usr/lib/python2.7/socket.py", line 447, in readline >> data = self._sock.recv(self._rbufsize) >> timeout: timed out >> >> >> >> Avik Pal >> Bengal Engineering & Scieence University,Shibpur >> github:https://github.com/avikpal >> IRC:- irc://freenode/avikp,isnick >> twitter:-https://twitter.com/avikpalme >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > > From 24.sneha at gmail.com Sat Apr 13 21:41:39 2013 From: 24.sneha at gmail.com (Sneha Priscilla) Date: Sun, 14 Apr 2013 01:11:39 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: I'm using Ubuntu 11.04 and Python 2.7.4. Installing the web UI was a piece of cake. On Sun, Apr 14, 2013 at 1:06 AM, Avik Pal wrote: > yep, tried all those things. can you tell me which os you are using and > the version of python that you got? > > Avik Pal > Bengal Engineering & Scieence University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > > > > On 14 April 2013 00:49, Sneha Priscilla <24.sneha at gmail.com> wrote: > >> Hey Avik, >> >> Why don't you try re-branching the repository again and start over? Plus >> buildout instruction needs sudo access as far as I know, so make sure >> you're doing that. >> >> Priscilla >> >> >> On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal wrote: >> >>> Hello, >>> >>> I was following the steps given at >>> >>> http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut >>> I am getting the following error every time. please help. >>> >>> An internal error occured due to a bug in either zc.buildout or in a >>> recipe being used: >>> Traceback (most recent call last): >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>> line 1923, in main >>> getattr(buildout, command)(args) >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>> line 604, in install >>> installed_files = self[part]._call(recipe.install) >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>> line 1358, in _call >>> return f() >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>> line 126, in install >>> reqs, ws = self.working_set() >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>> line 84, in working_set >>> allow_hosts=self.allow_hosts) >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>> line 782, in install >>> return installer.install(specs, working_set) >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>> line 626, in install >>> for dist in self._get_dist(requirement, ws): >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>> line 448, in _get_dist >>> dist, avail = self._satisfied(requirement) >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>> line 204, in _satisfied >>> return None, self._obtain(req, source) >>> File >>> >>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>> line 372, in _obtain >>> if index.obtain(requirement) is None: >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 341, in obtain >>> self.prescan(); self.find_packages(requirement) >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 326, in find_packages >>> self.scan_url(self.index_url + requirement.unsafe_name+'/') >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 673, in scan_url >>> self.process_url(url, True) >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 224, in process_url >>> page = self.process_index(url, page) >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 301, in process_index >>> self.scan_url(new_url) >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 673, in scan_url >>> self.process_url(url, True) >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 202, in process_url >>> f = self.open_url(url, "Download error on %s: %%s -- Some packages >>> may >>> not be found!" % url) >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 611, in open_url >>> return open_with_auth(url) >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 807, in _socket_timeout >>> return func(*args, **kwargs) >>> File >>> >>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>> line 854, in open_with_auth >>> fp = urllib2.urlopen(request) >>> File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen >>> return _opener.open(url, data, timeout) >>> File "/usr/lib/python2.7/urllib2.py", line 400, in open >>> response = self._open(req, data) >>> File "/usr/lib/python2.7/urllib2.py", line 418, in _open >>> '_open', req) >>> File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain >>> result = func(*args) >>> File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open >>> return self.do_open(httplib.HTTPConnection, req) >>> File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open >>> r = h.getresponse(buffering=True) >>> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse >>> response.begin() >>> File "/usr/lib/python2.7/httplib.py", line 407, in begin >>> version, status, reason = self._read_status() >>> File "/usr/lib/python2.7/httplib.py", line 365, in _read_status >>> line = self.fp.readline() >>> File "/usr/lib/python2.7/socket.py", line 447, in readline >>> data = self._sock.recv(self._rbufsize) >>> timeout: timed out >>> >>> >>> >>> Avik Pal >>> Bengal Engineering & Scieence University,Shibpur >>> github:https://github.com/avikpal >>> IRC:- irc://freenode/avikp,isnick >>> twitter:-https://twitter.com/avikpalme >>> _______________________________________________ >>> Mailman-Developers mailing list >>> Mailman-Developers at python.org >>> http://mail.python.org/mailman/listinfo/mailman-developers >>> Mailman FAQ: http://wiki.list.org/x/AgA3 >>> Searchable Archives: >>> http://www.mail-archive.com/mailman-developers%40python.org/ >>> Unsubscribe: >>> http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com >>> >>> Security Policy: http://wiki.list.org/x/QIA9 >>> >> >> > From avikpal.me at gmail.com Sat Apr 13 21:47:43 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Sun, 14 Apr 2013 01:17:43 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: actually I'm using ubuntu 12.04, will try installing on a 11.04 vm. but still don 't know whats behind the errors. Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 14 April 2013 01:11, Sneha Priscilla <24.sneha at gmail.com> wrote: > I'm using Ubuntu 11.04 and Python 2.7.4. Installing the web UI was a piece > of cake. > > > On Sun, Apr 14, 2013 at 1:06 AM, Avik Pal wrote: > >> yep, tried all those things. can you tell me which os you are using and >> the version of python that you got? >> >> Avik Pal >> Bengal Engineering & Scieence University,Shibpur >> github:https://github.com/avikpal >> IRC:- irc://freenode/avikp,isnick >> twitter:-https://twitter.com/avikpalme >> >> >> >> >> >> On 14 April 2013 00:49, Sneha Priscilla <24.sneha at gmail.com> wrote: >> >>> Hey Avik, >>> >>> Why don't you try re-branching the repository again and start over? Plus >>> buildout instruction needs sudo access as far as I know, so make sure >>> you're doing that. >>> >>> Priscilla >>> >>> >>> On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal wrote: >>> >>>> Hello, >>>> >>>> I was following the steps given at >>>> >>>> http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut >>>> I am getting the following error every time. please help. >>>> >>>> An internal error occured due to a bug in either zc.buildout or in a >>>> recipe being used: >>>> Traceback (most recent call last): >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>> line 1923, in main >>>> getattr(buildout, command)(args) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>> line 604, in install >>>> installed_files = self[part]._call(recipe.install) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>> line 1358, in _call >>>> return f() >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>>> line 126, in install >>>> reqs, ws = self.working_set() >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>>> line 84, in working_set >>>> allow_hosts=self.allow_hosts) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>> line 782, in install >>>> return installer.install(specs, working_set) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>> line 626, in install >>>> for dist in self._get_dist(requirement, ws): >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>> line 448, in _get_dist >>>> dist, avail = self._satisfied(requirement) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>> line 204, in _satisfied >>>> return None, self._obtain(req, source) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>> line 372, in _obtain >>>> if index.obtain(requirement) is None: >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 341, in obtain >>>> self.prescan(); self.find_packages(requirement) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 326, in find_packages >>>> self.scan_url(self.index_url + requirement.unsafe_name+'/') >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 673, in scan_url >>>> self.process_url(url, True) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 224, in process_url >>>> page = self.process_index(url, page) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 301, in process_index >>>> self.scan_url(new_url) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 673, in scan_url >>>> self.process_url(url, True) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 202, in process_url >>>> f = self.open_url(url, "Download error on %s: %%s -- Some packages >>>> may >>>> not be found!" % url) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 611, in open_url >>>> return open_with_auth(url) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 807, in _socket_timeout >>>> return func(*args, **kwargs) >>>> File >>>> >>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>> line 854, in open_with_auth >>>> fp = urllib2.urlopen(request) >>>> File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen >>>> return _opener.open(url, data, timeout) >>>> File "/usr/lib/python2.7/urllib2.py", line 400, in open >>>> response = self._open(req, data) >>>> File "/usr/lib/python2.7/urllib2.py", line 418, in _open >>>> '_open', req) >>>> File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain >>>> result = func(*args) >>>> File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open >>>> return self.do_open(httplib.HTTPConnection, req) >>>> File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open >>>> r = h.getresponse(buffering=True) >>>> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse >>>> response.begin() >>>> File "/usr/lib/python2.7/httplib.py", line 407, in begin >>>> version, status, reason = self._read_status() >>>> File "/usr/lib/python2.7/httplib.py", line 365, in _read_status >>>> line = self.fp.readline() >>>> File "/usr/lib/python2.7/socket.py", line 447, in readline >>>> data = self._sock.recv(self._rbufsize) >>>> timeout: timed out >>>> >>>> >>>> >>>> Avik Pal >>>> Bengal Engineering & Scieence University,Shibpur >>>> github:https://github.com/avikpal >>>> IRC:- irc://freenode/avikp,isnick >>>> twitter:-https://twitter.com/avikpalme >>>> _______________________________________________ >>>> Mailman-Developers mailing list >>>> Mailman-Developers at python.org >>>> http://mail.python.org/mailman/listinfo/mailman-developers >>>> Mailman FAQ: http://wiki.list.org/x/AgA3 >>>> Searchable Archives: >>>> http://www.mail-archive.com/mailman-developers%40python.org/ >>>> Unsubscribe: >>>> http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com >>>> >>>> Security Policy: http://wiki.list.org/x/QIA9 >>>> >>> >>> >> > From 24.sneha at gmail.com Sat Apr 13 21:48:43 2013 From: 24.sneha at gmail.com (Sneha Priscilla) Date: Sun, 14 Apr 2013 01:18:43 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: It should work irrespective of the OS. Wait for sometime until someone experienced answers your question. On Sun, Apr 14, 2013 at 1:17 AM, Avik Pal wrote: > actually I'm using ubuntu 12.04, will try installing on a 11.04 vm. but > still don 't know whats behind the errors. > > > Avik Pal > Bengal Engineering & Scieence University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > > > > On 14 April 2013 01:11, Sneha Priscilla <24.sneha at gmail.com> wrote: > >> I'm using Ubuntu 11.04 and Python 2.7.4. Installing the web UI was a >> piece of cake. >> >> >> On Sun, Apr 14, 2013 at 1:06 AM, Avik Pal wrote: >> >>> yep, tried all those things. can you tell me which os you are using and >>> the version of python that you got? >>> >>> Avik Pal >>> Bengal Engineering & Scieence University,Shibpur >>> github:https://github.com/avikpal >>> IRC:- irc://freenode/avikp,isnick >>> twitter:-https://twitter.com/avikpalme >>> >>> >>> >>> >>> >>> On 14 April 2013 00:49, Sneha Priscilla <24.sneha at gmail.com> wrote: >>> >>>> Hey Avik, >>>> >>>> Why don't you try re-branching the repository again and start over? >>>> Plus buildout instruction needs sudo access as far as I know, so make sure >>>> you're doing that. >>>> >>>> Priscilla >>>> >>>> >>>> On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal wrote: >>>> >>>>> Hello, >>>>> >>>>> I was following the steps given at >>>>> >>>>> http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut >>>>> I am getting the following error every time. please help. >>>>> >>>>> An internal error occured due to a bug in either zc.buildout or in a >>>>> recipe being used: >>>>> Traceback (most recent call last): >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>>> line 1923, in main >>>>> getattr(buildout, command)(args) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>>> line 604, in install >>>>> installed_files = self[part]._call(recipe.install) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>>> line 1358, in _call >>>>> return f() >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>>>> line 126, in install >>>>> reqs, ws = self.working_set() >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>>>> line 84, in working_set >>>>> allow_hosts=self.allow_hosts) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>> line 782, in install >>>>> return installer.install(specs, working_set) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>> line 626, in install >>>>> for dist in self._get_dist(requirement, ws): >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>> line 448, in _get_dist >>>>> dist, avail = self._satisfied(requirement) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>> line 204, in _satisfied >>>>> return None, self._obtain(req, source) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>> line 372, in _obtain >>>>> if index.obtain(requirement) is None: >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 341, in obtain >>>>> self.prescan(); self.find_packages(requirement) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 326, in find_packages >>>>> self.scan_url(self.index_url + requirement.unsafe_name+'/') >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 673, in scan_url >>>>> self.process_url(url, True) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 224, in process_url >>>>> page = self.process_index(url, page) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 301, in process_index >>>>> self.scan_url(new_url) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 673, in scan_url >>>>> self.process_url(url, True) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 202, in process_url >>>>> f = self.open_url(url, "Download error on %s: %%s -- Some packages >>>>> may >>>>> not be found!" % url) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 611, in open_url >>>>> return open_with_auth(url) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 807, in _socket_timeout >>>>> return func(*args, **kwargs) >>>>> File >>>>> >>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>> line 854, in open_with_auth >>>>> fp = urllib2.urlopen(request) >>>>> File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen >>>>> return _opener.open(url, data, timeout) >>>>> File "/usr/lib/python2.7/urllib2.py", line 400, in open >>>>> response = self._open(req, data) >>>>> File "/usr/lib/python2.7/urllib2.py", line 418, in _open >>>>> '_open', req) >>>>> File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain >>>>> result = func(*args) >>>>> File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open >>>>> return self.do_open(httplib.HTTPConnection, req) >>>>> File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open >>>>> r = h.getresponse(buffering=True) >>>>> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse >>>>> response.begin() >>>>> File "/usr/lib/python2.7/httplib.py", line 407, in begin >>>>> version, status, reason = self._read_status() >>>>> File "/usr/lib/python2.7/httplib.py", line 365, in _read_status >>>>> line = self.fp.readline() >>>>> File "/usr/lib/python2.7/socket.py", line 447, in readline >>>>> data = self._sock.recv(self._rbufsize) >>>>> timeout: timed out >>>>> >>>>> >>>>> >>>>> Avik Pal >>>>> Bengal Engineering & Scieence University,Shibpur >>>>> github:https://github.com/avikpal >>>>> IRC:- irc://freenode/avikp,isnick >>>>> twitter:-https://twitter.com/avikpalme >>>>> _______________________________________________ >>>>> Mailman-Developers mailing list >>>>> Mailman-Developers at python.org >>>>> http://mail.python.org/mailman/listinfo/mailman-developers >>>>> Mailman FAQ: http://wiki.list.org/x/AgA3 >>>>> Searchable Archives: >>>>> http://www.mail-archive.com/mailman-developers%40python.org/ >>>>> Unsubscribe: >>>>> http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com >>>>> >>>>> Security Policy: http://wiki.list.org/x/QIA9 >>>>> >>>> >>>> >>> >> > From 24.sneha at gmail.com Sat Apr 13 21:50:09 2013 From: 24.sneha at gmail.com (Sneha Priscilla) Date: Sun, 14 Apr 2013 01:20:09 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: Plus, just a piece of advice, use something like http://pastebin.com/ to show your stack trace! :) On Sun, Apr 14, 2013 at 1:18 AM, Sneha Priscilla <24.sneha at gmail.com> wrote: > It should work irrespective of the OS. Wait for sometime until someone > experienced answers your question. > > > On Sun, Apr 14, 2013 at 1:17 AM, Avik Pal wrote: > >> actually I'm using ubuntu 12.04, will try installing on a 11.04 vm. but >> still don 't know whats behind the errors. >> >> >> Avik Pal >> Bengal Engineering & Scieence University,Shibpur >> github:https://github.com/avikpal >> IRC:- irc://freenode/avikp,isnick >> twitter:-https://twitter.com/avikpalme >> >> >> >> >> >> On 14 April 2013 01:11, Sneha Priscilla <24.sneha at gmail.com> wrote: >> >>> I'm using Ubuntu 11.04 and Python 2.7.4. Installing the web UI was a >>> piece of cake. >>> >>> >>> On Sun, Apr 14, 2013 at 1:06 AM, Avik Pal wrote: >>> >>>> yep, tried all those things. can you tell me which os you are using and >>>> the version of python that you got? >>>> >>>> Avik Pal >>>> Bengal Engineering & Scieence University,Shibpur >>>> github:https://github.com/avikpal >>>> IRC:- irc://freenode/avikp,isnick >>>> twitter:-https://twitter.com/avikpalme >>>> >>>> >>>> >>>> >>>> >>>> On 14 April 2013 00:49, Sneha Priscilla <24.sneha at gmail.com> wrote: >>>> >>>>> Hey Avik, >>>>> >>>>> Why don't you try re-branching the repository again and start over? >>>>> Plus buildout instruction needs sudo access as far as I know, so make sure >>>>> you're doing that. >>>>> >>>>> Priscilla >>>>> >>>>> >>>>> On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> I was following the steps given at >>>>>> >>>>>> http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut >>>>>> I am getting the following error every time. please help. >>>>>> >>>>>> An internal error occured due to a bug in either zc.buildout or in a >>>>>> recipe being used: >>>>>> Traceback (most recent call last): >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>>>> line 1923, in main >>>>>> getattr(buildout, command)(args) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>>>> line 604, in install >>>>>> installed_files = self[part]._call(recipe.install) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>>>> line 1358, in _call >>>>>> return f() >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>>>>> line 126, in install >>>>>> reqs, ws = self.working_set() >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>>>>> line 84, in working_set >>>>>> allow_hosts=self.allow_hosts) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 782, in install >>>>>> return installer.install(specs, working_set) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 626, in install >>>>>> for dist in self._get_dist(requirement, ws): >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 448, in _get_dist >>>>>> dist, avail = self._satisfied(requirement) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 204, in _satisfied >>>>>> return None, self._obtain(req, source) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 372, in _obtain >>>>>> if index.obtain(requirement) is None: >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 341, in obtain >>>>>> self.prescan(); self.find_packages(requirement) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 326, in find_packages >>>>>> self.scan_url(self.index_url + requirement.unsafe_name+'/') >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 673, in scan_url >>>>>> self.process_url(url, True) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 224, in process_url >>>>>> page = self.process_index(url, page) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 301, in process_index >>>>>> self.scan_url(new_url) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 673, in scan_url >>>>>> self.process_url(url, True) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 202, in process_url >>>>>> f = self.open_url(url, "Download error on %s: %%s -- Some >>>>>> packages may >>>>>> not be found!" % url) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 611, in open_url >>>>>> return open_with_auth(url) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 807, in _socket_timeout >>>>>> return func(*args, **kwargs) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 854, in open_with_auth >>>>>> fp = urllib2.urlopen(request) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen >>>>>> return _opener.open(url, data, timeout) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 400, in open >>>>>> response = self._open(req, data) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 418, in _open >>>>>> '_open', req) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain >>>>>> result = func(*args) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open >>>>>> return self.do_open(httplib.HTTPConnection, req) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open >>>>>> r = h.getresponse(buffering=True) >>>>>> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse >>>>>> response.begin() >>>>>> File "/usr/lib/python2.7/httplib.py", line 407, in begin >>>>>> version, status, reason = self._read_status() >>>>>> File "/usr/lib/python2.7/httplib.py", line 365, in _read_status >>>>>> line = self.fp.readline() >>>>>> File "/usr/lib/python2.7/socket.py", line 447, in readline >>>>>> data = self._sock.recv(self._rbufsize) >>>>>> timeout: timed out >>>>>> >>>>>> >>>>>> >>>>>> Avik Pal >>>>>> Bengal Engineering & Scieence University,Shibpur >>>>>> github:https://github.com/avikpal >>>>>> IRC:- irc://freenode/avikp,isnick >>>>>> twitter:-https://twitter.com/avikpalme >>>>>> _______________________________________________ >>>>>> Mailman-Developers mailing list >>>>>> Mailman-Developers at python.org >>>>>> http://mail.python.org/mailman/listinfo/mailman-developers >>>>>> Mailman FAQ: http://wiki.list.org/x/AgA3 >>>>>> Searchable Archives: >>>>>> http://www.mail-archive.com/mailman-developers%40python.org/ >>>>>> Unsubscribe: >>>>>> http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com >>>>>> >>>>>> Security Policy: http://wiki.list.org/x/QIA9 >>>>>> >>>>> >>>>> >>>> >>> >> > From sreyanth at gmail.com Sat Apr 13 21:51:04 2013 From: sreyanth at gmail.com (Sreyanth) Date: Sun, 14 Apr 2013 01:21:04 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: Hey Avik Try building again. Your internet connection may sometime pose problem too. It worked fine for me when tried for the second time! On Sun, Apr 14, 2013 at 1:17 AM, Avik Pal wrote: > actually I'm using ubuntu 12.04, will try installing on a 11.04 vm. but > still don 't know whats behind the errors. > > > Avik Pal > Bengal Engineering & Scieence University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > > > > On 14 April 2013 01:11, Sneha Priscilla <24.sneha at gmail.com> wrote: > > > I'm using Ubuntu 11.04 and Python 2.7.4. Installing the web UI was a > piece > > of cake. > > > > > > On Sun, Apr 14, 2013 at 1:06 AM, Avik Pal wrote: > > > >> yep, tried all those things. can you tell me which os you are using and > >> the version of python that you got? > >> > >> Avik Pal > >> Bengal Engineering & Scieence University,Shibpur > >> github:https://github.com/avikpal > >> IRC:- irc://freenode/avikp,isnick > >> twitter:-https://twitter.com/avikpalme > >> > >> > >> > >> > >> > >> On 14 April 2013 00:49, Sneha Priscilla <24.sneha at gmail.com> wrote: > >> > >>> Hey Avik, > >>> > >>> Why don't you try re-branching the repository again and start over? > Plus > >>> buildout instruction needs sudo access as far as I know, so make sure > >>> you're doing that. > >>> > >>> Priscilla > >>> > >>> > >>> On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal > wrote: > >>> > >>>> Hello, > >>>> > >>>> I was following the steps given at > >>>> > >>>> > http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut > >>>> I am getting the following error every time. please help. > >>>> > >>>> An internal error occured due to a bug in either zc.buildout or in a > >>>> recipe being used: > >>>> Traceback (most recent call last): > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > >>>> line 1923, in main > >>>> getattr(buildout, command)(args) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > >>>> line 604, in install > >>>> installed_files = self[part]._call(recipe.install) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > >>>> line 1358, in _call > >>>> return f() > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", > >>>> line 126, in install > >>>> reqs, ws = self.working_set() > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", > >>>> line 84, in working_set > >>>> allow_hosts=self.allow_hosts) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>> line 782, in install > >>>> return installer.install(specs, working_set) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>> line 626, in install > >>>> for dist in self._get_dist(requirement, ws): > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>> line 448, in _get_dist > >>>> dist, avail = self._satisfied(requirement) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>> line 204, in _satisfied > >>>> return None, self._obtain(req, source) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>> line 372, in _obtain > >>>> if index.obtain(requirement) is None: > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 341, in obtain > >>>> self.prescan(); self.find_packages(requirement) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 326, in find_packages > >>>> self.scan_url(self.index_url + requirement.unsafe_name+'/') > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 673, in scan_url > >>>> self.process_url(url, True) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 224, in process_url > >>>> page = self.process_index(url, page) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 301, in process_index > >>>> self.scan_url(new_url) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 673, in scan_url > >>>> self.process_url(url, True) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 202, in process_url > >>>> f = self.open_url(url, "Download error on %s: %%s -- Some packages > >>>> may > >>>> not be found!" % url) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 611, in open_url > >>>> return open_with_auth(url) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 807, in _socket_timeout > >>>> return func(*args, **kwargs) > >>>> File > >>>> > >>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>> line 854, in open_with_auth > >>>> fp = urllib2.urlopen(request) > >>>> File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen > >>>> return _opener.open(url, data, timeout) > >>>> File "/usr/lib/python2.7/urllib2.py", line 400, in open > >>>> response = self._open(req, data) > >>>> File "/usr/lib/python2.7/urllib2.py", line 418, in _open > >>>> '_open', req) > >>>> File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain > >>>> result = func(*args) > >>>> File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open > >>>> return self.do_open(httplib.HTTPConnection, req) > >>>> File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open > >>>> r = h.getresponse(buffering=True) > >>>> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse > >>>> response.begin() > >>>> File "/usr/lib/python2.7/httplib.py", line 407, in begin > >>>> version, status, reason = self._read_status() > >>>> File "/usr/lib/python2.7/httplib.py", line 365, in _read_status > >>>> line = self.fp.readline() > >>>> File "/usr/lib/python2.7/socket.py", line 447, in readline > >>>> data = self._sock.recv(self._rbufsize) > >>>> timeout: timed out > >>>> > >>>> > >>>> > >>>> Avik Pal > >>>> Bengal Engineering & Scieence University,Shibpur > >>>> github:https://github.com/avikpal > >>>> IRC:- irc://freenode/avikp,isnick > >>>> twitter:-https://twitter.com/avikpalme > >>>> _______________________________________________ > >>>> Mailman-Developers mailing list > >>>> Mailman-Developers at python.org > >>>> http://mail.python.org/mailman/listinfo/mailman-developers > >>>> Mailman FAQ: http://wiki.list.org/x/AgA3 > >>>> Searchable Archives: > >>>> http://www.mail-archive.com/mailman-developers%40python.org/ > >>>> Unsubscribe: > >>>> > http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com > >>>> > >>>> Security Policy: http://wiki.list.org/x/QIA9 > >>>> > >>> > >>> > >> > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/sreyanth%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- *Yours Sincerely* * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From avikpal.me at gmail.com Sat Apr 13 21:54:45 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Sun, 14 Apr 2013 01:24:45 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: I also think so...but you know after spending several hours I have started to think of all those "out box things" and also several distribution and package version changed with the 12.04 LTS like zope, storm distribute etc. Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 14 April 2013 01:18, Sneha Priscilla <24.sneha at gmail.com> wrote: > It should work irrespective of the OS. Wait for sometime until someone > experienced answers your question. > > > On Sun, Apr 14, 2013 at 1:17 AM, Avik Pal wrote: > >> actually I'm using ubuntu 12.04, will try installing on a 11.04 vm. but >> still don 't know whats behind the errors. >> >> >> Avik Pal >> Bengal Engineering & Scieence University,Shibpur >> github:https://github.com/avikpal >> IRC:- irc://freenode/avikp,isnick >> twitter:-https://twitter.com/avikpalme >> >> >> >> >> >> On 14 April 2013 01:11, Sneha Priscilla <24.sneha at gmail.com> wrote: >> >>> I'm using Ubuntu 11.04 and Python 2.7.4. Installing the web UI was a >>> piece of cake. >>> >>> >>> On Sun, Apr 14, 2013 at 1:06 AM, Avik Pal wrote: >>> >>>> yep, tried all those things. can you tell me which os you are using and >>>> the version of python that you got? >>>> >>>> Avik Pal >>>> Bengal Engineering & Scieence University,Shibpur >>>> github:https://github.com/avikpal >>>> IRC:- irc://freenode/avikp,isnick >>>> twitter:-https://twitter.com/avikpalme >>>> >>>> >>>> >>>> >>>> >>>> On 14 April 2013 00:49, Sneha Priscilla <24.sneha at gmail.com> wrote: >>>> >>>>> Hey Avik, >>>>> >>>>> Why don't you try re-branching the repository again and start over? >>>>> Plus buildout instruction needs sudo access as far as I know, so make sure >>>>> you're doing that. >>>>> >>>>> Priscilla >>>>> >>>>> >>>>> On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> I was following the steps given at >>>>>> >>>>>> http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut >>>>>> I am getting the following error every time. please help. >>>>>> >>>>>> An internal error occured due to a bug in either zc.buildout or in a >>>>>> recipe being used: >>>>>> Traceback (most recent call last): >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>>>> line 1923, in main >>>>>> getattr(buildout, command)(args) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>>>> line 604, in install >>>>>> installed_files = self[part]._call(recipe.install) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >>>>>> line 1358, in _call >>>>>> return f() >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>>>>> line 126, in install >>>>>> reqs, ws = self.working_set() >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >>>>>> line 84, in working_set >>>>>> allow_hosts=self.allow_hosts) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 782, in install >>>>>> return installer.install(specs, working_set) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 626, in install >>>>>> for dist in self._get_dist(requirement, ws): >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 448, in _get_dist >>>>>> dist, avail = self._satisfied(requirement) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 204, in _satisfied >>>>>> return None, self._obtain(req, source) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >>>>>> line 372, in _obtain >>>>>> if index.obtain(requirement) is None: >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 341, in obtain >>>>>> self.prescan(); self.find_packages(requirement) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 326, in find_packages >>>>>> self.scan_url(self.index_url + requirement.unsafe_name+'/') >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 673, in scan_url >>>>>> self.process_url(url, True) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 224, in process_url >>>>>> page = self.process_index(url, page) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 301, in process_index >>>>>> self.scan_url(new_url) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 673, in scan_url >>>>>> self.process_url(url, True) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 202, in process_url >>>>>> f = self.open_url(url, "Download error on %s: %%s -- Some >>>>>> packages may >>>>>> not be found!" % url) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 611, in open_url >>>>>> return open_with_auth(url) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 807, in _socket_timeout >>>>>> return func(*args, **kwargs) >>>>>> File >>>>>> >>>>>> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >>>>>> line 854, in open_with_auth >>>>>> fp = urllib2.urlopen(request) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen >>>>>> return _opener.open(url, data, timeout) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 400, in open >>>>>> response = self._open(req, data) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 418, in _open >>>>>> '_open', req) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain >>>>>> result = func(*args) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open >>>>>> return self.do_open(httplib.HTTPConnection, req) >>>>>> File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open >>>>>> r = h.getresponse(buffering=True) >>>>>> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse >>>>>> response.begin() >>>>>> File "/usr/lib/python2.7/httplib.py", line 407, in begin >>>>>> version, status, reason = self._read_status() >>>>>> File "/usr/lib/python2.7/httplib.py", line 365, in _read_status >>>>>> line = self.fp.readline() >>>>>> File "/usr/lib/python2.7/socket.py", line 447, in readline >>>>>> data = self._sock.recv(self._rbufsize) >>>>>> timeout: timed out >>>>>> >>>>>> >>>>>> >>>>>> Avik Pal >>>>>> Bengal Engineering & Scieence University,Shibpur >>>>>> github:https://github.com/avikpal >>>>>> IRC:- irc://freenode/avikp,isnick >>>>>> twitter:-https://twitter.com/avikpalme >>>>>> _______________________________________________ >>>>>> Mailman-Developers mailing list >>>>>> Mailman-Developers at python.org >>>>>> http://mail.python.org/mailman/listinfo/mailman-developers >>>>>> Mailman FAQ: http://wiki.list.org/x/AgA3 >>>>>> Searchable Archives: >>>>>> http://www.mail-archive.com/mailman-developers%40python.org/ >>>>>> Unsubscribe: >>>>>> http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com >>>>>> >>>>>> Security Policy: http://wiki.list.org/x/QIA9 >>>>>> >>>>> >>>>> >>>> >>> >> > From sreyanth at gmail.com Sat Apr 13 22:00:40 2013 From: sreyanth at gmail.com (Sreyanth) Date: Sun, 14 Apr 2013 01:30:40 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: They might have changed. I agree. But we expect them to maintain compatibility (they call it reverse compatibility I guess, I forgot!) with the previous versions. So, it shouldn't be a problem anyhow. If the problem still persists, there are experts in the community who will be able to help you out! :) On Sun, Apr 14, 2013 at 1:24 AM, Avik Pal wrote: > I also think so...but you know after spending several hours I have started > to think of all those "out box things" and also several distribution and > package version changed with the 12.04 LTS like zope, storm distribute > etc. > > > Avik Pal > Bengal Engineering & Scieence University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > > > > On 14 April 2013 01:18, Sneha Priscilla <24.sneha at gmail.com> wrote: > > > It should work irrespective of the OS. Wait for sometime until someone > > experienced answers your question. > > > > > > On Sun, Apr 14, 2013 at 1:17 AM, Avik Pal wrote: > > > >> actually I'm using ubuntu 12.04, will try installing on a 11.04 vm. but > >> still don 't know whats behind the errors. > >> > >> > >> Avik Pal > >> Bengal Engineering & Scieence University,Shibpur > >> github:https://github.com/avikpal > >> IRC:- irc://freenode/avikp,isnick > >> twitter:-https://twitter.com/avikpalme > >> > >> > >> > >> > >> > >> On 14 April 2013 01:11, Sneha Priscilla <24.sneha at gmail.com> wrote: > >> > >>> I'm using Ubuntu 11.04 and Python 2.7.4. Installing the web UI was a > >>> piece of cake. > >>> > >>> > >>> On Sun, Apr 14, 2013 at 1:06 AM, Avik Pal > wrote: > >>> > >>>> yep, tried all those things. can you tell me which os you are using > and > >>>> the version of python that you got? > >>>> > >>>> Avik Pal > >>>> Bengal Engineering & Scieence University,Shibpur > >>>> github:https://github.com/avikpal > >>>> IRC:- irc://freenode/avikp,isnick > >>>> twitter:-https://twitter.com/avikpalme > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> On 14 April 2013 00:49, Sneha Priscilla <24.sneha at gmail.com> wrote: > >>>> > >>>>> Hey Avik, > >>>>> > >>>>> Why don't you try re-branching the repository again and start over? > >>>>> Plus buildout instruction needs sudo access as far as I know, so > make sure > >>>>> you're doing that. > >>>>> > >>>>> Priscilla > >>>>> > >>>>> > >>>>> On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal >wrote: > >>>>> > >>>>>> Hello, > >>>>>> > >>>>>> I was following the steps given at > >>>>>> > >>>>>> > http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut > >>>>>> I am getting the following error every time. please help. > >>>>>> > >>>>>> An internal error occured due to a bug in either zc.buildout or in a > >>>>>> recipe being used: > >>>>>> Traceback (most recent call last): > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > >>>>>> line 1923, in main > >>>>>> getattr(buildout, command)(args) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > >>>>>> line 604, in install > >>>>>> installed_files = self[part]._call(recipe.install) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", > >>>>>> line 1358, in _call > >>>>>> return f() > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", > >>>>>> line 126, in install > >>>>>> reqs, ws = self.working_set() > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", > >>>>>> line 84, in working_set > >>>>>> allow_hosts=self.allow_hosts) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>>>> line 782, in install > >>>>>> return installer.install(specs, working_set) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>>>> line 626, in install > >>>>>> for dist in self._get_dist(requirement, ws): > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>>>> line 448, in _get_dist > >>>>>> dist, avail = self._satisfied(requirement) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>>>> line 204, in _satisfied > >>>>>> return None, self._obtain(req, source) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", > >>>>>> line 372, in _obtain > >>>>>> if index.obtain(requirement) is None: > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 341, in obtain > >>>>>> self.prescan(); self.find_packages(requirement) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 326, in find_packages > >>>>>> self.scan_url(self.index_url + requirement.unsafe_name+'/') > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 673, in scan_url > >>>>>> self.process_url(url, True) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 224, in process_url > >>>>>> page = self.process_index(url, page) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 301, in process_index > >>>>>> self.scan_url(new_url) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 673, in scan_url > >>>>>> self.process_url(url, True) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 202, in process_url > >>>>>> f = self.open_url(url, "Download error on %s: %%s -- Some > >>>>>> packages may > >>>>>> not be found!" % url) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 611, in open_url > >>>>>> return open_with_auth(url) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 807, in _socket_timeout > >>>>>> return func(*args, **kwargs) > >>>>>> File > >>>>>> > >>>>>> > "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", > >>>>>> line 854, in open_with_auth > >>>>>> fp = urllib2.urlopen(request) > >>>>>> File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen > >>>>>> return _opener.open(url, data, timeout) > >>>>>> File "/usr/lib/python2.7/urllib2.py", line 400, in open > >>>>>> response = self._open(req, data) > >>>>>> File "/usr/lib/python2.7/urllib2.py", line 418, in _open > >>>>>> '_open', req) > >>>>>> File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain > >>>>>> result = func(*args) > >>>>>> File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open > >>>>>> return self.do_open(httplib.HTTPConnection, req) > >>>>>> File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open > >>>>>> r = h.getresponse(buffering=True) > >>>>>> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse > >>>>>> response.begin() > >>>>>> File "/usr/lib/python2.7/httplib.py", line 407, in begin > >>>>>> version, status, reason = self._read_status() > >>>>>> File "/usr/lib/python2.7/httplib.py", line 365, in _read_status > >>>>>> line = self.fp.readline() > >>>>>> File "/usr/lib/python2.7/socket.py", line 447, in readline > >>>>>> data = self._sock.recv(self._rbufsize) > >>>>>> timeout: timed out > >>>>>> > >>>>>> > >>>>>> > >>>>>> Avik Pal > >>>>>> Bengal Engineering & Scieence University,Shibpur > >>>>>> github:https://github.com/avikpal > >>>>>> IRC:- irc://freenode/avikp,isnick > >>>>>> twitter:-https://twitter.com/avikpalme > >>>>>> _______________________________________________ > >>>>>> Mailman-Developers mailing list > >>>>>> Mailman-Developers at python.org > >>>>>> http://mail.python.org/mailman/listinfo/mailman-developers > >>>>>> Mailman FAQ: http://wiki.list.org/x/AgA3 > >>>>>> Searchable Archives: > >>>>>> http://www.mail-archive.com/mailman-developers%40python.org/ > >>>>>> Unsubscribe: > >>>>>> > http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com > >>>>>> > >>>>>> Security Policy: http://wiki.list.org/x/QIA9 > >>>>>> > >>>>> > >>>>> > >>>> > >>> > >> > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/sreyanth%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- *Yours Sincerely* * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From avikpal.me at gmail.com Sat Apr 13 22:07:27 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Sun, 14 Apr 2013 01:37:27 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: ya thanx sneha and sreyanth, hopfully other community members will be able to help me out of this situation. Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 14 April 2013 01:30, Sreyanth wrote: > They might have changed. I agree. But we expect them to maintain > compatibility (they call it reverse compatibility I guess, I forgot!) with > the previous versions. > So, it shouldn't be a problem anyhow. If the problem still persists, there > are experts in the community who will be able to help you out! :) > > > On Sun, Apr 14, 2013 at 1:24 AM, Avik Pal wrote: > >> I also think so...but you know after spending several hours I have started >> to think of all those "out box things" and also several distribution and >> package version changed with the 12.04 LTS like zope, storm distribute >> etc. >> >> >> Avik Pal >> Bengal Engineering & Scieence University,Shibpur >> github:https://github.com/avikpal >> IRC:- irc://freenode/avikp,isnick >> twitter:-https://twitter.com/avikpalme >> >> >> >> >> >> On 14 April 2013 01:18, Sneha Priscilla <24.sneha at gmail.com> wrote: >> >> > It should work irrespective of the OS. Wait for sometime until someone >> > experienced answers your question. >> > >> > >> > On Sun, Apr 14, 2013 at 1:17 AM, Avik Pal wrote: >> > >> >> actually I'm using ubuntu 12.04, will try installing on a 11.04 vm. but >> >> still don 't know whats behind the errors. >> >> >> >> >> >> Avik Pal >> >> Bengal Engineering & Scieence University,Shibpur >> >> github:https://github.com/avikpal >> >> IRC:- irc://freenode/avikp,isnick >> >> twitter:-https://twitter.com/avikpalme >> >> >> >> >> >> >> >> >> >> >> >> On 14 April 2013 01:11, Sneha Priscilla <24.sneha at gmail.com> wrote: >> >> >> >>> I'm using Ubuntu 11.04 and Python 2.7.4. Installing the web UI was a >> >>> piece of cake. >> >>> >> >>> >> >>> On Sun, Apr 14, 2013 at 1:06 AM, Avik Pal >> wrote: >> >>> >> >>>> yep, tried all those things. can you tell me which os you are using >> and >> >>>> the version of python that you got? >> >>>> >> >>>> Avik Pal >> >>>> Bengal Engineering & Scieence University,Shibpur >> >>>> github:https://github.com/avikpal >> >>>> IRC:- irc://freenode/avikp,isnick >> >>>> twitter:-https://twitter.com/avikpalme >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> On 14 April 2013 00:49, Sneha Priscilla <24.sneha at gmail.com> wrote: >> >>>> >> >>>>> Hey Avik, >> >>>>> >> >>>>> Why don't you try re-branching the repository again and start over? >> >>>>> Plus buildout instruction needs sudo access as far as I know, so >> make sure >> >>>>> you're doing that. >> >>>>> >> >>>>> Priscilla >> >>>>> >> >>>>> >> >>>>> On Sun, Apr 14, 2013 at 12:26 AM, Avik Pal > >wrote: >> >>>>> >> >>>>>> Hello, >> >>>>>> >> >>>>>> I was following the steps given at >> >>>>>> >> >>>>>> >> http://wiki.list.org/display/DEV/A+5+minute+guide+to+get+the+Mailman+web+UI+runningbut >> >>>>>> I am getting the following error every time. please help. >> >>>>>> >> >>>>>> An internal error occured due to a bug in either zc.buildout or in >> a >> >>>>>> recipe being used: >> >>>>>> Traceback (most recent call last): >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >> >>>>>> line 1923, in main >> >>>>>> getattr(buildout, command)(args) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >> >>>>>> line 604, in install >> >>>>>> installed_files = self[part]._call(recipe.install) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", >> >>>>>> line 1358, in _call >> >>>>>> return f() >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >> >>>>>> line 126, in install >> >>>>>> reqs, ws = self.working_set() >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", >> >>>>>> line 84, in working_set >> >>>>>> allow_hosts=self.allow_hosts) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> >>>>>> line 782, in install >> >>>>>> return installer.install(specs, working_set) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> >>>>>> line 626, in install >> >>>>>> for dist in self._get_dist(requirement, ws): >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> >>>>>> line 448, in _get_dist >> >>>>>> dist, avail = self._satisfied(requirement) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> >>>>>> line 204, in _satisfied >> >>>>>> return None, self._obtain(req, source) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", >> >>>>>> line 372, in _obtain >> >>>>>> if index.obtain(requirement) is None: >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 341, in obtain >> >>>>>> self.prescan(); self.find_packages(requirement) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 326, in find_packages >> >>>>>> self.scan_url(self.index_url + requirement.unsafe_name+'/') >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 673, in scan_url >> >>>>>> self.process_url(url, True) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 224, in process_url >> >>>>>> page = self.process_index(url, page) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 301, in process_index >> >>>>>> self.scan_url(new_url) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 673, in scan_url >> >>>>>> self.process_url(url, True) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 202, in process_url >> >>>>>> f = self.open_url(url, "Download error on %s: %%s -- Some >> >>>>>> packages may >> >>>>>> not be found!" % url) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 611, in open_url >> >>>>>> return open_with_auth(url) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 807, in _socket_timeout >> >>>>>> return func(*args, **kwargs) >> >>>>>> File >> >>>>>> >> >>>>>> >> "/home/avik/proj/mailman/eggs/distribute-0.6.36-py2.7.egg/setuptools/package_index.py", >> >>>>>> line 854, in open_with_auth >> >>>>>> fp = urllib2.urlopen(request) >> >>>>>> File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen >> >>>>>> return _opener.open(url, data, timeout) >> >>>>>> File "/usr/lib/python2.7/urllib2.py", line 400, in open >> >>>>>> response = self._open(req, data) >> >>>>>> File "/usr/lib/python2.7/urllib2.py", line 418, in _open >> >>>>>> '_open', req) >> >>>>>> File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain >> >>>>>> result = func(*args) >> >>>>>> File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open >> >>>>>> return self.do_open(httplib.HTTPConnection, req) >> >>>>>> File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open >> >>>>>> r = h.getresponse(buffering=True) >> >>>>>> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse >> >>>>>> response.begin() >> >>>>>> File "/usr/lib/python2.7/httplib.py", line 407, in begin >> >>>>>> version, status, reason = self._read_status() >> >>>>>> File "/usr/lib/python2.7/httplib.py", line 365, in _read_status >> >>>>>> line = self.fp.readline() >> >>>>>> File "/usr/lib/python2.7/socket.py", line 447, in readline >> >>>>>> data = self._sock.recv(self._rbufsize) >> >>>>>> timeout: timed out >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Avik Pal >> >>>>>> Bengal Engineering & Scieence University,Shibpur >> >>>>>> github:https://github.com/avikpal >> >>>>>> IRC:- irc://freenode/avikp,isnick >> >>>>>> twitter:-https://twitter.com/avikpalme >> >>>>>> _______________________________________________ >> >>>>>> Mailman-Developers mailing list >> >>>>>> Mailman-Developers at python.org >> >>>>>> http://mail.python.org/mailman/listinfo/mailman-developers >> >>>>>> Mailman FAQ: http://wiki.list.org/x/AgA3 >> >>>>>> Searchable Archives: >> >>>>>> http://www.mail-archive.com/mailman-developers%40python.org/ >> >>>>>> Unsubscribe: >> >>>>>> >> http://mail.python.org/mailman/options/mailman-developers/24.sneha%40gmail.com >> >>>>>> >> >>>>>> Security Policy: http://wiki.list.org/x/QIA9 >> >>>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >> >> > >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/sreyanth%40gmail.com >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > > > > -- > *Yours Sincerely* > * > * > *Mora Sreyantha Chary* > *Computer Engineering '14* > *National Institute of Technology Karnataka* > *Surathkal, India 575 025* > From mark at msapiro.net Sat Apr 13 22:22:01 2013 From: mark at msapiro.net (Mark Sapiro) Date: Sat, 13 Apr 2013 13:22:01 -0700 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: Message-ID: Avik Pal wrote: >ya thanx sneha and sreyanth, hopfully other community members will be able >to help me out of this situation. The actual exception in your original post was a timeout trying to open an internet connection to retrieve some package. Are you behind some firewall or is there some other problem preventing you from connecting to PyPI or wherever it's going? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From avikpal.me at gmail.com Sun Apr 14 06:04:32 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Sun, 14 Apr 2013 09:34:32 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: actually I think the following is behind all these... *Download error on http://ish.io/projects/show/restish: [Errno -2] Name or service not known -- Some packages may not be found!* Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 14 April 2013 01:52, Mark Sapiro wrote: > Avik Pal wrote: > > >ya thanx sneha and sreyanth, hopfully other community members will be able > >to help me out of this situation. > > > The actual exception in your original post was a timeout trying to open > an internet connection to retrieve some package. Are you behind some > firewall or is there some other problem preventing you from connecting > to PyPI or wherever it's going? > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > > From pscollins at uchicago.edu Sun Apr 14 10:52:49 2013 From: pscollins at uchicago.edu (Patrick Collins) Date: Sun, 14 Apr 2013 03:52:49 -0500 Subject: [Mailman-Developers] GSOC Introduction and Bugfix Question Message-ID: Hello, My name is Patrick Collins and I'm a second year computer science major, interested in applying to GSOC via Mailman. I was thinking of taking on the following projects: 1) Boilerplate stripper 2) Log monitor I imagine that (1) would be pretty easy to implement and take no more than a week. (2) is more complex, but offers extensibility: I could aim to implement a log monitor that, at the bare minimum, warns the admins in case of trouble, and then go on to extend it with some visualizations like graphs that track subscribers/country, posts/subscriber, and so on. This would give admins the added benefit of being able to clearly see trends in the logs that indicate issues. Would this be be complex enough to merit a GSOC project? Additionally, I wanted to implement a simple bug fix in order to get familiar with the Launchpad system. https://bugs.launchpad.net/mailman/+bug/558274 seems to be a pretty quick one. Should I write tests to accompany my changes to the code, or just perform them on my own machine? Should there be some documentation of testing submitted? -- Patrick Collins University of Chicago '15 From avikpal.me at gmail.com Sun Apr 14 17:11:15 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Sun, 14 Apr 2013 20:41:15 +0530 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: at last resolved the issue, had to install *lazr.smtptest 2.0* separately, now as the the web UI is up and running its time to try out few things :) Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 14 April 2013 09:34, Avik Pal wrote: > actually I think the following is behind all these... > > *Download error on http://ish.io/projects/show/restish: [Errno -2] Name > or service not known -- Some packages may not be found!* > > Avik Pal > Bengal Engineering & Scieence University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > > > > On 14 April 2013 01:52, Mark Sapiro wrote: > >> Avik Pal wrote: >> >> >ya thanx sneha and sreyanth, hopfully other community members will be >> able >> >to help me out of this situation. >> >> >> The actual exception in your original post was a timeout trying to open >> an internet connection to retrieve some package. Are you behind some >> firewall or is there some other problem preventing you from connecting >> to PyPI or wherever it's going? >> >> -- >> Mark Sapiro The highway is for gamblers, >> San Francisco Bay Area, California better use your sense - B. Dylan >> >> > From paul at boddie.org.uk Sun Apr 14 23:51:52 2013 From: paul at boddie.org.uk (Paul Boddie) Date: Sun, 14 Apr 2013 23:51:52 +0200 Subject: [Mailman-Developers] Wiki Migration Update Message-ID: <201304142351.52687.paul@boddie.org.uk> Hello, It's been about a month or so since my last update, so here's a quick summary of the work done since that time on the content migration from Confluence to MoinMoin. First of all, I've uploaded a newer version of the content here: http://mmwiki.boddie.org.uk/ Not only has the conversion itself been improved, but the underlying Confluence content has also been updated, so it should reflect the existing Wiki (at http://wiki.list.org/) a bit more accurately. I also fought with mod_rewrite to at least make most pages accessible, particularly the FAQ-related pages which contain question marks in their titles (and with which mod_rewrite has an as-yet-unfixed problem). A lot of the recent work has gone into fixing the XHTML-based markup used by Confluence since version 4. In particular, links, lists, tables and preformatted regions have seen quite a bit of attention, and various pages should now look better due to this. For example: http://mmwiki.boddie.org.uk/DOC/Mailman_2.1_Members_Manual (Note that the above page has some fairly strange nested lists at the end where the bulletpoints have been added manually as "o" characters after newlines, and this is not an artifact of my conversion, but the markup is handled sufficiently well to retain the desired visual effect.) Some things that were mentioned as problem areas have been improved: http://mmwiki.boddie.org.uk/DEV/A_5_Minute_Guide_to_Get_the_Mailman_Web_UI_Running_%28only_for_development%29 (This now has properly preformatted regions, although I aim to improve the handling of single-line regions still further.) http://mmwiki.boddie.org.uk/COM/donate_to_the_GNU_Mailman_project (This now has a fixed link, although the Confluence markup version still needs improving to work correctly.) http://mmwiki.boddie.org.uk/DOC/Frequently_Asked_Questions (The links to questions now have tidier labels, and my mod_rewrite hacks mean that you can follow them on the hosted site.) A few things haven't yet been done since the last update that were worth mentioning before: http://mmwiki.boddie.org.uk/COM/Organizations_that_use_Mailman (I still need to make sure that the anchors are generated for the headings.) Author information is still not preserved in the page import process, but this will be tested in future. I want to start implementing macro support and to tidy up the way comments are presented on pages. It might also be nice to have a list of attachments on pages that have them, and I will take a look to see how Confluence tends to present such things. As I have mentioned previously, the source code for the converter can be found here: http://hgweb.boddie.org.uk/ConfluenceConverter/ Please take a look at your favourite pages and let me know where improvements can be made to the conversion process. Paul From stephen at xemacs.org Mon Apr 15 06:31:55 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 13:31:55 +0900 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> Message-ID: <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> Sreyanth writes: > Also, I would like to hear more about : Boilerplate stripper AND Better > content-filtering / handling error messages. > ?Boilerplate stripping is trivial to understand. But, can anyone elaborate > on Better content-filtering / handling error messages? But boilerplate stripping is not necessarily trivial to implement, because it's not always clear what boilerplate is. I think it might be a good idea to save it off and provide a link rather than discard it, which leads to interesting questions of storage, shared links for true boilerplate (storage compression of repeatedly encountered text, yes, but more important the link will turn purple so you don't need to click on it in the next message from that user!), and user interface in general. Content filtering is mostly going to be about MIME handling: choice of the appropriate text/* part and things like that, removing images/video/etc where the list prohibits them, converting HTML/ wordprocessor attachments to plain text, removing MIME parts whose Content-Type doesn't match filename or perhaps file(1) magic in the content, etc. I can also imagine content filtering (or scoring!) based on word choice ("WTF" OK, spelling it out not :-). Also content filtering based on stripping out the quoted from top-posts and replacing them with links (after checking that the quoted material is indeed available in the archive!) All coming with on/off options, at least for those who remember the IBM 360saurus and other dinosaurs and still prefer mail to web. :-) Error messages (I think this means delivery status notifications (DSN) from mail servers) are a similar kind of problem to text-based filtering, though somewhat more stylized. Steve From iampratiksarkar at gmail.com Mon Apr 15 07:39:55 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Sun, 14 Apr 2013 22:39:55 -0700 Subject: [Mailman-Developers] anti-spam filter Message-ID: Hi guys, Is the anti-spam/abuse filter still being seriously considered as a gsoc project this year? From p at sys4.de Mon Apr 15 08:26:38 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Mon, 15 Apr 2013 08:26:38 +0200 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130415062636.GA9945@sys4.de> * Stephen J. Turnbull : > Sreyanth writes: > > > Also, I would like to hear more about : Boilerplate stripper AND Better > > content-filtering / handling error messages. > > ?Boilerplate stripping is trivial to understand. But, can anyone elaborate > > on Better content-filtering / handling error messages? > > But boilerplate stripping is not necessarily trivial to implement, > because it's not always clear what boilerplate is. I think it might > be a good idea to save it off and provide a link rather than discard > it, which leads to interesting questions of storage, shared links for > true boilerplate (storage compression of repeatedly encountered text, > yes, but more important the link will turn purple so you don't need to > click on it in the next message from that user!), and user interface > in general. > > Content filtering is mostly going to be about MIME handling: choice of > the appropriate text/* part and things like that, removing > images/video/etc where the list prohibits them, converting HTML/ > wordprocessor attachments to plain text, removing MIME parts whose > Content-Type doesn't match filename or perhaps file(1) magic in the > content, etc. Just to mention it: IF we are going to add MILTER functionality, a MILTER would be perfect to do MIME handling. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stephen at xemacs.org Mon Apr 15 08:42:46 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 15:42:46 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: Message-ID: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> Pratik Sarkar writes: > Is the anti-spam/abuse filter still being seriously considered as a > gsoc project this year? I would say so, yes. Personally, I am fundamentally opposed to it; I think it's wrong in principle (filtering of this kind should be done by the incoming MTA) and inappropriate for the 3.0 release. *But* there is clearly user demand for it, and among the actually signed-up mentors[1] there are at least two who have shown some support for it. OTOH, you should be aware that while nobody has a veto except Barry (Terri has one ex oficio but I doubt she'd exercise it if Barry was in favor), it's likely that projects that all the mentors favor will be ranked higher, and looking at the quality of posts from students so far there is going to be competition for Mailman slots. If you have a solid proposal for anti-spam already worked out (the idea the guy proposed with a Bayesian filter based on word triads comes close to what I'd call solid, see also Terri's post where she proposed a schedule for the kind of additional detail needed), then that's probably your best bet. But if you're looking for a project and you think that anti-spam is cool but that's all the thinking you've done so far, I'd say it's very risky proposal. There are a lot of things we need in UI (both subscriber-oriented and admin-oriented) that are interesting and higher-priority. Footnotes: [1] Terri (org admin), Barry (project lead), Wacky, Pingu, Florian, and me. From sreyanth at gmail.com Mon Apr 15 09:35:12 2013 From: sreyanth at gmail.com (Sreyanth) Date: Mon, 15 Apr 2013 13:05:12 +0530 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Mon, Apr 15, 2013 at 10:01 AM, Stephen J. Turnbull wrote: > Sreyanth writes: > > > Also, I would like to hear more about : Boilerplate stripper AND Better > > content-filtering / handling error messages. > > ?Boilerplate stripping is trivial to understand. But, can anyone > elaborate > > on Better content-filtering / handling error messages? > > But boilerplate stripping is not necessarily trivial to implement, > because it's not always clear what boilerplate is. I think it might > be a good idea to save it off and provide a link rather than discard > it, which leads to interesting questions of storage, shared links for > true boilerplate (storage compression of repeatedly encountered text, > yes, but more important the link will turn purple so you don't need to > click on it in the next message from that user!), and user interface > in general. > > Yep! But, how about this? Just hide the boilerplate in the email, giving a link to click. When clicked, use js to unhide the boilerplate. This would not anyhow require separate storage. Suggest me something if this is bad! ?? > Content filtering is mostly going to be about MIME handling: choice of > the appropriate text/* part and things like that, removing > images/video/etc where the list prohibits them, converting HTML/ > wordprocessor attachments to plain text, removing MIME parts whose > Content-Type doesn't match filename or perhaps file(1) magic in the > content, etc. > This is cool! I have done converting the HTML attachments to plain text in one of my projects, but never worked on wordprocessor attachments (some library for Python should be there, will check!). So, for this, we will additionally have to provide the admin with various other options, like to prohibit images, videos etc. Instead an additional option may also be given to the admin like using the boilerplate idea you proposed. Store them somewhere, display these as attachments to the email.?? Correct me if I am in the wrong path! > I can also imagine content filtering (or scoring!) based on word > choice ("WTF" OK, spelling it out not :-). Also content filtering > based on stripping out the quoted from top-posts and replacing them > with links (after checking that the quoted material is indeed > available in the archive!) All coming with on/off options, at least > for those who remember the IBM 360saurus and other dinosaurs and still > prefer mail to web. :-) > So, we will have to index the archives properly so that even if the post is not quoted properly in the email, it must be linked to the appropriate material. This will be cool, isnt it??? > Error messages (I think this means delivery status notifications (DSN) > from mail servers) are a similar kind of problem to text-based > filtering, though somewhat more stylized. > > ?Okay! So, these error messages are to be notified to the sender in a more user understandable fashion. This is what you meant right?? > Steve > -- *Yours Sincerely* * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From pabs at debian.org Mon Apr 15 09:45:52 2013 From: pabs at debian.org (Paul Wise) Date: Mon, 15 Apr 2013 15:45:52 +0800 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: Message-ID: On Mon, Apr 15, 2013 at 1:39 PM, Pratik Sarkar wrote: > Is the anti-spam/abuse filter still being seriously considered > as a gsoc project this year? One really useful thing to have would be removal of spam from archives. Here are some examples of how it has been done before: The Debian lists server isn't based on mailman but has a collaborative method of removing spam from the archive: https://wiki.debian.org/Teams/ListMaster/ListArchiveSpam Indymedia implemented a mailman 2 patch for this a number of years ago: https://lists.indymedia.org/patches/imc-10-mmid_hide_posts.patch -- bye, pabs http://bonedaddy.net/pabs3/ From sreyanth at gmail.com Mon Apr 15 09:43:47 2013 From: sreyanth at gmail.com (Sreyanth) Date: Mon, 15 Apr 2013 13:13:47 +0530 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: <20130415062636.GA9945@sys4.de> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415062636.GA9945@sys4.de> Message-ID: On Mon, Apr 15, 2013 at 11:56 AM, Patrick Ben Koetter

wrote: > * Stephen J. Turnbull : > > Sreyanth writes: > > > > > Also, I would like to hear more about : Boilerplate stripper AND > Better > > > content-filtering / handling error messages. > > > ?Boilerplate stripping is trivial to understand. But, can anyone > elaborate > > > on Better content-filtering / handling error messages? > > > > But boilerplate stripping is not necessarily trivial to implement, > > because it's not always clear what boilerplate is. I think it might > > be a good idea to save it off and provide a link rather than discard > > it, which leads to interesting questions of storage, shared links for > > true boilerplate (storage compression of repeatedly encountered text, > > yes, but more important the link will turn purple so you don't need to > > click on it in the next message from that user!), and user interface > > in general. > > > > Content filtering is mostly going to be about MIME handling: choice of > > the appropriate text/* part and things like that, removing > > images/video/etc where the list prohibits them, converting HTML/ > > wordprocessor attachments to plain text, removing MIME parts whose > > Content-Type doesn't match filename or perhaps file(1) magic in the > > content, etc. > > Just to mention it: > IF we are going to add MILTER functionality, a MILTER would be perfect to > do MIME handling. > > ?If we are going to add a MILTER functionality, even anti-spam filters can be at the least implemented. Isn't it? Some days ago, we were discussing about MILTERs in anti-spam context right. Now, a piece of anti-spam AND anti-abuse can be implemented at this level! I have implemented a binary Bayesian classifier which classifies an email either spam or not spam. Using it, making use of the main keywords in the email as vectors and learning from the reportedly-spam emails from the logs, we can implement this. After classifying an email as spam, we can display a line, may be, as "This may be spam. Please be careful while clicking on links or replying to this email with sensitive information!". So, using this we can enhance the usage of MILTER at the same time doing the MIME handling. Correct me if I am wrong. :)? > p at rick > > -- > [*] sys4 AG > > http://sys4.de, +49 (89) 30 90 46 64 > Franziskanerstra?e 15, 81669 M?nchen > > Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 > Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer > Aufsichtsratsvorsitzender: Joerg Heidrich > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/sreyanth%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 -- *Yours Sincerely* * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From stephen at xemacs.org Mon Apr 15 10:55:33 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 17:55:33 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: Message-ID: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> Paul Wise writes: > One really useful thing to have would be removal of spam from > archives. Here are some examples of how it has been done before: That's a great idea (and a great example of why I don't think spam-filtering technology should be embedded in a Mailman handler! :-) From stephen at xemacs.org Mon Apr 15 10:59:03 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 17:59:03 +0900 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415062636.GA9945@sys4.de> Message-ID: <87r4ic41bc.fsf@uwakimon.sk.tsukuba.ac.jp> Sreyanth writes: > I have implemented a binary Bayesian classifier which classifies an email > either spam or not spam. Is it better than (or at least different from) SpamBayes (available on PyPI), which some third parties already use in Mailman installations? Let's not reinvent wheels, here! From pabs3 at bonedaddy.net Mon Apr 15 11:07:59 2013 From: pabs3 at bonedaddy.net (Paul Wise) Date: Mon, 15 Apr 2013 17:07:59 +0800 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Mon, Apr 15, 2013 at 4:55 PM, Stephen J. Turnbull wrote: > That's a great idea (and a great example of why I don't think > spam-filtering technology should be embedded in a Mailman handler! :-) I think you might not be thinking expansively enough, it allows more flexibility to do spam filtering in multiple places. Different lists will have different definitions of what is an unwanted message. On the Debian lists we don't want anything about shares, liquidity and financial instruments, but some economists will find messages about source code and copyright law unwanted on their lists. A list for pharmacists will probably want to accept messages containing usually-spammy keywords. A generic mailing-list-hoster hosting all 3 of those will want to allow most mail and add per-list filters. -- bye, pabs http://bonedaddy.net/pabs3/ From iampratiksarkar at gmail.com Mon Apr 15 11:21:33 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Mon, 15 Apr 2013 02:21:33 -0700 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Mon, Apr 15, 2013 at 2:19 AM, Pratik Sarkar wrote: > Yes,thats why the language model concept will come in handy. > The classifier will be trained on the existing archives. > For eg,in debian list,the classifier will be trained on a training set > containing copyright law,source code,etc.Whereas in an economists list,the > classifier will be trained on the training set containing economics-related > emails and conversations. > Whenever an economics-related message will try to enter the debian > list,the classifier would check the language model of the > (economics)message with the classifier and would classify it correctly as > the sentence construction/language model of a technical message(debian list > message) and an economics message varies a lot. > Please feel free to as me,if I have not been able to explain my views. > Regards > Pratik > > > On Mon, Apr 15, 2013 at 2:07 AM, Paul Wise wrote: > >> On Mon, Apr 15, 2013 at 4:55 PM, Stephen J. Turnbull wrote: >> >> > That's a great idea (and a great example of why I don't think >> > spam-filtering technology should be embedded in a Mailman handler! :-) >> >> I think you might not be thinking expansively enough, it allows more >> flexibility to do spam filtering in multiple places. Different lists >> will have different definitions of what is an unwanted message. On the >> Debian lists we don't want anything about shares, liquidity and >> financial instruments, but some economists will find messages about >> source code and copyright law unwanted on their lists. A list for >> pharmacists will probably want to accept messages containing >> usually-spammy keywords. A generic mailing-list-hoster hosting all 3 >> of those will want to allow most mail and add per-list filters. >> >> -- >> bye, >> pabs >> >> http://bonedaddy.net/pabs3/ >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/iampratiksarkar%40gmail.com >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > > From p at sys4.de Mon Apr 15 11:28:26 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Mon, 15 Apr 2013 11:28:26 +0200 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130415092825.GC15104@sys4.de> * Stephen J. Turnbull : > Pratik Sarkar writes: > > > Is the anti-spam/abuse filter still being seriously considered as a > > gsoc project this year? > > I would say so, yes. Personally, I am fundamentally opposed to it; I > think it's wrong in principle (filtering of this kind should be done > by the incoming MTA) and inappropriate for the 3.0 release. *But* > there is clearly user demand for it, and among the actually signed-up > mentors[1] there are at least two who have shown some support for it. > > OTOH, you should be aware that while nobody has a veto except Barry > (Terri has one ex oficio but I doubt she'd exercise it if Barry was in > favor), it's likely that projects that all the mentors favor will be > ranked higher, and looking at the quality of posts from students so > far there is going to be competition for Mailman slots. > > If you have a solid proposal for anti-spam already worked out (the > idea the guy proposed with a Bayesian filter based on word triads > comes close to what I'd call solid, see also Terri's post where she > proposed a schedule for the kind of additional detail needed), then > that's probably your best bet. > > But if you're looking for a project and you think that anti-spam is > cool but that's all the thinking you've done so far, I'd say it's very > risky proposal. There are a lot of things we need in UI (both > subscriber-oriented and admin-oriented) that are interesting and > higher-priority. Perhaps the integration could create an interface itself that makes it easy to add other filters in the future. I am thinking Postfix 'content filter', which uses SMTP/LMTP to send messages to an external filter and they send it back then using SMTP. The first Mailman content filter could be the one you proposed. Others could add their filter later. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From fte08eas at student.lu.se Mon Apr 15 12:04:29 2013 From: fte08eas at student.lu.se (Elias Assarsson) Date: Mon, 15 Apr 2013 12:04:29 +0200 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: <20130411161155.6a90c2e5@anarchist> References: <51652B5E.60200@student.lu.se> <20130410141418.1e614d12@anarchist> <5166AAF7.2060805@student.lu.se> <20130411161155.6a90c2e5@anarchist> Message-ID: <516BD0AD.9010508@student.lu.se> I appreciate the help in trying to understand the configuration systems in MM2 and MM3. 2013-04-11 22:11, Barry Warsaw skrev: > On Apr 11, 2013, at 02:22 PM, Elias Assarsson wrote: > >>> * MM2's configuration file is a Python file which really must be imported >>> in order to get a valid set of values. MM3's configuration file is a stack >>> of .ini-style files. >> I am trying to find and understand the configuration files so that I know >> what that that needs to be migrated and to what form. Is the MM2 >> configuration you refer to mainly Mailman/mm_cfg.py and MM3 configuration >> files src/mailman/config/* and /src/mailman//*? > Yes, in a deployed Mailman 2, mm_cfg.py will contain the system configuration > settings. These override the settings from Defaults.py so a good way to > explore is to use `bin/withlist` and `import mm_cfg` at the Python prompt. I have had a look through bin/withlist, mm_cfg.py and Defaults.py to get a feel for the format of the configuration. Why is bin/withlist relevant for configuration migration? In that it is way to learn about configuration? > > In MM3, we use lazr.config, which is essentially a configparser type package > that allows for stacks of configurations, with pushing and popping values on > this stack. > > http://pythonhosted.org/lazr.config/ > > The src/mailman/config/schema.cfg file is at the bottom of the stack and > defines the schema, obviously :). From there, src/mailman/config/mailman.cfg > essentially instantiates this schema, providing all the defaults. In the > testing environment, src/mailman/testing/testing.cfg gets pushed on top of > that (and many tests push and pop micro-overrides). In a deployed system, the > site's mailman.cfg is on the top of the stack and so can override anything. > There are various places this mailman.cfg file is searched; see > src/mailman/core/initialize.py for all the gory details. I have had a quick look at the lazr.config documentation and also checked out mailman.cfg, testing.cfg and initialize.cfg in trying to understand the system. I guess configuration migration scripts should have tests, e.g. to test if some particular MM2 configuration is migrated to the expected MM3 form. If this is so, what would be an appropriate way to collect a MM2 configuration and the expected MM3 form? - Elias From stephen at xemacs.org Mon Apr 15 12:08:24 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 19:08:24 +0900 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> Sreyanth writes: > Just hide the boilerplate in the email, giving a link to > click. When clicked, use js to unhide the boilerplate. This would > not anyhow require separate storage. Suggest me something if this > is bad! The main point of sharing links is not storage compression; it's that the link identifies boilerplate you've seen before by changing appearance. You won't be able to tell if JS is used. From sreyanth at gmail.com Mon Apr 15 12:11:51 2013 From: sreyanth at gmail.com (Sreyanth) Date: Mon, 15 Apr 2013 15:41:51 +0530 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Mon, Apr 15, 2013 at 3:38 PM, Stephen J. Turnbull wrote: > Sreyanth writes: > > > Just hide the boilerplate in the email, giving a link to > > click. When clicked, use js to unhide the boilerplate. This would > > not anyhow require separate storage. Suggest me something if this > > is bad! > > The main point of sharing links is not storage compression; it's that > the link identifies boilerplate you've seen before by changing > appearance. You won't be able to tell if JS is used. > ?I concur.? I I understood it before too. I proposed this JS approach, as I wondered why people would anyhow look at the boilerplate! (That's the main point of the project right!)? -- *Yours Sincerely* * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From stephen at xemacs.org Mon Apr 15 12:22:42 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 19:22:42 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130415092825.GC15104@sys4.de> References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> Message-ID: <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> Patrick Ben Koetter writes: > Perhaps the integration could create an interface itself that makes > it easy to add other filters in the future. I am thinking Postfix > 'content filter', which uses SMTP/LMTP to send messages to an > external filter and they send it back then using SMTP. > > The first Mailman content filter could be the one you > proposed. Others could add their filter later. I don't see any Mailman-specific issues in content-based spam filtering, though, and very little Mailman-specific coding. I also worry about reinventing the wheel, with corners. Ie, why do we think a GSoC student is going to be able to do something that's worth putting up again the very effective filters with huge user bases like SpamAssassin and SpamBayes that are already out there? Wouldn't a half-baked summer project just sit there and bitrot? OTOH, a generic interface to the Mailman REST API, which could be used by Sendmail milters or whatever else is out there and somewhat standard (is it Postfix or Exim that can handle milters? I forget), with example implementation of a milter that checks whether the poster is subscribed by asking Mailman, would be useful as an extension to any MTA used with Mailman. Or Terri's through-the-web interface to the Mailman Handler pipeline(s), with an example Handler installable from PyPI which wraps SpamAssassin or SpamBayes. From iampratiksarkar at gmail.com Mon Apr 15 12:23:36 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Mon, 15 Apr 2013 03:23:36 -0700 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <1366021134.13793.316.camel@chianamo> References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> <1366021134.13793.316.camel@chianamo> Message-ID: What is this signature.asc file.? Regards Pratik On Mon, Apr 15, 2013 at 3:18 AM, Paul Wise wrote: > On Mon, 2013-04-15 at 02:19 -0700, Pratik Sarkar wrote: > > > Yes,thats why the language model concept will come in handy. > > Please reply to the list, your message was not seen by anyone other than > me. > > -- > bye, > pabs > > http://bonedaddy.net/pabs3/ > From stephen at xemacs.org Mon Apr 15 12:29:18 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 19:29:18 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87k3o43x4x.fsf@uwakimon.sk.tsukuba.ac.jp> Paul Wise writes: > A generic mailing-list-hoster hosting all 3 of those will want to > allow most mail and add per-list filters. Of course. SpamAssassin at least can already do that (it allows mailbox-specific rules). A properly set up Bayesian filter will notice that certain criteria are associated with list addressees. (Not as effective as explicit training on each individual corpus, but I was never very good about that, and I doubt few list admins will bother.) Can we really do better than those? If not, we should concentrate on (1) exporting policy based on Mailman databases to MTAs via milters, and/or (2) importing existing functionality from existing spam filters such as SpamAssassin to Mailman. The big problem with (2) is that it has already been done; it's only a few lines in a Handler. Steve From pabs3 at bonedaddy.net Mon Apr 15 12:30:02 2013 From: pabs3 at bonedaddy.net (Paul Wise) Date: Mon, 15 Apr 2013 18:30:02 +0800 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> <1366021134.13793.316.camel@chianamo> Message-ID: On Mon, Apr 15, 2013 at 6:23 PM, Pratik Sarkar wrote: > What is this signature.asc file.? See RFC 2015. -- bye, pabs http://bonedaddy.net/pabs3/ From stephen at xemacs.org Mon Apr 15 12:43:45 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 19:43:45 +0900 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87ip3o3wgu.fsf@uwakimon.sk.tsukuba.ac.jp> Sreyanth writes: > I I understood it before too. I proposed this JS approach, as I wondered > why people would anyhow look at the boilerplate! (That's the main point of > the project right!)? Depends on your definition of "boilerplate". I consider quoted mailing lists' footers and those stupid corporate legal notices to be boilerplate, and they're generally far more annoying than "sent from my iPhone" because they often separate the text from attachments and the like by more than a few lines. Worse, quoted footers are often actively harmful, because they contain "unsubscribe" links for somebody else (but all the user can see is "click here to unsubscribe"). From barry at list.org Mon Apr 15 12:52:33 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 15 Apr 2013 06:52:33 -0400 Subject: [Mailman-Developers] Hi from a student interested in a GSoC project In-Reply-To: <516BD0AD.9010508@student.lu.se> References: <51652B5E.60200@student.lu.se> <20130410141418.1e614d12@anarchist> <5166AAF7.2060805@student.lu.se> <20130411161155.6a90c2e5@anarchist> <516BD0AD.9010508@student.lu.se> Message-ID: <20130415065233.5bc6cf8c@limelight> On Apr 15, 2013, at 12:04 PM, Elias Assarsson wrote: >I have had a look through bin/withlist, mm_cfg.py and Defaults.py to get a >feel for the format of the configuration. Why is bin/withlist relevant for >configuration migration? In that it is way to learn about configuration? Yes. >I guess configuration migration scripts should have tests, e.g. to test if >some particular MM2 configuration is migrated to the expected MM3 form. If >this is so, what would be an appropriate way to collect a MM2 configuration >and the expected MM3 form? Yes, *all* new code should have tests. :) You probably should build a MM2 system, work through the site defaults and create a few mailing lists. Also build a MM3 system and see what you'd need to do to convert your MM2 lists to MM3 lists. -Barry From barry at list.org Mon Apr 15 13:02:40 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 15 Apr 2013 07:02:40 -0400 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130415070240.589f1be9@limelight> On Apr 15, 2013, at 01:05 PM, Sreyanth wrote: >Just hide the boilerplate in the email, giving a link to click. When clicked, >use js to unhide the boilerplate. This would not anyhow require separate >storage. Suggest me something if this is bad! ?? A problem you would have to work out with this approach is how to properly integrate this when a site/list could have multiple enabled archivers, some or all of which are remote to the Mailman system. Remember that in MM3 we won't have an integrated archiver like Pipermail that we can just assume will be there. Even the web ui isn't as tightly integrated any more. This doesn't mean you can't still do this. See for example the additions our stable url proposed add-ons to RFC 5064 as an example of how you would have to think about this. -Barry From rkw at DATAPLEX.NET Mon Apr 15 13:11:17 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Mon, 15 Apr 2013 06:11:17 -0500 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <092E7F0C-7D34-441B-ABE9-94BCA2237743@DATAPLEX.NET> FWIW, I tend to support Stephen's view with respect to usefulness and interface strategy. Wacky On Apr 15, 2013, at 5:22 AM, Stephen J. Turnbull wrote: > Patrick Ben Koetter writes: > >> Perhaps the integration could create an interface itself that makes >> it easy to add other filters in the future. I am thinking Postfix >> 'content filter', which uses SMTP/LMTP to send messages to an >> external filter and they send it back then using SMTP. >> >> The first Mailman content filter could be the one you >> proposed. Others could add their filter later. > > I don't see any Mailman-specific issues in content-based spam > filtering, though, and very little Mailman-specific coding. I also > worry about reinventing the wheel, with corners. Ie, why do we think > a GSoC student is going to be able to do something that's worth > putting up again the very effective filters with huge user bases like > SpamAssassin and SpamBayes that are already out there? Wouldn't a > half-baked summer project just sit there and bitrot? > > OTOH, a generic interface to the Mailman REST API, which could be used > by Sendmail milters or whatever else is out there and somewhat > standard (is it Postfix or Exim that can handle milters? I forget), > with example implementation of a milter that checks whether the poster > is subscribed by asking Mailman, would be useful as an extension to > any MTA used with Mailman. > > Or Terri's through-the-web interface to the Mailman Handler > pipeline(s), with an example Handler installable from PyPI which wraps > SpamAssassin or SpamBayes. > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From barry at list.org Mon Apr 15 13:26:09 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 15 Apr 2013 07:26:09 -0400 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: <20130415072609.5bbc3d24@limelight> On Apr 14, 2013, at 12:49 AM, Sneha Priscilla wrote: >Why don't you try re-branching the repository again and start over? Plus >buildout instruction needs sudo access as far as I know, so make sure >you're doing that. You shouldn't need root to do a buildout and run the tests. You can probably even build a test deployment without root, but you may need it if you're going to bind to privileged ports, although neither the REST http server, nor LMTP server listen to privileged ports by default. Obviously, you'd need root if you were going to install a deployed system to certain parts of the file system. -Barry From barry at list.org Mon Apr 15 13:30:16 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 15 Apr 2013 07:30:16 -0400 Subject: [Mailman-Developers] error installing the mailman web UI In-Reply-To: References: Message-ID: <20130415073016.7e63c9f3@limelight> On Apr 14, 2013, at 09:34 AM, Avik Pal wrote: >actually I think the following is behind all these... > >*Download error on http://ish.io/projects/show/restish: [Errno -2] Name or >service not known -- Some packages may not be found!* Buildout should be able to grab restish from PyPI: https://pypi.python.org/pypi/restish/0.12.1 I've seen the errors you're describing (ish.io is gone I think), but it then successfully downloads it from the above URL. -Barry From iampratiksarkar at gmail.com Mon Apr 15 13:32:22 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Mon, 15 Apr 2013 04:32:22 -0700 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <092E7F0C-7D34-441B-ABE9-94BCA2237743@DATAPLEX.NET> References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> <092E7F0C-7D34-441B-ABE9-94BCA2237743@DATAPLEX.NET> Message-ID: Can someone please give me a link of the existing mailman spam filter techniques.? Pratik On Mon, Apr 15, 2013 at 4:11 AM, Richard Wackerbarth wrote: > FWIW, > > I tend to support Stephen's view with respect to usefulness and interface > strategy. > > Wacky > > On Apr 15, 2013, at 5:22 AM, Stephen J. Turnbull > wrote: > > > Patrick Ben Koetter writes: > > > >> Perhaps the integration could create an interface itself that makes > >> it easy to add other filters in the future. I am thinking Postfix > >> 'content filter', which uses SMTP/LMTP to send messages to an > >> external filter and they send it back then using SMTP. > >> > >> The first Mailman content filter could be the one you > >> proposed. Others could add their filter later. > > > > I don't see any Mailman-specific issues in content-based spam > > filtering, though, and very little Mailman-specific coding. I also > > worry about reinventing the wheel, with corners. Ie, why do we think > > a GSoC student is going to be able to do something that's worth > > putting up again the very effective filters with huge user bases like > > SpamAssassin and SpamBayes that are already out there? Wouldn't a > > half-baked summer project just sit there and bitrot? > > > > OTOH, a generic interface to the Mailman REST API, which could be used > > by Sendmail milters or whatever else is out there and somewhat > > standard (is it Postfix or Exim that can handle milters? I forget), > > with example implementation of a milter that checks whether the poster > > is subscribed by asking Mailman, would be useful as an extension to > > any MTA used with Mailman. > > > > Or Terri's through-the-web interface to the Mailman Handler > > pipeline(s), with an example Handler installable from PyPI which wraps > > SpamAssassin or SpamBayes. > > > > _______________________________________________ > > Mailman-Developers mailing list > > Mailman-Developers at python.org > > http://mail.python.org/mailman/listinfo/mailman-developers > > Mailman FAQ: http://wiki.list.org/x/AgA3 > > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > > > Security Policy: http://wiki.list.org/x/QIA9 > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/iampratiksarkar%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From 24.sneha at gmail.com Mon Apr 15 18:08:38 2013 From: 24.sneha at gmail.com (Sneha Priscilla) Date: Mon, 15 Apr 2013 21:38:38 +0530 Subject: [Mailman-Developers] Interested in - Better User Settings Management Message-ID: Hey everyone! I had introduced myself earlier on this list but I guess I can do it again. :) I'm Sneha Priscilla, a final year undergraduate CS student from Bangalore, India. I was a previous Summer of Code student for Systers last year and I have in the process, worked with a lot of Mailman too. I am especially interested in Postorius and would like to implement one of the following ideas this summer: 1.Better User Settings Management 2.Design interface "themes" for specific types of list 3.Enhance list style capabilities The one I am most interested in is 'Better User Settings Management'. I'd like to know if it's a feasible idea for this summer and how important is it on the list of project ideas for Mailman this year? Also, since this is something which could also serve as an important Systers project, is it viable to be joint mentored by Systers and Mailman? I have worked with both Mailman versions 2.1 and 3, so I feel I have a nice idea about working on user settings and I hopefully, will be able to do justice to this project if given a chance. You can see one of my bug fixes related to Postorius here :https://code.launchpad.net/~stylistica/postorius/inline_helptext IRC and Launchpad id: stylistica Thanks in advance! Regards, Sneha Priscilla From macobo at ut.ee Mon Apr 15 19:19:29 2013 From: macobo at ut.ee (Karl-Aksel Puulmann) Date: Mon, 15 Apr 2013 20:19:29 +0300 Subject: [Mailman-Developers] GSoC 2013 (GNU Mailman) - logs&rss Message-ID: Back from Germany with new ideas and 3 bronze medals won by our contestants. :) Sadly I haven't heard anything about those merge requests I made a week ago. Anyways, onto ideas. I'll be applying to either one of the following ideas, please comment on the open questions. = RSS access to access to archives = Should this be a part of Mailman core as a pipeline (similar to NNTP) or is something else envisioned? = Log monitor = Are there any previous projects that you had in mind that implement this as suggested? Should this only be used for emailing reports of oddities (monthly/weekly reports) or perhaps could be integrated into a dashboard displaying the recent statistics? Which of those two ideas is currently higher on the priority list (including perhaps Administrative email message log)? All the best Karl (macobo on IRC) On Sun, Apr 7, 2013 at 3:24 PM, Karl-Aksel Puulmann wrote: > On Thu, Apr 4, 2013 at 8:56 PM, Barry Warsaw wrote: > >> On Apr 04, 2013, at 07:59 PM, Karl-Aksel Puulmann wrote: >> >> >I'm Karl, 2nd year bachelors CS student from University of Tartu in >> >Estonia. >> >> Welcome Karl! FWIW, I'm utc-4 but feel free to ping me on #mailman >> during any >> overlapping time. >> >> -Barry >> >> > Will do, as I have some questions about a few issues on the tracker. But > as I will be away in Germany next week, that'll have to be saved for next > weekend or later. > > I tried to fix two of the bugs on the tracker, and am waiting for the > inevitable feedback on code style etc. Please review. :) > > About GSoC project ideas - right now working on mailman core seems to be > more fun. Which brings me to the following question - which of the projects > and bug fixes would benefit Mailman the most, making MM3 more stable and > increasing adoption by mailing lists? > > Thanks to all the helpful people on IRC (florianf, terri, mspiro and > others) who have helped me with various issues I ran into. > > All the best > Karl > > [1]: > https://code.launchpad.net/~macobo/mailman/macobo-rcpt-stage-reject/+merge/157516 > [2]: > https://code.launchpad.net/~macobo/mailman/macobo-conf_sort_option/+merge/157469 > From mark at msapiro.net Mon Apr 15 21:03:47 2013 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 15 Apr 2013 12:03:47 -0700 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> <092E7F0C-7D34-441B-ABE9-94BCA2237743@DATAPLEX.NET> Message-ID: <516C4F13.8070606@msapiro.net> On 4/15/2013 4:32 AM, Pratik Sarkar wrote: > Can someone please give me a link of the existing mailman spam filter > techniques.? SpamBayes integration Spamassassin integration -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From flo.fuchs at gmail.com Mon Apr 15 21:31:56 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Mon, 15 Apr 2013 21:31:56 +0200 Subject: [Mailman-Developers] Interested in - Better User Settings Management In-Reply-To: References: Message-ID: 2013/4/15 Sneha Priscilla <24.sneha at gmail.com>: > Hey everyone! Hey Sneha! > I had introduced myself earlier on this list but I guess I can do it again. > :) Sorry about that -- and thanks for writing again! > I'm Sneha Priscilla, a final year undergraduate CS student from > Bangalore, India. I was a previous Summer of Code student for Systers > last year and I have in the process, worked with a lot of Mailman > too. > > > I am especially interested in Postorius and would like to implement > one of the following ideas this summer: > 1.Better User Settings Management > 2.Design interface "themes" for specific types of list > 3.Enhance list style capabilities > The one I am most interested in is 'Better User Settings Management'. > I'd like to know if it's a feasible idea for this summer and how > important is it on the list of project ideas for Mailman this year? It is certainly a priority in Postorius, because it's one of the last missing features (and we're about to add a very simple version of it in order to be able to release a new beta soon). The core's architecture allows very fine-grained changes to a user's settings, like (but not limited to) the ones described on our ideas page. Because of that complexity, one very important aspect of this project is to come up with interface ideas that will not leave people confused (or how Terri described it: the "user interface nightmare"). So I think it could make for a very interesting summer project, both on the architectural as well as on the coding side. Overall I'd say it's probably one of the more important project ideas, although that definitely depends on who you ask. ;-) Also, it's a bit difficult to categorize the project ideas in "important" and "less important" ones. Even a project idea that hasn't been in the center of our attention can suddenly become very important if we receive a great application for it. > Also, since this is something which could also serve as an important > Systers project, is it viable to be joint mentored by Systers and > Mailman? Not officially. We'd have to decide if Systers or the PSF would be the "official" mentoring org. That being said: What's the harm in more than one person giving advice to a student? As long as there is one person who is formally in charge and is committed to keep the ball rolling. Cheers Florian From flo.fuchs at gmail.com Mon Apr 15 22:08:05 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Mon, 15 Apr 2013 22:08:05 +0200 Subject: [Mailman-Developers] A feature I'd like to emphasize for GSoC ... In-Reply-To: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: 2013/4/12 Stephen J. Turnbull : > ... is *convenient access to logs via the admin interface*. This could be something for a HTML(5)/JS/CSS savvy person (to make it *really* convenient). I'm thinking scrollable log views, that interactively append/prepend entries to the view, depending on your position in the log file. Or commenting on log snippets and saving them for further review. Florian > > Steve > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/f%40state-of-mind.de > > Security Policy: http://wiki.list.org/x/QIA9 From flo.fuchs at gmail.com Mon Apr 15 23:31:22 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Mon, 15 Apr 2013 23:31:22 +0200 Subject: [Mailman-Developers] Interested in - Better User Settings Management In-Reply-To: References: Message-ID: 2013/4/15 Robin Jeffries : > As someone from Systers, we would probably accept her under the Systers > umbrella (though if we get a lot of great applicants and are feeling > squeezed, we might come to you to ask you to use one of your slots for her, > but Terri is the one to broker that), but we'd want to make sure there was > someone on the mailman side who could help her get unstuck at various > points. We did that last year and it worked well. (as you know, Florian) Oh, I hope I didn't sound skeptical... This particular project (being a Postorius one) affects both Terri and me, and both of us have been mentoring for Systers as well as Mailman. So either way this should be especially easy. But as you said, Terri is probably in the best position to make a suggestion regarding which org's slot to use. Florian > > Robin > > > On Mon, Apr 15, 2013 at 12:31 PM, Florian Fuchs wrote: >> >> 2013/4/15 Sneha Priscilla <24.sneha at gmail.com>: >> > Hey everyone! >> >> >> > Also, since this is something which could also serve as an important >> > Systers project, is it viable to be joint mentored by Systers and >> > Mailman? >> >> Not officially. We'd have to decide if Systers or the PSF would be the >> "official" mentoring org. >> That being said: What's the harm in more than one person giving advice >> to a student? As long as there is one person who is formally in charge >> and is committed to keep the ball rolling. >> >> >> Cheers >> Florian >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/robin%40jeffries.org >> >> Security Policy: http://wiki.list.org/x/QIA9 > > From robin at jeffries.org Mon Apr 15 22:27:42 2013 From: robin at jeffries.org (Robin Jeffries) Date: Mon, 15 Apr 2013 13:27:42 -0700 Subject: [Mailman-Developers] Interested in - Better User Settings Management In-Reply-To: References: Message-ID: As someone from Systers, we would probably accept her under the Systers umbrella (though if we get a lot of great applicants and are feeling squeezed, we might come to you to ask you to use one of your slots for her, but Terri is the one to broker that), but we'd want to make sure there was someone on the mailman side who could help her get unstuck at various points. We did that last year and it worked well. (as you know, Florian) Robin On Mon, Apr 15, 2013 at 12:31 PM, Florian Fuchs wrote: > 2013/4/15 Sneha Priscilla <24.sneha at gmail.com>: > > Hey everyone! > > > > Also, since this is something which could also serve as an important > > Systers project, is it viable to be joint mentored by Systers and > > Mailman? > > Not officially. We'd have to decide if Systers or the PSF would be the > "official" mentoring org. > That being said: What's the harm in more than one person giving advice > to a student? As long as there is one person who is formally in charge > and is committed to keep the ball rolling. > > > Cheers > Florian > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/robin%40jeffries.org > > Security Policy: http://wiki.list.org/x/QIA9 > From adam-mailman at amyl.org.uk Tue Apr 16 00:13:08 2013 From: adam-mailman at amyl.org.uk (Adam McGreggor) Date: Mon, 15 Apr 2013 23:13:08 +0100 Subject: [Mailman-Developers] A feature I'd like to emphasize for GSoC ... In-Reply-To: References: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130415221308.GC3728@hendricks.amyl.org.uk> On Mon, Apr 15, 2013 at 10:08:05PM +0200, Florian Fuchs wrote: > 2013/4/12 Stephen J. Turnbull : > > ... is *convenient access to logs via the admin interface*. > > This could be something for a HTML(5)/JS/CSS savvy person (to make it > *really* convenient). I'm thinking scrollable log views, that > interactively append/prepend entries to the view, depending on your > position in the log file. Or commenting on log snippets and saving > them for further review. And with a nice regexp tool for hunting out the beasties in question? (or maybe lolcat stylee?) A -- "Cabbage-- the opinions of taxi drivers." (new definitions, from `I'm Sorry I Haven't a Clue') From barry at list.org Tue Apr 16 03:44:19 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 15 Apr 2013 21:44:19 -0400 Subject: [Mailman-Developers] GSoC 2013 (GNU Mailman) - logs&rss In-Reply-To: References: Message-ID: <20130415214419.55fe775f@limelight> On Apr 15, 2013, at 08:19 PM, Karl-Aksel Puulmann wrote: >Sadly I haven't heard anything about those merge requests I made a week ago. Yes, sorry, I've been swamped. -Barry From pabs3 at bonedaddy.net Tue Apr 16 04:53:11 2013 From: pabs3 at bonedaddy.net (Paul Wise) Date: Tue, 16 Apr 2013 10:53:11 +0800 Subject: [Mailman-Developers] A feature I'd like to emphasize for GSoC ... In-Reply-To: References: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Tue, Apr 16, 2013 at 4:08 AM, Florian Fuchs wrote: > This could be something for a HTML(5)/JS/CSS savvy person (to make it > *really* convenient). I'm thinking scrollable log views, that > interactively append/prepend entries to the view, depending on your > position in the log file. Or commenting on log snippets and saving > them for further review. As long these are based on progressive enhancement, sounds good. The web interface should continue to work for browsers that don't support CSS, JavaScript, AJAX, images, Flash, SVG or any of the other stuff that was added to HTML. https://en.wikipedia.org/wiki/Progressive_enhancement -- bye, pabs From markoupetr at gmail.com Tue Apr 16 09:51:43 2013 From: markoupetr at gmail.com (Peter Markou) Date: Tue, 16 Apr 2013 10:51:43 +0300 Subject: [Mailman-Developers] GSOC 2013 : Web Posting Interface In-Reply-To: References: Message-ID: Hi again community, As I was checking https://bugs.launchpad.net/mailman/ to give it a shot and fix any bug, I noticed this bug https://bugs.launchpad.net/mailman/+bug/1104505 tagged with "wishlist". The exact bug description is : "There should be an interface for posting to a list from the Web UI". After about 2 months Terri made this wishlist one of the proposed projects for GSoC. This got me thinking and drew the conclusion that Mailman is considering a lot the users feedback/suggestions. So since Mailman is looking for students who are able to contribute useful features, I'm going to have a look at all bugs/wishlists and collect the ones that are related to "Web Posting Interface" to a certain extent (a good starting point to get a clue about what features the users find useful). After all, in my opinion, these suggestions should guide us in refining the project to function as users expect to (features suggested by developers should also have an impact). Looking forward to hear any suggestion in order to include them in my application proposal as described here: http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013 in the "Web Posting Interface" project description. On Tue, Apr 9, 2013 at 9:49 AM, Peter Markou wrote: > Hello to all the members of the community. > My name is Peter Markou and I'm currently > running the last year of my university course. > I'm deeply interested in implementing the > "Web Posting Interface". > Since I've maintained and integrated phpBB > forum in a european project that I participated > about 8 months ago(MUTW - Multinational Undergraduate Team Work > http://mutw.praxisnetwork.eu/ ) I think I will be able > to offer some additional features to make it a replacement > for phpBB. Thanks in advance for your time and looking > forward to discuss any further details. > From stephen at xemacs.org Tue Apr 16 10:36:28 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 16 Apr 2013 17:36:28 +0900 Subject: [Mailman-Developers] Interested in - Better User Settings Management In-Reply-To: References: Message-ID: <87ehea50tv.fsf@uwakimon.sk.tsukuba.ac.jp> Sneha Priscilla writes: > The one I am most interested in is 'Better User Settings Management'. > I'd like to know if it's a feasible idea for this summer and how > important is it on the list of project ideas for Mailman this year? As far as I'm concerned, definitely yes (since "better" involves "as much work as you can do"! :-) and "pretty darn important", since that's one of the main things we need to get uptake on Mailman 3. > Also, since this is something which could also serve as an important > Systers project, is it viable to be joint mentored by Systers and > Mailman? "Viable," you'd have to ask Carol @ GSoC about that. Since Money is involved (even though a small amount for the orgs), I suspect the answer is "formally, you must choose one" (and get advice as you please during the summer). My *advice* is that I think that you're likely to find that Mailman and Systers will have different ideas about what is important. I think you are best off doing your project with one point of view rather than trying to balance them. From stephen at xemacs.org Tue Apr 16 10:38:16 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 16 Apr 2013 17:38:16 +0900 Subject: [Mailman-Developers] A feature I'd like to emphasize for GSoC ... In-Reply-To: References: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87d2tu50qv.fsf@uwakimon.sk.tsukuba.ac.jp> Florian Fuchs writes: > 2013/4/12 Stephen J. Turnbull : > > ... is *convenient access to logs via the admin interface*. > > This could be something for a HTML(5)/JS/CSS savvy person (to make it > *really* convenient). I'm thinking scrollable log views, that > interactively append/prepend entries to the view, depending on your > position in the log file. Or commenting on log snippets and saving > them for further review. BTW, I've updated the wiki entry for this project suggestion. From stephen at xemacs.org Tue Apr 16 11:02:57 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 16 Apr 2013 18:02:57 +0900 Subject: [Mailman-Developers] A feature I'd like to emphasize for GSoC ... In-Reply-To: References: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87bo9e4zlq.fsf@uwakimon.sk.tsukuba.ac.jp> Paul Wise writes: > On Tue, Apr 16, 2013 at 4:08 AM, Florian Fuchs wrote: > > > This could be something for a HTML(5)/JS/CSS savvy person (to make it > > *really* convenient). I'm thinking scrollable log views, that > > interactively append/prepend entries to the view, depending on your > > position in the log file. Or commenting on log snippets and saving > > them for further review. > > As long these are based on progressive enhancement, sounds good. I think "no thanks to interactive convenience", unless the candidate is *really* good at Zen Garden stuff (ie, won't need to spend any time thinking about progressive enhancement, let alone learning any CSS or JS). Convenience can only be realized if logs can be read and effective queries are easy for users to generate, and the replies are easy to understand, too. Let's leave Web2.0 stuff for next summer, please. Remember, one of the top reasons for doing this is so that the admin can forward the log(s) to mailman-users for analysis. Besides interactive access, I'd also like to be able to script (curl, wget) the queries, so as far as I'm concerned the only web design needed for the reply is text = slurp_log(log, filter) html = """\ Log: {0}

Log: {0}

{1}
""".format(str(filter), test) The *hard* and *interesting* part to me is the query API (ie, what the REST looks like, and that it be flexible enough to support enhanced queries in the future) and UI (what you'll use from Postorius). Even getting access to the files is going to be non-trivial if we go after MTA logs too. We're going to need to convince admins that Mailman should be allowed to read those logs, so we're going to need to think about preventing users from writing queries that allow them to find out who's getting mail from pr0N sites and Kim Jong Un. I'd also like the code to do the MTA message ID threading to get the whole MTA history of a message. Steve From stephen at xemacs.org Tue Apr 16 11:03:56 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 16 Apr 2013 18:03:56 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> <092E7F0C-7D34-441B-ABE9-94BCA2237743@DATAPLEX.NET> Message-ID: <87a9oy4zk3.fsf@uwakimon.sk.tsukuba.ac.jp> Pratik Sarkar writes: > Can someone please give me a link of the existing mailman spam filter > techniques.? Besides the links Mark sent, Mailman 2 site admin pages under "Privacy" will explain how to use the already integrated regexp-based filtering. From stephen at xemacs.org Tue Apr 16 11:12:57 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 16 Apr 2013 18:12:57 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <878v4i4z52.fsf@uwakimon.sk.tsukuba.ac.jp> I'm copying to Mailman-Developers to ensure the other mentors are aware of your plans. Pratik Sarkar writes: > Actually my college semester exams are starting from next week thats why I > couldn't spend much time on this proposal.Since you gave a positive > feedback,I will start working on it. OK, but remember, AFAIK Paul (who proposed the approach) is not a mentor[1], and I'm only one of 6 or 7, so I can't promise you'll get a slot. > And is there any coding challenge I need to take for gsoc for mailman? We don't have an explicit challenge AFAIK, but either a fairly detailed design (optionally including pseudocode) or some a pointer to code that we can use to assess your skills would be very welcome. Footnotes: [1] Yet? What do you say, Paul? :-) Don't you want a T-shirt? :-) From sreyanth at gmail.com Tue Apr 16 11:17:19 2013 From: sreyanth at gmail.com (Sreyanth) Date: Tue, 16 Apr 2013 14:47:19 +0530 Subject: [Mailman-Developers] Fwd: Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip3o3wgu.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Mon, Apr 15, 2013 at 4:13 PM, Stephen J. Turnbull wrote: > Sreyanth writes: > > > I I understood it before too. I proposed this JS approach, as I wondered > > why people would anyhow look at the boilerplate! (That's the main point > of > > the project right!)? > > Depends on your definition of "boilerplate". I consider quoted > mailing lists' footers and those stupid corporate legal notices to be > boilerplate, and they're generally far more annoying than "sent from > my iPhone" because they often separate the text from attachments and > the like by more than a few lines. > > Worse, quoted footers are often actively harmful, because they contain > "unsubscribe" links for somebody else (but all the user can see is > "click here to unsubscribe"). > > I completely forgot this "unsubscribe" links! Thanks for reminding :-) Will add in the application! Also, I want to know if I am in a correct path to make " Boilerplate stripping AND better content filtering / handling error messages" a worthy GSoC project.?? All this discussion here in the mailing lists encourages me to learn more and do more and I am looking forward to submit my application for an early review! -- *Yours Sincerely* * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From p at sys4.de Tue Apr 16 11:19:50 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Tue, 16 Apr 2013 11:19:50 +0200 Subject: [Mailman-Developers] A feature I'd like to emphasize for GSoC ... In-Reply-To: <87bo9e4zlq.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> <87bo9e4zlq.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130416091949.GB12260@sys4.de> * Stephen J. Turnbull : > Paul Wise writes: > > On Tue, Apr 16, 2013 at 4:08 AM, Florian Fuchs wrote: > > > > > This could be something for a HTML(5)/JS/CSS savvy person (to make it > > > *really* convenient). I'm thinking scrollable log views, that > > > interactively append/prepend entries to the view, depending on your > > > position in the log file. Or commenting on log snippets and saving > > > them for further review. > > > > As long these are based on progressive enhancement, sounds good. > > I think "no thanks to interactive convenience", unless the candidate > is *really* good at Zen Garden stuff (ie, won't need to spend any time > thinking about progressive enhancement, let alone learning any CSS or > JS). Convenience can only be realized if logs can be read and > effective queries are easy for users to generate, and the replies are > easy to understand, too. Let's leave Web2.0 stuff for next summer, > please. Let me add some inconvenience: Log contains privacy related information. I'd expect a log view/research interface to limit output to what the person/role may look at and no more. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stephen at xemacs.org Tue Apr 16 11:49:06 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 16 Apr 2013 18:49:06 +0900 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip3o3wgu.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <8761zm4xgt.fsf@uwakimon.sk.tsukuba.ac.jp> CC-ing Mailman Developers to keep all mentors up to date on what you're thinking. Sreyanth writes: > On Mon, Apr 15, 2013 at 4:13 PM, Stephen J. Turnbull wrote: > > Worse, quoted footers are often actively harmful, because they contain > > "unsubscribe" links for somebody else (but all the user can see is > > "click here to unsubscribe"). > > > I completely forgot this "unsubscribe" links! Thanks for reminding :-) > Will add in the application! > Also, I want to know if I am in a correct path to make " Boilerplate > stripping AND better content filtering / handling error messages" a worthy > GSoC project. I don't want to discourage you, but I'd have to say you're not really making a lot of progress. In brief, an "excellent" project proposal is (IMO, other mentors may have other opinions) is composed of four parts: 1. A "theme" that can be stated in one or two lines. You have that in "Boilerplate stripping and content filtering", although I would be more attracted to "Message body filtering, such as boilerplate stripping or hiding". (It's more general in a way that probably doesn't require more complex code.) Improving your theme is worth doing (steal others' words if you're not a native speaker!), because it makes it easier for the mentors to grasp your project. But a theme is all you really have shown at this point. 2. A statement of requirements. What will the *users* see that you've accomplished? This is most important. (As explained below, we don't expect students to be able to do 3 or 4 below without mentoring -- but it's nice to have.) 3. A design, or statement of the changes to be made to the code to satisfy the requirements. (Ie, what other *developers* will see!) Much of the design can be done in cooperation with the mentors, but it's very attractive if you are aware of similar projects or features (expecially if they're already part of Mailman 2 or 3!) 4. A schedule. In more detail, here's something I drafted at the Mailman Sprint. I can say that Terri got a kick out of it, but remember, it's not official, it's just my opinion. (I really should put it on the wiki, but I'm not sure where or even how -- does Confluence do ReST?) Note that it is a *draft*, and especially the URLs may be wrong or inaccurate. ==================================== Student Proposals Augmenting Mailman ==================================== SPAM us at mailman-developers at python.org. How Not to SPAM: ---------------- Don't do this:: To: mailman-developers at python.org Subject: I want a Mailman mentor for GSoC Dear Mailman: I want to be a GSoC student. Please take me, please, please, please, please? Yours truly, A. N. Unlikely Candidate Like all GSoC projects, we're looking for students who care about Mailman, and want to improve it. The application above screams "I need money!" Yes, we know that, and we don't blame you one bit. But there are candidates who we can tell will do a good job for us. They need money too, you know. How to SPAM: ------------ Briefly, you need to 1. Find out about Mailman. 2. Decide which part of Mailman you want to contribute to. 3. Decide on a particular task, *i.e.*, a problem to solve. 4. Propose a plan to solve the problem: a. Tell us who you are. b. Tell us what you want to do for Mailman. c. Tell us what code you will write to get it done. d. Tell us when you'll get it done. This is what software professionals do over and over again at work. (Except for part 4a, but they have to do that once, just like you.) One of the purposes of the GSoC is to provide mentors who can help you accomplish those preliminary tasks, as well as produce excellent finished code. A Few Hints ----------- Step 1: Find out about Mailman_. Mailman started out as a mailing list manager: it keeps a database of lists, subscribers, and posts; moderates and distributes the posts to the subscribers; and maintains an archive of past posts that users can access by mail or web. It is written in the Python_ programming language. You are welcome to ask questions either on the Mailman `users' list`_ or on the Mailman `developers' list`_. We prefer to mentor students who have shown interest in working on Mailman before applying (but it's not at all required). .. _Mailman: http://www.list.org/ .. _Python: http://www.python.org/ .. _users' list: http://lists.python.org/mailman-users .. _developers' list: http://lists.python.org/mailman-developers **Do** feel free to ask the mentor team about early drafts of your proposal. You must write it yourself, but we can tell you if you're writing about the right things, and if you are providing the right details. Step 2. Decide where you'd like to contribute. The current version, Mailman 2, is an integrated application written in Python 2. However, it's become dated, and we are nearly ready to beta test a Mailman 3 system. `Mailman 3`_ is currently written in a dialect of Python 2 intended to make a transition to Python 3 easy. It is divided into three subprojects: - `Mailman 3 core`_: implements the list and subscriber databases, and the mail distribution functionality. **FIXMYURL** - `Postorius`_: a RESTful_ web interface for managing the list and subscriber databases. **FIXMYURL** - `HyperKitty`_: the archive manager and web interface to the archives. **FIXMYURL** While the developers all communicate with each other, especially in developing shared APIs, actual design and coding takes place in smaller groups organized around these subprojects. Each subproject has a list of `suggested tasks`_. **FIXMYURL** .. _Mailman 3: http://wiki.mailman.org/ .. _Mailman 3 core: http://wiki.mailman.org/ .. _Postorius: http://wiki.mailman.org/ .. _HyperKitty: http://wiki.mailman.org/ .. _RESTful: https://advanced-python.readthedocs.org/en/latest/rest/what-is-rest.html .. _suggested tasks: http://wiki.mailman.org/ Step 3. Pick a task. Why you pick a task is up to you. But feel free to ask the mentors or the lists. Maybe you just want the help the developers by doing something we'd do otherwise. Ask on the `developers' list`_ for our priorities and advice on what's actually do-able in the GSoC time frame. Maybe you want to implement a new feature. Check Launchpad_ for feature requests, and maybe review the `users' list`_ archives and the FAQ_. **FIXMYURL** Ask the developers if it can be done in time. .. _FAQ: http://wiki.list.org/ Step 4. SPAM_ us! See also `How Not to SPAM`_. .. _SPAM: #how-to-spam-in-detail .. _How Not to SPAM: #how-not-to-spam How to SPAM, in Detail ---------------------- "SPAM" is a "Student Proposal Augmenting Mailman." Perhaps you've contributed to an old-style open source project or done an internship where developers hack on code and simply commit it to the mainline repository when they're pretty sure there aren't any bugs in the new code. Sure, Linus did it that way -- but don't expect to get *your* code in *his* kernel that way! The trend is to more formal processes including explicit design and review stages. Pragmatically, GSoC requires some formality because Google requires us to report on interns' progress and review their code. But we also need some documentation to ensure that we make the best use of the few interns we are assigned, and we're used to it because many changes to our favorite language, Python, require `Python Enhancement Proposals`_. .. _Python Enhancement Proposals: http://www.python.org/dev/peps/ A SPAM is not as formal as a PEP. However, you should take care in preparing yours, because it's one of the few things we have to go on in deciding which students to recommend to Google. A SPAM should contain certain specific information, there's a lot of more or less optional information you can include to improve both your chances of acceptance and the project itself, and it probably won't hurt to organize it as described below. Do **not** worry if you can't write an accurate design or plan yet. Planning and design are arts, and probably the most important things a mentor can help a student to learn to do. That's really what GSoC is about. And, of course, some students won't know that much about Mailman to start with. So just do the best you can now to give us an idea of what you want to do, and your mentors will help you whip it into shape during the "get to know each other" preliminary phase. But if you *do* come up with a good design and schedule by yourself, that's very attractive to mentoring orgs (not just Mailman, believe me!) What follows are some hints on what to aim for in your proposal. Self-Introduction ................. Tell us about yourself. Full name, school affiliation, year in school, of course. Previous programming experience, including languages used and publicly documented projects you've worked on, if any. URLs for any web resources about yourself are useful, including anything from Facebook pages to GitHub projects. You will eventually need a LaunchPad_ login to publish and submit your contribution for review and integration. That and your email address, as well as a phone number, are essential contact information. It is often helpful to have a stable IRC nick that you can use on the Mailman channel (#mailman on Freenode). .. _LaunchPad: http://launchpad.net/ Coding Proposal: Theme ...................... Robert Townsend wrote in *Up the Organization*: "[Top management] knows that any proposal affecting their business can be explained in one minute." My colleagues were horrified to be compared to management of any ilk, but the principle holds. It's both possible, and important, to be able to express what your proposal is about relatively briefly. Provide both a one-line summary that can serve as a title for your project, as well as a slightly more detailed (around 5 lines) explanation of how you propose to accomplish your task. We have a number of `suggested tasks`_. You are welcome to select one that appeals to you. If you *do* choose one of these tasks, your ideas about the task, and an original description of the task, are important to convince us you have thought about the task. Don't just copy our description into your proposal. Rewrite it in your own words, adding your own ideas. Coding Proposal: Design ....................... This is a very high-level design. You can express it as prose or as pseudo-code. More important than an exact description of the code you propose to write is showing us *how little* you need to do to accomplish your goal. Tell us about existing modules on PyPI_ or GitHub_ that can do "90% of the work". (Don't worry, you'll be wrong. There will be *plenty* of work left for you to do.) Explain how these modules and your implementation will hook into the rest of Mailman. You're not done with the design, yet. You *will* need to refine it somewhat, in order to complete your schedule_. .. _PyPI: http://pypi.python.org/ .. _GitHub: http://github.com/ .. _schedule: #coding-proposal-schedule Coding Proposal: Schedule ......................... Perhaps unlike anything you've done before, GSoC has several deadlines that you *must* meet or *you will get fired* (*i.e.*, **not paid**). While many curmudgeons_ will argue that *getting fired* is a "valuable educational experience", it reflects as badly on your mentors as it does on you. We do **not** want that to happen. We have come close to firing a student at the midterm in the past; it *can* happen. Accurate scheduling is important to *you*. First, review the official GSoC deadlines. They are posted on the `GSoC site`_. Add them to your personal GSoC schedule. Second, for each official GSoC deadline, add a date one week before to confer with your mentor about your progress in general, and any specific milestones you need to complete to pass your reviews. This is a date where, if your mentors don't get in touch with you, you must get in touch with them. If you can't reach the mentors, get in touch with the Mailman org representative, Terri Oda. Add these dates to your personal GSoC schedule. Third, set milestones for yourself. A *milestone* is a subtask that can be objectively verified. For example, "coding is 50% complete" is *not* a milestone, because you don't know how much code you have left to write, you can only estimate it inaccurately. "Coding is complete, documentation is complete, unit and integration tests all pass, branch pushed to Launchpad, and a merge request filed" is a milestone, because all of the above are easily verified events. In fact, this is the most important milestone, which you should schedule for September 16 right now, because it is the criterion for passing your final review. In a perfect world, a student would set about one milestone per week, which is pretty convenient for both our evaluations and your self-evaluations. However, as a practical matter, setting milestones depends on a detailed design, and you probably won't produce a detailed design until after you've been approved for the program. In your application to us, it is very attractive if at least you are able to provide your midterm milestone. In order to produce milestones other than final completion, you need to have a somewhat detailed design. At this point, go back and revise your design, then pick some specific task and set it as a milestone. To get to your midterm milestone, think of it as the set of your earlier milestones. As you add each milestone, consider how much work that set of milestones represents. If that's not enough to constitute half the work, revise your design again, and set another milestone. **Do** feel free to ask the Mailman mentor team about your milestones. Few developers find it a natural way to think about development at first. We can and will help you with it as you develop your proposal. The fourth step in scheduling is the most important. Look at your private schedule, and see what is going to cause conflicts with your GSoC schedule. Maybe you have a summer course at school. Or your school's spring examination schedule conflicts with GSoC. Maybe your older brother is getting married on the other side of the world from where you are. We don't need to know *what* it is, but you need to tell us *when* it is, and *adjust your GSoC schedule* to accommodate your private plans. Of course you can't change the GSoC deadlines, and there's really no leeway there for us, either. What *you* can do is to adjust the timing of your milestones, and what *we* can do is adjust our expectations for the midterm review. Note that this is an adjustment in timing, not an adjustment in the size of the completed project expected. Software work is like that for many developers: sometimes they work a 5x8 week, and other times they work a 15+15+10+0+0 week. *We* understand that reality. But *you* need to think in terms of picking up the pace before and after if you take time off. On the other hand, if your time off is too great, you will **not** be able to catch up. Really -- we've been there. And be realistic. If you have extensive other plans for your summer, you probably should give up on GSoC. Google expects you to treat your internship as a full-time job during the coding period. (Of course you can always change your other plans if GSoC is important to you!) Be honest with yourself, and be honest with us. You will have a much more enjoyable *and profitable* summer that way. If your plan crowds too many milestones into the first half or second half of the summer, we won't believe you can do it, and you won't be asked to participate. If you pretend that you can keep a better balance than is realistic given other commitments, you will fail a review. We've seen it happen before. .. _curmudgeons: http://en.wikipedia.org/wiki/curmudgeon .. _GSoC site: http://google-melange.com/ From barry at list.org Tue Apr 16 15:39:09 2013 From: barry at list.org (Barry Warsaw) Date: Tue, 16 Apr 2013 09:39:09 -0400 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: <8761zm4xgt.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip3o3wgu.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zm4xgt.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130416093909.00dd8de4@anarchist> On Apr 16, 2013, at 06:49 PM, Stephen J. Turnbull wrote: >In more detail, here's something I drafted at the Mailman Sprint. >I can say that Terri got a kick out of it, but remember, it's not >official, it's just my opinion. (I really should put it on the wiki, >but I'm not sure where or even how -- does Confluence do ReST?) I love it too, but no, afaik Confluence doesn't do reST. I hope we're not far from converting to Moin though. -Barry From iampratiksarkar at gmail.com Tue Apr 16 18:12:08 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Tue, 16 Apr 2013 09:12:08 -0700 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <878v4i4z52.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> <878v4i4z52.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: I am working on the proposal.And how many slots are there for the filter project? On Tue, Apr 16, 2013 at 2:12 AM, Stephen J. Turnbull wrote: > I'm copying to Mailman-Developers to ensure the other mentors are > aware of your plans. > > Pratik Sarkar writes: > > > Actually my college semester exams are starting from next week thats > why I > > couldn't spend much time on this proposal.Since you gave a positive > > feedback,I will start working on it. > > OK, but remember, AFAIK Paul (who proposed the approach) is not a > mentor[1], and I'm only one of 6 or 7, so I can't promise you'll get a > slot. > > > And is there any coding challenge I need to take for gsoc for mailman? > > We don't have an explicit challenge AFAIK, but either a fairly > detailed design (optionally including pseudocode) or some a pointer to > code that we can use to assess your skills would be very welcome. > > > Footnotes: > [1] Yet? What do you say, Paul? :-) Don't you want a T-shirt? :-) > > From terri at zone12.com Tue Apr 16 18:25:59 2013 From: terri at zone12.com (Terri Oda) Date: Tue, 16 Apr 2013 10:25:59 -0600 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: <20130416093909.00dd8de4@anarchist> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip3o3wgu.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zm4xgt.fsf@uwakimon.sk.tsukuba.ac.jp> <20130416093909.00dd8de4@anarchist> Message-ID: <516D7B97.6070905@zone12.com> On 13-04-16 7:39 AM, Barry Warsaw wrote: > On Apr 16, 2013, at 06:49 PM, Stephen J. Turnbull wrote: > >> In more detail, here's something I drafted at the Mailman Sprint. >> I can say that Terri got a kick out of it, but remember, it's not >> official, it's just my opinion. (I really should put it on the wiki, >> but I'm not sure where or even how -- does Confluence do ReST?) > I love it too, but no, afaik Confluence doesn't do reST. I hope we're not far > from converting to Moin though. > I love it three. ;) If anyone's got time to fix the URLs to minimize confusion, I'm happy to have it be our official docs. Pandoc should be able to convert rest->wiki http://johnmacfarlane.net/pandoc/ (it won't quite be confluence wiki, but if I recall correctly they're close enough) Or, if it's less trouble, feel free to stick it on the Python wiki (which is moin) under SummerOfCode; I was hoping to link to it anyhow as advice to help students, so it is actually going to be relevant to python students in general. Terri From mark at msapiro.net Tue Apr 16 19:02:47 2013 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 16 Apr 2013 10:02:47 -0700 Subject: [Mailman-Developers] A feature I'd like to emphasize for GSoC ... In-Reply-To: <20130416091949.GB12260@sys4.de> References: <87hajc5rwx.fsf@uwakimon.sk.tsukuba.ac.jp> <87bo9e4zlq.fsf@uwakimon.sk.tsukuba.ac.jp> <20130416091949.GB12260@sys4.de> Message-ID: <516D8437.90201@msapiro.net> On 4/16/2013 2:19 AM, Patrick Ben Koetter wrote: > > Let me add some inconvenience: Log contains privacy related information. I'd > expect a log view/research interface to limit output to what the person/role > may look at and no more. This is an excellent point. It can be very useful for list admins or domain admins to view logs for various purposes, but multi-user shared hosting services (think cPanel) will be very reluctant to let these people see logs unless what they can see is restricted to their own list/domain. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From sreyanth at gmail.com Tue Apr 16 20:31:10 2013 From: sreyanth at gmail.com (Sreyanth) Date: Wed, 17 Apr 2013 00:01:10 +0530 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: <8761zm4xgt.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip3o3wgu.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zm4xgt.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Tue, Apr 16, 2013 at 3:19 PM, Stephen J. Turnbull wrote: > CC-ing Mailman Developers to keep all mentors up to date on what > you're thinking. > > Sreyanth writes: > > On Mon, Apr 15, 2013 at 4:13 PM, Stephen J. Turnbull < > stephen at xemacs.org>wrote: > > > > Worse, quoted footers are often actively harmful, because they contain > > > "unsubscribe" links for somebody else (but all the user can see is > > > "click here to unsubscribe"). > > > > > I completely forgot this "unsubscribe" links! Thanks for reminding :-) > > Will add in the application! > > Also, I want to know if I am in a correct path to make " Boilerplate > > stripping AND better content filtering / handling error messages" a > worthy > > GSoC project. > > I don't want to discourage you, but I'd have to say you're not really > making a lot of progress. ?Thanks for your feedback. I was actually just starting off with what the Mailman developer community is looking for under this project. I wanted to know more about how things work inside the community. I have just started working on my proposal and would be able to post it and blog about it on my site by this weekend and we can discuss on the proposal to make it more feasible. :) On the same lines, I would like to thank you, Barry, Terri and others? who gave their valuable inputs on this idea. > In brief, an "excellent" project proposal > is (IMO, other mentors may have other opinions) is composed of four > parts: > > 1. A "theme" that can be stated in one or two lines. You have that in > "Boilerplate stripping and content filtering", although I would be > more attracted to "Message body filtering, such as boilerplate > stripping or hiding". (It's more general in a way that probably > doesn't require more complex code.) Improving your theme is worth > doing (steal others' words if you're not a native speaker!), > because it makes it easier for the mentors to grasp your project. > > But a theme is all you really have shown at this point. > > 2. A statement of requirements. What will the *users* see that you've > accomplished? This is most important. (As explained below, we > don't expect students to be able to do 3 or 4 below without > mentoring -- but it's nice to have.) > > 3. A design, or statement of the changes to be made to the code to > satisfy the requirements. (Ie, what other *developers* will see!) > Much of the design can be done in cooperation with the mentors, but > it's very attractive if you are aware of similar projects or > features (expecially if they're already part of Mailman 2 or 3!) > > 4. A schedule. > > In more detail, here's something I drafted at the Mailman Sprint. > I can say that Terri got a kick out of it, but remember, it's not > official, it's just my opinion. (I really should put it on the wiki, > but I'm not sure where or even how -- does Confluence do ReST?) Note > that it is a *draft*, and especially the URLs may be wrong or > inaccurate. > > > ==================================== > Student Proposals Augmenting Mailman > ==================================== > > SPAM us at mailman-developers at python.org. > > How Not to SPAM: > ---------------- > > Don't do this:: > > To: mailman-developers at python.org > Subject: I want a Mailman mentor for GSoC > > Dear Mailman: > > I want to be a GSoC student. Please take me, please, please, > please, please? > > Yours truly, > A. N. Unlikely Candidate > > Like all GSoC projects, we're looking for students who care about > Mailman, and want to improve it. The application above screams "I > need money!" Yes, we know that, and we don't blame you one bit. But > there are candidates who we can tell will do a good job for us. They > need money too, you know. > > How to SPAM: > ------------ > > Briefly, you need to > > 1. Find out about Mailman. > > 2. Decide which part of Mailman you want to contribute to. > > 3. Decide on a particular task, *i.e.*, a problem to solve. > > 4. Propose a plan to solve the problem: > > a. Tell us who you are. > > b. Tell us what you want to do for Mailman. > > c. Tell us what code you will write to get it done. > > d. Tell us when you'll get it done. > > This is what software professionals do over and over again at work. > (Except for part 4a, but they have to do that once, just like you.) > One of the purposes of the GSoC is to provide mentors who can help you > accomplish those preliminary tasks, as well as produce excellent > finished code. > > A Few Hints > ----------- > > Step 1: Find out about Mailman_. Mailman started out as a mailing > list manager: it keeps a database of lists, subscribers, and posts; > moderates and distributes the posts to the subscribers; and maintains > an archive of past posts that users can access by mail or web. It is > written in the Python_ programming language. You are welcome to ask > questions either on the Mailman `users' list`_ or on the Mailman > `developers' list`_. We prefer to mentor students who have shown > interest in working on Mailman before applying (but it's not at all > required). > > .. _Mailman: http://www.list.org/ > .. _Python: http://www.python.org/ > .. _users' list: http://lists.python.org/mailman-users > .. _developers' list: http://lists.python.org/mailman-developers > > **Do** feel free to ask the mentor team about early drafts of your > proposal. You must write it yourself, but we can tell you if you're > writing about the right things, and if you are providing the right > details. > > Step 2. Decide where you'd like to contribute. The current version, > Mailman 2, is an integrated application written in Python 2. However, > it's become dated, and we are nearly ready to beta test a Mailman 3 > system. `Mailman 3`_ is currently written in a dialect of Python 2 > intended to make a transition to Python 3 easy. It is divided into > three subprojects: > > - `Mailman 3 core`_: implements the list and subscriber databases, and > the mail distribution functionality. **FIXMYURL** > > - `Postorius`_: a RESTful_ web interface for managing the list and > subscriber databases. **FIXMYURL** > > - `HyperKitty`_: the archive manager and web interface to the > archives. **FIXMYURL** > > While the developers all communicate with each other, especially in > developing shared APIs, actual design and coding takes place in > smaller groups organized around these subprojects. Each subproject > has a list of `suggested tasks`_. **FIXMYURL** > > .. _Mailman 3: http://wiki.mailman.org/ > .. _Mailman 3 core: http://wiki.mailman.org/ > .. _Postorius: http://wiki.mailman.org/ > .. _HyperKitty: http://wiki.mailman.org/ > .. _RESTful: > https://advanced-python.readthedocs.org/en/latest/rest/what-is-rest.html > .. _suggested tasks: http://wiki.mailman.org/ > > Step 3. Pick a task. Why you pick a task is up to you. But feel free > to ask the mentors or the lists. Maybe you just want the help the > developers by doing something we'd do otherwise. Ask on the > `developers' list`_ for our priorities and advice on what's actually > do-able in the GSoC time frame. Maybe you want to implement a new > feature. Check Launchpad_ for feature requests, and maybe review the > `users' list`_ archives and the FAQ_. **FIXMYURL** Ask the developers > if it can be done in time. > > .. _FAQ: http://wiki.list.org/ > > Step 4. SPAM_ us! > > See also `How Not to SPAM`_. > > .. _SPAM: #how-to-spam-in-detail > .. _How Not to SPAM: #how-not-to-spam > > How to SPAM, in Detail > ---------------------- > > "SPAM" is a "Student Proposal Augmenting Mailman." Perhaps you've > contributed to an old-style open source project or done an internship > where developers hack on code and simply commit it to the mainline > repository when they're pretty sure there aren't any bugs in the new > code. Sure, Linus did it that way -- but don't expect to get *your* > code in *his* kernel that way! The trend is to more formal processes > including explicit design and review stages. Pragmatically, GSoC > requires some formality because Google requires us to report on > interns' progress and review their code. But we also need some > documentation to ensure that we make the best use of the few interns > we are assigned, and we're used to it because many changes to our > favorite language, Python, require `Python Enhancement Proposals`_. > > .. _Python Enhancement Proposals: http://www.python.org/dev/peps/ > > A SPAM is not as formal as a PEP. However, you should take care in > preparing yours, because it's one of the few things we have to go on > in deciding which students to recommend to Google. A SPAM should > contain certain specific information, there's a lot of more or less > optional information you can include to improve both your chances of > acceptance and the project itself, and it probably won't hurt to > organize it as described below. > > Do **not** worry if you can't write an accurate design or plan yet. > Planning and design are arts, and probably the most important things a > mentor can help a student to learn to do. That's really what GSoC is > about. And, of course, some students won't know that much about > Mailman to start with. So just do the best you can now to give us an > idea of what you want to do, and your mentors will help you whip it > into shape during the "get to know each other" preliminary phase. > > But if you *do* come up with a good design and schedule by yourself, > that's very attractive to mentoring orgs (not just Mailman, believe > me!) What follows are some hints on what to aim for in your > proposal. > > Self-Introduction > ................. > > Tell us about yourself. Full name, school affiliation, year in > school, of course. Previous programming experience, including > languages used and publicly documented projects you've worked on, if > any. URLs for any web resources about yourself are useful, including > anything from Facebook pages to GitHub projects. > > You will eventually need a LaunchPad_ login to publish and submit your > contribution for review and integration. That and your email address, > as well as a phone number, are essential contact information. It is > often helpful to have a stable IRC nick that you can use on the > Mailman channel (#mailman on Freenode). > > .. _LaunchPad: http://launchpad.net/ > > Coding Proposal: Theme > ...................... > > Robert Townsend wrote in *Up the Organization*: "[Top management] > knows that any proposal affecting their business can be explained in > one minute." My colleagues were horrified to be compared to > management of any ilk, but the principle holds. It's both possible, > and important, to be able to express what your proposal is about > relatively briefly. Provide both a one-line summary that can serve as > a title for your project, as well as a slightly more detailed (around > 5 lines) explanation of how you propose to accomplish your task. > > We have a number of `suggested tasks`_. You are welcome to select one > that appeals to you. If you *do* choose one of these tasks, > your ideas about the task, and an original description of the task, > are important to convince us you have thought about the task. Don't > just copy our description into your proposal. Rewrite it in your own > words, adding your own ideas. > > Coding Proposal: Design > ....................... > > This is a very high-level design. You can express it as prose or as > pseudo-code. More important than an exact description of the code you > propose to write is showing us *how little* you need to do to > accomplish your goal. Tell us about existing modules on PyPI_ or > GitHub_ that can do "90% of the work". (Don't worry, you'll be wrong. > There will be *plenty* of work left for you to do.) Explain how these > modules and your implementation will hook into the rest of Mailman. > > You're not done with the design, yet. You *will* need to refine it > somewhat, in order to complete your schedule_. > > .. _PyPI: http://pypi.python.org/ > .. _GitHub: http://github.com/ > .. _schedule: #coding-proposal-schedule > > Coding Proposal: Schedule > ......................... > > Perhaps unlike anything you've done before, GSoC has several deadlines > that you *must* meet or *you will get fired* (*i.e.*, **not paid**). > While many curmudgeons_ will argue that *getting fired* is a "valuable > educational experience", it reflects as badly on your mentors as it > does on you. We do **not** want that to happen. We have come close > to firing a student at the midterm in the past; it *can* happen. > Accurate scheduling is important to *you*. > > First, review the official GSoC deadlines. They are posted on the > `GSoC site`_. Add them to your personal GSoC schedule. > > Second, for each official GSoC deadline, add a date one week before to > confer with your mentor about your progress in general, and any > specific milestones you need to complete to pass your reviews. This > is a date where, if your mentors don't get in touch with you, you must > get in touch with them. If you can't reach the mentors, get in touch > with the Mailman org representative, Terri Oda. Add these dates > to your personal GSoC schedule. > > Third, set milestones for yourself. A *milestone* is a subtask that > can be objectively verified. For example, "coding is 50% complete" is > *not* a milestone, because you don't know how much code you have left > to write, you can only estimate it inaccurately. "Coding is complete, > documentation is complete, unit and integration tests all pass, branch > pushed to Launchpad, and a merge request filed" is a milestone, > because all of the above are easily verified events. In fact, this is > the most important milestone, which you should schedule for September > 16 right now, because it is the criterion for passing your final > review. > > In a perfect world, a student would set about one milestone per week, > which is pretty convenient for both our evaluations and your > self-evaluations. However, as a practical matter, setting milestones > depends on a detailed design, and you probably won't produce a > detailed design until after you've been approved for the program. In > your application to us, it is very attractive if at least you are able > to provide your midterm milestone. > > In order to produce milestones other than final completion, you need > to have a somewhat detailed design. At this point, go back and revise > your design, then pick some specific task and set it as a milestone. > To get to your midterm milestone, think of it as the set of your > earlier milestones. As you add each milestone, consider how much work > that set of milestones represents. If that's not enough to constitute > half the work, revise your design again, and set another milestone. > > **Do** feel free to ask the Mailman mentor team about your > milestones. Few developers find it a natural way to think about > development at first. We can and will help you with it as you develop > your proposal. > > The fourth step in scheduling is the most important. Look at your > private schedule, and see what is going to cause conflicts with your > GSoC schedule. Maybe you have a summer course at school. Or your > school's spring examination schedule conflicts with GSoC. Maybe your > older brother is getting married on the other side of the world from > where you are. We don't need to know *what* it is, but you need to > tell us *when* it is, and *adjust your GSoC schedule* to accommodate > your private plans. Of course you can't change the GSoC deadlines, > and there's really no leeway there for us, either. What *you* can do > is to adjust the timing of your milestones, and what *we* can do is > adjust our expectations for the midterm review. > > Note that this is an adjustment in timing, not an adjustment in the > size of the completed project expected. Software work is like that > for many developers: sometimes they work a 5x8 week, and other times > they work a 15+15+10+0+0 week. *We* understand that reality. But > *you* need to think in terms of picking up the pace before and after > if you take time off. On the other hand, if your time off is too > great, you will **not** be able to catch up. Really -- we've been > there. > > And be realistic. If you have extensive other plans for your summer, > you probably should give up on GSoC. Google expects you to treat your > internship as a full-time job during the coding period. (Of course > you can always change your other plans if GSoC is important to you!) > > Be honest with yourself, and be honest with us. You will have a much > more enjoyable *and profitable* summer that way. If your plan crowds > too many milestones into the first half or second half of the summer, > we won't believe you can do it, and you won't be asked to participate. > If you pretend that you can keep a better balance than is realistic > given other commitments, you will fail a review. We've seen it happen > before. > > .. _curmudgeons: http://en.wikipedia.org/wiki/curmudgeon > .. _GSoC site: http://google-melange.com/ > > > -- *Yours Sincerely* * * *Mora Sreyantha Chary* *Computer Engineering '14* *National Institute of Technology Karnataka* *Surathkal, India 575 025* From avikpal.me at gmail.com Tue Apr 16 21:13:43 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Wed, 17 Apr 2013 00:43:43 +0530 Subject: [Mailman-Developers] GSOC 2013 project discussion Message-ID: Hello, regarding the anti-spam feature in mailman I would like to say that I have almost completed development of a rudimentary classifier implementing Naive Bayes, but I need to train my classifier with some test data and also look into the precision recall. but now I need a labeled dataset to check these,any idea where I can get one?(though I know that naive Bayes is not going to be a good classifier but I am looking at it right now just as a start). also I would like to propose an idea of my own. Many of us set the preference in mailman to get all the emails of a day batched together, but sometimes this means we miss important mails(though we get it at the end of the day but we miss the moment)----important to the community, or my own interest, discussion on something I also have discussed upon in my previous mails, delivery of these mails instantly to the subscriber so that he can also join at that very moment may come out to be a very useful feature. Thus person gets to set two options 1.receive batched mails only. 2.receive batched mails with important mails delivered instantly. please take a look and let me know. Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme From rkw at DATAPLEX.NET Tue Apr 16 21:32:55 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Tue, 16 Apr 2013 14:32:55 -0500 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: References: Message-ID: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> An interesting suggestion -- A couple of things to consider: How do you identify "important" messages? Will you deliver these messages twice -- first as important and then, later, as a part of the digest ? On Apr 16, 2013, at 2:13 PM, Avik Pal wrote: > also I would like to propose an idea of my own. Many of us set the > preference in mailman to get all the emails of a day batched together, but > sometimes this means we miss important mails(though we get it at the end of > the day but we miss the moment)----important to the community, or my own > interest, discussion on something I also have discussed upon in my previous > mails, delivery of these mails instantly to the subscriber so that he can > also join at that very moment may come out to be a very useful feature. > Thus person gets to set two options > 1.receive batched mails only. > 2.receive batched mails with important mails delivered instantly. From flo.fuchs at gmail.com Tue Apr 16 23:13:32 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Tue, 16 Apr 2013 23:13:32 +0200 Subject: [Mailman-Developers] Regarding Authentication of REST API In-Reply-To: References: Message-ID: Hi Manish, hi everyone, 2013/4/10 Manish Gill : > For the GSoC REST API project, I've been wondering about how > authentication would work. > > OAuth is a way to go if we want authenticated/signed requests. I have a > few questions regarding that. > > - Will Mailman core become an OAuth provider, with Postorius/API being > the consumers? Probably not the core itself, but possibly another yet-to-be-written application that Postorius, Hyperkitty and other clients could use. We had a long discussion on this list whether to build a central application to store user data that can be accessed by the different Mailman-related applications. While we haven't decided yet whether or how to proceed, this would possibly be the right context for that. > - If the answer to the above is no, is the plan to support populer OAuth > providers like Facebook/Twitter ? Like we discussed on IRC earlier, it would be nice if a site running Mailman could act as an oAuth provider. Especially since the thought of a FLOSS mailing list manager requiring an account with a commercial oAuth service provider to use its API might seem a little odd. But implementing both the provider as well as the client is probably way beyond the scope of this GSoC project. Especially since authentication is only one aspect of it. > (If not, can you guys please explain how would the authentication > protocol really work?) > > - Since Postorius is already using Mozilla Persona, can that also be > used to provide authentication to API clients? Probably not Persona, which is meant to be used in the context of a browser. But are we sure oAuth is our only option in an API context? Are there other opinions? > - Am I over-thinking this? :) I don't think so. It's not exactly obvious. BTW, the oauthlib documentation has a nice overview over the different oAuth workflows [1]. Florian [1] https://oauthlib.readthedocs.org/en/latest/oauth_1_versus_oauth_2.html From stephen at xemacs.org Wed Apr 17 06:31:40 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 17 Apr 2013 13:31:40 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> <878v4i4z52.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87sj2p3hhv.fsf@uwakimon.sk.tsukuba.ac.jp> Pratik Sarkar writes: > I am working on the proposal.And how many slots are there for the filter > project? There are no slots for the filter project as such. The whole Mailman project has slots, and they are somewhat fluid, since we operate under the umbrella of the Python Software Foundation. For further details, like actual numbers, Terri Oda (PSF org admin) is authoritative. Please be patient on this, as Terri is *very* busy right now. Also, you may not get hard numbers until the selection is actually made, since the PSF has to balance many projects. From stephen at xemacs.org Wed Apr 17 06:33:25 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 17 Apr 2013 13:33:25 +0900 Subject: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion] In-Reply-To: <516D7B97.6070905@zone12.com> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <5167148C.4000701@zone12.com> <874nf85s90.fsf@uwakimon.sk.tsukuba.ac.jp> <87mwt03y3r.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip3o3wgu.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zm4xgt.fsf@uwakimon.sk.tsukuba.ac.jp> <20130416093909.00dd8de4@anarchist> <516D7B97.6070905@zone12.com> Message-ID: <87r4i93hey.fsf@uwakimon.sk.tsukuba.ac.jp> Terri Oda writes: > Or, if it's less trouble, feel free to stick it on the Python wiki > (which is moin) under SummerOfCode; I was hoping to link to it anyhow as > advice to help students, so it is actually going to be relevant to > python students in general. I think that's the way to go, but I may not have time to research the URLs until Saturday. If somebody want to beat me to it, feel free. Please post the URL here, though! Steve From mgill25 at outlook.com Wed Apr 17 11:15:23 2013 From: mgill25 at outlook.com (Manish Gill) Date: Wed, 17 Apr 2013 14:45:23 +0530 Subject: [Mailman-Developers] Regarding Authentication of REST API In-Reply-To: References: Message-ID: On 04/17/2013 02:43 AM, Florian Fuchs wrote: > Hi Manish, hi everyone, > > 2013/4/10 Manish Gill : >> For the GSoC REST API project, I've been wondering about how >> authentication would work. >> >> OAuth is a way to go if we want authenticated/signed requests. I have a >> few questions regarding that. >> >> - Will Mailman core become an OAuth provider, with Postorius/API being >> the consumers? > Probably not the core itself, but possibly another yet-to-be-written > application that Postorius, Hyperkitty and other clients could use. We > had a long discussion on this list whether to build a central > application to store user data that can be accessed by the different > Mailman-related applications. While we haven't decided yet whether or > how to proceed, this would possibly be the right context for that. That makes sense. > >> - If the answer to the above is no, is the plan to support populer OAuth >> providers like Facebook/Twitter ? > Like we discussed on IRC earlier, it would be nice if a site running > Mailman could act as an oAuth provider. Especially since the thought > of a FLOSS mailing list manager requiring an account with a commercial > oAuth service provider to use its API might seem a little odd. But > implementing both the provider as well as the client is probably way > beyond the scope of this GSoC project. Especially since authentication > is only one aspect of it. Indeed! This could be made easy if we don't have to take care of the provider implementation ourselves, like we discussed. If a third party library exists that could be used to provide this functionality, it would make things much easier. :) >> (If not, can you guys please explain how would the authentication >> protocol really work?) >> >> - Since Postorius is already using Mozilla Persona, can that also be >> used to provide authentication to API clients? > Probably not Persona, which is meant to be used in the context of a browser. > > But are we sure oAuth is our only option in an API context? Are there > other opinions? Hmm. I don't know much about it. I looked at Tastypie, and it provides HTTP Basic Auth [1]. Much simpler, but probably much less secure as well. [1] http://django-tastypie.readthedocs.org/en/latest/authentication.html > BTW, the oauthlib documentation has a nice overview over the different > oAuth workflows [1]. > > > Florian > > [1] https://oauthlib.readthedocs.org/en/latest/oauth_1_versus_oauth_2.html > > Cool! :) -- - Manish Gill Naeblis on Freenode @mgill25 on Twitter/Github From varunsharmalive at gmail.com Wed Apr 17 12:00:30 2013 From: varunsharmalive at gmail.com (varun sharma) Date: Wed, 17 Apr 2013 15:30:30 +0530 Subject: [Mailman-Developers] Working on "Better User Settings Management" project idea Message-ID: Hi, I am working on "better user settings management" for past few weeks. As i have worked on django earlier also, so i am enjoying it.I branched out postorius few weeks back and sent a merge request but after having discussion with terri and florian, i now know that extension of django's User class in not a good idea, so i am now concentrating on client object and its methods. *I have few queries regarding the project:* 1. Should i concentrate on resolving the existing bugs or adding new features right now ? 2. There are some menu links which do not have any pages linked with them, should i create them and add the desired functionality ? Thanks Varun Sharma From avikpal.me at gmail.com Wed Apr 17 13:16:05 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Wed, 17 Apr 2013 16:46:05 +0530 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> Message-ID: for identifying an important message a classifier will be implemented. and thanks for pointing out the issue regarding the delivery of the message, if it is delivered twice then the existing implementation of delivery is sufficient, but if we want to deliver it only once then for each person we need to maintain a database of important mails/threads to him(or vice-versa) and while sending check against that database. but this is going to raise some normalization issues which are to be taken care of by careful designing. Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 17 April 2013 01:02, Richard Wackerbarth wrote: > An interesting suggestion -- A couple of things to consider: > > How do you identify "important" messages? > > Will you deliver these messages twice -- first as important and then, > later, as a part of the digest ? > > > On Apr 16, 2013, at 2:13 PM, Avik Pal wrote: > > also I would like to propose an idea of my own. Many of us set > the > > preference in mailman to get all the emails of a day batched together, > but > > sometimes this means we miss important mails(though we get it at the end > of > > the day but we miss the moment)----important to the community, or my own > > interest, discussion on something I also have discussed upon in my > previous > > mails, delivery of these mails instantly to the subscriber so that he can > > also join at that very moment may come out to be a very useful feature. > > Thus person gets to set two options > > 1.receive batched mails only. > > 2.receive batched mails with important mails delivered instantly. > > From rkw at DATAPLEX.NET Wed Apr 17 14:07:15 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Wed, 17 Apr 2013 07:07:15 -0500 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> Message-ID: <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> In evaluating a proposal, we need to look at a number of factors: First, will it work? -- Does the proposed design accomplish the stated objective? Next: Is it useful? And: Can the candidate be expected to accomplish the task within the allotted time frame? Finally: Is it the best use for our limited resources (funding, mentor time, etc.)? If your presentation makes it easier to answer each of those questions in a positive manner, it will increase the likelihood that it will get funded. On Apr 17, 2013, at 6:16 AM, Avik Pal wrote: > for identifying an important message a classifier will be implemented. and thanks for pointing out the issue regarding the delivery of the message, if it is delivered twice then the existing implementation of delivery is sufficient, but if we want to deliver it only once then for each person we need to maintain a database of important mails/threads to him(or vice-versa) and while sending check against that database. but this is going to raise some normalization issues which are to be taken care of by careful designing. > > Avik Pal > Bengal Engineering & Science University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > > > > On 17 April 2013 01:02, Richard Wackerbarth wrote: > An interesting suggestion -- A couple of things to consider: > > How do you identify "important" messages? > > Will you deliver these messages twice -- first as important and then, later, as a part of the digest ? > > > On Apr 16, 2013, at 2:13 PM, Avik Pal wrote: > > also I would like to propose an idea of my own. Many of us set the > > preference in mailman to get all the emails of a day batched together, but > > sometimes this means we miss important mails(though we get it at the end of > > the day but we miss the moment)----important to the community, or my own > > interest, discussion on something I also have discussed upon in my previous > > mails, delivery of these mails instantly to the subscriber so that he can > > also join at that very moment may come out to be a very useful feature. > > Thus person gets to set two options > > 1.receive batched mails only. > > 2.receive batched mails with important mails delivered instantly. > > From avikpal.me at gmail.com Wed Apr 17 14:56:53 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Wed, 17 Apr 2013 18:26:53 +0530 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> Message-ID: thanks a lot for the information. Thing is that I don't think that the Spam classifier by itself is going to be big enough so I came up with this idea. Actually I also need to know what the community wants, regarding the e-mail delivery. and regarding the classifier I don't think that it is not going to be a problem at all( from my end, with my previous experience in machine learning NLP, just we need a database for the subscribers where classifier data for them is going to be stored) but the most important thing is what you have pointed out "Is it the best use for our limited resources (funding, mentor time, etc.)?" I am looking forward to Barry, Terri in this regard. Meanwhile It would be much appreciated if someone can direct me to an labeled dataset available on line. Also somebody was talking about legal aspects in some countries and also the fact that the classification to be done in MTA only. Here I have a suggestion, after submitting, whenever an email is classified as Spam, we store it in a separate archive and after the end of the day send them a mail telling "this is the digest for all the mails that Mailman thinks to be Spam" the subscriber may go there and can view them and also can mark them as not Spam, which will help the learning algorithm to work on the decision boundary and also the precision recall are also to be found out which upon adjusting the boundary or after being marked by majority(in simple words) as not Spam will be incorporated back into the main archive and will be sent as a part of the main digest then. Emails which stays as Spam will be dropped after a month Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 17 April 2013 17:37, Richard Wackerbarth wrote: > In evaluating a proposal, we need to look at a number of factors: > > First, will it work? -- Does the proposed design accomplish the stated > objective? > Next: Is it useful? > And: Can the candidate be expected to accomplish the task within the > allotted time frame? > Finally: Is it the best use for our limited resources (funding, mentor > time, etc.)? > > If your presentation makes it easier to answer each of those questions in > a positive manner, it will increase the likelihood that it will get funded. > > > On Apr 17, 2013, at 6:16 AM, Avik Pal wrote: > > for identifying an important message a classifier will be implemented. and > thanks for pointing out the issue regarding the delivery of the message, if > it is delivered twice then the existing implementation of delivery is > sufficient, but if we want to deliver it only once then for each person we > need to maintain a database of important mails/threads to him(or > vice-versa) and while sending check against that database. but this is > going to raise some normalization issues which are to be taken care of by > careful designing. > > Avik Pal > Bengal Engineering & Science University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > > > > On 17 April 2013 01:02, Richard Wackerbarth wrote: > >> An interesting suggestion -- A couple of things to consider: >> >> How do you identify "important" messages? >> >> Will you deliver these messages twice -- first as important and then, >> later, as a part of the digest ? >> >> >> On Apr 16, 2013, at 2:13 PM, Avik Pal wrote: >> > also I would like to propose an idea of my own. Many of us set >> the >> > preference in mailman to get all the emails of a day batched together, >> but >> > sometimes this means we miss important mails(though we get it at the >> end of >> > the day but we miss the moment)----important to the community, or my own >> > interest, discussion on something I also have discussed upon in my >> previous >> > mails, delivery of these mails instantly to the subscriber so that he >> can >> > also join at that very moment may come out to be a very useful feature. >> > Thus person gets to set two options >> > 1.receive batched mails only. >> > 2.receive batched mails with important mails delivered instantly. >> >> > > From terri at zone12.com Wed Apr 17 17:32:12 2013 From: terri at zone12.com (Terri Oda) Date: Wed, 17 Apr 2013 09:32:12 -0600 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> Message-ID: <516EC07C.7060503@zone12.com> On 13-04-17 6:56 AM, Avik Pal wrote: > Meanwhile It would be much appreciated if someone can direct me to > an labeled dataset available on line. > Leaving aside entirely the question of whether we should (or will) support any project that requires learning on this scale, as a former anti-spam researcher, I can at least answer this question. Unfortunately, the answer is largely "good luck with that" -- good labelled email data is surprisingly hard to come by, and that challenge is one of the reasons I stopped doing research in that area. When I was doing anti-spam research, the only viable public classified ham/spam set was the SpamAssassin one. I don't believe it's been maintained with modern messages and at this point it may be useless. Shortly after I left the field, people started using the Enron data set, which is pretty well classified by now, but again, is pretty long in the tooth. Given that you're going to want to be classifying mailing list data, you may have to produce some synthetic data sets using information from publicly available mailing lists (e.g. the public archives of mailman-developers are available) and combining them with other data sources (e.g. publicly available collections of spam). This won't have a whole lot of interesting sub-labels (some lists will have more than others, depending on their use of dlists/topics/pre-classification by the sender) and a synthetic set is generally regarded as a poor information source for reproducible results, but it could be enough in a pinch given that you're adding a feature rather than publishing scientific work. Note that the GSoC timeline doesn't allow time for finding and creating such a set, so if you're going to use one, you should determine in advance what you'll be using and and be able to provide a link to the completely-ready-for-gsoc set in your proposal. Terri From terri at zone12.com Wed Apr 17 17:57:36 2013 From: terri at zone12.com (Terri Oda) Date: Wed, 17 Apr 2013 09:57:36 -0600 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <87sj2p3hhv.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> <878v4i4z52.fsf@uwakimon.sk.tsukuba.ac.jp> <87sj2p3hhv.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <516EC670.5080706@zone12.com> On 13-04-16 10:31 PM, Stephen J. Turnbull wrote: > Pratik Sarkar writes: > > > I am working on the proposal.And how many slots are there for the filter > > project? > > There are no slots for the filter project as such. The whole Mailman > project has slots, and they are somewhat fluid, since we operate under > the umbrella of the Python Software Foundation. For further details, > like actual numbers, Terri Oda (PSF org admin) is authoritative. > > Please be patient on this, as Terri is *very* busy right now. Also, > you may not get hard numbers until the selection is actually made, > since the PSF has to balance many projects. > Stephen's right -- we don't pre-set our slots. We're basically going to rank the applications we get in the order of "most want to mentor" to "least want to mentor" with some adjustments for the availability and interests of specific mentors so each mentor will get a project and student who suits them, then we'll accept the top X students. Probably we will not use more than one slot for any particular project this year, but that discussion will happen among mentors during selection. How many slots will we have? Luckily for you, despite being busy, I've had to answer this question a few times already, so here's cut-and-paste from another email to one of the mentors who asked: --- The short answer is "it depends on what Google will give us." The PSF, in the past, has taken on around 35 students. But this year, we've grown by at least 50% in terms of projects. We may, by the time applications open, have as many as 19 sub-projects working under our umbrella. I'm hoping to request and receive at least enough slots for each project to get two students if they have the mentors to support this, so the rule of thumb for now can probably be that you'll all get slots for up to 2 students. For the projects participating for the first time this year, probably 1-2 slots is the right number. But some of our projects are veteran GSoC participants who have significantly more than the minimum 3 required mentors. Those projects, hopefully, will be able to get more slots since I know they can and will successfully support more students. In previous years, Google's been great about letting us have all the slots we tell them we want and can support, but with the amount of growth we've had this year, I'm hesitant to make any promises. --- Mailman is one of these veteran projects. I believe we currently have 6 mentors listed, but given my schedule for this year I'll be at most a co-mentor, and given his usually busy schedule Barry may do the same, so my guess is that at most, we'll be able to justify asking for 4 slots this year. Terri From avikpal.me at gmail.com Wed Apr 17 18:02:03 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Wed, 17 Apr 2013 21:32:03 +0530 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: <516EC07C.7060503@zone12.com> References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> <516EC07C.7060503@zone12.com> Message-ID: thanks a lot Terri, I think I will go with the Enron email dataset and they are to be cross validated against publicly available legitimate mailing list mails and Spam and (hopefully) python's regular expressions will help me a lot building the synthetic set. Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 17 April 2013 21:02, Terri Oda wrote: > > On 13-04-17 6:56 AM, Avik Pal wrote: > >> Meanwhile It would be much appreciated if someone can direct me >> to >> an labeled dataset available on line. >> >> Leaving aside entirely the question of whether we should (or will) > support any project that requires learning on this scale, as a former > anti-spam researcher, I can at least answer this question. > > Unfortunately, the answer is largely "good luck with that" -- good > labelled email data is surprisingly hard to come by, and that challenge is > one of the reasons I stopped doing research in that area. > > When I was doing anti-spam research, the only viable public classified > ham/spam set was the SpamAssassin one. I don't believe it's been > maintained with modern messages and at this point it may be useless. > > Shortly after I left the field, people started using the Enron data set, > which is pretty well classified by now, but again, is pretty long in the > tooth. > > Given that you're going to want to be classifying mailing list data, you > may have to produce some synthetic data sets using information from > publicly available mailing lists (e.g. the public archives of > mailman-developers are available) and combining them with other data > sources (e.g. publicly available collections of spam). This won't have a > whole lot of interesting sub-labels (some lists will have more than others, > depending on their use of dlists/topics/pre-**classification by the > sender) and a synthetic set is generally regarded as a poor information > source for reproducible results, but it could be enough in a pinch given > that you're adding a feature rather than publishing scientific work. > > Note that the GSoC timeline doesn't allow time for finding and creating > such a set, so if you're going to use one, you should determine in advance > what you'll be using and and be able to provide a link to the > completely-ready-for-gsoc set in your proposal. > > Terri > > > From avikpal.me at gmail.com Wed Apr 17 18:10:06 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Wed, 17 Apr 2013 21:40:06 +0530 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> <516EC07C.7060503@zone12.com> Message-ID: > On 17 April 2013 21:02, Terri Oda wrote: > >> >> On 13-04-17 6:56 AM, Avik Pal wrote: >> >>> Meanwhile It would be much appreciated if someone can direct >>> me to >>> an labeled dataset available on line. >>> >>> Leaving aside entirely the question of whether we should (or will) >> support any project that requires learning on this scale, as a former >> anti-spam researcher, I can at least answer this question. >> >> Unfortunately, the answer is largely "good luck with that" -- good >> labelled email data is surprisingly hard to come by, and that challenge is >> one of the reasons I stopped doing research in that area. >> >> >> > > Don't lose hope Terri, after digging for a couple of hours came across this and its pretty much updated. http://untroubled.org/spam/ From terri at zone12.com Wed Apr 17 18:35:10 2013 From: terri at zone12.com (Terri Oda) Date: Wed, 17 Apr 2013 10:35:10 -0600 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> <516EC07C.7060503@zone12.com> Message-ID: <516ECF3E.7080301@zone12.com> On 13-04-17 10:10 AM, Avik Pal wrote: > Don't lose hope Terri, after digging for a couple of hours came across > this and its pretty much updated. http://untroubled.org/spam/ Finding sources of spam (like that one) isn't that hard; it's finding sources of legit email combined with spam and classified and processed in the same way that's challenging. As I said, you can combine a spam source like this with a publicly available mailing list to make a synthetic set, but scientifically speaking, those aren't really preferred ways to handle data because they come from multiple sources. The problem is that when you have multiple sources it sometimes becomes too easy for a classifier to classify on less-than-useful features for future use. For example, one might classify on the fact that the list address won't appear in any of the To: or Cc: lines in the spam data because it comes from a different source, the fact that many of the spams will be from different time periods, the fact that the spam data is anonymized differently from any list data you might have, etc. You will wind up doing a lot of work to normalize the data sets to avoid these classifiers (and we're talking weeks of really boring work here, potentially, that you need to start Right Now if you're going to be using such a set), and you run the risk of missing out on features that would have been useful in a single-source set that have been completely obliterated by the synthetic data set. Terri From avikpal.me at gmail.com Wed Apr 17 18:51:34 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Wed, 17 Apr 2013 22:21:34 +0530 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: <516ECF3E.7080301@zone12.com> References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> <516EC07C.7060503@zone12.com> <516ECF3E.7080301@zone12.com> Message-ID: ya I get your point, but see these are part of any machine learning project, and feature extraction has to be done considering the synthetic data set. On 17 April 2013 22:05, Terri Oda wrote: > > > Finding sources of spam (like that one) isn't that hard; it's finding > sources of legit email combined with spam and classified and processed in > the same way that's challenging. As I said, you can combine a spam source > like this with a publicly available mailing list to make a synthetic set, > but scientifically speaking, those aren't really preferred ways to handle > data because they come from multiple sources. > > > well in this regard the only thing I can do is keep looking, I am also aware that coming from different sources can make them skewed but again these things are never perfect and there are always scope for betterment, I think that our aim should be to implement a rudimentary classifier with fairly good performance to start with. From terri at zone12.com Wed Apr 17 20:28:41 2013 From: terri at zone12.com (Terri Oda) Date: Wed, 17 Apr 2013 12:28:41 -0600 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> <516EC07C.7060503@zone12.com> <516ECF3E.7080301@zone12.com> Message-ID: <516EE9D9.70802@zone12.com> I'm glad you're somewhat aware of the issues. I frequently encounter folk who aren't aware of the issues in machine learning, so your "don't lose hope" email set off all kinds of warning bells in my head. Going back to GSoC-specific stuff: - Enron is a very old data set - If you're going to use it, you need to be prepared to defend that choice. I'm not sure it's a choice that can be defended at all, knowing the field. It's probably not only an old data set, but a completely counter-productive one given the space in which Mailman operates. So here's some things to think about: (1) I want some justification of how this is going to be relevant to the problem you're trying to solve, which is "helping classify spam emails sent to a mailing list that the MTA was unable to classify" (2) Many existing classifiers that run at the MTA level have already used the enron data set, so chances are any features you learn will either already have been incorporated. I have severe concerns that any new features you learn will result in over-fitting. How can you believe that yet another classifier trained on the same data will be worth the processing overhead and resulting delays in mail delivery when it seems likely that any improvement will be incremental at best? (3) Enron is not going to help you make use of any list-specific features. How can you use this data set to produce something that is useful to Mailman, going beyond what any MTA-level spam filter can do? (Note that we've been telling people to do spam filtering at the MTA level for years and years and years; justifying this is not going to be an easy task) (4) If you're going to do cross-validation with other data to make claims that the final classifier will be relevant to list data, how is that data going to be obtained, processed, and used? (5) Unless you've got a plan for making extensive use of the fact that you're classifying mailing list data and not general email, you're pretty much wasting our time since we are only interested in projects relevant to Mailman. To be completely honest, I'm still seeing "student project for data mining class" level thinking here, and that's not going to be good enough for us. Especially considering that you didn't even know about the most common data sets for this problem, I'm concerned that you haven't yet reached the skill and experience necessary for us to seriously consider a classifier as even a small part of a GSoC project. We have to give priority to students who we are convinced can finish their projects, and it seems like there's too many chances of you getting stuck on finding data and using it correctly on a problem that is actually meaningful to Mailman and not just a general classification task. Terri On 13-04-17 10:51 AM, Avik Pal wrote: > ya I get your point, but see these are part of any machine learning > project, and feature extraction has to be done considering the > synthetic data set. > > > On 17 April 2013 22:05, Terri Oda > wrote: > > > > Finding sources of spam (like that one) isn't that hard; it's > finding sources of legit email combined with spam and classified > and processed in the same way that's challenging. As I said, you > can combine a spam source like this with a publicly available > mailing list to make a synthetic set, but scientifically speaking, > those aren't really preferred ways to handle data because they > come from multiple sources. > > > well in this regard the only thing I can do is keep looking, I am > also aware that coming from different sources can make them skewed but > again these things are never perfect and there are always scope for > betterment, I think that our aim should be to implement a rudimentary > classifier with fairly good performance to start with. From stephen at xemacs.org Wed Apr 17 19:06:19 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 18 Apr 2013 02:06:19 +0900 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> Message-ID: <87li8h2ik4.fsf@uwakimon.sk.tsukuba.ac.jp> Avik Pal writes: > Meanwhile It would be much appreciated if someone can direct me to > an labeled dataset available on line. By "labelled" you mean pre-classified into spam vs ham? I see you already found one, but you could also check the SpamBayes and SpamAssassin distributions. > Here I have a suggestion, after submitting, whenever an email is > classified as Spam, we store it in a separate archive and after the > end of the day send them a mail telling "this is the digest for all > the mails that Mailman thinks to be Spam" the subscriber may go > there and can view them and also can mark them as not Spam, I suggest that you present this as an option for users who want to tune the filters, and as something that can be used pre-release to develop the initial parameters for the distributed classifier. Although Bayesian classifiers do offer the option to train or tune your personal classifier on a local corpus, most users just stick with the distribution parameters plus self-training. It's pretty effective (surprisingly so to me). I guess the logic is that spammers aren't terribly creative. > Emails which stays as Spam will be dropped after a month Let's think carefully about that. Everybody deletes the spam; that's why you started by asking for a labelled dataset, because nobody keeps one around. Somebody really ought to do the public service of collecting a corpus. Of course, if you do arrange to keep it around, it's going to need to be an option that sites and list owners can disable. From barry at list.org Wed Apr 17 21:09:13 2013 From: barry at list.org (Barry Warsaw) Date: Wed, 17 Apr 2013 15:09:13 -0400 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: References: Message-ID: <20130417150913.197a5199@anarchist> On Apr 17, 2013, at 12:43 AM, Avik Pal wrote: >also I would like to propose an idea of my own. Many of us set the preference >in mailman to get all the emails of a day batched together, but sometimes >this means we miss important mails(though we get it at the end of the day but >we miss the moment)----important to the community, or my own interest, >discussion on something I also have discussed upon in my previous mails, >delivery of these mails instantly to the subscriber so that he can also join >at that very moment may come out to be a very useful feature. I'm skeptical that you'll be able to do automated classification of important messages for individual users. I'm not even sure I can do that as a human reading a particular mailing list. Also, what's important to me today may or may not be important to me tomorrow, and vice versa. Mailman has a (probably little known) feature where a list owner can send an urgent message that bypasses digests. If it has an Urgent: header with a matching password, it goes straight through. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From sharma.adwait at gmail.com Wed Apr 17 23:21:13 2013 From: sharma.adwait at gmail.com (Adwait Sharma) Date: Thu, 18 Apr 2013 02:51:13 +0530 Subject: [Mailman-Developers] Query regarding GSoC project Message-ID: Hello ! By way of introduction I am *Adwait Sharma,* a final year computer science undergraduate from Bangalore, India who code a lot specially in Python. The idea I found most interesting in GSoC 2013 is Boilerplate stripper - http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013#GoogleSummerofCode2013-Boilerplatestripper I am planning to use regular expressions for this, obviously script will be in python :) Even though the project can be finished before the end of GSoC period. I would like to add *some more features* to it. 1. I am interested in adding Natural Language Processing techniques which will check for the word (attach, attached, attachment(s)) in the mail and will cross check if there is any file attached to it. If not, it will come up with a dialogue box saying, "*you forgot to attach a file*" 2. Apart from this, I would like to work on censorship as well which will work as a *dirty word filter*. and many more *Artificial Intelligence tools* in mailman. About me : I have *4 years* of programming experience in *Python*, using mailman extensively. I was a speaker at *PyCon India 2012 * http://in.pycon.org/2012/funnel/pyconindia2012/36-artificial-intelligence-using-python. I would like to thank PSF for sponsoring me to *PyCon US 2013* where I took a lightning talk. I'm hoping to work with Mailman in this year's GSoC as I already have a experience of AI & NLP and this would serve as an great opportunity and chance to work with Mailman formally and prove my skills. I can be reached via email : *sharma(dot)adwait(at)gmail(dot)com* Looking forward to your reply. Kind regards, Adwait -- *Adwait Sharma ~NeO~* Learner | Geek | Hacker | Blogger | Open Source Software Developer Email: sharma.adwait at gmail.com Site: http://www.ThePirado.com [image: Blog RSS] Contact Phone: +91-9164665550 From terri at zone12.com Thu Apr 18 02:21:17 2013 From: terri at zone12.com (Terri Oda) Date: Wed, 17 Apr 2013 18:21:17 -0600 Subject: [Mailman-Developers] Architecture for extra profile info Message-ID: <516F3C7D.1070509@zone12.com> Background for folk new to this discussion: Currently, all user information is stored in Mailman core, but it's minimal: a real name, a set of email addresses, subscription info, and preferences. Barry suggests that it should stay minimal: only the things Mailman needs to know to correctly deliver mail (which actually doesn't include "real name" but let's leave that as a legacy item for the moment) It's pretty likely that future features of Mailman will want to attach extra information to users. Some of it will be social-y stuff like user icons for HyperKitty to display in the archives. Other things include metrics like "when did this person last post to the list?" or "how many posts have they made over the lifetime of this list?" One thing I know of is that Systers requires a short essay for all new subscribers, explaining why they want to join the list. (And they're considering porting this feature to Postorius, which means we potentially want an answer to "where will the extra profile data get stored?" before their students start coding.) So... I think we've sort of agreed that it would be best if whatever we built just had a rest interface and hyperkitty/postorius/whatever would talk to it through there, and could share data that way, but we need a simple prototype that folk (particularly students) will be able to start using, and there's still some internal architecture decisions that need to be made. Does anyone have time to build such a thing or write up some short architectural documents so a student could build such a thing in relatively short order? It doesn't have to be the perfect final design, but we probably need a basic starter api for adding, accessing, editing and possibly even removing profile data. Thoughts? Terri From avikpal.me at gmail.com Thu Apr 18 06:03:46 2013 From: avikpal.me at gmail.com (Avik Pal) Date: Thu, 18 Apr 2013 09:33:46 +0530 Subject: [Mailman-Developers] GSOC 2013 project discussion In-Reply-To: <87li8h2ik4.fsf@uwakimon.sk.tsukuba.ac.jp> References: <04CDCC46-7630-4EE7-B8A1-91E2F1FE989E@DATAPLEX.NET> <7E46E575-E5D4-4A68-A0D6-F49C75D41BE3@DATAPLEX.NET> <87li8h2ik4.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: thanks a lot Stephen for all the suggestions :) Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 17 April 2013 22:36, Stephen J. Turnbull wrote: > Avik Pal writes: > > > Meanwhile It would be much appreciated if someone can direct me to > > an labeled dataset available on line. > > By "labelled" you mean pre-classified into spam vs ham? I see you > already found one, but you could also check the SpamBayes and > SpamAssassin distributions. > > > Here I have a suggestion, after submitting, whenever an email is > > classified as Spam, we store it in a separate archive and after the > > end of the day send them a mail telling "this is the digest for all > > the mails that Mailman thinks to be Spam" the subscriber may go > > there and can view them and also can mark them as not Spam, > > I suggest that you present this as an option for users who want to > tune the filters, and as something that can be used pre-release to > develop the initial parameters for the distributed classifier. > Although Bayesian classifiers do offer the option to train or tune > your personal classifier on a local corpus, most users just stick with > the distribution parameters plus self-training. It's pretty effective > (surprisingly so to me). I guess the logic is that spammers aren't > terribly creative. > > > Emails which stays as Spam will be dropped after a month > > Let's think carefully about that. Everybody deletes the spam; that's > why you started by asking for a labelled dataset, because nobody keeps > one around. Somebody really ought to do the public service of > collecting a corpus. Of course, if you do arrange to keep it around, > it's going to need to be an option that sites and list owners can > disable. > > From varunsharmalive at gmail.com Thu Apr 18 06:36:32 2013 From: varunsharmalive at gmail.com (varun sharma) Date: Thu, 18 Apr 2013 10:06:32 +0530 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <516F3C7D.1070509@zone12.com> References: <516F3C7D.1070509@zone12.com> Message-ID: +1 for this, i am desperately waiting for someone to do do this :) On Thu, Apr 18, 2013 at 5:51 AM, Terri Oda wrote: > Background for folk new to this discussion: > > Currently, all user information is stored in Mailman core, but it's > minimal: a real name, a set of email addresses, subscription info, and > preferences. Barry suggests that it should stay minimal: only the things > Mailman needs to know to correctly deliver mail (which actually doesn't > include "real name" but let's leave that as a legacy item for the moment) > > It's pretty likely that future features of Mailman will want to attach > extra information to users. Some of it will be social-y stuff like user > icons for HyperKitty to display in the archives. Other things include > metrics like "when did this person last post to the list?" or "how many > posts have they made over the lifetime of this list?" One thing I know of > is that Systers requires a short essay for all new subscribers, explaining > why they want to join the list. (And they're considering porting this > feature to Postorius, which means we potentially want an answer to "where > will the extra profile data get stored?" before their students start > coding.) > > So... > > I think we've sort of agreed that it would be best if whatever we built > just had a rest interface and hyperkitty/postorius/whatever would talk to > it through there, and could share data that way, but we need a simple > prototype that folk (particularly students) will be able to start using, > and there's still some internal architecture decisions that need to be made. > > Does anyone have time to build such a thing or write up some short > architectural documents so a student could build such a thing in relatively > short order? It doesn't have to be the perfect final design, but we > probably need a basic starter api for adding, accessing, editing and > possibly even removing profile data. > > Thoughts? > > Terri > > > ______________________________**_________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/**mailman/listinfo/mailman-**developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/** > mailman-developers%40python.**org/ > Unsubscribe: http://mail.python.org/**mailman/options/mailman-** > developers/varunsharmalive%**40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From flo.fuchs at gmail.com Thu Apr 18 08:19:57 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Thu, 18 Apr 2013 08:19:57 +0200 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <516F3C7D.1070509@zone12.com> References: <516F3C7D.1070509@zone12.com> Message-ID: Hi, some first thoughts on it: 1) It should be self-contained. Meaning: It should not depend on any mailman/mailman.client/postorius/hyperkitty packages. 2) Like the core, it should implement a Python-based webserver. It doesn't need to run on Ports 80/443, so we don't have to care about one of the popular web servers already listening to those ports. Also, we definitely don't want admins to have to follow another "how to setup mailman.users with Apache/mod_wsgi" howto. It should just run if someone says "start" (Maybe it can even hook into $ bin/mailman start? I know, that contradicts item 1. ...). 3) It doesn't need Django. Since it will not deliver any HTML (except an oAuth login form -- see 5.) and it doesn't need to be integrated into any existing web site, we can choose a more light-weight framework. 4) Adding new content types for user records should be easy, but clearly defined. We don't know what information applications need to store. Icons, essays, avatars, IRC handles, Twiter names, ... So we might think about using a schema-less database, but: We don't want to make it possible to just manipulate the result JSON and POST it back to the resource, possible deleting things other apps need. So adding new information types should be a separate process. 5) It should implement an oAuth provider. This could be used for API authenticaion and to log into Postorius/Hyperkitty (even on other servers. Hint: Reputation!) We won't need this for a first prototype though. For now we're probably fine with 6) 6) Like the core it should be accessible with BasicAuth from localhost. Ideally, in the future, it should be accessible both via BasicAuth from localhost and via oAuth from the outside world... Please correct me where I'm being stupid! Florian 2013/4/18 Terri Oda : > Background for folk new to this discussion: > > Currently, all user information is stored in Mailman core, but it's minimal: > a real name, a set of email addresses, subscription info, and preferences. > Barry suggests that it should stay minimal: only the things Mailman needs to > know to correctly deliver mail (which actually doesn't include "real name" > but let's leave that as a legacy item for the moment) > > It's pretty likely that future features of Mailman will want to attach extra > information to users. Some of it will be social-y stuff like user icons for > HyperKitty to display in the archives. Other things include metrics like > "when did this person last post to the list?" or "how many posts have they > made over the lifetime of this list?" One thing I know of is that Systers > requires a short essay for all new subscribers, explaining why they want to > join the list. (And they're considering porting this feature to Postorius, > which means we potentially want an answer to "where will the extra profile > data get stored?" before their students start coding.) > > So... > > I think we've sort of agreed that it would be best if whatever we built just > had a rest interface and hyperkitty/postorius/whatever would talk to it > through there, and could share data that way, but we need a simple prototype > that folk (particularly students) will be able to start using, and there's > still some internal architecture decisions that need to be made. > > Does anyone have time to build such a thing or write up some short > architectural documents so a student could build such a thing in relatively > short order? It doesn't have to be the perfect final design, but we > probably need a basic starter api for adding, accessing, editing and > possibly even removing profile data. > > Thoughts? > > Terri > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/f%40state-of-mind.de > > Security Policy: http://wiki.list.org/x/QIA9 From xuwang at gmail.com Thu Apr 18 09:39:31 2013 From: xuwang at gmail.com (Xu Wang) Date: Thu, 18 Apr 2013 00:39:31 -0700 Subject: [Mailman-Developers] Build a mailman3 development virtual machine with Vagrant and Chef Message-ID: Hi there, Although mailman3 installation is well documented, but putting everything together is still not a trivial job. To ease the task, I made a mailman3 chef cookbook and vagrant file: https://github.com/xuwang/mailman3-vbox If you already had virtualbox/vagrant installed on your mac or pc, build a working mailman3 vm should only take a few minutes. I'm sure there will be things I missed or got it wrong, but it just a starter. Hope this helps, Xu Wang From rahul.nbg at gmail.com Thu Apr 18 10:26:20 2013 From: rahul.nbg at gmail.com (Rahul Gaur) Date: Thu, 18 Apr 2013 13:56:20 +0530 Subject: [Mailman-Developers] GSOC 2013 : Authenticated REST-API in Postorius/Django In-Reply-To: References: <05BC1D67-31B0-41AA-82C2-8A47AB7AF7BD@NFSNet.org> Message-ID: Hi , Sorry for late reply, there's lot of college work on hand. > Thanks for creating the two examples! I took a quick look at them and > it looks like both frameworks are pretty easy to use if a resource is > based on a typical Django model. The Mailman resources OTOH are not > retrieved from a database but from another API -- Mailman's core REST > API that is only served locally (postorius uses the mailman.client > library to access that data). Which of the two frameworks would you > consider more flexible if you want to get your data from anything else > but a Django DB model (I *think* Tastypie has different classes for > model resources and everythingelse-resources)? > I looked into Tastypie and yes the model resources is just more tightly wrapped class which focus on django ORM and then there's this resources class which could be used for non orm models. I also went through a similar example on Tastypie, for using nosql db for resources. Here is a link to the example I found on Tastypie docs [0] . They are linking Riak for data resources, and Riak client used here communicates with core Risk for data sources with the APIs. So, I believe similar thing could be achieved with our project. > > *$ curl http://127.0.0.1:8080/formpost/ -H 'Accept:text/html' > > > > The Django-rest-frameworks API's returns web browsable representation of > > the data by default :) > > Sounds nice! :-) > > > So , the confusion I am facing here is that the Postorius is already mature > > and stable app and while playing with the two frameworks for API's what I > > figured out is that if I opt for django-rest-framework , I would need to > > write new Views ? > > But if I go for TastyPie , I won't have to touch the existing Views and > > write authenticated API's. > > > > Hence to implement authenticated API's , which one would be better ? > > Hard to say - judging by how they're used in the examples, both look > good. But there are some more things you might want to consider before > choosing one: > > Authentication methods > Which auth methods are provided out of the box by the frameworks? And > which one(s) would you like to support (OAuth(2), BasicAuth over SSL, > ...)? I guess choosing and implementing authentication could become > one of the major sub-tasks of that project. Talking about authentication, django-restframework has a wrapper for OAuth2 out of the box, but I will have to check how to use non form data sources. I know there must be a solution for that, but then again in django-rest framework REST APIs are handled from the views.py. I don't really want to fiddle with existing views.py in Postorius. Maybe we could discuss more about this , accounting what would be better in the long run. Personally I would be sticking to Tastypie for authentication as well. If we plan to support Oauth2 , it could also be achieved with Tastypie. Couple of days back I saw some similar examples on github. I would try to provide some examples on the topics to support my proposal. Hopefully I would be able to do that by this weekend . > > Support for Django 1.5 and Python3: Isn't the support for Python 3 just experimental in django 1.5 .? And neither of the two third party frameworks support python 3 fully in their current release. Django rest framework has some amount of experimental support but not sure about Tastypie. > At some point in the not so fare future mailman and postorius will be > ported to Python3. Django 1.5 already supports Python3, so it would be > a bonus point if the rest framework supports it, too (or if their > developers are making efforts to do so...). I know the support for python 3 is essential to the project, but as of now sticking I would prefer to sticking to Python 2.7.x for now. I would be having a month in hand before official GSOC CODING period starts. During this time I would be getting stared with the project so that there would be some substantial code ready before official coding time starts. I guess its roughly two months from now , so if there's a release for these frameworks with python 3 support . Porting it wouldn't be that much of an issue. > Another idea (not sure if it's a good one): Maybe also take a closer > look at Django's class-based views. They are capable of routing > requests to the same URI to different dispatch methods, depending on > the HTTP method. So that might be also be an interesting option. > Yes, this is similar to what class based views in django-restframework offers. To be honest, I don't really have much idea about django class based views.I am looking into this and I would let you know. Also I have couple of other thinks I would like to discus. Maybe we could discuss more about this over the irc. Regards! Rahul From stephen at xemacs.org Thu Apr 18 10:28:29 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 18 Apr 2013 17:28:29 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> Message-ID: <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> Main comment: Sounds like LDAP to me. Florian Fuchs writes: > 5) It should implement an oAuth provider. I don't see this. Mailman is an auth consumer. The only people Mailman can provide auth for are the site admins. Everybody else is more or less untrustworthy. I can see that there are applications where it would be useful to have an auth provider bundled with Mailman, but I think implementing it is somebody else's job. > This could be used for API authenticaion and to log into > Postorius/Hyperkitty I think generic auth provider is overkill for these purposes, and a trap for anybody who thinks we know enough about crypto/security to do this stuff well. From flo.fuchs at gmail.com Thu Apr 18 11:27:22 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Thu, 18 Apr 2013 11:27:22 +0200 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: 2013/4/18 Stephen J. Turnbull : > Florian Fuchs writes: > > > 5) It should implement an oAuth provider. > > I don't see this. Mailman is an auth consumer. The only people > Mailman can provide auth for are the site admins. Everybody else is > more or less untrustworthy. > > I can see that there are applications where it would be useful to have > an auth provider bundled with Mailman, but I think implementing it is > somebody else's job. > > > This could be used for API authenticaion and to log into > > Postorius/Hyperkitty > > I think generic auth provider is overkill for these purposes, and a > trap for anybody who thinks we know enough about crypto/security to do > this stuff well. I agree it's probably not easy. And, yes, maybe we need someone to help us with that. But maybe we can take a moment to think about the usefulness of such a feature and the possibilities this might open up, rather than dismissing the use of a certain technology right off the bat. If we're unsure we can implement this in a secure way, we can still say no to this. Also, who says such a feature would be enabled by default? We can add it as an "experts only" thing and leave it up to the admins to make sure they use it in a secure environment. I remember several discussions during PyCon(s) and on IRC where scenarios of different mailman instances talking to each other came up. Of course that doesn't mean implementing a generic oAuth provider is the only answer to this. If there are better and easier solutions, fine. Luckily going beyond localhost isn't something we need for the prototype that Terri suggested to be built for GSoC. Florian From rkw at DATAPLEX.NET Thu Apr 18 15:14:45 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Thu, 18 Apr 2013 08:14:45 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Whoa! Perhaps I don't understand oAuth. I thought that oAuth (and persona, kerberos, etc.) were protocols whereby one system (the provider) furnishes credentials for a second system (the client) to some third system (the consumer). By configuration, the consumer trusts that the provider has verified the client's identity and furnished appropriate credentials. Thus, when the client presents credentials in an interaction with the consumer, the consumer provides services on the basis of the credentials. If we assume that we distribute the MM implementation to include more than the two (core and web UI) systems by having, for example, a user manager, there might be an argument for passing around such credentials. However, the design that has been followed thus far does not have the client communicating directly with the consumer system. Instead, the web UI interacts as an agent for the client. In this model, we have implicitly trusted the agent to properly represent the client and also screen client requests in accordance with system policy. Thus, although we need some level of authentication of the agent, there is no need for third party credentials such as those implemented in oAuth. A connection on "localhost" is a form of credential. Trusting that the OS design restricts access to the connection, and trusting the applications running on that host provides a level of identification and trust. There is no reason why alternate channels cannot be substituted as long as a means of identification (such as shared secrets) is utilized. In those cases, the security of the communication channel and the trustworthiness of the agent system need to be considered. However, in a logical sense, the interaction is the same as one using the localhost channel. On Apr 18, 2013, at 4:27 AM, Florian Fuchs wrote: > 2013/4/18 Stephen J. Turnbull : >> Florian Fuchs writes: >> >>> 5) It should implement an oAuth provider. >> >> I don't see this. Mailman is an auth consumer. The only people >> Mailman can provide auth for are the site admins. Everybody else is >> more or less untrustworthy. >> >> I can see that there are applications where it would be useful to have >> an auth provider bundled with Mailman, but I think implementing it is >> somebody else's job. >> >>> This could be used for API authentication and to log into >>> Postorius/Hyperkitty >> >> I think generic auth provider is overkill for these purposes, and a >> trap for anybody who thinks we know enough about crypto/security to do >> this stuff well. > > I agree it's probably not easy. And, yes, maybe we need someone to > help us with that. > > But maybe we can take a moment to think about the usefulness of such a > feature and the possibilities this might open up, rather than > dismissing the use of a certain technology right off the bat. If we're > unsure we can implement this in a secure way, we can still say no to > this. Also, who says such a feature would be enabled by default? We > can add it as an "experts only" thing and leave it up to the admins to > make sure they use it in a secure environment. > > I remember several discussions during PyCon(s) and on IRC where > scenarios of different mailman instances talking to each other came > up. Of course that doesn't mean implementing a generic oAuth provider > is the only answer to this. If there are better and easier solutions, > fine. > > Luckily going beyond localhost isn't something we need for the > prototype that Terri suggested to be built for GSoC. > > Florian > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From richard at NFSNet.org Thu Apr 18 16:03:06 2013 From: richard at NFSNet.org (Richard Wackerbarth) Date: Thu, 18 Apr 2013 09:03:06 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> Message-ID: <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> On Apr 18, 2013, at 1:19 AM, Florian Fuchs wrote: > Hi, > > some first thoughts on it: > > 1) It should be self-contained. > Meaning: It should not depend on any > mailman/mailman.client/postorius/hyperkitty packages. Agreed > 2) Like the core, it should implement a Python-based webserver. > It doesn't need to run on Ports 80/443, so we don't have to care about > one of the popular web servers already listening to those ports. > Also, we definitely don't want admins to have to follow another "how > to setup mailman.users with Apache/mod_wsgi" howto. > It should just run if someone says "start" (Maybe it can even hook > into $ bin/mailman start? I know, that contradicts item 1. ...). Where it runs is something that should be easily configurable as a part of installation. To simplify installation, I would advocate that all of the MM subsystem interfaces become self-documentating. By doing so, much of the configuration could be done through a "discovery" mechanism. That would provide an easy way to have modular plug-in extensions. It would also provide a way for Postorius to automatically adapt to the addition of configuration settings or user fields. > 3) It doesn't need Django. > Since it will not deliver any HTML (except an oAuth login form -- see > 5.) and it doesn't need to be integrated into any existing web site, > we can choose a more light-weight framework. Here I take exception. Dismissing Django is a restriction that unnecessarily affects the ease of implementation and, in the common case, complicates the integration of the functionality. Although it could be implemented without Django, it could also be implemented as a Django "app". An instance of a django server can then serve the functionality. As an alternative, where appropriate, this "app" would directly "drop in" to an instance of Postorius or an enterprise website. One of the advantages of Django is that it can be used as a rapid prototyping mechanism. Simplified interfaces to the data are "free" and more elaborate ones can be added in an incremental fashion. Also, rather than writing custom modules for things such as authentication and REST interfaces, there is the large community of third-party extensions which readily integrate to provide that functionality. > 4) Adding new content types for user records should be easy, but > clearly defined. > We don't know what information applications need to store. Icons, > essays, avatars, IRC handles, Twiter names, ... > So we might think about using a schema-less database, but: We don't > want to make it possible to just manipulate the result JSON and POST > it back to the resource, possible deleting things other apps need. So > adding new information types should be a separate process. > > 5) It should implement an oAuth provider. > This could be used for API authenticaion and to log into > Postorius/Hyperkitty (even on other servers. Hint: Reputation!) > We won't need this for a first prototype though. For now we're > probably fine with 6) > > 6) Like the core it should be accessible with BasicAuth from localhost. > Ideally, in the future, it should be accessible both via BasicAuth > from localhost and via oAuth from the outside world... > > Please correct me where I'm being stupid! > > Florian > 2013/4/18 Terri Oda : >> Background for folk new to this discussion: >> >> Currently, all user information is stored in Mailman core, but it's minimal: >> a real name, a set of email addresses, subscription info, and preferences. >> Barry suggests that it should stay minimal: only the things Mailman needs to >> know to correctly deliver mail (which actually doesn't include "real name" >> but let's leave that as a legacy item for the moment) I would advocate that this "User" module make it appear as if stores the entire "record" for the user. In the implementation, it could actually store parts of the user information in multiple databases (one of which could be the MM-core). This would also allow the option of having the MM-core become a client of this User module, just as it now relies on an external message store. >> It's pretty likely that future features of Mailman will want to attach extra >> information to users. Some of it will be social-y stuff like user icons for >> HyperKitty to display in the archives. Other things include metrics like >> "when did this person last post to the list?" or "how many posts have they >> made over the lifetime of this list?" One thing I know of is that Systers >> requires a short essay for all new subscribers, explaining why they want to >> join the list. (And they're considering porting this feature to Postorius, >> which means we potentially want an answer to "where will the extra profile >> data get stored?" before their students start coding.) >> >> So... >> >> I think we've sort of agreed that it would be best if whatever we built just >> had a rest interface and hyperkitty/postorius/whatever would talk to it >> through there, and could share data that way, but we need a simple prototype >> that folk (particularly students) will be able to start using, and there's >> still some internal architecture decisions that need to be made. >> >> Does anyone have time to build such a thing or write up some short >> architectural documents so a student could build such a thing in relatively >> short order? It doesn't have to be the perfect final design, but we >> probably need a basic starter api for adding, accessing, editing and >> possibly even removing profile data. >> >> Thoughts? >> >> Terri From iampratiksarkar at gmail.com Thu Apr 18 17:37:11 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Thu, 18 Apr 2013 21:07:11 +0530 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <516EC670.5080706@zone12.com> References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> <878v4i4z52.fsf@uwakimon.sk.tsukuba.ac.jp> <87sj2p3hhv.fsf@uwakimon.sk.tsukuba.ac.jp> <516EC670.5080706@zone12.com> Message-ID: Thank you Terri and Stephen for your replies and explaining the whole procedure of selection. Coming back to the project,do we need to write patches for the SpamAssassin or SpamBayes(which are integrated in Mailman) or do we need to start over and make a whole new filter and start reinventing the wheels? And somewhere I read that the filtering of messages posted by non-subscribed members are done manually.Can it be changed to be filtered automatically somehow? Regards, Pratik From stephen at xemacs.org Thu Apr 18 18:07:59 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 01:07:59 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <878v4f3jq8.fsf@uwakimon.sk.tsukuba.ac.jp> Florian Fuchs writes: > But maybe we can take a moment to think about the usefulness of such a > feature and the possibilities this might open up, rather than > dismissing the use of a certain technology right off the bat. I'm not dismissing the use; I'm saying an authentication provider is out of scope for Mailman, for several reasons. An auth client is something we clearly need. The mail auth we use (plain text passwords) is clearly weak, and we need to improve that. Web auth is probably OK for most sites if we use HTTP Basic Auth over HTTPS. OAuth (or "social auth" or Persona) is clearly a major convenience for users, and probably significantly increases security all by itself (no need for Post-Its with the Mailman password you almost never use on your monitor bezel). Both HTTPS+BasicAuth and OAuth have the advantage that Mailman can delegate the handling of transmission of secrets over the Internet to professions. Given that HTTPS+BasicAuth is probably good enough for most of our users, and most of the rest will want to use Google/Yahoo/whatever rather than YAP, where's the need for a provider? And I *am* thinking about the possibilities implementing a provider opens up. Namely, the possibility that we'll distribute buggy code. :-P > I remember several discussions during PyCon(s) and on IRC where > scenarios of different mailman instances talking to each other came > up. Of course that doesn't mean implementing a generic oAuth provider > is the only answer to this. If there are better and easier solutions, > fine. The OAuth2 protocol spec (RFC 6749) makes obscure reference to "scope" and tokens acceptable to multiple resource servers. But I don't think sharing authorizations is something that a provider deals with. From stephen at xemacs.org Thu Apr 18 18:20:55 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 01:20:55 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87sj2s41h6.fsf@uwakimon.sk.tsukuba.ac.jp> <878v4i4z52.fsf@uwakimon.sk.tsukuba.ac.jp> <87sj2p3hhv.fsf@uwakimon.sk.tsukuba.ac.jp> <516EC670.5080706@zone12.com> Message-ID: <877gjz3j4o.fsf@uwakimon.sk.tsukuba.ac.jp> Pratik Sarkar writes: > Coming back to the project,do we need to write patches for the > SpamAssassin or SpamBayes(which are integrated in Mailman) No. You write a Handler which delegates to those external packages. A Handler is a sort of plug-in. > or do we need to start over and make a whole new filter and start > reinventing the wheels? Don't do that unless you think you can write a filter that does a significantly better job on some dimension than the existing ones. > And somewhere I read that the filtering of messages posted by > non-subscribed members are done manually. That's incorrect. Mailman's mail processing is architected as a pipeline of Handlers. Each Handler can act as a filter, or edit the message, or perform a side effect (delivering the mail to subscribers or saving it in an archive), then passes the message on to the next Handler (unless it raises an exception). Filters have a selection of options: throw away the message silently (DISCARD), return to sender (REJECT), quarantine (HOLD), or pass on for further processing. (The words in all uppercase are actually exception names.) One of the filters checks whether the mail is from a subscriber, but it is not otherwise special, and in principle both subscribers and non-subscribers may be subject to various filters. From terri at zone12.com Thu Apr 18 18:26:25 2013 From: terri at zone12.com (Terri Oda) Date: Thu, 18 Apr 2013 10:26:25 -0600 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> Message-ID: <51701EB1.1080804@zone12.com> On 13-04-18 8:03 AM, Richard Wackerbarth wrote: > On Apr 18, 2013, at 1:19 AM, Florian Fuchs wrote: > >> 1) It should be self-contained. >> Meaning: It should not depend on any >> mailman/mailman.client/postorius/hyperkitty packages. > Agreed I agree about not needing postorious/hyperkitty, but I think a case could be made that interdependence with mailman core and mailman client might make sense. > I would advocate that this "User" module make it appear as if stores the entire "record" for the user. > In the implementation, it could actually store parts of the user information in multiple databases (one of which could be the MM-core). > > This would also allow the option of having the MM-core become a client of this User module, just as it now relies on an external message store. Two options occur to me: 1. The user module is what mm-core uses for all user stuff. I thinks case, case we have to be *much* more conservative about dependencies. I think Django would be right out as a dependency for Mailman core, for example. Plus, we're going to have to care a lot more about speed and all. 2. The user module reads from mm-core and augments it. This gives us the ability to supplement mm-core without impeding speed. We discussed possibly filling in the blanks with respect to things like the user preferences (which are currently set by membership, by user and by email address... but a lot of those return an empty set when queried if nothing is set directly there...) so this is maybe something we already want. Conceptually, #1 is probably easier because everything will be in one place, but if we do #2 right, we can make it just as conceptually easy for HyperKitty/Postorius/etc. without impeding Barry's core dev at all. That does mean in case 2 that Mailman Extraneous User Stuff is going to depend on Mailman Core, though. My preference is #2: a) It doesn't add any dependencies to Mailman core. b) It doesn't require big changes in Mailman Core. Given that core is pretty much ready to release, now would be a bad time for changes, and I'm just not sure we can justify that amount of work for the types of features that will be built on the extraneous user stuff. c) It will be much easier to rip this out and replace it when we better understand our actual needs. (e.g. Right now, I think a case could be made that a quick mock-up in django would be fine, but I suspect that requiring django for some potential applications would border on ridiculous) We're probably going to be running around with a bit of a hack job for the user database in the near future (either done by a student who needs it in a hurry or done by one of the core devs to support a student who needs it in a hurry) so while I don't like to design on the assumption it's going to go wrong, I think in this case planning for a redesign might be prudent because it's pretty clear we don't actually know all our requirements. Terri From stephen at xemacs.org Thu Apr 18 18:42:05 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 01:42:05 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <8761zj3i5e.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > Whoa! Perhaps I don't understand oAuth. I thought that oAuth (and > persona, kerberos, etc.) were protocols whereby one system (the > provider) furnishes credentials for a second system (the client) to > some third system (the consumer). That's correct. > If we assume that we distribute the MM implementation to include > more than the two (core and web UI) systems by having, for example, > a user manager, there might be an argument for passing around such > credentials. But the does provide a user manager, and the "extra profile info" is in fact intended to be a user manager external to the core. > Thus, although we need some level of authentication of the agent, > there is no need for third party credentials such as those > implemented in oAuth. The point is that in many cases we would like to dispense with the agent authentication process altogether, and let a third party manage that. This is perfectly acceptable in the case of open subscription lists where we simply want to ensure that only the subscriber can change their subscriptions. For example, a person subscribing a Gmail account to use that account's credentials rather than creating new owns inside of Mailman -- which we trust only because the person demonstrates in a roundabout way that they can access that mailbox. OAuth allows us to make that check directly in real time. I suppose that what Florian is thinking is that some owners want *closed* subscription processes, and therefore want to control the authentication process themselves. I agree that that is a valid and likely (if unusual) use case. I just think it's better for such users to go find a provider implementation themselves, rather than offer them something that I know *I* can't properly design or review, and haven't seen credentials from anyone else on the team that they can do it, either. > There is no reason why alternate channels [to a connection from > localhost authorized by the OS] cannot be substituted as long as a > means of identification (such as shared secrets) is utilized. Sure, but didn't you notice the elephant in the room as you swept it under the rug? The implementation of "alternate channels" matters *a lot*, and it's not trivial. From rkw at DATAPLEX.NET Thu Apr 18 18:54:07 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Thu, 18 Apr 2013 11:54:07 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <8761zj3i5e.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zj3i5e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Apr 18, 2013, at 11:42 AM, "Stephen J. Turnbull" wrote: > Richard Wackerbarth writes: >> There is no reason why alternate channels [to a connection from >> localhost authorized by the OS] cannot be substituted as long as a >> means of identification (such as shared secrets) is utilized. > > Sure, but didn't you notice the elephant in the room as you swept it > under the rug? The implementation of "alternate channels" matters *a > lot*, and it's not trivial. Just because something is important or non-trivial to implement properly does not imply that it is difficult for us to utilize it. Rather than developing our own, we can, and should, leverage the efforts of "the professionals" and use the tools that they provide (such as https and oAuth, etc.). Certainly, the proper administration of each, and every, host is an essential element to prevent access "on the coat tails" of the trusted agents. But that also applies to the "localhost" implementation. From follybeachris at gmail.com Thu Apr 18 19:10:28 2013 From: follybeachris at gmail.com (Chris Cargile) Date: Thu, 18 Apr 2013 13:10:28 -0400 Subject: [Mailman-Developers] Setting up a VM. In-Reply-To: <51698325.5090808@zone12.com> References: <51698325.5090808@zone12.com> Message-ID: On Sat, Apr 13, 2013 at 12:09 PM, Terri Oda wrote: > > go ahead and get a VM set up; > > Chris, who was working on such a thing before, seems to have gotten stuck > somewhere.... You probably also want to talk to other incoming students > who've already set up their dev environment since they'll have any issues > they encountered fresh in their minds! > I seem to get stuck completing the buildout process..I am not able to get mailman core configured/running correctly (so far),as after running /bin/test -vv, the tests don't succeed, showing that I get errors from tests: SMTPLayer, Layer: mailman.testing.layers.RESTLayer.. The cause in both cases seems to stem from failing at the SMTPLayer, where it shows: [Error98] Address already in use. "mailman conf" shows that smtp port setting is:25. I tried "netstat -plntu | grep :25" to see if there were any processes using that port and there are not any. I think I found info. sometime back regarding the error message but have not resolved it, so I can get postorius to run in the browser but am stuck at mailman core part. > I'm cc'ing Chris so he can share any advice > > . RE: sharing a development image, Xu Wang posted on that today, which I look forward to exploring... thanks (and for bearing with me during my env't set up) I did research into sharing a development machine-image using Amazon EC2,S3 API/buckets, but left that alone for now due to possible cost factors as well as potentially better alternatives - though, I'd be glad to work together with anyone interested in an EC2 dev-env't setup. Send me an email or IRC :) From stephen at xemacs.org Thu Apr 18 19:21:53 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 02:21:53 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zj3i5e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <8738un3gb2.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > On Apr 18, 2013, at 11:42 AM, "Stephen J. Turnbull" wrote: > > > Richard Wackerbarth writes: > > >> There is no reason why alternate channels [to a connection from > >> localhost authorized by the OS] cannot be substituted as long as a > >> means of identification (such as shared secrets) is utilized. > > > > Sure, but didn't you notice the elephant in the room as you swept it > > under the rug? The implementation of "alternate channels" matters *a > > lot*, and it's not trivial. > > Just because something is important or non-trivial to implement > properly does not imply that it is difficult for us to utilize it. > Rather than developing our own, we can, and should, leverage the > efforts of "the professionals" and use the tools that they provide > (such as https and oAuth, etc.). > > Certainly, the proper administration of each, and every, host is an > essential element to prevent access "on the coat tails" of the > trusted agents. But that also applies to the "localhost" > implementation. I don't understand what you're advocating, your comments are way too general. My position is that secure authentication and authorization is a hard problem, and we should avoid doing that as much as possible (partly because as far as I know none of us are experts). No channels that few sites will use, ditto OAuth providers. Concentrate on a couple of channels with specific, well-understood, universal (or at least very common) use cases. The channels I have in mind are (1) shell access, (2) Basic Auth over HTTPS for people who need to control access fairly tightly, and (3) OAuth and/or Persona clients allowing authentication by any of a number of public providers for user (especially subscriber) convenience. I'm not wedded to any of those (except (1), for obvious reasons), but I don't think it's a good idea to extend the list if we can avoid it. From rkw at DATAPLEX.NET Thu Apr 18 19:22:14 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Thu, 18 Apr 2013 12:22:14 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <8761zj3i5e.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zj3i5e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <2260206D-E7A7-4AFB-A645-1A0CFDCEEB06@DATAPLEX.NET> On Apr 18, 2013, at 11:42 AM, "Stephen J. Turnbull" wrote: > Richard Wackerbarth writes: > >> Whoa! Perhaps I don't understand oAuth. I thought that oAuth (and >> persona, kerberos, etc.) were protocols whereby one system (the >> provider) furnishes credentials for a second system (the client) to >> some third system (the consumer). > > That's correct. > >> If we assume that we distribute the MM implementation to include >> more than the two (core and web UI) systems by having, for example, >> a user manager, there might be an argument for passing around such >> credentials. > > But the does provide a user manager, and the "extra profile info" is > in fact intended to be a user manager external to the core. > >> Thus, although we need some level of authentication of the agent, >> there is no need for third party credentials such as those >> implemented in oAuth. > > The point is that in many cases we would like to dispense with the > agent authentication process altogether, and let a third party manage > that. This is perfectly acceptable in the case of open subscription > lists where we simply want to ensure that only the subscriber can > change their subscriptions. For example, a person subscribing a Gmail > account to use that account's credentials rather than creating new > owns inside of Mailman -- which we trust only because the person > demonstrates in a roundabout way that they can access that mailbox. > OAuth allows us to make that check directly in real time. I have no problem with, and actually encourage, that we act as a consumer of oAuth credentials. However, the issue here is whether we should be provider of oAuth credentials (which might then be presented to some outside, totally unrelated, entity. From rkw at DATAPLEX.NET Thu Apr 18 19:27:27 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Thu, 18 Apr 2013 12:27:27 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <8738un3gb2.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zj3i5e.fsf@uwakimon.sk.tsukuba.ac.jp> <8738un3gb2.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4D2BFD37-424D-45F8-A0CC-E9B850BCC8A2@DATAPLEX.NET> On Apr 18, 2013, at 12:21 PM, "Stephen J. Turnbull" wrote: > Richard Wackerbarth writes: >> On Apr 18, 2013, at 11:42 AM, "Stephen J. Turnbull" wrote: >> >>> Richard Wackerbarth writes: >> >>>> There is no reason why alternate channels [to a connection from >>>> localhost authorized by the OS] cannot be substituted as long as a >>>> means of identification (such as shared secrets) is utilized. >>> >>> Sure, but didn't you notice the elephant in the room as you swept it >>> under the rug? The implementation of "alternate channels" matters *a >>> lot*, and it's not trivial. >> >> Just because something is important or non-trivial to implement >> properly does not imply that it is difficult for us to utilize it. >> Rather than developing our own, we can, and should, leverage the >> efforts of "the professionals" and use the tools that they provide >> (such as https and oAuth, etc.). >> >> Certainly, the proper administration of each, and every, host is an >> essential element to prevent access "on the coat tails" of the >> trusted agents. But that also applies to the "localhost" >> implementation. > > I don't understand what you're advocating, your comments are way too > general. > > My position is that secure authentication and authorization is a hard > problem, and we should avoid doing that as much as possible (partly > because as far as I know none of us are experts). No channels that > few sites will use, ditto OAuth providers. Concentrate on a couple of > channels with specific, well-understood, universal (or at least very > common) use cases. > > The channels I have in mind are (1) shell access, (2) Basic Auth over > HTTPS for people who need to control access fairly tightly, and (3) > OAuth and/or Persona clients allowing authentication by any of a > number of public providers for user (especially subscriber) > convenience. I'm not wedded to any of those (except (1), for obvious > reasons), but I don't think it's a good idea to extend the list if we > can avoid it. Perhaps I didn't understand you. I thought that you were advocating the omission of any channels other than "shell" and "localhost". I was trying to point out that HTTPS, oAuth, etc. should be equally viable (and they don't REQUIRE that the components reside on the shame host). From richard at NFSNet.org Thu Apr 18 19:12:13 2013 From: richard at NFSNet.org (Richard Wackerbarth) Date: Thu, 18 Apr 2013 12:12:13 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <51701EB1.1080804@zone12.com> References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> <51701EB1.1080804@zone12.com> Message-ID: <2082041F-2CCD-4F7B-84E4-F36B9B37A7BA@NFSNet.org> On Apr 18, 2013, at 11:26 AM, Terri Oda wrote: > On 13-04-18 8:03 AM, Richard Wackerbarth wrote: >> On Apr 18, 2013, at 1:19 AM, Florian Fuchs wrote: >> >>> 1) It should be self-contained. >>> Meaning: It should not depend on any >>> mailman/mailman.client/postorius/hyperkitty packages. >> Agreed > > I agree about not needing postorious/hyperkitty, but I think a case could be made that interdependence with mailman core and mailman client might make sense. Clearly, if you are going to follow (2) below, you will need to interface to MM-core for the portion that is stored there. >> I would advocate that this "User" module make it appear as if stores the entire "record" for the user. >> In the implementation, it could actually store parts of the user information in multiple databases (one of which could be the MM-core). >> >> This would also allow the option of having the MM-core become a client of this User module, just as it now relies on an external message store. Notice that I indicate that this is an option. Although, in the long run, I think it is the right way to go, getting there can be a migration and not something that needs to happen all at once. > > Two options occur to me: > > 1. The user module is what mm-core uses for all user stuff. > > I thinks case, case we have to be *much* more conservative about dependencies. I think Django would be right out as a dependency for Mailman core, for example. I thought that we were viewing this as a service accessed through a REST interface. MM core need never know if it is implemented using Django or FORTRAN or ... Why is depending on some part of Django prohibited when core uses zope, storm, ... ? Dependencies are just dependencies. As long as they are stable and readily available, why not use those that are useful? > Plus, we're going to have to care a lot more about speed and all. Much of the user information is not needed in the message handler (where speed is a consideration). Functions such as administration-by-mail that need to access the information are much less critical. Further, I will argue that those functions should be factored out of the "core" and exist as separate components. > 2. The user module reads from mm-core and augments it. > > This gives us the ability to supplement mm-core without impeding speed. We discussed possibly filling in the blanks with respect to things like the user preferences (which are currently set by membership, by user and by email address... but a lot of those return an empty set when queried if nothing is set directly there...) so this is maybe something we already want. > > Conceptually, #1 is probably easier because everything will be in one place, but if we do #2 right, we can make it just as conceptually easy for HyperKitty/Postorius/etc. without impeding Barry's core dev at all. That does mean in case 2 that Mailman Extraneous User Stuff is going to depend on Mailman Core, though. > > My preference is #2: I would agree except to the extent that Postorius references the user information. I believe that all user data should be accessed from the same source. I consider it poor design for the consumer of data to have to keep track of where particular parts of the data are stored. > a) It doesn't add any dependencies to Mailman core. > b) It doesn't require big changes in Mailman Core. Given that core is pretty much ready to release, now would be a bad time for changes, and I'm just not sure we can justify that amount of work for the types of features that will be built on the extraneous user stuff. > c) It will be much easier to rip this out and replace it when we better understand our actual needs. (e.g. Right now, I think a case could be made that a quick mock-up in django would be fine, but I suspect that requiring django for some potential applications would border on ridiculous) > > We're probably going to be running around with a bit of a hack job for the user database in the near future (either done by a student who needs it in a hurry or done by one of the core devs to support a student who needs it in a hurry) so while I don't like to design on the assumption it's going to go wrong, I think in this case planning for a redesign might be prudent because it's pretty clear we don't actually know all our requirements. > > Terri From flo.fuchs at gmail.com Thu Apr 18 20:54:59 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Thu, 18 Apr 2013 20:54:59 +0200 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: 2013/4/18 Richard Wackerbarth : > Whoa! Perhaps I don't understand oAuth. I thought that oAuth (and persona, kerberos, etc.) were protocols whereby one system (the provider) furnishes credentials for a second system (the client) to some third system (the consumer). > > By configuration, the consumer trusts that the provider has verified the client's identity and furnished appropriate credentials. > Thus, when the client presents credentials in an interaction with the consumer, the consumer provides services on the basis of the credentials. > > If we assume that we distribute the MM implementation to include more than the two (core and web UI) systems by having, for example, a user manager, there might be an argument for passing around such credentials. I was primarily thinking about the (future) authenticated REST API in Postorius. In that scenario a third party web app that we don't know would request an access token from the user profile store (provider) as well a user's email address. It would then use that access token for requests to the Postorius API (consumer). If the instance of the user store does not act as provider, we would either: 1) effectively require every api user to have an account with some other oauth provider. or 2) use some other authentication method. 1) is odd in my opinion. Mailman is free software. And there are not too many oauth providers that match that philosophy. 2) would be totally great if we found something that doesn't require a user to provide his credentials to said 3rd party app. I don't know. Maybe I'm missing something. And of course I agree with Stephen: If it's too hairy in terms of security, we should not do it. Anyway, we're talking about something that is absolutely not needed for what we want to achieve *right now*, which is a profile data store that Postorius/HK/etc can access from localhost (or maybe from an internal network. Or IP-restricted through SSL. Or... ?). Florian From p at sys4.de Thu Apr 18 21:05:07 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Thu, 18 Apr 2013 21:05:07 +0200 Subject: [Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion In-Reply-To: <20130412095939.66420805@anarchist> References: <87li8p5869.fsf@uwakimon.sk.tsukuba.ac.jp> <20130412062826.GA12030@sys4.de> <20130412095939.66420805@anarchist> Message-ID: <20130418190507.GD19472@sys4.de> * Barry Warsaw : > On Apr 12, 2013, at 08:28 AM, Patrick Ben Koetter wrote: > > >I think it would be real nice to have a MILTER interface at LMTP server level > >to allow mail modification as required. Mailman runs in large environments and > >all the 'large organizations' I have worked asked my team and me to customize > >how mail is processed. MILTER is a great interface to modify mail. > > Do you mean a hook in Mailman's LMTP server process? I thought about that in > my previous message but decided not to mention it because it's not clear to me > how performant Mailman's current smtpd-based (read: async) LMTP server is. > What I mean is, I'm not sure how much additional work we want the LMTP server > to do. > > It would be cool if someone did some performance testing of the LMTP > implementation, and it would be cool if someone tried to add some hooks into > that server. It might also be interesting to look into alternative > implementations. Another reason to push for getting Mailman 3 onto Python 3 > would be the ability to leverage Guido's Tulip work for better async IO > performance. We did a quick test and blew 10.000 messages into Mailman 3's LMTP server. The hardware was/is a Pentium 2, 2 GB RAM machine with desktop discs - way below current server hardware. It took the test 25 min. to submit all messages: real 25m10.041s user 0m4.872s sys 0m7.700s That makes an average of 400 msg/min or 6,6 msg/sec Robert, who did the tests, Ralf and I agree that this is "way enough" for LMTP server performance. If we add a MILTER interface, the milter applications hooked into the LMTP servers receiving process will slow down the income rate. The impact depends on what the specific application tests or what kind of modification it applies to the message. In general MILTERs are designed to work in memory only. No message will need to be written to a disc, which usually is the most expensive operation during mail processing. At the moment we (at sys4.de) don't think it needs further testing, but we offer to do so if you have reason to do so. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From flo.fuchs at gmail.com Thu Apr 18 21:29:56 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Thu, 18 Apr 2013 21:29:56 +0200 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> Message-ID: 2013/4/18 Richard Wackerbarth : > On Apr 18, 2013, at 1:19 AM, Florian Fuchs wrote: >> 3) It doesn't need Django. >> Since it will not deliver any HTML (except an oAuth login form -- see >> 5.) and it doesn't need to be integrated into any existing web site, >> we can choose a more light-weight framework. > > Here I take exception. Dismissing Django is a restriction that unnecessarily affects the ease of implementation and, in the common case, complicates the integration of the functionality. > > Although it could be implemented without Django, it could also be implemented as a Django "app". > An instance of a django server can then serve the functionality. As an alternative, where appropriate, this "app" would directly "drop in" to an instance of Postorius or an enterprise website. > > One of the advantages of Django is that it can be used as a rapid prototyping mechanism. Simplified interfaces to the data are "free" and more elaborate ones can be added in an incremental fashion. > Also, rather than writing custom modules for things such as authentication and REST interfaces, there is the large community of third-party extensions which readily integrate to provide that functionality. It's not that I don't want to use Django. I just wanted to point out that we won't need much of it for a pure JSON API. OTOH adding a new dependency by using another framework is probably not a good idea either. So if we want to keep the number of dependencies low, the alternative would probably be to use no framework at all and use restish for the, well, REST stuff (or whatever library the core will be using in the future). Florian > I would advocate that this "User" module make it appear as if stores the entire "record" for the user. > In the implementation, it could actually store parts of the user information in multiple databases (one of which could be the MM-core). +1 From terri at zone12.com Thu Apr 18 21:34:05 2013 From: terri at zone12.com (Terri Oda) Date: Thu, 18 Apr 2013 13:34:05 -0600 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <51704AAD.90300@zone12.com> On 13-04-18 2:28 AM, Stephen J. Turnbull wrote: > Main comment: Sounds like LDAP to me. Actually, this is a really important comment. I was sort of wondering that too when I started writing the description. LDAP is a moderately frequently requested feature already. Would it make sense to use that probably rather than rest, or make a rest framework that talked to ldap under the hood? Terri From flo.fuchs at gmail.com Thu Apr 18 21:39:46 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Thu, 18 Apr 2013 21:39:46 +0200 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <51701EB1.1080804@zone12.com> References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> <51701EB1.1080804@zone12.com> Message-ID: 2013/4/18 Terri Oda : > We're probably going to be running around with a bit of a hack job for the > user database in the near future (either done by a student who needs it in a > hurry or done by one of the core devs to support a student who needs it in a > hurry) so while I don't like to design on the assumption it's going to go > wrong, I think in this case planning for a redesign might be prudent because > it's pretty clear we don't actually know all our requirements. I think in this case it makes sense to spend a more time on the API as seen from the outside (urls, methods, json responses) than the actual implementation. If the API is good, it's totally acceptable if the underlying implementation will be redesigned later. Florian From p at sys4.de Thu Apr 18 21:40:07 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Thu, 18 Apr 2013 21:40:07 +0200 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: Message-ID: <20130418194006.GE19472@sys4.de> * Pratik Sarkar : > Hi guys, > Is the anti-spam/abuse filter still being seriously considered > as a gsoc project this year? I - personal opinion - think adding a anti-spam/abuse filter is a good idea. I do however stronly oppose against a hardcoded implemenation that e.g. integrates SpamAssassin only or a new Bayesian filter. Why? (Please don't take this personal...) Reinvention I think it is a waste of time to reinvent (Bayes) something that has been there for many years. Proprietary implementation I also think it is no good service to those who will also want to expand Mailmans capabilities to do a product specific implementation instead of building a generic interface. I think, if Mailman should become able to filter spam and viruses, we should use a standardized interface (e.g. MILTER, SMTP/LMTP transport) that allows _any_ kind of mail filter to be hooked into the receiving process. Then anyone would be able to add their secret sauce when it comes to process email. This way the (new) Bayesian filter, which has been suggested on the list, could interface with Mailman too. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From terri at zone12.com Thu Apr 18 21:51:57 2013 From: terri at zone12.com (Terri Oda) Date: Thu, 18 Apr 2013 13:51:57 -0600 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130418194006.GE19472@sys4.de> References: <20130418194006.GE19472@sys4.de> Message-ID: <51704EDD.6040001@zone12.com> On 13-04-18 1:40 PM, Patrick Ben Koetter wrote: > I - personal opinion - think adding a anti-spam/abuse filter is a good > idea. I do however stronly oppose against a hardcoded implemenation > that e.g. integrates SpamAssassin only or a new Bayesian filter. I hope that no one was seriously considering that level of hardcoding. What we are almost certainly talking about is setting up a handler (I think Stephen estimated this to be around 10 lines of code). I do think we'll want some related UI options so that list admins could do supervised learning as well, but since that's mostly a "mark this as spam" as an option other than just "reject this posting" (which could also indicate a duplicate or someone posting from a work address or whatever) and we should be able to generalize the hooks to send that information to whatever system(s) are in use. Terri From terri at zone12.com Thu Apr 18 21:54:03 2013 From: terri at zone12.com (Terri Oda) Date: Thu, 18 Apr 2013 13:54:03 -0600 Subject: [Mailman-Developers] Build a mailman3 development virtual machine with Vagrant and Chef In-Reply-To: References: Message-ID: <51704F5B.9070102@zone12.com> Yeay, thanks! If you've got time, could you make sure your vm is linked on the mailman wiki docs? You can replace the comment I have on the GSoC page for sure, and there's probably a good place to link it in a couple "setting up your dev environment" pages. Terri On 13-04-18 1:39 AM, Xu Wang wrote: > Hi there, > > Although mailman3 installation is well documented, but putting everything > together is still not a trivial job. > > To ease the task, I made a mailman3 chef cookbook and vagrant file: > > https://github.com/xuwang/mailman3-vbox > > If you already had virtualbox/vagrant installed on your mac or pc, build a > working mailman3 vm should only take a few minutes. > > I'm sure there will be things I missed or got it wrong, but it just a > starter. > > Hope this helps, > > Xu Wang > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/terri%40zone12.com > > Security Policy: http://wiki.list.org/x/QIA9 > From rkw at DATAPLEX.NET Thu Apr 18 21:56:08 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Thu, 18 Apr 2013 14:56:08 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> <51701EB1.1080804@zone12.com> Message-ID: <76D68763-1F9D-4BE9-9C1D-C5FDB2C415B5@DATAPLEX.NET> On Apr 18, 2013, at 2:39 PM, Florian Fuchs wrote: > 2013/4/18 Terri Oda : >> We're probably going to be running around with a bit of a hack job for the >> user database in the near future (either done by a student who needs it in a >> hurry or done by one of the core devs to support a student who needs it in a >> hurry) so while I don't like to design on the assumption it's going to go >> wrong, I think in this case planning for a redesign might be prudent because >> it's pretty clear we don't actually know all our requirements. > > I think in this case it makes sense to spend a more time on the API as > seen from the outside (urls, methods, json responses) than the actual > implementation. If the API is good, it's totally acceptable if the > underlying implementation will be redesigned later. I agree and I would advocate that that API model be based on the interface that you can readily get from a django database model and the django-rest-framework. There is a big advantage to being able to readily edit the model and have the corresponding changes to the interface automatically generated. Since HK and Postorius already use Django, you are not adding any significant dependancies. And you can leverage a existing code base for the implementation details. From rkw at DATAPLEX.NET Thu Apr 18 22:10:08 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Thu, 18 Apr 2013 15:10:08 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> Message-ID: <5539434F-B1D8-4C8C-8617-E62DC33662C4@DATAPLEX.NET> Yes, yes. Please re-invent the wheel once again. And while you are at it, you might just remove the dependancies on zope and storm, etc. I think that you are missing the point that, at this time, this is intended to provide the capabilities that MM-core chooses not to implement. Those website components (HK and Postorius) are being driven by Django and you only make it more difficult to implement them when choose a different schema to model/present the data. On Apr 18, 2013, at 2:29 PM, Florian Fuchs wrote: > 2013/4/18 Richard Wackerbarth : >> On Apr 18, 2013, at 1:19 AM, Florian Fuchs wrote: >>> 3) It doesn't need Django. >>> Since it will not deliver any HTML (except an oAuth login form -- see >>> 5.) and it doesn't need to be integrated into any existing web site, >>> we can choose a more light-weight framework. >> >> Here I take exception. Dismissing Django is a restriction that unnecessarily affects the ease of implementation and, in the common case, complicates the integration of the functionality. >> >> Although it could be implemented without Django, it could also be implemented as a Django "app". >> An instance of a django server can then serve the functionality. As an alternative, where appropriate, this "app" would directly "drop in" to an instance of Postorius or an enterprise website. >> >> One of the advantages of Django is that it can be used as a rapid prototyping mechanism. Simplified interfaces to the data are "free" and more elaborate ones can be added in an incremental fashion. >> Also, rather than writing custom modules for things such as authentication and REST interfaces, there is the large community of third-party extensions which readily integrate to provide that functionality. > > It's not that I don't want to use Django. I just wanted to point out > that we won't need much of it for a pure JSON API. OTOH adding a new > dependency by using another framework is probably not a good idea > either. So if we want to keep the number of dependencies low, the > alternative would probably be to use no framework at all and use > restish for the, well, REST stuff (or whatever library the core will > be using in the future). > > Florian > >> I would advocate that this "User" module make it appear as if stores the entire "record" for the user. >> In the implementation, it could actually store parts of the user information in multiple databases (one of which could be the MM-core). > > +1 From richard at NFSNet.org Thu Apr 18 22:26:16 2013 From: richard at NFSNet.org (Richard Wackerbarth) Date: Thu, 18 Apr 2013 15:26:16 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <19FB3F9E-6BD9-4AC6-ADFE-06350DA93C3C@NFSNet.org> On Apr 18, 2013, at 1:54 PM, Florian Fuchs wrote: > 2013/4/18 Richard Wackerbarth : >> Whoa! Perhaps I don't understand oAuth. I thought that oAuth (and persona, kerberos, etc.) were protocols whereby one system (the provider) furnishes credentials for a second system (the client) to some third system (the consumer). >> >> By configuration, the consumer trusts that the provider has verified the client's identity and furnished appropriate credentials. >> Thus, when the client presents credentials in an interaction with the consumer, the consumer provides services on the basis of the credentials. >> >> If we assume that we distribute the MM implementation to include more than the two (core and web UI) systems by having, for example, a user manager, there might be an argument for passing around such credentials. > > I was primarily thinking about the (future) authenticated REST API in > Postorius. In that scenario a third party web app that we don't know > would request an access token from the user profile store (provider) > as well a user's email address. It would then use that access token > for requests to the Postorius API (consumer). > > If the instance of the user store does not act as provider, we would either: > > 1) effectively require every api user to have an account with some > other oauth provider. > > or > > 2) use some other authentication method. > > > 1) is odd in my opinion. Mailman is free software. And there are not > too many oauth providers that match that philosophy. > 2) would be totally great if we found something that doesn't require a > user to provide his credentials to said 3rd party app. Look at what happens now. The user contacts Postorius and presents either some oAuth/BrowserID-style credential or it presents a username/password. In exchange, Postorius issues a local credential (session key). When the username/password path is chosen, Postorius acts as an agent for the user and verifies the login. In this case, Postorius must "see" the underlying credential (username/password pair) to be able to present it to the authenticator. In the oAuth path, Postorius never sees the underlying credential, but exchanges the presented one for its local credential. Since we consider the user manager to be a part of the MM complex, what have we gained by hiding the underlying credential from the web interface? Therefore, I would suggest that we need to be able to accept third-party (oAuth) credentials and also locally presented ones. I don't see where we gain an advantage by exposing the authenticator as a separate oAuth provider. > I don't know. Maybe I'm missing something. And of course I agree with > Stephen: If it's too hairy in terms of security, we should not do it. > > Anyway, we're talking about something that is absolutely not needed > for what we want to achieve *right now*, which is a profile data store > that Postorius/HK/etc can access from localhost (or maybe from an > internal network. Or IP-restricted through SSL. Or... ?). > > Florian From p at sys4.de Thu Apr 18 22:32:21 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Thu, 18 Apr 2013 22:32:21 +0200 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <51704EDD.6040001@zone12.com> References: <20130418194006.GE19472@sys4.de> <51704EDD.6040001@zone12.com> Message-ID: <20130418203221.GH19472@sys4.de> * Terri Oda : > > On 13-04-18 1:40 PM, Patrick Ben Koetter wrote: > >I - personal opinion - think adding a anti-spam/abuse filter is a > >good idea. I do however stronly oppose against a hardcoded > >implemenation that e.g. integrates SpamAssassin only or a new > >Bayesian filter. > > I hope that no one was seriously considering that level of > hardcoding. What we are almost certainly talking about is setting > up a handler (I think Stephen estimated this to be around 10 lines > of code). And that handler would be - excuse my ignorance I don't program - a Python function to handle a Python program? Could the handler pass a message over to e.g. SpamAssassin (Perl) or openDKIM (c code) or any other non Python program without the need to add any additional (Python) code? p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From flo.fuchs at gmail.com Thu Apr 18 23:33:42 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Thu, 18 Apr 2013 23:33:42 +0200 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <5539434F-B1D8-4C8C-8617-E62DC33662C4@DATAPLEX.NET> References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> <5539434F-B1D8-4C8C-8617-E62DC33662C4@DATAPLEX.NET> Message-ID: 2013/4/18 Richard Wackerbarth : > Yes, yes. Please re-invent the wheel once again. > > And while you are at it, you might just remove the dependancies on zope and storm, etc. > > I think that you are missing the point that, at this time, this is intended to provide the capabilities that MM-core chooses not to implement. > Those website components (HK and Postorius) are being driven by Django and you only make it more difficult to implement them when choose a different schema to model/present the data. I don't want to re-invent the wheel and I don't advocate dismissing Django. Django is probably perfect for the prototype. It might also be perfect for the final thing. But we don't know what database we're gonna use in the end (which determines if the Django ORM still fits), or if we'll use many HTML templates etc. If we end up not using many of the things that make Django great, we might wanna think about if we want to use it. And might still come to the conclusion that we do. All I'm saying is: The environment for such an app could be a little different to Postorius/HK, so let's spend some thought on it. Is this such a bad idea? Florian From rkw at DATAPLEX.NET Thu Apr 18 23:48:02 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Thu, 18 Apr 2013 16:48:02 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> <5539434F-B1D8-4C8C-8617-E62DC33662C4@DATAPLEX.NET> Message-ID: <1418DE60-2F62-4809-AF39-FF9705BE8CE5@DATAPLEX.NET> What I am suggesting is that we use the JSON format and URLs that are generated by this combination. If you later decide to implement it in a different framework, you can still use the same external representation. On Apr 18, 2013, at 4:33 PM, Florian Fuchs wrote: > 2013/4/18 Richard Wackerbarth : >> Yes, yes. Please re-invent the wheel once again. >> >> And while you are at it, you might just remove the dependancies on zope and storm, etc. >> >> I think that you are missing the point that, at this time, this is intended to provide the capabilities that MM-core chooses not to implement. >> Those website components (HK and Postorius) are being driven by Django and you only make it more difficult to implement them when choose a different schema to model/present the data. > > I don't want to re-invent the wheel and I don't advocate dismissing > Django. Django is probably perfect for the prototype. It might also be > perfect for the final thing. But we don't know what database we're > gonna use in the end (which determines if the Django ORM still fits), > or if we'll use many HTML templates etc. If we end up not using many > of the things that make Django great, we might wanna think about if we > want to use it. And might still come to the conclusion that we do. All > I'm saying is: The environment for such an app could be a little > different to Postorius/HK, so let's spend some thought on it. Is this > such a bad idea? > > Florian From mark at msapiro.net Fri Apr 19 00:42:07 2013 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 18 Apr 2013 15:42:07 -0700 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130418203221.GH19472@sys4.de> References: <20130418194006.GE19472@sys4.de> <51704EDD.6040001@zone12.com> <20130418203221.GH19472@sys4.de> Message-ID: <517076BF.1040401@msapiro.net> On 4/18/2013 1:32 PM, Patrick Ben Koetter wrote: > * Terri Oda : >> >> I hope that no one was seriously considering that level of >> hardcoding. What we are almost certainly talking about is setting >> up a handler (I think Stephen estimated this to be around 10 lines >> of code). > > And that handler would be - excuse my ignorance I don't program - a Python > function to handle a Python program? Could the handler pass a message over to > e.g. SpamAssassin (Perl) or openDKIM (c code) or any other non Python program > without the need to add any additional (Python) code? In 10 lines? Maybe. If SpamAssassin is spamd, the Python socket module is part of the standard library. OpenDKIM includes a milter interface. There are Python milter shims available Perhaps your point is that this is more than 10 lines of code which is probably true, but interfacing to some specific spam filter plugin architecture from a Mailman chain rule module does not seem to me to be a big project. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Fri Apr 19 02:04:50 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 18 Apr 2013 20:04:50 -0400 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130418203221.GH19472@sys4.de> References: <20130418194006.GE19472@sys4.de> <51704EDD.6040001@zone12.com> <20130418203221.GH19472@sys4.de> Message-ID: <20130418200450.524b8f69@anarchist> On Apr 18, 2013, at 10:32 PM, Patrick Ben Koetter wrote: >And that handler would be - excuse my ignorance I don't program - a Python >function to handle a Python program? Could the handler pass a message over to >e.g. SpamAssassin (Perl) or openDKIM (c code) or any other non Python program >without the need to add any additional (Python) code? In Mailman 3, it would be a class implementing an interface. I still think this wouldn't be a handler, but a rule. Although there's no such distinction in MM2, there is in MM3: rules don't modify the message, handlers do. E.g. the IRule interface has a check() method that takes three arguments, the mailing list, the message, and the metadata. It returns a boolean specifying whether the rule matches or not. Thus a SpamAssassin rule might look like: @implementer(IRule) class SpamAssassin: # See IRule for details. name = 'spamassassin' description = _('See if SpamAssassin thinks this message is spam') record = True def check(self, mlist, msg, msgdata): spam_score = shell_out_to_sa(msg) msgdata['spamassassin_score'] = spam_score return spam_score > some_threshold -Barry From barry at list.org Fri Apr 19 02:10:05 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 18 Apr 2013 20:10:05 -0400 Subject: [Mailman-Developers] Setting up a VM. In-Reply-To: References: <51698325.5090808@zone12.com> Message-ID: <20130418201005.7742989a@anarchist> On Apr 18, 2013, at 01:10 PM, Chris Cargile wrote: >The cause in both cases seems to stem from failing at the SMTPLayer, where >it shows: [Error98] Address already in use. This almost always means that some runner process from a previous test didn't exit. I've tried to bulletproof the process management in MM3, but sometimes the processes can hang around, keeping ports in use. >"mailman conf" shows that smtp port setting is:25. Remember though that the test infrastructure pushes a small configuration on top of that, which `mailman conf` wouldn't show (that would be an interesting enhancement :). Look at src/mailman/testing/testing.cfg for details. In particular, it uses unprivileged ports for the testing SMTP and LMTP servers (9025 and 9024 respectively). Use ps to find and kill the processes and the ports will free up. Cheers, -Barry From barry at list.org Fri Apr 19 02:16:47 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 18 Apr 2013 20:16:47 -0400 Subject: [Mailman-Developers] GSOC 2013 : Authenticated REST-API in Postorius/Django In-Reply-To: References: <05BC1D67-31B0-41AA-82C2-8A47AB7AF7BD@NFSNet.org> Message-ID: <20130418201647.02bd8730@anarchist> On Apr 18, 2013, at 01:56 PM, Rahul Gaur wrote: >Talking about authentication, django-restframework has a wrapper for OAuth2 >out of the box, but I will have to check how to use non form data sources. I keep hearing terrible things about OAuth2. OAuth1 is nice and simple and there are plenty of good libraries in many different languages to support it, so we want to think about which version of OAuth to support carefully. Maybe there are security issues involved. >I know the support for python 3 is essential to the project, but as of now >sticking I would prefer to sticking to Python 2.7.x for now. As a practical matter, MM3.0 will be Python 2.7. The core can't be ported until restish is ported, and I don't have high hopes for Storm ever being ported, so we'd have to also switch to SQLAlchemy or some other Python 3 compatible ORM (if there is another one). That's way too much work for the 3.0 release. What I would say though, is to be as careful as you can so that you will be able to port to Python 3 when the time comes. This means making your code `python2.7 -3` clean and being *very* certain of your bytes vs. strings story. Essentially, you want to write your code to be bi-lingual without of course using any Python 3 idioms. This page may help: https://wiki.ubuntu.com/Python/3 -Barry From p at sys4.de Fri Apr 19 02:40:29 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Fri, 19 Apr 2013 02:40:29 +0200 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <517076BF.1040401@msapiro.net> References: <20130418194006.GE19472@sys4.de> <51704EDD.6040001@zone12.com> <20130418203221.GH19472@sys4.de> <517076BF.1040401@msapiro.net> Message-ID: <20130419004028.GA6902@sys4.de> * Mark Sapiro : > On 4/18/2013 1:32 PM, Patrick Ben Koetter wrote: > > * Terri Oda : > >> > >> I hope that no one was seriously considering that level of > >> hardcoding. What we are almost certainly talking about is setting > >> up a handler (I think Stephen estimated this to be around 10 lines > >> of code). > > > > And that handler would be - excuse my ignorance I don't program - a Python > > function to handle a Python program? Could the handler pass a message over to > > e.g. SpamAssassin (Perl) or openDKIM (c code) or any other non Python program > > without the need to add any additional (Python) code? > > In 10 lines? Maybe. > > If SpamAssassin is spamd, the Python socket module is part of the > standard library. > > OpenDKIM includes a milter interface. There are Python milter shims > available > > Perhaps your point is that this is more than 10 lines of code which is > probably true, but interfacing to some specific spam filter plugin > architecture from a Mailman chain rule module does not seem to me to be > a big project. My original point was that we should use MILTERs because we shouldn't reinvent what already has been implemented very well and we shouldn't come up with a proprietary solution, because that will make it harder to expand Mailman and also less versatile as a tool. Let me add this too: I think we miss out some important customers if we stop at "but you only need to add code to get what you want". Mail systems are created and run by postmasters in most cases. Even though there's a strong movement at the moment towards 'devops', the majority of sysops/postmasters can't program. If we stop at a programmable interface - which I understand is what you and Terri suggest - they will not be able to use Mailman in more complex setups, because integrating additional mail processing components would require them to program. I suggest to go further and add a MILTER because this way all customers involved - admins and developers - will get what they want/need: - Admins can check if there's a MILTER out there that already does what the mail processing chain needs. If there is, they connect Mailman and the MILTER via a well know interface (UNIX domain socket/TCP socket) and be done. No need to program. Configuration only. Most of the tools used in everyday postmastering provide a MILTER interface to interact with other mail processing components. - Developers will have full access (LMTP session data, mail header, mail body) to a message via the MILTER protocol. They can create their specialized application in any programing language that suits the job. As you said, Python brings (about 16) tons of code to get you started easily. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From barry at list.org Fri Apr 19 03:00:25 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 18 Apr 2013 21:00:25 -0400 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130419004028.GA6902@sys4.de> References: <20130418194006.GE19472@sys4.de> <51704EDD.6040001@zone12.com> <20130418203221.GH19472@sys4.de> <517076BF.1040401@msapiro.net> <20130419004028.GA6902@sys4.de> Message-ID: <20130418210025.7c53b151@anarchist> On Apr 19, 2013, at 02:40 AM, Patrick Ben Koetter wrote: >My original point was that we should use MILTERs because we shouldn't reinvent >what already has been implemented very well and we shouldn't come up with a >proprietary solution, because that will make it harder to expand Mailman and >also less versatile as a tool. I don't know the milter API, but from a quick look at milter.org, it seems like there's a C libmilter library. It looks like there are several options for integrating this with Python, none of which I've actually looked at yet: https://pypi.python.org/pypi?%3Aaction=search&term=milter&submit=search From your description, it sounds like you might not even need this for our purposes. Just open a socket, shove the right bunch of bytes down it, then read and parse the response. In any case, I'm not opposed to the *idea* of supporting milters in MM3, but I think that's a long way from "yes, let's do it". A lot more analysis and design is necessary, so it would be cool if someone is really interested in pursing this. >I think we miss out some important customers if we stop at "but you only need >to add code to get what you want". > >Mail systems are created and run by postmasters in most cases. Even though >there's a strong movement at the moment towards 'devops', the majority of >sysops/postmasters can't program. > >If we stop at a programmable interface - which I understand is what you and >Terri suggest - they will not be able to use Mailman in more complex setups, >because integrating additional mail processing components would require them >to program. Well, it would require *someone* to program, but I hope not the sysadmin unless they really want to. One of the design goals of MM3 is to allow for much easier extensibility. For some sense of what I envision, look at the way HyperKitty hooks into MM3. You grab some code someone else has written, stick it in some place on the file system, set a configuration variable or two, and restart. The nice thing is that this all then becomes re-usable with no source hacking required, which makes things much better than in MM2. -Barry From stephen at xemacs.org Fri Apr 19 03:06:58 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 10:06:58 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <4D2BFD37-424D-45F8-A0CC-E9B850BCC8A2@DATAPLEX.NET> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zj3i5e.fsf@uwakimon.sk.tsukuba.ac.jp> <8738un3gb2.fsf@uwakimon.sk.tsukuba.ac.jp> <4D2BFD37-424D-45F8-A0CC-E9B850BCC8A2@DATAPLEX.NET> Message-ID: <871ua72urx.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > Perhaps I didn't understand you. I thought that you were > advocating the omission of any channels other than "shell" and > "localhost". I'm saying that we should make appropriate Mailman components be OAuth clients (subject to site policy, per component), but try to avoid providing *any* authentication ourselves (localhost relies on existing shell access mechanisms via OS or SSH etc, HTTP Basic auth relies on Apache or other webserver, OAuth we'll have to build in, but the actual authentication is done by 3rd party providers). I suppose we'll have to provide moderation-by-password-in-headers and the traditional triple-handshake-by-mail for backward compatibility. Ah - I missed a very important channel: secure mail via OpenPGP. We need something like that (but again, the actual auth/auth process is done by GPG or PGP, we just rely on the token (signature) provided as a valid identification of a user). > I was trying to point out that HTTPS, oAuth, etc. should be equally > viable (and they don't REQUIRE that the components reside on the > shame host). I don't think anybody is opposed to exploring distributed architecture, and that implies securing inter-component communications. The question is how much of the security architecture we should provide ourselves. I advocate restricting that to the bare minimum, by which I mean "we don't *do* anything we don't *need* to do ourselves", not "we don't reimplement functionality that is available in Python packages or 3rd-party libraries we can wrap". From stephen at xemacs.org Fri Apr 19 03:08:35 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 10:08:35 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <2260206D-E7A7-4AFB-A645-1A0CFDCEEB06@DATAPLEX.NET> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <8761zj3i5e.fsf@uwakimon.sk.tsukuba.ac.jp> <2260206D-E7A7-4AFB-A645-1A0CFDCEEB06@DATAPLEX.NET> Message-ID: <87zjwv1g4s.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > I have no problem with, and actually encourage, that we act as a > consumer of oAuth credentials. +1 > However, the issue here is whether we should be provider of oAuth > credentials (which might then be presented to some outside, totally > unrelated, entity. -1 From barry at list.org Fri Apr 19 03:20:13 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 18 Apr 2013 21:20:13 -0400 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <51704AAD.90300@zone12.com> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <51704AAD.90300@zone12.com> Message-ID: <20130418212013.79100b8a@anarchist> Terri, thanks for kicking off this discussion! On Apr 18, 2013, at 01:34 PM, Terri Oda wrote: >On 13-04-18 2:28 AM, Stephen J. Turnbull wrote: >> Main comment: Sounds like LDAP to me. >Actually, this is a really important comment. I was sort of wondering that >too when I started writing the description. LDAP is a moderately frequently >requested feature already. Would it make sense to use that probably rather >than rest, or make a rest framework that talked to ldap under the hood? I agree that this is a really interesting suggestion. What's interesting to me about it is that this acknowledges that administrative control of this extra user information may fall to folks not at all directly involved in mailing list administration. You can imagine that a company would already have their LDAP database of additional user information, from postal addresses, to phone numbers, to IRC handles, pictures and icons, birthdays, etc. Further, I can imagine that if that company were to be running MM3 and HyperKitty, they'd probably not want to duplicate that information in multiple databases, they'd just want to pull the pictures and IRC nicks out of LDAP. For their purposes, their LDAP database *is* their user database. Of course, some of that information may also be used in part or in whole for the core, e.g. display names (which actually are useful in places, such as crafting the From header for a virgin notification). Now, at least conceptually, I've always thought about the user database as one of the three pillars of information in the core, the other two being the list database and the message store. This is why for example subscriptions do not just link a foreign key for the mlist to a foreign key to the address, because this would not allow for separation of these two pillars[1]. Furthermore, the use of interfaces internally is meant to allow for alternative implementations of the lowest database layer of these pillars, without affecting any functionality above it. So let's say even in the core, user passwords, display names, and preferences were kept in LDAP. An alternative implementation of user.py would pull the data in from there and as long as it satisfied the syntactic and semantic APIs of the IUser interface, everything else in MM3 would continue to work. You could extend this to the idea expressed elsewhere that the core could use this external user database. Why not? The external user database could be accessible over REST or a via a database server, etc. We'd need an implementation of the underlying data objects that could talk to this user database and satisfy the interfaces, and that should be all you'd need. Making it performant enough for the core is just an engineering exercise . -Barry [1] Whether any of this works or has the right separation is another matter entirely. ;) For example, looking at it now, I'm not so sure Members shouldn't be part of the MailingList pillar rather than the user database pillar. From stephen at xemacs.org Fri Apr 19 03:22:38 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 10:22:38 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> Florian Fuchs writes: > If the instance of the user store does not act as provider, we would either: > > 1) effectively require every api user to have an account with some > other oauth provider. Most people do. Sites that care can bring up their own provider. AFAIK that's not terribly hard, and I doubt that Mailman can make it much easier. But we shouldn't encourage it. Requiring a specific provider sucks if not necessary for security reasons; users just end up managing yet another password and get no benefit. Adding a new provider, even if not required, is likely to cause the same problem if the user later signs up with a well-known provider. The point of OAuth for Mailman is that it's nearly formally equivalent to the traditional 3-part handshake (request subscription, receive token by mail to subscribed address, return token thus proving you are owner of the address), since most OAuth providers are email providers. Perfect fit. > or > > 2) use some other authentication method. What's wrong with HTTPS + Basic Auth? > Anyway, we're talking about something that is absolutely not needed > for what we want to achieve *right now*, which is a profile data store > that Postorius/HK/etc can access from localhost (or maybe from an > internal network. Or IP-restricted through SSL. Or... ?). OAuth (or Persona, etc) access for subscribers (to Postorius and HyperKitty) using subscribers' existing OAuth provider is not *absolutely* needed, but would be a very big plus. Users hate managing passwords for resources they don't care much about security. Once we've got that, I would think it's a relatively small step (in terms of code) to implementing for inter-component communications. The security audit required might be quite a bit of work, so we might not recommend deploying the feature in "high" security contexts. From stephen at xemacs.org Fri Apr 19 03:25:28 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 10:25:28 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <19FB3F9E-6BD9-4AC6-ADFE-06350DA93C3C@NFSNet.org> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <19FB3F9E-6BD9-4AC6-ADFE-06350DA93C3C@NFSNet.org> Message-ID: <87wqrz1fcn.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > Since we consider the user manager to be a part of the MM complex, > what have we gained by hiding the underlying credential from the > web interface? Security. See the OAuth 2.0 spec (RFC 6749) which recommends (at SHOULD level) this practice. From turnbull at sk.tsukuba.ac.jp Fri Apr 19 03:28:47 2013 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 10:28:47 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130418200450.524b8f69@anarchist> References: <20130418194006.GE19472@sys4.de> <51704EDD.6040001@zone12.com> <20130418203221.GH19472@sys4.de> <20130418200450.524b8f69@anarchist> Message-ID: <87vc7j1f74.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > I still think this wouldn't be a handler, but a rule. Is this distinction implemented and enforced by the API? If not, it's going to be hard to persuade myself to make the distinction in discussion. Ie, I'll probably just go on calling everything a Handler unless somebody asks "don't you really mean a Rule?" From stephen at xemacs.org Fri Apr 19 05:07:35 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 19 Apr 2013 12:07:35 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <20130418212013.79100b8a@anarchist> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <51704AAD.90300@zone12.com> <20130418212013.79100b8a@anarchist> Message-ID: <87r4i71amg.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > What's interesting to me about it is that this acknowledges that > administrative control of this extra user information may fall to > folks not at all directly involved in mailing list administration. I think this is crucial to World Domination^W^WMailman 3 uptake. > Now, at least conceptually, I've always thought about the user > database as one of the three pillars of information in the core, > the other two being the list database and the message store. I like this conceptual division, but I admit I haven't thought too carefully about what goes where (eg, you mention Members). From rkw at DATAPLEX.NET Fri Apr 19 05:09:19 2013 From: rkw at DATAPLEX.NET (Richard Wackerbarth) Date: Thu, 18 Apr 2013 22:09:19 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87wqrz1fcn.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <19FB3F9E-6BD9-4AC6-ADFE-06350DA93C3C@NFSNet.org> <87wqrz1fcn.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <389AEBEB-AB11-4DF7-87D2-4B899D83734F@DATAPLEX.NET> On Apr 18, 2013, at 8:25 PM, Stephen J. Turnbull wrote: > Richard Wackerbarth writes: > >> Since we consider the user manager to be a part of the MM complex, >> what have we gained by hiding the underlying credential from the >> web interface? > > Security. See the OAuth 2.0 spec (RFC 6749) which recommends (at > SHOULD level) this practice. RFC 6749 addresses the implementation of an OAuth authorization system. In this context, SHOULD refers to the implementation of this RFC. It does not imply that other authorization schemes also need to meet those same criteria. As for security, exposing the authorization server to direct Internet access is, in itself, a security weak point. From xuwang at gmail.com Fri Apr 19 09:16:20 2013 From: xuwang at gmail.com (Xu Wang) Date: Fri, 19 Apr 2013 00:16:20 -0700 Subject: [Mailman-Developers] Build a mailman3 development virtual machine with Vagrant and Chef In-Reply-To: <51704F5B.9070102@zone12.com> References: <51704F5B.9070102@zone12.com> Message-ID: I'm not sure I have the permission to edit the wiki page. If anyone with the permission and know how, please feel free to add the link. Xu Wang On Thu, Apr 18, 2013 at 12:54 PM, Terri Oda wrote: > Yeay, thanks! > > If you've got time, could you make sure your vm is linked on the mailman > wiki docs? You can replace the comment I have on the GSoC page for sure, > and there's probably a good place to link it in a couple "setting up your > dev environment" pages. > > Terri > > > On 13-04-18 1:39 AM, Xu Wang wrote: > >> Hi there, >> >> Although mailman3 installation is well documented, but putting everything >> together is still not a trivial job. >> >> To ease the task, I made a mailman3 chef cookbook and vagrant file: >> >> https://github.com/xuwang/**mailman3-vbox >> >> If you already had virtualbox/vagrant installed on your mac or pc, build a >> working mailman3 vm should only take a few minutes. >> >> I'm sure there will be things I missed or got it wrong, but it just a >> starter. >> >> Hope this helps, >> >> Xu Wang >> ______________________________**_________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/**mailman/listinfo/mailman-**developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: http://www.mail-archive.com/** >> mailman-developers%40python.**org/ >> Unsubscribe: http://mail.python.org/**mailman/options/mailman-** >> developers/terri%40zone12.com >> >> Security Policy: http://wiki.list.org/x/QIA9 >> >> > From iane at sussex.ac.uk Fri Apr 19 13:48:44 2013 From: iane at sussex.ac.uk (Ian Eiloart) Date: Fri, 19 Apr 2013 11:48:44 +0000 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 15 Apr 2013, at 11:22, Stephen J. Turnbull wrote: > OTOH, a generic interface to the Mailman REST API, which could be used > by Sendmail milters or whatever else is out there and somewhat > standard (is it Postfix or Exim that can handle milters? I forget), > with example implementation of a milter that checks whether the poster > is subscribed by asking Mailman, would be useful as an extension to > any MTA used with Mailman. I think Mailman supports SMTP/LMTP calls to discover whether a sender is permitted to post to a list, doesn't it? Exim doesn't handle Milters, but can do the calls forward. Provided Mailman is making the judgement, and issuing L/SMTP rejects at L/SMTP time before accepting the message, Exim is fine. Content filtering *could* also be done at L/SMTP time. I think that where the Mailman and the MTA installations are managed by the same person or organisation, then the better place to have content filtering performed is at the MTA, but there might be exceptions to this. For example, a medical mailing list might want to be more liberal with regard to drugs that are commonly marketed in spam. Conversely, a list might have a particular subscriber demographic that makes it more sensitive to bad language. Or perhaps different lists might have different primary languages, and therefore different views on the value of messages in that language. So, I can see that different lists on the same system might have different requirements for spam filtering. However, the solution is probably to provide hooks into Spamassassin, or another existing spam solution, and to provide ways that list owners can manage a configuration file on a per list basis. -- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148 From p at sys4.de Fri Apr 19 14:13:22 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Fri, 19 Apr 2013 14:13:22 +0200 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130419121321.GA27855@sys4.de> * Ian Eiloart : > On 15 Apr 2013, at 11:22, Stephen J. Turnbull wrote: > > > OTOH, a generic interface to the Mailman REST API, which could be used > > by Sendmail milters or whatever else is out there and somewhat > > standard (is it Postfix or Exim that can handle milters? I forget), > > with example implementation of a milter that checks whether the poster > > is subscribed by asking Mailman, would be useful as an extension to > > any MTA used with Mailman. > > I think Mailman supports SMTP/LMTP calls to discover whether a sender is permitted to post to a list, doesn't it? > > Exim doesn't handle Milters, but can do the calls forward. Provided Mailman is making the judgement, and issuing L/SMTP rejects at L/SMTP time before accepting the message, Exim is fine. It would be great if Exim as third MTA in the OSS troika of MTAs would. > Content filtering *could* also be done at L/SMTP time. I think that where the Mailman and the MTA installations are managed by the same person or organisation, then the better place to have content filtering performed is at the MTA, but there might be exceptions to this. > > For example, a medical mailing list might want to be more liberal with regard to drugs that are commonly marketed in spam. Conversely, a list might have a particular subscriber demographic that makes it more sensitive to bad language. Or perhaps different lists might have different primary languages, and therefore different views on the value of messages in that language. ACK > So, I can see that different lists on the same system might have different requirements for spam filtering. However, the solution is probably to provide hooks into Spamassassin, or another existing spam solution, and to provide ways that list owners can manage a configuration file on a per list basis. ACK. SpamAssassin knows how to apply different policies per recipient(domain). It is possible to provide policies via file config, SQL or LDAP. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From richard at NFSNet.org Fri Apr 19 14:18:10 2013 From: richard at NFSNet.org (Richard Wackerbarth) Date: Fri, 19 Apr 2013 07:18:10 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <20130418212013.79100b8a@anarchist> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <51704AAD.90300@zone12.com> <20130418212013.79100b8a@anarchist> Message-ID: <2A819EC3-EA60-4E49-B5F9-32328F72AC1E@NFSNet.org> Barry, Are you starting to come around to the concept that I have been advocating for a long time? In particular, rather than "owning" the user information, "core" simply views it. I assume that you are reluctant to store "foreign keys" to external databases because you worry that consistency might be broken when another process alters the database. I ask that you consideration the following as it applies to that view: First, as viewed from the other side (the enterprise, etc.), your same concerns apply to accessing their "user". For example, when you store the user password, how do they assure availability and consistency? Second, particularly if you separate the administration functions, "core" message handling should, at most, rarely care about the "user" information. The subscription information, which seldom changes, should contain sufficient information to handle the time-critical task of message dispatch. Richard On Apr 18, 2013, at 8:20 PM, Barry Warsaw wrote: > Terri, thanks for kicking off this discussion! > > On Apr 18, 2013, at 01:34 PM, Terri Oda wrote: > >> On 13-04-18 2:28 AM, Stephen J. Turnbull wrote: >>> Main comment: Sounds like LDAP to me. > >> Actually, this is a really important comment. I was sort of wondering that >> too when I started writing the description. LDAP is a moderately frequently >> requested feature already. Would it make sense to use that probably rather >> than rest, or make a rest framework that talked to ldap under the hood? > > I agree that this is a really interesting suggestion. > > What's interesting to me about it is that this acknowledges that > administrative control of this extra user information may fall to folks not at > all directly involved in mailing list administration. > > You can imagine that a company would already have their LDAP database of > additional user information, from postal addresses, to phone numbers, to IRC > handles, pictures and icons, birthdays, etc. Further, I can imagine that if > that company were to be running MM3 and HyperKitty, they'd probably not want > to duplicate that information in multiple databases, they'd just want to pull > the pictures and IRC nicks out of LDAP. For their purposes, their LDAP > database *is* their user database. > > Of course, some of that information may also be used in part or in whole for > the core, e.g. display names (which actually are useful in places, such as > crafting the From header for a virgin notification). > > Now, at least conceptually, I've always thought about the user database as one > of the three pillars of information in the core, the other two being the list > database and the message store. This is why for example subscriptions do not > just link a foreign key for the mlist to a foreign key to the address, because > this would not allow for separation of these two pillars[1]. > > Furthermore, the use of interfaces internally is meant to allow for > alternative implementations of the lowest database layer of these pillars, > without affecting any functionality above it. So let's say even in the core, > user passwords, display names, and preferences were kept in LDAP. An > alternative implementation of user.py would pull the data in from there and as > long as it satisfied the syntactic and semantic APIs of the IUser interface, > everything else in MM3 would continue to work. > > You could extend this to the idea expressed elsewhere that the core could use > this external user database. Why not? The external user database could be > accessible over REST or a via a database server, etc. We'd need an > implementation of the underlying data objects that could talk to this user > database and satisfy the interfaces, and that should be all you'd need. > Making it performant enough for the core is just an engineering exercise > . > > -Barry > > [1] Whether any of this works or has the right separation is another matter > entirely. ;) For example, looking at it now, I'm not so sure Members > shouldn't be part of the MailingList pillar rather than the user database > pillar. > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From mark at msapiro.net Fri Apr 19 15:21:09 2013 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 19 Apr 2013 06:21:09 -0700 Subject: [Mailman-Developers] Build a mailman3 development virtualmachine with Vagrant and Chef In-Reply-To: Message-ID: Xu Wang wrote: >I'm not sure I have the permission to edit the wiki page. If anyone with >the permission and know how, please feel free to add the link. As it says at the upper left of the wiki home (dashboard) page "to add or edit content you must sign up and log in, and you must also request write permission for your user name by sending a note to the Mailman Steering Committee. (sorry it's the only way to control wiki spam)." You now have permission. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Fri Apr 19 16:30:44 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 19 Apr 2013 10:30:44 -0400 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <87vc7j1f74.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20130418194006.GE19472@sys4.de> <51704EDD.6040001@zone12.com> <20130418203221.GH19472@sys4.de> <20130418200450.524b8f69@anarchist> <87vc7j1f74.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130419103044.640c84c6@anarchist> On Apr 19, 2013, at 10:28 AM, Stephen J. Turnbull wrote: >Barry Warsaw writes: > > > I still think this wouldn't be a handler, but a rule. > >Is this distinction implemented and enforced by the API? It is. There are two distinct interfaces, IHandler and IRule, with different signatures. It's also enforced by design and implementation, since rules are generally attached to actions via links in a chain and get executed during incoming processing, whereas handlers are processed sequentially without interruption, and only after the message has been accepted for posting. >If not, it's going to be hard to persuade myself to make the distinction in >discussion. Ie, I'll probably just go on calling everything a Handler unless >somebody asks "don't you really mean a Rule?" There are several ways to think about it, but I guess the easiest one is "does this chunk of processing change the message or make a non-destructive moderation decision?" Or more simply, "is it moderation or modification?" If it's moderation, then it's a rule. If it's modification then it's a handler. (There are some corner cases in each, but it's a good general distinction.) Thus a chunk that asks "does this message contain administrivia?" is a moderation question and thus a rule. A chunk that adds an informative footer to the message is a modification handler. Aside: I wanted to find a better name than "handler" for the modification bits, but I failed. I would have rather done away with the term "handler" in MM3 so suggestions are welcome for the term naming the units that plug into the modification pipeline. The interfaces are important too because MM3 initialization will auto-discover both rules and handlers and stick them in different collections. If we ever expose this in the REST API, then you would be able to compose them in different ways, appropriate for the interface. Hope that helps. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Fri Apr 19 16:39:40 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 19 Apr 2013 10:39:40 -0400 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130419103940.774a633f@anarchist> On Apr 19, 2013, at 11:48 AM, Ian Eiloart wrote: >I think Mailman supports SMTP/LMTP calls to discover whether a sender is >permitted to post to a list, doesn't it? MM3's LMTP server currently only does a limited sanity check on the messages. E.g. does the To: field name an existing mailing list[1] >Exim doesn't handle Milters, but can do the calls forward. Provided Mailman >is making the judgement, and issuing L/SMTP rejects at L/SMTP time before >accepting the message, Exim is fine. As a side note, right now only Postfix is officially supported, mostly because that's what I use so I can easily debug it. I would love to have contributions to support at least Exim and Sendmail out of the box. If you're an expert willing to contribute that code, please get in touch. >Content filtering *could* also be done at L/SMTP time. I think that where the >Mailman and the MTA installations are managed by the same person or >organisation, then the better place to have content filtering performed is at >the MTA, but there might be exceptions to this. Currently, I'm trying to keep the processing that the LMTP server does at acceptance time to a minimum, just because I'm concerned about its single threaded performance. While it does async I/O, and it runs in a separate process, time consuming processing for a single message will still block acceptance of all other messages. The answer to this is to somehow multiplex the LMTP server, but ideally without using multiple threads (MM3 is currently single threaded everywhere). In any case, this would also be interesting to work on. -Barry [1] I just noticed https://bugs.launchpad.net/mailman/+bug/1170726 From barry at list.org Fri Apr 19 16:57:28 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 19 Apr 2013 10:57:28 -0400 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <2A819EC3-EA60-4E49-B5F9-32328F72AC1E@NFSNet.org> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <51704AAD.90300@zone12.com> <20130418212013.79100b8a@anarchist> <2A819EC3-EA60-4E49-B5F9-32328F72AC1E@NFSNet.org> Message-ID: <20130419105728.760144f7@anarchist> On Apr 19, 2013, at 07:18 AM, Richard Wackerbarth wrote: >Are you starting to come around to the concept that I have been advocating >for a long time? It's always been part of my thinking, and it's most evident in the use of interfaces internally. Time will tell whether we've accomplished that or not. However, it's also important to realize that probably the vast majority of users who roll their own just install what we give them and go (well, most likely with their stack of distro patches). So I think in that case, it still makes sense for the core to provide, by default, a minimally useful user database, as it does today. >I ask that you consideration the following as it applies to that view: First, >as viewed from the other side (the enterprise, etc.), your same concerns >apply to accessing their "user". For example, when you store the user >password, how do they assure availability and consistency? It's a great question, which I cannot answer. My opinion is that we have to let such enterprise users drive that development, while we give them an architecture that supports their use cases. >Second, particularly if you separate the administration functions, "core" >message handling should, at most, rarely care about the "user" >information. The subscription information, which seldom changes, should >contain sufficient information to handle the time-critical task of message >dispatch. Our users (specifically IUser) is extremely minimal. A user consists of a display name, a password[1], an id, and a "created on" date. There are some additional methods or properties which are just convenient APIs for queries and membership updates, but really it's not much more than a way to collate a set of registered addresses under one unit. -Barry [1] With Persona, we can *probably* get rid of the password. It's only going to be used to do authenticated email commands, but it sucks for security. It's also probably true that few people actually use the email command interface - heck even I rarely use it. It's important to keep, but I think it would be better to base that on OpenPGP rather than plain text passwords. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Fri Apr 19 17:06:04 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 19 Apr 2013 11:06:04 -0400 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> Message-ID: <20130419110604.24bf286f@anarchist> My main suggestion for now is to be very careful and not over-engineer the user database component. Provide something minimal that fits the bill and has a minimum of security, e.g. basic-auth over localhost, and possibly https. For now, I think it would be fine as a Django app if that makes things easier, but also remember how much pain we had at the sprint trying to get Postorius and HyperKitty deployed together (how's that coming by the way?). OTOH, do the easiest thing that will allow our GSoC students to succeed but that doesn't box us in later. E.g. providing a REST API makes sense, and it's okay if there aren't fancy UI to change the schema (unless it's easy using Django). Eventually OAuth is a good idea and I'm not aware of anything else that fits the bill as well, for authenticated scripting of REST APIs. But we probably don't need it for now. One important requirement is that for any data that is kept in both the core and the user database, we must have a way of keeping them in sync. The easiest way of doing that I think is to allow two way communication between them so that if data changes in the core (e.g. an address gets verified by reply instead of link-click), the core can inform the user database of this event. Eventually, we can think about how the core would just share that information or delegate to the user database, but for now, and for the GSoC students, it's probably overkill. -Barry From barry at list.org Fri Apr 19 17:12:03 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 19 Apr 2013 11:12:03 -0400 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <51701EB1.1080804@zone12.com> References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> <51701EB1.1080804@zone12.com> Message-ID: <20130419111203.1febe3b1@anarchist> On Apr 18, 2013, at 10:26 AM, Terri Oda wrote: >I thinks case, case we have to be *much* more conservative about >dependencies. I think Django would be right out as a dependency for Mailman >core, for example. Plus, we're going to have to care a lot more about speed >and all. Definitely. However (and I'm not saying it has to be this way ;) if the user database were a Django app, and there were REST APIs that the core could call to do updates, then when something in the core changes, an event would get triggered that would make the RESTish call to update the user database. Similarly, when something in the user database changes that the core needs to know about, it would call the core's REST API to make that change. We would have to design something that would be robust enough to deal with failures, but I think it could be done. >2. The user module reads from mm-core and augments it. > >This gives us the ability to supplement mm-core without impeding speed. We >discussed possibly filling in the blanks with respect to things like the user >preferences (which are currently set by membership, by user and by email >address... but a lot of those return an empty set when queried if nothing is >set directly there...) so this is maybe something we already want. > >Conceptually, #1 is probably easier because everything will be in one place, >but if we do #2 right, we can make it just as conceptually easy for >HyperKitty/Postorius/etc. without impeding Barry's core dev at all. That >does mean in case 2 that Mailman Extraneous User Stuff is going to depend on >Mailman Core, though. Which I think is fine for now. Without the core, what do you have? :) >a) It doesn't add any dependencies to Mailman core. >b) It doesn't require big changes in Mailman Core. Given that core is pretty >much ready to release, now would be a bad time for changes, and I'm just not >sure we can justify that amount of work for the types of features that will >be built on the extraneous user stuff. >c) It will be much easier to rip this out and replace it when we better >understand our actual needs. (e.g. Right now, I think a case could be made >that a quick mock-up in django would be fine, but I suspect that requiring >django for some potential applications would border on ridiculous) >We're probably going to be running around with a bit of a hack job for the >user database in the near future (either done by a student who needs it in a >hurry or done by one of the core devs to support a student who needs it in a >hurry) so while I don't like to design on the assumption it's going to go >wrong, I think in this case planning for a redesign might be prudent because >it's pretty clear we don't actually know all our requirements. +1 -Barry From barry at list.org Fri Apr 19 17:12:24 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 19 Apr 2013 11:12:24 -0400 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <6A718BFE-A8FC-4943-AAD6-8DE68BCD7F15@NFSNet.org> <51701EB1.1080804@zone12.com> Message-ID: <20130419111224.173646f0@anarchist> On Apr 18, 2013, at 09:39 PM, Florian Fuchs wrote: >I think in this case it makes sense to spend a more time on the API as >seen from the outside (urls, methods, json responses) than the actual >implementation. If the API is good, it's totally acceptable if the >underlying implementation will be redesigned later. +1 -Barry From barry at list.org Fri Apr 19 17:16:39 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 19 Apr 2013 11:16:39 -0400 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130419111639.249f0241@anarchist> On Apr 19, 2013, at 10:22 AM, Stephen J. Turnbull wrote: > > 2) use some other authentication method. > >What's wrong with HTTPS + Basic Auth? Oh, one thing about HTTPS + Basic Auth. Right now, MM3 core has only one admin user and password for access to its REST API. So it can't be used to authenticate individual users, only the "root" user of the REST API. -Barry From terri at zone12.com Fri Apr 19 23:27:38 2013 From: terri at zone12.com (Terri Oda) Date: Fri, 19 Apr 2013 15:27:38 -0600 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130419004028.GA6902@sys4.de> References: <20130418194006.GE19472@sys4.de> <51704EDD.6040001@zone12.com> <20130418203221.GH19472@sys4.de> <517076BF.1040401@msapiro.net> <20130419004028.GA6902@sys4.de> Message-ID: <5171B6CA.7090603@zone12.com> On 13-04-18 6:40 PM, Patrick Ben Koetter wrote: > If we stop at a programmable interface - which I understand is what > you and Terri suggest - they will not be able to use Mailman in more > complex setups, because integrating additional mail processing > components would require them to program. Oi, I hope that's not what it sounded like I was advocating. I want a fair bit more than just an generic interface. In my ideal world, the deliverable for this project will be "everything a sysadmin needs to integrate SpamBayes, SpamAssassin, and maybe 2 other related mail filtering tools" (e.g. ClamAV?) This includes an installer for each one or at least a set of INSTALL_$SPAM_SOLUTION.TXT files and all the work necessary so that they have about 2 steps that amount to "install the thing" and "enable the thing in mailman by flipping this switch/changing this config option." So I think in that respect, Patrick and I are on the same page. I don't think we're saying exactly "oh, all sysadmins could do this in a few minutes" so much as "is this really worth a whole summer of mentoring time given that most of our mentors could write it in 2 weeks themselves if they had need?" The question is whether it's a viable GSoC project, not so much whether it's a thing that would be useful to someone, someday. *I* still think that there's potential for a good GSoC proposal that includes spam filtering, assuming the student understands that this is more of an integration project (NOT a "build a spam filter!" project) and the student is willing to pair this with some other useful features in order to be sure of having 12 weeks of work. Terri PS - milters sound like something any students interested in this project should read up on and mention in their proposals why they did/didn't choose to go that route. It's a good way to show that you've been paying attention to discussions here! :) From xuwang at gmail.com Sat Apr 20 05:16:11 2013 From: xuwang at gmail.com (Xu Wang) Date: Fri, 19 Apr 2013 20:16:11 -0700 Subject: [Mailman-Developers] Build a mailman3 development virtualmachine with Vagrant and Chef In-Reply-To: References: Message-ID: Thanks Mark. I added a section, "Virtual Machine with Vagrant and Chef instructions" on http://wiki.list.org/display/DEV/Mac+Mailman+development+setup+guide Xu Wang On Fri, Apr 19, 2013 at 6:21 AM, Mark Sapiro wrote: > Xu Wang wrote: > > >I'm not sure I have the permission to edit the wiki page. If anyone with > >the permission and know how, please feel free to add the link. > > > As it says at the upper left of the wiki home (dashboard) page "to add > or edit content you must sign up and log in, and you must also request > write permission for your user name by sending a note to the Mailman > Steering Committee. (sorry it's the only way to control wiki spam)." > > You now have permission. > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > > From stephen at xemacs.org Sat Apr 20 06:15:34 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 20 Apr 2013 13:15:34 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <389AEBEB-AB11-4DF7-87D2-4B899D83734F@DATAPLEX.NET> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <19FB3F9E-6BD9-4AC6-ADFE-06350DA93C3C@NFSNet.org> <87wqrz1fcn.fsf@uwakimon.sk.tsukuba.ac.jp> <389AEBEB-AB11-4DF7-87D2-4B899D83734F@DATAPLEX.NET> Message-ID: <87obd925y1.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > On Apr 18, 2013, at 8:25 PM, Stephen J. Turnbull wrote: > > Richard Wackerbarth writes: > >> Since we consider the user manager to be a part of the MM complex, > >> what have we gained by hiding the underlying credential from the > >> web interface? > > > > Security. See the OAuth 2.0 spec (RFC 6749) which recommends (at > > SHOULD level) this practice. > > RFC 6749 addresses the implementation of an OAuth authorization > system. > > In this context, SHOULD refers to the implementation of this RFC. > > It does not imply that other authorization schemes also need to > meet those same criteria. True, but logically irrelevant: it does not mean that the recommendations of the RFC are irrelevant in the design of authorization schemes. OAuth has reasons for recommending that you avoid passing user- supplied credentials around. These include (among others I haven't the imagination to supply here, I'm sure) (1) privacy of the user (the component the user wants access to may have no need to know who the user is) (2) security of the user credentials (leaking the auth token does not allow an attacker to access all resources the user is authorized for) (3) the accessed component doesn't know the importance of the user credentials (the user may have used the same password in several systems, or the user may have very high privileges in other parts of this system), increasing the risk involved in (2). I see no reason to suppose Mailman is not subject to these considerations. (3) is certainly relevant, as I know that I am simultaneously a site owner, a list owner, a moderator, and a subscriber for some lists. There is a plausible workflow in which a user (who happens to be a list owner) who is browsing the archives is given a subscriber-level token, and when they decide to access a list admin page, they need to re-authenticate to upgrade their authorization to list owner. If they then access a non-admin page, or they time out before accessing another admin page, the owner-level token is downgraded to a subscriber token. (This is like sudo on POSIX systems.) I'm not arguing that Mailman should default to such a workflow, but I think we should be able to support it. Passing around user credentials makes it harder to audit such a workflow. > As for security, exposing the authorization server to direct > Internet access is, in itself, a security weak point. I wouldn't say necessarily "weak", but it certainly expands the attack surface that needs to be defended. That's why I don't want to include an auth server implemented by security amateurs (us!) From stephen at xemacs.org Sat Apr 20 06:22:46 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 20 Apr 2013 13:22:46 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130419103044.640c84c6@anarchist> References: <20130418194006.GE19472@sys4.de> <51704EDD.6040001@zone12.com> <20130418203221.GH19472@sys4.de> <20130418200450.524b8f69@anarchist> <87vc7j1f74.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419103044.640c84c6@anarchist> Message-ID: <87mwst25m1.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > There are several ways to think about it, but I guess the easiest > one is "does this chunk of processing change the message or make a > non-destructive moderation decision?" Or more simply, "is it > moderation or modification?" At least for me, the concept is not a problem. The issue is (mental) duck-typing, that's all. > Aside: I wanted to find a better name than "handler" for the modification > bits, but I failed. I would have rather done away with the term "handler" in > MM3 so suggestions are welcome for the term naming the units that plug into > the modification pipeline. +1 From stephen at xemacs.org Sat Apr 20 06:26:37 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 20 Apr 2013 13:26:37 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130419103940.774a633f@anarchist> References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419103940.774a633f@anarchist> Message-ID: <87li8d25fm.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > I would love to have contributions to support at least Exim and > Sendmail out of the box. If you're an expert willing to contribute > that code, please get in touch. I'm not an Exim expert, but my production[1] system uses Exim. I'm working (slowly) on Mailman 3 integration. Footnotes: [1] It's part of the daily workflow, but high availability is not a requirement. :-) From stephen at xemacs.org Sat Apr 20 06:30:24 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 20 Apr 2013 13:30:24 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <20130419105728.760144f7@anarchist> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <51704AAD.90300@zone12.com> <20130418212013.79100b8a@anarchist> <2A819EC3-EA60-4E49-B5F9-32328F72AC1E@NFSNet.org> <20130419105728.760144f7@anarchist> Message-ID: <87k3nx259b.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > It's also probably true that few people actually use the email command > interface - heck even I rarely use it. Emacs and (some) Debian folks do, though. It's not random, there are whole constituencies (small) out there for this feature. From stephen at xemacs.org Sat Apr 20 06:40:42 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 20 Apr 2013 13:40:42 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <20130419110604.24bf286f@anarchist> References: <516F3C7D.1070509@zone12.com> <20130419110604.24bf286f@anarchist> Message-ID: <87ip3h24s5.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > but also remember how much pain we had at the sprint trying to get Postorius > and HyperKitty deployed together (how's that coming by the way?). I got mail from Yarko, does that help? :-) I hope to get back to that this week, now that I've initialized my classes. Dunno if Yarko has done anything, or if anyone else is working on it. > Eventually OAuth is a good idea and I'm not aware of anything else > that fits the bill as well, for authenticated scripting of REST > APIs. But we probably don't need it for now. We don't *need* anything we don't already have :-), but I'd like to see OAuth and Persona for subscribers. > One important requirement is that for any data that is kept in both the core > and the user database, we must have a way of keeping them in sync. The > easiest way of doing that I think is to allow two way communication between > them so that if data changes in the core (e.g. an address gets verified by > reply instead of link-click), the core can inform the user database of this > event. This isn't going to fly for enterprise usage. I mean, you can inform the enterprise DB, but it's highly unlikely to do anything useful with the information. I think in the core "user" has to be a rather abstract class, which provides links to profile information (perhaps in a Mailman-supplied component, or in an enterprise DB, or both[1]), and to list-related information. Footnotes: [1] UI note: if both, enterprises may want "official" information to be visually distinguished from "unofficial" information. From xuwang at gmail.com Sat Apr 20 10:11:09 2013 From: xuwang at gmail.com (Xu Wang) Date: Sat, 20 Apr 2013 01:11:09 -0700 Subject: [Mailman-Developers] Build a mailman3 development virtualmachine with Vagrant and Chef In-Reply-To: References: Message-ID: I made a sub-page under development home for how to build a VM: http://wiki.list.org/display/DEV/Build+Mailman3+Virtual+Machine+with+Vagrant+and+Chef I also add a link to this page in the GSoC 2013 page, in Getting Started section. I also removed the section I added in the Mac Mailman setup guide page because I think it dose not belong to that page. Xu Wang On Fri, Apr 19, 2013 at 8:16 PM, Xu Wang wrote: > Thanks Mark. > > I added a section, "Virtual Machine with Vagrant and Chef instructions" > on http://wiki.list.org/display/DEV/Mac+Mailman+development+setup+guide > > Xu Wang > > > On Fri, Apr 19, 2013 at 6:21 AM, Mark Sapiro wrote: > >> Xu Wang wrote: >> >> >I'm not sure I have the permission to edit the wiki page. If anyone with >> >the permission and know how, please feel free to add the link. >> >> >> As it says at the upper left of the wiki home (dashboard) page "to add >> or edit content you must sign up and log in, and you must also request >> write permission for your user name by sending a note to the Mailman >> Steering Committee. (sorry it's the only way to control wiki spam)." >> >> You now have permission. >> >> -- >> Mark Sapiro The highway is for gamblers, >> San Francisco Bay Area, California better use your sense - B. Dylan >> >> > From iampratiksarkar at gmail.com Sat Apr 20 14:44:52 2013 From: iampratiksarkar at gmail.com (Pratik Sarkar) Date: Sat, 20 Apr 2013 18:14:52 +0530 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <87li8d25fm.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419103940.774a633f@anarchist> <87li8d25fm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Okay so what should a gsoc student concentrate on for the project? 1.a standardized interface (e.g. MILTER, SMTP/LMTP transport) 2.Handler which delegates to external spam filtering packages 3.A totally new spam filter 4.An interface where users can manually tag "this mail is a spam" (which remain unfiltered) to improve existing spam database. On Sat, Apr 20, 2013 at 9:56 AM, Stephen J. Turnbull wrote: > Barry Warsaw writes: > > > I would love to have contributions to support at least Exim and > > Sendmail out of the box. If you're an expert willing to contribute > > that code, please get in touch. > > I'm not an Exim expert, but my production[1] system uses Exim. I'm > working (slowly) on Mailman 3 integration. > > Footnotes: > [1] It's part of the daily workflow, but high availability is not a > requirement. :-) > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/iampratiksarkar%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From stephen at xemacs.org Sun Apr 21 19:11:23 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 22 Apr 2013 02:11:23 +0900 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419103940.774a633f@anarchist> <87li8d25fm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <874nez24hw.fsf@uwakimon.sk.tsukuba.ac.jp> Pratik Sarkar writes: > Okay so what should a gsoc student concentrate on for the project? Writing the proposal! > 1.a standardized interface (e.g. MILTER, SMTP/LMTP transport) Very important. In the case of a filter proposal, which one is up to you (both are important to Mailman because milter is more flexible, but it's not available in Exim). > 2.Handler which delegates to external spam filtering packages Much preferred to 3 by pretty much all the Mailman developers who have spoken up. > 3.A totally new spam filter IMHO, unless it includes a facility for sending Black Helicopters to shut down spammers "permanently", don't bother. SpamAssassin and SpamBayes are good, and both can be trained -- it's just that users don't want to go to that much effort. I see very little room for a breakthrough (defined as "clearly going to be at least as good SA or SB, and near zero effort to train to be better") in this area by any of the students who have proposed them: clearly, none of them could be called "experts" on spam or on classifiers yet. (And that matters, both are pretty big subjects that will require a lot of weeks to learn about.) If someone really wants to do this, start now and come back next year with a very precise proposal of how you're going to construct the classifier and why it will be a breakthrough (as defined above). It's worth doing -- not only will you get the GSoC, but any degree of success will make you a "name" in the field.[1] > 4.An interface where users can manually tag "this mail is a spam" (which > remain unfiltered) to improve existing spam database. Define "users". For most of the definitions I can think of, though, it's a TAGUI (They Aren't Gonna Use It), so why bother? One of two exceptions is that I could see an *inverse* to tagging spam, where site owners provide the service of running a trainer over the ham in existing archives. Implementing such a trainer is *very* difficult, however, because it's very likely to result in over-training unless very accurately tuned, and it must account for the bias of having no spam in the training corpus. The other exception would be implementing tagging as part of a moderation interface. This is going to be an even weirder corpus, since it's the boundary of spam and ham. It could easily end up "thrashing", ie, picking up small irrelevant differences, and making detection of both spam and ham worse. So this would require a lot of empirical evidence to convince people to use it in production. (Ie, hard, boring work actually looking at spam.) Footnotes: [1] http://www.youtube.com/watch?v=MGhEEuE56cY From raj.abhilash1 at gmail.com Mon Apr 22 08:28:34 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Mon, 22 Apr 2013 11:58:34 +0530 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <20130410093912.GY16254@beskar.mdcc.cx> References: <20130410093912.GY16254@beskar.mdcc.cx> Message-ID: Hi all, Can you tell about who is going to mentor this(OpenPGP integration with mailman) so that I can discuss a few things about the application? Also others can you please give me a few suggestion about proposal on the idea that is discussed in this[1] thread. [1]: http://mail.python.org/pipermail/mailman-developers/2013-April/022675.html Thanks, On Wed, Apr 10, 2013 at 3:09 PM, Joost van Baal-Ili? < joostvb-mailman-developers at mdcc.cx> wrote: > Hi Abhilash Raj, > > Abhilash Raj raj.abhilash1 at gmail.com schreef: > >On Sun, Apr 7, 2013 at 4:47 AM, Daniel Kahn Gillmor > >wrote: > >> On 04/06/2013 06:53 PM, Paul Wise wrote: > >> > On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: > >> > > >> >> I am a undergrad student interested in OpenPGP integration in mailman > >> >> as a GSOC project this summer. > > >> neat, i'm glad to hear it! > > Be aware however: it's not an easy task, as Daniel Kahn Gillmor pointed > out. > > >> > I'm not sure about the scope of your project but you may want to > >> > review some prior efforts: > >> > > >> > http://schleuder2.nadir.org/ > >> > http://www.synacklabs.net/projects/crypt-ml/ > >> > >> see also: > >> > >> http://non-gnu.uvt.nl/mailman-pgp-smime/ > >> http://sels.ncsa.illinois.edu/ > >> > >> > My pet favourite feature from the lurker mail archiver is showing > >> > photos from OpenPGP keys in the archive pages. > >> > >Thanks for these links. I am currently going through these projects to > >figure out the implementation part of the OpenPGP into mailman. Also > trying > >to use the mailman-php-smime patch to figure out how it is implemented. > > The Mailman Secure List Server Patch hasn't been touched since 2010-09. > It's a > patch for mailman-2.1.15, not for the development branch. However, > studying it > will surely give you some inspiration. Some code might be reusable too. > > If you'd like to discuss details of this patch, you're invited to join the > list > at ssls-dev at ulm.ccc.de. > > I'd be glad to help you dealing with the work. > > Bye, > > Joost > > -- > http://mdcc.cx/ x http://ad1810.com/ > -- Abhilash Raj From stephen at xemacs.org Mon Apr 22 10:33:50 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 22 Apr 2013 17:33:50 +0900 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <20130410093912.GY16254@beskar.mdcc.cx> Message-ID: <87r4i3ynf5.fsf@uwakimon.sk.tsukuba.ac.jp> Abhilash Raj writes: > Can you tell about who is going to mentor this(OpenPGP integration with > mailman) I would guess the official mentors are likely to be myself and Wacky (Richard Wackerbarth). Joost isn't official (why not? -- you get a T-shirt! :-) but he has expressed interest and offered help. Lack of a secure through-the-mail channel for several aspects of Mailman is a pain point for many users, though, so I suspect there will be a lot of interest (including suggestions and even code contributions) from non-mentors. I strongly suggest that you keep the discussion on this list for that reason. I will also try to be available on IRC Freenode #mailman as yaseppochi for the next two days (more or less 1am to 1pm UTC), and intermittently after that. Other mentors will probably be there, too. > so that I can discuss a few things about the application? Also > others can you please give me a few suggestion about proposal on > the idea that is discussed in this[1] thread. > > [1]: > http://mail.python.org/pipermail/mailman-developers/2013-April/022675.html > > Thanks, > > > On Wed, Apr 10, 2013 at 3:09 PM, Joost van Baal-Ili? < > joostvb-mailman-developers at mdcc.cx> wrote: > > > Hi Abhilash Raj, > > > > Abhilash Raj raj.abhilash1 at gmail.com schreef: > > >On Sun, Apr 7, 2013 at 4:47 AM, Daniel Kahn Gillmor > > >wrote: > > >> On 04/06/2013 06:53 PM, Paul Wise wrote: > > >> > On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: > > >> > > > >> >> I am a undergrad student interested in OpenPGP integration in mailman > > >> >> as a GSOC project this summer. > > > > >> neat, i'm glad to hear it! > > > > Be aware however: it's not an easy task, as Daniel Kahn Gillmor pointed > > out. > > > > >> > I'm not sure about the scope of your project but you may want to > > >> > review some prior efforts: > > >> > > > >> > http://schleuder2.nadir.org/ > > >> > http://www.synacklabs.net/projects/crypt-ml/ > > >> > > >> see also: > > >> > > >> http://non-gnu.uvt.nl/mailman-pgp-smime/ > > >> http://sels.ncsa.illinois.edu/ > > >> > > >> > My pet favourite feature from the lurker mail archiver is showing > > >> > photos from OpenPGP keys in the archive pages. > > >> > > >Thanks for these links. I am currently going through these projects to > > >figure out the implementation part of the OpenPGP into mailman. Also > > trying > > >to use the mailman-php-smime patch to figure out how it is implemented. > > > > The Mailman Secure List Server Patch hasn't been touched since 2010-09. > > It's a > > patch for mailman-2.1.15, not for the development branch. However, > > studying it > > will surely give you some inspiration. Some code might be reusable too. > > > > If you'd like to discuss details of this patch, you're invited to join the > > list > > at ssls-dev at ulm.ccc.de. > > > > I'd be glad to help you dealing with the work. > > > > Bye, > > > > Joost > > > > -- > > http://mdcc.cx/ x http://ad1810.com/ > > > > > > -- > Abhilash Raj > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/stephen%40xemacs.org > > Security Policy: http://wiki.list.org/x/QIA9 From rkw at dataplex.net Mon Apr 22 13:24:30 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Mon, 22 Apr 2013 06:24:30 -0500 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <87r4i3ynf5.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20130410093912.GY16254@beskar.mdcc.cx> <87r4i3ynf5.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: I echo Stephen's comments. Although I try to lurk on the #mailman channel most of the time, being half a world away from him, I am most likely to be at the keyboard after 1100 UTC and before 0200 UTC. However, I strongly suggest that you begin more specific questions on this mailing list. Richard "Wacky" Wackerbarth On Apr 22, 2013, at 3:33 AM, Stephen J. Turnbull wrote: > Abhilash Raj writes: > >> Can you tell about who is going to mentor this(OpenPGP integration with >> mailman) > > I would guess the official mentors are likely to be myself and Wacky > (Richard Wackerbarth). Joost isn't official (why not? -- you get a > T-shirt! :-) but he has expressed interest and offered help. > > Lack of a secure through-the-mail channel for several aspects of > Mailman is a pain point for many users, though, so I suspect there > will be a lot of interest (including suggestions and even code > contributions) from non-mentors. I strongly suggest that you keep the > discussion on this list for that reason. > > I will also try to be available on IRC Freenode #mailman as yaseppochi > for the next two days (more or less 1am to 1pm UTC), and > intermittently after that. Other mentors will probably be there, too. > >> so that I can discuss a few things about the application? Also >> others can you please give me a few suggestion about proposal on >> the idea that is discussed in this[1] thread. >> >> [1]: >> http://mail.python.org/pipermail/mailman-developers/2013-April/022675.html >> >> Thanks, >> >> >> On Wed, Apr 10, 2013 at 3:09 PM, Joost van Baal-Ili? < >> joostvb-mailman-developers at mdcc.cx> wrote: >> >>> Hi Abhilash Raj, >>> >>> Abhilash Raj raj.abhilash1 at gmail.com schreef: >>>> On Sun, Apr 7, 2013 at 4:47 AM, Daniel Kahn Gillmor >>>> wrote: >>>>> On 04/06/2013 06:53 PM, Paul Wise wrote: >>>>>> On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: >>>>>> >>>>>>> I am a undergrad student interested in OpenPGP integration in mailman >>>>>>> as a GSOC project this summer. >>> >>>>> neat, i'm glad to hear it! >>> >>> Be aware however: it's not an easy task, as Daniel Kahn Gillmor pointed >>> out. >>> >>>>>> I'm not sure about the scope of your project but you may want to >>>>>> review some prior efforts: >>>>>> >>>>>> http://schleuder2.nadir.org/ >>>>>> http://www.synacklabs.net/projects/crypt-ml/ >>>>> >>>>> see also: >>>>> >>>>> http://non-gnu.uvt.nl/mailman-pgp-smime/ >>>>> http://sels.ncsa.illinois.edu/ >>>>> >>>>>> My pet favourite feature from the lurker mail archiver is showing >>>>>> photos from OpenPGP keys in the archive pages. >>>>> >>>> Thanks for these links. I am currently going through these projects to >>>> figure out the implementation part of the OpenPGP into mailman. Also >>> trying >>>> to use the mailman-php-smime patch to figure out how it is implemented. >>> >>> The Mailman Secure List Server Patch hasn't been touched since 2010-09. >>> It's a >>> patch for mailman-2.1.15, not for the development branch. However, >>> studying it >>> will surely give you some inspiration. Some code might be reusable too. >>> >>> If you'd like to discuss details of this patch, you're invited to join the >>> list >>> at ssls-dev at ulm.ccc.de. >>> >>> I'd be glad to help you dealing with the work. >>> >>> Bye, >>> >>> Joost >>> >>> -- >>> http://mdcc.cx/ x http://ad1810.com/ >>> >> >> >> >> -- >> Abhilash Raj >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/stephen%40xemacs.org >> >> Security Policy: http://wiki.list.org/x/QIA9 > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From pabs at debian.org Mon Apr 22 13:29:03 2013 From: pabs at debian.org (Paul Wise) Date: Mon, 22 Apr 2013 19:29:03 +0800 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: Message-ID: On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: > I am a undergrad student interested in OpenPGP integration in mailman as a > GSOC project this summer. Here is a semi-related idea; use OpenPGP instead of passwords for authentication to the web interface, possibly using monkeysphere: http://web.monkeysphere.info/ -- bye, pabs http://wiki.debian.org/PaulWise From rkw at dataplex.net Mon Apr 22 13:39:42 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Mon, 22 Apr 2013 06:39:42 -0500 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: Message-ID: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> Although there might be a place for the use of OpenPGP for identification of users to the WebUI, such a project would not, in itself, be sufficiently complex for a GSoC project. If you are interested in such an effort, it would need to be combined with other (preferably related) aspects of authentication such as identification of submitted email messages. On Apr 22, 2013, at 6:29 AM, Paul Wise wrote: > On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: > >> I am a undergrad student interested in OpenPGP integration in mailman as a >> GSOC project this summer. > > Here is a semi-related idea; use OpenPGP instead of passwords for > authentication to the web interface, possibly using monkeysphere: > > http://web.monkeysphere.info/ > > -- > bye, > pabs > > http://wiki.debian.org/PaulWise > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From iane at sussex.ac.uk Mon Apr 22 17:44:25 2013 From: iane at sussex.ac.uk (Ian Eiloart) Date: Mon, 22 Apr 2013 15:44:25 +0000 Subject: [Mailman-Developers] anti-spam filter In-Reply-To: <20130419103940.774a633f@anarchist> References: <87txn847mh.fsf@uwakimon.sk.tsukuba.ac.jp> <20130415092825.GC15104@sys4.de> <87li8k3xfx.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419103940.774a633f@anarchist> Message-ID: <1040106B-EA10-4714-BD2E-C22344CE3E47@sussex.ac.uk> On 19 Apr 2013, at 15:39, Barry Warsaw wrote: > On Apr 19, 2013, at 11:48 AM, Ian Eiloart wrote: > >> I think Mailman supports SMTP/LMTP calls to discover whether a sender is >> permitted to post to a list, doesn't it? > > MM3's LMTP server currently only does a limited sanity check on the messages. > E.g. does the To: field name an existing mailing list[1] The "To: field"? Does that mean the argument of the "RCPT TO" command in the LMTP session? Or does it mean the "To:" message header? The two aren't necessarily related. And, does it not also check the argument of the "MAIL FROM" command? To ensure that the sender is permitted to send to the list specified in RCPT TO. That check is hugely important. It's what keeps mailing lists spam free. >> Exim doesn't handle Milters, but can do the calls forward. Provided Mailman >> is making the judgement, and issuing L/SMTP rejects at L/SMTP time before >> accepting the message, Exim is fine. > > As a side note, right now only Postfix is officially supported, mostly because > that's what I use so I can easily debug it. I would love to have > contributions to support at least Exim and Sendmail out of the box. If you're > an expert willing to contribute that code, please get in touch. > >> Content filtering *could* also be done at L/SMTP time. I think that where the >> Mailman and the MTA installations are managed by the same person or >> organisation, then the better place to have content filtering performed is at >> the MTA, but there might be exceptions to this. > > Currently, I'm trying to keep the processing that the LMTP server does at > acceptance time to a minimum, just because I'm concerned about its single > threaded performance. That's a very good argument for limiting the checks to the RCPT TO phase. Exim can call forward at MAIL FROM, and reject the message if necessary without ever seeing the message body. > While it does async I/O, and it runs in a separate > process, time consuming processing for a single message will still block > acceptance of all other messages. > > The answer to this is to somehow multiplex the LMTP server, but ideally > without using multiple threads (MM3 is currently single threaded everywhere). > In any case, this would also be interesting to work on. > > -Barry > > [1] I just noticed https://bugs.launchpad.net/mailman/+bug/1170726 > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/iane%40sussex.ac.uk > > Security Policy: http://wiki.list.org/x/QIA9 -- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148 From varunsharmalive at gmail.com Mon Apr 22 20:10:00 2013 From: varunsharmalive at gmail.com (varun sharma) Date: Mon, 22 Apr 2013 23:40:00 +0530 Subject: [Mailman-Developers] mailman query Message-ID: Hi, While working on postorius "Better User Settings Management" for gsoc i came up with few queries that i need to ask from core developers and barry. 1. Is there a way to filter the object of MailmanUser just like django have. (eg: to get distinct list_ids from mm_user.subscriptions) 2. from where can i get the possible values for each preference of mm_user.preferences. Thanks in advance. Thanks Varun Sharma From follybeachris at gmail.com Tue Apr 23 18:26:10 2013 From: follybeachris at gmail.com (Chris Cargile) Date: Tue, 23 Apr 2013 12:26:10 -0400 Subject: [Mailman-Developers] Setting up a VM. In-Reply-To: <20130418201005.7742989a@anarchist> References: <51698325.5090808@zone12.com> <20130418201005.7742989a@anarchist> Message-ID: python did seem to be running processes tying up port 9025 and killing allowed me to succeed beyond those errors but upon re-running the tests, my system always hangs at: test_not_enough_parameters (mailman.rest.tests.test_users.TestLogin) Is there any reason off-hand someone can see why? The log file associated with test -vv is here if anyone would like to see: http://test3.ncmacharleston.org/log423 On Thu, Apr 18, 2013 at 8:10 PM, Barry Warsaw wrote: > On Apr 18, 2013, at 01:10 PM, Chris Cargile wrote: > > >The cause in both cases seems to stem from failing at the SMTPLayer, where > >it shows: [Error98] Address already in use. > > This almost always means that some runner process from a previous test > didn't > exit. I've tried to bulletproof the process management in MM3, but > sometimes > the processes can hang around, keeping ports in use. > > >"mailman conf" shows that smtp port setting is:25. > > Remember though that the test infrastructure pushes a small configuration > on > top of that, which `mailman conf` wouldn't show (that would be an > interesting > enhancement :). Look at src/mailman/testing/testing.cfg for details. In > particular, it uses unprivileged ports for the testing SMTP and LMTP > servers > (9025 and 9024 respectively). > > Use ps to find and kill the processes and the ports will free up. > > Cheers, > -Barry > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/follybeachris%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From stephen at xemacs.org Tue Apr 23 18:54:19 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 24 Apr 2013 01:54:19 +0900 Subject: [Mailman-Developers] Setting up a VM. In-Reply-To: References: <51698325.5090808@zone12.com> <20130418201005.7742989a@anarchist> Message-ID: <87d2tlyypw.fsf@uwakimon.sk.tsukuba.ac.jp> Chris Cargile writes: > test_not_enough_parameters > (mailman.rest.tests.test_users.TestLogin) Either the test or the code is borked. Last I heard (at the sprint), we didn't know why. Comment out that test so that the rest of the test suite can run. From barry at list.org Tue Apr 23 19:21:33 2013 From: barry at list.org (Barry Warsaw) Date: Tue, 23 Apr 2013 13:21:33 -0400 Subject: [Mailman-Developers] Setting up a VM. In-Reply-To: <87d2tlyypw.fsf@uwakimon.sk.tsukuba.ac.jp> References: <51698325.5090808@zone12.com> <20130418201005.7742989a@anarchist> <87d2tlyypw.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130423132133.73ba6fc7@limelight.wooz.org> On Apr 24, 2013, at 01:54 AM, Stephen J. Turnbull wrote: >Chris Cargile writes: > > > test_not_enough_parameters > > (mailman.rest.tests.test_users.TestLogin) > >Either the test or the code is borked. Last I heard (at the sprint), >we didn't know why. Comment out that test so that the rest of the >test suite can run. Yeah, I spent some time looking at this one, but couldn't figure it out. -Barry From raj.abhilash1 at gmail.com Thu Apr 25 00:14:27 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Thu, 25 Apr 2013 03:44:27 +0530 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> Message-ID: Hi all, I made a small list[1] of deliverable for this project and required changes in mailman for it. Can you all please review it and comment on how can it be improved. Also there are two points that I am not able to think on, 1) When a message is decrypted and then passed on between the queues, it creates a security threat for the cleartext message is being held in memory, even for a small time in between the runners. 2) Which one is the best standard to be implemented for encryption/signing of the email? [1]: https://gist.github.com/maxking/5455462 Thanks On Mon, Apr 22, 2013 at 5:09 PM, Richard Wackerbarth wrote: > Although there might be a place for the use of OpenPGP for identification > of users to the WebUI, such a project would not, in itself, be sufficiently > complex for a GSoC project. If you are interested in such an effort, it > would need to be combined with other (preferably related) aspects of > authentication such as identification of submitted email messages. > > > On Apr 22, 2013, at 6:29 AM, Paul Wise wrote: > > > On Sun, Apr 7, 2013 at 5:19 AM, Abhilash Raj wrote: > > > >> I am a undergrad student interested in OpenPGP integration in mailman > as a > >> GSOC project this summer. > > > > Here is a semi-related idea; use OpenPGP instead of passwords for > > authentication to the web interface, possibly using monkeysphere: > > > > http://web.monkeysphere.info/ > > > > -- > > bye, > > pabs > > > > http://wiki.debian.org/PaulWise > > _______________________________________________ > > Mailman-Developers mailing list > > Mailman-Developers at python.org > > http://mail.python.org/mailman/listinfo/mailman-developers > > Mailman FAQ: http://wiki.list.org/x/AgA3 > > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > > > Security Policy: http://wiki.list.org/x/QIA9 > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- Abhilash Raj From stephen at xemacs.org Thu Apr 25 04:05:40 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 25 Apr 2013 11:05:40 +0900 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> Message-ID: <87vc7bxt3f.fsf@uwakimon.sk.tsukuba.ac.jp> Abhilash Raj writes: > I made a small list[1] > [1]: https://gist.github.com/maxking/5455462 I strongly recommend that you put this in your proposal on Melange. The mentors will all see it on the mentors' list that way, and you won't get caught short at deadline when Melange crashes.[1] If you want to keep a public copy of your proposal, that's very cool (and if you're accepted, you *must* keep a blog as well as publicly commit your code so you may as well start now IMO). (Speaking for myself) I don't have a problem with you posting it here, especially if you post only excerpts of new and changed content from your proposal (even though that would duplicate the Melange mail for us mentors). However, you might want to see what Barry (as The Big Boss) and the mentors say about posting your proposal, especially more than once a day; others might be more sensitive to an increased amount of mail. Footnotes: [1] Based on past experience, that's *when* Melange crashes, not "if" Melange crashes. :-/ From stefan.schlott at ulm.ccc.de Thu Apr 25 10:36:47 2013 From: stefan.schlott at ulm.ccc.de (Stefan Schlott) Date: Thu, 25 Apr 2013 10:36:47 +0200 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> Message-ID: <5178EB1F.5080907@ulm.ccc.de> On 25.04.2013 00:14, Abhilash Raj wrote: > 1) When a message is decrypted and then passed on between the queues, it > creates a security threat for the cleartext message is being held in > memory, even for a small time in between the runners. The Mailman server holds the key to decrypt _every_ incoming message. So if the server is compromised, a message temporarily held in memory is the least of your problems :-) Stefan. From dkg at fifthhorseman.net Thu Apr 25 15:35:41 2013 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Thu, 25 Apr 2013 21:35:41 +0800 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <5178EB1F.5080907@ulm.ccc.de> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> Message-ID: <5179312D.7010704@fifthhorseman.net> On 04/25/2013 04:36 PM, Stefan Schlott wrote: > On 25.04.2013 00:14, Abhilash Raj wrote: > >> 1) When a message is decrypted and then passed on between the queues, it >> creates a security threat for the cleartext message is being held in >> memory, even for a small time in between the runners. > > The Mailman server holds the key to decrypt _every_ incoming message. So > if the server is compromised, a message temporarily held in memory is > the least of your problems :-) abhilash might have meant that there is a concern that a decrypted message could be stored *on disk* in one of the queues, not just in memory. This could be a problem if an adversary gets access to the disk and can get access to the backing storage, even after the files have been unlinked from the filesystem (since unlinking files doesn't guarantee removal of all traces from the backing storage). Of course, if the secret key for the list is kept without a passphrase on the same filesystem or on a separate filesystem on the same backing storage, then your risk is elevated to begin with. Abhilash, i don't see any mention in your proposal of how you plan to deal with the secret key material. will there be a way for mailman to use a secret key that is stored in a password-protected form? If so, how? --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1027 bytes Desc: OpenPGP digital signature URL: From raj.abhilash1 at gmail.com Thu Apr 25 21:10:58 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Fri, 26 Apr 2013 00:40:58 +0530 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <5179312D.7010704@fifthhorseman.net> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> Message-ID: On Thu, Apr 25, 2013 at 7:05 PM, Daniel Kahn Gillmor wrote: > On 04/25/2013 04:36 PM, Stefan Schlott wrote: > > On 25.04.2013 00:14, Abhilash Raj wrote: > > > >> 1) When a message is decrypted and then passed on between the queues, it > >> creates a security threat for the cleartext message is being held in > >> memory, even for a small time in between the runners. > > > > The Mailman server holds the key to decrypt _every_ incoming message. So > > if the server is compromised, a message temporarily held in memory is > > the least of your problems :-) > > abhilash might have meant that there is a concern that a decrypted > message could be stored *on disk* in one of the queues, not just in > memory. This could be a problem if an adversary gets access to the disk > and can get access to the backing storage, even after the files have > been unlinked from the filesystem (since unlinking files doesn't > guarantee removal of all traces from the backing storage). > > Of course, if the secret key for the list is kept without a passphrase > on the same filesystem or on a separate filesystem on the same backing > storage, then your risk is elevated to begin with. > > Abhilash, i don't see any mention in your proposal of how you plan to > deal with the secret key material. will there be a way for mailman to > use a secret key that is stored in a password-protected form? If so, how? > > Well I am not quite proficient in cryptography but I tried to answer how could it be done and have updated on the same link[1]. Here is a copy of only that part: One of the biggest issues of any cryptographic procedure is to secure and manage the keys. Firstly for the lists, when the list is created by the owner the keypair is generated by mailman in some time(because when i was trying to create one using gnupg, it asked me to wait for sometime and keep doing some work to get threshold entropy. Although in reality I don't have much idea about how the keys are created, but I am guessing that it somewhere uses the random bits from the memory of the host where key is created and thus required a threshold entropy for the proper randomization of the key. On virtualised Linux systems, this can often be achieved by installing the rng-tools package.) and is stored in the database against the name of the list. It will then be available for download to the subscribers. [python-gnupg][2] also allows one to encrypt/decrypt using the keys that are protected by a paraphrase. Such paraphrase though would then be stored in cleartext format in database. Though this poses a security thread but even if you encrypt and store the paraphrase, you can only slow the process of decryption once the server is compromised since the private-paraphrase-encryption key will also be needed to be stored somewhere on the local disk. The pub-keys added by the users will be stored in different table(having many to one relationship with users) and will be used whenever there is a need to encrypt or verify_signature. [1]: https://gist.github.com/maxking/5455462#key-management > --dkg > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- Abhilash Raj From barry at list.org Thu Apr 25 23:28:32 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 25 Apr 2013 17:28:32 -0400 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <20130410093912.GY16254@beskar.mdcc.cx> <87r4i3ynf5.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130425172832.36da17c6@anarchist> On Apr 22, 2013, at 06:24 AM, Richard Wackerbarth wrote: >I echo Stephen's comments. Although I try to lurk on the #mailman channel >most of the time, being half a world away from him, I am most likely to be at >the keyboard after 1100 UTC and before 0200 UTC. We chatted on #mailman a few days ago, and hopefully it was helpful. In general I'm always on #mailman during working hours UTC-4 (currently), but you will have to ping my nick to get my attention. See the channel topic for details. -Barry From stefan.schlott at ulm.ccc.de Fri Apr 26 14:09:41 2013 From: stefan.schlott at ulm.ccc.de (Stefan Schlott) Date: Fri, 26 Apr 2013 14:09:41 +0200 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <5179312D.7010704@fifthhorseman.net> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> Message-ID: <517A6E85.6030005@ulm.ccc.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 25.04.2013 15:35, Daniel Kahn Gillmor wrote: > abhilash might have meant that there is a concern that a decrypted > message could be stored *on disk* in one of the queues, not just > in memory. Of course, it's a good idea to decrypt the data as late as possible in order to avoid unnecessary mistakes. When does mailman store received messages on disk? I can think of the following: - - swapping. Either you request "non-swappable" memory from your OS (might be tricky in Python), or you encrypt your swap device with a new, randomly generated key on every startup. - - mailinglist archive. You simply shouldn't keep a (decrypted) archive on the server. - - disk queue. I don't remember if mailman persists received (but not yet sent) mails on disk. Addressing the last point, you can either choose to decrypt the mail in a later stage, or (if this is a bad idea for performance reasons) deal with this problem with an adequate system configuration, although this is tricky and certainly error-prone. But I think it could be done by excluding the queue from backup (unless, of course, the backup is encrypted, which you should do anyway) and having an encrypted file system. Stefan. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlF6boUACgkQ/fRK6HX9cHTzSACgm5bbYbTpmQ0PZAL9+VCwvcMR hR8An2dFewlP/w3TJejzST3Fp1f4xD+9 =in7V -----END PGP SIGNATURE----- From stefan at ploing.de Fri Apr 26 14:21:45 2013 From: stefan at ploing.de (Stefan Schlott) Date: Fri, 26 Apr 2013 14:21:45 +0200 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> Message-ID: <517A7159.9080909@ploing.de> On 25.04.2013 21:10, Abhilash Raj wrote: >> Abhilash, i don't see any mention in your proposal of how you plan to >> deal with the secret key material. will there be a way for mailman to >> use a secret key that is stored in a password-protected form? If so, how? >> >> Well I am not quite proficient in cryptography but I tried to answer how > could it be done and have updated on the same link[1]. Here is a copy of > only that part: > > One of the biggest issues of any cryptographic procedure is to secure and > manage the keys. Firstly for the lists, when the list is created by the > owner the keypair is generated by mailman in some time(because when i was > trying to create one using gnupg, it asked me to wait for sometime and keep > doing some work to get threshold entropy. Although in reality I don't have > much idea about how the keys are created, but I am guessing that it > somewhere uses the random bits from the memory of the host where key is > created and thus required a threshold entropy for the proper randomization > of the key. On virtualised Linux systems, this can often be achieved by > installing the rng-tools package.) and is stored in the database against > the name of the list. It will then be available for download to the May I suggest that mailman doesn't create the list key by itself, but ask the list maintainer to upload a public/private key pair (if no crypto hardware is used, see below)? On a virtualized system, getting real randomness is tricky. > subscribers. [python-gnupg][2] also allows one to encrypt/decrypt using the > keys that are protected by a paraphrase. Such paraphrase though would then > be stored in cleartext format in database. Though this poses a security > thread but even if you encrypt and store the paraphrase, you can only slow > the process of decryption once the server is compromised since the > private-paraphrase-encryption key will also be needed to be stored > somewhere on the local disk. I would distinguish the following two scenarios: 1. The list is not-so-high-sec that you can risk storing the secret key without a password (which is the equivalent to storing the passphrase in the database). 2. Your list has elevated security requirements. In this case, you can use gpg-agent to manage the secret key (and its passphrase). This would require the sysadmin to start the gpg-agent and enter the list's passphrase before firing up mailman (or mailman could queue incoming mails until the secret key becomes available). This would open you the option to have the mailing list's secret key on a hardware token (e.g. the CryptoStick http://shop.kernelconcepts.de/product_info.php?cPath=1_26&products_id=133). Stefan. From barry at list.org Fri Apr 26 20:45:19 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 26 Apr 2013 14:45:19 -0400 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <517A6E85.6030005@ulm.ccc.de> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A6E85.6030005@ulm.ccc.de> Message-ID: <20130426144519.4ffbddfe@anarchist> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On Apr 26, 2013, at 02:09 PM, Stefan Schlott wrote: >- disk queue. I don't remember if mailman persists received (but not >yet sent) mails on disk. > >Addressing the last point, you can either choose to decrypt the mail >in a later stage, or (if this is a bad idea for performance reasons) >deal with this problem with an adequate system configuration, although >this is tricky and certainly error-prone. But I think it could be done >by excluding the queue from backup (unless, of course, the backup is >encrypted, which you should do anyway) and having an encrypted file >system. Yes, Mailman caches the messages and the metadata as it transfers the message from queue to queue. These two pieces of information are what make up the .pck (Python pickle) files in the queue directories, so for example, after the message has been moderated, it lives in a pck file until the modification runner picks it up for processing. One option, which might suck performance-wise, would be to decrypt the message multiple times. Thus the moderation queue runner would decrypt the message if it needs to make moderation decisions based on the encrypted payload (it may not need to though, since a lot can be discerned from the headers and other cleartext information). If it decides that the message is okay to post, it would not store the decrypted message in the queue, but instead the original message with the encrypted payload. The next queue runner would then also have to decrypt the message to do its processing. OTOH, maybe that's all security theater. If the Mailman system's private key is available to an attacker, then having the encrypted message on disk temporarily is probably not going to stop them from decrypting it. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBCAAGBQJRess/AAoJEBJutWOnSwa//rcQALx/p1Ba8a4CZWCzL2FGW+PZ 80mP+prL44VisScEJopqxx2vzCmzRNo8w0uH7UwKc2DQ4Bl8O+LdBoZs3UdZAB/9 dgSxIAMFsy78TnVngif3Ps5gESdQUAuLijkViHJGePcKNDXMYMV4hBzeqKZxCj+Q Y1NxyJLLeuLrt3HEvQy4TAmWFA/r4UGG5QM249orv2iOtXeHlGMD+IUi4pqyolY6 qzK6WirEh+ntGLvsXHuIBSxpidG9UvRe4XmLT7/fVAUO6X5EuTBdk9NgT9d+Pw+Z eslyngqPOf2MvV/wKLzZFytblGFog7pLOkOPbJ1UzI+KxIf8K4LMlEUG5mo2IGY+ 7vOZgsD9dxzJ2kX0uk1SFR4b23jWZhrYwHAC/k03x2l3FoMvdUqb5/9+nf6C+/4K ZyeB+exOD33TkKtTSx5iZ8HEO/1vCsENFESLeZ5M79cXQJKihyRMiAQfHXzQfR65 XZ0lCG4SB3c0QmhBSqWTxdNP01In0YcD0E5S+1JlP7HbCRhKTU0oHy45rMVSwKfC h1luVZe74Ecuy0foL2gcNObJG6GrXEsAUfYXL5TIy8vSff5VuNVyP4j0Xq7pmPxN XzEt0Vyyc3GTrHbBbnkX1gM4W3icxSHCt9mvCDZ8Civ46qR2pJjkTg6laPtHfLWB 02sufu7o47Z3xcGM4rbq =pajO -----END PGP SIGNATURE----- From terri at zone12.com Fri Apr 26 20:55:33 2013 From: terri at zone12.com (Terri Oda) Date: Fri, 26 Apr 2013 12:55:33 -0600 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <20130426144519.4ffbddfe@anarchist> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A6E85.6030005@ulm.ccc.de> <20130426144519.4ffbddfe@anarchist> Message-ID: <517ACDA5.8030100@zone12.com> On 04/26/2013 12:45 PM, Barry Warsaw wrote: > OTOH, maybe that's all security theater. If the Mailman system's private key > is available to an attacker, then having the encrypted message on disk > temporarily is probably not going to stop them from decrypting it. I've been wondering about that... is there any time when the encrypted message on disk would be available but the private key not? - snapshot backups of Mailman queues but not the key - corrupted filesystems - unusual permissions that allow access to the queues but not the key - mailman is only allowed to deal with encrypted messages when someone inserts the key which is stored on another physical device? It's probably best to keep things encrypted as much as possible just in case there is a threat model we're not thinking of, but unless we're doing more to protect the key, I'm not sure we're gaining much. Terri From raj.abhilash1 at gmail.com Fri Apr 26 21:02:02 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Sat, 27 Apr 2013 00:32:02 +0530 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <20130426144519.4ffbddfe@anarchist> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A6E85.6030005@ulm.ccc.de> <20130426144519.4ffbddfe@anarchist> Message-ID: <517ACF2A.4030808@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Everyone was sending singed messages so i thought to send one too ;-), Though my public keys are not available at any key-server. On Saturday 27 April 2013 12:15 AM, Barry Warsaw wrote: > On Apr 26, 2013, at 02:09 PM, Stefan Schlott wrote: > > > - disk queue. I don't remember if mailman persists received (but not > > yet sent) mails on disk. > > > Addressing the last point, you can either choose to decrypt the mail > > in a later stage, or (if this is a bad idea for performance reasons) > > deal with this problem with an adequate system configuration, although > > this is tricky and certainly error-prone. But I think it could be done > > by excluding the queue from backup (unless, of course, the backup is > > encrypted, which you should do anyway) and having an encrypted file > > system. > > Yes, Mailman caches the messages and the metadata as it transfers the message > from queue to queue. These two pieces of information are what make up the > .pck (Python pickle) files in the queue directories, so for example, after the > message has been moderated, it lives in a pck file until the modification > runner picks it up for processing. > One option, which might suck performance-wise, would be to decrypt the message > multiple times. Thus the moderation queue runner would decrypt the message if > it needs to make moderation decisions based on the encrypted payload (it may > not need to though, since a lot can be discerned from the headers and other > cleartext information). If it decides that the message is okay to post, it > would not store the decrypted message in the queue, but instead the original > message with the encrypted payload. The next queue runner would then also > have to decrypt the message to do its processing. I did think about this part but discarded it on base that is it really worth it to decrypt the message multiple times? While talking to Stephen he suggested that keys could be stored in a more secure database than the main database whose permissions are much higher. So accessing the keys from multiple points( once from each queue ) may increase the chances of attacker getting success? > OTOH, maybe that's all security theater. If the Mailman system's private key > is available to an attacker, then having the encrypted message on disk > temporarily is probably not going to stop them from decrypting it. > That always remains the risk that if one part of the server is compromised its easier for the attacker to access other parts but still should we not try to secure both( private key and decrypted message ) so as to increase complication for attacker? > -Barry > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 Thanks Abhilash -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJRes8qAAoJEPVZtmCk10dUwNIH/jsLoEfFHqu6kFpwgkp+vjC+ sTR8f8QYovkARvaAhOSSlgFwCQw9dQnIwzIkitOQCxtdpQMSr4JJPpvw9AaeY/ik /C+IGg18/ypfOA4FxK/T75ZpincxovB+HkTNS0xwTbyhr3/5KfwqYdC6PcF6f/Ea 5Drqsr7QwQO3X+pv30aoDunJ6/th2P1p1PgM2juBUdtpXPwL0FFTa9QkcAoRv9Sx V7e+ofu7nWF6M7dKDP7eYIJDL7oiNJJTSiz+VdiK7FqfgSq6UUMvoTgyd0l2NDZr MSiS8Kq1Hcm/C/RpUOiEuZzTBNw5nPMBx8fKWtcyo6TTrmQNy3mOHCAnCsoT4po= =Lk6Z -----END PGP SIGNATURE----- From terri at zone12.com Fri Apr 26 21:36:31 2013 From: terri at zone12.com (Terri Oda) Date: Fri, 26 Apr 2013 13:36:31 -0600 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <20130419111639.249f0241@anarchist> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> Message-ID: <517AD73F.4010702@zone12.com> So... What have we decided? :) From my fuzzy "I read my email on a plane after delta woke me up at 3am to tell me my flight was cancelled" level of recollection.... The few things I we actually agreed on: - We like the idea of a rest interface. - That interface/API should probably be decided now and relatively permanently - then implementation details can be changed later as necessary (so it doesn't matter if we start with django or whether mailman-user reads from mailman-core or vice versa, as long as it gets done and fulfills the promises of the API) - something something oauth something Can someone sketch out what that REST interface will look like as an actual architectural document that we can comment on? I don't care who does this; we just need somewhere to start, so any subscriber to this list can probably summarize the emails into a useful document. Terri From stefan.schlott at ulm.ccc.de Fri Apr 26 22:00:18 2013 From: stefan.schlott at ulm.ccc.de (Stefan Schlott) Date: Fri, 26 Apr 2013 22:00:18 +0200 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <517ACDA5.8030100@zone12.com> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A6E85.6030005@ulm.ccc.de> <20130426144519.4ffbddfe@anarchist> <517ACDA5.8030100@zone12.com> Message-ID: <517ADCD2.8010506@ulm.ccc.de> On 26.04.2013 20:55, Terri Oda wrote: > I've been wondering about that... is there any time when the encrypted > message on disk would be available but the private key not? As already pointed out, there are (at least) two ways to avoid an unprotected secret key (or the corresponding pass phrase, respectively) on disk: - Keep the passphrase only in RAM (e.g. entering it when starting mailman or by using gpg-agent) - Having the secret key on a smartcard Stefan. From raj.abhilash1 at gmail.com Sat Apr 27 02:00:50 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Sat, 27 Apr 2013 05:30:50 +0530 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <517AD73F.4010702@zone12.com> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> Message-ID: Hi all, I wrote a brief summary[1] of this thread. Its in the form of notes sorted according to participants and a small summary at the end( showing off whatever I could understand reading the thread twice from head to toe ). However I might have misunderstood ( or not understood at all ) or missed a few things, forgive me for that. [1]: https://gist.github.com/maxking/5471211 Thanks, Abhilash On Sat, Apr 27, 2013 at 1:06 AM, Terri Oda wrote: > So... What have we decided? :) > > From my fuzzy "I read my email on a plane after delta woke me up at 3am to > tell me my flight was cancelled" level of recollection.... > > The few things I we actually agreed on: > > - We like the idea of a rest interface. > > - That interface/API should probably be decided now and relatively > permanently > > - then implementation details can be changed later as necessary (so it > doesn't matter if we start with django or whether mailman-user reads from > mailman-core or vice versa, as long as it gets done and fulfills the > promises of the API) > > - something something oauth something > > Can someone sketch out what that REST interface will look like as an > actual architectural document that we can comment on? I don't care who > does this; we just need somewhere to start, so any subscriber to this list > can probably summarize the emails into a useful document. > > Terri > > > > ______________________________**_________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/**mailman/listinfo/mailman-**developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/** > mailman-developers%40python.**org/ > Unsubscribe: http://mail.python.org/**mailman/options/mailman-** > developers/raj.abhilash1%**40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- Abhilash Raj From rkw at dataplex.net Sat Apr 27 06:32:20 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Fri, 26 Apr 2013 23:32:20 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> Message-ID: Abhilash, Thanks for the summary. Let me add a bit about what I think we should present in this interface. First, it should be RESTful and self documenting. The format of information delivered will be controlled by the http Accept: header The standard representation for communication between various modules of the MM system (core, user-manager,webUI, etc.) will be "application/hal+json" https://server.example.com/mailman/list/ABCDEFG/attribute/join_address returns the email address to which subscription requests should be sent. https://server.example.com/mailman/list/ABCDEFG/attributes https://server.example.com/mailman/attribute/posting_address/test_list at example.com would return the URI representing the list https://server.example.com/mailman/attribute/posting_address/test_list at example.com/attribute/join_address might return test_list-join at example.com See https://gist.github.com/hjr3/2289546 (an E-commerce platform specification) to get a representative idea of the json formatting. I envision the simplified model to have the following core resources: user - A person address - An email address each address is owned by one person credential - An authentication binding associating a user with an identifier such as username/password, browserid, or oAuth list - A mailing list. Lists have attributes that apply to the list and also have a set of preferences which supplies defaults for subscription preferences subscription - the binding of an email address to a list. attributes - a dictionary of the parameters that characterize the list preferences - a dictionary of the parameters that apply to a subscription I am suggesting two primary differences from the MM core style of representation. First, rather than having multiple attributes and preferences as "columns" in a table, I would portray them as multiple rows in a table where the attribute names are a key that is used to select the corresponding value. Additionally, I would generalize the grouping of lists into a hierarchical tree that represents the enterprise organization rather than aspects of the internet namespace. By treating the preferences as a table of key-value pairs, we can easily extend the list of elements with plug-in modules without having to change the data handling or display mechanisms. Richard On Apr 26, 2013, at 7:00 PM, Abhilash Raj wrote: > Hi all, > I wrote a brief summary[1] of this thread. Its in the form of notes sorted > according to participants and a small summary at the end( showing off > whatever I could understand reading the thread twice from head to toe ). > However I might have misunderstood ( or not understood at all ) or missed a > few things, forgive me for that. > > [1]: https://gist.github.com/maxking/5471211 > > Thanks, > Abhilash > > > On Sat, Apr 27, 2013 at 1:06 AM, Terri Oda wrote: > >> So... What have we decided? :) >> >> From my fuzzy "I read my email on a plane after delta woke me up at 3am to >> tell me my flight was cancelled" level of recollection.... >> >> The few things I we actually agreed on: >> >> - We like the idea of a rest interface. >> >> - That interface/API should probably be decided now and relatively >> permanently >> >> - then implementation details can be changed later as necessary (so it >> doesn't matter if we start with Django or whether mailman-user reads from >> mailman-core or vice versa, as long as it gets done and fulfills the >> promises of the API) >> >> - something something oauth something >> >> Can someone sketch out what that REST interface will look like as an >> actual architectural document that we can comment on? I don't care who >> does this; we just need somewhere to start, so any subscriber to this list >> can probably summarize the emails into a useful document. >> >> Terri >> >> >> >> ______________________________**_________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/**mailman/listinfo/mailman-**developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: http://www.mail-archive.com/** >> mailman-developers%40python.**org/ >> Unsubscribe: http://mail.python.org/**mailman/options/mailman-** >> developers/raj.abhilash1%**40gmail.com >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > > > > -- > Abhilash Raj > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From stephen at xemacs.org Sat Apr 27 06:45:37 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 13:45:37 +0900 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <517A7159.9080909@ploing.de> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A7159.9080909@ploing.de> Message-ID: <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> Stefan Schlott writes: > 2. Your list has elevated security requirements. In this case, you can > use gpg-agent to manage the secret key (and its passphrase). I don't understand what threat you propose to address in this way. It's true that you can prevent the attacker from getting access to the key (using agent forwarding or a token, it need not be on the exposed host at all), but we're assuming he has access to the host and the Mailman process. At a minimum you need some kind of privilege separation mechanism within Mailman. I'd recommend a postfix-style separate process which does all cryptographic work. But this might be a very large performance hit. From stephen at xemacs.org Sat Apr 27 07:36:58 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 14:36:58 +0900 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <20130426144519.4ffbddfe@anarchist> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A6E85.6030005@ulm.ccc.de> <20130426144519.4ffbddfe@anarchist> Message-ID: <87bo90y1ol.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > OTOH, maybe that's all security theater. If the Mailman system's > private key is available to an attacker, then having the encrypted > message on disk temporarily is probably not going to stop them from > decrypting it. It's worse than that. The attacker doesn't need the key, he just needs to be able to suborn the Mailman process. There is a scenario where the attacker might want access to the key itself, and that's if he wants to use it somewhere else for some reason (ie, to spoof that Mailman server). But I think the primary scenario is that the attacker just wants access to list traffic, and for that the ability to install a "rule" or "handler" is sufficient in the current architecture. I think we should assume that the Mailman host is secure[1], and worry about how Mailman itself provides an attack surface. Footnotes: [1] I know that that assumption is incorrect. Nevertheless, I don't see what Mailman can do about it without a complete redesign starting from the assumption of encrypted messages whose plain text must be exposed as briefly as possible. From stephen at xemacs.org Sat Apr 27 07:51:18 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 14:51:18 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> Message-ID: <87a9oky10p.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > subscription - the binding of an email address to a list. Also preferences are bound here. (This is not the only kind of thing that preferences can be bound to, but experience shows that we need per-subscription preferences.) > First, rather than having multiple attributes and preferences as > "columns" in a table, I would portray them as multiple rows in a > table where the attribute names are a key that is used to select > the corresponding value. I have no clue what that sentence means, but I suppose that you mean that the current representation is something like (in mock-ABNF) member := address fullname *attribute and you want to change that to member := address fullname preferences preferences := *attribute > Additionally, I would generalize the grouping of lists into a > hierarchical tree that represents the enterprise organization > rather than aspects of the internet namespace. Example? > By treating the preferences as a table of key-value pairs, we can > easily extend the list of elements with plug-in modules without > having to change the data handling or display mechanisms. Ah so you don't mean what I wrote above, you want to represent preferences as a table with row = preference-owning-entity att_name att_key ? From dkg at fifthhorseman.net Sat Apr 27 08:01:43 2013 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Sat, 27 Apr 2013 14:01:43 +0800 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <87bo90y1ol.fsf@uwakimon.sk.tsukuba.ac.jp> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A6E85.6030005@ulm.ccc.de> <20130426144519.4ffbddfe@anarchist> <87bo90y1ol.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <517B69C7.5080601@fifthhorseman.net> On 04/27/2013 01:36 PM, Stephen J. Turnbull wrote: > without a complete redesign starting > from the assumption of encrypted messages whose plain text must > be exposed as briefly as possible. At least one project suggests that it may be possible to operate an encrypted mailing list such that the automated remailing daemon does not have any access to the cleartext body of the messages, and the mailing list members don't need to do any key management of other members of the list. SELS does this through some interesting cryptographic techniques, and was actually built on top of unmodified mailman, afaict: http://sels.ncsa.illinois.edu/ If you're interested in looking for ways that mailman could provide list members with message content protection even in the face of an exploitable bug in mailman itself, this might be an interesting approach to consider (e.g. perhaps SELS could be revived and integrated directly). for the record: I have never run an SELS server, and have never read the code. I just think it's an interesting idea. just a thought, --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1027 bytes Desc: OpenPGP digital signature URL: From dkg at fifthhorseman.net Sat Apr 27 08:05:50 2013 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Sat, 27 Apr 2013 14:05:50 +0800 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A7159.9080909@ploing.de> <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <517B6ABE.8030401@fifthhorseman.net> On 04/27/2013 12:45 PM, Stephen J. Turnbull wrote: > Stefan Schlott writes: > > > 2. Your list has elevated security requirements. In this case, you can > > use gpg-agent to manage the secret key (and its passphrase). > > I don't understand what threat you propose to address in this way. > It's true that you can prevent the attacker from getting access to the > key (using agent forwarding or a token, it need not be on the exposed > host at all), but we're assuming he has access to the host and the > Mailman process. If mailman is storing messages on-disk in an encrypted form, Stefan's proposal mitigates the threat of an adversary with offline access to the disk (e.g. in the event of server theft or seizure) -- no additional message content will be revealed if such an adversary scrapes the contents of the disk. --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1027 bytes Desc: OpenPGP digital signature URL: From stephen at xemacs.org Sat Apr 27 08:53:56 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 15:53:56 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> Message-ID: <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> Abhilash Raj writes: > Hi all, > I wrote a brief summary[1] of this thread. You've misinterpreted or mistyped a couple things I wrote: I'm not against OAuth in general, just against Mailman being an OAuth *provider*, or bundling one, because we can't support it properly. Users should get auth stuff from somebody whose primary interest is security stuff. When I write authentication and authorization should be avoided, I don't mean Mailman doesn't authenticate and authorize users. I mean the implementation should be delegated to robust, well-tested modules or external applications (eg, Apache) whereever possible. The last quote needs to be fleshed out. "This practice" refers to not exposing keys and other secrets to the whole application, including cooperating processes. If authentication can be done in one place and an internal session or one-time authorization be granted, that's what should be done, rather than exposing user credentials to other parts of the application to do their own authentication and authorization. In making such a summary, I think it would be better to organize by topic. Eg, a partial outline: REST API for extended user profiles Authorization Trusting local connections HTTP Basic OAuth - Recommended for "outside world" [Florian] - Advocates including an OAuth provider as a non-default, experts-only option [Florian] - Opposes bundling an OAuth provider [Stephen] - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for now? [Stephen] - Don't need an OAuth provider to share authorizations (in fact, at least in OAuth 2.0, providers don't provide sharing at all) [Stephen] - Implementing OAuth provider doesn't provide the benefits of OAuth (ie, add an OAuth provider means users have yet another set of credentials to manage) [Stephen] - OAuth architecture = provider + client + consumer [Richard] - Agrees to Mailman as OAuth consumer, not provider [Richard] - OAuth may be overengineering, at first [Barry] Database schemas Database implementations Wire Protocol etc... Also, in that format it's easy to reorganize: REST API for extended user profiles Authorization Trusting local connections HTTP Basic OAuth - OAuth architecture = provider + client + consumer [Richard] - Use client in Mailman? - Recommended for "outside world" [Florian] - Agrees to Mailman as OAuth consumer, not provider [Richard] - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for now? [Stephen] - OAuth may be overengineering, at first [Barry] - Use provider in Mailman? - Advocates including an OAuth provider as a non-default, experts-only option [Florian] - Opposes bundling an OAuth provider [Stephen, Richard] - Implementing OAuth provider doesn't provide the benefits of OAuth (ie, add an OAuth provider means users have yet another set of credentials to manage) [Stephen] - Don't need an OAuth provider to share authorizations (in fact, at least in OAuth 2.0, providers don't provide sharing at all) [Stephen] Database schemas Database implementations Wire Protocol etc... By the way, don't go out of your way to reorganize what you've already done, except as it's useful to you. Gradually improve it as it helps you to recall discussion. From turnbull at sk.tsukuba.ac.jp Sat Apr 27 09:42:15 2013 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 16:42:15 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> Message-ID: <877gjoxvvs.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > https://server.example.com/mailman/list/ABCDEFG/attribute/join_address > returns the email address to which subscription requests should be > sent. "ABCDEFG" is what? The list? I think the short prefix /mailman/ should be reserved for traditional and anonymous requests. /mailman/rest_api/ or some such for the REST API. > https://server.example.com/mailman/list/ABCDEFG/attributes This is what you mean by self-documenting? Presumably it returns not only the attribute names, but their ranges (types) and docstrings? > https://server.example.com/mailman/attribute/posting_address/test_list at example.com would return the URI representing the list Why have you changed the order of components here? > https://server.example.com/mailman/attribute/posting_address/test_list at example.com/attribute/join_address might return test_list-join at example.com Huh? This isn't a syntax error? From xuwang at gmail.com Sat Apr 27 09:42:50 2013 From: xuwang at gmail.com (Xu Wang) Date: Sat, 27 Apr 2013 00:42:50 -0700 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: For OAuth , you need to implement token based auth in API, as well as a provider service because there is no open OAuth provider service for third party API out there :-( The provider service has to implement application registration, user auth, token validation and distribution (for oauth 1a, access token need to be pushed to or shared with API). In my view, OAuth really belongs to an enterprise system, rather than a specific api or one application. It requires a existing robust web-based auth system on the provider's site. This will also make the API less "independent" because it is depending on the provider service and provider's auth system. With that said, the API could have a plugin that support "token-based" authorization, with an out-of-band token distribution/validation and management service, whatever it would be. Only if there is a strong 3-legged auth use case, should one push to this controversy route. Xu Wang On Fri, Apr 26, 2013 at 11:53 PM, Stephen J. Turnbull wrote: > Abhilash Raj writes: > > > Hi all, > > I wrote a brief summary[1] of this thread. > > You've misinterpreted or mistyped a couple things I wrote: > > I'm not against OAuth in general, just against Mailman being an OAuth > *provider*, or bundling one, because we can't support it properly. > Users should get auth stuff from somebody whose primary interest is > security stuff. > > When I write authentication and authorization should be avoided, I > don't mean Mailman doesn't authenticate and authorize users. I mean > the implementation should be delegated to robust, well-tested modules > or external applications (eg, Apache) whereever possible. > > The last quote needs to be fleshed out. "This practice" refers to not > exposing keys and other secrets to the whole application, including > cooperating processes. If authentication can be done in one place and > an internal session or one-time authorization be granted, that's what > should be done, rather than exposing user credentials to other parts > of the application to do their own authentication and authorization. > > In making such a summary, I think it would be better to organize by > topic. Eg, a partial outline: > > REST API for extended user profiles > Authorization > Trusting local connections > HTTP Basic > OAuth > - Recommended for "outside world" [Florian] > - Advocates including an OAuth provider as a non-default, > experts-only option [Florian] > - Opposes bundling an OAuth provider [Stephen] > - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for > now? [Stephen] > - Don't need an OAuth provider to share authorizations (in fact, > at least in OAuth 2.0, providers don't provide sharing at all) > [Stephen] > - Implementing OAuth provider doesn't provide the benefits of > OAuth (ie, add an OAuth provider means users have yet another > set of credentials to manage) [Stephen] > - OAuth architecture = provider + client + consumer [Richard] > - Agrees to Mailman as OAuth consumer, not provider [Richard] > - OAuth may be overengineering, at first [Barry] > Database schemas > Database implementations > Wire Protocol > etc... > > Also, in that format it's easy to reorganize: > > REST API for extended user profiles > Authorization > Trusting local connections > HTTP Basic > OAuth > - OAuth architecture = provider + client + consumer [Richard] > - Use client in Mailman? > - Recommended for "outside world" [Florian] > - Agrees to Mailman as OAuth consumer, not provider [Richard] > - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for > now? [Stephen] > - OAuth may be overengineering, at first [Barry] > - Use provider in Mailman? > - Advocates including an OAuth provider as a non-default, > experts-only option [Florian] > - Opposes bundling an OAuth provider [Stephen, Richard] > - Implementing OAuth provider doesn't provide the benefits of > OAuth (ie, add an OAuth provider means users have yet another > set of credentials to manage) [Stephen] > - Don't need an OAuth provider to share authorizations (in fact, > at least in OAuth 2.0, providers don't provide sharing at all) > [Stephen] > Database schemas > Database implementations > Wire Protocol > etc... > > By the way, don't go out of your way to reorganize what you've already > done, except as it's useful to you. Gradually improve it as it helps > you to recall discussion. > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/xuwang%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From xuwang at gmail.com Sat Apr 27 10:00:42 2013 From: xuwang at gmail.com (Xu Wang) Date: Sat, 27 Apr 2013 01:00:42 -0700 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Here is my take on the basic system requirements and design issues: System Components: * A RESTful API - a mini-server that servers restful calls. * A persistent store - a schemaless or relaxed datasource, e.g. Mongodb * An authn/authz service to support api authn/authz and account management options for authn: - no auth, open to localhost, off load the AC to clients. - base auth, username/pwd, requires https and minimum client effort. - HMAC auth, requires clients to sign the requests with shared secrets, e.g. oauth1 and AWS S3 auth. Needs out-of-band secretes and token management and distribution. options for authz (privileges are http methods combined with endpoint/scope): - role based, i.e. privileges are associated with role, work with base auth. - owner based, i.e. privileges are associated with user, work with base auth. - token based, i.e. privileges are associated with token, see OAuth and HMAC auth Resource/Data model servered by API: * TBD, means data model changes "as-we-go". A few initial data type should be given as a start point, or as examples: - users - mailing_lists - subscriptions - user_profiles - accounts etc. * data presentation should be HATEOAS enabled. * content-type, applicaion/json, xml, html, etc. * etag should be used to support caching control and concurrency control. * each resource servered by the api may have a simple validation schema, i.e. in some sort of DSL. Implementation Consideration: * Small footprint. * The API mechanism should decoupled with the resource data model to allow maximum flexibility. * Due to the decoupling of API and the resource data model, the API may only have limited support for advanced or customized quires. * It is a "garbage-in, garbage-out" service, i.e. no or minmum data manipulation by the service. E.g. if you post in a clear texted password with user's data, it will stay clear in the database, and return back as plain text when someone gets. * Service oriented, i.e runs as an independent first-class service. * DRY, use other good open source packages whenever possible. * In Python :-) Relations to other system components: * Open... and RESTful On Fri, Apr 26, 2013 at 11:53 PM, Stephen J. Turnbull wrote: > Abhilash Raj writes: > > > Hi all, > > I wrote a brief summary[1] of this thread. > > You've misinterpreted or mistyped a couple things I wrote: > > I'm not against OAuth in general, just against Mailman being an OAuth > *provider*, or bundling one, because we can't support it properly. > Users should get auth stuff from somebody whose primary interest is > security stuff. > > When I write authentication and authorization should be avoided, I > don't mean Mailman doesn't authenticate and authorize users. I mean > the implementation should be delegated to robust, well-tested modules > or external applications (eg, Apache) whereever possible. > > The last quote needs to be fleshed out. "This practice" refers to not > exposing keys and other secrets to the whole application, including > cooperating processes. If authentication can be done in one place and > an internal session or one-time authorization be granted, that's what > should be done, rather than exposing user credentials to other parts > of the application to do their own authentication and authorization. > > In making such a summary, I think it would be better to organize by > topic. Eg, a partial outline: > > REST API for extended user profiles > Authorization > Trusting local connections > HTTP Basic > OAuth > - Recommended for "outside world" [Florian] > - Advocates including an OAuth provider as a non-default, > experts-only option [Florian] > - Opposes bundling an OAuth provider [Stephen] > - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for > now? [Stephen] > - Don't need an OAuth provider to share authorizations (in fact, > at least in OAuth 2.0, providers don't provide sharing at all) > [Stephen] > - Implementing OAuth provider doesn't provide the benefits of > OAuth (ie, add an OAuth provider means users have yet another > set of credentials to manage) [Stephen] > - OAuth architecture = provider + client + consumer [Richard] > - Agrees to Mailman as OAuth consumer, not provider [Richard] > - OAuth may be overengineering, at first [Barry] > Database schemas > Database implementations > Wire Protocol > etc... > > Also, in that format it's easy to reorganize: > > REST API for extended user profiles > Authorization > Trusting local connections > HTTP Basic > OAuth > - OAuth architecture = provider + client + consumer [Richard] > - Use client in Mailman? > - Recommended for "outside world" [Florian] > - Agrees to Mailman as OAuth consumer, not provider [Richard] > - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for > now? [Stephen] > - OAuth may be overengineering, at first [Barry] > - Use provider in Mailman? > - Advocates including an OAuth provider as a non-default, > experts-only option [Florian] > - Opposes bundling an OAuth provider [Stephen, Richard] > - Implementing OAuth provider doesn't provide the benefits of > OAuth (ie, add an OAuth provider means users have yet another > set of credentials to manage) [Stephen] > - Don't need an OAuth provider to share authorizations (in fact, > at least in OAuth 2.0, providers don't provide sharing at all) > [Stephen] > Database schemas > Database implementations > Wire Protocol > etc... > > By the way, don't go out of your way to reorganize what you've already > done, except as it's useful to you. Gradually improve it as it helps > you to recall discussion. > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/xuwang%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From xuwang at gmail.com Sat Apr 27 10:21:23 2013 From: xuwang at gmail.com (Xu Wang) Date: Sat, 27 Apr 2013 01:21:23 -0700 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Here is an example of what it might look like for "user" resource returned by the api (without any auth): curl http://localhost:5000/api/v1/users/517b5560f84a4b13d239fc59/ HTTP/1.0 200 OK Content-Type: application/json Content-Length: 1232 Cache-Control: max-age=20,must-revalidate Expires: Sat, 27 Apr 2013 08:07:04 GMT ETag: 8252e88a72eea8fd4c93aa57435a3857f618d5d1 Last-Modified: Sat, 27 Apr 2013 04:34:40 UTC Date: Sat, 27 Apr 2013 08:03:44 GMT { "_id": "517b5560f84a4b13d239fc59", "_links": { "collection": { "href": "localhost:5000/api/v1/users/", "title": "users" }, "parent": { "href": "localhost:5000/api/v1", "title": "home" }, "self": { "href": "localhost:5000/api/v1/users/517b5560f84a4b13d239fc59/", "title": "user" } }, "created": "Sat, 27 Apr 2013 04:34:40 UTC", "email": "swesgymt at baqlwrzxqfjpofpy.nl", "firstname": "wtegrglnub", "lastname": "bjsbvjrn", "roles": "level_3", "subscriptions": [ { "href": "localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc56/", "options": { "option_0": "qifqk", "option_1": "qyqyx", "option_2": "dirkf", "option_3": "yjrtv", "option_4": "mljew" }, "ref": "517b5560f84a4b13d239fc56", "title": "mailing list" }, { "href": "localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc58/", "options": { "option_0": "aixqy", "option_1": "triwy", "option_2": "aponq", "option_3": "xmorj", "option_4": "szmig" }, "ref": "517b5560f84a4b13d239fc58", "title": "mailing list" }, { "href": "localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc54/", "options": { "option_0": "ltops", "option_1": "bojfl", "option_2": "qjsyl", "option_3": "ndtof", "option_4": "diass" }, "ref": "517b5560f84a4b13d239fc54", "title": "mailing list" } ], "updated": "Sat, 27 Apr 2013 04:34:40 UTC" } and let's follow the last of mailing list link in the users.subscriptions: $ curl -i 'localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc54/' HTTP/1.0 200 OK Content-Type: application/json Content-Length: 711 Cache-Control: max-age=20,must-revalidate Expires: Sat, 27 Apr 2013 08:05:00 GMT ETag: eea6cfa4fc6311a1ea3c5c4189597ab962369d34 Last-Modified: Sat, 27 Apr 2013 04:34:40 UTC Date: Sat, 27 Apr 2013 08:04:40 GMT { "_id": "517b5560f84a4b13d239fc54", "_links": { "collection": { "href": "localhost:5000/api/v1/mlists/", "title": "mlists" }, "parent": { "href": "localhost:5000/api/v1", "title": "home" }, "self": { "href": "localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc54/", "title": "mailing list" } }, "address": "auxriarr at rdrfzfmvluylkegy.ca", "created": "Sat, 27 Apr 2013 04:34:40 UTC", "description": "krtejcbwmedzftdvjwagmbqkkiajubnzezxstahexvkrjncecdwsyfjlbobgjuxevwgflxlnemqtqcjz", "option": { "option_0": "vwmum" }, "owners": [ { "href": "localhost:5000/api/v1/users/517b5560f84a4b13d239fc59/", "name": "wtegrglnub bjsbvjrn", "ref": "517b5560f84a4b13d239fc59", "title": "user" } ], "updated": "Sat, 27 Apr 2013 04:34:40 UTC" } On Sat, Apr 27, 2013 at 1:00 AM, Xu Wang wrote: > Here is my take on the basic system requirements and design issues: > > System Components: > * A RESTful API - > a mini-server that servers restful calls. > * A persistent store - a schemaless or relaxed datasource, e.g. Mongodb > * An authn/authz service to support api authn/authz and account > management > options for authn: > - no auth, open to localhost, off load the AC to clients. > - base auth, > username/pwd, requires https and minimum client effort. > - HMAC auth, > requires clients to sign the requests with shared secrets, e.g. oauth1 and > AWS S3 auth. Needs out-of-band secretes and token management and > distribution. > options for authz (privileges are http methods combined with > endpoint/scope): > - role based, i.e. privileges are associated with role, work > with base auth. > - owner based, i.e. privileges are associated with user, work > with base auth. > - token based, i.e. privileges are associated with token, see > OAuth and HMAC auth > > Resource/Data model servered by API: > * TBD, means data model changes "as-we-go". > A few initial data type should be given as a start point, or as > examples: > - users > - mailing_lists > - subscriptions > - user_profiles > - accounts > etc. > * data presentation should be HATEOAS > enabled. > * content-type, applicaion/json, xml, html, etc. > * etag should be used to support caching control and concurrency > control. > * each resource servered by the api may have a simple validation > schema, i.e. in some sort of DSL. > > Implementation Consideration: > * Small footprint. > * The API mechanism should decoupled with the resource data model to > allow maximum flexibility. > * Due to the decoupling of API and the resource data model, the API > may only have limited support for advanced or customized quires. > * It is a "garbage-in, garbage-out" service, i.e. no or minmum data > manipulation by the service. E.g. if you post in a clear texted password > with user's data, it will stay clear in the database, and return back as > plain text when someone gets. > * Service oriented, i.e runs as an independent first-class service. > * DRY, use other good open source packages whenever possible. > * In Python :-) > > Relations to other system components: > * Open... and RESTful > > > On Fri, Apr 26, 2013 at 11:53 PM, Stephen J. Turnbull wrote: > >> Abhilash Raj writes: >> >> > Hi all, >> > I wrote a brief summary[1] of this thread. >> >> You've misinterpreted or mistyped a couple things I wrote: >> >> I'm not against OAuth in general, just against Mailman being an OAuth >> *provider*, or bundling one, because we can't support it properly. >> Users should get auth stuff from somebody whose primary interest is >> security stuff. >> >> When I write authentication and authorization should be avoided, I >> don't mean Mailman doesn't authenticate and authorize users. I mean >> the implementation should be delegated to robust, well-tested modules >> or external applications (eg, Apache) whereever possible. >> >> The last quote needs to be fleshed out. "This practice" refers to not >> exposing keys and other secrets to the whole application, including >> cooperating processes. If authentication can be done in one place and >> an internal session or one-time authorization be granted, that's what >> should be done, rather than exposing user credentials to other parts >> of the application to do their own authentication and authorization. >> >> In making such a summary, I think it would be better to organize by >> topic. Eg, a partial outline: >> >> REST API for extended user profiles >> Authorization >> Trusting local connections >> HTTP Basic >> OAuth >> - Recommended for "outside world" [Florian] >> - Advocates including an OAuth provider as a non-default, >> experts-only option [Florian] >> - Opposes bundling an OAuth provider [Stephen] >> - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for >> now? [Stephen] >> - Don't need an OAuth provider to share authorizations (in fact, >> at least in OAuth 2.0, providers don't provide sharing at all) >> [Stephen] >> - Implementing OAuth provider doesn't provide the benefits of >> OAuth (ie, add an OAuth provider means users have yet another >> set of credentials to manage) [Stephen] >> - OAuth architecture = provider + client + consumer [Richard] >> - Agrees to Mailman as OAuth consumer, not provider [Richard] >> - OAuth may be overengineering, at first [Barry] >> Database schemas >> Database implementations >> Wire Protocol >> etc... >> >> Also, in that format it's easy to reorganize: >> >> REST API for extended user profiles >> Authorization >> Trusting local connections >> HTTP Basic >> OAuth >> - OAuth architecture = provider + client + consumer [Richard] >> - Use client in Mailman? >> - Recommended for "outside world" [Florian] >> - Agrees to Mailman as OAuth consumer, not provider [Richard] >> - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for >> now? [Stephen] >> - OAuth may be overengineering, at first [Barry] >> - Use provider in Mailman? >> - Advocates including an OAuth provider as a non-default, >> experts-only option [Florian] >> - Opposes bundling an OAuth provider [Stephen, Richard] >> - Implementing OAuth provider doesn't provide the benefits of >> OAuth (ie, add an OAuth provider means users have yet another >> set of credentials to manage) [Stephen] >> - Don't need an OAuth provider to share authorizations (in fact, >> at least in OAuth 2.0, providers don't provide sharing at all) >> [Stephen] >> Database schemas >> Database implementations >> Wire Protocol >> etc... >> >> By the way, don't go out of your way to reorganize what you've already >> done, except as it's useful to you. Gradually improve it as it helps >> you to recall discussion. >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/xuwang%40gmail.com >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > > From rkw at dataplex.net Sat Apr 27 12:45:15 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sat, 27 Apr 2013 05:45:15 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87a9oky10p.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87a9oky10p.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4BEE8046-C21A-4CE1-ADB6-89D056F8D8C0@dataplex.net> On Apr 27, 2013, at 12:51 AM, "Stephen J. Turnbull" wrote: > Richard Wackerbarth writes: > >> subscription - the binding of an email address to a list. > > Also preferences are bound here. (This is not the only kind of thing > that preferences can be bound to, but experience shows that we need > per-subscription preferences.) Complete agreement. > >> First, rather than having multiple attributes and preferences as >> "columns" in a table, I would portray them as multiple rows in a >> table where the attribute names are a key that is used to select >> the corresponding value. > > I have no clue what that sentence means, but I suppose that you mean > that the current representation is something like (in mock-ABNF) > > member := address fullname *attribute > > and you want to change that to > > member := address fullname preferences > preferences := *attribute > >> Additionally, I would generalize the grouping of lists into a >> hierarchical tree that represents the enterprise organization >> rather than aspects of the internet namespace. > > Example? > >> By treating the preferences as a table of key-value pairs, we can >> easily extend the list of elements with plug-in modules without >> having to change the data handling or display mechanisms. > > Ah so you don't mean what I wrote above, you want to represent > preferences as a table with > > row = preference-owning-entity att_name att_key > > ? Correct. But isn't it row = preference-owning-entity att_name att_value ? From rkw at dataplex.net Sat Apr 27 13:03:10 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sat, 27 Apr 2013 06:03:10 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <877gjoxvvs.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <877gjoxvvs.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <59FFC477-C978-4F11-865B-9EB1674346F3@dataplex.net> On Apr 27, 2013, at 2:42 AM, "Stephen J. Turnbull" wrote: > Richard Wackerbarth writes: > >> https://server.example.com/mailman/list/ABCDEFG/attribute/join_address >> returns the email address to which subscription requests should be >> sent. > > "ABCDEFG" is what? The list? Yes. But note that it is some pk provided by the list store. It may not have any recognizable relationship to other characteristics of the list ( such as a "common name" or FQDN ) > > I think the short prefix /mailman/ should be reserved for traditional > and anonymous requests. /mailman/rest_api/ or some such for the > REST API. I don't disagree. I would reserve /.well_known/mailman/ as an access point that delivers the "root" of the REST API tree and, otherwise, make no assumption about the location of it in URI space. > >> https://server.example.com/mailman/list/ABCDEFG/attributes > > This is what you mean by self-documenting? No. This is an access point for the document. If fetched as JSON or XML, it would be machine parseable. If fetched as HTML/text, it would be human readable. > Presumably it returns not only the attribute names, but their ranges (types) and docstrings? > >> https://server.example.com/mailman/attribute/posting_address/test_list at example.com would return the URI representing the list > > Why have you changed the order of components here? Because I don't know the pk for the list. I wish to look it up. > >> https://server.example.com/mailman/attribute/posting_address/test_list at example.com/attribute/join_address might return test_list-join at example.com > > Huh? This isn't a syntax error? Probably is syntactically incorrect. The email address would need to be URL escaped. From stephen at xemacs.org Sat Apr 27 13:18:35 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 20:18:35 +0900 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <517B6ABE.8030401@fifthhorseman.net> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A7159.9080909@ploing.de> <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> <517B6ABE.8030401@fifthhorseman.net> Message-ID: <8738ucxlv8.fsf@uwakimon.sk.tsukuba.ac.jp> Daniel Kahn Gillmor writes: > If mailman is storing messages on-disk in an encrypted form, Stefan's > proposal mitigates the threat of an adversary with offline access to the > disk (e.g. in the event of server theft or seizure) OK, it does that. But in the event of that kind of threat, I think you also need to protect the logs and lists. I guess you can deal with the logs by the simple expedient of not keeping them. From rkw at dataplex.net Sat Apr 27 13:27:55 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sat, 27 Apr 2013 06:27:55 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: I think that we are advocating the same approach. Your example is better tailored to actual MM resources and can be substituted for the one that I referenced. Note: I am "speaking" conceptually. The actual hard design of the details would be the scope of work for a GSoC project. The easiest way to present all of the details is to actually create a reference implementation. :) On Apr 27, 2013, at 3:21 AM, Xu Wang wrote: > Here is an example of what it might look like for "user" resource returned > by the api (without any auth): > > curl http://localhost:5000/api/v1/users/517b5560f84a4b13d239fc59/ > HTTP/1.0 200 OK > Content-Type: application/json > Content-Length: 1232 > Cache-Control: max-age=20,must-revalidate > Expires: Sat, 27 Apr 2013 08:07:04 GMT > ETag: 8252e88a72eea8fd4c93aa57435a3857f618d5d1 > Last-Modified: Sat, 27 Apr 2013 04:34:40 UTC > Date: Sat, 27 Apr 2013 08:03:44 GMT > > { > "_id": "517b5560f84a4b13d239fc59", > "_links": { > "collection": { > "href": "localhost:5000/api/v1/users/", > "title": "users" > }, > "parent": { > "href": "localhost:5000/api/v1", > "title": "home" > }, > "self": { > "href": > "localhost:5000/api/v1/users/517b5560f84a4b13d239fc59/", > "title": "user" > } > }, > "created": "Sat, 27 Apr 2013 04:34:40 UTC", > "email": "swesgymt at baqlwrzxqfjpofpy.nl", > "firstname": "wtegrglnub", > "lastname": "bjsbvjrn", > "roles": "level_3", > "subscriptions": [ > { > "href": > "localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc56/", > "options": { > "option_0": "qifqk", > "option_1": "qyqyx", > "option_2": "dirkf", > "option_3": "yjrtv", > "option_4": "mljew" > }, > "ref": "517b5560f84a4b13d239fc56", > "title": "mailing list" > }, > { > "href": > "localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc58/", > "options": { > "option_0": "aixqy", > "option_1": "triwy", > "option_2": "aponq", > "option_3": "xmorj", > "option_4": "szmig" > }, > "ref": "517b5560f84a4b13d239fc58", > "title": "mailing list" > }, > { > "href": > "localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc54/", > "options": { > "option_0": "ltops", > "option_1": "bojfl", > "option_2": "qjsyl", > "option_3": "ndtof", > "option_4": "diass" > }, > "ref": "517b5560f84a4b13d239fc54", > "title": "mailing list" > } > ], > "updated": "Sat, 27 Apr 2013 04:34:40 UTC" > } > > and let's follow the last of mailing list link in the users.subscriptions: > > $ curl -i 'localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc54/' > > HTTP/1.0 200 OK > Content-Type: application/json > Content-Length: 711 > Cache-Control: max-age=20,must-revalidate > Expires: Sat, 27 Apr 2013 08:05:00 GMT > ETag: eea6cfa4fc6311a1ea3c5c4189597ab962369d34 > Last-Modified: Sat, 27 Apr 2013 04:34:40 UTC > Date: Sat, 27 Apr 2013 08:04:40 GMT > > { > "_id": "517b5560f84a4b13d239fc54", > "_links": { > "collection": { > "href": "localhost:5000/api/v1/mlists/", > "title": "mlists" > }, > "parent": { > "href": "localhost:5000/api/v1", > "title": "home" > }, > "self": { > "href": > "localhost:5000/api/v1/mlists/517b5560f84a4b13d239fc54/", > "title": "mailing list" > } > }, > "address": "auxriarr at rdrfzfmvluylkegy.ca", > "created": "Sat, 27 Apr 2013 04:34:40 UTC", > "description": > "krtejcbwmedzftdvjwagmbqkkiajubnzezxstahexvkrjncecdwsyfjlbobgjuxevwgflxlnemqtqcjz", > "option": { > "option_0": "vwmum" > }, > "owners": [ > { > "href": > "localhost:5000/api/v1/users/517b5560f84a4b13d239fc59/", > "name": "wtegrglnub bjsbvjrn", > "ref": "517b5560f84a4b13d239fc59", > "title": "user" > } > ], > "updated": "Sat, 27 Apr 2013 04:34:40 UTC" > } > > > > On Sat, Apr 27, 2013 at 1:00 AM, Xu Wang wrote: > >> Here is my take on the basic system requirements and design issues: >> >> System Components: >> * A RESTful API - >> a mini-server that servers restful calls. >> * A persistent store - a schemaless or relaxed datasource, e.g. Mongodb >> * An authn/authz service to support api authn/authz and account >> management >> options for authn: >> - no auth, open to localhost, off load the AC to clients. >> - base auth, >> username/pwd, requires https and minimum client effort. >> - HMAC auth, >> requires clients to sign the requests with shared secrets, e.g. oauth1 and >> AWS S3 auth. Needs out-of-band secretes and token management and >> distribution. >> options for authz (privileges are http methods combined with >> endpoint/scope): >> - role based, i.e. privileges are associated with role, work >> with base auth. >> - owner based, i.e. privileges are associated with user, work >> with base auth. >> - token based, i.e. privileges are associated with token, see >> OAuth and HMAC auth >> >> Resource/Data model servered by API: >> * TBD, means data model changes "as-we-go". >> A few initial data type should be given as a start point, or as >> examples: >> - users >> - mailing_lists >> - subscriptions >> - user_profiles >> - accounts >> etc. >> * data presentation should be HATEOAS >> enabled. >> * content-type, applicaion/json, xml, html, etc. >> * etag should be used to support caching control and concurrency >> control. >> * each resource servered by the api may have a simple validation >> schema, i.e. in some sort of DSL. >> >> Implementation Consideration: >> * Small footprint. >> * The API mechanism should decoupled with the resource data model to >> allow maximum flexibility. >> * Due to the decoupling of API and the resource data model, the API >> may only have limited support for advanced or customized quires. >> * It is a "garbage-in, garbage-out" service, i.e. no or minmum data >> manipulation by the service. E.g. if you post in a clear texted password >> with user's data, it will stay clear in the database, and return back as >> plain text when someone gets. >> * Service oriented, i.e runs as an independent first-class service. >> * DRY, use other good open source packages whenever possible. >> * In Python :-) >> >> Relations to other system components: >> * Open... and RESTful >> >> >> On Fri, Apr 26, 2013 at 11:53 PM, Stephen J. Turnbull wrote: >> >>> Abhilash Raj writes: >>> >>>> Hi all, >>>> I wrote a brief summary[1] of this thread. >>> >>> You've misinterpreted or mistyped a couple things I wrote: >>> >>> I'm not against OAuth in general, just against Mailman being an OAuth >>> *provider*, or bundling one, because we can't support it properly. >>> Users should get auth stuff from somebody whose primary interest is >>> security stuff. >>> >>> When I write authentication and authorization should be avoided, I >>> don't mean Mailman doesn't authenticate and authorize users. I mean >>> the implementation should be delegated to robust, well-tested modules >>> or external applications (eg, Apache) whereever possible. >>> >>> The last quote needs to be fleshed out. "This practice" refers to not >>> exposing keys and other secrets to the whole application, including >>> cooperating processes. If authentication can be done in one place and >>> an internal session or one-time authorization be granted, that's what >>> should be done, rather than exposing user credentials to other parts >>> of the application to do their own authentication and authorization. >>> >>> In making such a summary, I think it would be better to organize by >>> topic. Eg, a partial outline: >>> >>> REST API for extended user profiles >>> Authorization >>> Trusting local connections >>> HTTP Basic >>> OAuth >>> - Recommended for "outside world" [Florian] >>> - Advocates including an OAuth provider as a non-default, >>> experts-only option [Florian] >>> - Opposes bundling an OAuth provider [Stephen] >>> - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for >>> now? [Stephen] >>> - Don't need an OAuth provider to share authorizations (in fact, >>> at least in OAuth 2.0, providers don't provide sharing at all) >>> [Stephen] >>> - Implementing OAuth provider doesn't provide the benefits of >>> OAuth (ie, add an OAuth provider means users have yet another >>> set of credentials to manage) [Stephen] >>> - OAuth architecture = provider + client + consumer [Richard] >>> - Agrees to Mailman as OAuth consumer, not provider [Richard] >>> - OAuth may be overengineering, at first [Barry] >>> Database schemas >>> Database implementations >>> Wire Protocol >>> etc... >>> >>> Also, in that format it's easy to reorganize: >>> >>> REST API for extended user profiles >>> Authorization >>> Trusting local connections >>> HTTP Basic >>> OAuth >>> - OAuth architecture = provider + client + consumer [Richard] >>> - Use client in Mailman? >>> - Recommended for "outside world" [Florian] >>> - Agrees to Mailman as OAuth consumer, not provider [Richard] >>> - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for >>> now? [Stephen] >>> - OAuth may be overengineering, at first [Barry] >>> - Use provider in Mailman? >>> - Advocates including an OAuth provider as a non-default, >>> experts-only option [Florian] >>> - Opposes bundling an OAuth provider [Stephen, Richard] >>> - Implementing OAuth provider doesn't provide the benefits of >>> OAuth (ie, add an OAuth provider means users have yet another >>> set of credentials to manage) [Stephen] >>> - Don't need an OAuth provider to share authorizations (in fact, >>> at least in OAuth 2.0, providers don't provide sharing at all) >>> [Stephen] >>> Database schemas >>> Database implementations >>> Wire Protocol >>> etc... >>> >>> By the way, don't go out of your way to reorganize what you've already >>> done, except as it's useful to you. Gradually improve it as it helps >>> you to recall discussion. >>> _______________________________________________ >>> Mailman-Developers mailing list >>> Mailman-Developers at python.org >>> http://mail.python.org/mailman/listinfo/mailman-developers >>> Mailman FAQ: http://wiki.list.org/x/AgA3 >>> Searchable Archives: >>> http://www.mail-archive.com/mailman-developers%40python.org/ >>> Unsubscribe: >>> http://mail.python.org/mailman/options/mailman-developers/xuwang%40gmail.com >>> >>> Security Policy: http://wiki.list.org/x/QIA9 >>> >> >> > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From rkw at dataplex.net Sat Apr 27 13:33:03 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sat, 27 Apr 2013 06:33:03 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1D101B0C-6BB0-420F-A87F-EAC7D20AFDDC@dataplex.net> I, and I think Stephen, are advocating the use of oAuth as a "login" method. Just as BrowserID provides a third party identifier, Google, Twitter, Facebook, etc. provide similar service through oAuth protocols. MM should be configurable to accept those, or other enterprise-provided identifiers. On Apr 27, 2013, at 2:42 AM, Xu Wang wrote: > For OAuth , you need to implement token > based auth in API, as well as a provider service because there is no open > OAuth provider service for third party API out there :-( > > The provider service has to implement application registration, user auth, > token validation and distribution (for oauth 1a, access token need to be > pushed to or shared with API). > > In my view, OAuth really belongs to an enterprise system, rather than a > specific api or one application. It requires a existing robust web-based > auth system on the provider's site. > > This will also make the API less "independent" because it is depending on > the provider service and provider's auth system. > > With that said, the API could have a plugin that support "token-based" > authorization, with an out-of-band token distribution/validation and > management service, whatever it would be. > > Only if there is a strong 3-legged auth use case, should one push to this > controversy route. > > Xu Wang > > > On Fri, Apr 26, 2013 at 11:53 PM, Stephen J. Turnbull wrote: > >> Abhilash Raj writes: >> >>> Hi all, >>> I wrote a brief summary[1] of this thread. >> >> You've misinterpreted or mistyped a couple things I wrote: >> >> I'm not against OAuth in general, just against Mailman being an OAuth >> *provider*, or bundling one, because we can't support it properly. >> Users should get auth stuff from somebody whose primary interest is >> security stuff. >> >> When I write authentication and authorization should be avoided, I >> don't mean Mailman doesn't authenticate and authorize users. I mean >> the implementation should be delegated to robust, well-tested modules >> or external applications (eg, Apache) whereever possible. >> >> The last quote needs to be fleshed out. "This practice" refers to not >> exposing keys and other secrets to the whole application, including >> cooperating processes. If authentication can be done in one place and >> an internal session or one-time authorization be granted, that's what >> should be done, rather than exposing user credentials to other parts >> of the application to do their own authentication and authorization. >> >> In making such a summary, I think it would be better to organize by >> topic. Eg, a partial outline: >> >> REST API for extended user profiles >> Authorization >> Trusting local connections >> HTTP Basic >> OAuth >> - Recommended for "outside world" [Florian] >> - Advocates including an OAuth provider as a non-default, >> experts-only option [Florian] >> - Opposes bundling an OAuth provider [Stephen] >> - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for >> now? [Stephen] >> - Don't need an OAuth provider to share authorizations (in fact, >> at least in OAuth 2.0, providers don't provide sharing at all) >> [Stephen] >> - Implementing OAuth provider doesn't provide the benefits of >> OAuth (ie, add an OAuth provider means users have yet another >> set of credentials to manage) [Stephen] >> - OAuth architecture = provider + client + consumer [Richard] >> - Agrees to Mailman as OAuth consumer, not provider [Richard] >> - OAuth may be overengineering, at first [Barry] >> Database schemas >> Database implementations >> Wire Protocol >> etc... >> >> Also, in that format it's easy to reorganize: >> >> REST API for extended user profiles >> Authorization >> Trusting local connections >> HTTP Basic >> OAuth >> - OAuth architecture = provider + client + consumer [Richard] >> - Use client in Mailman? >> - Recommended for "outside world" [Florian] >> - Agrees to Mailman as OAuth consumer, not provider [Richard] >> - OAuth necessary? Why isn't HTTPS + Basic Auth good enough for >> now? [Stephen] >> - OAuth may be overengineering, at first [Barry] >> - Use provider in Mailman? >> - Advocates including an OAuth provider as a non-default, >> experts-only option [Florian] >> - Opposes bundling an OAuth provider [Stephen, Richard] >> - Implementing OAuth provider doesn't provide the benefits of >> OAuth (ie, add an OAuth provider means users have yet another >> set of credentials to manage) [Stephen] >> - Don't need an OAuth provider to share authorizations (in fact, >> at least in OAuth 2.0, providers don't provide sharing at all) >> [Stephen] >> Database schemas >> Database implementations >> Wire Protocol >> etc... >> >> By the way, don't go out of your way to reorganize what you've already >> done, except as it's useful to you. Gradually improve it as it helps >> you to recall discussion. >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/xuwang%40gmail.com >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From stephen at xemacs.org Sat Apr 27 13:59:23 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 20:59:23 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> Xu Wang writes: > For OAuth , you need to > implement token based auth in API, as well as a provider service > because there is no open OAuth provider service for third party API > out there :-( No, we don't *need* to implement *anything*. We implement what we want to. That's the nature of a volunteer project. I'm recommending against becoming a provider because I think it's too hard to provide the enterprise-strength implementation you focus on. The client side is much easier. > The provider service has to implement application registration, > user auth, token validation and distribution (for oauth 1a, access > token need to be pushed to or shared with API). Yes, if I understand you correctly. > In my view, OAuth really belongs to an enterprise system, rather > than a specific api or one application. It requires a existing > robust web-based auth system on the provider's site. It very much depends on how accurate the identification of the client needs to be. For an open subscription service where the only thing you really need to do is confirm ownership of the subscribed address, any provider the user trusts is good enough. > This will also make the API less "independent" because it is depending on > the provider service and provider's auth system. Why? The whole point of the OAuth protocol is that the authentication is delegated to an OAuth client, which contacts the provider, and if successful returns a token to the customer. The customer then interprets that token as it likes. The tokens have a specific, well-known form, so the authentication subsystem is completely independent of the application API. There does need to be coupling to the UI, to provide a channel to the OAuth provider. > Only if there is a strong 3-legged auth use case, should one push to this > controversy route. I don't see 3-legged auth as the killer app for OAuth here. Rather, I see convenience for subscribers as the main feature, although it might also be useful for moderators and other admins. Steve From rkw at dataplex.net Sat Apr 27 14:02:10 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sat, 27 Apr 2013 07:02:10 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> Message-ID: On Apr 26, 2013, at 11:32 PM, Richard Wackerbarth wrote: > Additionally, I would generalize the grouping of lists into a hierarchical tree that represents the enterprise organization rather than aspects of the internet namespace. Stephen, Barry has introduced a small amount of hierarchy into the grouping of lists. (E.g. The domain) Within that hierarchy, he at least alludes to the delegation of authority to list owners. It is not necessary to have more than a flat collection of lists. In fact, that approach has the advantage that each list can be allowed to have individual configuration parameters. At the present, he has attached the "base web URL" to the email_domain. Instead, I would like to have the ability to set that on a per-list basis. There is no structural constraint that requires lists to share such common values, but I do acknowledge that it is expedient for the system to be able to generate the list specific value by some algorithm that is applied to a class of lists. Rather than defining a specific structure, I would substitute a more generic structure defining collections of lists. This tree could be configured to represent any organizational hierarchy that the installation desires. Various persons could then be granted administrative rights over particular subtrees. Algorithmic constraints can be assigned to a collection insuring that each list therein complies with the enterprise schema. From stephen at xemacs.org Sat Apr 27 14:08:22 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 21:08:22 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <4BEE8046-C21A-4CE1-ADB6-89D056F8D8C0@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87a9oky10p.fsf@uwakimon.sk.tsukuba.ac.jp> <4BEE8046-C21A-4CE1-ADB6-89D056F8D8C0@dataplex.net> Message-ID: <87zjwkw4zt.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > > Ah so you don't mean what I wrote above, you want to represent > > preferences as a table with > > > > row = preference-owning-entity att_name att_key > Correct. But isn't it > > row = preference-owning-entity att_name att_value Yes. From stephen at xemacs.org Sat Apr 27 15:19:35 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 22:19:35 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <59FFC477-C978-4F11-865B-9EB1674346F3@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <877gjoxvvs.fsf@uwakimon.sk.tsukuba.ac.jp> <59FFC477-C978-4F11-865B-9EB1674346F3@dataplex.net> Message-ID: <87y5c4w1p4.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > > "ABCDEFG" is what? The list? > > Yes. But note that it is some pk provided by the list store. "pk" ? > >> https://server.example.com/mailman/attribute/posting_address/test_list at example.com would return the URI representing the list > > > > Why have you changed the order of components here? > > Because I don't know the pk for the list. I wish to look it up. So "attribute/posting_address/test_list at example.com" is a query that means "give me a way to indicate this list"? These URIs are pretty ugly. Surely we can do better? From stefan.schlott at ulm.ccc.de Sat Apr 27 15:22:17 2013 From: stefan.schlott at ulm.ccc.de (Stefan Schlott) Date: Sat, 27 Apr 2013 15:22:17 +0200 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A7159.9080909@ploing.de> <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <517BD109.3020805@ulm.ccc.de> On 27.04.2013 06:45, Stephen J. Turnbull wrote: > > 2. Your list has elevated security requirements. In this case, you can > > use gpg-agent to manage the secret key (and its passphrase). > > I don't understand what threat you propose to address in this way. > It's true that you can prevent the attacker from getting access to the > key (using agent forwarding or a token, it need not be on the exposed > host at all), but we're assuming he has access to the host and the > Mailman process. The gpg-agent approach protects you from all storage-related attacks: - unencrypted backups - physical access to the harddrive - etc. It does not protect from attackers who have access to the contents of the computer's RAM: - raw memory access and scanning for the secret key (requires root) - memory dump via DMA-enabled interfaces (firewire, pc-card, ...) - cold boot attacks Stefan. From rkw at dataplex.net Sat Apr 27 15:34:42 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sat, 27 Apr 2013 08:34:42 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87y5c4w1p4.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <877gjoxvvs.fsf@uwakimon.sk.tsukuba.ac.jp> <59FFC477-C978-4F11-865B-9EB1674346F3@dataplex.net> <87y5c4w1p4.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <21D2DD23-ACA1-4822-8CE4-5D4B706A7595@dataplex.net> On Apr 27, 2013, at 8:19 AM, "Stephen J. Turnbull" wrote: > Richard Wackerbarth writes: > >>> "ABCDEFG" is what? The list? >> >> Yes. But note that it is some pk provided by the list store. > > "pk" ? Primary Key == An identifier of the server's choice that identifies a unique instance of the specified resource. It is important to note that the client CANNOT rely on any particular scheme for mapping other keys to this identifier. >>>> https://server.example.com/mailman/attribute/posting_address/test_list at example.com would return the URI representing the list >>> >>> Why have you changed the order of components here? >> >> Because I don't know the pk for the list. I wish to look it up. > > So "attribute/posting_address/test_list at example.com" is a query that > means "give me a way to indicate this list"? Yes > These URIs are pretty ugly. Surely we can do better? Perhaps we can. But we need to conform to the HAL framework style. As I understand it, the consumer understands the semantics of various links, but the actual links are "discovered" as a part of previous queries and not "constructed". As such, the links depend more on the ease of implementation rather than their "beauty". Perhaps Xu has experience and can enlighten us. From rkw at dataplex.net Sat Apr 27 15:40:42 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sat, 27 Apr 2013 08:40:42 -0500 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: <517BD109.3020805@ulm.ccc.de> References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A7159.9080909@ploing.de> <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> <517BD109.3020805@ulm.ccc.de> Message-ID: I don't think that "we" have the expertise to create a "secure" system. At best, we can adopt good practices and provide an obscured traffic stream. I consider anything more to be beyond the scope of the MM project. On Apr 27, 2013, at 8:22 AM, Stefan Schlott wrote: > On 27.04.2013 06:45, Stephen J. Turnbull wrote: > >>> 2. Your list has elevated security requirements. In this case, you can >>> use gpg-agent to manage the secret key (and its passphrase). >> >> I don't understand what threat you propose to address in this way. >> It's true that you can prevent the attacker from getting access to the >> key (using agent forwarding or a token, it need not be on the exposed >> host at all), but we're assuming he has access to the host and the >> Mailman process. > > The gpg-agent approach protects you from all storage-related attacks: > - unencrypted backups > - physical access to the harddrive > - etc. > > It does not protect from attackers who have access to the contents of > the computer's RAM: > - raw memory access and scanning for the secret key (requires root) > - memory dump via DMA-enabled interfaces (firewire, pc-card, ...) > - cold boot attacks > > > Stefan From stephen at xemacs.org Sat Apr 27 16:15:52 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 27 Apr 2013 23:15:52 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> Message-ID: <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > It is not necessary to have more than a flat collection of lists. I don't know how it will be represented, but we *do* need to support virtual hosting, where the mailman administrator delegates site administration to the owner of the virtual host. > In fact, that approach has the advantage that each list can be > allowed to have individual configuration parameters. At the > present, he has attached the "base web URL" to the > email_domain. Instead, I would like to have the ability to set that > on a per-list basis. I think that's reasonable. > Rather than defining a specific structure, I would substitute a > more generic structure defining collections of lists. This tree > could be configured to represent any organizational hierarchy that > the installation desires. Such flexibility has benefits and costs. How many of our users will need/want this flexibility? (Genuine question. I personally don't want it, so it's hard for me to estimate.) From stephen at xemacs.org Sat Apr 27 17:36:33 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 28 Apr 2013 00:36:33 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <21D2DD23-ACA1-4822-8CE4-5D4B706A7595@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <877gjoxvvs.fsf@uwakimon.sk.tsukuba.ac.jp> <59FFC477-C978-4F11-865B-9EB1674346F3@dataplex.net> <87y5c4w1p4.fsf@uwakimon.sk.tsukuba.ac.jp> <21D2DD23-ACA1-4822-8CE4-5D4B706A7595@dataplex.net> Message-ID: <87vc78vvcu.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > Primary Key == An identifier of the server's choice that identifies > a unique instance of the specified resource. It is important to > note that the client CANNOT rely on any particular scheme for > mapping other keys to this identifier. That's true for resources that were added after the client was written, but there's no reason I can see why we shouldn't make it easy for people to write URIs by hand if they know something about Mailman. > > These URIs are pretty ugly. Surely we can do better? > > Perhaps we can. But we need to conform to the HAL framework style. Well, I haven't studied it carefully, but the examples on Mike Kelly's pages are pretty enough. > As I understand it, the consumer understands the semantics of > various links, but the actual links are "discovered" as a part of > previous queries and not "constructed". As such, the links depend > more on the ease of implementation rather than their "beauty". >From the consumer's point of view, yes, the links are discovered. But the producer constructs them. From xuwang at gmail.com Sat Apr 27 21:38:58 2013 From: xuwang at gmail.com (Xu Wang) Date: Sat, 27 Apr 2013 12:38:58 -0700 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sat, Apr 27, 2013 at 4:59 AM, Stephen J. Turnbull wrote: > Xu Wang writes: > > > For OAuth , you need to > > implement token based auth in API, as well as a provider service > > because there is no open OAuth provider service for third party API > > out there :-( > > No, we don't *need* to implement *anything*. We implement what we > want to. That's the nature of a volunteer project. > Loud and clear. > > I'm recommending against becoming a provider because I think it's too > hard to provide the enterprise-strength implementation you focus on. > The client side is much easier. > Agreed. > > In my view, OAuth really belongs to an enterprise system, rather > > than a specific api or one application. It requires a existing > > robust web-based auth system on the provider's site. > > It very much depends on how accurate the identification of the client > needs to be. For an open subscription service where the only thing > you really need to do is confirm ownership of the subscribed address, > any provider the user trusts is good enough. > The problem is how do you "confirm ownership of the subscribed address" when a request coming with an access token. In another words, unless your are the provider, how do you validate the access token issued by a provider at the resource endpoint? > > This will also make the API less "independent" because it is depending > on > > the provider service and provider's auth system. > > Why? The whole point of the OAuth protocol is that the authentication > is delegated to an OAuth client, which contacts the provider, and if > successful returns a token to the customer. The customer then > interprets that token as it likes. The tokens have a specific, > well-known form, so the authentication subsystem is completely > independent of the application API. > > I may misunderstood you, but OAuth doesn't specify the format or content of the access token. It is left to the implementation of a provider. This is true for both OAuth 1.a and OAuth 2. To control the access of a resource, a provider have to authenticate both client and the resource owner, issue an access token if authenticated resource owner authorized the resource access for a specific client (authenticated by a client id and a secret). When the access token coming with a request at the API, only the provider or an agent of the provider knows what it is. What I'm saying is that if OAuth is supported, mailman will be a part of provider because it holds and protects the user's data. > There does need to be coupling to the UI, to provide a channel to the > OAuth provider. > > > Only if there is a strong 3-legged auth use case, should one push to > this > > controversy route. > > I don't see 3-legged auth as the killer app for OAuth here. Rather, I > see convenience for subscribers as the main feature, although it might > also be useful for moderators and other admins. > > Stev From xuwang at gmail.com Sat Apr 27 21:59:49 2013 From: xuwang at gmail.com (Xu Wang) Date: Sat, 27 Apr 2013 12:59:49 -0700 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <21D2DD23-ACA1-4822-8CE4-5D4B706A7595@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <877gjoxvvs.fsf@uwakimon.sk.tsukuba.ac.jp> <59FFC477-C978-4F11-865B-9EB1674346F3@dataplex.net> <87y5c4w1p4.fsf@uwakimon.sk.tsukuba.ac.jp> <21D2DD23-ACA1-4822-8CE4-5D4B706A7595@dataplex.net> Message-ID: On Sat, Apr 27, 2013 at 6:34 AM, Richard Wackerbarth wrote: > > On Apr 27, 2013, at 8:19 AM, "Stephen J. Turnbull" > wrote: > > > Richard Wackerbarth writes: > > > >>> "ABCDEFG" is what? The list? > >> > >> Yes. But note that it is some pk provided by the list store. > > > > "pk" ? > > Primary Key == An identifier of the server's choice that identifies a > unique instance of the specified resource. > It is important to note that the client CANNOT rely on any particular > scheme for mapping other keys to this identifier. > > >>>> > https://server.example.com/mailman/attribute/posting_address/test_list at example.comwould return the URI representing the list > >>> > >>> Why have you changed the order of components here? > >> > >> Because I don't know the pk for the list. I wish to look it up. > > > > So "attribute/posting_address/test_list at example.com" is a query that > > means "give me a way to indicate this list"? > > Yes > > > These URIs are pretty ugly. Surely we can do better? > > Perhaps we can. But we need to conform to the HAL framework style. > > As I understand it, the consumer understands the semantics of various > links, but the actual links are "discovered" as a part of previous queries > and not "constructed". As such, the links depend more on the ease of > implementation rather than their "beauty". > > Perhaps Xu has experience and can enlighten us. > > The rules of good endpoint is very simple. * For first-class resources collections endpoint: //// e.g. http://list.mailman.org/api/v1/users document endpoint: ///// e.g. http://list.mailman.org/api/v1/users/ID1234567/ * Don't use path as tools of deep query, use query string for query e.g. http://list.mailman.org/api/v1/users/?subscriptions="mailing_list_abc" * Use hyperlinks for references between first-class resources in documents: e.g. curl localhost:5000/api/v1/users/?max_results=2 | python -mjson.tool { "_items": [ { "_id": "517b88e4f84a4b15f756a1af", "_links": { "self": { "href": "localhost:5000/api/v1/users/517b88e4f84a4b15f756a1af/", "title": "user" } }, ... }, { "_id": "517b88e4f84a4b15f756a1b0", "_links": { "self": { "href": "localhost:5000/api/v1/users/517b88e4f84a4b15f756a1b0/", "title": "user" } }, ... ], "_links": { "next": { "href": "localhost:5000/api/v1/users/?max_results=2&page=2", "title": "next page" }, "parent": { "href": "localhost:5000/api/v1", "title": "home" }, "self": { "href": "localhost:5000/api/v1/users/", "title": "users" } } } From xuwang at gmail.com Sat Apr 27 22:18:43 2013 From: xuwang at gmail.com (Xu Wang) Date: Sat, 27 Apr 2013 13:18:43 -0700 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sat, Apr 27, 2013 at 4:27 AM, Richard Wackerbarth wrote: > I think that we are advocating the same approach. > Your example is better tailored to actual MM resources and can be > substituted for the one that I referenced. > Note: I am "speaking" conceptually. The actual hard design of the details > would be the scope of work for a GSoC project. > The easiest way to present all of the details is to actually create a > reference implementation. :) > > Totally agreed. To be clear, what I gave was only some random test data of a testing data schema from an early stage prototype for the purpose of concepture discussion . Neither to be the real use case nor to be a substitution of any existing working components. I started with MySQL, but after playing around a bit, I do think a NoSQL database is a good fit for this project. It will simplify the task quite a bit. From stephen at xemacs.org Sun Apr 28 09:15:19 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 28 Apr 2013 16:15:19 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> Xu Wang writes: > The problem is how do you "confirm ownership of the subscribed address" > when a request coming with an access token. You don't. That was done when the OAuth ID was linked to the address, using the usual 3-step handshake (submit the association, receive an email containing a secret, confirm ownership by replying with the secret). > In another words, unless your are the provider, how do you validate > the access token issued by a provider at the resource endpoint? I'm not sure what you mean. In the security context (as others), "validate" means mostly "check the syntax". But a valid token is assumed to be authentic (otherwise the provided has been suborned). Remember, a provider does not *authorize* anything. A provider offers a channel of communication to the resource owner, who does the authorization. Ie, provider tells owner "client tells me consumer wants to access the X Files", and on success returns a token to the client indicating "I've confirmed that you are indeed talking to the owner, and by the way, *she* says it's OK to let consumer look at them." > > Why? The whole point of the OAuth protocol is that the authentication > > is delegated to an OAuth client, which contacts the provider, and if > > successful returns a token to the customer. The customer then > > interprets that token as it likes. The tokens have a specific, > > well-known form, so the authentication subsystem is completely > > independent of the application API. > > > I may misunderstood you, but OAuth doesn't specify the format or > content of the access token. It is left to the implementation of a > provider. This is true for both OAuth 1.a and OAuth 2. I'd have to look again to be 100% sure, but I'm pretty sure that what the provider returns is well-defined. What the client gives to the application is application-defined. > To control the access of a resource, a provider have to authenticate both > client and the resource owner, issue an access token if authenticated > resource owner authorized the resource access for a specific client > (authenticated by a client id and a secret). When the access token coming > with a request at the API, only the provider or an agent of the provider > knows what it is. Again IIRC, the provider doesn't know what the token is. The *client* does. The provider just passes a message saying "Access to the X Files, which you own, have been requested. Is that OK?" In practice, the common case is that consumer == owner, so effectively the message boils down to, "Click OK if you are you". > What I'm saying is that if OAuth is supported, mailman will be a part of > provider because it holds and protects the user's data. OAuth providers don't know about the protected data. OAuth consumers do. The provider-client combination only offers a communication channel. That's what is so elegant about the OAuth concept. Steve From rkw at dataplex.net Sun Apr 28 14:36:13 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sun, 28 Apr 2013 07:36:13 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> On Apr 27, 2013, at 9:15 AM, Stephen J. Turnbull wrote: > Richard Wackerbarth writes: > >> It is not necessary to have more than a flat collection of lists. > > I don't know how it will be represented, but we *do* need to support > virtual hosting, where the mailman administrator delegates site > administration to the owner of the virtual host. A list is a first class object. Each list has its own set of parameters which characterize it. One portion of those characteristics is the administrative permissions associated with the creation and alteration of the list. Groupings of lists simply provides a "shorthand" for the description of characteristics which are common to the group. >> In fact, that approach has the advantage that each list can be >> allowed to have individual configuration parameters. At the >> present, he has attached the "base web URL" to the >> email_domain. Instead, I would like to have the ability to set that >> on a per-list basis. > > I think that's reasonable. > >> Rather than defining a specific structure, I would substitute a >> more generic structure defining collections of lists. This tree >> could be configured to represent any organizational hierarchy that >> the installation desires. > > Such flexibility has benefits and costs. How many of our users will > need/want this flexibility? (Genuine question. I personally don't > want it, so it's hard for me to estimate.) The advantage in using the generic structure is that it does not impose any pre-supposed structure on the collection of lists. Note that "core" doesn't NEED the structure to function. The structure can be imposed by having the interface agent impose constraints on the members of a group of lists and mapping an operation on the group into that operation on each of its members. If we use a MPTT key as the list/list-group identifier, we get the generic hierarchy as a byproduct. The only real issue is how we would express constraints that get delegated. But we need to address that issue for any structure that we establish. Virtual hosting is the primary example of the need for a hierarchical administrative structure. However, it can also be useful in corporate structures where the email address space might be partitioned in a quite different manner. From a design perspective, the "core" message handler should be distinct from the configuration administrator. Message handling uses the current value of various configuration parameters. It need not, and should not be concerned with the mechanics related to the setting or modification of those parameters. Since these parameters seldom change, an effective caching mechanism would address access performance issues. From rkw at dataplex.net Sun Apr 28 14:54:10 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sun, 28 Apr 2013 07:54:10 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87vc78vvcu.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <877gjoxvvs.fsf@uwakimon.sk.tsukuba.ac.jp> <59FFC477-C978-4F11-865B-9EB1674346F3@dataplex.net> <87y5c4w1p4.fsf@uwakimon.sk.tsukuba.ac.jp> <21D2DD23-ACA1-4822-8CE4-5D4B706A7595@dataplex.net> <87vc78vvcu.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Apr 27, 2013, at 10:36 AM, Stephen J. Turnbull wrote: > Richard Wackerbarth writes: >> Primary Key == An identifier of the server's choice that identifies >> a unique instance of the specified resource. It is important to >> note that the client CANNOT rely on any particular scheme for >> mapping other keys to this identifier. > > That's true for resources that were added after the client was > written, but there's no reason I can see why we shouldn't make it easy > for people to write URIs by hand if they know something about Mailman. Being able to "write URIs by hand" is a violation of the HAL design because it locks the interface into a particular implementation. The design principal is that "things will change". Although there is an advantage in having URIs that convey semantic value, (for example using a "short name" to designate one of the lists on the server), we should leave that mapping to some configuration aspect of the particular installation. Remember that the REST URIs are intended to be mechanisms for automated agents to access resources. They are not intended to be the substitute for a user's search interface. From rkw at dataplex.net Sun Apr 28 15:03:35 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sun, 28 Apr 2013 08:03:35 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Apr 28, 2013, at 2:15 AM, Stephen J. Turnbull wrote: > Xu Wang writes: >> The problem is how do you "confirm ownership of the subscribed address" >> when a request coming with an access token. > > You don't. That was done when the OAuth ID was linked to the address, > using the usual 3-step handshake (submit the association, receive an > email containing a secret, confirm ownership by replying with the secret). In many installations, the linking may not require the email handshake. An installation may choose to "trust" that the third party issuing the access credential has already performed sufficient vetting of the association. I'm thinking of things like BrowserID credentials or Google/Twitter/Facebook issued credentials. However, that is a local "policy" whose decision involves a tradeoff between the level of assurance and the ease in establishing the association. From turnbull at sk.tsukuba.ac.jp Sun Apr 28 18:15:15 2013 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 01:15:15 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> Message-ID: <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > Groupings of lists simply provides a "shorthand" for the > description of characteristics which are common to the group. You don't need to teach Grandma how to suck intensional vs. extensional Python eggs. > > Such flexibility has benefits and costs. How many of our users will > > need/want this flexibility? (Genuine question. I personally don't > > want it, so it's hard for me to estimate.) > > The advantage in using the generic structure is that it does not > impose any pre-supposed structure on the collection of lists. Ie, the advantage of being generic is that it's generic. > Note that "core" doesn't NEED the structure to function. Of course not, the only structure we NEED is a tape that's lightyears long, and a finite automaton that Turing designed in the 40s IIRC. Sorry for the sarcasm (well, a little bit sorry), but in deciding on design principles we need to evaluate simplicity, practicality, etc from the point of view of the people involved. The question is whether structure makes the program easier for users to use and for developers to create and maintain. I think the tried and true hierarchy (site -> virtual hosts -> lists) matches some a natural hierarchy in administrative responsibility and user understanding of this corner of the Web. That makes it easier to think about requirements and write code to satisfy them. > The structure can be imposed by having the interface agent impose > constraints on the members of a group of lists and mapping an > operation on the group into that operation on each of its members. Yes, and we'll have to create a language to express those constraints and an interpreter to enforce them, *and* create objects like "site" == Mailman instance and "vhost" out of the abstract groups. Costs, benefits, which is bigger? > If we use a MPTT key "Ministry of Posts, Telegraph, and Telephone"? "mildly paranoic teletype"? > as the list/list-group identifier, we get the generic hierarchy as > a byproduct. No, that's not enough. Sites and virtual hosts have attributes besides lists (websites, the "mailman" list, owner and admin addresses, ...). There should be objects that correspond to those concepts to carry those attributes. > The only real issue is how we would express constraints that get > delegated. > > But we need to address that issue for any structure that we > establish. True, but it's not a question of need to or not, it's a question of costs and benefits. Generic code is more expensive to create and maintain in many cases. I suspect it will be here, as well. Without use cases for the generality, it's hard to see the benefits. When do you expect the generality to be of use? > Virtual hosting is the primary example of the need for a > hierarchical administrative structure. However, it can also be > useful in corporate structures where the email address space might > be partitioned in a quite different manner. > > From a design perspective, the "core" message handler should be > distinct from the configuration administrator. > > Message handling uses the current value of various configuration > parameters. It need not, and should not be concerned with the > mechanics related to the setting or modification of those > parameters. > > Since these parameters seldom change, an effective caching > mechanism would address access performance issues. I have no clue what you're trying to express here. I don't see any performance issues, except that programmers whose brains have exploded from cognitive overload tend not to produce double-digit KLOC/hour. My concern is entirely whether the design you propose simplifies the job of meeting the common requirement of creating a (1 to 3 level) hierarchical organization of lists (the third level being the children of umbrella lists), or serves important use cases without significantly complexifying the job of serving the common case. I don't see a simplification for the reasons expressed above. What are the use cases for a more generic structure? From stephen at xemacs.org Sun Apr 28 19:39:39 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 02:39:39 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <877gjoxvvs.fsf@uwakimon.sk.tsukuba.ac.jp> <59FFC477-C978-4F11-865B-9EB1674346F3@dataplex.net> <87y5c4w1p4.fsf@uwakimon.sk.tsukuba.ac.jp> <21D2DD23-ACA1-4822-8CE4-5D4B706A7595@dataplex.net> <87vc78vvcu.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87k3nmwo4k.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > Being able to "write URIs by hand" is a violation of the HAL design > because it locks the interface into a particular implementation. Sorry, that's an un-Pythonic way to think. If HAL really requires URIs that only a machine can deal with, let's junk it. "If the implementation is hard to explain, it's a bad idea." But I really don't think HAL requires that. From rkw at dataplex.net Sun Apr 28 21:13:01 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sun, 28 Apr 2013 14:13:01 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> On Apr 28, 2013, at 11:15 AM, Stephen J. Turnbull wrote: > Richard Wackerbarth writes: >> Note that "core" doesn't NEED the structure (of list heiarchy) to function. > in deciding on design principles we need to evaluate simplicity, practicality, etc > from the point of view of the people involved. Agreed. > I think the tried and true hierarchy (site -> virtual hosts -> lists) matches some a > natural hierarchy in administrative responsibility and user understanding of this corner of the Web. That matches ONE organizational hierarchy. There is no reason that you cannot use a more general structure and still follow that organizational structure. > That makes it easier to think about requirements and write code to satisfy them. Not necessarily. >> The structure can be imposed by having the interface agent impose >> constraints on the members of a group of lists and mapping an >> operation on the group into that operation on each of its members. > > Yes, and we'll have to create a language to express those constraints > and an interpreter to enforce them, *and* create objects like "site" > == Mailman instance and "vhost" out of the abstract groups. Costs, > benefits, which is bigger? > >> If we use a MPTT key > > "Ministry of Posts, Telegraph, and Telephone"? "mildly paranoic teletype"? Modified Preorder Tree Traversal > Sites and virtual hosts have attributes besides lists (websites, the "mailman" list, owner and admin > addresses, ...). There should be objects that correspond to those concepts to carry those attributes. I think that I view it differently. EVERY list has, for example, an "admin address". If there is only one list, to the user(administrator) that attribute belongs to the (only) list. It is only when you have a collection of lists that share a common value that a hierarchy adds value. In that case, by defining a default value to the node representing the collection and specifying "inherit from parent" in all of the sub-nodes, you can, administratively simplify the maintenance of the value to be used in each instance. >> The only real issue is how we would express constraints that get delegated. >> >> But we need to address that issue for any structure that we establish. > > True, but it's not a question of need to or not, it's a question of > costs and benefits. Generic code is more expensive to create and > maintain in many cases. I suspect it will be here, as well. Yes, initially generating a more generic structure than the ad hoc one in place (which doesn't even attempt to address delegation) is a bit more expensive for the first few parameters. However, in the log run I suspect that it won't be more expensive. Creating a single, reusable mechanism for the specification of delegation of authority will actually produce an easier-to-maintain scheme. And, ad hoc approaches are notorious for becoming unmaintainable. > Without use cases for the generality, it's hard to see the benefits. When do you expect the generality to be of use? > >> Virtual hosting is the primary example of the need for a >> hierarchical administrative structure. However, it can also be >> useful in corporate structures where the email address space might >> be partitioned in a quite different manner. There is no need to make a distinction between the levels in your list hierarchy tree. Treating each level as a specific case of a generic structure allows reuse of a common code base. >> From a design perspective, the "core" message handler should be >> distinct from the configuration administrator. >> >> Message handling uses the current value of various configuration >> parameters. It need not, and should not be concerned with the >> mechanics related to the setting or modification of those >> parameters. >> >> Since these parameters seldom change, an effective caching >> mechanism would address access performance issues. > > I have no clue what you're trying to express here. This is addressing the modularity of the system, separating functionality and utilizing particular views of the stored data rather than dictating how the data is stored and manipulated. > I don't see any performance issues, except that programmers whose brains have exploded > from cognitive overload tend not to produce double-digit KLOC/hour. > My concern is entirely whether the design you propose simplifies the > job of meeting the common requirement of creating a (1 to 3 level) > hierarchical organization of lists (the third level being the children > of umbrella lists), or serves important use cases without > significantly complexifying the job of serving the common case. I argue that it simplifies the case that you mention because it provides a uniform way to deal with all of the aspects, especially the delegation aspects which are not addressed in the present scheme. > I don't see a simplification for the reasons expressed above. > > What are the use cases for a more generic structure? From xuwang at gmail.com Sun Apr 28 21:42:08 2013 From: xuwang at gmail.com (Xu Wang) Date: Sun, 28 Apr 2013 12:42:08 -0700 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sun, Apr 28, 2013 at 11:27 AM, Stephen J. Turnbull wrote: > Xu Wang writes: > > On Sun, Apr 28, 2013 at 12:15 AM, Stephen J. Turnbull < > stephen at xemacs.org> > > wrote: > > > Xu Wang writes: > > > > > > > The problem is how do you "confirm ownership of the subscribed > > > > address" when a request coming with an access token. > > > > > > You don't. That was done when the OAuth ID was linked to the address, > > > using the usual 3-step handshake (submit the association, receive an > > > email containing a secret, confirm ownership by replying with the > secret). > > > > I'm not sure what you mean by "OAuth ID". > > In the case of Mailman's use cases, it will typically be an email > address at Gmail, Yahoo, or Launchpad, or some other well-known OpenID > provider. Often, but not necessarily, it will be the subscribed address. > > OpenID is about authentication. OAuth is about authorisation. What are we talking about here? Support to OpenID or support to OAuth or both? > As a resource server, the API need to validate the access token but > > how to validate the access token is not part of OAuth. > > This is some of the excess complexity (if you prefer, "enterprise > readiness") in OAuth 2.0. In Mailman usage, the resource that the > consumer gets access to will be a user ID. Eg, I might be "user: > stephen; name: Stephen Turnbull; email: stephen at xemacs.org; > email:stephen.turnbull.fw at u.tsukuba.ac.jp; owner: > fs-phil at turnbull.sk.tsukuba.ac.jp; owner: xemacs-beta at xemacs.org ...; > OpenID: stephenjturnbull at gmail.com". So when I login via OpenID > (which uses the OAuth v1 protocol AFAIK), No OpenID does not uses OAuth protocol. > the provider (GMail) asks me > to verify who I am by presenting my credential (most users will think > of this as "getting the login screen"), and in turn it testifies to > the client that I own that email at GMail, and the consumer (eg, the > HyperKitty app) then looks it up, identifies me, logs me in as > "stephen", and gives me a cookie (session auth token). The > authentication *is* the validation here. > Mailman can't use GMail as OAuth provider to protect mailman's own resource as a general authorization solution. OAuth v2 envisions more complex scenarios where the client may present > the same token repeatedly, or consumers might hang on to them and > present them to the resource owner repeatedly, or even pass them on to > other consumers. In those cases, yes, there will need to be > validation etc going on. For example, the token might be a (ID, URI, > expiration-date) tuple encrypted with the resource owner's private > key. Being able to decrypt and getting a syntactically correct, > unexpired tuple would then be validation of an authorization to fetch > that URI. (In this particular case I'm thinking of "any valid user is > OK".) This does require prior coordination between the provider and > the resource owner on token format and exchange of keys. > > But AFAICS Mailman's currently envisioned applications don't require > this complexity. > As you put it, mailman don't have to support OAuth, but if it dose, it's better stick to the protocol. > Do you have any ideas for Mailman that might require something more > fine-grained than just "this user is logged in"? > I'm new to mailman and have a very limited knowledge about what kind of "fine-grained" authorization it may need other than authentication. (read: Xu had no clue ...) >From the api implementation point of view, a role base authorization can be bundled in, which allows admin or resource owner to associate user with roles and roles with endpoints/methods. From stephen at xemacs.org Mon Apr 29 01:54:29 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 08:54:29 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> Message-ID: <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > Yes, initially generating a more generic structure than the ad hoc > one in place (which doesn't even attempt to address delegation) Aha! Something that looks like a concrete use case! But what is "delegation"? I mean, "who delegates what to whom? And why does Mailman need to address it? What code needs to be written to address it? The rest of your post is just a reiteration of your religious belief that generic is good. I know the theory, but without use cases I will devote my efforts elsewhere, and ignore complaints that my code or my advisee's code is insufficiently general or that it ignores a theoretical design that has no code or use case to back it up. From stephen at xemacs.org Mon Apr 29 02:18:25 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 09:18:25 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87bo8yw5ny.fsf@uwakimon.sk.tsukuba.ac.jp> Xu Wang writes: > No OpenID does not uses OAuth protocol. My mistake. Let's talk about what Mailman needs, then. OpenID (or an equivalent based on a more general system) is all that Mailman needs as far as I can see. AIUI, the resources that can be accessed or mutated, in order of frequency of use, are 1. Moderation queues. Moderators need access. List owners grant access on the basis of identity of the moderator. 2. Subscriptions. Subscribers and list owners need access. Subscribers usually own the subscribed address and have a privacy (no spam) right. (In employment and education cases, subscribers may have restricted rights, but ID == authorization holds.) List owners are delegated authority from site (or vhost) admin. They may have "legal" (or "precedent") rights to the subscriber list and archives. Either way, ID is authorization. 3. Lists. List owners need access. List owners are delegated authority from site (or vhost) admin. ID is authorization. 4. Virtual hosts (vhosts). A namespace for lists based on a domain name (DNS MX). Delegated by site admins. ID is authorization. 5. Site (collection of vhosts). A physical system. Delegated to site admin from outside. ID is authorization. Corner cases: (1) People with multiple roles may prefer to have to re-login (or "su") use "higher" privilege. "su" or "sudo" mechanisms can address this. (2) Delegators may wish to de-authorize delegatees. An ID-based authorization means some lag. This can be mitigated by granting authorization with short expiration, but that has its issues too. > Mailman can't use GMail as OAuth provider to protect mailman's own > resource as a general authorization solution. Evidently it can't use it at all, if OpenID isn't based on OAuth (and the OAuth protocol exposed to GMail users). > As you put it, mailman don't have to support OAuth, but if it dose, it's > better stick to the protocol. I don't see how anything general I said doesn't stick to protocol. *Validation* is a syntactic operation. Contacting the resource owner and asking "is this token one of yours?" is one way to validate. (Traditional 3-way handshake for validating mail addresses when subscribing is exactly this.) But using encryption or signatures is another way. > I'm new to mailman and have a very limited knowledge about what > kind of "fine-grained" authorization it may need other than > authentication. (read: Xu had no clue ...) > > >From the api implementation point of view, a role base authorization can > be bundled in, which allows admin or resource owner to associate user with > roles and roles with endpoints/methods. AFAICS this is all we need, see above. From rkw at dataplex.net Mon Apr 29 02:43:59 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sun, 28 Apr 2013 19:43:59 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> I am thinking of "delegation" in the context of administering list properties and preferences. Assume a hierarchy of lists handled by one (or more) servers that comprise a MM system. Various "persons" could be assigned authority to make changes that affect lists within portions of the tree. those changes might in things such as setting the default language or the ability to create a new list. In the virtual hosting scenario, provides a number of possible opportunities to illustrate how this might be used. The "root" of the tree covers all of the lists. Under that top node, we might create nodes for "Customer Plans", for example, "Bronze", "Silver", "Gold" and "Platinum". Each of these nodes would specify some limits that applies to the level of service. Let us assume that the Bronze service plan allows the customer to have only lists set up by the Provider staff. That user might still be allowed to set other parameters, for example, the subscription policy and the"welcome message". Under the Silver Plan, a customer might be allowed to set up new lists within their own email-domain. In both of these cases, a subordinate node would be created under the appropriate "plan" node for the list(s) of that customer. And under those nodes, we would find various individual lists. Now, within our permissions system, we would grant the customer administrator permissions to alter certain parts of the list parameters. In my example case, the right to create new lists would not be granted to the customers in the Bronze plan, but would be granted to those in the Silver plan. Another right is the ability to permit the administrator to associate additional persons to nodes within their portion of the tree. By doing so, that administrator "delegates" the ability to perform certain operations to this added person. Note: I am not trying to present the complete set of details, only trying to indicate how such a framework might be utilized. The reason that a generic mechanism has value is that the same type of "permissions" logic is utilized for any parameter describing the list behavior. Whether you are selecting the default user preference to "receive digest" or specifying the email address or website URL through which someone can "unsubscribe", except for the filtering on permissible data values, the mechanisms needed to handle an input form are very similar. Richard On Apr 28, 2013, at 6:54 PM, Stephen J. Turnbull wrote: > Richard Wackerbarth writes: > >> Yes, initially generating a more generic structure than the ad hoc >> one in place (which doesn't even attempt to address delegation) > > Aha! Something that looks like a concrete use case! But what is > "delegation"? I mean, "who delegates what to whom? And why does > Mailman need to address it? What code needs to be written to address > it? > > The rest of your post is just a reiteration of your religious belief > that generic is good. > > I know the theory, but without use cases I will devote my efforts > elsewhere, and ignore complaints that my code or my advisee's code is > insufficiently general or that it ignores a theoretical design that > has no code or use case to back it up. From rkw at dataplex.net Mon Apr 29 03:05:53 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sun, 28 Apr 2013 20:05:53 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <3E60A872-BD1E-4649-8E94-020D411FD43C@dataplex.net> On Apr 28, 2013, at 6:54 PM, Stephen J. Turnbull wrote: > The rest of your post is just a reiteration of your religious belief that generic is good. Call it "religion" if you wish. It is based on DECADES of experience, many of which involved reworking existing code to handle some changed conditions. > I know the theory, but without use cases I will devote my efforts > elsewhere, and ignore complaints that my code or my advisee's code is > insufficiently general or that it ignores a theoretical design that > has no code or use case to back it up. Where is your design to handle the delegation of (restricted) permissions to alter list settings, create new lists, add moderators or administrators, etc.? I am, at least, proposing a framework which would be able to address these issues. This sounds as if you are using the "But you haven't actually built the entire system, therefore I can dismiss anything that you propose as a design concept" excuse to dismiss any ideas that are "Not Invented Here" From rkw at dataplex.net Mon Apr 29 03:47:54 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sun, 28 Apr 2013 20:47:54 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: My understanding of the use of oAuth to provide "login" information from Google, Twitter, etc. is based on the following description provided by Google. Below is a trivial example of how to use Google's OAuth 2.0 endpoint to gain access to a Google API. It's a Python web application running on App Engine. The flow of the example is fairly straightforward: When the application loads, it shows the you a "Login" link. When you click that link, you are asked to login to Google and asked to release basic account information to the application (user consent). If you grant consent, the application receives an access token. Once it has the access token, the application presents the access token to the Google API that provides basic account information (https://www.googleapis.com/oauth2/v1/userinfo) The application renders the basic account information in a simple table. In particular, a user grants our server the permission to access some information pertaining to the user as it is stored by Google and accessed through their API for the purpose. Based on that information, our server grants permission to the user thus identified and based on authorization data stored in our system. Once we have identified the user, our system can use tokens such as session keys, or other mechanisms, to maintain the association. Richard On Apr 28, 2013, at 2:42 PM, Xu Wang wrote: > On Sun, Apr 28, 2013 at 11:27 AM, Stephen J. Turnbull wrote: > >> Xu Wang writes: >>> On Sun, Apr 28, 2013 at 12:15 AM, Stephen J. Turnbull < >> stephen at xemacs.org> >>> wrote: >>>> Xu Wang writes: >>>> >>>>> The problem is how do you "confirm ownership of the subscribed >>>>> address" when a request coming with an access token. >>>> >>>> You don't. That was done when the OAuth ID was linked to the address, >>>> using the usual 3-step handshake (submit the association, receive an >>>> email containing a secret, confirm ownership by replying with the >> secret). >>> >>> I'm not sure what you mean by "OAuth ID". >> >> In the case of Mailman's use cases, it will typically be an email >> address at Gmail, Yahoo, or Launchpad, or some other well-known OpenID >> provider. Often, but not necessarily, it will be the subscribed address. >> >> > OpenID is about authentication. OAuth is about authorisation. > What are we talking about here? Support to OpenID or support to OAuth or > both? > >> As a resource server, the API need to validate the access token but >>> how to validate the access token is not part of OAuth. >> >> This is some of the excess complexity (if you prefer, "enterprise >> readiness") in OAuth 2.0. In Mailman usage, the resource that the >> consumer gets access to will be a user ID. Eg, I might be "user: >> stephen; name: Stephen Turnbull; email: stephen at xemacs.org; >> email:stephen.turnbull.fw at u.tsukuba.ac.jp; owner: >> fs-phil at turnbull.sk.tsukuba.ac.jp; owner: xemacs-beta at xemacs.org ...; >> OpenID: stephenjturnbull at gmail.com". So when I login via OpenID >> (which uses the OAuth v1 protocol AFAIK), > > No OpenID does not uses OAuth protocol. > > >> the provider (GMail) asks me >> to verify who I am by presenting my credential (most users will think >> of this as "getting the login screen"), and in turn it testifies to >> the client that I own that email at GMail, and the consumer (eg, the >> HyperKitty app) then looks it up, identifies me, logs me in as >> "stephen", and gives me a cookie (session auth token). The >> authentication *is* the validation here. >> > > Mailman can't use GMail as OAuth provider to protect mailman's own resource > as a general authorization solution. > > OAuth v2 envisions more complex scenarios where the client may present >> the same token repeatedly, or consumers might hang on to them and >> present them to the resource owner repeatedly, or even pass them on to >> other consumers. In those cases, yes, there will need to be >> validation etc going on. For example, the token might be a (ID, URI, >> expiration-date) tuple encrypted with the resource owner's private >> key. Being able to decrypt and getting a syntactically correct, >> unexpired tuple would then be validation of an authorization to fetch >> that URI. (In this particular case I'm thinking of "any valid user is >> OK".) This does require prior coordination between the provider and >> the resource owner on token format and exchange of keys. >> >> But AFAICS Mailman's currently envisioned applications don't require >> this complexity. >> > > As you put it, mailman don't have to support OAuth, but if it dose, it's > better stick to the protocol. > > >> Do you have any ideas for Mailman that might require something more >> fine-grained than just "this user is logged in"? >> > > I'm new to mailman and have a very limited knowledge about what kind of > "fine-grained" authorization it may need other than authentication. (read: > Xu had no clue ...) > > From the api implementation point of view, a role base authorization can > be bundled in, which allows admin or resource owner to associate user with > roles and roles with endpoints/methods. > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From stephen at xemacs.org Mon Apr 29 04:58:04 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 11:58:04 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> Message-ID: <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > The "root" of the tree covers all of the lists. Under that top > node, we might create nodes for "Customer Plans", for example, > "Bronze", "Silver", "Gold" and "Platinum". Each of these nodes > would specify some limits that applies to the level of service. But here the limits apply to *principals*[1], not lists, AFAICS. Sure, you can record them on the lists, but you'll still have to check the principal's ID/authorization (presumably site admins can do anything they want, even to a list owned by a Bronze customer who can't do it for himself. Note how you repeatedly write "[principal] be allowed": > Let us assume that the Bronze service plan allows the customer to > have only lists set up by the Provider staff. That user might > still be allowed to set other parameters, for example, the > subscription policy and the"welcome message". Under the Silver > Plan, a customer might be allowed to set up new lists within their > own email-domain. > > In both of these cases, a subordinate node would be created under > the appropriate "plan" node for the list(s) of that customer. And > under those nodes, we would find various individual lists. I could see this organization being usable in a case where a customer wants to have some Bronze-plan lists and some Gold-plan lists. However, I could also abstract "customer" as a principal which is a group of accounts with a common billing address. Then accounts would be authorized, and the customer would log in as an account-user as appropriate, rather than log in as a customer-user. The user interface could provide a su-like way to switch accounts of a single customer, and/or a sudo-like way to work with "foreign" accounts. It's not obvious to me that your way of doing things is better for this use-case. If it's not plausible that it's better (I make no claim that it's not at this point), it fails to justify an experiment with a generic organization for lists. > Another right is the ability to permit the administrator to > associate additional persons to nodes within their portion of the > tree. By doing so, that administrator "delegates" the ability to > perform certain operations to this added person. This is more plausible as a use-case, since the "certain nodes" need not be all the nodes for that administrator. OTOH, we already have "list owner" and "moderator" roles, and those *are* attached to each list. It's not clear to me that we need to provide for adding additional roles, but I'll keep it in mind. Footnotes: [1] A generic term covering users, groups of users, accounts, and in general "entities that can be authorized". From xuwang at gmail.com Mon Apr 29 05:26:17 2013 From: xuwang at gmail.com (Xu Wang) Date: Sun, 28 Apr 2013 20:26:17 -0700 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Well, it is about how a third party web application can get user's profile data from google as oauth client, like OpenID, it's little help on the access control of a third party RESTful API. As oauth supported google's userinfo API, one need to present a valid google's oauth access token to get access. s/google/mailman/g on above statement, it will be true too. If we are talking about enable OpenID or get user's profiles from google, facebook, tweeter, etc. as an OAuth client in postorius, it should not be a big deal because the client libs are readily available from those providers, but it is different from claiming mailman's api supports OAuth. On Sun, Apr 28, 2013 at 6:47 PM, Richard Wackerbarth wrote: > My understanding of the use of oAuth to provide "login" information from > Google, Twitter, etc. is based on the following description provided by > Google. > > Below is a trivial example of how to use Google's OAuth 2.0 endpoint to > gain access to a Google API. It's a Python web application running on App > Engine. The flow of the example is fairly straightforward: > > 1. When the application loads, it shows the you a "Login" link. > 2. When you click that link, you are asked to login to Google and > asked to release basic account information to the application (user > consent). > 3. If you grant consent, the application receives an access token. > 4. Once it has the access token, the application presents the access > token to the Google API that provides basic account information ( > https://www.googleapis.com/oauth2/v1/userinfo) > 5. The application renders the basic account information in a simple > table. > > In particular, a user grants our server the permission to access some > information pertaining to the user as it is stored by Google and accessed > through their API for the purpose. > > Based on that information, our server grants permission to the user thus > identified and based on authorization data stored in our system. Once we > have identified the user, our system can use tokens such as session keys, > or other mechanisms, to maintain the association. > > Richard > > On Apr 28, 2013, at 2:42 PM, Xu Wang wrote: > > On Sun, Apr 28, 2013 at 11:27 AM, Stephen J. Turnbull >wrote: > > Xu Wang writes: > > On Sun, Apr 28, 2013 at 12:15 AM, Stephen J. Turnbull < > > stephen at xemacs.org> > > wrote: > > Xu Wang writes: > > The problem is how do you "confirm ownership of the subscribed > address" when a request coming with an access token. > > > You don't. That was done when the OAuth ID was linked to the address, > using the usual 3-step handshake (submit the association, receive an > email containing a secret, confirm ownership by replying with the > > secret). > > > I'm not sure what you mean by "OAuth ID". > > > In the case of Mailman's use cases, it will typically be an email > address at Gmail, Yahoo, or Launchpad, or some other well-known OpenID > provider. Often, but not necessarily, it will be the subscribed address. > > > OpenID is about authentication. OAuth is about authorisation. > What are we talking about here? Support to OpenID or support to OAuth or > both? > > As a resource server, the API need to validate the access token but > > how to validate the access token is not part of OAuth. > > > This is some of the excess complexity (if you prefer, "enterprise > readiness") in OAuth 2.0. In Mailman usage, the resource that the > consumer gets access to will be a user ID. Eg, I might be "user: > stephen; name: Stephen Turnbull; email: stephen at xemacs.org; > email:stephen.turnbull.fw at u.tsukuba.ac.jp; owner: > fs-phil at turnbull.sk.tsukuba.ac.jp; owner: xemacs-beta at xemacs.org ...; > OpenID: stephenjturnbull at gmail.com". So when I login via OpenID > (which uses the OAuth v1 protocol AFAIK), > > > No OpenID does not uses OAuth protocol. > > > the provider (GMail) asks me > to verify who I am by presenting my credential (most users will think > of this as "getting the login screen"), and in turn it testifies to > the client that I own that email at GMail, and the consumer (eg, the > HyperKitty app) then looks it up, identifies me, logs me in as > "stephen", and gives me a cookie (session auth token). The > authentication *is* the validation here. > > > Mailman can't use GMail as OAuth provider to protect mailman's own resource > as a general authorization solution. > > OAuth v2 envisions more complex scenarios where the client may present > > the same token repeatedly, or consumers might hang on to them and > present them to the resource owner repeatedly, or even pass them on to > other consumers. In those cases, yes, there will need to be > validation etc going on. For example, the token might be a (ID, URI, > expiration-date) tuple encrypted with the resource owner's private > key. Being able to decrypt and getting a syntactically correct, > unexpired tuple would then be validation of an authorization to fetch > that URI. (In this particular case I'm thinking of "any valid user is > OK".) This does require prior coordination between the provider and > the resource owner on token format and exchange of keys. > > But AFAICS Mailman's currently envisioned applications don't require > this complexity. > > > As you put it, mailman don't have to support OAuth, but if it dose, it's > better stick to the protocol. > > > Do you have any ideas for Mailman that might require something more > fine-grained than just "this user is logged in"? > > > I'm new to mailman and have a very limited knowledge about what kind of > "fine-grained" authorization it may need other than authentication. (read: > Xu had no clue ...) > > From the api implementation point of view, a role base authorization can > be bundled in, which allows admin or resource owner to associate user with > roles and roles with endpoints/methods. > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 > > > From stephen at xemacs.org Mon Apr 29 05:28:25 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 12:28:25 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <3E60A872-BD1E-4649-8E94-020D411FD43C@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <3E60A872-BD1E-4649-8E94-020D411FD43C@dataplex.net> Message-ID: <8761z6vwva.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > > On Apr 28, 2013, at 6:54 PM, Stephen J. Turnbull wrote: > > The rest of your post is just a reiteration of your religious > > belief that generic is good. > > Call it "religion" if you wish. It is based on DECADES of > experience, So are most religions, but for some reason they all have exceptions where the Power chooses not to perform miracles. > many of which involved reworking existing code to > handle some changed conditions. Of course you do. Anybody with five years of experience has run into that. But with decades of experience, surely you've also run into projects that never saw the light of day because the implementers overengineered Grand Designs which didn't address user requirements. > Where is your design to handle the delegation of (restricted) > permissions to alter list settings, create new lists, add > moderators or administrators, etc.? Mailman 2.1. I've seen no real reason to suppose we need more. What people repeatedly request on Mailman channels (in my, eminently fallible, recollection) is more *power* for existing administrator roles (viewing logs), more power for the user role (searching archives), and more intuitive UIs for both. Barry's design for Mailman 3 addresses those needs by making it a heck of a lot easier to experiment with additional UIs. An example of a use case I don't like is the "Like" buttons (or something like that) the HyperKitty guys are putting in. Nobody has requested them on Mailman channels that I can recall. But social networks are booming, and any visit to YouTube will provide evidence that people do use those "Like" buttons, and comment on them. I have no ground to stand on there. I'm sure those buttons will be appreciated and used by lots of subscribers. That use case is valid, whatever my personal feeling is. But I've never seen (IMEFR) a request for more flexibility in constructing an administrative organization for a Mailman site. > I am, at least, proposing a framework which would be able to > address these issues. My point is, "Don't tell me about theoretical issues. What use cases are you addressing?" If users aren't going to use your framework, there's no point in building it. > This sounds as if you are using the "But you haven't actually built > the entire system, therefore I can dismiss anything that you > propose as a design concept" excuse to dismiss any ideas that are > "Not Invented Here" You're not listened if that's what it sounds like to you. I haven't once asked you for running code. I've asked you for examples of what Real Users[tm] might want to run the proposed code for. If you want to build it with your own resources, I have no problem with that. If you want me to help, or if you want me to support your design to other developers (including GSoC interns), I need to know what it's for, besides addressing issues that I do perceive in software engineering theory, but not in Mailman practice. Steve From rkw at dataplex.net Mon Apr 29 05:45:11 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sun, 28 Apr 2013 22:45:11 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Apr 28, 2013, at 9:58 PM, Stephen J. Turnbull wrote: > Richard Wackerbarth writes: > >> The "root" of the tree covers all of the lists. Under that top >> node, we might create nodes for "Customer Plans", for example, >> "Bronze", "Silver", "Gold" and "Platinum". Each of these nodes >> would specify some limits that applies to the level of service. > > But here the limits apply to *principals*[1], not lists, AFAICS. That is the case in my hypothetical example. However, the mechanism applies to any property of the lists such as a default value for some user preference. In my example, the property is, perhaps, "Administrator can/cannot create a new list" Equally, the content of a "Greeting Message" or the "Default Language", or any other property can be treated in the same manner. > I could see this organization being usable in a case where ... > It's not obvious to me that your way of doing things is better for > this use-case. If it's not plausible that it's better (I make no > claim that it's not at this point), it fails to justify an experiment > with a generic organization for lists. You seem to be missing the point that "one size fits all", or in this case, one hierarchy, is not a flexible strategy. By having a generic structure and allowing each installation to customize the hierarchy to fix their individual need, we provide a mechanism that better suits the needs of a larger group of installations. >> Another right is the ability to permit the administrator to >> associate additional persons to nodes within their portion of the >> tree. By doing so, that administrator "delegates" the ability to >> perform certain operations to this added person. > > This is more plausible as a use-case, since the "certain nodes" need > not be all the nodes for that administrator. OTOH, we already have > "list owner" and "moderator" roles, and those *are* attached to each > list. It's not clear to me that we need to provide for adding > additional roles, but I'll keep it in mind. I'm advocating that we attach the roles (whatever they may be) to an entire collection of lists. The purpose of a hierarchical structure, whether one fixed structure or a generic recursive one, is to provide a mechanism to assist the "principal" in managing a number of lists that have common properties. Actually, by establishing a flexible hierarchy, it may be possible to eliminate some of the functionally similar roles. For example, a "Domain Administrator" might be nothing more than a "List Administrator" attached to a higher node in the tree of Lists. From stephen at xemacs.org Mon Apr 29 06:07:16 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 13:07:16 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> Xu Wang writes: > As oauth supported google's userinfo API, one need to present a valid > google's oauth access token to get access. > s/google/mailman/g on above statement, it will be true too. I disagree, in the sense that Google (as an OAuth provider) is in the business of *providing* enterprise workflows such as AppEngine. That's why they need to be an OAuth provider. Mailman is a support function for workflows (enterprise or otherwise). So it's not a "Mailman" token. It's an token, and the enterprise, not Mailman, should be the provider. If Mailman provides, then we have to take responsibility for foreseeing enterprise needs. If we go Wacky's route and make everything as generic as possible, we may need the power of OAuth to handle all that genericity. (We may also then need another 5 years to release Mailman 3....) But if we stick to the current role-based authorization model with a small fixed set of roles, then OpenID-like workflows (whether implemented via OAuth protocol or otherwise) should be enough. If a site demands more control over authentication than public OpenID providers can afford, then Basic Auth over HTTPS fits into the "user has role" authorization model as well as OpenID does. I don't see a need for Mailman to provide an authentication provider, and there are serious downsides to the proliferation of authentication providers. Steve From rkw at dataplex.net Mon Apr 29 06:24:26 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Sun, 28 Apr 2013 23:24:26 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Steve, Here I agree with you. It is useful for MM to be able to accept enterprise information when it is available. OAuth is a mechanism that will be useful for some enterprises. To the general public, being able to use enterprise identification from common sources such as Google or Twitter, is a "friendly" way identify a user and allow them to log into a MM system. Within a MM installation, OAuth could be used in a more robust distributed implementation. However for our purposes, much simpler schemes such as Basic or Digest Auth is more than adequate for the intercommunication between components such as "core", "postorius", a message store, etc. Richard On Apr 28, 2013, at 11:07 PM, "Stephen J. Turnbull" wrote: > Xu Wang writes: > >> As oauth supported google's userinfo API, one need to present a valid >> google's oauth access token to get access. >> s/google/mailman/g on above statement, it will be true too. > > I disagree, in the sense that Google (as an OAuth provider) is in the > business of *providing* enterprise workflows such as AppEngine. > That's why they need to be an OAuth provider. Mailman is a support > function for workflows (enterprise or otherwise). > > So it's not a "Mailman" token. It's an token, and the > enterprise, not Mailman, should be the provider. If Mailman provides, > then we have to take responsibility for foreseeing enterprise needs. > > If we go Wacky's route and make everything as generic as possible, we > may need the power of OAuth to handle all that genericity. (We may > also then need another 5 years to release Mailman 3....) But if we > stick to the current role-based authorization model with a small fixed > set of roles, then OpenID-like workflows (whether implemented via > OAuth protocol or otherwise) should be enough. > > If a site demands more control over authentication than public OpenID > providers can afford, then Basic Auth over HTTPS fits into the "user > has role" authorization model as well as OpenID does. I don't see a > need for Mailman to provide an authentication provider, and there are > serious downsides to the proliferation of authentication providers. > > Steve From stephen at xemacs.org Mon Apr 29 06:59:33 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 13:59:33 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <871u9uvsne.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > You seem to be missing the point that "one size fits all", or in > this case, one hierarchy, is not a flexible strategy. Sorry, that's false, and there's plenty of evidence in the archives that I've acknowledged that point. But "flexible" is not an absolute, not even if you're discussing female gymnasts. My point is that IMHO it's flexible *enough*. > I'm advocating that we attach the roles (whatever they may be) to > an entire collection of lists. I know what you're advocating, and I agree with the general theory of constructing for flexibility. I just don't see a need for it. > Actually, by establishing a flexible hierarchy, it may be possible > to eliminate some of the functionally similar roles. For example, a > "Domain Administrator" might be nothing more than a "List > Administrator" attached to a higher node in the tree of Lists. Sure. That's the main point of a generic model, sharing code to handle cases that are similar enough to handle with the same code. However, I don't think it's true that they're that similar. By the principle of subsidiarity, there are things that a List Administrator does that a Domain Administrator normally shouldn't do (authorizing moderators). But there are others that both should be able to do (clearing queues of broken messages, removing legally objectionable messages from archives). So to handle both attributes that are subject to subsidiarity and those that aren't, you need as much code as in the current model, *plus* in your model you need to make the code generic *and* provide rules for the existing use cases. That just doesn't sound like a bargain to me in the absence of a few plausible *and concrete* use cases that require more flexibility. Steve From markoupetr at gmail.com Mon Apr 29 10:39:34 2013 From: markoupetr at gmail.com (Peter Markou) Date: Mon, 29 Apr 2013 11:39:34 +0300 Subject: [Mailman-Developers] Web Posting Interface: Additional Features. Message-ID: Hello everyone in the community. As you may or may not know, I've applied a proposal in Melange for Web Posting Interface. The description of the project idea states: "the interface itself may not take the whole summer", so I've come up with some features that I would like to implement for Mailman web interface. These features are listed below: ** 3.) Keyword Summary, 6.) User Profiles(support for name, photo, bio, past posts & files, statistics about how people post), 10.) Top X Threads of all-time, 11.) Topics to be Wary Of, 12.) User's Filter Tools, 14.) Keyword-Based Thread Browse, 17.) List Monthly Health, 22.) Mentioned in thread refs. I would appreciate any feedback from developers/users, about which of them would be useful and fit well with Mailman web interface. ** The numbering comes from the source I've found these ideas: http://blog.linuxgrrl.com/2012/03/13/mailman-brainstorm/ , http://blog.linuxgrrl.com/2012/03/14/mailman-brainstorm-2/ . From iane at sussex.ac.uk Mon Apr 29 11:40:45 2013 From: iane at sussex.ac.uk (Ian Eiloart) Date: Mon, 29 Apr 2013 09:40:45 +0000 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A7159.9080909@ploing.de> <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> <517BD109.3020805@ulm.ccc.de> Message-ID: On 27 Apr 2013, at 14:40, Richard Wackerbarth wrote: > I don't think that "we" have the expertise to create a "secure" system. At best, we can adopt good practices and provide an obscured traffic stream. I consider anything more to be beyond the scope of the MM project. > Also, what kind of secure list would have automated processing of message content as a requirement? If a message is gpg encrypted, then every sender would require the public keys of every recipient, would they not? Which means that a PKI for the list holders is required. Currently outside of Mailman's scope, but if it exists, then presumably senders would be required to cryptographically sign every message. All the list needs to do is verify the signature before redistributing. THAT is going to be the main body processing requirement. > On Apr 27, 2013, at 8:22 AM, Stefan Schlott wrote: > >> On 27.04.2013 06:45, Stephen J. Turnbull wrote: >> >>>> 2. Your list has elevated security requirements. In this case, you can >>>> use gpg-agent to manage the secret key (and its passphrase). >>> >>> I don't understand what threat you propose to address in this way. >>> It's true that you can prevent the attacker from getting access to the >>> key (using agent forwarding or a token, it need not be on the exposed >>> host at all), but we're assuming he has access to the host and the >>> Mailman process. >> >> The gpg-agent approach protects you from all storage-related attacks: >> - unencrypted backups >> - physical access to the harddrive >> - etc. >> >> It does not protect from attackers who have access to the contents of >> the computer's RAM: >> - raw memory access and scanning for the secret key (requires root) >> - memory dump via DMA-enabled interfaces (firewire, pc-card, ...) >> - cold boot attacks >> >> >> Stefan > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/iane%40sussex.ac.uk > > Security Policy: http://wiki.list.org/x/QIA9 -- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148 From iane at sussex.ac.uk Mon Apr 29 12:02:23 2013 From: iane at sussex.ac.uk (Ian Eiloart) Date: Mon, 29 Apr 2013 10:02:23 +0000 Subject: [Mailman-Developers] Web Posting Interface: Additional Features. In-Reply-To: References: Message-ID: <310D9154-D0A9-46AC-A28F-DC237CFBACD5@sussex.ac.uk> On 29 Apr 2013, at 09:39, Peter Markou wrote: > Hello everyone in the community. As you may or may not know, > I've applied a proposal in Melange for Web Posting Interface. > The description of the project idea states: "the interface > itself may not take the whole summer", so I've come up with > some features that I would like to implement for Mailman web > interface. These features are listed below: ** > 3.) Keyword Summary, > 6.) User Profiles(support for name, photo, bio, past posts & files, > statistics about how people post), > 10.) Top X Threads of all-time, > 11.) Topics to be Wary Of, > 12.) User's Filter Tools, > 14.) Keyword-Based Thread Browse, > 17.) List Monthly Health, > 22.) Mentioned in thread refs. > > I would appreciate any feedback from developers/users, about > which of them would be useful and fit well with Mailman web > interface. Hi, The first thing to remember is that there are at least three types of user: Site admins List admins List members And, of course there will be a variety of levels of technical skill within each user set. I imagine that a Web Posting Interface will be for List Members. They'll need to be able to access their personal profile, list archives, and a form for composing and sending a message to the list. In my view, the essential features for that are: 1. A nice editing tool, that a list admin can configure to send ONLY unformatted text. That should probably be the default for new lists, for new members, and for new messages. 2. The tool should do it's utmost to preserve threads when a member is replying to a message, but not otherwise. 3. If the message is a reply, then the rest of the thread messages should have some visibility: to discourage members from replying before they've seen the whole thread. 4. There should be a link to a profile page, but it would be neat if the profile page could leverage some existing identity. Various options exist, but given that we have an email address, we can often get some profile information from the email service provider, for example when they're openid providers. Don't work on this until 1-3 are complete. 5. There should be a link to a list archive. An archive browser is probably a project of its own. My view is that the keyword and topic type stuff all belong in (5), if anywhere at all. In the message posting page, they'd be unnecessary clutter. -- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148 From rkw at dataplex.net Mon Apr 29 13:42:16 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Mon, 29 Apr 2013 06:42:16 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <871u9uvsne.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9uvsne.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <3D7B92D1-F665-4727-8ED0-30CE3E1152E3@dataplex.net> On Apr 28, 2013, at 11:59 PM, Stephen J. Turnbull wrote: > Richard Wackerbarth writes: > My point is that IMHO it's flexible *enough*. > >> I'm advocating that we attach the roles (whatever they may be) to >> an entire collection of lists. > > I know what you're advocating, and I agree with the general theory of > constructing for flexibility. I just don't see a need for it. One point that I had overlooked is that you have already acknowledged an additional "layer" between "domain" and "list". So you have at least three layers. Do you really think that it is more difficult to implement a general recursive tree than it is to implement those layers? Another case for generality is permissions. I don't propose to know all of the parameters that will be associated with a list. In fact, I am sure that they will change over time. There has already been expressed a desire to have plug-in extensions (which might add some additional parameters of their own). However, from the view of the admin UI, all parameters share the common characteristic that they either can, or cannot, be altered by the . Using the generic abstraction that every parameter is a case of the Parameter class, and subclassing that to provide commonly seen variations (for example, "boolean") allows us implement a flexible structure that does not require handler recoding as the set of parameters changes. From stefan.schlott at ulm.ccc.de Mon Apr 29 14:12:51 2013 From: stefan.schlott at ulm.ccc.de (Stefan Schlott) Date: Mon, 29 Apr 2013 14:12:51 +0200 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A7159.9080909@ploing.de> <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> <517BD109.3020805@ulm.ccc.de> Message-ID: <517E63C3.4090309@ulm.ccc.de> On 29.04.2013 11:40, Ian Eiloart wrote: > Also, what kind of secure list would have automated processing of > message content as a requirement? imho you're asking the wrong question ;-) _All_ network communication should be encrypted, it is a pity that mail encryption is so little adopted. > If a message is gpg encrypted, then > every sender would require the public keys of every recipient, would > they not? No. The idea here that the recipient of a mail sent to a mailing list is the (trusted) mailman server, thus the only key needed is the mailing list public key. Mailman has access to its secret key, decrypts the incoming message and re-encrypts it for each recipient. Stefan. From stephen at xemacs.org Mon Apr 29 14:28:27 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 21:28:27 +0900 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A7159.9080909@ploing.de> <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> <517BD109.3020805@ulm.ccc.de> Message-ID: <87zjwhv7v8.fsf@uwakimon.sk.tsukuba.ac.jp> Ian Eiloart writes: > Also, what kind of secure list would have automated processing of > message content as a requirement? Precisely, a list that wants to avoid this requirement: > If a message is gpg encrypted, then every sender would require the > public keys of every recipient, would they not? The idea is that senders use the list's public key. The list holds those public keys, and uses them to re-encrypt the message on a recipient-by-recipient basis after decrypting with its own private key. The discussion has been about how to deal with attacks on (a) the list's private key (including offline attacks on the hard drive) and on (b) the temporarily decrypted text (which could end up in the clear for a long time in a queue file or if Mailman crashes). From Richard at Damon-Family.org Mon Apr 29 14:30:35 2013 From: Richard at Damon-Family.org (Richard Damon) Date: Mon, 29 Apr 2013 08:30:35 -0400 Subject: [Mailman-Developers] GSOC Project idea: OpenPGP integration In-Reply-To: References: <6DCA73B6-171A-42BF-89CC-23E0E4C1BB2C@dataplex.net> <5178EB1F.5080907@ulm.ccc.de> <5179312D.7010704@fifthhorseman.net> <517A7159.9080909@ploing.de> <87d2tgy426.fsf@uwakimon.sk.tsukuba.ac.jp> <517BD109.3020805@ulm.ccc.de> Message-ID: <517E67EB.3000108@Damon-Family.org> On 4/29/13 5:40 AM, Ian Eiloart wrote: > Also, what kind of secure list would have automated processing of > message content as a requirement? If a message is gpg encrypted, then > every sender would require the public keys of every recipient, would > they not? Which means that a PKI for the list holders is required. > Currently outside of Mailman's scope, but if it exists, then > presumably senders would be required to cryptographically sign every > message. All the list needs to do is verify the signature before > redistributing. THAT is going to be the main body processing requirement. That is one way, the other is you send the message encrypted to the list's public key, and the list decrypts the message and then reencrypts to each recipient's public key. (In many cases this doesn't actually require decrypting/reencrypting the whole message, just the session key block). The list could also check any signature, and sign messages with valid signatures with it's key. That way, subscribers don't need any other subscriber's public key. In fact, I think the list could even be set up anonymous so you might not even know who anyone else was, just that the list has validated that the message came from someone on the list. -- Richard Damon From stephen at xemacs.org Mon Apr 29 15:12:33 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 22:12:33 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <3D7B92D1-F665-4727-8ED0-30CE3E1152E3@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9uvsne.fsf@uwakimon.sk.tsukuba.ac.jp> <3D7B92D1-F665-4727-8ED0-30CE3E1152E3@dataplex.net> Message-ID: <87y5c1v5tq.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > So you have at least three layers. Do you really think that it is > more difficult to implement a general recursive tree than it is to > implement those layers? No. What I think is difficult is to implement a general recursive tree *and* an API for expressing properties of nodes that make them equivalent to the specialized hierarchy *and* a usable interface for specifying and managing those properties by non-programmer users (including admins). > Another case for generality is permissions. I don't propose to know > all of the parameters that will be associated with a list. You're pushing on a string here. I'm not opposed to generality in general. ;-) What's more important to me than whether you know all of the parameters that will be associated with a list :-) is that I'm pretty sure I don't want Postorius's views to know. That leads to ugly and unmaintainable code. I also don't want users to be seeing data they don't need to see, or shouldn't see, and they mustn't be able to change parameters they shouldn't change. These requirements can be fulfilled in a number of ways, including customization by extension modules. For example, there could be (extensible) lists of parameters attached to each role, a list for each permission. I think that's kind of fragile; better to attach an ACL to each permission. (I think that's basically what you have in mind, too.) From rkw at dataplex.net Mon Apr 29 21:30:01 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Mon, 29 Apr 2013 14:30:01 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87y5c1v5tq.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9uvsne.fsf@uwakimon.sk.tsukuba.ac.jp> <3D7B92D1-F665-4727-8ED0-30CE3E1152E3@dataplex.net> <87y5c1v5tq.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <800AED25-ED93-4212-B7D6-DC198C0356FF@dataplex.net> On Apr 29, 2013, at 8:12 AM, Stephen J. Turnbull wrote: > Richard Wackerbarth writes: > >> So you have at least three layers. Do you really think that it is >> more difficult to implement a general recursive tree than it is to >> implement those layers? > > No. What I think is difficult is to implement a general recursive tree > *and* an API for expressing properties of nodes that make them > equivalent to the specialized hierarchy *and* a usable interface for > specifying and managing those properties by non-programmer users > (including admins). There are two aspects of using a "generic tree". The first is some customization of the tree to fit the installation's delegation model. Here, I would suggest that "sample base installations" be made available so that the installation of a workable tree structure is nothing more than copying a supplied configuration base. "Power users" should be able to understand the structure and customize it to fit their particular situation. The other aspect is "how to display" them, particularly because of inheritance within the tree. I don't see "display the tree of lists" as a problem. There are a number of reasonable implementations available for adoption. IMHO, the key to displaying particular the individual parameters is to use a consistent presentation style and "focusing" on a particular node in the tree. That style can be driven by css using class selectors on each individual item. As for "domain" vs "list", I view them as just two versions of the same thing -- namely a node with a dictionary of properties. By limiting access permission, even though they are present (in a virtual sense), and by modeling each list as having all of the properties of itself and the MM-domain which contains it, we can use one mechanism to handle the both the domain administrator and the list administrator. The only distinction is the point of focus. >> Another case for generality is permissions. I don't propose to know >> all of the parameters that will be associated with a list. > > You're pushing on a string here. I'm not opposed to generality in > general. ;-) What's more important to me than whether you know all of > the parameters that will be associated with a list :-) is that I'm > pretty sure I don't want Postorius's views to know. Postorius need only know about those items that should be viewed or modified once the system is running. > That leads to ugly and unmaintainable code. Or you can treat them in a generic manner, even make them "discoverable", and attach the appropriate ACL information. :) This is, in effect, the approach that django takes with its "Model" and "Admin" constructs. One reason that I advocate attaching the parameters to the list-tree nodes as a dictionary is so that it is easy to add or delete items without having to change the data transfer structure. > I also don't want users to be seeing data they don't need to see, or > shouldn't see, and they mustn't be able to change parameters they > shouldn't change. Agreed. One purpose of a list hierarchy is to make it easier to specify and maintain these attributes. > These requirements can be fulfilled in a number of > ways, including customization by extension modules. For example, > there could be (extensible) lists of parameters attached to each role, > a list for each permission. > > I think that's kind of fragile; better to attach an ACL to each > permission. (I think that's basically what you have in mind, too.) From terri at zone12.com Mon Apr 29 22:06:37 2013 From: terri at zone12.com (Terri Oda) Date: Mon, 29 Apr 2013 14:06:37 -0600 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <517ED2CD.9010507@zone12.com> 1. I would like to remind/tell you all that FOR THIS RELEASE, we will ONLY be supporting Mozilla Persona and passwords as authentication methods. This is not up for discussion at this time; it's a choice we've made to in order to help move the release forwards. 2. As such, any discussion of enterprise use of google or twitter or what have you is pretty much academic. I am satisfied if our authentication plan for methods other than Persona and passwords is "And then the magical python gnomes associate an external account with an internal mailman account!" (I'm not being entirely facetious here; libraries such as django-social-auth are likely to make this sufficiently trivial that it might as well be gnomes.) 3. While I agree that thinking about these things in advance is useful, I think this discussion is starting to crowd out discussions of time-sensitive things like students' GSoC applications and architectural choices that directly impact how they write said applications. 4. Let's be realistic: No one is going to plumb everything carefully through with good architectural work in time for it to be useful to GSoC students starting in a few weeks. So... can we please backburner parts of this discussion until after GSoC so that we don't make it impossible for anyone to start on any project involving extra profile info? I don't want to bog down a student with the need to make this choice and work with these decisions at this stage. For the purposes of GSoC this summer: Authentication for the purposes of the extra profile info data store will be provided by either Persona through the web interface or passwords. Yes, even internally for the REST interface: passwords+localhost are good enough for Mailman Core right now, so it's good enough for anything else we do short-term. We can re-open the floor to better ideas either after GSoC, after the Mailman suite releases, or if Barry vetoes my decision. ;) Terri On 13-04-28 10:24 PM, Richard Wackerbarth wrote: > Steve, > > Here I agree with you. > > It is useful for MM to be able to accept enterprise information when it is available. > OAuth is a mechanism that will be useful for some enterprises. > To the general public, being able to use enterprise identification from common sources such as Google or Twitter, is a "friendly" way identify a user and allow them to log into a MM system. > > Within a MM installation, OAuth could be used in a more robust distributed implementation. However for our purposes, much simpler schemes such as Basic or Digest Auth is more than adequate for the intercommunication between components such as "core", "postorius", a message store, etc. > > Richard > > On Apr 28, 2013, at 11:07 PM, "Stephen J. Turnbull" wrote: > >> Xu Wang writes: >> >>> As oauth supported google's userinfo API, one need to present a valid >>> google's oauth access token to get access. >>> s/google/mailman/g on above statement, it will be true too. >> I disagree, in the sense that Google (as an OAuth provider) is in the >> business of *providing* enterprise workflows such as AppEngine. >> That's why they need to be an OAuth provider. Mailman is a support >> function for workflows (enterprise or otherwise). >> >> So it's not a "Mailman" token. It's an token, and the >> enterprise, not Mailman, should be the provider. If Mailman provides, >> then we have to take responsibility for foreseeing enterprise needs. >> >> If we go Wacky's route and make everything as generic as possible, we >> may need the power of OAuth to handle all that genericity. (We may >> also then need another 5 years to release Mailman 3....) But if we >> stick to the current role-based authorization model with a small fixed >> set of roles, then OpenID-like workflows (whether implemented via >> OAuth protocol or otherwise) should be enough. >> >> If a site demands more control over authentication than public OpenID >> providers can afford, then Basic Auth over HTTPS fits into the "user >> has role" authorization model as well as OpenID does. I don't see a >> need for Mailman to provide an authentication provider, and there are >> serious downsides to the proliferation of authentication providers. >> >> Steve > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/terri%40zone12.com > > Security Policy: http://wiki.list.org/x/QIA9 > From rkw at dataplex.net Mon Apr 29 23:25:31 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Mon, 29 Apr 2013 16:25:31 -0500 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: <517ED2CD.9010507@zone12.com> References: <516F3C7D.1070509@zone12.com> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> Message-ID: On Apr 29, 2013, at 3:06 PM, Terri Oda wrote: > 1. I would like to remind/tell you all that FOR THIS RELEASE, we will ONLY be supporting Mozilla Persona and passwords as authentication methods. This is not up for discussion at this time; it's a choice we've made to in order to help move the release forwards. That doesn't bother me as long as we recognize that the list of acceptable methods be extensible. In other words, adding a new method would require only the addition of the necessary handlers, forms, etc. and NOT require the re-engineering of the code that handles BrowserID or username/password. > 2. As such, any discussion of enterprise use of google or twitter or what have you is pretty much academic. I am satisfied if our authentication plan for methods other than Persona and passwords is "And then the magical python gnomes associate an external account with an internal mailman account!" (I'm not being entirely facetious here; libraries such as django-social-auth are likely to make this sufficiently trivial that it might as well be gnomes.) Agreed. > Authentication for the purposes of the extra profile info data store will be provided by either Persona through the web interface or passwords. Yes, even internally for the REST interface: passwords+localhost are good enough for Mailman Core right now, so it's good enough for anything else we do short-term. I'll go further than that and suggest, for the purpose of GSoC: 1) The USER module will be implemented as a Django 1.5 app. 2) In addition to a custom user model (https://docs.djangoproject.com/en/dev/topics/auth/customizing/#auth-custom-user), it will provide a Credential verification service. The service will store the associations between users and the parameters used to identify them -- protocol, identifier, confirmation. If the credentials match, the service will return a user_identifier which will be used to access additional information about the user. 3) The user_identifier will be of a form chosen by the USER module. Other modules (such as MM-core) may associate alternate identifiers (in their namespace) and set or retrieve them. When installed as a part of a MM system, at the installer's discretion, additional user profile information may also be added. 4) Both the User information and the Credential service will be exposed on a RESTful interface. 5) This Django app may be installed as a part of a Postorius interface or run as a standalone service. In the future alternate implementations might be considered, but this will do for now. It also has the advantage that multiple students can work on things affecting the user record without the other student's work becoming a stopping point to their progress. Then, as they get them working, they can be merged into the postorius system. From terri at zone12.com Mon Apr 29 23:35:46 2013 From: terri at zone12.com (Terri Oda) Date: Mon, 29 Apr 2013 15:35:46 -0600 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> Message-ID: <517EE7B2.2000509@zone12.com> On 13-04-29 3:25 PM, Richard Wackerbarth wrote: > I'll go further than that and suggest, for the purpose of GSoC: [snip details] These are good if you're going to do the implementation. If you're not, I'm happy to leave some of these details up to the person who actually writes the code. Terri From stephen at xemacs.org Tue Apr 30 06:57:51 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 30 Apr 2013 13:57:51 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <800AED25-ED93-4212-B7D6-DC198C0356FF@dataplex.net> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9uvsne.fsf@uwakimon.sk.tsukuba.ac.jp> <3D7B92D1-F665-4727-8ED0-30CE3E1152E3@dataplex.net> <87y5c1v5tq.fsf@uwakimon.sk.tsukuba.ac.jp> <800AED25-ED93-4212-B7D6-DC198C0356FF@dataplex.net> Message-ID: <87k3nkvcmo.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > There are two aspects of using a "generic tree". The first is some > customization of the tree to fit the installation's delegation > model. Here, I would suggest that "sample base installations" Once again, you're making an argument based on theory that nobody disagrees with, certainly not me, and not even attempting to address my request for use cases. We're not making progress. Let's just agree to disagree, and accept that there's a rather high probability that we can't work together. In commenting, I'll play the loyal opposition, and we can hope other developers will keep me honest. Regards, Steve From stephen at xemacs.org Tue Apr 30 07:28:38 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 30 Apr 2013 14:28:38 +0900 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: <517ED2CD.9010507@zone12.com> References: <516F3C7D.1070509@zone12.com> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> Message-ID: <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> Terri Oda writes: > 1. I would like to remind/tell you all that FOR THIS RELEASE, we will > ONLY be supporting Mozilla Persona and passwords as authentication > methods. > 3. While I agree that thinking about these things in advance is useful, > I think this discussion is starting to crowd out discussions of > time-sensitive things like students' GSoC applications and > architectural choices that directly impact how they write said > applications. Hold on! Who's "we" and what is being released? There are at least 3 more or less separate projects here (Mailman core, Postorius, and HyperKitty). For login of the webapps Postorius and HyperKitty, I certainly agree that aiming releasing them simultaneously with core 3.0, focusing on a particular authn mechanism (and given work done at the sprints, it should be Persona) is appropriate. But Mailman 3.0 is *not* the only item on the agenda here. We have a pile of GSoC students who want to implement various forms of authn for Mailman core functionality (secure lists) and some REST API (not clear to me which subproject these adhere to, the core Mailman REST API or a new one to be provided by Postorius). AFAICT, *Persona is not appropriate* for these use cases, as it implies interactive use of a full-featured web browser. Are you saying that (despite our ideas page) these proposals are a priori nearly unacceptable? It's also rather late in the process to change the rules on the students: if experience is any guide, Melange availability is going to start deteriorating soon. > So... can we please backburner parts of this discussion until after > GSoC so that we don't make it impossible for anyone to start on any > project involving extra profile info? I don't want to bog down a > student with the need to make this choice and work with these > decisions at this stage. Actually, there aren't any students explicitly discussing extra profile information in their proposals. They talk airily about "extended REST API" or similarly generic terms. None talk about storage or representation of profile information. (OK, it was 3am, my memory is fallible. I don't remember any, though.) > For the purposes of GSoC this summer: > > Authentication for the purposes of the extra profile info data store > will be provided by either Persona through the web interface or > passwords. OK. > Yes, even internally for the REST interface: passwords+localhost > are good enough for Mailman Core right now, so it's good enough for > anything else we do short-term. That's going to strongly bias decision-making against at least one REST API proposal, which was made in good faith. It also formally seems to rule out the secure lists proposal, which obviously depends on an off-line non-localhost authn protocol (based on OpenPGP or S/MIME). Is that your intention? Regards, From rkw at dataplex.net Tue Apr 30 12:22:40 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Tue, 30 Apr 2013 05:22:40 -0500 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: <87k3nkvcmo.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9uvsne.fsf@uwakimon.sk.tsukuba.ac.jp> <3D7B92D1-F665-4727-8ED0-30CE3E1152E3@dataplex.net> <87y5c1v5tq.fsf@uwakimon.sk.tsukuba.ac.jp> <800AED25-ED93-4212-B7D6-DC198C0356FF@dataplex.net> <87k3nkvcmo.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Steve, In my opinion, the normal use case doesn't even need the generality of "domains". Now that we are talking about only the few percent remaining installations, I have to seriously ask how many of those will not be handled by "power users"? My concern is that, in your effort to "protect" the less sophisticated, you are excluding some very few, but extremely "powerful" users. I share the goal of "world domination". I don't want to exclude anyone if we can avoid it. Richard On Apr 29, 2013, at 11:57 PM, Stephen J. Turnbull wrote: > Richard Wackerbarth writes: > >> There are two aspects of using a "generic tree". The first is some >> customization of the tree to fit the installation's delegation >> model. Here, I would suggest that "sample base installations" > > Once again, you're making an argument based on theory that nobody > disagrees with, certainly not me, and not even attempting to address > my request for use cases. > > We're not making progress. Let's just agree to disagree, and accept > that there's a rather high probability that we can't work together. > In commenting, I'll play the loyal opposition, and we can hope other > developers will keep me honest. > > Regards, > Steve > From rkw at dataplex.net Tue Apr 30 12:42:01 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Tue, 30 Apr 2013 05:42:01 -0500 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Steve, Again, we agree. (Something must be wrong :) ) I think that it is important that "we" (the senior developers and mentors) assure that we maintain a structure whereby each of the student proposals can proceed at the same time. Thus, for example, someone working on the User interface can do so without requiring the "secure" mail features. At the same time, we provide a place (Eg. the Credentials store) that can be used to hold public keys associated with individual subscribers. Now I will admit that I haven't thought about the criteria associated with permitting someone to add a public key-->user association. But, within whatever policy is needed, we have a method to handle the storage and the framework within which the system can "GetUserByCredential()", etc. Richard On Apr 30, 2013, at 12:28 AM, Stephen J. Turnbull wrote: > Terri Oda writes: > >> 1. I would like to remind/tell you all that FOR THIS RELEASE, we will >> ONLY be supporting Mozilla Persona and passwords as authentication >> methods. > >> 3. While I agree that thinking about these things in advance is useful, >> I think this discussion is starting to crowd out discussions of >> time-sensitive things like students' GSoC applications and >> architectural choices that directly impact how they write said >> applications. > > Hold on! Who's "we" and what is being released? There are at least 3 > more or less separate projects here (Mailman core, Postorius, and > HyperKitty). For login of the webapps Postorius and HyperKitty, I > certainly agree that aiming releasing them simultaneously with core > 3.0, focusing on a particular authn mechanism (and given work done at > the sprints, it should be Persona) is appropriate. > > But Mailman 3.0 is *not* the only item on the agenda here. We have a > pile of GSoC students who want to implement various forms of authn for > Mailman core functionality (secure lists) and some REST API (not clear > to me which subproject these adhere to, the core Mailman REST API or a > new one to be provided by Postorius). AFAICT, *Persona is not > appropriate* for these use cases, as it implies interactive use of a > full-featured web browser. > > Are you saying that (despite our ideas page) these proposals are a > priori nearly unacceptable? It's also rather late in the process to > change the rules on the students: if experience is any guide, Melange > availability is going to start deteriorating soon. > >> So... can we please backburner parts of this discussion until after >> GSoC so that we don't make it impossible for anyone to start on any >> project involving extra profile info? I don't want to bog down a >> student with the need to make this choice and work with these >> decisions at this stage. > > Actually, there aren't any students explicitly discussing extra profile > information in their proposals. They talk airily about "extended REST > API" or similarly generic terms. None talk about storage or > representation of profile information. (OK, it was 3am, my memory is > fallible. I don't remember any, though.) > > >> For the purposes of GSoC this summer: >> >> Authentication for the purposes of the extra profile info data store >> will be provided by either Persona through the web interface or >> passwords. > > OK. > >> Yes, even internally for the REST interface: passwords+localhost >> are good enough for Mailman Core right now, so it's good enough for >> anything else we do short-term. > > That's going to strongly bias decision-making against at least one > REST API proposal, which was made in good faith. > > It also formally seems to rule out the secure lists proposal, which > obviously depends on an off-line non-localhost authn protocol (based > on OpenPGP or S/MIME). Is that your intention? > > Regards, From p at sys4.de Tue Apr 30 12:47:50 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Tue, 30 Apr 2013 12:47:50 +0200 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: References: <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130430104749.GH386@sys4.de> Richard, * Richard Wackerbarth : > Again, we agree. (Something must be wrong :) ) I think that it is important that "we" (the senior developers and mentors) assure that we maintain a structure whereby each of the student proposals can proceed at the same time. I believe you have been asked very politely not to continue this thread yesterday and give way for more pressing topics such as GSoC. Can you please do so? p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From rkw at dataplex.net Tue Apr 30 13:14:25 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Tue, 30 Apr 2013 06:14:25 -0500 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: <20130430104749.GH386@sys4.de> References: <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> <20130430104749.GH386@sys4.de> Message-ID: <54DFA9E0-9AC5-4623-B740-9AA364EA4E0F@dataplex.net> Patrick, This IS about GSoC. If we accept Terri's position, as stated and without question, then we are, de facto, eliminating some of the proposals based only on subject matter. The students making these proposals are doing so in good faith, and often based on "ideas" provided by our organization. I consider that doing so is totally inappropriate. As I state, we need to maintain a structure whereby each of the proposed projects can proceed without being blocked by another project, but done in such a manner that they can be integrated into the whole as they develop into functioning code. Richard On Apr 30, 2013, at 5:47 AM, Patrick Ben Koetter

wrote: > Richard, > > * Richard Wackerbarth : >> Again, we agree. (Something must be wrong :) ) I think that it is important that "we" (the senior developers and mentors) assure that we maintain a structure whereby each of the student proposals can proceed at the same time. > > I believe you have been asked very politely not to continue this thread > yesterday and give way for more pressing topics such as GSoC. Can you please > do so? > > p at rick > > -- > [*] sys4 AG > > http://sys4.de, +49 (89) 30 90 46 64 > Franziskanerstra?e 15, 81669 M?nchen > > Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 > Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer > Aufsichtsratsvorsitzender: Florian Kirstein > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/rkw%40dataplex.net > > Security Policy: http://wiki.list.org/x/QIA9 From tom.browder at gmail.com Tue Apr 30 17:07:54 2013 From: tom.browder at gmail.com (Tom Browder) Date: Tue, 30 Apr 2013 10:07:54 -0500 Subject: [Mailman-Developers] Using Mailman scripts from cgi Message-ID: I'm probably going in the wrong direction, but I'm trying to create my own page for subscribers on an Ubuntu server. Before I get my cgi form working, I'm trying to use the command line script "add_members" and can't get the right syntax for using stdin. I've tried: $ ./add_member -r - listname joe at test.org and $ ./add_member -r - listname < joe at test.org but neither works. I just get the command's help output. I am in the bin dir with the script and I'm a member of the list group. What am I doing wrong? Thanks, -Tom From stephen at xemacs.org Tue Apr 30 17:14:26 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 01 May 2013 00:14:26 +0900 Subject: [Mailman-Developers] Architecture for extra profile info In-Reply-To: References: <516F3C7D.1070509@zone12.com> <87d2ts2qfm.fsf@uwakimon.sk.tsukuba.ac.jp> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <87wqrovz3b.fsf@uwakimon.sk.tsukuba.ac.jp> <0259DB9B-AB82-42C2-8E27-285D65F454AB@dataplex.net> <87li82ws18.fsf@uwakimon.sk.tsukuba.ac.jp> <86057C72-795B-49D8-94D7-BD2AD74A2A8B@dataplex.net> <87d2tew6ru.fsf@uwakimon.sk.tsukuba.ac.jp> <857AB8B1-88B5-4019-91E7-0E74E64F7116@dataplex.net> <877gjmvy9v.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9uvsne.fsf@uwakimon.sk.tsukuba.ac.jp> <3D7B92D1-F665-4727-8ED0-30CE3E1152E3@dataplex.net> <87y5c1v5tq.fsf@uwakimon.sk.tsukuba.ac.jp> <800AED25-ED93-4212-B7D6-DC198C0356FF@dataplex.net> <87k3nkvcmo.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87bo8wuk31.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > In my opinion, the normal use case doesn't even need the generality > of "domains". I'm not going to argue with you. Let's go get some code written. Steve From terri at zone12.com Tue Apr 30 17:31:10 2013 From: terri at zone12.com (Terri Oda) Date: Tue, 30 Apr 2013 09:31:10 -0600 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> References: <516F3C7D.1070509@zone12.com> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <517FE3BE.5050506@zone12.com> On 13-04-29 11:28 PM, Stephen J. Turnbull wrote: > Actually, there aren't any students explicitly discussing extra profile > information in their proposals. They talk airily about "extended REST > API" or similarly generic terms. None talk about storage or > representation of profile information. (OK, it was 3am, my memory is > fallible. I don't remember any, though.) > Oh! I think I understand some of the confusion now. I thought I'd said at the beginning why I was starting this thread at all, which is primarily because Systers has a student project that needs extra profile info. There are also a collection of students with minor features that would use the data store that they've used to pad out projects that might otherwise be too short (For example, see some of the ideas Peter Markou mentioned in his recent post). But now that you mention it, I've been talking to a lot of folk on IRC and they often don't tell me if they're applying with Mailman or Systers, so maybe there's more of them with Systers than with Mailman? For those not familiar, Systers runs a heavily modified version of Mailman 2.1. They'd like to switch to Mailman 3 shortly after we release Mailman Suite (whenever that's going to be), but they've got a few features that they'll need to either have integrated to core Mailman or maintained as branches. This year, they've got a project set up to port one of them to Mailman 3/Postorius as part of preparing them for that. The specific feature they're hoping to add this year stores essays that people write when they subscribe. It's currently done with a database grafted on to Mailman 2.1 that was being used for another extended feature, but obviously if we're going to mentor such a project for Mailman 3, we'd like to use something at least approximating a real data store design. The problem is that the Systers mentors are mostly used to systers-mailman, based on 2.1, and I'm not sure anyone will have the Mailman 3.0 knowledge to guide a student through a halfway reasonable design, so I asked here in order to help them out so that their student could start on the essays (likely to be the easier part of her project) right at the beginning of the GSoC period. If we can't come to a moderate consensus on that design, I should be telling the Systers folk that this project won't be able to run this year, but I honestly figured we'd have a simple design after a couple of posts... I wasn't counting on authentication blowing up to a huge argument, and I don't have time for it do so if we want to make a good decision about this specific project. So yeah, totally cool to be thinking about the more advanced projects in general, but I started this thread due to a specific need for a "good enough approximation for this Mailman suite release" version of the extra data store, and I still need a consensus on that that's either implemented or simple enough for a student to do in the first week of GSoC. I think we're pretty close to that point, but I'm not sure it's yet described in a way that we can hand it off safely to a student and get a REST API that actually meaningfully matches the discussion within a couple of days. Probably we need an actual set of doctests for the student to work against next and then we'll be there. Incidentally, I've been trying to get the students interested in this particular Systers project to come to this mailing list and join in the discussion, but it's become *very* intimidating to take part in this thread, which is the other reason I need folk to take a step back. Terri From rkw at dataplex.net Tue Apr 30 17:47:38 2013 From: rkw at dataplex.net (Richard Wackerbarth) Date: Tue, 30 Apr 2013 10:47:38 -0500 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: <517FE3BE.5050506@zone12.com> References: <516F3C7D.1070509@zone12.com> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> <517FE3BE.5050506@zone12.com> Message-ID: <4FFD0D2C-0592-4FF6-9463-E27B06A99E9F@dataplex.net> Terri, For Systers, is it better to try to access Postorius through its REST API or by directly including it as a django app? In particular, using Django 1.5 and the custom user module feature that it provides along with the scheme that I outlined earlier provides an easy path to developing and integrating features such as the essays into the user profile. Basically, we share a common User model. Someone can start by using the standard django.contrib model and ignore the Postorius specific parts. I am willing to implement the pieces (I estimate needing only a day or two if I do a simple version), but I also think that it would be an appropriate task for one of the students. Richard On Apr 30, 2013, at 10:31 AM, Terri Oda wrote: > On 13-04-29 11:28 PM, Stephen J. Turnbull wrote: >> Actually, there aren't any students explicitly discussing extra profile >> information in their proposals. They talk airily about "extended REST >> API" or similarly generic terms. None talk about storage or >> representation of profile information. (OK, it was 3am, my memory is >> fallible. I don't remember any, though.) >> > > Oh! I think I understand some of the confusion now. I thought I'd said at the beginning why I was starting this thread at all, which is primarily because Systers has a student project that needs extra profile info. There are also a collection of students with minor features that would use the data store that they've used to pad out projects that might otherwise be too short (For example, see some of the ideas Peter Markou mentioned in his recent post). But now that you mention it, I've been talking to a lot of folk on IRC and they often don't tell me if they're applying with Mailman or Systers, so maybe there's more of them with Systers than with Mailman? > > > For those not familiar, Systers runs a heavily modified version of Mailman 2.1. They'd like to switch to Mailman 3 shortly after we release Mailman Suite (whenever that's going to be), but they've got a few features that they'll need to either have integrated to core Mailman or maintained as branches. This year, they've got a project set up to port one of them to Mailman 3/Postorius as part of preparing them for that. > > The specific feature they're hoping to add this year stores essays that people write when they subscribe. It's currently done with a database grafted on to Mailman 2.1 that was being used for another extended feature, but obviously if we're going to mentor such a project for Mailman 3, we'd like to use something at least approximating a real data store design. The problem is that the Systers mentors are mostly used to systers-mailman, based on 2.1, and I'm not sure anyone will have the Mailman 3.0 knowledge to guide a student through a halfway reasonable design, so I asked here in order to help them out so that their student could start on the essays (likely to be the easier part of her project) right at the beginning of the GSoC period. If we can't come to a moderate consensus on that design, I should be telling the Systers folk that this project won't be able to run this year, but I honestly figured we'd have a simple design after a couple of posts... I wasn't counting on authentication blowing up to a huge argument, and I don't have time for it do so if we want to make a good decision about this specific project. > > So yeah, totally cool to be thinking about the more advanced projects in general, but I started this thread due to a specific need for a "good enough approximation for this Mailman suite release" version of the extra data store, and I still need a consensus on that that's either implemented or simple enough for a student to do in the first week of GSoC. I think we're pretty close to that point, but I'm not sure it's yet described in a way that we can hand it off safely to a student and get a REST API that actually meaningfully matches the discussion within a couple of days. Probably we need an actual set of doctests for the student to work against next and then we'll be there. > > Incidentally, I've been trying to get the students interested in this particular Systers project to come to this mailing list and join in the discussion, but it's become *very* intimidating to take part in this thread, which is the other reason I need folk to take a step back. > > Terri > > From stephen at xemacs.org Tue Apr 30 17:52:54 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 01 May 2013 00:52:54 +0900 Subject: [Mailman-Developers] About authentication (was Re: Architecture for extra profile info) In-Reply-To: <54DFA9E0-9AC5-4623-B740-9AA364EA4E0F@dataplex.net> References: <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> <20130430104749.GH386@sys4.de> <54DFA9E0-9AC5-4623-B740-9AA364EA4E0F@dataplex.net> Message-ID: <87a9oguiax.fsf@uwakimon.sk.tsukuba.ac.jp> > On Apr 30, 2013, at 5:47 AM, Patrick Ben Koetter

wrote: > > I believe you have been asked very politely not to continue this > > thread yesterday and give way for more pressing topics such as > > GSoC. Can you please do so? This is in rather bad taste. Richard and I have been actively involved in mentoring the GSoC students, and we care about them. Please respect that, at least to the extent of reading the messages we write about the project. Richard Wackerbarth writes: > I consider that [a priori ruling out submitted proposals] is > totally inappropriate. I won't go so far as to say "totally" inappropriate. We do have to get some work done, and if the consensus is that strict adherence to the letter of Terri's post is the best way to move along ... I'll deal with it. I hope we can find a compromise, though. Steve From stephen at xemacs.org Tue Apr 30 18:07:48 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 01 May 2013 01:07:48 +0900 Subject: [Mailman-Developers] Using Mailman scripts from cgi In-Reply-To: References: Message-ID: <877gjkuhm3.fsf@uwakimon.sk.tsukuba.ac.jp> Tom Browder writes: > I've tried: > $ ./add_member -r - listname joe at test.org > > and > > $ ./add_member -r - listname < joe at test.org > > but neither works. I just get the command's help output. > > I am in the bin dir with the script and I'm a member of the list group. echo joe at test.org | ./add_member -r - mailman works for me. (Tip: If I don't have a test list handy, I use the mailman list.) From stephen at xemacs.org Tue Apr 30 20:23:17 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 01 May 2013 03:23:17 +0900 Subject: [Mailman-Developers] Really: architecting extra profile info [was: About authentication...] In-Reply-To: <517FE3BE.5050506@zone12.com> References: <516F3C7D.1070509@zone12.com> <87y5cf1fhd.fsf@uwakimon.sk.tsukuba.ac.jp> <20130419111639.249f0241@anarchist> <517AD73F.4010702@zone12.com> <878v44xy4b.fsf@uwakimon.sk.tsukuba.ac.jp> <871u9wxjz8.fsf@uwakimon.sk.tsukuba.ac.jp> <87ppxfw2go.fsf@uwakimon.sk.tsukuba.ac.jp> <87ip36wlwg.fsf@uwakimon.sk.tsukuba.ac.jp> <874neqvv2j.fsf@uwakimon.sk.tsukuba.ac.jp> <517ED2CD.9010507@zone12.com> <87ip34vb7d.fsf@uwakimon.sk.tsukuba.ac.jp> <517FE3BE.5050506@zone12.com> Message-ID: <874nenvpwq.fsf@uwakimon.sk.tsukuba.ac.jp> Terri Oda writes: > Oh! I think I understand some of the confusion now. I thought I'd > said at the beginning why I was starting this thread at all, which > is primarily because Systers has a student project that needs extra > profile info. That was a long time ago, and ... none of the students actually applying to Mailman has expressed direct interest in the extra profile info. To the contrary, a couple have commented that crypto really interests them. stylistica posted here and has been on IRC off and on, but hasn't applied through Mailman. dardie will need a profile API, I think, but *her* proposal just showed up in the last hour or so and is an early draft. > The problem is ... somebody who knows the requirements needs to explain them on *Mailman* channels. We (more precisely, you, 'cause I don't know who to ask) need to get the principals in here, or in a conversation off-list. Then we'll work it out.