From barry at python.org Fri Jan 2 05:40:33 2015 From: barry at python.org (Barry Warsaw) Date: Thu, 1 Jan 2015 23:40:33 -0500 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> Message-ID: <20150101234033.0e5cee7a@anarchist> On Dec 27, 2014, at 01:16 PM, Florian Fuchs wrote: >So far we haven't found a perfect solution for testing packages that >rely heavily on the MM3 REST API. As far as mailman.client, Postorius >and HK are concerned, testing without *some* core integration doesn't >make too much sense IMO, because any changes in the API would go unnoticed. I'm playing with a solution that involves the use of the 'vcr' package, which as its name implies (well, at least for us, um, mature programmers ;) is a facility to record and replay HTTP traffic. It's compatible with both Python 2 and 3 and is available on the Cheeseshop. So the basic idea is to set up an MM3 installation (`tox --notest -r` suffices), then start up its REST server with a fresh mailman.db, and run the test suite against it. This will capture the traffic in a yaml file, which we could ship with mailman.client. Then to run the test suite, you'd use vcr in replay mode against the captured yaml file. Any time the API changes, you'd have to re-record the traffic against a fresh database. I have a branch that's working in this direction and it's promising, but I've also had to rewrite the testing infrastructure for mailman.client, and make some other changes. It also points out what I think are some bugs in Mailman core, so I'm working those out on my py3 branch in parallel. It's not yet ready for review, but if you want to keep an eye on things, look here: lp:~barry/mailman.client/bilingual Feel free to provide feedback on the approach, or the changes so far, but I will submit a merge proposal when/if it's ready to go. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From sumanah at panix.com Fri Jan 2 06:14:41 2015 From: sumanah at panix.com (Sumana Harihareswara) Date: Fri, 02 Jan 2015 00:14:41 -0500 Subject: [Mailman-Developers] new wiki, get-togethers, and 3.0 blockers Message-ID: <54A62941.7010801@panix.com> Hi Mailman developers! I am becoming one of you :) and have gotten as far as a first patch.[0] Congrats on the new 3.0b5 release! A few questions: * I saw on http://wiki.list.org that we're in the process of moving to a new wiki, and I saw https://mail.python.org/pipermail/mailman-developers/2014-October/024001.html in which I heard that we're still in the process of finishing up the conversion. Is there anything you need to help make this happen? Money for SSL certs, wiki gardening time, or anything else I can help with? * I'd love to get to hack with some of you in person. Are any of you planning on going to the PyCon sprints in Montreal April 13-16? https://us.pycon.org/2015/community/sprints/ I live in New York City and can get to DC and Boston pretty easily in case anyone would like to have a little hacking weekend. * I saw in the https://mail.python.org/pipermail/mailman-developers/2014-November/024056.html conversation that I should be looking at https://bugs.launchpad.net/postorius/+bugs?field.tag=mailman3-suite-blocker for Mailman 3 blockers. It looks like there are 13 blocker bugs left; is that still right? Thanks all! [0] https://code.launchpad.net/~sumanah/mailman/mailman/+merge/244077 -- Sumana Harihareswara http://brainwane.net From mark at msapiro.net Fri Jan 2 19:35:56 2015 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 02 Jan 2015 10:35:56 -0800 Subject: [Mailman-Developers] new wiki, get-togethers, and 3.0 blockers In-Reply-To: <54A62941.7010801@panix.com> References: <54A62941.7010801@panix.com> Message-ID: <54A6E50C.6020803@msapiro.net> On 01/01/2015 09:14 PM, Sumana Harihareswara wrote: > > * I'd love to get to hack with some of you in person. Are any of you > planning on going to the PyCon sprints in Montreal April 13-16? Several of us (me for one) are planning to sprint on Mailman at Pycon. Please join us if you can. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Fri Jan 2 22:38:51 2015 From: barry at list.org (Barry Warsaw) Date: Fri, 2 Jan 2015 16:38:51 -0500 Subject: [Mailman-Developers] new wiki, get-togethers, and 3.0 blockers In-Reply-To: <54A62941.7010801@panix.com> References: <54A62941.7010801@panix.com> Message-ID: <20150102163851.4f46b517@anarchist> On Jan 02, 2015, at 12:14 AM, Sumana Harihareswara wrote: >* I saw on http://wiki.list.org that we're in the process of moving to a new >wiki, and I saw >https://mail.python.org/pipermail/mailman-developers/2014-October/024001.html >in which I heard that we're still in the process of finishing up the >conversion. Is there anything you need to help make this happen? Money for >SSL certs, wiki gardening time, or anything else I can help with? I need to ping John again since he owns the domain. I think we just need to get the cert and flip the DNS and then we'll be done. For now, wiki.mailman.psf.io will get you to the new Moin-based wiki. We would certainly appreciate some gardening and maybe even a bit of styling. >* I'd love to get to hack with some of you in person. Are any of you >planning on going to the PyCon sprints in Montreal April 13-16? Yep. >* I saw in the >https://mail.python.org/pipermail/mailman-developers/2014-November/024056.html >conversation that I should be looking at >https://bugs.launchpad.net/postorius/+bugs?field.tag=mailman3-suite-blocker >for Mailman 3 blockers. It looks like there are 13 blocker bugs left; is that >still right? I'll leave the Postorious discussion to Florian. For the core, I want to merge the py3 port, update mailman.client to not require `mailman` package imports, and re-visit the subscription policy branch. It would be really awesome if we can do a final release of the full suite at Pycon. >[0] https://code.launchpad.net/~sumanah/mailman/mailman/+merge/244077 Cool. I saw this before, but thanks for the reminder. I'll take a look at the branch after I land the py3 port. Cheers, -Barry From flo.fuchs at gmail.com Sat Jan 3 22:16:17 2015 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Sat, 03 Jan 2015 22:16:17 +0100 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <20150101234033.0e5cee7a@anarchist> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> Message-ID: <54A85C21.6050301@gmail.com> Am 02.01.2015 um 05:40 schrieb Barry Warsaw: > On Dec 27, 2014, at 01:16 PM, Florian Fuchs wrote: > >> So far we haven't found a perfect solution for testing packages that >> rely heavily on the MM3 REST API. As far as mailman.client, Postorius >> and HK are concerned, testing without *some* core integration doesn't >> make too much sense IMO, because any changes in the API would go unnoticed. > > I'm playing with a solution that involves the use of the 'vcr' package, which > as its name implies (well, at least for us, um, mature programmers ;) is a > facility to record and replay HTTP traffic. It's compatible with both Python > 2 and 3 and is available on the Cheeseshop. I took a quick look at vcr and it really looks promising. Although apparently it doesn't support VHS, which is very sad. But we can still use it for testing I guess... :-) > So the basic idea is to set up an MM3 installation (`tox --notest -r` > suffices), then start up its REST server with a fresh mailman.db, and run the > test suite against it. This will capture the traffic in a yaml file, which > we could ship with mailman.client. > > Then to run the test suite, you'd use vcr in replay mode against the captured > yaml file. Any time the API changes, you'd have to re-record the traffic > against a fresh database. That sounds fantastic! Do you think it might make sense to ship the traffic dump with mailman instead of mailman.client (and work the recording into the test suite somehow)? That way there would only be a single file to observe for changes in the REST API. Sure, a change in the yaml file doesn't necessarily mean a change in the API (might as well be just an additional test case or different test data). But still, if the yaml file stays the same between commits, that would be a nice guarantee that the API stayed stable. > I have a branch that's working in this direction and it's promising, but I've > also had to rewrite the testing infrastructure for mailman.client, and make > some other changes. It also points out what I think are some bugs in Mailman > core, so I'm working those out on my py3 branch in parallel. > > It's not yet ready for review, but if you want to keep an eye on things, look > here: lp:~barry/mailman.client/bilingual > > Feel free to provide feedback on the approach, or the changes so far, but I > will submit a merge proposal when/if it's ready to go. Thanks for starting the port, Barry. Looking forward to seeing that merge proposal (although I don't expect I'll have a too much to add... ;-). Florian From flo.fuchs at gmail.com Sat Jan 3 22:43:01 2015 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Sat, 03 Jan 2015 22:43:01 +0100 Subject: [Mailman-Developers] new wiki, get-togethers, and 3.0 blockers In-Reply-To: <54A62941.7010801@panix.com> References: <54A62941.7010801@panix.com> Message-ID: <54A86265.9030901@gmail.com> Am 02.01.2015 um 06:14 schrieb Sumana Harihareswara: > * I'd love to get to hack with some of you in person. Are any of you > planning on going to the PyCon sprints in Montreal April 13-16? Yup. > * I saw in the > https://mail.python.org/pipermail/mailman-developers/2014-November/024056.html > conversation that I should be looking at > https://bugs.launchpad.net/postorius/+bugs?field.tag=mailman3-suite-blocker > for Mailman 3 blockers. It looks like there are 13 blocker bugs left; is > that still right? Some of them already have patches attached to them and just haven't been merged yet, and a few others are already in progress. So there are currently 5 bugs with no one assigned to it: https://bugs.launchpad.net/postorius/+bug/987100 https://bugs.launchpad.net/postorius/+bug/1062963 https://bugs.launchpad.net/postorius/+bug/1004049 https://bugs.launchpad.net/postorius/+bug/1006356 https://bugs.launchpad.net/postorius/+bug/1201150 Would be great if any of these would spark your interest! :-) Florian From flo.fuchs at gmail.com Sat Jan 3 22:54:51 2015 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Sat, 03 Jan 2015 22:54:51 +0100 Subject: [Mailman-Developers] Python 3 In-Reply-To: <20141228182740.00e6613e@anthem> References: <20141222134734.698db176@anthem> <20141226161128.408d9ead@anarchist.wooz.org> <8738814v2u.fsf@uwakimon.sk.tsukuba.ac.jp> <20141226231846.17eb0ab8@anarchist.wooz.org> <549E62C4.2020602@gmail.com> <20141228182740.00e6613e@anthem> Message-ID: <54A8652B.40606@gmail.com> Am 29.12.2014 um 00:27 schrieb Barry Warsaw: > On Dec 27, 2014, at 08:41 AM, Florian Fuchs wrote: > >> How about a third option, a generic "pub/sub" or "event" archiver, >> implemented in py3? It would meet all the above criteria (archivers on other >> systems, no dependency on py3 -- or Python for that matter). >> >> We could start supporting one backend (zeromq for instance) and maybe >> add more later. > > This would be pretty cool. Would you be interested in writing an IArchiver > implementation for that? Sure. I don't know how much interest there is out there in something like this. But I think I like the idea enough to just do it anyway. :-) Florian From flo.fuchs at gmail.com Sat Jan 3 23:42:13 2015 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Sat, 03 Jan 2015 23:42:13 +0100 Subject: [Mailman-Developers] Python 3 In-Reply-To: <20141228182648.76d40d96@anthem> References: <20141222134734.698db176@anthem> <20141226161128.408d9ead@anarchist.wooz.org> <8738814v2u.fsf@uwakimon.sk.tsukuba.ac.jp> <20141228182648.76d40d96@anthem> Message-ID: <54A87045.5080608@gmail.com> Am 29.12.2014 um 00:26 schrieb Barry Warsaw: > On Dec 28, 2014, at 01:51 PM, Aurelien Bompard wrote: > >>> As I mentioned, I think LMTP *could* work, but REST (inside HK) could >>> work too. Aurelien, what do you think? >> >> I'd go with REST, it seems more flexible and we already have nice libraries >> for it. > > REST will be very nice, and I think with something like Falcon, pretty easy to > implement. You probably don't need the object-based (i.e. restish-like) > traversal stuff, but if so, I'd be happy to split that out into a separately > library that HK could use (and keep it bilingual). Keep it simple, but think > through the resource tree and leave open some opportunities for expanding the > API later, both for use by MM3 and perhaps other scripting applications. Are you planning to implement that in a way similar to the IArchiver interface? Like, say, having a REST-based equivalent for each of the IArchiver's `list_url`, `permalink` and `archive_message` methods? I think it would be nice if it worked more or less the same as the an IArchiver class, not only on the implementation side but also as far as the configuration is concerned. Although allowing multiple, potentially slow-answering, REST-based archivers could be a trap performance-wise (if those requests are not made concurrently). Florian From andrew.stuart at supercoders.com.au Sat Jan 3 23:44:21 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sun, 4 Jan 2015 09:44:21 +1100 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <20150101234033.0e5cee7a@anarchist> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> Message-ID: <63C07C1D-2CBC-4130-AF60-DDCC25F03ADB@supercoders.com.au> FWIW some notes on mitmproxy and ngrep - helpful tools when intercepting and recording HTTP traffic. mitmproxy/mitmdump: I know doesn?t serve the same purpose as VCR but its helpful to be able to capture and see what is going on between server and client when Mailman tests are running. https://mitmproxy.org/doc/mitmdump.html mitmproxy itself seems to have Python 3 as its next major initiative. I configured Mailman tests to run through a proxy (I sent an email to this list a few weeks back with instructions for how to do this). Then: mitmdump -p 7001 -P http://127.0.0.1:9001 -w outfile Captures traffic to outfile. It can be loaded back into mitmproxy and modified and replayed. **ngrep ngrep is useful when digging deeper. it will show you traffic in real time on a given port. for example sudo ngrep -W byline -d lo port 9001 will show what is happening on the Mailman REST API port when configured for port 9001 Dunno if this is useful for anyone but everything helps when digging into HTTP. as On 2 Jan 2015, at 3:40 pm, Barry Warsaw wrote: On Dec 27, 2014, at 01:16 PM, Florian Fuchs wrote: > So far we haven't found a perfect solution for testing packages that > rely heavily on the MM3 REST API. As far as mailman.client, Postorius > and HK are concerned, testing without *some* core integration doesn't > make too much sense IMO, because any changes in the API would go unnoticed. I'm playing with a solution that involves the use of the 'vcr' package, which as its name implies (well, at least for us, um, mature programmers ;) is a facility to record and replay HTTP traffic. It's compatible with both Python 2 and 3 and is available on the Cheeseshop. So the basic idea is to set up an MM3 installation (`tox --notest -r` suffices), then start up its REST server with a fresh mailman.db, and run the test suite against it. This will capture the traffic in a yaml file, which we could ship with mailman.client. Then to run the test suite, you'd use vcr in replay mode against the captured yaml file. Any time the API changes, you'd have to re-record the traffic against a fresh database. I have a branch that's working in this direction and it's promising, but I've also had to rewrite the testing infrastructure for mailman.client, and make some other changes. It also points out what I think are some bugs in Mailman core, so I'm working those out on my py3 branch in parallel. It's not yet ready for review, but if you want to keep an eye on things, look here: lp:~barry/mailman.client/bilingual Feel free to provide feedback on the approach, or the changes so far, but I will submit a merge proposal when/if it's ready to go. Cheers, -Barry _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From andrew.stuart at supercoders.com.au Sun Jan 4 00:06:50 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sun, 4 Jan 2015 10:06:50 +1100 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <20150101234033.0e5cee7a@anarchist> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> Message-ID: <395EC375-D931-4C44-A16C-B26ED3FE6B06@supercoders.com.au> My understanding is that VCR will capture the HTTP test request/responses and comparison to some baseline will indicate that the API is behaving as expected. Forgive me if I?m misunderstanding something fundamental about the VCR based testing approach. Some questions to helop me understand: Are the HTTP responses going to be the same between runs and between systems? How deep are the comparisons of the HTTP responses? Are you thinking of doing byte level comparisions of the entire response data, including headers etc? Often the contents of the HTTP headers vary for a whole bunch of reasons. Also if there is a proxy in the middle that can easily rewrite the whole request and response. I?m guessing a byte level comparison of the responses would fail. If it was the content of the responses being compared then that?s likely to work but that?s not really what VCR is about, right? Am I understanding the VCR approach correctly? as On 2 Jan 2015, at 3:40 pm, Barry Warsaw wrote: On Dec 27, 2014, at 01:16 PM, Florian Fuchs wrote: > So far we haven't found a perfect solution for testing packages that > rely heavily on the MM3 REST API. As far as mailman.client, Postorius > and HK are concerned, testing without *some* core integration doesn't > make too much sense IMO, because any changes in the API would go unnoticed. I'm playing with a solution that involves the use of the 'vcr' package, which as its name implies (well, at least for us, um, mature programmers ;) is a facility to record and replay HTTP traffic. It's compatible with both Python 2 and 3 and is available on the Cheeseshop. So the basic idea is to set up an MM3 installation (`tox --notest -r` suffices), then start up its REST server with a fresh mailman.db, and run the test suite against it. This will capture the traffic in a yaml file, which we could ship with mailman.client. Then to run the test suite, you'd use vcr in replay mode against the captured yaml file. Any time the API changes, you'd have to re-record the traffic against a fresh database. I have a branch that's working in this direction and it's promising, but I've also had to rewrite the testing infrastructure for mailman.client, and make some other changes. It also points out what I think are some bugs in Mailman core, so I'm working those out on my py3 branch in parallel. It's not yet ready for review, but if you want to keep an eye on things, look here: lp:~barry/mailman.client/bilingual Feel free to provide feedback on the approach, or the changes so far, but I will submit a merge proposal when/if it's ready to go. Cheers, -Barry _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From andrew.stuart at supercoders.com.au Sun Jan 4 00:21:26 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sun, 4 Jan 2015 10:21:26 +1100 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <20150101234033.0e5cee7a@anarchist> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> Message-ID: <42B54F9D-E85E-4B17-9352-5BC788C6A304@supercoders.com.au> I feel like prefacing all my questions with an apology if it?s a dumb question. Anyway... What?s the advantage of using VCR over a set of tests that use Kenneth Reitz requests library to send HTTP queries to the API and have a look at the response to see if it contains the expected HTTP codes/data? thanks On 2 Jan 2015, at 3:40 pm, Barry Warsaw wrote: On Dec 27, 2014, at 01:16 PM, Florian Fuchs wrote: > So far we haven't found a perfect solution for testing packages that > rely heavily on the MM3 REST API. As far as mailman.client, Postorius > and HK are concerned, testing without *some* core integration doesn't > make too much sense IMO, because any changes in the API would go unnoticed. I'm playing with a solution that involves the use of the 'vcr' package, which as its name implies (well, at least for us, um, mature programmers ;) is a facility to record and replay HTTP traffic. It's compatible with both Python 2 and 3 and is available on the Cheeseshop. So the basic idea is to set up an MM3 installation (`tox --notest -r` suffices), then start up its REST server with a fresh mailman.db, and run the test suite against it. This will capture the traffic in a yaml file, which we could ship with mailman.client. Then to run the test suite, you'd use vcr in replay mode against the captured yaml file. Any time the API changes, you'd have to re-record the traffic against a fresh database. I have a branch that's working in this direction and it's promising, but I've also had to rewrite the testing infrastructure for mailman.client, and make some other changes. It also points out what I think are some bugs in Mailman core, so I'm working those out on my py3 branch in parallel. It's not yet ready for review, but if you want to keep an eye on things, look here: lp:~barry/mailman.client/bilingual Feel free to provide feedback on the approach, or the changes so far, but I will submit a merge proposal when/if it's ready to go. Cheers, -Barry _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From jeff at jab.org Mon Jan 5 00:10:00 2015 From: jeff at jab.org (Jeff Breidenbach) Date: Sun, 4 Jan 2015 15:10:00 -0800 Subject: [Mailman-Developers] X-Message-ID-Hash header (was Re: Python 3) In-Reply-To: References: <20141222134734.698db176@anthem> <20141226161128.408d9ead@anarchist.wooz.org> <8738814v2u.fsf@uwakimon.sk.tsukuba.ac.jp> <20141226231846.17eb0ab8@anarchist.wooz.org> <87y4pt37gr.fsf@uwakimon.sk.tsukuba.ac.jp> <20141228175108.1d905288@anthem> <87ppb32r83.fsf@uwakimon.sk.tsukuba.ac.jp> <20141229130206.18c3e335@anthem> Message-ID: I should probably report the experience of mail-archive.com, which has supported a homebrew message-id hash algorithm for over 6 years. Only one organization has ever used it (LibreOffice). They put it in the Archived-At headers, not message footers. However, links did end up getting manually referenced in message bodies 575 times. Folks here might want to take a look at how people used it. Christian Lohmaier from LibreOffice is a good contact if anyone wants to talk to him directly. The homebrew has algorithm turned out to be a pretty bad design. Base-64 got us in trouble. One memorable bug involved hashes prepended with a minus sign. This caused havoc with the search engine resolving the links, who interpreted the minus sign as a negation operator. Blending the List-Post address into the hash was supposed to resolve crossposts. In practice it complicated things enough that LibreOffice incorrectly computed some of the hashes. There were also times when mail-archive.com wasn't doing that great a job either of running everything smoothly. Finally, we really get in trouble when asked to merge multiple archives, due to a mailing list migrating from server to server. It happens, and those migrations often change both List-Post and List-Id. Blending either into the hash is a headache. So the homebrew hash was dumb idea, and will gracefully retire in favor of the standard mailman hash algorithm. It was pretty neat to see some usage, though. And Archived-At still looks like a winner. Cheers, Jeff Alogithm: https://www.mail-archive.com/faq.html#listserver Usage: https://www.mail-archive.com/search?l=all&q=go.mail-archive.com From kon.nalpa at gmail.com Mon Jan 5 19:09:12 2015 From: kon.nalpa at gmail.com (Konstantinos N) Date: Mon, 05 Jan 2015 20:09:12 +0200 Subject: [Mailman-Developers] Introduction and GSoC project requests Message-ID: <54AAD348.4020005@gmail.com> Hello mailman devs and Happy New Year. Since this summer i have been looking at the mailman project as it was the first project that attracted me, being a tool i use at my university to collaborate with my fellow sysadmins. I am generally interested in participating in GSoC at some time but first i would like to start contributing in some way so i can develop better programming skills,learn code researching and bugfixing and generally give back to software that helps me do my work better. The last days i asked around in the irc and gathered resources,links and started learning more about the mailman project.In detail i read about the architecture of mailman 3, differences with mailman 2 and tried reading almost anything i could find helpful in the wiki so i can better understand how mailman really works. I haven't set up my own instance of mailman at the moment because of some hardware problems but i have started reading some code in general, plus looking at the easy bugs listed in the wiki. On to the requests now. I saw two project ideas for GSoC 2014 that really interested me and i would like some guidance around them so i could read more on them. Could someone give me some status on the implementation of them at the moment? Were these ideas for mailman 3, mailman 2 or both? The 2 ideas are 1. Full anonymization 2. No-logging Mode Thank you in advance Konstantinos N From barry at list.org Tue Jan 6 04:24:29 2015 From: barry at list.org (Barry Warsaw) Date: Mon, 5 Jan 2015 22:24:29 -0500 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <20150101234033.0e5cee7a@anarchist> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> Message-ID: <20150105222429.1a8bef26@marathon> On Jan 01, 2015, at 11:40 PM, Barry Warsaw wrote: >I'm playing with a solution that involves the use of the 'vcr' package So this branch is now ready for review. https://code.launchpad.net/~barry/mailman.client/bilingual/+merge/245537 I even added documentation on how to run the test to record the yaml file containing the HTTP traffic. There are tox rules for doing both the recording, and for running the test suite with the recorded traffic against both Python 2 and Python 3. I've only tested 2.7 and 3.4 since I don't currently have good builds of 2.6, 3.2, or 3.3, but the tox.ini file should support those. There are now no imports of any `mailman` modules. I did have to add a new top-level `queues` resource to the REST API which will allow you to inject messages, query, and delete messages from the various queues. I thought this was a pretty handy feature anyway, and it was easy to add, so that's now in the Mailman 3 trunk. Which btw, as you might notice, I did go ahead and merge the py3 branch. I think with this mailman.client approach and a REST-based HK IArchiver implementation (TBD), it was safe enough to do, and I didn't want to have to continue to implement new features in both branches (e.g. the queues resource). Please have a look and let me know what you think. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From barry at list.org Tue Jan 6 04:26:56 2015 From: barry at list.org (Barry Warsaw) Date: Mon, 5 Jan 2015 22:26:56 -0500 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <42B54F9D-E85E-4B17-9352-5BC788C6A304@supercoders.com.au> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> <42B54F9D-E85E-4B17-9352-5BC788C6A304@supercoders.com.au> Message-ID: <20150105222656.3fc9f6fb@marathon> On Jan 04, 2015, at 10:21 AM, Andrew Stuart wrote: >What?s the advantage of using VCR over a set of tests that use Kenneth Reitz >requests library to send HTTP queries to the API and have a look at the >response to see if it contains the expected HTTP codes/data? The latter approach requires a running MM3 core REST server. The vcr approach only requires that when the core's REST API changes, i.e. in order to generate a new tape.yaml file. The nice thing about vcr is that it integrates with all kinds of HTTP libraries, including httplib2 which is what mailman.client uses, so the test suite just adds a simple context manager to go through vcr when in replay mode. Cheers, -Barry From barry at list.org Tue Jan 6 04:30:02 2015 From: barry at list.org (Barry Warsaw) Date: Mon, 5 Jan 2015 22:30:02 -0500 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <395EC375-D931-4C44-A16C-B26ED3FE6B06@supercoders.com.au> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> <395EC375-D931-4C44-A16C-B26ED3FE6B06@supercoders.com.au> Message-ID: <20150105223002.553a57d4@marathon> On Jan 04, 2015, at 10:06 AM, Andrew Stuart wrote: >Are the HTTP responses going to be the same between runs and between systems? Yes, they should be. If you think about it, GETs are supposed to be idempotent, so those won't change (even the etags) between requests. As it turns out, if replayed in the same order they were recorded (which I think is guaranteed by a full tox run via nose2, as my mailman.client branch adds), then even non-idempotent requests will have the same responses. >How deep are the comparisons of the HTTP responses? Are you thinking of >doing byte level comparisions of the entire response data, including headers >etc? I know that vcr has some support for tweaking the match between requests and responses, but I didn't find those to be necessary so I didn't dig into that very deeply. Cheers, -Barry From barry at list.org Tue Jan 6 04:33:14 2015 From: barry at list.org (Barry Warsaw) Date: Mon, 5 Jan 2015 22:33:14 -0500 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <63C07C1D-2CBC-4130-AF60-DDCC25F03ADB@supercoders.com.au> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> <63C07C1D-2CBC-4130-AF60-DDCC25F03ADB@supercoders.com.au> Message-ID: <20150105223314.0c4de74b@marathon> On Jan 04, 2015, at 09:44 AM, Andrew Stuart wrote: >mitmproxy/mitmdump: I know doesn?t serve the same purpose as VCR but its >helpful to be able to capture and see what is going on between server and >client when Mailman tests are running. mitm* look pretty cool. mailman.client needs to be bilingual (Python 2 and 3) so these might end up being interesting once they port mitm* to Python 3 (unless it doesn't matter because they're never imported, I guess). Cheers, -Barry From barry at list.org Tue Jan 6 04:37:02 2015 From: barry at list.org (Barry Warsaw) Date: Mon, 5 Jan 2015 22:37:02 -0500 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <54A85C21.6050301@gmail.com> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> <54A85C21.6050301@gmail.com> Message-ID: <20150105223702.75e9fa45@marathon> On Jan 03, 2015, at 10:16 PM, Florian Fuchs wrote: >I took a quick look at vcr and it really looks promising. Although >apparently it doesn't support VHS, which is very sad. But we can still >use it for testing I guess... :-) :) >That sounds fantastic! Do you think it might make sense to ship the >traffic dump with mailman instead of mailman.client (and work the >recording into the test suite somehow)? That way there would only be a >single file to observe for changes in the REST API. Sure, a change in >the yaml file doesn't necessarily mean a change in the API (might as >well be just an additional test case or different test data). But still, >if the yaml file stays the same between commits, that would be a nice >guarantee that the API stayed stable. It's possible. One reason I'm shipping it with mailman.client in my merge proposal is because once it's generated, there's no second guessing in mmclient about where the yaml file lives. If we shipped it with core, you'd have to have some way to tell mmclient where to find the yaml file, since different developers may be working in completely different file system locations of course. But do let me know what you think about the approach in my mp. I'm definitely not opposed to moving it if it makes sense. Of course we also talked about making mmclient an actual subpackage of core and placing it in core's tree. That wasn't very convenient at the time, but maybe that's worth a revisit at some point. Cheers, -Barry From aurelien at bompard.org Tue Jan 6 13:37:02 2015 From: aurelien at bompard.org (Aurelien Bompard) Date: Tue, 6 Jan 2015 13:37:02 +0100 Subject: [Mailman-Developers] Python 3 In-Reply-To: <54A87045.5080608@gmail.com> References: <20141222134734.698db176@anthem> <20141226161128.408d9ead@anarchist.wooz.org> <8738814v2u.fsf@uwakimon.sk.tsukuba.ac.jp> <20141228182648.76d40d96@anthem> <54A87045.5080608@gmail.com> Message-ID: I must also consider how much work it would be to just port HyperKitty & KittyStore to Python3. All things considered, it may very well be faster and more reliable. A. From barry at list.org Tue Jan 6 13:48:48 2015 From: barry at list.org (Barry Warsaw) Date: Tue, 6 Jan 2015 07:48:48 -0500 Subject: [Mailman-Developers] Python 3 In-Reply-To: References: <20141222134734.698db176@anthem> <20141226161128.408d9ead@anarchist.wooz.org> <8738814v2u.fsf@uwakimon.sk.tsukuba.ac.jp> <20141228182648.76d40d96@anthem> <54A87045.5080608@gmail.com> Message-ID: <20150106074848.38798792@limelight.wooz.org> On Jan 06, 2015, at 01:37 PM, Aurelien Bompard wrote: >I must also consider how much work it would be to just port HyperKitty & >KittyStore to Python3. All things considered, it may very well be faster and >more reliable. Of course, I encourage any porting to Python 3 just on principle :) and certainly if I can help, let me know. But you may still want to consider a more distributed communication protocol between core and HK. What if a site wanted to run them on different VMs for example? Cheers, -Barry From flo.fuchs at gmail.com Tue Jan 6 14:11:37 2015 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Tue, 06 Jan 2015 14:11:37 +0100 Subject: [Mailman-Developers] Python 3 In-Reply-To: <20150106074848.38798792@limelight.wooz.org> References: <20141222134734.698db176@anthem> <20141226161128.408d9ead@anarchist.wooz.org> <8738814v2u.fsf@uwakimon.sk.tsukuba.ac.jp> <20141228182648.76d40d96@anthem> <54A87045.5080608@gmail.com> <20150106074848.38798792@limelight.wooz.org> Message-ID: <54ABDF09.4080507@gmail.com> Am 06.01.2015 um 13:48 schrieb Barry Warsaw: > On Jan 06, 2015, at 01:37 PM, Aurelien Bompard wrote: > >> I must also consider how much work it would be to just port HyperKitty & >> KittyStore to Python3. All things considered, it may very well be faster and >> more reliable. > > Of course, I encourage any porting to Python 3 just on principle :) and > certainly if I can help, let me know. But you may still want to consider a > more distributed communication protocol between core and HK. What if a site > wanted to run them on different VMs for example? Another thing we shouldn't forget is that Hyperkitty or Postorius might not be the only apps living in a Django project. Some people might choose to integrate them into their long-running Django sites, with lots of Py2-app dependencies. Of course it would be easier for us to just port our apps to Py3. But as far as I remember integratability was the main argument in favor of keeping our Django apps bilingual when we discussed the topic at Pycon last year. But then, hey, one *could* also argue we ignore these issues in order to make a point. Florian From aurelien at bompard.org Tue Jan 6 14:55:16 2015 From: aurelien at bompard.org (Aurelien Bompard) Date: Tue, 6 Jan 2015 14:55:16 +0100 Subject: [Mailman-Developers] Python 3 In-Reply-To: <54ABDF09.4080507@gmail.com> References: <20141222134734.698db176@anthem> <20141226161128.408d9ead@anarchist.wooz.org> <8738814v2u.fsf@uwakimon.sk.tsukuba.ac.jp> <20141228182648.76d40d96@anthem> <54A87045.5080608@gmail.com> <20150106074848.38798792@limelight.wooz.org> <54ABDF09.4080507@gmail.com> Message-ID: > > What if a site wanted to run them on different VMs for example? > Yeah, but certainly there should be a limit to that. What if a site wanted to run each mailman runner on a different VM for example? Oh well, I guess there's no other way. I hope it's not going to be a mess to configure, I'd like to avoid having the admin set HK's base URL in 3 different places. Another thing we shouldn't forget is that Hyperkitty or Postorius might > not be the only apps living in a Django project. Some people might > choose to integrate them into their long-running Django sites, with lots > of Py2-app dependencies. > Very good point. Thanks. Aur?lien From kon.nalpa at gmail.com Tue Jan 6 18:39:11 2015 From: kon.nalpa at gmail.com (Konstantinos N) Date: Tue, 06 Jan 2015 19:39:11 +0200 Subject: [Mailman-Developers] Introduction and GSoC project requests In-Reply-To: <54AAD348.4020005@gmail.com> References: <54AAD348.4020005@gmail.com> Message-ID: <54AC1DBF.10008@gmail.com> Hello again Mailman developers. I have received some more guidance so i wanted to correct some mistakes in my previous email. On 01/05/2015 08:09 PM, Konstantinos N wrote: > On to the requests now. I saw two project ideas for GSoC 2014 that > really interested me and i would like some guidance around them so i > could read more on them. Could someone give me some status on the > implementation of them at the moment? Were these ideas for mailman 3, > mailman 2 or both? > The 2 ideas are > 1. Full anonymization [2] > 2. No-logging Mode [3] I read again the GSoC wiki page for 2014 [1] and saw that Mailman 2 development has been frozen so the projects i was asking about are indeed for Mailman 3. Still, i would like all the help you could give me on these 2 subjects. Specifically: Is anyone working on implementing either of these ideas right now? Are there any more specifications i could read or similar features in other projects or services that i could use as reference? Thank you again Konstantinos N [1]http://wiki.list.org/display/DEV/Google+Summer+of+Code+2014 [2]http://wiki.list.org/display/DEV/Google+Summer+of+Code+2014#GoogleSummerofCode2014-Fullanonymization [3]http://wiki.list.org/display/DEV/Google+Summer+of+Code+2014#GoogleSummerofCode2014-No-loggingmode From tanstaafl at libertytrek.org Tue Jan 6 18:48:20 2015 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Tue, 06 Jan 2015 12:48:20 -0500 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <20141230143938.48b96570@marathon> References: <20141230143938.48b96570@marathon> Message-ID: <54AC1FE4.2040506@libertytrek.org> On 12/30/2014 2:39 PM, Barry Warsaw wrote: > * "A" release, which remains on Python 2.7 > * "B" release, which is only compatible with Python 3.4 So, wheezy admins will be left out in the cold. Bummer... :( From mark at msapiro.net Tue Jan 6 19:04:43 2015 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 06 Jan 2015 10:04:43 -0800 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <54AC1FE4.2040506@libertytrek.org> References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> Message-ID: <54AC23BB.8030801@msapiro.net> On 01/06/2015 09:48 AM, Tanstaafl wrote: > On 12/30/2014 2:39 PM, Barry Warsaw wrote: >> * "A" release, which remains on Python 2.7 >> * "B" release, which is only compatible with Python 3.4 > > So, wheezy admins will be left out in the cold. > > Bummer... :( See the last paragraph of Barry's post at . Multiple Python versions can coexist happily side by side. For example, on my production CentOS 5 server, I have (uptime included as partial explanation of why it's CentOS 5) [mark at sbh16 ~]$ uptime 09:55:41 up 1821 days, 17:04, 1 user, load average: 0.00, 0.03, 0.08 [mark at sbh16 ~]$ python -V Python 2.4.3 [mark at sbh16 ~]$ python2.7 -V Python 2.7.9 [mark at sbh16 ~]$ python3 -V Python 3.4.2 I submit that for quite some time it will be significantly easier to install an alternate Python from source than it will be to install and configure and migrate to any Mailman 3, so just consider the Python installation a small part of the total. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Tue Jan 6 20:26:09 2015 From: barry at list.org (Barry Warsaw) Date: Tue, 6 Jan 2015 14:26:09 -0500 Subject: [Mailman-Developers] Python 3 In-Reply-To: References: <20141222134734.698db176@anthem> <20141226161128.408d9ead@anarchist.wooz.org> <8738814v2u.fsf@uwakimon.sk.tsukuba.ac.jp> <20141228182648.76d40d96@anthem> <54A87045.5080608@gmail.com> <20150106074848.38798792@limelight.wooz.org> <54ABDF09.4080507@gmail.com> Message-ID: <20150106142609.63cacc7e@limelight.wooz.org> On Jan 06, 2015, at 02:55 PM, Aurelien Bompard wrote: >> > What if a site wanted to run them on different VMs for example? >> > >Yeah, but certainly there should be a limit to that. What if a site wanted to >run each mailman runner on a different VM for example? There's already some support for that, in a way. The runners with queues already employ an algorithm to safely split the queue space up into 2^n separate runners without the need for locking. They do of course have to have access to the same directory (e.g. over NFS). Also, now that MM3 is backed by SQL, the use of a client/server db like PostgreSQL eliminates another co-location bottleneck. I wouldn't say this latter is thoroughly fleshed out, and I'm not even sure there's a compelling use case, but it may indeed be possible. >Oh well, I guess there's no other way. I hope it's not going to be a mess to >configure, I'd like to avoid having the admin set HK's base URL in 3 >different places. Agreed. Configuration complexity among all the various components is definitely something to be wary of. Cheers, -Barry From barry at list.org Tue Jan 6 21:50:04 2015 From: barry at list.org (Barry Warsaw) Date: Tue, 6 Jan 2015 15:50:04 -0500 Subject: [Mailman-Developers] X-Message-ID-Hash header (was Re: Python 3) In-Reply-To: References: <20141222134734.698db176@anthem> <20141226161128.408d9ead@anarchist.wooz.org> <8738814v2u.fsf@uwakimon.sk.tsukuba.ac.jp> <20141226231846.17eb0ab8@anarchist.wooz.org> <87y4pt37gr.fsf@uwakimon.sk.tsukuba.ac.jp> <20141228175108.1d905288@anthem> <87ppb32r83.fsf@uwakimon.sk.tsukuba.ac.jp> <20141229130206.18c3e335@anthem> Message-ID: <20150106155004.6713f762@anarchist.wooz.org> On Jan 04, 2015, at 03:10 PM, Jeff Breidenbach wrote: >So the homebrew hash was dumb idea, and will gracefully >retire in favor of the standard mailman hash algorithm. It >was pretty neat to see some usage, though. And Archived-At >still looks like a winner. Definitely. X-Message-ID-Hash was just a way to elaborate on RFC 5064 $3.2, and because Base 32 was chosen, there isn't any need to worry about needing to escaping non-ascii characters in the Message-ID, nor case/letter/number ambiguity, and still have a reasonably short-ish, human readable URL. Mailman guarantees that any message it forwards will have a Message-ID, although it's usually the case that something upstream will also make such a guarantee. The X-Message-ID-Hash is a pure convenience. We could omit it since we still create and add an Archived-At header with the hash as the last path component, and of course with a well-documented algorithm, it can always be calculated from the Message-ID. It's just handy to include it for other system components that want to combine it easily with RFC 2369's List-Archive header. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From flo.fuchs at gmail.com Tue Jan 6 23:11:44 2015 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Tue, 06 Jan 2015 23:11:44 +0100 Subject: [Mailman-Developers] MM3 REST API testing [was: Python 3] In-Reply-To: <20150105222429.1a8bef26@marathon> References: <20141222134734.698db176@anthem> <549EA333.6000607@gmail.com> <20150101234033.0e5cee7a@anarchist> <20150105222429.1a8bef26@marathon> Message-ID: <54AC5DA0.7040107@gmail.com> Am 06.01.2015 um 04:24 schrieb Barry Warsaw: > On Jan 01, 2015, at 11:40 PM, Barry Warsaw wrote: > >> I'm playing with a solution that involves the use of the 'vcr' package > > So this branch is now ready for review. > > https://code.launchpad.net/~barry/mailman.client/bilingual/+merge/245537 > > I even added documentation on how to run the test to record the yaml file > containing the HTTP traffic. There are tox rules for doing both the > recording, and for running the test suite with the recorded traffic against > both Python 2 and Python 3. I've only tested 2.7 and 3.4 since I don't > currently have good builds of 2.6, 3.2, or 3.3, but the tox.ini file should > support those. > > There are now no imports of any `mailman` modules. I did have to add a new > top-level `queues` resource to the REST API which will allow you to inject > messages, query, and delete messages from the various queues. I thought this > was a pretty handy feature anyway, and it was easy to add, so that's now in > the Mailman 3 trunk. Which btw, as you might notice, I did go ahead and merge > the py3 branch. I think with this mailman.client approach and a REST-based HK > IArchiver implementation (TBD), it was safe enough to do, and I didn't want to > have to continue to implement new features in both branches (e.g. the queues > resource). > > Please have a look and let me know what you think. I haven't had the time to look at it today. But will definitely do it tomorrow. Thanks a lot! Florian From barry at list.org Wed Jan 7 00:38:49 2015 From: barry at list.org (Barry Warsaw) Date: Tue, 6 Jan 2015 18:38:49 -0500 Subject: [Mailman-Developers] Some raw thinking on user level authorization and authentication In-Reply-To: References: Message-ID: <20150106183849.18bb82ab@anarchist.wooz.org> Hi Andrew, thanks for starting to look at this. On Dec 17, 2014, at 09:29 AM, Andrew Stuart wrote: >step 1: authentication >?- HTTP client provides username and password >?- username and password sent to Mailman REST server >?- if Mailman REST server authenticates username and password: >?- authentication token returned to client MM3's REST API has the resource /users//login which you can POST to with form data `cleartext_password` as the authentication step. You get back a 204 if the password matches, and a 403 if it does not. >step 2: authorization >-- examines inbound requests >?- if request contains valid authentication information, continue >?- examine requested Mailman resource >-? send REST request to Mailman REST server to find out if this user is >-allowed to do this resource request This is the trickiest part because we don't currently codify what a particular user is or is not allowed to do. We'll need to do this carefully, and the question is whether that information should be kept in the core or in the proxy layer. I wonder if there is prior art or best practices on how best to specify authorizations to REST resources and methods. >?- requires execution of the Mailman REST API as a wsgi application, which is >potentially complex and more tightly coupled than desirable and would require >implementation into the Mailman code base It already is a wsgi application actually. But I think the bigger problem is that we want admins to be comfortable exposing the full admin API only on localhost, and they'll need to expose the proxy on a public IP, so it's likely that the proxy would run on a separate machine/VM from core. Cheers, -Barry From stephen at xemacs.org Wed Jan 7 01:28:11 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 07 Jan 2015 09:28:11 +0900 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <54AC1FE4.2040506@libertytrek.org> References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> Message-ID: <874ms31lk4.fsf@uwakimon.sk.tsukuba.ac.jp> Tanstaafl writes: > On 12/30/2014 2:39 PM, Barry Warsaw wrote: > > * "A" release, which remains on Python 2.7 > > * "B" release, which is only compatible with Python 3.4 > > So, wheezy admins will be left out in the cold. There may be free software, but There Ain't No Such Thing As A Free Lunch, man! People who depend on package managers take that risk. This is especially true of the Debian "paleolithic" distribution. You want rock-like stability, you best be prepared to use Stone-Age tech. Get out of your cave and kill yourself some sabertooth fur if you feel a chill. Yabba dabba doo! :-) Note that I use Debian stable when possible myself, but I'm not a "Debian stable admin", I'm an admin who chooses the best[1] distro for the job, and that is usually Debian stable. The point for me is that it provides a very stable platform for locally-configured and -built installations of mission-critical software that I'm willing to baby- sit. Over the occasional period I have to upgrade to testing; a very few times I've had to pin versions of a few packages in testing. One can also take on the packaging task oneself, or persuade the packagers to do so. Often it's not that hard to do the latter ... they want the same things their users do (at least, that's usually the case for packages in popular distros). Really, you should be lobbying Debian, not Mailman. Footnotes: [1] Which includes considerations of my time for evaluation of "best" and switching costs. From pabs3 at bonedaddy.net Wed Jan 7 02:36:08 2015 From: pabs3 at bonedaddy.net (Paul Wise) Date: Wed, 7 Jan 2015 09:36:08 +0800 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <54AC1FE4.2040506@libertytrek.org> References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> Message-ID: On Wed, Jan 7, 2015 at 1:48 AM, Tanstaafl wrote: > So, wheezy admins will be left out in the cold. I expect Debian jessie will become Debian stable soon enough. -- bye, pabs http://bonedaddy.net/pabs3/ From tanstaafl at libertytrek.org Wed Jan 7 08:09:34 2015 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Wed, 07 Jan 2015 02:09:34 -0500 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> Message-ID: <54ACDBAE.7060001@libertytrek.org> On 1/6/2015 8:36 PM, Paul Wise wrote: > On Wed, Jan 7, 2015 at 1:48 AM, Tanstaafl wrote: > >> So, wheezy admins will be left out in the cold. > > I expect Debian jessie will become Debian stable soon enough. Not for those wanting to avoid systemd. From pabs3 at bonedaddy.net Wed Jan 7 08:13:06 2015 From: pabs3 at bonedaddy.net (Paul Wise) Date: Wed, 7 Jan 2015 15:13:06 +0800 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <54ACDBAE.7060001@libertytrek.org> References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> <54ACDBAE.7060001@libertytrek.org> Message-ID: On Wed, Jan 7, 2015 at 3:09 PM, Tanstaafl wrote: > Not for those wanting to avoid systemd. systemd is optional and easy to avoid in Debian. -- bye, pabs http://bonedaddy.net/pabs3/ From tanstaafl at libertytrek.org Wed Jan 7 10:06:15 2015 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Wed, 07 Jan 2015 04:06:15 -0500 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> <54ACDBAE.7060001@libertytrek.org> Message-ID: <54ACF707.5010205@libertytrek.org> On 1/7/2015 2:13 AM, Paul Wise wrote: > On Wed, Jan 7, 2015 at 3:09 PM, Tanstaafl wrote: >> Not for those wanting to avoid systemd. > systemd is optional and easy to avoid in Debian. Until it isn't, and that is the overriding concern, and with very good reason considering the relatively short track record that systemd has. From geoff at QuiteLikely.com Wed Jan 7 10:21:29 2015 From: geoff at QuiteLikely.com (Geoff Shang) Date: Wed, 7 Jan 2015 11:21:29 +0200 (IST) Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <54AC1FE4.2040506@libertytrek.org> References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> Message-ID: On Tue, 6 Jan 2015, Tanstaafl wrote: > On 12/30/2014 2:39 PM, Barry Warsaw wrote: >> * "A" release, which remains on Python 2.7 >> * "B" release, which is only compatible with Python 3.4 > > So, wheezy admins will be left out in the cold. Wheezy has Python 2.7. Geoff. From tanstaafl at libertytrek.org Wed Jan 7 11:36:27 2015 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Wed, 07 Jan 2015 05:36:27 -0500 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> Message-ID: <54AD0C2B.90601@libertytrek.org> On 1/7/2015 4:21 AM, Geoff Shang wrote: > On Tue, 6 Jan 2015, Tanstaafl wrote: > >> On 12/30/2014 2:39 PM, Barry Warsaw wrote: >>> * "A" release, which remains on Python 2.7 >>> * "B" release, which is only compatible with Python 3.4 >> >> So, wheezy admins will be left out in the cold. > > Wheezy has Python 2.7. And 3.2... and that is the problem. From barry at list.org Wed Jan 7 16:37:51 2015 From: barry at list.org (Barry Warsaw) Date: Wed, 7 Jan 2015 10:37:51 -0500 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <54AD0C2B.90601@libertytrek.org> References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> <54AD0C2B.90601@libertytrek.org> Message-ID: <20150107103751.179edaa4@anarchist.wooz.org> On Jan 07, 2015, at 05:36 AM, Tanstaafl wrote: >And 3.2... and that is the problem. Let me ask again: if you have to install MM3 from source anyway (and probably many of its dependencies), why is it also a problem to install Python 3.4 from source? I've heard that Debian is working on Ubuntu-like PPAs. That would seem like a fruitful way of providing the entire stack for older distros. Cheers, -Barry From pabs3 at bonedaddy.net Wed Jan 7 16:42:40 2015 From: pabs3 at bonedaddy.net (Paul Wise) Date: Wed, 7 Jan 2015 23:42:40 +0800 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <20150107103751.179edaa4@anarchist.wooz.org> References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> <54AD0C2B.90601@libertytrek.org> <20150107103751.179edaa4@anarchist.wooz.org> Message-ID: On Wed, Jan 7, 2015 at 11:37 PM, Barry Warsaw wrote: > I've heard that Debian is working on Ubuntu-like PPAs. That would seem like a > fruitful way of providing the entire stack for older distros. What about just a backport of Python 3.4 (and any future mailman3 package)? http://backports.debian.org/ -- bye, pabs http://bonedaddy.net/pabs3/ From barry at list.org Wed Jan 7 16:48:12 2015 From: barry at list.org (Barry Warsaw) Date: Wed, 7 Jan 2015 10:48:12 -0500 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> <54AD0C2B.90601@libertytrek.org> <20150107103751.179edaa4@anarchist.wooz.org> Message-ID: <20150107104812.6316f54f@limelight.wooz.org> On Jan 07, 2015, at 11:42 PM, Paul Wise wrote: >What about just a backport of Python 3.4 (and any future mailman3 package)? > >http://backports.debian.org/ For sure, Python 3.4 should be in backports, IMHO. Cheers, -Barry From stephen at xemacs.org Thu Jan 8 03:43:52 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 08 Jan 2015 11:43:52 +0900 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <20150107103751.179edaa4@anarchist.wooz.org> References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> <54AD0C2B.90601@libertytrek.org> <20150107103751.179edaa4@anarchist.wooz.org> Message-ID: <87zj9uyot3.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > I've heard that Debian is working on Ubuntu-like PPAs. That would > seem like a fruitful way of providing the entire stack for older > distros. This is already possible to some extent, as I mentioned earlier: one can enable more recent versions of packages from the Debian repositories, or (as I do) upgrade the distribution but pin packages which you need to be invariant to specific versions. It's also easy to build Debian packages in most cases if you just want to use more recent versions of upstream source. And there really isn't any difficulty in creating a PPA per se, you just add it to your sources list. The software can already handle it I'm pretty sure. The problem is that PPAs just aren't acceptable in many cases for two reasons: (1) they are by definition not subject to distribution QA, and they may not be acceptable to enterprise QA, and (2) if you go to a PPA for Python 3.4, you will also need a 3.4-specific compile of each Python package you depend on, which means packages for those too in most distributions (and specifically in Debian, I believe). (And I'll address (3) "effort" below.) In the case of (2), you can use the source debs and pretty easily modify them to use Python 3.4 instead of the specified Python (in fact, there probably is a generic source deb for this purpose). Or you can backport Mailman to Python 3.2. But either means that there is *zero* QA for Python 3.4 vs 3.2 (or whatever) differences. Someone who is using Debian stable evidently cares about that risk, and cares *a lot*. Personally, I don't see why we should put in the effort to deal with this. If there are a lot of admins on wheezy systems who are systemd refuseniks (and I sympathize with them, systemd as described in Lennart's blogs is a great idea, but it suffers from mission creep on a Vietnam-sized scale), then there's enough manpower to create a "Mailman 3 on wheezy" PPA, and mailman-devel and mailman-users (not to forget python-list and maybe even python-dev) will be there to help with Mailman-specific issues in the packages. If there aren't, then why should Mailman supply the manpower to satisfy a small number cranky admins? I suppose most admins will trust that by the time Debian stable gets systemd and Mailman 3, both Mailman and systemd will be good and stable themselves. So as I wrote to Tanstaafl before, TISATAFS but TANSTAAFL! (NB FVO F (speech) != F (beer).) Somebody will pay for this lunch; in this case I think the diners should do it. Mailman itself would get nothing but pain and a big "thank you" (ie, hot air). From tanstaafl at libertytrek.org Thu Jan 8 06:29:32 2015 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Thu, 08 Jan 2015 00:29:32 -0500 Subject: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5 In-Reply-To: <20150107103751.179edaa4@anarchist.wooz.org> References: <20141230143938.48b96570@marathon> <54AC1FE4.2040506@libertytrek.org> <54AD0C2B.90601@libertytrek.org> <20150107103751.179edaa4@anarchist.wooz.org> Message-ID: <54AE15BC.7050409@libertytrek.org> On 1/7/2015 10:37 AM, Barry Warsaw wrote: > Let me ask again: if you have to install MM3 from source anyway (and probably > many of its dependencies), why is it also a problem to install Python 3.4 from > source? Maybe I missed that question (sorry, currently out of the country)... Good point, hadn't considered it... Sorry for the noise... From barry at list.org Thu Jan 8 23:15:13 2015 From: barry at list.org (Barry Warsaw) Date: Thu, 8 Jan 2015 17:15:13 -0500 Subject: [Mailman-Developers] Dark Mail Message-ID: <20150108171513.776ff650@limelight.wooz.org> Something interesting to keep an eye on, especially to see if it might relate to encrypted mailing lists. http://arstechnica.com/security/2015/01/lavabit-founder-wants-to-make-dark-e-mail-secure-by-default/ Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From sumanah at panix.com Fri Jan 9 22:50:47 2015 From: sumanah at panix.com (Sumana Harihareswara) Date: Fri, 09 Jan 2015 16:50:47 -0500 Subject: [Mailman-Developers] Postorius merge proposals: translatability & accuracy in templates Message-ID: <54B04D37.1010504@panix.com> I've proposed two new branches for merge into Postorius: * https://code.launchpad.net/~sumanah/postorius/bug-1157947/+merge/245979 fixes bug # 1157947 by finding all user-visible occurrences of hard-coded English strings in templates and templatizing them for translatability. * https://code.launchpad.net/~sumanah/postorius/metrics-template/+merge/245965 fixes bug #1409033 by correcting the messages in a template associated with three metrics. I'd welcome any reviews. Thanks! (And I also see that Aur?lien and others have a few outstanding merge proposals at https://code.launchpad.net/postorius/+activereviews , in case you want to work FIFO.) -- Sumana Harihareswara http://brainwane.net From andrew.stuart at supercoders.com.au Sat Jan 10 00:52:09 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sat, 10 Jan 2015 10:52:09 +1100 Subject: [Mailman-Developers] REST API returning etags as slashed quoted Message-ID: <822635C7-2C56-4445-8B3C-A4DF5238A3F6@supercoders.com.au> The REST API is returning etags as slashed quoted. Just wondering if this is intentional, does not seem consistent with the behaviour of the other fields returned from the API. i.e. "http_etag": "\"d6d174c76c5fbb5f8a69b648ecd3b27b53a5d27e\?", thanks From andrew.stuart at supercoders.com.au Sat Jan 10 00:58:28 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sat, 10 Jan 2015 10:58:28 +1100 Subject: [Mailman-Developers] REST API returning value of password field in user record Message-ID: <179F9097-63E5-48B3-8929-5A7379642A2C@supercoders.com.au> I?m aware that it?s not the actual cleartext password. From a security perspective should even salted and hashed passwords should stay behind the API or might there be a need for something on the other side of the API to access that field? thanks From andrew.stuart at supercoders.com.au Sat Jan 10 02:30:31 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sat, 10 Jan 2015 12:30:31 +1100 Subject: [Mailman-Developers] REST API Difference between PUT and PATCH? Message-ID: <817DDCC5-FE01-4293-8E41-19AEB6C67997@supercoders.com.au> The user endpoint supports PUT and PATCH methods. PATCH does a partial update of the user?s configuration PUT does a full update of the user?s configuration It seems that for user, the field that can be modified between these two methods are display_name and cleartext_password What is the difference exactly between partial and full update? Are the two fields required in one case and optional in the other? thanks From barry at list.org Sat Jan 10 05:03:46 2015 From: barry at list.org (Barry Warsaw) Date: Fri, 9 Jan 2015 23:03:46 -0500 Subject: [Mailman-Developers] REST API Difference between PUT and PATCH? In-Reply-To: <817DDCC5-FE01-4293-8E41-19AEB6C67997@supercoders.com.au> References: <817DDCC5-FE01-4293-8E41-19AEB6C67997@supercoders.com.au> Message-ID: <20150109230346.0169211a@anarchist.wooz.org> On Jan 10, 2015, at 12:30 PM, Andrew Stuart wrote: >What is the difference exactly between partial and full update? >Are the two fields required in one case and optional in the other? Exactly right. For all resources, PATCH allows the fields to be optional while PUT requires the fields to be present. Cheers, -Barry From barry at list.org Sat Jan 10 05:05:29 2015 From: barry at list.org (Barry Warsaw) Date: Fri, 9 Jan 2015 23:05:29 -0500 Subject: [Mailman-Developers] REST API returning value of password field in user record In-Reply-To: <179F9097-63E5-48B3-8929-5A7379642A2C@supercoders.com.au> References: <179F9097-63E5-48B3-8929-5A7379642A2C@supercoders.com.au> Message-ID: <20150109230529.6ca8df09@anarchist.wooz.org> On Jan 10, 2015, at 10:58 AM, Andrew Stuart wrote: >I?m aware that it?s not the actual cleartext password. > >From a security perspective should even salted and hashed passwords should >stay behind the API or might there be a need for something on the other side >of the API to access that field? Keeping in mind that the core's REST API is a privileged API, only to be exposed over localhost, it is intended to make the hashed password field available. For a public facing proxy, I would expect this field to be filtered out. Cheers, -Barry From barry at list.org Sat Jan 10 05:07:10 2015 From: barry at list.org (Barry Warsaw) Date: Fri, 9 Jan 2015 23:07:10 -0500 Subject: [Mailman-Developers] REST API returning etags as slashed quoted In-Reply-To: <822635C7-2C56-4445-8B3C-A4DF5238A3F6@supercoders.com.au> References: <822635C7-2C56-4445-8B3C-A4DF5238A3F6@supercoders.com.au> Message-ID: <20150109230710.0a4e4a7f@anarchist.wooz.org> On Jan 10, 2015, at 10:52 AM, Andrew Stuart wrote: >The REST API is returning etags as slashed quoted. > >Just wondering if this is intentional, does not seem consistent with the >behaviour of the other fields returned from the API. > >i.e. > >"http_etag": "\"d6d174c76c5fbb5f8a69b648ecd3b27b53a5d27e\?", It's intentional, but I don't remember why. I think I remember reading that the HTTP ETag header (which we actually don't set in the HTTP response, but probably should) is quoted. Cheers, -Barry From stephen at xemacs.org Sat Jan 10 05:37:47 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 10 Jan 2015 13:37:47 +0900 Subject: [Mailman-Developers] English in UTF-8 [was: Does mailman 2.1.18 support hebrew language ?] In-Reply-To: <54B00017.9010503@msapiro.net> References: <2F60D2B899AFBC4FAF660B2D9845BE8F0DD80A49@CIO-KRC-D1MBX06.osuad.osu.edu> <54B00017.9010503@msapiro.net> Message-ID: <87vbkfz1wk.fsf@uwakimon.sk.tsukuba.ac.jp> On Mailman-Users, Mark Sapiro writes: > 3) You can set Mailman's character set for English to utf-8 by putting > > add_language('en', 'English (USA)', 'utf-8') > > in mm_cfg.py (and restarting Mailman). The downside of this is the > bodies of Mailman generated messages including plain digests will be > base64 encoded and will not be readable by non-MIME aware MUAs. Does Mailman 3/Python 3 have this limitation? (One of my MUAs is grep .... :-) Steve From andrew.stuart at supercoders.com.au Sat Jan 10 05:42:30 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sat, 10 Jan 2015 15:42:30 +1100 Subject: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful Message-ID: <6FD6FABB-53C1-4EB1-AF02-EE7675A479CF@supercoders.com.au> Successful requests to the REST API return a 2xx HTTP code to indicate success either with or without JSON data depending on the context of the request. At least one (I?ve not checked the othes yet) 4xx error returned from the REST API returns a plain text error message rather than a JSON error message. For example attempting to create a domain that exists returns a 400 error with the response text ?Domain exists?. Do you think this matters? Probably not much but does mean handling valid responses goes down a JSON path and handling error message goes down a plain text path. JSON error messages might arguable have been somewhat more consistent and programmable. thanks From stephen at xemacs.org Sat Jan 10 05:57:38 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 10 Jan 2015 13:57:38 +0900 Subject: [Mailman-Developers] REST API returning value of password field in user record In-Reply-To: <179F9097-63E5-48B3-8929-5A7379642A2C@supercoders.com.au> References: <179F9097-63E5-48B3-8929-5A7379642A2C@supercoders.com.au> Message-ID: <87twzzz0zh.fsf@uwakimon.sk.tsukuba.ac.jp> Andrew Stuart writes: > From a security perspective should even salted and hashed passwords > should stay behind the API or might there be a need for something > on the other side of the API to access that field? At present the REST API is available only on localhost (at least by default), so it's not that big a risk (yes, I understand defense in depth, but there's a need of corresponding importance). In the absence of a proper authz/authn module inside of Mailman itself, I don't see a real alternative to making that data available to mailman.client, and thus making it possible for other user apps (HyperKitty, Postorius) to get authorization to access a specific user's data. In the long run we need to do something about this. However, Mailman has operated based on passing around *cleartext* passwords by *email* for decades, with no serious issues that I know of. If Barry is serious about World Domination, we need to fix this, but I don't see a huge hurry. Steve From mark at msapiro.net Sat Jan 10 05:59:40 2015 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 09 Jan 2015 20:59:40 -0800 Subject: [Mailman-Developers] English in UTF-8 [was: Does mailman 2.1.18 support hebrew language ?] In-Reply-To: <87vbkfz1wk.fsf@uwakimon.sk.tsukuba.ac.jp> References: <2F60D2B899AFBC4FAF660B2D9845BE8F0DD80A49@CIO-KRC-D1MBX06.osuad.osu.edu> <54B00017.9010503@msapiro.net> <87vbkfz1wk.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <54B0B1BC.4050703@msapiro.net> On 01/09/2015 08:37 PM, Stephen J. Turnbull wrote: > On Mailman-Users, Mark Sapiro writes: > > > 3) You can set Mailman's character set for English to utf-8 by putting > > > > add_language('en', 'English (USA)', 'utf-8') > > > > in mm_cfg.py (and restarting Mailman). The downside of this is the > > bodies of Mailman generated messages including plain digests will be > > base64 encoded and will not be readable by non-MIME aware MUAs. > > Does Mailman 3/Python 3 have this limitation? > > (One of my MUAs is grep .... :-) I feel your pain. The short answer is Yes. Python's email package encodes things based on character set. It chooses between Quoted Printable, base64 or whichever gives the shorter result based on the dictionary at email.charset.CHARSETS which maps charset names to a tuple of (header_encoding, body_encoding, output_codec) for utf-8 this is (SHORTEST, BASE64, 'utf-8') See email/charset.py for details. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From stephen at xemacs.org Sat Jan 10 09:19:46 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 10 Jan 2015 17:19:46 +0900 Subject: [Mailman-Developers] English in UTF-8 In-Reply-To: <54B0B1BC.4050703@msapiro.net> References: <2F60D2B899AFBC4FAF660B2D9845BE8F0DD80A49@CIO-KRC-D1MBX06.osuad.osu.edu> <54B00017.9010503@msapiro.net> <87vbkfz1wk.fsf@uwakimon.sk.tsukuba.ac.jp> <54B0B1BC.4050703@msapiro.net> Message-ID: <87ppanyrml.fsf@uwakimon.sk.tsukuba.ac.jp> Mark Sapiro writes: > I feel your pain. :-) It's possible to do something about it, though. > The short answer is Yes. Python's email package encodes things based on > character set. Thanks for the quick response. I'll take a look at this. At latest, I'll sprint on it. :-) From barry at list.org Sat Jan 10 19:36:20 2015 From: barry at list.org (Barry Warsaw) Date: Sat, 10 Jan 2015 13:36:20 -0500 Subject: [Mailman-Developers] REST API returning value of password field in user record In-Reply-To: <87twzzz0zh.fsf@uwakimon.sk.tsukuba.ac.jp> References: <179F9097-63E5-48B3-8929-5A7379642A2C@supercoders.com.au> <87twzzz0zh.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20150110133620.57103e39@anarchist.wooz.org> On Jan 10, 2015, at 01:57 PM, Stephen J. Turnbull wrote: >In the long run we need to do something about this. However, Mailman >has operated based on passing around *cleartext* passwords by *email* >for decades, with no serious issues that I know of. Just to be, ahem, clear, by default MM3 does not store cleartext passwords. It uses passlib to hash the passwords set via the REST API and only stores those hashed passwords internally. From src/mailman/config/passlib.cfg: [passlib] # This is the output of passlibs.apps.custom_app_context.to_string(). # See http://packages.python.org/passlib/index.html for details. schemes = sha512_crypt, sha256_crypt default = sha512_crypt all__vary_rounds = 0.1 sha256_crypt__min_rounds = 80000 sha512_crypt__min_rounds = 60000 admin__sha256_crypt__min_rounds = 160000 admin__sha512_crypt__min_rounds = 120000 It's possible to configure passlib to use an 'identity' hash which would leave passwords in the clear, but you'd have to go out of your way to do so. The key 'cleartext_password' is only used with PUT/PATCH on the user resource. The idea being that the client doesn't hash the password, but instead passes it in cleartext to core over REST. The core then hashes it and stores it on the user record. The JSON representation of the user resource has a 'password' key which contains the hashed password. So when you GET a user, you get it hashed, i.e. exactly what's stored on the user record. User resources have a 'login' subresource which you can POST to to verify a password. This also takes a cleartext_password, presumably as you'd receive from a login form. Cheers, -Barry From barry at list.org Sat Jan 10 19:39:23 2015 From: barry at list.org (Barry Warsaw) Date: Sat, 10 Jan 2015 13:39:23 -0500 Subject: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful In-Reply-To: <6FD6FABB-53C1-4EB1-AF02-EE7675A479CF@supercoders.com.au> References: <6FD6FABB-53C1-4EB1-AF02-EE7675A479CF@supercoders.com.au> Message-ID: <20150110133923.413d27e5@anarchist.wooz.org> On Jan 10, 2015, at 03:42 PM, Andrew Stuart wrote: >Successful requests to the REST API return a 2xx HTTP code to indicate >success either with or without JSON data depending on the context of the >request. > >At least one (I?ve not checked the othes yet) 4xx error returned from the >REST API returns a plain text error message rather than a JSON error message. > >For example attempting to create a domain that exists returns a 400 error >with the response text ?Domain exists?. > >Do you think this matters? Probably not much but does mean handling valid >responses goes down a JSON path and handling error message goes down a plain >text path. JSON error messages might arguable have been somewhat more >consistent and programmable. Some of this has to do with the historical legacy of the tools we've used to implement the REST API (first, lazr.restful, then restish, and now hopefully finally, falcon). IIRC it wasn't possible to set a JSON body response on error codes in lazr.restful and that just got inherited throughout the other migrations. I don't know if it matters. I suppose we'll have to see what client authors think. Changing it would be an API change though, so perhaps we'd need to wait until we rev the REST API. Cheers, -Barry From andrew.stuart at supercoders.com.au Sat Jan 10 20:06:41 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sun, 11 Jan 2015 06:06:41 +1100 Subject: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful In-Reply-To: <20150110133923.413d27e5@anarchist.wooz.org> References: <6FD6FABB-53C1-4EB1-AF02-EE7675A479CF@supercoders.com.au> <20150110133923.413d27e5@anarchist.wooz.org> Message-ID: I don?t think it?s a big deal in the short term because the plain text error message can easily be read from the response. The only thing that?s somewhat odd is that the content-type is application/json even though the content isn?t (for error messages). On 11 Jan 2015, at 5:39 am, Barry Warsaw wrote: On Jan 10, 2015, at 03:42 PM, Andrew Stuart wrote: > Successful requests to the REST API return a 2xx HTTP code to indicate > success either with or without JSON data depending on the context of the > request. > > At least one (I?ve not checked the othes yet) 4xx error returned from the > REST API returns a plain text error message rather than a JSON error message. > > For example attempting to create a domain that exists returns a 400 error > with the response text ?Domain exists?. > > Do you think this matters? Probably not much but does mean handling valid > responses goes down a JSON path and handling error message goes down a plain > text path. JSON error messages might arguable have been somewhat more > consistent and programmable. Some of this has to do with the historical legacy of the tools we've used to implement the REST API (first, lazr.restful, then restish, and now hopefully finally, falcon). IIRC it wasn't possible to set a JSON body response on error codes in lazr.restful and that just got inherited throughout the other migrations. I don't know if it matters. I suppose we'll have to see what client authors think. Changing it would be an API change though, so perhaps we'd need to wait until we rev the REST API. Cheers, -Barry _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From barry at list.org Sat Jan 10 20:10:47 2015 From: barry at list.org (Barry Warsaw) Date: Sat, 10 Jan 2015 14:10:47 -0500 Subject: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful In-Reply-To: References: <6FD6FABB-53C1-4EB1-AF02-EE7675A479CF@supercoders.com.au> <20150110133923.413d27e5@anarchist.wooz.org> Message-ID: <20150110141047.24afca44@anarchist.wooz.org> On Jan 11, 2015, at 06:06 AM, Andrew Stuart wrote: >I don?t think it?s a big deal in the short term because the plain text error >message can easily be read from the response. > >The only thing that?s somewhat odd is that the content-type is >application/json even though the content isn?t (for error messages). That, at least, is a bug. Care to file one? Cheers, -Barry From andrew.stuart at supercoders.com.au Sat Jan 10 20:23:43 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sun, 11 Jan 2015 06:23:43 +1100 Subject: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful In-Reply-To: <20150110141047.24afca44@anarchist.wooz.org> References: <6FD6FABB-53C1-4EB1-AF02-EE7675A479CF@supercoders.com.au> <20150110133923.413d27e5@anarchist.wooz.org> <20150110141047.24afca44@anarchist.wooz.org> Message-ID: Hmmm. Happy to do so if you like. Would you mind pointing me in the direction of some instructions for filing bugs please? FWIW I?d probably favor a REST API that is behaving improperly by returning plain text error responses (i.e. the current situation) rather than a REST API that returns application/json content type for valid responses and text/plain for errors. Once that gets baked into release 1 it?ll be challenging to correct back to application/json later because people will write clients to expect text/plain for errors. It?s a whole lot easier conceptually to deal with a REST API that always returns the same content-type regardless of the outcome of the API call. Some REST tools don?t have any mechanism for defining mixed response type APIs which leads to some weird outcomes when querying successfully versus getting errors. In short, I?d leave as-is and eventually turn the error text into json, but obviously Barry you should say what you want. as On 11 Jan 2015, at 6:10 am, Barry Warsaw wrote: On Jan 11, 2015, at 06:06 AM, Andrew Stuart wrote: > I don?t think it?s a big deal in the short term because the plain text error > message can easily be read from the response. > > The only thing that?s somewhat odd is that the content-type is > application/json even though the content isn?t (for error messages). That, at least, is a bug. Care to file one? Cheers, -Barry _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From raj.abhilash1 at gmail.com Sat Jan 10 20:31:31 2015 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Sun, 11 Jan 2015 01:01:31 +0530 Subject: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful In-Reply-To: References: <6FD6FABB-53C1-4EB1-AF02-EE7675A479CF@supercoders.com.au> <20150110133923.413d27e5@anarchist.wooz.org> <20150110141047.24afca44@anarchist.wooz.org> Message-ID: <54B17E13.9080208@gmail.com> On Sunday 11 January 2015 12:53 AM, Andrew Stuart wrote: > Hmmm. > > Happy to do so if you like. Would you mind pointing me in the direction of some instructions for filing bugs please? https://bugs.launchpad.net/mailman/3.0/+bugs -- thanks, Abhilash Raj From andrew.stuart at supercoders.com.au Sat Jan 10 22:51:18 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sun, 11 Jan 2015 08:51:18 +1100 Subject: [Mailman-Developers] REST API first pass at Swagger spec Message-ID: Attached is a first pass at a Swagger spec for the REST API. You can find out about Swagger at http://swagger.io The Swagger spec that I am working on is attached to this message or can be found here: http://www.mailripper.com/api-docs.json If you want to see the attached spec in action against the Mailman REST API, go to http://www.mailripper.com/swagger-ui/index.html Give it a try at http://www.mailripper.com/swagger-ui/index.html - you are welcome to change anything, it?s a test Mailman server and you can add or delete anything - nothing you do will be a problem. There?s still a fair bit to do to finalise this but most of the API queries are working - enough for you to see how it works. Any comments or feedback welcome. as -------------- next part -------------- A non-text attachment was scrubbed... Name: api-docs.json Type: application/json Size: 64793 bytes Desc: not available URL: From stephen at xemacs.org Sun Jan 11 06:23:24 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 11 Jan 2015 14:23:24 +0900 Subject: [Mailman-Developers] REST API first pass at Swagger spec In-Reply-To: References: Message-ID: <87a91pzy9f.fsf@uwakimon.sk.tsukuba.ac.jp> Andrew Stuart writes: > Attached is a first pass at a Swagger spec for the REST API. > > You can find out about Swagger at http://swagger.io tmc;wr[1] Please explain briefly what Swagger is and why you're using it rather than some similar product. Two lines is enough. I have ZERO intention of paying attention to what you're doing; my technical plate is full, and Barry's the boss anyway. But if some new person or a GSoC student seems to have an interest in "that kind of thing" I'd like to know enough to say "go talk to Andrew". Footnotes: [1] Too many clicks; won't read. From andrew.stuart at supercoders.com.au Sun Jan 11 06:40:54 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sun, 11 Jan 2015 16:40:54 +1100 Subject: [Mailman-Developers] REST API first pass at Swagger spec In-Reply-To: <87a91pzy9f.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87a91pzy9f.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: The Swagger spec defines a way to document the methods, properties and return values of a REST API using JSON, and also provides a front end user interface to turn that JSON into a functioning user interface to directly query the API. There seems to be a few choices for documentation REST APIs at the moment (http://apiary.io & http://www.mashery.com/product/io-docs) but Swagger appears to have the industry momentum. I?ve put together a Swagger spec mainly because I?ve experienced enough pain with REST APIs in the past to know that development goes much, much more smoothly when you have a single place that shows all the available API methods, their parameters and return values, ideally live so you can see it working. Instead of digging through source code and written documentation to find how the REST API works, it?s all right there in front of you. Once the REST API is documented in JSON, a range of things become possible including writing tests, command lines interfaces, clients and servers that ?build themselves? to a greater or lesser extent directly from the JSON REST API spec. Sorry, that was five long lines :-) On 11 Jan 2015, at 4:23 pm, Stephen J. Turnbull wrote: Andrew Stuart writes: > Attached is a first pass at a Swagger spec for the REST API. > > You can find out about Swagger at http://swagger.io tmc;wr[1] Please explain briefly what Swagger is and why you're using it rather than some similar product. Two lines is enough. I have ZERO intention of paying attention to what you're doing; my technical plate is full, and Barry's the boss anyway. But if some new person or a GSoC student seems to have an interest in "that kind of thing" I'd like to know enough to say "go talk to Andrew". Footnotes: [1] Too many clicks; won't read. From stephen at xemacs.org Sun Jan 11 07:26:23 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 11 Jan 2015 15:26:23 +0900 Subject: [Mailman-Developers] REST API first pass at Swagger spec In-Reply-To: References: <87a91pzy9f.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <877fwtzvcg.fsf@uwakimon.sk.tsukuba.ac.jp> Andrew Stuart writes: > The Swagger spec defines a way to document the methods, properties > and return values of a REST API using JSON, Aha. The Systers' students will like that, I think! Thank you! From barry at list.org Mon Jan 12 02:19:18 2015 From: barry at list.org (Barry Warsaw) Date: Sun, 11 Jan 2015 20:19:18 -0500 Subject: [Mailman-Developers] REST API first pass at Swagger spec In-Reply-To: References: Message-ID: <20150111201918.2c0409d9@anarchist.wooz.org> On Jan 11, 2015, at 08:51 AM, Andrew Stuart wrote: >Attached is a first pass at a Swagger spec for the REST API. > >You can find out about Swagger at http://swagger.io > >The Swagger spec that I am working on is attached to this message or can be >found here: http://www.mailripper.com/api-docs.json > >If you want to see the attached spec in action against the Mailman REST API, >go to http://www.mailripper.com/swagger-ui/index.html > >Give it a try at http://www.mailripper.com/swagger-ui/index.html - you are >welcome to change anything, it?s a test Mailman server and you can add or >delete anything - nothing you do will be a problem. > >There?s still a fair bit to do to finalise this but most of the API queries >are working - enough for you to see how it works. It looks interesting. I have a little experience with WADL with the Launchpad API. There was a lot of infrastructure built into lazr.restful to automatically generate the WADL, but it was so cumbersome that I was ecstatic to remove it all for restish (at the time). My big question is how to make sure that the swagger spec doesn't get out of date to the implementation. I definitely don't want to add a lot of cruft to the implementation - the light weight of the current falcon-based implementation is a huge benefit. It would be okay (maybe ideal) if some how the swagger spec was integrated with the core's test suite, so that at least we'd see failures if the API were modified (e.g. new resources and methods added) without the spec being similarly updated. Ultimately I think if it isn't much effort to keep up to date, and is useful to consumers of the API, it could be a good thing, but I don't really have any opinions about swagger as compared to alternatives. Cheers, -Barry From barry at list.org Mon Jan 12 02:22:40 2015 From: barry at list.org (Barry Warsaw) Date: Sun, 11 Jan 2015 20:22:40 -0500 Subject: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful In-Reply-To: References: <6FD6FABB-53C1-4EB1-AF02-EE7675A479CF@supercoders.com.au> <20150110133923.413d27e5@anarchist.wooz.org> <20150110141047.24afca44@anarchist.wooz.org> Message-ID: <20150111202240.682ae1fc@anarchist.wooz.org> On Jan 11, 2015, at 06:23 AM, Andrew Stuart wrote: >Happy to do so if you like. Would you mind pointing me in the direction of >some instructions for filing bugs please? > >FWIW I?d probably favor a REST API that is behaving improperly by returning >plain text error responses (i.e. the current situation) rather than a REST >API that returns application/json content type for valid responses and >text/plain for errors. Once that gets baked into release 1 it?ll be >challenging to correct back to application/json later because people will >write clients to expect text/plain for errors. > >It?s a whole lot easier conceptually to deal with a REST API that always >returns the same content-type regardless of the outcome of the API call. > >Some REST tools don?t have any mechanism for defining mixed response type >APIs which leads to some weird outcomes when querying successfully versus >getting errors. In short, I?d leave as-is and eventually turn the error text >into json, but obviously Barry you should say what you want. I think at least the Content-Type should be accurate. Your response sounds a little contradictory, but let me see if I can summarize it. From a practicality view point, you prefer the current implementation, even if it isn't conceptually pure, at least for now. That works for me because then I don't have to worry about changing anything. :) I'm open to re-evaluating that in some future Mailman release. Cheers, -Barry From andrew.stuart at supercoders.com.au Mon Jan 12 02:27:50 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Mon, 12 Jan 2015 12:27:50 +1100 Subject: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful In-Reply-To: <20150111202240.682ae1fc@anarchist.wooz.org> References: <6FD6FABB-53C1-4EB1-AF02-EE7675A479CF@supercoders.com.au> <20150110133923.413d27e5@anarchist.wooz.org> <20150110141047.24afca44@anarchist.wooz.org> <20150111202240.682ae1fc@anarchist.wooz.org> Message-ID: Yes I?d favor leave as is, and at some stage in future convert the error messages to JSON. On 12 Jan 2015, at 12:22 pm, Barry Warsaw wrote: On Jan 11, 2015, at 06:23 AM, Andrew Stuart wrote: > Happy to do so if you like. Would you mind pointing me in the direction of > some instructions for filing bugs please? > > FWIW I?d probably favor a REST API that is behaving improperly by returning > plain text error responses (i.e. the current situation) rather than a REST > API that returns application/json content type for valid responses and > text/plain for errors. Once that gets baked into release 1 it?ll be > challenging to correct back to application/json later because people will > write clients to expect text/plain for errors. > > It?s a whole lot easier conceptually to deal with a REST API that always > returns the same content-type regardless of the outcome of the API call. > > Some REST tools don?t have any mechanism for defining mixed response type > APIs which leads to some weird outcomes when querying successfully versus > getting errors. In short, I?d leave as-is and eventually turn the error text > into json, but obviously Barry you should say what you want. I think at least the Content-Type should be accurate. Your response sounds a little contradictory, but let me see if I can summarize it. From a practicality view point, you prefer the current implementation, even if it isn't conceptually pure, at least for now. That works for me because then I don't have to worry about changing anything. :) I'm open to re-evaluating that in some future Mailman release. Cheers, -Barry _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From stephen at xemacs.org Mon Jan 12 02:46:30 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 12 Jan 2015 10:46:30 +0900 Subject: [Mailman-Developers] REST API first pass at Swagger spec In-Reply-To: <20150111201918.2c0409d9@anarchist.wooz.org> References: <20150111201918.2c0409d9@anarchist.wooz.org> Message-ID: <874mrwzs7d.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > My big question is how to make sure that the swagger spec doesn't > get out of date to the implementation. Doesn't Andrew's use case answer that? I mean, if the Swagger spec is out of date, the tests based on it should fail. If the tests don't fail, they're incomplete anyway, and that's a different problem that no spec language can solve for you. As a matter of process, if Swagger is readable enough, we make it *the* spec, and you don't accept pull requests for the REST implementation without a patch to the spec. If Swagger isn't usable in that fashion, I suppose it will go away again like WADL as soon as Andrew loses interest in keeping it up to date. ;-) From andrew.stuart at supercoders.com.au Mon Jan 12 02:47:16 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Mon, 12 Jan 2015 12:47:16 +1100 Subject: [Mailman-Developers] REST API first pass at Swagger spec In-Reply-To: <20150111201918.2c0409d9@anarchist.wooz.org> References: <20150111201918.2c0409d9@anarchist.wooz.org> Message-ID: <89469EF3-9E6A-47F2-BE59-BE1BC4D50E7B@supercoders.com.au> With Swagger you can either: 1: integrate the spec into the API source code or 2: have one single standalone JSON file that contains the whole specification (api-docs.json in this case) I?m not a fan of integrating the Swagger specification into the code - seems to make for a VERY large amount of work, a huge amount of complexity and massive potential for introduction of bugs. I?ve taken the approach of a single static JSON file - MUCH more simple. All Mailman would need is to include this file (api-docs.json) as a static file that can be read or if there is no static file server in Mailman (is there? I haven?t looked) maybe create an api-docs.json endpoint in the API that returns the static file. Any tools that want to use the api-docs.json spec file don?t need to be included in Mailman core. The api-docs.json spec file would need to be kept up to date when the Mailman API changes but it?s really straightforward - building it in the first place is just tedious research on the methods and properties but maintaining it should be a matter of a few minutes work to update it when the API changes. I?ll need to finish it up. I?m not sure what to do about getting a pair of eyes to QA verify it, but maybe we can cross that bridge after I?ve finished it. On 12 Jan 2015, at 12:19 pm, Barry Warsaw wrote: On Jan 11, 2015, at 08:51 AM, Andrew Stuart wrote: > Attached is a first pass at a Swagger spec for the REST API. > > You can find out about Swagger at http://swagger.io > > The Swagger spec that I am working on is attached to this message or can be > found here: http://www.mailripper.com/api-docs.json > > If you want to see the attached spec in action against the Mailman REST API, > go to http://www.mailripper.com/swagger-ui/index.html > > Give it a try at http://www.mailripper.com/swagger-ui/index.html - you are > welcome to change anything, it?s a test Mailman server and you can add or > delete anything - nothing you do will be a problem. > > There?s still a fair bit to do to finalise this but most of the API queries > are working - enough for you to see how it works. It looks interesting. I have a little experience with WADL with the Launchpad API. There was a lot of infrastructure built into lazr.restful to automatically generate the WADL, but it was so cumbersome that I was ecstatic to remove it all for restish (at the time). My big question is how to make sure that the swagger spec doesn't get out of date to the implementation. I definitely don't want to add a lot of cruft to the implementation - the light weight of the current falcon-based implementation is a huge benefit. It would be okay (maybe ideal) if some how the swagger spec was integrated with the core's test suite, so that at least we'd see failures if the API were modified (e.g. new resources and methods added) without the spec being similarly updated. Ultimately I think if it isn't much effort to keep up to date, and is useful to consumers of the API, it could be a good thing, but I don't really have any opinions about swagger as compared to alternatives. Cheers, -Barry _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From andrew.stuart at supercoders.com.au Mon Jan 12 02:55:51 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Mon, 12 Jan 2015 12:55:51 +1100 Subject: [Mailman-Developers] REST API first pass at Swagger spec In-Reply-To: <874mrwzs7d.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20150111201918.2c0409d9@anarchist.wooz.org> <874mrwzs7d.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <9CE3512E-96D4-4062-9CFE-B9CE312A5704@supercoders.com.au> " as soon as Andrew loses interest in keeping it up to date.? ;-) I was never interested in keeping it up to date in the first place but I can?t stand doing development without API documentation, a necessary first step to coding to the API. I do like your idea about changes to the API not being complete until they have been implemented into the JSON spec. On 12 Jan 2015, at 12:46 pm, Stephen J. Turnbull wrote: Barry Warsaw writes: > My big question is how to make sure that the swagger spec doesn't > get out of date to the implementation. Doesn't Andrew's use case answer that? I mean, if the Swagger spec is out of date, the tests based on it should fail. If the tests don't fail, they're incomplete anyway, and that's a different problem that no spec language can solve for you. As a matter of process, if Swagger is readable enough, we make it *the* spec, and you don't accept pull requests for the REST implementation without a patch to the spec. If Swagger isn't usable in that fashion, I suppose it will go away again like WADL as soon as Andrew loses interest in keeping it up to date. ;-) _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From andrew.stuart at supercoders.com.au Mon Jan 12 03:27:58 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Mon, 12 Jan 2015 13:27:58 +1100 Subject: [Mailman-Developers] Fwd: Your confirmation is required to leave the Mailman-Developers mailing list References: Message-ID: <437DDFCF-6C04-41EF-9CFA-B6C1DCBF8EAC@supercoders.com.au> errr?. is this someone saying they would prefer I wasn?t on this list? I?d rather hear directly from someone if you feel that way. Begin forwarded message: From: mailman-developers-confirm+e1a7bed1fbf00db07dc7590dcbdba1e364ecc240 at python.org To: andrew.stuart at supercoders.com.au Subject: Your confirmation is required to leave the Mailman-Developers mailing list Reply-To: mailman-developers-confirm+e1a7bed1fbf00db07dc7590dcbdba1e364ecc240 at python.org Date: 12 January 2015 1:23:31 pm AEDT Mailing list removal confirmation notice for mailing list Mailman-Developers We have received a request from 68.196.224.128 for the removal of your email address, "andrew.stuart at supercoders.com.au" from the mailman-developers at python.org mailing list. To confirm that you want to be removed from this mailing list, simply reply to this message, keeping the Subject: header intact. Or visit this web page: https://mail.python.org/mailman/confirm/mailman-developers/e1a7bed1fbf00db07dc7590dcbdba1e364ecc240 Or include the following line -- and only the following line -- in a message to mailman-developers-request at python.org: confirm e1a7bed1fbf00db07dc7590dcbdba1e364ecc240 Note that simply sending a `reply' to this message should work from most mail readers, since that usually leaves the Subject: line in the right form (additional "Re:" text in the Subject: is okay). If you do not wish to be removed from this list, please simply disregard this message. If you think you are being maliciously removed from the list, or have any other questions, send them to mailman-developers-owner at python.org. From mark at msapiro.net Mon Jan 12 04:26:48 2015 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 11 Jan 2015 19:26:48 -0800 Subject: [Mailman-Developers] Fwd: Your confirmation is required to leave the Mailman-Developers mailing list In-Reply-To: <437DDFCF-6C04-41EF-9CFA-B6C1DCBF8EAC@supercoders.com.au> References: <437DDFCF-6C04-41EF-9CFA-B6C1DCBF8EAC@supercoders.com.au> Message-ID: <54B33EF8.1050209@msapiro.net> On 01/11/2015 06:27 PM, Andrew Stuart wrote: > errr?. is this someone saying they would prefer I wasn?t on this list? I?d rather hear directly from someone if you feel that way. Unlikely it's that. More likely, its because you've replied to posts without removing the msg_footer from the quoted material in your reply. See, e.g., . Someone sees your post and thinks that's their unsubscribe link. Be glad you didn't do what I once unthinkingly did at . Every time Google, Bing, etc crawl that page, I get an unsub confirmation. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tom.browder at gmail.com Tue Jan 13 22:35:10 2015 From: tom.browder at gmail.com (Tom Browder) Date: Tue, 13 Jan 2015 15:35:10 -0600 Subject: [Mailman-Developers] MM 3: Virtual Environment Required? Message-ID: I want to install, and use, MM 3. I'm on a single-user, Debian 7 server, and control the whole thing. Is it important to use a virtual environment? I'm sure I will be updating from the development branch until the stable release, and I am prepared to have interruptions in service along the way. Note that I may eventually allow other users, but I will probably remain the supreme administrator. Thanks. -Tom From andrew.stuart at supercoders.com.au Wed Jan 14 04:06:54 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Wed, 14 Jan 2015 14:06:54 +1100 Subject: [Mailman-Developers] Your confirmation is required to leave the Mailman-Developers mailing list In-Reply-To: <54B33EF8.1050209@msapiro.net> References: <437DDFCF-6C04-41EF-9CFA-B6C1DCBF8EAC@supercoders.com.au> <54B33EF8.1050209@msapiro.net> Message-ID: <8BDBF06A-1D6F-453C-BC84-40A51BE81F5B@supercoders.com.au> Should Mailman be stripping these out as part of its processing? On 12 Jan 2015, at 2:26 pm, Mark Sapiro wrote: On 01/11/2015 06:27 PM, Andrew Stuart wrote: > errr?. is this someone saying they would prefer I wasn?t on this list? I?d rather hear directly from someone if you feel that way. Unlikely it's that. More likely, its because you've replied to posts without removing the msg_footer from the quoted material in your reply. See, e.g., . Someone sees your post and thinks that's their unsubscribe link. Be glad you didn't do what I once unthinkingly did at . Every time Google, Bing, etc crawl that page, I get an unsub confirmation. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From postmaster at damon-family.org Wed Jan 14 05:32:11 2015 From: postmaster at damon-family.org (Richard Damon) Date: Tue, 13 Jan 2015 23:32:11 -0500 Subject: [Mailman-Developers] Your confirmation is required to leave the Mailman-Developers mailing list In-Reply-To: <8BDBF06A-1D6F-453C-BC84-40A51BE81F5B@supercoders.com.au> References: <437DDFCF-6C04-41EF-9CFA-B6C1DCBF8EAC@supercoders.com.au> <54B33EF8.1050209@msapiro.net> <8BDBF06A-1D6F-453C-BC84-40A51BE81F5B@supercoders.com.au> Message-ID: <54B5F14B.9020603@damon-family.org> Why would it? There isn't anything special about the link that would tell mailman to do so. Ideally, people would have signature separators that would cause the footer to be trimmed, but that isn't mailman's job. (like Mark's did for me). On 1/13/15 10:06 PM, Andrew Stuart wrote: > Should Mailman be stripping these out as part of its processing? > > On 12 Jan 2015, at 2:26 pm, Mark Sapiro wrote: > > On 01/11/2015 06:27 PM, Andrew Stuart wrote: >> errr?. is this someone saying they would prefer I wasn?t on this list? I?d rather hear directly from someone if you feel that way. > > Unlikely it's that. More likely, its because you've replied to posts > without removing the msg_footer from the quoted material in your reply. > See, e.g., > . > > Someone sees your post and thinks that's their unsubscribe link. > > Be glad you didn't do what I once unthinkingly did at > . > Every time Google, Bing, etc crawl that page, I get an unsub confirmation. > From stephen at xemacs.org Wed Jan 14 15:00:41 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 14 Jan 2015 23:00:41 +0900 Subject: [Mailman-Developers] Your confirmation is required to leave the Mailman-Developers mailing list In-Reply-To: <8BDBF06A-1D6F-453C-BC84-40A51BE81F5B@supercoders.com.au> References: <437DDFCF-6C04-41EF-9CFA-B6C1DCBF8EAC@supercoders.com.au> <54B33EF8.1050209@msapiro.net> <8BDBF06A-1D6F-453C-BC84-40A51BE81F5B@supercoders.com.au> Message-ID: <87twztxy0m.fsf@uwakimon.sk.tsukuba.ac.jp> Andrew Stuart writes: > Should Mailman be stripping these out as part of its processing? It could try, but it wouldn't be 100% successful, and it would surely break DKIM signatures. A third-party add-on, sure. A Mailman feature, I'm -1 on that. From barry at list.org Wed Jan 14 23:30:23 2015 From: barry at list.org (Barry Warsaw) Date: Wed, 14 Jan 2015 17:30:23 -0500 Subject: [Mailman-Developers] Your confirmation is required to leave the Mailman-Developers mailing list In-Reply-To: <87twztxy0m.fsf@uwakimon.sk.tsukuba.ac.jp> References: <437DDFCF-6C04-41EF-9CFA-B6C1DCBF8EAC@supercoders.com.au> <54B33EF8.1050209@msapiro.net> <8BDBF06A-1D6F-453C-BC84-40A51BE81F5B@supercoders.com.au> <87twztxy0m.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20150114173023.02a22b32@anarchist.wooz.org> On Jan 14, 2015, at 11:00 PM, Stephen J. Turnbull wrote: >It could try, but it wouldn't be 100% successful, and it would surely >break DKIM signatures. A third-party add-on, sure. A Mailman >feature, I'm -1 on that. Right. I don't think it's a good idea for Mailman to attempt to strip body content by default, especially if it can't be sure that it will always get it correct. I do envision the ability for add-ons in Mailman 3, and that would be a route for folks to experiment with. Cheers, -Barry From raj.abhilash1 at gmail.com Sat Jan 17 18:27:29 2015 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Sat, 17 Jan 2015 22:57:29 +0530 Subject: [Mailman-Developers] MM 3: Virtual Environment Required? In-Reply-To: References: Message-ID: <54BA9B81.5030901@gmail.com> Hi Tom, On Wednesday 14 January 2015 03:05 AM, Tom Browder wrote: > I want to install, and use, MM 3. I'm on a single-user, Debian 7 > server, and control the whole thing. > > Is it important to use a virtual environment? I'm sure I will be > updating from the development branch until the stable release, and I > am prepared to have interruptions in service along the way. I don't think you would need a virtualenv unless you are developing mailman. You can simply download the source and run `python3.4 setup.py install` to install the latest development version or the 3.0.0b5 release. > Note that I may eventually allow other users, but I will probably > remain the supreme administrator. > > Thanks. > -- thanks, Abhilash Raj From terri at toybox.ca Mon Jan 19 09:26:31 2015 From: terri at toybox.ca (Terri Oda) Date: Mon, 19 Jan 2015 00:26:31 -0800 Subject: [Mailman-Developers] MM 3: Virtual Environment Required? In-Reply-To: References: Message-ID: <54BCBFB7.5080607@toybox.ca> On 2015-01-13, 1:35 PM, Tom Browder wrote: > I want to install, and use, MM 3. I'm on a single-user, Debian 7 > server, and control the whole thing. > > Is it important to use a virtual environment? I'm sure I will be > updating from the development branch until the stable release, and I > am prepared to have interruptions in service along the way. It *can* make your life easier if you're running multiple versions of python (which is why developers often like to use it) but it's definitely not required (even then!). As a core dev, I don't use it on some of my development machines. Terri From andrew.stuart at supercoders.com.au Mon Jan 19 14:09:13 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Tue, 20 Jan 2015 00:09:13 +1100 Subject: [Mailman-Developers] Mailman user identification questions Message-ID: I am working on authenticating users, just want to clarify a few things in concrete. Does every Mailman user have a single, unique userid? Is a Mailman userid secret or public information - i.e. any reason not to use the userid in, for example, a cookie that is potentially public information? Is it possible to be a Mailman user WITHOUT having a related Mailman userid i.e. somehow you can use Mailman with an email address that has no corresponding Mailman userid? Do all email addresses map back to a Mailman userid, or is it possible for email addresses to be in some sort of ?orphan/floating state? in which they are not related to a userid? Is it possible for a Mailman user to log in via their email address and password, or only via their userid and password? thanks as From sumanah at panix.com Tue Jan 20 17:38:51 2015 From: sumanah at panix.com (Sumana Harihareswara) Date: Tue, 20 Jan 2015 11:38:51 -0500 Subject: [Mailman-Developers] installation problems blocking a new developer Message-ID: <54BE849B.2080502@panix.com> Hi all! Thank you for your IRC help and for merging several of my patches. :) I'm running into installation problems with Mailman and mailman.client . The Mailman core one, I can work around, but the client bug is stopping me from getting a running dev environment so I can fix bugs in Postorius: https://bugs.launchpad.net/mailman/+bug/1411435 after initial install, master runner crashes when config file is missing https://bugs.launchpad.net/mailman.client/+bug/1411653 Installation fails because of bdist_egg I've been improving http://wiki.mailman.psf.io/DEV/A%205%20minute%20guide%20to%20get%20the%20Mailman%20web%20UI%20running as I go, and have gotten help from /src/mailman/docs/START.rst and the branch https://bazaar.launchpad.net/~raj-abhilash1/mailman/docs/view/head:/src/mailman/docs/WebUIin5.rst . But I am a bit at sea here, and stuck on this bdist_egg problem. Thanks, Sumana -- Sumana Harihareswara http://brainwane.net From raj.abhilash1 at gmail.com Tue Jan 20 18:06:38 2015 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Tue, 20 Jan 2015 22:36:38 +0530 Subject: [Mailman-Developers] installation problems blocking a new developer In-Reply-To: <54BE849B.2080502@panix.com> References: <54BE849B.2080502@panix.com> Message-ID: <54BE8B1E.7000909@gmail.com> Hi Sumana, On Tuesday 20 January 2015 10:08 PM, Sumana Harihareswara wrote: > Hi all! Thank you for your IRC help and for merging several of my > patches. :) Thanks to you for helping with bugs and your effort on improving the wiki is very much appreciated. > I'm running into installation problems with Mailman and mailman.client . > The Mailman core one, I can work around, but the client bug is stopping > me from getting a running dev environment so I can fix bugs in Postorius: > > https://bugs.launchpad.net/mailman/+bug/1411435 > after initial install, master runner crashes when config file is missing I confirm this bug and have faced it myself, but still haven't looked into what is causing this. The work around you mentioned on the bug-report seems to solve the problem. > https://bugs.launchpad.net/mailman.client/+bug/1411653 > Installation fails because of bdist_egg This is not really a bug. MailmanClient is not compatible with python3 yet. Barry has ported it to be bilingual, but it is yet to be merged in the trunk. So if you care to give Barry's branch a spin you can download it from here: lp:~barry/mailman.client/bilingual . Or else you can try running mailman.client with python2.7 and you should not face any problem. > I've been improving > http://wiki.mailman.psf.io/DEV/A%205%20minute%20guide%20to%20get%20the%20Mailman%20web%20UI%20running > as I go, and have gotten help from /src/mailman/docs/START.rst and the > branch > https://bazaar.launchpad.net/~raj-abhilash1/mailman/docs/view/head:/src/mailman/docs/WebUIin5.rst > . But I am a bit at sea here, and stuck on this bdist_egg problem. And I think I have mentioned it in the branch above that mailman.client should be run in (py2) environment. Please let me know it is not clear, I would try to improve the documentation. > Thanks, > Sumana > -- thanks, Abhilash Raj From barry at list.org Tue Jan 20 21:02:42 2015 From: barry at list.org (Barry Warsaw) Date: Tue, 20 Jan 2015 15:02:42 -0500 Subject: [Mailman-Developers] installation problems blocking a new developer In-Reply-To: <54BE849B.2080502@panix.com> References: <54BE849B.2080502@panix.com> Message-ID: <20150120150242.490eaba4@anarchist.wooz.org> On Jan 20, 2015, at 11:38 AM, Sumana Harihareswara wrote: >https://bugs.launchpad.net/mailman/+bug/1411435 >after initial install, master runner crashes when config file is missing This is fixed in trunk now! Cheers, -Barry From flo.fuchs at gmail.com Tue Jan 20 22:13:51 2015 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Tue, 20 Jan 2015 22:13:51 +0100 Subject: [Mailman-Developers] installation problems blocking a new developer In-Reply-To: <54BE849B.2080502@panix.com> References: <54BE849B.2080502@panix.com> Message-ID: <54BEC50F.1090208@gmail.com> Am 20.01.2015 um 17:38 schrieb Sumana Harihareswara: > Hi all! Thank you for your IRC help and for merging several of my > patches. :) Thanks for your contributions! > https://bugs.launchpad.net/mailman.client/+bug/1411653 > Installation fails because of bdist_egg That should be fixed as of 15 minutes ago. A fresh bzr checkout from mailman.client now has support for Python3 and is installable inside the same virtualenv as the mailman (core) package. (Thanks again, Barry...) *But*: Installing it into a py3 venv means it will *not* work (yet) with Postorius or Hyperkitty, since both apps are still running on Py2 only. mailman.client communicates with the Mailman core only via HTTP, so they don't have to be version compatible. Postorius and HK on the other hand import mailmanclient directly, so they do need to live in the same venv. So in order to get a dev environment for the whole suite running, the best solution (for now) is to: - install mailman into a py3 venv and run it from there - install mailman.client, Postorius and/or Hyperkitty into a py2 venv Hope that helps. Florian From barry at list.org Wed Jan 21 00:50:14 2015 From: barry at list.org (Barry Warsaw) Date: Tue, 20 Jan 2015 18:50:14 -0500 Subject: [Mailman-Developers] REST API first pass at Swagger spec In-Reply-To: <89469EF3-9E6A-47F2-BE59-BE1BC4D50E7B@supercoders.com.au> References: <20150111201918.2c0409d9@anarchist.wooz.org> <89469EF3-9E6A-47F2-BE59-BE1BC4D50E7B@supercoders.com.au> Message-ID: <20150120185014.64250026@anarchist.wooz.org> On Jan 12, 2015, at 12:47 PM, Andrew Stuart wrote: >I?m not a fan of integrating the Swagger specification into the code - seems >to make for a VERY large amount of work, a huge amount of complexity and >massive potential for introduction of bugs. Agreed. >I?ve taken the approach of a single static JSON file - MUCH more simple. All >Mailman would need is to include this file (api-docs.json) as a static file >that can be read or if there is no static file server in Mailman (is there? I >haven?t looked) maybe create an api-docs.json endpoint in the API that >returns the static file. Any tools that want to use the api-docs.json spec >file don?t need to be included in Mailman core. MM3 does not have a static file server, but I think we could serve the file as an API endpoint. Perhaps //api-docs.json or if that's difficult to implement, //system/api-docs.json. >The api-docs.json spec file would need to be kept up to date when the Mailman >API changes but it?s really straightforward - building it in the first place >is just tedious research on the methods and properties but maintaining it >should be a matter of a few minutes work to update it when the API changes. I worry a little bit about the extra work to keep it up-to-date when I change the API, but maybe it won't be so bad in practice. I wonder if there are any automated testing tools that can be integrated in the test suite. I.e. something that grabs the api-docs.json and then makes calls against the API to verify endpoints, arguments, and return values. Do you know of any such tool? >I?ll need to finish it up. I?m not sure what to do about getting a pair of >eyes to QA verify it, but maybe we can cross that bridge after I?ve finished >it. Sounds good! Cheers, -Barry From andrew.stuart at supercoders.com.au Wed Jan 21 14:02:07 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Thu, 22 Jan 2015 00:02:07 +1100 Subject: [Mailman-Developers] How can I run the REST tests against live servers? In-Reply-To: <20141211193426.3c93c7b1@marathon> References: <2D09B264-75B1-48C8-9A03-084738B57144@supercoders.com.au> <20141211162752.504ed099@limelight.wooz.org> <20141211193426.3c93c7b1@marathon> Message-ID: <28EC1222-5C8C-43A3-B0DE-9034C0AEED5A@supercoders.com.au> Barry - would you like the authenticating proxy to be part of Mailman core or a third party project? It?s close to being functionally operational (which is to say working but in a raw state). as On 12 Dec 2014, at 11:34 am, Barry Warsaw wrote: On Dec 12, 2014, at 08:35 AM, Andrew Stuart wrote: > I?m writing an authenticating proxy for the Mailman REST API and want to make > sure everything works as expected against real infrastructure > (Postgres/Postfix/Sqlalchemy/Falcon). We definitely want such a proxy. It's always been our intent that the core's REST API is an adminipstrative API with full access. It should only be exposed on localhost or other highly secure interface. A proxy server such as you describe would be available on a public interface and would allow people to script the management of their list memberships and list administrations. > I was hoping that just pointing the REST tests request port to my proxy would > work but no dice - lots of failures - trying to diagnose the failures has led > me back to trying to get the test suite to work against the live servers. > > I?m wondering if maybe the shortest route here might not be for me to write a > new set of tests which purely talk REST and don?t touch the back end directly > at all. Except that?s not a short route. :-) > > Trying to bend the existing tests to meet my needs is looking harder than I > have time to deal with. Even in this case, I wouldn't expect the standard unittests to work. For example, if a normal user were to try to access the list admin API, you'd expect that to fail. I think one approach would be to continue to use the test servers, but have a backdoor connection which would let you set up the state before each test and reset the state after each test. Then the tests themselves would use the authenticated API to flex the things it can and shouldn't be able to do. It would be possible for example to expose some helper resources in the admin API which would not normally be exposed in production. Something under a top level /tests resource perhaps. Cheers, -Barry _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From barry at list.org Wed Jan 21 16:57:04 2015 From: barry at list.org (Barry Warsaw) Date: Wed, 21 Jan 2015 10:57:04 -0500 Subject: [Mailman-Developers] How can I run the REST tests against live servers? In-Reply-To: <28EC1222-5C8C-43A3-B0DE-9034C0AEED5A@supercoders.com.au> References: <2D09B264-75B1-48C8-9A03-084738B57144@supercoders.com.au> <20141211162752.504ed099@limelight.wooz.org> <20141211193426.3c93c7b1@marathon> <28EC1222-5C8C-43A3-B0DE-9034C0AEED5A@supercoders.com.au> Message-ID: <20150121105704.369e885d@anarchist.wooz.org> On Jan 22, 2015, at 12:02 AM, Andrew Stuart wrote: >Barry - would you like the authenticating proxy to be part of Mailman core or >a third party project? For now, I think it should not be distributed as part of core. Maybe after the 3.0 release. The code could still be donated to the FSF and managed under the GNU Mailman project in a separate source repository. That's kind of your call; let me know if you'd like to do that. >It?s close to being functionally operational (which is to say working but in >a raw state). Very cool. Thanks for working on this important piece. Cheers, -Barry From godricglow at gmail.com Thu Jan 22 08:28:21 2015 From: godricglow at gmail.com (Pranjal Yadav) Date: Thu, 22 Jan 2015 12:58:21 +0530 Subject: [Mailman-Developers] Introductory mail Message-ID: Hello I'm a programming enthusiast and recently started working in python about 6 months back, I have had experience in cryptography and many machine learning techniques. I started reading about mailman few weeks back and I have setup a local and server dev env successfully. I also learned about previous year's GSOC project and I can see mailman core is already ported to ver 3.4 however potorius / hyperkitty / mailman.client needs to be done. Also test suite would be interesting work to couple with all this. I would like to know if these 2 projects can be coupled for this year's work or not? Apart from that I also found few bugs while setting up env which was mainly due to (var) folder creation while running mailman server from different locatin serving various instances. It seems unless I do it from path/to/mailman it returns an error. I will go through the code in over next few weeks and till then I can expect a reply for above projects. PS : The documentation for mailman setup is outdated I guess, reported that to @maxking -- Pranjal Yadav Final Year Undergraduate Student Department of Metallurgical and Materials Engineering IIT Kharagpur From rajeevs1992 at gmail.com Fri Jan 23 10:45:58 2015 From: rajeevs1992 at gmail.com (Rajeev S) Date: Fri, 23 Jan 2015 15:15:58 +0530 Subject: [Mailman-Developers] Introductory mail In-Reply-To: References: Message-ID: On Thu, Jan 22, 2015 at 12:58 PM, Pranjal Yadav wrote: > Hello > > I'm a programming enthusiast and recently started working in > python about 6 months back, I have had experience in cryptography and many > machine learning techniques. I started reading about mailman few weeks back > and I have setup a local and server dev env successfully. > > I also learned about previous year's GSOC project and I can see > mailman core is already ported to ver 3.4 however potorius / hyperkitty / > mailman.client needs to be done. Also test suite would be interesting work > to couple with all this. I would like to know if these 2 projects can be > coupled for this year's work or not? > > Apart from that I also found few bugs while setting up env which > was mainly due to (var) folder creation while running mailman server from > different locatin serving various instances. It seems unless I do it from > path/to/mailman it returns an error. I will go through the code in over > next few weeks and till then I can expect a reply for above projects. > > I guess this is not a bug. There is a setting in the mailman.cfg that tells mailman where to put in the var directory. Look for var_dir under [paths.dev] PS : The documentation for mailman setup is outdated I guess, reported that > to @maxking > > > -- > Pranjal Yadav > Final Year Undergraduate Student > Department of Metallurgical and Materials Engineering > IIT Kharagpur > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > https://mail.python.org/mailman/options/mailman-developers/rajeevs1992%40gmail.com > > Security Policy: http://wiki.list.org/x/QIA9 > From raj.abhilash1 at gmail.com Fri Jan 23 11:49:09 2015 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Fri, 23 Jan 2015 16:19:09 +0530 Subject: [Mailman-Developers] Introductory mail In-Reply-To: References: Message-ID: <54C22725.6020604@gmail.com> Hi Pranjal! On Thursday 22 January 2015 12:58 PM, Pranjal Yadav wrote: > Hello > > I'm a programming enthusiast and recently started working in > python about 6 months back, I have had experience in cryptography and many > machine learning techniques. I started reading about mailman few weeks back > and I have setup a local and server dev env successfully. > > I also learned about previous year's GSOC project and I can see > mailman core is already ported to ver 3.4 however potorius / hyperkitty / > mailman.client needs to be done. Also test suite would be interesting work > to couple with all this. I would like to know if these 2 projects can be > coupled for this year's work or not? It is not decided yet, but it is expected that postorius and hyperkitty would be ported to be bilingual by this year's pycon. I suggest you look for other project, the list on the wiki will be updated soon. > Apart from that I also found few bugs while setting up env which > was mainly due to (var) folder creation while running mailman server from > different locatin serving various instances. It seems unless I do it from > path/to/mailman it returns an error. I will go through the code in over > next few weeks and till then I can expect a reply for above projects. The var folder creation in the currect directory is a design pattern built for developers. So if mailman does not see any working instance of itself in the current directory it starts a new instance with `var` directory in the new location. You can change this behavior in mailman.cfg, look src/mailman/config/mailman.cfg for [paths.*] sections. > PS : The documentation for mailman setup is outdated I guess, reported that > to @maxking > > -- thanks, Abhilash Raj -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From barry at list.org Fri Jan 23 14:49:56 2015 From: barry at list.org (Barry Warsaw) Date: Fri, 23 Jan 2015 08:49:56 -0500 Subject: [Mailman-Developers] Introductory mail In-Reply-To: References: Message-ID: <20150123084956.46ed26d3@limelight.wooz.org> On Jan 23, 2015, at 03:15 PM, Rajeev S wrote: >I guess this is not a bug. There is a setting in the mailman.cfg that tells >mailman where to put in the var directory. This has also changed recently in bzr trunk, hopefully for the better. Cheers, -Barry From andrew.stuart at supercoders.com.au Sat Jan 24 04:12:04 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sat, 24 Jan 2015 14:12:04 +1100 Subject: [Mailman-Developers] Who is the "site administrator"? Message-ID: Is this defined in any way? Is there any concept that a given Mailman user has ?site admin rights?? Or is the concept of the ?site administrator? not realised in the application and just considered to be someone with operating system level access to the system that Mailman is running on? thanks From terri at toybox.ca Sat Jan 24 05:13:41 2015 From: terri at toybox.ca (Terri Oda) Date: Fri, 23 Jan 2015 20:13:41 -0800 Subject: [Mailman-Developers] Who is the "site administrator"? In-Reply-To: References: Message-ID: <54C31BF5.2000901@toybox.ca> On 2015-01-23, 7:12 PM, Andrew Stuart wrote: > Is this defined in any way? > > Is there any concept that a given Mailman user has ?site admin rights?? > > Or is the concept of the ?site administrator? not realised in the application and just considered to be someone with operating system level access to the system that Mailman is running on? In Mailman 2, there's no admin "users" in a typical sense but the site administrator is someone who knows the global password for a site and/or has shell access. The global password lets you log in as an admin to any list, and depending on your settings may let you do things like create and delete lists. And yes, it's really just one shared password In Mailman 3 there's a much more nuanced user system and you can have users with site privileges (and there's no longer a need to share passwords). Terri From andrew.stuart at supercoders.com.au Sat Jan 24 06:05:46 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sat, 24 Jan 2015 16:05:46 +1100 Subject: [Mailman-Developers] Who is the "site administrator"? In-Reply-To: <54C31BF5.2000901@toybox.ca> References: <54C31BF5.2000901@toybox.ca> Message-ID: <42C69B23-5B7A-45F0-8F5B-FA63B720BD90@supercoders.com.au> >>you can have users with site privileges I had a look at the data model but couldn?t see this. Would you mind please pointing me in the direction of where user site privilege information is stored? I can see that there is the concept of ?Rosters? but rosters seem to apply to lists and not at a higher level i.e. as far as I can see there?s no way to apply a roster to the system rather than to a list. The main thing I?m looking for is whether there is an authorisation concept that operates at a higher level than the list. I wonder is there the concept of some sort of ?special? mailing list that is different or hidden or privileged in some way? There is a reference in the documentation to a ?site list? but I coudn?t find much more about it. If there is a ?special? list then maybe site wide user priveleges can be stored against it. I can see there is a site_owner in the Mailman config file although this just seems to be an email address to get bounces in certain circumstances, is it used for anything? as On 24 Jan 2015, at 3:13 pm, Terri Oda wrote: On 2015-01-23, 7:12 PM, Andrew Stuart wrote: > Is this defined in any way? > > Is there any concept that a given Mailman user has ?site admin rights?? > > Or is the concept of the ?site administrator? not realised in the application and just considered to be someone with operating system level access to the system that Mailman is running on? In Mailman 2, there's no admin "users" in a typical sense but the site administrator is someone who knows the global password for a site and/or has shell access. The global password lets you log in as an admin to any list, and depending on your settings may let you do things like create and delete lists. And yes, it's really just one shared password In Mailman 3 there's a much more nuanced user system and you can have users with site privileges (and there's no longer a need to share passwords). Terri _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From terri at toybox.ca Sat Jan 24 06:16:04 2015 From: terri at toybox.ca (Terri Oda) Date: Fri, 23 Jan 2015 21:16:04 -0800 Subject: [Mailman-Developers] Who is the "site administrator"? In-Reply-To: <42C69B23-5B7A-45F0-8F5B-FA63B720BD90@supercoders.com.au> References: <54C31BF5.2000901@toybox.ca> <42C69B23-5B7A-45F0-8F5B-FA63B720BD90@supercoders.com.au> Message-ID: <54C32A94.4090306@toybox.ca> On 2015-01-23, 9:05 PM, Andrew Stuart wrote: >>> you can have users with site privileges > > I had a look at the data model but couldn?t see this. Would you mind please pointing me in the direction of where user site privilege information is stored? I'm not sure the concept is visible in Mailman Core, but I'm pretty sure it's in Postorius. (And if you're bored, you can look back in the archives to the huge discussion we had about unifying the user data store... ;) ) From stephen at xemacs.org Sat Jan 24 08:45:42 2015 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 24 Jan 2015 16:45:42 +0900 Subject: [Mailman-Developers] Introductory mail In-Reply-To: References: Message-ID: <87lhkstye1.fsf@uwakimon.sk.tsukuba.ac.jp> Pranjal Yadav writes: > I also learned about previous year's GSOC project and I > can see mailman core is already ported to ver 3.4 however potorius > / hyperkitty / mailman.client needs to be done. Also test suite > would be interesting work to couple with all this. I would like to > know if these 2 projects can be coupled for this year's work or > not? If you think of them as two projects, then they probably are. As a mentor, I won't touch that -- most students have enough trouble just getting started on *one* project. Also, note that projects that are just adding to a test suite are not eligible for Google support. > Apart from that I also found few bugs while setting up > env which was mainly due to (var) folder creation while running > mailman server from different locatin serving various instances. It > seems unless I do it from path/to/mailman it returns an error. I > will go through the code in over next few weeks and till then I can > expect a reply for above projects. You should go over the mailing list archives first. var folder creation was discussed extensively a couple of months ago. In general, if you're going to propose a project not on our list for *this year* (which hasn't been published yet), it is usually helpful to review the past two year's threads on the mailing lists (both mailman-developers and mailman-users) to get an idea of what the current needs are considered to be, and what things the developers are addressing already. From sumanah at panix.com Sat Jan 24 13:31:08 2015 From: sumanah at panix.com (Sumana Harihareswara) Date: Sat, 24 Jan 2015 07:31:08 -0500 Subject: [Mailman-Developers] Who is the "site administrator"? In-Reply-To: <42C69B23-5B7A-45F0-8F5B-FA63B720BD90@supercoders.com.au> References: <54C31BF5.2000901@toybox.ca> <42C69B23-5B7A-45F0-8F5B-FA63B720BD90@supercoders.com.au> Message-ID: <54C3908C.1090204@panix.com> On 01/24/2015 12:05 AM, Andrew Stuart wrote: >>> you can have users with site privileges > > I had a look at the data model but couldn?t see this. Would you mind please pointing me in the direction of where user site privilege information is stored? > > I can see that there is the concept of ?Rosters? but rosters seem to apply to lists and not at a higher level i.e. as far as I can see there?s no way to apply a roster to the system rather than to a list. > > The main thing I?m looking for is whether there is an authorisation concept that operates at a higher level than the list. > > I wonder is there the concept of some sort of ?special? mailing list that is different or hidden or privileged in some way? There is a reference in the documentation to a ?site list? but I coudn?t find much more about it. If there is a ?special? list then maybe site wide user priveleges can be stored against it. > > I can see there is a site_owner in the Mailman config file although this just seems to be an email address to get bounces in certain circumstances, is it used for anything? > > > > as > > > > On 24 Jan 2015, at 3:13 pm, Terri Oda wrote: > > On 2015-01-23, 7:12 PM, Andrew Stuart wrote: >> Is this defined in any way? >> >> Is there any concept that a given Mailman user has ?site admin rights?? >> >> Or is the concept of the ?site administrator? not realised in the application and just considered to be someone with operating system level access to the system that Mailman is running on? > > In Mailman 2, there's no admin "users" in a typical sense but the site administrator is someone who knows the global password for a site and/or has shell access. The global password lets you log in as an admin to any list, and depending on your settings may let you do things like create and delete lists. And yes, it's really just one shared password > > In Mailman 3 there's a much more nuanced user system and you can have users with site privileges (and there's no longer a need to share passwords). > > Terri Just yesterday I spoke with Barry and straightened out my own understanding of the privilege levels within Mailman. Here's the hierarchy as I understand it (the first level has the most power and each successive level has less power): 1) site administrator - has command-line (shell) access to the machine. (The website's system administrator, usually.) Has ultimate power! Can invoke "mailman stop", for instance, to stop mailman services. 2) Mailman administrator - does not have command line access to the machine, but can use the web interface to change everything within the Mailman installation on that site (that is, can do anything Postorius lets you!). Can create, change, and delete domains, lists, users, and so on. 3) domain administrator - Any given list belongs to 1 and only 1 domain, which correlates to an actual internet domain name, such as mozilla.org. A domain administrator can create, change/administer, and delete lists. If you set up multiple domains, you can, e.g., have one domain administrator who has power over the lists at the 70 lists at example.com, and another domain administrator for the 40 lists at example.org. 4) list administrator - can change list settings such as preferred language, digest threshold, footer, etc., and moderate messages, manage users, etc. 5) list moderator - cannot change the list's settings as a list administrator could, but can allow/reject messages on moderated lists or that trip certain rules, approve subscriptions, ban users, etc. 6) user - can change their own user preferences I do not know where all this is laid out in the code, though. And I am totally willing to be told I have gotten the nomenclature wrong! -- Sumana Harihareswara http://brainwane.net From andrew.stuart at supercoders.com.au Sat Jan 24 13:46:39 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sat, 24 Jan 2015 23:46:39 +1100 Subject: [Mailman-Developers] Who is the "site administrator"? In-Reply-To: <54C3908C.1090204@panix.com> References: <54C31BF5.2000901@toybox.ca> <42C69B23-5B7A-45F0-8F5B-FA63B720BD90@supercoders.com.au> <54C3908C.1090204@panix.com> Message-ID: <47988937-A8DA-45E7-AF4D-D9CA0C3F6AB2@supercoders.com.au> Having dug around the models and data structures a bit I am wondering if some of some of these constructs are not Mailman constructs but Postorius constructs, which is to say, third party application level contstructs rather than Mailman core constructs. Specifically the ones you refer to ?domain administrator? and ?mailman administrator? don?t seem to be represented in the Mailman core code. Perhaps I?m wrong of course, I?ve just done a little ham fisted glancing over the code but so far I can?t see how a user would get to become ?domain administrator? or ?mailman administrator?. Which is fine of course - no problem with these being application level constructs - I?m just trying to understand because I?m writing some authorisation code and I need to know how to authorise a user to the correct level of their privilege. thanks as On 24 Jan 2015, at 11:31 pm, Sumana Harihareswara wrote: On 01/24/2015 12:05 AM, Andrew Stuart wrote: >>> you can have users with site privileges > > I had a look at the data model but couldn?t see this. Would you mind please pointing me in the direction of where user site privilege information is stored? > > I can see that there is the concept of ?Rosters? but rosters seem to apply to lists and not at a higher level i.e. as far as I can see there?s no way to apply a roster to the system rather than to a list. > > The main thing I?m looking for is whether there is an authorisation concept that operates at a higher level than the list. > > I wonder is there the concept of some sort of ?special? mailing list that is different or hidden or privileged in some way? There is a reference in the documentation to a ?site list? but I coudn?t find much more about it. If there is a ?special? list then maybe site wide user priveleges can be stored against it. > > I can see there is a site_owner in the Mailman config file although this just seems to be an email address to get bounces in certain circumstances, is it used for anything? > > > > as > > > > On 24 Jan 2015, at 3:13 pm, Terri Oda wrote: > > On 2015-01-23, 7:12 PM, Andrew Stuart wrote: >> Is this defined in any way? >> >> Is there any concept that a given Mailman user has ?site admin rights?? >> >> Or is the concept of the ?site administrator? not realised in the application and just considered to be someone with operating system level access to the system that Mailman is running on? > > In Mailman 2, there's no admin "users" in a typical sense but the site administrator is someone who knows the global password for a site and/or has shell access. The global password lets you log in as an admin to any list, and depending on your settings may let you do things like create and delete lists. And yes, it's really just one shared password > > In Mailman 3 there's a much more nuanced user system and you can have users with site privileges (and there's no longer a need to share passwords). > > Terri Just yesterday I spoke with Barry and straightened out my own understanding of the privilege levels within Mailman. Here's the hierarchy as I understand it (the first level has the most power and each successive level has less power): 1) site administrator - has command-line (shell) access to the machine. (The website's system administrator, usually.) Has ultimate power! Can invoke "mailman stop", for instance, to stop mailman services. 2) Mailman administrator - does not have command line access to the machine, but can use the web interface to change everything within the Mailman installation on that site (that is, can do anything Postorius lets you!). Can create, change, and delete domains, lists, users, and so on. 3) domain administrator - Any given list belongs to 1 and only 1 domain, which correlates to an actual internet domain name, such as mozilla.org. A domain administrator can create, change/administer, and delete lists. If you set up multiple domains, you can, e.g., have one domain administrator who has power over the lists at the 70 lists at example.com, and another domain administrator for the 40 lists at example.org. 4) list administrator - can change list settings such as preferred language, digest threshold, footer, etc., and moderate messages, manage users, etc. 5) list moderator - cannot change the list's settings as a list administrator could, but can allow/reject messages on moderated lists or that trip certain rules, approve subscriptions, ban users, etc. 6) user - can change their own user preferences I do not know where all this is laid out in the code, though. And I am totally willing to be told I have gotten the nomenclature wrong! -- Sumana Harihareswara http://brainwane.net From andrew.stuart at supercoders.com.au Sat Jan 24 14:08:09 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sun, 25 Jan 2015 00:08:09 +1100 Subject: [Mailman-Developers] Who is the "site administrator"? In-Reply-To: <54C32A94.4090306@toybox.ca> References: <54C31BF5.2000901@toybox.ca> <42C69B23-5B7A-45F0-8F5B-FA63B720BD90@supercoders.com.au> <54C32A94.4090306@toybox.ca> Message-ID: <8F32BC16-38D9-4753-ABBB-28844CD2AE8D@supercoders.com.au> >>>you can look back in the archives to the huge discussion we had about unifying the user data store.. Is there search to find that discussion or do I have to manually review all the threads? Any idea what year it was in? thanks On 24 Jan 2015, at 4:16 pm, Terri Oda wrote: On 2015-01-23, 9:05 PM, Andrew Stuart wrote: >>> you can have users with site privileges > > I had a look at the data model but couldn?t see this. Would you mind please pointing me in the direction of where user site privilege information is stored? I'm not sure the concept is visible in Mailman Core, but I'm pretty sure it's in Postorius. (And if you're bored, you can look back in the archives to the huge discussion we had about unifying the user data store... ;) ) From sumanah at panix.com Sat Jan 24 14:10:02 2015 From: sumanah at panix.com (Sumana Harihareswara) Date: Sat, 24 Jan 2015 08:10:02 -0500 Subject: [Mailman-Developers] Who is the "site administrator"? In-Reply-To: <8F32BC16-38D9-4753-ABBB-28844CD2AE8D@supercoders.com.au> References: <54C31BF5.2000901@toybox.ca> <42C69B23-5B7A-45F0-8F5B-FA63B720BD90@supercoders.com.au> <54C32A94.4090306@toybox.ca> <8F32BC16-38D9-4753-ABBB-28844CD2AE8D@supercoders.com.au> Message-ID: <54C399AA.1060406@panix.com> The searchable archives are at http://www.mail-archive.com/mailman-developers%40python.org/ . (Linked to from https://mail.python.org/mailman/listinfo/mailman-developers but I know that's a big chunk of text.) (Have I mentioned I cannot wait for HyperKitty and its built-in search?) -Sumana On 01/24/2015 08:08 AM, Andrew Stuart wrote: >>>> you can look back in the archives to the huge discussion we had about unifying the user data store.. > > Is there search to find that discussion or do I have to manually review all the threads? Any idea what year it was in? > > thanks > > > > On 24 Jan 2015, at 4:16 pm, Terri Oda wrote: > > On 2015-01-23, 9:05 PM, Andrew Stuart wrote: >>>> you can have users with site privileges >> >> I had a look at the data model but couldn?t see this. Would you mind please pointing me in the direction of where user site privilege information is stored? > > I'm not sure the concept is visible in Mailman Core, but I'm pretty sure it's in Postorius. > > (And if you're bored, you can look back in the archives to the huge discussion we had about unifying the user data store... ;) ) > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/sumanah%40panix.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- Sumana Harihareswara http://brainwane.net From andrew.stuart at supercoders.com.au Sat Jan 24 14:40:53 2015 From: andrew.stuart at supercoders.com.au (Andrew Stuart) Date: Sun, 25 Jan 2015 00:40:53 +1100 Subject: [Mailman-Developers] How can I run the REST tests against live servers? In-Reply-To: <20150121105704.369e885d@anarchist.wooz.org> References: <2D09B264-75B1-48C8-9A03-084738B57144@supercoders.com.au> <20141211162752.504ed099@limelight.wooz.org> <20141211193426.3c93c7b1@marathon> <28EC1222-5C8C-43A3-B0DE-9034C0AEED5A@supercoders.com.au> <20150121105704.369e885d@anarchist.wooz.org> Message-ID: Sure thing that?s fine. FYI the Mailman REST authentication proxy works in this way: ** Python 3 only ** built using Falcon ** the proxy reads the structure of the Mailman 3 REST API by parsing the Mailman 3 Swagger API specification ** each Mailman 3 REST API URL/HTTP method call combination is defined in the Swagger spec and has a unique OperationID ** the proxy intercepts all inbound requests to the Mailman 3 REST API, this is done through Falcon?s ?sink? capability ** the proxy does a regex match on the inbound URL and looks up the operationID to work out which Mailman 3 REST API method is being called ** in the proxy a set of request handlers is defined for each Mailman 3 API operation ** the inbound request is pushed through the request handlers, typically the final request handler proxies the authenticated request to the Mailman REST API, unless one of the request handlers aborts the processing due to some sort of auth issue A typical example of one API operation mapping to its set of handlers looks something like this: {?UsersGet?: [validateJSONAuthToken, refreshJSONAuthToken, userMustbeListAdmin, proxyRequestToMailman]} In the above example the inbound request goes through a handler that validates its JWT authentication token, a handler that refreshes its JWT token, a handler that checks to ensure that the requesting user (as identified from the JWT token) has the appropriate rights to carry out the requested Mailman 3 operation, then the request is proxied to the Mailman REST API and the result is send back to the client/requestor. JSON web token authentication is automatically layered on top of the API. Typically you would want the authenticating proxy to sit behind a reverse proxy such as HAproxy or Nginx which would do gzip compression and SSL. Most of this is working. as On 22 Jan 2015, at 2:57 am, Barry Warsaw wrote: On Jan 22, 2015, at 12:02 AM, Andrew Stuart wrote: > Barry - would you like the authenticating proxy to be part of Mailman core or > a third party project? For now, I think it should not be distributed as part of core. Maybe after the 3.0 release. The code could still be donated to the FSF and managed under the GNU Mailman project in a separate source repository. That's kind of your call; let me know if you'd like to do that. > It?s close to being functionally operational (which is to say working but in > a raw state). Very cool. Thanks for working on this important piece. Cheers, -Barry _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 From terri at toybox.ca Sat Jan 24 21:37:31 2015 From: terri at toybox.ca (Terri Oda) Date: Sat, 24 Jan 2015 12:37:31 -0800 Subject: [Mailman-Developers] Who is the "site administrator"? In-Reply-To: <8F32BC16-38D9-4753-ABBB-28844CD2AE8D@supercoders.com.au> References: <54C31BF5.2000901@toybox.ca> <42C69B23-5B7A-45F0-8F5B-FA63B720BD90@supercoders.com.au> <54C32A94.4090306@toybox.ca> <8F32BC16-38D9-4753-ABBB-28844CD2AE8D@supercoders.com.au> Message-ID: <54C4028B.2000302@toybox.ca> On 2015-01-24, 5:08 AM, Andrew Stuart wrote: >>>> you can look back in the archives to the huge discussion we had about unifying the user data store.. > > Is there search to find that discussion or do I have to manually review all the threads? Any idea what year it was in? The data store discussion was I think two years ago around the time of pycon. I started it with a thread entitled "Architecture for extra profile info" https://mail.python.org/pipermail/mailman-developers/2013-April/022852.html (As Sumana says, mail-archive is usually the best bet for searching the mailman archives. I searched locally, though.) But I'm serious that it's only worth reading if you're incredibly bored. It became a huge architecture discussion that only ended on the lists with an "agree to disagree." (and I got bombarded with further comments about it on IRC for months after the list discussion ended, so I'm actually kind of still cranky about it.) I think it's still a thing we need to look in to more seriously, but be warned that it's a topic folk have been passionate about in the past, to the point where progress was not made. On the other hand, one of the people involved in that discussion has moved on to other things and the rest of us do collaborative work together more frequently, so it might be easier to reach a solution now. Terri From mark at msapiro.net Sat Jan 24 22:53:38 2015 From: mark at msapiro.net (Mark Sapiro) Date: Sat, 24 Jan 2015 13:53:38 -0800 Subject: [Mailman-Developers] Heads up on pending Mailman 2.1.19 release Message-ID: <54C41462.3000309@msapiro.net> I will soon be packaging and announcing the first release candidate for Mailman 2.1.19. The details of changes since the 2.1.18-1 release are in the attached README. The reason for this heads up is there are many things in this release that have i18n impacts and I want to give as much notice as possible of these changes so that people who are interested in updating templates and the message catalogs for a particular language can hopefully get their changes in in time for the final release. If you are interested in helping with this, whether or not you are a current translator, see . Many of the changes have to do with the fact that for several years, I have been maintaining the officially defunct 2.2 branch in parallel with 2.1. This branch had a number of features and fixes that had never been put in the 2.1 branch because of their i18n impacts. I have now decided to backport all those changes and finally abandon the 2.2 branch. These changes affect the message catalog and add a new adminaddrchgack.txt template and make a minor change to the admindbdetails.html template. I have also implemented new features that weren't in the 2.2 branch, but add new strings to the web admin UI. See the attached README for more on what all these changes are. The mailman.pot message template has been updated with all these changes and msgmerged with the various mailman.po files. All of this can be retrieved from the bazaar branch at at rev. 1520 (or later). -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- 2.2 Branch Backports (released in conjunction with 2.1.19) The following New Features and Bug Fixes have been in an "unofficial, never to be released" Mailman 2.2 branch for several years. Until now, they were never implemented on the official 2.1 branch because of their i18n impacts. Given that there have been a number of i18n impacting changes due to DMARC mitigations in the last few releases, it has been decided to backport these as well. All of these changes have been running in production on several lists for years without problems other than untranslated strings, so they should be reasonably "bug free". New Features - There is a new list attribute 'subscribe_auto_approval' which is a list of email addresses and regular expressions matching email addresses whose subscriptions are exempt from admin approval. (LP: #266609) - Confirmed member change of address is logged in the 'subscribe' log, and if admin_notify_mchanges is true, a notice is sent to the list owner using a new adminaddrchgack.txt template. - Added an 'automate' option to bin/newlist to send the notice to the admin without the prompt. - The processing of Topics regular expressions has changed. Previously the Topics regexp was compiled in verbose mode but not documented as such which caused some confusion. Also, the documentation indicated that topic keywords could be entered one per line, but these entries were not handled properly. Topics regexps are now compiled in non-verbose mode and multi-line entries are 'ored'. Existing Topics regexps will be converted when the list is updated so they will continue to work. - Added real name display to the web roster. (LP: #266754) Bug fixes and other patches - Changed the response to an invalid confirmation to be more generic. Not all confirmations are subscription requests. - Changed the default nonmember_rejection_notice to be more user friendly. (LP: #418728) - Added "If you are a list member" qualification to some messages from the options login page. (LP: #266442) - Changed the 'Approve' wording in the admindbdetails.html template to 'Accept/Approve' for better agreement with the button labels. - Added '(by thread)' to the previous and next message links in the archive to emphasize that even if you got to the message from a subject, date or author index, previous and next are still by thread. 2.1.19 (xx-xxx-xxxx) New Features - There is a new list attribute dmarc_wrapped_message_text and a DEFAULT_DMARC_WRAPPED_MESSAGE_TEXT setting to set the default for new lists. This text is added to a message which is wrapped because of dmarc_moderation_action in a separate text/plain part that precedes the message/rfc822 part containing the original message. It can be used to provide an explanation of why the message was wrapped or similar info. - There is a new list attribute equivalent_domains and a DEFAULT_EQUIVALENT_DOMAINS setting to set the default for new lists which in turn defaults to the empty string. This provides a way to specify one or more groups of domains, e.g., mac.com, me.com, icloud.com, which are considered equivalent for validating list membership for posting and moderation purposes. - There is a new WEB_HEAD_ADD setting to specify text to be added to the section of Mailman's internally generated web pages. This doesn't apply to pages built from templates, but in those cases, custom templates can be created. (LP: #1409396) - There is a new DEFAULT_SUBSCRIBE_OR_INVITE setting. Set this to Yes to make the default selection on the admin Mass Subscriptions page Invite rather than Subscribe. (LP: #1404511) - There is a new list attribute in the Bounce processing section. bounce_notify_owner_on_bounce_increment if set to Yes will cause Mailman to notify the list owner on every bounce that increments a list member's score but doesn't result in a probe or disable. There is a new configuration setting setting DEFAULT_BOUNCE_NOTIFY_OWNER_ON_BOUNCE_INCREMENT to set the default for new lists. This in turn defaults to No. (LP: #1382150) Changed behavior - Mailman's log files, request.pck files and heldmsg-* files are no longer created world readable to protect against access by untrusted local users. Note that permissions on existing log files won't be changed so if you are concerned about this and don't rotate logs or have a logrotate process that creates new log files instead of letting Mailman create them, you will need to address that. (LP: #1327404) Other changes - The Python Powered logo image has been replaced in the misc/ directory in the source distribution. Depending on how you've installed these images, you may need to copy PythonPowered.png from the misc/ directory in the source or from the $prefix/icons/ installed directory to another location for your web server. (LP: #1408575) i18n - The Russian templates have been updated by Danil Smirnov. (LP: #1403462) - The Japanese translation has been updated by SATOH Fumiyasu. (LP: #1402989) - A minor change in the French translation of a listinfo subscribe form message has been made. (LP: #1331194) Bug fixes and other patches - When applying DMARC mitigations, CookHeaders now adds the original From: to Cc: rather than Reply-To: in some cases to make MUA 'reply' and 'reply all' more consistent with the non-DMARC cases. (LP: #1407098) - The Subject: of the list welcome message wasn't always in the user's preferred language. Fixed. (LP: #1400988) - Accept email command in Subject: prefixed with Re: or similar with no intervening space. (LP: #1400200) - Fixed a UnicodeDecodeError that could occur in the web admin interface if 'text' valued attributes have unicode values. (LP: #1397170) - We now catch the NotAMemberError exception thrown if an authenticated unsubscribe is submitted from the user options page for a nonmember. (LP: #1390653) - Fixed an archiving bug that would cause messages with 'Subject: Re:' only to be indexed in the archives without a link to the message. (LP: #1388614) - The vette log entry for a message discarded by a handler now includes the list name and the name of the handler. (LP: #558096) - The options CGI now rejects all but HTTP GET and POST requests. (LP: #1372199) - A list's poster password will now be accepted on an Urgent: header. (LP: #1371678) - Fixed a bug which caused a setting of 2 for REMOVE_DKIM_HEADERS to be ignored. (LP: #1363278) - Renamed messages/sr/readme.sr to README.sr. (LP: #1360616) - Moved the dmarc_moderation_action checks from the Moderate handler to the SpamDetect handler so that the Reject and Discard actions will be done before the message might be held by header_filter_rules, and the Wrap Message and Munge From actions will be done on messages held by header_filter_rules if the message is approved. (LP: #1334450) -