From maxking at asynchronous.in Fri Dec 1 01:16:01 2017 From: maxking at asynchronous.in (Abhilash Raj) Date: Thu, 30 Nov 2017 22:16:01 -0800 Subject: [Mailman-Developers] 3.1.x REST API enhancements around preferred email address In-Reply-To: <20171120172013.GC33354@paprika.localdomain> References: <20171120172013.GC33354@paprika.localdomain> Message-ID: <1512108961.11845.12.camel@asynchronous.in> Hi Scott, On Mon, 2017-11-20 at 11:20 -0600, Scott Koranda wrote: > Hello, > > I am interested in contributing enhancements around preferred address > to the REST API for version 3.1.x. > > To prove out my use case(s) and familiarize myself with the code I have > made changes already. Here are some example curl command line > invocations and output to illustrate: > > 1) Create an address for an existing user and mark that address as the > user's preferred address: > > curl -u restadmin:restpass \ > -X POST \ > -d email=skoranda at example.nil \ > -d preferred=1 \ > http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8/addr > esses > This returns a '201 Created'. > > 2) Return the preferred address, if set, for a user: > > curl -u restadmin:restpass \ > -X GET \ > http://mystestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8 | > python -m json.tool > > { > "created_on": "2017-11-11T22:20:45.950949", > "display_name": "Scott Koranda", > "http_etag": "\"26bc0d9f21eb145248884aedae4c1ffd00b608d2\"", > "is_server_owner": false, > "password": > "$6$rounds=656000$gWGfwpajfZ7rVn5O$Sl559B2TgtpJWXA2i67G5ukjzkV6iTp4NgP.6FJpMFM > UTDdDXULAmwdN8YW92w87EdctgqFqAUkUqS6.EOTCz/", > "preferred_address": "skoranda at example.nil", > "self_link": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b1 > 8882a6d8", > "user_id": "e57ecfd0c0c74a319ac958b18882a6d8" > } > > 3) Update/patch a user to set a preferred address: > > curl -u restadmin:restpass \ > -X PATCH \ > -d preferred_address=skoranda at example.nil \ > http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8 > > This returns a '204 No Content'. > > 4) Get all addresses for a user with the preferred address (if set) having > the attribute 'preferred' set to 'true': > > curl -u restadmin:restpass \ > -X GET \ > http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8/addr > esses | python -m json.tool > > { > "entries": [ > { > "email": "skoranda at example.nil", > "http_etag": "\"492824d150f9f11e32d530d3cad0f76422bddb48\"", > "original_email": "skoranda01 at example.nil", > "preferred": true, > "registered_on": "2017-11-13T22:35:17.232180", > "self_link": "http://mytestbed.com:8001/3.1/addresses/skoranda at exa > mple.nil", > "user": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac95 > 8b18882a6d8", > "verified_on": "2017-11-20T16:48:51.738145" > }, > { > "email": "skoranda01 at example.nil", > "http_etag": "\"f0f28c1c6a925a595269d3fe21ef7fc1515d7670\"", > "original_email": "skoranda01 at example.nil", > "registered_on": "2017-11-20T16:46:20.941262", > "self_link": "http://mytestbed.com:8001/3.1/addresses/skoranda01 at e > xample.nil", > "user": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac95 > 8b18882a6d8", > "verified_on": "2017-11-20T16:46:20.941595" > }, > ], > "http_etag": "\"89c46bd312c6f1a344e6e6180aed6f11b62e6048\"", > "start": 0, > "total_size": 2 > } > > Before I submit a pull request I would be grateful for any comments > or feedback on the functionality as illustrated above, or any other comments > or feedback. I think these all seem like reasonable additions. - an extra `preferred` boolean attribute when creating an address is good, I couldn't find an endpoint to actually change the preferred_address of a user, so this is good. - I think instead of embedding the `preferred_address` in user (/user/), we could have a sub-resource that represents the preferred address, if any exists. so something like /user/ > I will also begin the "copyright assignment" paperwork as explained at > > https://wiki.list.org/DEV/Home Great! > > Thank you for your consideration. Thank you for taking time to contribute to Mailman! :) -- thanks, Abhilash Raj From geneshuman at gmail.com Fri Dec 1 03:23:57 2017 From: geneshuman at gmail.com (Gene Shuman) Date: Fri, 1 Dec 2017 00:23:57 -0800 Subject: [Mailman-Developers] ARC Message-ID: Hi all, I'm writing again about the ARC PR that I've been working on that I'd like to get reviewed & merged. I've been in touch with Abhilash Raj about this and he's graciously agreed to put some energy into it when he has time. But in general, I've put some thought into it, and have come up with a general plan to streamline this review, to make it as simple as possible for Abhilash or anybody else with the interest or bandwidth to help. First the are 4 major components of the review: - Barry Warsaw did a preliminary review which focused mostly on the code syntax & organization, and I've correspondently updated the code. So first point is verifying this, which should be easy. - Next is verifying that the way in which I've approached this problem is appropriately in line with how mailman is abstractly structured. Also easy. - Next again is verifyomg that the documentation is sufficient from a users(or developers) perspective. I've recently updated the documentation, and believe it to be sufficient. To test this, somebody with possibly marginal familiarity with ARC needs to just try to setup a Mailman server with ARC functionality. This is closely related to the next point. - Finally is verifying that the code behaves cryptographically correctly This last point is clearly what most imperatively needs to be right, and I'll devote a bit of this message towards it. Honestly, I don't think it should be that difficult for somebody to convince themselves Mailman is handling ARC correctly, even knowing little about it. Depending on the level of paranoia here(any of which are justifiable,) there are several routes to going about this. But first an overview of how this PR implements ARC. This may be a bit long winded, so hopefully bear with me. So this PR does ARC signing & validation via a library of mine, authheaders , which basically aggregates & passes along email authentication & signing functions to other libraries. It's quite simple & its easy to see that it's not modifying the results its getting from the other libraries. In particular, with respect to ARC, its using a fork of the library dkimpy . The fork has passed review, and afaik will be merged & released any day now. The fork brings the dkimpy library's ARC implementation(which I wrote) from 99% compliance with the ARC test suite (which I also authored), to 100%. So it's absolutely demonstrably clear that the library that my ARC Mailman PR is using is cryptographically correct. The test suite itself has also been validated from several other directions, so at this point is authoritative. In particular, I've worked with the engineer at Google responsible for their ARC implementation, Brandon Long, on this, and I have also worked in depth with the major authors of the ARC rfc to clarify the protocol, illuminate edge cases, and bring the document to completion. But in particular, wrt Google, I've personally verified that their ARC implementation is 100% compliant with the spec & the test suite. And their ARC implementation is live and is validating & signing 100% of their email traffic atm. So here is my simple proposal for validating the cryptographic validity of my PR. Set up a Mailman server with ARC configured properly. Add some google email addresses to the outbound list, send a bunch of emails, and inspect the headers to see that ARC appropriately verifies. Then take the same Mailman server and turn of ARC and see that the messages fail. As a further step you can set up a nested sequence of Mailman servers, with some or all or none of them ARC enabled, and then check the results. I personally think it is within reason to trust Google as an authority here, as I've personally followed their development, and know that their implementation is 100% compliant. As an additional level of security here, we can combine this with direct validation fo dkimpy. I can directly show somebody how to test dkimpy against the ARC test suite themselves. I don't think this is necessary, as I've done it hundreds of times, but in 30m I can help somebody check this for themselves personally Another thing to note. There are tests added to the Mailman test suite that directly test the crypto. They aren't comprehensive, but they are there & they pass. So to be summarize, I'm perfectly confident that that the ARC implementation in Mailman is rock solid, as I've personally written the intermediary layer, the actual ARC implementation in dkimpy, the ARC test suite, and have been a non-trivial contributor to the specification itself. And with any of the above steps necessary for this validation, I'm available to help. What ever I can do to make this as painless as possible, I'm available for. Now to the next level of paranoia, which is the bottom level, and would prove beyond a shadow of a doubt that Mailman's ARC implementation is absolutely cryptographically compliant. First, a bit about the ARC test suite. It's a bunch of yaml files that specifies various messages, edge cases, etc, and what the output should be. It's designed abstractly so that you can test it against any system which is ARC capable, by just creating a system of translation layers between the test suite & the software being tested. It's been tested against dkimpy in such a manner. It's also been tested against googles ARC implementation, OpenARC, and others. It's quite general. It would be possible to directly run the test suite against Mailman. This would be a bit awkward, as its a mail server, as opposed to a library, but it could be done. And this literally would be bullet proof evidence that the system is behaving cryptographically correctly. However the translation layer may be a pain to write. But I haven't put too much thought into that. I personally think that the argument that the library, dkimpy, which the ARC PR is currently using passes test-suite validation is strong enough. But I'm an expert wrt ARC, and I understand that argument via claims of expertise aren't really all that valid. So the bottom line is this. Either I, or somebody else can wire up some ARC enabled Mailman servers, point them at Google, and check that things check out, using Google's ARC implementation as a reference. And possibly to demonstrate running the ARC test suite against dkimpy. The other option is making a harness to directly run the ARC test suite against mailman. This is doable, and would be a hassle, but is not out of the question. I could likely personally do this, and would happily do so. Thank you to anybody who has read this far. Please let me know your thoughts. And don't hesitate with any questions about ARC or DMARC, or other email authentication protocols. I'm a near encyclopedic resource about these things at this point, and am happy to share my knowledge. Looking forward to working with guys! Regards, =Gene From turnbull.stephen.fw at u.tsukuba.ac.jp Fri Dec 1 05:21:49 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 1 Dec 2017 19:21:49 +0900 Subject: [Mailman-Developers] ARC PR for Mailman 3 In-Reply-To: References: Message-ID: <23073.11581.533790.348516@turnbull.sk.tsukuba.ac.jp> Gene Shuman writes: > Over the past few months I've developed an patch for Mailman 3, > giving it ARC > email > authentication capabilities. The work is (99%)completed, and it just needs > somebody to do a final review of the ARC functionality itself so I can > implement any requested changes. So at this point we have two complete ARC patches, yours and Aditya Divekar's from last year's GSoC which I mentored, and I'm pretty sure somebody else is working on it. I think somebody else mentioned a patch, too. > I had been working with Barry Warsaw, I wish he'd mentioned it.... > who had done a preliminary stylistic review, and subsequently, I've > eliminated all raised issues. The branch is rebased as of last > week, and passes all of qa, except one instance, which is something > I'd like to discuss with somebody. (It fails a style test for line > lengths in a test file which contains a bunch of cryptographic > hashes). I don't understand. Python offers several ways to deal with this, such as sha_values = ["0123456789012345678901234567890123456789"] foo(sha_value[0]) or if they're really long, adjacent strings will concatenate: foo("01234567890123456789" "01234567890123456789") I would think either would be fine, depending on your preferred style. Abhilash Raj (who just received the baton from Barry) would have the final say. I guess the main question I have at this point is whether you've participated in any of the interoperability testing that the ARC developers have done. I'll take a look at your MR and see if there are any features that differ across the implementations. Right now I'm buried until the 13th for sure, and more likely until Christmas. I definitely will be able to work on it during the holiday vacation, though. Steve -- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull at sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN From geneshuman at gmail.com Wed Dec 13 03:35:08 2017 From: geneshuman at gmail.com (Gene Shuman) Date: Wed, 13 Dec 2017 00:35:08 -0800 Subject: [Mailman-Developers] ARC PR for Mailman 3 In-Reply-To: <23073.11581.533790.348516@turnbull.sk.tsukuba.ac.jp> References: <23073.11581.533790.348516@turnbull.sk.tsukuba.ac.jp> Message-ID: HI Stephen, great to get in touch with you. On Fri, Dec 1, 2017 at 2:21 AM, Stephen J. Turnbull < turnbull.stephen.fw at u.tsukuba.ac.jp> wrote: > Gene Shuman writes: > > > Over the past few months I've developed an patch for Mailman 3, > > giving it ARC > > email > > authentication capabilities. The work is (99%)completed, and it just > needs > > somebody to do a final review of the ARC functionality itself so I can > > implement any requested changes. > > So at this point we have two complete ARC patches, yours and Aditya > Divekar's from last year's GSoC which I mentored, and I'm pretty sure > somebody else is working on it. I think somebody else mentioned a > patch, too. > So I know very well the other competing ARC PR that you're referring to. I investigated it closely before I started own my work on ARC for Mailman. It's pretty clear to me, and probably anybody else who takes a closer look at it that it has basically 0 chance of getting merged into Mailman. There are two major issues at hand. One, that it is based of of a somewhat sloppy custom fork of dkimpy, that would never have had a chance to make its way into the main library itself. My ARC code on the other hand is already merged into that library and released. There is still a minor patch waiting to be merged(just pathological edge cases), which has been approved and is just waiting on the maintainer of the library to have a bit of time to complete the merge & release. I've been in close discussion with the maintainer(Scott Kitterman) through this entire process, and it's on track for merging as soon as he can spare some time, which my understanding will be not too long from now. My best guess will be some time in January at the latest, but I can reach out to him and see if he can confirm this. The second point is that the original PR is basically dead. The code is not active, has not been tested in any interops and will certainly not validation reasonably against the reference ARC test suite, as the specification itself has changed significantly since that PR was created(and for multiple other reasons). It is my personally recommendation that that PR should just be discarded. (Sincere apologies to the author if you're on this mailing list, no offense intended) However I'm curious about the other patch you mentioned. If this is the one from ValiMail, this is also me. I don't work for them any more, but I am now contracting with them to get this ARC code which has been basically ready for 6-8+ months reviewed & merged. If there is another competing developmental effort for ARC underway, I strongly recommend that they halt their efforts. My patch is complete, has passed a first review stage, and has been validated extensively, technically. There is quite literally no reason at all I can imagine for anybody else to be working on this, as, pending feedback, It's basically done. But if you are aware of another attempt at ARC in Mailman, I would *very* much appreciate it if you point me at it and its developers. > > > I had been working with Barry Warsaw, > > I wish he'd mentioned it.... > > > who had done a preliminary stylistic review, and subsequently, I've > > eliminated all raised issues. The branch is rebased as of last > > week, and passes all of qa, except one instance, which is something > > I'd like to discuss with somebody. (It fails a style test for line > > lengths in a test file which contains a bunch of cryptographic > > hashes). > > I don't understand. Python offers several ways to deal with this, > such as > > sha_values = ["0123456789012345678901234567890123456789"] > foo(sha_value[0]) > > or if they're really long, adjacent strings will concatenate: > > foo("01234567890123456789" > "01234567890123456789") > > I would think either would be fine, depending on your preferred > style. Abhilash Raj (who just received the baton from Barry) would > have the final say. > I entirely understand this. But these types of solutions just seem a bit awkward to me. Compensating for line lengths this way, just to satisfy a stylistic requirement, I personally believe would reduce the quality of the code. I understand the reason for line length limits just fine. But to split a 500 character cryptographic hex string into segments just to appease to static analyzer seem like it imposes a reduction of code quality that entirely misses the spirit of the underlying rule. An argument could then be made to just put these crypto strings into files. But there are maybe ~15 violations of this nature. And I'd guess we don't think we want to clutter the file system up with ~15+ random, very specific crypto files. For fragility reason if no other. But I'm 100% behind whatever the community thinks is best practice here. I personally advocate for simply turning off the line length restriction in the 2 relevant test files. After that I would advocate concatenating shorter strings, which wouldn't be too bad. After that moving those strings to files, which I do kind of think is a bad idea, but whatever. Honestly any of these are fine, and would probably in totally take me no more than an hour or so to implement. So whatever approach you guys prefer, I have no problem with, and will do so ASAP. Just let me know what is preferred. I guess the main question I have at this point is whether you've > participated in any of the interoperability testing that the ARC > developers have done. > Unequivocally, yes. I personally wrote the ARC test suite. I worked directly with Kurt Anderson and several others on elucidating all of the corner cases of the ARC RFC to make sure the test suite(an the spec itself) was bullet proof, and AFAIK, it is now accepted as the industry standard reference by which to validate all ARC implementations. I worked with Google to make sure their ARC implementation passed the test suite. I contributed significantly, and directly worked with the author of OpenARC, an ARC milter written in C. As of maybe a few weeks ago, it now completely validates against the test suite & as such is considered production ready. And as I'd mentioned, I solely wrote the ARC implementation in dkimpy, which my Mailman patch directly relies on, and of course made sure it was 100% ARC specification compliant. > I'll take a look at your MR and see if there are any features that > differ across the implementations. Right now I'm buried until the > 13th for sure, and more likely until Christmas. I definitely will be > able to work on it during the holiday vacation, though. > Fantastic. To be perfectly honest, I've probably personally written more ARC code than anybody else in the world. I've outlined several proposals for conveying the cryptographic integrity of this software to the Mailman community, in a previous email. If you have any thoughts on the various approaches, please chime in. My personal hope is to have a fully qualified/validated/etc PR ready to merge into Mailman by the end of next January(a date I made up but which seems feasible). I have the bandwidth and can help out in any and all ways in the mean time. And Abhilash Raj has also said that he is going to try to find some time help finish this process as well. Steve > Thank you greatly for your response, and all of your time in general. Regards, =Gene > > > -- > Associate Professor Division of Policy and Planning Science > http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information > Email: turnbull at sk.tsukuba.ac.jp University of Tsukuba > Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN > From pmlists at free.fr Wed Dec 13 07:35:42 2017 From: pmlists at free.fr (=?utf-8?Q?Peter_M=C3=BCnster?=) Date: Wed, 13 Dec 2017 13:35:42 +0100 Subject: [Mailman-Developers] ARC PR for Mailman 3 References: <23073.11581.533790.348516@turnbull.sk.tsukuba.ac.jp> Message-ID: <878te6u7oh.fsf@free.fr> On Wed, Dec 13 2017, Gene Shuman wrote: >> foo("01234567890123456789" >> "01234567890123456789") > > Compensating for line lengths this way, just to satisfy a stylistic > requirement, I personally believe would reduce the quality of the > code. Hi, I personally believe that the limitation of the line length would *increase* the quality of the code. (Please, no lines with 500 characters in a text file.) Kind regards, -- Peter From mark at msapiro.net Thu Dec 14 20:43:47 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 14 Dec 2017 17:43:47 -0800 Subject: [Mailman-Developers] Gitlab CI jobs stuck. Message-ID: <5929d680-af09-8b55-db65-2a0251a9617e@msapiro.net> CI on Gitlab seems to be stuck. Somehow "docker-runner: using maxking/mailman-ci-runner image." is not available. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From maxking at asynchronous.in Thu Dec 14 20:55:41 2017 From: maxking at asynchronous.in (Abhilash Raj) Date: Thu, 14 Dec 2017 17:55:41 -0800 Subject: [Mailman-Developers] Gitlab CI jobs stuck. In-Reply-To: <5929d680-af09-8b55-db65-2a0251a9617e@msapiro.net> References: <5929d680-af09-8b55-db65-2a0251a9617e@msapiro.net> Message-ID: <1513302941.27683.10.camel@asynchronous.in> On Thu, 2017-12-14 at 17:43 -0800, Mark Sapiro wrote: > CI on Gitlab seems to be stuck. Somehow "docker-runner: using > maxking/mailman-ci-runner image." is not available. Should now be working! > -- thanks, Abhilash Raj From mark at msapiro.net Thu Dec 14 22:25:04 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 14 Dec 2017 19:25:04 -0800 Subject: [Mailman-Developers] Gitlab CI jobs stuck. In-Reply-To: <1513302941.27683.10.camel@asynchronous.in> References: <5929d680-af09-8b55-db65-2a0251a9617e@msapiro.net> <1513302941.27683.10.camel@asynchronous.in> Message-ID: <1e37d44c-f21a-f804-25bf-6e863fe7b4bf@msapiro.net> On 12/14/2017 05:55 PM, Abhilash Raj wrote: > > Should now be working! Yes, it is. Thanks. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From 455750938 at qq.com Fri Dec 15 08:43:49 2017 From: 455750938 at qq.com (=?gb18030?B?y+/WvtPC?=) Date: Fri, 15 Dec 2017 21:43:49 +0800 Subject: [Mailman-Developers] mailman var directory pip3 Message-ID: Hi, I installed mailman3.1.1 using pip3 on debian9.3. Where is the var directory of mailman?Thank you. From tc at tobiasconradi.com Fri Dec 15 14:03:44 2017 From: tc at tobiasconradi.com (Tobias Conradi) Date: Fri, 15 Dec 2017 20:03:44 +0100 Subject: [Mailman-Developers] Date of Mailman 3 Suite going Python 3 only after Django dropped Python 2 Message-ID: https://www.djangoproject.com/weblog/2017/dec/02/django-20-released/ 1) Is there a date for when Mailman Suite will switch to Python 3 only, so user are not required to install two Python versions? 2) Is there a date for a switch to Django 2? -- Tobias Conradi Rheinsberger Str. 18 10115 Berlin Germany https://tobiasconradi.com From mark at msapiro.net Fri Dec 15 15:15:42 2017 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 15 Dec 2017 12:15:42 -0800 Subject: [Mailman-Developers] mailman var directory pip3 In-Reply-To: References: Message-ID: <6b5f2793-1b00-2b80-f23d-1a30ed56c116@msapiro.net> On 12/15/2017 05:43 AM, ??? wrote: > Hi, I installed mailman3.1.1 using pip3 on debian9.3. > Where is the var directory of mailman?Thank you. In the home directory of the user that runs 'mailman'. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From simon at hannaweb.eu Fri Dec 15 18:23:40 2017 From: simon at hannaweb.eu (Simon Hanna) Date: Sat, 16 Dec 2017 00:23:40 +0100 Subject: [Mailman-Developers] mailman var directory pip3 In-Reply-To: <6b5f2793-1b00-2b80-f23d-1a30ed56c116@msapiro.net> References: <6b5f2793-1b00-2b80-f23d-1a30ed56c116@msapiro.net> Message-ID: On 12/15/2017 09:15 PM, Mark Sapiro wrote: > On 12/15/2017 05:43 AM, ??? wrote: >> Hi, I installed mailman3.1.1 using pip3 on debian9.3. >> Where is the var directory of mailman?Thank you. > > In the home directory of the user that runs 'mailman'. That's where it should be (by convention). It will be created in the current directory that you run any mailman command from, unless you specify a config directory. The recommended setup is a user called mailman owning /var/lib/mailman where the var directory would be placed. From simon at hannaweb.eu Fri Dec 15 18:43:10 2017 From: simon at hannaweb.eu (Simon Hanna) Date: Sat, 16 Dec 2017 00:43:10 +0100 Subject: [Mailman-Developers] Date of Mailman 3 Suite going Python 3 only after Django dropped Python 2 In-Reply-To: References: Message-ID: <23f1f90f-7cf3-196f-c6cf-84404ef5080a@hannaweb.eu> On 12/15/2017 08:03 PM, Tobias Conradi wrote: > https://www.djangoproject.com/weblog/2017/dec/02/django-20-released/ > > 1) Is there a date for when Mailman Suite will switch to Python 3 only, > so user are not required to install two Python versions? AFAIK there are merge requests for all the parts, the only hold-up is testing to make sure they actualy work. You can search for "Python 3" in the gitlab issue trackers for more information. > 2) Is there a date for a switch to Django 2? Since Django 2 only supports Python3 support for it will most probably be added by merging the Python3 branches. I haven't gone over all the details, but I don't think there are any changes that still require porting... From 455750938 at qq.com Sun Dec 17 04:59:12 2017 From: 455750938 at qq.com (=?gb18030?B?y+/WvtPC?=) Date: Sun, 17 Dec 2017 17:59:12 +0800 Subject: [Mailman-Developers] no module named django Message-ID: Hi, when I init uwsgi, I find a problem "importerror no module named django". I install postorius following installation in http://postorius.readthedocs.io/en/latest/setup.html, and visit http://192.168.1.96:8001 it workes well after run: $ sudo python manage.py runserver 0.0.0.0:8001. Do not know what is about development in the page: http://postorius.readthedocs.io/en/latest/development.html, so I pass it. Then go to deployment in http://postorius.readthedocs.io/en/latest/deployment.html. I creat file /home/lists/uwsgi.ini and put in it: --- [uwsgi] chdir = /home/lists/postorius/sample_project #virtualenv = /srv/django/mailman/env #socket = /run/uwsgi/mailman.sock wsgi-file = wsgi.py master = true process = 4 threads = 2 vacuum = true plugin = python2 uid = http gid = http --- then run: $ uwsgi --ini /home/lists/uwsgi.ini it gives the error "importerror, no module named django". I think it is the problem of the below 2 line: #virtualenv = /srv/django/mailman/env #socket = /run/uwsgi/mailman.sock but docs do not mention how to set about this 2 line, so I just comment out the 2 line. I also tried with mailman-suite, also have the same problem. can you help with me? will you think to write a installing instruction with command to follow for people totally don't know django in future? Thank you. From simon at hannaweb.eu Sun Dec 17 08:33:12 2017 From: simon at hannaweb.eu (Simon Hanna) Date: Sun, 17 Dec 2017 14:33:12 +0100 Subject: [Mailman-Developers] no module named django In-Reply-To: References: Message-ID: On 12/17/2017 10:59 AM, ??? wrote: > Hi, when I init uwsgi, I find a problem "importerror no module named django". > > > I install postorius following installation in http://postorius.readthedocs.io/en/latest/setup.html, and visit http://192.168.1.96:8001 it workes well after run: $ sudo python manage.py runserver 0.0.0.0:8001. That is very bad. You shouldn't run that as root! www-data or http or whatever your web user is, is the right user for that. I wonder why you use 0.0.0.0:8001 8001 is the default port of Mailman core, you probably don't want to use that. You need to have core on your system for postorius to actually do anything... > Do not know what is about development in the page: http://postorius.readthedocs.io/en/latest/development.html, so I pass it. Do you want to setup for development? If yes, the guide should be more than enough to tell you what needs to be done. If you follow the link to https://wiki.list.org/DEV/HowToContributeGit you should be able to do that. > Then go to deployment in http://postorius.readthedocs.io/en/latest/deployment.html. This somewhat contradicts the above url.. > I creat file /home/lists/uwsgi.ini and put in it: > --- > [uwsgi] > chdir = /home/lists/postorius/sample_project > #virtualenv = /srv/django/mailman/env > #socket = /run/uwsgi/mailman.sock > > wsgi-file = wsgi.py > master = true > process = 4 > threads = 2 > vacuum = true > plugin = python2 > > uid = http > gid = http > > --- > > > then run: > $ uwsgi --ini /home/lists/uwsgi.ini > > > it gives the error "importerror, no module named django". > > > I think it is the problem of the below 2 line: > #virtualenv = /srv/django/mailman/env since you managed to install postorius and get it running, you either installed it systemwide which would work with it commented out, or you actually created a virtualenv in which case you should know where the location of that is. (virtualenv is preferred, you only use the system packages, if they are installes using your package manager and the application is too. Otherwise you will run into incompatibilties sooner or later. > #socket = /run/uwsgi/mailman.sock This is the socket that is created for the webserver to connect to. I'm actually not sure what happens if you don't include that... I doubt uwsgi would actually start. > > but docs do not mention how to set about this 2 line, so I just comment out the 2 line. > > > I also tried with mailman-suite, also have the same problem. > > > can you help with me? > will you think to write a installing instruction with command to follow for people totally don't know django in future? It's not entirely clear what you actually want to do, so it's hard to help... If you want it for production I suggest you use the docker images provided by maxking or wait a couple of weeks until the packages get into debian backports, where you can just run one command and then get a running system. For dev, the mentioned link should be more than enough to get you up and running. If you have issues from following a guide, please mention that so they can be improved... From simon at hannaweb.eu Mon Dec 18 10:28:09 2017 From: simon at hannaweb.eu (Simon Hanna) Date: Mon, 18 Dec 2017 16:28:09 +0100 Subject: [Mailman-Developers] no module named django In-Reply-To: References: Message-ID: Replying back to the list :-) > #virtualenv? = /srv/django/mailman/env > since you managed to install postorius and get it running, you either > installed it systemwide which would work with it commented out, or you > actually created a virtualenv in which case you should know where the > location of that is. (virtualenv is preferred, you only use the system > packages, if they are installes using your package manager and the > application is too. Otherwise you will run into incompatibilties sooner > or later. > > totally do not know about virtualenv, just follow the guide to install > postorius. > when runserver, it can visit by 192.168.1.96, when "ctrl+c", > 192.168.1.96 not work. virtualenv is a python concept, you should be able to find information online about that. It's about how and where you install python dependencies. > > > #socket? = /run/uwsgi/mailman.sock > This is the socket that is created for the webserver to connect to. I'm > actually not sure what happens if you don't include that... I doubt > uwsgi would actually start. > > installed uwsgi and it seems running, I run $ uwsgi, there is no error. > but can't find /run/uwsgi/ directory, usually mailman.sock can be at > which directory? > By convention sockets are placed in /run/ you can put them wherever you want. > I tried with docker, here is its guide with 6 steps: > --- > $ mkdir -p /opt/mailman/core > $ mkdir -p /opt/mailman/web > $ git clone https://github.com/maxking/docker-mailman > $ cd docker-mailman > # Change some configuration variables as mentioned above. > $ docker-compose up -d > ---- > But about configuration in step 5, I can't > find?/opt/mailman/core/mailman-extra.cfg > and?/opt/mailman/web/settings_local.py, and can't find it in any > directory from /home/lists/docker-mailman/. > so do not know what to with it. is it to create these two files? > Yes, you have to manually create them. The idea is that there are default settings and you would override them in additional files. The default settings are found here https://github.com/maxking/docker-mailman/blob/master/web/mailman-web/settings.py https://github.com/maxking/docker-mailman/blob/master/core/assets/mailman.cfg There are some settings that you have to overwrite, as mentioned in the readme of https://github.com/maxking/docker-mailman You do so, by specifing them in the respective files that you create in /opt/mailman/ on your host machine. You can search for the settings in the original files in the github repo and then use the same syntax in your new files. > OK, maybe it is realistic for me to wait for debian backport. That will definelty mean less work for you, and you will have it running directly on the machine managed by your package manager. This will probably be easier with upgrades as well. From maxking at asynchronous.in Mon Dec 18 15:21:05 2017 From: maxking at asynchronous.in (Abhilash Raj) Date: Mon, 18 Dec 2017 12:21:05 -0800 Subject: [Mailman-Developers] Date of Mailman 3 Suite going Python 3 only after Django dropped Python 2 In-Reply-To: <23f1f90f-7cf3-196f-c6cf-84404ef5080a@hannaweb.eu> References: <23f1f90f-7cf3-196f-c6cf-84404ef5080a@hannaweb.eu> Message-ID: <1513628465.1922348.1209183144.5C3D57ED@webmail.messagingengine.com> On Fri, Dec 15, 2017, at 03:43 PM, Simon Hanna wrote: > On 12/15/2017 08:03 PM, Tobias Conradi wrote: > > https://www.djangoproject.com/weblog/2017/dec/02/django-20-released/ > > > > 1) Is there a date for when Mailman Suite will switch to Python 3 only, > > so user are not required to install two Python versions? > AFAIK there are merge requests for all the parts, the only hold-up is > testing to make sure they actualy work. You can search for "Python 3" in > the gitlab issue trackers for more information. There exists merge request for all the django applications. As Simon mentioned, I am the guy stalling for some more testing :) My personal deadline is the end-of-this-year, so I'd say sometime by end of January would be a reasonable timeframe for an alpha release of Python 3 based Django apps. > > 2) Is there a date for a switch to Django 2? > Since Django 2 only supports Python3 support for it will most probably > be added by merging the Python3 branches. I haven't gone over all the > details, but I don't think there are any changes that still require > porting... This will be done as a part of the Python 3 port, so next releases should support Django-2.0 out of the box. -- Abhilash Raj maxking at asynchronous.in From geneshuman at gmail.com Tue Dec 19 16:45:48 2017 From: geneshuman at gmail.com (Gene Shuman) Date: Tue, 19 Dec 2017 16:45:48 -0500 Subject: [Mailman-Developers] ARC PR for Mailman 3 In-Reply-To: <878te6u7oh.fsf@free.fr> References: <23073.11581.533790.348516@turnbull.sk.tsukuba.ac.jp> <878te6u7oh.fsf@free.fr> Message-ID: Reasonable enough, will update the PR shortly after hollidays. Regards, =Gene On Dec 13, 2017 10:34 AM, "Peter M?nster" wrote: On Wed, Dec 13 2017, Gene Shuman wrote: >> foo("01234567890123456789" >> "01234567890123456789") > > Compensating for line lengths this way, just to satisfy a stylistic > requirement, I personally believe would reduce the quality of the > code. Hi, I personally believe that the limitation of the line length would *increase* the quality of the code. (Please, no lines with 500 characters in a text file.) Kind regards, -- Peter _______________________________________________ Mailman-Developers mailing list Mailman-Developers at python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/ mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman- developers/geneshuman%40gmail.com Security Policy: http://wiki.list.org/x/QIA9 From maxking at asynchronous.in Wed Dec 27 03:49:26 2017 From: maxking at asynchronous.in (Abhilash Raj) Date: Wed, 27 Dec 2017 00:49:26 -0800 Subject: [Mailman-Developers] Handling additions to REST API in client side Message-ID: <1514364566.2747.14.camel@asynchronous.in> Hi All, Core's REST API is versioned and any change that break backwards-compatibility cause the version to bump so that clients can take care of that. However, one question that I have been thinking about recently is how to handle additions to REST API that don't necessarily break the backwards compatibility. For example, Core added `max_message_size` attribute to MailingList's REST endpoint, but it hasn't made into any released version yet. Also, Postorius added max_message_size in `Message Acceptance` settings. The problem here is that the entire PUT/PATCH request is going to fail if the currently running version of Core doesn't have `max_message_size` attribute exposed (Unknown Attribute Error). There is no easy way to check for whether the Core has this attribute as API is versioned at 3.1 for both cases. So, how do we actually handle this and maybe future cases like this? -- thanks, Abhilash Raj -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 866 bytes Desc: This is a digitally signed message part URL: From maxking at asynchronous.in Thu Dec 28 02:16:06 2017 From: maxking at asynchronous.in (Abhilash Raj) Date: Wed, 27 Dec 2017 23:16:06 -0800 Subject: [Mailman-Developers] Postorius 1.1.2 security release Message-ID: <1514445366.370.24.camel@asynchronous.in> Hi Everyone, I am pleased to announce that Postorius 1.1.2 is released and is up on PyPI[1]. This release fixes a security bug that sets the password of a user in Core to their display name. It is recommended that you upgrade to this version. Postorius (Django) and Mailman Core both have different notion of "user" and "password". When a user account in created in Postorius, it creates a user in Core using the REST API. This bug, causes the password of user created in Core to be set to their display name instead. However, as of now, there are no use cases of the user password in Core and it is present only for historical reasons. So, while this bug is a serious one, it wouldn't result in any real-world exploit. Along with the bug-fix, this release includes a new command that resets *all* user passwords in Core to a random value. Again, there are no use cases of these passwords so resetting *all* of them isn't going to cause any inconvenience to users. This command should be run after the upgrade: $ cd mailman-suite/mailman-suite_project/ $ python manage.py reset_passwords Python 2.7 is the only supported Python version for this release. All versions of Django <=1.11 is supported. For more information about GNU Mailman and Postorius, please see our website: http://list.org The source code is available on Gitlab: https://gitlab.com/mailman/ [1]: https://pypi.org/project/postorius -- thanks, Abhilash Raj -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 866 bytes Desc: This is a digitally signed message part URL: From simon at hannaweb.eu Thu Dec 28 11:33:44 2017 From: simon at hannaweb.eu (Simon Hanna) Date: Thu, 28 Dec 2017 17:33:44 +0100 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: <1514364566.2747.14.camel@asynchronous.in> References: <1514364566.2747.14.camel@asynchronous.in> Message-ID: On 12/27/2017 09:49 AM, Abhilash Raj wrote: > Hi All, > > Core's REST API is versioned and any change that break backwards-compatibility > cause the version to bump so that clients can take care of that. > > However, one question that I have been thinking about recently is how to handle > additions to REST API that don't necessarily break the backwards compatibility. > > For example, Core added `max_message_size` attribute to MailingList's REST > endpoint, but it hasn't made into any released version yet. Also, Postorius > added max_message_size in `Message Acceptance` settings. The problem here is > that the entire PUT/PATCH request is going to fail if the currently running > version of Core doesn't have `max_message_size` attribute exposed (Unknown > Attribute Error). > > There is no easy way to check for whether the Core has this attribute as API is > versioned at 3.1 for both cases. > > So, how do we actually handle this and maybe future cases like this? How about defining it this way: * The result of queries can be viewed as dictionaries * New endpoints (urls) can be added anytime * No endpoint is removed without a version bump * Existing dict keys will not be dropped without a version bump * The format of values assigned to existing keys will not change without a version bump * New keys (and values) can be added anytime This would make things like exposing new stuff easy. Clients that don't know about a field can just ignore it. From mark at msapiro.net Thu Dec 28 12:19:14 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 28 Dec 2017 09:19:14 -0800 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: References: <1514364566.2747.14.camel@asynchronous.in> Message-ID: <0798851c-40de-2906-2fc3-2d9f8b04eee5@msapiro.net> On 12/28/2017 08:33 AM, Simon Hanna wrote: > * New keys (and values) can be added anytime > > This would make things like exposing new stuff easy. Clients that don't > know about a field can just ignore it. But that doesn't address the original issue which is that when a new key/value is added and a client is updated to use it, the client has no way to require a core API that provides it. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Thu Dec 28 15:25:50 2017 From: barry at list.org (Barry Warsaw) Date: Thu, 28 Dec 2017 15:25:50 -0500 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: <1514364566.2747.14.camel@asynchronous.in> References: <1514364566.2747.14.camel@asynchronous.in> Message-ID: <98422230-8A68-413E-9489-AC60F76E9B70@list.org> Abhilash Raj wrote: > Core's REST API is versioned and any change that break backwards-compatibility > cause the version to bump so that clients can take care of that. The 3.1 version bump happened because there was no backward compatible way to handle the UUID as int vs hex string change required for some JavaScript libraries. It couldn't be autodetected in all cases. In general though, we've mostly be able to manage changes in a backward compatible way. > However, one question that I have been thinking about recently is how to handle > additions to REST API that don't necessarily break the backwards compatibility. This problem is analogous to adding arguments to functions. This is usually handled in backward compatible ways in Python by appending new arguments to the end of the parameter list and giving them default values. Or to use keyword-only arguments. The analogy breaks down in one specific case though. > For example, Core added `max_message_size` attribute to MailingList's REST > endpoint, but it hasn't made into any released version yet. Also, Postorius > added max_message_size in `Message Acceptance` settings. The problem here is > that the entire PUT/PATCH request is going to fail if the currently running > version of Core doesn't have `max_message_size` attribute exposed (Unknown > Attribute Error). PATCH won't fail because it allows for partial representations. PUT does fail because it requires the entire new representation to be included in the request (it's a complete replacement). This is where the analogy to function arguments break down. I don't really know of any good way to handle this that still conforms to REST principles. I don't think we want to rev the API in these cases since that'll result in a lot of version churn. > There is no easy way to check for whether the Core has this attribute as API is > versioned at 3.1 for both cases. > > So, how do we actually handle this and maybe future cases like this? Simon suggests: * The result of queries can be viewed as dictionaries * New endpoints (urls) can be added anytime * No endpoint is removed without a version bump * Existing dict keys will not be dropped without a version bump * The format of values assigned to existing keys will not change without a version bump * New keys (and values) can be added anytime This is pretty much the criteria I've used in the past, and it works well enough in practice except for the PUT exception. A couple of thoughts on how to handle this include, using PATCH in preference to PUT, using PUT but catch any exception then fall back to PATCH, do a GET first to get the list of keys. None of those are great options, although some caching might help. Mailman's REST API is very dynamic so we don't even have a static representation of it that can be queried. I did a quick scan of RESTful Web APIs (Richardson & Amundsen - my go to bible for REST design philosophy) and didn't find a specific discussion on this topic. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From maxking at asynchronous.in Thu Dec 28 19:50:55 2017 From: maxking at asynchronous.in (Abhilash Raj) Date: Thu, 28 Dec 2017 16:50:55 -0800 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: <98422230-8A68-413E-9489-AC60F76E9B70@list.org> References: <1514364566.2747.14.camel@asynchronous.in> <98422230-8A68-413E-9489-AC60F76E9B70@list.org> Message-ID: <1514508655.370.36.camel@asynchronous.in> On Thu, 2017-12-28 at 15:25 -0500, Barry Warsaw wrote: > Abhilash Raj wrote: > > > Core's REST API is versioned and any change that break backwards- > > compatibility > > cause the version to bump so that clients can take care of that. > > The 3.1 version bump happened because there was no backward compatible > way to handle the UUID as int vs hex string change required for some > JavaScript libraries. It couldn't be autodetected in all cases. In > general though, we've mostly be able to manage changes in a backward > compatible way. > > > However, one question that I have been thinking about recently is how to > > handle > > additions to REST API that don't necessarily break the backwards > > compatibility. > > This problem is analogous to adding arguments to functions. This is > usually handled in backward compatible ways in Python by appending new > arguments to the end of the parameter list and giving them default > values. Or to use keyword-only arguments. The analogy breaks down in > one specific case though. > > > For example, Core added `max_message_size` attribute to MailingList's REST > > endpoint, but it hasn't made into any released version yet. Also, Postorius > > added max_message_size in `Message Acceptance` settings. The problem here is > > that the entire PUT/PATCH request is going to fail if the currently running > > version of Core doesn't have `max_message_size` attribute exposed (Unknown > > Attribute Error). > > PATCH won't fail because it allows for partial representations. PUT > does fail because it requires the entire new representation to be > included in the request (it's a complete replacement). This is where > the analogy to function arguments break down. > > I don't really know of any good way to handle this that still conforms > to REST principles. I don't think we want to rev the API in these cases > since that'll result in a lot of version churn. I agree that we don't want to rev the API for additions. PATCH won't fail when running partial updates, but it won't silently drop the attributes that it doesn't support. So from client side, there is no real way to understand when to include that attribute which was added in a later version. > > > There is no easy way to check for whether the Core has this attribute as API > > is > > versioned at 3.1 for both cases. > > > > So, how do we actually handle this and maybe future cases like this? > > Simon suggests: > > * The result of queries can be viewed as dictionaries > * New endpoints (urls) can be added anytime > * No endpoint is removed without a version bump > * Existing dict keys will not be dropped without a version bump > * The format of values assigned to existing keys will not change without > a version bump > * New keys (and values) can be added anytime > > This is pretty much the criteria I've used in the past, and it works > well enough in practice except for the PUT exception. A couple of > thoughts on how to handle this include, using PATCH in preference to > PUT, using PUT but catch any exception then fall back to PATCH, do a GET > first to get the list of keys. None of those are great options, > although some caching might help. As I mentioned using PATCH doesn't really solve the problem as we don't know when we can update that particular attribute, however, it does help update other attributes in the same resource. Exception caching might work, but then it would probably involve parsing the error message to find out which field is the problem before making a PATCH request without that attribute. Doing a GET first sounds expensive to me, we would definitely need some caching to be able to use this idea. We need some way to associate attributes with a minimum Core version, which we can get from `/system` endpoint. Although, for now, it probably is an overkill. I will do some static stuff to take care of this. > > Mailman's REST API is very dynamic so we don't even have a static > representation of it that can be queried. I did a quick scan of RESTful > Web APIs (Richardson & Amundsen - my go to bible for REST design > philosophy) and didn't find a specific discussion on this topic. How bad do you think would it be for Core to silently drop extra attributes and only use the ones that it needs? -- thanks, Abhilash Raj -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 866 bytes Desc: This is a digitally signed message part URL: From mark at msapiro.net Thu Dec 28 21:23:52 2017 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 28 Dec 2017 18:23:52 -0800 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: <1514508655.370.36.camel@asynchronous.in> References: <1514364566.2747.14.camel@asynchronous.in> <98422230-8A68-413E-9489-AC60F76E9B70@list.org> <1514508655.370.36.camel@asynchronous.in> Message-ID: <5a90dd78-5fde-0f62-395f-1bffba94c068@msapiro.net> On 12/28/2017 04:50 PM, Abhilash Raj wrote: > > How bad do you think would it be for Core to silently drop extra attributes and > only use the ones that it needs? The problem with that approach is it leads to "I set it in Postorius but it didn't take effect" kinds of issues. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Dec 28 22:08:38 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 29 Dec 2017 12:08:38 +0900 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: <1514364566.2747.14.camel@asynchronous.in> References: <1514364566.2747.14.camel@asynchronous.in> Message-ID: <23109.45494.694158.15688@turnbull.sk.tsukuba.ac.jp> Abhilash Raj writes: > There is no easy way to check for whether the Core has this > attribute as API is versioned at 3.1 for both cases. Barry has spoken (later, in a parallel thread) to the effect that this isn't a real problem "most" of the time. But to me it seems like the obvious ways are (1) to expose the whole API (as an HTTP collection attached at the version number -- Barry says this is hard, though), (2) to somehow ensure that client-side REST calls are transactional (I don't think this is possible), or (3) to provide the client a way to probe for an API that the server guarantees is a no-op, except for returning SUCCESS or NOTFOUND. Steve From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Dec 28 22:09:23 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 29 Dec 2017 12:09:23 +0900 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: <1514508655.370.36.camel@asynchronous.in> References: <1514364566.2747.14.camel@asynchronous.in> <98422230-8A68-413E-9489-AC60F76E9B70@list.org> <1514508655.370.36.camel@asynchronous.in> Message-ID: <23109.45539.498989.814420@turnbull.sk.tsukuba.ac.jp> Abhilash Raj writes: > How bad do you think would it be for Core to silently drop extra > attributes and only use the ones that it needs? I have no idea what the semantics of that would be. You'd need to come up with rules about semantics of adding attributes that guarantee this works as expected, but if you could, you probably wouldn't need the new attributes in the first place. I think it would be better for Core to fail with a reason of unknown attribute and allow the client to retry. But if the client isn't prepared to retry, there's no existing guarantee that the client can rollback the transaction. We'll just have to document that clients need to be prepared to rollback if they use attributes not documented for the version they require. Of course this is mitigated by the fact that in general clients wouldn't be keeping substantial state anyway. From simon at hannaweb.eu Fri Dec 29 06:01:57 2017 From: simon at hannaweb.eu (Simon Hanna) Date: Fri, 29 Dec 2017 12:01:57 +0100 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: <0798851c-40de-2906-2fc3-2d9f8b04eee5@msapiro.net> References: <1514364566.2747.14.camel@asynchronous.in> <0798851c-40de-2906-2fc3-2d9f8b04eee5@msapiro.net> Message-ID: <13e4efc2-c215-c9fa-5cb8-cdd55478751f@hannaweb.eu> On 12/28/2017 06:19 PM, Mark Sapiro wrote: > On 12/28/2017 08:33 AM, Simon Hanna wrote: > >> * New keys (and values) can be added anytime >> >> This would make things like exposing new stuff easy. Clients that don't >> know about a field can just ignore it. > > But that doesn't address the original issue which is that when a new > key/value is added and a client is updated to use it, the client has no > way to require a core API that provides it. > Up until now I was under the impression that the clients (Postorius/Hyperkitty) always require the latest version of core. Do we really want to add the burden of having to care for different api versions in the clients? If we would start from scratch something like http://www.coreapi.org/ might be feasible, adding that now would more or less require a complete rewrite of Postorius. In my view backward compatibiliy should be applied so that old clients can still access newer versions of core. I don't think it's a good idea to require Postorius to be compatible with older(all?) versions of core. From turnbull.stephen.fw at u.tsukuba.ac.jp Fri Dec 29 07:32:16 2017 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 29 Dec 2017 21:32:16 +0900 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: <13e4efc2-c215-c9fa-5cb8-cdd55478751f@hannaweb.eu> References: <1514364566.2747.14.camel@asynchronous.in> <0798851c-40de-2906-2fc3-2d9f8b04eee5@msapiro.net> <13e4efc2-c215-c9fa-5cb8-cdd55478751f@hannaweb.eu> Message-ID: <23110.13776.25176.105334@turnbull.sk.tsukuba.ac.jp> Simon Hanna writes: > Up until now I was under the impression that the clients > (Postorius/Hyperkitty) always require the latest version of core. We don't require that the client be Postorius or Hyperkitty though. While in one sense we're not responsible for what third party clients do *at all*, in another I don't think we should make it hard for them. It's also possible that a third party (hello, Systers!) might have modified versions of the code, and not realize that their "don't fix it 'cause it ain't broken" core isn't compatible with their up-to-the-minute Postorius that they pulled to get Abhilash's security patch. > Do we really want to add the burden of having to care for different api > versions in the clients? I don't think that's the question. The question is, "what can we do to make things easier for developers both in the Mailman project and out?" > In my view backward compatibiliy should be applied so that old clients > can still access newer versions of core. > I don't think it's a good idea to require Postorius to be compatible > with older(all?) versions of core. We don't. Only the current version. The problem is that "version" is a potentially large set. That *is* *our* fault (looking straight at Barry with one forearm on the emergency exit crashbar ;-). Steve From barry at list.org Fri Dec 29 10:10:35 2017 From: barry at list.org (Barry Warsaw) Date: Fri, 29 Dec 2017 10:10:35 -0500 Subject: [Mailman-Developers] Handling additions to REST API in client side In-Reply-To: <1514508655.370.36.camel@asynchronous.in> References: <1514364566.2747.14.camel@asynchronous.in> <98422230-8A68-413E-9489-AC60F76E9B70@list.org> <1514508655.370.36.camel@asynchronous.in> Message-ID: On Dec 28, 2017, at 19:50, Abhilash Raj wrote: > PATCH won't fail when running partial updates, but it won't silently drop the > attributes that it doesn't support. So from client side, there is no real way to > understand when to include that attribute which was added in a later version. Yes, I see the problem with that. > We need some way to associate attributes with a minimum Core version, which we > can get from `/system` endpoint. Although, for now, it probably is an overkill. > I will do some static stuff to take care of this. What happens when someone runs from say git head though? /system/version won?t keep up with that, and besides, it?s going to become an increasingly big compatibility matrix over time. > How bad do you think would it be for Core to silently drop extra attributes and > only use the ones that it needs? I think it could be problematic, as Steve points out. I don?t really have a good answer, and I?m wondering what the state of the art is. Maybe we should ask some experts for advice (we do have some friends who are experts in this). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: