From public at thesofa.de Sat Sep 6 20:07:57 2014 From: public at thesofa.de (Alexander Hachmann) Date: Sat, 06 Sep 2014 20:07:57 +0200 Subject: [Mailman-Developers] JAVA API to administer Mailman Message-ID: <540B4D7D.2040000@thesofa.de> Hello, I am happily using Mailman. So first of all a big thanks to all of you developers!! I am currently developing a community platform for a peergroup. There I would like to integrate mailman for list management. So my question is, whether there already is a liabrary written in Java that connects to Mailman? This way I would save development effort. The Version I am using is 2.1.4. Thanks and regards, Alex From stephen at xemacs.org Sun Sep 7 02:02:13 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 07 Sep 2014 09:02:13 +0900 Subject: [Mailman-Developers] JAVA API to administer Mailman In-Reply-To: <540B4D7D.2040000@thesofa.de> References: <540B4D7D.2040000@thesofa.de> Message-ID: <87oaus70xm.fsf@uwakimon.sk.tsukuba.ac.jp> Alexander Hachmann writes: > So my question is, whether there already is a liabrary written in > Java that connects to Mailman? This way I would save development > effort. For Mailman 2, there probably is no specific library in any language, because Mailman 2 is a Web 1.0 application that just presents HTML to clients (presumably browsers). So any language that can deal with HTTP, parse HTML, and "scrape the screens" could be used. But it could be annoying if it needs to deal with multiple Mailman versions or multiple languages, as there is no provision for introspection, so you would need to prepare for each version or language separately. For Mailman 3, there is the REST API which can be used, and there are several Python applications that provide alternative APIs and UIs in development. It's possible that those can be run on the JVM using the Jython implementation of Python, but I don't know if Jython can be embedded in a Java application. (It is possible for Jython to call other Java libraries, though, so if you're lucky you might be able to translate your main program to Python and do most development in Java.) > The Version I am using is 2.1.4. Really? That is very old (close to a decade), has security and other flaws, and does not address recent problems with Internet mail (specifically "DMARC") at all. You would be well-advised to upgrade to 2.1.18-1, if you are not going to make the jump to Mailman 3 (which is still in beta). Steve From barry at list.org Sun Sep 7 19:44:47 2014 From: barry at list.org (Barry Warsaw) Date: Sun, 7 Sep 2014 13:44:47 -0400 Subject: [Mailman-Developers] Resetting the database for each test In-Reply-To: References: Message-ID: <20140907134447.35c36dd8@anarchist.wooz.org> I hope Abhilash doesn't mind my following up to the mailing list. This is useful information that should be searchable in the archives. Abhilash is working on a port of the core from Storm to SQLAlchemy as the Python ORM layer. As an aside, this port plus the one I have pending for replacing restish with falcon, would allow us to start the port of Mailman 3 to Python 3. On Sep 07, 2014, at 06:15 AM, Abhilash Raj wrote: >I am facing a problem where I get this error: > > mailman.interfaces.domain.BadDomainSpecificationError: Duplicate email >host: example.com > >while running the tests, now I suspect this probably because the database is >not being reset in between tests. This is exactly right. Here's how the test suite ensures a clean system for each test. Note that this includes resetting the database, but also taking care of things like clearing out the queue directories and other persistent state. The first place to look is in the test layers. These are a Zope-ism adopted by nose2 (the test runner that MM3 uses) that provides for fixture-like resource management. https://nose2.readthedocs.org/en/latest/plugins/layers.html MM3's layers are defined in src/mailman/testing/layers.py. The most common layer is ConfigLayer which ensures that the configuration subsystem is initialized for the tests. ConfigLayer's testTearDown() is run at the end of every test that uses the ConfigLayer or any of its subclass layers. This method is pretty simple; it just calls reset_the_world() which is where all the good stuff happens. See src/mailman/testing/helpers.py. Most of that method should be pretty obvious; the call to config.db._reset() is where the database gets cleared out. _reset() is actually defined in src/mailman/database/factory.,py and placed onto the database object in DatabaseTestingFactory.create(). So how does _reset() work? First, it rolls back the database just to throw away any in progress transactions. It then calls back into the specific database implementations to give them a chance to do some pre-rest actions. It'll do something similar after the next step - it'll make a post-reset callback and then commit any outstanding transactions. The real heart of the method is in the call to ModelMeta._reset()[1]. ModelMeta is the metaclass for all ORM classes, via the Model base class used everywhere. The idea here is that each ORM class registers itself in ModelMeta._class_registry(), as long as the ORM class doesn't have a PRESERVE=False flag, indicating that particular table should *not* be reset after every test (currently only the Version table has this setting). Now you can see what ModelMeta._reset() does. It iterates through every registered ORM class, and removes every row in the associated database. I do it this way instead of dropping every table because it's actually more difficult to re-establish the schema than it is to remove the rows. So the trick for the SQLAlchemy port is to figure out both how to identify which tables to clear out, and how to drop all of those table's rows. Hope that makes sense! Cheers, -Barry [1] I just noticed that ModelMeta._reset() calls _pre_reset() again. That's probably a lurking bug that should be fixed. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From barry at list.org Sun Sep 7 19:48:19 2014 From: barry at list.org (Barry Warsaw) Date: Sun, 7 Sep 2014 13:48:19 -0400 Subject: [Mailman-Developers] JAVA API to administer Mailman In-Reply-To: <87oaus70xm.fsf@uwakimon.sk.tsukuba.ac.jp> References: <540B4D7D.2040000@thesofa.de> <87oaus70xm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20140907134819.757818c7@anarchist.wooz.org> On Sep 07, 2014, at 09:02 AM, Stephen J. Turnbull wrote: >For Mailman 3, there is the REST API which can be used, and there are >several Python applications that provide alternative APIs and UIs in >development. It's possible that those can be run on the JVM using the >Jython implementation of Python, but I don't know if Jython can be >embedded in a Java application. (It is possible for Jython to call >other Java libraries, though, so if you're lucky you might be able to >translate your main program to Python and do most development in >Java.) Of course, the whole point to exposing a REST API is that bindings can be easily written in any language, since it's just HTTP and JSON. One caveat though is that the core's REST API is an unauthenticated "administrative" API that should only ever be exposed on localhost. The idea is that Postorius or other equivalent would proxy the admin API through an authentication layer. Cheers, -Barry From nicolas at karageuzian.com Wed Sep 10 22:51:55 2014 From: nicolas at karageuzian.com (Nicolas Karageuzian) Date: Wed, 10 Sep 2014 22:51:55 +0200 Subject: [Mailman-Developers] PyCharm License for mailman team Message-ID: <5410B9EB.7050606@karageuzian.com> Hi all, I've applied for the pyCharm opensource licence and recieved it with the pleasant surprise that it can be shared away with whole contributor's team. As Jetbrains kindly ask, this key should not be published over unsecure channels (public mailing list or forum), but I can send it on email request. Please feel free to ask, prefer using direct reply as i'll track the thread. Regards -- Nicolas Karageuzian From barry at list.org Thu Sep 11 00:11:41 2014 From: barry at list.org (Barry Warsaw) Date: Wed, 10 Sep 2014 18:11:41 -0400 Subject: [Mailman-Developers] PyCharm License for mailman team In-Reply-To: <5410B9EB.7050606@karageuzian.com> References: <5410B9EB.7050606@karageuzian.com> Message-ID: <20140910181141.564892b2@anarchist.wooz.org> On Sep 10, 2014, at 10:51 PM, Nicolas Karageuzian wrote: >I've applied for the pyCharm opensource licence and recieved it with the >pleasant surprise that it can be shared away with whole contributor's team. > >As Jetbrains kindly ask, this key should not be published over unsecure >channels (public mailing list or forum), but I can send it on email request. > >Please feel free to ask, prefer using direct reply as i'll track the thread. Thanks Nicolas! -Barry From cp at ccil.org Tue Sep 16 02:08:12 2014 From: cp at ccil.org (Chuck Peters) Date: Mon, 15 Sep 2014 20:08:12 -0400 Subject: [Mailman-Developers] Deb packages for Mailman 3 suite, beta 1 preview Message-ID: <20140916000812.GA14009@ccil.org> Has anyone been working on Debian/Ubuntu packages? Debian Jessie/testing will be frozen Nov 5th and the package should be in unstable for at least two weeks (Oct 24th). It's a bit late to make it into the upcoming Ubuntu release, but it would be nice have packages available in a Launchpad PPA (Personal Package Archive). 1. https://release.debian.org/jessie/freeze_policy.html 2. https://wiki.ubuntu.com/UtopicUnicorn/ReleaseSchedule Thanks, Chuck From stephen at xemacs.org Tue Sep 16 07:30:04 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 16 Sep 2014 14:30:04 +0900 Subject: [Mailman-Developers] Deb packages for Mailman 3 suite, beta 1 preview In-Reply-To: <20140916000812.GA14009@ccil.org> References: <20140916000812.GA14009@ccil.org> Message-ID: <87mwa02kv7.fsf@uwakimon.sk.tsukuba.ac.jp> Chuck Peters writes: > Has anyone been working on Debian/Ubuntu packages? -0 Sounds like a serious sinkhole for developer time to me. Every time I've had a beta package (not Mailman) taken up on one of the distributions I've had to deal with way too many n00b questions and "when is this feature going to be done?" or "why isn't this feature from stable in the beta?" to make the additional testing worthwhile. From pabs3 at bonedaddy.net Tue Sep 16 09:20:48 2014 From: pabs3 at bonedaddy.net (Paul Wise) Date: Tue, 16 Sep 2014 15:20:48 +0800 Subject: [Mailman-Developers] Deb packages for Mailman 3 suite, beta 1 preview In-Reply-To: <20140916000812.GA14009@ccil.org> References: <20140916000812.GA14009@ccil.org> Message-ID: On Tue, Sep 16, 2014 at 8:08 AM, Chuck Peters wrote: > Has anyone been working on Debian/Ubuntu packages? >From our discussions at DebConf14 it sounds like Barry Warsaw is working on this but I don't think it will be ready in time. I also think it is far too early to package mailman3 in a stable distro release at this time. -- bye, pabs http://bonedaddy.net/pabs3/ From barry at list.org Tue Sep 16 17:29:35 2014 From: barry at list.org (Barry Warsaw) Date: Tue, 16 Sep 2014 11:29:35 -0400 Subject: [Mailman-Developers] Deb packages for Mailman 3 suite, beta 1 preview In-Reply-To: References: <20140916000812.GA14009@ccil.org> Message-ID: <20140916112935.2968de9d@anarchist.wooz.org> On Sep 16, 2014, at 03:20 PM, Paul Wise wrote: >From our discussions at DebConf14 it sounds like Barry Warsaw is >working on this but I don't think it will be ready in time. I also >think it is far too early to package mailman3 in a stable distro >release at this time. I wouldn't say "working on this" but maybe "want to work on this". ;) However I agree that it's too early to put MM3 in any stable distribution channel. I think the core would be relatively easy to package up, though I'm much less experienced in correctly packaging Django apps (Postorius and Hyperkitty). There was a web-app session at Debconf14 so I think it might even be better to wait and see what the outcome of that is. For Debian, it could go to experimental which would be a clear sign that it's available to play with, but not deemed stable. For Ubuntu, it would probably be best to stick it in a PPA. My RPM experience is way too rusty to admit having it. Cheers, -Barry From saurabh.singhal.apc11 at itbhu.ac.in Tue Sep 16 20:59:17 2014 From: saurabh.singhal.apc11 at itbhu.ac.in (Saurabh Singhal) Date: Tue, 16 Sep 2014 18:59:17 +0000 Subject: [Mailman-Developers] Looking for Contribution Message-ID: Hi Sir, For the past few days I've been looking mailman project and am very much willing to participate in the integration of mailman3 and hyperkitty and the development of both.I have an experience in SMTP servers also as trying to develop my own SMTP server using postfix and also a basic knowledge about email verification mechanisms.I have also worked on django before for few of my projects. It would be a pleasure to work with the mailman team and if possible please let me know from where I should start. Regards, Saurabh Singhal. From stephen at xemacs.org Wed Sep 17 02:33:51 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 17 Sep 2014 09:33:51 +0900 Subject: [Mailman-Developers] Looking for Contribution In-Reply-To: References: Message-ID: <878ulj2ihc.fsf@uwakimon.sk.tsukuba.ac.jp> Saurabh Singhal writes: > It would be a pleasure to work with the mailman team and if > possible please let me know from where I should start. Start here: http://wiki.list.org/display/DEV/Home and here for recent lists of projects we considered doable in two months by a single, often new to Mailman, developer: http://wiki.list.org/display/DEV/Google+Summer+of+Code+2014 http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013 Before starting work on any of the GSoC projects, check if work has already been done on them. Ask here, or if you really prefer to do it yourself, you can browse the related branches at https://code.launchpad.net/mailman/ Note that the wiki is changing platforms in the *very* near future. There will be an announcement here, I'm sure, but due to this your wiki credentials may suddenly stop working. The problem is simply that it's not possible to move the password database; any profile you create on the existing wiki will be moved to the new one automatically. You just have to reregister a password. HTH Steve From barry at list.org Tue Sep 23 02:41:49 2014 From: barry at list.org (Barry Warsaw) Date: Mon, 22 Sep 2014 20:41:49 -0400 Subject: [Mailman-Developers] Experimental SQLAlchemy port available Message-ID: <20140922204149.7b428cd9@anarchist.wooz.org> Hi Mailman Hackers! I wanted to let you know about some very cool work that Abhilash Raj has been doing, and to request your help to review and test it. You might know that for a long time, Mailman 3's ORM (Object Relational Mapper) layer was provided by Storm[1]. There were a lot of things to like about Storm, especially because it was such a nice, thin layer over SQL so it was very transparent to use and easy to debug. It had some downsides though, and as the project lost momentum, those downsides became more apparent. These included lack of any Python 3 support, and bugs in the PostgreSQL layer. As I eventually want MM3 to be a Python 3 application, and it looked like Storm would never be ported, I have for a while desperately wanted to port MM3 to SQLAlchemy. At one point early in the MM3 fork, it *was* using SA, but I had some problems with that (details are lost, and unimportant) and switched to Storm. Now, I think it's clear that SA is the best ORM available for Python, and it is Python 3 compatible[2]. Abhilash has done a fantastic port of the MM3 trunk to SA. I have taken his branch, fixed a few test failures, and cleaned up a few things. But really, all the credit goes to Abhilash for his great work. I have not yet merged the SA branch into trunk, for a few reasons. Abhilash is investigating Alembic as the schema migration layer, and I have not had time to test the branch against PostgreSQL. I would also like to invite folks who have deployed MM3 to give the branch some testing, especially to make sure that the new SA-derived schema is compatible with the hand-written schema used by Storm. Adopting SA will mean a break in upgrade guarantees. We won't provide migrations from 3.0b4's schema to 3.0b5, which is why I want them to be effectively compatible. After 3.0b5 is released, we'll likely use Alembic to manage any future schema changes. While I'd rather not make this break, I'm convinced that the SA community can handle all this much better than our crufty pile of schema migration code. If you are able, please test this branch: $ bzr branch lp:~barry/mailman/abhilash That's essentially the 3.0 trunk with Abhilash's branch merged in, and my test fixes and clean ups on top of that. I'd like to merge this to trunk, definitely for 3.0b5, so let's say in the next week or so. Let us know how it goes! Cheers, -Barry [1] https://storm.canonical.com/ [2] We are very likely going to adopt Falcon as a replacement for restish. The latter suffers the same problems as Storm; it's not Python 3 compatible, and it is effectively unmaintained upstream. Falcon is really great, and with the following changes, it will easily support restish-style object-based traversal. Storm and restish are the last Python 2-only dependencies of MM3 core. https://github.com/racker/falcon/pull/307 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From terri at toybox.ca Wed Sep 24 05:58:35 2014 From: terri at toybox.ca (Terri Oda) Date: Tue, 23 Sep 2014 20:58:35 -0700 Subject: [Mailman-Developers] Experimental SQLAlchemy port available In-Reply-To: <20140922204149.7b428cd9@anarchist.wooz.org> References: <20140922204149.7b428cd9@anarchist.wooz.org> Message-ID: <5422416B.2040209@toybox.ca> On 2014-09-22, 5:41 PM, Barry Warsaw wrote: > Abhilash has done a fantastic port of the MM3 trunk to SA. I have taken his > branch, fixed a few test failures, and cleaned up a few things. But really, > all the credit goes to Abhilash for his great work. Hurrah Abhilash! Glad to see one more barrier fall on the road to python3. :) I'm probably not going to get a chance to test this myself this week (I'm buried in prep for a work meeting in europe next week) but I'm really excited to hear that SA is working! Terri