From dkg at fifthhorseman.net Mon Jul 1 00:01:21 2013 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Sun, 30 Jun 2013 18:01:21 -0400 Subject: [Mailman-Developers] A list of discussion topics: GSoC OpenPGP Integration In-Reply-To: <87haghy10p.fsf@uwakimon.sk.tsukuba.ac.jp> References: <878v2cc4mw.fsf@phoenix.mshall.iitkgp.ernet.in> <87ppvn5nh9.fsf@uwakimon.sk.tsukuba.ac.jp> <87li5ve84j.fsf@alice.fifthhorseman.net> <87ppv6yj8h.fsf@uwakimon.sk.tsukuba.ac.jp> <51CDAFE0.10700@fifthhorseman.net> <87haghy10p.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <51D0AAB1.7000405@fifthhorseman.net> On 06/29/2013 12:49 AM, Stephen J. Turnbull wrote: > Daniel Kahn Gillmor writes: > > OpenPGP certifications should attest to people's identities; those > > identities should have permissions in mailman the same way that > > non-cryptographically-verifiable identities have permissions in > > mailman. > > > > The semantics are simple and graspable if we say "for list X, rely on > > OpenPGP certifications from the following keys to prove cryptographic > > identity". > > So you're suggesting that the *only* policies that should be > automatically implementable via certified key are > > (0) let this guy upload this key (and implicitly create a User if > needed), but he can't do *anything* else (not even subscribe) > until the list owner explicitly adds authorizations, > > (0.5) this guy gets the intersection of sets of privileges I ever want > to grant to anybody, and > > (1) this guy is co-owner of this list Maybe we're not talking about the same thing. OpenPGP certification should be identity certification, and nothing else. trying to extend OpenPGP certification to mean something other than identity certification sounds like a bad idea to me -- it breaks all kinds of other assumptions within the OpenPGP world. I was thinking that the baseline is: 0) each e-mail list has a set of "identity certifiers" 1) each "identity certifier" is itself an OpenPGP primary key fingerprint (or, the primary key itself). 2) subscribers to an OpenPGP-enabled mailman mailing list subscribe, unsubscribe, receive, and send mails as usual (though messages not signed with valid keys will not be re-sent to the list). 3) if a signed message comes in, the server checks to make sure that the message is signed properly with a key that is certified (by one of the list's "identity certifiers") to belong to the person in the message's "From:" header, *and* that person is a known subscriber to the list. getting fancier, subscription and unsubscription messages, preference-changing messages, etc, could need to be signed by a valid, certified key as well. so when you say "certified key" above, i think you're talking about what is known as a "valid" key -- that is, the relevant user ID is bound to its primary key by a certification made by one of my trusted identity certifiers. In this model, the only special policy that is conferred upon an OpenPGP keyholder by the list administrator is "is this key one of the list's trusted identity certifiers or not?" > > i can see how this would be useful, but it means that there is more > > fiddly tracking of the validity state of each (key,userid) pairing > > that needs to be done to make this possible. > > I agree it's fiddly, I agree it's not in scope for Abhilash's GSoC > project, but for Mark's[1] sake I think we need to notify users whose > status changes from valid to invalid of the reason for the change. I hear you :) > > I suspect that the most common change from valid to not-valid will > > be an expired key or an expired certification (e.g. if the list > > owner's certification of the key expires). for the latter case, i > > can imagine that the certifier (the list admin) might want to be > > notified as well. > > I would interpret a certification expiry differently: as the period of > time for which permission to register the key is valid. If we want an > expiry for User authentication, probably a generic tool for managing > that in Mailman itself would be sufficient for this purpose and useful > elsewhere. certification expiry means "i am willing to claim that this key belongs to this person for N months; if it's later than N months, and you don't see a newer certification from me, please don't rely on my claim any more". I think it would be a mistake to interpret that any other way, since that is the default interpretation of other pre-existing OpenPGP clients that will be seeing these same certifications. i hope this helps clarify my perspective -- i think i'm pushing for something simpler, not more complex; i think simplicity is one of the critical factors in making this stuff comprehensible to regular e-mail list administrators. Another (as you mention above) is clear, concise, and clean reporting about what is going on! --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1027 bytes Desc: OpenPGP digital signature URL: From stephen at xemacs.org Mon Jul 1 07:58:26 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 01 Jul 2013 14:58:26 +0900 Subject: [Mailman-Developers] A list of discussion topics: GSoC OpenPGP Integration In-Reply-To: <51D0AAB1.7000405@fifthhorseman.net> References: <878v2cc4mw.fsf@phoenix.mshall.iitkgp.ernet.in> <87ppvn5nh9.fsf@uwakimon.sk.tsukuba.ac.jp> <87li5ve84j.fsf@alice.fifthhorseman.net> <87ppv6yj8h.fsf@uwakimon.sk.tsukuba.ac.jp> <51CDAFE0.10700@fifthhorseman.net> <87haghy10p.fsf@uwakimon.sk.tsukuba.ac.jp> <51D0AAB1.7000405@fifthhorseman.net> Message-ID: <877ghayg71.fsf@uwakimon.sk.tsukuba.ac.jp> Daniel Kahn Gillmor writes: > Maybe we're not talking about the same thing. OpenPGP certification > should be identity certification, and nothing else. trying to extend > OpenPGP certification to mean something other than identity > certification sounds like a bad idea to me -- it breaks all kinds of > other assumptions within the OpenPGP world. > > I was thinking that the baseline is: > > 0) each e-mail list has a set of "identity certifiers" Yes. > 1) each "identity certifier" is itself an OpenPGP primary key > fingerprint (or, the primary key itself). Yes. Probably the primary key. > 2) subscribers to an OpenPGP-enabled mailman mailing list subscribe, > unsubscribe, receive, and send mails as usual (though messages not > signed with valid keys will not be re-sent to the list). Not necessarily. It may be necessary to sign admin messages as well (subscribe, unsubscribe, set vacation).... Your phrasing presumes that the only role of Users (the internal representation of an identity) is to act as a subscriber who reads and posts. The way I think of it is that Users may have several roles (read, post, moderate, admin) for each list. Each of these roles may be certified by a different "agent" of the owner, where agents are represented by different keys. > 3) if a signed message comes in, the server checks to make sure that > the message is signed properly with a key that is certified (by one of > the list's "identity certifiers") to belong to the person in the > message's "From:" header, *and* that person is a known subscriber to the > list. "Known subscriber" doesn't really make sense in the Mailman 3 world. There are Users, they can be "members" of a list (ie, known to the list) and they can have roles (reader, poster, etc). It's not clear to me that requiring posters to have the reader role in this world is the right way to determine membership. > so when you say "certified key" above, i think you're talking about what > is known as a "valid" key -- that is, the relevant user ID is bound to > its primary key by a certification made by one of my trusted identity > certifiers. That seems to be nonsense, though. Key-to-UID binding is done by the user database; it's only meaningful inside of Mailman. A signature on a key is not a way of making an authenticated link to a UID in Mailman, AFAICS. It's only a of certifying that the signer knows the keyholder and that this is that keyholder's key, and that the signer trusts the keyholder not to allow others to use her key. That doesn't mean (in the absence of actually binding a UID to the key *in* the certification) that this keyholder corresponds to any given UID. So this interpretation is useful only if a new UID is being assigned to that key in this transaction. It doesn't work if we are binding a new key to an existing UID. I also don't see why, in a web-of-trust model, you would want to use that definition of "valid". (If the key is determined to be none of corrupt, expired, or revoked, I would call that "valid".) A key not signed by a trusted certification key would be (completely) untrusted, but you can imagine various degrees of trust. For example, in a member recommendation model, you might allow any member's key to certify the reader role but not the poster (or vice-versa for a list handling privacy issues!) Having validity depend on trust makes the concept of validity quite ambiguous. > > I would interpret a certification expiry differently: as the period of > > time for which permission to register the key is valid. If we want an > > expiry for User authentication, probably a generic tool for managing > > that in Mailman itself would be sufficient for this purpose and useful > > elsewhere. > > certification expiry means "i am willing to claim that this key belongs > to this person for N months; if it's later than N months, and you don't > see a newer certification from me, please don't rely on my claim any > more". I think it would be a mistake to interpret that any other > way, I don't interpret it any other way. I'm assuming that once the key is registered, Mailman and the list owner take responsibility for trusting keys, and they no longer rely on the certification at all. If the list owner wants to use the "certification" in authenticating the User, that would be OK. But then the temptation to use an infinite expiry would be strong (unless a mechanism is provided to make re-signing convenient -- but not *too* convenient: sorry, no "re-sign all" button will be provided!). With infinite expiry certification is meaningless in the long run (in the long run we're all dead, and a signed message from someone known to be dead should be a clue...). > since that is the default interpretation of other pre-existing > OpenPGP clients that will be seeing these same certifications. Why are they delegating so much power to the certifications? That seems very strange to me. I certainly wouldn't stop trusting a key that had been used to sign messages to my list daily just because a third party certification from 6 months ago expired! From dkg at fifthhorseman.net Mon Jul 1 20:36:54 2013 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Mon, 01 Jul 2013 14:36:54 -0400 Subject: [Mailman-Developers] A list of discussion topics: GSoC OpenPGP Integration In-Reply-To: <877ghayg71.fsf@uwakimon.sk.tsukuba.ac.jp> References: <878v2cc4mw.fsf@phoenix.mshall.iitkgp.ernet.in> <87ppvn5nh9.fsf@uwakimon.sk.tsukuba.ac.jp> <87li5ve84j.fsf@alice.fifthhorseman.net> <87ppv6yj8h.fsf@uwakimon.sk.tsukuba.ac.jp> <51CDAFE0.10700@fifthhorseman.net> <87haghy10p.fsf@uwakimon.sk.tsukuba.ac.jp> <51D0AAB1.7000405@fifthhorseman.net> <877ghayg71.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <51D1CC46.6020507@fifthhorseman.net> On 07/01/2013 01:58 AM, Stephen J. Turnbull wrote: > > 2) subscribers to an OpenPGP-enabled mailman mailing list subscribe, > > unsubscribe, receive, and send mails as usual (though messages not > > signed with valid keys will not be re-sent to the list). > > Not necessarily. It may be necessary to sign admin messages as well > (subscribe, unsubscribe, set vacation).... > > Your phrasing presumes that the only role of Users (the internal > representation of an identity) is to act as a subscriber who reads and > posts. The way I think of it is that Users may have several roles > (read, post, moderate, admin) for each list. Each of these roles may > be certified by a different "agent" of the owner, where agents are > represented by different keys. It sounds to me like you're trying to use OpenPGP keys for authorization here, rather than for authentication. I think that's a mistake. OpenPGP's so-called "web of trust" is a network of identity certifications; it does not typically include authorization information. This is limited scope is actually a feature, for (at least) three reasons: 0) OpenPGP certifications are typically public -- you do not always want to publish your authorization decisions. 1) because OpenPGP certifications are statements about identity, you can perform some reasoned inference about them (e.g. following reasonable chains of authenticated certifications) that would be much harder to do if the semantics of any given certification could potentially indicate authorization as well as authentication. 2) because the certifications are just statements about identity, they are easier for most users to make; this is because it is simpler to consider identity on its own than to consider both identity and authorization bundled together. More certifications means a richer pool to draw from when trying to calculate authentication. > "Known subscriber" doesn't really make sense in the Mailman 3 world. > There are Users, they can be "members" of a list (ie, known to the > list) and they can have roles (reader, poster, etc). It's not clear > to me that requiring posters to have the reader role in this world is > the right way to determine membership. i'm just suggesting that those roles are authorization statements about users and lists. I apologize for not knowing the official mailman terminology here, so if you can help me translate, i would be very grateful :) I think those authorization statements should *stay the same* in mailman lists that use OpenPGP as they are in mailman lists that do not use OpenPGP. > > so when you say "certified key" above, i think you're talking about what > > is known as a "valid" key -- that is, the relevant user ID is bound to > > its primary key by a certification made by one of my trusted identity > > certifiers. > > That seems to be nonsense, though. Key-to-UID binding is done by the > user database; it's only meaningful inside of Mailman. The key-to-uid binding is done by the OpenPGP keyring. this is the entire point of an OpenPGP keyring: given a signed message and a from address, can you confirm that this message really came from the stated from address? > A signature on > a key is not a way of making an authenticated link to a UID in > Mailman, AFAICS. It's only a of certifying that the signer knows the > keyholder and that this is that keyholder's key, and that the signer > trusts the keyholder not to allow others to use her key. That doesn't > mean (in the absence of actually binding a UID to the key *in* the > certification) that this keyholder corresponds to any given UID. I don't think i understand what you're saying. when Alice makes an OpenPGP certification on Bob's key ("Alice signs Bob's key"), she is explicitly binding Bob's User ID to his public key. So saying "in the absence of actually binding a UID to the key in the certification" doesn't make sense to me. > I also don't see why, in a web-of-trust model, you would want to use > that definition of "valid". (If the key is determined to be none of > corrupt, expired, or revoked, I would call that "valid".) A key not > signed by a trusted certification key would be (completely) untrusted, > but you can imagine various degrees of trust. For example, in a > member recommendation model, you might allow any member's key to > certify the reader role but not the poster (or vice-versa for a list > handling privacy issues!) Having validity depend on trust makes the > concept of validity quite ambiguous. It's not surprising that these terms still confuse people; they were muddled even in the GPG documentation up until a few years ago. It's worth clarifying them explicitly: * A literal key (nothing but a key, no UIDs, no self-sigs, etc -- not an OpenPGP certificate) that is well-formed is "syntactically valid". * a syntactically-valid key whose certifications we are willing to rely on when evaluating other keys and user IDs is a "trusted" key. This is the same thing as saying that the key has "full ownertrust". * a (key,userID) pair that has been certified by a trusted key is "valid" -- that is, we believe that the person identified by the User ID is the person who holds the specified key. If the plan is to use GnuPG's OpenPGP keyring as a backend for this project, it's probably worth adopting the same terminology. > I don't interpret it any other way. I'm assuming that once the key is > registered, Mailman and the list owner take responsibility for > trusting keys, and they no longer rely on the certification at all. Why? Why not keep relying on the keyring for that? Who is going to be responsible for checking for expiration, revocation, etc? It seems like continuing to use the keyring specifically for this relationship is the simpler and cleaner. > If the list owner wants to use the "certification" in authenticating > the User, that would be OK. But then the temptation to use an > infinite expiry would be strong (unless a mechanism is provided to > make re-signing convenient -- but not *too* convenient: sorry, no > "re-sign all" button will be provided!). With infinite expiry > certification is meaningless in the long run (in the long run we're > all dead, and a signed message from someone known to be dead should be > a clue...). if they're known to be dead, they should be unsusbcribed from the list :P Most people currently make OpenPGP certifications that have no expiry -- because there is an assumption that people's identities and keys don't change very often. This is not a universal practice, but it's pretty widespread. > > since that is the default interpretation of other pre-existing > > OpenPGP clients that will be seeing these same certifications. > > Why are they delegating so much power to the certifications? That > seems very strange to me. I certainly wouldn't stop trusting a > key that had been used to sign messages to my list daily just because > a third party certification from 6 months ago expired! if that third party was the only trusted certifier for the list, the list absolutely should stop believing that the given key belongs to the given user when the certification expires. why should the list believe anything different? Anyway, i hope this makes my perspective on this clear: i think you should be relying on OpenPGP certifications for authentication, and you should rely on your list's keyring to understand key-to-userID mappings. And keep the authentication associations (roles that relate users to lists) as they currently are in mailman. I realize that it sounds like these suggestions are not in line with your current plans, so maybe i'm just causing noise; if you want me to step back from the discussion, please let me know -- i certainly don't want to get in the way of the project getting done. Regards, --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1027 bytes Desc: OpenPGP digital signature URL: From barry at list.org Mon Jul 1 23:44:36 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 1 Jul 2013 17:44:36 -0400 Subject: [Mailman-Developers] Mailman search function (was?? Re: Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python)) In-Reply-To: References: Message-ID: <20130701174436.11cd9e81@anarchist> On Jun 29, 2013, at 10:33 PM, Chris Cargile wrote: >FWIW, I'd also like very much if anyone could provide a few lines recap of >the activity on the ModernArchiving topic under "initiatives and proposals" >as I feel my thoughts on searching/archiving tie into the >objectives/requirements listed there: >http://wiki.list.org/display/DEV/ModernArchiving > >More concisely, where I have search capability, my goal of integrating more >'robustly' is I think one that will rely upon a channel of communication >being established with the archiving mechanisms, if I understand the mm 2.0 >codebase well enough. Hi Chris, I really think searching is pretty closely tied to the archiver. While the Mailman core may keep a central message store, it won't be directly tied to archiving or searching. We've tried to keep the connection between Mailman and the archivers loose, certainly much looser than between MM2.1 and Pipermail. There are lots of reasons for that, including the ability to define multiple archivers for a single Mailman instance, and the ability to define remote archivers (e.g. mail-archive.com). This is the primary motivation for the stable URL proposal . As for our eventually hoped-for replacement to Pipermail, Hyperkitty, I think it would be wonderful for it to have a built-in search. -Barry From p at sys4.de Tue Jul 2 00:44:15 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Tue, 2 Jul 2013 00:44:15 +0200 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 Message-ID: <20130701224414.GC1767@sys4.de> Greetings, I am writing on behalf of a group of companies and single persons, who would like to see a limited feature set of the DMARC? standard supported by Mailman 3. Since I know we're all eager to get MM3 out as soon as possible and any additional new feature request stands against that I've contacted Barry offlist and asked if he'd agree that the companies involved pay us, sys4?, to implement the feature. He did and we also agreed to dedicate a significant part of the payment to mailman's FSF donation account. Before we take out to write code, I would like to ask mailman-developers how it should be done to fit best into Mailman's architecture. Here are the DMARC features that should go into Mailman 3: - don't allow email that comes from a domain with a DMRAC record of p=reject - take ownership of the email and send it with a From: using the domain of the mailing list. (There's a patch for this for Mailman 2.1, which might might be helpful for Mailman 3.) - find the authentication-results header and rewrite it as an Original-Authentication-header: http://tools.ietf.org/html/draft-kucherawy-original-authres-00.html Speaking of an RFC written by Murray Kucherawy. I've contacted Murray in advance and asked him to assist in case we had any questions regarding his RFC(s). He subscribed and ready to help. I hope I was able to bring all parties required together to make a Mailman DMARC implementation come true and I am curious to hear what you have to say. p at rick ?) DMARC adds policies and more to DKIM, which gives digitally signed identity to sender domains and has become a cornerstone to reputation based mail management. Having DMARC support in Mailman 3 is - in my eyes - a reason for postmasters (not end-users) to upgrade their MM 2.x installation as soon as possible. ?) In case you ask yourself, who "sys4" is: We've accompanied Mailman 3 development since Pycon in Chicago (in 2008 ?). Florian Fuchs, whom I don't have to introduce, works with us as well Ralf Hildebrandt and I. We, Ralf and I, are on the python.org postmaster team. Also with sys4 are Antoine Nguyen and Christian Roesser - both are exerienced Python programmers and both have spent significant time developing email applications. -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From stephen at xemacs.org Tue Jul 2 06:04:09 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 02 Jul 2013 13:04:09 +0900 Subject: [Mailman-Developers] A list of discussion topics: GSoC OpenPGP Integration In-Reply-To: <51D1CC46.6020507@fifthhorseman.net> References: <878v2cc4mw.fsf@phoenix.mshall.iitkgp.ernet.in> <87ppvn5nh9.fsf@uwakimon.sk.tsukuba.ac.jp> <87li5ve84j.fsf@alice.fifthhorseman.net> <87ppv6yj8h.fsf@uwakimon.sk.tsukuba.ac.jp> <51CDAFE0.10700@fifthhorseman.net> <87haghy10p.fsf@uwakimon.sk.tsukuba.ac.jp> <51D0AAB1.7000405@fifthhorseman.net> <877ghayg71.fsf@uwakimon.sk.tsukuba.ac.jp> <51D1CC46.6020507@fifthhorseman.net> Message-ID: <87y59pwqti.fsf@uwakimon.sk.tsukuba.ac.jp> Daniel Kahn Gillmor writes: > On 07/01/2013 01:58 AM, Stephen J. Turnbull wrote: > > The way I think of it is that Users may have several roles (read, > > post, moderate, admin) for each list. Each of these roles may be > > certified by a different "agent" of the owner, where agents are > > represented by different keys. > > It sounds to me like you're trying to use OpenPGP keys for > authorization here, rather than for authentication. I think that's > a mistake. A certification is a statement that "*I* know *this* key to be associated with *that* person, and only that person". Depending on the value of "I", "you" may extend different levels of trust to "that person". You're already down the rabbit hole, my dear Alice. The question is whether to eat the red mushroom. ;-) > OpenPGP's so-called "web of trust" is a network of identity > certifications; it does not typically include authorization > information. There's no authorization information in the web of trust. The authorization decision is made on the basis of being presented a signed key; that's all Mailman has to go on. Depending on the identity of the signer *Mailman* may grant additional authorizations. Consider the example I gave of subscription by member recommendation. > This is limited scope is actually a feature, for (at least) three > reasons: > > 0) OpenPGP certifications are typically public If you don't want to expose different "roles" as OpenPGP identities, don't. Nevertheless, the capability is available in OpenPGP: the identity-to-key mapping is not one-to-one. > 1) because OpenPGP certifications are statements about identity, you > can perform some reasoned inference about them The whole point of the suggestion is based on this fact. There's no authorization in the signatures. Mailman makes decisions about the authorization to grant based on the identity(s) of the signer(s). > 2) because the certifications are just statements about identity, they > are easier for most users to make; Sure, but some users are smarter than the average bear, or have requirements beyond those of a typical panda. > > "Known subscriber" doesn't really make sense in the Mailman 3 world. > > There are Users, they can be "members" of a list (ie, known to the > > list) and they can have roles (reader, poster, etc). It's not clear > > to me that requiring posters to have the reader role in this world is > > the right way to determine membership. > > i'm just suggesting that those roles are authorization statements about > users and lists. They are. We're just confused about "user". I mean Mailman User, you mean PGP userID. They're not equivalent, at least not at the stage of adding a key to a User in Mailman. > > > so when you say "certified key" above, i think you're talking about what > > > is known as a "valid" key -- that is, the relevant user ID is bound to > > > its primary key by a certification made by one of my trusted identity > > > certifiers. > > > > That seems to be nonsense, though. Key-to-UID binding is done by the > > user database; it's only meaningful inside of Mailman. > > The key-to-uid binding is done by the OpenPGP keyring. this is the > entire point of an OpenPGP keyring: given a signed message and a from > address, can you confirm that this message really came from the stated > from address? No, in Mailman 3 it is not, and cannot be, internal to OpenPGP because addresses are *not* Users. There is a many-to-one (address-to-User) mapping (I hope; if it's many-to-many, we can probably dodge that bullet by allowing sets of Users in many places we'd normally expect a User). However, binding an email address to a User is a Mailman operation, and at the point of adding an email to a User, in the scenario I'm thinking of the only thing Mailman has to go on is the association of a key to an email. If this is the initial email for that User, there's no problem. But for additional emails, there *is* a problem. The identification of existing emails with the email to be added is not necessarily guaranteed by the key presented. We need to think carefully about how this works (or can be subverted). > I don't think i understand what you're saying. when Alice makes an > OpenPGP certification on Bob's key ("Alice signs Bob's key"), she is > explicitly binding Bob's User ID to his public key. So saying "in the > absence of actually binding a UID to the key in the certification" > doesn't make sense to me. Mailman UID != GPG UID, at least not at present, and probably not for the foreseeable future. > It's not surprising that these terms still confuse people; they were > muddled even in the GPG documentation up until a few years ago. It's > worth clarifying them explicitly: > > * A literal key (nothing but a key, no UIDs, no self-sigs, etc -- not > an OpenPGP certificate) that is well-formed is "syntactically valid". > > * a syntactically-valid key whose certifications we are willing to rely > on when evaluating other keys and user IDs is a "trusted" key. This is > the same thing as saying that the key has "full ownertrust". > > * a (key,userID) pair that has been certified by a trusted key is > "valid" -- that is, we believe that the person identified by the User ID > is the person who holds the specified key. > > If the plan is to use GnuPG's OpenPGP keyring as a backend for this > project, it's probably worth adopting the same terminology. Not so fast. The official position of the GNU Project is that standards are optional in GNU software. We should adopt the standard terminology as used in the OpenPGP RFCs and any other non-vendor- specific documentation. If GPG pretty reliably conforms, we can use the GPG documentation as a reference since it's free and easily available. > > I don't interpret it any other way. I'm assuming that once the key is > > registered, Mailman and the list owner take responsibility for > > trusting keys, and they no longer rely on the certification at all. > > Why? Why not keep relying on the keyring for that? Who said anything about not relying on the keyring? I'm saying that Mailman should not rely on the *certification* that supported adding this key, once the key is added. Unless it wants to. > > With infinite expiry certification is meaningless in the long run > > (in the long run we're all dead, and a signed message from > > someone known to be dead should be a clue...). > > if they're known to be dead, they should be unsusbcribed from the list :P In security applications, we don't rely on "shoulds," now do we? > Most people currently make OpenPGP certifications that have no expiry -- > because there is an assumption that people's identities and keys don't > change very often. This is not a universal practice, but it's pretty > widespread. Sure, but we're specifically discussing the case where the certification *does* expire. > if that third party was the only trusted certifier for the list, the > list absolutely should stop believing that the given key belongs to the > given user when the certification expires. why should the list believe > anything different? Because it's the truth that the given key still does belong to the given user? And this truth is known to all subscribers of the list because each has separately authenticated that user in their personal keyrings, without expiry? > Anyway, i hope this makes my perspective on this clear: i think you > should be relying on OpenPGP certifications for authentication, and you > should rely on your list's keyring to understand key-to-userID mappings. > And keep the authentication associations (roles that relate users to > lists) as they currently are in mailman. All three are fine by me, but they're not enough by themselves. One problem is associating OpenPGP userIDs with Mailman Users. They are not and cannot be the same. Another problem is associating authorizations with Users in the complex trust environment of OpenPGP. > I realize that it sounds like these suggestions are not in line with > your current plans, I don't have any plans. I'm trying to figure out what the requirements are here. The model of OpenPGP as a network of individuals, each represented by a single userID-email pair, is not a priori appropriate for mailing lists, and especially not for Mailman 3 which has many features designed for handling one User's memberships in multiple lists possibly using various email addresses across different domains typically representing different organizations. > so maybe i'm just causing noise; No, not at all. I'm a professional in the field of drawing inferences about beliefs, but I don't know very much about OpenPGP specifically. Obviously we need to have Mailman semantics and terminology correspond to OpenPGP semantics and terminology where they are supposed to be equivalent. From barry at list.org Tue Jul 2 17:47:49 2013 From: barry at list.org (Barry Warsaw) Date: Tue, 2 Jul 2013 11:47:49 -0400 Subject: [Mailman-Developers] A list of discussion topics: GSoC OpenPGP Integration In-Reply-To: <87y59pwqti.fsf@uwakimon.sk.tsukuba.ac.jp> References: <878v2cc4mw.fsf@phoenix.mshall.iitkgp.ernet.in> <87ppvn5nh9.fsf@uwakimon.sk.tsukuba.ac.jp> <87li5ve84j.fsf@alice.fifthhorseman.net> <87ppv6yj8h.fsf@uwakimon.sk.tsukuba.ac.jp> <51CDAFE0.10700@fifthhorseman.net> <87haghy10p.fsf@uwakimon.sk.tsukuba.ac.jp> <51D0AAB1.7000405@fifthhorseman.net> <877ghayg71.fsf@uwakimon.sk.tsukuba.ac.jp> <51D1CC46.6020507@fifthhorseman.net> <87y59pwqti.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130702114749.230ce52b@anarchist> On Jul 02, 2013, at 01:04 PM, Stephen J. Turnbull wrote: >No, in Mailman 3 it is not, and cannot be, internal to OpenPGP because >addresses are *not* Users. There is a many-to-one (address-to-User) >mapping (I hope; if it's many-to-many, we can probably dodge that >bullet by allowing sets of Users in many places we'd normally expect a >User). You're correct, although technically addresses don't have to be associated with users. But if they are, they can only be associated with a single user. Users can control multiple addresses. While unlinked addresses are supported by the model, I don't think there's any case where unlinked addresses are really exposed in any meaningful way that a user or admin can utilize. So I think if we can associate OpenPGP user ids to email addresses, that will almost always imply an association to a user. >However, binding an email address to a User is a Mailman operation, and at >the point of adding an email to a User, in the scenario I'm thinking of the >only thing Mailman has to go on is the association of a key to an email. If >this is the initial email for that User, there's no problem. > >But for additional emails, there *is* a problem. The identification of >existing emails with the email to be added is not necessarily guaranteed by >the key presented. We need to think carefully about how this works (or can >be subverted). Very definitely. While it's easy to associate an address with an existing user, it's not entirely clear how we can do that in a secure way. -Barry From truongvunghia at gmail.com Fri Jul 5 23:31:43 2013 From: truongvunghia at gmail.com (VuNghia Truong) Date: Fri, 5 Jul 2013 22:31:43 +0100 Subject: [Mailman-Developers] Message Handling Workflow Message-ID: Hi everyone, Could any one please help me to answers these questions: - Is the handling workflow b.t version 2 and 3 the same or different? - How the file system are organized, how these pieces of codes fit together? Thanks! Nghia Truong From stephen at xemacs.org Sat Jul 6 11:50:13 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 06 Jul 2013 18:50:13 +0900 Subject: [Mailman-Developers] Message Handling Workflow In-Reply-To: References: Message-ID: <87wqp4uiei.fsf@uwakimon.sk.tsukuba.ac.jp> VuNghia Truong writes: > Could any one please help me to answers these questions: > - Is the handling workflow b.t version 2 and 3 the same or different? It's different. V2 has a single pipeline that both checks the mail for conformance to list regulations and also cleans up the post (removing forbidden content types) and adds list-specific content (mail headers, leader and trailer parts). V3 has two separate pipelines, one for approving the post (using "chains" of "rules") and one for preparting for distribution (using a "pipeline" of "handlers"). Be carefule because the terminology resembles that used in V2 but the architecture is definitely different. > - How the file system are organized, how these pieces of codes fit > together? The /var hierarchy for Mailman 3 is quite similar to that of Mailman 2. I'm not sure what the installed code looks like; we currently recommend running in a virtualenv, basically everything in the source tree. The source hierarchy is quite different. There are altogether 4 separate source trees: - Mailman 3 core: the post distribution and databases - mailman.client: helper library for accessing database and configuration using the REST (HTTP) interface - Postorius: the web admin interface (including user subscription options), which depends on mailman.client - HyperKitty: the archive management and web access interface. Note that Postorius and HyperKitty are designed to function as independent modules, and are basically independent projects. From franck at peachymango.org Sun Jul 7 04:00:15 2013 From: franck at peachymango.org (Franck Martin) Date: Sat, 6 Jul 2013 21:00:15 -0500 (CDT) Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> Message-ID: <179302616.174482.1373162415199.JavaMail.zimbra@peachymango.org> Greetings, Patrick asked me to introduce a bit why DMARC and mailman. In one year DMARC has gained good support (60% of worldwide mailboxes are protected with DMARC http://www.dmarc.org/news/press_release_20130206.html), but like others I'm worried about the long tail. This is the reason some of the people working with DMARC.org have been sponsoring the openDMARC implementation to make it available on a large set of mail servers (cf http://www.trusteddomain.org/opendmarc/ for a list of sponsors). Some openDMARC packages are now available and I expect to see them as part of GNU/Linux distros anytime soon. Similarly, I'm interested to offer the option to list administrators to transition to a behavior that makes the lists safe/working/compatible with DMARC. As Patrick explained, there are about 3 possibilities, while I'm interested more in some than others (I personally experimented with the patch to mailman 2.1), it is only fair to offer the 3 options and let the list administrator choose the one more suitable for his/her needs. Once Patrick has a better understanding on how to best implement these 3 options, it will be easy, like for openDMARC, to sponsor the work to make it as part of mailman3. I know that several DMARC.org members have shown interest to do so. In an other year, with the help of the mailman community, we can progress more in the fight against fake emails. While this may sound like a sales pitch, there has not been so much excitement in email for a long time. Franck Martin https://www.linkedin.com/in/franckmartin ----- Original Message ----- From: "Patrick Ben Koetter"

To: "Mailman Developers" Sent: Monday, July 1, 2013 3:44:15 PM Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 Greetings, I am writing on behalf of a group of companies and single persons, who would like to see a limited feature set of the DMARC? standard supported by Mailman 3. Since I know we're all eager to get MM3 out as soon as possible and any additional new feature request stands against that I've contacted Barry offlist and asked if he'd agree that the companies involved pay us, sys4?, to implement the feature. He did and we also agreed to dedicate a significant part of the payment to mailman's FSF donation account. Before we take out to write code, I would like to ask mailman-developers how it should be done to fit best into Mailman's architecture. Here are the DMARC features that should go into Mailman 3: - don't allow email that comes from a domain with a DMRAC record of p=reject - take ownership of the email and send it with a From: using the domain of the mailing list. (There's a patch for this for Mailman 2.1, which might might be helpful for Mailman 3.) - find the authentication-results header and rewrite it as an Original-Authentication-header: http://tools.ietf.org/html/draft-kucherawy-original-authres-00.html Speaking of an RFC written by Murray Kucherawy. I've contacted Murray in advance and asked him to assist in case we had any questions regarding his RFC(s). He subscribed and ready to help. I hope I was able to bring all parties required together to make a Mailman DMARC implementation come true and I am curious to hear what you have to say. p at rick From superuser at gmail.com Sun Jul 7 08:02:16 2013 From: superuser at gmail.com (Murray S. Kucherawy) Date: Sat, 6 Jul 2013 23:02:16 -0700 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <20130701224414.GC1767@sys4.de> References: <20130701224414.GC1767@sys4.de> Message-ID: On Mon, Jul 1, 2013 at 3:44 PM, Patrick Ben Koetter

wrote: > Before we take out to write code, I would like to ask mailman-developers > how it > should be done to fit best into Mailman's architecture. Here are the DMARC > features that should go into Mailman 3: > > - don't allow email that comes from a domain with a DMRAC record of > p=reject > - take ownership of the email and send it with a From: using the > domain of the mailing list. (There's a patch for this for Mailman 2.1, > which > might might be helpful for Mailman 3.) > - find the authentication-results header and rewrite it as an > Original-Authentication-header: > http://tools.ietf.org/html/draft-kucherawy-original-authres-00.html > I'm all for experimentation and being adaptive as new things come along, and I'm obviously supportive of the DMARC effort. That said, I hope that these are going to be configurable options, defaulted "off" for backward compatibility. This is because: (a) the second bullet above is a significant departure from current use (as I understand it), and fails the test of least surprise if we were going to suddenly see that MM3 does things quite differently than previous versions or, indeed, other packages; and (b) I'm uneasy about Original-Authentication-Results. As far as I'm aware there's only a single, proprietary implementation. Its proponents have explained the logic to me several times, but I remain unconvinced. I'm all for experimentation in order to provide data for future efforts, so I don't really object, but this shouldn't be taken as a well-vetted proposal just because there's an (expired) draft about it. -MSK From stephen at xemacs.org Sun Jul 7 16:56:15 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 07 Jul 2013 23:56:15 +0900 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> Message-ID: <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> Murray S. Kucherawy writes: Hey, I've been trying to find the superuser at Gmail, and it turns out I've known him all along! Would you please fix the breakage where copies of one's posts aren't delivered to yourself, which is a real PITA when trying to diagnose Gmail problems? (BTW, did you try to sign up as root? or Administrator? ;-) > I'm all for experimentation and being adaptive as new things come along, > and I'm obviously supportive of the DMARC effort. That said, I hope that > these are going to be configurable options, defaulted "off" for backward > compatibility. I second your concerns in principle, but I don't think you need to worry about these features being anything but optional. I suspect that a majority of our user don't actually have the fine control over their DNS needed to implement DMARC. > (a) the second bullet above is a significant departure from current > use Violating the RFC by munging From (for heaven's sake! have we learned nothing from the sad history of Reply-To munging?) is simply not acceptable for most purposes, although perfectly acceptable for advertising. ;-) Steve From franck at peachymango.org Mon Jul 8 09:26:23 2013 From: franck at peachymango.org (Franck Martin) Date: Mon, 8 Jul 2013 02:26:23 -0500 (CDT) Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> Yes, these are options and they should be off. It is important when you introduce options, you do not change past behavior automatically. 1) may not be necessary, if mailman recognizes the bounce message as in section http://tools.ietf.org/html/draft-kucherawy-dmarc-base-00#section-15.8 eg "550 5.7.1 Email rejected per DMARC policy for example.com" and does not increase the unsubscribe/bounce counter for the receiving email address. I suppose MM3 bounce processing is better than with MM2, so this may be already addressed. Some people have requested this feature, so it is fair to include it, rather than them having to tweak the associated MTA (which some do not have control). 2) mailman has one option: reply to poster or reply to list. I think the first one is default, but as mentioned, many list admins, enable reply to list. I won't enter in the details, but my experience is that reply to list increases participation with less technically savvy people and I read why people should not do that (but they do)... Anyhow, the option is there, so I prefer not to choose on the behalf of the List Admin, and it is best to not change this setting when the list Admins enables 2) If the From: contains the posting email of the mailing list, one would think that the default becomes reply to the list, but this is where Reply-To: can be used. If the list is set to reply to poster, the email address of the poster is added to the reply-to header If the list is set to reply to the list, then the email address of the list plus the email address of the poster is added to the reply-to header. The RFC allows as many email addresses there as needed. So if no reply-to header is present, otherwise the correct email addresses are added to the list. When you click the reply button in your MUA, you achieve same results when option 2) is on or off. Reply to all button, behaves the same in all cases. So no munging is done, unless the List is already set for munging and the reply to will contain the original author of the email retransmitted by MM3. 3) This draft has been on the table for a while, as Murray points, one large mailbox provider uses it in a proprietary way, but similar to what is in Murray's draft. So there is experience, and as far as I know they still do not think it is a bad idea. Nevertheless, I think mailman should not do the email authentication part, but be able to recognize "true" authentication-result headers coming from the MTA mailman uses and be able to rewrite them as an OAR. This keep the logic simple, and should be enabled if the MTA can control Authentication-Results headers and remove spoofed ones. Personally I think 3) is more complicated than 2) to put in place correctly as it requires tight configuration between MM3 and the MTA. I hope this helps alleviate concerns. PS: Stephen, Murray does not work for Google, and therefore cannot change the way gmail works. What you describe is a gmail "feature". ----- Original Message ----- From: "Stephen J. Turnbull" To: "Murray S. Kucherawy" Cc: "Mailman Developers" Sent: Sunday, July 7, 2013 7:56:15 AM Subject: Re: [Mailman-Developers] Adding DMARC support for Mailman 3 Murray S. Kucherawy writes: Hey, I've been trying to find the superuser at Gmail, and it turns out I've known him all along! Would you please fix the breakage where copies of one's posts aren't delivered to yourself, which is a real PITA when trying to diagnose Gmail problems? (BTW, did you try to sign up as root? or Administrator? ;-) > I'm all for experimentation and being adaptive as new things come along, > and I'm obviously supportive of the DMARC effort. That said, I hope that > these are going to be configurable options, defaulted "off" for backward > compatibility. I second your concerns in principle, but I don't think you need to worry about these features being anything but optional. I suspect that a majority of our user don't actually have the fine control over their DNS needed to implement DMARC. > (a) the second bullet above is a significant departure from current > use Violating the RFC by munging From (for heaven's sake! have we learned nothing from the sad history of Reply-To munging?) is simply not acceptable for most purposes, although perfectly acceptable for advertising. ;-) From stephen at xemacs.org Mon Jul 8 11:01:43 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 08 Jul 2013 18:01:43 +0900 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> Message-ID: <87d2qtv30o.fsf@uwakimon.sk.tsukuba.ac.jp> Franck Martin writes: > If the From: contains the posting email of the mailing list, one > would think that the default becomes reply to the list, but this is > where Reply-To: can be used. Most users do not display Reply-To; many cannot (at least not at their level of technical skill). This means that they get no indication of the author of a message unless the author signs the body of the mail, which often isn't done, and is impossible to enforce. So this setting is simply unacceptable except on announce/advertising lists (where Reply-To is usually set to some other address anyway) and on anonymous lists (which as far as I know are relatively rare). If this option becomes a popular filter on large mail hosts, discussion lists (ie, the kind of mailing list that Mailman was originally intended to serve) will take a severe, perhaps fatal, blow. > I hope this helps alleviate concerns. Unfortunately, it doesn't. Imposition of authentication designed for personal and other direct mail on aggregate-and-forward services like Mailman is a knife at our throat in principle. Despite obvious goodwill on the part of IETF and others involved in the discussions, I have seen no proposal that works well for discussion lists as currently constituted. And, even more unfortunate, there is ample evidence that the large freemail services (including AOL), not to mention many inexpensive hosting services, consider the possibility that even one spam gets through a knife at *their* throats, while non-delivery of legitimate mail is no problem because it can always be blamed on somebody else.[1] I fear that any halfway plausible plan (even one as flawed as Gmail's handling of "own posts") could get quick and widespread adoption if Mailman offers such features, and operators of discussion lists will be blamed for their poor attitude toward spam-fighting when they resist using them. One might also speculate that the big portal operators are strategicly hostile to independently operated mailing lists, as they would like people to use their forum services instead. But that's purely speculation. None of this is your fault or Murray's, or DMARC's or DKIM's (despite its yahoo roots). It is, however, a reality that *we* face. > PS: Stephen, Murray does not work for Google, and therefore cannot > change the way gmail works. I'm aware of that. I was ribbing him for his choice of mailbox at Gamil. He obviously thinks it's funny; nobody imposed it on him! Footnotes: [1] For example, my University's Information and Media Center (which intercepts all SMTP connections, and likes to think of itself as in the same class as the MIT Media Lab) labelled Murray's mail "[Suspected Spam]" in the Subject field (which I removed in the interest of remaining friends). I can't even rely on the "[Spam]" label when applied by their incompetent filters. :-( From superuser at gmail.com Mon Jul 8 15:57:03 2013 From: superuser at gmail.com (Murray S. Kucherawy) Date: Mon, 8 Jul 2013 06:57:03 -0700 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> Message-ID: On Mon, Jul 8, 2013 at 12:26 AM, Franck Martin wrote: > 1) may not be necessary, if mailman recognizes the bounce message as in > section > http://tools.ietf.org/html/draft-kucherawy-dmarc-base-00#section-15.8 > eg "550 5.7.1 Email rejected per DMARC policy for example.com" > and does not increase the unsubscribe/bounce counter for the receiving > email address. I suppose MM3 bounce processing is better than with MM2, so > this may be already addressed. > Some people have requested this feature, so it is fair to include it, > rather than them having to tweak the associated MTA (which some do not have > control). > I don't think the idea of telling people to include or go look for a particular substring in the SMTP reply text will ultimately work in a standards document, which relegates this logic to the realm of heuristics. We've already seen resistance to that effect on the IETF lists. We'd be better off trying to register some enhanced status codes and asking the community to begin using those. > > 3) This draft has been on the table for a while, as Murray points, one > large mailbox provider uses it in a proprietary way, but similar to what is > in Murray's draft. So there is experience, and as far as I know they still > do not think it is a bad idea. Nevertheless, I think mailman should not do > the email authentication part, but be able to recognize "true" > authentication-result headers coming from the MTA mailman uses and be able > to rewrite them as an OAR. This keep the logic simple, and should be > enabled if the MTA can control Authentication-Results headers and remove > spoofed ones. Personally I think 3) is more complicated than 2) to put in > place correctly as it requires tight configuration between MM3 and the MTA. > Well, one (or two) parties have experience with OAR. It would be nice if this was broader, but that there is not after all this time is something I take to mean it's not a pain point others are feeling. It might work great for MM3, to be sure, but they'd effectively be the broad-scale pioneers here. -MSK From superuser at gmail.com Mon Jul 8 15:58:19 2013 From: superuser at gmail.com (Murray S. Kucherawy) Date: Mon, 8 Jul 2013 06:58:19 -0700 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <87d2qtv30o.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> <87d2qtv30o.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Mon, Jul 8, 2013 at 2:01 AM, Stephen J. Turnbull wrote: > > PS: Stephen, Murray does not work for Google, and therefore cannot > > change the way gmail works. > > I'm aware of that. I was ribbing him for his choice of mailbox at > Gamil. He obviously thinks it's funny; nobody imposed it on him! > I'm amazed I was able to get it. It's resulted in a lot of kudos, but also some very interesting spam. -MSK From franck at peachymango.org Mon Jul 8 17:26:27 2013 From: franck at peachymango.org (Franck Martin) Date: Mon, 8 Jul 2013 10:26:27 -0500 (CDT) Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> <87d2qtv30o.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <695810514.23099.1373297187561.JavaMail.zimbra@peachymango.org> ----- Original Message ----- > From: "Stephen J. Turnbull" > To: "Franck Martin" > Cc: "Mailman Developers" > Sent: Monday, July 8, 2013 2:01:43 AM > Subject: Re: [Mailman-Developers] Adding DMARC support for Mailman 3 > > Franck Martin writes: > > > If the From: contains the posting email of the mailing list, one > > would think that the default becomes reply to the list, but this is > > where Reply-To: can be used. > > Most users do not display Reply-To; many cannot (at least not at their > level of technical skill). This means that they get no indication of > the author of a message unless the author signs the body of the mail, > which often isn't done, and is impossible to enforce. So this setting > is simply unacceptable except on announce/advertising lists (where > Reply-To is usually set to some other address anyway) and on anonymous > lists (which as far as I know are relatively rare). > > If this option becomes a popular filter on large mail hosts, > discussion lists (ie, the kind of mailing list that Mailman was > originally intended to serve) will take a severe, perhaps fatal, blow. This is speculation, and broad fear of the future... The current practice for a postmaster is to trust (or not) emails from specific mailing lists, not who post them to the list. Adding DKIM to the list and taking ownership will only improve it. > > > I hope this helps alleviate concerns. > You should really read the code of the patch for MM2 and try it. From franck at peachymango.org Mon Jul 8 17:34:03 2013 From: franck at peachymango.org (Franck Martin) Date: Mon, 8 Jul 2013 10:34:03 -0500 (CDT) Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> Message-ID: <1364338180.23523.1373297643936.JavaMail.zimbra@peachymango.org> ----- Original Message ----- > From: "Murray S. Kucherawy" > To: "Franck Martin" > Cc: "Stephen J. Turnbull" , "Mailman Developers" > > Sent: Monday, July 8, 2013 6:57:03 AM > Subject: Re: [Mailman-Developers] Adding DMARC support for Mailman 3 > On Mon, Jul 8, 2013 at 12:26 AM, Franck Martin < franck at peachymango.org > > wrote: > > 1) may not be necessary, if mailman recognizes the bounce message as in > > section > > http://tools.ietf.org/html/draft-kucherawy-dmarc-base-00#section-15.8 > > > eg "550 5.7.1 Email rejected per DMARC policy for example.com " > > > and does not increase the unsubscribe/bounce counter for the receiving > > email > > address. I suppose MM3 bounce processing is better than with MM2, so this > > may be already addressed. > > > Some people have requested this feature, so it is fair to include it, > > rather > > than them having to tweak the associated MTA (which some do not have > > control). > > I don't think the idea of telling people to include or go look for a > particular substring in the SMTP reply text will ultimately work in a > standards document, which relegates this logic to the realm of heuristics. > We've already seen resistance to that effect on the IETF lists. We'd be > better off trying to register some enhanced status codes and asking the > community to begin using those. Mailman 2 read already the substring, and it is common practices amongst ESP to do that to better classify bounces, at least between soft and hard bounces ( http://www.boogietools.com/Products/Windows/BounceStudioEnterprise/Email-Bounce-Categories.asp ) I suspect this process has been improved in MM3, but I have not had a look at the code. The purpose is for mailman to recognize 5.7.x bounces and not count them against the recipient (or as soft bounce). Hard bounce is the default for any non recognized bounce. Finally, yes, DMARC should register X.7.17 on http://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xml From mark at msapiro.net Mon Jul 8 19:49:06 2013 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 08 Jul 2013 10:49:06 -0700 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <1364338180.23523.1373297643936.JavaMail.zimbra@peachymango.org> References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> <1364338180.23523.1373297643936.JavaMail.zimbra@peachymango.org> Message-ID: <51DAFB92.1010302@msapiro.net> On 07/08/2013 08:34 AM, Franck Martin wrote: > > I suspect this process has been improved in MM3, but I have not had a look at the code. The purpose is for mailman to recognize 5.7.x bounces and not count them against the recipient (or as soft bounce). Hard bounce is the default for any non recognized bounce. MM3 uses flufl.bounce for bounce processing. The process, algorithms and heuristics are essentially identical to MM2.1 In the case of an RFC 3464 compliant DSN, MM does not look at the Status field at all. It only looks at the Action field. Actions beginning with 'fail' result in a 'permanent failure' determination. Actions beginning with 'delayed' result in a 'temporary failure' determination. All other actions result in the bounce being ignored. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From franck at peachymango.org Mon Jul 8 19:52:20 2013 From: franck at peachymango.org (Franck Martin) Date: Mon, 8 Jul 2013 12:52:20 -0500 (CDT) Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> <1364338180.23523.1373297643936.JavaMail.zimbra@peachymango.org> <51DAF8BD.40806@msapiro.net> Message-ID: <1494365035.32841.1373305940888.JavaMail.zimbra@peachymango.org> ----- Original Message ----- > From: "Mark Sapiro" > To: "Franck Martin" > Sent: Monday, July 8, 2013 10:37:01 AM > Subject: Re: [Mailman-Developers] Adding DMARC support for Mailman 3 > > On 07/08/2013 08:34 AM, Franck Martin wrote: > > > > I suspect this process has been improved in MM3, but I have not had a look > > at the code. The purpose is for mailman to recognize 5.7.x bounces and not > > count them against the recipient (or as soft bounce). Hard bounce is the > > default for any non recognized bounce. > > > MM3 uses flufl.bounce for bounce > processing. The process, algorithms and heuristics are essentially > identical to MM2.1 > > In the case of an RFC 3464 compliant DSN, MM does not look at the Status > field at all. It only looks at the Action field. Actions beginning with > 'fail' result in a 'permanent failure' determination. Actions beginning > with 'delayed' result in a 'temporary failure' determination. All other > actions result in the bounce being ignored. > I see in MM2 that regex are used to classify the bounces: http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/files/head:/Mailman/Bouncers/ which is described in broad terms here: http://www.esosoft.com/support/mailinglist/mailman/bounce.html It seems flufl has the same structure http://bazaar.launchpad.net/~mailman-coders/flufl.bounce/trunk/files/head:/flufl/bounce/_detectors/ but I just read the code quickly... From mark at msapiro.net Mon Jul 8 20:37:29 2013 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 08 Jul 2013 11:37:29 -0700 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <1494365035.32841.1373305940888.JavaMail.zimbra@peachymango.org> References: <20130701224414.GC1767@sys4.de> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> <1364338180.23523.1373297643936.JavaMail.zimbra@peachymango.org> <51DAF8BD.40806@msapiro.net> <1494365035.32841.1373305940888.JavaMail.zimbra@peachymango.org> Message-ID: <51DB06E9.3090400@msapiro.net> On 07/08/2013 10:52 AM, Franck Martin wrote: > > I see in MM2 that regex are used to classify the bounces: > http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/files/head:/Mailman/Bouncers/ Yes and no. Bounce processing in the non-VERP case works by processing the DSN through a series of heuristic recognizers. A recognizer returns a possibly empty list of bouncing addresses or a special STOP signal. If the STOP signal is returned, it means "I recognize this, but ignore it" and processing of this bounce stops. If a non-empty list is returned, bounces are scored for those addresses and processing stops. If an empty list is returned, processing continues to the next recognizer. Several of the heuristic recognizers use regexps in the recognition process, but ideally, all DSNs would be RFC 3464 compliant and the recognizer for these looks only at the Action field to determine what to do. Of course in the real world, not all DSNs are RFC 3464 compliant, so we have a bunch of other recognizers. This is different in the VERP case in that the bouncing address is always obtained from the envelope recipient and the recognozers are only called to determine if the bounce is fatal or not. MM3 is different in that its recognizers return two sets of addresses, one for permanent and one for temporary failures. > which is described in broad terms here: > http://www.esosoft.com/support/mailinglist/mailman/bounce.html And which is not strictly correct for MM2.1 at least because in MM2.1, contrary to the documentation, there are no 'soft bounces'. Either delivery failed, even for a full mailbox, in which case we score a 'hard bounce', or it didn't (yet) in which case we score nothing. > It seems flufl has the same structure http://bazaar.launchpad.net/~mailman-coders/flufl.bounce/trunk/files/head:/flufl/bounce/_detectors/ In essence, the structure and process are the same, but it does add the temporary failure notion. It would certainly be possible to add recognition of a DMARC policy rejection and not report it as a bounce for a recipient (or fix the bugs one by one as they are reported), although I suspect it would be some while before the dust settles and most all the non-compliant MTAs bounces are properly classified. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From stephen at xemacs.org Tue Jul 9 03:39:14 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 09 Jul 2013 10:39:14 +0900 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <695810514.23099.1373297187561.JavaMail.zimbra@peachymango.org> References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> <87d2qtv30o.fsf@uwakimon.sk.tsukuba.ac.jp> <695810514.23099.1373297187561.JavaMail.zimbra@peachymango.org> Message-ID: <87bo6cv7el.fsf@uwakimon.sk.tsukuba.ac.jp> Franck Martin writes: > This is speculation, I said so myself. The fact that I'm paranoid just means I've read Bellovin and Cheswick. > and broad fear of the future... No, it's an extrapolation of my own occasional experience, and the frequent pain of others that I see on this list, and the observed behavior of sysadmins, some of whom I respect but who are under extreme pressure to stop spam, and others who are less competent. It's a very specific fear of a broken standard that may be imposed on us by powerful third parties. It's possible that mailing lists as we know them today are an anachronism, that they themselves are fundamentally broken in a world where spam constitutes 90% of all email traffic, and we should let them go rest in peace. I can accept that. There may be no solution that allows both existing mailing list customs to continue and provides socially acceptable levels of spam prevention. If so, so be it. > > > I hope this helps alleviate concerns. > > > You should really read the code of the patch for MM2 and try it. I don't need to when you suggest violating basic RFCs to make DMARC work better. Sometimes it's appropriate to "take ownership of From". DMARC is not a valid reason to do so, and I'm not going to try that. And what good does trying a patch do? I fear a *social* problem, not that your patch will make Mailman technically unable to receive or send mail. If the latter happens, we debug the patch. Minor irritation, no more than that. From stephen at xemacs.org Tue Jul 9 14:51:16 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 09 Jul 2013 21:51:16 +0900 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <695810514.23099.1373297187561.JavaMail.zimbra@peachymango.org> References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> <87d2qtv30o.fsf@uwakimon.sk.tsukuba.ac.jp> <695810514.23099.1373297187561.JavaMail.zimbra@peachymango.org> Message-ID: <8761wjvquz.fsf@uwakimon.sk.tsukuba.ac.jp> Franck Martin writes: > The current practice for a postmaster is to trust (or not) emails > from specific mailing lists, not who post them to the list. Really? I thought they trusted SMTP connections from specified MTAs (IP addresses). (More precisely, folks who seem to be running legitimate lists who run into problems generally find that their IP is blocked, not any identification of the list.) Anyway, List-Id is trivial to forge; I wouldn't trust it. > Adding DKIM to the list and taking ownership will only improve it. DKIM is fine, if postmasters actually do trust lists. Just use List-Id as one of the signed headers and add your own DKIM signature. Done, no need to violate RFC 5322. So I went back and re-read the DMARC spec (more carefully than I did a year ago, it seems, because it seems to be a rather different document than the one I remember reading :-/), and it seems to me that From- munging is not only a bad idea from the point of view of mailing list custom and RFC 5322 conformance, but it violates the spirit of DMARC as well. DMARC is a framework for implementing, evaluating, and improving sender policies at the domain level. It insists (correctly, for the intended application of anti-phishing) on using From and nothing else. In most cases the primary users[1] of DMARC (institutions that handle private data, whose domain names are well-known -- at least to correspondents -- and can be used for phishing) want to ensure that only messages originating from their domain can use their domain name, or at least that non-technical users can be given a very obvious indication that something funny is going on if a "From" using their domain name originated from a different domain. But they *want* their domain names seen. They don't want them munged. ***** But this philosophical discussion isn't really convincing even to me. I'd like to see examples of real use cases for DMARC, and the recommended policy settings for them. Footnotes: [1] The users whose requirements are reflected in DMARC's specific requirements. From sm at resistor.net Wed Jul 10 01:05:28 2013 From: sm at resistor.net (SM) Date: Tue, 09 Jul 2013 16:05:28 -0700 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <87bo6cv7el.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> <87d2qtv30o.fsf@uwakimon.sk.tsukuba.ac.jp> <695810514.23099.1373297187561.JavaMail.zimbra@peachymango.org> <87bo6cv7el.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <6.2.5.6.2.20130709155251.0af66328@resistor.net> Hi Stephen, At 18:39 08-07-2013, Stephen J. Turnbull wrote: >work better. Sometimes it's appropriate to "take ownership of From". There is a case where the mailing list administrator configured the list to take ownership of the "From". Telling people that it was not a good idea never works. It's easier to wait for the denial of service (which happened) and watch the complaints to pour in. Regards, -sm From barry at list.org Wed Jul 10 05:15:32 2013 From: barry at list.org (Barry Warsaw) Date: Tue, 9 Jul 2013 23:15:32 -0400 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <20130701224414.GC1767@sys4.de> References: <20130701224414.GC1767@sys4.de> Message-ID: <20130709231532.10e9ce66@anarchist> On Jul 02, 2013, at 12:44 AM, Patrick Ben Koetter wrote: >Before we take out to write code, I would like to ask mailman-developers how >it should be done to fit best into Mailman's architecture. Here are the DMARC >features that should go into Mailman 3: > >- don't allow email that comes from a domain with a DMRAC record of p=reject >- take ownership of the email and send it with a From: using the > domain of the mailing list. (There's a patch for this for Mailman 2.1, which > might might be helpful for Mailman 3.) >- find the authentication-results header and rewrite it as an > Original-Authentication-header: > http://tools.ietf.org/html/draft-kucherawy-original-authres-00.html In this message, I'll talk about implementation strategies. In very general terms, what I think you're looking at is writing an extension/plugin that provides additional rules and handlers, and then modifying or extending the built-in processing chains and pipelines to consult those new rules and handlers. In this way, DMARC support (or some subset of it) can start out as an unofficial extension, which then based on experience may eventually make it into the standard distribution. I've spoken many times about the architecture for moderation (rules and chains) and modification (pipeline of handlers), so I won't go into more detail here. Please start with my Pycon 2012 talk and feel free to ask any questions you may still have. In some sense there is overlap with how I would integrate any other anti-spam tool in MM3. Most of us agree that such things are better done in the MTA, but it's just not always possible to do that. List admins have more control over how their lists are configured than over their MTA, so I do see value in providing these services in MM3, as plugins. For #1 you would have a rule that can answer the question of DMARC disposition. Rules output binary results, and if this rule hits, it would run an action, probably to discard the message, although it could also hold it or reject/bounce it. I doubt you'd want to accept the message. If the rule misses then processing continues along as normal. Munging the From header is done by a processing handler, after the message has already been accepted for posting, and is being prepared for delivery to the list membership. Similarly, the Authentication-Results rewrite would be done in the same or a different handler. With respect to both headers, I'm assuming that the munging is the same for every recipient; there's a different hook point for personalization of outgoing messages. -Barry From barry at list.org Wed Jul 10 05:19:12 2013 From: barry at list.org (Barry Warsaw) Date: Tue, 9 Jul 2013 23:19:12 -0400 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> Message-ID: <20130709231912.2f8f103c@anarchist> On Jul 06, 2013, at 11:02 PM, Murray S. Kucherawy wrote: >I'm all for experimentation and being adaptive as new things come along, >and I'm obviously supportive of the DMARC effort. That said, I hope that >these are going to be configurable options, defaulted "off" for backward >compatibility. This is because: Current MM3 doesn't have a great way to expose plugin configuration options to the REST API, which is what a web ui would access to control this stuff. I've thought about it a lot, so I think I know how I'd do it, but there's details to be worked out and code to be written around this. >(a) the second bullet above is a significant departure from current use (as >I understand it), and fails the test of least surprise if we were going to >suddenly see that MM3 does things quite differently than previous versions >or, indeed, other packages; and Mailman has support for anonymous lists, which can rewrite the From header to substitute the mailing list's posting address for the original author's address. I frankly don't know how much that feature is used in MM2.1, if at all. -Barry From barry at list.org Wed Jul 10 05:32:17 2013 From: barry at list.org (Barry Warsaw) Date: Tue, 9 Jul 2013 23:32:17 -0400 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> Message-ID: <20130709233217.06867951@anarchist> On Jul 08, 2013, at 02:26 AM, Franck Martin wrote: >2) mailman has one option: reply to poster or reply to list. I think the >first one is default, but as mentioned, many list admins, enable reply to >list. Actually, Reply-To munging is a bit more complicated. There is a boolean flag that indicates whether any existing Reply-To headers should first be stripped, the default being not to strip. Then there is a ternary option to specify what kind of munging should happen. The default is not to munge Reply-To. The other two options are to put the mailing list's posting address in the Reply-To, or to put an explicit header (which the list admin can enter) into the Reply-To. I am strongly in the Don't Munge camp, so the default is to leave all these headers alone. Mostly, the munge options were added to silence the vocal Munge advocates. Using an explicit header has a valid use case: redirecting replies to an announce-only list posting to a discussion mailing list. Of course, the effects of any of these headers is heavily dependent on the end user's MUA and the user's own habits. I think peoples' opinion on these contentious issues is almost always driven by the MUA they use. Mine has very nice buttons to do the appropriate kind of reply (although often Gmail conspires against doing the right thing... so what's new there? ;). -Barry From barry at list.org Wed Jul 10 05:45:07 2013 From: barry at list.org (Barry Warsaw) Date: Tue, 9 Jul 2013 23:45:07 -0400 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <179302616.174482.1373162415199.JavaMail.zimbra@peachymango.org> References: <20130701224414.GC1767@sys4.de> <179302616.174482.1373162415199.JavaMail.zimbra@peachymango.org> Message-ID: <20130709234507.7aaf0275@anarchist> On Jul 06, 2013, at 09:00 PM, Franck Martin wrote: >Similarly, I'm interested to offer the option to list administrators to >transition to a behavior that makes the lists safe/working/compatible with >DMARC. As Patrick explained, there are about 3 possibilities, while I'm >interested more in some than others (I personally experimented with the patch >to mailman 2.1), it is only fair to offer the 3 options and let the list >administrator choose the one more suitable for his/her needs. Once Patrick >has a better understanding on how to best implement these 3 options, it will >be easy, like for openDMARC, to sponsor the work to make it as part of >mailman3. I know that several DMARC.org members have shown interest to do so. One other thing to keep in mind. Most list administrators have no clue how to configure their lists. Just like most technologies, they'll use whatever defaults get shipped. Postmaster are more clueful for sure (our own python.org ones being tip top :) and some of them do get involved in list configurations. MM3 supports "list styles" which are essentially composable settings applied when a mailing list is created. If there could even be such a thing as a "DMARC style", it would only need to touch the DMARC related options, and this style could be shipped in the previously mentioned plugin. -Barry From stephen at xemacs.org Wed Jul 10 09:56:36 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 10 Jul 2013 16:56:36 +0900 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <6.2.5.6.2.20130709155251.0af66328@resistor.net> References: <20130701224414.GC1767@sys4.de> <87mwpyv2pc.fsf@uwakimon.sk.tsukuba.ac.jp> <1896492635.10289.1373268383234.JavaMail.zimbra@peachymango.org> <87d2qtv30o.fsf@uwakimon.sk.tsukuba.ac.jp> <695810514.23099.1373297187561.JavaMail.zimbra@peachymango.org> <87bo6cv7el.fsf@uwakimon.sk.tsukuba.ac.jp> <6.2.5.6.2.20130709155251.0af66328@resistor.net> Message-ID: <8738rmvoej.fsf@uwakimon.sk.tsukuba.ac.jp> SM writes: > Hi Stephen, > At 18:39 08-07-2013, Stephen J. Turnbull wrote: > >work better. Sometimes it's appropriate to "take ownership of From". > > There is a case where the mailing list administrator configured the > list to take ownership of the "From". Telling people that it was not > a good idea never works. It's easier to wait for the denial of > service (which happened) and watch the complaints to pour in. It doesn't work on people who convince themselves it's the only way to solve their problems. It often does work on people who are grasping at straws, and know it. You can often convince the latter that they'll only make their problems worse. From franck at peachymango.org Wed Jul 10 11:01:28 2013 From: franck at peachymango.org (Franck Martin) Date: Wed, 10 Jul 2013 02:01:28 -0700 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> <20130709231532.10e9ce66@anarchist> Message-ID: On Jul 9, 2013, at 8:15 PM, Barry Warsaw wrote: > On Jul 02, 2013, at 12:44 AM, Patrick Ben Koetter wrote: > >> Before we take out to write code, I would like to ask mailman-developers how >> it should be done to fit best into Mailman's architecture. Here are the DMARC >> features that should go into Mailman 3: >> > > For #1 you would have a rule that can answer the question of DMARC > disposition. Rules output binary results, and if this rule hits, it would run > an action, probably to discard the message, although it could also hold it or > reject/bounce it. I doubt you'd want to accept the message. If the rule > misses then processing continues along as normal. Good idea, to held the message for approval. It is harder to know what you don't know... Holding the message for approval would give a chance for the list admin to guide the subscriber. > > Munging the From header is done by a processing handler, after the message has > already been accepted for posting, and is being prepared for delivery to the > list membership. Similarly, the Authentication-Results rewrite would be done > in the same or a different handler. With respect to both headers, I'm > assuming that the munging is the same for every recipient; there's a different > hook point for personalization of outgoing messages. > I thought about munging the From Header on a sender basis (if it has a DMARC policy) but that makes the list oscillating between two states therefore making life difficult for the recipient (rules to sort messages). From franck at peachymango.org Wed Jul 10 11:03:14 2013 From: franck at peachymango.org (Franck Martin) Date: Wed, 10 Jul 2013 02:03:14 -0700 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> <20130709231912.2f8f103c@anarchist> Message-ID: On Jul 9, 2013, at 8:19 PM, Barry Warsaw wrote: > On Jul 06, 2013, at 11:02 PM, Murray S. Kucherawy wrote: > >> >> (a) the second bullet above is a significant departure from current use (as >> I understand it), and fails the test of least surprise if we were going to >> suddenly see that MM3 does things quite differently than previous versions >> or, indeed, other packages; and > > Mailman has support for anonymous lists, which can rewrite the From header to > substitute the mailing list's posting address for the original author's > address. I frankly don't know how much that feature is used in MM2.1, if at > all. > This code was useful to do the patch for MM2.1. I too doubt it is much in use, after all we gave away our privacy long time ago on all the social networks and cloud services... but I guess it answered a need otherwise it would not have been there. From stephen at xemacs.org Wed Jul 10 20:23:29 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 11 Jul 2013 03:23:29 +0900 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <20130709231532.10e9ce66@anarchist> References: <20130701224414.GC1767@sys4.de> <20130709231532.10e9ce66@anarchist> Message-ID: <87y59etgta.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > For #1 you would have a rule that can answer the question of DMARC > disposition. Rules output binary results, This is somewhat problematic. DMARC results are potentially trivalent. If action is "reject" and pct is less than 100, some hits are "rejects" and some are "quarantine". Misses are misses. So I guess you do this with a chain of two rules, the first one verifying the message and if that hits (ie, verification fails) the second one rolls the dice for pct. > and if this rule hits, it would run an action, probably to discard > the message, although it could also hold it or reject/bounce it. Silent discards without content analysis make me queasy. I guess we can work around that by doing DMARC checks after the content checks, although the draft implies the DMARC checks should be done early. Or we could reject, but unfortunately we can't reject in the SMTP transaction, so we need to issue a DSN. That makes me really queasy, because DSNs for illegitimate mail suck all around. In case of a quarantine, maybe this should go into a separate queue that silently waits for a moderator to look at the messages, and discards them after a reasonable period of time (maybe two weeks?) So they'd be there if somebody asks for a lost message, but otherwise no bother. Steve From franck at peachymango.org Thu Jul 11 04:24:51 2013 From: franck at peachymango.org (Franck Martin) Date: Wed, 10 Jul 2013 19:24:51 -0700 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: References: <20130701224414.GC1767@sys4.de> <20130709231532.10e9ce66@anarchist> <87y59etgta.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <325075E8-9B03-423C-A1B4-A02EA8C80A82@peachymango.org> We are not asking mailman to do the work of DMARC here. There is openDMARC for that. On Jul 10, 2013, at 11:23 AM, Stephen J. Turnbull wrote: > Barry Warsaw writes: > >> For #1 you would have a rule that can answer the question of DMARC >> disposition. Rules output binary results, > > This is somewhat problematic. DMARC results are potentially > trivalent. If action is "reject" and pct is less than 100, some hits > are "rejects" and some are "quarantine". Misses are misses. So I > guess you do this with a chain of two rules, the first one verifying > the message and if that hits (ie, verification fails) the second one > rolls the dice for pct. > >> and if this rule hits, it would run an action, probably to discard >> the message, although it could also hold it or reject/bounce it. > > Silent discards without content analysis make me queasy. I guess we > can work around that by doing DMARC checks after the content checks, > although the draft implies the DMARC checks should be done early. Or > we could reject, but unfortunately we can't reject in the SMTP > transaction, so we need to issue a DSN. That makes me really queasy, > because DSNs for illegitimate mail suck all around. > > In case of a quarantine, maybe this should go into a separate queue > that silently waits for a moderator to look at the messages, and > discards them after a reasonable period of time (maybe two weeks?) So > they'd be there if somebody asks for a lost message, but otherwise no > bother. From turnbull at sk.tsukuba.ac.jp Thu Jul 11 07:50:02 2013 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 11 Jul 2013 14:50:02 +0900 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <325075E8-9B03-423C-A1B4-A02EA8C80A82@peachymango.org> References: <20130701224414.GC1767@sys4.de> <20130709231532.10e9ce66@anarchist> <87y59etgta.fsf@uwakimon.sk.tsukuba.ac.jp> <325075E8-9B03-423C-A1B4-A02EA8C80A82@peachymango.org> Message-ID: <87sizltzlh.fsf@uwakimon.sk.tsukuba.ac.jp> Franck Martin writes: > We are not asking mailman to do the work of DMARC here. There is > openDMARC for that. Of course you are, in feature #1. Unless you take it literally (reject all email that comes from such a domain, *including* email that would authenticate correctly in a full DMARC implementation). I think that the appropriate interpretation of that feature request is that "in some cases, Mailman needs to play the role of MTA in the DMARC protocol." Anything less than a full implementation is a denial of service and screws everybody: the domain owners, the recipients, and the Mailman site admins, list admins, and moderators. And us, who will take the lion's share of PRs for it even if it's a third-party module. Steve From barry at list.org Thu Jul 11 21:52:34 2013 From: barry at list.org (Barry Warsaw) Date: Thu, 11 Jul 2013 15:52:34 -0400 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <87y59etgta.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20130701224414.GC1767@sys4.de> <20130709231532.10e9ce66@anarchist> <87y59etgta.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130711155234.2979e6ef@anarchist> On Jul 11, 2013, at 03:23 AM, Stephen J. Turnbull wrote: >Barry Warsaw writes: > > > For #1 you would have a rule that can answer the question of DMARC > > disposition. Rules output binary results, > >This is somewhat problematic. DMARC results are potentially >trivalent. If action is "reject" and pct is less than 100, some hits >are "rejects" and some are "quarantine". Misses are misses. So I >guess you do this with a chain of two rules, the first one verifying >the message and if that hits (ie, verification fails) the second one >rolls the dice for pct. While ugly, that might be the best we can do for now. I have thought about adding an action to links for when the rule misses, the default being 'Defer' (i.e the next link in the chain executes as normal). That would at least give you more control over each step in the chain. But handling more than two cases quickly gets into ugliness. Another possibility is to collapse the reject/quarantine "hit" into a single boolean result. Rules can add key/values to the metadata dictionary, so you could imagine that a hit wouldn't jump directly to the Reject or Hold chains. Instead it would jump to a custom (terminal) chain that made the more specific determination of whether to reject or hold the message. > > and if this rule hits, it would run an action, probably to discard > > the message, although it could also hold it or reject/bounce it. > >Silent discards without content analysis make me queasy. Of course, we'd likely log and fire an event, so at least it wouldn't happen completely silently. >I guess we can work around that by doing DMARC checks after the content >checks, although the draft implies the DMARC checks should be done early. Or >we could reject, but unfortunately we can't reject in the SMTP transaction, >so we need to issue a DSN. That makes me really queasy, because DSNs for >illegitimate mail suck all around. Yep. There is some limited ability to do additional checking at LMTP time, but this isn't pluggable currently. >In case of a quarantine, maybe this should go into a separate queue >that silently waits for a moderator to look at the messages, and >discards them after a reasonable period of time (maybe two weeks?) So >they'd be there if somebody asks for a lost message, but otherwise no >bother. Currently there's only one moderation queue, but it can be set up to auto-discard held requests after a period of time. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From terri at zone12.com Fri Jul 12 02:22:53 2013 From: terri at zone12.com (Terri Oda) Date: Thu, 11 Jul 2013 18:22:53 -0600 Subject: [Mailman-Developers] Online Mailman Hackathon: July 14 0500-2300UTC Message-ID: <51DF4C5D.1070500@zone12.com> Hey all, We're going to do a virtual hackathon day for Mailman suite (Mailman/Postorius/Hyperkitty) on Saturday and you're all invited. I especially suggest that the GSoC students for both Mailman and Systers take advantage of this day and show up to ask questions and get some time with the mentors. Where: on IRC (#mailman on freenode) It's a virtual hackathon! When: Sunday July 14 0500 to 2300 UTC Who: anyone interested in Mailman dev As before, you're not expected to stay the whole time: people will drop in and out through that period as their time zone and other commitments permit. I, for example, will be on for a little bit at the beginning, then will be back around UTC 1600 until the end. We'll be doing a variety of things: bug triage and fixing, discussion of architecture, new feature development, helping each other with any blocking problems, spouting off crazy new ideas, code review and merging, etc. We're especially hoping to make sure we clear any issues we can relating to GSoC projects, but there's plenty of work to go around. New folk are welcome too. Hope to "see" you there! Terri PS - Here's a link to help you figure out what those times would be in your time zone. You can click "change cities" to add your own local time to the list: http://www.timeanddate.com/worldclock/meetingtime.html?iso=20130714&p1=394&p2=37&p3=248&p4=263&p5=1038 From mark at msapiro.net Fri Jul 12 02:51:13 2013 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 11 Jul 2013 17:51:13 -0700 Subject: [Mailman-Developers] Mailman 2.1.16 release. Message-ID: <51DF5301.7070802@msapiro.net> FYI, I will be releasing 2.1.16rc1 in a few days with a target of a final release in early September. I believe the release will be very solid and stable. The main purpose of the candidate release is to expose any i18n changes so that translators can submit any updates and get them in the final release. There are a few new features, contributed programs, i18n changes and bug fixes, all of which will be announced when the candidate is released. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: OpenPGP digital signature URL: From stephen at xemacs.org Fri Jul 12 04:56:41 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 12 Jul 2013 11:56:41 +0900 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <20130711155234.2979e6ef@anarchist> References: <20130701224414.GC1767@sys4.de> <20130709231532.10e9ce66@anarchist> <87y59etgta.fsf@uwakimon.sk.tsukuba.ac.jp> <20130711155234.2979e6ef@anarchist> Message-ID: <87li5ctriu.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > On Jul 11, 2013, at 03:23 AM, Stephen J. Turnbull wrote: > >This is somewhat problematic. DMARC results are potentially > >trivalent. If action is "reject" and pct is less than 100, some hits > >are "rejects" and some are "quarantine". Misses are misses. So I > >guess you do this with a chain of two rules, the first one verifying > >the message and if that hits (ie, verification fails) the second one > >rolls the dice for pct. > > While ugly, that might be the best we can do for now. Verbose, yes. Is it really ugly, though? I don't know how much you were directly influenced by iptables and SIEVE, but the idea of rule chains as a way to very flexibly configure filters has been implemented many times. The model is very simple and completely flexible. > Instead it would jump to a custom (terminal) chain that made the > more specific determination of whether to reject or hold the > message. This is pretty much what I was suggesting. > >Silent discards without content analysis make me queasy. > > Of course, we'd likely log and fire an event, so at least it wouldn't happen > completely silently. No, but it might be many days before the originator gets around to asking why their message hasn't appeared. > Yep. There is some limited ability to do additional checking at LMTP time, > but this isn't pluggable currently. Does LMTP provide the necessary ability to reject? Steve From barry at list.org Fri Jul 12 15:59:51 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 12 Jul 2013 09:59:51 -0400 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <87li5ctriu.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20130701224414.GC1767@sys4.de> <20130709231532.10e9ce66@anarchist> <87y59etgta.fsf@uwakimon.sk.tsukuba.ac.jp> <20130711155234.2979e6ef@anarchist> <87li5ctriu.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130712095951.09574305@anarchist> On Jul 12, 2013, at 11:56 AM, Stephen J. Turnbull wrote: > > Yep. There is some limited ability to do additional checking at LMTP time, > > but this isn't pluggable currently. > >Does LMTP provide the necessary ability to reject? Not reject in the Mailman sense of sending a bounce message (possibly DSN formatted), but it could refuse to accept the message from the LMTP client. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rob at colorist.org Fri Jul 12 20:13:48 2013 From: rob at colorist.org (Rob Lingelbach) Date: Fri, 12 Jul 2013 13:13:48 -0500 Subject: [Mailman-Developers] [Mailman-Announce] Mailman 2.1.16 release. In-Reply-To: <51DF5301.7070802@msapiro.net> References: <51DF5301.7070802@msapiro.net> Message-ID: <1681BE97-B456-4673-A4BB-3E97C2B679C8@colorist.org> As a long-time user of Mailman, running a list of 2000+ subscribers, thank you Mark for all your work. Is there any pertinent news regarding a new major release of Mailman? On Jul 11, 2013, at 7:51 PM, Mark Sapiro wrote: > FYI, I will be releasing 2.1.16rc1 in a few days with a target of a > final release in early September. > -- Rob Lingelbach http://rob.colorist.org From mark at msapiro.net Fri Jul 12 21:54:47 2013 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 12 Jul 2013 12:54:47 -0700 Subject: [Mailman-Developers] [Mailman-Announce] Mailman 2.1.16 release. In-Reply-To: <1681BE97-B456-4673-A4BB-3E97C2B679C8@colorist.org> References: <51DF5301.7070802@msapiro.net> <1681BE97-B456-4673-A4BB-3E97C2B679C8@colorist.org> Message-ID: <51E05F07.5020706@msapiro.net> On 07/12/2013 11:13 AM, Rob Lingelbach wrote: > As a long-time user of Mailman, running a list of 2000+ subscribers, thank you Mark for all your work. Is there any pertinent > news regarding a new major release of Mailman? Work on Mailman 3, Postorius and Hyperkitty is progressing, but at the moment I'm not aware of any firm release dates. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Fri Jul 12 22:11:55 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 12 Jul 2013 16:11:55 -0400 Subject: [Mailman-Developers] [Mailman-Announce] Mailman 2.1.16 release. In-Reply-To: <51E05F07.5020706@msapiro.net> References: <51DF5301.7070802@msapiro.net> <1681BE97-B456-4673-A4BB-3E97C2B679C8@colorist.org> <51E05F07.5020706@msapiro.net> Message-ID: <20130712161155.0311a619@anarchist> On Jul 12, 2013, at 12:54 PM, Mark Sapiro wrote: >On 07/12/2013 11:13 AM, Rob Lingelbach wrote: >> As a long-time user of Mailman, running a list of 2000+ subscribers, thank you Mark for all your work. Is there any pertinent >> news regarding a new major release of Mailman? > >Work on Mailman 3, Postorius and Hyperkitty is progressing, but at the >moment I'm not aware of any firm release dates. Come to the hackathon announced by Terri and help us! :) We'll probably also do another hackathon next weekend, same time and place. -Barry From stephen at xemacs.org Sun Jul 14 11:36:27 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 14 Jul 2013 18:36:27 +0900 Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 In-Reply-To: <20130712095951.09574305@anarchist> References: <20130701224414.GC1767@sys4.de> <20130709231532.10e9ce66@anarchist> <87y59etgta.fsf@uwakimon.sk.tsukuba.ac.jp> <20130711155234.2979e6ef@anarchist> <87li5ctriu.fsf@uwakimon.sk.tsukuba.ac.jp> <20130712095951.09574305@anarchist> Message-ID: <87a9lptrdw.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > On Jul 12, 2013, at 11:56 AM, Stephen J. Turnbull wrote: > > > > Yep. There is some limited ability to do additional checking > > > at LMTP time, but this isn't pluggable currently. > > > >Does LMTP provide the necessary ability to reject? > > Not reject in the Mailman sense of sending a bounce message (possibly DSN > formatted), but it could refuse to accept the message from the LMTP client. That's what I meant. With luck, then, the LMTP rejection could happen during the MTA's SMTP transaction. From mark at msapiro.net Sun Jul 14 22:45:43 2013 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 14 Jul 2013 13:45:43 -0700 Subject: [Mailman-Developers] Mailman 2.1.16 release. In-Reply-To: <51DF5301.7070802@msapiro.net> References: <51DF5301.7070802@msapiro.net> Message-ID: <51E30DF7.8020306@msapiro.net> I am pleased to announce the first release candidate for Mailman 2.1.16. Python 2.4 is the minimum supported, but Python 2.7 is recommended. This release has a few new features and several bug fixed as detailed in the attached README. Mailman is free software for managing email mailing lists and e-newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites. For more information, please see: http://www.list.org http://www.gnu.org/software/mailman http://mailman.sourceforge.net/ Mailman 2.1.16rc1 can be downloaded from https://launchpad.net/mailman/2.1/ http://ftp.gnu.org/gnu/mailman/ https://sourceforge.net/projects/mailman/ I anticipate that the 2.1.16 final release will be in early September. Bugs found between now and then will be fixed if possible, but I hope that most if not all changes between now and then will be i18n updates. Please send any updates to the templates and/or message catalogs in the 2.1.16rc1 release directly to me no later than 1 Sept 2013. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- 2.1.16rc1 (14-Aug-2013) New Features - Setting digest_size_threshhold to zero now means no digests will be sent based on size instead of a digest being sent with every post. (LP: #558274) - There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET which will put a dynamically generated, hidden hash in the listinfo subscribe form and check it upon submission. Setting this will prevent automated processes (bots) from successfully POSTing web subscribes without first retrieving and parsing the form from the listinfo page. The form must also be submitted no later than FORM_LIFETIME nor no earlier than SUBSCRIBE_FORM_MIN_TIME after retrieval. Note that enabling this will break any static subscribe forms on your site. See the description in Defaults.py for more info. (LP: #1082746) - add_members now has an option to add members with mail delivery disabled by admin. (LP: #1070574) - IncomingRunner now logs rejected messages to the vette log. (LP: #1068837) - The name of the mailmanctl master lock file is now congigurable via the mm_cfg.py setting MASTER_LOCK_FILE. (LP: #1082308) - list_lists now has an option to list only lists with public archives. (LP: #1082711) Contributed programs - A new import_majordomo_into_mailman.pl script has been contributed by Geoff Mayes. (LP: #1129742) - A new "sitemap" bash script has been contributed by Tomasz Chmielewski to generate a sitemap.xml file of an installation's public archives for submission to search engines. i18n - A Farsi (Persian) translation has been added thanks to Javad Hoseini and Mahyar Moghimi. - Fixed several misspelled or garbled string replacements in the Spanish message catalog. (LP: #1160138) - pt_BR message catalog has two new and an updated message per Hugo Koji Kobayashi. (LP: #1138578) - German message catalog has been updated per Ralf Hildebrandt. - Corrected typo in templates/it/private.html. Bug Fixes and other patches - Added "message_id" to the interpolation dictionary for the Article.html template. (LP: #725498) - Changed the admin GUI to report only the bad entries in a list of email addresses if any are bad. (LP: #558253) - Added logging for template errors in HyperArch.py. (LP: #558254) - Added more explanation to the bad owner address message from bin/newlist. (LP: #1200763) - Fixed a bug causing the admin web interface to fail CSRF checking if the list name contains a '+' character. (LP: #1190802) - Fixed bin/mailmanctl -s to not remove the master lock if it can't be determined to be truly stale. (LP: #1189558) - It is no longer possible to add 'invalid' addresses to the ban_list and the *_these_nonmembers filters from the check boxes on the admindb interface. (LP: #1187201) - Backported recognition for mail.ru DSNs and minor bug fixes from lp:flufl.bounce. (LP: #1074592, LP: #1079249 and #1079254) - Defended against buggy web servers that don't include an empty QUERY_STRING in the CGI environment. (LP: #1160647) - The Switchboard.finish() method now logs the text of the exception when it fails to unlink/preserve a .bak file. (LP: #1165589) - The pending (un)subscriptions waiting approval are now sorted by email address in the admindb interface as intended. (LP: #1164160) - The subscribe log entry for a bin/add_members subscribe now identifies bin/add_members as the source. (LP: #1161642) - Fixed a bug where the Subject: of the user notification of a bin/remove_members unsubscribe was not in the user's language. (LP: #1161445) - Fixed a bug where BounceRunner could create and leave behind zero length bounce-events files. (LP: #1161610) - Added recognition for another Yahoo bounce format. (LP: #1157961) - Changed configure's method for getting Python's include directory from distutils.sysconfig.get_config_var('CONFINCLUDEPY') to distutils.sysconfig.get_python_inc(). (LP: #1098162) - Added an Auto-Generated: header to password reminders. (LP: #558240) - Fixed a bug where non-ascii characters in the real name in a subscription request could throw a UnicodeEncodeError upon subscription approval and perhaps in other situations too. (LP: #1047100) - The query fragments send_unsub_notifications_to_list_owner and send_unsub_ack_to_this_batch will now assume default values if not set in mass unsubscribe URLs. (LP: #1032378) - Replaced utf-8 encoded characters in newly added German templates with HTML entities. (LP: #1018208) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: OpenPGP digital signature URL: From terri at zone12.com Sun Jul 14 23:52:21 2013 From: terri at zone12.com (Terri Oda) Date: Sun, 14 Jul 2013 15:52:21 -0600 Subject: [Mailman-Developers] Mailman 3.0 todo list Message-ID: <51E31D95.50609@zone12.com> I've started a new "things left to do before Mailman 3 releases" list over at http://wiki.list.org/display/DEV/Mailman+3.0 Mostly this is so the next time someone asks me "So, when are you releasing?" I can at least say "We don't have a date yet, but here's the remaining todo list" but also because I think our wiki is rather information-poor on the subject of the current status of mailman 3 suite. I've populated the list with some todos leftover from PyCon, but I only really pulled out the postorius ones. Barry, Aurelian, maybe you could populate the Mailman and Hyperkitty lists? Terri From stephen at xemacs.org Mon Jul 15 06:21:11 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Jul 2013 13:21:11 +0900 Subject: [Mailman-Developers] Mailman 3.0 todo list In-Reply-To: <51E31D95.50609@zone12.com> References: <51E31D95.50609@zone12.com> Message-ID: <874nbwtpvs.fsf@uwakimon.sk.tsukuba.ac.jp> Terri Oda writes: > I've started a new "things left to do before Mailman 3 releases" list > over at > http://wiki.list.org/display/DEV/Mailman+3.0 > > Mostly this is so the next time someone asks me "So, when are you > releasing?" I can at least say "We don't have a date yet, but here's the > remaining todo list" but also because I think our wiki is rather > information-poor on the subject of the current status of mailman 3 suite. > > I've populated the list with some todos leftover from PyCon, but I only > really pulled out the postorius ones. Barry, Aurelian, maybe you could > populate the Mailman and Hyperkitty lists? Good idea! The only thing I can think of offhand for core is Exim support. (Sendmail support, I suppose, but nobody I know uses Sendmail, and I can't do it. Exim on the other hand is on my personal list.) We should contact the GSoC students and ask them for suggestions. Of course, whether they are actually TODOs is to be decided by project leads. Do we have an RFE page that random wiki users could be allowed to add to? From geoff at QuiteLikely.com Mon Jul 15 10:52:36 2013 From: geoff at QuiteLikely.com (Geoff Shang) Date: Mon, 15 Jul 2013 11:52:36 +0300 (IDT) Subject: [Mailman-Developers] Mailman 3.0 todo list In-Reply-To: <51E31D95.50609@zone12.com> References: <51E31D95.50609@zone12.com> Message-ID: Hi, I was just about to ask about the status of Mailman 3. What I'm looking for is some idea of what it can do already. For example, it has all its features but needs to be administered from the command line, or it's missing important feature X, or it has everything but some switches can't be adjusted easily yet. The reason why I'm asking is that I'm in need of some Mm3 features and was wondering if it was ready for limited deployment, and if so, what those limits currently are. Thanks, Geoff. From stephen at xemacs.org Mon Jul 15 16:28:18 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Jul 2013 23:28:18 +0900 Subject: [Mailman-Developers] Mailman 3.0 todo list In-Reply-To: References: <51E31D95.50609@zone12.com> Message-ID: <87wqorsxrx.fsf@uwakimon.sk.tsukuba.ac.jp> Geoff Shang writes: > The reason why I'm asking is that I'm in need of some Mm3 features > and was wondering if it was ready for limited deployment, and if > so, what those limits currently are. The list server is in beta, when combined with Postfix as the MTA. Interfaces to other MTAs have not yet been created yet. The archiver and the web interface are at best early beta, but there are running prototypes. You can substitute a 3rd party service like mail-archive.com if that is satisfactory. The User model remains somewhat incomplete. By that I mean that we don't have a successful integration of Mailman users with "enterprise" user databases. The linkage seems a bit ad hoc in the prototypes that GSoC students are working on. There are no deployments I know of, although Barry may. (I'm leaving out people who are playing with betas; by "deployment" I mean "supporting real work".) The GSoC mentors and students are discussing setting one up that we can experiment either. HTH Steve From cnulk at scu.edu Mon Jul 15 16:57:57 2013 From: cnulk at scu.edu (Chris Nulk) Date: Mon, 15 Jul 2013 07:57:57 -0700 Subject: [Mailman-Developers] Mailman 3.0 todo list In-Reply-To: <874nbwtpvs.fsf@uwakimon.sk.tsukuba.ac.jp> References: <51E31D95.50609@zone12.com> <874nbwtpvs.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <51E40DF5.2040605@scu.edu> On 7/14/2013 9:21 PM, Stephen J. Turnbull wrote: > Terri Oda writes: > > I've started a new "things left to do before Mailman 3 releases" list > > over at > > http://wiki.list.org/display/DEV/Mailman+3.0 > > > > Mostly this is so the next time someone asks me "So, when are you > > releasing?" I can at least say "We don't have a date yet, but here's the > > remaining todo list" but also because I think our wiki is rather > > information-poor on the subject of the current status of mailman 3 suite. > > > > I've populated the list with some todos leftover from PyCon, but I only > > really pulled out the postorius ones. Barry, Aurelian, maybe you could > > populate the Mailman and Hyperkitty lists? > > Good idea! > > The only thing I can think of offhand for core is Exim support. > (Sendmail support, I suppose, but nobody I know uses Sendmail, and I Sendmail support would be nice. > can't do it. Exim on the other hand is on my personal list.) > > We should contact the GSoC students and ask them for suggestions. Of > course, whether they are actually TODOs is to be decided by project > leads. Do we have an RFE page that random wiki users could be allowed > to add to? > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/cnulk%40scu.edu > > Security Policy: http://wiki.list.org/x/QIA9 From terri at zone12.com Mon Jul 15 18:44:36 2013 From: terri at zone12.com (Terri Oda) Date: Mon, 15 Jul 2013 10:44:36 -0600 Subject: [Mailman-Developers] Mailman 3.0 todo list In-Reply-To: <87wqorsxrx.fsf@uwakimon.sk.tsukuba.ac.jp> References: <51E31D95.50609@zone12.com> <87wqorsxrx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <51E426F4.7040900@zone12.com> On 13-07-15 8:28 AM, Stephen J. Turnbull wrote: > There are no deployments I know of, although Barry may. (I'm leaving > out people who are playing with betas; by "deployment" I mean > "supporting real work".) The GSoC mentors and students are discussing > setting one up that we can experiment either. Postorius had a bug from someone running a 2000-person local music list on Mailman 3, so there's at least one fairly significant deployment that I know of. The thing that usually stops people from trying the Mailman 3 suite out is that we have no upgrade path from mailman 2 to mailman 3 at the moment. I believe there's a conversion script for archives, but you'd have to write your own script to migrate users and list settings. Terri From barry at list.org Tue Jul 16 00:44:07 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 15 Jul 2013 18:44:07 -0400 Subject: [Mailman-Developers] Mailman 3.0 todo list In-Reply-To: <51E31D95.50609@zone12.com> References: <51E31D95.50609@zone12.com> Message-ID: <20130715184407.6d676dd5@anarchist> On Jul 14, 2013, at 03:52 PM, Terri Oda wrote: >I've started a new "things left to do before Mailman 3 releases" list over at >http://wiki.list.org/display/DEV/Mailman+3.0 Thanks Terri. I'll spend some time pouring over that page w.r.t. the core, updating it, removing old to-do items, and filling in some details with my notes and tracker bugs. Anybody interested in following details should subscribe to that wiki page. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Tue Jul 16 00:45:54 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 15 Jul 2013 18:45:54 -0400 Subject: [Mailman-Developers] Mailman 3.0 todo list In-Reply-To: <87wqorsxrx.fsf@uwakimon.sk.tsukuba.ac.jp> References: <51E31D95.50609@zone12.com> <87wqorsxrx.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130715184554.71452ddf@anarchist> On Jul 15, 2013, at 11:28 PM, Stephen J. Turnbull wrote: >There are no deployments I know of, although Barry may. (I'm leaving >out people who are playing with betas; by "deployment" I mean >"supporting real work".) The GSoC mentors and students are discussing >setting one up that we can experiment either. Several months ago, I had a lot of discussions with someone who is using the core in their own project. I don't believe that work is public, so I'm not going to name the user, but it's different than the one Terri mentioned. -Barry From barry at list.org Tue Jul 16 00:52:27 2013 From: barry at list.org (Barry Warsaw) Date: Mon, 15 Jul 2013 18:52:27 -0400 Subject: [Mailman-Developers] Mailman 3.0 todo list In-Reply-To: <874nbwtpvs.fsf@uwakimon.sk.tsukuba.ac.jp> References: <51E31D95.50609@zone12.com> <874nbwtpvs.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130715185227.6630e671@anarchist> On Jul 15, 2013, at 01:21 PM, Stephen J. Turnbull wrote: >The only thing I can think of offhand for core is Exim support. >(Sendmail support, I suppose, but nobody I know uses Sendmail, and I >can't do it. Exim on the other hand is on my personal list.) I'd very much love to have Exim and Sendmail support. If you're an expert in one of those MTAs, please help us by contributing code and documentation for integrating them. Based on the Postfix integration, I suspect it will be mostly documentation and integration with alias creation. Integration is LMTP from MTA->MM3 and SMTP from MM3->MTA so I think there's probably little or nothing that has to happen at that level, except document how to set up your MTA to work properly. The most complicated part of the Postfix integration is making sure the aliases are set up correct so that when new mailing lists are added to MM3, the MTA recognizes them. That may require a bit of code, but I think the APIs are well documented. I'd also like to see better PostgreSQL and or MySQL-family integration. SQLite is the default, and we have PostgreSQL support already, although a few tests fail with PG. I would welcome PG fixes, and additional support for any database that Storm supports. (Note that at some point we may ditch Storm for SQLAlchemy, but I don't think that will majorly affect the set of FLOSS supported databases). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From stephen at xemacs.org Tue Jul 16 04:16:04 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 16 Jul 2013 11:16:04 +0900 Subject: [Mailman-Developers] Mailman 3.0 todo list In-Reply-To: <20130715185227.6630e671@anarchist> References: <51E31D95.50609@zone12.com> <874nbwtpvs.fsf@uwakimon.sk.tsukuba.ac.jp> <20130715185227.6630e671@anarchist> Message-ID: <87txjvs10b.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > On Jul 15, 2013, at 01:21 PM, Stephen J. Turnbull wrote: > > >The only thing I can think of offhand for core is Exim support. > >(Sendmail support, I suppose, but nobody I know uses Sendmail, and I > >can't do it. Exim on the other hand is on my personal list.) > > I'd very much love to have Exim and Sendmail support. If you're an > expert I'm not an Exim expert, but "on my personal list means "I'm going to start working on it tomorrow [or a little later due to Terri's recent edict to GSoCers]". I'll be happy to compare notes or pair program or whatever floats boats with anybody else interested. Steve From flo.fuchs at gmail.com Thu Jul 18 23:30:07 2013 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Thu, 18 Jul 2013 23:30:07 +0200 Subject: [Mailman-Developers] Mailman Hackathon this Sunday (July 21 2013) Message-ID: <51E85E5F.9080403@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi there, We're going to do another IRC Hackathon this Sunday for all things Mailman! This time we'll start around 0700 UTC and will stay until 2300 UTC. Please join us, no matter if you need advice, want to have some code reviewed (GSoC midterms are approaching fast!) or just like to hang and see what everyone else is working on. Most of the Mailman mentors and developers are going to be there (we're distributed pretty much around the whole planet, so not everyone's gonna be there all the time...). So save the date: Sunday July 21 2013 0700 - 2300 UTC on #mailman (on freenode). Please note that while current GSoC students (both under the PSF and Systers) are especially encouraged to join, this event is for everyone interested in developing anything Mailman-related. Oh, and here's the link to that awesome timezone converter Terri dug up: http://www.timeanddate.com/worldclock/meetingtime.html?day=21&month=7&year=2013&p1=394&p2=37&p3=248&p4=263&p5=1038&iv=0 Hope to see you on Sunday! Florian -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJR6F5fAAoJEEceGbPdavl7P1EH/0cROuMotLQbvz/yM1nkT5kn I37EtxW+1rXQXzotL1HZxSmFaI85iXR9bOhLqcKZkSswIL5HhUlXeIztx216RpSh A09K3gu6u7kFOvITkaZIA/26r0uNLbTyGsqjLZqCJbqtpSF6FHBqQdsAFQD8fNsq BiOP1VEv76Q2Cdaxyd4eDzmdH635Feb1xf3s8UvaDuILWQebZlGgxAhJmwbpe96H E4lKAzMJD4bH3NpImWa98EAi1OqmBULtzEChB6t76jt3eEp3xUpIm8zMM+CA0xsf q+AxOaMR11I+y1iPHX54xatS0UelBhQIearm0jSHsSqjATgArf8u6J7LNdkxyrY= =qBFY -----END PGP SIGNATURE----- From paul at boddie.org.uk Fri Jul 19 01:15:39 2013 From: paul at boddie.org.uk (Paul Boddie) Date: Fri, 19 Jul 2013 01:15:39 +0200 Subject: [Mailman-Developers] Wiki Migration Update Message-ID: <201307190115.39468.paul@boddie.org.uk> Hello, Once again, I've had some time to push the wiki migration along. As always, the results can be found here: http://mmwiki.boddie.org.uk/ I've updated the archived content to that of 16th July, so the translated content should reflect the existing wiki fairly accurately. Users ----- Big changes this time have happened around the handling of user information. I have modified the MoinMoin package installer to preserve author/editor details, a capability which for some reason had been removed from MoinMoin. I also need to preserve timestamps as well so that the history of each page makes a bit more sense, but at this point you should be able to see individual contributions: http://mmwiki.boddie.org.uk/DEV/LogoSubmissions?action=info The XML export archives do not contain user details over and above who edited which page (or who uploaded which attachment), but I have written scripts to obtain usernames from the archives, to fetch user details from the existing wiki, and to create accounts for users in MoinMoin. An alternative might involve getting a database query run on the server to provide user details: https://confluence.atlassian.com/display/CONFKB/How+to+Export+User+Data+to+CSV+in+Confluence There's also some kind of export functionality in Atlassian OnDemand, whatever that is: https://confluence.atlassian.com/display/AOD/Exporting+wiki+data But the solution above may be good enough: it just creates an account for each existing user and employs the same username, full name (or alias) and e-mail address, and sets a random password. It also obtains the URL of any profile picture, with these user details obtained by scraping the home page of each user on the existing wiki. How people then log into their accounts is something we can decide: e-mailing the random password is not exactly secure, but they could reset their account in the MoinMoin interface instead (which involves a reset e-mail that isn't too secure either, but it's the slightly better choice). Redirects --------- One thing we also discussed were the redirects that occur for "tiny URLs" and where page identifiers are employed. For example: http://mmwiki.boddie.org.uk/x/AgA3 http://mmwiki.boddie.org.uk/pages/viewpage.action?pageId=3604482 A mapping of page identifiers is extracted from the exported data, and this mapping is used to support a simple redirection script that looks up the appropriate page name and redirects to that page. One thing it does not yet do is to redirect to a specific revision, however. To Do ----- I always provide a list of things that still need doing, so here are some familiar items: The way comments are presented on pages still needs improving. I may write a macro to include the comment pages in a nicer way and maybe even to allow new comments to be added. Meanwhile, the comment pages should now only be editable by their original authors (by applying ACLs). It might also be nice to have a list of attachments on pages that have them, and I will take a look to see how Confluence tends to present such things. User home pages should probably be populated and have things like profile images (if provided), activity indicators, and maybe the dashboard functionality should be emulated, too. In Conclusion ------------- As I have mentioned previously, the source code for the converter can be found here: http://hgweb.boddie.org.uk/ConfluenceConverter/ Please take a look at your favourite pages and let me know where improvements can be made to the conversion process. I usually apologise for pages with question marks at the end of their names, which is a lot of the FAQ, but I've implemented a mod_rewrite hack that lets you view them, so if your passion is for FAQ fidelity then please take this opportunity to have a look. I hope this is of interest! Paul From mark at msapiro.net Sat Jul 20 02:51:48 2013 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 19 Jul 2013 17:51:48 -0700 Subject: [Mailman-Developers] Mailman 2.1.16 release. In-Reply-To: <51E30DF7.8020306@msapiro.net> References: <51DF5301.7070802@msapiro.net> <51E30DF7.8020306@msapiro.net> Message-ID: <51E9DF24.6030600@msapiro.net> I am pleased to announce the second release candidate for Mailman 2.1.16. Python 2.4 is the minimum supported, but Python 2.7 is recommended. This release has a few new features and several bug fixed as detailed in the attached README. Differences from the first candidate are the inclusion of the admindb summary sorting options feature and the 'author is list' feature and a bug fix for lp:1203200. The Farsi translation has also been updated. Mailman is free software for managing email mailing lists and e-newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites. For more information, please see: http://www.list.org http://www.gnu.org/software/mailman http://mailman.sourceforge.net/ Mailman 2.1.16rc2 can be downloaded from https://launchpad.net/mailman/2.1/ http://ftp.gnu.org/gnu/mailman/ https://sourceforge.net/projects/mailman/ I anticipate that the 2.1.16 final release will be in early September. Bugs found between now and then will be fixed if possible, but I hope that most if not all changes between now and then will be i18n updates. Please send any updates to the templates and/or message catalogs in the 2.1.16rc2 release directly to me no later than 1 Sept 2013. Note that there is a minor glitch in the tarball in that the NEWS file dates the release as xx-Aug-2013 instead of 19-Jul-2013, but I didn't think it was worth uploading new files for that. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- 2.1.16rc2 (19-Jul-2013) New Features - There is a new list attribute author_is_list to rewrite the From: header of posts replacing the posters address with that of the list for compatability with DMARC and or ADSP. There is a new mm_cfg.py setting DEFAULT_AUTHOR_IS_LIST to control the default for new lists, and the existing REMOVE_DKIM_HEADERS setting has been extended to allow removing those headers only for author_is_list = Yes lists. This feature must be enabled by setting ALLOW_AUTHOR_IS_LIST to Yes in mm_cfg.py. - There is a new DISPLAY_HELD_SUMMARY_SORT_BUTTONS setting which if set in mm_cfg.py will display a set of radio buttons in the admindb held message summary to select how the held messages are sorted and grouped for display. The exact setting determines the default grouping and sorting. See the description in Defaults.py for details. - Setting digest_size_threshhold to zero now means no digests will be sent based on size instead of a digest being sent with every post. (LP: #558274) - There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET which will put a dynamically generated, hidden hash in the listinfo subscribe form and check it upon submission. Setting this will prevent automated processes (bots) from successfully POSTing web subscribes without first retrieving and parsing the form from the listinfo page. The form must also be submitted no later than FORM_LIFETIME nor no earlier than SUBSCRIBE_FORM_MIN_TIME after retrieval. Note that enabling this will break any static subscribe forms on your site. See the description in Defaults.py for more info. (LP: #1082746) - add_members now has an option to add members with mail delivery disabled by admin. (LP: #1070574) - IncomingRunner now logs rejected messages to the vette log. (LP: #1068837) - The name of the mailmanctl master lock file is now congigurable via the mm_cfg.py setting MASTER_LOCK_FILE. (LP: #1082308) - list_lists now has an option to list only lists with public archives. (LP: #1082711) Contributed programs - A new import_majordomo_into_mailman.pl script has been contributed by Geoff Mayes. (LP: #1129742) - A new "sitemap" bash script has been contributed by Tomasz Chmielewski to generate a sitemap.xml file of an installation's public archives for submission to search engines. i18n - A Farsi (Persian) translation has been added thanks to Javad Hoseini and Mahyar Moghimi. - Fixed several misspelled or garbled string replacements in the Spanish message catalog. (LP: #1160138) - pt_BR message catalog has two new and an updated message per Hugo Koji Kobayashi. (LP: #1138578) - German message catalog has been updated per Ralf Hildebrandt. - Corrected typo in templates/it/private.html. Bug Fixes and other patches - Fixed cron/disabled to send a fresh cookie when notifying disabled members. (LP: #1203200) - Added "message_id" to the interpolation dictionary for the Article.html template. (LP: #725498) - Changed the admin GUI to report only the bad entries in a list of email addresses if any are bad. (LP: #558253) - Added logging for template errors in HyperArch.py. (LP: #558254) - Added more explanation to the bad owner address message from bin/newlist. (LP: #1200763) - Fixed a bug causing the admin web interface to fail CSRF checking if the list name contains a '+' character. (LP: #1190802) - Fixed bin/mailmanctl -s to not remove the master lock if it can't be determined to be truly stale. (LP: #1189558) - It is no longer possible to add 'invalid' addresses to the ban_list and the *_these_nonmembers filters from the check boxes on the admindb interface. (LP: #1187201) - Backported recognition for mail.ru DSNs and minor bug fixes from lp:flufl.bounce. (LP: #1074592, LP: #1079249 and #1079254) - Defended against buggy web servers that don't include an empty QUERY_STRING in the CGI environment. (LP: #1160647) - The Switchboard.finish() method now logs the text of the exception when it fails to unlink/preserve a .bak file. (LP: #1165589) - The pending (un)subscriptions waiting approval are now sorted by email address in the admindb interface as intended. (LP: #1164160) - The subscribe log entry for a bin/add_members subscribe now identifies bin/add_members as the source. (LP: #1161642) - Fixed a bug where the Subject: of the user notification of a bin/remove_members unsubscribe was not in the user's language. (LP: #1161445) - Fixed a bug where BounceRunner could create and leave behind zero length bounce-events files. (LP: #1161610) - Added recognition for another Yahoo bounce format. (LP: #1157961) - Changed configure's method for getting Python's include directory from distutils.sysconfig.get_config_var('CONFINCLUDEPY') to distutils.sysconfig.get_python_inc(). (LP: #1098162) - Added an Auto-Generated: header to password reminders. (LP: #558240) - Fixed a bug where non-ascii characters in the real name in a subscription request could throw a UnicodeEncodeError upon subscription approval and perhaps in other situations too. (LP: #1047100) - The query fragments send_unsub_notifications_to_list_owner and send_unsub_ack_to_this_batch will now assume default values if not set in mass unsubscribe URLs. (LP: #1032378) - Replaced utf-8 encoded characters in newly added German templates with HTML entities. (LP: #1018208) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: OpenPGP digital signature URL: From raj.abhilash1 at gmail.com Wed Jul 24 23:44:51 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Thu, 25 Jul 2013 03:14:51 +0530 Subject: [Mailman-Developers] GSOC Progress report Message-ID: <51F04AD3.3040201@gmail.com> Hi all, I am sorry for this delayed post, this is a brief report on my progress on gsoc project(OpenPGP integration with GNU Mailman). I have added a new rule called as 'signature'(src/mailman/rules/signature.py) which verifies if the signature in the email is valid. There is a strict implementation of RFC 3156 for the messages that we accept, only 'multipart/signed' with exactly two parts one of which is 'application/pgp-signature' is accepted, rest are held/discarded/bounced as per the list policy(whose implementation is still remaining). The signature rule uses 'gpg' utility(src/mailman/utilities/gpg.py) for verification of the signature. The storage and usage of keys is not implemented yet, for now only one key is used i.e. in the tests, the key is added as test data. The testing of this module was interesting work for me, the test-data gets copied from the 'var/gpg' directory to the temp var directory created on-the-fly during configLayer setup in tests. The work on 'sign' handler(src/mailman/handlers/sign.py) is under progress right now, it signs the text in the message body with the list's secret key. One the issues for this implementation is the structure of message that goes out. We decided to leave the sender's signature and add list's signature so that if someone wants to verify sender's signature he can do it easily. But there are issues with the structure of the message which would support more than one signatures simultaneously. There are options like one in this[1] document and other suggestion from Steve and Daniel. Also there has been some issues with pushing my version of code versioned in git to launchpad so the most recent code is available at my github repo here[2]. Efforts are on to push the code to lp and send a pull request soon. [1]: http://tools.ietf.org/html/draft-ietf-openpgp-multsig-02 [2]: https://github.com/maxking/mailman From raj.abhilash1 at gmail.com Wed Jul 24 23:45:21 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Thu, 25 Jul 2013 03:15:21 +0530 Subject: [Mailman-Developers] GSOC Progress report Message-ID: <51F04AF1.6060908@gmail.com> Hi all, I am sorry for this delayed post, this is a brief report on my progress on gsoc project(OpenPGP integration with GNU Mailman). I have added a new rule called as 'signature'(src/mailman/rules/signature.py) which verifies if the signature in the email is valid. There is a strict implementation of RFC 3156 for the messages that we accept, only 'multipart/signed' with exactly two parts one of which is 'application/pgp-signature' is accepted, rest are held/discarded/bounced as per the list policy(whose implementation is still remaining). The signature rule uses 'gpg' utility(src/mailman/utilities/gpg.py) for verification of the signature. The storage and usage of keys is not implemented yet, for now only one key is used i.e. in the tests, the key is added as test data. The testing of this module was interesting work for me, the test-data gets copied from the 'var/gpg' directory to the temp var directory created on-the-fly during configLayer setup in tests. The work on 'sign' handler(src/mailman/handlers/sign.py) is under progress right now, it signs the text in the message body with the list's secret key. One the issues for this implementation is the structure of message that goes out. We decided to leave the sender's signature and add list's signature so that if someone wants to verify sender's signature he can do it easily. But there are issues with the structure of the message which would support more than one signatures simultaneously. There are options like one in this[1] document and other suggestion from Steve and Daniel. Also there has been some issues with pushing my version of code versioned in git to launchpad so the most recent code is available at my github repo here[2]. Efforts are on to push the code to lp and send a pull request soon. [1]: http://tools.ietf.org/html/draft-ietf-openpgp-multsig-02 [2]: https://github.com/maxking/mailman -- Abhilash Raj From mgill25 at outlook.com Thu Jul 25 12:50:11 2013 From: mgill25 at outlook.com (Manish Gill) Date: Thu, 25 Jul 2013 16:20:11 +0530 Subject: [Mailman-Developers] GSoC Progress Report - Authenticated REST API Message-ID: -----BEGIN PGP MESSAGE----- Charset: ISO-8859-1 Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ hQEMA1FCth+r/vDlAQf+KQRfH5VjI5JZlTUVQOjlaXceS0R4jwFEQvfMZMhBSasn mAOSEeDbkJtKf4PbOqMg5TVj1UMj9PHSGO35vPVQJvC/CbK2+rV8Tyi/97rBkWbG 0PcpTTa+PhcOIfQuydl4TOEhqFvM+qQTukWZSl36liygFIB6Ji1Zuw5zFYouz63C MAZLckxwsmB6wI+y//06WjXtiLA+0i5mLkzp8fFft11v4+ZNE/F2Euvn9wZHJjzX BCFb5kwqhiAdIwY5bj3R9DSLeM4Z+aPc1ZYZ0OlL3UqP20vQBxtDadHymZ9SJcep i692zoDxaTtohkMqPqH19ifyHVZtAo9ifksZrEO0QdLqAV+piFXz/F3ILk5lFk8l 5836tQeR1cVpF1bDQvvxwfpzzpqr5BStnIDTHIWNHHYzlKTMxe7ieOksu6BEyusR WS+w87T9r29Sl9u9+tHN2XFqNN9OCAIYQbtJVow0Mk2ThxapJire93ztwvZCPwJ7 WorharPCJL2imkN2Pyf+a8fQImeatbFiWqUqJgauBcnVxF661q/v5L+W8ITUnaLz tbTJS8QBlQnEirO2ZdDYoRcnsWP1Truo8mhUKIS+RMp4xsmyURfS5D3N6uMGCBex qV0W4fQegQ6MlgHS3UNn/7gF1TL6oB0aZ735ZJMfVBnXal4uisfmofES5CJR6iSL UPiyskucp4vKemdetLl5cQDoP/6Q/uXYbNCL/alx+HDbeMJ/HlNV+8n/WeK77j/8 +drXhSMjUSzENlrKMzh3nvbp7DoLHIExMvQVYixG464z2tIR/W+ekG2vgbxj4r/1 4CFrI4/lu624MC+TKOnHCl4Q97SAFhFwTQVhH/J7taYdpdfDB0wbxJyt22p1n2xP XdkfwFxr1HDJEWT6XzKMr/TxJWcmzxsh0qFXXvXEn5jy8a85S4vXtYQotTs8xCj1 XMr5WPDowHDH0N/TEfidbCPtk9/Gl0EBkJk3ude69HyELRf65+pLgoJLqeXQQDbd zwIwC64ZtiTAFMC9uGSh899XJlNW32h3uGkWzgCzh4RSaiyMgAIVoWMLQ0ACo0Ez FJHokNMYfMH83Tom6d8wegdm1ZTFC8cSQal1Dp8uNMk7uL53OWIQlinO3CWsdBOf cOey3yZgzMKO1TVYWNZtPuNAXrI+98Ur/ZE4hfWgT15C1ga8FBHM/G56WTUNxY3r v2HY8JuwRDZiH2xO4JPjlSQN4ohtJ8D/jvTKIjY5n4plweRhRV9DLbFz6sfwqdFn k+gZAX7mn9d+l0zZ2FTkAmIdsC5CL9x8jylk/gxJkYkYl8dw2KNUkHUekUbGX2pK uj9i2NUkSkSCWuWhMoe7ZNyb/b5uUW61FzYwD3bVggEcfDlWao17D9TFJi1nTK9l pbLdui20iwO4U717/UDdw8InliWncwKKmA5yPEFL5HGXyzqp8G16DBm0LFJszEUI /1xaTIFBIUIEciSXFkoDvfPJA5lrLQdBifrab1CUuiUDVOGPZS7r8ob44XCE2bU3 HDmdT8ifvEh3+ZHtX03vguGNi1JUML6bCyNFIV+RXWO0igKq0EgFu8RmUBc6PXJD 2X5/Kd7StIVntn1EYhPzV3bIR7ZUS3jxP4OklwkLuo+xxcL5A65QjhEPOX/ojoOQ bU0QEywNr32oqB+1ZrmehsWabgbou6AnvPfeUUf2Sa26LvhEDKQ4kfGXh2YKGfAC wMBv8ahCBFfO4hzRWwtErPPsZRFkabuvxsk2SR65TSUrLH1RCK0l/sUebCa+HOuU yZ9y83VABQHB0GID/m+jVodtUWOZf125mfHg2OipKwRjRxRaNZgzWdLVGs+QW9mz R6g7/5ZEGZyDGjUMLrrc5GNORm2eBoHomlzJA8yvHSR87JpOIv252ipjcpqdSr0k Ghsp3lltAufDvWJNZwdtxK7gqWP51fHkzfC5CQYrUFAA/wskv0RVIbILMDO4YqEG Cmh99Xr/Nl9kUNvSaqSRjpH0nIw+Unc4OL5EOvKABqgm2yoRD/W4W3NIpQkMxBBI faBHdaMJhH9xRpHMXZXVXGYa+Opabml5jwvv2SRTGbiDU5OLvICrt8ETF9qoOgOR QhOyXOU4zHZmf4fiBy1ZV0S3 =gbbw -----END PGP MESSAGE----- From mgill25 at outlook.com Thu Jul 25 13:36:23 2013 From: mgill25 at outlook.com (Manish Gill) Date: Thu, 25 Jul 2013 17:06:23 +0530 Subject: [Mailman-Developers] GSoC Progress Report - Authenticated REST API Message-ID: Apologies about the encrypted message before. Hello, I've been documenting my progress weekly at my blog [1]. I'll give an overview of the work done. So far, me and Richard have created a new Django app mm-rest [2], which is the base of the new REST API, modified Postorius to use the app's models, which will communicate with MM-Core to get the data and expose it via Django-Rest-Framework. The approach we have been taking is to have mm-rest mirror the models exposed by the MM-Core alongside adding any of its own modifications. This makes it easy to integrate Django and any extensions/libraries for the API models. This also lets mm-rest handle the primary User model and any authentication/authorization that goes along with it. The design choices have been outlined in this [3] document. Postorius code has also been modified [4] to use these models instead of MM-client. This lets Postorius do its essential jobs even if there is no available connection to the Core. The issue that we have been working on right now is syncing. Changes b/w the Local mm-rest database and core database need to be properly synced. The data in mm-rest essentially acts like a cache for the core, so there needs to be proper integration how long that data is there for (expiration time etc), and how do we handle situations where the core database is modified without any knowledge to the local one. That's about it for now. [1] http://manishgill.blogspot.in/ [2] https://launchpad.net/mm-rest [3] http://bazaar.launchpad.net/~mgill25/mm-rest/trunk/view/head:/website/public_rest/doc/API_design.rst [4] https://code.launchpad.net/~mgill25/postorius/m-trunk/+merge/175549 From barry at list.org Fri Jul 26 21:59:14 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 26 Jul 2013 15:59:14 -0400 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <201307190115.39468.paul@boddie.org.uk> References: <201307190115.39468.paul@boddie.org.uk> Message-ID: <20130726155914.5ecfd817@anarchist> On Jul 19, 2013, at 01:15 AM, Paul Boddie wrote: >Once again, I've had some time to push the wiki migration along. As always, >the results can be found here: > >http://mmwiki.boddie.org.uk/ > >I've updated the archived content to that of 16th July, so the translated >content should reflect the existing wiki fairly accurately. Thanks very much for continuing to work on this Paul. I think it's looking great and I'd love to start coming up with a plan to migrate officially. Input from top wiki editors is key, so perhaps Mark, Terri, Steve and others can chime in. I can tell you though that my motivation to garden the current wiki is pretty low, and I think I'd be more motivated to clean it up after the Moin migration. If the other wiki editors feel the same way, then we should seize the opportunity to get the migration started. What features are missing from Moin that would prevent us from migrating, or make it more painful than staying with Confluence for now? >But the solution above may be good enough: it just creates an account for >each existing user and employs the same username, full name (or alias) and >e-mail address, and sets a random password. It also obtains the URL of any >profile picture, with these user details obtained by scraping the home page >of each user on the existing wiki. How people then log into their accounts is >something we can decide: e-mailing the random password is not exactly secure, >but they could reset their account in the MoinMoin interface instead (which >involves a reset e-mail that isn't too secure either, but it's the slightly >better choice). I'm happy with just asking everyone to reset their password. We can't do it on the demo site because email is disabled currently. What can we do about setting up the ACLs for editing? I'm happy if we start by enabling a small group to start with and having folks who want to edit pages re-request enabling their editing permissions. >One thing we also discussed were the redirects that occur for "tiny URLs" and >where page identifiers are employed. For example: > >http://mmwiki.boddie.org.uk/x/AgA3 >http://mmwiki.boddie.org.uk/pages/viewpage.action?pageId=3604482 > >A mapping of page identifiers is extracted from the exported data, and this >mapping is used to support a simple redirection script that looks up the >appropriate page name and redirects to that page. One thing it does not yet >do is to redirect to a specific revision, however. I think that's fine. I don't mind if redirects to anything other than the latest revision is enabled. Mark might disagree. I guess we won't be able to use tiny urls after the migration though, right? >I always provide a list of things that still need doing, so here are some >familiar items: > >The way comments are presented on pages still needs improving. I may write a >macro to include the comment pages in a nicer way and maybe even to allow new >comments to be added. Meanwhile, the comment pages should now only be >editable by their original authors (by applying ACLs). > >It might also be nice to have a list of attachments on pages that have them, >and I will take a look to see how Confluence tends to present such things. > >User home pages should probably be populated and have things like profile >images (if provided), activity indicators, and maybe the dashboard >functionality should be emulated, too. Would this work be better tackled before an official migration or can some of these things be done after the fact? >I hope this is of interest! It *is*, very much so. Thanks again! -Barry From mark at msapiro.net Fri Jul 26 23:05:27 2013 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 26 Jul 2013 14:05:27 -0700 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <20130726155914.5ecfd817@anarchist> References: <201307190115.39468.paul@boddie.org.uk> <20130726155914.5ecfd817@anarchist> Message-ID: <51F2E497.1050200@msapiro.net> On 07/26/2013 12:59 PM, Barry Warsaw wrote: > On Jul 19, 2013, at 01:15 AM, Paul Boddie wrote: > >> Once again, I've had some time to push the wiki migration along. As always, >> the results can be found here: >> >> http://mmwiki.boddie.org.uk/ >> >> I've updated the archived content to that of 16th July, so the translated >> content should reflect the existing wiki fairly accurately. > > Thanks very much for continuing to work on this Paul. I think it's looking > great and I'd love to start coming up with a plan to migrate officially. > Input from top wiki editors is key, so perhaps Mark, Terri, Steve and others > can chime in. Yes, thank you Paul. I too think it looks really good. > I can tell you though that my motivation to garden the current > wiki is pretty low, and I think I'd be more motivated to clean it up after the > Moin migration. If the other wiki editors feel the same way, then we should > seize the opportunity to get the migration started. +1 > What features are missing from Moin that would prevent us from migrating, or > make it more painful than staying with Confluence for now? I think we can do alright with a switch. I support a Moin wiki for my bike club, and I think we'll be OK. I have a bit to add below. >> But the solution above may be good enough: it just creates an account for >> each existing user and employs the same username, full name (or alias) and >> e-mail address, and sets a random password. It also obtains the URL of any >> profile picture, with these user details obtained by scraping the home page >> of each user on the existing wiki. How people then log into their accounts is >> something we can decide: e-mailing the random password is not exactly secure, >> but they could reset their account in the MoinMoin interface instead (which >> involves a reset e-mail that isn't too secure either, but it's the slightly >> better choice). The reset email shouldn't be a big issue if the user is prepared to receive and act on it. The reset token is only valid during the window from when it is sent to when it is used. > I'm happy with just asking everyone to reset their password. We can't do it > on the demo site because email is disabled currently. > > What can we do about setting up the ACLs for editing? I'm happy if we start > by enabling a small group to start with and having folks who want to edit > pages re-request enabling their editing permissions. It's fairly simple to add ACLs to pages that give one or more groups various access rights. Groups are just wiki pages with a list of the login names of the members of that group and so are administered by the members of the group that can modify that page. I think we can deal with that by making a small 'admin' group and then having people at large request permission (i.e. addition to the larger 'authors' group as they do now. There is an issue on my own Moin wiki in that people who are logged in but not members of any group can create and edit pages which don't have ACLs. At one point I was discovered by wiki spammers. I 'fixed' that by using Moin's textcha facility to effectively require a password to create an account. It could be solved more readily by not giving 'Known' users default (or any) write access. >> One thing we also discussed were the redirects that occur for "tiny URLs" and >> where page identifiers are employed. For example: >> >> http://mmwiki.boddie.org.uk/x/AgA3 >> http://mmwiki.boddie.org.uk/pages/viewpage.action?pageId=3604482 >> >> A mapping of page identifiers is extracted from the exported data, and this >> mapping is used to support a simple redirection script that looks up the >> appropriate page name and redirects to that page. One thing it does not yet >> do is to redirect to a specific revision, however. > > I think that's fine. I don't mind if redirects to anything other than the > latest revision is enabled. Mark might disagree. I do not disagree. I see no need to redirect those URLs to other than the current page. > I guess we won't be able to use tiny urls after the migration though, right? I think that's right, and it's unfortunate, but I can live with it (or implement a tiny url feature for Moin). >> I always provide a list of things that still need doing, so here are some >> familiar items: >> >> The way comments are presented on pages still needs improving. I may write a >> macro to include the comment pages in a nicer way and maybe even to allow new >> comments to be added. Meanwhile, the comment pages should now only be >> editable by their original authors (by applying ACLs). >> >> It might also be nice to have a list of attachments on pages that have them, >> and I will take a look to see how Confluence tends to present such things. >> >> User home pages should probably be populated and have things like profile >> images (if provided), activity indicators, and maybe the dashboard >> functionality should be emulated, too. > > Would this work be better tackled before an official migration or can some of > these things be done after the fact? I think after the fact, at least for the home page stuff, is fine. >> I hope this is of interest! > > It *is*, very much so. Thanks again! Yes, absolutely +1 -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Fri Jul 26 23:25:12 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 26 Jul 2013 17:25:12 -0400 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <51F2E497.1050200@msapiro.net> References: <201307190115.39468.paul@boddie.org.uk> <20130726155914.5ecfd817@anarchist> <51F2E497.1050200@msapiro.net> Message-ID: <20130726172512.2884e089@anarchist> On Jul 26, 2013, at 02:05 PM, Mark Sapiro wrote: >> I can tell you though that my motivation to garden the current wiki is >> pretty low, and I think I'd be more motivated to clean it up after the Moin >> migration. If the other wiki editors feel the same way, then we should >> seize the opportunity to get the migration started. > >+1 That's encouraging! :) I think the general plan is to host the wiki on the python.org infrastructure. They're already hosting the Python and Jython wikis and Noah seemed amenable to the idea at the last Pycon. If there are no objections, I'll reach out to the infrastructure team to see what they'd need. I'll have to contact John and Matt to do any required DNS changes. I think the rough task list would be: * Get python.org to assign us an IP address * Map wiki-new.list.org to that IP * Freeze all edits on wiki.list.org * Have Paul do one more migration that he's happy with * Get pdo to install that on wiki-new * Test, test, test * Move the DNS for wiki.list.org to the new IP * Decommission the Confluence instance >I think we can do alright with a switch. I support a Moin wiki for my >bike club, and I think we'll be OK. I have a bit to add below. Would you and/or Paul want shell access to the new Moin instance? I don't know if that's possible, but if you do, I'd make that part of my request to infrastructure. >There is an issue on my own Moin wiki in that people who are logged in >but not members of any group can create and edit pages which don't have >ACLs. At one point I was discovered by wiki spammers. I 'fixed' that by >using Moin's textcha facility to effectively require a password to >create an account. It could be solved more readily by not giving 'Known' >users default (or any) write access. Is that possible? I think we'd definitely want to do that. Also, I guess 'unknown' users would also not have any write access, correct? >> I guess we won't be able to use tiny urls after the migration though, right? > >I think that's right, and it's unfortunate, but I can live with it (or >implement a tiny url feature for Moin). Yay for open source! :) I love the tiny urls. I think it would make a great addition to Moin in general. >I think after the fact, at least for the home page stuff, is fine. Cool. -Barry From mark at msapiro.net Fri Jul 26 23:55:00 2013 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 26 Jul 2013 14:55:00 -0700 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <20130726172512.2884e089@anarchist> References: <201307190115.39468.paul@boddie.org.uk> <20130726155914.5ecfd817@anarchist> <51F2E497.1050200@msapiro.net> <20130726172512.2884e089@anarchist> Message-ID: <51F2F034.2090907@msapiro.net> On 07/26/2013 02:25 PM, Barry Warsaw wrote: > On Jul 26, 2013, at 02:05 PM, Mark Sapiro wrote: > > If there are no objections, I'll reach out to the infrastructure team to see > what they'd need. I'll have to contact John and Matt to do any required DNS > changes. I think the rough task list would be: > > * Get python.org to assign us an IP address > * Map wiki-new.list.org to that IP > * Freeze all edits on wiki.list.org > * Have Paul do one more migration that he's happy with > * Get pdo to install that on wiki-new > * Test, test, test > * Move the DNS for wiki.list.org to the new IP > * Decommission the Confluence instance Sounds good. Re: timing, I am off line much of now through Aug 30, so for me at least, 'Test, test, test' shouldn't begin before September. > Would you and/or Paul want shell access to the new Moin instance? I don't > know if that's possible, but if you do, I'd make that part of my request to > infrastructure. It might be helpful, but the more critical things are that probably at least one of us should be in the superuser list (a configuration setting) and there should be an AdminGroup with all rights and we should be in it. Of course we can do this with shell access to the configuration file. >> There is an issue on my own Moin wiki in that people who are logged in >> but not members of any group can create and edit pages which don't have >> ACLs. At one point I was discovered by wiki spammers. I 'fixed' that by >> using Moin's textcha facility to effectively require a password to >> create an account. It could be solved more readily by not giving 'Known' >> users default (or any) write access. > > Is that possible? I think we'd definitely want to do that. Also, I guess > 'unknown' users would also not have any write access, correct? My Moin installation is dumb in this respect. I don't remember why we wanted to set it up the way we did, maybe so a user could create their own home page before being added to a group, but in this age of wiki spam it's probably not a good idea. And yes, we can be as loose or as tight as we want. The page at gives the story, but the short answer is we can require a user to be logged in and a member of some specific group before they can create or modify any content. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From paul at boddie.org.uk Sat Jul 27 00:38:22 2013 From: paul at boddie.org.uk (Paul Boddie) Date: Sat, 27 Jul 2013 00:38:22 +0200 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <20130726155914.5ecfd817@anarchist> References: <201307190115.39468.paul@boddie.org.uk> <20130726155914.5ecfd817@anarchist> Message-ID: <201307270038.23292.paul@boddie.org.uk> On Friday 26. July 2013 21.59.14 Barry Warsaw wrote: > [Stuff already discussed] > What features are missing from Moin that would prevent us from migrating, > or make it more painful than staying with Confluence for now? I think only the actual wiki editors can answer this. I imagine that the dashboard features would be nice to have. [...] > I'm happy with just asking everyone to reset their password. We can't do > it on the demo site because email is disabled currently. I think e-mail is enabled on the python.org system, so they should be able to reset their passwords. [...] > >I always provide a list of things that still need doing, so here are some > >familiar items: > > > >The way comments are presented on pages still needs improving. I may write > >a macro to include the comment pages in a nicer way and maybe even to > >allow new comments to be added. Meanwhile, the comment pages should now > >only be editable by their original authors (by applying ACLs). I've now made some support for comments, including adding new ones. It might be wise to add textcha integration for this, though, but I already support ACLs and thus don't allow the posting of comments on pages where a user cannot write to that page, even though the comments are actually subpages. > >It might also be nice to have a list of attachments on pages that have > >them, and I will take a look to see how Confluence tends to present such > >things. > > > >User home pages should probably be populated and have things like profile > >images (if provided), activity indicators, and maybe the dashboard > >functionality should be emulated, too. > > Would this work be better tackled before an official migration or can some > of these things be done after the fact? I don't think making user pages would be too difficult, but populating them with fancy features would probably end up happening later. Paul From paul at boddie.org.uk Sat Jul 27 00:42:40 2013 From: paul at boddie.org.uk (Paul Boddie) Date: Sat, 27 Jul 2013 00:42:40 +0200 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <20130726172512.2884e089@anarchist> References: <201307190115.39468.paul@boddie.org.uk> <51F2E497.1050200@msapiro.net> <20130726172512.2884e089@anarchist> Message-ID: <201307270042.40816.paul@boddie.org.uk> On Friday 26. July 2013 23.25.12 Barry Warsaw wrote: > On Jul 26, 2013, at 02:05 PM, Mark Sapiro wrote: > > I think the general plan is to host the wiki on the python.org > infrastructure. They're already hosting the Python and Jython wikis and > Noah seemed amenable to the idea at the last Pycon. > > If there are no objections, I'll reach out to the infrastructure team to > see what they'd need. I'll have to contact John and Matt to do any > required DNS changes. I think the rough task list would be: > > * Get python.org to assign us an IP address > * Map wiki-new.list.org to that IP > * Freeze all edits on wiki.list.org > * Have Paul do one more migration that he's happy with > * Get pdo to install that on wiki-new > * Test, test, test > * Move the DNS for wiki.list.org to the new IP > * Decommission the Confluence instance We can iterate on the migration a few times before freezing and doing the final migration. The testing part is the important part, of course. ;-) > >I think we can do alright with a switch. I support a Moin wiki for my > >bike club, and I think we'll be OK. I have a bit to add below. > > Would you and/or Paul want shell access to the new Moin instance? I don't > know if that's possible, but if you do, I'd make that part of my request to > infrastructure. It would probably help a lot. > >There is an issue on my own Moin wiki in that people who are logged in > >but not members of any group can create and edit pages which don't have > >ACLs. At one point I was discovered by wiki spammers. I 'fixed' that by > >using Moin's textcha facility to effectively require a password to > >create an account. It could be solved more readily by not giving 'Known' > >users default (or any) write access. > > Is that possible? I think we'd definitely want to do that. Also, I guess > 'unknown' users would also not have any write access, correct? We can define a group of trusted editors and give them full rights. For everyone else, there is a range of options including making them ask for permission on the mailing list (or mailing an administrator directly) before being able to write anything, making them log in and possibly having "textcha" (challenge question) verification of their edits, or just having textchas preventing people from spamming. (Textchas don't work well on the Python Wiki at the moment because the questions seem to be easily guessed and haven't been fixed to address this, as far as I can tell.) I see that Mark has probably summarised this better than I have. :-) > >> I guess we won't be able to use tiny urls after the migration though, > >> right? > > > >I think that's right, and it's unfortunate, but I can live with it (or > >implement a tiny url feature for Moin). > > Yay for open source! :) I love the tiny urls. I think it would make a > great addition to Moin in general. Some kind of "compression" of a page name might be enough to provide a similar feature without requiring some kind of special index, but something using an index could also be done if necessary. Paul From paul at boddie.org.uk Sat Jul 27 00:59:09 2013 From: paul at boddie.org.uk (Paul Boddie) Date: Sat, 27 Jul 2013 00:59:09 +0200 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <51F2F034.2090907@msapiro.net> References: <201307190115.39468.paul@boddie.org.uk> <20130726172512.2884e089@anarchist> <51F2F034.2090907@msapiro.net> Message-ID: <201307270059.09987.paul@boddie.org.uk> On Friday 26. July 2013 23.55.00 Mark Sapiro wrote: > > Sounds good. Re: timing, I am off line much of now through Aug 30, so > for me at least, 'Test, test, test' shouldn't begin before September. Leaving any serious activity for a few weeks would suit me, too. [...] > My Moin installation is dumb in this respect. I don't remember why we > wanted to set it up the way we did, maybe so a user could create their > own home page before being added to a group, but in this age of wiki > spam it's probably not a good idea. On the subject of home pages, users can create their own after creating an account, but it is also possible to impose textchas on the registration process itself. The global ACLs may also prevent the creation of home pages through things like the MyPages action, but I haven't really checked this. Paul From mark at msapiro.net Sat Jul 27 02:07:18 2013 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 26 Jul 2013 17:07:18 -0700 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <201307270059.09987.paul@boddie.org.uk> References: <201307190115.39468.paul@boddie.org.uk> <20130726172512.2884e089@anarchist> <51F2F034.2090907@msapiro.net> <201307270059.09987.paul@boddie.org.uk> Message-ID: <51F30F36.8010609@msapiro.net> On 07/26/2013 03:59 PM, Paul Boddie wrote: > > On the subject of home pages, users can create their own after creating an > account, but it is also possible to impose textchas on the registration > process itself. That is in fact what I do. I have one TextCha question, "What is the account creator password?" and the answer is given to people out of band. I also have a SuperGroup whose members are all the other groups, and (indirect) members of the SuperGroup are exempt from TextChas. This way, one needs to establish contact before registering, and once registered and added to at least one group, they can proceed with minimal hinderance. > The global ACLs may also prevent the creation of home pages > through things like the MyPages action, but I haven't really checked this. I'm pretty sure this is the case. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Sat Jul 27 02:23:41 2013 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 26 Jul 2013 17:23:41 -0700 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <201307270038.23292.paul@boddie.org.uk> References: <201307190115.39468.paul@boddie.org.uk> <20130726155914.5ecfd817@anarchist> <201307270038.23292.paul@boddie.org.uk> Message-ID: <51F3130D.5040100@msapiro.net> On 07/26/2013 03:38 PM, Paul Boddie wrote: > On Friday 26. July 2013 21.59.14 Barry Warsaw wrote: > >> What features are missing from Moin that would prevent us from migrating, >> or make it more painful than staying with Confluence for now? > > I think only the actual wiki editors can answer this. I imagine that the > dashboard features would be nice to have. I'm only one data point, but pretty much the only thing I use the dashboard for is seeing the recently updated pages, both to see what's going on and check for spam. I'm perfectly happy with Moin's RecentChanges page for this. [...] > I've now made some support for comments, including adding new ones. It might > be wise to add textcha integration for this, though, but I already support > ACLs and thus don't allow the posting of comments on pages where a user cannot > write to that page, even though the comments are actually subpages. Cool on the comments support. On the ACLs, are you assuming acl_hierarchic is True or are you just duplicating the parent page's ACL? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Sat Jul 27 04:47:13 2013 From: barry at list.org (Barry Warsaw) Date: Fri, 26 Jul 2013 22:47:13 -0400 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <51F2F034.2090907@msapiro.net> References: <201307190115.39468.paul@boddie.org.uk> <20130726155914.5ecfd817@anarchist> <51F2E497.1050200@msapiro.net> <20130726172512.2884e089@anarchist> <51F2F034.2090907@msapiro.net> Message-ID: <20130726224713.59d85426@anarchist> On Jul 26, 2013, at 02:55 PM, Mark Sapiro wrote: >Sounds good. Re: timing, I am off line much of now through Aug 30, so >for me at least, 'Test, test, test' shouldn't begin before September. Works for me too. What I'll do in the meantime is get the ball rolling with infrastructure, try to get an IP assigned, and get the wiki-new.list.org hostname assigned to it. We can do all the rest later. -Barry From paul at boddie.org.uk Sat Jul 27 15:14:21 2013 From: paul at boddie.org.uk (Paul Boddie) Date: Sat, 27 Jul 2013 15:14:21 +0200 Subject: [Mailman-Developers] Wiki Migration Update In-Reply-To: <51F3130D.5040100@msapiro.net> References: <201307190115.39468.paul@boddie.org.uk> <201307270038.23292.paul@boddie.org.uk> <51F3130D.5040100@msapiro.net> Message-ID: <201307271514.22157.paul@boddie.org.uk> On Saturday 27. July 2013 02.23.41 Mark Sapiro wrote: > > I'm only one data point, but pretty much the only thing I use the > dashboard for is seeing the recently updated pages, both to see what's > going on and check for spam. I'm perfectly happy with Moin's > RecentChanges page for this. Right. It might be nice to have a per-user RecentChanges macro showing only a particular user's edits, and I've been meaning to look into this for a while, anyway. [...] > Cool on the comments support. > > On the ACLs, are you assuming acl_hierarchic is True or are you just > duplicating the parent page's ACL? I actually set an ACL that lets only the author of the comment edit the comment subpage. This obviously doesn't let others add replies to an individual comment, however, so maybe a more sophisticated approach is necessary. Paul From raj.abhilash1 at gmail.com Wed Jul 31 08:43:49 2013 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Wed, 31 Jul 2013 12:13:49 +0530 Subject: [Mailman-Developers] GSOC Midterm Report Message-ID: <51F8B225.1060504@gmail.com> Hi all, This is report of my GSoC project - OpenPG integration with GNU Mailman for midterm. The code is updated at my launchpad branch[1]. Accomplishments: * A signature rule(src/mailman/rules/signature.py) to check for OpenPGP signature in a message, incase the signature is found it tries to verify the message using gpg utility. * A gpg utility(src/mailman/utilities/gpg.py) to perform all the signature related tasks using `python-gnupg`(v0.3.4). It signs and verifies the signature. * A signmessage handler(src/mailman/handlers/signmessage.py) to sign the outgoing message using the gpg utility.(Signing a message is still incomplete in the sense we do not have a way to store and use keys. For now it has a pubring.gpg and secring.gpg inside (var/gpg/). It will be replaced with actual keyrings in future.) * Tests for each of the three modules above. Testing gpg.py was a small challenge(for me). Issues and Roadblocks: * Signature verification using `python-gnupg` was a PITA to me for sometime. The way it accepts the string and signature for detached signature is not documented at all and is converse of what I could think of. It just occurred to me to try out the other possibility which turned out to be the right way. * Version control was also a issue that I encountered. Initially I started to work on git mirror of mailman. When I tried to port the code back to bazaar and push to github it took me(and others) a lot of brainstorming. Finally I did that using bzr-git. * Deciding the structure of the signed message that we were going to send out. Initially it was decided to leave sender's signature intact so that if someone wants to verify it he can do it, but there can-not be two 'pgp-signature' parts in a conventional multipart/signed message. I wrote the code to follow an internet-draft[2] i discovered one-day. But then I commented out the code and also added another format mutipart/alternative { multipart/signed { text/plain, application/pgp-signature } multipart/signed { text/plain, application/pgp-signature } } Some furthur assistance and research on which format do MUAs support the most should be implemented. * python-gnupg does not provide any way to search for keys based on key-data like email-address. I need some more work to get this working. Future Plan: The next plans for this project include testing all the above parts thoroughly and then moving on to creating a PKI for the key. Also the settings for the list-manager to adjust a few options related to signed lists - like time limit(decided from date in signature) to consider a message old, and also implementing this in code. Schedule: My work remains two to three days behind the schedule i proposed. I hope I will be able to give some extra time and get back to schedule soon. [1]: https://code.launchpad.net/~raj-abhilash1/mailman/master [2]: http://tools.ietf.org/html/draft-ietf-openpgp-multsig-02 --- Abhilash Raj