From anirudhdahiya9 at gmail.com Wed Jun 1 06:30:59 2016 From: anirudhdahiya9 at gmail.com (Anirudh Dahiya) Date: Wed, 1 Jun 2016 16:00:59 +0530 Subject: [Mailman-Developers] GSoc Project : Doubt regarding listarchiverset Message-ID: Hi I am have built an interface(IArchiver) as part of my gsoc project,which is supposed to work on the basis of message queues. I am intending to use the same interface for multiple archives, using separate queues for each archive in this system. So for this I am registering different archivers(say ArchiverA and ArchiverB) in the config(mailman.cfg), with their class parameter being the same. But it has come to my notice, that the Archiver runner while enumerating the archieverset(IListArchiverSet), builds the set based on their class name, i.e. it builds a dictionary where the key is determined by the class of the archiver interface, and not the actual archive. I feel this makes us use a separate interface for each archive, and my design goes against that(my design wants to use the same class for multiple archives). Is there a special reason such a design was chosen, and if my idea seems fine, should I go ahead and make some changes in the mailman core. If so, please provide suggestions on how you would like me to go about it. Thanks Anirudh Dahiya From harshitbansal2015 at gmail.com Wed Jun 1 14:17:22 2016 From: harshitbansal2015 at gmail.com (Harshit Bansal) Date: Wed, 1 Jun 2016 23:47:22 +0530 Subject: [Mailman-Developers] New Interface Message-ID: Hi, I am thinking of adding a new interface named 'IStyleable' to the mailman core. All the styleable attributes from the current 'IMailingList' interface will be moved to this new interface. The current 'IMailingList' interface does not provide documentation for all the styleable attributes. This new interface will contain all styleable interfaces arranged in an orderly way with proper documentation. This new interface then can be extended by the existing 'IMailingList' interface and the new 'IStyle' interface that I am writing. Why I felt the need of a new interface? For constructing a new style I am taking a dictionary as an input to the constructor which is supposed to contain all the necessary styleable attributes and their corresponding values as key value pairs and to check whether the dictionary contains all the styleable attributes or not I will need to have a list of all the styleable attributes and that can be done very easily if I have an interface containning all the styleable attributes. I know there are other methods of getting a list of all the attributes but they are neither pythonic nor stable as all of them depend on python internals which are liable to change with the version of python. I had asked Barry for his comments but as he is busy in PyCon so he asked me to redirect my query to mailing list. As this was blocking so I in the meantime I wrote the new interface which can be found here: https://gitlab.com/_Harshit_/mailman/commit/6c94e220cd5f5ba857fa003bdc17919ca6f9217b Please provide your comments/suggestions on whether should I continue with this or not. Thanks, Harshit Bansal. From barry at list.org Fri Jun 3 15:40:29 2016 From: barry at list.org (Barry Warsaw) Date: Fri, 3 Jun 2016 12:40:29 -0700 Subject: [Mailman-Developers] New Interface In-Reply-To: References: Message-ID: <20160603124029.1b3a43ff.barry@wooz.org> Thanks for posting this Harshit. This is a useful discussion to have, and I have some thoughts, but first a little history. In MM2.1, the MailList class is composed of a number of mixins, each of which implements a general chunk of functionality, such as NNTP or digests, etc. I really didn't like the mixin structure, so for MM3, I collapsed all the style attributes into a single IMailingList interface. As you've noticed, some of the attributes in the model class aren't described in the interface. Those are mostly the legacy attributes which should eventually go away (because they are unused or represent features we don't want to support), or converted to better database types (see the PickleType for accept_these_nonmembers). It's gotten a lot better, but there are still some holdovers from the old MM2.1 code, so more opportunities to clean up and modernize. I've always wanted styles to be composable, by which I mean that a style can describe just a partial set of attributes. For now I'll call these stylets. One stylet might define how autoresponses work, while another might describe how digests work. In a sense, this is analogous to how the composition was broken down in MM2.1, but without all the ugly mixin stuff. The current IStyles support this, but of course they are only applied to mailing lists when they are created, and changing a style does not change any list to which the style was applied. On Jun 01, 2016, at 11:47 PM, Harshit Bansal wrote: >I am thinking of adding a new interface named 'IStyleable' to the >mailman core. All the styleable attributes from the current >'IMailingList' interface will be moved to this new interface. The >current 'IMailingList' interface does not provide documentation for >all the styleable attributes. This new interface will contain all >styleable interfaces arranged in an orderly way with proper >documentation. This new interface then can be extended by the existing >'IMailingList' interface and the new 'IStyle' interface that I am >writing. I think this makes sense. You're refactoring out the interface for the attributes of a mailing list apart from the functionality that a list provides. There may be some attributes which aren't style related, e.g. created_at, and perhaps some that shouldn't get refactored because they identify the list (e.g. list_name and list_id), but we can discuss that. Once you have that refactored, then I think you need a way to say "this stylet sets attributes A, B, and C, but doesn't touch X, Y, and Z". That gives you back the composability without having to worry about partial interfaces; i.e. all IStylet (actual name TBD) interfaces *can* change any style attribute, but it's okay if they change only a subset. Then it may be possible to do a lot of cool things on stylets, such as impose permissions on changing or using one; recording the stylets used with a particular mailing list, so if you change the stylet, the mailing list automatically changes, etc. >Why I felt the need of a new interface? For constructing a new style I am >taking a dictionary as an input to the constructor which is supposed to >contain all the necessary styleable attributes and their corresponding values >as key value pairs and to check whether the dictionary contains all the >styleable attributes or not I will need to have a list of all the styleable >attributes and that can be done very easily if I have an interface >containning all the styleable attributes. I know there are other methods of >getting a list of all the attributes but they are neither pythonic nor stable >as all of them depend on python internals which are liable to change with the >version of python. Let me see if I understand: you want to query the IStyleable interface to know what attributes are allowed in a style. It makes sense in that case to use a separate interface just for that purpose, without cluttering it up with all the other interface bits specifically for mailing list operation (e.g. get_roster(), subscribe(), and immutable attributes like created_at and list_id). The only other comment on the above paragraph has to do with requiring a style to contain *all* the attributes. How would you handle the composability in that case? Cheers, -Barry From stephen at xemacs.org Sat Jun 4 04:11:43 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 4 Jun 2016 17:11:43 +0900 Subject: [Mailman-Developers] Postorius issue tracker labels Message-ID: <22354.36159.520114.975742@turnbull.sk.tsukuba.ac.jp> Hi, all I just went through the labels in the Postorius issue tracker https://gitlab.com/mailman/postorius/labels and documented the undocumented as well as changing a couple of labels. Comments very welcome. There are two labels whose purpose I'm unclear on, "subscription" and "ui". I think "ui" should be split into "appearance" (CSS) and "workflow" (what operations can be done from which forms). If so, I think "subscription" really should be merged into "workflow" (at least the open issue should be). WDYT? Steve From harshitbansal2015 at gmail.com Sat Jun 4 10:08:30 2016 From: harshitbansal2015 at gmail.com (Harshit Bansal) Date: Sat, 4 Jun 2016 19:38:30 +0530 Subject: [Mailman-Developers] New Interface In-Reply-To: <20160603124029.1b3a43ff.barry@wooz.org> References: <20160603124029.1b3a43ff.barry@wooz.org> Message-ID: Hi, Thanks for replying in spite of being busy! > As you've noticed, some of the attributes in the model class aren't > described > in the interface. Those are mostly the legacy attributes which should > eventually go away (because they are unused or represent features we don't > want to support), or converted to better database types (see the PickleType > for accept_these_nonmembers). It's gotten a lot better, but there are > still > some holdovers from the old MM2.1 code, so more opportunities to clean up > and > modernize. Would you like to keep these attributes(those which are unused and we don't want to support) or should these be removed? > I've always wanted styles to be composable, by which I mean that a style > can > describe just a partial set of attributes. For now I'll call these > stylets. > One stylet might define how autoresponses work, while another might > describe > how digests work. In a sense, this is analogous to how the composition was > broken down in MM2.1, but without all the ugly mixin stuff. > > The current IStyles support this, but of course they are only applied to > mailing lists when they are created, and changing a style does not change > any > list to which the style was applied. >From core's point of view this idea seems to be fine but I am unable to think of any way of representing this idea in Postorius(since new styles would be created there)? Earlier while discussing we decided to make style composable in terms of other styles(inheritance and fallback mechanism). For example, if a user wants to modify only the the autoresponses attributes of Style1, then, he may create a child style of Style1 and change only the attributes related to the autoresponses and rest of the attributes will take their values from the parent style(Style1). Also is there any advantage of making styles composable in terms of stylets? >>I am thinking of adding a new interface named 'IStyleable' to the >>mailman core. All the styleable attributes from the current >>'IMailingList' interface will be moved to this new interface. The >>current 'IMailingList' interface does not provide documentation for >>all the styleable attributes. This new interface will contain all >>styleable interfaces arranged in an orderly way with proper >>documentation. This new interface then can be extended by the existing >>'IMailingList' interface and the new 'IStyle' interface that I am >>writing. > > I think this makes sense. You're refactoring out the interface for the > attributes of a mailing list apart from the functionality that a list > provides. There may be some attributes which aren't style related, > e.g. created_at, and perhaps some that shouldn't get refactored because > they > identify the list (e.g. list_name and list_id), but we can discuss that. The new interface will contain only styleable attributes and not any immutable attributes like created_at. All the immutable attributes will reside in the existing 'IMailingList' interface. New interface(WIP): https://gitlab.com/_Harshit_/mailman/commit/6c94e220cd5f5ba857fa003bdc17919ca6f9217b > Then it may be possible to do a lot of cool things on stylets, such as > impose > permissions on changing or using one; recording the stylets used with a > particular mailing list, so if you change the stylet, the mailing list > automatically changes, etc. How will be the permissions enforced without authorization system? > Let me see if I understand: you want to query the IStyleable interface to > know > what attributes are allowed in a style. It makes sense in that case to use > a > separate interface just for that purpose, without cluttering it up with all > the other interface bits specifically for mailing list operation > (e.g. get_roster(), subscribe(), and immutable attributes like created_at > and > list_id). As I mentioned above that the new interface is going to have only styleable attributes. > The only other comment on the above paragraph has to do with requiring a > style > to contain *all* the attributes. How would you handle the composability in > that case? All attributes will be required only in case the style doesn't inherit from any other base style. In case the style inherits from some other base style then the dictionary(argument to the constructor) will contain only those attributes which the user wants to change and rest of the attributes would be taken from the parent style. Also could you suggest some default value(such as 'DEFER') for styleable attributes that would trigger the fall back mechanism to take values from the parent style. Thanks, Harshit Bansal. From terri at toybox.ca Sat Jun 4 11:46:28 2016 From: terri at toybox.ca (Terri Oda) Date: Sat, 4 Jun 2016 08:46:28 -0700 Subject: [Mailman-Developers] Postorius issue tracker labels In-Reply-To: <22354.36159.520114.975742@turnbull.sk.tsukuba.ac.jp> References: <22354.36159.520114.975742@turnbull.sk.tsukuba.ac.jp> Message-ID: <5752F7D4.5070605@toybox.ca> On 2016-06-04 1:11 AM, Stephen J. Turnbull wrote: > Hi, all > > I just went through the labels in the Postorius issue tracker > > https://gitlab.com/mailman/postorius/labels > > and documented the undocumented as well as changing a couple of > labels. Comments very welcome. > > There are two labels whose purpose I'm unclear on, "subscription" and > "ui". I think "ui" should be split into "appearance" (CSS) and > "workflow" (what operations can be done from which forms). If so, I > think "subscription" really should be merged into "workflow" (at least > the open issue should be). It looks good to me, and I especially like the proposed UI split. I think the "subscription" tag can go away: it was used to flag bugs related to subscribing and unsubscribing, but I don't think that's a super useful category in and of itself right now . I'm not sure "rest" is the way it's currently being used either: it might be more useful to rename it to be like the wait-for-mailman tag to indicate bugs that occur due to mailmanclient. Terri From barry at list.org Sat Jun 4 13:32:43 2016 From: barry at list.org (Barry Warsaw) Date: Sat, 4 Jun 2016 10:32:43 -0700 Subject: [Mailman-Developers] Postorius issue tracker labels In-Reply-To: <5752F7D4.5070605@toybox.ca> References: <22354.36159.520114.975742@turnbull.sk.tsukuba.ac.jp> <5752F7D4.5070605@toybox.ca> Message-ID: <20160604103243.58b73933.barry@wooz.org> On Jun 04, 2016, at 08:46 AM, Terri Oda wrote: >I'm not sure "rest" is the way it's currently being used either: it might be >more useful to rename it to be like the wait-for-mailman tag to indicate bugs >that occur due to mailmanclient. 'wait-for-mailman' is just a little weird given that we use the term 'mailman' for the whole umbrella project. 'wait-for-core' is a little better but it doesn't accurately describe blockers on mailman.client. 'wait-for-api'? I'm also okay with leaving it as it us until/unless we figure out something better. -Barry From barry at list.org Sat Jun 4 15:30:01 2016 From: barry at list.org (Barry Warsaw) Date: Sat, 4 Jun 2016 12:30:01 -0700 Subject: [Mailman-Developers] New Interface In-Reply-To: References: <20160603124029.1b3a43ff.barry@wooz.org> Message-ID: <20160604123001.7d646fa8.barry@wooz.org> On Jun 04, 2016, at 07:38 PM, Harshit Bansal wrote: >Would you like to keep these attributes(those which are unused and we >don't want to support) or should these be removed? For now, keep them. It would probably make sense to open an issue on gitlab listing the unused attributes to delete. We can mark that as a tech-debt issue and address it outside of the context of your work. >From core's point of view this idea seems to be fine but I am unable to think >of any way of representing this idea in Postorius(since new styles would be >created there)? Yes, the ui issues are challenging. ;) I think it would be fairly easy to represent in REST. >Earlier while discussing we decided to make style composable in terms of >other styles(inheritance and fallback mechanism). For example, if a user >wants to modify only the the autoresponses attributes of Style1, then, he may >create a child style of Style1 and change only the attributes related to the >autoresponses and rest of the attributes will take their values from the >parent style(Style1). Also is there any advantage of making styles >composable in terms of stylets? It allows the admin to only be concerned with the attributes defined in the stylet. It can be fairly difficult otherwise to figure out what a specific stylet modifies. >The new interface will contain only styleable attributes and not any >immutable attributes like created_at. All the immutable attributes >will reside in the existing 'IMailingList' interface. Cool. >> Then it may be possible to do a lot of cool things on stylets, such as >> impose permissions on changing or using one; recording the stylets used >> with a particular mailing list, so if you change the stylet, the mailing >> list automatically changes, etc. > >How will be the permissions enforced without authorization system? Postorius would enforce it for now. When we have the authproxy, it will do it for scripts. >> The only other comment on the above paragraph has to do with requiring a >> style to contain *all* the attributes. How would you handle the >> composability in that case? > >All attributes will be required only in case the style doesn't inherit >from any other base style. In case the style inherits from some other >base style then the dictionary(argument to the constructor) will >contain only those attributes which the user wants to change and rest >of the attributes would be taken from the parent style. > >Also could you suggest some default value(such as 'DEFER') for >styleable attributes that would trigger the fall back mechanism to >take values from the parent style. That's tricky. If we weren't interfacing to a database (i.e. in pure Python), we'd have a marker object, e.g. `INHERIT = object()` but we have to worry about database column types. I'm not sure what the right thing to do there is. -Barry From harshitbansal2015 at gmail.com Sat Jun 4 16:31:55 2016 From: harshitbansal2015 at gmail.com (Harshit Bansal) Date: Sun, 5 Jun 2016 02:01:55 +0530 Subject: [Mailman-Developers] New Interface In-Reply-To: <20160604123001.7d646fa8.barry@wooz.org> References: <20160603124029.1b3a43ff.barry@wooz.org> <20160604123001.7d646fa8.barry@wooz.org> Message-ID: Hi, On 04-Jun-2016 1:11 am, "Barry Warsaw" wrote: >I've always wanted styles to be composable, by which I mean that a style can >describe just a partial set of attributes.? For now I'll call these stylets. >One stylet might define how autoresponses work, while another might describe >how digests work.? In a sense, this is analogous to how the composition was >broken down in MM2.1, but without all the ugly mixin stuff. How can this composability be achieved in a model? I mean that the size of a table is fixed in a model and if a style describes just a partial set of attributes then what would be stored in other attributes? Also how would this partial style be applied to a mailing list? Problems that are blocking me for past two-three days: Solved: 1: Getting a list of all styleable attributes. Since you have approved the new interface this problem is solved. Unsolved: 1: How to trigger the fallback mechanism? 2: Current 'IStyle' interface allows us to define either a partial style(stylet, describing just a partial set of attributes) or a complete style. But is that possible with a model? 3: If I go for style composabiltiy as described by you then how would I represent that idea in REST and Postorius? Thanks, Harshit Bansal From simon.hanna at serve-me.info Sat Jun 4 18:23:27 2016 From: simon.hanna at serve-me.info (Simon Hanna) Date: Sun, 05 Jun 2016 00:23:27 +0200 Subject: [Mailman-Developers] Postorius issue tracker labels In-Reply-To: <20160604103243.58b73933.barry@wooz.org> References: <22354.36159.520114.975742@turnbull.sk.tsukuba.ac.jp> <5752F7D4.5070605@toybox.ca> <20160604103243.58b73933.barry@wooz.org> Message-ID: <47C1159D-E9D1-4F50-8989-8EDC640E84F2@serve-me.info> >>I'm not sure "rest" is the way it's currently being used either: it >might be >>more useful to rename it to be like the wait-for-mailman tag to >indicate bugs >>that occur due to mailmanclient. > >'wait-for-mailman' is just a little weird given that we use the term >'mailman' >for the whole umbrella project. 'wait-for-core' is a little better but >it >doesn't accurately describe blockers on mailman.client. >'wait-for-api'? I'm >also okay with leaving it as it us until/unless we figure out something >better. In most cases I think we would have to wait for both core and mailmanclient. Thats why I don't think we need two labels for it. In my opinion it would be better to have only one label and always link to an issue in mailmanclient and/or core From terri at toybox.ca Sat Jun 4 20:34:31 2016 From: terri at toybox.ca (Terri Oda) Date: Sat, 4 Jun 2016 17:34:31 -0700 Subject: [Mailman-Developers] Postorius issue tracker labels In-Reply-To: <47C1159D-E9D1-4F50-8989-8EDC640E84F2@serve-me.info> References: <22354.36159.520114.975742@turnbull.sk.tsukuba.ac.jp> <5752F7D4.5070605@toybox.ca> <20160604103243.58b73933.barry@wooz.org> <47C1159D-E9D1-4F50-8989-8EDC640E84F2@serve-me.info> Message-ID: <57537397.2060307@toybox.ca> On 2016-06-04 3:23 PM, Simon Hanna wrote: > >>> I'm not sure "rest" is the way it's currently being used either: it >> might be >>> more useful to rename it to be like the wait-for-mailman tag to >> indicate bugs >>> that occur due to mailmanclient. >> >> 'wait-for-mailman' is just a little weird given that we use the term >> 'mailman' >> for the whole umbrella project. 'wait-for-core' is a little better but >> it >> doesn't accurately describe blockers on mailman.client. >> 'wait-for-api'? I'm >> also okay with leaving it as it us until/unless we figure out something >> better. > > In most cases I think we would have to wait for both core and mailmanclient. Thats why I don't think we need two labels for it. > In my opinion it would be better to have only one label and always link to an issue in mailmanclient and/or core We actually changed it to be just "blocked" so it can refer to any thing we might be waiting on, including things just in client, things in core and client, and other issues, since that turned out to be useful for triage. Terri From stephen at xemacs.org Mon Jun 6 02:35:53 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 6 Jun 2016 15:35:53 +0900 Subject: [Mailman-Developers] New Interface In-Reply-To: References: <20160603124029.1b3a43ff.barry@wooz.org> <20160604123001.7d646fa8.barry@wooz.org> Message-ID: <22357.6601.342438.129683@turnbull.sk.tsukuba.ac.jp> Harshit Bansal writes: > How can this composability be achieved in a model? I mean that the > size of a table is fixed in a model and if a style describes just a > partial set of attributes then what would be stored in other > attributes? (1) In a NoSQL database (MongoDB, for example), you can store a partial record with no preparation. This is a good thing and a bad thing, depending on how the now-you-see-them-now-you-don't fields are used. (2) SQLAlchemy (and the underlying RDBMSes) allow you to have optional fields. This is implemented by storing NULL in the missing field. > Also how would this partial style be applied to a mailing list? A style would be a list of stylets (NOTE! none of the identifiers below are real! All names have been changed to protect my ignorance!): s1 = { notmetoo_default: True, nomail_default: False } s2 = { notmetoo_default: False, ack_default: True } newstyle = [s1, s2] def apply_style(style, mlist): try: mlist.start_transaction() for s in style: # common attributes will overwrite earlier values mlist.apply_stylet(s) # make sure all attributes are set to sane values mlist.validate_style() mlist.commit_transaction() except Exception: mlist.rollback_transaction() > 2: Current 'IStyle' interface allows us to define either a partial > style(stylet, describing just a partial set of attributes) or a > complete style. But is that possible with a model? There should be either an SQLAlchemy option for each attribute to be optional, or maybe the inverse attribute of required. Use that option on the model attributes. > 3: If I go for style composabiltiy as described by you then how would > I represent that idea in REST and Postorius? Basically the same way. From adityadivekar03 at gmail.com Wed Jun 8 14:50:53 2016 From: adityadivekar03 at gmail.com (Aditya Divekar) Date: Wed, 8 Jun 2016 11:50:53 -0700 Subject: [Mailman-Developers] GSoC project - Implement module to process ARC headers Message-ID: Hi! It's been sometime since the GSoC coding period started, and I apologize for putting in this mail so late! The GSoC project is divided into two repositories. ARC implementation - https://gitlab.com/adityadivekar/arc refactored dkimpy package - https://gitlab.com/adityadivekar/sign-message The original dkimpy package hosted on launchpad provides the functions for DKIM signing and verifying of a message. This module extends the functionality by adding support for the ARC set of headers. Now the next step would be to change the namespace and make it more general to accommodate the ARC capability. That is the module now provides the capability for ARC protocol implementation but still retains its 'DKIM' namespace. Since this would involve quite a few changes, and could easily break the setup, I would like your thoughts on this if there are any good practices to follow here. The way I can think of doing this is to grep all files and change the names and then clear all the cache files. Thanks Aditya From barry at list.org Wed Jun 8 18:48:12 2016 From: barry at list.org (Barry Warsaw) Date: Wed, 8 Jun 2016 18:48:12 -0400 Subject: [Mailman-Developers] New Interface In-Reply-To: <22357.6601.342438.129683@turnbull.sk.tsukuba.ac.jp> References: <20160603124029.1b3a43ff.barry@wooz.org> <20160604123001.7d646fa8.barry@wooz.org> <22357.6601.342438.129683@turnbull.sk.tsukuba.ac.jp> Message-ID: <20160608184812.69cfacf1.barry@wooz.org> Steve's got it right. Just a few other thoughts. On Jun 06, 2016, at 03:35 PM, Stephen J. Turnbull wrote: >(2) SQLAlchemy (and the underlying RDBMSes) allow you to have optional > fields. This is implemented by storing NULL in the missing field. I think we'd prefer this method, i.e. that the stylets would be stored in the existing database, and we'd use NULLs to indicate that a particular style variable is not set in the stylet. Thus, when that stylet is applied, it would just skip over that attribute. > > 3: If I go for style composabiltiy as described by you then how would > > I represent that idea in REST and Postorius? > >Basically the same way. Just sketching things out (IANAWD :) You'd be able to create new stylets, giving them a name. For each variable in a stylet you'd be able to say whether the variable is enabled or not. If it's not, then the stylet won't change that variable. If it is enabled then it can be set in that stylet. You probably want to be able to clone and delete stylets too. You'd then have a second interface which would be for composability. I think you should be able to select named stylets and include them in your style (which would also be named). You'd be able to reorder the stylets in a style, add and remove them from the stack, etc. Bonus would be to keep track of which styles and/or stylets apply to a particular list so you could do interesting things like: * find out which lists use(d) a particular style or stylet * change an existing mailing list by changing a style or stylet (which differs from how things work today. Of course styles and stylets would be exposed in REST, and we'd have to discuss where those resources live inside the tree. But we can do that later, once/if the basic model sketched out here and in Steve's response ends up making sense. Cheers, -Barry From adam at amyl.org.uk Thu Jun 9 06:09:53 2016 From: adam at amyl.org.uk (Adam McGreggor) Date: Thu, 9 Jun 2016 11:09:53 +0100 Subject: [Mailman-Developers] One click unsub draft Message-ID: <20160609100953.GO4266@hendricks.amyl.org.uk> May be of interest to some; although I think I recall this is what Mailmans do, although not enforcing https. https://github.com/Lockhead/oneclick/blob/master/draft-herkula-oneclick.txt -- "You've got to remember most anarchists don't get up that early" ? Tony Harris, (chairman of the Metropolitan Police Authority, describing events during the morning of May Day 2001) From stephen at xemacs.org Fri Jun 10 02:29:42 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 10 Jun 2016 15:29:42 +0900 Subject: [Mailman-Developers] [Email-SIG] Some parsing/generation issues of email in Python 3 In-Reply-To: <2142887.cIH3R3lDk4@xrated> References: <4466958.y32vL1X9xu@xrated> <2142887.cIH3R3lDk4@xrated> Message-ID: <22362.24150.266657.519411@turnbull.sk.tsukuba.ac.jp> Hans-Peter Jansen writes: > > I'm willing to bet it will converge to "robust in practice" much > > faster than the previous design did. > > I will take your word on that. Why take my word when you can take my money instead? :-) Keep on with the porting project! > > Patience and understanding of how opensource software development > > works, perhaps. > > Okay, as already said, I'm sorry for sounding overly harsh. Yeah, me, too. I actually saw Barry's and David's posts while composing and decided I didn't have much to add after all, but apparently I managed to send without noticing (bad Emacs! bad Emacs!) before deleting the draft. Sorry about that; you didn't deserve multiple reprimands. > Usually, when I report such problems nowadays, I add a patch > proposal for fixing the issue, but these issues were overwhelming > me. Needless to mention the complexity of the email package itself > and my reluctance of studying RFCs. The email RFCs are probably not worth studying. They're not only quite complex, on many points they are controversial. Leave that to us. ;-) Merely identifying problems (thanks for the test cases, by the way!) is a great contribution. From khushbuparakh at hotmail.com Sat Jun 11 17:14:35 2016 From: khushbuparakh at hotmail.com (Khushbu Parakh) Date: Sat, 11 Jun 2016 21:14:35 +0000 Subject: [Mailman-Developers] Postorius: Search Plugin for List Message-ID: Hi, I am working on search plugin for lists and wondering what would be the best approach, using Ajax or reloading the page and which JS files. Do I need to write urls with the handler for JSON?I have made the UI for search list. What is the best way to approach. Best, Khushbu Parakhhttp://khushbu.strikingly.com/ Arya College Of Engineering and ITLinkedin: http://linkedin.com/in/khushbuparakh at khushbuparakh From simon.hanna at serve-me.info Sat Jun 11 20:57:48 2016 From: simon.hanna at serve-me.info (Simon Hanna) Date: Sun, 12 Jun 2016 02:57:48 +0200 Subject: [Mailman-Developers] Postorius: Search Plugin for List In-Reply-To: References: Message-ID: <47bfbcc5-4b73-6b90-765e-9c8dc975ab6d@serve-me.info> On 06/11/2016 11:14 PM, Khushbu Parakh wrote: > Hi, > I am working on search plugin for lists I think this would be a nice addition to Postorius. > and wondering what would be the best approach, > using Ajax or reloading the page and which JS files. > Do I need to write urls with the handler for JSON? > I have made the UI for search list. > What is the best way to approach. It depends. What functions would your search plugin support, how would it be used? I guess it also depends on what functions of mailmanclient you would use. Actually I think that neither mailmanclient nor core expose (efficient) methods that can be used to search for lists. From stephen at xemacs.org Sun Jun 12 11:56:58 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 13 Jun 2016 00:56:58 +0900 Subject: [Mailman-Developers] Postorius: Search Plugin for List In-Reply-To: References: Message-ID: <22365.34378.874557.681622@turnbull.sk.tsukuba.ac.jp> Khushbu Parakh writes: > I am working on search plugin for lists You're proposing to find information about lists, not posts, right? What search criteria do you propose? Just name ~= posting address ~= List-Id? > and wondering what would be the best approach, using Ajax Think about why you use AJAX. AJAX makes the most sense when you have large data that can be updated in pieces, or when you have interactive work that needs to be done on the server. (Javascript can be used entirely browser-side to improve UI interactivity, but that's not really AJAX.) Does your proposed plugin fit either of those categories? Do you envision generalization that would fit? I don't know how true it is these days, but traditionally Mailman has been used by many folks who refuse to use Javascript, so your plugin should be able to fall back to straight HTML (with or without CSS, for the sake of text browsers and blind users). That's not an absolute requirement, but do avoid anything really fancy that just won't work if you don't have everything enabled in the browser. > or reloading the page and which JS files. Do I need to write urls > with the handler for JSON? I have made the UI for search list. Screen shot, please! (Most important, it shows us a lot about the amount of user interaction you are proposing.) > What is the best way to approach. You tell us. ;-) (Your taste in technology counts, too!) Steve From khushbuparakh at hotmail.com Sun Jun 12 13:51:18 2016 From: khushbuparakh at hotmail.com (Khushbu Parakh) Date: Sun, 12 Jun 2016 17:51:18 +0000 Subject: [Mailman-Developers] Postorius: Search Plugin for List In-Reply-To: <22365.34378.874557.681622@turnbull.sk.tsukuba.ac.jp> References: , <22365.34378.874557.681622@turnbull.sk.tsukuba.ac.jp> Message-ID: Hi, So right now UI is just in HTML with no CSS. I will add CSS later to make it in perfect position. I tried to keep it simpler. What I think is AJAX is better option cause reloading the page just to get search results is something we don't like. I mentioned JS because all I want to know is backend process behind search, nothing related to UI and proposing criteria is list_name only. If list_id is useful we can show it too. PFA Thanks, Steve, Best, Khushbu Parakhhttp://khushbu.strikingly.com/ Arya College Of Engineering and ITLinkedin: http://linkedin.com/in/khushbuparakh at khushbuparakh > Date: Mon, 13 Jun 2016 00:56:58 +0900 > From: stephen at xemacs.org > To: khushbuparakh at hotmail.com > CC: mailman-developers at python.org > Subject: [Mailman-Developers] Postorius: Search Plugin for List > > Khushbu Parakh writes: > > > I am working on search plugin for lists > > You're proposing to find information about lists, not posts, right? > What search criteria do you propose? Just name ~= posting address ~= > List-Id? > > > and wondering what would be the best approach, using Ajax > > Think about why you use AJAX. AJAX makes the most sense when you have > large data that can be updated in pieces, or when you have interactive > work that needs to be done on the server. (Javascript can be used > entirely browser-side to improve UI interactivity, but that's not > really AJAX.) > > Does your proposed plugin fit either of those categories? Do you > envision generalization that would fit? > > I don't know how true it is these days, but traditionally Mailman has > been used by many folks who refuse to use Javascript, so your plugin > should be able to fall back to straight HTML (with or without CSS, for > the sake of text browsers and blind users). That's not an absolute > requirement, but do avoid anything really fancy that just won't work > if you don't have everything enabled in the browser. > > > or reloading the page and which JS files. Do I need to write urls > > with the handler for JSON? I have made the UI for search list. > > Screen shot, please! > > (Most important, it shows us a lot about the amount of user > interaction you are proposing.) > > > What is the best way to approach. > > You tell us. ;-) (Your taste in technology counts, too!) > > Steve > -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot from 2016-06-12 10-31-06.png Type: image/png Size: 66702 bytes Desc: not available URL: From adam-mailman at amyl.org.uk Sun Jun 12 14:15:26 2016 From: adam-mailman at amyl.org.uk (Adam McGreggor) Date: Sun, 12 Jun 2016 19:15:26 +0100 Subject: [Mailman-Developers] Postorius: Search Plugin for List In-Reply-To: <22365.34378.874557.681622@turnbull.sk.tsukuba.ac.jp> References: <22365.34378.874557.681622@turnbull.sk.tsukuba.ac.jp> Message-ID: <20160612181525.GR4266@hendricks.amyl.org.uk> On Mon, Jun 13, 2016 at 12:56:58AM +0900, Stephen J. Turnbull wrote: > Khushbu Parakh writes: > > I am working on search plugin for lists > > You're proposing to find information about lists, not posts, right? > What search criteria do you propose? Just name ~= posting address ~= > List-Id? It might be nice from my point of view to be able to search by admins, owners, moderators, as well as (sub)domain (which may be via posting address). -- "Bernard: It's another of those irregular verbs. I hold confidential briefings, you leak, he's been charged under section 2A of the Official Secrets Act." From stephen at xemacs.org Tue Jun 14 15:24:08 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 15 Jun 2016 04:24:08 +0900 Subject: [Mailman-Developers] GSoC project - Implement module to process ARC headers In-Reply-To: References: Message-ID: <22368.23000.857798.790637@turnbull.sk.tsukuba.ac.jp> Hi! I'm sorry that I haven't responded before, it's been very busy catching up to $DAYJOB since getting back. > Now the next step would be to change the namespace and make it more > general to accommodate the ARC capability. That is the module now > provides the capability for ARC protocol implementation but still > retains its 'DKIM' namespace. Since this would involve quite a few > changes, and could easily break the setup, I would like your > thoughts on this if there are any good practices to follow here. I don't understand what you're worried about. Of course if you change all the names, especially with some sort of automatic process, there's a pretty good chance of introducing typos. We fix them as we find them, that's all. > The way I can think of doing this is to grep all files and change > the names and then clear all the cache files. Do you mean __pycache__? That will happen automatically for all files with updated API calls. If you haven't updated the source to use the changed API, the calls will break anyway. The only time this is a worry is if you repurpose an existing name to different semantics, and the answer there is "don't do that," because it's problematic anyway. But these will be new names, so shouldn't cause that kind of problem. Re interop: I have a server with a fixed IP which could just pipe mail to a Python script that reads it and passes the message to the ARC module for processing. Then the script can send it back out if necessary. From adityadivekar03 at gmail.com Wed Jun 15 00:01:04 2016 From: adityadivekar03 at gmail.com (Aditya Divekar) Date: Wed, 15 Jun 2016 09:31:04 +0530 Subject: [Mailman-Developers] GSoC project - Implement module to process ARC headers In-Reply-To: <22368.23000.857798.790637@turnbull.sk.tsukuba.ac.jp> References: <22368.23000.857798.790637@turnbull.sk.tsukuba.ac.jp> Message-ID: > > > I don't understand what you're worried about. Of course if you change > all the names, especially with some sort of automatic process, there's > a pretty good chance of introducing typos. We fix them as we find > them, that's all. > Yes, I went ahead and made the changes. I have a separate module 'arcpy' installed in my python dist-packages now which is basically the refactored 'dkimpy' package. I don't think its the best of names since it doesn't give the idea of presence of DKIM signing/verifying features. Any ideas for the same? I have a server with a fixed IP which could just pipe mail to a Python > script that reads it and passes the message to the ARC module for > processing. Then the script can send it back out if necessary. > That should work for us! Also we would need to have a DMARC and DKIM record set up though for the signing domain. The procedures for that are easy, and are outlined in detail on the net. So if we have a domain name associated with the ip, I'll go ahead and make the TXT records. Else, we'll have to create a new one. I'm not aware of any way to avoid this. Let me know if you are aware of any :) Aditya From anirudhdahiya9 at gmail.com Sat Jun 18 00:56:01 2016 From: anirudhdahiya9 at gmail.com (Anirudh Dahiya) Date: Sat, 18 Jun 2016 10:26:01 +0530 Subject: [Mailman-Developers] GSOC: Progress, community review and blog post Message-ID: Hi My Gsoc project Message queue based email archiver has progressed quite well. The detailed progress and journey had been documented by me in this blog post - http://codefullofsummer.blogspot.in/ Also, I would like to share my work and ask for community review - https://gitlab.com/anirudhdahiya9/RabbitMQMessageQueueArchiver/tree/pubMQArchiver https://gitlab.com/anirudhdahiya9/mailman/tree/MQArchiver Regards Anirudh Dahiya (irc spark) From stephen at xemacs.org Sat Jun 18 06:38:07 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 18 Jun 2016 19:38:07 +0900 Subject: [Mailman-Developers] GSoC project - Implement module to process ARC headers In-Reply-To: References: <22368.23000.857798.790637@turnbull.sk.tsukuba.ac.jp> Message-ID: <22373.9359.579521.793923@turnbull.sk.tsukuba.ac.jp> Aditya Divekar writes: > So if we have a domain name associated with the ip, I'll go ahead > and make the TXT records. Else, we'll have to create a new one. > I'm not aware of any way to avoid this. Let me know if you are > aware of any :) Ah, that's a problem. I don't have control of the DNS yet. We'd need to interact with Mark. Anyway it's too late for this one, there were procedures to be taken earlier in the week. I'll get you on the arc-interop list, there should be another one in the summer at the pace they've been going. > Yes, I went ahead and made the changes. Good! > I have a separate module 'arcpy' installed in my python > dist-packages now which is basically the refactored 'dkimpy' > package. I don't think its the best of names since it doesn't give > the idea of presence of DKIM signing/verifying features. Any ideas > for the same? Not really. Keywords that might be useful aside from ARC, DKIM, and DMARC: mail, post, verify, track, key, sign, identity. Ah -- one brainstorm -- kimpy = "key identified mail.py". As far as words go, you could just keep dkimpy, too, since ARC depends on domain keys for its semantics, as well as on DKIM for its signing protocol. "adkimpy"? Lots of double meanings there: Aditya, ARC, augmented, domain, divikar. :-) I was surprised by the news, but Google also chose dkimpy for its implementation of ARC. I haven't had time to look at it, but maybe early next week (not publicly licensed, sorry, no URL). Steve From adityadivekar03 at gmail.com Sat Jun 18 12:49:48 2016 From: adityadivekar03 at gmail.com (Aditya Divekar) Date: Sat, 18 Jun 2016 22:19:48 +0530 Subject: [Mailman-Developers] GSoC project - Implement module to process ARC headers In-Reply-To: <22373.9359.579521.793923@turnbull.sk.tsukuba.ac.jp> References: <22368.23000.857798.790637@turnbull.sk.tsukuba.ac.jp> <22373.9359.579521.793923@turnbull.sk.tsukuba.ac.jp> Message-ID: On Sat, Jun 18, 2016 at 4:08 PM, Stephen J. Turnbull wrote: > Ah, that's a problem. I don't have control of the DNS yet. We'd need > to interact with Mark. So should a new conversation be started with Mark for our request? > I'll get you on the > arc-interop list, there should be another one in the summer at the > pace they've been going. > Okay! > Not really. Keywords that might be useful aside from ARC, DKIM, and > DMARC: mail, post, verify, track, key, sign, identity. > > Ah -- one brainstorm -- kimpy = "key identified mail.py". > > That sounds good! The `mail` keyword gave an idea - `emailauthentication` As far as words go, you could just keep dkimpy, too, since ARC depends > on domain keys for its semantics, as well as on DKIM for its signing > protocol. "adkimpy"? Lots of double meanings there: Aditya, ARC, > augmented, domain, divikar. :-) > > Hahaha! In case we choose that, that can be our inside joke. Narrowing it down, we have the options - dkimpy, kimpy, adkimpy ( meaning augmented/arc - domain ;) ), emailauthentication, and any others with keywords you specified. > I was surprised by the news, but Google also chose dkimpy for its > implementation of ARC. I haven't had time to look at it, but maybe > early next week (not publicly licensed, sorry, no URL). > I had read on the arc list that an OpenARC implementation was being developed by Murray Kucherawy. So I suppose that we may have the news of it being released on the list soon. Aditya From stephen at xemacs.org Sun Jun 19 04:36:50 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 19 Jun 2016 17:36:50 +0900 Subject: [Mailman-Developers] GSoC project - Implement module to process ARC headers In-Reply-To: References: <22368.23000.857798.790637@turnbull.sk.tsukuba.ac.jp> <22373.9359.579521.793923@turnbull.sk.tsukuba.ac.jp> Message-ID: <22374.22946.733506.543457@turnbull.sk.tsukuba.ac.jp> Aditya Divekar writes: > So should a new conversation be started with Mark for > our request? I/we need to find out exactly what we need. We may want to do this through email-test.python.org (no-existent, for example, python.org is the relevant part, or maybe a private domain of someone's), but we have to be careful that it doesn't conflict with other uses. > That sounds good! The `mail` keyword gave an idea - > `emailauthentication` I think that would need to be shortened, but mailauth would work and has a good rhythm. :-) Of course it may be "taken". > I had read on the arc list that an OpenARC implementation was being > developed by Murray Kucherawy. So I suppose that we may have the > news of it being released on the list soon. Yes, but Murray's OpenDKIM is a C library, and I expect OpenArc is too. I don't think we should be using a C library because we're not an MTA, and I suspect it will be making assumptions (eg, availability of SPF) that we shouldn't. In any case, the more independent implementations the better. Steve From adityadivekar03 at gmail.com Sun Jun 19 06:10:32 2016 From: adityadivekar03 at gmail.com (Aditya Divekar) Date: Sun, 19 Jun 2016 15:40:32 +0530 Subject: [Mailman-Developers] GSoC project - Implement module to process ARC headers In-Reply-To: <22374.22946.733506.543457@turnbull.sk.tsukuba.ac.jp> References: <22368.23000.857798.790637@turnbull.sk.tsukuba.ac.jp> <22373.9359.579521.793923@turnbull.sk.tsukuba.ac.jp> <22374.22946.733506.543457@turnbull.sk.tsukuba.ac.jp> Message-ID: > > > I/we need to find out exactly what we need. We may want to do this > through email-test.python.org (no-existent, for example, python.org is > the relevant part, or maybe a private domain of someone's), but we > have to be careful that it doesn't conflict with other uses. email-test.python.org sounds apt. And since you think its good we could ping Mark about the same. Also, since its for our interop, we will be freeing it up again after the testing period ( something along the line of a few months ) making it available again. > I think that would need to be shortened, but mailauth would work and > has a good rhythm. :-) Of course it may be "taken". > > Perfect! I checked PyPI and didn't find any such packages. So at least till now no one has laid claim to it on PyPI though "taken" might mean other things I understand. (Still, the doubt remains whether the original author would like to get the name changed to something different. ) > Yes, but Murray's OpenDKIM is a C library, and I expect OpenArc is > too. I don't think we should be using a C library because we're not... > > In any case, the more independent implementations the better. > Yes, right. I somehow missed that C fact. I'll like to see the code once its out, and see the approach used. Not for our purposes, but just curious :) Aditya From bajwa.nischay at gmail.com Tue Jun 21 04:39:09 2016 From: bajwa.nischay at gmail.com (Nischay Singh) Date: Tue, 21 Jun 2016 08:39:09 +0000 Subject: [Mailman-Developers] Begginer Level Bugs Message-ID: Hi all! This is Nischay. I'm new to the open source community. Looking forward to actively contributing to mailman. I read the guidelines on the website and found that the bugs are classified for beginners. Can someone please tell me where to look for such bugs. Thank you! -- Nischay Singh Student B.E.(Hons) Computer Science BITS Pilani From mark at msapiro.net Tue Jun 21 13:43:38 2016 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 21 Jun 2016 10:43:38 -0700 Subject: [Mailman-Developers] Begginer Level Bugs In-Reply-To: References: Message-ID: <57697CCA.5010704@msapiro.net> On 06/21/2016 01:39 AM, Nischay Singh wrote: > I read the guidelines on the website and found that the bugs are classified > for beginners. Can someone please tell me where to look for such bugs. Go to and look for issues tagged 'beginner friendly'. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From aditi.medha96 at gmail.com Wed Jun 22 19:09:21 2016 From: aditi.medha96 at gmail.com (ADITI GUPTA) Date: Thu, 23 Jun 2016 04:39:21 +0530 Subject: [Mailman-Developers] Hyperkitty set up Message-ID: Hello, I am trying to connect hyperkitty to mailman. I have cloned and set up mailman-hyperkitty and made changes in the mailman_hyperkitty.cfg and mailman.cfg files. But, still I am unable to see anything in hyperkitty(except the login page).I have injected mails to a list using mailman inject -f email.txt test at example.net but couldn't see anything there except the login page. "No archived lists yet" is shown there. I checked the MailingList model in hyperkitty through command line and found that it is empty. Regards, Aditi From harshitbansal2015 at gmail.com Thu Jun 23 17:44:42 2016 From: harshitbansal2015 at gmail.com (Harshit Bansal) Date: Fri, 24 Jun 2016 03:14:42 +0530 Subject: [Mailman-Developers] GSOC:Populating two default styles Message-ID: Hi, I was trying to populate the two default styles in the database. For this purpose, I was trying to use ConfigurationUpdatedEvent as used in the earlier style manager but I am recieving following error: Traceback (most recent call last): File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/testing/layers.py", line 87, in setUp initialize.initialize_1(INHIBIT_CONFIG_FILE) File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/core/initialize.py", line 118, in initialize_1 mailman.config.config.load(config_path) File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/config/config.py", line 102, in load self._post_process() File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/config/config.py", line 125, in _post_process notify(ConfigurationUpdatedEvent(self)) File "/home/harshit/Drive1/dev/Mailman/env/lib/python3.4/site-packages/zope.event-4.1.0-py3.4.egg/zope/event/__init__.py", line 31, in notify subscriber(event) File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/model/styletmanager.py", line 105, in handle_ConfigurationUpdatedEvent getUtility(IStyletManager).populate() File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/config/config.py", line 87, in __getattr__ return getattr(self._config, name) File "/home/harshit/Drive1/dev/Mailman/env/lib/python3.4/site-packages/lazr.config-2.1-py3.4.egg/lazr/config/_config.py", line 513, in __getattr__ raise AttributeError("No section or category named %s." % name) AttributeError: No section or category named db. It seems that the database layer hasn't got initialized by the time this event was fired. Is there any way of fixing this or any other way using which I can populate the database when mailman starts? Thanks, Harshit Bansal From barry at list.org Sat Jun 25 16:24:41 2016 From: barry at list.org (Barry Warsaw) Date: Sat, 25 Jun 2016 16:24:41 -0400 Subject: [Mailman-Developers] GSOC:Populating two default styles In-Reply-To: References: Message-ID: <20160625162441.342685d5.barry@wooz.org> On Jun 24, 2016, at 03:14 AM, Harshit Bansal wrote: >I was trying to populate the two default styles in the database. For >this purpose, I was trying to use ConfigurationUpdatedEvent as used in >the earlier style manager but I am recieving following error: [...] >AttributeError: No section or category named db. > >It seems that the database layer hasn't got initialized by the time >this event was fired. Is there any way of fixing this or any other way >using which I can populate the database when mailman starts? That's right. Getting the initialization sequence correct is pretty tricky, and it's true that the configuration subsystem is initialized before the database connection is created. If you think about it, it has to be this way since there are settings in the config files that control how the database is set up. What I think this means for you is that you can't use the ConfigurationUpdatedEvent to set up your stylets until the database is set up. Unfortunately, right now there aren't any events fired when initialization is completely, but just the other day Aurelien and I sketched out an event-based plugin mechanism for better HK integration, and one of the things I realized we needed was some additional event notification for the various stages of initialization. Right now, your best hope is to use the [mailman]post_hook which gets run during initialize_3() and is guaranteed to be run after both the configuration subsystem and the database connection is set up. Eventually the pre_hook and post_hook will be deprecated because they aren't really set up well for plugins, and we'll have events for the various phases of initialization, so you could register a handler for one of those events. If you decide to keep the ConfigurationUpdatedEvent handler (e.g. because you want to do other stylet things if the configuration changes once the system is up and running), just check for the existence of the config.db attribute, or catch the AttributeError. If the attribute is missing, you know the ConfigurationUpdatedEvent is too early for your purposes. Hope that helps. -Barry