From barry at list.org Wed Oct 8 17:40:16 2014 From: barry at list.org (Barry Warsaw) Date: Wed, 8 Oct 2014 11:40:16 -0400 Subject: [Mailman-Developers] Changes coming to the GNU Mailman wiki Message-ID: <20141008114016.7f09dd4a@anarchist.wooz.org> Hi folks. I'm happy to announce some changes to the GNU Mailman infrastructure that has been a long time coming. For many years, our wiki hosted at wiki.list.org has been running on a donated instance of Atlassian Confluence. We are extremely grateful to both Atlassian and the hosting provider Contegix for their support over the years. However, Confluence is not free software and because Mailman is a GNU project, it is incumbent on the project to utilize free software throughout our infrastructure. Thus I am here to announce the imminent switch of our wiki to a new Moin server. Huge, huge thanks go to Paul Boddie for the incredible amount of work he's put into the conversion process. Paul has been able to import most if not all of the content, along with most of the account names, into the new wiki. Because of the difference in organizing principles between Confluence and Moin, some post conversion gardening will be needed, and eventually we'd like to improve the style, but the conversion has really been done masterfully. Of course, passwords have not been imported into the new wiki, so if you were an approved author of the old wiki, you'll need to do a password reset to gain control of your new wiki account. However, the group permissions have been imported, so if you were able to edit the old wiki, you should be able to edit the new wiki. As always, contact mailman-cabal at python.org if you need assistance. We're doing the last few fiddly bits of the conversion, so you may see some down time while things fall into place. We still need to set up the SSL certificates and twiddle the DNS, so please be aware of this. I'll make another announcement when everything should be settled. In the meantime, I've disabled write permissions to the Confluence wiki. Huge thanks also go out to the Python Software Foundation and the PSF infrastructure team for donating the resources that the new wiki is running on. Thanks too to the other Mailpersons who have helped review the moin conversion process. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From stephen at xemacs.org Thu Oct 9 08:12:47 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 09 Oct 2014 15:12:47 +0900 Subject: [Mailman-Developers] Changes coming to the GNU Mailman wiki In-Reply-To: <20141008114016.7f09dd4a@anarchist.wooz.org> References: <20141008114016.7f09dd4a@anarchist.wooz.org> Message-ID: <87y4sppy8w.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > Thus I am here to announce the imminent switch of our wiki to a new > Moin server. Hurray! > Huge, huge thanks go to Paul Boddie for the incredible amount of > work he's put into the conversion process. Yes indeed! It's been a huge project, taking what, about a year and a half (IIRC it was discussed at PyCon Sprints in 2013)! Thanks again Paul! Steve From raj.abhilash1 at gmail.com Thu Oct 9 18:05:04 2014 From: raj.abhilash1 at gmail.com (Abhilash Raj) Date: Thu, 9 Oct 2014 21:35:04 +0530 Subject: [Mailman-Developers] Unofficial git mirror of GNU Mailman 3 Message-ID: Hi all, We have setup a unofficial git mirror[1] for GNU Mailman 3 on gitorious.org. I have added mailman-core and will shortly add postorius and mailman.client too. The mirrors would be updated daily from the official launchpad branch. This mirror is just for people who want to work personally on git, the work flow for contributing would remain same (atleast for now ) i.e. via pull request through launchpad. No pull requests would be accepted on gitorious. [1]: https://gitorious.org/mailman/mailman thanks, Abhilash Raj From tc at tobiasconradi.com Thu Oct 9 21:29:59 2014 From: tc at tobiasconradi.com (Tobias Conradi) Date: Thu, 9 Oct 2014 21:29:59 +0200 Subject: [Mailman-Developers] Hyperkitty - archive URLs - month with leading zero Message-ID: The demo server shows https://lists.stg.fedoraproject.org/archives/list/announce at lists.fedoraproject.org/2014/8/ The "8" seems to represent a month, but having a two digit month in the URL would bring it closer to the ISO 8601 format. Two digit month can be seen in various places: https://fedorahosted.org/hyperkitty/ shows URLs for blogs: http://blog.linuxgrrl.com/2010/03/16/a-rich-web-interface-for-mailing-lists/ http://blog.linuxgrrl.com/2012/02/29/7750-pixels-of-mailing-list-thread/ http://blog.linuxgrrl.com/2012/03/13/mailman-brainstorm/ http://blog.linuxgrrl.com/2012/03/14/mailman-brainstorm-2/ Same style is used by some blog hosting services, e.g. google for - wordpress.com 2014/01/01 - blogspot.com 2014/01 Some newspaper store it that way, google for - nytimes.com 2014/01/01 - washingtonpost.com 2014/01/01 -- Tobias Conradi Rheinsberger Str. 18 10115 Berlin Germany http://tobiasconradi.com From glibersat at unisson.co Wed Oct 15 12:46:57 2014 From: glibersat at unisson.co (Guillaume Libersat) Date: Wed, 15 Oct 2014 12:46:57 +0200 Subject: [Mailman-Developers] Translation Message-ID: <543E50A1.9070303@unisson.co> Hi, I'd like to start translating postorius in french, where should I start? I've cloned the repository and I'm ready to generate the catalogs. Can I just translate it and submit it somewhere? Thanks, Guillaume PS: Please cc, I'm not subscribed to the list -- Fabrique collective de biens communs -- http://unisson.co Faire un don : https://www.gittip.com/UnissonCo/ From stephen at xemacs.org Wed Oct 15 19:49:46 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 16 Oct 2014 02:49:46 +0900 Subject: [Mailman-Developers] Translation In-Reply-To: <543E50A1.9070303@unisson.co> References: <543E50A1.9070303@unisson.co> Message-ID: <87r3y9kyth.fsf@uwakimon.sk.tsukuba.ac.jp> Guillaume Libersat writes: > I'd like to start translating postorius in french, where should I start? > I've cloned the repository and I'm ready to generate the catalogs. Can I > just translate it and submit it somewhere? Thank you! The usual procedure is to set up an account on Launchpad, push your code to your account as a branch of the mainline, and make a merge request through Launchpad. You can also announce your branch and your progress here to get testing. Once you have a pretty significant fraction of the work done, you can announce on mailman-users, as that has many more subscribers and presumably a lot more French speakers, too. Steve From glibersat at sigill.org Wed Oct 15 23:15:44 2014 From: glibersat at sigill.org (Guillaume Libersat) Date: Wed, 15 Oct 2014 23:15:44 +0200 Subject: [Mailman-Developers] Translation In-Reply-To: <87r3y9kyth.fsf@uwakimon.sk.tsukuba.ac.jp> References: <543E50A1.9070303@unisson.co> <87r3y9kyth.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <543EE400.4050002@sigill.org> Hi Steve, Le 15/10/2014 19:49, Stephen J. Turnbull a ?crit : > The usual procedure is to set up an account on Launchpad, push your > code to your account as a branch of the mainline, and make a merge > request through Launchpad. You can also announce your branch and your > progress here to get testing. Once you have a pretty significant > fraction of the work done, you can announce on mailman-users, as that > has many more subscribers and presumably a lot more French speakers, > too. Ok, thanks for your advice! I've done exactly what you said, here's the beginning of the work: https://code.launchpad.net/~glibersat/postorius/i18n-french I hope to finish the translation in a few days. I've also added a few "trans" tags so more strings are now internationalized. Thanks! Guillaume PS: I'm now subscribed to this list. From stephen at xemacs.org Thu Oct 16 02:49:03 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 16 Oct 2014 09:49:03 +0900 Subject: [Mailman-Developers] Translation In-Reply-To: <543EE400.4050002@sigill.org> References: <543E50A1.9070303@unisson.co> <87r3y9kyth.fsf@uwakimon.sk.tsukuba.ac.jp> <543EE400.4050002@sigill.org> Message-ID: <87oatcltz4.fsf@uwakimon.sk.tsukuba.ac.jp> Guillaume Libersat writes: > Ok, thanks for your advice! I've done exactly what you said, here's the > beginning of the work: > https://code.launchpad.net/~glibersat/postorius/i18n-french Great! > I hope to finish the translation in a few days. I've also added a few > "trans" tags so more strings are now internationalized. AFAIK Mailman in the past has given free reign to the translators on existing translations, but adding new translations (if I understand what you mean by "adding 'trans' tags") is subject to review by other developers, and eventually Florian and Terri. I don't know if it applies to the Postorius code, but I know I've run into situations where it was undesirable to use the internationalization mechanism to translate certain strings. It is very helpful to the maintainers when reviewing and integrating if you make separate commits for the translations themselves, and for changes to the originals. It's even better if you keep separate kinds of changes in separate branches. (This is very cheap for Launchpad; they use "stacked branches" to minimize the work done to create a new branch.) It occurs to me that if you are adding new translations, it might be a good idea even to do a separate commit for each one, including both the markup at the original string and the French translation. Note that I'm not a Postorius maintainer, nor the project lead, and I've also not done translation work myself. So I don't know how much burden these guidelines put on you. They aren't "rules" (at least not until Florian or Terri says so!), but it would be helpful to the project if you consider using them in your work to make the integration side go more smoothly. Thank you for your consideration, and thanks again for the translations! From dkg at fifthhorseman.net Mon Oct 27 14:59:49 2014 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Mon, 27 Oct 2014 09:59:49 -0400 Subject: [Mailman-Developers] Mailman introducing spurious References: or In-Reply-To: headers? Message-ID: <87ioj5wr3e.fsf@alice.fifthhorseman.net> hi mailman folks-- over on dns-privacy at ietf.org, one of the participants (Hosnieh Rafiee, cc'ed here) suggests that mailman appears to be introducing spurious References: and In-Reply-To: headers (see the attached message below for some of the discussion. Can you confirm whether this is a function of mailman (e.g. to try to merge threads between people whose MUAs don't properly set In-Reply-To: or References:? If so, is there anything we can do to avoid it happening in the future? I believe the ietf.org mailman installation is running 2.1.15, fwiw. Regards, --dkg -------------- next part -------------- An embedded message was scrubbed... From: Hosnieh Rafiee Subject: Re: [dns-privacy] What about CGA-TSIG as a solution for DNS privacy? Date: Mon, 27 Oct 2014 10:12:39 +0000 Size: 6745 URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 948 bytes Desc: not available URL: From aurelien at bompard.org Mon Oct 27 16:30:40 2014 From: aurelien at bompard.org (Aurelien Bompard) Date: Mon, 27 Oct 2014 16:30:40 +0100 Subject: [Mailman-Developers] Hyperkitty - archive URLs - month with leading zero In-Reply-To: References: Message-ID: Hey Tobias, Sorry for not replying to your email but I did read it. having a two digit month in the URL would bring it closer to the ISO 8601 > format. > Using leading zeros in URLs is something I had already noticed and wanted to do, but I guess I never wrote it down when I thought about it and thus, forgot it ;-) FYI it's already possible to use leading zeros in the URLs, they are just not generated that way by the UI. I'll make sure I write it down this time, thanks for your email. Aur?lien From aurelien at bompard.org Mon Oct 27 16:36:37 2014 From: aurelien at bompard.org (Aurelien Bompard) Date: Mon, 27 Oct 2014 16:36:37 +0100 Subject: [Mailman-Developers] mailman.client and UUIDs for user.user_id Message-ID: Hey folks, Mailman stores user_ids as UUIDs in the database, and they are converted on the fly to integers by the REST API. However, the mailmanclient library do not convert them back to UUID. Do you think this is something that it should do, or should I handle this conversion in my own code? FYI, I'm attaching a very simple patch which does this conversion in mailmanclient (4 lines changed). Aur?lien -------------- next part -------------- A non-text attachment was scrubbed... Name: mmc.patch Type: text/x-patch Size: 1082 bytes Desc: not available URL: From mark at msapiro.net Mon Oct 27 19:33:56 2014 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 27 Oct 2014 11:33:56 -0700 Subject: [Mailman-Developers] Mailman introducing spurious References: or In-Reply-To: headers? In-Reply-To: <87ioj5wr3e.fsf@alice.fifthhorseman.net> References: <87ioj5wr3e.fsf@alice.fifthhorseman.net> Message-ID: <544E9014.2040909@msapiro.net> On 10/27/2014 06:59 AM, Daniel Kahn Gillmor wrote: > > over on dns-privacy at ietf.org, one of the participants (Hosnieh Rafiee, > cc'ed here) suggests that mailman appears to be introducing spurious > References: and In-Reply-To: headers (see the attached message below for > some of the discussion. > > Can you confirm whether this is a function of mailman (e.g. to try to > merge threads between people whose MUAs don't properly set In-Reply-To: > or References:? If so, is there anything we can do to avoid it > happening in the future? Mailman does not do this. Mailman, at least standard GNU Mailman, does not alter References: or In-Reply-To: headers in any way. Mailman's pipermail archiver makes use of these headers in determining threading in its archives, but it neither adds nor removes them or anything from them and anything it does has no effect on delivered messages in any case. The only thing in Mailman which manipulates these headers is the DMARC mitigation wrap message option introduced in Mailman 2.1.16 which merely copies these if they exist from the wrapped message to the outer wrapping message. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From dkg at fifthhorseman.net Mon Oct 27 19:41:33 2014 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Mon, 27 Oct 2014 14:41:33 -0400 Subject: [Mailman-Developers] Mailman introducing spurious References: or In-Reply-To: headers? In-Reply-To: <544E9014.2040909@msapiro.net> References: <87ioj5wr3e.fsf@alice.fifthhorseman.net> <544E9014.2040909@msapiro.net> Message-ID: <544E91DD.3010401@fifthhorseman.net> On 10/27/2014 02:33 PM, Mark Sapiro wrote: > On 10/27/2014 06:59 AM, Daniel Kahn Gillmor wrote: >> >> over on dns-privacy at ietf.org, one of the participants (Hosnieh Rafiee, >> cc'ed here) suggests that mailman appears to be introducing spurious >> References: and In-Reply-To: headers (see the attached message below for >> some of the discussion. >> >> Can you confirm whether this is a function of mailman (e.g. to try to >> merge threads between people whose MUAs don't properly set In-Reply-To: >> or References:? If so, is there anything we can do to avoid it >> happening in the future? > > Mailman does not do this. Mailman, at least standard GNU Mailman, does > not alter References: or In-Reply-To: headers in any way. Thanks for the followup, Mark. --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 949 bytes Desc: OpenPGP digital signature URL: From mark at msapiro.net Tue Oct 28 02:01:36 2014 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 27 Oct 2014 18:01:36 -0700 Subject: [Mailman-Developers] Mailman introducing spurious References: or In-Reply-To: headers? In-Reply-To: <544E9014.2040909@msapiro.net> References: <87ioj5wr3e.fsf@alice.fifthhorseman.net> <544E9014.2040909@msapiro.net> Message-ID: <544EEAF0.4050703@msapiro.net> On 10/27/2014 11:33 AM, Mark Sapiro wrote: > Mailman's pipermail archiver makes use of these headers in determining > threading in its archives, but it neither adds nor removes them or > anything from them and anything it does has no effect on delivered > messages in any case. I subsequently noticed in the message attached to the original post in this thread On 10/27/2014 03:12 AM, Hosnieh Rafiee wrote: ... > No not always. I saw a lot of such misleading information by mailman before in other groups as well. > > It appears that mailman uses a fuzzy approach and if the subject line has any similar words to the subject line of the previous thread, it categorized it under the same thread. > > As I said I have started a completely new message. So the header of my message was quite new and not carry any information for the old thread. > > For instance this one is another example of such problem. Y > [dns-privacy] Authenticating the resolver, Paul Hoffman > Re: [dns-privacy] Authenticating the resolver, Wes Hardaker > Re: [dns-privacy] Authenticating the resolver, Paul Hoffman As I wrote, Mailman makes use of In-Reply-To: and References: headers in determining threading in pipermail archives. It also does some Subject: header matching to augment threading decisions although not nearly as cavalier as "any similar words to the subject line of the previous thread", so unrelated posts can be threaded together in the archives, but this most often if not always occurs when something is posted as a reply to an unrelated post and thus is sent to Mailman with In-Reply-To: and/or References: referring to the unrelated thread. In any case, as I said previously, Mailman does not ever add In-Reply-To: or References: headers to messages which it delivers. If Hosnieh Rafiee's statement "I have started a completely new message. So the header of my message was quite new and not carry any information for the old thread." is correct and if his delivered post contained In-Reply-To: and References: headers referencing the old thread, I can only imagine that perhaps his own MUA was responsible in some way or possibly some non-Mailman process or Mailman modification is involved on the Mailman host that added them, but I'm certain that standard Mailman does not. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From tc at tobiasconradi.com Tue Oct 28 02:41:44 2014 From: tc at tobiasconradi.com (Tobias Conradi) Date: Tue, 28 Oct 2014 02:41:44 +0100 Subject: [Mailman-Developers] Hyperkitty - archive URLs - month with leading zero In-Reply-To: References: Message-ID: On Mon, Oct 27, 2014 at 4:30 PM, Aurelien Bompard wrote: > Using leading zeros in URLs is something I had already noticed and wanted to > do, but I guess I never wrote it down when I thought about it and thus, > forgot it ;-) > FYI it's already possible to use leading zeros in the URLs, they are just > not generated that way by the UI. > I'll make sure I write it down this time, thanks for your email. Aur?lien, thanks a lot. Waiting to see the UI change. It's several years now, that I wait for YYYYMM for mailman archives. I think many people will benefit from having it. Thanks a lot for all the work with programming the software! Tobias -- Tobias Conradi Rheinsberger Str. 18 10115 Berlin Germany http://tobiasconradi.com From Richard at Damon-Family.org Tue Oct 28 02:42:04 2014 From: Richard at Damon-Family.org (Richard Damon) Date: Mon, 27 Oct 2014 18:42:04 -0700 Subject: [Mailman-Developers] Mailman introducing spurious References: or In-Reply-To: headers? In-Reply-To: <544EEAF0.4050703@msapiro.net> References: <87ioj5wr3e.fsf@alice.fifthhorseman.net> <544E9014.2040909@msapiro.net> <544EEAF0.4050703@msapiro.net> Message-ID: <544EF46C.4000203@Damon-Family.org> On 10/27/14, 6:01 PM, Mark Sapiro wrote: > If Hosnieh Rafiee's statement "I have started a completely new message. > So the header of my message was quite new and not carry any information > for the old thread." is correct and if his delivered post contained > In-Reply-To: and References: headers referencing the old thread, I can > only imagine that perhaps his own MUA was responsible in some way or > possibly some non-Mailman process or Mailman modification is involved on > the Mailman host that added them, but I'm certain that standard Mailman > does not. My take on a statement like that (with a result that it threads under some other message) was that the poster took an unrelated message, and pressed "Reply" to get the list address in the To: line, and then deleted everything, and is just assume that be deleting the message and changing the subject that "of course" the system should know it is a new message. I have seen this done many times by people who will claim they started a new message (and many time it still contain the old message because their system hid quoted message almost completely from them). -- Richard Damon From stephen at xemacs.org Tue Oct 28 07:54:08 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 28 Oct 2014 15:54:08 +0900 Subject: [Mailman-Developers] Mailman introducing spurious References: or In-Reply-To: headers? In-Reply-To: <544EEAF0.4050703@msapiro.net> References: <87ioj5wr3e.fsf@alice.fifthhorseman.net> <544E9014.2040909@msapiro.net> <544EEAF0.4050703@msapiro.net> Message-ID: <87mw8g4rcf.fsf@uwakimon.sk.tsukuba.ac.jp> Mark Sapiro writes: > On 10/27/2014 11:33 AM, Mark Sapiro wrote: Thanks, Mark. You saved me a lot of words. > On 10/27/2014 03:12 AM, Hosnieh Rafiee wrote: > ... > > No not always. I saw a lot of such misleading information by > > mailman before in other groups as well. First I've heard of "misleading misinformation" (and I care a lot about threading; I would notice and follow up). > > For instance this one is another example of such problem. Y > > [dns-privacy] Authenticating the resolver, Paul Hoffman > > Re: [dns-privacy] Authenticating the resolver, Wes Hardaker > > Re: [dns-privacy] Authenticating the resolver, Paul Hoffman This appears to be a cut and paste of the archived mail from a browser, so it would be a Pipermail issue (note that the IETF's pipermail seems to be modified, though). Still, I don't see *any* problem there, and there are no threading header fields presented, so there's no way to diagnose. > As I wrote, Mailman makes use of In-Reply-To: and References: > headers in determining threading in pipermail archives. It also > does some Subject: header matching to augment threading decisions Mark's description here is somewhat ambiguous. First, to specifically describe header matching, it removes well-known prefixes (Re:, Fwd:, a few others) and list tags/serial numbers when enclosed in square brackets, then trims leading and trailing space. The result must match exactly. Second, "augment" does not mean "add". Pipermail does *not* "add the similar subjects to the thread." What it does do is group threads with the same subject (after trimming as above) together, and then sort thread groups by date. Conceptually, each individual thread has a separate root. This behavior is strongly preferred by users precisely because the exact match described above is usually due to a user who cut and pasted headers or whose MUA doesn't add reference headers. It's in http://www.jwz.org/doc/threading.html by Jamie Zawinski, the author of Netscape's threading code, and I believe that algorithm was adopted by RFC 5256 for IMAP. I suspect that this is what Hosnieh Rafiee is seeing: separate threads grouped by subject, and appearing to be a single thread because he expects strict sorting by date, and therefore the same-subject threads should appear together only by chance of very close dates. From hosnieh.rafiee at huawei.com Tue Oct 28 08:47:06 2014 From: hosnieh.rafiee at huawei.com (Hosnieh Rafiee) Date: Tue, 28 Oct 2014 07:47:06 +0000 Subject: [Mailman-Developers] Mailman introducing spurious References: or In-Reply-To: headers? In-Reply-To: <544EEAF0.4050703@msapiro.net> References: <87ioj5wr3e.fsf@alice.fifthhorseman.net> <544E9014.2040909@msapiro.net> <544EEAF0.4050703@msapiro.net> Message-ID: <814D0BFB77D95844A01CA29B44CBF8A7A4DAE6@lhreml513-mbb.china.huawei.com> Hi Mark, I have tried even to forward and reply (change the sender to myself) of another message to myself. The header did not contain any reply to. Whatever happens is in IETF part and not the MUA in my side. But in any case, I have started a new message. My message did not have any similarities to the other message except, it might be a word "privacy" in subject or content. Besides that there was no similarity. Best, Hosnieh > -----Original Message----- > From: Mark Sapiro [mailto:mark at msapiro.net] > Sent: Tuesday, October 28, 2014 2:02 AM > To: Daniel Kahn Gillmor; mailman developers list > Cc: Hosnieh Rafiee > Subject: Re: [Mailman-Developers] Mailman introducing spurious > References: or In-Reply-To: headers? > > On 10/27/2014 11:33 AM, Mark Sapiro wrote: > > > Mailman's pipermail archiver makes use of these headers in > determining > > threading in its archives, but it neither adds nor removes them or > > anything from them and anything it does has no effect on delivered > > messages in any case. > > > I subsequently noticed in the message attached to the original post in > this thread > > On 10/27/2014 03:12 AM, Hosnieh Rafiee wrote: > ... > > No not always. I saw a lot of such misleading information by mailman > before in other groups as well. > > > > It appears that mailman uses a fuzzy approach and if the subject line > has any similar words to the subject line of the previous thread, it > categorized it under the same thread. > > > > As I said I have started a completely new message. So the header of > my message was quite new and not carry any information for the old > thread. > > > > For instance this one is another example of such problem. Y > > [dns-privacy] Authenticating the resolver, Paul Hoffman > > Re: [dns-privacy] Authenticating the resolver, Wes Hardaker > > Re: [dns-privacy] Authenticating the resolver, Paul Hoffman > > > As I wrote, Mailman makes use of In-Reply-To: and References: headers > in determining threading in pipermail archives. It also does some > Subject: > header matching to augment threading decisions although not nearly as > cavalier as "any similar words to the subject line of the previous > thread", so unrelated posts can be threaded together in the archives, > but this most often if not always occurs when something is posted as a > reply to an unrelated post and thus is sent to Mailman with In-Reply-To: > and/or References: referring to the unrelated thread. > > In any case, as I said previously, Mailman does not ever add > In-Reply-To: or References: headers to messages which it delivers. > > If Hosnieh Rafiee's statement "I have started a completely new message. > So the header of my message was quite new and not carry any information > for the old thread." is correct and if his delivered post contained > In-Reply-To: and References: headers referencing the old thread, I can > only imagine that perhaps his own MUA was responsible in some way or > possibly some non-Mailman process or Mailman modification is involved > on the Mailman host that added them, but I'm certain that standard > Mailman does not. > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Tue Oct 28 18:26:56 2014 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 28 Oct 2014 10:26:56 -0700 Subject: [Mailman-Developers] Mailman introducing spurious References: or In-Reply-To: headers? In-Reply-To: <814D0BFB77D95844A01CA29B44CBF8A7A4DAE6@lhreml513-mbb.china.huawei.com> References: <87ioj5wr3e.fsf@alice.fifthhorseman.net> <544E9014.2040909@msapiro.net> <544EEAF0.4050703@msapiro.net> <814D0BFB77D95844A01CA29B44CBF8A7A4DAE6@lhreml513-mbb.china.huawei.com> Message-ID: <544FD1E0.3050702@msapiro.net> On 10/28/2014 12:47 AM, Hosnieh Rafiee wrote: > > I have tried even to forward and reply (change the sender to myself) of another message to myself. The header did not contain any reply to. Whatever happens is in IETF part and not the MUA in my side. > > But in any case, I have started a new message. My message did not have any similarities to the other message except, it might be a word "privacy" in subject or content. Besides that there was no similarity. Exactly what do you mean by "started a new message"? If you mean you selected 'new', 'write', 'mail', 'compose' or whatever you do in your mail client to compose a brand new message, then that message should be sent with no References: or In-Reply-To; headers. On the other hand, if you start with 'reply', 'reply-all', 'reply-list' or similar, even if you subsequently remove and replace everything in the visible message headers and body, the message you send is still a reply and will contain whatever References: and/or In-Reply-To; headers that your client normally includes in replies. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From flo.fuchs at gmail.com Wed Oct 29 14:08:21 2014 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Wed, 29 Oct 2014 14:08:21 +0100 Subject: [Mailman-Developers] mailman.client and UUIDs for user.user_id In-Reply-To: References: Message-ID: <5450E6C5.0@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Aur?lien, On 10/27/2014 04:36 PM, Aurelien Bompard wrote: > Hey folks, > > Mailman stores user_ids as UUIDs in the database, and they are > converted on the fly to integers by the REST API. However, the > mailmanclient library do not convert them back to UUID. Do you > think this is something that it should do, or should I handle this > conversion in my own code? I think mailmanclient should not expose a different value as the REST API does (the fact that it's an int representation of a uuid isn't exactly obvious from the outside, so a non-explicit conversion could potentially lead to some confusion). But how about we add a `uuid` property to the user object which exposes the original uuid value? Florian -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJUUObFAAoJEEceGbPdavl7GsgH/2PsrcCdwKCMcOWAqsci0XYN 458TrvIbgRolMMlW9RogMgcavNJUSaWwB9nSjjU3UV6Z2bCbRJEhBXSce6tNEbJF XZZ5Cdy8I7gGXhfmeP1/+enDSO+lB655KngXBZdhP1IeYME49z1yJnqH6bafsM1e 9YOecu0nVvjUFbpMwVdJVVPhcml0D/qVIcN0wD5ZXfO01m+bx2yTt2cuUkrj7SMt b756OQIcYIBJW3rqr+5kTr/FDCbsN1TrH921k8ZcgthfxMpmYeSwueeIizjFyf0H VAxa83l4uLyzK1LDtXZ4mkimMHIQhSvpGNSJdhPBVLW8oGomnqpPSdu/w/l5KtY= =Lyma -----END PGP SIGNATURE----- From aurelien at bompard.org Wed Oct 29 17:27:19 2014 From: aurelien at bompard.org (Aurelien Bompard) Date: Wed, 29 Oct 2014 17:27:19 +0100 Subject: [Mailman-Developers] mailman.client and UUIDs for user.user_id In-Reply-To: <5450E6C5.0@gmail.com> References: <5450E6C5.0@gmail.com> Message-ID: Hi Florian! I think mailmanclient should not expose a different value as the REST > API does (the fact that it's an int representation of a uuid isn't > exactly obvious from the outside, so a non-explicit conversion could > potentially lead to some confusion). > Hmm, to me the current situation is more confusing, since the user_id property I get from the REST API is different from the one I find in the database. I guess it depends on the reason the UUID is serialized. If it's to avoid being messed up during the transport, it's a transport issue and I think the library should convert it back (but the original ascii string should not be a problem so it's probably not that. Anyone remembers why the REST API exposes the int value instead of the string? It happened in commit 7043 but the reason is not given in the commit message. > But how about we add a `uuid` property to the user object which > exposes the original uuid value? > Not sure that's useful, converting it back to an UUID is easy enough ( uuid.UUID(int=value) ) A. From aurelien at bompard.org Wed Oct 29 17:43:58 2014 From: aurelien at bompard.org (Aurelien Bompard) Date: Wed, 29 Oct 2014 17:43:58 +0100 Subject: [Mailman-Developers] mailman.client and UUIDs for user.user_id In-Reply-To: <545117EC.8040100@gmail.com> References: <5450E6C5.0@gmail.com> <545117EC.8040100@gmail.com> Message-ID: > If the user_id value changes depending on whether > you are using the API directly via HTTP or through mailmanclient, that > might be hard to understand. > Ah, very good point. I'll do the conversion myself then I guess. A. From flo.fuchs at gmail.com Wed Oct 29 17:38:04 2014 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Wed, 29 Oct 2014 17:38:04 +0100 Subject: [Mailman-Developers] mailman.client and UUIDs for user.user_id In-Reply-To: References: <5450E6C5.0@gmail.com> Message-ID: <545117EC.8040100@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/29/2014 05:27 PM, Aurelien Bompard wrote: > Hi Florian! > > I think mailmanclient should not expose a different value as the > REST API does (the fact that it's an int representation of a uuid > isn't exactly obvious from the outside, so a non-explicit > conversion could potentially lead to some confusion). > > > Hmm, to me the current situation is more confusing, since the > user_id property I get from the REST API is different from the one > I find in the database. Yeah, I know what you mean. From a Mailman-developer's view that *is* confusing. But the view of non-MM dev, who is just a user of mailmanclient and/or the REST API (and doesn't know the innards of Mailman's database) might be different. If the user_id value changes depending on whether you are using the API directly via HTTP or through mailmanclient, that might be hard to understand. > I guess it depends on the reason the UUID is serialized. If it's > to avoid being messed up during the transport, it's a transport > issue and I think the library should convert it back (but the > original ascii string should not be a problem so it's probably not > that. Anyone remembers why the REST API exposes the int value > instead of the string? It happened in commit 7043 but the reason is > not given in the commit message. Good question. @Barry...? > But how about we add a `uuid` property to the user object which > exposes the original uuid value? > > > Not sure that's useful, converting it back to an UUID is easy > enough ( uuid.UUID(int=value) ) OK ;-) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJUURfsAAoJEEceGbPdavl7IxcIAKSp3XQdIW7jyzO67cqsf2+O Q7hXqttQdXCxYriu/BfihOrc+REVihMnE78Hb2JRq/mTc1Waa7wHIEQyM9SAUB+A 9y+oZb75GHrVPONNd89Xs5PODpyjOuFLrg+no4zHRr/CTrH6wgLrQC5EoNgjQsla dN25XAvVNGt2x9uJqLiaETwj7nE++wE8SdlhOlVT+PDZfktPPztCfO4n7TzV2BlU eTUNAxqMKaypd3IjpXpsY8tFUMQKJmilqaja2vXjk067KskPPD48C2CAcSybpWbC LOp8KWW3Uhxi+eHyLWvaqP24MbE92ic5TOmVVVK6926pPAs1uUM7PjR72Xk6dIA= =as2h -----END PGP SIGNATURE----- From barry at list.org Wed Oct 29 18:40:37 2014 From: barry at list.org (Barry Warsaw) Date: Wed, 29 Oct 2014 13:40:37 -0400 Subject: [Mailman-Developers] mailman.client and UUIDs for user.user_id In-Reply-To: References: <5450E6C5.0@gmail.com> Message-ID: <20141029134037.1e819ddc@anarchist.wooz.org> On Oct 29, 2014, at 05:27 PM, Aurelien Bompard wrote: >Hmm, to me the current situation is more confusing, since the user_id >property I get from the REST API is different from the one I find in the >database. It's really the same value, just a different representation of it. The conversion UUIDs to integers is completely reversible. >I guess it depends on the reason the UUID is serialized. If it's to avoid >being messed up during the transport, it's a transport issue and I think the >library should convert it back (but the original ascii string should not be a >problem so it's probably not that. Anyone remembers why the REST API exposes >the int value instead of the string? It happened in commit 7043 but the >reason is not given in the commit message. I don't remember exactly why I changed it to the int representation, but it could have been some limitation (valid or not) on restish. The log message doesn't describe that because IIRC, the REST API change happened at the same time that internally user ids were changed to UUIDs. So I probably tried a few things and settled on ints before I committed that change. I could have been trying to optimize the wire format, given that UUID.int is 16 bytes whereas UUID.hex is 32 bytes. Or maybe I was just being dumb. (Not mutually exclusive with the above. ;) We could certainly change the JSON for a user to return the string (specifically, UUID().hex) but we'd need to return both for backward compatibility, if we care about that. Maybe we don't if we're just promising for now that mailmanclient is the level of API guarantee, and we make both changes at the same time. I can probably figure out how to accept either ints or hex strings as input for identifying the user, but it might be tricky. So Aurelien and Florian, what do you think? Should we use the hex representation of the UUID and should I keep backward compatibility. Cheers, -Barry From glibersat at sigill.org Thu Oct 30 00:09:56 2014 From: glibersat at sigill.org (Guillaume Libersat) Date: Thu, 30 Oct 2014 00:09:56 +0100 Subject: [Mailman-Developers] Translation In-Reply-To: <87oatcltz4.fsf@uwakimon.sk.tsukuba.ac.jp> References: <543E50A1.9070303@unisson.co> <87r3y9kyth.fsf@uwakimon.sk.tsukuba.ac.jp> <543EE400.4050002@sigill.org> <87oatcltz4.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <545173C4.2090401@sigill.org> Hi, Le 16/10/2014 02:49, Stephen J. Turnbull a ?crit : > AFAIK Mailman in the past has given free reign to the translators on > existing translations, but adding new translations (if I understand > what you mean by "adding 'trans' tags") is subject to review by other > developers, and eventually Florian and Terri. I don't know if it > applies to the Postorius code, but I know I've run into situations > where it was undesirable to use the internationalization mechanism to > translate certain strings. > > It is very helpful to the maintainers when reviewing and integrating > if you make separate commits for the translations themselves, and for > changes to the originals. It's even better if you keep separate kinds > of changes in separate branches. (This is very cheap for Launchpad; > they use "stacked branches" to minimize the work done to create a new > branch.) > > It occurs to me that if you are adding new translations, it might be a > good idea even to do a separate commit for each one, including both > the markup at the original string and the French translation. > > Note that I'm not a Postorius maintainer, nor the project lead, and > I've also not done translation work myself. So I don't know how much > burden these guidelines put on you. They aren't "rules" (at least not > until Florian or Terri says so!), but it would be helpful to the > project if you consider using them in your work to make the > integration side go more smoothly. > > Thank you for your consideration, and thanks again for the > translations! > I've done exactly what you said (splitting commits between effective translation work and code/template instrumentation) and submitted a Merge Request there ( https://code.launchpad.net/~glibersat/postorius/i18n-french/+merge/240066 ). I hope it's gonna be merged soon :) Cheers, Guillaume -- http://sigill.org From vtico at yandex.com Wed Oct 29 18:32:20 2014 From: vtico at yandex.com (Vijay Tico) Date: Wed, 29 Oct 2014 21:02:20 +0330 Subject: [Mailman-Developers] mailman-bundler-3.0b1 error - Does it have an HTTPS login page? Message-ID: <545124A4.7080008@yandex.com> Hi, After installation of mailman-bundler-3.0b1 according to the instructions in README.rst and running the web interface, the web server runs and responds HTTP requests. The problem is that when I click the Login button on the top right of the page, I am redirected to this url: https://localhost:8000/archives/accounts/login The server naturally doesn't respond to this, as its an HTTPS request being sent to an HTTP server. I tried some hacks, but could not get it working (I don't know django at all). How can I solve this? Why has this happened? Bests From stephen at xemacs.org Thu Oct 30 04:47:40 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 30 Oct 2014 12:47:40 +0900 Subject: [Mailman-Developers] Translation In-Reply-To: <545173C4.2090401@sigill.org> References: <543E50A1.9070303@unisson.co> <87r3y9kyth.fsf@uwakimon.sk.tsukuba.ac.jp> <543EE400.4050002@sigill.org> <87oatcltz4.fsf@uwakimon.sk.tsukuba.ac.jp> <545173C4.2090401@sigill.org> Message-ID: <87ppda2p7n.fsf@uwakimon.sk.tsukuba.ac.jp> Guillaume Libersat writes: > I've done exactly what you said (splitting commits between effective > translation work and code/template instrumentation) and submitted a > Merge Request there ( > https://code.launchpad.net/~glibersat/postorius/i18n-french/+merge/240066 ). Thank you! My, that was fast! > I hope it's gonna be merged soon :) Me too, but I don't "own" that repo. Florian? Terri? Can you review soon? Steve From aurelien at bompard.org Thu Oct 30 16:08:30 2014 From: aurelien at bompard.org (Aurelien Bompard) Date: Thu, 30 Oct 2014 16:08:30 +0100 Subject: [Mailman-Developers] mailman.client and UUIDs for user.user_id In-Reply-To: <20141029134037.1e819ddc@anarchist.wooz.org> References: <5450E6C5.0@gmail.com> <20141029134037.1e819ddc@anarchist.wooz.org> Message-ID: > So Aurelien and Florian, what do you think? Should we use the hex > representation of the UUID and should I keep backward compatibility. > If we can't see any valid reason for converting to ints today, I would personally vote for not converting in the REST API, and for mailman.client to return the UUID object (not the hex string), because I think a python client library should expose python idoms and objects when possible. If we do that, keeping backwards compatibility could be useful indeed, and you can start from my change here: http://bazaar.launchpad.net/~abompard/mailman.client/uuids/revision/58 But FYI, I have already adapted my code in HyperKitty to do the conversion to UUIDs, so I really don't mind if we keep the status quo. Let's just do what's best in the long run. A. From flo.fuchs at gmail.com Thu Oct 30 21:13:01 2014 From: flo.fuchs at gmail.com (Florian Fuchs) Date: Thu, 30 Oct 2014 21:13:01 +0100 Subject: [Mailman-Developers] Translation In-Reply-To: <87ppda2p7n.fsf@uwakimon.sk.tsukuba.ac.jp> References: <543E50A1.9070303@unisson.co> <87r3y9kyth.fsf@uwakimon.sk.tsukuba.ac.jp> <543EE400.4050002@sigill.org> <87oatcltz4.fsf@uwakimon.sk.tsukuba.ac.jp> <545173C4.2090401@sigill.org> <87ppda2p7n.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <54529BCD.1070404@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/30/2014 04:47 AM, Stephen J. Turnbull wrote: > Guillaume Libersat writes: > >> I've done exactly what you said (splitting commits between >> effective translation work and code/template instrumentation) and >> submitted a Merge Request there ( >> https://code.launchpad.net/~glibersat/postorius/i18n-french/+merge/240066 >> ). > > Thank you! My, that was fast! +1 Thank you very much! > >> I hope it's gonna be merged soon :) > > Me too, but I don't "own" that repo. > > Florian? Terri? Can you review soon? Yes, I can do a review. I'll let you know once it's merged... Florian -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJUUpvNAAoJEEceGbPdavl7Qp0H/3+KUV0ebSg7H+UcJRuLac1J kOV1FeQrY4WwDevF4T21fKCGcCsrMpNIWqYIP457eovNkvmXIAJ6u0F7scmdQiNi 4WGmzFgdU+4g64lEzm9eJmzCg0t8GhdHUnPLWIFgh5k29ukrpU4UPZQvhFH7wBgl +7euN6BQUKYVIcfo8yKnU8OdVHg8BsOCuu0YuwZ7r1ckIK+rdaEtYktYnHkmc8Ek NM/s2AXiG4ppVAl0cgbG63ECpnYxy3ca0d9tAgYPWLm5db6lniOjLnOUEnW+hWeK DUf7xZRllu8yuapd86w2aXRNg1tfKQk0KldUKZWUK8pfUjVULvFcwjWSeFsDI7s= =PRo0 -----END PGP SIGNATURE----- From phil at ultimate.com Fri Oct 31 19:53:25 2014 From: phil at ultimate.com (Phil Budne) Date: Fri, 31 Oct 2014 14:53:25 -0400 Subject: [Mailman-Developers] Patches to mailman-2.1.18 to use FreeBSD "kqueue" Message-ID: <201410311853.s9VIrPFF078529@ultimate.com> This may be of greatest interest to FreeBSD & MacOS "port" maintainers.. I've been running these patches to mailman mailman-2.1.18.1 to use "kqueue" to watch the queue directories since August. As you can see the idle components don't accumulate any CPU time (they don't have to poll their queue directory): mail% ps axwww | grep mailman 1067 - IWs 0:00.00 /usr/local/bin/python2.7 /usr/local/mailman/bin/mailmanctl -s -q start 1068 - IW 0:00.00 /usr/local/bin/python2.7 /usr/local/mailman/bin/qrunner --runner=ArchRunner:0:1 -s 1069 - I 0:02.76 /usr/local/bin/python2.7 /usr/local/mailman/bin/qrunner --runner=BounceRunner:0:1 -s 1070 - IW 0:00.00 /usr/local/bin/python2.7 /usr/local/mailman/bin/qrunner --runner=CommandRunner:0:1 -s 1071 - I 0:02.85 /usr/local/bin/python2.7 /usr/local/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s 1072 - IW 0:00.00 /usr/local/bin/python2.7 /usr/local/mailman/bin/qrunner --runner=NewsRunner:0:1 -s 1073 - I 0:04.67 /usr/local/bin/python2.7 /usr/local/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s 1074 - IW 0:00.00 /usr/local/bin/python2.7 /usr/local/mailman/bin/qrunner --runner=VirginRunner:0:1 -s 1075 - IW 0:00.00 /usr/local/bin/python2.7 /usr/local/mailman/bin/qrunner --runner=RetryRunner:0:1 -s [this particular set of processes have been running since Oct 5th] It looks like the "inotify" facility on Linux might do the job, but it's not among Python's "included batteries" (yet) I did a little bit of jiggering to periodic wakeups, I can't remember at this point if it was all "strictly necessary".. I'd be happy to have the work picked up (but would appreciate credit for the initial work). --- Runner.py.orig 2014-07-11 15:01:26.000000000 -0400 +++ Runner.py 2014-08-15 11:24:02.000000000 -0400 @@ -15,9 +15,14 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. -"""Generic queue runner class. +"""Generic queue runner class. (hacked for FreeBSD by phil at ultimate.com) """ +USE_KQUEUE = True + +import os # for kqueue +import select # for kqueue +import errno # for kqueue import time import traceback from cStringIO import StringIO @@ -44,6 +49,7 @@ class Runner: QDIR = None SLEEPTIME = mm_cfg.QRUNNER_SLEEP_TIME + PERIODIC = 0 def __init__(self, slice=None, numslices=1): self._kids = {} @@ -53,6 +59,21 @@ # Create the shunt switchboard self._shunt = Switchboard(mm_cfg.SHUNTQUEUE_DIR) self._stop = False + # BEGIN phil at ultimate.com + self.kq = None + # See if BSD/OSX 'kqueue' can be used for dir watching: + if USE_KQUEUE and getattr(select, 'kqueue', None): + self._kqueue_init() + # Linux has inotify, but not among included batteries + # if other mechanisms implemented, should be hidden in a + # WatchDir() class, maybe should be in any case?! + + # pulled up from BounceMixin + if self.PERIODIC > 0: + self._nextaction = time.time() + self.PERIODIC + else: + self._nextaction = -1 + # END phil at ultimate.com def __repr__(self): return '<%s at %s>' % (self.__class__.__name__, id(self)) @@ -70,8 +91,8 @@ filecnt = self._oneloop() # Do the periodic work for the subclass. BAW: this # shouldn't be called here. There should be one more - # _doperiodic() call at the end of the _oneloop() loop. - self._doperiodic() + # _checkperiodic() call at the end of the _oneloop() loop. + self._checkperiodic() # If the stop flag is set, we're done. if self._stop: break @@ -146,7 +167,7 @@ self._switchboard.finish(filebase, preserve=True) # Other work we want to do each time through the loop Utils.reap(self._kids, once=True) - self._doperiodic() + self._checkperiodic() if self._shortcircuit(): break return len(files) @@ -243,15 +264,31 @@ """ raise NotImplementedError - def _doperiodic(self): - """Do some processing `every once in a while'. + # BEGIN phil at ultimate.com + def _checkperiodic(self): + """Called every once in a while both from the Runner's main loop, and + from the Runner's hash slice processing loop. To check if time + to run periodic processing. - Called every once in a while both from the Runner's main loop, and - from the Runner's hash slice processing loop. You can do whatever + If self.PERIODIC is positive, calls subclass _doperiodic() method + every self.PERIODIC seconds. + """ + if self.PERIODIC <= 0: + return + now = time.time() + if self._nextaction > now: + return + self._nextaction = now + self.PERIODIC + # Run the subclass _myperiodic method: + self._myperiodic() + + def _myperiodic(self): + """Do some processing `every once in a while' + (every self.PERIODIC seconds) You can do whatever special periodic processing you want here, and the return value is irrelevant. """ - pass + syslog('error', 'empty _myperiodic called') def _snooze(self, filecnt): """Sleep for a little while. @@ -263,8 +300,68 @@ """ if filecnt or self.SLEEPTIME <= 0: return + if USE_KQUEUE and self.kq: + self._kqueue_snooze() + return time.sleep(self.SLEEPTIME) + # BEGIN phil at ultimate.com + def _kqueue_init(self): + """Setup a kqueue fd listening for VNODE WRITE events in self.QDIR""" + self.kq = None + self.kq_dirfd = -1 + try: + self.kq = select.kqueue() + self.kq_dirfd = os.open(self.QDIR, os.O_RDONLY) + ev = select.kevent(self.kq_dirfd, + filter=select.KQ_FILTER_VNODE, + flags =select.KQ_EV_ADD|select.KQ_EV_CLEAR, + fflags=select.KQ_NOTE_WRITE) + # install ev, return no events, don't sleep + self.kq.control([ev], 0, 0) + return True + except Exception, e: + if self.kq: + self.kq.close() + self.kq = None + if self.kq_dirfd != -1: + os.close(self.kq_dirfd) + self.kq_dirfd = -1 + return False + + def _kqueue_snooze(self): + """Sleep until activity in self.QDIR + or time to call _myperiodic() + """ + wait = None # default to indefinite + if self.PERIODIC > 0 and self._nextaction > 0: + wait = self._nextaction - time.time() + # periodic processing overdue, just return + if wait < 0: + return + if USE_KQUEUE and self.kq: + try: + syslog('phil', 'calling kq.control wait=%s', wait) + # no changes, return at most 10 events (should only return 1) + events = self.kq.control([], 10, wait) + syslog('phil', 'kq.control returned %d', len(events)) + # ignore events!! + time.sleep(5.0) # TEMP!!!! + return + except KeyboardInterrupt: + raise + except OSError, exc: + # suppress EINTR, like time.sleep + if exc.errno == errno.EINTR: + return + syslog('phil', 'kq.control exception: %s', exc) + #raise + # fall thru..... + if wait is None or wait > self.SLEEPTIME: + wait = self.SLEEPTIME + time.sleep(wait) + # END phil at ultimate.com + def _shortcircuit(self): """Return a true value if the individual file processing loop should exit before it's finished processing each message in the current slice --- BounceRunner.py.orig 2014-07-11 15:01:26.000000000 -0400 +++ BounceRunner.py 2014-08-14 09:55:00.000000000 -0400 @@ -46,8 +46,18 @@ -class BounceMixin: - def __init__(self): +# phil at ultimate.com: the only users of BounceMixin +# used "Runner" as a base, so refactoring as BounceRunnerBase +# to avoid MRO end-runs (and added pain for PERIODIC), +# seems to require fewer explicit parent class calls +# (__init__, _cleanup) too! + +class BounceRunnerBase(Runner): + PERIODIC = mm_cfg.REGISTER_BOUNCES_EVERY + + def __init__(self, slice=None, numslices=1): + Runner.__init__(self, slice, numslices) + # Registering a bounce means acquiring the list lock, and it would be # too expensive to do this for each message. Instead, each bounce # runner maintains an event log which is essentially a file with @@ -84,7 +94,6 @@ mm_cfg.DATA_DIR, 'bounce-events-%05d.pck' % os.getpid()) self._bounce_events_fp = None self._bouncecnt = 0 - self._nextaction = time.time() + mm_cfg.REGISTER_BOUNCES_EVERY def _queue_bounces(self, listname, addrs, msg): today = time.localtime()[:3] @@ -135,14 +144,12 @@ def _cleanup(self): if self._bouncecnt > 0: self._register_bounces() + Runner._cleanup(self) - def _doperiodic(self): - now = time.time() - if self._nextaction > now or self._bouncecnt == 0: - return - # Let's go ahead and register the bounces we've got stored up - self._nextaction = now + mm_cfg.REGISTER_BOUNCES_EVERY - self._register_bounces() + def _myperiodic(self): + """called every self.PERIODIC seconds""" + if self._bouncecnt > 0: + self._register_bounces() def _probe_bounce(self, mlist, token): locked = mlist.Locked() @@ -161,13 +168,9 @@ -class BounceRunner(Runner, BounceMixin): +class BounceRunner(BounceRunnerBase): QDIR = mm_cfg.BOUNCEQUEUE_DIR - def __init__(self, slice=None, numslices=1): - Runner.__init__(self, slice, numslices) - BounceMixin.__init__(self) - def _dispose(self, mlist, msg, msgdata): # Make sure we have the most up-to-date state mlist.Load() @@ -258,14 +261,6 @@ # addrs = filter(None, addrs) # MAS above filter moved up so we don't try to queue an empty list. self._queue_bounces(mlist.internal_name(), addrs, msg) - - _doperiodic = BounceMixin._doperiodic - - def _cleanup(self): - BounceMixin._cleanup(self) - Runner._cleanup(self) - - def verp_bounce(mlist, msg): bmailbox, bdomain = Utils.ParseEmail(mlist.GetBouncesEmail()) --- OutgoingRunner.py.orig 2014-07-11 15:01:26.000000000 -0400 +++ OutgoingRunner.py 2014-07-26 11:42:18.000000000 -0400 @@ -31,12 +31,12 @@ from Mailman import LockFile from Mailman.Queue.Runner import Runner from Mailman.Queue.Switchboard import Switchboard -from Mailman.Queue.BounceRunner import BounceMixin +from Mailman.Queue.BounceRunner import BounceRunnerBase from Mailman.Logging.Syslog import syslog # This controls how often _doperiodic() will try to deal with deferred # permanent failures. It is a count of calls to _doperiodic() -DEAL_WITH_PERMFAILURES_EVERY = 10 +DEAL_WITH_PERMFAILURES_EVERY = 10 # PLB not used? try: True, False @@ -46,12 +46,11 @@ -class OutgoingRunner(Runner, BounceMixin): +class OutgoingRunner(BounceRunnerBase): QDIR = mm_cfg.OUTQUEUE_DIR def __init__(self, slice=None, numslices=1): - Runner.__init__(self, slice, numslices) - BounceMixin.__init__(self) + BounceRunnerBase.__init__(self, slice, numslices) # We look this function up only at startup time modname = 'Mailman.Handlers.' + mm_cfg.DELIVERY_MODULE mod = __import__(modname) @@ -131,9 +130,3 @@ self.__retryq.enqueue(msg, msgdata) # We've successfully completed handling of this message return False - - _doperiodic = BounceMixin._doperiodic - - def _cleanup(self): - BounceMixin._cleanup(self) - Runner._cleanup(self) From barry at list.org Fri Oct 31 21:28:19 2014 From: barry at list.org (Barry Warsaw) Date: Fri, 31 Oct 2014 16:28:19 -0400 Subject: [Mailman-Developers] Patches to mailman-2.1.18 to use FreeBSD "kqueue" In-Reply-To: <201410311853.s9VIrPFF078529@ultimate.com> References: <201410311853.s9VIrPFF078529@ultimate.com> Message-ID: <20141031162819.7ba64d11@anarchist.wooz.org> On Oct 31, 2014, at 02:53 PM, Phil Budne wrote: >This may be of greatest interest to FreeBSD & MacOS "port" >maintainers.. I've been running these patches to mailman >mailman-2.1.18.1 to use "kqueue" to watch the queue directories since >August. > >It looks like the "inotify" facility on Linux might do the job, but >it's not among Python's "included batteries" (yet) I'd be supportive of a patch to MM3 to support kqueue/inotify. There appear to be several inotify modules available on PyPI, and even at least two in Debian (with pyinotify looking to have both Python 2 and 3 support). Other than that, I don't know which one is "better". Would you be interested in putting together a merge proposal for MM3? Cheers, -Barry