From rocky at panix.com Sun Jan 1 00:40:56 2006 From: rocky at panix.com (R. Bernstein) Date: Sat, 31 Dec 2005 18:40:56 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator Message-ID: <17335.5896.569912.550753@panix3.panix.com> Note: the basic info of the below feature request has been posted here: http://sourceforge.net/tracker/index.php?func=detail&aid=1394592&group_id=103&atid=350103 I received an weird but interesting weird email the other day that got me thinking about moderation of GNU mailing lists. Here's a paraphrase the email: To: Moderator Slacker Subject: Bug-slacker-project post from xxx at gnu.org requires approval As list administrator, your authorization is requested ... [you know rest] [mail message] Dear Slacker: We've rather belatedly realized that a you've been ignoring the moderation messages of your mailing list which now has hundreds or thousands of held messages and/or a whole lot of spam in the archives. If you would like to volunteer to take out the trash for this or *any* of the 25 or so lists mentioned below, please email me. Although somehow I don't think there are going to be too many takers of this fabulous offer, the email does have I think one useful idea in it. Basically for any slacker-moderated list, it's probably okay to let just about anyone do the moderation. After all, just about any human can easily detect spam from legitimate posts. As someone who's been doing GPL projects for a long time, things have been getting tougher. In the good old days, one gave an email address for support and the email address was used for support on the project. Nowadays the email address is used to send viruses, spam and phishing requests, and to use as the return address for viruses, and spam to others. Okay. So now this moderation thing on mailing lists was then added. That then gave the person offering support the additional burden to act as a trash man for the list (in addition to his/her own personal increased spam/viruses). The thing that always struck me as wrong about moderated lists is that for the convenience of the poster --- and often this is someone who is a novice asking for help (and sometimes in a not even in a very respectful way) -- burden is usually put on the people who might be able to help. I think of this as the N to one burden problem: a little burden (by not having to register to post) is reduced on N people (often novices), at the expense of adding N times extra burden on the "moderator(s)" (someone who is often an expert and is 1 in number). It strikes me as a poor use of the expert's time. Actually, I'm probably not the only one who feels that way, hence the result cited above. So it might be nice to have a box or flag for such a mailing list that allows anyone who is registered in the mailing list have the pleasure of doing email moderation. I suppose this could be subject for abuse too (discard valid posts and accept spam), but I have a feeling that to first order approximaton this would be a big help. And doesn't mailman already have ways of watching users or moderators, and revoking moderation by the administrator or whatnot? Thanks for considering this. From brad at stop.mail-abuse.org Sun Jan 1 01:27:36 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Sat, 31 Dec 2005 18:27:36 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <17335.5896.569912.550753@panix3.panix.com> References: <17335.5896.569912.550753@panix3.panix.com> Message-ID: At 6:40 PM -0500 2005-12-31, R. Bernstein wrote: > So it might be nice to have a box or flag for such a mailing list that > allows anyone who is registered in the mailing list have the pleasure > of doing email moderation. You can list as many moderators for a list as you like. But there's a problem with multiple moderators, one that we have on the mailman-users and mailman-developers lists ourselves -- in addition to many other lists hosted on python.org. In short, the problem is getting all the moderators to follow the same moderation policy. Even if you have agreed on a moderation policy, there is still a certain amount of judgement required, and Barry might feel one way regarding a given post, JC might feel a different way, and I might be somewhere on the fence -- or any other combination of the various people involved. And that's when we all agree on the policy that should be implemented, or remember what it was that we all agreed on several months ago. > I suppose this could be subject for abuse too (discard valid posts and > accept spam), but I have a feeling that to first order approximaton > this would be a big help. And doesn't mailman already have ways of > watching users or moderators, and revoking moderation by the > administrator or whatnot? No, there is no monitoring of the moderators -- within their limited set of actions they are allowed to perform, their actions are absolute, and for the most part are not reversible -- once a message is discarded, it is gone and there's nothing you can do to get it back. The same is true of the list administrators. You could always re-subscribe someone if they've been unsubscribed by someone else, but that's about it. The kind of monitoring you're talking about would add significant additional load on the system, and would force the administrators to do a lot more work to keep checking up on the moderators, etc.... This would further concentrate the workload on an even smaller group of people, which I think is precisely the sort of thing you were trying to eliminate. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From lists05 at equinephotoart.com Sun Jan 1 03:56:10 2006 From: lists05 at equinephotoart.com (JC Dill) Date: Sat, 31 Dec 2005 18:56:10 -0800 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: References: <17335.5896.569912.550753@panix3.panix.com> Message-ID: <43B744CA.7090607@equinephotoart.com> Brad Knowles wrote: > But there's a problem with multiple moderators, one that we have > on the mailman-users and mailman-developers lists ourselves -- in > addition to many other lists hosted on python.org. In short, the > problem is getting all the moderators to follow the same moderation > policy. > > Even if you have agreed on a moderation policy, there is still a > certain amount of judgement required, and Barry might feel one way > regarding a given post, JC might feel a different way, and I might be > somewhere on the fence -- or any other combination of the various > people involved. That's a very interesting and accurate observation. In fact, the moderated post that started this thread is one that I don't think I would have approved for posting to this list! I felt mildly (but not strongly) that this was a discussion that should probably take place on -users first, because it's a discussion about the possibility of changing the way the mailman list is used (by it's users) rather than a discussion about "developing" a new feature, per se. But, it's not totally off-topic for -dev so Brad is also "correct" for approving it. The main problem I think Rocky is experiencing is the problem of absent moderators, period. Rather than some automated method of turning the moderator tasks over to others, I suggest that a better way is to more closely oversee pending moderator tasks so that the list owner and the list server administrator receive notices when a moderator's queue has not been recently attended to, and address the lack of moderation while the queue is still small and relatively fresh. To this end, I suggest a list server setting and a per-list setting for sending email notices (to the list server admin and to the list owner, respectively) daily, listing the number and type of stale moderation requests for each list. My suggestion is that the default state of this setting be set to "on" and it can be toggled off as desired, and that the default is to start sending these emails once daily when there are pending moderator items in the queue that are older than 3 days (72 hours). List server admins and list owners could then adjust these settings based on their list traffic etc. It might also be useful to provide a cc field for the list-owner setting so that when the owner is going to be absent from list management duties this message (and all other messages that go to the list owner) could be sent to a co-owner, and to allow the owner address to be set to nomail when there's a co-owner address entered. jc From brad at stop.mail-abuse.org Sun Jan 1 05:22:58 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Sat, 31 Dec 2005 22:22:58 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <43B744CA.7090607@equinephotoart.com> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> Message-ID: At 6:56 PM -0800 2005-12-31, JC Dill wrote: > That's a very interesting and accurate observation. In fact, the > moderated post that started this thread is one that I don't think I > would have approved for posting to this list! I felt mildly (but not > strongly) that this was a discussion that should probably take place on > -users first, because it's a discussion about the possibility of > changing the way the mailman list is used (by it's users) rather than a > discussion about "developing" a new feature, per se. But, it's not > totally off-topic for -dev so Brad is also "correct" for approving it. Okay, now that is truly weird. I thought it was kind of off-topic myself, but I thought that it would be one that either you or Barry would have approved of, so I approved it on that basis. I guess that just goes to show that you shouldn't over-think the process too much. ;) > The main problem I think Rocky is experiencing is the problem of absent > moderators, period. Rather than some automated method of turning the > moderator tasks over to others, I suggest that a better way is to more > closely oversee pending moderator tasks so that the list owner and the > list server administrator receive notices when a moderator's queue has > not been recently attended to, and address the lack of moderation while > the queue is still small and relatively fresh. We're certainly seeing some issues of absentee moderators on some of the lists at python.org, where my new version of the "mmdsr" script (see ) is showing that some lists have as many as 100 messages waiting in the queue to be moderated, and some of those messages date back to May of 2005. I think that this is a problem that needs to be addressed within the Mailman package, and not just something that can be observed externally through tools like "mmdsr". However, I am not yet sure what would be the best way to resolve this issue. I would like to see more discussion on that topic, although I'm not sure that mailman-developers is the best place to do that. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From rocky at panix.com Sun Jan 1 02:11:21 2006 From: rocky at panix.com (R. Bernstein) Date: Sat, 31 Dec 2005 20:11:21 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: References: <17335.5896.569912.550753@panix3.panix.com> Message-ID: <17335.11321.968871.894940@panix3.panix.com> Brad Knowles writes: > You can list as many moderators for a list as you like. Fine. One just needs a way for people who are members of a list to be able to volunteer to be a moderator. > But there's a problem with multiple moderators, one that we have > on the mailman-users and mailman-developers lists ourselves -- in > addition to many other lists hosted on python.org. In short, the > problem is getting all the moderators to follow the same moderation > policy. You are way too sophisticated. Again, we are talking about general help, info, and user lists. The policy for all of these lists is simple: if you think it's spam, discard. If not, accept. So now let's go through the false-positive cases: 1. A post is not spam but is discarded. Not a big deal. For whatever reason the poster didn't manage to convince the moderator it was not spam. The poster can join the list, or try again and maybe another moderator will approve. This is an improvement on the current situation cited where no posts go through. 2. A post is spam and let through. Still not a big deal. Recent versions allow for deleting archive messages. Still there is the annoyance to others on the list for getting unwanted spam. An obvious solution for someone annoyed would be to volunteer to do some (or more) of the moderation of it. Or drop out of the list. Still better than what happens now, which is basically no posts get through and most certainly those that are spam-infested either don't have any members or maybe they have their own spam filters. > No, there is no monitoring of the moderators ... > The kind of monitoring you're talking about would add significant > additional load ... Okay. If nothing simple can be done, so be it. Personally, I think just adding a button that allows a registered user to be a moderator would grealy improve the situation described previously. The problem of getting humans to do moderation is not theoretical, but real. And basically I think that a self-moderated system by users would largely work. To some extent, I think this is proven by wikis. From rocky at panix.com Sun Jan 1 07:50:56 2006 From: rocky at panix.com (R. Bernstein) Date: Sun, 1 Jan 2006 01:50:56 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> Message-ID: <17335.31696.282198.15573@panix3.panix.com> Brad Knowles writes: > Okay, now that is truly weird. I thought it was kind of > off-topic myself, but I thought that it would be one that either you > or Barry would have approved of, so I approved it on that basis. What I find truly weird is all the discussion of the moderation process on mailman-developers not to mention perhaps the moderation itself. Please allow me explain why I initially posted to mailman-developer. (I have no idea if this will make it to the mailman-developers list given what I'm reading so far. As I write this, my first reply to Brad Knowles is, as far as I know, being held for moderation. But I've received two other emails from that list on the thread, one of which seems to reinterpret the problem I stated.) I wanted to request a new feature. I looked at http://www.gnu.org/software/mailman/bugs.html to find out the proper way to do so. I submitted a feature request on sourceforge.net. But I also read at the URL mentioned: It is also recommended that you email a note about your submission to the mailman-developers mailing list, but don't rely on just the email to get the attention of the Mailman developers. So I do that. I have to subscribe to the mailing list first. Not to be able to post -- but to have it looked at by a moderator for posting. A little bit involved, but okay. I'm actually am lucky that the initial post was accepted. As part of the feature request, I describe the motivation for why I think the feature helpful. And now it seems as though whether or not the *moderators* think the way to get this accomplished is by adding a feature (as I guess I am not convincingly making my case) or as some other way to set up a mailing list now determines whether or not I can discuss or even *defend* my request for a feature on mailman-developers. Finally, there is a bit of irony with respect to the spam-infested moderator-lacking list which got me to request this and the views and treatment of the mailman-developers list. As the co-maintainer (but not the primary or main author) of that mailing list with the too much spam per valid posting, when the issue arose I suggested exactly the tight-fisted approach that seems to be in effect on mailman-developers list. That is, make people register if they want to post. (Actually, I wasn't going to then suggest moderating them *after* registering so I guess I am a bit more liberal.) However the main author felt very strongly that people should not have to subscribe to a group in order to ask for help or post bugs on the mailing list he set up. And interestingly, the person who sent the garbage-man solicitation to GNU developers feels exactly the same way. But here's the part that is very relevant here: others may not share the mailman-developer moderator's view of how mailing lists should be managed or maintained. I've mentioned two people above who probably don't in a strong way. It's possible or probable that in those other 25 or so lists there are more. And an important goal of my software projects is to allow others to do things in ways that maybe I don't necessarily personally use. So again, in sum although the *moderators* may decide that the way to handle the problem described is using mailman a different way -- again -- I don't (yet). It's a feature request. > I guess that just goes to show that you shouldn't over-think the > process too much. ;) Amen. From lists05 at equinephotoart.com Sun Jan 1 09:05:42 2006 From: lists05 at equinephotoart.com (JC Dill) Date: Sun, 01 Jan 2006 00:05:42 -0800 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <17335.31696.282198.15573@panix3.panix.com> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> Message-ID: <43B78D56.10502@equinephotoart.com> R. Bernstein wrote: > Please allow me explain why I initially posted to mailman-developer. Your reasons make perfect sense. I don't want you to think I was saying your reasons were "wrong" when I mentioned in my prior post that I might not have approved your initial post. Just that there is room for interpretation about if a post is "on-topic" or not, or if it's being posted to the right list or if there's a better list for the particular discussion. My point was that even moderators who are trying to follow the same "policy" and who have very similar opinions about when a post is on- or off-topic can still disagree sometimes. > (I have no idea if this will make it to the mailman-developers list > given what I'm reading so far. As I write this, my first reply to Brad > Knowles is, as far as I know, being held for moderation. But I've > received two other emails from that list on the thread, one of which > seems to reinterpret the problem I stated.) > > I wanted to request a new feature. I looked at > http://www.gnu.org/software/mailman/bugs.html to find out the proper > way to do so. I submitted a feature request on sourceforge.net. But I > also read at the URL mentioned: > > It is also recommended that you email a note about your submission > to the mailman-developers mailing list, but don't rely on just the > email to get the attention of the Mailman developers. There still exist some disconnects between some of the webpages (that were often written years ago) and the mailing list policies that have been changed or updated in more recent times. A bit of history might help you to understand how we got to where we are today. When I first joined mailman-users, it was an open list that non-members could post to. Unfortunately, it started getting a lot of spam. First Barry tried filtering out the spam and letting all "non-spam" thru to the list, but still a lot of spam got thru. Then Barry solicited volunteers to help administrate the list (and I volunteered), and we started approving non-member (non-spam) posts, and rejecting non-member (spam) posts. At this point 2 new issues surfaced. 1) The percentage of spam kept increasing until it was much too difficult for the owners and moderators to keep up with the moderation duties. Some days we would have upwards of 100 spam posts that had slipped past the initial spam filters and were in the moderation queue, and perhaps only a handful of on-topic posts that needed to be approved. 2) Some -users were replying "to the list" only - assuming that the person who asked was subscribed to the list and would see the reply - meanwhile the user didn't get their question answered (so they thought) and then posted the same question again. I take the blame (for good or bad) of persuading Barry that due to the changing environment it was not unreasonable to require a mailman user to subscribe to the mailman-users list before posting. Once Barry agreed we changed the list to reject all non-member posts. If a mailman user had a question for -users, they had to subscribe and confirm before posting. New users on that list are not moderated. When we made this change we had to change the list info pages, the welcome message, and the text describing the mailman-users link on many different webpages. Getting this all sorted out took several months and there may still be links out there that imply that mailman-users goes to a "person" rather than being an address for a discussion list where one has to subscribe before posting. So, that's how we ultimately dealt with the moderation problem you presently have when we encountered it on the mailman-users list. The mailman-developers list is different. The primary purpose of -dev is for discussion about "development" of mailman code. Feature requests are a gray area - are we discussing code development, or are we discussing "using" mailman? Many feature request *discussions* are better held on -users where more actual "users" of mailman can participate. We also had problems with discussions that weren't actually about developing mailman becoming a predominant part of the list traffic on -dev, overwhelming the subscribers on -dev that only wanted to discuss actual development issues. But, as you noted, the feature request process on sourceforge suggests you mail the -dev list when you submit a new request. We haven't quite figured out how to address this item with our current list policies. I'd love to hear your suggestions on this topic! > So I do that. I have to subscribe to the mailing list first. Not to > be able to post -- but to have it looked at by a moderator for > posting. A little bit involved, but okay. I'm actually am lucky that > the initial post was accepted. The primary reason that new subscribers to -dev are moderated is that for a while we were getting a lot of users who thought that their particular problem with mailman was the type of problem that they need "advanced help" with and that the people who could help would only be found on -dev. But they hadn't read the documentation, searched the FAQ, searched the -users archive, or asked on -users first. A high percentage of these "help me" posts belong on -users. A high percentage of first posts by new subscribers falls into this category. When this problem first surfaced we tried to resolve it a different way. For a few months we added text to the confirmation message asking the subscriber to email the list admin address with their reason for wanting to join -dev, so that we could quickly approve their subscription request. Once their subscription request was approved, then they could post immediately (no moderation of the subscriber or post). A very high percentage of subscription requests were never followed up, even when we (the list admins, primarily Brad and myself) repeatedly emailed the subscriber asking for their reason for joining so we could approve their membership request. Finally we decided that this wasn't working so we changed to the current process. This allows us to head off new subscribers who shouldn't be posting to -dev, but makes it easier for people to subscribe for non-posting reasons (to lurk on the development discussion), and only introduces a small delay to the first post by first-time on-topic posters. Normally, when the first moderated post is on-topic and thus approved we also clear the user's moderation flag. Brad didn't do that with your first post but it might have just been something he overlooked - I've overlooked doing that myself a few times. > As part of the feature request, I describe the motivation for why I > think the feature helpful. And now it seems as though whether or not > the *moderators* think the way to get this accomplished is by adding a > feature (as I guess I am not convincingly making my case) or as some > other way to set up a mailing list now determines whether or not I can > discuss or even *defend* my request for a feature on > mailman-developers. I assure you that the moderators are NOT trying to interfere with your ability to make your case and engage in discussion. Our role in this is ONLY to ensure that the discussion takes place on the correct list, -users or -dev. The reason for sending some discussions to -users is to keep -dev from being flooded with posts that aren't about development, which drives some people off the list and ultimately results in fewer people developing mailman (which would be a Bad Thing, right?). There is nothing sinister about our role, and I apologize if we weren't more clear about this in our prior posts. > Finally, there is a bit of irony with respect to the spam-infested > moderator-lacking list which got me to request this and the views and > treatment of the mailman-developers list. It's very hard in today's spam infested email environment to develop systems and policies that keep spam out and let on-topic discussions in without occasionally introducing delays of some sort. As you can see, we have tried several systems on these lists over the years, and it's still not a perfect system. > As the co-maintainer (but not the primary or main author) of that > mailing list with the too much spam per valid posting, when the issue > arose I suggested exactly the tight-fisted approach that seems to be > in effect on mailman-developers list. That is, make people register if > they want to post. (Actually, I wasn't going to then suggest > moderating them *after* registering so I guess I am a bit more > liberal.) Your suggested new policy is the very policy we currently have in effect on -users, so IMHO it's an excellent policy for that type of list. :-) > However the main author felt very strongly that people should not have > to subscribe to a group in order to ask for help or post bugs on the > mailing list he set up. And interestingly, the person who sent the > garbage-man solicitation to GNU developers feels exactly the same way. Barry used to feel that way too. Very strongly, in fact. See above for why we changed to the solution we now use. We have been very happy with the results since we made the change. > But here's the part that is very relevant here: others may not share > the mailman-developer moderator's view of how mailing lists should be > managed or maintained. Brad and I help moderate and administrate these lists but we try very VERY hard to make our decisions based on what Barry has told us he wants. When we have felt that the list configuration or policy should change we have had lengthy 3-way email threads with Barry to work out something that Barry approves of, and then only when Barry approves have we made the changes. So in the end, these are still Barry's lists and they are being run in a way that Barry approves. Brad and I help deal with the daily administrivia so that Barry and other key developers can devote their spare time on actually developing mailman code, instead of squandering their precious time on administrivia matters. > And an important goal of my software projects is to allow others to do > things in ways that maybe I don't necessarily personally use. So > again, in sum although the *moderators* may decide that the way to > handle the problem described is using mailman a different way -- again > -- I don't (yet). It's a feature request. Fair enough! Here's a suggestion that might solve the problem on your lists in the fashion you propose - create a moderator's list. Moderation requests are sent to that list. Anyone can subscribe/confirm to the moderator's list. The welcome message would give them the moderator username and password for the main list. Now they can receive the moderator requests and act on them. For example, your main list is foo-list at example.com. Create a list called foo-moderators at example.com. Add foo-moderators at example.com to the moderators for foo-list. J Doe subscribes and confirm to foo-moderators. The welcome message tells J. Doe that the moderator's username is foo-mod and the password is foobar. When a non-member post is sent to foo-list, it goes out to all subscribers of foo-moderators. J. Doe (or any other subscriber of foo-moderators) can click on the link to the admin page, enter the username of foo-mod and the password of foobar, and then approve or reject each held post. One risk is that a spammer may find it worthwhile to join foo-moderators so that they can approve their spam to be sent on to your list. Since the spammer could be any of the subscribers of foo-moderators it would be hard to find out which one was the spammer and remove them (and of course the spammer could just resubscribe with another email address). There may be other drawbacks with this suggestion and if so I'm sure that someone will let us all know. :-) Leap second passed by a few minutes ago. Happy New Year everyone! jc - volunteer moderator and admin for mailman-user and -dev From rmg at terc.edu Sun Jan 1 09:48:38 2006 From: rmg at terc.edu (Robby Griffin) Date: Sun, 1 Jan 2006 03:48:38 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> Message-ID: <26fa8931af20ac19a1dbe55d1de8235e@terc.edu> On Dec 31, 2005, at 23:22, Brad Knowles wrote: > some lists have as many as 100 messages waiting in > the queue to be moderated, and some of those messages date back to > May of 2005. I think that this is a problem that needs to be > addressed within the Mailman package, and not just something that can > be observed externally through tools like "mmdsr". Here's what I've done for somewhat unrelated reasons: - patch bin/discard to support rejecting held messages and providing rejection comments. - add a cron job that rejects held messages older than 10 days, with the following comment: "Your message was automatically rejected after being on hold too long without moderator action." The reason I set out to expire held messages was not because of absentee moderators, but because of overworked moderators who had trouble with the sheer workload of deleting spam from batches of lists they run. With expiration in place, they can simply pick out and approve the few legitimate posts every so often, leaving the rest to rot. I mention this because you're on the subject of ridiculously long-held messages, which my changes eliminate entirely. But also, for lists with absentee moderators, held message expiration should have two harmless effects: 1. removing all held spam eventually, by way of removing all old held posts. 2. informing legitimate moderated senders that the moderator is likely absent, and definitely did not discard their post. So if they know of some other way to post without incurring moderation or some other way to contact the moderator they should use it. If I had this to do over, I would probably say the timeout and action (discard, reject with configurable comment, approve) for expiring held messages ought to be configurable sitewide and/or per-list rather than hardwired in a cron job. Some people take longer vacations than others, and for some lists the failure mode of rejecting all posts is undesirable. I would also want to consider the relationship of any such configuration with mm_cfg.PENDING_REQUEST_LIFE. It appears that some subscription requests and confirmation cookies for held posts may already expire on their own schedule, and that the rationale for this might inform the design of held message expiration. --Robby From tkikuchi at is.kochi-u.ac.jp Sun Jan 1 10:22:48 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Sun, 01 Jan 2006 18:22:48 +0900 Subject: [Mailman-Developers] RELEASED Mailman 2.1.7 In-Reply-To: References: Message-ID: <43B79F68.9070708@is.kochi-u.ac.jp> John W. Baxter wrote: > > Shouldn't Mark Shapiro be recognized in the ACKNOWLEDGEMENTS file now? > Yes. Mark Sapiro _is_ in the ACKNOWLEDEMENTS. I've just added his name in top page of the http://mailman.sourceforge.net/ It'll appear in list.org and gnu mirror sites later. I just want to say my thanks to Mark again now, and I thank you all for your cooperation. -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From rocky at panix.com Sun Jan 1 18:15:29 2006 From: rocky at panix.com (R. Bernstein) Date: Sun, 1 Jan 2006 12:15:29 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <43B78D56.10502@equinephotoart.com> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> <43B78D56.10502@equinephotoart.com> Message-ID: <17336.3633.210464.849880@panix3.panix.com> Thanks also for the suggestion of setting up a list just to send out moderator passwords. I'll pass that suggestion and the one by Robby on global detection of mailing-neglect back to the the GNU discussion group. I hope that will help. Should they go that route, I'll try to withdraw the sourceforge feature request. Where I think something inside GNU Mailman could be a little better than a second list is that the integration could enforce that the email associated with person logging in to the webpage or sending moderation by email is also *currently* subscribed to the list***. The theory here is that spammers don't want to receive the spam they spew. ***If I have this correct, where GNU mailman seems to differ from say sourceforge bug and feature trackers is that in GNU Mailman where there is a password associated with a moderator and an administrator *account*, in sourceforge tracker, there is a moderator or administrator *flag* is associated with a account(s) to grant access. So to moderate or administer an account one uses one's selected user account and password. As a result, is easier to effect such an enforcement described above. - - - I guess sometimes things are not what they may seem initally, so many thanks for the detailed explanation; it all makes sense. It is also interesting to learn that the GNU mailman mailing lists have the same problems as other GNU lists. But it sounds like the GNU mailman lists have very dedicated moderators. Again at the risk of beating this horse dead, what we're looking for is a way for mailing lists to distribute the burden of moderation such as by having the mailing list be more self moderating as it appears that the wiki works. (I could be wrong here about the wiki.) The observation is that right now, a number of mailing lists at least GNU mailing lists are just getting neglected, and this suggests something is wrong. Maybe it's just a global misunderstanding of how to set up a general help list (e.g. a documentation change), but I have a feeling it's not just that. I don't know how to or have a suggestion as to how to deal with concerns of the getting discussions to the right user/developer group or what should be indicated when making a feature request. However I do see the wisdom in discussing things in the right venue. After all, what's important is getting ideas and solving this problem, not bothering or using up the bandwidth of the wrong people. So if this discussion should be moved to the user list, please let me know. Again thanks. From brad at stop.mail-abuse.org Sun Jan 1 19:51:26 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Sun, 1 Jan 2006 12:51:26 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <26fa8931af20ac19a1dbe55d1de8235e@terc.edu> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <26fa8931af20ac19a1dbe55d1de8235e@terc.edu> Message-ID: At 3:48 AM -0500 2006-01-01, Robby Griffin wrote: > Here's what I've done for somewhat unrelated reasons: > > - patch bin/discard to support rejecting held messages > and providing rejection comments. > > - add a cron job that rejects held messages older than 10 days, > with the following comment: > > "Your message was automatically rejected after being on hold too > long without moderator action." I like both of these modifications. Have you already submitted patches for them to SourceForge? If not, could I talk you into doing that? I'd certainly like to apply these modifications to the other site I help administer, and I'd like to talk to Barry about incorporating these features on python.org. > If I had this to do over, I would probably say the timeout and > action (discard, reject with configurable comment, approve) for > expiring held messages ought to be configurable sitewide and/or > per-list rather than hardwired in a cron job. Agreed. But I would think that this would be a relatively minor enhancement over the original modification. > I would also want to consider the relationship of any such > configuration with mm_cfg.PENDING_REQUEST_LIFE. It appears that > some subscription requests and confirmation cookies for held posts > may already expire on their own schedule, and that the rationale > for this might inform the design of held message expiration. That's also a good idea. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From brad at stop.mail-abuse.org Sun Jan 1 20:00:36 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Sun, 1 Jan 2006 13:00:36 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <17336.3633.210464.849880@panix3.panix.com> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> <43B78D56.10502@equinephotoart.com> <17336.3633.210464.849880@panix3.panix.com> Message-ID: At 12:15 PM -0500 2006-01-01, R. Bernstein wrote: > ***If I have this correct, where GNU mailman seems to differ from say > sourceforge bug and feature trackers is that in GNU Mailman where > there is a password associated with a moderator and an administrator > *account*, in sourceforge tracker, there is a moderator or > administrator *flag* is associated with a account(s) to grant > access. So to moderate or administer an account one uses one's > selected user account and password. As a result, is easier to effect > such an enforcement described above. The current version of Mailman does not really properly support a database, which I think would be required for the kind of features you're talking about. Yes, there are at least one or two database MemberAdaptors of one sort or another, but all they do is take the existing Mailman method of working and adapt that to be compatible with databases, whereas for the kinds of features you're talking about, the reverse would really be required. The next major version of Mailman (Mailman3) will be much more database-aware, and this would be an excellent idea to get onto their plate now. > Again at the risk of beating this horse dead, what we're looking for > is a way for mailing lists to distribute the burden of moderation such > as by having the mailing list be more self moderating as it appears > that the wiki works. (I could be wrong here about the wiki.) Check the recent news about WikiPedia. The lesson is that any project which grows sufficiently large, will have such issues if they are permissive in terms of what they allow their rank-and-file subscribers to do. I certainly occasionally hear about similar problems with the MoinMoin wiki on python.org, and it's not anywhere near as large as WikiPedia. On another site I help administer, we take a more restrictive approach with TWiki -- accounts are free and automatically given, but you at least have to sign up for an account before you are allowed to modify anything, and certain pages are locked down as to which administrative groups are allowed to modify them -- and I haven't heard of any such issues there. On the other hand, we're also much smaller than the MoinMoin wiki on python.org, so it's hard to say what is a result of our tighter security measures and what is a result of our being a lot smaller. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From lists05 at equinephotoart.com Sun Jan 1 20:18:54 2006 From: lists05 at equinephotoart.com (JC Dill) Date: Sun, 01 Jan 2006 11:18:54 -0800 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <17336.3633.210464.849880@panix3.panix.com> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> <43B78D56.10502@equinephotoart.com> <17336.3633.210464.849880@panix3.panix.com> Message-ID: <43B82B1E.7080002@equinephotoart.com> R. Bernstein wrote: > I guess sometimes things are not what they may seem initally, so many > thanks for the detailed explanation; it all makes sense. It is also > interesting to learn that the GNU mailman mailing lists have the same > problems as other GNU lists. But it sounds like the GNU mailman lists > have very dedicated moderators. I don't write code - my role in projects is as a product manager. Helping administer/moderate the mailman lists is one of the ways I can help "contribute" to mailman. > Again at the risk of beating this horse dead, what we're looking for > is a way for mailing lists to distribute the burden of moderation such > as by having the mailing list be more self moderating as it appears > that the wiki works. (I could be wrong here about the wiki.) There are a lot of differences between editing a wiki and moderating a list. Most people edit a wiki by adding content in areas where they have knowledge. Sometimes this contribution is prodded by going to the wiki to *get* the information and discovering that it's not there, and realizing that they are well suited for putting it there. There's also immediate positive reinforcement, their words are now on the web for all to see. When they need to refer someone to the content they can say "go to the wiki" and their words will be there for the other person to refer to. So they get value (something they made is "there" and can be used later) and recognition from this activity, and that positively reinforces their efforts and encourages them to do more of it. The process of digging thru spam to find on-topic posts that should go to a mailing list is not nearly so rewarding. (The term "Thankless task" comes to mind.) Without receiving reminders that there are posts waiting for moderation, there is no event to nudge moderators to go moderate. Once they have moderated the posts, there is no recognition that they discarded all that spam, that they were the ones who "freed" the held posts for delivery on to the list. I don't think you will get a lot of participation from a wide range of your list membership. I think that you will find that a very small number of your list members regularly do the moderation duties and that occasional moderation by other list members is very very rare. Another option for the solution I proposed is to just give out the moderator's username and password in the footer of the main list, in addition to having a moderators list. Now a would-be moderator has 2 different ways they can participate - they can just randomly log in from time to time to see if there is anything that needs moderating, or they can subscribe to the moderator's list to receive notices when there are posts that need moderating. You still don't have any way to track who moderated the post, but you would make participation easier because they don't have to subscribe to the moderator's list to get the username and password. In your original proposal you suggested a box or flag that allows anyone who is a subscriber to the list to moderate the list, which provides the individual activity tracking. If we were to add this, I think that this should be a *member* setting (mod-access) rather than a *list* setting, and then it can be added to the new member config options. A list owner could then configure the list to have new members automatically included in the people who can moderate the list, and the mod-access flag can be turned off (or back on) member by member for existing members. > The observation is that right now, a number of mailing lists at least GNU > mailing lists are just getting neglected, and this suggests something > is wrong. Maybe it's just a global misunderstanding of how to set up a > general help list (e.g. a documentation change), but I have a feeling > it's not just that. Moderating a mailing list is a different type of activity than contributing code. For best results you need to enlist, encourage, and reward (with recognition from the list owners or key developers) the people who do this work, even if you do all the reward and encouragement in private email to your helpers. This goes a long way towards keeping them happy. I do this work on the mailman lists as a service to the mailman community, but when Barry says "thanks" that's a real motivator for me to continue. I'm pretty sure Brad feels similarly. > I don't know how to or have a suggestion as to how to deal with > concerns of the getting discussions to the right user/developer group > or what should be indicated when making a feature request. However I > do see the wisdom in discussing things in the right venue. After all, > what's important is getting ideas and solving this problem, not > bothering or using up the bandwidth of the wrong people. So if this > discussion should be moved to the user list, please let me know. The -dev list is fairly quiet right now, so it's not a bad thing to have this discussion here. You might want to post on -users as well, you will reach more people, different people, and get some additional perspectives on your problem and possible solutions. -users is a better place to drum up support for the feature request - if we get many other list owners who say "hey, we could use this on our lists too" then that helps move the feature request priority up and encourages a developer to consider writing the code. jc From rmg at terc.edu Sun Jan 1 21:41:04 2006 From: rmg at terc.edu (Robby Griffin) Date: Sun, 1 Jan 2006 15:41:04 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <26fa8931af20ac19a1dbe55d1de8235e@terc.edu> Message-ID: On Jan 1, 2006, at 13:51, Brad Knowles wrote: > At 3:48 AM -0500 2006-01-01, Robby Griffin wrote: > >> Here's what I've done for somewhat unrelated reasons: >> >> - patch bin/discard to support rejecting held messages >> and providing rejection comments. >> >> - add a cron job that rejects held messages older than 10 days, >> with the following comment: >> >> "Your message was automatically rejected after being on hold too >> long without moderator action." > > I like both of these modifications. Have you already submitted > patches for them to SourceForge? If not, could I talk you into doing > that? I'll go ahead and post what I have as it relates to 2.1.4, and point out that held message expiration in 2.1.7 already exists but could use minor improvements. I don't have a lot of time to work on it right now, and as I started to think about putting together clean diffs against 2.1.7, I observed that my changes are *almost* obsolete: sometime after 2.1.4, a per-list config parameter for "max_days_to_hold" was added, and bin/checkdbs uses it to expire held messages. But it supports only discard, and not reject/comment or approve. Also, it does not deal with stale held messages whose lists have since been deleted. See patch #1394949. --Robby From barry at python.org Mon Jan 2 02:57:03 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 01 Jan 2006 20:57:03 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <43B744CA.7090607@equinephotoart.com> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> Message-ID: <1136167023.2302.61.camel@geddy.wooz.org> On Sat, 2005-12-31 at 18:56 -0800, JC Dill wrote: > The main problem I think Rocky is experiencing is the problem of absent > moderators, period. Rather than some automated method of turning the > moderator tasks over to others, I suggest that a better way is to more > closely oversee pending moderator tasks so that the list owner and the > list server administrator receive notices when a moderator's queue has > not been recently attended to, and address the lack of moderation while > the queue is still small and relatively fresh. One of the problems that I have with the moderation workflow is that I have to log into every list I'm going to moderate, and then that login authentication is lost when I kill my browser. I don't know how common my experience is but I've been terrible lately in moderating the dozen or whatever lists I'm an admin for. If it was just one list, and if I didn't have to go through the login dance every time, I think I'd do more moderating. Or if I could log in once and get to all my lists, that would be much better too. Of course, that requires the federated user database that MM3 is all about. When I do have tme to moderate all my lists, it takes me a long time to do so, which is why I don't do it very often. Aside from the login issues, I think the admindb web interface is just so crappy that it's really hard to easily separate the wheat from the chaff. I find that my typical approach is to scan the summary, opening any potential ham in a tab window ($1M to whoever thought up /that/ particular browser feature!). Then I approve all the hams and go back to the summary list, using Skip Montanaro's (IIRC) awesome "discard-all-deferred" feature to finish up the list. So I'd be interested to hear ideas for improving the admindb interface. Ultimately, my dream is to have an IMAP interface to the admin queue, then I could just move the ham to one special folder and just delete the spam. I'm not sure exactly how to do rejects, but a "reply" or "forward" is probably good enough. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060101/3bb0806c/attachment.pgp From barry at python.org Mon Jan 2 02:58:16 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 01 Jan 2006 20:58:16 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> Message-ID: <1136167096.2303.63.camel@geddy.wooz.org> On Sat, 2005-12-31 at 22:22 -0600, Brad Knowles wrote: > Okay, now that is truly weird. I thought it was kind of > off-topic myself, but I thought that it would be one that either you > or Barry would have approved of, so I approved it on that basis. I definitely think it's on-topic for mm-dev. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060101/bdd3639c/attachment-0001.pgp From barry at python.org Mon Jan 2 03:14:36 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 01 Jan 2006 21:14:36 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <43B78D56.10502@equinephotoart.com> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> <43B78D56.10502@equinephotoart.com> Message-ID: <1136168077.2302.70.camel@geddy.wooz.org> First let me say that I think JC and Brad are doing a great job moderating the lists, and I /greatly/ appreciate their help with this! Second, I think there's one more use case that might work well for general help lists like mailman-users (but not mailman-developers). There should be a way for non-members to "self-moderate", essentially using a technique similar to subscription confirmations. If you were a non-member poster who replied to the confirmation, your message would go through. Non-confirmed posts would be automatically discarded or bounced after a certain amount of time. Come to think of it, a list like mailman-developers could use a variant similar to the confirm-and-approve for subscriptions. Admins would only see confirmed messages in their queue. At that point, most spam should be deleted and the moderator only as to decide whether the message is on-topic or not (something that will always be a judgment call). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060101/0270ba5b/attachment.pgp From barry at python.org Mon Jan 2 03:16:54 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 01 Jan 2006 21:16:54 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <17336.3633.210464.849880@panix3.panix.com> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> <43B78D56.10502@equinephotoart.com> <17336.3633.210464.849880@panix3.panix.com> Message-ID: <1136168214.2302.73.camel@geddy.wooz.org> On Sun, 2006-01-01 at 12:15 -0500, R. Bernstein wrote: > ***If I have this correct, where GNU mailman seems to differ from say > sourceforge bug and feature trackers is that in GNU Mailman where > there is a password associated with a moderator and an administrator > *account*, in sourceforge tracker, there is a moderator or > administrator *flag* is associated with a account(s) to grant > access. So to moderate or administer an account one uses one's > selected user account and password. As a result, is easier to effect > such an enforcement described above. Yes. This sucks about Mailman, and it's something we plan to fix in MM3 by having a real user database. ;) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060101/3441d78b/attachment.pgp From brad at stop.mail-abuse.org Mon Jan 2 05:10:23 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Sun, 1 Jan 2006 22:10:23 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <1136168077.2302.70.camel@geddy.wooz.org> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> <43B78D56.10502@equinephotoart.com> <1136168077.2302.70.camel@geddy.wooz.org> Message-ID: At 9:14 PM -0500 2006-01-01, Barry Warsaw wrote: > Come to think of it, a list like mailman-developers could use a variant > similar to the confirm-and-approve for subscriptions. Admins would only > see confirmed messages in their queue. At that point, most spam should > be deleted and the moderator only as to decide whether the message is > on-topic or not (something that will always be a judgment call). If confirmations were required for posting by non-members (before the messages would make it into a moderation queue), I think that would pretty much completely solve all the moderation problems that I have personal experience with. But then we're getting dangerously close to tools like Active Spam Killer or TMDA, which I am generally violently opposed to. Maybe those kinds of tools are appropriate for mailing list use but not personal use, I dunno.... I'll have to think long and hard on that topic. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From brad at stop.mail-abuse.org Mon Jan 2 05:05:33 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Sun, 1 Jan 2006 22:05:33 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <1136167023.2302.61.camel@geddy.wooz.org> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> Message-ID: At 8:57 PM -0500 2006-01-01, Barry Warsaw wrote: > One of the problems that I have with the moderation workflow is that I > have to log into every list I'm going to moderate, and then that login > authentication is lost when I kill my browser. That's why I never kill my browser anymore. > I find that my > typical approach is to scan the summary, opening any potential ham in a > tab window ($1M to whoever thought up /that/ particular browser > feature!). Then I approve all the hams and go back to the summary list, > using Skip Montanaro's (IIRC) awesome "discard-all-deferred" feature to > finish up the list. I do exactly the same. For smaller lists, or lists with lower moderation load, that works okay. For the larger ones, I'd like to see something like Skip's "mmfold.py" script that could run locally on the same server where the lists are located, so that no use of a web browser is required, and so that the program could directly access the Python pickles in question. That would make it a lot easier for me to manage some of the lists on the other main site where I volunteer. > So I'd be interested to hear ideas for improving the admindb interface. The admindb interface is one thing that needs improvement. But I'd also like to see alternatives we could use that completely avoid the use of any web browser, etc.... > Ultimately, my dream is to have an IMAP interface to the admin queue, > then I could just move the ham to one special folder and just delete the > spam. I'm not sure exactly how to do rejects, but a "reply" or > "forward" is probably good enough. IMAP would probably be an improvement over what we have now, assuming you've got a decent IMAP client -- that's not necessarily a valid assumption. But my experience is that IMAP falls down too (especially depending on the IMAP server implementation), and you need something even scalable when that happens. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From jwblist at loricamail.com Mon Jan 2 08:31:21 2006 From: jwblist at loricamail.com (John W. Baxter) Date: Sun, 01 Jan 2006 23:31:21 -0800 Subject: [Mailman-Developers] RELEASED Mailman 2.1.7 In-Reply-To: <43B79F68.9070708@is.kochi-u.ac.jp> Message-ID: On 1/1/06 1:22 AM, "Tokio Kikuchi" wrote: > John W. Baxter wrote: >> >> Shouldn't Mark Shapiro be recognized in the ACKNOWLEDGEMENTS file now? >> > > Yes. Mark Sapiro _is_ in the ACKNOWLEDEMENTS. I've just added his name > in top page of the http://mailman.sourceforge.net/ It'll appear in > list.org and gnu mirror sites later. YIKES. I have been reading Mark's name for a long time now, and my mind has continuously inserted an "h" after the "S". So I did a search for the incorrect name I expected so find. Sorry, Mark! --John From stephen at xemacs.org Tue Jan 3 03:45:59 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 03 Jan 2006 11:45:59 +0900 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <17336.3633.210464.849880@panix3.panix.com> (R. Bernstein's message of "Sun, 1 Jan 2006 12:15:29 -0500") References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> <43B78D56.10502@equinephotoart.com> <17336.3633.210464.849880@panix3.panix.com> Message-ID: <87wthim3ko.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "R" == R Bernstein writes: R> Where I think something inside GNU Mailman could be a little R> better than a second list is that the integration could enforce R> that the email associated with person logging in to the webpage R> or sending moderation by email is also *currently* subscribed R> to the list***. The theory here is that spammers don't want to R> receive the spam they spew. This is little help. The spammer simply uses a throwaway address that they have no intention of reading. That's part of the spammers' manual by now. Total cost per week: $0.23. R> But it sounds like the GNU mailman lists have very dedicated R> moderators. They do. It's not an accident. Part of the reason is that Mailman is Python culture. Python's development community is very business- oriented; they have a healthy respect for good administration. It's not just a question of of eating your own dogfood; it's question of making the big dogs who eat your food feel welcome to come in and help with the process, and paying them due respect even though they can't even cook up spam (the Monty Pythonic kind). This is resoundingly successful for Mailman because developing list software is its business, but as Brad occasionally mentions he also works on the general Python lists, too. I don't think that's an accident. And, of course, the Python and Mailman communities are large. The typical GNU project (or Sourceforge, for that matter) _can't_ work this way. Usually the mailing list owner and moderator is also the project lead. Mailing list management (including the parts that AI is not yet I enough for, like spam management) is not what they imagine themselves doing. I don't blame them, but I also have no ideas for reducing the burden to the levels they imagine. R> Again at the risk of beating this horse dead, what we're R> looking for is a way for mailing lists to distribute the burden R> of moderation such as by having the mailing list be more self R> moderating as it appears that the wiki works. (I could be wrong R> here about the wiki.) You're wrong about comparing the mailing list to the wiki. :-) Wikis have a substantially lower burden on both ends. First, spamming wikis is both more expensive (wiki input has no equivalent to RFC 2821) and less productive for spammers. Email (and netnews) spamming is push; the readers get it whether they ask for it (that particular post, of course they asked for the list or newsgroup) or not. Wikis are pull; the readers are in an unreceptive mood: they're looking for something else _specifically_ when they follow a link. Not only that, but they have the power to do something about *that particular spam*. This means that the half-life of spam on a wiki is short; in mail, it lives until somebody (or a smart-enough spam-filter) reads that copy. In other words, the "casual moderator" gets the satisfaction of nuking the spammer that bit her on the wiki, whereas with a mailing list it's "paying forward" to the other users for _future_ spams ... those user have already received this one. Not as satisfying. R> The observation is that right now, a number of mailing lists at R> least GNU mailing lists are just getting neglected, and this R> suggests something is wrong. Maybe it's just a global R> misunderstanding of how to set up a general help list (e.g. a R> documentation change), but I have a feeling it's not just that. It is "just that". You need dedicated staff to handle spam on an open list. That's all there is to it. Spammers are a hostile entity, always looking for ways to break down your security, banging on your door every day. Unlike GNU volunteer moderators, they get paid in hard cash for what they do. And it's a lot more fun to screw the moderators than to be one (for the perverts who enjoy breaking into things, anyway). So there's only so much a machine can do to combat that. All of the above, though stated as fact for emphasis, is of course IMO only. R> So if this discussion should be moved to the user list, please R> let me know. I think you'll get much more fruitful discussion there. The Mailman Developers list is primarily about how to implement the specification. "Stop Spam" is not a specification, it's a requirement that will interact with others to constrain the specification. Although many of us on Mailman Developers are active in list administration and moderation, you'll get a much broader and more casual audience on Mailman Users ... and your whole point is to find ways to get the casual moderator to be more active. That's also a requirement, not a specification. :-) -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From stephen at xemacs.org Tue Jan 3 04:02:30 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 03 Jan 2006 12:02:30 +0900 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: (Brad Knowles's message of "Sun, 1 Jan 2006 22:10:23 -0600") References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> <43B78D56.10502@equinephotoart.com> <1136168077.2302.70.camel@geddy.wooz.org> Message-ID: <87sls6m2t5.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Brad" == Brad Knowles writes: Brad> But then we're getting dangerously close to tools like Brad> Active Spam Killer or TMDA, Technically, yes. Brad> which I am generally violently opposed to. Brad> Maybe those kinds of tools are appropriate for mailing Brad> list use but not personal use, I dunno.... I wouldn't make that distinction. Rather, I would say, does this address have to be well-known and open? (Bug and help lists, yes.) Can potential users be informed that the address is protected by a challenge-response system? (In the README and on the home page, yes.) Is it better than the alternative of requiring them to subscribe (and optionally set themselves to no-mail)? (Not clear, if subject to further moderation.) Although I quit TMDA as soon as I realized I'd forgotten to whitelist my Mom, I'd still consider using it for the address I publish on my home page, _if it were uniquely for people who view that page_. Ie, if I don't ever write mail from that address (or perhaps only in response to inquiries to that address). Of course IMHO FWIW YMMV. HTH. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From stephen at xemacs.org Tue Jan 3 04:21:38 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 03 Jan 2006 12:21:38 +0900 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <1136167023.2302.61.camel@geddy.wooz.org> (Barry Warsaw's message of "Sun, 01 Jan 2006 20:57:03 -0500") References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> Message-ID: <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "BAW" == Barry Warsaw writes: BAW> One of the problems that I have with the moderation workflow BAW> is that I have to log into every list I'm going to moderate, BAW> and then that login authentication is lost when I kill my BAW> browser. I simply have a local file that puts each of about 18 (way too many, but no time to clean up :-( ) XEmacs lists in a separate frame. I would assume I could use the http://USER:PWD at lists.DOMAIN.TLD/lists/admindb/LIST form to autologin. Haven't tried, I'm anal-compulsive about password protection, and my browser usually lives for a couple months at a time. BAW> I find that my typical approach is to scan the summary, BAW> opening any potential ham I assume you mean "ham = on-topic, spam = off-topic, possibly but not necessarily UCE"? Our lists don't have a topicality problem, so I don't think I've ever had need to open a post. The summary subject invariably shows spam vs. ham. Unfortunately, you can't open a subscription request. I know, the text doesn't show much in most cases, but we could look for header forging. I don't understand why this distinction was made; would it cost resources to give access to the subscription requests? My only problem with the current interface form is that the form is a little too big to be conveniently used in a format of 3 columns of 6 rows of frames. So from my point of view improving the admin interface is not that high-priority. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From brad at stop.mail-abuse.org Tue Jan 3 06:25:29 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Mon, 2 Jan 2006 23:25:29 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: At 12:21 PM +0900 2006-01-03, Stephen J. Turnbull wrote: > Our lists don't have a topicality problem, so I don't think I've ever > had need to open a post. The summary subject invariably shows spam > vs. ham. I can't speak for Barry or anyone else, but when I use Mailman to handle mailing lists for webmaster, postmaster, etc..., there is a 99% chance that any one particular message is spam, and I have to take a look at the remaining 1% to see if they've managed to forge a plausible subject line on something that we would rather not be allowed through to the list. Since one known tactic of spammers is to take list archives and slice off message bodies and replace them with their own, I think that this is a necessary step -- for me, on the mailing lists I help manage, the way I help to manage them. > My only problem with the current interface form is that the form is a > little too big to be conveniently used in a format of 3 columns of 6 > rows of frames. Which I think is part of why Skip created mmfold.py. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From stephen at xemacs.org Tue Jan 3 09:32:21 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 03 Jan 2006 17:32:21 +0900 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: (Brad Knowles's message of "Mon, 2 Jan 2006 23:25:29 -0600") References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <87k6dhn23u.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Brad" == Brad Knowles writes: Brad> I can't speak for Barry or anyone else, but when I use Brad> Mailman to handle mailing lists for webmaster, postmaster, Brad> etc..., there is a 99% chance that any one particular Brad> message is spam, and I have to take a look at the remaining Brad> 1% to see if they've managed to forge a plausible subject Brad> line on something that we would rather not be allowed Brad> through to the list. If it's really necessary to look so that you can get from 99% to 100%, then there needs to be some improvement in the UI. Suggestion: provide a spam-oriented (rather than netiquette-oriented) moderator screen option. 1. All the ban-author options should go, they take up _way_ too much space. In all the lists I've ever subscribed to, I've only seen one case where they would have been appropriate, and that guy quickly learned not only how to change from, sender, and envelope, but also his IP addresses. I've never seen a case on XEmacs lists (ie, in long but limited-scope experience as a moderator). There are lists where identifiable senders _are_ a problem, so this should be optional. I think everybody has a spam problem, though---I'd vote for the spam-oriented layout as the default. 2. I would sort/group on (prefix-scrubbed) subject rather than sender; non-whitelisted senders rarely send more than one topic in a short period, although they often get frustrated by moderation and mail system delays and repost. And spammers often use multiple fake authors for multiple tries of a given subject. Both would be conveniently trapped by this sort. 3. Where possible, the information _and the controls_ for a single entry should be on a single line. I think it's reasonable to assume as a default that the moderator has at least a 1024px width screen and can read small enough type to put 100 characters on a line, and provide an alternative format for people who moderate from their cellphones or need inch-high fonts. (Obviously with >80 characters per line you need the vertical rules to separate fields.) I'd try arranging as | Sender | CCs | controls | Subject | to optimize eye and mouse movement. The Defer/Approve/Reject/Discard options can be enormously compressed by getting rid of the tags. For graphical browsers, use rotated text (SVG or prerotated PNGs) in column headers, repeated every 20 rows (#rows should be configurable). For non-graphical browsers, use initials (D/A/R/X). 4. It would be very nice if it could be arranged that each column was of a fixed width but with a horizontal scrollbar every twenty rows or so. This would allow most froms to be identified in 15-20 characters, but convenient scrolling if you wanted to see more. I'm not sure how to arrange that in HTML, offhand, but I suspect it can be done for common browsers. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From nigel.metheringham at dev.intechnology.co.uk Tue Jan 3 10:09:47 2006 From: nigel.metheringham at dev.intechnology.co.uk (Nigel Metheringham) Date: Tue, 03 Jan 2006 09:09:47 +0000 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <1136279387.9433.5.camel@localhost.localdomain> On Tue, 2006-01-03 at 12:21 +0900, Stephen J. Turnbull wrote: > I would assume I could use the > > http://USER:PWD at lists.DOMAIN.TLD/lists/admindb/LIST > > form to autologin. Haven't tried, I'm anal-compulsive about password > protection, and my browser usually lives for a couple months at a time. its a form so not quite like that - try http://lists.DOMAIN.TLD/lists/admindb/LIST?adminpw=yourpassword Personally I have a bookmarks folder within Firefox with all the lists I handle as bookmarks within that set up with password. I then just right click on the folder, "open in tabs" and have all the list admin interfaces opened at once... [I'm reasonably relaxed about having the list moderation or admin passwords available in clear on my box - I think its a reasonable trade off compared to the other options. I *always* log off my box (so close the browser) when I'm going to be away for more than a couple of hours, so keeping a browser running for days is not an option] Nigel. -- [ Nigel Metheringham Nigel.Metheringham at InTechnology.co.uk ] [ - Comments in this message are my own and not ITO opinion/policy - ] From brad at stop.mail-abuse.org Tue Jan 3 14:14:59 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Tue, 3 Jan 2006 07:14:59 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <87k6dhn23u.fsf@tleepslib.sk.tsukuba.ac.jp> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> <87k6dhn23u.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: At 5:32 PM +0900 2006-01-03, Stephen J. Turnbull wrote: > 1. All the ban-author options should go, they take up _way_ too much > space. In all the lists I've ever subscribed to, I've only seen one > case where they would have been appropriate, and that guy quickly > learned not only how to change from, sender, and envelope, but > also his IP addresses. I've never seen a case on XEmacs lists > (ie, in long but limited-scope experience as a moderator). I find that I use those functions pretty frequently on the various lists that I help administer at ntp.isc.org. I don't think I've used them yet in helping to moderate lists at python.org, however. > There are lists where identifiable senders _are_ a problem, so > this should be optional. I think everybody has a spam problem, > though---I'd vote for the spam-oriented layout as the default. I could support that view. For myself, I would turn those options back on, but that would be more of a personal thing. For me, the problem is not one of screen space. The problem is one of server performance. I can hit "page down" pretty frequently and get through a list of messages reasonably fast. But when I've got a thousand messages queued up for moderation, my entire machine grinds to a halt as it tries to render that page -- very, very, very, very, very slowly. This is why I want those command-line server-side tools. > 2. I would sort/group on (prefix-scrubbed) subject rather than > sender; Agreed. > 3. Where possible, the information _and the controls_ for a single > entry should be on a single line. I think it's reasonable to > assume as a default that the moderator has at least a 1024px width > screen Now there, I disagree. My machine is a few years old, but there are still plenty of laptops being built and shipped today that have relatively small screens, and laptops are quickly becoming the default computer instead of desktops. Morever, we can't know the typeface/font size choices that the moderator has made in their web browser, so what may fit on your one line may wind up being effectively three poorly formatted lines for someone who is visually impaired. > The Defer/Approve/Reject/Discard options can be enormously > compressed by getting rid of the tags. Based on my other disagreements above, I think it's pretty obvious that anything built on top of those premises would get further and further into the "bad idea" category. > 4. It would be very nice if it could be arranged that each column was > of a fixed width but with a horizontal scrollbar every twenty rows > or so. IMO, this violates the concept of getting everything onto one line, so that you can compress things as much as physically possible. Either go with an idea or don't, but don't go with an idea and then muck it up at the last moment. From everything you've said, I think you would like Skip's mmfold.py script. I think you should check it out. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From stephen at xemacs.org Tue Jan 3 15:38:36 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 03 Jan 2006 23:38:36 +0900 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: (Brad Knowles's message of "Tue, 3 Jan 2006 07:14:59 -0600") References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> <87k6dhn23u.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <87acedml5f.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Brad" == Brad Knowles writes: > 3. Where possible, the information _and the controls_ for a single > entry should be on a single line. I think it's reasonable to > assume as a default that the moderator has at least a 1024px width > screen Now there, I disagree. My machine is a few years old, but there are still plenty of laptops being built and shipped today that have relatively small screens, and laptops are quickly becoming the default computer instead of desktops. Ah, the disadvantages of living in Japan, Inc. The option of a laptop with less than 1024x768 hasn't been available to me for several years in this country. (Might have something to do with kanji requiring high resolution, too.) Morever, we can't know the typeface/font size choices that the moderator has made in their web browser, so what may fit on your one line may wind up being effectively three poorly formatted lines for someone who is visually impaired. Isn't that what I just said (and you snipped)? I'm well-aware of the problem; I have to read Japanese, which (for comfort as a non-native) requires fonts with approximately twice the minimum resolution I can reasonably use for English. My reason for proposing that as default, though, is that if somebody requires bigger fonts or smaller screen, then really, shouldn't somebody with good eyes or equipment volunteer for that burden? Of course sometimes they gotta or they wanna, so we *must* keep the current format (or an incremental improvement on it) as an option. But I would expect that those with even mild impairment would self-select out of that job on average. Brad> From everything you've said, I think you would like Brad> Skip's mmfold.py script. I think you should check it out. I already do like it. Most of the moderators I know don't have the necessary shell access, though. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From erling.hellenas at runbox.com Tue Jan 3 13:19:09 2006 From: erling.hellenas at runbox.com (=?iso-8859-1?Q?Erling_Hellen=E4s?=) Date: Tue, 3 Jan 2006 13:19:09 +0100 Subject: [Mailman-Developers] Mailman/mnoGoSearch Integration Message-ID: <001701c6105f$e6a86150$fc01a8c0@home.local> Hi all ! I have created a mailgroup where you can look at how Mailman and mnoGoSearch can work together. In the mailgroup you can also get instructions and most of the programs needed. We can help each other with issues related to the integration of Mailman and mnoGoSearch. We can also discuss how to continue development related to this. Subscribe here: http://erlinghellenas.no-ip.com/mailman/listinfo/mmi Welcome ! Cheers, Erling Hellen?s From barry at python.org Tue Jan 3 18:03:39 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 03 Jan 2006 12:03:39 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <17335.31696.282198.15573@panix3.panix.com> <43B78D56.10502@equinephotoart.com> <1136168077.2302.70.camel@geddy.wooz.org> Message-ID: <1136307819.31078.14.camel@geddy.wooz.org> On Sun, 2006-01-01 at 22:10 -0600, Brad Knowles wrote: > But then we're getting dangerously close to tools like Active > Spam Killer or TMDA, which I am generally violently opposed to. I think they're different use cases. My main problem with such tools is when people email me first, I try to be helpful, and then have to go through extra dances to get that response to them. That wouldn't be the case with Mailman. In our case, we're actually reducing the dance people have to go through to get their message posted. The biggest danger is from abuse by replybots though, so we have to think about that. It's also a potential problem with subscription requests, and I see those and what I'm proposing as being very closely related. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060103/14b4398e/attachment.pgp From barry at python.org Tue Jan 3 18:07:15 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 03 Jan 2006 12:07:15 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> Message-ID: <1136308035.31063.18.camel@geddy.wooz.org> On Sun, 2006-01-01 at 22:05 -0600, Brad Knowles wrote: > For the larger ones, I'd like to see something like Skip's > "mmfold.py" script that could run locally on the same server where > the lists are located, so that no use of a web browser is required, > and so that the program could directly access the Python pickles in > question. That would make it a lot easier for me to manage some of > the lists on the other main site where I volunteer. I'm actually thinking we need /less/ magic in command line scripts, especially for typical user and admin tasks, because I think increasingly, fewer people have access to the command line (or know what to do with it when they've got it). > The admindb interface is one thing that needs improvement. But > I'd also like to see alternatives we could use that completely avoid > the use of any web browser, etc.... I'm keen on the idea of making Mailman access available via xmlrpc or somesuch, and then we can provide scripts that can be run on the client w/o requiring a browser. > IMAP would probably be an improvement over what we have now, > assuming you've got a decent IMAP client -- that's not necessarily a > valid assumption. But my experience is that IMAP falls down too > (especially depending on the IMAP server implementation), and you > need something even scalable when that happens. True. (Aside: why do all mail clients suck so much? :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060103/89b5d7a1/attachment.pgp From barry at python.org Tue Jan 3 18:13:49 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 03 Jan 2006 12:13:49 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <1136308429.31062.22.camel@geddy.wooz.org> On Tue, 2006-01-03 at 12:21 +0900, Stephen J. Turnbull wrote: > BAW> I find that my typical approach is to scan the summary, > BAW> opening any potential ham > > I assume you mean "ham = on-topic, spam = off-topic, possibly but not > necessarily UCE"? In this context, yep! > Our lists don't have a topicality problem, so I don't think I've ever > had need to open a post. The summary subject invariably shows spam > vs. ham. Often, but not always. I probably find myself tricked about 10% of the time. ;) > Unfortunately, you can't open a subscription request. I > know, the text doesn't show much in most cases, but we could look for > header forging. I don't understand why this distinction was made; > would it cost resources to give access to the subscription requests? Probably not. I'm sure there was a "good" reason buried in history, but it's not a bad idea to revisit this. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060103/1344023b/attachment.pgp From barry at python.org Tue Jan 3 18:21:51 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 03 Jan 2006 12:21:51 -0500 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <1136279387.9433.5.camel@localhost.localdomain> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> <1136279387.9433.5.camel@localhost.localdomain> Message-ID: <1136308911.31078.28.camel@geddy.wooz.org> On Tue, 2006-01-03 at 09:09 +0000, Nigel Metheringham wrote: > its a form so not quite like that - try > http://lists.DOMAIN.TLD/lists/admindb/LIST?adminpw=yourpassword > > Personally I have a bookmarks folder within Firefox with all the lists I > handle as bookmarks within that set up with password. I then just right > click on the folder, "open in tabs" and have all the list admin > interfaces opened at once... Brilliant! > [I'm reasonably relaxed about having the list moderation or admin > passwords available in clear on my box - I think its a reasonable trade > off compared to the other options. I *always* log off my box (so close > the browser) when I'm going to be away for more than a couple of hours, > so keeping a browser running for days is not an option] I'm also not worried about it. I almost never work on a multi-user box anyway. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060103/c4a87770/attachment.pgp From lists05 at equinephotoart.com Tue Jan 3 19:13:57 2006 From: lists05 at equinephotoart.com (JC Dill) Date: Tue, 03 Jan 2006 10:13:57 -0800 Subject: [Mailman-Developers] Access options for command line tasks In-Reply-To: <1136308035.31063.18.camel@geddy.wooz.org> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <1136308035.31063.18.camel@geddy.wooz.org> Message-ID: <43BABEE5.80100@equinephotoart.com> Barry Warsaw wrote: >I'm actually thinking we need /less/ magic in command line scripts, >especially for typical user and admin tasks, because I think >increasingly, fewer people have access to the command line (or know what >to do with it when they've got it). > > Wearing my product manager's hat here - I request that EVERY administrative function be usable in all 3 methods - via the command line, via email, and via the admin webpage. The underlying function would always be the command line task but it could then be called via the other 2 methods. If this is not already a feature of MM3, I'm formally making it a request now. Please??? As a work-around for those who don't have direct command line access, and for ease of implementation in the web UI, I suggest we build a tool that takes a line of text via web form input, processes it on the command line, and spits the resulting command line text back out on a web page (concatenated with previous (in the past x minutes) commands and their results from this user, e.g. a screenshot of the command line interface). This tool would be a work-around for all the places where we haven't built a more elegant web interface for the given command line tool/function. The server admin should have options to enable or disable this access when installing mailman, or when installing/configuring each new list. Are there security implications with this that are not easily addressed? Finally, to address Barry's concern that many people don't know what to do with the command line - even if they don't know what to do, it's still useful for them to have access to it. I've done a lot of work on other people's PCs where I access the command line and type a few commands and this gets me info to help me solve their problem. In some cases I talk them thru using the command line themselves (e.g. tell them how to do a ping, or tracert). If this tool was not available at all, it would make it much harder for me to help them fix their problems. I expect that similar things will happen when experienced mailing list server admins try to help novices, so we avoid designing a system without the tools that the experienced server admins need to get the job done. jc From brad at stop.mail-abuse.org Wed Jan 4 05:31:10 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Tue, 3 Jan 2006 22:31:10 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <87acedml5f.fsf@tleepslib.sk.tsukuba.ac.jp> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <87oe2um1x9.fsf@tleepslib.sk.tsukuba.ac.jp> <87k6dhn23u.fsf@tleepslib.sk.tsukuba.ac.jp> <87acedml5f.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: At 11:38 PM +0900 2006-01-03, Stephen J. Turnbull wrote: > My reason for proposing that as default, though, is that if somebody > requires bigger fonts or smaller screen, then really, shouldn't > somebody with good eyes or equipment volunteer for that burden? I don't think you can make that assumption. Many people are just barely getting by on their own, and we've even got a lot of hosting sites running Plesk or crap like cPanel or MacOS X Server, or with heavily customized "normal" Mailman configurations, and yet they get absolutely no site administrator support at all. Any default change like this would have to be something that a list moderator could configure for themselves, without any intervention on the part of the list administrator, and especially without any intervention on the part of the site administrator. > But I would expect that those with even mild impairment would > self-select out of that job on average. When you're in a ghetto and no way out, when the power & gas company starts sending you invoices that are not intelligible and your heating bill goes up by a factor of ten in a single month, what do you think people are going to do? > Brad> From everything you've said, I think you would like > Brad> Skip's mmfold.py script. I think you should check it out. > > I already do like it. Most of the moderators I know don't have the > necessary shell access, though. All you need is shell access somewhere. It does everything over HTTP, so it doesn't have to be run on the server where the list is hosted. That should make the job a lot easier, albeit still not as simple as pointing their existing favourite web browser at the appropriate admindb page. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From brad at stop.mail-abuse.org Wed Jan 4 05:38:21 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Tue, 3 Jan 2006 22:38:21 -0600 Subject: [Mailman-Developers] On allowing any list member to be an email moderator In-Reply-To: <1136308035.31063.18.camel@geddy.wooz.org> References: <17335.5896.569912.550753@panix3.panix.com> <43B744CA.7090607@equinephotoart.com> <1136167023.2302.61.camel@geddy.wooz.org> <1136308035.31063.18.camel@geddy.wooz.org> Message-ID: At 12:07 PM -0500 2006-01-03, Barry Warsaw wrote: > I'm actually thinking we need /less/ magic in command line scripts, > especially for typical user and admin tasks, because I think > increasingly, fewer people have access to the command line (or know what > to do with it when they've got it). The command-line scripts are not for your average joe-user sites. The command-line scripts are for people like me, Chuq, Skip, and anyone else who operates or may want to operate a larger site, and has not written such a tool for themselves -- at least, not yet. That is, unless you can build an admin interface that can deal with hundreds of thousands of queued messages without killing both the server and the client. > I'm keen on the idea of making Mailman access available via xmlrpc or > somesuch, and then we can provide scripts that can be run on the client > w/o requiring a browser. I'm not opposed to that, but I'd want to make sure those kinds of tools could also be run on the server where the list is hosted. >> IMAP would probably be an improvement over what we have now, >> assuming you've got a decent IMAP client -- that's not necessarily a >> valid assumption. But my experience is that IMAP falls down too >> (especially depending on the IMAP server implementation), and you >> need something even scalable when that happens. > > True. (Aside: why do all mail clients suck so much? :) All mail servers suck, too. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From j.e.vanbaal at uvt.nl Wed Jan 4 16:57:33 2006 From: j.e.vanbaal at uvt.nl (Joost van Baal) Date: Wed, 4 Jan 2006 16:57:33 +0100 Subject: [Mailman-Developers] preferred documentation format, sources for documentation in admin/www Message-ID: <20060104155733.GA29152@banach.uvt.nl> Hi, I'd like to document my patch for making Mailman OpenPGP and S/MIME aware ( http://non-gnu.uvt.nl/pub/mailman/ ). I'd like this documentation to integrate nicely with the official Mailman documenation. What is the preferred format for creating documentation? I believe it's LaTeX. However, the sources for mailman-{admin,install,member}.{ps,pdf,txt} are neither in cvs/sf/mailman/mailman/admin/www nor in mailman-2.1.7.tgz . It'd help me if these sources got published somewhere. Thanks, Bye, Joost -- Joost van Baal http://abramowitz.uvt.nl/ Tilburg University j.e.vanbaal at uvt.nl The Netherlands -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060104/028b0a42/attachment.pgp From msapiro at value.net Wed Jan 4 17:58:48 2006 From: msapiro at value.net (Mark Sapiro) Date: Wed, 4 Jan 2006 08:58:48 -0800 Subject: [Mailman-Developers] Serious Archiving problem in 2.1.7 Message-ID: There is a potentially serious archiving problem in 2.1.7 involving a UnicodeError when obscuring email addresses in the body of posts destined for the .txt file. The problem is reported at and was also reported by a different user on the mailman-users list at . I observed one manifestation of this myself and mentioned it to Tokio and Barry, but I didn't have time to follow up before going away last week. The problem can abort a bin/arch run and can also result in a list post not being archived (it will be shunted in this case). I have worked up a patch which avoids the problem in all the cases I have seen. (My original patch simply ignored the UnicodeError which allowed unobscured email addresses in the .txt file for the problem messages - my current patch avoids the UnicodeError so addresses are properly obscured). The patch is attached to the above tracker item. The purpose of this post is twofold. First, I would like independent review of the patch, as I still don't fully understand the reason for the error in all cases and the i18n implications. Second, other than committing the patch to CVS, is there a mechanism for making it more widely visible. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at python.org Wed Jan 4 20:11:01 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 04 Jan 2006 14:11:01 -0500 Subject: [Mailman-Developers] Serious Archiving problem in 2.1.7 In-Reply-To: References: Message-ID: <1136401861.10342.20.camel@geddy.wooz.org> On Wed, 2006-01-04 at 08:58 -0800, Mark Sapiro wrote: > The purpose of this post is twofold. First, I would like independent > review of the patch, as I still don't fully understand the reason for > the error in all cases and the i18n implications. I'll try to find some time to do this, if Tokio doesn't get to it first. > Second, other than committing the patch to CVS, is there a mechanism > for making it more widely visible. Yes; we can make the patch available on the website and announce it on mailman-announce. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060104/b00cdc7c/attachment.pgp From lionel at mamane.lu Wed Jan 4 22:01:56 2006 From: lionel at mamane.lu (Lionel Elie Mamane) Date: Wed, 4 Jan 2006 22:01:56 +0100 Subject: [Mailman-Developers] subscription_cancel in Cgi/confirm.py lacks SIGTERM parachute Message-ID: <20060104210156.GA14809@capsaicin.mamane.lu> Hi, I noticed that in 2.1.6, subscription_cancel in Mailman/Cgi/confirm.py doesn't have the same SIGTERM unlocking parachute that subscription_confirm has; is this done on purpose, and why? Is there no risk of leaving a stale lock with what subscription_cancel does? Without more info, I'd say this is "simply" oversight and that it would be needed. Your opinion? -- Lionel From tkikuchi at is.kochi-u.ac.jp Thu Jan 5 01:00:38 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Thu, 05 Jan 2006 09:00:38 +0900 Subject: [Mailman-Developers] Serious Archiving problem in 2.1.7 In-Reply-To: <43BC5F6E.5020702@is.kochi-u.ac.jp> References: <1136401861.10342.20.camel@geddy.wooz.org> <43BC5F6E.5020702@is.kochi-u.ac.jp> Message-ID: <43BC61A6.8060400@is.kochi-u.ac.jp> Tokio Kikuchi wrote: >> >>>The purpose of this post is twofold. First, I would like independent >>>review of the patch, as I still don't fully understand the reason for >>>the error in all cases and the i18n implications. >> >> >>I'll try to find some time to do this, if Tokio doesn't get to it first. > > > Yeah, I'll look at it NOW. :-) At my first glance, it will only work for english lists in which the replace parameter _(' at ') is the same both in unicode and ascii. I'll try to test and generate i18n patch today. -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From tkikuchi at is.kochi-u.ac.jp Thu Jan 5 00:51:10 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Thu, 05 Jan 2006 08:51:10 +0900 Subject: [Mailman-Developers] Serious Archiving problem in 2.1.7 In-Reply-To: <1136401861.10342.20.camel@geddy.wooz.org> References: <1136401861.10342.20.camel@geddy.wooz.org> Message-ID: <43BC5F6E.5020702@is.kochi-u.ac.jp> Barry Warsaw wrote: > On Wed, 2006-01-04 at 08:58 -0800, Mark Sapiro wrote: > > >>The purpose of this post is twofold. First, I would like independent >>review of the patch, as I still don't fully understand the reason for >>the error in all cases and the i18n implications. > > > I'll try to find some time to do this, if Tokio doesn't get to it first. Yeah, I'll look at it NOW. :-) > > >>Second, other than committing the patch to CVS, is there a mechanism >>for making it more widely visible. > > > Yes; we can make the patch available on the website and announce it on > mailman-announce. > > -Barry -- ???? tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ ?780-8520 ????????????? From msapiro at value.net Thu Jan 5 04:02:09 2006 From: msapiro at value.net (Mark Sapiro) Date: Wed, 4 Jan 2006 19:02:09 -0800 Subject: [Mailman-Developers] Serious Archiving problem in 2.1.7 In-Reply-To: <43BC61A6.8060400@is.kochi-u.ac.jp> Message-ID: Tokio Kikuchi wrote: > >At my first glance, it will only work for english lists in which the >replace parameter _(' at ') is the same both in unicode and ascii. > >I'll try to test and generate i18n patch today. I see the point. Unfortunately I only tested English and French and in French, _(' at ') translates to ' at '. Here is my suggestion for a revised patch (tested with en, es, fr and ja - I can't read the Japanese, but it 'looks' OK): --- f:/Mailman/mailman-2.1.7/Mailman/Archiver/HyperArch.py 2005-12-30 10:50:07.000000000 -0800 +++ f:/test-mailman/Mailman/Archiver/HyperArch.py 2006-01-04 18:21:50.750000000 -0800 @@ -569,16 +569,21 @@ if d['_message_id']: headers.append('Message-ID: %(_message_id)s') body = EMPTYSTRING.join(self.body) - if isinstance(body, types.UnicodeType): - body = body.encode(Utils.GetCharSet(self._lang), 'replace') + # MAS: Coerce the body to Unicode and replace any invalid characters. + langcset = Utils.GetCharSet(self._lang) + if not isinstance(body, types.UnicodeType): + body = body.decode(langcset, 'replace') if mm_cfg.ARCHIVER_OBSCURES_EMAILADDRS: otrans = i18n.get_translation() try: i18n.set_language(self._lang) body = re.sub(r'([-+,.\w]+)@([-+.\w]+)', - '\g<1>' + _(' at ') + '\g<2>', body) + '\g<1>' + unicode(_(' at '), langcset) + + '\g<2>', body) finally: i18n.set_translation(otrans) + # MAS: Return body to character set of article. + body = body.encode(langcset, 'replace') return NL.join(headers) % d + '\n\n' + body + '\n' def _set_date(self, message): -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tkikuchi at is.kochi-u.ac.jp Thu Jan 5 04:00:05 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Thu, 05 Jan 2006 12:00:05 +0900 Subject: [Mailman-Developers] Serious Archiving problem in 2.1.7 In-Reply-To: <43BC61A6.8060400@is.kochi-u.ac.jp> References: <1136401861.10342.20.camel@geddy.wooz.org> <43BC5F6E.5020702@is.kochi-u.ac.jp> <43BC61A6.8060400@is.kochi-u.ac.jp> Message-ID: <43BC8BB5.8030907@is.kochi-u.ac.jp> > At my first glance, it will only work for english lists in which the > replace parameter _(' at ') is the same both in unicode and ascii. > > I'll try to test and generate i18n patch today. > Done. I'll commit it in the CVS if no problems. I really hate to read code in the Archiver directory. :-( We'd rather develope new archiver from scratch for 2.2 (or 3.0). New design should be in XHTML/CSS/UTF-8. -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From chris at scorpion.nl Thu Jan 5 11:50:43 2006 From: chris at scorpion.nl (Christiaan den Besten) Date: Thu, 5 Jan 2006 11:50:43 +0100 Subject: [Mailman-Developers] Mailman 2.1.5 'listinfo' command slow ... Message-ID: <024401c611e5$e05e3090$3d64880a@speedy> Hi all ! Could someone please tell me why the "config.pck"'s of every list is completely read, copy'ed to config.pck.last etc on a '/usr/bin/python -S /usr/lib/mailman/scripts/driver listinfo' call. We have some lists with a lot of recipients which results in a combined size of all config.pck's upto 40Mb. Having to write this much data on a 'simple' (?) listinfo call makes the call very slow (5s or something like that ...). Why does it update the configs? Should it not only read info (list names/descriptions). Perhaps it would be a nice idea to split the config and the recipients in separate files to prevent this ? Or has all this been fixed in upstream versions already? Any hint/tips would be very appreciated! bye, Chris From brad at stop.mail-abuse.org Fri Jan 6 00:50:33 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Thu, 5 Jan 2006 17:50:33 -0600 Subject: [Mailman-Developers] preferred documentation format, sources for documentation in admin/www In-Reply-To: <20060104155733.GA29152@banach.uvt.nl> References: <20060104155733.GA29152@banach.uvt.nl> Message-ID: At 4:57 PM +0100 2006-01-04, Joost van Baal wrote: > I'd like to document my patch for making Mailman OpenPGP and S/MIME > aware ( http://non-gnu.uvt.nl/pub/mailman/ ). Ideally, the patch and documentation should be uploaded to the SourceForge tracker. That way Tokio, Mark, and Barry can look over it and try to get that incorporated into the baseline code. > I'd like this > documentation to integrate nicely with the official Mailman > documenation. What is the preferred format for creating documentation? As for the preferred documentation format, I'll let someone who knows more about that subject provide an answer for you. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From tkikuchi at is.kochi-u.ac.jp Fri Jan 6 01:01:45 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Fri, 06 Jan 2006 09:01:45 +0900 Subject: [Mailman-Developers] Serious Archiving problem in 2.1.7 In-Reply-To: References: Message-ID: <43BDB369.5080901@is.kochi-u.ac.jp> Thank you Mark. It looks like we reached the same conclusion. One note: I prefer to use unicode() than decode() because the latter is not supported in Python 2.1 and looks like fuzzy in declaring unicode string (or else). Mark Sapiro wrote: > Tokio Kikuchi wrote: > >>At my first glance, it will only work for english lists in which the >>replace parameter _(' at ') is the same both in unicode and ascii. >> >>I'll try to test and generate i18n patch today. > > > > I see the point. Unfortunately I only tested English and French and in > French, _(' at ') translates to ' at '. Here is my suggestion for a > revised patch (tested with en, es, fr and ja - I can't read the > Japanese, but it 'looks' OK): > > --- f:/Mailman/mailman-2.1.7/Mailman/Archiver/HyperArch.py > 2005-12-30 10:50:07.000000000 -0800 > +++ f:/test-mailman/Mailman/Archiver/HyperArch.py 2006-01-04 > 18:21:50.750000000 -0800 > @@ -569,16 +569,21 @@ > if d['_message_id']: > headers.append('Message-ID: %(_message_id)s') > body = EMPTYSTRING.join(self.body) > - if isinstance(body, types.UnicodeType): > - body = body.encode(Utils.GetCharSet(self._lang), 'replace') > + # MAS: Coerce the body to Unicode and replace any invalid > characters. > + langcset = Utils.GetCharSet(self._lang) > + if not isinstance(body, types.UnicodeType): > + body = body.decode(langcset, 'replace') > if mm_cfg.ARCHIVER_OBSCURES_EMAILADDRS: > otrans = i18n.get_translation() > try: > i18n.set_language(self._lang) > body = re.sub(r'([-+,.\w]+)@([-+.\w]+)', > - '\g<1>' + _(' at ') + '\g<2>', body) > + '\g<1>' + unicode(_(' at '), langcset) > + + '\g<2>', body) > finally: > i18n.set_translation(otrans) > + # MAS: Return body to character set of article. > + body = body.encode(langcset, 'replace') > return NL.join(headers) % d + '\n\n' + body + '\n' > > def _set_date(self, message): > -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From barry at python.org Fri Jan 6 03:03:39 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 05 Jan 2006 21:03:39 -0500 Subject: [Mailman-Developers] preferred documentation format, sources for documentation in admin/www In-Reply-To: <20060104155733.GA29152@banach.uvt.nl> References: <20060104155733.GA29152@banach.uvt.nl> Message-ID: <1136513019.15418.89.camel@geddy.wooz.org> On Wed, 2006-01-04 at 16:57 +0100, Joost van Baal wrote: > I'd like to document my patch for making Mailman OpenPGP and S/MIME > aware ( http://non-gnu.uvt.nl/pub/mailman/ ). I'd like this > documentation to integrate nicely with the official Mailman > documenation. What is the preferred format for creating documentation? > > I believe it's LaTeX. However, the sources for > mailman-{admin,install,member}.{ps,pdf,txt} are neither in > cvs/sf/mailman/mailman/admin/www nor in mailman-2.1.7.tgz . It'd help > me if these sources got published somewhere. LaTeX is the preferred format for the documentation source. We process our .tex files with the mkhowto program from the Python distribution. These rules apply for Mailman too: http://www.python.org/doc/current/doc/doc.html The source files are available in CVS, here: http://cvs.sourceforge.net/viewcvs.py/mailman/mailman/doc/?only_with_tag=Release_2_1-maint Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060105/7af1a213/attachment.pgp From barry at python.org Fri Jan 6 03:09:14 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 05 Jan 2006 21:09:14 -0500 Subject: [Mailman-Developers] Serious Archiving problem in 2.1.7 In-Reply-To: <43BC8BB5.8030907@is.kochi-u.ac.jp> References: <1136401861.10342.20.camel@geddy.wooz.org> <43BC5F6E.5020702@is.kochi-u.ac.jp> <43BC61A6.8060400@is.kochi-u.ac.jp> <43BC8BB5.8030907@is.kochi-u.ac.jp> Message-ID: <1136513354.15418.94.camel@geddy.wooz.org> On Thu, 2006-01-05 at 12:00 +0900, Tokio Kikuchi wrote: > I really hate to read code in the Archiver directory. :-( > We'd rather develope new archiver from scratch for 2.2 (or 3.0). New > design should be in XHTML/CSS/UTF-8. Oh most definitely. The current archiver is a bastard bolt-on from a branch of an unmaintained separate project. ;) We've discussed archivers on this list many times, and I've begged for a champion for the archiver with little luck. Maybe next time will be the charm. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060105/fc5f1d13/attachment.pgp From msapiro at value.net Fri Jan 6 04:54:03 2006 From: msapiro at value.net (Mark Sapiro) Date: Thu, 5 Jan 2006 19:54:03 -0800 Subject: [Mailman-Developers] Mailman 2.1.5 'listinfo' command slow ... In-Reply-To: <024401c611e5$e05e3090$3d64880a@speedy> Message-ID: Christiaan den Besten wrote: > >Could someone please tell me why the "config.pck"'s of every list is completely read, copy'ed to config.pck.last etc on a >'/usr/bin/python -S /usr/lib/mailman/scripts/driver listinfo' call. listinfo instantiates every list which involves reading the config.pck because it needs to know whether the list is 'advertised' and in a virtual hosts environment whether the list 'belongs' to the listinfo host. It also needs to get the list's real_name and description for listing on the listinfo page. >We have some lists with a lot of recipients which results in a combined size of all config.pck's upto 40Mb. Having to write this >much data on a 'simple' (?) listinfo call makes the call very slow (5s or something like that ...). > >Why does it update the configs? Should it not only read info (list names/descriptions). Perhaps it would be a nice idea to split the >config and the recipients in separate files to prevent this ? Normally the process you describe, reading the config.pck, pickling to a temp file, renaming config.pck -> config.pck.last and renaming the temp file -> config.pck, only occurs when the list is locked and saved, neither of which are done by listinfo. The only other time any writing is done is when the config.pck is missing or corrupt and one of the fallback files config.pck.last, config.db or config.db.last is used. Are you sure the command /usr/bin/python -S /usr/lib/mailman/scripts/driver listinfo does any writing of config.pck files. It doesn't in my 2.1.7 installation, and I don't see why 2.1.5 would be any different. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From j.e.vanbaal at uvt.nl Fri Jan 6 06:42:49 2006 From: j.e.vanbaal at uvt.nl (Joost van Baal) Date: Fri, 6 Jan 2006 06:42:49 +0100 Subject: [Mailman-Developers] preferred documentation format, sources for documentation in admin/www In-Reply-To: <1136513019.15418.89.camel@geddy.wooz.org> References: <20060104155733.GA29152@banach.uvt.nl> <1136513019.15418.89.camel@geddy.wooz.org> Message-ID: <20060106054249.GT28159@banach.uvt.nl> Hi, Brad and Barry: thanks for your reply! On Thu, Jan 05, 2006 at 09:03:39PM -0500, Barry Warsaw wrote: > On Wed, 2006-01-04 at 16:57 +0100, Joost van Baal wrote: > > > I'd like to document my patch for making Mailman OpenPGP and S/MIME > > aware ( http://non-gnu.uvt.nl/pub/mailman/ ). I'd like this > > documentation to integrate nicely with the official Mailman > > documenation. What is the preferred format for creating documentation? > > > > I believe it's LaTeX. However, the sources for > > mailman-{admin,install,member}.{ps,pdf,txt} are neither in > > cvs/sf/mailman/mailman/admin/www nor in mailman-2.1.7.tgz . It'd help > > me if these sources got published somewhere. > > LaTeX is the preferred format for the documentation source. We process > our .tex files with the mkhowto program from the Python distribution. > These rules apply for Mailman too: > > http://www.python.org/doc/current/doc/doc.html > > The source files are available in CVS, here: > > http://cvs.sourceforge.net/viewcvs.py/mailman/mailman/doc/?only_with_tag=Release_2_1-maint Oops, of course. Apparently I got confused by the rsync calls in the Makefile in admin/www, and stopped searching too early. I'll start documenting my Secure List Server patch using this framework (it's patch #1167696, BTW). Bye, Joost -- Joost van Baal http://abramowitz.uvt.nl/ Tilburg University j.e.vanbaal at uvt.nl The Netherlands -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060106/eba8e5c8/attachment.pgp From barry at python.org Wed Jan 11 04:23:18 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 10 Jan 2006 22:23:18 -0500 Subject: [Mailman-Developers] [Mailman-i18n] A better way of doing templates? In-Reply-To: References: <1132786513.20488.153.camel@geddy.wooz.org> <43851AF3.7060303@is.kochi-u.ac.jp> <1133910469.15748.60.camel@geddy.wooz.org> <43962E84.30804@is.kochi-u.ac.jp> <20051207082005.GB15456@rezo.net> Message-ID: <1136949798.10327.52.camel@geddy.wooz.org> On Wed, 2005-12-07 at 13:08 +0100, Bj?rn Vermo wrote: > May I suggest a zero-lingual base version? The only way to make software > truly language independent is to avoid the concept of a "default language". > If everybody has to get a language file, developers will be forced to > think about localization issues. That's difficult. IIUC what you propose is a more 'catgets' style than a 'gettext' style of coding. E.g. where each human readable message in the source is given a number so that even English messages must be looked up in the catalog. The alternative is what Mailman currently uses -- the English message serves as the lookup key in the catalog. Ages ago we experimented with both styles. I highly dislike message numbers and prefer gettext, although admittedly I'm probably biased by being a native English speaker. As for Fil's opinion on splitting the distribution up by languages, I kind of agree. I think we should continue to produce fat distributions with all languages, possibly providing an English-only (i.e. no catalogs) distribution as a thin alternative. I'm still +1 though on Tokio's changes to allow the site admin to /install/ Mailman with fewer languages. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060110/73c54ba7/attachment.pgp From erling.hellenas at runbox.com Wed Jan 11 14:56:50 2006 From: erling.hellenas at runbox.com (=?iso-8859-1?Q?Erling_Hellen=E4s?=) Date: Wed, 11 Jan 2006 14:56:50 +0100 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? Message-ID: <001701c616b6$df6401d0$fc01a8c0@home.local> Hi all ! It seems like some kind of automatic unsubscribe would be a nice maillist function. It could relieve much of the burden of keeping the member list up to date from the administrators. 1. There could be an inactivity warning after a configurable time of inactivity. If the user doesn't answer, he is automatically unsubsribed. 2. There could also be a possibility to send an activity check mail to all members. Those who doesn't answer are automatically unsubscribed. Those who are unsubscribed get a polite letter telling that they are welcome back, of course. Erling Hellen?s From benson at music.mcgill.ca Wed Jan 11 18:41:26 2006 From: benson at music.mcgill.ca (David Benson) Date: Wed, 11 Jan 2006 12:41:26 -0500 Subject: [Mailman-Developers] creating a list of moderators Message-ID: <200601111741.k0BHfQNK018366@mailscan2.cc.mcgill.ca> Hi, I'm a site administrator and I'd like to be able to automatically compile a list of 'list moderators' so that I can send them all announcements. I can't find a way to do this in the current version, and I saw similar functionality being requested on the 'wishlist', so I assume that it has yet to be implemented. I'm new to Mailman, but I'd like to try writing a script to do this. Can anyone advise me on how to go about it/where to start? Any ideas? Many thanks! Dave Benson From rmg at terc.edu Wed Jan 11 20:41:04 2006 From: rmg at terc.edu (Robby Griffin) Date: Wed, 11 Jan 2006 14:41:04 -0500 Subject: [Mailman-Developers] creating a list of moderators In-Reply-To: <200601111741.k0BHfQNK018366@mailscan2.cc.mcgill.ca> References: <200601111741.k0BHfQNK018366@mailscan2.cc.mcgill.ca> Message-ID: Um, take a look at `bin/list_owners -m`. It might be nice to have a sort of live mailing list for them rather than a list of their addresses, though. --Robby On Jan 11, 2006, at 12:41, David Benson wrote: > I'm a site administrator and I'd like to be able to automatically > compile a > list of 'list moderators' so that I can send them all announcements. I > can't find a way to do this in the current version, and I saw similar > functionality being requested on the 'wishlist', so I assume that it > has yet > to be implemented. > > I'm new to Mailman, but I'd like to try writing a script to do this. > Can > anyone advise me on how to go about it/where to start? From jag at fsf.org Wed Jan 11 20:48:34 2006 From: jag at fsf.org (Joshua Ginsberg) Date: Wed, 11 Jan 2006 14:48:34 -0500 Subject: [Mailman-Developers] creating a list of moderators In-Reply-To: <200601111741.k0BHfQNK018366@mailscan2.cc.mcgill.ca> References: <200601111741.k0BHfQNK018366@mailscan2.cc.mcgill.ca> Message-ID: <1137008914.13112.26.camel@localhost.localdomain> Something like this? import cPickle, os, os.path, sys MAILMANHOME='/var/mailman' ;# change to suit sys.path.append(MAILMANHOME) moderators = [] for dir in os.listdir(os.path.join(MAILMANHOME, 'lists')): o = cPickle.load(open(os.path.join(MAILMANHOME, 'lists', dir, 'config.pck'))) for email in o['moderator']: email in moderators or moderators.append(email) return moderators On Wed, 2006-01-11 at 12:41 -0500, David Benson wrote: > Hi, > > I'm a site administrator and I'd like to be able to automatically compile a > list of 'list moderators' so that I can send them all announcements. I > can't find a way to do this in the current version, and I saw similar > functionality being requested on the 'wishlist', so I assume that it has yet > to be implemented. > > I'm new to Mailman, but I'd like to try writing a script to do this. Can > anyone advise me on how to go about it/where to start? > > Any ideas? > > Many thanks! > > > Dave Benson > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/jag%40fsf.org > > Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp > -- Joshua Ginsberg Free Software Foundation - Senior Systems Administrator -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060111/de2de163/attachment.pgp From hmm at heeg.de Wed Jan 11 20:44:26 2006 From: hmm at heeg.de (Hans-Martin Mosner) Date: Wed, 11 Jan 2006 20:44:26 +0100 Subject: [Mailman-Developers] creating a list of moderators In-Reply-To: <200601111741.k0BHfQNK018366@mailscan2.cc.mcgill.ca> References: <200601111741.k0BHfQNK018366@mailscan2.cc.mcgill.ca> Message-ID: <43C5601A.1050300@heeg.de> David Benson wrote: >Hi, > >I'm a site administrator and I'd like to be able to automatically compile a >list of 'list moderators' so that I can send them all announcements. I >can't find a way to do this in the current version, and I saw similar >functionality being requested on the 'wishlist', so I assume that it has yet >to be implemented. > > It's already mostly there: The command "list_owners -m" prints all owners and moderators of all mailing lists on your machine. Getting these into a mailing list should not be that hard... Cheers, Hans-Martin From wilder at eskimo.com Thu Jan 12 04:52:28 2006 From: wilder at eskimo.com (Dan Wilder) Date: Wed, 11 Jan 2006 19:52:28 -0800 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? In-Reply-To: <001701c616b6$df6401d0$fc01a8c0@home.local> References: <001701c616b6$df6401d0$fc01a8c0@home.local> Message-ID: <20060112035228.GA32197@eskimo.com> On Wed, Jan 11, 2006 at 02:56:50PM +0100, Erling Hellen?s wrote: > Hi all ! > > It seems like some kind of automatic unsubscribe would be a nice maillist > function. It could relieve much of the burden of keeping the member list up > to date from the administrators. > > 1. There could be an inactivity warning after a configurable time of > inactivity. If the user doesn't answer, he is automatically unsubsribed. > 2. There could also be a possibility to send an activity check mail to all > members. Those who doesn't answer are automatically unsubscribed. Personally I'd call that a big nuisance. As a long-time lurker on the list, who nonetheless reads it, I'd much prefer not to be dropped off just because I have not much to say. I'd expect this is not an uncommon situation. Would that fewer people with little to say, would say little, on some other lists I have been on. Having administered lists of up to 15k subscribers using Mailman, I'll also submit that unsubscribing people with legitimate and still-active email addresses isn't much of a burden. Certainly not compared with the burdens of - unsubscribing people whose email boxes are no longer there but whose former ISPs file all identifying marks off the bounce messages, for example, the email address of the intended recipient: From: Postmaster at im.clueless.com Your recent mail to a user at im.clueless.com was not deliverable. Please resend the message. - spam, spam, spam, and spam - locating and unsubscribing people who can't be troubled to unsubscribe but who instead start submitting anonymous complaints to the likes of SpamCop - locating and unsubscribing people who request to be unsubscribed but who are not subscribers, and who are not technically savvy enough to send you all the received headers so you could at least hazard a guess as to the route (forwards, mailing lists subscribed to mailing lists etc) by which the mail reached them The check of all members that really helps in my experience is the monthly password reminder. It reminds a lot of people who are not interested any longer to unsubcribe, and it elicits bounces from inactive mailboxes, most of which are susceptible to diagnosis by Mailman's excellent bounce handling. -- Dan Wilder From erling.hellenas at runbox.com Thu Jan 12 15:24:42 2006 From: erling.hellenas at runbox.com (=?iso-8859-1?Q?Erling_Hellen=E4s?=) Date: Thu, 12 Jan 2006 15:24:42 +0100 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? In-Reply-To: <20060112035228.GA32197@eskimo.com> Message-ID: <000201c61783$ee02e6c0$fc01a8c0@home.local> Hi all ! Thanks for the comment Dan. Automatic unsubscription would be optional, of course. This was not about unsubscribing people who have nothing to say, just about asking inactive people if they still want to be subscribers, and require that they press Reply and Send if they want. It seems if those maillists of yours only contained the maybe 1 k active participants instead of 15 k, maybe all those other problems would disappear into thin air. Cheers, Erling Hellen?s From jag at fsf.org Thu Jan 12 17:33:32 2006 From: jag at fsf.org (Joshua Ginsberg) Date: Thu, 12 Jan 2006 11:33:32 -0500 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? In-Reply-To: <20060112035228.GA32197@eskimo.com> References: <001701c616b6$df6401d0$fc01a8c0@home.local> <20060112035228.GA32197@eskimo.com> Message-ID: <1137083613.13112.44.camel@localhost.localdomain> Perhaps an interesting compromise might be to add to config.pck a key "last_post" whose value is a dictionary of email:time 9-tuple pairs. That way, folks like Erling could write a script to go ahead and do this fu, and it might be an interesting metric for other purposes as well. -jag On Wed, 2006-01-11 at 19:52 -0800, Dan Wilder wrote: > On Wed, Jan 11, 2006 at 02:56:50PM +0100, Erling Hellen?s wrote: > > Hi all ! > > > > It seems like some kind of automatic unsubscribe would be a nice maillist > > function. It could relieve much of the burden of keeping the member list up > > to date from the administrators. > > > > 1. There could be an inactivity warning after a configurable time of > > inactivity. If the user doesn't answer, he is automatically unsubsribed. > > 2. There could also be a possibility to send an activity check mail to all > > members. Those who doesn't answer are automatically unsubscribed. > > Personally I'd call that a big nuisance. > > As a long-time lurker on the list, who nonetheless reads it, I'd much > prefer not to be dropped off just because I have not much to say. I'd > expect this is not an uncommon situation. > > Would that fewer people with little to say, would say little, on some > other lists I have been on. > > Having administered lists of up to 15k subscribers using Mailman, I'll > also submit that unsubscribing people with legitimate and still-active > email addresses isn't much of a burden. Certainly not compared with > the burdens of > > - unsubscribing people whose email boxes are no longer there but whose > former ISPs file all identifying marks off the bounce messages, for > example, the email address of the intended recipient: > > From: Postmaster at im.clueless.com > > Your recent mail to a user at im.clueless.com was not deliverable. > Please resend the message. > > - spam, spam, spam, and spam > > - locating and unsubscribing people who can't be troubled to unsubscribe > but who instead start submitting anonymous complaints to the likes > of SpamCop > > - locating and unsubscribing people who request to be unsubscribed > but who are not subscribers, and who are not technically savvy > enough to send you all the received headers so you could at least > hazard a guess as to the route (forwards, mailing lists subscribed > to mailing lists etc) by which the mail reached them > > The check of all members that really helps in my experience is the > monthly password reminder. It reminds a lot of people who are not > interested any longer to unsubcribe, and it elicits bounces from > inactive mailboxes, most of which are susceptible to diagnosis > by Mailman's excellent bounce handling. > -- Joshua Ginsberg Free Software Foundation - Senior Systems Administrator -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060112/c3c925ec/attachment.pgp From msapiro at value.net Fri Jan 13 02:51:48 2006 From: msapiro at value.net (Mark Sapiro) Date: Thu, 12 Jan 2006 17:51:48 -0800 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? In-Reply-To: <1137083613.13112.44.camel@localhost.localdomain> Message-ID: Joshua Ginsberg wrote: > >Perhaps an interesting compromise might be to add to config.pck a key >"last_post" whose value is a dictionary of email:time 9-tuple pairs. >That way, folks like Erling could write a script to go ahead and do this >fu, and it might be an interesting metric for other purposes as well. This could prove to be a real burden to sites that have LARGE lists. Adding yet another dictionary with 100,000 keys to the list object could have significant impact. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From jwblist at loricamail.com Fri Jan 13 03:08:42 2006 From: jwblist at loricamail.com (John W. Baxter) Date: Thu, 12 Jan 2006 18:08:42 -0800 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? In-Reply-To: Message-ID: On 1/12/06 5:51 PM, "Mark Sapiro" wrote: > Joshua Ginsberg wrote: >> >> Perhaps an interesting compromise might be to add to config.pck a key >> "last_post" whose value is a dictionary of email:time 9-tuple pairs. >> That way, folks like Erling could write a script to go ahead and do this >> fu, and it might be an interesting metric for other purposes as well. > > This could prove to be a real burden to sites that have LARGE lists. > Adding yet another dictionary with 100,000 keys to the list object > could have significant impact. Aimed primarily at original poster, whose name I don't remember. What is so hard about parsing the lines like the following in the post log for the information? Jan 11 14:31:59 2006 (8997) post to from user at example.com, size=3320, message-id=, success Run a script before the log is rotated away which maintains a database keyed by posting address and containing a last_post_date field. Now and then query the database for posters who have posted more recently than xxx days ago, gets a list of all members, and reports the set difference of all-members - recent-posters. Action taken could be any of the ones discussed. This really doesn't seem like it should be part of Mailman, although it might have standing as a contributed extra goodie. I would expect it to have a low download count. --John From erling.hellenas at runbox.com Fri Jan 13 04:13:41 2006 From: erling.hellenas at runbox.com (=?us-ascii?Q?Erling_Hellenas?=) Date: Fri, 13 Jan 2006 04:13:41 +0100 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? In-Reply-To: Message-ID: <000801c617ef$5b6f9620$fc01a8c0@home.local> HI all ! This is just an idea for an addition I think would be useful. I think that normally if you create some piece of information with a computer you should also see to that it is taken away when it is no longer needed. As I see it everything will get much easier if you have only the active users in your databases, instead of having many times that amount of inactive people who don't even read the mails. Cheers, Erling Hellenas From brad at stop.mail-abuse.org Fri Jan 13 11:36:50 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Fri, 13 Jan 2006 11:36:50 +0100 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? In-Reply-To: <000801c617ef$5b6f9620$fc01a8c0@home.local> References: <000801c617ef$5b6f9620$fc01a8c0@home.local> Message-ID: At 4:13 AM +0100 2006-01-13, Erling Hellenas wrote: > This is just an idea for an addition I think would be useful. I think that > normally if you create some piece of information with a computer you should > also see to that it is taken away when it is no longer needed. In some cases, that may be appropriate. However, it is totally inappropriate to assume that this is correct in all cases. > As I see it everything will get much easier if you have only the active > users in your databases, instead of having many times that amount of > inactive people who don't even read the mails. For most mailing lists, 99.9999% of the subscribership is silent -- they read, but they never post. In fact, many mailing lists are announcement-only -- subscribers are not allowed to post, even if they want to. On lists where subscribers are allowed to post, if you then force them to periodically respond to an automated query, you will find that many of them decide it's not worth the hassle and they will go somewhere else. You obviously feel that this feature is desirable for your purposes, and therefore you want the Mailman developers to add this functionality. The problem is that Mailman is an open source project and the developers do not have the time to implement even a small fraction of the features they consider to be useful, much less the features that might benefit only a tiny fraction of the community. Mailman is an open-source project. If you want to add code to the system to provide such a feature, or arrange for someone else to do that for you, then you would be welcome to have that code uploaded to the appropriate page on the SourceForge web site and the Mailman developers would consider whether or not to use that code (or create their own code) to add that feature at some point in time in the future. Otherwise, I wouldn't hold your breath. I certainly doubt that anyone is going to give this idea any strong consideration so long as we're still talking about Mailman 2.x and Python pickles for list data and meta-data storage -- the additional data storage and operational performance costs would be too great for far too many lists. Maybe once we get into Mailman 3 (a.k.a., "MM3") and having a real back-end database, something like this might be somewhat more feasible -- although I doubt it would be any more attractive or broadly useful to the whole community. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From barry at python.org Fri Jan 13 14:41:56 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 13 Jan 2006 08:41:56 -0500 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? In-Reply-To: References: Message-ID: <948AB4C8-34CA-44E5-8948-E4FC779F6178@python.org> On Jan 12, 2006, at 8:51 PM, Mark Sapiro wrote: > Joshua Ginsberg wrote: >> >> Perhaps an interesting compromise might be to add to config.pck a key >> "last_post" whose value is a dictionary of email:time 9-tuple pairs. >> That way, folks like Erling could write a script to go ahead and >> do this >> fu, and it might be an interesting metric for other purposes as well. > > This could prove to be a real burden to sites that have LARGE lists. > Adding yet another dictionary with 100,000 keys to the list object > could have significant impact. Definitely. But it would be easier in MM3 (although we wouldn't store 9-tuples, we'd probably store ISO formatted datetimes). -Barry From erling.hellenas at runbox.com Fri Jan 13 15:10:49 2006 From: erling.hellenas at runbox.com (=?us-ascii?Q?Erling_Hellenas?=) Date: Fri, 13 Jan 2006 15:10:49 +0100 Subject: [Mailman-Developers] Inactivity deletion of maillist users ? In-Reply-To: Message-ID: <001101c6184b$27ed6140$fc01a8c0@home.local> Hi all ! Comment to Brad. Thanks for your comments. > This is just an idea for an addition I think would be useful. I think that > normally if you create some piece of information with a computer you should > also see to that it is taken away when it is no longer needed. Brad Knowles: In some cases, that may be appropriate. However, it is totally inappropriate to assume that this is correct in all cases. I think it seems perfectly clear from my statement that I am not assuming that this is correct in all cases. "For most mailing lists, 99.9999% of the subscribership is silent -- they read, but they never post. In fact, many mailing lists are announcement-only -- subscribers are not allowed to post, even if they want to." My assumption is that of the 99.9999 % a considerable part don't even read the mails. " On lists where subscribers are allowed to post, if you then force them to periodically respond to an automated query, you will find that many of them decide it's not worth the hassle and they will go somewhere else." If two clicks every year or half a year is too much trouble to stay in the list they might not be very important to the list owners either. " You obviously feel that this feature is desirable for your purposes, and therefore you want the Mailman developers to add this functionality. The problem is that Mailman is an open source project and the developers do not have the time to implement even a small fraction of the features they consider to be useful, much less the features that might benefit only a tiny fraction of the community." No, this is not so important to me personally, but might help me a tiny bit. For the moment I am not considering doing the implementation, but I might well do other Mailman additions. Cheers, Erling Hellenas From adrian at whatifnet.com Fri Jan 13 14:54:37 2006 From: adrian at whatifnet.com (Adrian Wells) Date: Fri, 13 Jan 2006 08:54:37 -0500 Subject: [Mailman-Developers] Mysql MemberAdaptor and Mailman 2.1.6 Message-ID: Hello all. I am working with MySQL 3.23.58 (Ver 11.18) on a Fedora Core 3 system, Mailman 2.1.6, and the Mysql MemberAdaptor. Today, this system experienced a problem while using the list_lists command line tool which seems to be related to the Mysql MemberAdaptor. The following error was provided: Fatal error connecting to MySQL database mailman (1040): Too many connections Exception exceptions.AttributeError: "MysqlMemberships instance has no attribute 'cursor'" in > ignored To resolve this problem, I restarted the MySQL server (which resides on the same host as the Mailman and Postfix server). Below is a MySQL 'show processlist' of the processes running (?) when this error was displayed. Could this be an issue with the Mysql MemberAdaptor not releasing connections? (I realize this list does not solely exist to provide support for patches, but I also know there are individuals that have worked with this patch who have frequented the list in the past.) Thank you, -Adrian mysql> SHOW PROCESSLIST; +-------+-----------+-----------+---------+---------+---------+-------+------------------+ | Id | User | Host | db | Command | Time | State | Info | +-------+-----------+-----------+---------+---------+---------+-------+------------------+ | 14504 | mailmandb | localhost | mailman | Sleep | 1513375 | | NULL | | 14508 | mailmandb | localhost | mailman | Sleep | 1920366 | | NULL | | 14545 | mailmandb | localhost | mailman | Sleep | 196702 | | NULL | | 14546 | mailmandb | localhost | mailman | Sleep | 196704 | | NULL | | 14547 | mailmandb | localhost | mailman | Sleep | 196700 | | NULL | | 14551 | mailmandb | localhost | mailman | Sleep | 1430519 | | NULL | | 14552 | mailmandb | localhost | mailman | Sleep | 1430521 | | NULL | | 14553 | mailmandb | localhost | mailman | Sleep | 1430518 | | NULL | | 14554 | mailmandb | localhost | mailman | Sleep | 38720 | | NULL | | 14555 | mailmandb | localhost | mailman | Sleep | 38720 | | NULL | | 14556 | mailmandb | localhost | mailman | Sleep | 38719 | | NULL | | 14557 | mailmandb | localhost | mailman | Sleep | 38705 | | NULL | | 14558 | mailmandb | localhost | mailman | Sleep | 38704 | | NULL | | 14559 | mailmandb | localhost | mailman | Sleep | 38703 | | NULL | | 14759 | mailmandb | localhost | mailman | Sleep | 1741488 | | NULL | | 14858 | mailmandb | localhost | mailman | Sleep | 18618 | | NULL | | 14907 | mailmandb | localhost | mailman | Sleep | 1809176 | | NULL | | 15012 | mailmandb | localhost | mailman | Sleep | 1789990 | | NULL | | 15013 | mailmandb | localhost | mailman | Sleep | 63692 | | NULL | | 15033 | mailmandb | localhost | mailman | Sleep | 64803 | | NULL | | 15034 | mailmandb | localhost | mailman | Sleep | 64802 | | NULL | | 15042 | mailmandb | localhost | mailman | Sleep | 1779391 | | NULL | | 15095 | mailmandb | localhost | mailman | Sleep | 1787571 | | NULL | | 15096 | mailmandb | localhost | mailman | Sleep | 58763 | | NULL | | 15116 | mailmandb | localhost | mailman | Sleep | 62932 | | NULL | | 15117 | mailmandb | localhost | mailman | Sleep | 62931 | | NULL | | 15128 | mailmandb | localhost | mailman | Sleep | 1787008 | | NULL | | 15130 | mailmandb | localhost | mailman | Sleep | 64207 | | NULL | | 15177 | mailmandb | localhost | mailman | Sleep | 64724 | | NULL | | 15178 | mailmandb | localhost | mailman | Sleep | 64724 | | NULL | | 15198 | mailmandb | localhost | mailman | Sleep | 1785738 | | NULL | | 15199 | mailmandb | localhost | mailman | Sleep | 63444 | | NULL | | 15259 | mailmandb | localhost | mailman | Sleep | 1292509 | | NULL | | 15260 | mailmandb | localhost | mailman | Sleep | 81444 | | NULL | | 15261 | mailmandb | localhost | mailman | Sleep | 81443 | | NULL | | 15262 | mailmandb | localhost | mailman | Sleep | 1782744 | | NULL | | 15263 | mailmandb | localhost | mailman | Sleep | 1782743 | | NULL | | 15264 | mailmandb | localhost | mailman | Sleep | 1782742 | | NULL | | 15265 | mailmandb | localhost | mailman | Sleep | 64804 | | NULL | | 15516 | mailmandb | localhost | mailman | Sleep | 898199 | | NULL | | 15658 | mailmandb | localhost | mailman | Sleep | 1080278 | | NULL | | 15659 | mailmandb | localhost | mailman | Sleep | 173981 | | NULL | | 15660 | mailmandb | localhost | mailman | Sleep | 304728 | | NULL | | 15661 | mailmandb | localhost | mailman | Sleep | 173980 | | NULL | | 15662 | mailmandb | localhost | mailman | Sleep | 1513762 | | NULL | | 15663 | mailmandb | localhost | mailman | Sleep | 1513755 | | NULL | | 15664 | mailmandb | localhost | mailman | Sleep | 304427 | | NULL | | 15665 | mailmandb | localhost | mailman | Sleep | 1513757 | | NULL | | 15666 | mailmandb | localhost | mailman | Sleep | 1513753 | | NULL | | 15667 | mailmandb | localhost | mailman | Sleep | 281688 | | NULL | | 15668 | mailmandb | localhost | mailman | Sleep | 1466194 | | NULL | | 15669 | mailmandb | localhost | mailman | Sleep | 1081178 | | NULL | | 15670 | mailmandb | localhost | mailman | Sleep | 304428 | | NULL | | 15671 | mailmandb | localhost | mailman | Sleep | 173983 | | NULL | | 15709 | mailmandb | localhost | mailman | Sleep | 47519 | | NULL | | 15710 | mailmandb | localhost | mailman | Sleep | 47519 | | NULL | | 15711 | mailmandb | localhost | mailman | Sleep | 47518 | | NULL | | 19847 | mailmandb | localhost | mailman | Sleep | 51998 | | NULL | | 19848 | mailmandb | localhost | mailman | Sleep | 51997 | | NULL | | 19849 | mailmandb | localhost | mailman | Sleep | 51996 | | NULL | | 20164 | mailmandb | localhost | mailman | Sleep | 684161 | | NULL | | 21367 | mailmandb | localhost | mailman | Sleep | 52922 | | NULL | | 21368 | mailmandb | localhost | mailman | Sleep | 52921 | | NULL | | 22439 | mailmandb | localhost | mailman | Sleep | 64749 | | NULL | | 22440 | mailmandb | localhost | mailman | Sleep | 65439 | | NULL | | 22441 | mailmandb | localhost | mailman | Sleep | 64725 | | NULL | | 22453 | mailmandb | localhost | mailman | Sleep | 64208 | | NULL | | 22454 | mailmandb | localhost | mailman | Sleep | 62933 | | NULL | | 22455 | mailmandb | localhost | mailman | Sleep | 58764 | | NULL | | 22456 | mailmandb | localhost | mailman | Sleep | 63693 | | NULL | | 22457 | mailmandb | localhost | mailman | Sleep | 63445 | | NULL | | 22463 | mailmandb | localhost | mailman | Sleep | 62560 | | NULL | | 22464 | mailmandb | localhost | mailman | Sleep | 62562 | | NULL | | 22465 | mailmandb | localhost | mailman | Sleep | 62559 | | NULL | | 22576 | root | localhost | NULL | Query | 0 | NULL | SHOW PROCESSLIST | +-------+-----------+-----------+---------+---------+---------+-------+------------------+ 75 rows in set (0.00 sec) From slave at codegrunt.com Fri Jan 13 17:53:12 2006 From: slave at codegrunt.com (Ron Brogden) Date: Fri, 13 Jan 2006 08:53:12 -0800 Subject: [Mailman-Developers] Mysql MemberAdaptor and Mailman 2.1.6 In-Reply-To: References: Message-ID: <200601130853.12245.slave@codegrunt.com> On Friday 13 January 2006 05:54, Adrian Wells wrote: > Fatal error connecting to MySQL database mailman (1040): Too many > connections > Exception exceptions.AttributeError: "MysqlMemberships instance has no > attribute 'cursor'" in > ignored > > To resolve this problem, I restarted the MySQL server (which resides on > the same host as the Mailman and Postfix server). Below is a MySQL 'show > processlist' of the processes running (?) when this error was displayed. This is a common issue and affects any script that uses persistent connections to MySQL (PHP scripts for example). What you can do is on the MySQL daemon side adjust the timeout for idle connections so you do not have stale connections sitting around indefinitely tying up resources. It's been a while since I have had to tweak these but I believe that the main one in this case is "wait_timeout" which is how long the server will wait for data from the client. The default is 8 hours I think which is ridiculous. Try something like 120 seconds to start then move lower if you need to (assuming that Mailman will not take more than n seconds to send data after opening a connection). You can also increase the concurrent number of allowed connections if you want assuming your system can handle that but from the looks of it, the wait timeout is probably your main foe here. http://dev.mysql.com/doc/refman/5.0/en/show-variables.html You will need to either set this on the command line when you start up MySQL or place it in the my.cnf file if you want it to stick around after a reboot. Cheers -- CODEgrunt slave at codegrunt.com / http://codegrunt.com From fil at rezo.net Fri Jan 13 18:29:17 2006 From: fil at rezo.net (Fil) Date: Fri, 13 Jan 2006 18:29:17 +0100 Subject: [Mailman-Developers] Mysql MemberAdaptor and Mailman 2.1.6 In-Reply-To: <200601130853.12245.slave@codegrunt.com> References: <200601130853.12245.slave@codegrunt.com> Message-ID: <20060113172917.GX19696@rezo.net> > > Fatal error connecting to MySQL database mailman (1040): Too many > > connections > > Exception exceptions.AttributeError: "MysqlMemberships instance has no > > attribute 'cursor'" in > > ignored > > > > To resolve this problem, I restarted the MySQL server (which resides on > > the same host as the Mailman and Postfix server). Below is a MySQL 'show > > processlist' of the processes running (?) when this error was displayed. I think I have solved this issue in my version of the MySQLMemberAdaptor; it is available at http://trac.rezo.net/trac/rezo/browser/Mailman/ -- Fil From tkikuchi at is.kochi-u.ac.jp Sun Jan 15 03:59:37 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Sun, 15 Jan 2006 11:59:37 +0900 Subject: [Mailman-Developers] [Mailman-Users] Problems with uuencoded attachments In-Reply-To: References: Message-ID: <43C9BA99.90308@is.kochi-u.ac.jp> Mark Sapiro wrote: >> File "/usr/lib/python2.3/uu.py", line 139, in decode >> sys.stderr.write("Warning: %s\n" % str(v)) >> File "/usr/lib/mailman/Mailman/Logging/MultiLogger.py", line 45, in write >> _logexc(logger, msg) >> File "/usr/lib/mailman/Mailman/Logging/Utils.py", line 22, in _logexc >> sys.__stderr__.write('Logging error: %s\n' % logger) >>IOError: [Errno 32] Broken pipe > > I think this could be fixed by changing > "/usr/lib/mailman/pythonlib/email/Message.py", line 223 from > > uu.decode(StringIO(payload+'\n'), sfp) > > to > > uu.decode(StringIO(payload+'\n'), sfp, quiet=True) > > There should be other chances that Python builtin modules spew warnings to sys.stderr. How about this patch for Logging/Utils.py to write these messages into syslog facility. -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Utils.py.patch Url: http://mail.python.org/pipermail/mailman-developers/attachments/20060115/fedc6190/attachment.asc From camberwell at camberwellcarrot.net Sun Jan 15 04:01:12 2006 From: camberwell at camberwellcarrot.net (Camberwell) Date: Sun, 15 Jan 2006 03:01:12 +0000 Subject: [Mailman-Developers] Dealing with DomainKeys and DKIM Message-ID: <43C9BAF8.5020902@camberwellcarrot.net> Joe Peterson Sat, 10 Sep 2005 12:03:55 -0700 I've recently been testing DomainKeys (http://antispam.yahoo.com/domainkeys) and DKIM (which is supposedly a merging of DomainKeys with Cisco's scheme. I am using dk-milter and dkim-milter with sendmail. What this does is add two header lines to outgoing email that allow the receiver to determine the authenticity of the sender... Anyway, since I run a Mailman system too, I figured this might be a problem. Indeed it is, since the header lines get passed through, and when the check is done, it indicates a failure. DomainKeys recommends mail lists regenerate the keys rather than pass them through. What I tried was pretty simple: Mailman doesn't have to deal with these things itself, but if it strips the old keys from the header, the keys will be regenerated on the way out by the MTA, thereby making the whole process clean. So the receiver of the email can at least verify that the mail came from the host hosting Mailman. I suppose Mailman could also check email on the way in for valid keys if it wanted, but that's another subject... I patched Handlers/Cleanse.py as follows: 49a50,55 > # JGP: Remove all "DomainKeys" type header lines, since we want these > # to be regenerated by the MTA when this message is sent out. We need > # to let new such keys be generated because Mailman alters the message, > # and the old keys would be seen as invalid by the receiver. > del msg['domainkey-signature'] > del msg['dkim-signature'] I wanted to pass this by the developers here and see if: 1) This is a reasonable thing to do (or maybe have an option, or even a way to strip selected headers in the config?) 2) If this is the right place to do it. -Thanks, Joe -------------------------------------------------------------- Good day all, I have recently started using dkim-milter myself, and i have made these adjustments to my Cleanse.py to get around this very problem and it works great, alltho i have another little problem..... When i send a mail to list-owner at mysite, if there is a dkim-signature allready in the header (in my case my mail is signed) my dkim-milter trys to verify it instead of signing it on the way back out to the list owner. i hope that makes sense. basicaly i would like to know what i can edit to remove the dkim-signature from ALL incoming mail, not just mail to be bounced to the list. thanks in advance Martin Airs (Camberwell) From brad at stop.mail-abuse.org Sun Jan 15 04:56:12 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Sun, 15 Jan 2006 04:56:12 +0100 Subject: [Mailman-Developers] Dealing with DomainKeys and DKIM In-Reply-To: <43C9BAF8.5020902@camberwellcarrot.net> References: <43C9BAF8.5020902@camberwellcarrot.net> Message-ID: At 3:01 AM +0000 2006-01-15, Camberwell wrote: > Anyway, since I run a Mailman system too, I figured this might be a > problem. Indeed it is, since the header lines get passed through, and > when the check is done, it indicates a failure. DomainKeys recommends > mail lists regenerate the keys rather than pass them through. Did you check Mailman version 2.1.7? It was released recently and I believe that handling DomainKeys/DKIM was one of the things that was addressed. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From msapiro at value.net Sun Jan 15 16:40:29 2006 From: msapiro at value.net (Mark Sapiro) Date: Sun, 15 Jan 2006 07:40:29 -0800 Subject: [Mailman-Developers] [Mailman-Users] Problems with uuencoded attachments In-Reply-To: <43C9BA99.90308@is.kochi-u.ac.jp> Message-ID: Tokio Kikuchi wrote: > >There should be other chances that Python builtin modules spew warnings >to sys.stderr. How about this patch for Logging/Utils.py to write these >messages into syslog facility. The patch looks good to me, and I think is a good thing in general. It's clearly better than supressing warnings one at a time. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at python.org Sun Jan 15 16:49:47 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 15 Jan 2006 10:49:47 -0500 Subject: [Mailman-Developers] [Mailman-Users] Problems with uuencoded attachments In-Reply-To: <43C9BA99.90308@is.kochi-u.ac.jp> References: <43C9BA99.90308@is.kochi-u.ac.jp> Message-ID: <91129E11-A54A-4236-BA17-8C2CD0B96782@python.org> On Jan 14, 2006, at 9:59 PM, Tokio Kikuchi wrote: > Mark Sapiro wrote: > > >>> File "/usr/lib/python2.3/uu.py", line 139, in decode >>> sys.stderr.write("Warning: %s\n" % str(v)) >>> File "/usr/lib/mailman/Mailman/Logging/MultiLogger.py", line 45, >>> in write >>> _logexc(logger, msg) >>> File "/usr/lib/mailman/Mailman/Logging/Utils.py", line 22, in >>> _logexc >>> sys.__stderr__.write('Logging error: %s\n' % logger) >>> IOError: [Errno 32] Broken pipe > >> I think this could be fixed by changing >> "/usr/lib/mailman/pythonlib/email/Message.py", line 223 from >> uu.decode(StringIO(payload+'\n'), sfp) >> to >> uu.decode(StringIO(payload+'\n'), sfp, >> quiet=True) > > > There should be other chances that Python builtin modules spew > warnings to sys.stderr. How about this patch for Logging/Utils.py > to write these messages into syslog facility. The only problem is that currently Mailman does not use the syslog module, and I'm uncomfortable with adding it for this one situation (are there others?). We should definitely be passing the quiet flag to uu.decode(), but OTOH maybe we should also be testing whether sys.__stderr__ is detached and then redirecting that to logs/errors? -Barry From msapiro at value.net Sun Jan 15 17:10:26 2006 From: msapiro at value.net (Mark Sapiro) Date: Sun, 15 Jan 2006 08:10:26 -0800 Subject: [Mailman-Developers] Dealing with DomainKeys and DKIM In-Reply-To: Message-ID: Brad Knowles wrote: > > Did you check Mailman version 2.1.7? It was released recently >and I believe that handling DomainKeys/DKIM was one of the things >that was addressed. Brad is correct, but it is addressed exactly as described in the OP, namely by modifying Cleanse.py to delete 'domainkey-signature' and 'dkim-signature' headers from the message. The problem with this is that 'Cleanse' is not in OWNER_PIPELINE so the headers don't get deleted from messages to listname-owner. There are a couple of possibilities to address this. The first is easy, but wrong. Add OWNER_PIPELINE.insert(1,'Cleanse') to mm_cfg.py to add 'Cleanse' after 'SpamDetect' in OWNER_PIPELINE. The reason this is wrong is that if a list is anonymous, the owner will have to refer to the 'post' log to find out who the message was from as Cleanse will replace From: and Reply-To: with the list address and remove Sender: and X-Originating-Email:. A better idea is to remove the 'domainkey-signature' and 'dkim-signature' headers in ToOutgoing.py which should get them out of all messages. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at python.org Sun Jan 15 17:52:46 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 15 Jan 2006 11:52:46 -0500 Subject: [Mailman-Developers] Dealing with DomainKeys and DKIM In-Reply-To: References: Message-ID: <462BE12F-9A15-48A3-A57F-4A0EAF92BBD6@python.org> On Jan 15, 2006, at 11:10 AM, Mark Sapiro wrote: > Brad Knowles wrote: >> >> Did you check Mailman version 2.1.7? It was released recently >> and I believe that handling DomainKeys/DKIM was one of the things >> that was addressed. > > Brad is correct, but it is addressed exactly as described in the OP, > namely by modifying Cleanse.py to delete 'domainkey-signature' and > 'dkim-signature' headers from the message. > > The problem with this is that 'Cleanse' is not in OWNER_PIPELINE so > the > headers don't get deleted from messages to listname-owner. > > There are a couple of possibilities to address this. The first is > easy, > but wrong. Add > > OWNER_PIPELINE.insert(1,'Cleanse') > > to mm_cfg.py to add 'Cleanse' after 'SpamDetect' in OWNER_PIPELINE. > The > reason this is wrong is that if a list is anonymous, the owner will > have to refer to the 'post' log to find out who the message was from > as Cleanse will replace From: and Reply-To: with the list address and > remove Sender: and X-Originating-Email:. > > A better idea is to remove the 'domainkey-signature' and > 'dkim-signature' headers in ToOutgoing.py which should get them out of > all messages. Or to add a simpler handler (maybe something called CleanseDKIM.py?) and add that to OWNER_PIPELINE, possibly refactoring out those from Cleanse.py so we don't duplicate code. -Barry From msapiro at value.net Sun Jan 15 18:41:29 2006 From: msapiro at value.net (Mark Sapiro) Date: Sun, 15 Jan 2006 09:41:29 -0800 Subject: [Mailman-Developers] Dealing with DomainKeys and DKIM In-Reply-To: Message-ID: Camberwell wrote: > >i have added that line to my mm_cfg.py and it works perfectly thankyou >very much. sorry i'm not brilliant at python but how might i go about >removing the signatures in ToOutgoing.py. your first solution works fine >for me as my list will never be anonymous so thanks again You could add the following (watch out for wrapped lines) # Remove any "DomainKeys" (or similar) header lines. The values contained # in these header lines are intended to be used by the recipient to detect # forgery or tampering in transit, and the modifications made by Mailman # to the headers and body of the message will cause these keys to appear # invalid. Removing them will at least avoid this misleading result, and # it will also give the MTA the opportunity to regenerate valid keys # originating at the Mailman server for the outgoing message. del msg['domainkey-signature'] del msg['dkim-signature'] immediately following the line def process(mlist, msg, msgdata): in Mailman/Handlers/ToOutgoing.py. Note to developers: It seems we should move this from Cleanse.py to ToOutgoing.py for exactly the reasons expressed earlier in this thread, but I'm a little uneasy about mucking with headers in ToOutgoing as that isn't its purpose. Any comments? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From msapiro at value.net Sun Jan 15 20:28:28 2006 From: msapiro at value.net (Mark Sapiro) Date: Sun, 15 Jan 2006 11:28:28 -0800 Subject: [Mailman-Developers] Dealing with DomainKeys and DKIM In-Reply-To: Message-ID: Mark Sapiro wrote: > >Note to developers: > >It seems we should move this from Cleanse.py to ToOutgoing.py for >exactly the reasons expressed earlier in this thread, but I'm a little >uneasy about mucking with headers in ToOutgoing as that isn't its >purpose. Any comments? And Barry Warsaw commented previously: >Or to add a simpler handler (maybe something called CleanseDKIM.py?) >and add that to OWNER_PIPELINE, possibly refactoring out those from >Cleanse.py so we don't duplicate code. Barry's comment seems to me to be the way to go. I'll work up a patch. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From camberwell at camberwellcarrot.net Sun Jan 15 06:29:27 2006 From: camberwell at camberwellcarrot.net (Camberwell) Date: Sun, 15 Jan 2006 05:29:27 +0000 Subject: [Mailman-Developers] Dealing with DomainKeys and DKIM Message-ID: <43C9DDB7.5040300@camberwellcarrot.net> Joe Peterson Sat, 10 Sep 2005 12:03:55 -0700 I've recently been testing DomainKeys (http://antispam.yahoo.com/domainkeys) and DKIM (which is supposedly a merging of DomainKeys with Cisco's scheme. I am using dk-milter and dkim-milter with sendmail. What this does is add two header lines to outgoing email that allow the receiver to determine the authenticity of the sender... Anyway, since I run a Mailman system too, I figured this might be a problem. Indeed it is, since the header lines get passed through, and when the check is done, it indicates a failure. DomainKeys recommends mail lists regenerate the keys rather than pass them through. What I tried was pretty simple: Mailman doesn't have to deal with these things itself, but if it strips the old keys from the header, the keys will be regenerated on the way out by the MTA, thereby making the whole process clean. So the receiver of the email can at least verify that the mail came from the host hosting Mailman. I suppose Mailman could also check email on the way in for valid keys if it wanted, but that's another subject... I patched Handlers/Cleanse.py as follows: 49a50,55 > # JGP: Remove all "DomainKeys" type header lines, since we want these > # to be regenerated by the MTA when this message is sent out. We need > # to let new such keys be generated because Mailman alters the message, > # and the old keys would be seen as invalid by the receiver. > del msg['domainkey-signature'] > del msg['dkim-signature'] I wanted to pass this by the developers here and see if: 1) This is a reasonable thing to do (or maybe have an option, or even a way to strip selected headers in the config?) 2) If this is the right place to do it. -Thanks, Joe -------------------------------------------------------------- Good day all, I have recently started using dkim-milter myself, and i have made these adjustments to my Cleanse.py to get around this very problem and it works great, alltho i have another little problem..... When i send a mail to list-owner at mysite, if there is a dkim-signature allready in the header (in my case my mail is signed) my dkim-milter trys to verify it instead of signing it on the way back out to the list owner. i hope that makes sense. basicaly i would like to know what i can edit to remove the dkim-signature from ALL incoming mail, not just mail to be bounced to the list. i have tried the latest version 2.1.7 and it does indeed remove the domainkey signature on mail that is for the list, and my MTA does indeed re insert its own domainkey signature, BUT the domainkey signature is NOT removed for mail to list-owner. how can i remove these headers? thanks in advance Martin Airs (Camberwell) -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3553 bytes Desc: S/MIME Cryptographic Signature Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060115/2dc794ae/attachment.bin From tkikuchi at is.kochi-u.ac.jp Mon Jan 16 01:25:08 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Mon, 16 Jan 2006 09:25:08 +0900 Subject: [Mailman-Developers] [Mailman-Users] Problems with uuencoded attachments In-Reply-To: <91129E11-A54A-4236-BA17-8C2CD0B96782@python.org> References: <43C9BA99.90308@is.kochi-u.ac.jp> <91129E11-A54A-4236-BA17-8C2CD0B96782@python.org> Message-ID: <43CAE7E4.2060801@is.kochi-u.ac.jp> Barry Warsaw wrote: > On Jan 14, 2006, at 9:59 PM, Tokio Kikuchi wrote: > > >>Mark Sapiro wrote: >> >> >> >>>>File "/usr/lib/python2.3/uu.py", line 139, in decode >>>> sys.stderr.write("Warning: %s\n" % str(v)) >>>>File "/usr/lib/mailman/Mailman/Logging/MultiLogger.py", line 45, >>>>in write >>>> _logexc(logger, msg) >>>>File "/usr/lib/mailman/Mailman/Logging/Utils.py", line 22, in >>>>_logexc >>>> sys.__stderr__.write('Logging error: %s\n' % logger) >>>>IOError: [Errno 32] Broken pipe >> >>>I think this could be fixed by changing >>>"/usr/lib/mailman/pythonlib/email/Message.py", line 223 from >>> uu.decode(StringIO(payload+'\n'), sfp) >>>to >>> uu.decode(StringIO(payload+'\n'), sfp, >>>quiet=True) >> >> >>There should be other chances that Python builtin modules spew >>warnings to sys.stderr. How about this patch for Logging/Utils.py >>to write these messages into syslog facility. > > > The only problem is that currently Mailman does not use the syslog > module, and I'm uncomfortable with adding it for this one situation > (are there others?). We should definitely be passing the quiet flag > to uu.decode(), but OTOH maybe we should also be testing whether > sys.__stderr__ is detached and then redirecting that to logs/errors? > In usual mailman qrunner execs, stderr is logged into logs/errors. It is the additional tee_to_real_stderr in LogStdErr() setting which wants to print the error into real stderr. Isn't it safe to put the tee_to_real_stderr value 0 in bin/qrunner script ? -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From barry at python.org Mon Jan 16 02:31:49 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 15 Jan 2006 20:31:49 -0500 Subject: [Mailman-Developers] [Mailman-Users] Problems with uuencoded attachments In-Reply-To: <43CAE7E4.2060801@is.kochi-u.ac.jp> References: <43C9BA99.90308@is.kochi-u.ac.jp> <91129E11-A54A-4236-BA17-8C2CD0B96782@python.org> <43CAE7E4.2060801@is.kochi-u.ac.jp> Message-ID: <1137375109.5898.34.camel@geddy.wooz.org> On Mon, 2006-01-16 at 09:25 +0900, Tokio Kikuchi wrote: > In usual mailman qrunner execs, stderr is logged into logs/errors. It > is the additional tee_to_real_stderr in LogStdErr() setting which wants > to print the error into real stderr. > > Isn't it safe to put the tee_to_real_stderr value 0 in bin/qrunner script ? Ideally, tee_to_real_stderr would be !AS_SUBPROC (i.e. tee when not running qrunner under mailmanctl). That would have to be done in main() after processing the command line arguments. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060115/b1be120f/attachment.pgp From tkikuchi at is.kochi-u.ac.jp Mon Jan 16 03:03:47 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Mon, 16 Jan 2006 11:03:47 +0900 Subject: [Mailman-Developers] [Mailman-Users] Problems with uuencoded attachments In-Reply-To: <1137375109.5898.34.camel@geddy.wooz.org> References: <43C9BA99.90308@is.kochi-u.ac.jp> <91129E11-A54A-4236-BA17-8C2CD0B96782@python.org> <43CAE7E4.2060801@is.kochi-u.ac.jp> <1137375109.5898.34.camel@geddy.wooz.org> Message-ID: <43CAFF03.9010401@is.kochi-u.ac.jp> Barry Warsaw wrote: > On Mon, 2006-01-16 at 09:25 +0900, Tokio Kikuchi wrote: > > >>In usual mailman qrunner execs, stderr is logged into logs/errors. It >>is the additional tee_to_real_stderr in LogStdErr() setting which wants >>to print the error into real stderr. >> >>Isn't it safe to put the tee_to_real_stderr value 0 in bin/qrunner script ? > > > Ideally, tee_to_real_stderr would be !AS_SUBPROC (i.e. tee when not > running qrunner under mailmanctl). That would have to be done in main() > after processing the command line arguments. > Yeah, I noticed that too and played around. How about this patch. I assumed running qrunner independently is for debugging purpose and quit redirecting stderr into logs/error. -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: qrunner.patch Url: http://mail.python.org/pipermail/mailman-developers/attachments/20060116/1a7ac806/attachment.asc From claw at kanga.nu Tue Jan 17 09:01:59 2006 From: claw at kanga.nu (J C Lawrence) Date: Tue, 17 Jan 2006 00:01:59 -0800 Subject: [Mailman-Developers] Feature Request: message_id as replaceable token for footers etc. Message-ID: <26006.1137484919@kanga.nu> Barry, Long time no talk. Whaddya think of adding message_id as a replaceable token for footers and the like? This would allow footers to be constructed which direct URL-reference their own post in an NNTP archive. -- J C Lawrence They said, "You have a blue guitar, ---------(*) You do not play things as they are." claw at kanga.nu The man replied, "Things as they are http://www.kanga.nu/~claw/ Are changed upon the blue guitar." From stephen at xemacs.org Tue Jan 17 10:52:11 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 17 Jan 2006 18:52:11 +0900 Subject: [Mailman-Developers] Feature Request: message_id as replaceable token for footers etc. In-Reply-To: <26006.1137484919@kanga.nu> (J. C. Lawrence's message of "Tue, 17 Jan 2006 00:01:59 -0800") References: <26006.1137484919@kanga.nu> Message-ID: <87zmlv9o6s.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "J" == J C Lawrence writes: J> Long time no talk. Whaddya think of adding message_id as a J> replaceable token for footers and the like? This would allow J> footers to be constructed which direct URL-reference their own J> post in an NNTP archive. Or in a mailing list archive designed to take advantage of the feature (AFAIK none exist yet, but so what?) +1 -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From brad at stop.mail-abuse.org Wed Jan 18 17:00:52 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Wed, 18 Jan 2006 17:00:52 +0100 Subject: [Mailman-Developers] Feature Request: message_id as replaceable token for footers etc. In-Reply-To: <26006.1137484919@kanga.nu> References: <26006.1137484919@kanga.nu> Message-ID: At 12:01 AM -0800 2006-01-17, J C Lawrence wrote: > Long time no talk. Whaddya think of adding message_id as a replaceable > token for footers and the like? This would allow footers to be > constructed which direct URL-reference their own post in an NNTP > archive. I like the concept, but the only problem is that message-ids are not guaranteed unique. I'd like to see a guaranteed unique id be generated on the system, and then have that used as the new message-id, displayable in the footers, used as the index reference in the mailing list archives, etc.... I've been agitating for this one for a while. ;) -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 LOPSA member since December 2005. See . From claw at kanga.nu Wed Jan 18 17:20:15 2006 From: claw at kanga.nu (J C Lawrence) Date: Wed, 18 Jan 2006 08:20:15 -0800 Subject: [Mailman-Developers] Feature Request: message_id as replaceable token for footers etc. In-Reply-To: Message from Brad Knowles of "Wed, 18 Jan 2006 17:00:52 +0100." References: <26006.1137484919@kanga.nu> Message-ID: <28229.1137601215@kanga.nu> On Wed, 18 Jan 2006 17:00:52 +0100 Brad Knowles wrote: > At 12:01 AM -0800 2006-01-17, J C Lawrence wrote: >> Long time no talk. Whaddya think of adding message_id as a >> replaceable token for footers and the like? This would allow footers >> to be constructed which direct URL-reference their own post in an >> NNTP archive. > I like the concept, but the only problem is that message-ids are not > guaranteed unique. I'd like to see a guaranteed unique id be > generated on the system, and then have that used as the new > message-id, displayable in the footers, used as the index reference in > the mailing list archives, etc.... While true (the non-guarantee), in working across a quarter million messages in the archives of my various lists I've hit less than a score of duplicate or missing Message-IDs. That's a small enough problem that I'm willing to just eat the conflicts. -- J C Lawrence They said, "You have a blue guitar, ---------(*) You do not play things as they are." claw at kanga.nu The man replied, "Things as they are http://www.kanga.nu/~claw/ Are changed upon the blue guitar." From carbonnb at gmail.com Fri Jan 27 04:50:17 2006 From: carbonnb at gmail.com (Bryan Carbonnell) Date: Thu, 26 Jan 2006 22:50:17 -0500 Subject: [Mailman-Developers] Sitewide headers/footers & XHTML Compliant Web UI In-Reply-To: References: Message-ID: I have just uploaded a patch to Sourforge that allows you to set sitewide headers and footers for the Mailman 2.1.7 web UI, [ 1415956 ] Sitewide headers/footers & XHTML Compliant Web UI This patch was borne out of a request I received to make the Mailman UI fit the look of the web site. This patch allows you to set a site wide header and footer. This allows you to pretty much make the MM UI look like any other site. While I was at it I also made the web UI XHTML compliant. Once you patch your source and install it, all you need to do is edit the html files in the templates/en directory. Most of the pages will get the header and footers from the site-header.html and site-footer.html files, but some of the HTML files already contain theor own header/footer so you will need ot edit some of these files as well. Since this also adds XHTML compliance, this superceeds patch #116035 You can dowload it (and a separate version that works with the ht://dig integration patches) from: https://sourceforge.net/tracker/index.php?func=detail&aid=1415956&group_id=103&atid=300103 You can see in use at http://databaseadvisors.com/mailman/listinfo -- Bryan Carbonnell - carbonnb at gmail.com Life's journey is not to arrive at the grave safely in a well preserved body, but rather to skid in sideways, totally worn out, shouting "What a great ride!" From msapiro at value.net Fri Jan 27 21:52:17 2006 From: msapiro at value.net (Mark Sapiro) Date: Fri, 27 Jan 2006 12:52:17 -0800 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change to Handlers/SpamDetect.py isincomplete In-Reply-To: Message-ID: Dan Astoorian posted to the bug tracker: >Specifically: it is (IMHO) probably incorrect to apply >the "Hold" action to messages which were not submitted >to the list address. For example, a message sent to >"listname-owner" and matched by a "Hold" >header_filter_rule should not be held for approval >(irrespective of whether the message was administrivia >or not). Good point. >(It is not clear to the end user whether approving such >a held message would cause the message to be delivered >to the original "listname-owner" address, or whether it >would be sent to the list itself.) Definitely the message received by the user in this case is misleading, but the held message if approved will go to the owners/moderators only. I think Dan knows this. >I suspect (but have not verified) that similar issues >may exist for other mail paths (e.g., -request, -admin, >-(un)subscribe). Unless you are seeing something I'm not, this is not the case. The issue is that SpamDetect.py is in both the GLOBAL_PIPELINE and the OWNER_PIPELINE, but messages to -request|join|leave|subscribe|unsubscribe|admin|bounces|confirm are not processed through SpamDetect.py and are never held for any reason. >Probably the most sensible thing to do with messages >that match a "Hold" rule would be to hold the message >only if the "tolist" key is set; otherwise, do one of >the following: >a) continue on to the next rule, as though the "Hold" >rule failed to match the message; >b) accept the message; or >c) discard the message. > >It's not clear to me which of these options makes the >most sense. Discarding the message is probably unwise, >as a "hold" rule in the first place suggests that the >administrator does not consider all mail that matches >the rule to be expendable. I think b) accept the message is the correct action since the 'hold rule' implies the owner/moderator wants to see the message and accepting the message sends it to the owners/moderators. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From msapiro at value.net Fri Jan 27 23:30:25 2006 From: msapiro at value.net (Mark Sapiro) Date: Fri, 27 Jan 2006 14:30:25 -0800 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change to Handlers/SpamDetect.py isincomplete In-Reply-To: <200601272129.k0RLTUw3014440@ecf.utoronto.ca> Message-ID: Dan Astoorian wrote: >On Fri, 27 Jan 2006 12:52:17 PST, Mark Sapiro writes: >> >> Definitely the message received by the user in this case is misleading, >> but the held message if approved will go to the owners/moderators >> only. I think Dan knows this. > >Actually, I didn't. I'm a co-moderator of a mailing list that I didn't >initially set up, so my experience with Mailman is fairly limited. I've >never previously encountered a held message that was to any mail path >other than the list itself, and had never had cause to find out whether >other types of mail ever got held up for approval. In my declaration >that "it is not clear to the end user...", I was the end user in >question :-) OK >(Perhaps the moderation queue web interface should indicate which queue >the held message is in? Currently the moderator has to infer this >information from the message headers.) Actually, as you implicitly point out, this isn't necessary since only list posts should be in the moderation queue in the first place. The only other messages that currently get held are messages to the owner that get caught by a Spam filter 'hold' rule, and these shouldn't be held. At least that's how it seems pending other input. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at python.org Sat Jan 28 00:22:59 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 27 Jan 2006 18:22:59 -0500 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change to Handlers/SpamDetect.py isincomplete In-Reply-To: References: Message-ID: <1138404179.10387.3.camel@geddy.wooz.org> On Fri, 2006-01-27 at 14:30 -0800, Mark Sapiro wrote: > Actually, as you implicitly point out, this isn't necessary since only > list posts should be in the moderation queue in the first place. The > only other messages that currently get held are messages to the owner > that get caught by a Spam filter 'hold' rule, and these shouldn't be > held. At least that's how it seems pending other input. That seems right to me. It doesn't make any sense to hold a message ultimately destined for the list owner or moderator because those are the people who have to approve the message (what? so they can then read it again in their inbox?). If in response to a certain spam designation the owner destined message is discarded, that's fine. But it shouldn't be held, and rejecting spam is almost always a bad idea. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060127/76b7b7c3/attachment.pgp From tkikuchi at is.kochi-u.ac.jp Sat Jan 28 03:06:31 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Sat, 28 Jan 2006 11:06:31 +0900 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change to Handlers/SpamDetect.py isincomplete In-Reply-To: <1138404179.10387.3.camel@geddy.wooz.org> References: <1138404179.10387.3.camel@geddy.wooz.org> Message-ID: <43DAD1A7.8040407@is.kochi-u.ac.jp> Barry Warsaw wrote: > That seems right to me. It doesn't make any sense to hold a message > ultimately destined for the list owner or moderator because those are > the people who have to approve the message (what? so they can then read > it again in their inbox?). > > If in response to a certain spam designation the owner destined message > is discarded, that's fine. But it shouldn't be held, and rejecting spam > is almost always a bad idea. OK, Barry. I've come up with this patch (for the current CVS). If its OK, I want to start up for the release of 2.1.8a1. -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: SpamDetect.py.patch-20060128.txt Url: http://mail.python.org/pipermail/mailman-developers/attachments/20060128/f1496965/attachment.txt From msapiro at value.net Sat Jan 28 03:45:41 2006 From: msapiro at value.net (Mark Sapiro) Date: Fri, 27 Jan 2006 18:45:41 -0800 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change toHandlers/SpamDetect.py isincomplete In-Reply-To: <43DAD1A7.8040407@is.kochi-u.ac.jp> Message-ID: Tokio Kikuchi wrote: > >OK, Barry. I've come up with this patch (for the current CVS). >If its OK, I want to start up for the release of 2.1.8a1. The pass through of the 'hold' action if the message is to -owner seems right to me, but discarding instead of rejecting a 'reject' action if the message is to -owner seems wrong. I think we should not change the disposition for a 'reject' action. The rule can be for lots of purposes, not just spam and if the owner has configured the rule to reject the message, I don't think we should discard it just because it is to -owner and not to the list. I have one idea for 2.1.8a1 before we wrap it up. I'll address that in a separate post. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tkikuchi at is.kochi-u.ac.jp Sat Jan 28 04:57:55 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Sat, 28 Jan 2006 12:57:55 +0900 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change toHandlers/SpamDetect.py isincomplete In-Reply-To: References: Message-ID: <43DAEBC3.30607@is.kochi-u.ac.jp> Mark Sapiro wrote: > Tokio Kikuchi wrote: > >>OK, Barry. I've come up with this patch (for the current CVS). >>If its OK, I want to start up for the release of 2.1.8a1. > > > > The pass through of the 'hold' action if the message is to -owner seems > right to me, but discarding instead of rejecting a 'reject' action if > the message is to -owner seems wrong. Well, then it should be passed. Rejection makes it another loop of rejection notices if an initial poster forges the From: address as list-owner. Yes, I tested this. Nothing could stop this if the mailman qrunner wasn't stopped. (Maybe growing size of rejection notice would hit the MTA limit.) > > I think we should not change the disposition for a 'reject' action. The > rule can be for lots of purposes, not just spam and if the owner has > configured the rule to reject the message, I don't think we should > discard it just because it is to -owner and not to the list. > > I have one idea for 2.1.8a1 before we wrap it up. I'll address that in > a separate post. > OK. -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From msapiro at value.net Sat Jan 28 05:19:56 2006 From: msapiro at value.net (Mark Sapiro) Date: Fri, 27 Jan 2006 20:19:56 -0800 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change toHandlers/SpamDetect.pyisincomplete In-Reply-To: <43DAEBC3.30607@is.kochi-u.ac.jp> Message-ID: Tokio Kikuchi wrote: >Mark Sapiro wrote: >> >> The pass through of the 'hold' action if the message is to -owner seems >> right to me, but discarding instead of rejecting a 'reject' action if >> the message is to -owner seems wrong. > >Well, then it should be passed. Rejection makes it another loop of >rejection notices if an initial poster forges the From: address as >list-owner. Yes, I tested this. Nothing could stop this if the mailman >qrunner wasn't stopped. (Maybe growing size of rejection notice would >hit the MTA limit.) Ahh. I hadn't thought of that. Maybe discard it, but put something in the comment about the potential loop if we reject. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From msapiro at value.net Sat Jan 28 06:10:53 2006 From: msapiro at value.net (Mark Sapiro) Date: Fri, 27 Jan 2006 21:10:53 -0800 Subject: [Mailman-Developers] Scrubber mungs quoted-printable revisited. Message-ID: To refresh, see and . The problem with set_payload() creating a message for which a subsequent get_payload did 'too much' decoding was fixed by having Scrubber.py add a 'X-Mailman-Scrubbed: Yes' header upon doing a set_payload(), and then in various places where there are subsequent get_payload() calls, setting the decode flag according to the presence or absence of the header. I actually like the header as an explanation of content change along the lines of X-Content-Filtered-By:, but I've been uneasy about setting the get_payload() decode flag based on the presence or absence of 'X-Mailman-Scrubbed: Yes', although it seems to work in all cases we've tried. Recently, I looked in more detail at the actual set_payload() method in the email library and I have at least a vague understanding of the problem. The problem and my understanding are reported at . I have suggested a patch there which I call a 'Hint at possible fix'. This patch could be applied in Scrubber.py. The patch to Scrubber.py would add to the end of the def replace_payload_by_text(msg, text, charset): definition making the whole definition def replace_payload_by_text(msg, text, charset): # TK: This is a common function in replacing the attachment and the main # message by a text (scrubbing). Also, add a flag indicating it has been # scrubbed. del msg['content-type'] del msg['content-transfer-encoding'] msg.set_payload(text, charset) msg['X-Mailman-Scrubbed'] = 'Yes' if msg.get('content-transfer-encoding') == 'quoted-printable': cset = msg.get_charset() if cset: msg._payload = cset.body_encode(msg._payload) msg._charset = None The advantage to doing this in Scrubber.py and unconditionally setting the decode flag for subsequent get_payload() calls is it makes the whole process insensitive to whether or not or when Python email bug # 1409455 is fixed. If the bug is fixed, the payload will be encoded and msg.get_charset() above will return None so the payload won't be encoded a second time. The additional code above can be removed at some point after we're sure the email library used by Mailman is fixed. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From msapiro at value.net Sat Jan 28 19:06:39 2006 From: msapiro at value.net (Mark Sapiro) Date: Sat, 28 Jan 2006 10:06:39 -0800 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change toHandlers/SpamDetect.pyisincomplete In-Reply-To: <200601281713.k0SHDbvQ020918@ecf.utoronto.ca> Message-ID: Dan Astoorian wrote: > >Personally, I'm of the opinion that it's wrong ever to use a reject rule >for spam filtering in the first place, since so much spam has forged >sender information anyway, but debating that is well beyond the scope of >how this bug should be fixed. I think we all agree that 'spam' should be discarded or maybe held for examination by a human, but never rejected. The problem I see is that while header_filter_rules are in the Spam filters section of the admin interface and enforced by a module named SpamDetect, they can actually be used in other ways to reject messages which are not 'spam' per se. >However, I agree with Mark that discarding instead of rejecting feels >wrong. > >Would checking the x-beenthere: headers to avoid the loop not be a far >cleaner solution, or is there some reason I've overlooked why that >wouldn't work? X-Been-There: won't work because it's never added in messages to -owner. Perhaps we could test if msg.get_sender() == mlist.GetOwnerEmail(), and discard in that case only, but I'd want a somewhat different test in case the sender's domain were not identical - something like if msg.get_sender().split('@')[0] == \ mlist.GetOwnerEmail().split('@')[0]: -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From msapiro at value.net Sat Jan 28 19:23:06 2006 From: msapiro at value.net (Mark Sapiro) Date: Sat, 28 Jan 2006 10:23:06 -0800 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change toHandlers/SpamDetect.pyisincomplete In-Reply-To: Message-ID: Mark Sapiro wrote: > >Perhaps we could test if msg.get_sender() == mlist.GetOwnerEmail(), and >discard in that case only, but I'd want a somewhat different test in >case the sender's domain were not identical - something like > > if msg.get_sender().split('@')[0] == \ > mlist.GetOwnerEmail().split('@')[0]: Nope. This won't do either. Consider that someone spoofs a message from list1-owner to list2-owner and this matches a list2 reject rule. Reject is sent from list2-owner (rejects come from -owner) to list1-owner and this matches a list1 reject rule. Reject is sent from list1-owner to list2-owner and we have a ping-pong loop. If we're going to test sender, it would have to be something like if msg.get_sender().split('@')[0].endswith('-owner'): -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tkikuchi at is.kochi-u.ac.jp Sun Jan 29 01:07:23 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Sun, 29 Jan 2006 09:07:23 +0900 Subject: [Mailman-Developers] Scrubber mungs quoted-printable revisited. In-Reply-To: References: Message-ID: <43DC073B.9080902@is.kochi-u.ac.jp> Hi Mark, Mark Sapiro wrote: > Recently, I looked in more detail at the actual set_payload() method in > the email library and I have at least a vague understanding of the > problem. The problem and my understanding are reported at > . > I have suggested a patch there which I call a 'Hint at possible fix'. > This patch could be applied in Scrubber.py. > It's nice that the problem was tracked down. Thank you! > The patch to Scrubber.py would add to the end of the > > def replace_payload_by_text(msg, text, charset): > > definition making the whole definition > > def replace_payload_by_text(msg, text, charset): > # TK: This is a common function in replacing the attachment and the > main > # message by a text (scrubbing). Also, add a flag indicating it > has been > # scrubbed. > del msg['content-type'] > del msg['content-transfer-encoding'] > msg.set_payload(text, charset) > msg['X-Mailman-Scrubbed'] = 'Yes' > if msg.get('content-transfer-encoding') == 'quoted-printable': > cset = msg.get_charset() > if cset: > msg._payload = cset.body_encode(msg._payload) > msg._charset = None > I'm rather uneasy with _varname attribute is manupulated everywhere in the application code. Maybe we should override the email.Message behaviour by overriding in Mailman.Message. Also I will add 'base64' in encoding check and back out the patches related to the 'X-Mailman-Scruber'. > > The advantage to doing this in Scrubber.py and unconditionally setting > the decode flag for subsequent get_payload() calls is it makes the > whole process insensitive to whether or not or when Python email bug # > 1409455 is fixed. If the bug is fixed, the payload will be encoded and > msg.get_charset() above will return None so the payload won't be > encoded a second time. The additional code above can be removed at > some point after we're sure the email library used by Mailman is fixed. > > -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From msapiro at value.net Sun Jan 29 02:15:57 2006 From: msapiro at value.net (Mark Sapiro) Date: Sat, 28 Jan 2006 17:15:57 -0800 Subject: [Mailman-Developers] Put admin_member_chunksize in GUI Message-ID: Is there any reason to not put admin_member_chunksize in the GUI? It doesn't do the job requested in , but it helps. Is there a fear that some unsuspecting list owner will set it too big? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tkikuchi at is.kochi-u.ac.jp Sun Jan 29 02:54:32 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Sun, 29 Jan 2006 10:54:32 +0900 Subject: [Mailman-Developers] Put admin_member_chunksize in GUI In-Reply-To: References: Message-ID: <43DC2058.7090902@is.kochi-u.ac.jp> Mark Sapiro wrote: > Is there any reason to not put admin_member_chunksize in the GUI? > > It doesn't do the job requested in > , > but it helps. > > Is there a fear that some unsuspecting list owner will set it too big? > No reason, maybe. But, I just don't want to add a new language catalog entry and re-generate whole .po files for 2.2 now. :-( -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From msapiro at value.net Sun Jan 29 03:05:32 2006 From: msapiro at value.net (Mark Sapiro) Date: Sat, 28 Jan 2006 18:05:32 -0800 Subject: [Mailman-Developers] Put admin_member_chunksize in GUI In-Reply-To: <43DC2058.7090902@is.kochi-u.ac.jp> Message-ID: Tokio Kikuchi wrote: >No reason, maybe. But, I just don't want to add a new language catalog >entry and re-generate whole .po files for 2.2 now. :-( Actually, that's probably just as well. I started looking at it and the obvious place to put it is on the membership list page, but that isn't quite as simple as adding one more entry in a Gui module. So we'll save it for after 2.1.8. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From stephen at xemacs.org Sun Jan 29 15:13:57 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 29 Jan 2006 23:13:57 +0900 Subject: [Mailman-Developers] [ mailman-Bugs-1416853 ] Jan 14 change toHandlers/SpamDetect.pyisincomplete In-Reply-To: (Mark Sapiro's message of "Sat, 28 Jan 2006 10:06:39 -0800") References: Message-ID: <877j8j3yvu.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Mark" == Mark Sapiro writes: Mark> The problem I see is that while header_filter_rules are in Mark> the Spam filters section of the admin interface and enforced Mark> by a module named SpamDetect, they can actually be used in Mark> other ways to reject messages which are not 'spam' per se. I'm tempted to say that in that case the logic should be moved to a separate module, which is imported to SpamDetect with the restriction that REJECT is not available, while another Handler is created for PolicyFilters. They maintain separate rule lists etc. Sound like a lot of work when simply documenting that spam should _never_ be rejected should be enough. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From tkikuchi at is.kochi-u.ac.jp Tue Jan 31 02:37:25 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Tue, 31 Jan 2006 10:37:25 +0900 Subject: [Mailman-Developers] [Mailman-Users] any info on this reported exploit? In-Reply-To: <43D93815.8010009@is.kochi-u.ac.jp> References: <43D93815.8010009@is.kochi-u.ac.jp> Message-ID: <43DEBF55.6020504@is.kochi-u.ac.jp> Tokio Kikuchi wrote: >> http://www.securityfocus.com/bid/16248/discuss >> GNU Mailman Large Date Data Denial Of Service Vulnerability >> GNU Mailman is prone to a denial of service attack. This issue affects >> the >> email date parsing functionality of Mailman. (snip) >> 06.3.18 CVE: CVE-2005-4153 (snip) > > Mailman-2.1.7 is not vulnerable to this issue. > We may have to patch against this email package parsedate bug. I've just uploaded a patch on SF tracker. Please someone review this before I commit in the CVS (this weekend, maybe). https://sourceforge.net/tracker/?func=add&group_id=103&atid=300103 Cheers, -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From msapiro at value.net Tue Jan 31 02:48:46 2006 From: msapiro at value.net (Mark Sapiro) Date: Mon, 30 Jan 2006 17:48:46 -0800 Subject: [Mailman-Developers] [Mailman-Users] any info on this reportedexploit? In-Reply-To: <43DEBF55.6020504@is.kochi-u.ac.jp> Message-ID: Tokio Kikuchi wrote: > >We may have to patch against this email package parsedate bug. >I've just uploaded a patch on SF tracker. Please someone review this >before I commit in the CVS (this weekend, maybe). >https://sourceforge.net/tracker/?func=add&group_id=103&atid=300103 Above URI is not correct :-( See -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tkikuchi at is.kochi-u.ac.jp Tue Jan 31 03:02:05 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Tue, 31 Jan 2006 11:02:05 +0900 Subject: [Mailman-Developers] [Mailman-Users] any info on this reportedexploit? In-Reply-To: References: Message-ID: <43DEC51D.1050707@is.kochi-u.ac.jp> Mark Sapiro wrote: > Tokio Kikuchi wrote: > >>We may have to patch against this email package parsedate bug. >>I've just uploaded a patch on SF tracker. Please someone review this >>before I commit in the CVS (this weekend, maybe). >>https://sourceforge.net/tracker/?func=add&group_id=103&atid=300103 > > > Above URI is not correct :-( > > See > > Oops, sorry! :-P -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From msapiro at value.net Tue Jan 31 03:39:09 2006 From: msapiro at value.net (Mark Sapiro) Date: Mon, 30 Jan 2006 18:39:09 -0800 Subject: [Mailman-Developers] [Mailman-Users] any info on this reportedexploit? In-Reply-To: <43DEBF55.6020504@is.kochi-u.ac.jp> Message-ID: Tokio Kikuchi wrote: > >We may have to patch against this email package parsedate bug. >I've just uploaded a patch on SF tracker. Please someone review this >before I commit in the CVS (this weekend, maybe). I have looked at the patch in the tracker. Caveat: I haven't tested anything - this is just based on my reading. I think the patch is good. The issue I see is that Scrubber.py may not currently be doing the right thing if parsedate() returns None. Consider the attached patch for Scrubber.py in addition to the patch in the tracker. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Scrubber.patch.txt Url: http://mail.python.org/pipermail/mailman-developers/attachments/20060130/871b2038/attachment.txt From tkikuchi at is.kochi-u.ac.jp Tue Jan 31 05:12:09 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Tue, 31 Jan 2006 13:12:09 +0900 Subject: [Mailman-Developers] [Mailman-Users] any info on this reported exploit? In-Reply-To: References: Message-ID: <43DEE399.3010605@is.kochi-u.ac.jp> Mark Sapiro wrote: > Tokio Kikuchi wrote: > >>We may have to patch against this email package parsedate bug. >>I've just uploaded a patch on SF tracker. Please someone review this >>before I commit in the CVS (this weekend, maybe). > > > I have looked at the patch in the tracker. > > Caveat: I haven't tested anything - this is just based on my reading. > > I think the patch is good. The issue I see is that Scrubber.py may not > currently be doing the right thing if parsedate() returns None. > > Consider the attached patch for Scrubber.py in addition to the patch in > the tracker. Well, the logic may be unclear but calculate_attachment_dir() tries again to guess the real date of message arrival because it may be called from bin/arch. I think the code should be cleaned up but since we are now dealing with the email parsedate bug and it should be safe to limit our patch to this purpose. -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/