From dragondm@nexus.Integral.org Fri May 1 15:20:41 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Fri, 1 May 1998 09:20:41 -0500 (CDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) Message-ID: Aha!! a mailman developer's list!! Apparently I was in the same position as Ken Manheimer, with regards to MailMan. When the origional Mailman web sites & lists dissapeared, and didn't come back, I thought the author had abandoned development, and continued hacking at MailMan for my own use. I have made numerous changes to the code as a result. Now that I am aware that MailMan is being actively developed, I would like to see what I can do about merging these versions together. I started with a copy of Mailman 1.0b1. Because I was unsure of the status of MailMan, I have been calling my version DMM (the Dragon's version of MailMan), and the python modules I have added start with 'dmm_' I haven't yet had the time to download & go over the 1.0b2.0 (that is the most current version ?) sources, to see what things I've done may be redundant, (I will this weekend) but here is a list of the changes that I've made (that I can think of so far): (in roughly chronological order) Added Makefile to make & install all the wrappers. Earlier versions of this were posted to the previous MailMan dev list. Things have been improved since. Removed all dependance on MailMan being in /home/mailman. I made sure all scripts were always being run as the MailMan user, then used the pwd module to find the current user's home directory. Everything else is found from there. (I did this so I could have two MailMan installs on one machine, a 'production' server, which runs several lists, and a development server) Re-did logging system. Removed all /tmp logging. Created module that redirects errors to files in ~/logs Error log files are named for the scripts (thus errors from admin show up in ~/logs/admin.err) This is done automatically by importing the logging module. (major) Compleatly changed the way outgoing mail gets sent. Did much re-writing of mm_message & mm_deliver modules. Both OutgoingMessage & IncomingMessage are now derived from a common base class, ListMessage. Sending is accomplished by calling send method of message object. Mail is no longer sent by piping to 'sendmail'! Instead, outgoing messages are punted to a mailserver (currently always localhost) via SMTP. I wrote an SMTP module, smtplib to do this. smtplib is now part of the python standard library as of 1.5.1 Added random-mnemnonic password generator. Instead of passwords like 'qZ', AOL-like random-word passwords are generated (like 'cold-blue-tiger') Added multiple subject: prefix removal and Re: compaction changes subjects like: "Re: Prefix Re: Prefix Re Something" to "Prefix Re[3]: Something" (major) Changed the way user data is stored (still tying up loose ends on this) Created a User class. User data is no longer stored in the config.db file. Instead there is a shelf full of User objects. The main Mailinglist object contains an instance of a UserDB class that manages storing, retrieving & quering this database. Most of the *User* methods of the Mailinglist class have become methods of the User class. Changed the way user options are handled. Options are referenced by name (a string) and can be Boolean or "multiple-choice" values. Info on available options & allowed values for them are available via methods of the User objects. I compacted the 'Digests', "DisableDelivery" and "EnableMIME" options into one multiple-choice "delivery_mode" option." There have prolly been soem bugfixes too. Hopefully all this work wasn't redundant :> -The Dragon De Monsyne From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri May 1 16:05:59 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 1 May 1998 11:05:59 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) References: Message-ID: <13641.57859.522754.975416@anthem.CNRI.Reston.Va.US> Dragon, Very cool that you have continued hacking on Mailman. I think we should definitely work together to get a single merged version. One of the things that we hope to do RSN is get an anonymous CVS read-only tree up and running. That will hopefully keep developers as painfully up-to-date as they want, and encourage people to submit patches against the current devel versions. >>>>> "TDDM" == The Dragon De Monsyne writes: TDDM> Added Makefile to make & install all the wrappers. Earlier TDDM> versions of this were posted to the previous MailMan dev TDDM> list. Things have been improved since. TDDM> Removed all dependance on MailMan being in /home/mailman. I TDDM> made sure all scripts were always being run as the MailMan TDDM> user, then used the pwd module to find the current user's TDDM> home directory. Everything else is found from there. (I did TDDM> this so I could have two MailMan installs on one machine, a TDDM> 'production' server, which runs several lists, and a TDDM> development server) I'm currently working on autoconf'ing the installation process. I've got the src directory mostly working and just need to add a few more things before I check stuff in. I'd really like for you to be able to say ./configure --prefix=/not/home/mailman and have everything Just Work. The default $prefix will be /home/mailman. TDDM> (major) Compleatly changed the way outgoing mail gets sent. TDDM> Did much re-writing of mm_message & mm_deliver modules. Both TDDM> OutgoingMessage & IncomingMessage are now derived from a TDDM> common base class, ListMessage. Sending is accomplished by TDDM> calling send method of message object. Mail is no longer TDDM> sent by piping to 'sendmail'! Instead, outgoing messages are TDDM> punted to a mailserver (currently always localhost) via TDDM> SMTP. I wrote an SMTP module, smtplib to do this. smtplib is TDDM> now part of the python standard library as of 1.5.1 Very cool. I know this last bit was high on Ken's hit list. TDDM> Added random-mnemnonic password generator. Instead of TDDM> passwords like 'qZ', AOL-like random-word passwords are TDDM> generated (like 'cold-blue-tiger') Also very cool. I was thinking about using the implementation of RFC1751 from Andrew Kuchling's PCT to do something similar. Andrew and I have talked about what he can do to release the non-export controlled parts of the PCT separately. TDDM> Hopefully all this work wasn't redundant :> I don't think so. You've done some cool stuff that others are attacking or want to attack. Things may take a while to accrete as people discover that Mailman is alive and kicking. But it's really good to see so many people interested in Mailman's future! -Barry From klm@python.org Fri May 1 17:28:03 1998 From: klm@python.org (Ken Manheimer) Date: Fri, 1 May 1998 12:28:03 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: Message-ID: Mr dragon, It looks like we've got a nice synergy here - a number of the items you attacked are things i wanted to do, but did not yet do, while i (and now we) have taken care of a number of other things. (Most of the big things are mentioned in the notes i included in the distributions.) On Fri, 1 May 1998, The Dragon De Monsyne wrote: > I have made numerous changes to the code as a result. Now that I > am aware that MailMan is being actively developed, I would like to see > what I can do about merging these versions together. As barry mentioned, we're planning to make a cvs view of our version available across the net sometime soon. I should add that we will need to clear some admin issues first, so it's not guaranteed, but likely. > I started with a copy of Mailman 1.0b1. Because I was unsure of > the status of MailMan, I have been calling my version DMM (the Dragon's > version of MailMan), and the python modules I have added start with 'dmm_' I suspect that we started at the same place. Not sure whether there were any micro-releases or such that would lead to version skew. > I haven't yet had the time to download & go over the 1.0b2.0 (that > is the most current version ?) sources, to see what things I've done may > be redundant, (I will this weekend) but here is a list of the changes that > I've made (that I can think of so far): That is the latest version. I may release a patch with my latest hacking over the weekend, if i have time. ( Mostly bug fixes, some refinements - mime digest format, bounce handling, admin-option help.) > Removed all dependance on MailMan being in /home/mailman. > I made sure all scripts were always being run as the MailMan user, > then used the pwd module to find the current user's home directory. > Everything else is found from there. (I did this so I could have two > MailMan installs on one machine, a 'production' server, which runs several > lists, and a development server) Relaxing the /home/mailman rigidity is high on my list! My ideal would be to have as much as possible be in a package or use relative paths (which have the same effect - independence from location of the root). Unfortunately, i have a few other things on my agenda before that, so i'm not sure when i'll get to it. (High up is gettings things like the admin pages behind a cookied password, another item is making sure that a single mailman install can use different addresses for the host for different lists - os, eg, ISP clients with their own domains can use their domains for their mailman lists...) (I'm confused about how working with respect to the ~mailman homedir enables you to have to install's, btw.) > Re-did logging system. Removed all /tmp logging. Created module that > redirects errors to files in ~/logs Error log files are named for > the scripts (thus errors from admin show up in ~/logs/admin.err) This is > done automatically by importing the logging module. This is a place where we completely overlapped - and came up with similar answers, at least in the abstract. I also used ~mailman/logs, but am using a few categories of logs - bounce, vette, subscribe, digest, and error. Error gets the errors from bugs in any of the scripts, so you can just check the file periodically to see if anything is malfunctioning in a way that is caught... I guess we'll have to reconcile the two approaches. > (major) Compleatly changed the way outgoing mail gets sent. > Did much re-writing of mm_message & mm_deliver modules. Both > OutgoingMessage & IncomingMessage are now derived from a common base > class, ListMessage. Sending is accomplished by calling send method of > message object. > Mail is no longer sent by piping to 'sendmail'! Instead, outgoing > messages are punted to a mailserver (currently always localhost) > via SMTP. I wrote an SMTP module, smtplib to do this. smtplib is now part > of the python standard library as of 1.5.1 As barry also mentioned, this is one of the things up there on my "desired" list - but it never was high enough urgency to percolate to the top. (John actually sent me an smtplib.py that he was playing with, but i never took the step of hooking it up.) I'm glad you've done this, and that it uses something you've gotten into the distribution! > Added random-mnemnonic password generator. Instead of passwords like > 'qZ', AOL-like random-word passwords are generated (like > 'cold-blue-tiger') That's nice! People won't have to visit the site to either select reasonable passwords, or have their unmemorable one's sent to them... > Added multiple subject: prefix removal and Re: compaction > changes subjects like: > "Re: Prefix Re: Prefix Re Something" > to "Prefix Re[3]: Something" Me too, except i didn't do any munging of the Re, and in particular leave the Re before the prefix so mail readers sort threads appropriately. I'd rather avoid the fancy Re stuff, with the idea that mailman should do as little intrusion as possible to the content. > (major) Changed the way user data is stored (still tying up loose ends on > this) > Created a User class. User data is no longer stored in the > config.db file. Instead there is a shelf full of User objects. > The main Mailinglist object contains an instance of a UserDB class that > manages storing, retrieving & quering this database. Most of the *User* > methods of the Mailinglist class have become methods of the User class. If you're saying what i think you're saying, this is a real biggie, one which i've dearly wanted to see happen if i didn't get to it myself. (I think john had something like it going in a later, lost version of mailman, and i know a number of us have expressed interest in it.) > Changed the way user options are handled. Options are referenced by name > (a string) and can be Boolean or "multiple-choice" values. Info on > available options & allowed values for them are available via methods > of the User objects. I compacted the 'Digests', "DisableDelivery" and > "EnableMIME" options into one multiple-choice "delivery_mode" option." Sounds cool, provided there's no scaleup-defeating performance hits. We've got some people on this list interested in handling lists of several thousand members... > There have prolly been soem bugfixes too. > > Hopefully all this work wasn't redundant :> Mostly not! ken manheimer klm@python.org From viega@list.org Fri May 1 22:04:38 1998 From: viega@list.org (John Viega) Date: Fri, 1 May 1998 14:04:38 -0700 Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: ; from The Dragon De Monsyne on Fri, May 01, 1998 at 09:20:41AM -0500 References: Message-ID: <19980501140438.B13559@list.org> On Fri, May 01, 1998 at 09:20:41AM -0500, The Dragon De Monsyne wrote: > > Aha!! a mailman developer's list!! > > Apparently I was in the same position as Ken Manheimer, with regards to > MailMan. When the origional Mailman web sites & lists dissapeared, and > didn't come back, I thought the author had abandoned development, and > continued hacking at MailMan for my own use. Well, Ken and I were still in contact. I guess it's too bad you didn't contact me then, as we could have had your work integrated already and made things a bit easier on you. > Added Makefile to make & install all the wrappers. Earlier versions of > this were posted to the previous MailMan dev list. Things have > been improved since. This one has been done. > Removed all dependance on MailMan being in /home/mailman. This one I was going to do in the near future. I am glad that I did not :) > (major) Compleatly changed the way outgoing mail gets sent. > Did much re-writing of mm_message & mm_deliver modules. Both > OutgoingMessage & IncomingMessage are now derived from a common base > class, ListMessage. Sending is accomplished by calling send method of > message object. > Mail is no longer sent by piping to 'sendmail'! Instead, outgoing > messages are punted to a mailserver (currently always localhost) > via SMTP. I wrote an SMTP module, smtplib to do this. smtplib is now part > of the python standard library as of 1.5.1 Hmm, I wrote a basic smtplib a few months ago with the intent of using that instead of a call to sendmail, and I think I saw a cvs checkin from Ken that actually replaced the sendmail call. > > Added random-mnemnonic password generator. Instead of passwords like > 'qZ', AOL-like random-word passwords are generated (like > 'cold-blue-tiger') Cool. > Added multiple subject: prefix removal and Re: compaction > changes subjects like: > "Re: Prefix Re: Prefix Re Something" > to "Prefix Re[3]: Something" You might want to check the archives of this list on this topic, as we've discussed it recently, and agreed that we don't find it desirable for us. However, in thinking about it, I see no reason why it couldn't be added as an option if the list administrator seems to prefer it. > (major) Changed the way user data is stored (still tying up loose ends on > this) > Created a User class. User data is no longer stored in the > config.db file. Instead there is a shelf full of User objects. > The main Mailinglist object contains an instance of a UserDB class that > manages storing, retrieving & quering this database. Most of the *User* > methods of the Mailinglist class have become methods of the User class. That's good. I'd been moving in that direction when that disk crash occured. > Changed the way user options are handled. Options are referenced by name > (a string) and can be Boolean or "multiple-choice" values. Info on > available options & allowed values for them are available via methods > of the User objects. I compacted the 'Digests', "DisableDelivery" and > "EnableMIME" options into one multiple-choice "delivery_mode" option." Sounds reasonable, though I'd like to see it... > Hopefully all this work wasn't redundant :> For the most part, not! My only concern is keeping the amount of work necessary for integration on your part as low as possible. John From viega@list.org Fri May 1 22:16:24 1998 From: viega@list.org (John Viega) Date: Fri, 1 May 1998 14:16:24 -0700 Subject: [Mailman-developers] Web page Message-ID: <19980501141624.C13559@list.org> I'm going to be constructing a new Mailman web page at list.org in the next couple of days (probably after I finish my last final exam monday afternoon). One thing the old site had that I would like to reinstate is a list of all changes we would like to make that have not yet been made. To this end, I'd like people to send wishlists to the list, and for the group to discuss items on the wishlist. Some things that come to mind that are important for me include: 1) Having an optional (threaded) daemon mode that will obviate mailman having to start up multiple times. I've already seen lists where the administrator doesn't have any bounce handling on, and completely ignores all error messages. On a large list, error handling scripts get run frightingly often. 2) The user options should include a way of saying, "If I'm on the TO or CC list, don't send me another copy of that message!" 3) Nice, web-based, in place documentation. 4) Bounce detection needs some major work. I'm going to also look at this one soon. I'd like to also keep track of who is persuing what, so as to avoid overlap of effort. Any other suggestions on what should or should not show up on the Mailman home page are also welcome, as are contributions. From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri May 1 22:23:18 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 1 May 1998 17:23:18 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) References: <19980501140438.B13559@list.org> Message-ID: <13642.15485.769242.827267@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: >> Removed all dependance on MailMan being in /home/mailman. JV> This one I was going to do in the near future. I am glad that JV> I did not :) I'd like to see this stuff when its available so I can integrate it with the autoconf stuff I've been doing. I'm very near the point where the entire package can be easily configured and installed in the standard GNU way. Right now I have stuff set up so that $(prefix) defaults to /home/mailman but for reasons I won't go into, I actually have to configure with --prefix=/export/home/mailman (Ken will know why :-). -Barry From viega@list.org Fri May 1 22:23:12 1998 From: viega@list.org (John Viega) Date: Fri, 1 May 1998 14:23:12 -0700 Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: ; from Ken Manheimer on Fri, May 01, 1998 at 12:28:03PM -0400 References: Message-ID: <19980501142312.D13559@list.org> On Fri, May 01, 1998 at 12:28:03PM -0400, Ken Manheimer wrote: > I suspect that we started at the same place. Not sure whether there > were any micro-releases or such that would lead to version skew. If the version numbers were the same, they were the same. Any time I did a release, even if it was to a single person only, I changed the version number. > (I'm confused about how working with respect to the ~mailman homedir > enables you to have to install's, btw.) Well, if you have 2 installs and 2 sets of binaries, and 2 disjoint sets of lists, and 2 different cgi paths (ie, http://www.xxx.org/mailman-dev/) then I don't see why there would be any problem with it. > As barry also mentioned, this is one of the things up there on my > "desired" list - but it never was high enough urgency to percolate to > the top. (John actually sent me an smtplib.py that he was playing with, > but i never took the step of hooking it up.) I'm glad you've done this, > and that it uses something you've gotten into the distribution! Ah, I would have sworn you already did this one, Ken :) John From roy@endeavor.med.nyu.edu Fri May 1 22:26:54 1998 From: roy@endeavor.med.nyu.edu (Roy Smith) Date: Fri, 01 May 1998 17:26:54 -0400 Subject: [Mailman-developers] Web page Message-ID: <1895883.3103032414@qwerky.med.nyu.edu> Not sure if it does this already, but here's my biggest mailing list headache... We have our lists (run under majordomo, which we want to get rid of) set up that if you're not subscribed, you can't submit. This is the most obvious anti-spam measure, and goes a good way towards cutting down abuse. Problem is, people have multiple email addresses, or they change, or one MUA formats is slightly differently than another MUA they use, etc. It would be really cool to be able to say something like "accept postings from this person, if any of the following are on the From: line" Roy Smith New York University School of Medicine 550 First Avenue, New York, NY 10016 From viega@list.org Fri May 1 22:26:11 1998 From: viega@list.org (Mr. + Ms. John Viega) Date: Fri, 1 May 1998 14:26:11 -0700 Subject: [Mailman-developers] Web page In-Reply-To: <1895883.3103032414@qwerky.med.nyu.edu>; from Roy Smith on Fri, May 01, 1998 at 05:26:54PM -0400 References: <1895883.3103032414@qwerky.med.nyu.edu> Message-ID: <19980501142611.E13559@list.org> On Fri, May 01, 1998 at 05:26:54PM -0400, Roy Smith wrote: > Not sure if it does this already, but here's my biggest mailing list > headache... > > We have our lists (run under majordomo, which we want to get rid of) set up > that if you're not subscribed, you can't submit. This is the most obvious > anti-spam measure, and goes a good way towards cutting down abuse. Problem > is, people have multiple email addresses, or they change, or one MUA formats > is slightly differently than another MUA they use, etc. > > It would be really cool to be able to say something like "accept postings > from this person, if any of the following are on the From: line" Yes, both things are in Mailman: You can set an option that restricts non-members from posting (their posts get set aside for approval). You can list people from whom mail is always accepted without question. I don't think these two features have any sort of negative interaction. John From dragondm@nexus.Integral.org Sat May 2 03:51:50 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Fri, 1 May 1998 21:51:50 -0500 (CDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: Message-ID: On Fri, 1 May 1998, Ken Manheimer wrote: > Mr dragon, > > It looks like we've got a nice synergy here - a number of the items you > attacked are things i wanted to do, but did not yet do, while i (and now > we) have taken care of a number of other things. (Most of the big > things are mentioned in the notes i included in the distributions.) > > On Fri, 1 May 1998, The Dragon De Monsyne wrote: > > > I have made numerous changes to the code as a result. Now that I > > am aware that MailMan is being actively developed, I would like to see > > what I can do about merging these versions together. > > As barry mentioned, we're planning to make a cvs view of our version > available across the net sometime soon. I should add that we will need > to clear some admin issues first, so it's not guaranteed, but likely. Good! > > Removed all dependance on MailMan being in /home/mailman. > > I made sure all scripts were always being run as the MailMan user, > > then used the pwd module to find the current user's home directory. > > Everything else is found from there. (I did this so I could have two > > MailMan installs on one machine, a 'production' server, which runs several > > lists, and a development server) > > Relaxing the /home/mailman rigidity is high on my list! My ideal would > be to have as much as possible be in a package or use relative paths > (which have the same effect - independence from location of the root). > Unfortunately, i have a few other things on my agenda before that, so > i'm not sure when i'll get to it. (High up is gettings things like the > admin pages behind a cookied password, another item is making sure that > a single mailman install can use different addresses for the host for > different lists - os, eg, ISP clients with their own domains can use > their domains for their mailman lists...) > > (I'm confused about how working with respect to the ~mailman homedir > enables you to have to install's, btw.) I have two mailman users, 'mailman' & 'mm', and separate base urls, etc. The neccesary magic to make it all work is this line at the begining of the scripts: sys.path.append(os.path.join(pwd.getpwuid(os.geteuid())[5], "mailman","modules")) A bit ugly, but it works.( a package would be more elegant, of course. ) > > > Re-did logging system. Removed all /tmp logging. Created module that > > redirects errors to files in ~/logs Error log files are named for > > the scripts (thus errors from admin show up in ~/logs/admin.err) This is > > done automatically by importing the logging module. > > This is a place where we completely overlapped - and came up with > similar answers, at least in the abstract. I also used ~mailman/logs, > but am using a few categories of logs - bounce, vette, subscribe, > digest, and error. Error gets the errors from bugs in any of the > scripts, so you can just check the file periodically to see if anything > is malfunctioning in a way that is caught... > > I guess we'll have to reconcile the two approaches. Shouldn't be too difficult. What I did was very simple. Basically just re-directing stderr to a file in ~/logs named for sys.argv[0] The syslogish thing you have (now that I look at it) is a much more compleate solution. > > > (major) Compleatly changed the way outgoing mail gets sent. > > Did much re-writing of mm_message & mm_deliver modules. Both > > OutgoingMessage & IncomingMessage are now derived from a common base > > class, ListMessage. Sending is accomplished by calling send method of > > message object. > > Mail is no longer sent by piping to 'sendmail'! Instead, outgoing > > messages are punted to a mailserver (currently always localhost) > > via SMTP. I wrote an SMTP module, smtplib to do this. smtplib is now part > > of the python standard library as of 1.5.1 > > As barry also mentioned, this is one of the things up there on my > "desired" list - but it never was high enough urgency to percolate to > the top. (John actually sent me an smtplib.py that he was playing with, > but i never took the step of hooking it up.) I'm glad you've done this, > and that it uses something you've gotten into the distribution! This was one of my first 'big' projects with mm. It helped preformance quite a bit on my system. > > > Added random-mnemnonic password generator. Instead of passwords like > > 'qZ', AOL-like random-word passwords are generated (like > > 'cold-blue-tiger') > > That's nice! People won't have to visit the site to either select > reasonable passwords, or have their unmemorable one's sent to them... Yup. It's fairly simple (just based on whrandom) but it works nicely. > > > Added multiple subject: prefix removal and Re: compaction > > changes subjects like: > > "Re: Prefix Re: Prefix Re Something" > > to "Prefix Re[3]: Something" > > Me too, except i didn't do any munging of the Re, and in particular > leave the Re before the prefix so mail readers sort threads > appropriately. I'd rather avoid the fancy Re stuff, with the idea that > mailman should do as little intrusion as possible to the content. Yah. I had a list with it, and they kept getting "prefix Re: prefix Re: prefix ...." I munge the subject before the prefix gets put on the outgoing message, so the prefix winds up first. The problem with having the Re first, is then filtering dosen't work. > > > (major) Changed the way user data is stored (still tying up loose ends on > > this) > > Created a User class. User data is no longer stored in the > > config.db file. Instead there is a shelf full of User objects. > > The main Mailinglist object contains an instance of a UserDB class that > > manages storing, retrieving & quering this database. Most of the *User* > > methods of the Mailinglist class have become methods of the User class. > > If you're saying what i think you're saying, this is a real biggie, one > which i've dearly wanted to see happen if i didn't get to it myself. (I > think john had something like it going in a later, lost version of > mailman, and i know a number of us have expressed interest in it.) Yup. I may be tweaking the code a bit on this before diving in in earnest to try to integrate things (I have a circular ref I'd like to eliminate, and I just saw a very nice way to do so today. ) but basically my idea was this: I wanted to have hierarchy of objects here. You start with a Server object that manages all server-wide info. you can get Mailinglist objects from the Server object (like Server.GetList("foolist") ), that handle all list-wide info, and you get User objects from the Mailinglist that handle all User specific stuff. Right now, I've just finished/am finishing the User object addition, and havent gotten to the Server object yet. > > > Changed the way user options are handled. Options are referenced by name > > (a string) and can be Boolean or "multiple-choice" values. Info on > > available options & allowed values for them are available via methods > > of the User objects. I compacted the 'Digests', "DisableDelivery" and > > "EnableMIME" options into one multiple-choice "delivery_mode" option." > > Sounds cool, provided there's no scaleup-defeating performance hits. > We've got some people on this list interested in handling lists of > several thousand members... > Hmmm.... Well, I will admit, flexibility was higher in my mind than preformance, however the UserDB object that manages quering the shelf full of User objects does do some caching using dictionaries. That should make the impact minmimal. -The Dragon De Monsyne From klm@python.org Sat May 2 20:51:15 1998 From: klm@python.org (Ken Manheimer) Date: Sat, 2 May 1998 15:51:15 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: Message-ID: On Fri, 1 May 1998, The Dragon De Monsyne wrote: > On Fri, 1 May 1998, Ken Manheimer wrote: > > > (I'm confused about how working with respect to the ~mailman homedir > > enables you to have to install's, btw.) > > I have two mailman users, 'mailman' & 'mm', and separate base > urls, etc. The neccesary magic to make it all work is this line at the > begining of the scripts: Ah, a separate user. Cool. > sys.path.append(os.path.join(pwd.getpwuid(os.geteuid())[5], > "mailman","modules")) > > A bit ugly, but it works.( a package would be more elegant, of course. ) Not bad. If you look at mailman/cgi/private, andrew kuchling did something along the same lines. You implemented something more refined by getting the user id of calling process. Using relative paths wherever possible (using packages for the modules and relative paths for scripts and wrapper executables) is my favorite prospect. > Yah. I had a list with it, and they kept getting > "prefix Re: prefix Re: prefix ...." I munge the subject before the prefix > gets put on the outgoing message, so the prefix winds up first. The > problem with having the Re first, is then filtering dosen't work. Not sure what you're saying, but i implemented something here that leaves the "re" alone, but still filters fine to avoid cascading prefixes. Small hack, with no apparent problems and many messages under the bridge (or something:). > > > (major) Changed the way user data is stored (still tying up loose ends on > > > this) > > > Created a User class. User data is no longer stored in the > > > [...] > > If you're saying what i think you're saying, this is a real biggie, one > Yup. I may be tweaking the code a bit on this before diving in in earnest > [...] > this: I wanted to have hierarchy of objects here. You start with a > Server object that manages all server-wide info. you can get Mailinglist Sounds like this will also feed in well to john's planned creation of a threaded persistent server mode. It also sounds like it'll provide a good basis for another feature i really want - user profiles. The simplest use of these profiles would be to contain a user's preferred password, which a user shares across lists on a site. They'd just have to change that password, rather than going through the process for every list to which they subscribe. (List-specific passwords could still be available, so the user wouldn't *have* to use the same password for all.) Also, delivery address could become a secondary aspect of the user account, rather than the key by which it is identified. This way, the user could change the delivery address without having to unsubscribe and resubscribe. (They'd have to go through a confirmation process that proves they're in charge of the new address, to prevent mischief makers from subscribing at a valid address and then changing to some arbitrary other address.) And the profile could be used for "incidental" member info, like name, organization, etc, to provide background for the mailing list community - etc... Anyway, it all sounds real good! ken manheimer klm@python.org From klm@python.org Sat May 2 21:02:54 1998 From: klm@python.org (Ken Manheimer) Date: Sat, 2 May 1998 16:02:54 -0400 (EDT) Subject: [Mailman-developers] Web page In-Reply-To: <19980501142611.E13559@list.org> Message-ID: On Fri, 1 May 1998, Mr. + Ms. John Viega wrote: > On Fri, May 01, 1998 at 05:26:54PM -0400, Roy Smith wrote: > > Not sure if it does this already, but here's my biggest mailing list > > headache... > > > > We have our lists (run under majordomo, which we want to get rid of) set up > > that if you're not subscribed, you can't submit. This is the most obvious > > anti-spam measure, and goes a good way towards cutting down abuse. Problem > > is, people have multiple email addresses, or they change, or one MUA formats > > is slightly differently than another MUA they use, etc. > > > > It would be really cool to be able to say something like "accept postings > > from this person, if any of the following are on the From: line" > > Yes, both things are in Mailman: > > You can set an option that restricts non-members from posting (their > posts get set aside for approval). > > You can list people from whom mail is always accepted without question. > > I don't think these two features have any sort of negative interaction. I agree with roy that this feature would be useful(*). Use of the always-accepted list of addresses for this purpose doesn't quite cut it for me - it requires list-administrator intervention for what really is a user-specific option. I could imagine on a big list, with lots of subscriptions happening all the time, the list admin would be kept too busy just adding the always-accepted addrs. I suspect this will be another job for the much anticipated (:-) user profiles. ((*) I must say, the require_explicit_destination option seems to be a remarkably effective spam filter. I sometimes want to advertise it heavily (not via spam:), but then think that if everyone in the world were using it the spammers would concentrate more on the bulk mailers that specify the destinations in the headers, so i don't harp on it too much...) ken manheimer klm@python.org From klm@python.org Sat May 2 21:20:53 1998 From: klm@python.org (Ken Manheimer) Date: Sat, 2 May 1998 16:20:53 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: <19980501142312.D13559@list.org> Message-ID: On Fri, 1 May 1998, John Viega wrote: > > As barry also mentioned, this is one of the things up there on my > > "desired" list - but it never was high enough urgency to percolate to > > the top. (John actually sent me an smtplib.py that he was playing with, > > but i never took the step of hooking it up.) I'm glad you've done this, > > and that it uses something you've gotten into the distribution! > > Ah, I would have sworn you already did this one, Ken :) I did add your smtplib.py to the distribution. I bet my checkin message was not clear about the fact that i hadn't actually hooked it up yet. (I think my notes mention hooking it up as a todo, though.) BTW, i see a note in the dragon's 1.5.1-included smtplib.py which says it "doesn't do esmtp (yet)". I remembered seeing a pipelining esmtp module going by the newsgroup, and sure enough a deja news search shows "FREE: Python ESMTP Module by me!" from daniel robbins. Maybe this code would provide a leg up, at least as an example, on incorporating the command pipelining extension to smtplib.py? ken manheimer klm@python.org From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Sat May 2 22:09:38 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Sat, 2 May 1998 17:09:38 -0400 (EDT) Subject: [Mailman-developers] Web page References: <1895883.3103032414@qwerky.med.nyu.edu> <19980501142611.E13559@list.org> Message-ID: <13643.35665.905958.716734@anthem.CNRI.Reston.Va.US> >>>>> "MJV" == Ms John Viega writes: MJV> You can set an option that restricts non-members from posting MJV> (their posts get set aside for approval). MJV> You can list people from whom mail is always accepted without MJV> question. MJV> I don't think these two features have any sort of negative MJV> interaction. Although it might be cool, once there's a real user database, to allow users to set up aliases for themselves. I.e. their mail could come from multiple addresses. You'd still have to verify through each given address though. -Barry From klm@python.org Mon May 4 19:50:40 1998 From: klm@python.org (Ken Manheimer) Date: Mon, 4 May 1998 14:50:40 -0400 (EDT) Subject: [Mailman-developers] mailman 1.0b3 patch in the ftp dir Message-ID: <199805041850.OAA11723@glyph.CNRI.Reston.Va.US> As promised (though slightly late), i put together a mostly-fixes patch release, bringing mailman to 1.0b3. It takes care of some significant (but not showstopper) bugs that the intermediate patch i released did not do. See the README file in the ftp dir: ftp://ftp.python.org/pub/python/contrib/Network/mailman A couple of notes. First, i'm sad to say that the patch is with respect to 1.0b2, not taking into account the intermediate patch i released. When you apply it you may have to ignore some "reversed or already applied patch" prompts, and/or do some reconciliation by hand. I'm sorry about this - i am still ironing out my release techniques. (My lesson in this case was to *always* add a symbolic tag to the version control logs when you release a patch. I didn't do so for the intermediate patch, and the option of trying to correct the logs retroactively is too error prone to reliably improve matters.) This may be the last release we'll do here at python.org, now that john is nearing having time to devote to mailman maintenance, and in particular, to reactivation of the mailman web site, www.list.org. (I've set the default value for MAILMAN_URL in mm_defaults.py to that value, in anticipation of this event.) Also, i'm hoping to tone down my time on mailman a bit, to pay better attention to my proper duties (and proper weekends), and take on just some specific todos in mailman development itself. It sounds like we have the beginnings of a small cadre around the net who have serious interest in mailman development, and a start at organizing the effort to avoid overlap and ease integration. If you have development plans and want to participate, be sure to send email to john (viega@list.org) with your list of personal-priority todos, so we can have a coordinated effort here... Yay! ken manheimer, 05/04/1998 klm@python.org From klm@python.org Tue May 5 18:00:33 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 5 May 1998 13:00:33 -0400 (EDT) Subject: [Mailman-developers] Re: anyone got mailman running on freebsd ? In-Reply-To: <6il5oh$l0p$1@nnrp1.dejanews.com> References: <6il5oh$l0p$1@nnrp1.dejanews.com> Message-ID: <13647.9309.893202.35064@glyph.CNRI.Reston.Va.US> A message in the python newsgroup points out a likely portability problem with mailman under FreeBSD, in particular with file locking mechanisms. sweeting@neuronet.com.my writes: > Sorry to trouble you but I've run into a problem setting up > mailman (python mailling list software) on FreeBSD and wonder > if there are any special config for FBSD ? > [...] > However, accessing any of the URLs to manage the lists > returned server config errors. The apache logs said were > almost as unhelpful : > > [Tue May 5 03:10:07 1998] access to /home/mailman/cgi-bin/listinfo failed for > 202.184.153.17, reason: Premature end of script headers > [Tue May 5 03:10:18 1998] access to /home/mailman/cgi-bin/admin failed for > 202.184.153.17, reason: Premature end of script headers (Note for mailman-developers, including me - it may be worth noting in the troubleshooting section of the README about Apache (and other?) error log notices something to the effect of "premature end of script headers" meaning that the script in question probably bombed before producing any, or sufficient, output. And, it occurs to me, it may be worth breaking out the troubleshooting section of the README into a separate file, and eventually as a section of a prospective admin guide.) > Trying to interact from the python interpreter also > produced errors : > >>> import sys > [...] > File "/usr/local/lib/python1.5/posixfile.py", line 190, in lock > flock = fcntl.fcntl(self._file_.fileno(), cmd, flock) > IOError: (22, 'Invalid argument') We mailman developers need to look into this issue, in particular guido's suggestion that: Guido: > A much more portable way to do file locking is to use fcntl.lockf()! Clearly we need some fairly portable solution, and it probably makes sense to tackle this now. Barry has an article by jamie zawinski pointing out the pitfalls of locking, with a conclusion (i gather) that use of a directory (with a leading '.' dot in its name) is the best portable answer. I'd be interested to hear whether anyone has a fairly clear-win solution - one that's simple and portable - and/or whether just switching over to fcntl.lockf() is the right thing. Maybe andrew's discussions (in that thread) about revision of the posixfile interfaces should play in here... Ken From akuchlin@cnri.reston.va.us Tue May 5 19:31:18 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Tue, 5 May 1998 14:31:18 -0400 (EDT) Subject: [Mailman-developers] Re: anyone got mailman running on freebsd ? In-Reply-To: <13647.9309.893202.35064@glyph.CNRI.Reston.Va.US> References: <6il5oh$l0p$1@nnrp1.dejanews.com> <13647.9309.893202.35064@glyph.CNRI.Reston.Va.US> Message-ID: <13647.23013.946909.814142@newcnri.cnri.reston.va.us> Ken Manheimer writes: >Clearly we need some fairly portable solution, and it probably makes >sense to tackle this now. Barry has an article by jamie zawinski >pointing out the pitfalls of locking, with a conclusion (i gather) >that use of a directory (with a leading '.' dot in its name) is the >best portable answer. I'd be interested to hear whether anyone has a >fairly clear-win solution - one that's simple and portable - and/or >whether just switching over to fcntl.lockf() is the right thing. >Maybe andrew's discussions (in that thread) about revision of the >posixfile interfaces should play in here... The person sent me the relevant header file; the flock structure is: struct flock { off_t l_start; /* starting offset */ off_t l_len; /* len = 0 means until end of file */ pid_t l_pid; /* lock owner */ short l_type; /* lock type: read/write, etc. */ short l_whence; /* type of l_start */ }; I don't have a FreeBSD machine to fiddle with, so I can't really help suggest a fix. The struct format in posixfile.py is: if sys.platform in ('netbsd1', 'freebsd2', 'freebsd3'): flock = struct.pack('lxxxxlxxxxlhh', \ l_start, l_len, os.getpid(), l_type, l_whence) Perhaps the xxxx padding is unnecessary or something, but a FreeBSD person will have to look into it. For Python, probably the best solution is to change posixfile.py to use lockf(), and possibly change the interface in other ways. Best to discuss this on the main list. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ But I'm getting too old for the trick, which troubles me, for the sea is in my blood like a fever and I don't know how I can leave. -- Jim's envoi, in SANDMAN #53: "Hob's Leviathan" From klm@python.org Tue May 5 20:28:28 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 5 May 1998 15:28:28 -0400 (EDT) Subject: [Mailman-developers] More mailman locking issues Message-ID: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> Another issue with mailman locking - i think it's coarse grained, and needs to be refined. For example, this morning someone noticed that the umbrella listinfo page on python.org was hanging, because, it turns out, one of the lists was left in a locked state. (Not sure how that happened - it usually doesn't.) Well, the umbrella listinfo (and admin) pages open the lists only for the sake of determining which are public ones - no writing is actually required, so a lock isn't really required. It might be sufficient to offer a way to open a list "read-only", such that no locking is done. I think it would be better, however, to refine the locking system such that lists are not locked until they enter an operation that changes the list data in a way that will eventually need to be written. This would probably take a more effort than a "read-only" kluge, but may not be too bad. I guess this is one for the todo list. (Wish i had gobs of time to continue to hack on a these sorts of things - they're not very big or hard, but there is a bunch of them, and they need to be done right.) Ken From viega@list.org Tue May 5 20:37:31 1998 From: viega@list.org (John Viega) Date: Tue, 5 May 1998 12:37:31 -0700 Subject: [Mailman-developers] More mailman locking issues In-Reply-To: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US>; from Ken Manheimer on Tue, May 05, 1998 at 03:28:28PM -0400 References: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> Message-ID: <19980505123731.A12842@list.org> Well, if one of the lists was left unlocked, there was probably a bug... Anyway, WRT. locking, I think that several things need to happen minimum: 1) Locks should insert a time stamp. After x seconds (30?) you can lose the lock. Anything such as the archiving software should use that time to copy the archive over to a tmp file and nulling out the old file so that other procs can overwrite (Not that hard). 2) A version of Load() should be written that doesn't call Lock() (call it LoadUnlocked() I guess). Change things that don't care to use it (Even easier). 3) Make sure Lock() uses portable calls (might require a bit of research). I can look into this stuff next week, but if anyone wants to do it sooner, let us know and feel free... John On Tue, May 05, 1998 at 03:28:28PM -0400, Ken Manheimer wrote: > Another issue with mailman locking - i think it's coarse grained, and > needs to be refined. For example, this morning someone noticed that > the umbrella listinfo page on python.org was hanging, because, it > turns out, one of the lists was left in a locked state. (Not sure how > that happened - it usually doesn't.) Well, the umbrella listinfo (and > admin) pages open the lists only for the sake of determining which are > public ones - no writing is actually required, so a lock isn't really > required. > > It might be sufficient to offer a way to open a list "read-only", such > that no locking is done. I think it would be better, however, to > refine the locking system such that lists are not locked until they > enter an operation that changes the list data in a way that will > eventually need to be written. This would probably take a more effort > than a "read-only" kluge, but may not be too bad. > > I guess this is one for the todo list. (Wish i had gobs of time to > continue to hack on a these sorts of things - they're not very big or > hard, but there is a bunch of them, and they need to be done right.) > > Ken > > ------------------------------------------------------ > Mailman-developers maillist - Mailman-developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Tue May 5 20:45:47 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 5 May 1998 15:45:47 -0400 (EDT) Subject: [Mailman-developers] More mailman locking issues References: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> Message-ID: <13647.26901.437384.156650@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> It might be sufficient to offer a way to open a list KM> "read-only", such that no locking is done. I think it would KM> be better, however, to refine the locking system such that KM> lists are not locked until they enter an operation that KM> changes the list data in a way that will eventually need to be KM> written. This would probably take a more effort than a KM> "read-only" kluge, but may not be too bad. Of course, much of these file-locking problems go away once you have a long-lived mailman server. Then again, you'll have threading issues to deal with :-). I haven't looked at what it is that Mailman is actually trying to lock (too busy interspersing autoconf hacking time in with Real Work), but I'm guess from what I've read that you're just trying to keep multiple writers from clobbering each other (and readers from reading inconsistent data when a writer is writing). It sounds like you don't really need to lock the file, but instead to lock out access to a shared resource, and you want this to work across NFS. I wasn't able to dig up the Netscape Mail + File Locking article that Jamie Zawinski wrote. I don't seem to have it in my bookmarks or easily grepable and a web search turned up only stale links. I emailed Jaime to see if he's got an updated link and will forward that if I get a response. If I remember correctly, the article basically justifies the point of view that the best portable way to do resource locking across NFS is by creating a file with O_CREAT | O_EXCL. That's what's meant by "dot-file" locking because you usually create a .file that is mostly invisible. Since the check-for-existance and file-creation steps are atomic (on Solaris at least), you won't every have two processes step on each other. The additional advantage is that just rm'ing the dot-file is enough to unlock the system. The disadvantage is that this operation may not be atomic for older versions of NFS, although I believe it is for NFS 3 (can't find a reference to this right now though). On the other hand, creation of directories *is* atomic across all versions of NFS, AFAIK, so perhaps it makes more sense to create a dot-directory that locks the resource. You'd have to use mkdir() and check for EEXIST. You'd basically busy-loop in either case waiting for the lock to be given up. So creating a dot-directory representing a lock is probably the most portable, safest, and easiest way to do it. -Barry From tismer@appliedbiometrics.com Tue May 5 21:16:49 1998 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Tue, 05 May 1998 22:16:49 +0200 Subject: [Mailman-developers] More mailman locking issues References: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> <13647.26901.437384.156650@anthem.CNRI.Reston.Va.US> Message-ID: <354F73B1.F4DD56B5@appliedbiometrics.com> Barry A. Warsaw wrote: [a lot :] > On the other hand, creation of directories *is* atomic across all > versions of NFS, AFAIK, so perhaps it makes more sense to create a > dot-directory that locks the resource. You'd have to use mkdir() and > check for EEXIST. You'd basically busy-loop in either case waiting > for the lock to be given up. > > So creating a dot-directory representing a lock is probably the > most portable, safest, and easiest way to do it. Another easy way is to simply use the "lockfile" utility which comes with procmail. Procmail has its own install script to check for the locking abilities of the particular installation and does intense testing, and I never got any problems with it when I was using smartlist, which is a procmail based minidomo. (BTW, reading the smartlist source can cause serious damage to your brain, it is write-only-coded procmail and shell). If one needs a quick and immediately working solution, I can recommend this, at least give it a slot in your "tips&tricks" box. -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaiserin-Augusta-Allee 101 : *Starship* http://starship.skyport.net 10553 Berlin : PGP key -> http://pgpkeys.mit.edu we're tired of banana software - shipped green, ripens at home From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed May 6 20:36:53 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 6 May 1998 15:36:53 -0400 (EDT) Subject: [Mailman-Developers] Re: [Mailman-CVS] CVS: 'mailman/modules mm_digest.py' References: <13643.35405.507841.64622@anthem.CNRI.Reston.Va.US> Message-ID: <13648.47311.397569.686639@anthem.CNRI.Reston.Va.US> [Ken, I hope you don't mind me changing the follow up to the developers list. Folks, this thread started in response to a checkin that strips certain headers... -BAW] >>>>> "KM" == Ken Manheimer writes: >> Digest messages now keep all headers except 'received', >> 'errors-to', and all 'x-*' ones (and any continuations of >> these). >> should x-* headers be stripped? the user can use them to add >> interesting tidbits that probably ought to be preserved. what >> is the rationale as to which headers get stripped and which >> remain? KM> If you take a look at a non-mime digest (or a mime digest in a KM> reader that doesn't minimize the headers to just essential KM> ones) you'll see that it doesn't take much to have the headers KM> outweigh, and obscure, the messages. My sense is that KM> sticking to official headers - and not all of them at that - KM> is all that can be justified. The default settings for most KM> mail readers, minimizing out even things like organization and KM> sender, gives credence to this view. And people reading the KM> non-mime digests don't have the option of mimimizing the KM> headers display... In general, I like a policy kind of like the Hippocratic oath of mailers, "above all do minimal munging". I agree with you that headers may end up outweighing content in RFC934 digests, so there has to be tradeoffs. I personally don't much like digest mailing lists, so maybe I'm not the right person to say, but I also am a little uncomfortable that digest subscribers get a different message than non-digest subscribers. My mental model is that the digest is just the sum of all the message a non-digest subscriber would see, packaged conveniently, and sent out less frequently. If I were to view a MIME digest in a MIME savvy MUA (e.g. VM/XEmacs), I could unpack the digest and view the individual articles, and it would look no different than if I was a non-digest reader. Perhaps this can't be acheived in practice, and maybe it really does make sense to strip or change some headers. But I guess I don't like the idea of stripping out headers that users take special pains to craft, like Reply-to and most X-* headers (and not just 'cause I like the ones I add). I don't care if most MUA hide this stuff. Those headers can be useful! I know that VM's author (Kyle Jones) takes special care to include as many headers as possible when creating digests, precisely because those headers are occasionally useful. Maybe there's a distinction to be made b/w MIME and non-MIME digests? I can see more of a rationale for stripping headers in RFC934 style digests. -Barry From viega@list.org Thu May 7 03:06:37 1998 From: viega@list.org (John Viega) Date: Wed, 6 May 1998 19:06:37 -0700 Subject: [Mailman-Developers] Big announcement! Message-ID: <19980506190637.A24164@list.org> I've got a few things to announce. First, I've come up w/ a big Mailman TODO list that is now on the web. If you want to tackle something off that list, just let this mailing list know, and go at it! The TODO list is at: http://www.list.org/todo.html Please submit additions, corrections, etc. to me. Second, as of our next release, Mailman will be released under the Gnu GPL. Third, Mailman is now GNU software. From the next release forward, it will be distributed by the FSF. There will be a second web page on www.gnu.org, which will almost certainly be a mirror of the list.org web page. We're going to wait around 3-4 weeks before our next release. We want to give everyone time to get their work incorporated into this first big release (at which time, we'll advertise the software fairly heavily on the net). John From tdyas@remus.rutgers.edu Thu May 7 06:29:36 1998 From: tdyas@remus.rutgers.edu (Tom Dyas) Date: Thu, 7 May 1998 01:29:36 -0400 (EDT) Subject: [Mailman-Developers] Big announcement! In-Reply-To: <19980506190637.A24164@list.org> Message-ID: On Wed, 6 May 1998, John Viega wrote: > I've got a few things to announce. First, I've come up w/ a big > Mailman TODO list that is now on the web. If you want to tackle > something off that list, just let this mailing list know, and go at > it! > > The TODO list is at: http://www.list.org/todo.html > > Please submit additions, corrections, etc. to me. What about support for a "confirm" command similar to listserv? While the password scheme is nice, it seems possible for a bad person to spoof an email from an unsuspecting user with the password in the command after they have used the WWW interface. FYI, I help run the Linux development lists on vger.rutgers.edu in my spare time. I have seen _many_ forged requests to Majordomo that come in as requests for approval. If mailman were to send out a unique value determined only by it and only sent to the subscriber, then there is a much better guarantee that it was the subscriber who confirmed the subscription request. An secondary feature would be to send out a confirmation request to all current subscribers for them to stay on the list like once a month. The Bugtraq list does this using ListServ. While not many lists would use it, it would be nice. I can hack on some of this but are there any relevant parts of mailman I should consider for this? I have programmed in Python before so I just need pointers to right files and such. Btw, I am going to try out mailman on a CVS commits list (vger cvs is home to linux/sparc) that I am going to make on vger. Vger uses the zmailer MTA. Is there any way to make mailman use an "outgoing"-style list alias? Zmailer is very good about sending out mail if it has all of the destinations at once since it can group nearby domains and such to send to exploder sites. Tom From scott@chronis.pobox.com Thu May 7 06:38:11 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 7 May 1998 01:38:11 -0400 Subject: [Mailman-Developers] Big announcement! In-Reply-To: ; from Tom Dyas on Thu, May 07, 1998 at 01:29:36AM -0400 References: <19980506190637.A24164@list.org> Message-ID: <19980507013811.03723@chronis.icgroup.com> On Thu, May 07, 1998 at 01:29:36AM -0400, Tom Dyas wrote: | On Wed, 6 May 1998, John Viega wrote: | | > I've got a few things to announce. First, I've come up w/ a big | > Mailman TODO list that is now on the web. If you want to tackle | > something off that list, just let this mailing list know, and go at | > it! | > | > The TODO list is at: http://www.list.org/todo.html | > | > Please submit additions, corrections, etc. to me. | | What about support for a "confirm" command similar to listserv? While the | password scheme is nice, it seems possible for a bad person to spoof an | email from an unsuspecting user with the password in the command after | they have used the WWW interface. this is in the works by me. if i understand correctly, the consensus was to require confirmations. i have them working in an offshoot branch of mailman, and plan to put together and post patches against the most recent distribution next week sometime. in the meantime, sorry, i'm real busy with other stuff. | FYI, I help run the Linux development lists on vger.rutgers.edu in my | spare time. I have seen _many_ forged requests to Majordomo that come in | as requests for approval. If mailman were to send out a unique value | determined only by it and only sent to the subscriber, then there is a | much better guarantee that it was the subscriber who confirmed the | subscription request. that's just what it will do. | An secondary feature would be to send out a confirmation request to all | current subscribers for them to stay on the list like once a month. The | Bugtraq list does this using ListServ. While not many lists would use it, | it would be nice. hmmm, seems like an idea. what do you do with the members who don't respond in 2 weeks because they're on vacation? | I can hack on some of this but are there any relevant parts of mailman I | should consider for this? I have programmed in Python before so I just | need pointers to right files and such. i've already got it done, and just need to put it to the newest release. no reason to be redundant about development... | Btw, I am going to try out mailman on a CVS commits list (vger cvs is home | to linux/sparc) that I am going to make on vger. Vger uses the zmailer | MTA. Is there any way to make mailman use an "outgoing"-style list alias? | Zmailer is very good about sending out mail if it has all of the | destinations at once since it can group nearby domains and such to send to | exploder sites. not that i know with the addresses per se, but you can set the maximum outgoing connections for each list. for vmailer, it would probably be good to just have that number be <= the size of the list. same effect. BTW. vmailer is great, can't wait till it goes beta... scott From tdyas@remus.rutgers.edu Thu May 7 07:01:41 1998 From: tdyas@remus.rutgers.edu (Tom Dyas) Date: Thu, 7 May 1998 02:01:41 -0400 (EDT) Subject: [Mailman-Developers] Big announcement! In-Reply-To: <19980507013811.03723@chronis.icgroup.com> Message-ID: On Thu, 7 May 1998, Scott wrote: > | What about support for a "confirm" command similar to listserv? While the > | password scheme is nice, it seems possible for a bad person to spoof an > | email from an unsuspecting user with the password in the command after > | they have used the WWW interface. > > this is in the works by me. if i understand correctly, the consensus > was to require confirmations. i have them working in an offshoot > branch of mailman, and plan to put together and post patches against > the most recent distribution next week sometime. in the meantime, Cool. > | An secondary feature would be to send out a confirmation request to all > | current subscribers for them to stay on the list like once a month. The > | Bugtraq list does this using ListServ. While not many lists would use it, > | it would be nice. > > hmmm, seems like an idea. what do you do with the members who don't > respond in 2 weeks because they're on vacation? I have only seen Bugtraq actually use this feature. It is a very large list with a reasonable amount of traffic. I think for such lists the "you lose" policy is in effect for such people since they can always resubscribe when they come back and can view a WWW archive for info they missed. > not that i know with the addresses per se, but you can set the maximum > outgoing connections for each list. for vmailer, it would probably be > good to just have that number be <= the size of the list. same > effect. > > BTW. vmailer is great, can't wait till it goes beta... vmailer? I could probably just write a script that runs out of cron every 15 minutes that checks to see if any *-outgoing files need to be regenerated. Tom From viega@list.org Thu May 7 13:10:40 1998 From: viega@list.org (John Viega) Date: Thu, 7 May 1998 05:10:40 -0700 Subject: [Mailman-Developers] Big announcement! In-Reply-To: ; from Tom Dyas on Thu, May 07, 1998 at 01:29:36AM -0400 References: <19980506190637.A24164@list.org> Message-ID: <19980507051040.A27198@list.org> On Thu, May 07, 1998 at 01:29:36AM -0400, Tom Dyas wrote: > What about support for a "confirm" command similar to listserv? While the > password scheme is nice, it seems possible for a bad person to spoof an > email from an unsuspecting user with the password in the command after > they have used the WWW interface. If I understand you correctly, what you're looking for is there. A confirmation step was added recently. I think it needs to be improved upon to the point where you can just reply to the message to confirm (as long as some string is in the subject, say). For things other than subscribe, the password is necessary from the web, and the email address is checked from mail commands (I don't think the password is). > An secondary feature would be to send out a confirmation request to all > current subscribers for them to stay on the list like once a month. The > Bugtraq list does this using ListServ. While not many lists would use it, > it would be nice. Sure, that's a good idea for large lists. I think you should be able to set the period, though. I run a few large lists where I'd like to do that 2 times a year only. I'll add it to the todo list today. > I can hack on some of this but are there any relevant parts of mailman I > should consider for this? I have programmed in Python before so I just > need pointers to right files and such. Well, the core of mailman is in the modules directory. The maillist file is perhaps the best place to start; subsystems such as digest handling and bounce detection are in other files in that directory, and are included by multiple inheritance. The commands that interact with the email addresses, the cgi scripts, etc. All live in different subdirectories. They all end up instantiating mailing lists to do their work. > Btw, I am going to try out mailman on a CVS commits list (vger cvs is home > to linux/sparc) that I am going to make on vger. Vger uses the zmailer > MTA. Is there any way to make mailman use an "outgoing"-style list alias? > Zmailer is very good about sending out mail if it has all of the > destinations at once since it can group nearby domains and such to send to > exploder sites. I don't know if I understand this one. Do you want mylist-outgoing to be the sender address as opposed to mylist-admin? John From dragondm@nexus.Integral.org Thu May 7 14:52:52 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Thu, 7 May 1998 08:52:52 -0500 (CDT) Subject: [Mailman-Developers] Big announcement! In-Reply-To: <19980507051040.A27198@list.org> Message-ID: On Thu, 7 May 1998, John Viega wrote: > On Thu, May 07, 1998 at 01:29:36AM -0400, Tom Dyas wrote: > > > Btw, I am going to try out mailman on a CVS commits list (vger cvs is home > > to linux/sparc) that I am going to make on vger. Vger uses the zmailer > > MTA. Is there any way to make mailman use an "outgoing"-style list alias? > > Zmailer is very good about sending out mail if it has all of the > > destinations at once since it can group nearby domains and such to send to > > exploder sites. > > I don't know if I understand this one. Do you want mylist-outgoing to > be the sender address as opposed to mylist-admin? I think what he is talking about is working the way majordomo does, with an alias file with all the subscriber names in it, so the MTA can work various sorting & delivery magic. to Tom: I think that kind of mode of operation would be troublesome, fortunately, there is another way. I have a divergent version of Mailman 1.0b1 that has been modifed to send outgoing mail by punting it to the local mailserver via SMTP. This gives the MTA all the info it needs to do various sorting/routing magic. I intend to sit down someday soon, and see what I can do to integrate this back into the main Mailman source tree. -The Dragon De Monsyne From klm@python.org Thu May 7 17:30:54 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 7 May 1998 12:30:54 -0400 (EDT) Subject: [Mailman-Developers] Re: Todo list questions In-Reply-To: <19980506160759.A23032@list.org> Message-ID: [Greets. The following message comes out of a side discussion with barry and john, but it contains some substantial explanations of a few issues that i think will interest at least some others, so i'm cc'ing the entire list.] On Wed, 6 May 1998, John Viega wrote: > I'm looking at merging the TODO lists. I have questions. > > You say that address matching isn't working, giving the example > klm@parrot.python.org should match klm@python.org. I wrote code to do > that sort of matching, and it used to work great. Is it not still > there? Or does it not work for some functions? Or what? It is there, and it does happen to work for that particular example. However, it does not accept some that the original did. In particular, the original version was much too accepting on one hand: oldmm_utils.AddressesMatch('klm@perl.org', 'klm@python.org') => 1 because of the matching ".org". Likewise: oldmm_utils.AddressesMatch('klm@kgb.ukraine.ru', 'klm@politech.ukraine.ru') => 1 This was actually a serious problem - before, people who had the same login name as someone already on the list, and who happened to have the same top level domain, were simply refused subscription with a "you're already subscribed" message. That (and tracebacks when either of the arguments was an unqualified login names, eg 'klm' instead of 'klm@python.org') prompted me to make my revisions. My new version does the right thing when the account names are the same and one domain contains the other: mm_utils.AddressesMatch('klm@parrot.python.org', 'klm@python.org') => 1 What my version does *not* do is accept sibling hosts in a domain as the same email addr: mm_utils.AddressesMatch('klm@parrot.python.org', 'klm@larch.python.org') => 0 Of course, this is the right thing when we're talking about the top level domain. However, it's open for debate whether we want to accept common account names on sibling hosts within a local naming authority (second level domain, eg 'python.org') as being the same account. My read of the situation is that matching there might typically be correct for small domains (like python.org or cnri.reston.va.us) and *not* correct for moderate or large ones (eg, sun.com or reston.va.us or ukraine.ru). So i opted for the conservative approach - accept matches for domain containment, but not sibling hosts within a domain. I expect there's more to be exploited in the specifics of the addresses. Then again, i could make a table of cases with items that have the same structure and which should be accepted as matches in some cases but not in others. So i'm not convinced that this is a sufficient mechanism for, eg, recognizing postings from maillist members - i think we would need a mechanism where members explicitly designate alternate posting addresses. In any case, we need to think about it more - i think that's what i said on my todo list entry... > What would a "which" mailcmd do? Tell the user all the public maillists of which they are a member. (It would also mention that there are private maillist memberships, if any, and the command might could take an optional password, which would be used to include any of the private maillists which that password qualifies.) > Isn't the confirmation code already in place? (BTW, if I haven't said When 'web_subscribe_requires_confirmation' is set to "Requestor confirms via email", which is what we're talking about, then a message is sent to the delivery address the prospective member specified. That subscription-confirmation message is really just a subscription template - instructions for subscribing, with the exact subscription line that they can just send back to the reply address. As such, i don't really consider it a confirmation mechanism, really it's just a customized (with the users selected password) template for email subscription. This is fine when the desired delivery address is the same as the address the user would be sending the "confirmation". Where it breaks down is when the user's delivery address is different than the address from which they send their "confirmation" (subscription). If we allowed a user to subscribe for delivery to an arbitrary address, then we're fucked - mischief makers can wreak havoc. That's why we don't. What we would need to do that is a real subscription confirmation mechanism, where a modest random string (a nonce) is sent to the delivery address. The user can send back that string from any address, proving that they have control over (really, just access to - but that's the best we can do, anyway) the delivery address. The problem is that we currently have no infrastructure for tracking dispatched nonces - the "confirmation" mechanism is currently stateless, which is why i keep putting the "confirmation" in quotes. I was interested in doing something like that, but scott cotton seemed game to do something like it, so i layed out the situation and he said he'd do it (and the membership management stuff). Apparently he has, but couldn't offer patches, only an entire new copy of the system (including other changes, as well), and i haven't had the time or energy to delve into it to unravel the changes. This is what he's been referring to the last couple of days, when people bring up these issues, when saying he's going to do it when he gets a moment (or two:) to do it. This nice thing is, the specific goal scott and i initially discussed was alternate-address subscription confirmation, where a user could subscribe to a delivery address (eg, klm@acm.org) from a different address (eg, klm@python.org). I expect this will be useful for this other issue i was talking about yesterday, enlistment of redistribution addresses. The only problem is that it requires the delivery address to receive the nonce, and some redistribution addresses may already be active and populated, which would require the redistribution administrator to either intercept the confirmation request, or just let it go to their redistribution audience. Oh well. The upshot is that the current requestor-confirms-via-email mechanism isn't really a confirmation mechanism, it's a deliver-instructions-for- subscribing-via-email mechanism. And it looks like we're getting a real confirmation mechanism, soon. > it yet, the confirmation mechanism needs to be a LOT easier to use. > All you should have to do is hit reply. Novices aren't going to get > it to work the way it currently is.) Yes, you did mention this, and my understanding of that is that the user should be able to just hit reply and send the resulting message. (This will apply equally for a real, nonce-based confirmation mechanism.) This should be fairly easy to do, i just haven't yet wanted to spend time thinking about how to identify significant pitfalls, and avoid them. There are some. For example, how about when someone hits reply, cites the entire confirmation template, but adds at the top or bottom "What the hell is going on, i never asked to subscribe, why did i get sent this?!" It would be Not Good (tm) if our mechanism saw the subscription confirmation message cited in its entirety and took it as confirmation, ay? You-thought-you-were-asking-simple-questions-ay'ly yers, Ken From viega@list.org Thu May 7 17:52:19 1998 From: viega@list.org (John Viega) Date: Thu, 7 May 1998 09:52:19 -0700 Subject: [Mailman-Developers] Re: Todo list questions In-Reply-To: ; from Ken Manheimer on Thu, May 07, 1998 at 12:30:54PM -0400 References: <19980506160759.A23032@list.org> Message-ID: <19980507095219.A28772@list.org> On Thu, May 07, 1998 at 12:30:54PM -0400, Ken Manheimer wrote: > It is there, and it does happen to work for that particular example. > However, it does not accept some that the original did. In particular, > the original version was much too accepting on one hand: > > oldmm_utils.AddressesMatch('klm@perl.org', > 'klm@python.org') => 1 That would be a bug, if so. It was intended to *require* the last two names to match. > because of the matching ".org". Likewise: > > oldmm_utils.AddressesMatch('klm@kgb.ukraine.ru', > 'klm@politech.ukraine.ru') => 1 I never really cosidered that one, but it should be easy to fix :) > Of course, this is the right thing when we're talking about the top > level domain. However, it's open for debate whether we want to accept > common account names on sibling hosts within a local naming authority > (second level domain, eg 'python.org') as being the same account. I would say it should be an option defaulting to yes. > My read of the situation is that matching there might typically be > correct for small domains (like python.org or cnri.reston.va.us) and > *not* correct for moderate or large ones (eg, sun.com or reston.va.us or > ukraine.ru). So i opted for the conservative approach - accept matches > for domain containment, but not sibling hosts within a domain. Well, take advantage of the info you know. In foreign addresses, require 3 matches. In .us addresses, require 4. For sun.com, I would leave it at two. It's a single company, they tend to have aliases like jim.gosling@sun.com, which should match @foof.engr.sun.com. I have never really heard of anyones having big problems w/ majordomo lists that do this sort of fuzzy matching. I think it makes things a lot easier on people who forget which machine they were on when they subscribed, etc... (which I do all the time). > for, eg, recognizing postings from maillist members - i think we would > need a mechanism where members explicitly designate alternate posting > addresses. I think that's useful no matter what, but not a very high priority. > > What would a "which" mailcmd do? > > Tell the user all the public maillists of which they are a member. (It > would also mention that there are private maillist memberships, if any, > and the command might could take an optional password, which would be > used to include any of the private maillists which that password > qualifies.) Ah, ok. > What we would need to do that is a real subscription confirmation > mechanism, where a modest random string (a nonce) is sent to the > delivery address. The user can send back that string from any address, > proving that they have control over (really, just access to - but that's > the best we can do, anyway) the delivery address. Right, that's what I had in mind... > The problem is that we currently have no infrastructure for tracking > dispatched nonces - the "confirmation" mechanism is currently stateless, > which is why i keep putting the "confirmation" in quotes. Ok. > I was interested in doing something like that, but scott cotton seemed > game to do something like it, so i layed out the situation and he said > he'd do it (and the membership management stuff). Apparently he has, > but couldn't offer patches, only an entire new copy of the system > (including other changes, as well), and i haven't had the time or energy > to delve into it to unravel the changes. This is what he's been > referring to the last couple of days, when people bring up these issues, > when saying he's going to do it when he gets a moment (or two:) to do > it. Ok. > The upshot is that the current requestor-confirms-via-email mechanism > isn't really a confirmation mechanism, it's a deliver-instructions-for- > subscribing-via-email mechanism. And it looks like we're getting a real > confirmation mechanism, soon. Well, if it's not a confirmation mechanism of some sort, then why was it added in the first place? > For example, how about when someone hits reply, cites the entire > confirmation template, but adds at the top or bottom "What the hell is > going on, i never asked to subscribe, why did i get sent this?!" It > would be Not Good (tm) if our mechanism saw the subscription > confirmation message cited in its entirety and took it as confirmation, > ay? Well, if the title is: REPLY TO CONFIRM YOUR SUBSCRIPTION [SOMECODE#] Then you would hope people won't make that mistake. I'm sure some people will nonetheless :) John From scott@chronis.pobox.com Thu May 7 19:46:55 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 7 May 1998 14:46:55 -0400 Subject: [Mailman-Developers] Re: Todo list questions In-Reply-To: <19980507095219.A28772@list.org>; from John Viega on Thu, May 07, 1998 at 09:52:19AM -0700 References: <19980506160759.A23032@list.org> <19980507095219.A28772@list.org> Message-ID: <19980507144655.16636@chronis.icgroup.com> my personal vote on this is to have the confirmation set up so that you can just reply to it to trigger subscription. scott On Thu, May 07, 1998 at 09:52:19AM -0700, John Viega wrote: | > For example, how about when someone hits reply, cites the entire | > confirmation template, but adds at the top or bottom "What the hell is | > going on, i never asked to subscribe, why did i get sent this?!" It | > would be Not Good (tm) if our mechanism saw the subscription | > confirmation message cited in its entirety and took it as confirmation, | > ay? | | Well, if the title is: REPLY TO CONFIRM YOUR SUBSCRIPTION [SOMECODE#] | | Then you would hope people won't make that mistake. | I'm sure some people will nonetheless :) | | John | | ------------------------------------------------------ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers From scott@chronis.pobox.com Thu May 7 19:48:25 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 7 May 1998 14:48:25 -0400 Subject: [Mailman-Developers] Re: Todo list questions Message-ID: <19980507144825.54162@chronis.icgroup.com> On Thu, May 07, 1998 at 09:52:19AM -0700, John Viega wrote: | On Thu, May 07, 1998 at 12:30:54PM -0400, Ken Manheimer wrote: [stuff about address matching] just wanted to put in my $.02 about address matching. i don't think any special mechanism other than string.lower(string.strip(address1)) == string.lower(string.strip(address2)) should be used. therer are some instances , for example, 2 of my own address, where it is perfectly valid for user@furtherqualified.domain.dom to be a different address than user@domain.dom. in addition when you start taking into account how many parts you should compare based on individual top level domains, you're hard wiring knowledge that can change daily. my vote is to have addresses match only if they are the same username and have the same doman part, case folded. scott From viega@list.org Thu May 7 19:53:30 1998 From: viega@list.org (John Viega) Date: Thu, 7 May 1998 11:53:30 -0700 Subject: [Mailman-Developers] Re: Todo list questions Message-ID: <19980507115330.B29602@list.org> On Thu, May 07, 1998 at 02:43:24PM -0400, Scott wrote: > just wanted to put in my $.02 about address matching. i don't think > any special mecahnsim other than > > string.lower(string.strip(address1)) == > string.lower(string.strip(address2)) should be used. therer are some > instances , for example, 2 of my own address, where it is perfectly valid > for user@furtherqualified.domain.dom to be a different address than > user@domain.dom. > > in addition when you start taking into account how many parts you > should compare based on individual top level domains, you're hard > wiring knowledge that can change daily. > > my vote is to have addresses match only if they are the same username > and have the same doman part, case folded. > Well, I agree with you, but I know people who will not. Do you really think it's a bad idea to let the site admin change it if he prefers? John From tismer@appliedbiometrics.com Thu May 7 20:16:41 1998 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Thu, 07 May 1998 21:16:41 +0200 Subject: [Mailman-Developers] Re: Todo list questions References: <19980507115330.B29602@list.org> Message-ID: <35520899.D1F01C9D@appliedbiometrics.com> John Viega wrote: > > On Thu, May 07, 1998 at 02:43:24PM -0400, Scott wrote: > > > just wanted to put in my $.02 about address matching. i don't think > > any special mecahnsim other than > > > > string.lower(string.strip(address1)) == > > string.lower(string.strip(address2)) should be used. therer are some > > instances , for example, 2 of my own address, where it is perfectly valid > > for user@furtherqualified.domain.dom to be a different address than > > user@domain.dom. > > > > in addition when you start taking into account how many parts you > > should compare based on individual top level domains, you're hard > > wiring knowledge that can change daily. > > > > my vote is to have addresses match only if they are the same username > > and have the same doman part, case folded. > > > > Well, I agree with you, but I know people who will not. Do you really > think it's a bad idea to let the site admin change it if he prefers? If that feature is there why not. But I don't believe there is an urgent need for this. Wouldn't it be the more general solution to allow for an alias list which explicitly says which addresses are considered equal? We would define multiple ingoing addresses and one outgoing which is the "nominal" address. Since there are many special cases (Aaron Watters used to use many completely different addresses, for istance), the domain mangling would not solve all requirements, and this one is needed anyway. ciao - pirx -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaiserin-Augusta-Allee 101 : *Starship* http://starship.skyport.net 10553 Berlin : PGP key -> http://pgpkeys.mit.edu we're tired of banana software - shipped green, ripens at home From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri May 8 22:26:22 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 8 May 1998 17:26:22 -0400 (EDT) Subject: [Mailman-Developers] Re: Todo list questions References: <19980506160759.A23032@list.org> Message-ID: <13651.30182.475430.18450@anthem.CNRI.Reston.Va.US> <> I wonder if DNS MX records could be used to help out here? You could strip lefthand components until you find an MX record. Do this for both addresses and then compare MX records. Of course, you have the disadvantages we've talked about before for tying in with DNS... Barring that, you might just want to steal Majordomo's algorithm since it seems to work in practice ;-). <> To avoid quoting/citing hell, you'll probably have to put the confirmation magic in the Subject: header, so a simply reply will contain that data largely unchanged (module Re:'s). In the *body* of the message you can put some explanatory text saying By replying to this message you will be subscribed to the foobar@thingie.com mailing list. Do not modify the Subject header! If you do not want to be subscribed to foobar@thingie.com just ignore and/or delete this message. For more information about the foobar@thingie.com mailing list, please see That ought to be sufficient for 99% of moderately clueful people. :-) -Barry From viega@list.org Sat May 9 04:58:39 1998 From: viega@list.org (John Viega) Date: Fri, 8 May 1998 20:58:39 -0700 Subject: [Mailman-Developers] Mailman users Message-ID: <19980508205839.A7992@list.org> Could I ask everyone to join the mailing list mailman-users@python.org? When we start making big public announcements, it would be nice to have a couple of people on the list to help field questions. Also, there's a pretty lame Mailman web page up on www.list.org. Any contributions for the web page of any sort would be appreciated. Please contact me about it. John From klm@python.org Sat May 9 05:05:53 1998 From: klm@python.org (Ken Manheimer) Date: Sat, 9 May 1998 00:05:53 -0400 (EDT) Subject: [Mailman-Developers] Mailman users In-Reply-To: <19980508205839.A7992@list.org> Message-ID: > Also, there's a pretty lame Mailman web page up on www.list.org. Any > contributions for the web page of any sort would be appreciated. > Please contact me about it. (I have to say, i really like the picture, though i don't find it to exactly be a logo. In fact, i find it kinda psychedelic, like some of the plastic-hyperreal stuff i used to see in Heavy Metal comics years ago. But i'm a bit worn out today - maybe it's just flashbacks...-) Ken From klm@python.org Tue May 12 18:09:30 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 12 May 1998 13:09:30 -0400 (EDT) Subject: [Mailman-Developers] Quandry 2: Alternate hosts and sendmail Message-ID: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US> I'm hitting a bit of a quandry in determining the _correct_ setting for the sender of the subscription confirmation-request message (the thing sent when web_subscribe_requires_confirmation is "Requestor confirms via email"), and am wondering if anyone has a simple, clear answer. The problem is in some conflicting constraints in the confirmation-request return address, which can be resolved in most cases by judicious use of the sender and reply-to message fields, but which apparently cannot be resolved simply in a way that satisfies all MUAs. Here are the constraints: 1 Regular replies to the confirmation request should be directed to the list's "-request" address (where it will be processed by the mailcmd script). 2 A slightly more obscure constraint is that errors should be directed to the -admin address (or perhaps something else), *not* to the "-request" addr. And, unfortunately, setting the "errors-to" header to the -admin addr is not sufficient - in certain cases, eg when the remote address specified by the subscriber can't even be resolved in DNS, the sender receives the error notice. I think, but am not sure, that there are cases where the message makes it to the remote site but errors still get returned to the sender, not errors-to, addr. It is the last part that dictates what we're currently doing: Reply-to: -request addr Sender: -admin addr Errors-to: -admin addr This works fine in most cases - users hit "reply" in the MUA, and get the right response address. Bounced messages go to the -admin, who can diagnose the situation and/or ignore it. Note that i said "most cases" - i, as list administrator, have gotten a few errant replies from users. On investigation, it turns out that a *few* MUAs, eg /bin/mail on some platform(s) or other, do not respect the reply-to header. Dammit. So i guess my question is, is it worth trying a different approach that will allow both the reply-to *and* the sender to be the -request addr, just to accomodate the few mailers that don't respect reply-to? That would probably mean increasing the complexity of the mailcmd script significantly, to detect requests that are actually bounces of the confirmation-requests. I'm just starting to count how often this happens, but it seems like it could be a real problem for lists with lots of subscriptions... Is my explanation of the situation clear? Any strong ideas, suggestions? Ken From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Tue May 12 18:16:26 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 12 May 1998 13:16:26 -0400 (EDT) Subject: [Mailman-Developers] Quandry 2: Alternate hosts and sendmail References: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US> Message-ID: <13656.33682.522330.4854@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> Note that i said "most cases" - i, as list administrator, have KM> gotten a few errant replies from users. On investigation, it KM> turns out that a *few* MUAs, eg /bin/mail on some platform(s) KM> or other, do not respect the reply-to header. Dammit. KM> So i guess my question is, is it worth trying a different KM> approach that will allow both the reply-to *and* the sender to KM> be the -request addr, just to accomodate the few mailers that KM> don't respect reply-to? Brain dead MUAs will always be a fact of life and I doubt you'll ever eliminate all problems. Stick with best current practice and try to at least enumerate those systems that are broken (and suggest their users fix their systems :-). -Barry From klm@python.org Tue May 12 18:32:54 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 12 May 1998 13:32:54 -0400 (EDT) Subject: [Mailman-Developers] Question: Alternate hosts and sendmail Message-ID: <199805121732.NAA04868@glyph.CNRI.Reston.Va.US> [Oy - i managed to put the wrong subject line on the last message. This one really does concern alternate hosts and sendmail.] This question really is about sendmail configuration, though it is in pursuit of a mailman issue. The general problem is that some people run various maillists that are effectively in different domains, and they'd like to be able to run them from a single installation of mailman on a host that occupies all the domains. (This probably will be a major concern for ISPs who host subscriber domains - they probably want to *not* have to run a separate maillist system for each one!) In fact, mailman supports this - at the bottom of the General Options page for any list (in 1.0b3) you'll find settings for the preferred host name and URL that the list thinks it occupies. It works fine. It is my sendmail configuration that gets in the way, here. My problem is that my sendmail configuration mangles all outgoing messages from my host (eg, parrot.python.org) to look like they're coming from, eg, 'blat@python.org'. I want this mangling in general, but would like for it to *not* happen when messages are supposed to be coming from, eg, "cnri.reston.va.us". Unfortunately, i haven't sufficiently mastered sendmail setup to prevent it. All my poking and prodding has not prevailed. Can anyone out there with greater sendmail savvy point me to particular things to investigate? (We're running a range of sendmail 8.8.5 and 8.8.7 on various machines - i think the one where this really will be an issue is at 8.8.7.) Thanks for any clues! Ken From viega@list.org Tue May 12 18:45:50 1998 From: viega@list.org (John Viega) Date: Tue, 12 May 1998 10:45:50 -0700 Subject: [Mailman-Developers] Quandry 2: Alternate hosts and sendmail In-Reply-To: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US>; from Ken Manheimer on Tue, May 12, 1998 at 01:09:30PM -0400 Message-ID: <19980512104550.A2343@list.org> On Tue, May 12, 1998 at 01:09:30PM -0400, Ken Manheimer wrote: > So i guess my question is, is it worth trying a different approach > that will allow both the reply-to *and* the sender to be the -request > addr, just to accomodate the few mailers that don't respect reply-to? > That would probably mean increasing the complexity of the mailcmd > script significantly, to detect requests that are actually bounces of > the confirmation-requests. Two things. 1) You only have to do this on confirmations. Otherwise, people should be contacting the address on their own accord. So just set up a -confirm addess, make it the sender and the reply-to, and then see if the subject has the token in it, which it should not for a bounce. 2) You could always have confirmations require an "extra web step" instead of responding to an email. John From scott@chronis.pobox.com Tue May 12 22:13:53 1998 From: scott@chronis.pobox.com (Scott) Date: Tue, 12 May 1998 17:13:53 -0400 Subject: [Mailman-Developers] Quandry 2: Alternate hosts and sendmail In-Reply-To: <19980512104550.A2343@list.org>; from John Viega on Tue, May 12, 1998 at 10:45:50AM -0700 References: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US> <19980512104550.A2343@list.org> Message-ID: <19980512171353.47830@chronis.icgroup.com> hmm, this might make my life with confirmations more difficult... (without addressing this, a patch for confirmations is on the way shortly...) On Tue, May 12, 1998 at 10:45:50AM -0700, John Viega wrote: | | On Tue, May 12, 1998 at 01:09:30PM -0400, Ken Manheimer wrote: | | > So i guess my question is, is it worth trying a different approach | > that will allow both the reply-to *and* the sender to be the -request | > addr, just to accomodate the few mailers that don't respect reply-to? | | > That would probably mean increasing the complexity of the mailcmd | > script significantly, to detect requests that are actually bounces of | > the confirmation-requests. | | Two things. | | 1) You only have to do this on confirmations. Otherwise, people | should be contacting the address on their own accord. So just set up | a -confirm addess, make it the sender and the reply-to, and then see | if the subject has the token in it, which it should not for a bounce. I've seen subject lines of bounced messages maintain the subject line, prepending it with some tag or other. Is there any guarantee that scanning the subject will adequately determine what is and isn't a bounce? | 2) You could always have confirmations require an "extra web step" | instead of responding to an email. email confirmations are easier (since the user will not have to change from a MUA to a webrowser to do anything) and therefore more desirable. Any more ideas? I'm prone to follow Barry's suggestion of the current practice, but that may be biased by the fact that my work with adding email confirmation of subscriptions and subscriptions to alternate addresses works in the current framework... Scott From viega@list.org Tue May 12 22:32:36 1998 From: viega@list.org (John Viega) Date: Tue, 12 May 1998 14:32:36 -0700 Subject: [Mailman-Developers] Re: confirmations In-Reply-To: <19980512171353.47830@chronis.icgroup.com>; from Scott on Tue, May 12, 1998 at 05:13:53PM -0400 References: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US> <19980512104550.A2343@list.org> <19980512171353.47830@chronis.icgroup.com> Message-ID: <19980512143236.B2382@list.org> On Tue, May 12, 1998 at 05:13:53PM -0400, Scott wrote: > hmm, this might make my life with confirmations more difficult... > (without addressing this, a patch for confirmations is on the way > shortly...) > I've seen subject lines of bounced messages maintain the subject > line, prepending it with some tag or other. Is there any guarantee > that scanning the subject will adequately determine what is and isn't > a bounce? Well, let's say your subject is: List-name confirm (XYZZY-867-5309) If you say, "reply to this message, or send a message with subject XYZZY-867-5309", then You should be able to just see if the subject line is the same, the ID exactly, or if it has an Re: or RE: prepended. > | 2) You could always have confirmations require an "extra web step" > | instead of responding to an email. > > email confirmations are easier (since the user will not have to change > from a MUA to a webrowser to do anything) and therefore more > desirable. Sure, but if there is no better solution that will always work... > Any more ideas? I'm prone to follow Barry's suggestion of the current > practice, but that may be biased by the fact that my work with adding > email confirmation of subscriptions and subscriptions to alternate > addresses works in the current framework... Well, if Berkley mail doesn't respect Reply-To:, that's quite a problem. I know a lot of people who use it because it is quick and dirty (e.g., they don't want to wait for pine to load). I don't mind ignoring it, but it's going to be a problem anyway :) From klm@python.org Tue May 12 23:46:11 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 12 May 1998 18:46:11 -0400 (EDT) Subject: [Mailman-Developers] CGI variables question - hostname used in URL Message-ID: <199805122246.SAA06371@glyph.CNRI.Reston.Va.US> As i mentioned in my sendmail question, i have someone that will need to run mailman to handle multiple lists in different domains. (Thanks, scott, for the pointer to sendmails ruleset 1 - i'll be able to point the system guys in the right direction.) It looks like mailman handles that just fine, *except* for one addition i had made. The overarching listinfo and admin pages, which provide the roster of lists on the system, need to show only the (advertised) lists for that virtual system - not the ones that are effectively on a different system. My problem is that i seem to have only a server-specific mechanism for getting the hostname part of the URL by which the script was visited, and i'm wondering whether there's a more general way. Running apache (1.2.0), i see that HTTP_HOST has the info i need, and can compare that with the list's host_name (checking whether one includes the other or vice versa), excluding those lists that do not. However, it does not look like HTTP_HOST is by any means universal. The provisional thing i've implemented is to just ignore the list's host_name setting if no HTTP_HOST env var is found, and show all advertised lists in that case. What i'm wondering is whether there are other variable names i should check for other servers, or some more general mechanism. (It's nice to have a bunch of knowledgeable people on the list to tap into for info, btw.) Ken From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed May 13 00:09:16 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 12 May 1998 19:09:16 -0400 (EDT) Subject: [Mailman-Developers] CGI variables question - hostname used in URL References: <199805122246.SAA06371@glyph.CNRI.Reston.Va.US> Message-ID: <13656.54885.16779.208982@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> Running apache (1.2.0), i see that HTTP_HOST has the info i KM> need, and can compare that with the list's host_name (checking KM> whether one includes the other or vice versa), excluding those KM> lists that do not. However, it does not look like HTTP_HOST KM> is by any means universal. SERVER_NAME is the standard I think. From klm@python.org Wed May 13 00:15:20 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 12 May 1998 19:15:20 -0400 (EDT) Subject: [Mailman-Developers] CGI variables question - hostname used in URL In-Reply-To: <13656.54885.16779.208982@anthem.CNRI.Reston.Va.US> Message-ID: On Tue, 12 May 1998, Barry A. Warsaw wrote: > >>>>> "KM" == Ken Manheimer writes: > > KM> Running apache (1.2.0), i see that HTTP_HOST has the info i > KM> need, and can compare that with the list's host_name (checking > KM> whether one includes the other or vice versa), excluding those > KM> lists that do not. However, it does not look like HTTP_HOST > KM> is by any means universal. > > SERVER_NAME is the standard I think. That's the official name of the server - not the name by which the page was visited. Ie, on visiting http://parrot.cnri.reston.va.us/mailman/listinfo HTTP_HOST: parrot.cnri.reston.va.us SERVER_NAME: www.python.org I want to display the lists that belong to the effective host being visited (the virtual host?), so i need the info contained in HTTP_HOST. ken From scott@chronis.pobox.com Wed May 13 00:52:55 1998 From: scott@chronis.pobox.com (Scott) Date: Tue, 12 May 1998 19:52:55 -0400 Subject: [Mailman-Developers] confirmation patch Message-ID: <19980512195255.13311@chronis.icgroup.com> here is a patch that implements email confirmations. as was discussed previously on this list, all subscriptions go through the confirmation process. in addition, the subscribe mail command has changed to allow subsciptions like this: subscribe [password] [digest|nodigest] [address=] this will allow people to subscribe addresses other than the one they are sending from. Finally, regarding whether or not someone can be subscribed by just replying to the confirmation message, they can. they can also mail the command confirm this patch is against mailman 1.0b3. it does not address the issues regarding a -confirm address and bounced confirmation requests that were discussed today on this list. this patch consists of 2 parts: first, the patch, and second, a file called mm_pending.py to go in the mailman/modules directory. scott (coming soon: admin cgi to use cookie confirmation and way enhanced membership management page) --------------------------------file mm_pending.py---------------- """ module for handling pending subscriptions """ import os import sys import posixfile import marshal import time import whrandom import mm_cfg DB_PATH = mm_cfg.MAILMAN_DIR + "/misc/pending_subscriptions.db" LOCK_PATH = mm_cfg.LOCK_DIR + "/pending_subscriptions.lock" VERIFY_FMT = """\ %(listname)s -- confirmation of subscription -- request %(cookie)s You or someone (%(requestor)s) has requested that your email address (%(email)s) be subscribed to the %(listname)s mailling list at %(listaddress)s. If you wish to fulfill this request, please simply reply to this message, or mail %(request_addr)s with the following line, and only the following line in the message body: confirm %(cookie)s If you do not wish to subscribe to this list, please simply ignore or delete this message. """ # ' icky emacs font lock thing def get_pending(): " returns a dict containing pending information" try: fp = open(DB_PATH,"r" ) except IOError: return {} dict = marshal.load(fp) return dict def gencookie(p=None): if p is None: p = get_pending() while 1: newcookie = int(whrandom.random() * 1000000) if p.has_key(newcookie) or newcookie < 100000: continue return newcookie def set_pending(p): ou = os.umask(0) try: lock_file = posixfile.open(LOCK_PATH,'a+') finally: os.umask(ou) lock_file.lock('w|', 1) fp = open(DB_PATH, "w") marshal.dump(p, fp) fp.close() lock_file.lock("u") lock_file.close() def add2pending(email_addr, password, digest, cookie): ts = int(time.time()) processed = 0 p = get_pending() p[cookie] = (email_addr, password, digest, ts) set_pending(p) ----------------------------patches----------------------------- Index: cgi/subscribe =================================================================== RCS file: /usr/local/cvsroot/mailman/cgi/subscribe,v retrieving revision 1.2 diff -r1.2 subscribe 12c12 < import mm_utils, maillist, mm_err, mm_message, mm_cfg, htmlformat --- > import mm_utils, maillist, mm_err, mm_message, mm_cfg, mm_pending, htmlformat 20,38d19 < NEED_CONFIRM_NOTICE = """ < A request for subscription of your address to the %s@%s < mailing list has been received via the web%s < < This is a confirmation request, to prevent anyone from subscribing < you against your wishes. In order to complete this subscription you < must send a confirming email, to %s, by < replying to this mail, and including just the line: < < subscribe %s%s < < in the body or as the subject line. If you do not actually wish to < subscribe you need not do anything. Upon subscribing you will receive < a message welcoming you to the list and describing how to tailor your < account. < < Questions or comments? < Send them to """ + mm_cfg.MAILMAN_OWNER < 146d126 < 148,165c128,137 < list.AddMember(email, pw, digest, web_subscribe=1) < results = results + ("You have successfully been added. " < "You should receive confirmation by " < "e-mail within an hour. If you do not " < "receive confirmation, then the email " < "address you gave probably bounced, " < "in which case you should try again.

") < except mm_err.MMWebSubscribeRequiresConfirmation: < results = results + ("Confirmation from your email address is " < "required, to prevent anyone from covertly " < "subscribing you. Instructions are being " < "sent to you at %s." % email) < if os.environ.has_key('REMOTE_HOST'): < remote = ", from\n%s. " % os.environ['REMOTE_HOST'] < elif os.environ.has_key('REMOTE_ADDR'): < remote = ", from\n%s." % os.environ['REMOTE_ADDR'] < else: < remote = "." --- > results = results + ("Confirmation from your email address is " > "required, to prevent anyone from covertly " > "subscribing you. Instructions are being " > "sent to you at %s." % email) > if os.environ.has_key('REMOTE_HOST'): > remote = os.environ['REMOTE_HOST'] > elif os.environ.has_key('REMOTE_ADDR'): > remote = os.environ['REMOTE_ADDR'] > else: > remote = "." 170,179c142,154 < list.SendTextToUser(subject = 'Subscribing to %s' % list.real_name, < recipient = email, < sender = list.GetAdminEmail(), < text = (NEED_CONFIRM_NOTICE < % (list.real_name, < list.host_name, < remote, < list.GetRequestEmail(), < pw, < digesting)), --- > cookie = mm_pending.gencookie() > mm_pending.add2pending(email, pw, digest, cookie) > list.SendTextToUser(subject = "%s -- confirmation of subscription -- request %d" % \ > (list.real_name, cookie), > recipient = email, > sender = list.GetRequestEmail(), > text = mm_pending.VERIFY_FMT % ({"email": email, > "listaddress": list.GetListEmail(), > "listname": list.real_name, > "cookie": cookie, > "requestor": remote, > "request_addr": list.GetRequestEmail()}), > 183c158 < % list.GetAdminEmail()]) --- > % list.GetAdminEmail()]) 191,197c166,176 < except mm_err.MMNeedApproval, x: < results = results + ("Subscription was deferred " < "because:
%s

Your request must " < "be approved by the list admin. " < "You will receive email informing you " < "of the moderator's descision when they " < "get to your request.

" % x) --- > # > # deprecating this, it might be useful if we decide to > # allow approved based subscriptions without confirmation > # > ## except mm_err.MMNeedApproval, x: > ## results = results + ("Subscription was deferred " > ## "because:
%s

Your request must " > ## "be approved by the list admin. " > ## "You will receive email informing you " > ## "of the moderator's descision when they " > ## "get to your request.

" % x) 206a186,192 > > > > > > > Index: modules/maillist.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/maillist.py,v retrieving revision 1.2 retrieving revision 1.3 diff -r1.2 -r1.3 142,143d141 < self.web_subscribe_requires_confirmation = \ < mm_cfg.DEFAULT_WEB_SUBSCRIBE_REQUIRES_CONFIRMATION 312,322d309 < < ('web_subscribe_requires_confirmation', mm_cfg.Radio, < ('None', 'Requestor confirms via email', 'Admin approves'), 0, < 'What confirmation is required for on-the-web subscribes?', < < "This option determines whether web-initiated subscribes" < " require further confirmation, either from the subscribed" < " address or from the list administrator. Absence of" < " any confirmation makes web-based subscription a" < " tempting opportunity for mischievous subscriptions by third" < " parties."), Index: modules/mm_defaults.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/mm_defaults.py,v retrieving revision 1.2 retrieving revision 1.3 diff -r1.2 -r1.3 93,94d92 < # 1 for email subscription verification, 2 for admin confirmation: < DEFAULT_WEB_SUBSCRIBE_REQUIRES_CONFIRMATION = 1 Index: modules/mm_mailcmd.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/mm_mailcmd.py,v retrieving revision 1.2 diff -r1.2 mm_mailcmd.py 8,9c8,9 < import string, os, sys < import mm_message, mm_err, mm_cfg, mm_utils --- > import string, os, sys, re > import mm_message, mm_err, mm_cfg, mm_utils, mm_pending 41a42 > 'confirm': self.ProcessConfirmCmd, 50a52 > self.__NoMailCmdResponse = 0 84a87,94 > # > # check to see if confirmation request -- special handling > # > conf_pat = r"%s -- confirmation of subscription -- request (\d\d\d\d\d\d)" % \ > self.real_name > match = re.search(conf_pat, mail.body) > if match: > lines = ["confirm %s" % (match.group(1))] 103c113,114 < self.SendMailCmdResponse(mail) --- > if not self.__NoMailCmdResponse: > self.SendMailCmdResponse(mail) 358a370,372 > password = "" > address = "" > done_digest = 0 362,369c376,396 < elif len(args) == 1: < if string.lower(args[0]) == 'digest': < digest = 1 < password = "%s%s" % (mm_utils.GetRandomSeed(), < mm_utils.GetRandomSeed()) < elif string.lower(args[0]) == 'nodigest': < digest = 0 < password = "%s%s" % (mm_utils.GetRandomSeed(), --- > elif len(args) > 3: > self.AddError("Usage: subscribe [password] [digest|nodigest] [address=]") > return > else: > for arg in args: > if string.lower(arg) == 'digest' and not done_digest: > digest = 1 > done_digest = 1 > elif string.lower(arg) == 'nodigest' and not done_digest: > digest = 0 > done_digest = 1 > elif string.lower(arg)[:8] == 'address=' and not address: > address = string.lower(arg)[8:] > elif not password: > password = arg > else: > self.AddError("Usage: subscribe [password] " > "[digest|nodigest] [address=]") > return > if not password: > password = "%s%s" % (mm_utils.GetRandomSeed(), 371,372c398,415 < else: < password = args[0] --- > if not address: > pending_addr = mail.GetSender() > else: > pending_addr = address > cookie = mm_pending.gencookie() > mm_pending.add2pending(pending_addr, password, digest, cookie) > self.SendTextToUser(subject = "%s -- confirmation of subscription -- request %d" % \ > (self.real_name, cookie), > recipient = pending_addr, > sender = self.GetRequestEmail(), > text = mm_pending.VERIFY_FMT % ({"email": pending_addr, > "listaddress": self.GetListEmail(), > "listname": self.real_name, > "cookie": cookie, > "requestor": mail.GetSender(), > "request_addr": self.GetRequestEmail()})) > self.__NoMailCmdResponse = 1 > return 374,392d416 < elif len(args) == 2: < if string.lower(args[1]) == 'nodigest': < digest = 0 < password = args[0] < elif string.lower(args[1]) == 'digest': < digest = 1 < password = args[0] < elif string.lower(args[0]) == 'nodigest': < digest = 0 < password = args[1] < elif string.lower(args[0]) == 'digest': < digest = 1 < password = args[1] < else: < self.AddError("Usage: subscribe [password] [digest|nodigest]") < return < elif len(args) > 2: < self.AddError("Usage: subscribe [password] [digest|nodigest]") < return 393a418 > def FinishSubscribe(self, addr, password, digest): 395c420 < self.AddMember(mail.GetSender(), password, digest) --- > self.AddMember(addr, password, digest) 399c424 < mail.GetSender()) --- > addr) 410c435 < "(insecure address)" % mail.GetSender()) --- > "(insecure address)" % addr) 412c437 < self.AddError("%s is already a list member." % mail.GetSender()) --- > self.AddError("%s is already a list member." % addr) 418a444,470 > > > > def ProcessConfirmCmd(self, args, cmd, mail): > if len(args) != 1: > self.AddError("Usage: confirm \n") > return > try: > cookie = string.atoi(args[0]) > except: > self.AddError("Usage: confirm \n") > return > pending = mm_pending.get_pending() > if not pending.has_key(cookie): > self.AddError("Invalid confirmation number!\n" > "Please recheck the confirmation number and try again.") > return > (email_addr, password, digest, ts) = pending[cookie] > if self.open_subscribe: > self.ApprovedAddMember(email_addr, password, digest) > self.AddToResponse("Succeeded") > else: > self.AddRequest('add_member', digest, email_addr, password) > del pending[cookie] > mm_pending.set_pending(pending) > > 464c516 < subscribe [password] [digest-option] --- > subscribe [password] [digest-option] [address=

] 469,470c521,523 < To subscribe this way, you must subscribe from the account in < which you wish to receive mail. --- > If you wish to subscribe an address other than the address you send > this request from, you may specify "address=" (no brackets > around the email address, no quotes!) From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed May 13 04:32:00 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 12 May 1998 23:32:00 -0400 (EDT) Subject: [Mailman-Developers] CGI variables question - hostname used in URL References: <13656.54885.16779.208982@anthem.CNRI.Reston.Va.US> Message-ID: <13657.5148.16956.722331@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> That's the official name of the server - not the name by which KM> the page was visited. Darn. From scott@chronis.pobox.com Sat May 16 20:32:10 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 16 May 1998 15:32:10 -0400 Subject: [Mailman-Developers] Finally! admin cgi patches Message-ID: <19980516153210.19958@chronis.icgroup.com> I've finally managed to put together some patches for the admin cgi. Changes included are: * single login for access to list-specific administrative pages, as well as for access to making all changes, except changing the administrator password. this uses cookie authentication with a configurable lifetime (changeable in mm_defaults.ADMIN_COOKIE_LIFE). In order to get this to function, I had to change maillist.MailList.GetScriptURL to allways return full, absolute urls instead of relative ones. * membership management section for mass subscription now does error checking for the addresses entered and reports what addresses were successfully subscribed and which ones weren't and why. * membership management section has an added section where list members are shown in a table with checkboxes available for setting their options or unsubscribing them. There is a related configuration variable in mm_defaults.ADMIN_MEMBER_CHUNKSIZE which will cause lists with more members than that value to only display the first of that many members (sorted) and to display an index for accessing the remaining chunks of members. this patch should be applied against mailman-1.0b3 with the patches for subscription confirmation applied. it probably would work without the confirmation patches, but i haven't tried it and won't recommend it. scott Index: cgi/admin =================================================================== RCS file: /usr/local/cvsroot/mailman/cgi/admin,v retrieving revision 1.2 diff -r1.2 admin 12,13c12,15 < import os, cgi, string, crypt, types < import mm_utils, maillist, mm_cfg, mm_err --- > sys.path.append('.') > import os, cgi, string, crypt, types, time > import mm_utils, maillist, mm_cfg, mm_err, mm_mailcmd > import Cookie 30a33,82 > LOGIN_PAGE = """ > > > %(listname)s Administrative Authentication > > >
> %(message)s > > > > > > > > > > > >
> %(listname)s Administrative > Authentication >
List Administrative Password:
>
>
> """ > > # " <- icky emacs font-lock bug workaround > > def isAuthenticated(list, password=None): > if password is not None: # explicit login > try: > list.ConfirmAdminPassword(password) > except mm_err.MMBadPasswordError: > AddErrorMessage(doc, 'Error: Incorrect admin password.') > return 0 > c = Cookie.Cookie() > c[list_name] = list.password # its crypted so this should be ok > c[list_name]['expires'] = mm_cfg.ADMIN_COOKIE_LIFE > c[list_name]["path"] = "/mailman/" + list.GetScriptURL("admin") > print c # Output the cookie > > return 1 > if os.environ.has_key('HTTP_COOKIE'): > c = Cookie.Cookie( os.environ['HTTP_COOKIE'] ) > if c.has_key(list_name): > return (c[list_name].value == list.password) > return 0 > > 36c88 < global list_name, list_info --- > global list_name, list_info, doc 48,50c100,101 < < list_name = string.lower(list_info[0]) < --- > else: > list_name = string.lower(list_info[0]) 52d102 < 65c115 < --- > 68c118 < --- > global cgi_data 70,73c120,138 < if len(cgi_data.keys()): < if cgi_data.has_key('VARHELP'): < FormatOptionHelp(doc, cgi_data['VARHELP'].value, lst) < print doc.Format(bgcolor="#ffffff") --- > is_auth = 0 > if cgi_data.has_key("adminpw"): > is_auth = isAuthenticated(lst, cgi_data["adminpw"].value) > message = FontAttr("Sorry, wrong password. Try again.", > color="ff5060", size="+1").Format() > else: > is_auth = isAuthenticated(lst) > message = "" > if not is_auth: > print "Content-type: text/html\n\n" > print LOGIN_PAGE % ({"listname": list_name, > "path": os.environ.get("REQUEST_URI", "/mailman/admin"), > "message": message}) > return > > if len(cgi_data.keys()): > if cgi_data.has_key('VARHELP'): > FormatOptionHelp(doc, cgi_data['VARHELP'].value, lst) > print doc.Format(bgcolor="#ffffff") 75,79c140 < if not cgi_data.has_key('adminpw'): < AddErrorMessage(doc, < 'Error: You must supply the admin password to' < ' change options.') < else: --- > if (cgi_data.has_key('bounce_matching_headers')): 81,87c142,147 < lst.ConfirmAdminPassword(cgi_data['adminpw'].value) < ChangeOptions(lst, category, cgi_data, doc) < # Yuck. This shouldn't need to be here. < if not lst.digestable and not lst.nondigestable: < lst.nondigestable = 1 < except mm_err.MMBadPasswordError: < AddErrorMessage(doc, 'Error: Incorrect admin password.') --- > pairs = lst.parse_matching_header_opt() > except mm_err.MMBadConfigError, line: > AddErrorMessage(doc, > 'Warning: bad matching-header line' > ' (does it have the colon?)
    %s
', > line) 100,110c160,161 < < if len(cgi_data.keys()): < if (cgi_data.has_key('bounce_matching_headers')): < try: < pairs = lst.parse_matching_header_opt() < except mm_err.MMBadConfigError, line: < AddErrorMessage(doc, < 'Warning: bad matching-header line' < ' (does it have the colon?)
    %s
', < line) < --- > if len(cgi_data.keys()): > ChangeOptions(lst, category, cgi_data, doc) 220d270 < url = lst.GetScriptURL('admin') 225c275,276 < these_links.AddItem(Link(os.path.join(url, k), v)) --- > these_links.AddItem(Link("%s/%s" % (lst.GetScriptURL('admin'),k), > v)) 234c285,286 < form = Form(os.path.join(lst.GetScriptURL('admin'), category)) --- > form = Form("%s/%s" % (lst.GetScriptURL('admin'), > category_suffix)) 425,427d476 < header.AddRow([Bold("Subscribe and Unsubscribe Members")]) < header.AddCellInfo(max(header.GetCurrentRowIndex(), 0), 0, < colspan=2, bgcolor ="#FFF0D0") 429,444c478,540 < < container.AddItem('

Mass Subscribe Members

') < container.AddItem('

Enter one email address per line

') < container.AddItem(TextArea(name='subscribees', rows=20,cols=60,wrap=None)) < < container.AddItem('

To Unsubscribe Members...

') < container.AddItem(""" < To unsubscribe members you must use your admin password in place of the < user's password on the user's edit-options page. Visit their < edit-options page (via the roster page) and do the < unsubscribe procedure, providing the admin password instead of the < user's password. <

(Note that you can, alternately, set the subscriber's no-delivery < option to inhibit delivery of their messages, if you want to only < temporarily disable their delivery.)

""" < % lst.GetScriptURL('roster')) --- > user_table = Table(width="90%") > user_table.AddRow([Center(Header(4, "Membership List"))]) > user_table.AddCellInfo(user_table.GetCurrentRowIndex(), > user_table.GetCurrentCellIndex(), > bgcolor="#cccccc", colspan=8) > > members = {} > digests = {} > for member in lst.members: > members[member] = 1 > for member in lst.digest_members: > digests[member] = 1 > all = lst.members + lst.digest_members > if len(all) > mm_cfg.ADMIN_MEMBER_CHUNKSIZE: > chunks = mm_utils.chunkify(all) > if not cgi_data.has_key("chunk"): > chunk = 0 > else: > chunk = string.atoi(cgi_data["chunk"].value) > all = chunks[chunk] > footer = ("

To View other sections, " > "click on the appropriate range listed below") > chunk_indices = range(len(chunks)) > chunk_indices.remove(chunk) > buttons = [] > pi = os.environ["PATH_INFO"] > for ci in chunk_indices: > start, end = chunks[ci][0], chunks[ci][-1] > buttons.append(" from %s to %s " % \ > ( pi, ci, start, end)) > buttons = apply(UnorderedList, tuple(buttons)) > footer = footer + buttons.Format() + "

" > else: > all.sort() > footer = "

" > for member in all: > cells = [member + "" % (member), > "subscribed " +CheckBox(member + "_subscribed", "on", 1).Format(), > ] > if members.get(member): > cells.append("digest " + CheckBox(member + "_digest", "off", 0).Format()) > else: > cells.append("digest " + CheckBox(member + "_digest", "on", 1).Format()) > for opt in ("hide", "nomail", "ack", "norcv", "plain"): > if lst.GetUserOption(member, mm_mailcmd.option_info[opt]): > value = "on" > checked = 1 > else: > value = "off" > checked = 0 > box = CheckBox("%s_%s" % (member, opt), value, checked) > cells.append("%s %s" % (opt, box.Format())) > user_table.AddRow(cells) > container.AddItem(Center(user_table)) > container.AddItem(footer) > t = Table(width="90%") > t.AddRow([Center(Header(4, "Mass Subscribe Members"))]) > t.AddCellInfo(t.GetCurrentRowIndex(), > t.GetCurrentCellIndex(), > bgcolor="#cccccc", colspan=8) > container.AddItem(Center(t)) > container.AddItem(Center(TextArea(name='subscribees', rows=10,cols=60,wrap=None))) > container.AddItem(Center(" Enter One address per line

")) 448,471c544,561 < password_submit = Table(bgcolor="#99cccc", < border=0, cellspacing=0, cellpadding=2) < password_submit.AddRow([Center(Bold('To Submit Your Changes'))]) < password_submit.AddCellInfo(password_submit.GetCurrentRowIndex(), 0, < colspan=2) < password_submit.AddRow(['

Enter the administrator ' < 'password:
', < PasswordBox('adminpw')]) < password_submit.AddRow(['
And...
', < Bold(SubmitButton('submit', 'Submit Changes'))]) < change_pw_table = Table(bgcolor="#cccccc", border=0, < cellspacing=0, cellpadding=2) < change_pw_table.AddRow([Bold(Center('To Change The Administrator' < ' Password'))]) < change_pw_table.AddCellInfo(change_pw_table.GetCurrentRowIndex(), 0, < colspan=2) < change_pw_table.AddRow(['
Enter the new ' < 'password:
', < PasswordBox('newpw')]) < change_pw_table.AddRow(['
And also confirm it:
', < PasswordBox('confirmpw')]) < < password_stuff = Table(bgcolor="#99cccc") < password_stuff.AddRow([password_submit, change_pw_table]) --- > submit = Table(bgcolor="#99ccff", > border=0, cellspacing=0, cellpadding=2, width="100%") > submit.AddRow([Bold(SubmitButton('submit', 'Submit Your Changes'))]) > submit.AddCellInfo(submit.GetCurrentRowIndex(), 0, align="middle") > change_pw_table = Table(bgcolor="#99cccc", border=0, > cellspacing=0, cellpadding=2, width="90%") > change_pw_table.AddRow([Bold(Center('To Change The Administrator Password')), > '
Enter the new password:
', > PasswordBox('newpw'),]) > change_pw_table.AddCellInfo(0, 0, align="middle", colspan=2) > change_pw_table.AddRow(['
Enter the current password
', > PasswordBox('adminpw'), > '
Again to confirm it:
', > PasswordBox('confirmpw')]) > password_stuff = Container() > password_stuff.AddItem(change_pw_table) > password_stuff.AddItem("

") > password_stuff.AddItem(submit) 555c645,676 < if category != 'members': --- > confirmed = 0 > if cgi_info.has_key('newpw'): > if cgi_info.has_key('confirmpw'): > if cgi_info.has_key('adminpw'): > try: > lst.ConfirmAdminPassword(cgi_info['adminpw'].value) > confirmed = 1 > except mm_err.MMBadPasswordError: > m = "Error: incorrect administrator password" > document.AddItem(Header(3, Italic(FontAttr(m, color="ff5060")))) > confirmed = 0 > if confirmed: > new = cgi_info['newpw'].value > confirm = cgi_info['confirmpw'].value > if new == confirm: > lst.password = crypt.crypt(new, mm_utils.GetRandomSeed()) > dirty = 1 > else: > m = 'Error: Passwords did not match.' > document.AddItem(Header(3, Italic(FontAttr(m, color="ff5060")))) > > else: > m = 'Error: You must type in your new password twice.' > document.AddItem( > Header(3, Italic(FontAttr(m, color="ff5060")))) > # > # for some reason, the login page mangles important values for the list > # such as .real_name so we only process these changes if the category > # is not "members" and the request is not from the login page > # -scott 19980515 > # > if category != 'members' and not cgi_info.has_key("request_login"): 572a694 > print "property: ", property, "value: ", value 574a697,699 > # > # mass subscription processing for members category > # 577,578c702,713 < names = string.split(name_text, '\r\n') < for new_name in names: --- > name_text = string.replace(name_text, '\r', '') > names = string.split(name_text, '\n') > if '' in names: > names.remove('') > subscribe_success = [] > subscribe_errors = [] > for new_name in map(string.strip,names): > digest = 0 > if not lst.digestable: > digest = 0 > if not lst.nondigestable: > digest = 1 580,593c715,747 < #FIXME: The admin needs to be able to specify subscribe options < lst.AddMember(new_name, (mm_utils.GetRandomSeed() + < mm_utils.GetRandomSeed())) < dirty = 1 < #FIXME: Give some sort of an indication of which names didn't work, < # and why they didn't work... < except: < pass < if cgi_info.has_key('newpw'): < if cgi_info.has_key('confirmpw'): < new = cgi_info['newpw'].value < confirm = cgi_info['confirmpw'].value < if new == confirm: < lst.password = crypt.crypt(new, mm_utils.GetRandomSeed()) --- > lst.ApprovedAddMember(new_name, (mm_utils.GetRandomSeed() + > mm_utils.GetRandomSeed()), digest) > subscribe_success.append(new_name) > except mm_err.MMAlreadyAMember: > subscribe_errors.append((new_name, 'Already a member')) > > except mm_err.MMBadEmailError: > subscribe_errors.append((new_name, "Bad/Invalid email address")) > except mm_err.MMHostileAddress: > subscribe_errors.append((new_name, "Hostile Address (illegal characters)")) > if subscribe_success: > document.AddItem(Header(5, "Successfully Subscribed:")) > document.AddItem(apply(UnorderedList, tuple((subscribe_success)))) > document.AddItem("

") > if subscribe_errors: > document.AddItem(Header(5, "Error Subscribing:")) > items = map(lambda x: "%s -- %s" % (x[0], x[1]), subscribe_errors) > document.AddItem(apply(UnorderedList, tuple((items)))) > document.AddItem("

") > # > # do the user options for members category > # > if cgi_info.has_key('user'): > user = cgi_info["user"] > if type(user) is type([]): > users = [] > for ui in range(len(user)): > users.append(user[ui].value) > else: > users = [user.value] > for user in users: > if not cgi_info.has_key('%s_subscribed' % (user)): > lst.DeleteMember(user) 595,598c749,771 < else: < m = 'Error: Passwords did not match.' < document.AddItem( < Header(3, Italic(FontAttr(m, color="ff5060")))) --- > continue > if not cgi_info.has_key("%s_digest" % (user)): > if user in lst.digest_members: > list.digest_members.remove(user) > dirty = 1 > if user not in lst.members: > lst.members.append(user) > dirty = 1 > else: > if user not in lst.digest_members: > lst.digest_members.append(user) > dirty = 1 > if user in lst.members: > lst.members.remove(user) > dirty = 1 > > for opt in ("hide", "nomail", "ack", "norcv", "plain"): > if cgi_info.has_key("%s_%s" % (user, opt)): > lst.SetUserOption(user, mm_mailcmd.option_info[opt], 1) > dirty = 1 > else: > lst.SetUserOption(user, mm_mailcmd.option_info[opt], 0) > dirty = 1 600,603d772 < else: < m = 'Error: You must type in your new password twice.' < document.AddItem( < Header(3, Italic(FontAttr(m, color="ff5060")))) 622a792 > Index: cgi/private =================================================================== RCS file: /usr/local/cvsroot/mailman/cgi/private,v retrieving revision 1.1.1.1 diff -r1.1.1.1 private 62a63 > # " 68,75c69,76 < (?: / (?: \d{4} q \d\. )? # Match "/", and, optionally, 1998q1." < ( [^/]* ) /? # The SIG name < /[^/]*$ # The trailing 12345.html portion < ) | (?: < / ( [^/.]* ) # Match matrix-sig < (?:\.html)? # Optionally match .html < /? # Optionally match a trailing slash < $ # Must match to end of string --- > (?: / (?: \d{4} q \d\. )? # Match "/", and, optionally, 1998q1. > ( [^/]* ) /? # The SIG name > /[^/]*$ # The trailing 12345.html portion > ) | (?: > / ( [^/.]* ) # Match matrix-sig > (?:\.html)? # Optionally match .html > /? # Optionally match a trailing slash > $ # Must match to end of string 87,93d87 < #for i in ['/matrix-sig.html', '/1998q1.c++-sig/index.html', < # '/1998q1.string-sig/foobar.html', < # '/psa-members.html']: < # print i, `getListName(i)` < #sys.exit(0) < < ## sys.exit(0) Index: modules/htmlformat.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/htmlformat.py,v retrieving revision 1.1.1.1 diff -r1.1.1.1 htmlformat.py 391a392,396 > class CheckBox(InputObj): > def __init__(self, name, value, checked=0, **kws): > apply(InputObj.__init__, (self, name, "CHECKBOX", value, checked), kws) > > 446a452 > Index: modules/maillist.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/maillist.py,v retrieving revision 1.3 diff -r1.3 maillist.py 60,61c60,65 < return os.path.join(self.web_page_url, '%s/%s' % < (script_name, self._internal_name)) --- > if self.web_page_url: > prefix = self.web_page_url > else: > prefix = mm_cfg.DEFAULT_URL > return os.path.join(prefix, '%s/%s' % (script_name, > self._internal_name)) 566a571,572 > if not mm_utils.ValidEmail(name): > raise mm_err.MMBadEmailError Index: modules/mm_defaults.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/mm_defaults.py,v retrieving revision 1.3 diff -r1.3 mm_defaults.py 156a157,168 > > # > # how long the cookie authorizing administrative > # changes via the admin cgi lasts > # > ADMIN_COOKIE_LIFE = 60 * 20 # 20 minutes > > # > # how many members to display at a time > # on the admin cgi to unsubscribe them or change their options > # > ADMIN_MEMBER_CHUNKSIZE = 10 Index: modules/mm_utils.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/mm_utils.py,v retrieving revision 1.2 diff -r1.2 mm_utils.py 390a391,404 > > def chunkify(members, chunksize=mm_cfg.ADMIN_MEMBER_CHUNKSIZE): > """ > return a list of lists of members > """ > members.sort() > res = [] > while 1: > if not members: > break > chunk = members[:chunksize] > res.append(chunk) > members = members[chunksize:] > return res From scott@chronis.pobox.com Sat May 16 20:49:46 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 16 May 1998 15:49:46 -0400 Subject: [Mailman-Developers] Finally! admin cgi patches In-Reply-To: <19980516153210.19958@chronis.icgroup.com>; from Scott on Sat, May 16, 1998 at 03:32:10PM -0400 References: <19980516153210.19958@chronis.icgroup.com> Message-ID: <19980516154946.30972@chronis.icgroup.com> On Sat, May 16, 1998 at 03:32:10PM -0400, Scott wrote: | * membership management section for mass subscription now does | error checking for the addresses entered and reports what addresses | were successfully subscribed and which ones weren't and why. I just took a look at the TODO list at www.list.org and thought i should add a couple of quick notes. the mass subscription section skips the confirmation and admin approval steps, but still checks for valid email addresses and whether the member has already been subscribed. | | * membership management section has an added section where list | members are shown in a table with checkboxes available for setting | their options or unsubscribing them. There is a related | configuration variable in mm_defaults.ADMIN_MEMBER_CHUNKSIZE which | will cause lists with more members than that value to only display | the first of that many members (sorted) and to display an index for | accessing the remaining chunks of members. this section includes all members, hidden or not. scott From janne@avocado.pc.helsinki.fi Mon May 18 13:14:35 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 18 May 1998 15:14:35 +0300 Subject: [Mailman-Developers] Bouncing administrative mail Message-ID: It seems that in 1.0b1.2 there's a serious problem with bouncing administrative mail. I hope this is corrected in later versions and apologize if I'm reporting an old problem. We have a big list here with the administrator now outside of the local site (local \equiv where mailman is run). Last weekend the network connection to outside world was down. Administrative mail for the big list could not be delivered. As a result, some error messages where generated about pending mail ("undelivered for 4 hours" etc.), but apparently an Errors-To or some other header of the administrative messages caused the warnings to be delivered to the list-admin. These, of course, generated even more error messages about pending mail. As a result, I had 22000 messages in my mail queue this morning. -- Janne From scott@chronis.pobox.com Tue May 19 22:25:42 1998 From: scott@chronis.pobox.com (Scott) Date: Tue, 19 May 1998 17:25:42 -0400 Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: ; from Janne Sinkkonen on Mon, May 18, 1998 at 03:14:35PM +0300 References: Message-ID: <19980519172542.38756@chronis.icgroup.com> would it suffice to set the bounce options to "disable and don't notify me" when the list administrator address is not local? scott On Mon, May 18, 1998 at 03:14:35PM +0300, Janne Sinkkonen wrote: | | It seems that in 1.0b1.2 there's a serious problem with bouncing | administrative mail. I hope this is corrected in later versions and | apologize if I'm reporting an old problem. | | We have a big list here with the administrator now outside of the | local site (local \equiv where mailman is run). | | Last weekend the network connection to outside world was | down. Administrative mail for the big list could not be delivered. As | a result, some error messages where generated about pending mail | ("undelivered for 4 hours" etc.), but apparently an Errors-To or some | other header of the administrative messages caused the warnings to be | delivered to the list-admin. These, of course, generated even more error | messages about pending mail. | | As a result, I had 22000 messages in my mail queue this morning. | | -- | Janne | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers From janne@avocado.pc.helsinki.fi Wed May 20 09:40:07 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 20 May 1998 11:40:07 +0300 Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: Scott's message of "Tue, 19 May 1998 17:25:42 -0400" References: <19980519172542.38756@chronis.icgroup.com> Message-ID: Scott writes: > would it suffice to set the bounce options to "disable and don't > notify me" when the list administrator address is not local? > > scott As a temporary remedy, maybe yes (haven't tried it, because things are running smoothly again). But in the long run, of course, it would be nice to have the same functionality for remote administrators as for the local ones. :) -- Janne From viega@list.org Wed May 20 16:51:52 1998 From: viega@list.org (John Viega) Date: Wed, 20 May 1998 08:51:52 -0700 Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: ; from Janne Sinkkonen on Wed, May 20, 1998 at 11:40:07AM +0300 References: <19980519172542.38756@chronis.icgroup.com> Message-ID: <19980520085152.A29157@list.org> I would say that what needs to be done is that the same sort of loop detection needs to be performed on these mails as is done on regular list mail w/ the X-BeenThere header. It could also be done by adding an additional administrative address... John On Wed, May 20, 1998 at 11:40:07AM +0300, Janne Sinkkonen wrote: > Scott writes: > > > would it suffice to set the bounce options to "disable and don't > > notify me" when the list administrator address is not local? > > > > scott > > As a temporary remedy, maybe yes (haven't tried it, because things are > running smoothly again). But in the long run, of course, it would be > nice to have the same functionality for remote administrators as for > the local ones. :) > > -- > Janne > > _______________________________________________ > Mailman-Developers maillist - Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From klm@python.org Wed May 20 17:27:07 1998 From: klm@python.org (Ken Manheimer) Date: Wed, 20 May 1998 12:27:07 -0400 (EDT) Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: <19980520085152.A29157@list.org> Message-ID: On Wed, 20 May 1998, John Viega wrote: > I would say that what needs to be done is that the same sort of loop > detection needs to be performed on these mails as is done on regular > list mail w/ the X-BeenThere header. It could also be done by adding > an additional administrative address... If i'm thinking of the right bounce situation (will have to go back and look at the original problem statement), i think the right answer is to look at the intended recipients of the bounce, and if any are the same as the place where the new bounce notice would go - don't send it! I'll investigate. In addition, do most prominent logging - like in the "error" log, which only otherwise gets actual coding error tracebacks. (The idea being that this is an extraordinary error condition worth bringing to the attention of the mailman site administrator.) And incidentally, have people been checking their logs/error file occasionally? It should mostly be empty, but any tracebacks found there should be investigated. Ken From janne@avocado.pc.helsinki.fi Wed May 20 21:58:11 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 20 May 1998 23:58:11 +0300 Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: John Viega's message of "Wed, 20 May 1998 08:51:52 -0700" References: <19980519172542.38756@chronis.icgroup.com> <19980520085152.A29157@list.org> Message-ID: John Viega writes: > list mail w/ the X-BeenThere header. It could also be done by adding > an additional administrative address... The problem seems to arise from the sender and receiver of the administrative messages being identical. If 'list-admin' cannot be reached, a foreign or local MTA generates an error message and sends it to 'list-admin'. I don't see any fool-proof way to resolve this by including a header for loop detection, because the nondeliverable message is not necessarily included in an error message generated by an MTA. If this is true, then another administrative address is necessarily needed, and it must be always functional when the list-admin@mailman.site address is functional. That means that it must be local. It could be the mailman-owner, or a trashcan. Mailman-owner is a natural alternative, because the person being responsible of mailman certainly wants to know if a list administrator has disappeared. Now that we are on administrative addresses, another remotely related feature I would like to see is separation of the address which gets all the error messages due to nondeliverable list mail from the address which is mentioned as the place to ask help in case of trouble. Sorting out pleas for help from the pile of rubbish generated by MTAs is very tedious. -- Janne From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed May 20 22:02:51 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 20 May 1998 17:02:51 -0400 (EDT) Subject: [Mailman-Developers] configuration option `info' Message-ID: <13667.17517.801967.195815@anthem.CNRI.Reston.Va.US> [Just one for the to-do list... -BAW] The help for the `info' config option says: The text will be treated as html except that newlines newlines will be translated to
- so you can use links, preformatted text, etc, but don't put in carriage returns except where you mean to separate paragraphs. And review your changes - bad html (like some unterminated HTML constructs) can prevent display of the entire listinfo page. Other that the typo (`newlines' twice in the first sentence), it would actually be better if a more texinfo like approach was used. Instead of hard newlines being translated into
, translate *blank* lines (e.g. two newlines) into
. That way the text inside the tiny type-in box could be formatted a little easier. -Barry From klm@python.org Thu May 21 01:36:27 1998 From: klm@python.org (Ken Manheimer) Date: Wed, 20 May 1998 20:36:27 -0400 (EDT) Subject: [Mailman-Developers] Preventing admin-notice bounce loops! Message-ID: <199805210036.UAA07123@glyph.CNRI.Reston.Va.US> I think i've got the right solution to the nasty bounce loop by which janne sinkonnen got around 22,000 messages last weekend. The approach is what i suggested earlier today - check the bouncing address to see if it's among the list administrators, and if so log an error and *don't* send out another notice. I have tested it as far as i was able, and it seems to behave just the way we'd want. This patch also has a few other bounce improvements, one being important, in a slightly different manner: a copy of the offending bounce notification is included with the admin-notice when a user is being disabled or unsubscribed. This enables the admin to assess the correctness of the action - something otherwise impossible, since no other record of the bounce notice is retained. (Ultimately it would be nice to have some kind of cataloguing system for things like subscribe and admin-approval requests, since the entries can be discarded once the administrator has reviewed and decided their fate. For bounce notices, just sending them with the automatic action seems to make the most sense to me...) If you give this a try, please let me know how it goes. I don't think it depends on any other local changes, but would like to have that checked... See the checkin log entries, below, for more details. ken manheimer klm@python.org ---------------------------- revision 1.16 date: 1998/05/21 00:20:00; author: klm; state: Exp; lines: +65 -30 Prevent bounce loops when admin-action recipient is itself the bouncing address - before sending the notice, check whether the bouncing address is among the list owner addresses. And otherwise, include (attach) a copy of the actual bounce notice in the admin-action message. Otherwise there is no record the administrator can examine. Finally, send both failed and successful admin-action notifications to the list administrator. Before, failed action (e.g. inability to locate the specific user) were sent to the mailman site admin - this was while i was debugging the bounce handling, but it looks a lot better now. ---------------------------- revision 1.15 date: 1998/05/20 17:18:19; author: mailman; state: Exp; lines: +10 -10 Neglected to mention on last checkin: ScanMessage(): added final-step massage of all candidates to remove encompassing <, > angle brackets, if any, and check against already processed candidates. ---------------------------- revision 1.14 date: 1998/05/20 17:13:40; author: mailman; state: Exp; lines: +35 -17 Do not send "user disabled" notices to admin when user is already disabled. The idea is that numerous outstanding bounces do not each require renotification of the admin. DisableBouncingAddress(), RemoveBouncingAddress(): changed return signatures to indicate whether email notification is necessary for this action, and in Disable version, indicate not necessary when user is already disabled. In all other cases, notification is indicated, whatever the status. Index: mm_bouncer.py =================================================================== RCS file: /projects/cvsroot/mailman/modules/mm_bouncer.py,v retrieving revision 1.13 retrieving revision 1.16 diff -c -r1.13 -r1.16 *** mm_bouncer.py 1998/04/23 16:26:33 1.13 --- mm_bouncer.py 1998/05/21 00:20:00 1.16 *************** *** 1,6 **** "Handle delivery bounce messages, doing filtering when list is set for it." ! __version__ = "$Revision: 1.13 $" # It's possible to get the mail-list senders address (list-admin) in the # bounce list. You probably don't want to have list mail sent to that --- 1,6 ---- "Handle delivery bounce messages, doing filtering when list is set for it." ! __version__ = "$Revision: 1.16 $" # It's possible to get the mail-list senders address (list-admin) in the # bounce list. You probably don't want to have list mail sent to that *************** *** 53,59 **** if self.bounce_info.has_key(email): del self.bounce_info[email] ! def RegisterBounce(self, email): report = "%s: %s - " % (self.real_name, email) bouncees = self.bounce_info.keys() this_dude = mm_utils.FindMatchingAddresses(email, bouncees) --- 53,59 ---- if self.bounce_info.has_key(email): del self.bounce_info[email] ! def RegisterBounce(self, email, msg): report = "%s: %s - " % (self.real_name, email) bouncees = self.bounce_info.keys() this_dude = mm_utils.FindMatchingAddresses(email, bouncees) *************** *** 84,90 **** self.minimum_post_count_before_bounce_action) and difference > self.minimum_removal_date * 24 * 60 * 60): self.LogMsg("bounce", report + "exceeded limits") ! self.HandleBouncingAddress(addr) return else: post_count = (self.minimum_post_count_before_bounce_action - --- 84,90 ---- self.minimum_post_count_before_bounce_action) and difference > self.minimum_removal_date * 24 * 60 * 60): self.LogMsg("bounce", report + "exceeded limits") ! self.HandleBouncingAddress(addr, msg) return else: post_count = (self.minimum_post_count_before_bounce_action - *************** *** 107,113 **** return if difference > self.minimum_removal_date * 24 * 60 * 60: self.LogMsg("bounce", "exceeded limits (D)") ! self.HandleBouncingAddress(addr) return self.LogMsg("bounce", "digester lucked out") --- 107,113 ---- return if difference > self.minimum_removal_date * 24 * 60 * 60: self.LogMsg("bounce", "exceeded limits (D)") ! self.HandleBouncingAddress(addr, msg) return self.LogMsg("bounce", "digester lucked out") *************** *** 117,164 **** self._internal_name, addr) ! def HandleBouncingAddress(self, addr): """Disable or remove addr according to bounce_action setting.""" if self.automatic_bounce_action == 0: return elif self.automatic_bounce_action == 1: ! succeeded = self.DisableBouncingAddress(addr) did = "disabled" - send = 1 elif self.automatic_bounce_action == 2: ! succeeded = self.DisableBouncingAddress(addr) did = "disabled" send = 0 elif self.automatic_bounce_action == 3: ! succeeded = self.RemoveBouncingAddress(addr) ! did = "removed" send = 1 if send: if succeeded != 1: negative="not " - recipient = mm_cfg.MAILMAN_OWNER else: negative="" ! recipient = self.GetAdminEmail() ! text = ("This is a mailman maillist administrator notice.\n" ! "\n\tMaillist:\t%s\n" ! "\tMember:\t\t%s\n" ! "\tAction:\t\tSubscription %s%s.\n" ! "\tReason:\t\tExcessive or fatal bounces.\n" ! % (self.real_name, addr, negative, did)) if succeeded != 1: ! text = text + "\tBUT:\t\t%s\n\n" % succeeded ! else: ! text = text + "\n" if did == "disabled" and succeeded == 1: ! text = text + ( ! "You can reenable their subscription by visiting " ! "their options page\n" ! "(via %s) and using your\n" ! "list admin password to authorize the option change.\n\n" ! % self.GetScriptURL('listinfo')) ! text = text + ("Questions? Contact the mailman site admin,\n%s" ! % mm_cfg.MAILMAN_OWNER) if negative: negative = string.upper(negative) self.SendTextToUser(subject = ("%s member %s %s%s due to bounces" --- 117,198 ---- self._internal_name, addr) ! def HandleBouncingAddress(self, addr, msg): """Disable or remove addr according to bounce_action setting.""" if self.automatic_bounce_action == 0: return elif self.automatic_bounce_action == 1: ! # Only send if call works ok. ! (succeeded, send) = self.DisableBouncingAddress(addr) did = "disabled" elif self.automatic_bounce_action == 2: ! (succeeded, send) = self.DisableBouncingAddress(addr) did = "disabled" + # Never send. send = 0 elif self.automatic_bounce_action == 3: ! (succeeded, send) = self.RemoveBouncingAddress(addr) ! # Always send. send = 1 + did = "removed" if send: if succeeded != 1: negative="not " else: negative="" ! recipient = self.GetAdminEmail() ! if addr in self.owner + [recipient]: ! # Whoops! This is a bounce of a bounce notice - do not ! # perpetuate the bounce loop! Log it prominently and be ! # satisfied with that. ! self.LogMsg("error", ! "%s: Bounce recipient loop" ! " encountered!\n\t%s\n\tBad admin recipient: %s", ! self._internal_name, ! "(Ie, bounce notification addr, itself, bounces.)", ! addr) ! return ! import mimetools ! boundary = mimetools.choose_boundary() ! text = [""] ! text.append("(This MIME message should be" ! " readable as plain text.)") ! text.append("") ! text.append("--" + boundary) ! text.append("Content-type: text/plain; charset=us-ascii") ! text.append("") ! text.append("This is a mailman mailing list bounce action notice:") ! text.append("") ! text.append("\tMaillist:\t%s" % self.real_name) ! text.append("\tMember:\t\t%s" % addr) ! text.append("\tAction:\t\tSubscription %s%s." % (negative, did)) ! text.append("\tReason:\t\tExcessive or fatal bounces.") if succeeded != 1: ! text.append("\tBUT:\t\t%s\n" % succeeded) ! text.append("") if did == "disabled" and succeeded == 1: ! text.append("You can reenable their subscription by visiting " ! "their options page") ! text.append("(via %s) and using your" ! % self.GetScriptURL('listinfo')) ! text.append( ! "list admin password to authorize the option change.") ! text.append("") ! text.append("The triggering bounce notice is attached below.") ! text.append("") ! text.append("Questions? Contact the mailman site admin,") ! text.append("\t" + mm_cfg.MAILMAN_OWNER) ! ! text.append("") ! text.append("--" + boundary) ! text.append("Content-type: text/plain; charset=us-ascii") ! text.append("") ! text.append(string.join(msg.headers, '')) ! text.append("") ! text.append(mm_utils.QuotePeriods(msg.body)) ! text.append("") ! text.append("--" + boundary + "--") ! if negative: negative = string.upper(negative) self.SendTextToUser(subject = ("%s member %s %s%s due to bounces" *************** *** 166,209 **** negative, did)), recipient = recipient, sender = mm_cfg.MAILMAN_OWNER, ! add_headers = ["Errors-To: %s" ! % mm_cfg.MAILMAN_OWNER], ! text = text) def DisableBouncingAddress(self, addr): if not self.IsMember(addr): reason = "User not found." self.LogMsg("bounce", "%s: NOT disabled %s: %s", self.real_name, addr, reason) ! return reason try: ! self.SetUserOption(addr, mm_cfg.DisableDelivery, 1) ! self.LogMsg("bounce", "%s: disabled %s", self.real_name, addr) ! self.Save() ! return 1 except mm_err.MMNoSuchUserError: self.LogMsg("bounce", "%s: NOT disabled %s: %s", self.real_name, addr, mm_err.MMNoSuchUserError) self.ClearBounceInfo(addr) self.Save() ! return mm_err.MMNoSuchUserError def RemoveBouncingAddress(self, addr): if not self.IsMember(addr): reason = "User not found." self.LogMsg("bounce", "%s: NOT removed %s: %s", self.real_name, addr, reason) ! return reason try: self.DeleteMember(addr, "bouncing addr") self.LogMsg("bounce", "%s: removed %s", self.real_name, addr) self.Save() ! return 1 except mm_err.MMNoSuchUserError: self.LogMsg("bounce", "%s: NOT removed %s: %s", self.real_name, addr, mm_err.MMNoSuchUserError) self.ClearBounceInfo(addr) self.Save() ! return mm_err.MMNoSuchUserError # Return 0 if we couldn't make any sense of it, 1 if we handled it. def ScanMessage(self, msg): --- 200,259 ---- negative, did)), recipient = recipient, sender = mm_cfg.MAILMAN_OWNER, ! add_headers = [ ! "Errors-To: %s" % mm_cfg.MAILMAN_OWNER, ! "MIME-version: 1.0", ! "Content-type: multipart/mixed;" ! ' boundary="%s"' % boundary], ! text = string.join(text, '\n')) def DisableBouncingAddress(self, addr): + """Disable delivery for bouncing user address. + + Returning success and notification status.""" if not self.IsMember(addr): reason = "User not found." self.LogMsg("bounce", "%s: NOT disabled %s: %s", self.real_name, addr, reason) ! return reason, 1 try: ! if self.GetUserOption(addr, mm_cfg.DisableDelivery): ! # No need to send out notification if they're already disabled. ! self.LogMsg("bounce", ! "%s: already disabled %s", self.real_name, addr) ! return 1, 0 ! else: ! self.SetUserOption(addr, mm_cfg.DisableDelivery, 1) ! self.LogMsg("bounce", ! "%s: disabled %s", self.real_name, addr) ! self.Save() ! return 1, 1 except mm_err.MMNoSuchUserError: self.LogMsg("bounce", "%s: NOT disabled %s: %s", self.real_name, addr, mm_err.MMNoSuchUserError) self.ClearBounceInfo(addr) self.Save() ! return mm_err.MMNoSuchUserError, 1 def RemoveBouncingAddress(self, addr): + """Unsubscribe user with bouncing address. + + Returning success and notification status.""" if not self.IsMember(addr): reason = "User not found." self.LogMsg("bounce", "%s: NOT removed %s: %s", self.real_name, addr, reason) ! return reason, 1 try: self.DeleteMember(addr, "bouncing addr") self.LogMsg("bounce", "%s: removed %s", self.real_name, addr) self.Save() ! return 1, 1 except mm_err.MMNoSuchUserError: self.LogMsg("bounce", "%s: NOT removed %s: %s", self.real_name, addr, mm_err.MMNoSuchUserError) self.ClearBounceInfo(addr) self.Save() ! return mm_err.MMNoSuchUserError, 1 # Return 0 if we couldn't make any sense of it, 1 if we handled it. def ScanMessage(self, msg): *************** *** 268,274 **** messy_pattern_6 = regex.compile('^[ \t]*[^ ]+: User unknown.*$') messy_pattern_7 = regex.compile('^[^ ]+ - User currently disabled.*$') ! message_groked = 0 for line in string.split(relevant_text, '\n'): for pattern, action in simple_bounce_pats: --- 318,324 ---- messy_pattern_6 = regex.compile('^[ \t]*[^ ]+: User unknown.*$') messy_pattern_7 = regex.compile('^[^ ]+ - User currently disabled.*$') ! message_grokked = 0 for line in string.split(relevant_text, '\n'): for pattern, action in simple_bounce_pats: *************** *** 276,331 **** email = self.ExtractBouncingAddr(line) if action == REMOVE: candidates = candidates + string.split(email,',') ! message_groked = 1 continue elif action == BOUNCE: emails = string.split(email,',') for email_addr in emails: ! self.RegisterBounce(email_addr) ! message_groked = 1 continue else: ! message_groked = 1 continue # Now for the special case messages that are harder to parse... if (messy_pattern_1.match(line) <> -1 or messy_pattern_2.match(line) <> -1): username = string.split(line)[1] ! self.RegisterBounce('%s@%s' % (username, remote_host)) ! message_groked = 1 continue if (messy_pattern_3.match(line) <> -1 or messy_pattern_4.match(line) <> -1 or messy_pattern_5.match(line) <> -1): username = string.split(line)[1] candidates.append('%s@%s' % (username, remote_host)) ! message_groked = 1 continue if messy_pattern_6.match(line) <> -1: username = string.split(string.strip(line))[0][:-1] candidates.append('%s@%s' % (username, remote_host)) ! message_groked = 1 continue if messy_pattern_7.match(line) <> -1: username = string.split(string.strip(line))[0] candidates.append('%s@%s' % (username, remote_host)) ! message_groked = 1 continue did = [] for i in candidates: el = string.find(i, "...") ! if el != -1: i = i[:el] if i not in did: ! self.HandleBouncingAddress(i) did.append(i) ! return message_groked def ExtractBouncingAddr(self, line): email = regsub.splitx(line, '[^ \t@<>]+@[^ \t@<>]+\.[^ \t<>.]+')[1] if email[0] == '<': - # Remove what's within the angles. return regsub.splitx(email[1:], ">")[0] else: return email --- 326,384 ---- email = self.ExtractBouncingAddr(line) if action == REMOVE: candidates = candidates + string.split(email,',') ! message_grokked = 1 continue elif action == BOUNCE: emails = string.split(email,',') for email_addr in emails: ! self.RegisterBounce(email_addr, msg) ! message_grokked = 1 continue else: ! message_grokked = 1 continue # Now for the special case messages that are harder to parse... if (messy_pattern_1.match(line) <> -1 or messy_pattern_2.match(line) <> -1): username = string.split(line)[1] ! self.RegisterBounce('%s@%s' % (username, remote_host), msg) ! message_grokked = 1 continue if (messy_pattern_3.match(line) <> -1 or messy_pattern_4.match(line) <> -1 or messy_pattern_5.match(line) <> -1): username = string.split(line)[1] candidates.append('%s@%s' % (username, remote_host)) ! message_grokked = 1 continue if messy_pattern_6.match(line) <> -1: username = string.split(string.strip(line))[0][:-1] candidates.append('%s@%s' % (username, remote_host)) ! message_grokked = 1 continue if messy_pattern_7.match(line) <> -1: username = string.split(string.strip(line))[0] candidates.append('%s@%s' % (username, remote_host)) ! message_grokked = 1 continue did = [] for i in candidates: el = string.find(i, "...") ! if el != -1: ! i = i[:el] ! if len(i) > 1 and i[0] == '<': ! # Use stuff after open angle and before (optional) close: ! i = regsub.splitx(i[1:], ">")[0] if i not in did: ! self.HandleBouncingAddress(i, msg) did.append(i) ! return message_grokked def ExtractBouncingAddr(self, line): email = regsub.splitx(line, '[^ \t@<>]+@[^ \t@<>]+\.[^ \t<>.]+')[1] if email[0] == '<': return regsub.splitx(email[1:], ">")[0] else: return email From klm@python.org Thu May 21 18:56:36 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 21 May 1998 13:56:36 -0400 (EDT) Subject: [Mailman-Developers] Bounce action policy - not distinguish "user not found"? Message-ID: <199805211756.NAA09630@glyph.CNRI.Reston.Va.US> Here's a bounce-action policy question, primarily for john, but also i think useful for general discussion and information. Currently, "user not found"/"user unknown" bounces are treated a fatal problems, invoking immediate (disable or unsubscribe, according to list conf) action. Most other random types of bounces are treated as potentially transient, and just added to the members tally. It was suggested to me that it might be better to just handle them all as potentially transient, and only take the action when the tally exceeds the threshhold. Offhand, i don't see the *need* to treat the "user not found" bounces specially, though i can imagine they're a bit less likely to be transient than, eg, "spool dir full" bounces. I'm inclined to think it would be simpler and more sensible to just lump all the bounces together. Thoughts? Ken From janne@avocado.pc.helsinki.fi Thu May 21 23:45:57 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 22 May 1998 01:45:57 +0300 Subject: [Mailman-Developers] Administrators would like to see hidden addresses Message-ID: An administrator would like to see all the subscribed addresses, including the hidden ones. This would be necessary to check whether they match some of the bounce messages which were not handled automatically. -- Janne From scott@chronis.pobox.com Fri May 22 00:01:41 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 21 May 1998 19:01:41 -0400 Subject: [Mailman-Developers] Administrators would like to see hidden addresses In-Reply-To: ; from Janne Sinkkonen on Fri, May 22, 1998 at 01:45:57AM +0300 References: Message-ID: <19980521190141.64768@chronis.icgroup.com> the admin cgi patch i posted a few days ago does this. in addition, administrators can edit the options or unsubscribe members en masse with this patch. scott On Fri, May 22, 1998 at 01:45:57AM +0300, Janne Sinkkonen wrote: | | An administrator would like to see all the subscribed addresses, | including the hidden ones. This would be necessary to check whether | they match some of the bounce messages which were not handled | automatically. | | -- | Janne | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers From janne@avocado.pc.helsinki.fi Fri May 22 00:04:35 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 22 May 1998 02:04:35 +0300 Subject: [Mailman-Developers] Administrators would like to see hidden addresses In-Reply-To: Scott's message of "Thu, 21 May 1998 19:01:41 -0400" References: <19980521190141.64768@chronis.icgroup.com> Message-ID: Scott writes: > the admin cgi patch i posted a few days ago does this. > in addition, administrators can edit the options or unsubscribe > members en masse with this patch. OK, thanks. :) Does anyone happen to have an up-to-date snapshot of mailman available for downloading? I'm lagging badly behind with the patches. -- Janne From klm@python.org Fri May 22 01:25:42 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 21 May 1998 20:25:42 -0400 (EDT) Subject: [Mailman-Developers] Administrators would like to see hidden addresses In-Reply-To: Message-ID: On 22 May 1998, Janne Sinkkonen wrote: > Scott writes: > > > the admin cgi patch i posted a few days ago does this. > > in addition, administrators can edit the options or unsubscribe > > members en masse with this patch. > > OK, thanks. :) > > Does anyone happen to have an up-to-date snapshot of mailman available > for downloading? I'm lagging badly behind with the patches. Now that it looks like i have all the patches, i'm (sorta) hoping to take some time this weekend or early next week to integrate them. However, i'm going to be going out of time in around a week, and time is tight, so no guarantees... Ken From klm@python.org Thu May 28 21:53:35 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 28 May 1998 16:53:35 -0400 (EDT) Subject: [Mailman-Developers] Re: Alternate hosts and sendmail Message-ID: <199805282053.QAA08478@glyph.CNRI.Reston.Va.US> It looks like having mailman do smtp directly is going to resolve the problem i was having getting mailman to do "virtual hosting". And that increases the priority of getting the direct smtp connection in - i have some people here at cnri who need the capability, not urgently, but it'd be nice sooner rather than later... The problem i'm referring to is that mailman does the right thing sending out messages as if they come from the domain for which the maillist is configured, but sendmail by default rewrites the addresses so the maillist address looks like the primary domain for which sendmail is configured. The consensus was that reconfiguring sendmail to avoid that would be dirty - it entails rewriting one of the rulesets, which is fairly-to-very intrusive. Well, it turns out that this problem does not apply to mail relayed by direct smtp connections, hooray. That means no special instructions will be necessary for sendmail (or other mda) configuration, hooray. Oh, and incidentally, i'm going to be away for the next week and a day - no new releases, though barry and john may have some nice stuff to spring on you in the interim. I'm gone! (Hooray.) Ken From klm@python.org Fri May 29 00:56:31 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 28 May 1998 19:56:31 -0400 (EDT) Subject: [Mailman-Developers] Some improvement suggestions Message-ID: <199805282356.TAA09183@glyph.CNRI.Reston.Va.US> Some of the other people here at cnri are using mailman, and they've hit up against one small change needed. I thought i'd mention it here, in case anyone gets the chance and inclination to address it. I also have one other item which i would really like to do, but the eventuality of actually doing it may be less near than i'd like (especially since i'm going away for a week, and there's all that catch-up time on return). The first item is the ability to edit the new-member welcome message. We offer list managers the ability to edit the html pages, and they get to see the way the magic refs (eg, ) are used on the template to see how they could use them for their own versions. Well, there is a number of standard mail messages, including particularly the welcome message, that they may want to edit. (IN FACT, two different groups, separately, came to me today with the need to edit the welcome message - for different reasons! A sure sign...) I think we would have to change the format substitutions, currently all the "%s" order dependent style, to be "%(real_name)s" dictionary substitutions, and otherwise do the same kind of thing we do for the html templates. (Ah, i see barry's done that in some recent checkins.) There is a bunch of messages, some which may benefit from this treatment, other might just clutter the list admins choices. Along these lines, there's something much more encompassing i'd like to do or see done. I'd like to extend the handling of list admin options and html templates so list-specific versions are only squirreled away in list-specific space if and when the list admin changes them from the default. This way, the central administrator could change the template or default values, and the new value would show for all the lists that haven't tailored their own versions. This would certainly be suitable for something like the html pages, where site style changes need propagation, and also for stuff like the forbidden posters. It may not be suitable for some others. This should be easy for the template files - just don't stash list-specific copies until an edit is made (and the edited copy differs), and use the default template if no list-specific one exists. It'd also be good to offer a way in the edit interface to revert to the default version, wiping out the list-specific copy. For the list admin options, we'd need a new field in the options data structure specifying the *name* of the default value, and likewise some way for the gui option for the list admin to explicitly revert to the default. Maybe there should be some way to have a locked-in list-specific setting, even if it doesn't differ from the _current_ site default. And then all the routines that use the list admin options will have to mediate through a routine (or options mediator object) that takes the name of the option and gets the default value when suitable... Well, i hope this is at least slightly understandable, if quick - i gotta run. C ya later! Ken From klm@python.org Fri May 29 01:12:39 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 28 May 1998 20:12:39 -0400 (EDT) Subject: [Mailman-Developers] oh yeah, one more teeny tiny needed change Message-ID: <199805290012.UAA09317@glyph.CNRI.Reston.Va.US> One more small suggestion i got today - list managers should be able to override the monthly password reminder for their list members. Some lists are infrequent, announcement only lists, and there can be enough info in the header/footer about where to find the place to get their passwords that the password reminder is gratuitous. At least, that's the opinion of someone who's running such a list... briefly, ken From dragondm@nexus.Integral.org Fri May 29 02:45:32 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Thu, 28 May 1998 20:45:32 -0500 (CDT) Subject: [Mailman-Developers] Smtp delivery. Message-ID: I noticed the message about direct smtp delivery being needed for virtual hosting.... As I mentioned before, I have a version on mailman 1.0b1 that I am running that already does this. I am going to give a good shot at integrating that feature into the current (1.0b3?) source this weekend. (I want to anyway, as I want to upgrade my server. ) BTW, I also mentioned some user database changes in my variant of mailman... I'm dropping those, as I don't like the way I implemented them. I'll just try to incorporate the other features I added to 1.0b1 (smtp delivery being the biggie) into the current tree, and go from there. -The Dragon De Monsyne From cklempay@jerkweed.ml.org Fri May 29 03:29:17 1998 From: cklempay@jerkweed.ml.org (Corbett Klempay) Date: Thu, 28 May 1998 22:29:17 -0400 (EDT) Subject: No subject Message-ID: (I'm not sure if this should really be on mailman-users or not instead, but it seems somewhat relevant to this list as well)... Does anyone have any quantitative (or maybe even non-quantitative) idea of Mailman's performance, especially compared to direct competitors such as Majordomo? Do we know of any very high volume lists being run on Mailman (or is this even a good idea?). I imagine that something like ezmlm would probably obliterate both Majordomo as well as Mailman as far as performance, but I don't see the average list needing the kind of fiendish performance ezmlm provides (don't me wrong; ezmlm has a lot to recommend it); most people could (and would) trade ezmlm's speed for Mailman's slick web interface. But what I'm wondering is do we know how big this performance tradeoff is? Also, I saw that there was a mention on the TODO list about adding support for other MTAs (such as qmail). Has anyone implemented this on their own yet? Would providing support for qmail affect (enhance) performance much? I'm trying to (seeing how smooth Mailman is) see what kind of potential user base there will be; I'm wondering at what point the list becomes too speed-critical to allow use of Mailman. Any comments? ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "There exists in India no politician daring enough to attempt to explain to the masses that cows can be eaten." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From dragondm@nexus.Integral.org Fri May 29 03:06:15 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Thu, 28 May 1998 21:06:15 -0500 (CDT) Subject: [Mailman-Developers] Some improvement suggestions In-Reply-To: <199805282356.TAA09183@glyph.CNRI.Reston.Va.US> Message-ID: On Thu, 28 May 1998, Ken Manheimer wrote: > Some of the other people here at cnri are using mailman, and they've > hit up against one small change needed. I thought i'd mention it > here, in case anyone gets the chance and inclination to address it. I > also have one other item which i would really like to do, but the > eventuality of actually doing it may be less near than i'd like > (especially since i'm going away for a week, and there's all that > catch-up time on return). > > The first item is the ability to edit the new-member welcome message. > We offer list managers the ability to edit the html pages, and they > get to see the way the magic refs (eg, ) are used on the > template to see how they could use them for their own versions. Well, > there is a number of standard mail messages, including particularly > the welcome message, that they may want to edit. (IN FACT, two > different groups, separately, came to me today with the need to edit > the welcome message - for different reasons! A sure sign...) > > I think we would have to change the format substitutions, currently > all the "%s" order dependent style, to be "%(real_name)s" dictionary > substitutions, and otherwise do the same kind of thing we do for the > html templates. (Ah, i see barry's done that in some recent > checkins.) There is a bunch of messages, some which may benefit from > this treatment, other might just clutter the list admins choices. Hmm.. Has anyone looked at doing the whole shot with Digicool's DocumentTemplate modules? The do both html and text, and their DTML is alot more flexible than mailan's current 'magic-tag' system. It seems a natural. > Along these lines, there's something much more encompassing i'd like > to do or see done. I'd like to extend the handling of list admin > options and html templates so list-specific versions are only > squirreled away in list-specific space if and when the list admin > changes them from the default. This way, the central administrator > could change the template or default values, and the new value would > show for all the lists that haven't tailored their own versions. This > would certainly be suitable for something like the html pages, where > site style changes need propagation, and also for stuff like the > forbidden posters. It may not be suitable for some others. > > This should be easy for the template files - just don't stash > list-specific copies until an edit is made (and the edited copy > differs), and use the default template if no list-specific one > exists. It'd also be good to offer a way in the edit interface to > revert to the default version, wiping out the list-specific copy. > > For the list admin options, we'd need a new field in the options data > structure specifying the *name* of the default value, and likewise > some way for the gui option for the list admin to explicitly revert to > the default. Maybe there should be some way to have a locked-in > list-specific setting, even if it doesn't differ from the _current_ > site default. And then all the routines that use the list admin > options will have to mediate through a routine (or options mediator > object) that takes the name of the option and gets the default value > when suitable... > Hmm... Suggestion... In my UserDB module (which I' not abandoning for various other reasons) that stored a database of User objects for each list, I had something like this. The Mailinglist object held certain 'default' settings for users, and the User object contained oly those settings that were changed. the User objects used the Aquisition mdule (part of the Extention Class system) to fetch the info from the list object if they didn't have it. IMHO, I realy think mailman needs a class of persistant Server object (one for each virtual server) to store server-specific info which really dosen't belong in the mailinglist object. Such an object could store defaults in the same way as my UserDB did . It could also have methods for things like getting the list of lists. PS Anyone thought of using mailman with Bobo? -The Dragon De Monsyne From viega@list.org Fri May 29 03:41:17 1998 From: viega@list.org (John Viega) Date: Thu, 28 May 1998 19:41:17 -0700 Subject: [Mailman-Developers] Re: your mail In-Reply-To: ; from Corbett Klempay on Thu, May 28, 1998 at 10:29:17PM -0400 References: Message-ID: <19980528194117.B18099@list.org> On Thu, May 28, 1998 at 10:29:17PM -0400, Corbett Klempay wrote: > Does anyone have any quantitative (or maybe even non-quantitative) idea of > Mailman's performance, especially compared to direct competitors such as > Majordomo? It's more efficient than Majordomo, especially when doing bulk mailing. > Do we know of any very high volume lists being run on Mailman How high volume? I know of some lists with 5000+ users, they don't seem to have any problems. The biggest one I run has been over 2000 strong. > (or is this even a good idea?). I imagine that something like ezmlm would > probably obliterate both Majordomo as well as Mailman as far as > performance, but I don't see the average list needing the kind of fiendish > performance ezmlm provides (don't me wrong; ezmlm has a lot to recommend > it); most people could (and would) trade ezmlm's speed for Mailman's slick > web interface. But what I'm wondering is do we know how big this > performance tradeoff is? You'd do a lot better performance wise trading sendmail for qmail than Mailman for ezmlm. Also, within a few months we're going to have an option to keep mailman as a persistant server, to avoid the costs involved of always starting up. > Also, I saw that there was a mention on the TODO list about adding support > for other MTAs (such as qmail). Has anyone implemented this on their own > yet? Would providing support for qmail affect (enhance) performance much? I'm working on this feature as we speak. Like I said, the underlying mail transport is more important than the MLM software in terms of how many messages you can push through a machine. > I'm trying to (seeing how smooth Mailman is) see what kind of potential > user base there will be; I'm wondering at what point the list becomes too > speed-critical to allow use of Mailman. Any comments? Well, I think that the current way the locking is done might cause some problems on lists that have large, large numbers of users AND get tons of users. I don't know how large those numbers are yet, but the big list I run has gotten 100 posts in an hour before, without problem. The locking is going to eventually become more fine-grained, so even for the largest lists, I don't think it'll be a problem. John From scott@chronis.pobox.com Fri May 29 04:00:06 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 28 May 1998 23:00:06 -0400 Subject: [Mailman-Developers] Re: your mail In-Reply-To: ; from Corbett Klempay on Thu, May 28, 1998 at 10:29:17PM -0400 References: Message-ID: <19980528230006.22326@chronis.icgroup.com> On Thu, May 28, 1998 at 10:29:17PM -0400, Corbett Klempay wrote: | Does anyone have any quantitative (or maybe even non-quantitative) idea of | Mailman's performance, especially compared to direct competitors such as | Majordomo? not that i know of. | Do we know of any very high volume lists being run on Mailman | (or is this even a good idea?). I The highest volume that i have heard of is ~250. Over the next number of weeks, i plan to give mailman a test run for some large lists. If i'm lucky i'll get some lists with thousands of subscribers. I think large lists with mailman is a good idea, and that mailman's programming is structured in a way that if it cannot handle very large lists as is, it will be easy to accomodate the needed changes. I am certain it will become quite scalable over the summer :) | imagine that something like ezmlm would | probably obliterate both Majordomo as well as Mailman as far as | performance, but I don't see the average list needing the kind of fiendish | performance ezmlm provides (don't me wrong; ezmlm has a lot to recommend | it); most people could (and would) trade ezmlm's speed for Mailman's slick | web interface. But what I'm wondering is do we know how big this | performance tradeoff is? As an administrator of a very high volume list site, i can assure you that the efficiency of the MTA is so much more important than the efficiency of the mailling list manager that the efficiency of the mailling list manager can *almost* be ignored. As i see it, mailman's efficiency is OK now, and can and will be made more and more efficient by implementing better handling of locking the lists, general code cleanup, and perhaps a client-server model. Without any hard data, in many ways it seems like mailman will outperform majordomo with most list operations in loaded circumstances simply because it deals with a lot less file io, which is one the main things that tends to slow down a host that is delivering lots of mail. Also, with the upcoming use of deliveries via smtp, it will be much easier to have the web interface for mailman on one machine and the outgoing deliveries happen on another machine. To my knowledge, most really large listservs do this, and it would be quite difficult to get to work with majordomo. I'm not familiar with ezmlm, but I know that no matter how efficient it is, it almost certainly won't make a noticable difference compared to differences produced by how an MTA handles mail, and if it's difficult to have delivery occur on a separate machine with ezmlm, then it can't scale as well as a mailling list manager that allows for that separation. | | Also, I saw that there was a mention on the TODO list about adding support | for other MTAs (such as qmail). Has anyone implemented this on their own | yet? Would providing support for qmail affect (enhance) performance much? the upcoming smtp changes will make any MTA work for delivery. i don't think anyone has made something that will manage aliases for qmail, though. | I'm trying to (seeing how smooth Mailman is) see what kind of potential | user base there will be; I'm wondering at what point the list becomes too | speed-critical to allow use of Mailman. Any comments? Once mailman is past it's beta stage, i'm willing to bet that it will handle large lists very well. In the mean time, there may well be some unforeseen glitches. scott From scott@chronis.pobox.com Fri May 29 04:06:15 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 28 May 1998 23:06:15 -0400 Subject: [Mailman-Developers] Re: your mail In-Reply-To: <19980528194117.B18099@list.org>; from John Viega on Thu, May 28, 1998 at 07:41:17PM -0700 References: <19980528194117.B18099@list.org> Message-ID: <19980528230615.23149@chronis.icgroup.com> On Thu, May 28, 1998 at 07:41:17PM -0700, John Viega wrote: | On Thu, May 28, 1998 at 10:29:17PM -0400, Corbett Klempay wrote: | > Does anyone have any quantitative (or maybe even non-quantitative) idea of | > Mailman's performance, especially compared to direct competitors such as | > Majordomo? | | It's more efficient than Majordomo, especially when doing bulk mailing. | | > Do we know of any very high volume lists being run on Mailman | | How high volume? I know of some lists with 5000+ users, they don't | seem to have any problems. The biggest one I run has been over 2000 | strong. Wow! | > (or is this even a good idea?). I imagine that something like ezmlm would | > probably obliterate both Majordomo as well as Mailman as far as | > performance, but I don't see the average list needing the kind of fiendish | > performance ezmlm provides (don't me wrong; ezmlm has a lot to recommend | > it); most people could (and would) trade ezmlm's speed for Mailman's slick | > web interface. But what I'm wondering is do we know how big this | > performance tradeoff is? | | You'd do a lot better performance wise trading sendmail for qmail than | Mailman for ezmlm. just wanted to suggest vmailer as well as qmail. personally, i like vmailer a lot more. it's about as efficient as qmail for mailling list delivery, a lot more efficient than qmail for 1 to 1 delivery, it has sane logging, and is more effectively a sendmail drop in replacement. i'm using it with mailman, and it required no changes whatsoever to the distribution to work. scott From klm@python.org Fri May 29 06:29:56 1998 From: klm@python.org (Ken Manheimer) Date: Fri, 29 May 1998 01:29:56 -0400 (EDT) Subject: [Mailman-Developers] Smtp delivery. In-Reply-To: Message-ID: On Thu, 28 May 1998, The Dragon De Monsyne wrote: > I noticed the message about direct smtp delivery being needed for > virtual hosting.... > > As I mentioned before, I have a version on mailman 1.0b1 that I am > running that already does this. I am going to give a good shot at > integrating that feature into the current (1.0b3?) source this weekend. > (I want to anyway, as I want to upgrade my server. ) It looks like john was doing this, today, and all that will come out with barry's big autoconf changes within the next few weeks - maybe sooner - probably as 1.0b4. So you may not want to spend more time on it. > BTW, I also mentioned some user database changes in my variant of > mailman... I'm dropping those, as I don't like the way I implemented them. > I'll just try to incorporate the other features I added to 1.0b1 (smtp > delivery being the biggie) into the current tree, and go from there. If you're interested in incorporating the other things, you should check with barry about getting a current snapshot before doing to much hacking... Ken From roy@endeavor.med.nyu.edu Fri May 29 12:39:53 1998 From: roy@endeavor.med.nyu.edu (Roy Smith) Date: Fri, 29 May 1998 07:39:53 -0400 Subject: [Mailman-Developers] Re: your mail Message-ID: <26162.3105416393@qwerky.med.nyu.edu> "John Viega" wrote: > It's more efficient than Majordomo, especially when doing bulk mailing. This may be a naive question, but how can that be? Majordomo doesn't actually get involved in mail delivery (at least the versions I've used). It just maintains an address file which is read by sendmail (or whatever MTA you're using) to do the actual delivery. So what does it mean to say that majordomo isn't efficient at bulk mailing? From viega@list.org Fri May 29 13:21:15 1998 From: viega@list.org (John Viega) Date: Fri, 29 May 1998 05:21:15 -0700 Subject: [Mailman-Developers] Re: your mail In-Reply-To: <26162.3105416393@qwerky.med.nyu.edu>; from Roy Smith on Fri, May 29, 1998 at 07:39:53AM -0400 References: <26162.3105416393@qwerky.med.nyu.edu> Message-ID: <19980529052115.A25666@list.org> The assumption being that you're using sendmail, I should have said that. What happens is, majordomo uses a single sendmail connection. On lists of 2000 people, it can take hours to deliver to everyone, because w/ a list that large, there are going to be lots of machines that are temporarily inaccessable, etc. Sendmail just marches down the list of addresses, sending each one. If one of them blocks, sendmail will stall delivery for everyone for a few minutes. Mailman by default forks off several versions of sendmail at once. You can set the number of sendmails to be whatever you like. For a mailing list of mine that used to take several hours to completely deliver a mail, Mailman reduced the time to about 2 minutes. Make sense? On Fri, May 29, 1998 at 07:38:57AM -0400, Roy Smith wrote: > "John Viega" wrote: > > It's more efficient than Majordomo, especially when doing bulk mailing. > > This may be a naive question, but how can that be? > > Majordomo doesn't actually get involved in mail delivery (at least the > versions I've used). It just maintains an address file which is read by > sendmail (or whatever MTA you're using) to do the actual delivery. So what > does it mean to say that majordomo isn't efficient at bulk mailing? > > Roy Smith > New York University School of Medicine > 550 First Avenue, New York, NY 10016 > > From viega@list.org Fri May 29 13:37:06 1998 From: viega@list.org (John Viega) Date: Fri, 29 May 1998 05:37:06 -0700 Subject: [Mailman-Developers] Smtp delivery. In-Reply-To: ; from Ken Manheimer on Fri, May 29, 1998 at 01:29:56AM -0400 References: Message-ID: <19980529053706.B25666@list.org> On Fri, May 29, 1998 at 01:29:56AM -0400, Ken Manheimer wrote: > On Thu, 28 May 1998, The Dragon De Monsyne wrote: > > > I noticed the message about direct smtp delivery being needed for > > virtual hosting.... > > > > As I mentioned before, I have a version on mailman 1.0b1 that I am > > running that already does this. I am going to give a good shot at > > integrating that feature into the current (1.0b3?) source this weekend. > > (I want to anyway, as I want to upgrade my server. ) > > It looks like john was doing this, today, and all that will come out > with barry's big autoconf changes within the next few weeks - maybe > sooner - probably as 1.0b4. So you may not want to spend more time on > it. Yeah, I got smtp delivery done last night, and it seems to be working well. > > BTW, I also mentioned some user database changes in my variant of > > mailman... I'm dropping those, as I don't like the way I implemented them. > > I'll just try to incorporate the other features I added to 1.0b1 (smtp > > delivery being the biggie) into the current tree, and go from there. > > If you're interested in incorporating the other things, you should check > with barry about getting a current snapshot before doing to much > hacking... Poor Barry :) I can provide you with a snapshot as well. In fact, let me try to merge patches that have recently been posted to this list into the distribution, and then I'll put up a snapshot of the current version just for the people on this mailing list. It would not only be good for people wanting to integrate more changes, but it will be good for testing out newer code... John From viega@list.org Fri May 29 13:49:28 1998 From: viega@list.org (John Viega) Date: Fri, 29 May 1998 05:49:28 -0700 Subject: [Mailman-Developers] Some improvement suggestions In-Reply-To: ; from The Dragon De Monsyne on Thu, May 28, 1998 at 09:06:15PM -0500 References: <199805282356.TAA09183@glyph.CNRI.Reston.Va.US> Message-ID: <19980529054928.C25666@list.org> On Thu, May 28, 1998 at 09:06:15PM -0500, The Dragon De Monsyne wrote: > Hmm.. Has anyone looked at doing the whole shot with Digicool's > DocumentTemplate modules? The do both html and text, and their DTML is > alot more flexible than mailan's current 'magic-tag' system. It seems a > natural. Sounds good. I don't think anyone is persuing that right now, if you want to do it.. > IMHO, I realy think mailman needs a class of persistant Server > object (one for each virtual server) to store server-specific info which > really dosen't belong in the mailinglist object. Such an object could > store defaults in the same way as my UserDB did . It could also have > methods for things like getting the list of lists. Hmm, on one level it does belong in the mailing list object. For example, you might have the default set up to be: www.mymachine.com for both web stuff and list stuff, and someone might want to change it to be www.mymachine.com for the web page, and list.mymachine.com for the list. Or the other way around. I've seen a few lists use completely different domain names for the mail address and the web address. It's definitely a property of the list, although storing defaults on a per-server level would be good. I think I'd prioritize it fairly low, though, since creating new lists is not a frequently done thing. If you want to work on it, though, I do think it'd be a fine addition. From dragondm@nexus.Integral.org Fri May 29 13:23:54 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Fri, 29 May 1998 07:23:54 -0500 (CDT) Subject: [Mailman-Developers] Smtp delivery. In-Reply-To: Message-ID: On Fri, 29 May 1998, Ken Manheimer wrote: > > I noticed the message about direct smtp delivery being needed for > > virtual hosting.... > > > > As I mentioned before, I have a version on mailman 1.0b1 that I am > > running that already does this. I am going to give a good shot at > > integrating that feature into the current (1.0b3?) source this weekend. > > (I want to anyway, as I want to upgrade my server. ) > > It looks like john was doing this, today, and all that will come out > with barry's big autoconf changes within the next few weeks - maybe > sooner - probably as 1.0b4. So you may not want to spend more time on > it. Hmm.... > > > BTW, I also mentioned some user database changes in my variant of > > mailman... I'm dropping those, as I don't like the way I implemented them. > > I'll just try to incorporate the other features I added to 1.0b1 (smtp > > delivery being the biggie) into the current tree, and go from there. > > If you're interested in incorporating the other things, you should check > with barry about getting a current snapshot before doing to much > hacking... Yes. Where/how do I get the current system? -The Dragon De Monsyne From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri May 29 17:47:13 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 29 May 1998 12:47:13 -0400 (EDT) Subject: [Mailman-Developers] Smtp delivery. References: Message-ID: <13678.58953.409728.12328@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> If you're interested in incorporating the other things, you KM> should check with barry about getting a current snapshot KM> before doing to much hacking... One of the things I would really like to do, is to get a read-only public CVS tree available, so that it will be easier for developers such as Scott to integrate their changes, and stay as (painfully) up-to-date as they want :-). -Barry From viega@list.org Sat May 30 11:41:04 1998 From: viega@list.org (John Viega) Date: Sat, 30 May 1998 03:41:04 -0700 Subject: [Mailman-Developers] Cookies Message-ID: <19980530034104.A3940@list.org> I installed Scott's patches for confirmation and admin logins (thank god for ediff-buffers). I have a couple of questions mainly for Scott, but I think other people might be interested in discussing them. First, I don't know what the expiration time for cookies is, but the cookie didn't go away when I shut down my browser. Do you think that's good behavior? I'd like to not be implicitly logged in if someone else starts up my browser. Also, I've seen some sites that log people off automatically after 15 mins of inactivity on that site. Do you think that's a good idea? Second, if you don't have cookies on, changes don't get made. You get sent back to the login screen, and when you log back in, everything is the same. Should cookies really be required? Something that could be done to offer similar functionality yet not require cookies would be to have an "enter your password" box after the initial login, and put the password in the proper field as default text. While that may not be incredibly secure, it's not much worse than sending a plaintext password via httpd the first time only (although the password will be in the page source). Also, perhaps there should be a way to explicitly log out? I can't get logged out, even by turning off cookies! John From scott@chronis.pobox.com Sat May 30 19:23:02 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 30 May 1998 14:23:02 -0400 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530034104.A3940@list.org>; from John Viega on Sat, May 30, 1998 at 03:41:04AM -0700 References: <19980530034104.A3940@list.org> Message-ID: <19980530142302.24877@chronis.icgroup.com> After i released the admin patches, i realized that the way the cookies were put together was causing some problems. i rewrote the function isAuthenticated to work more like the same function in private. this change addresses much of what you are saying. i haven't posted about this till now because i've been quite busy with other things and unsure about the best way to submit a patch to a patch to a patch ;) On Sat, May 30, 1998 at 03:41:04AM -0700, John Viega wrote: | I installed Scott's patches for confirmation and admin logins (thank | god for ediff-buffers). I have a couple of questions mainly for | Scott, but I think other people might be interested in discussing | them. | | First, I don't know what the expiration time for cookies is, but the | cookie didn't go away when I shut down my browser. Do you think | that's good behavior? no. | I'd like to not be implicitly logged in if | someone else starts up my browser. Also, I've seen some sites that | log people off automatically after 15 mins of inactivity on that site. | Do you think that's a good idea? The cookies will not allow anyone to submit changes after the timeout period (defaulting to 20 minutes). I'm not sure how to portably force people to be logged off in any other way. | | Second, if you don't have cookies on, changes don't get made. You get | sent back to the login screen, and when you log back in, everything is | the same. Should cookies really be required? With the changed isAuthenticated function, an admin can enter the password on each screen to make changes, but will still have to log into each section separately :(. | Something that could be | done to offer similar functionality yet not require cookies would be | to have an "enter your password" box after the initial login, and put | the password in the proper field as default text. While that may not | be incredibly secure, it's not much worse than sending a plaintext | password via httpd the first time only (although the password will be | in the page source). | | Also, perhaps there should be a way to explicitly log out? that sounds like a good idea. my rewrite of the isAuthenticated function in the admin cgi follows: scott SECRET="monty" def isAuthenticated(list, password=None, SECRET="SECRET"): import base64, md5 if password is not None: # explicit login try: list.ConfirmAdminPassword(password) except mm_err.MMBadPasswordError: AddErrorMessage(doc, 'Error: Incorrect admin password.') return 0 token = md5.new(SECRET + list_name + SECRET).digest() token = base64.encodestring(token) token = string.strip(token) c = Cookie.Cookie() cookie_key = list_name + "-admin" c[cookie_key] = token c[cookie_key]['expires'] = mm_cfg.ADMIN_COOKIE_LIFE path = list.GetScriptURL("admin") path = path[string.find(path, "://") + 3:] path = path[string.find(path, "/"):] c[cookie_key]["path"] = path print c # Output the cookie return 1 if os.environ.has_key('HTTP_COOKIE'): c = Cookie.Cookie( os.environ['HTTP_COOKIE'] ) if c.has_key(list_name + "-admin"): inp = base64.decodestring(c[list_name + "-admin"].value) check = md5.new(SECRET+list_name+SECRET).digest() if inp == check: return 1 else: return 0 return 0 From viega@list.org Sat May 30 21:28:29 1998 From: viega@list.org (John Viega) Date: Sat, 30 May 1998 13:28:29 -0700 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530142302.24877@chronis.icgroup.com>; from Scott on Sat, May 30, 1998 at 02:23:02PM -0400 References: <19980530034104.A3940@list.org> <19980530142302.24877@chronis.icgroup.com> Message-ID: <19980530132829.C4221@list.org> On Sat, May 30, 1998 at 02:23:02PM -0400, Scott wrote: > > | I'd like to not be implicitly logged in if > | someone else starts up my browser. Also, I've seen some sites that > | log people off automatically after 15 mins of inactivity on that site. > | Do you think that's a good idea? > > The cookies will not allow anyone to submit changes after the timeout > period (defaulting to 20 minutes). I'm not sure how to portably force > people to be logged off in any other way. Well, what I'm talking about is the same thing, but if you press "submit" after the cookie runs out, you'll get an error message saying, "we've already logged you out due to inactivity", instead of just giving the login screen. Also, I noticed that it looks like the edithtml pages don't share the same cookie (and, in fact, aren't using cookies at all at the moment...) John From scott@chronis.pobox.com Sat May 30 21:37:38 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 30 May 1998 16:37:38 -0400 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530132829.C4221@list.org>; from John Viega on Sat, May 30, 1998 at 01:28:29PM -0700 References: <19980530034104.A3940@list.org> <19980530142302.24877@chronis.icgroup.com> <19980530132829.C4221@list.org> Message-ID: <19980530163738.06030@chronis.icgroup.com> On Sat, May 30, 1998 at 01:28:29PM -0700, John Viega wrote: | On Sat, May 30, 1998 at 02:23:02PM -0400, Scott wrote: | > | > | I'd like to not be implicitly logged in if | > | someone else starts up my browser. Also, I've seen some sites that | > | log people off automatically after 15 mins of inactivity on that site. | > | Do you think that's a good idea? | > | > The cookies will not allow anyone to submit changes after the timeout | > period (defaulting to 20 minutes). I'm not sure how to portably force | > people to be logged off in any other way. | | Well, what I'm talking about is the same thing, but if you press | "submit" after the cookie runs out, you'll get an error message | saying, "we've already logged you out due to inactivity", instead of | just giving the login screen. how do you tell the difference between a cookie running out and no cookie being submitted in the first place? | Also, I noticed that it looks like the edithtml pages don't share the | same cookie (and, in fact, aren't using cookies at all at the moment...) true. it seems like they should more for the sake of uniformity than anything else, as there's no reason to hide publicly accessible html pages from people. this shouldn't be too hard to do with the existing isAuthenticated function from the admin cgi. i don't think i'll have time to that before i go away June 1-10, but i'm willing to change edithtml when i get back if no one's done it yet. scott From viega@list.org Sat May 30 21:42:42 1998 From: viega@list.org (John Viega) Date: Sat, 30 May 1998 13:42:42 -0700 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530163738.06030@chronis.icgroup.com>; from Scott on Sat, May 30, 1998 at 04:37:38PM -0400 References: <19980530034104.A3940@list.org> <19980530142302.24877@chronis.icgroup.com> <19980530132829.C4221@list.org> <19980530163738.06030@chronis.icgroup.com> Message-ID: <19980530134242.D4221@list.org> On Sat, May 30, 1998 at 04:37:38PM -0400, Scott wrote: > > how do you tell the difference between a cookie running out and no > cookie being submitted in the first place? I think by checking to see if there are extra CGI parameters passed? If so, you can assume they were still on your page. > | Also, I noticed that it looks like the edithtml pages don't share the > | same cookie (and, in fact, aren't using cookies at all at the moment...) > > true. it seems like they should more for the sake of uniformity than > anything else, as there's no reason to hide publicly accessible html > pages from people. this shouldn't be too hard to do with the existing > isAuthenticated function from the admin cgi. i don't think i'll have > time to that before i go away June 1-10, but i'm willing to change > edithtml when i get back if no one's done it yet. Would it be possible to keep 1 central auth checking function? I also noticed that admindb needs a similar mechanism. Also, when we add a site administrator's UI, we'll need to use the same functionality again... John From scott@chronis.pobox.com Sat May 30 21:50:58 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 30 May 1998 16:50:58 -0400 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530134242.D4221@list.org>; from John Viega on Sat, May 30, 1998 at 01:42:42PM -0700 References: <19980530034104.A3940@list.org> <19980530142302.24877@chronis.icgroup.com> <19980530132829.C4221@list.org> <19980530163738.06030@chronis.icgroup.com> <19980530134242.D4221@list.org> Message-ID: <19980530165058.09702@chronis.icgroup.com> On Sat, May 30, 1998 at 01:42:42PM -0700, John Viega wrote: | On Sat, May 30, 1998 at 04:37:38PM -0400, Scott wrote: | > | > how do you tell the difference between a cookie running out and no | > cookie being submitted in the first place? | | I think by checking to see if there are extra CGI parameters passed? | If so, you can assume they were still on your page. that's a good idea. | > | Also, I noticed that it looks like the edithtml pages don't share the | > | same cookie (and, in fact, aren't using cookies at all at the moment...) | > | > true. it seems like they should more for the sake of uniformity than | > anything else, as there's no reason to hide publicly accessible html | > pages from people. this shouldn't be too hard to do with the existing | > isAuthenticated function from the admin cgi. i don't think i'll have | > time to that before i go away June 1-10, but i'm willing to change | > edithtml when i get back if no one's done it yet. | | Would it be possible to keep 1 central auth checking function? I also | noticed that admindb needs a similar mechanism. Also, when we add a | site administrator's UI, we'll need to use the same functionality | again... i'm sure it would be possible, and thought that maybe such things should go in a general mailman cgi library, maybe in modules/mm_cgilib.py? i'm not sure where the best place for that is. scott From viega@list.org Sun May 31 03:35:46 1998 From: viega@list.org (John Viega) Date: Sat, 30 May 1998 19:35:46 -0700 Subject: [Mailman-Developers] Pipermail Message-ID: <19980530193546.A8889@list.org> Has anyone modified pipermail's hypermail emulation to work using only modules that are compiled in the default Python distribution? There's one dependency I have in mind: pipermail currently uses bsddb, which is not compiled into Python 1.5. I think gdbm is compiled in by default, but the two interfaces aren't compatible. My biggest goal w/ Mailman has always been to have everything people would ever want to do be easy to do. That's why I'm pretty reluctant to distribute without built-in archiving - one thing I hated about Majordomo was having to go out and download extras and then figure out how to integrate them to get useful functionality. I also believe that many people who will want to use Mailman will be using Python for the first time. The default Python installation is really easy to build. But as soon as you tell people, "Hey, now you have to recompile, and install crypt. Oh, and now you have to go download pipermail. And before you use that, you have to find bsddbm on the net somewhere, and install it", that's going to send people running back to the comfort of Perl and Majordomo, even though they'll be doing the same sorts of things. To that end, I've put in provisions for passwords to use md5 if crypt is not found. BTW, do you guys think md5 should be the default, and crypt an option for those who add it to their config? The only real advantage of crypt is that it is less space intensive. Also, the way I have things set up at the moment, if someone starts w/ the default distribution and later recompiles Python to include crypt, all the old passwords won't work, since they'll be in the md5 digest format. John From scott@chronis.pobox.com Sun May 31 20:12:04 1998 From: scott@chronis.pobox.com (Scott) Date: Sun, 31 May 1998 15:12:04 -0400 Subject: [Mailman-Developers] nasty admin Cookie bug (and fix) Message-ID: <19980531151204.04788@chronis.icgroup.com> there is a bug in the admin cgi changes that i posted. the block of code making general list changes fires from some brownsers when you log on and press enter to submit the data instead of clicking the submit button. the reason is that the code knows to process the login when the button is pressed because of it's name. that data is not submitted from some browsers when you submit the form by pressing enter instead of clicking on the button. since not much data is submitted, it wipes out a lot of the list information. bad bad bad! anyway, the fix is simple enough (against the previous patch and the substituted isAuthenticted function). scott chronis 3:01pm $ cvs diff -r1.4 admin Index: admin =================================================================== RCS file: /usr/local/cvsroot/mailman/cgi/admin,v retrieving revision 1.4 diff -r1.4 admin 689c690,692 < if category != 'members' and not cgi_info.has_key("request_login"): --- > if category != 'members' and \ > not cgi_info.has_key("request_login") and \ > len(cgi_info.keys()) > 1: From dragondm@nexus.Integral.org Fri May 1 15:20:41 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Fri, 1 May 1998 09:20:41 -0500 (CDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) Message-ID: Aha!! a mailman developer's list!! Apparently I was in the same position as Ken Manheimer, with regards to MailMan. When the origional Mailman web sites & lists dissapeared, and didn't come back, I thought the author had abandoned development, and continued hacking at MailMan for my own use. I have made numerous changes to the code as a result. Now that I am aware that MailMan is being actively developed, I would like to see what I can do about merging these versions together. I started with a copy of Mailman 1.0b1. Because I was unsure of the status of MailMan, I have been calling my version DMM (the Dragon's version of MailMan), and the python modules I have added start with 'dmm_' I haven't yet had the time to download & go over the 1.0b2.0 (that is the most current version ?) sources, to see what things I've done may be redundant, (I will this weekend) but here is a list of the changes that I've made (that I can think of so far): (in roughly chronological order) Added Makefile to make & install all the wrappers. Earlier versions of this were posted to the previous MailMan dev list. Things have been improved since. Removed all dependance on MailMan being in /home/mailman. I made sure all scripts were always being run as the MailMan user, then used the pwd module to find the current user's home directory. Everything else is found from there. (I did this so I could have two MailMan installs on one machine, a 'production' server, which runs several lists, and a development server) Re-did logging system. Removed all /tmp logging. Created module that redirects errors to files in ~/logs Error log files are named for the scripts (thus errors from admin show up in ~/logs/admin.err) This is done automatically by importing the logging module. (major) Compleatly changed the way outgoing mail gets sent. Did much re-writing of mm_message & mm_deliver modules. Both OutgoingMessage & IncomingMessage are now derived from a common base class, ListMessage. Sending is accomplished by calling send method of message object. Mail is no longer sent by piping to 'sendmail'! Instead, outgoing messages are punted to a mailserver (currently always localhost) via SMTP. I wrote an SMTP module, smtplib to do this. smtplib is now part of the python standard library as of 1.5.1 Added random-mnemnonic password generator. Instead of passwords like 'qZ', AOL-like random-word passwords are generated (like 'cold-blue-tiger') Added multiple subject: prefix removal and Re: compaction changes subjects like: "Re: Prefix Re: Prefix Re Something" to "Prefix Re[3]: Something" (major) Changed the way user data is stored (still tying up loose ends on this) Created a User class. User data is no longer stored in the config.db file. Instead there is a shelf full of User objects. The main Mailinglist object contains an instance of a UserDB class that manages storing, retrieving & quering this database. Most of the *User* methods of the Mailinglist class have become methods of the User class. Changed the way user options are handled. Options are referenced by name (a string) and can be Boolean or "multiple-choice" values. Info on available options & allowed values for them are available via methods of the User objects. I compacted the 'Digests', "DisableDelivery" and "EnableMIME" options into one multiple-choice "delivery_mode" option." There have prolly been soem bugfixes too. Hopefully all this work wasn't redundant :> -The Dragon De Monsyne From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri May 1 16:05:59 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 1 May 1998 11:05:59 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) References: Message-ID: <13641.57859.522754.975416@anthem.CNRI.Reston.Va.US> Dragon, Very cool that you have continued hacking on Mailman. I think we should definitely work together to get a single merged version. One of the things that we hope to do RSN is get an anonymous CVS read-only tree up and running. That will hopefully keep developers as painfully up-to-date as they want, and encourage people to submit patches against the current devel versions. >>>>> "TDDM" == The Dragon De Monsyne writes: TDDM> Added Makefile to make & install all the wrappers. Earlier TDDM> versions of this were posted to the previous MailMan dev TDDM> list. Things have been improved since. TDDM> Removed all dependance on MailMan being in /home/mailman. I TDDM> made sure all scripts were always being run as the MailMan TDDM> user, then used the pwd module to find the current user's TDDM> home directory. Everything else is found from there. (I did TDDM> this so I could have two MailMan installs on one machine, a TDDM> 'production' server, which runs several lists, and a TDDM> development server) I'm currently working on autoconf'ing the installation process. I've got the src directory mostly working and just need to add a few more things before I check stuff in. I'd really like for you to be able to say ./configure --prefix=/not/home/mailman and have everything Just Work. The default $prefix will be /home/mailman. TDDM> (major) Compleatly changed the way outgoing mail gets sent. TDDM> Did much re-writing of mm_message & mm_deliver modules. Both TDDM> OutgoingMessage & IncomingMessage are now derived from a TDDM> common base class, ListMessage. Sending is accomplished by TDDM> calling send method of message object. Mail is no longer TDDM> sent by piping to 'sendmail'! Instead, outgoing messages are TDDM> punted to a mailserver (currently always localhost) via TDDM> SMTP. I wrote an SMTP module, smtplib to do this. smtplib is TDDM> now part of the python standard library as of 1.5.1 Very cool. I know this last bit was high on Ken's hit list. TDDM> Added random-mnemnonic password generator. Instead of TDDM> passwords like 'qZ', AOL-like random-word passwords are TDDM> generated (like 'cold-blue-tiger') Also very cool. I was thinking about using the implementation of RFC1751 from Andrew Kuchling's PCT to do something similar. Andrew and I have talked about what he can do to release the non-export controlled parts of the PCT separately. TDDM> Hopefully all this work wasn't redundant :> I don't think so. You've done some cool stuff that others are attacking or want to attack. Things may take a while to accrete as people discover that Mailman is alive and kicking. But it's really good to see so many people interested in Mailman's future! -Barry From klm@python.org Fri May 1 17:28:03 1998 From: klm@python.org (Ken Manheimer) Date: Fri, 1 May 1998 12:28:03 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: Message-ID: Mr dragon, It looks like we've got a nice synergy here - a number of the items you attacked are things i wanted to do, but did not yet do, while i (and now we) have taken care of a number of other things. (Most of the big things are mentioned in the notes i included in the distributions.) On Fri, 1 May 1998, The Dragon De Monsyne wrote: > I have made numerous changes to the code as a result. Now that I > am aware that MailMan is being actively developed, I would like to see > what I can do about merging these versions together. As barry mentioned, we're planning to make a cvs view of our version available across the net sometime soon. I should add that we will need to clear some admin issues first, so it's not guaranteed, but likely. > I started with a copy of Mailman 1.0b1. Because I was unsure of > the status of MailMan, I have been calling my version DMM (the Dragon's > version of MailMan), and the python modules I have added start with 'dmm_' I suspect that we started at the same place. Not sure whether there were any micro-releases or such that would lead to version skew. > I haven't yet had the time to download & go over the 1.0b2.0 (that > is the most current version ?) sources, to see what things I've done may > be redundant, (I will this weekend) but here is a list of the changes that > I've made (that I can think of so far): That is the latest version. I may release a patch with my latest hacking over the weekend, if i have time. ( Mostly bug fixes, some refinements - mime digest format, bounce handling, admin-option help.) > Removed all dependance on MailMan being in /home/mailman. > I made sure all scripts were always being run as the MailMan user, > then used the pwd module to find the current user's home directory. > Everything else is found from there. (I did this so I could have two > MailMan installs on one machine, a 'production' server, which runs several > lists, and a development server) Relaxing the /home/mailman rigidity is high on my list! My ideal would be to have as much as possible be in a package or use relative paths (which have the same effect - independence from location of the root). Unfortunately, i have a few other things on my agenda before that, so i'm not sure when i'll get to it. (High up is gettings things like the admin pages behind a cookied password, another item is making sure that a single mailman install can use different addresses for the host for different lists - os, eg, ISP clients with their own domains can use their domains for their mailman lists...) (I'm confused about how working with respect to the ~mailman homedir enables you to have to install's, btw.) > Re-did logging system. Removed all /tmp logging. Created module that > redirects errors to files in ~/logs Error log files are named for > the scripts (thus errors from admin show up in ~/logs/admin.err) This is > done automatically by importing the logging module. This is a place where we completely overlapped - and came up with similar answers, at least in the abstract. I also used ~mailman/logs, but am using a few categories of logs - bounce, vette, subscribe, digest, and error. Error gets the errors from bugs in any of the scripts, so you can just check the file periodically to see if anything is malfunctioning in a way that is caught... I guess we'll have to reconcile the two approaches. > (major) Compleatly changed the way outgoing mail gets sent. > Did much re-writing of mm_message & mm_deliver modules. Both > OutgoingMessage & IncomingMessage are now derived from a common base > class, ListMessage. Sending is accomplished by calling send method of > message object. > Mail is no longer sent by piping to 'sendmail'! Instead, outgoing > messages are punted to a mailserver (currently always localhost) > via SMTP. I wrote an SMTP module, smtplib to do this. smtplib is now part > of the python standard library as of 1.5.1 As barry also mentioned, this is one of the things up there on my "desired" list - but it never was high enough urgency to percolate to the top. (John actually sent me an smtplib.py that he was playing with, but i never took the step of hooking it up.) I'm glad you've done this, and that it uses something you've gotten into the distribution! > Added random-mnemnonic password generator. Instead of passwords like > 'qZ', AOL-like random-word passwords are generated (like > 'cold-blue-tiger') That's nice! People won't have to visit the site to either select reasonable passwords, or have their unmemorable one's sent to them... > Added multiple subject: prefix removal and Re: compaction > changes subjects like: > "Re: Prefix Re: Prefix Re Something" > to "Prefix Re[3]: Something" Me too, except i didn't do any munging of the Re, and in particular leave the Re before the prefix so mail readers sort threads appropriately. I'd rather avoid the fancy Re stuff, with the idea that mailman should do as little intrusion as possible to the content. > (major) Changed the way user data is stored (still tying up loose ends on > this) > Created a User class. User data is no longer stored in the > config.db file. Instead there is a shelf full of User objects. > The main Mailinglist object contains an instance of a UserDB class that > manages storing, retrieving & quering this database. Most of the *User* > methods of the Mailinglist class have become methods of the User class. If you're saying what i think you're saying, this is a real biggie, one which i've dearly wanted to see happen if i didn't get to it myself. (I think john had something like it going in a later, lost version of mailman, and i know a number of us have expressed interest in it.) > Changed the way user options are handled. Options are referenced by name > (a string) and can be Boolean or "multiple-choice" values. Info on > available options & allowed values for them are available via methods > of the User objects. I compacted the 'Digests', "DisableDelivery" and > "EnableMIME" options into one multiple-choice "delivery_mode" option." Sounds cool, provided there's no scaleup-defeating performance hits. We've got some people on this list interested in handling lists of several thousand members... > There have prolly been soem bugfixes too. > > Hopefully all this work wasn't redundant :> Mostly not! ken manheimer klm@python.org From viega@list.org Fri May 1 22:04:38 1998 From: viega@list.org (John Viega) Date: Fri, 1 May 1998 14:04:38 -0700 Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: ; from The Dragon De Monsyne on Fri, May 01, 1998 at 09:20:41AM -0500 References: Message-ID: <19980501140438.B13559@list.org> On Fri, May 01, 1998 at 09:20:41AM -0500, The Dragon De Monsyne wrote: > > Aha!! a mailman developer's list!! > > Apparently I was in the same position as Ken Manheimer, with regards to > MailMan. When the origional Mailman web sites & lists dissapeared, and > didn't come back, I thought the author had abandoned development, and > continued hacking at MailMan for my own use. Well, Ken and I were still in contact. I guess it's too bad you didn't contact me then, as we could have had your work integrated already and made things a bit easier on you. > Added Makefile to make & install all the wrappers. Earlier versions of > this were posted to the previous MailMan dev list. Things have > been improved since. This one has been done. > Removed all dependance on MailMan being in /home/mailman. This one I was going to do in the near future. I am glad that I did not :) > (major) Compleatly changed the way outgoing mail gets sent. > Did much re-writing of mm_message & mm_deliver modules. Both > OutgoingMessage & IncomingMessage are now derived from a common base > class, ListMessage. Sending is accomplished by calling send method of > message object. > Mail is no longer sent by piping to 'sendmail'! Instead, outgoing > messages are punted to a mailserver (currently always localhost) > via SMTP. I wrote an SMTP module, smtplib to do this. smtplib is now part > of the python standard library as of 1.5.1 Hmm, I wrote a basic smtplib a few months ago with the intent of using that instead of a call to sendmail, and I think I saw a cvs checkin from Ken that actually replaced the sendmail call. > > Added random-mnemnonic password generator. Instead of passwords like > 'qZ', AOL-like random-word passwords are generated (like > 'cold-blue-tiger') Cool. > Added multiple subject: prefix removal and Re: compaction > changes subjects like: > "Re: Prefix Re: Prefix Re Something" > to "Prefix Re[3]: Something" You might want to check the archives of this list on this topic, as we've discussed it recently, and agreed that we don't find it desirable for us. However, in thinking about it, I see no reason why it couldn't be added as an option if the list administrator seems to prefer it. > (major) Changed the way user data is stored (still tying up loose ends on > this) > Created a User class. User data is no longer stored in the > config.db file. Instead there is a shelf full of User objects. > The main Mailinglist object contains an instance of a UserDB class that > manages storing, retrieving & quering this database. Most of the *User* > methods of the Mailinglist class have become methods of the User class. That's good. I'd been moving in that direction when that disk crash occured. > Changed the way user options are handled. Options are referenced by name > (a string) and can be Boolean or "multiple-choice" values. Info on > available options & allowed values for them are available via methods > of the User objects. I compacted the 'Digests', "DisableDelivery" and > "EnableMIME" options into one multiple-choice "delivery_mode" option." Sounds reasonable, though I'd like to see it... > Hopefully all this work wasn't redundant :> For the most part, not! My only concern is keeping the amount of work necessary for integration on your part as low as possible. John From viega@list.org Fri May 1 22:16:24 1998 From: viega@list.org (John Viega) Date: Fri, 1 May 1998 14:16:24 -0700 Subject: [Mailman-developers] Web page Message-ID: <19980501141624.C13559@list.org> I'm going to be constructing a new Mailman web page at list.org in the next couple of days (probably after I finish my last final exam monday afternoon). One thing the old site had that I would like to reinstate is a list of all changes we would like to make that have not yet been made. To this end, I'd like people to send wishlists to the list, and for the group to discuss items on the wishlist. Some things that come to mind that are important for me include: 1) Having an optional (threaded) daemon mode that will obviate mailman having to start up multiple times. I've already seen lists where the administrator doesn't have any bounce handling on, and completely ignores all error messages. On a large list, error handling scripts get run frightingly often. 2) The user options should include a way of saying, "If I'm on the TO or CC list, don't send me another copy of that message!" 3) Nice, web-based, in place documentation. 4) Bounce detection needs some major work. I'm going to also look at this one soon. I'd like to also keep track of who is persuing what, so as to avoid overlap of effort. Any other suggestions on what should or should not show up on the Mailman home page are also welcome, as are contributions. From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri May 1 22:23:18 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 1 May 1998 17:23:18 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) References: <19980501140438.B13559@list.org> Message-ID: <13642.15485.769242.827267@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: >> Removed all dependance on MailMan being in /home/mailman. JV> This one I was going to do in the near future. I am glad that JV> I did not :) I'd like to see this stuff when its available so I can integrate it with the autoconf stuff I've been doing. I'm very near the point where the entire package can be easily configured and installed in the standard GNU way. Right now I have stuff set up so that $(prefix) defaults to /home/mailman but for reasons I won't go into, I actually have to configure with --prefix=/export/home/mailman (Ken will know why :-). -Barry From viega@list.org Fri May 1 22:23:12 1998 From: viega@list.org (John Viega) Date: Fri, 1 May 1998 14:23:12 -0700 Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: ; from Ken Manheimer on Fri, May 01, 1998 at 12:28:03PM -0400 References: Message-ID: <19980501142312.D13559@list.org> On Fri, May 01, 1998 at 12:28:03PM -0400, Ken Manheimer wrote: > I suspect that we started at the same place. Not sure whether there > were any micro-releases or such that would lead to version skew. If the version numbers were the same, they were the same. Any time I did a release, even if it was to a single person only, I changed the version number. > (I'm confused about how working with respect to the ~mailman homedir > enables you to have to install's, btw.) Well, if you have 2 installs and 2 sets of binaries, and 2 disjoint sets of lists, and 2 different cgi paths (ie, http://www.xxx.org/mailman-dev/) then I don't see why there would be any problem with it. > As barry also mentioned, this is one of the things up there on my > "desired" list - but it never was high enough urgency to percolate to > the top. (John actually sent me an smtplib.py that he was playing with, > but i never took the step of hooking it up.) I'm glad you've done this, > and that it uses something you've gotten into the distribution! Ah, I would have sworn you already did this one, Ken :) John From roy@endeavor.med.nyu.edu Fri May 1 22:26:54 1998 From: roy@endeavor.med.nyu.edu (Roy Smith) Date: Fri, 01 May 1998 17:26:54 -0400 Subject: [Mailman-developers] Web page Message-ID: <1895883.3103032414@qwerky.med.nyu.edu> Not sure if it does this already, but here's my biggest mailing list headache... We have our lists (run under majordomo, which we want to get rid of) set up that if you're not subscribed, you can't submit. This is the most obvious anti-spam measure, and goes a good way towards cutting down abuse. Problem is, people have multiple email addresses, or they change, or one MUA formats is slightly differently than another MUA they use, etc. It would be really cool to be able to say something like "accept postings from this person, if any of the following are on the From: line" Roy Smith New York University School of Medicine 550 First Avenue, New York, NY 10016 From viega@list.org Fri May 1 22:26:11 1998 From: viega@list.org (Mr. + Ms. John Viega) Date: Fri, 1 May 1998 14:26:11 -0700 Subject: [Mailman-developers] Web page In-Reply-To: <1895883.3103032414@qwerky.med.nyu.edu>; from Roy Smith on Fri, May 01, 1998 at 05:26:54PM -0400 References: <1895883.3103032414@qwerky.med.nyu.edu> Message-ID: <19980501142611.E13559@list.org> On Fri, May 01, 1998 at 05:26:54PM -0400, Roy Smith wrote: > Not sure if it does this already, but here's my biggest mailing list > headache... > > We have our lists (run under majordomo, which we want to get rid of) set up > that if you're not subscribed, you can't submit. This is the most obvious > anti-spam measure, and goes a good way towards cutting down abuse. Problem > is, people have multiple email addresses, or they change, or one MUA formats > is slightly differently than another MUA they use, etc. > > It would be really cool to be able to say something like "accept postings > from this person, if any of the following are on the From: line" Yes, both things are in Mailman: You can set an option that restricts non-members from posting (their posts get set aside for approval). You can list people from whom mail is always accepted without question. I don't think these two features have any sort of negative interaction. John From dragondm@nexus.Integral.org Sat May 2 03:51:50 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Fri, 1 May 1998 21:51:50 -0500 (CDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: Message-ID: On Fri, 1 May 1998, Ken Manheimer wrote: > Mr dragon, > > It looks like we've got a nice synergy here - a number of the items you > attacked are things i wanted to do, but did not yet do, while i (and now > we) have taken care of a number of other things. (Most of the big > things are mentioned in the notes i included in the distributions.) > > On Fri, 1 May 1998, The Dragon De Monsyne wrote: > > > I have made numerous changes to the code as a result. Now that I > > am aware that MailMan is being actively developed, I would like to see > > what I can do about merging these versions together. > > As barry mentioned, we're planning to make a cvs view of our version > available across the net sometime soon. I should add that we will need > to clear some admin issues first, so it's not guaranteed, but likely. Good! > > Removed all dependance on MailMan being in /home/mailman. > > I made sure all scripts were always being run as the MailMan user, > > then used the pwd module to find the current user's home directory. > > Everything else is found from there. (I did this so I could have two > > MailMan installs on one machine, a 'production' server, which runs several > > lists, and a development server) > > Relaxing the /home/mailman rigidity is high on my list! My ideal would > be to have as much as possible be in a package or use relative paths > (which have the same effect - independence from location of the root). > Unfortunately, i have a few other things on my agenda before that, so > i'm not sure when i'll get to it. (High up is gettings things like the > admin pages behind a cookied password, another item is making sure that > a single mailman install can use different addresses for the host for > different lists - os, eg, ISP clients with their own domains can use > their domains for their mailman lists...) > > (I'm confused about how working with respect to the ~mailman homedir > enables you to have to install's, btw.) I have two mailman users, 'mailman' & 'mm', and separate base urls, etc. The neccesary magic to make it all work is this line at the begining of the scripts: sys.path.append(os.path.join(pwd.getpwuid(os.geteuid())[5], "mailman","modules")) A bit ugly, but it works.( a package would be more elegant, of course. ) > > > Re-did logging system. Removed all /tmp logging. Created module that > > redirects errors to files in ~/logs Error log files are named for > > the scripts (thus errors from admin show up in ~/logs/admin.err) This is > > done automatically by importing the logging module. > > This is a place where we completely overlapped - and came up with > similar answers, at least in the abstract. I also used ~mailman/logs, > but am using a few categories of logs - bounce, vette, subscribe, > digest, and error. Error gets the errors from bugs in any of the > scripts, so you can just check the file periodically to see if anything > is malfunctioning in a way that is caught... > > I guess we'll have to reconcile the two approaches. Shouldn't be too difficult. What I did was very simple. Basically just re-directing stderr to a file in ~/logs named for sys.argv[0] The syslogish thing you have (now that I look at it) is a much more compleate solution. > > > (major) Compleatly changed the way outgoing mail gets sent. > > Did much re-writing of mm_message & mm_deliver modules. Both > > OutgoingMessage & IncomingMessage are now derived from a common base > > class, ListMessage. Sending is accomplished by calling send method of > > message object. > > Mail is no longer sent by piping to 'sendmail'! Instead, outgoing > > messages are punted to a mailserver (currently always localhost) > > via SMTP. I wrote an SMTP module, smtplib to do this. smtplib is now part > > of the python standard library as of 1.5.1 > > As barry also mentioned, this is one of the things up there on my > "desired" list - but it never was high enough urgency to percolate to > the top. (John actually sent me an smtplib.py that he was playing with, > but i never took the step of hooking it up.) I'm glad you've done this, > and that it uses something you've gotten into the distribution! This was one of my first 'big' projects with mm. It helped preformance quite a bit on my system. > > > Added random-mnemnonic password generator. Instead of passwords like > > 'qZ', AOL-like random-word passwords are generated (like > > 'cold-blue-tiger') > > That's nice! People won't have to visit the site to either select > reasonable passwords, or have their unmemorable one's sent to them... Yup. It's fairly simple (just based on whrandom) but it works nicely. > > > Added multiple subject: prefix removal and Re: compaction > > changes subjects like: > > "Re: Prefix Re: Prefix Re Something" > > to "Prefix Re[3]: Something" > > Me too, except i didn't do any munging of the Re, and in particular > leave the Re before the prefix so mail readers sort threads > appropriately. I'd rather avoid the fancy Re stuff, with the idea that > mailman should do as little intrusion as possible to the content. Yah. I had a list with it, and they kept getting "prefix Re: prefix Re: prefix ...." I munge the subject before the prefix gets put on the outgoing message, so the prefix winds up first. The problem with having the Re first, is then filtering dosen't work. > > > (major) Changed the way user data is stored (still tying up loose ends on > > this) > > Created a User class. User data is no longer stored in the > > config.db file. Instead there is a shelf full of User objects. > > The main Mailinglist object contains an instance of a UserDB class that > > manages storing, retrieving & quering this database. Most of the *User* > > methods of the Mailinglist class have become methods of the User class. > > If you're saying what i think you're saying, this is a real biggie, one > which i've dearly wanted to see happen if i didn't get to it myself. (I > think john had something like it going in a later, lost version of > mailman, and i know a number of us have expressed interest in it.) Yup. I may be tweaking the code a bit on this before diving in in earnest to try to integrate things (I have a circular ref I'd like to eliminate, and I just saw a very nice way to do so today. ) but basically my idea was this: I wanted to have hierarchy of objects here. You start with a Server object that manages all server-wide info. you can get Mailinglist objects from the Server object (like Server.GetList("foolist") ), that handle all list-wide info, and you get User objects from the Mailinglist that handle all User specific stuff. Right now, I've just finished/am finishing the User object addition, and havent gotten to the Server object yet. > > > Changed the way user options are handled. Options are referenced by name > > (a string) and can be Boolean or "multiple-choice" values. Info on > > available options & allowed values for them are available via methods > > of the User objects. I compacted the 'Digests', "DisableDelivery" and > > "EnableMIME" options into one multiple-choice "delivery_mode" option." > > Sounds cool, provided there's no scaleup-defeating performance hits. > We've got some people on this list interested in handling lists of > several thousand members... > Hmmm.... Well, I will admit, flexibility was higher in my mind than preformance, however the UserDB object that manages quering the shelf full of User objects does do some caching using dictionaries. That should make the impact minmimal. -The Dragon De Monsyne From klm@python.org Sat May 2 20:51:15 1998 From: klm@python.org (Ken Manheimer) Date: Sat, 2 May 1998 15:51:15 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: Message-ID: On Fri, 1 May 1998, The Dragon De Monsyne wrote: > On Fri, 1 May 1998, Ken Manheimer wrote: > > > (I'm confused about how working with respect to the ~mailman homedir > > enables you to have to install's, btw.) > > I have two mailman users, 'mailman' & 'mm', and separate base > urls, etc. The neccesary magic to make it all work is this line at the > begining of the scripts: Ah, a separate user. Cool. > sys.path.append(os.path.join(pwd.getpwuid(os.geteuid())[5], > "mailman","modules")) > > A bit ugly, but it works.( a package would be more elegant, of course. ) Not bad. If you look at mailman/cgi/private, andrew kuchling did something along the same lines. You implemented something more refined by getting the user id of calling process. Using relative paths wherever possible (using packages for the modules and relative paths for scripts and wrapper executables) is my favorite prospect. > Yah. I had a list with it, and they kept getting > "prefix Re: prefix Re: prefix ...." I munge the subject before the prefix > gets put on the outgoing message, so the prefix winds up first. The > problem with having the Re first, is then filtering dosen't work. Not sure what you're saying, but i implemented something here that leaves the "re" alone, but still filters fine to avoid cascading prefixes. Small hack, with no apparent problems and many messages under the bridge (or something:). > > > (major) Changed the way user data is stored (still tying up loose ends on > > > this) > > > Created a User class. User data is no longer stored in the > > > [...] > > If you're saying what i think you're saying, this is a real biggie, one > Yup. I may be tweaking the code a bit on this before diving in in earnest > [...] > this: I wanted to have hierarchy of objects here. You start with a > Server object that manages all server-wide info. you can get Mailinglist Sounds like this will also feed in well to john's planned creation of a threaded persistent server mode. It also sounds like it'll provide a good basis for another feature i really want - user profiles. The simplest use of these profiles would be to contain a user's preferred password, which a user shares across lists on a site. They'd just have to change that password, rather than going through the process for every list to which they subscribe. (List-specific passwords could still be available, so the user wouldn't *have* to use the same password for all.) Also, delivery address could become a secondary aspect of the user account, rather than the key by which it is identified. This way, the user could change the delivery address without having to unsubscribe and resubscribe. (They'd have to go through a confirmation process that proves they're in charge of the new address, to prevent mischief makers from subscribing at a valid address and then changing to some arbitrary other address.) And the profile could be used for "incidental" member info, like name, organization, etc, to provide background for the mailing list community - etc... Anyway, it all sounds real good! ken manheimer klm@python.org From klm@python.org Sat May 2 21:02:54 1998 From: klm@python.org (Ken Manheimer) Date: Sat, 2 May 1998 16:02:54 -0400 (EDT) Subject: [Mailman-developers] Web page In-Reply-To: <19980501142611.E13559@list.org> Message-ID: On Fri, 1 May 1998, Mr. + Ms. John Viega wrote: > On Fri, May 01, 1998 at 05:26:54PM -0400, Roy Smith wrote: > > Not sure if it does this already, but here's my biggest mailing list > > headache... > > > > We have our lists (run under majordomo, which we want to get rid of) set up > > that if you're not subscribed, you can't submit. This is the most obvious > > anti-spam measure, and goes a good way towards cutting down abuse. Problem > > is, people have multiple email addresses, or they change, or one MUA formats > > is slightly differently than another MUA they use, etc. > > > > It would be really cool to be able to say something like "accept postings > > from this person, if any of the following are on the From: line" > > Yes, both things are in Mailman: > > You can set an option that restricts non-members from posting (their > posts get set aside for approval). > > You can list people from whom mail is always accepted without question. > > I don't think these two features have any sort of negative interaction. I agree with roy that this feature would be useful(*). Use of the always-accepted list of addresses for this purpose doesn't quite cut it for me - it requires list-administrator intervention for what really is a user-specific option. I could imagine on a big list, with lots of subscriptions happening all the time, the list admin would be kept too busy just adding the always-accepted addrs. I suspect this will be another job for the much anticipated (:-) user profiles. ((*) I must say, the require_explicit_destination option seems to be a remarkably effective spam filter. I sometimes want to advertise it heavily (not via spam:), but then think that if everyone in the world were using it the spammers would concentrate more on the bulk mailers that specify the destinations in the headers, so i don't harp on it too much...) ken manheimer klm@python.org From klm@python.org Sat May 2 21:20:53 1998 From: klm@python.org (Ken Manheimer) Date: Sat, 2 May 1998 16:20:53 -0400 (EDT) Subject: [Mailman-developers] Found it! (I have a divergent version of MailMan) In-Reply-To: <19980501142312.D13559@list.org> Message-ID: On Fri, 1 May 1998, John Viega wrote: > > As barry also mentioned, this is one of the things up there on my > > "desired" list - but it never was high enough urgency to percolate to > > the top. (John actually sent me an smtplib.py that he was playing with, > > but i never took the step of hooking it up.) I'm glad you've done this, > > and that it uses something you've gotten into the distribution! > > Ah, I would have sworn you already did this one, Ken :) I did add your smtplib.py to the distribution. I bet my checkin message was not clear about the fact that i hadn't actually hooked it up yet. (I think my notes mention hooking it up as a todo, though.) BTW, i see a note in the dragon's 1.5.1-included smtplib.py which says it "doesn't do esmtp (yet)". I remembered seeing a pipelining esmtp module going by the newsgroup, and sure enough a deja news search shows "FREE: Python ESMTP Module by me!" from daniel robbins. Maybe this code would provide a leg up, at least as an example, on incorporating the command pipelining extension to smtplib.py? ken manheimer klm@python.org From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Sat May 2 22:09:38 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Sat, 2 May 1998 17:09:38 -0400 (EDT) Subject: [Mailman-developers] Web page References: <1895883.3103032414@qwerky.med.nyu.edu> <19980501142611.E13559@list.org> Message-ID: <13643.35665.905958.716734@anthem.CNRI.Reston.Va.US> >>>>> "MJV" == Ms John Viega writes: MJV> You can set an option that restricts non-members from posting MJV> (their posts get set aside for approval). MJV> You can list people from whom mail is always accepted without MJV> question. MJV> I don't think these two features have any sort of negative MJV> interaction. Although it might be cool, once there's a real user database, to allow users to set up aliases for themselves. I.e. their mail could come from multiple addresses. You'd still have to verify through each given address though. -Barry From klm@python.org Mon May 4 19:50:40 1998 From: klm@python.org (Ken Manheimer) Date: Mon, 4 May 1998 14:50:40 -0400 (EDT) Subject: [Mailman-developers] mailman 1.0b3 patch in the ftp dir Message-ID: <199805041850.OAA11723@glyph.CNRI.Reston.Va.US> As promised (though slightly late), i put together a mostly-fixes patch release, bringing mailman to 1.0b3. It takes care of some significant (but not showstopper) bugs that the intermediate patch i released did not do. See the README file in the ftp dir: ftp://ftp.python.org/pub/python/contrib/Network/mailman A couple of notes. First, i'm sad to say that the patch is with respect to 1.0b2, not taking into account the intermediate patch i released. When you apply it you may have to ignore some "reversed or already applied patch" prompts, and/or do some reconciliation by hand. I'm sorry about this - i am still ironing out my release techniques. (My lesson in this case was to *always* add a symbolic tag to the version control logs when you release a patch. I didn't do so for the intermediate patch, and the option of trying to correct the logs retroactively is too error prone to reliably improve matters.) This may be the last release we'll do here at python.org, now that john is nearing having time to devote to mailman maintenance, and in particular, to reactivation of the mailman web site, www.list.org. (I've set the default value for MAILMAN_URL in mm_defaults.py to that value, in anticipation of this event.) Also, i'm hoping to tone down my time on mailman a bit, to pay better attention to my proper duties (and proper weekends), and take on just some specific todos in mailman development itself. It sounds like we have the beginnings of a small cadre around the net who have serious interest in mailman development, and a start at organizing the effort to avoid overlap and ease integration. If you have development plans and want to participate, be sure to send email to john (viega@list.org) with your list of personal-priority todos, so we can have a coordinated effort here... Yay! ken manheimer, 05/04/1998 klm@python.org From klm@python.org Tue May 5 18:00:33 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 5 May 1998 13:00:33 -0400 (EDT) Subject: [Mailman-developers] Re: anyone got mailman running on freebsd ? In-Reply-To: <6il5oh$l0p$1@nnrp1.dejanews.com> References: <6il5oh$l0p$1@nnrp1.dejanews.com> Message-ID: <13647.9309.893202.35064@glyph.CNRI.Reston.Va.US> A message in the python newsgroup points out a likely portability problem with mailman under FreeBSD, in particular with file locking mechanisms. sweeting@neuronet.com.my writes: > Sorry to trouble you but I've run into a problem setting up > mailman (python mailling list software) on FreeBSD and wonder > if there are any special config for FBSD ? > [...] > However, accessing any of the URLs to manage the lists > returned server config errors. The apache logs said were > almost as unhelpful : > > [Tue May 5 03:10:07 1998] access to /home/mailman/cgi-bin/listinfo failed for > 202.184.153.17, reason: Premature end of script headers > [Tue May 5 03:10:18 1998] access to /home/mailman/cgi-bin/admin failed for > 202.184.153.17, reason: Premature end of script headers (Note for mailman-developers, including me - it may be worth noting in the troubleshooting section of the README about Apache (and other?) error log notices something to the effect of "premature end of script headers" meaning that the script in question probably bombed before producing any, or sufficient, output. And, it occurs to me, it may be worth breaking out the troubleshooting section of the README into a separate file, and eventually as a section of a prospective admin guide.) > Trying to interact from the python interpreter also > produced errors : > >>> import sys > [...] > File "/usr/local/lib/python1.5/posixfile.py", line 190, in lock > flock = fcntl.fcntl(self._file_.fileno(), cmd, flock) > IOError: (22, 'Invalid argument') We mailman developers need to look into this issue, in particular guido's suggestion that: Guido: > A much more portable way to do file locking is to use fcntl.lockf()! Clearly we need some fairly portable solution, and it probably makes sense to tackle this now. Barry has an article by jamie zawinski pointing out the pitfalls of locking, with a conclusion (i gather) that use of a directory (with a leading '.' dot in its name) is the best portable answer. I'd be interested to hear whether anyone has a fairly clear-win solution - one that's simple and portable - and/or whether just switching over to fcntl.lockf() is the right thing. Maybe andrew's discussions (in that thread) about revision of the posixfile interfaces should play in here... Ken From akuchlin@cnri.reston.va.us Tue May 5 19:31:18 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Tue, 5 May 1998 14:31:18 -0400 (EDT) Subject: [Mailman-developers] Re: anyone got mailman running on freebsd ? In-Reply-To: <13647.9309.893202.35064@glyph.CNRI.Reston.Va.US> References: <6il5oh$l0p$1@nnrp1.dejanews.com> <13647.9309.893202.35064@glyph.CNRI.Reston.Va.US> Message-ID: <13647.23013.946909.814142@newcnri.cnri.reston.va.us> Ken Manheimer writes: >Clearly we need some fairly portable solution, and it probably makes >sense to tackle this now. Barry has an article by jamie zawinski >pointing out the pitfalls of locking, with a conclusion (i gather) >that use of a directory (with a leading '.' dot in its name) is the >best portable answer. I'd be interested to hear whether anyone has a >fairly clear-win solution - one that's simple and portable - and/or >whether just switching over to fcntl.lockf() is the right thing. >Maybe andrew's discussions (in that thread) about revision of the >posixfile interfaces should play in here... The person sent me the relevant header file; the flock structure is: struct flock { off_t l_start; /* starting offset */ off_t l_len; /* len = 0 means until end of file */ pid_t l_pid; /* lock owner */ short l_type; /* lock type: read/write, etc. */ short l_whence; /* type of l_start */ }; I don't have a FreeBSD machine to fiddle with, so I can't really help suggest a fix. The struct format in posixfile.py is: if sys.platform in ('netbsd1', 'freebsd2', 'freebsd3'): flock = struct.pack('lxxxxlxxxxlhh', \ l_start, l_len, os.getpid(), l_type, l_whence) Perhaps the xxxx padding is unnecessary or something, but a FreeBSD person will have to look into it. For Python, probably the best solution is to change posixfile.py to use lockf(), and possibly change the interface in other ways. Best to discuss this on the main list. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ But I'm getting too old for the trick, which troubles me, for the sea is in my blood like a fever and I don't know how I can leave. -- Jim's envoi, in SANDMAN #53: "Hob's Leviathan" From klm@python.org Tue May 5 20:28:28 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 5 May 1998 15:28:28 -0400 (EDT) Subject: [Mailman-developers] More mailman locking issues Message-ID: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> Another issue with mailman locking - i think it's coarse grained, and needs to be refined. For example, this morning someone noticed that the umbrella listinfo page on python.org was hanging, because, it turns out, one of the lists was left in a locked state. (Not sure how that happened - it usually doesn't.) Well, the umbrella listinfo (and admin) pages open the lists only for the sake of determining which are public ones - no writing is actually required, so a lock isn't really required. It might be sufficient to offer a way to open a list "read-only", such that no locking is done. I think it would be better, however, to refine the locking system such that lists are not locked until they enter an operation that changes the list data in a way that will eventually need to be written. This would probably take a more effort than a "read-only" kluge, but may not be too bad. I guess this is one for the todo list. (Wish i had gobs of time to continue to hack on a these sorts of things - they're not very big or hard, but there is a bunch of them, and they need to be done right.) Ken From viega@list.org Tue May 5 20:37:31 1998 From: viega@list.org (John Viega) Date: Tue, 5 May 1998 12:37:31 -0700 Subject: [Mailman-developers] More mailman locking issues In-Reply-To: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US>; from Ken Manheimer on Tue, May 05, 1998 at 03:28:28PM -0400 References: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> Message-ID: <19980505123731.A12842@list.org> Well, if one of the lists was left unlocked, there was probably a bug... Anyway, WRT. locking, I think that several things need to happen minimum: 1) Locks should insert a time stamp. After x seconds (30?) you can lose the lock. Anything such as the archiving software should use that time to copy the archive over to a tmp file and nulling out the old file so that other procs can overwrite (Not that hard). 2) A version of Load() should be written that doesn't call Lock() (call it LoadUnlocked() I guess). Change things that don't care to use it (Even easier). 3) Make sure Lock() uses portable calls (might require a bit of research). I can look into this stuff next week, but if anyone wants to do it sooner, let us know and feel free... John On Tue, May 05, 1998 at 03:28:28PM -0400, Ken Manheimer wrote: > Another issue with mailman locking - i think it's coarse grained, and > needs to be refined. For example, this morning someone noticed that > the umbrella listinfo page on python.org was hanging, because, it > turns out, one of the lists was left in a locked state. (Not sure how > that happened - it usually doesn't.) Well, the umbrella listinfo (and > admin) pages open the lists only for the sake of determining which are > public ones - no writing is actually required, so a lock isn't really > required. > > It might be sufficient to offer a way to open a list "read-only", such > that no locking is done. I think it would be better, however, to > refine the locking system such that lists are not locked until they > enter an operation that changes the list data in a way that will > eventually need to be written. This would probably take a more effort > than a "read-only" kluge, but may not be too bad. > > I guess this is one for the todo list. (Wish i had gobs of time to > continue to hack on a these sorts of things - they're not very big or > hard, but there is a bunch of them, and they need to be done right.) > > Ken > > ------------------------------------------------------ > Mailman-developers maillist - Mailman-developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Tue May 5 20:45:47 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 5 May 1998 15:45:47 -0400 (EDT) Subject: [Mailman-developers] More mailman locking issues References: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> Message-ID: <13647.26901.437384.156650@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> It might be sufficient to offer a way to open a list KM> "read-only", such that no locking is done. I think it would KM> be better, however, to refine the locking system such that KM> lists are not locked until they enter an operation that KM> changes the list data in a way that will eventually need to be KM> written. This would probably take a more effort than a KM> "read-only" kluge, but may not be too bad. Of course, much of these file-locking problems go away once you have a long-lived mailman server. Then again, you'll have threading issues to deal with :-). I haven't looked at what it is that Mailman is actually trying to lock (too busy interspersing autoconf hacking time in with Real Work), but I'm guess from what I've read that you're just trying to keep multiple writers from clobbering each other (and readers from reading inconsistent data when a writer is writing). It sounds like you don't really need to lock the file, but instead to lock out access to a shared resource, and you want this to work across NFS. I wasn't able to dig up the Netscape Mail + File Locking article that Jamie Zawinski wrote. I don't seem to have it in my bookmarks or easily grepable and a web search turned up only stale links. I emailed Jaime to see if he's got an updated link and will forward that if I get a response. If I remember correctly, the article basically justifies the point of view that the best portable way to do resource locking across NFS is by creating a file with O_CREAT | O_EXCL. That's what's meant by "dot-file" locking because you usually create a .file that is mostly invisible. Since the check-for-existance and file-creation steps are atomic (on Solaris at least), you won't every have two processes step on each other. The additional advantage is that just rm'ing the dot-file is enough to unlock the system. The disadvantage is that this operation may not be atomic for older versions of NFS, although I believe it is for NFS 3 (can't find a reference to this right now though). On the other hand, creation of directories *is* atomic across all versions of NFS, AFAIK, so perhaps it makes more sense to create a dot-directory that locks the resource. You'd have to use mkdir() and check for EEXIST. You'd basically busy-loop in either case waiting for the lock to be given up. So creating a dot-directory representing a lock is probably the most portable, safest, and easiest way to do it. -Barry From tismer@appliedbiometrics.com Tue May 5 21:16:49 1998 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Tue, 05 May 1998 22:16:49 +0200 Subject: [Mailman-developers] More mailman locking issues References: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> <13647.26901.437384.156650@anthem.CNRI.Reston.Va.US> Message-ID: <354F73B1.F4DD56B5@appliedbiometrics.com> Barry A. Warsaw wrote: [a lot :] > On the other hand, creation of directories *is* atomic across all > versions of NFS, AFAIK, so perhaps it makes more sense to create a > dot-directory that locks the resource. You'd have to use mkdir() and > check for EEXIST. You'd basically busy-loop in either case waiting > for the lock to be given up. > > So creating a dot-directory representing a lock is probably the > most portable, safest, and easiest way to do it. Another easy way is to simply use the "lockfile" utility which comes with procmail. Procmail has its own install script to check for the locking abilities of the particular installation and does intense testing, and I never got any problems with it when I was using smartlist, which is a procmail based minidomo. (BTW, reading the smartlist source can cause serious damage to your brain, it is write-only-coded procmail and shell). If one needs a quick and immediately working solution, I can recommend this, at least give it a slot in your "tips&tricks" box. -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaiserin-Augusta-Allee 101 : *Starship* http://starship.skyport.net 10553 Berlin : PGP key -> http://pgpkeys.mit.edu we're tired of banana software - shipped green, ripens at home From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed May 6 20:36:53 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 6 May 1998 15:36:53 -0400 (EDT) Subject: [Mailman-Developers] Re: [Mailman-CVS] CVS: 'mailman/modules mm_digest.py' References: <13643.35405.507841.64622@anthem.CNRI.Reston.Va.US> Message-ID: <13648.47311.397569.686639@anthem.CNRI.Reston.Va.US> [Ken, I hope you don't mind me changing the follow up to the developers list. Folks, this thread started in response to a checkin that strips certain headers... -BAW] >>>>> "KM" == Ken Manheimer writes: >> Digest messages now keep all headers except 'received', >> 'errors-to', and all 'x-*' ones (and any continuations of >> these). >> should x-* headers be stripped? the user can use them to add >> interesting tidbits that probably ought to be preserved. what >> is the rationale as to which headers get stripped and which >> remain? KM> If you take a look at a non-mime digest (or a mime digest in a KM> reader that doesn't minimize the headers to just essential KM> ones) you'll see that it doesn't take much to have the headers KM> outweigh, and obscure, the messages. My sense is that KM> sticking to official headers - and not all of them at that - KM> is all that can be justified. The default settings for most KM> mail readers, minimizing out even things like organization and KM> sender, gives credence to this view. And people reading the KM> non-mime digests don't have the option of mimimizing the KM> headers display... In general, I like a policy kind of like the Hippocratic oath of mailers, "above all do minimal munging". I agree with you that headers may end up outweighing content in RFC934 digests, so there has to be tradeoffs. I personally don't much like digest mailing lists, so maybe I'm not the right person to say, but I also am a little uncomfortable that digest subscribers get a different message than non-digest subscribers. My mental model is that the digest is just the sum of all the message a non-digest subscriber would see, packaged conveniently, and sent out less frequently. If I were to view a MIME digest in a MIME savvy MUA (e.g. VM/XEmacs), I could unpack the digest and view the individual articles, and it would look no different than if I was a non-digest reader. Perhaps this can't be acheived in practice, and maybe it really does make sense to strip or change some headers. But I guess I don't like the idea of stripping out headers that users take special pains to craft, like Reply-to and most X-* headers (and not just 'cause I like the ones I add). I don't care if most MUA hide this stuff. Those headers can be useful! I know that VM's author (Kyle Jones) takes special care to include as many headers as possible when creating digests, precisely because those headers are occasionally useful. Maybe there's a distinction to be made b/w MIME and non-MIME digests? I can see more of a rationale for stripping headers in RFC934 style digests. -Barry From viega@list.org Thu May 7 03:06:37 1998 From: viega@list.org (John Viega) Date: Wed, 6 May 1998 19:06:37 -0700 Subject: [Mailman-Developers] Big announcement! Message-ID: <19980506190637.A24164@list.org> I've got a few things to announce. First, I've come up w/ a big Mailman TODO list that is now on the web. If you want to tackle something off that list, just let this mailing list know, and go at it! The TODO list is at: http://www.list.org/todo.html Please submit additions, corrections, etc. to me. Second, as of our next release, Mailman will be released under the Gnu GPL. Third, Mailman is now GNU software. From the next release forward, it will be distributed by the FSF. There will be a second web page on www.gnu.org, which will almost certainly be a mirror of the list.org web page. We're going to wait around 3-4 weeks before our next release. We want to give everyone time to get their work incorporated into this first big release (at which time, we'll advertise the software fairly heavily on the net). John From tdyas@remus.rutgers.edu Thu May 7 06:29:36 1998 From: tdyas@remus.rutgers.edu (Tom Dyas) Date: Thu, 7 May 1998 01:29:36 -0400 (EDT) Subject: [Mailman-Developers] Big announcement! In-Reply-To: <19980506190637.A24164@list.org> Message-ID: On Wed, 6 May 1998, John Viega wrote: > I've got a few things to announce. First, I've come up w/ a big > Mailman TODO list that is now on the web. If you want to tackle > something off that list, just let this mailing list know, and go at > it! > > The TODO list is at: http://www.list.org/todo.html > > Please submit additions, corrections, etc. to me. What about support for a "confirm" command similar to listserv? While the password scheme is nice, it seems possible for a bad person to spoof an email from an unsuspecting user with the password in the command after they have used the WWW interface. FYI, I help run the Linux development lists on vger.rutgers.edu in my spare time. I have seen _many_ forged requests to Majordomo that come in as requests for approval. If mailman were to send out a unique value determined only by it and only sent to the subscriber, then there is a much better guarantee that it was the subscriber who confirmed the subscription request. An secondary feature would be to send out a confirmation request to all current subscribers for them to stay on the list like once a month. The Bugtraq list does this using ListServ. While not many lists would use it, it would be nice. I can hack on some of this but are there any relevant parts of mailman I should consider for this? I have programmed in Python before so I just need pointers to right files and such. Btw, I am going to try out mailman on a CVS commits list (vger cvs is home to linux/sparc) that I am going to make on vger. Vger uses the zmailer MTA. Is there any way to make mailman use an "outgoing"-style list alias? Zmailer is very good about sending out mail if it has all of the destinations at once since it can group nearby domains and such to send to exploder sites. Tom From scott@chronis.pobox.com Thu May 7 06:38:11 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 7 May 1998 01:38:11 -0400 Subject: [Mailman-Developers] Big announcement! In-Reply-To: ; from Tom Dyas on Thu, May 07, 1998 at 01:29:36AM -0400 References: <19980506190637.A24164@list.org> Message-ID: <19980507013811.03723@chronis.icgroup.com> On Thu, May 07, 1998 at 01:29:36AM -0400, Tom Dyas wrote: | On Wed, 6 May 1998, John Viega wrote: | | > I've got a few things to announce. First, I've come up w/ a big | > Mailman TODO list that is now on the web. If you want to tackle | > something off that list, just let this mailing list know, and go at | > it! | > | > The TODO list is at: http://www.list.org/todo.html | > | > Please submit additions, corrections, etc. to me. | | What about support for a "confirm" command similar to listserv? While the | password scheme is nice, it seems possible for a bad person to spoof an | email from an unsuspecting user with the password in the command after | they have used the WWW interface. this is in the works by me. if i understand correctly, the consensus was to require confirmations. i have them working in an offshoot branch of mailman, and plan to put together and post patches against the most recent distribution next week sometime. in the meantime, sorry, i'm real busy with other stuff. | FYI, I help run the Linux development lists on vger.rutgers.edu in my | spare time. I have seen _many_ forged requests to Majordomo that come in | as requests for approval. If mailman were to send out a unique value | determined only by it and only sent to the subscriber, then there is a | much better guarantee that it was the subscriber who confirmed the | subscription request. that's just what it will do. | An secondary feature would be to send out a confirmation request to all | current subscribers for them to stay on the list like once a month. The | Bugtraq list does this using ListServ. While not many lists would use it, | it would be nice. hmmm, seems like an idea. what do you do with the members who don't respond in 2 weeks because they're on vacation? | I can hack on some of this but are there any relevant parts of mailman I | should consider for this? I have programmed in Python before so I just | need pointers to right files and such. i've already got it done, and just need to put it to the newest release. no reason to be redundant about development... | Btw, I am going to try out mailman on a CVS commits list (vger cvs is home | to linux/sparc) that I am going to make on vger. Vger uses the zmailer | MTA. Is there any way to make mailman use an "outgoing"-style list alias? | Zmailer is very good about sending out mail if it has all of the | destinations at once since it can group nearby domains and such to send to | exploder sites. not that i know with the addresses per se, but you can set the maximum outgoing connections for each list. for vmailer, it would probably be good to just have that number be <= the size of the list. same effect. BTW. vmailer is great, can't wait till it goes beta... scott From tdyas@remus.rutgers.edu Thu May 7 07:01:41 1998 From: tdyas@remus.rutgers.edu (Tom Dyas) Date: Thu, 7 May 1998 02:01:41 -0400 (EDT) Subject: [Mailman-Developers] Big announcement! In-Reply-To: <19980507013811.03723@chronis.icgroup.com> Message-ID: On Thu, 7 May 1998, Scott wrote: > | What about support for a "confirm" command similar to listserv? While the > | password scheme is nice, it seems possible for a bad person to spoof an > | email from an unsuspecting user with the password in the command after > | they have used the WWW interface. > > this is in the works by me. if i understand correctly, the consensus > was to require confirmations. i have them working in an offshoot > branch of mailman, and plan to put together and post patches against > the most recent distribution next week sometime. in the meantime, Cool. > | An secondary feature would be to send out a confirmation request to all > | current subscribers for them to stay on the list like once a month. The > | Bugtraq list does this using ListServ. While not many lists would use it, > | it would be nice. > > hmmm, seems like an idea. what do you do with the members who don't > respond in 2 weeks because they're on vacation? I have only seen Bugtraq actually use this feature. It is a very large list with a reasonable amount of traffic. I think for such lists the "you lose" policy is in effect for such people since they can always resubscribe when they come back and can view a WWW archive for info they missed. > not that i know with the addresses per se, but you can set the maximum > outgoing connections for each list. for vmailer, it would probably be > good to just have that number be <= the size of the list. same > effect. > > BTW. vmailer is great, can't wait till it goes beta... vmailer? I could probably just write a script that runs out of cron every 15 minutes that checks to see if any *-outgoing files need to be regenerated. Tom From viega@list.org Thu May 7 13:10:40 1998 From: viega@list.org (John Viega) Date: Thu, 7 May 1998 05:10:40 -0700 Subject: [Mailman-Developers] Big announcement! In-Reply-To: ; from Tom Dyas on Thu, May 07, 1998 at 01:29:36AM -0400 References: <19980506190637.A24164@list.org> Message-ID: <19980507051040.A27198@list.org> On Thu, May 07, 1998 at 01:29:36AM -0400, Tom Dyas wrote: > What about support for a "confirm" command similar to listserv? While the > password scheme is nice, it seems possible for a bad person to spoof an > email from an unsuspecting user with the password in the command after > they have used the WWW interface. If I understand you correctly, what you're looking for is there. A confirmation step was added recently. I think it needs to be improved upon to the point where you can just reply to the message to confirm (as long as some string is in the subject, say). For things other than subscribe, the password is necessary from the web, and the email address is checked from mail commands (I don't think the password is). > An secondary feature would be to send out a confirmation request to all > current subscribers for them to stay on the list like once a month. The > Bugtraq list does this using ListServ. While not many lists would use it, > it would be nice. Sure, that's a good idea for large lists. I think you should be able to set the period, though. I run a few large lists where I'd like to do that 2 times a year only. I'll add it to the todo list today. > I can hack on some of this but are there any relevant parts of mailman I > should consider for this? I have programmed in Python before so I just > need pointers to right files and such. Well, the core of mailman is in the modules directory. The maillist file is perhaps the best place to start; subsystems such as digest handling and bounce detection are in other files in that directory, and are included by multiple inheritance. The commands that interact with the email addresses, the cgi scripts, etc. All live in different subdirectories. They all end up instantiating mailing lists to do their work. > Btw, I am going to try out mailman on a CVS commits list (vger cvs is home > to linux/sparc) that I am going to make on vger. Vger uses the zmailer > MTA. Is there any way to make mailman use an "outgoing"-style list alias? > Zmailer is very good about sending out mail if it has all of the > destinations at once since it can group nearby domains and such to send to > exploder sites. I don't know if I understand this one. Do you want mylist-outgoing to be the sender address as opposed to mylist-admin? John From dragondm@nexus.Integral.org Thu May 7 14:52:52 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Thu, 7 May 1998 08:52:52 -0500 (CDT) Subject: [Mailman-Developers] Big announcement! In-Reply-To: <19980507051040.A27198@list.org> Message-ID: On Thu, 7 May 1998, John Viega wrote: > On Thu, May 07, 1998 at 01:29:36AM -0400, Tom Dyas wrote: > > > Btw, I am going to try out mailman on a CVS commits list (vger cvs is home > > to linux/sparc) that I am going to make on vger. Vger uses the zmailer > > MTA. Is there any way to make mailman use an "outgoing"-style list alias? > > Zmailer is very good about sending out mail if it has all of the > > destinations at once since it can group nearby domains and such to send to > > exploder sites. > > I don't know if I understand this one. Do you want mylist-outgoing to > be the sender address as opposed to mylist-admin? I think what he is talking about is working the way majordomo does, with an alias file with all the subscriber names in it, so the MTA can work various sorting & delivery magic. to Tom: I think that kind of mode of operation would be troublesome, fortunately, there is another way. I have a divergent version of Mailman 1.0b1 that has been modifed to send outgoing mail by punting it to the local mailserver via SMTP. This gives the MTA all the info it needs to do various sorting/routing magic. I intend to sit down someday soon, and see what I can do to integrate this back into the main Mailman source tree. -The Dragon De Monsyne From klm@python.org Thu May 7 17:30:54 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 7 May 1998 12:30:54 -0400 (EDT) Subject: [Mailman-Developers] Re: Todo list questions In-Reply-To: <19980506160759.A23032@list.org> Message-ID: [Greets. The following message comes out of a side discussion with barry and john, but it contains some substantial explanations of a few issues that i think will interest at least some others, so i'm cc'ing the entire list.] On Wed, 6 May 1998, John Viega wrote: > I'm looking at merging the TODO lists. I have questions. > > You say that address matching isn't working, giving the example > klm@parrot.python.org should match klm@python.org. I wrote code to do > that sort of matching, and it used to work great. Is it not still > there? Or does it not work for some functions? Or what? It is there, and it does happen to work for that particular example. However, it does not accept some that the original did. In particular, the original version was much too accepting on one hand: oldmm_utils.AddressesMatch('klm@perl.org', 'klm@python.org') => 1 because of the matching ".org". Likewise: oldmm_utils.AddressesMatch('klm@kgb.ukraine.ru', 'klm@politech.ukraine.ru') => 1 This was actually a serious problem - before, people who had the same login name as someone already on the list, and who happened to have the same top level domain, were simply refused subscription with a "you're already subscribed" message. That (and tracebacks when either of the arguments was an unqualified login names, eg 'klm' instead of 'klm@python.org') prompted me to make my revisions. My new version does the right thing when the account names are the same and one domain contains the other: mm_utils.AddressesMatch('klm@parrot.python.org', 'klm@python.org') => 1 What my version does *not* do is accept sibling hosts in a domain as the same email addr: mm_utils.AddressesMatch('klm@parrot.python.org', 'klm@larch.python.org') => 0 Of course, this is the right thing when we're talking about the top level domain. However, it's open for debate whether we want to accept common account names on sibling hosts within a local naming authority (second level domain, eg 'python.org') as being the same account. My read of the situation is that matching there might typically be correct for small domains (like python.org or cnri.reston.va.us) and *not* correct for moderate or large ones (eg, sun.com or reston.va.us or ukraine.ru). So i opted for the conservative approach - accept matches for domain containment, but not sibling hosts within a domain. I expect there's more to be exploited in the specifics of the addresses. Then again, i could make a table of cases with items that have the same structure and which should be accepted as matches in some cases but not in others. So i'm not convinced that this is a sufficient mechanism for, eg, recognizing postings from maillist members - i think we would need a mechanism where members explicitly designate alternate posting addresses. In any case, we need to think about it more - i think that's what i said on my todo list entry... > What would a "which" mailcmd do? Tell the user all the public maillists of which they are a member. (It would also mention that there are private maillist memberships, if any, and the command might could take an optional password, which would be used to include any of the private maillists which that password qualifies.) > Isn't the confirmation code already in place? (BTW, if I haven't said When 'web_subscribe_requires_confirmation' is set to "Requestor confirms via email", which is what we're talking about, then a message is sent to the delivery address the prospective member specified. That subscription-confirmation message is really just a subscription template - instructions for subscribing, with the exact subscription line that they can just send back to the reply address. As such, i don't really consider it a confirmation mechanism, really it's just a customized (with the users selected password) template for email subscription. This is fine when the desired delivery address is the same as the address the user would be sending the "confirmation". Where it breaks down is when the user's delivery address is different than the address from which they send their "confirmation" (subscription). If we allowed a user to subscribe for delivery to an arbitrary address, then we're fucked - mischief makers can wreak havoc. That's why we don't. What we would need to do that is a real subscription confirmation mechanism, where a modest random string (a nonce) is sent to the delivery address. The user can send back that string from any address, proving that they have control over (really, just access to - but that's the best we can do, anyway) the delivery address. The problem is that we currently have no infrastructure for tracking dispatched nonces - the "confirmation" mechanism is currently stateless, which is why i keep putting the "confirmation" in quotes. I was interested in doing something like that, but scott cotton seemed game to do something like it, so i layed out the situation and he said he'd do it (and the membership management stuff). Apparently he has, but couldn't offer patches, only an entire new copy of the system (including other changes, as well), and i haven't had the time or energy to delve into it to unravel the changes. This is what he's been referring to the last couple of days, when people bring up these issues, when saying he's going to do it when he gets a moment (or two:) to do it. This nice thing is, the specific goal scott and i initially discussed was alternate-address subscription confirmation, where a user could subscribe to a delivery address (eg, klm@acm.org) from a different address (eg, klm@python.org). I expect this will be useful for this other issue i was talking about yesterday, enlistment of redistribution addresses. The only problem is that it requires the delivery address to receive the nonce, and some redistribution addresses may already be active and populated, which would require the redistribution administrator to either intercept the confirmation request, or just let it go to their redistribution audience. Oh well. The upshot is that the current requestor-confirms-via-email mechanism isn't really a confirmation mechanism, it's a deliver-instructions-for- subscribing-via-email mechanism. And it looks like we're getting a real confirmation mechanism, soon. > it yet, the confirmation mechanism needs to be a LOT easier to use. > All you should have to do is hit reply. Novices aren't going to get > it to work the way it currently is.) Yes, you did mention this, and my understanding of that is that the user should be able to just hit reply and send the resulting message. (This will apply equally for a real, nonce-based confirmation mechanism.) This should be fairly easy to do, i just haven't yet wanted to spend time thinking about how to identify significant pitfalls, and avoid them. There are some. For example, how about when someone hits reply, cites the entire confirmation template, but adds at the top or bottom "What the hell is going on, i never asked to subscribe, why did i get sent this?!" It would be Not Good (tm) if our mechanism saw the subscription confirmation message cited in its entirety and took it as confirmation, ay? You-thought-you-were-asking-simple-questions-ay'ly yers, Ken From viega@list.org Thu May 7 17:52:19 1998 From: viega@list.org (John Viega) Date: Thu, 7 May 1998 09:52:19 -0700 Subject: [Mailman-Developers] Re: Todo list questions In-Reply-To: ; from Ken Manheimer on Thu, May 07, 1998 at 12:30:54PM -0400 References: <19980506160759.A23032@list.org> Message-ID: <19980507095219.A28772@list.org> On Thu, May 07, 1998 at 12:30:54PM -0400, Ken Manheimer wrote: > It is there, and it does happen to work for that particular example. > However, it does not accept some that the original did. In particular, > the original version was much too accepting on one hand: > > oldmm_utils.AddressesMatch('klm@perl.org', > 'klm@python.org') => 1 That would be a bug, if so. It was intended to *require* the last two names to match. > because of the matching ".org". Likewise: > > oldmm_utils.AddressesMatch('klm@kgb.ukraine.ru', > 'klm@politech.ukraine.ru') => 1 I never really cosidered that one, but it should be easy to fix :) > Of course, this is the right thing when we're talking about the top > level domain. However, it's open for debate whether we want to accept > common account names on sibling hosts within a local naming authority > (second level domain, eg 'python.org') as being the same account. I would say it should be an option defaulting to yes. > My read of the situation is that matching there might typically be > correct for small domains (like python.org or cnri.reston.va.us) and > *not* correct for moderate or large ones (eg, sun.com or reston.va.us or > ukraine.ru). So i opted for the conservative approach - accept matches > for domain containment, but not sibling hosts within a domain. Well, take advantage of the info you know. In foreign addresses, require 3 matches. In .us addresses, require 4. For sun.com, I would leave it at two. It's a single company, they tend to have aliases like jim.gosling@sun.com, which should match @foof.engr.sun.com. I have never really heard of anyones having big problems w/ majordomo lists that do this sort of fuzzy matching. I think it makes things a lot easier on people who forget which machine they were on when they subscribed, etc... (which I do all the time). > for, eg, recognizing postings from maillist members - i think we would > need a mechanism where members explicitly designate alternate posting > addresses. I think that's useful no matter what, but not a very high priority. > > What would a "which" mailcmd do? > > Tell the user all the public maillists of which they are a member. (It > would also mention that there are private maillist memberships, if any, > and the command might could take an optional password, which would be > used to include any of the private maillists which that password > qualifies.) Ah, ok. > What we would need to do that is a real subscription confirmation > mechanism, where a modest random string (a nonce) is sent to the > delivery address. The user can send back that string from any address, > proving that they have control over (really, just access to - but that's > the best we can do, anyway) the delivery address. Right, that's what I had in mind... > The problem is that we currently have no infrastructure for tracking > dispatched nonces - the "confirmation" mechanism is currently stateless, > which is why i keep putting the "confirmation" in quotes. Ok. > I was interested in doing something like that, but scott cotton seemed > game to do something like it, so i layed out the situation and he said > he'd do it (and the membership management stuff). Apparently he has, > but couldn't offer patches, only an entire new copy of the system > (including other changes, as well), and i haven't had the time or energy > to delve into it to unravel the changes. This is what he's been > referring to the last couple of days, when people bring up these issues, > when saying he's going to do it when he gets a moment (or two:) to do > it. Ok. > The upshot is that the current requestor-confirms-via-email mechanism > isn't really a confirmation mechanism, it's a deliver-instructions-for- > subscribing-via-email mechanism. And it looks like we're getting a real > confirmation mechanism, soon. Well, if it's not a confirmation mechanism of some sort, then why was it added in the first place? > For example, how about when someone hits reply, cites the entire > confirmation template, but adds at the top or bottom "What the hell is > going on, i never asked to subscribe, why did i get sent this?!" It > would be Not Good (tm) if our mechanism saw the subscription > confirmation message cited in its entirety and took it as confirmation, > ay? Well, if the title is: REPLY TO CONFIRM YOUR SUBSCRIPTION [SOMECODE#] Then you would hope people won't make that mistake. I'm sure some people will nonetheless :) John From scott@chronis.pobox.com Thu May 7 19:46:55 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 7 May 1998 14:46:55 -0400 Subject: [Mailman-Developers] Re: Todo list questions In-Reply-To: <19980507095219.A28772@list.org>; from John Viega on Thu, May 07, 1998 at 09:52:19AM -0700 References: <19980506160759.A23032@list.org> <19980507095219.A28772@list.org> Message-ID: <19980507144655.16636@chronis.icgroup.com> my personal vote on this is to have the confirmation set up so that you can just reply to it to trigger subscription. scott On Thu, May 07, 1998 at 09:52:19AM -0700, John Viega wrote: | > For example, how about when someone hits reply, cites the entire | > confirmation template, but adds at the top or bottom "What the hell is | > going on, i never asked to subscribe, why did i get sent this?!" It | > would be Not Good (tm) if our mechanism saw the subscription | > confirmation message cited in its entirety and took it as confirmation, | > ay? | | Well, if the title is: REPLY TO CONFIRM YOUR SUBSCRIPTION [SOMECODE#] | | Then you would hope people won't make that mistake. | I'm sure some people will nonetheless :) | | John | | ------------------------------------------------------ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers From scott@chronis.pobox.com Thu May 7 19:48:25 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 7 May 1998 14:48:25 -0400 Subject: [Mailman-Developers] Re: Todo list questions Message-ID: <19980507144825.54162@chronis.icgroup.com> On Thu, May 07, 1998 at 09:52:19AM -0700, John Viega wrote: | On Thu, May 07, 1998 at 12:30:54PM -0400, Ken Manheimer wrote: [stuff about address matching] just wanted to put in my $.02 about address matching. i don't think any special mechanism other than string.lower(string.strip(address1)) == string.lower(string.strip(address2)) should be used. therer are some instances , for example, 2 of my own address, where it is perfectly valid for user@furtherqualified.domain.dom to be a different address than user@domain.dom. in addition when you start taking into account how many parts you should compare based on individual top level domains, you're hard wiring knowledge that can change daily. my vote is to have addresses match only if they are the same username and have the same doman part, case folded. scott From viega@list.org Thu May 7 19:53:30 1998 From: viega@list.org (John Viega) Date: Thu, 7 May 1998 11:53:30 -0700 Subject: [Mailman-Developers] Re: Todo list questions Message-ID: <19980507115330.B29602@list.org> On Thu, May 07, 1998 at 02:43:24PM -0400, Scott wrote: > just wanted to put in my $.02 about address matching. i don't think > any special mecahnsim other than > > string.lower(string.strip(address1)) == > string.lower(string.strip(address2)) should be used. therer are some > instances , for example, 2 of my own address, where it is perfectly valid > for user@furtherqualified.domain.dom to be a different address than > user@domain.dom. > > in addition when you start taking into account how many parts you > should compare based on individual top level domains, you're hard > wiring knowledge that can change daily. > > my vote is to have addresses match only if they are the same username > and have the same doman part, case folded. > Well, I agree with you, but I know people who will not. Do you really think it's a bad idea to let the site admin change it if he prefers? John From tismer@appliedbiometrics.com Thu May 7 20:16:41 1998 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Thu, 07 May 1998 21:16:41 +0200 Subject: [Mailman-Developers] Re: Todo list questions References: <19980507115330.B29602@list.org> Message-ID: <35520899.D1F01C9D@appliedbiometrics.com> John Viega wrote: > > On Thu, May 07, 1998 at 02:43:24PM -0400, Scott wrote: > > > just wanted to put in my $.02 about address matching. i don't think > > any special mecahnsim other than > > > > string.lower(string.strip(address1)) == > > string.lower(string.strip(address2)) should be used. therer are some > > instances , for example, 2 of my own address, where it is perfectly valid > > for user@furtherqualified.domain.dom to be a different address than > > user@domain.dom. > > > > in addition when you start taking into account how many parts you > > should compare based on individual top level domains, you're hard > > wiring knowledge that can change daily. > > > > my vote is to have addresses match only if they are the same username > > and have the same doman part, case folded. > > > > Well, I agree with you, but I know people who will not. Do you really > think it's a bad idea to let the site admin change it if he prefers? If that feature is there why not. But I don't believe there is an urgent need for this. Wouldn't it be the more general solution to allow for an alias list which explicitly says which addresses are considered equal? We would define multiple ingoing addresses and one outgoing which is the "nominal" address. Since there are many special cases (Aaron Watters used to use many completely different addresses, for istance), the domain mangling would not solve all requirements, and this one is needed anyway. ciao - pirx -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaiserin-Augusta-Allee 101 : *Starship* http://starship.skyport.net 10553 Berlin : PGP key -> http://pgpkeys.mit.edu we're tired of banana software - shipped green, ripens at home From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri May 8 22:26:22 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 8 May 1998 17:26:22 -0400 (EDT) Subject: [Mailman-Developers] Re: Todo list questions References: <19980506160759.A23032@list.org> Message-ID: <13651.30182.475430.18450@anthem.CNRI.Reston.Va.US> <> I wonder if DNS MX records could be used to help out here? You could strip lefthand components until you find an MX record. Do this for both addresses and then compare MX records. Of course, you have the disadvantages we've talked about before for tying in with DNS... Barring that, you might just want to steal Majordomo's algorithm since it seems to work in practice ;-). <> To avoid quoting/citing hell, you'll probably have to put the confirmation magic in the Subject: header, so a simply reply will contain that data largely unchanged (module Re:'s). In the *body* of the message you can put some explanatory text saying By replying to this message you will be subscribed to the foobar@thingie.com mailing list. Do not modify the Subject header! If you do not want to be subscribed to foobar@thingie.com just ignore and/or delete this message. For more information about the foobar@thingie.com mailing list, please see That ought to be sufficient for 99% of moderately clueful people. :-) -Barry From viega@list.org Sat May 9 04:58:39 1998 From: viega@list.org (John Viega) Date: Fri, 8 May 1998 20:58:39 -0700 Subject: [Mailman-Developers] Mailman users Message-ID: <19980508205839.A7992@list.org> Could I ask everyone to join the mailing list mailman-users@python.org? When we start making big public announcements, it would be nice to have a couple of people on the list to help field questions. Also, there's a pretty lame Mailman web page up on www.list.org. Any contributions for the web page of any sort would be appreciated. Please contact me about it. John From klm@python.org Sat May 9 05:05:53 1998 From: klm@python.org (Ken Manheimer) Date: Sat, 9 May 1998 00:05:53 -0400 (EDT) Subject: [Mailman-Developers] Mailman users In-Reply-To: <19980508205839.A7992@list.org> Message-ID: > Also, there's a pretty lame Mailman web page up on www.list.org. Any > contributions for the web page of any sort would be appreciated. > Please contact me about it. (I have to say, i really like the picture, though i don't find it to exactly be a logo. In fact, i find it kinda psychedelic, like some of the plastic-hyperreal stuff i used to see in Heavy Metal comics years ago. But i'm a bit worn out today - maybe it's just flashbacks...-) Ken From klm@python.org Tue May 12 18:09:30 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 12 May 1998 13:09:30 -0400 (EDT) Subject: [Mailman-Developers] Quandry 2: Alternate hosts and sendmail Message-ID: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US> I'm hitting a bit of a quandry in determining the _correct_ setting for the sender of the subscription confirmation-request message (the thing sent when web_subscribe_requires_confirmation is "Requestor confirms via email"), and am wondering if anyone has a simple, clear answer. The problem is in some conflicting constraints in the confirmation-request return address, which can be resolved in most cases by judicious use of the sender and reply-to message fields, but which apparently cannot be resolved simply in a way that satisfies all MUAs. Here are the constraints: 1 Regular replies to the confirmation request should be directed to the list's "-request" address (where it will be processed by the mailcmd script). 2 A slightly more obscure constraint is that errors should be directed to the -admin address (or perhaps something else), *not* to the "-request" addr. And, unfortunately, setting the "errors-to" header to the -admin addr is not sufficient - in certain cases, eg when the remote address specified by the subscriber can't even be resolved in DNS, the sender receives the error notice. I think, but am not sure, that there are cases where the message makes it to the remote site but errors still get returned to the sender, not errors-to, addr. It is the last part that dictates what we're currently doing: Reply-to: -request addr Sender: -admin addr Errors-to: -admin addr This works fine in most cases - users hit "reply" in the MUA, and get the right response address. Bounced messages go to the -admin, who can diagnose the situation and/or ignore it. Note that i said "most cases" - i, as list administrator, have gotten a few errant replies from users. On investigation, it turns out that a *few* MUAs, eg /bin/mail on some platform(s) or other, do not respect the reply-to header. Dammit. So i guess my question is, is it worth trying a different approach that will allow both the reply-to *and* the sender to be the -request addr, just to accomodate the few mailers that don't respect reply-to? That would probably mean increasing the complexity of the mailcmd script significantly, to detect requests that are actually bounces of the confirmation-requests. I'm just starting to count how often this happens, but it seems like it could be a real problem for lists with lots of subscriptions... Is my explanation of the situation clear? Any strong ideas, suggestions? Ken From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Tue May 12 18:16:26 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 12 May 1998 13:16:26 -0400 (EDT) Subject: [Mailman-Developers] Quandry 2: Alternate hosts and sendmail References: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US> Message-ID: <13656.33682.522330.4854@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> Note that i said "most cases" - i, as list administrator, have KM> gotten a few errant replies from users. On investigation, it KM> turns out that a *few* MUAs, eg /bin/mail on some platform(s) KM> or other, do not respect the reply-to header. Dammit. KM> So i guess my question is, is it worth trying a different KM> approach that will allow both the reply-to *and* the sender to KM> be the -request addr, just to accomodate the few mailers that KM> don't respect reply-to? Brain dead MUAs will always be a fact of life and I doubt you'll ever eliminate all problems. Stick with best current practice and try to at least enumerate those systems that are broken (and suggest their users fix their systems :-). -Barry From klm@python.org Tue May 12 18:32:54 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 12 May 1998 13:32:54 -0400 (EDT) Subject: [Mailman-Developers] Question: Alternate hosts and sendmail Message-ID: <199805121732.NAA04868@glyph.CNRI.Reston.Va.US> [Oy - i managed to put the wrong subject line on the last message. This one really does concern alternate hosts and sendmail.] This question really is about sendmail configuration, though it is in pursuit of a mailman issue. The general problem is that some people run various maillists that are effectively in different domains, and they'd like to be able to run them from a single installation of mailman on a host that occupies all the domains. (This probably will be a major concern for ISPs who host subscriber domains - they probably want to *not* have to run a separate maillist system for each one!) In fact, mailman supports this - at the bottom of the General Options page for any list (in 1.0b3) you'll find settings for the preferred host name and URL that the list thinks it occupies. It works fine. It is my sendmail configuration that gets in the way, here. My problem is that my sendmail configuration mangles all outgoing messages from my host (eg, parrot.python.org) to look like they're coming from, eg, 'blat@python.org'. I want this mangling in general, but would like for it to *not* happen when messages are supposed to be coming from, eg, "cnri.reston.va.us". Unfortunately, i haven't sufficiently mastered sendmail setup to prevent it. All my poking and prodding has not prevailed. Can anyone out there with greater sendmail savvy point me to particular things to investigate? (We're running a range of sendmail 8.8.5 and 8.8.7 on various machines - i think the one where this really will be an issue is at 8.8.7.) Thanks for any clues! Ken From viega@list.org Tue May 12 18:45:50 1998 From: viega@list.org (John Viega) Date: Tue, 12 May 1998 10:45:50 -0700 Subject: [Mailman-Developers] Quandry 2: Alternate hosts and sendmail In-Reply-To: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US>; from Ken Manheimer on Tue, May 12, 1998 at 01:09:30PM -0400 Message-ID: <19980512104550.A2343@list.org> On Tue, May 12, 1998 at 01:09:30PM -0400, Ken Manheimer wrote: > So i guess my question is, is it worth trying a different approach > that will allow both the reply-to *and* the sender to be the -request > addr, just to accomodate the few mailers that don't respect reply-to? > That would probably mean increasing the complexity of the mailcmd > script significantly, to detect requests that are actually bounces of > the confirmation-requests. Two things. 1) You only have to do this on confirmations. Otherwise, people should be contacting the address on their own accord. So just set up a -confirm addess, make it the sender and the reply-to, and then see if the subject has the token in it, which it should not for a bounce. 2) You could always have confirmations require an "extra web step" instead of responding to an email. John From scott@chronis.pobox.com Tue May 12 22:13:53 1998 From: scott@chronis.pobox.com (Scott) Date: Tue, 12 May 1998 17:13:53 -0400 Subject: [Mailman-Developers] Quandry 2: Alternate hosts and sendmail In-Reply-To: <19980512104550.A2343@list.org>; from John Viega on Tue, May 12, 1998 at 10:45:50AM -0700 References: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US> <19980512104550.A2343@list.org> Message-ID: <19980512171353.47830@chronis.icgroup.com> hmm, this might make my life with confirmations more difficult... (without addressing this, a patch for confirmations is on the way shortly...) On Tue, May 12, 1998 at 10:45:50AM -0700, John Viega wrote: | | On Tue, May 12, 1998 at 01:09:30PM -0400, Ken Manheimer wrote: | | > So i guess my question is, is it worth trying a different approach | > that will allow both the reply-to *and* the sender to be the -request | > addr, just to accomodate the few mailers that don't respect reply-to? | | > That would probably mean increasing the complexity of the mailcmd | > script significantly, to detect requests that are actually bounces of | > the confirmation-requests. | | Two things. | | 1) You only have to do this on confirmations. Otherwise, people | should be contacting the address on their own accord. So just set up | a -confirm addess, make it the sender and the reply-to, and then see | if the subject has the token in it, which it should not for a bounce. I've seen subject lines of bounced messages maintain the subject line, prepending it with some tag or other. Is there any guarantee that scanning the subject will adequately determine what is and isn't a bounce? | 2) You could always have confirmations require an "extra web step" | instead of responding to an email. email confirmations are easier (since the user will not have to change from a MUA to a webrowser to do anything) and therefore more desirable. Any more ideas? I'm prone to follow Barry's suggestion of the current practice, but that may be biased by the fact that my work with adding email confirmation of subscriptions and subscriptions to alternate addresses works in the current framework... Scott From viega@list.org Tue May 12 22:32:36 1998 From: viega@list.org (John Viega) Date: Tue, 12 May 1998 14:32:36 -0700 Subject: [Mailman-Developers] Re: confirmations In-Reply-To: <19980512171353.47830@chronis.icgroup.com>; from Scott on Tue, May 12, 1998 at 05:13:53PM -0400 References: <199805121709.NAA04744@glyph.CNRI.Reston.Va.US> <19980512104550.A2343@list.org> <19980512171353.47830@chronis.icgroup.com> Message-ID: <19980512143236.B2382@list.org> On Tue, May 12, 1998 at 05:13:53PM -0400, Scott wrote: > hmm, this might make my life with confirmations more difficult... > (without addressing this, a patch for confirmations is on the way > shortly...) > I've seen subject lines of bounced messages maintain the subject > line, prepending it with some tag or other. Is there any guarantee > that scanning the subject will adequately determine what is and isn't > a bounce? Well, let's say your subject is: List-name confirm (XYZZY-867-5309) If you say, "reply to this message, or send a message with subject XYZZY-867-5309", then You should be able to just see if the subject line is the same, the ID exactly, or if it has an Re: or RE: prepended. > | 2) You could always have confirmations require an "extra web step" > | instead of responding to an email. > > email confirmations are easier (since the user will not have to change > from a MUA to a webrowser to do anything) and therefore more > desirable. Sure, but if there is no better solution that will always work... > Any more ideas? I'm prone to follow Barry's suggestion of the current > practice, but that may be biased by the fact that my work with adding > email confirmation of subscriptions and subscriptions to alternate > addresses works in the current framework... Well, if Berkley mail doesn't respect Reply-To:, that's quite a problem. I know a lot of people who use it because it is quick and dirty (e.g., they don't want to wait for pine to load). I don't mind ignoring it, but it's going to be a problem anyway :) From klm@python.org Tue May 12 23:46:11 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 12 May 1998 18:46:11 -0400 (EDT) Subject: [Mailman-Developers] CGI variables question - hostname used in URL Message-ID: <199805122246.SAA06371@glyph.CNRI.Reston.Va.US> As i mentioned in my sendmail question, i have someone that will need to run mailman to handle multiple lists in different domains. (Thanks, scott, for the pointer to sendmails ruleset 1 - i'll be able to point the system guys in the right direction.) It looks like mailman handles that just fine, *except* for one addition i had made. The overarching listinfo and admin pages, which provide the roster of lists on the system, need to show only the (advertised) lists for that virtual system - not the ones that are effectively on a different system. My problem is that i seem to have only a server-specific mechanism for getting the hostname part of the URL by which the script was visited, and i'm wondering whether there's a more general way. Running apache (1.2.0), i see that HTTP_HOST has the info i need, and can compare that with the list's host_name (checking whether one includes the other or vice versa), excluding those lists that do not. However, it does not look like HTTP_HOST is by any means universal. The provisional thing i've implemented is to just ignore the list's host_name setting if no HTTP_HOST env var is found, and show all advertised lists in that case. What i'm wondering is whether there are other variable names i should check for other servers, or some more general mechanism. (It's nice to have a bunch of knowledgeable people on the list to tap into for info, btw.) Ken From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed May 13 00:09:16 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 12 May 1998 19:09:16 -0400 (EDT) Subject: [Mailman-Developers] CGI variables question - hostname used in URL References: <199805122246.SAA06371@glyph.CNRI.Reston.Va.US> Message-ID: <13656.54885.16779.208982@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> Running apache (1.2.0), i see that HTTP_HOST has the info i KM> need, and can compare that with the list's host_name (checking KM> whether one includes the other or vice versa), excluding those KM> lists that do not. However, it does not look like HTTP_HOST KM> is by any means universal. SERVER_NAME is the standard I think. From klm@python.org Wed May 13 00:15:20 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 12 May 1998 19:15:20 -0400 (EDT) Subject: [Mailman-Developers] CGI variables question - hostname used in URL In-Reply-To: <13656.54885.16779.208982@anthem.CNRI.Reston.Va.US> Message-ID: On Tue, 12 May 1998, Barry A. Warsaw wrote: > >>>>> "KM" == Ken Manheimer writes: > > KM> Running apache (1.2.0), i see that HTTP_HOST has the info i > KM> need, and can compare that with the list's host_name (checking > KM> whether one includes the other or vice versa), excluding those > KM> lists that do not. However, it does not look like HTTP_HOST > KM> is by any means universal. > > SERVER_NAME is the standard I think. That's the official name of the server - not the name by which the page was visited. Ie, on visiting http://parrot.cnri.reston.va.us/mailman/listinfo HTTP_HOST: parrot.cnri.reston.va.us SERVER_NAME: www.python.org I want to display the lists that belong to the effective host being visited (the virtual host?), so i need the info contained in HTTP_HOST. ken From scott@chronis.pobox.com Wed May 13 00:52:55 1998 From: scott@chronis.pobox.com (Scott) Date: Tue, 12 May 1998 19:52:55 -0400 Subject: [Mailman-Developers] confirmation patch Message-ID: <19980512195255.13311@chronis.icgroup.com> here is a patch that implements email confirmations. as was discussed previously on this list, all subscriptions go through the confirmation process. in addition, the subscribe mail command has changed to allow subsciptions like this: subscribe [password] [digest|nodigest] [address=] this will allow people to subscribe addresses other than the one they are sending from. Finally, regarding whether or not someone can be subscribed by just replying to the confirmation message, they can. they can also mail the command confirm this patch is against mailman 1.0b3. it does not address the issues regarding a -confirm address and bounced confirmation requests that were discussed today on this list. this patch consists of 2 parts: first, the patch, and second, a file called mm_pending.py to go in the mailman/modules directory. scott (coming soon: admin cgi to use cookie confirmation and way enhanced membership management page) --------------------------------file mm_pending.py---------------- """ module for handling pending subscriptions """ import os import sys import posixfile import marshal import time import whrandom import mm_cfg DB_PATH = mm_cfg.MAILMAN_DIR + "/misc/pending_subscriptions.db" LOCK_PATH = mm_cfg.LOCK_DIR + "/pending_subscriptions.lock" VERIFY_FMT = """\ %(listname)s -- confirmation of subscription -- request %(cookie)s You or someone (%(requestor)s) has requested that your email address (%(email)s) be subscribed to the %(listname)s mailling list at %(listaddress)s. If you wish to fulfill this request, please simply reply to this message, or mail %(request_addr)s with the following line, and only the following line in the message body: confirm %(cookie)s If you do not wish to subscribe to this list, please simply ignore or delete this message. """ # ' icky emacs font lock thing def get_pending(): " returns a dict containing pending information" try: fp = open(DB_PATH,"r" ) except IOError: return {} dict = marshal.load(fp) return dict def gencookie(p=None): if p is None: p = get_pending() while 1: newcookie = int(whrandom.random() * 1000000) if p.has_key(newcookie) or newcookie < 100000: continue return newcookie def set_pending(p): ou = os.umask(0) try: lock_file = posixfile.open(LOCK_PATH,'a+') finally: os.umask(ou) lock_file.lock('w|', 1) fp = open(DB_PATH, "w") marshal.dump(p, fp) fp.close() lock_file.lock("u") lock_file.close() def add2pending(email_addr, password, digest, cookie): ts = int(time.time()) processed = 0 p = get_pending() p[cookie] = (email_addr, password, digest, ts) set_pending(p) ----------------------------patches----------------------------- Index: cgi/subscribe =================================================================== RCS file: /usr/local/cvsroot/mailman/cgi/subscribe,v retrieving revision 1.2 diff -r1.2 subscribe 12c12 < import mm_utils, maillist, mm_err, mm_message, mm_cfg, htmlformat --- > import mm_utils, maillist, mm_err, mm_message, mm_cfg, mm_pending, htmlformat 20,38d19 < NEED_CONFIRM_NOTICE = """ < A request for subscription of your address to the %s@%s < mailing list has been received via the web%s < < This is a confirmation request, to prevent anyone from subscribing < you against your wishes. In order to complete this subscription you < must send a confirming email, to %s, by < replying to this mail, and including just the line: < < subscribe %s%s < < in the body or as the subject line. If you do not actually wish to < subscribe you need not do anything. Upon subscribing you will receive < a message welcoming you to the list and describing how to tailor your < account. < < Questions or comments? < Send them to """ + mm_cfg.MAILMAN_OWNER < 146d126 < 148,165c128,137 < list.AddMember(email, pw, digest, web_subscribe=1) < results = results + ("You have successfully been added. " < "You should receive confirmation by " < "e-mail within an hour. If you do not " < "receive confirmation, then the email " < "address you gave probably bounced, " < "in which case you should try again.

") < except mm_err.MMWebSubscribeRequiresConfirmation: < results = results + ("Confirmation from your email address is " < "required, to prevent anyone from covertly " < "subscribing you. Instructions are being " < "sent to you at %s." % email) < if os.environ.has_key('REMOTE_HOST'): < remote = ", from\n%s. " % os.environ['REMOTE_HOST'] < elif os.environ.has_key('REMOTE_ADDR'): < remote = ", from\n%s." % os.environ['REMOTE_ADDR'] < else: < remote = "." --- > results = results + ("Confirmation from your email address is " > "required, to prevent anyone from covertly " > "subscribing you. Instructions are being " > "sent to you at %s." % email) > if os.environ.has_key('REMOTE_HOST'): > remote = os.environ['REMOTE_HOST'] > elif os.environ.has_key('REMOTE_ADDR'): > remote = os.environ['REMOTE_ADDR'] > else: > remote = "." 170,179c142,154 < list.SendTextToUser(subject = 'Subscribing to %s' % list.real_name, < recipient = email, < sender = list.GetAdminEmail(), < text = (NEED_CONFIRM_NOTICE < % (list.real_name, < list.host_name, < remote, < list.GetRequestEmail(), < pw, < digesting)), --- > cookie = mm_pending.gencookie() > mm_pending.add2pending(email, pw, digest, cookie) > list.SendTextToUser(subject = "%s -- confirmation of subscription -- request %d" % \ > (list.real_name, cookie), > recipient = email, > sender = list.GetRequestEmail(), > text = mm_pending.VERIFY_FMT % ({"email": email, > "listaddress": list.GetListEmail(), > "listname": list.real_name, > "cookie": cookie, > "requestor": remote, > "request_addr": list.GetRequestEmail()}), > 183c158 < % list.GetAdminEmail()]) --- > % list.GetAdminEmail()]) 191,197c166,176 < except mm_err.MMNeedApproval, x: < results = results + ("Subscription was deferred " < "because:
%s

Your request must " < "be approved by the list admin. " < "You will receive email informing you " < "of the moderator's descision when they " < "get to your request.

" % x) --- > # > # deprecating this, it might be useful if we decide to > # allow approved based subscriptions without confirmation > # > ## except mm_err.MMNeedApproval, x: > ## results = results + ("Subscription was deferred " > ## "because:
%s

Your request must " > ## "be approved by the list admin. " > ## "You will receive email informing you " > ## "of the moderator's descision when they " > ## "get to your request.

" % x) 206a186,192 > > > > > > > Index: modules/maillist.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/maillist.py,v retrieving revision 1.2 retrieving revision 1.3 diff -r1.2 -r1.3 142,143d141 < self.web_subscribe_requires_confirmation = \ < mm_cfg.DEFAULT_WEB_SUBSCRIBE_REQUIRES_CONFIRMATION 312,322d309 < < ('web_subscribe_requires_confirmation', mm_cfg.Radio, < ('None', 'Requestor confirms via email', 'Admin approves'), 0, < 'What confirmation is required for on-the-web subscribes?', < < "This option determines whether web-initiated subscribes" < " require further confirmation, either from the subscribed" < " address or from the list administrator. Absence of" < " any confirmation makes web-based subscription a" < " tempting opportunity for mischievous subscriptions by third" < " parties."), Index: modules/mm_defaults.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/mm_defaults.py,v retrieving revision 1.2 retrieving revision 1.3 diff -r1.2 -r1.3 93,94d92 < # 1 for email subscription verification, 2 for admin confirmation: < DEFAULT_WEB_SUBSCRIBE_REQUIRES_CONFIRMATION = 1 Index: modules/mm_mailcmd.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/mm_mailcmd.py,v retrieving revision 1.2 diff -r1.2 mm_mailcmd.py 8,9c8,9 < import string, os, sys < import mm_message, mm_err, mm_cfg, mm_utils --- > import string, os, sys, re > import mm_message, mm_err, mm_cfg, mm_utils, mm_pending 41a42 > 'confirm': self.ProcessConfirmCmd, 50a52 > self.__NoMailCmdResponse = 0 84a87,94 > # > # check to see if confirmation request -- special handling > # > conf_pat = r"%s -- confirmation of subscription -- request (\d\d\d\d\d\d)" % \ > self.real_name > match = re.search(conf_pat, mail.body) > if match: > lines = ["confirm %s" % (match.group(1))] 103c113,114 < self.SendMailCmdResponse(mail) --- > if not self.__NoMailCmdResponse: > self.SendMailCmdResponse(mail) 358a370,372 > password = "" > address = "" > done_digest = 0 362,369c376,396 < elif len(args) == 1: < if string.lower(args[0]) == 'digest': < digest = 1 < password = "%s%s" % (mm_utils.GetRandomSeed(), < mm_utils.GetRandomSeed()) < elif string.lower(args[0]) == 'nodigest': < digest = 0 < password = "%s%s" % (mm_utils.GetRandomSeed(), --- > elif len(args) > 3: > self.AddError("Usage: subscribe [password] [digest|nodigest] [address=]") > return > else: > for arg in args: > if string.lower(arg) == 'digest' and not done_digest: > digest = 1 > done_digest = 1 > elif string.lower(arg) == 'nodigest' and not done_digest: > digest = 0 > done_digest = 1 > elif string.lower(arg)[:8] == 'address=' and not address: > address = string.lower(arg)[8:] > elif not password: > password = arg > else: > self.AddError("Usage: subscribe [password] " > "[digest|nodigest] [address=]") > return > if not password: > password = "%s%s" % (mm_utils.GetRandomSeed(), 371,372c398,415 < else: < password = args[0] --- > if not address: > pending_addr = mail.GetSender() > else: > pending_addr = address > cookie = mm_pending.gencookie() > mm_pending.add2pending(pending_addr, password, digest, cookie) > self.SendTextToUser(subject = "%s -- confirmation of subscription -- request %d" % \ > (self.real_name, cookie), > recipient = pending_addr, > sender = self.GetRequestEmail(), > text = mm_pending.VERIFY_FMT % ({"email": pending_addr, > "listaddress": self.GetListEmail(), > "listname": self.real_name, > "cookie": cookie, > "requestor": mail.GetSender(), > "request_addr": self.GetRequestEmail()})) > self.__NoMailCmdResponse = 1 > return 374,392d416 < elif len(args) == 2: < if string.lower(args[1]) == 'nodigest': < digest = 0 < password = args[0] < elif string.lower(args[1]) == 'digest': < digest = 1 < password = args[0] < elif string.lower(args[0]) == 'nodigest': < digest = 0 < password = args[1] < elif string.lower(args[0]) == 'digest': < digest = 1 < password = args[1] < else: < self.AddError("Usage: subscribe [password] [digest|nodigest]") < return < elif len(args) > 2: < self.AddError("Usage: subscribe [password] [digest|nodigest]") < return 393a418 > def FinishSubscribe(self, addr, password, digest): 395c420 < self.AddMember(mail.GetSender(), password, digest) --- > self.AddMember(addr, password, digest) 399c424 < mail.GetSender()) --- > addr) 410c435 < "(insecure address)" % mail.GetSender()) --- > "(insecure address)" % addr) 412c437 < self.AddError("%s is already a list member." % mail.GetSender()) --- > self.AddError("%s is already a list member." % addr) 418a444,470 > > > > def ProcessConfirmCmd(self, args, cmd, mail): > if len(args) != 1: > self.AddError("Usage: confirm \n") > return > try: > cookie = string.atoi(args[0]) > except: > self.AddError("Usage: confirm \n") > return > pending = mm_pending.get_pending() > if not pending.has_key(cookie): > self.AddError("Invalid confirmation number!\n" > "Please recheck the confirmation number and try again.") > return > (email_addr, password, digest, ts) = pending[cookie] > if self.open_subscribe: > self.ApprovedAddMember(email_addr, password, digest) > self.AddToResponse("Succeeded") > else: > self.AddRequest('add_member', digest, email_addr, password) > del pending[cookie] > mm_pending.set_pending(pending) > > 464c516 < subscribe [password] [digest-option] --- > subscribe [password] [digest-option] [address=

] 469,470c521,523 < To subscribe this way, you must subscribe from the account in < which you wish to receive mail. --- > If you wish to subscribe an address other than the address you send > this request from, you may specify "address=" (no brackets > around the email address, no quotes!) From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed May 13 04:32:00 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 12 May 1998 23:32:00 -0400 (EDT) Subject: [Mailman-Developers] CGI variables question - hostname used in URL References: <13656.54885.16779.208982@anthem.CNRI.Reston.Va.US> Message-ID: <13657.5148.16956.722331@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> That's the official name of the server - not the name by which KM> the page was visited. Darn. From scott@chronis.pobox.com Sat May 16 20:32:10 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 16 May 1998 15:32:10 -0400 Subject: [Mailman-Developers] Finally! admin cgi patches Message-ID: <19980516153210.19958@chronis.icgroup.com> I've finally managed to put together some patches for the admin cgi. Changes included are: * single login for access to list-specific administrative pages, as well as for access to making all changes, except changing the administrator password. this uses cookie authentication with a configurable lifetime (changeable in mm_defaults.ADMIN_COOKIE_LIFE). In order to get this to function, I had to change maillist.MailList.GetScriptURL to allways return full, absolute urls instead of relative ones. * membership management section for mass subscription now does error checking for the addresses entered and reports what addresses were successfully subscribed and which ones weren't and why. * membership management section has an added section where list members are shown in a table with checkboxes available for setting their options or unsubscribing them. There is a related configuration variable in mm_defaults.ADMIN_MEMBER_CHUNKSIZE which will cause lists with more members than that value to only display the first of that many members (sorted) and to display an index for accessing the remaining chunks of members. this patch should be applied against mailman-1.0b3 with the patches for subscription confirmation applied. it probably would work without the confirmation patches, but i haven't tried it and won't recommend it. scott Index: cgi/admin =================================================================== RCS file: /usr/local/cvsroot/mailman/cgi/admin,v retrieving revision 1.2 diff -r1.2 admin 12,13c12,15 < import os, cgi, string, crypt, types < import mm_utils, maillist, mm_cfg, mm_err --- > sys.path.append('.') > import os, cgi, string, crypt, types, time > import mm_utils, maillist, mm_cfg, mm_err, mm_mailcmd > import Cookie 30a33,82 > LOGIN_PAGE = """ > > > %(listname)s Administrative Authentication > > >
> %(message)s > > > > > > > > > > > >
> %(listname)s Administrative > Authentication >
List Administrative Password:
>
>
> """ > > # " <- icky emacs font-lock bug workaround > > def isAuthenticated(list, password=None): > if password is not None: # explicit login > try: > list.ConfirmAdminPassword(password) > except mm_err.MMBadPasswordError: > AddErrorMessage(doc, 'Error: Incorrect admin password.') > return 0 > c = Cookie.Cookie() > c[list_name] = list.password # its crypted so this should be ok > c[list_name]['expires'] = mm_cfg.ADMIN_COOKIE_LIFE > c[list_name]["path"] = "/mailman/" + list.GetScriptURL("admin") > print c # Output the cookie > > return 1 > if os.environ.has_key('HTTP_COOKIE'): > c = Cookie.Cookie( os.environ['HTTP_COOKIE'] ) > if c.has_key(list_name): > return (c[list_name].value == list.password) > return 0 > > 36c88 < global list_name, list_info --- > global list_name, list_info, doc 48,50c100,101 < < list_name = string.lower(list_info[0]) < --- > else: > list_name = string.lower(list_info[0]) 52d102 < 65c115 < --- > 68c118 < --- > global cgi_data 70,73c120,138 < if len(cgi_data.keys()): < if cgi_data.has_key('VARHELP'): < FormatOptionHelp(doc, cgi_data['VARHELP'].value, lst) < print doc.Format(bgcolor="#ffffff") --- > is_auth = 0 > if cgi_data.has_key("adminpw"): > is_auth = isAuthenticated(lst, cgi_data["adminpw"].value) > message = FontAttr("Sorry, wrong password. Try again.", > color="ff5060", size="+1").Format() > else: > is_auth = isAuthenticated(lst) > message = "" > if not is_auth: > print "Content-type: text/html\n\n" > print LOGIN_PAGE % ({"listname": list_name, > "path": os.environ.get("REQUEST_URI", "/mailman/admin"), > "message": message}) > return > > if len(cgi_data.keys()): > if cgi_data.has_key('VARHELP'): > FormatOptionHelp(doc, cgi_data['VARHELP'].value, lst) > print doc.Format(bgcolor="#ffffff") 75,79c140 < if not cgi_data.has_key('adminpw'): < AddErrorMessage(doc, < 'Error: You must supply the admin password to' < ' change options.') < else: --- > if (cgi_data.has_key('bounce_matching_headers')): 81,87c142,147 < lst.ConfirmAdminPassword(cgi_data['adminpw'].value) < ChangeOptions(lst, category, cgi_data, doc) < # Yuck. This shouldn't need to be here. < if not lst.digestable and not lst.nondigestable: < lst.nondigestable = 1 < except mm_err.MMBadPasswordError: < AddErrorMessage(doc, 'Error: Incorrect admin password.') --- > pairs = lst.parse_matching_header_opt() > except mm_err.MMBadConfigError, line: > AddErrorMessage(doc, > 'Warning: bad matching-header line' > ' (does it have the colon?)
    %s
', > line) 100,110c160,161 < < if len(cgi_data.keys()): < if (cgi_data.has_key('bounce_matching_headers')): < try: < pairs = lst.parse_matching_header_opt() < except mm_err.MMBadConfigError, line: < AddErrorMessage(doc, < 'Warning: bad matching-header line' < ' (does it have the colon?)
    %s
', < line) < --- > if len(cgi_data.keys()): > ChangeOptions(lst, category, cgi_data, doc) 220d270 < url = lst.GetScriptURL('admin') 225c275,276 < these_links.AddItem(Link(os.path.join(url, k), v)) --- > these_links.AddItem(Link("%s/%s" % (lst.GetScriptURL('admin'),k), > v)) 234c285,286 < form = Form(os.path.join(lst.GetScriptURL('admin'), category)) --- > form = Form("%s/%s" % (lst.GetScriptURL('admin'), > category_suffix)) 425,427d476 < header.AddRow([Bold("Subscribe and Unsubscribe Members")]) < header.AddCellInfo(max(header.GetCurrentRowIndex(), 0), 0, < colspan=2, bgcolor ="#FFF0D0") 429,444c478,540 < < container.AddItem('

Mass Subscribe Members

') < container.AddItem('

Enter one email address per line

') < container.AddItem(TextArea(name='subscribees', rows=20,cols=60,wrap=None)) < < container.AddItem('

To Unsubscribe Members...

') < container.AddItem(""" < To unsubscribe members you must use your admin password in place of the < user's password on the user's edit-options page. Visit their < edit-options page (via the roster page) and do the < unsubscribe procedure, providing the admin password instead of the < user's password. <

(Note that you can, alternately, set the subscriber's no-delivery < option to inhibit delivery of their messages, if you want to only < temporarily disable their delivery.)

""" < % lst.GetScriptURL('roster')) --- > user_table = Table(width="90%") > user_table.AddRow([Center(Header(4, "Membership List"))]) > user_table.AddCellInfo(user_table.GetCurrentRowIndex(), > user_table.GetCurrentCellIndex(), > bgcolor="#cccccc", colspan=8) > > members = {} > digests = {} > for member in lst.members: > members[member] = 1 > for member in lst.digest_members: > digests[member] = 1 > all = lst.members + lst.digest_members > if len(all) > mm_cfg.ADMIN_MEMBER_CHUNKSIZE: > chunks = mm_utils.chunkify(all) > if not cgi_data.has_key("chunk"): > chunk = 0 > else: > chunk = string.atoi(cgi_data["chunk"].value) > all = chunks[chunk] > footer = ("

To View other sections, " > "click on the appropriate range listed below") > chunk_indices = range(len(chunks)) > chunk_indices.remove(chunk) > buttons = [] > pi = os.environ["PATH_INFO"] > for ci in chunk_indices: > start, end = chunks[ci][0], chunks[ci][-1] > buttons.append(" from %s to %s " % \ > ( pi, ci, start, end)) > buttons = apply(UnorderedList, tuple(buttons)) > footer = footer + buttons.Format() + "

" > else: > all.sort() > footer = "

" > for member in all: > cells = [member + "" % (member), > "subscribed " +CheckBox(member + "_subscribed", "on", 1).Format(), > ] > if members.get(member): > cells.append("digest " + CheckBox(member + "_digest", "off", 0).Format()) > else: > cells.append("digest " + CheckBox(member + "_digest", "on", 1).Format()) > for opt in ("hide", "nomail", "ack", "norcv", "plain"): > if lst.GetUserOption(member, mm_mailcmd.option_info[opt]): > value = "on" > checked = 1 > else: > value = "off" > checked = 0 > box = CheckBox("%s_%s" % (member, opt), value, checked) > cells.append("%s %s" % (opt, box.Format())) > user_table.AddRow(cells) > container.AddItem(Center(user_table)) > container.AddItem(footer) > t = Table(width="90%") > t.AddRow([Center(Header(4, "Mass Subscribe Members"))]) > t.AddCellInfo(t.GetCurrentRowIndex(), > t.GetCurrentCellIndex(), > bgcolor="#cccccc", colspan=8) > container.AddItem(Center(t)) > container.AddItem(Center(TextArea(name='subscribees', rows=10,cols=60,wrap=None))) > container.AddItem(Center(" Enter One address per line

")) 448,471c544,561 < password_submit = Table(bgcolor="#99cccc", < border=0, cellspacing=0, cellpadding=2) < password_submit.AddRow([Center(Bold('To Submit Your Changes'))]) < password_submit.AddCellInfo(password_submit.GetCurrentRowIndex(), 0, < colspan=2) < password_submit.AddRow(['

Enter the administrator ' < 'password:
', < PasswordBox('adminpw')]) < password_submit.AddRow(['
And...
', < Bold(SubmitButton('submit', 'Submit Changes'))]) < change_pw_table = Table(bgcolor="#cccccc", border=0, < cellspacing=0, cellpadding=2) < change_pw_table.AddRow([Bold(Center('To Change The Administrator' < ' Password'))]) < change_pw_table.AddCellInfo(change_pw_table.GetCurrentRowIndex(), 0, < colspan=2) < change_pw_table.AddRow(['
Enter the new ' < 'password:
', < PasswordBox('newpw')]) < change_pw_table.AddRow(['
And also confirm it:
', < PasswordBox('confirmpw')]) < < password_stuff = Table(bgcolor="#99cccc") < password_stuff.AddRow([password_submit, change_pw_table]) --- > submit = Table(bgcolor="#99ccff", > border=0, cellspacing=0, cellpadding=2, width="100%") > submit.AddRow([Bold(SubmitButton('submit', 'Submit Your Changes'))]) > submit.AddCellInfo(submit.GetCurrentRowIndex(), 0, align="middle") > change_pw_table = Table(bgcolor="#99cccc", border=0, > cellspacing=0, cellpadding=2, width="90%") > change_pw_table.AddRow([Bold(Center('To Change The Administrator Password')), > '
Enter the new password:
', > PasswordBox('newpw'),]) > change_pw_table.AddCellInfo(0, 0, align="middle", colspan=2) > change_pw_table.AddRow(['
Enter the current password
', > PasswordBox('adminpw'), > '
Again to confirm it:
', > PasswordBox('confirmpw')]) > password_stuff = Container() > password_stuff.AddItem(change_pw_table) > password_stuff.AddItem("

") > password_stuff.AddItem(submit) 555c645,676 < if category != 'members': --- > confirmed = 0 > if cgi_info.has_key('newpw'): > if cgi_info.has_key('confirmpw'): > if cgi_info.has_key('adminpw'): > try: > lst.ConfirmAdminPassword(cgi_info['adminpw'].value) > confirmed = 1 > except mm_err.MMBadPasswordError: > m = "Error: incorrect administrator password" > document.AddItem(Header(3, Italic(FontAttr(m, color="ff5060")))) > confirmed = 0 > if confirmed: > new = cgi_info['newpw'].value > confirm = cgi_info['confirmpw'].value > if new == confirm: > lst.password = crypt.crypt(new, mm_utils.GetRandomSeed()) > dirty = 1 > else: > m = 'Error: Passwords did not match.' > document.AddItem(Header(3, Italic(FontAttr(m, color="ff5060")))) > > else: > m = 'Error: You must type in your new password twice.' > document.AddItem( > Header(3, Italic(FontAttr(m, color="ff5060")))) > # > # for some reason, the login page mangles important values for the list > # such as .real_name so we only process these changes if the category > # is not "members" and the request is not from the login page > # -scott 19980515 > # > if category != 'members' and not cgi_info.has_key("request_login"): 572a694 > print "property: ", property, "value: ", value 574a697,699 > # > # mass subscription processing for members category > # 577,578c702,713 < names = string.split(name_text, '\r\n') < for new_name in names: --- > name_text = string.replace(name_text, '\r', '') > names = string.split(name_text, '\n') > if '' in names: > names.remove('') > subscribe_success = [] > subscribe_errors = [] > for new_name in map(string.strip,names): > digest = 0 > if not lst.digestable: > digest = 0 > if not lst.nondigestable: > digest = 1 580,593c715,747 < #FIXME: The admin needs to be able to specify subscribe options < lst.AddMember(new_name, (mm_utils.GetRandomSeed() + < mm_utils.GetRandomSeed())) < dirty = 1 < #FIXME: Give some sort of an indication of which names didn't work, < # and why they didn't work... < except: < pass < if cgi_info.has_key('newpw'): < if cgi_info.has_key('confirmpw'): < new = cgi_info['newpw'].value < confirm = cgi_info['confirmpw'].value < if new == confirm: < lst.password = crypt.crypt(new, mm_utils.GetRandomSeed()) --- > lst.ApprovedAddMember(new_name, (mm_utils.GetRandomSeed() + > mm_utils.GetRandomSeed()), digest) > subscribe_success.append(new_name) > except mm_err.MMAlreadyAMember: > subscribe_errors.append((new_name, 'Already a member')) > > except mm_err.MMBadEmailError: > subscribe_errors.append((new_name, "Bad/Invalid email address")) > except mm_err.MMHostileAddress: > subscribe_errors.append((new_name, "Hostile Address (illegal characters)")) > if subscribe_success: > document.AddItem(Header(5, "Successfully Subscribed:")) > document.AddItem(apply(UnorderedList, tuple((subscribe_success)))) > document.AddItem("

") > if subscribe_errors: > document.AddItem(Header(5, "Error Subscribing:")) > items = map(lambda x: "%s -- %s" % (x[0], x[1]), subscribe_errors) > document.AddItem(apply(UnorderedList, tuple((items)))) > document.AddItem("

") > # > # do the user options for members category > # > if cgi_info.has_key('user'): > user = cgi_info["user"] > if type(user) is type([]): > users = [] > for ui in range(len(user)): > users.append(user[ui].value) > else: > users = [user.value] > for user in users: > if not cgi_info.has_key('%s_subscribed' % (user)): > lst.DeleteMember(user) 595,598c749,771 < else: < m = 'Error: Passwords did not match.' < document.AddItem( < Header(3, Italic(FontAttr(m, color="ff5060")))) --- > continue > if not cgi_info.has_key("%s_digest" % (user)): > if user in lst.digest_members: > list.digest_members.remove(user) > dirty = 1 > if user not in lst.members: > lst.members.append(user) > dirty = 1 > else: > if user not in lst.digest_members: > lst.digest_members.append(user) > dirty = 1 > if user in lst.members: > lst.members.remove(user) > dirty = 1 > > for opt in ("hide", "nomail", "ack", "norcv", "plain"): > if cgi_info.has_key("%s_%s" % (user, opt)): > lst.SetUserOption(user, mm_mailcmd.option_info[opt], 1) > dirty = 1 > else: > lst.SetUserOption(user, mm_mailcmd.option_info[opt], 0) > dirty = 1 600,603d772 < else: < m = 'Error: You must type in your new password twice.' < document.AddItem( < Header(3, Italic(FontAttr(m, color="ff5060")))) 622a792 > Index: cgi/private =================================================================== RCS file: /usr/local/cvsroot/mailman/cgi/private,v retrieving revision 1.1.1.1 diff -r1.1.1.1 private 62a63 > # " 68,75c69,76 < (?: / (?: \d{4} q \d\. )? # Match "/", and, optionally, 1998q1." < ( [^/]* ) /? # The SIG name < /[^/]*$ # The trailing 12345.html portion < ) | (?: < / ( [^/.]* ) # Match matrix-sig < (?:\.html)? # Optionally match .html < /? # Optionally match a trailing slash < $ # Must match to end of string --- > (?: / (?: \d{4} q \d\. )? # Match "/", and, optionally, 1998q1. > ( [^/]* ) /? # The SIG name > /[^/]*$ # The trailing 12345.html portion > ) | (?: > / ( [^/.]* ) # Match matrix-sig > (?:\.html)? # Optionally match .html > /? # Optionally match a trailing slash > $ # Must match to end of string 87,93d87 < #for i in ['/matrix-sig.html', '/1998q1.c++-sig/index.html', < # '/1998q1.string-sig/foobar.html', < # '/psa-members.html']: < # print i, `getListName(i)` < #sys.exit(0) < < ## sys.exit(0) Index: modules/htmlformat.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/htmlformat.py,v retrieving revision 1.1.1.1 diff -r1.1.1.1 htmlformat.py 391a392,396 > class CheckBox(InputObj): > def __init__(self, name, value, checked=0, **kws): > apply(InputObj.__init__, (self, name, "CHECKBOX", value, checked), kws) > > 446a452 > Index: modules/maillist.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/maillist.py,v retrieving revision 1.3 diff -r1.3 maillist.py 60,61c60,65 < return os.path.join(self.web_page_url, '%s/%s' % < (script_name, self._internal_name)) --- > if self.web_page_url: > prefix = self.web_page_url > else: > prefix = mm_cfg.DEFAULT_URL > return os.path.join(prefix, '%s/%s' % (script_name, > self._internal_name)) 566a571,572 > if not mm_utils.ValidEmail(name): > raise mm_err.MMBadEmailError Index: modules/mm_defaults.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/mm_defaults.py,v retrieving revision 1.3 diff -r1.3 mm_defaults.py 156a157,168 > > # > # how long the cookie authorizing administrative > # changes via the admin cgi lasts > # > ADMIN_COOKIE_LIFE = 60 * 20 # 20 minutes > > # > # how many members to display at a time > # on the admin cgi to unsubscribe them or change their options > # > ADMIN_MEMBER_CHUNKSIZE = 10 Index: modules/mm_utils.py =================================================================== RCS file: /usr/local/cvsroot/mailman/modules/mm_utils.py,v retrieving revision 1.2 diff -r1.2 mm_utils.py 390a391,404 > > def chunkify(members, chunksize=mm_cfg.ADMIN_MEMBER_CHUNKSIZE): > """ > return a list of lists of members > """ > members.sort() > res = [] > while 1: > if not members: > break > chunk = members[:chunksize] > res.append(chunk) > members = members[chunksize:] > return res From scott@chronis.pobox.com Sat May 16 20:49:46 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 16 May 1998 15:49:46 -0400 Subject: [Mailman-Developers] Finally! admin cgi patches In-Reply-To: <19980516153210.19958@chronis.icgroup.com>; from Scott on Sat, May 16, 1998 at 03:32:10PM -0400 References: <19980516153210.19958@chronis.icgroup.com> Message-ID: <19980516154946.30972@chronis.icgroup.com> On Sat, May 16, 1998 at 03:32:10PM -0400, Scott wrote: | * membership management section for mass subscription now does | error checking for the addresses entered and reports what addresses | were successfully subscribed and which ones weren't and why. I just took a look at the TODO list at www.list.org and thought i should add a couple of quick notes. the mass subscription section skips the confirmation and admin approval steps, but still checks for valid email addresses and whether the member has already been subscribed. | | * membership management section has an added section where list | members are shown in a table with checkboxes available for setting | their options or unsubscribing them. There is a related | configuration variable in mm_defaults.ADMIN_MEMBER_CHUNKSIZE which | will cause lists with more members than that value to only display | the first of that many members (sorted) and to display an index for | accessing the remaining chunks of members. this section includes all members, hidden or not. scott From janne@avocado.pc.helsinki.fi Mon May 18 13:14:35 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 18 May 1998 15:14:35 +0300 Subject: [Mailman-Developers] Bouncing administrative mail Message-ID: It seems that in 1.0b1.2 there's a serious problem with bouncing administrative mail. I hope this is corrected in later versions and apologize if I'm reporting an old problem. We have a big list here with the administrator now outside of the local site (local \equiv where mailman is run). Last weekend the network connection to outside world was down. Administrative mail for the big list could not be delivered. As a result, some error messages where generated about pending mail ("undelivered for 4 hours" etc.), but apparently an Errors-To or some other header of the administrative messages caused the warnings to be delivered to the list-admin. These, of course, generated even more error messages about pending mail. As a result, I had 22000 messages in my mail queue this morning. -- Janne From scott@chronis.pobox.com Tue May 19 22:25:42 1998 From: scott@chronis.pobox.com (Scott) Date: Tue, 19 May 1998 17:25:42 -0400 Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: ; from Janne Sinkkonen on Mon, May 18, 1998 at 03:14:35PM +0300 References: Message-ID: <19980519172542.38756@chronis.icgroup.com> would it suffice to set the bounce options to "disable and don't notify me" when the list administrator address is not local? scott On Mon, May 18, 1998 at 03:14:35PM +0300, Janne Sinkkonen wrote: | | It seems that in 1.0b1.2 there's a serious problem with bouncing | administrative mail. I hope this is corrected in later versions and | apologize if I'm reporting an old problem. | | We have a big list here with the administrator now outside of the | local site (local \equiv where mailman is run). | | Last weekend the network connection to outside world was | down. Administrative mail for the big list could not be delivered. As | a result, some error messages where generated about pending mail | ("undelivered for 4 hours" etc.), but apparently an Errors-To or some | other header of the administrative messages caused the warnings to be | delivered to the list-admin. These, of course, generated even more error | messages about pending mail. | | As a result, I had 22000 messages in my mail queue this morning. | | -- | Janne | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers From janne@avocado.pc.helsinki.fi Wed May 20 09:40:07 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 20 May 1998 11:40:07 +0300 Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: Scott's message of "Tue, 19 May 1998 17:25:42 -0400" References: <19980519172542.38756@chronis.icgroup.com> Message-ID: Scott writes: > would it suffice to set the bounce options to "disable and don't > notify me" when the list administrator address is not local? > > scott As a temporary remedy, maybe yes (haven't tried it, because things are running smoothly again). But in the long run, of course, it would be nice to have the same functionality for remote administrators as for the local ones. :) -- Janne From viega@list.org Wed May 20 16:51:52 1998 From: viega@list.org (John Viega) Date: Wed, 20 May 1998 08:51:52 -0700 Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: ; from Janne Sinkkonen on Wed, May 20, 1998 at 11:40:07AM +0300 References: <19980519172542.38756@chronis.icgroup.com> Message-ID: <19980520085152.A29157@list.org> I would say that what needs to be done is that the same sort of loop detection needs to be performed on these mails as is done on regular list mail w/ the X-BeenThere header. It could also be done by adding an additional administrative address... John On Wed, May 20, 1998 at 11:40:07AM +0300, Janne Sinkkonen wrote: > Scott writes: > > > would it suffice to set the bounce options to "disable and don't > > notify me" when the list administrator address is not local? > > > > scott > > As a temporary remedy, maybe yes (haven't tried it, because things are > running smoothly again). But in the long run, of course, it would be > nice to have the same functionality for remote administrators as for > the local ones. :) > > -- > Janne > > _______________________________________________ > Mailman-Developers maillist - Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From klm@python.org Wed May 20 17:27:07 1998 From: klm@python.org (Ken Manheimer) Date: Wed, 20 May 1998 12:27:07 -0400 (EDT) Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: <19980520085152.A29157@list.org> Message-ID: On Wed, 20 May 1998, John Viega wrote: > I would say that what needs to be done is that the same sort of loop > detection needs to be performed on these mails as is done on regular > list mail w/ the X-BeenThere header. It could also be done by adding > an additional administrative address... If i'm thinking of the right bounce situation (will have to go back and look at the original problem statement), i think the right answer is to look at the intended recipients of the bounce, and if any are the same as the place where the new bounce notice would go - don't send it! I'll investigate. In addition, do most prominent logging - like in the "error" log, which only otherwise gets actual coding error tracebacks. (The idea being that this is an extraordinary error condition worth bringing to the attention of the mailman site administrator.) And incidentally, have people been checking their logs/error file occasionally? It should mostly be empty, but any tracebacks found there should be investigated. Ken From janne@avocado.pc.helsinki.fi Wed May 20 21:58:11 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 20 May 1998 23:58:11 +0300 Subject: [Mailman-Developers] Bouncing administrative mail In-Reply-To: John Viega's message of "Wed, 20 May 1998 08:51:52 -0700" References: <19980519172542.38756@chronis.icgroup.com> <19980520085152.A29157@list.org> Message-ID: John Viega writes: > list mail w/ the X-BeenThere header. It could also be done by adding > an additional administrative address... The problem seems to arise from the sender and receiver of the administrative messages being identical. If 'list-admin' cannot be reached, a foreign or local MTA generates an error message and sends it to 'list-admin'. I don't see any fool-proof way to resolve this by including a header for loop detection, because the nondeliverable message is not necessarily included in an error message generated by an MTA. If this is true, then another administrative address is necessarily needed, and it must be always functional when the list-admin@mailman.site address is functional. That means that it must be local. It could be the mailman-owner, or a trashcan. Mailman-owner is a natural alternative, because the person being responsible of mailman certainly wants to know if a list administrator has disappeared. Now that we are on administrative addresses, another remotely related feature I would like to see is separation of the address which gets all the error messages due to nondeliverable list mail from the address which is mentioned as the place to ask help in case of trouble. Sorting out pleas for help from the pile of rubbish generated by MTAs is very tedious. -- Janne From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed May 20 22:02:51 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 20 May 1998 17:02:51 -0400 (EDT) Subject: [Mailman-Developers] configuration option `info' Message-ID: <13667.17517.801967.195815@anthem.CNRI.Reston.Va.US> [Just one for the to-do list... -BAW] The help for the `info' config option says: The text will be treated as html except that newlines newlines will be translated to
- so you can use links, preformatted text, etc, but don't put in carriage returns except where you mean to separate paragraphs. And review your changes - bad html (like some unterminated HTML constructs) can prevent display of the entire listinfo page. Other that the typo (`newlines' twice in the first sentence), it would actually be better if a more texinfo like approach was used. Instead of hard newlines being translated into
, translate *blank* lines (e.g. two newlines) into
. That way the text inside the tiny type-in box could be formatted a little easier. -Barry From klm@python.org Thu May 21 01:36:27 1998 From: klm@python.org (Ken Manheimer) Date: Wed, 20 May 1998 20:36:27 -0400 (EDT) Subject: [Mailman-Developers] Preventing admin-notice bounce loops! Message-ID: <199805210036.UAA07123@glyph.CNRI.Reston.Va.US> I think i've got the right solution to the nasty bounce loop by which janne sinkonnen got around 22,000 messages last weekend. The approach is what i suggested earlier today - check the bouncing address to see if it's among the list administrators, and if so log an error and *don't* send out another notice. I have tested it as far as i was able, and it seems to behave just the way we'd want. This patch also has a few other bounce improvements, one being important, in a slightly different manner: a copy of the offending bounce notification is included with the admin-notice when a user is being disabled or unsubscribed. This enables the admin to assess the correctness of the action - something otherwise impossible, since no other record of the bounce notice is retained. (Ultimately it would be nice to have some kind of cataloguing system for things like subscribe and admin-approval requests, since the entries can be discarded once the administrator has reviewed and decided their fate. For bounce notices, just sending them with the automatic action seems to make the most sense to me...) If you give this a try, please let me know how it goes. I don't think it depends on any other local changes, but would like to have that checked... See the checkin log entries, below, for more details. ken manheimer klm@python.org ---------------------------- revision 1.16 date: 1998/05/21 00:20:00; author: klm; state: Exp; lines: +65 -30 Prevent bounce loops when admin-action recipient is itself the bouncing address - before sending the notice, check whether the bouncing address is among the list owner addresses. And otherwise, include (attach) a copy of the actual bounce notice in the admin-action message. Otherwise there is no record the administrator can examine. Finally, send both failed and successful admin-action notifications to the list administrator. Before, failed action (e.g. inability to locate the specific user) were sent to the mailman site admin - this was while i was debugging the bounce handling, but it looks a lot better now. ---------------------------- revision 1.15 date: 1998/05/20 17:18:19; author: mailman; state: Exp; lines: +10 -10 Neglected to mention on last checkin: ScanMessage(): added final-step massage of all candidates to remove encompassing <, > angle brackets, if any, and check against already processed candidates. ---------------------------- revision 1.14 date: 1998/05/20 17:13:40; author: mailman; state: Exp; lines: +35 -17 Do not send "user disabled" notices to admin when user is already disabled. The idea is that numerous outstanding bounces do not each require renotification of the admin. DisableBouncingAddress(), RemoveBouncingAddress(): changed return signatures to indicate whether email notification is necessary for this action, and in Disable version, indicate not necessary when user is already disabled. In all other cases, notification is indicated, whatever the status. Index: mm_bouncer.py =================================================================== RCS file: /projects/cvsroot/mailman/modules/mm_bouncer.py,v retrieving revision 1.13 retrieving revision 1.16 diff -c -r1.13 -r1.16 *** mm_bouncer.py 1998/04/23 16:26:33 1.13 --- mm_bouncer.py 1998/05/21 00:20:00 1.16 *************** *** 1,6 **** "Handle delivery bounce messages, doing filtering when list is set for it." ! __version__ = "$Revision: 1.13 $" # It's possible to get the mail-list senders address (list-admin) in the # bounce list. You probably don't want to have list mail sent to that --- 1,6 ---- "Handle delivery bounce messages, doing filtering when list is set for it." ! __version__ = "$Revision: 1.16 $" # It's possible to get the mail-list senders address (list-admin) in the # bounce list. You probably don't want to have list mail sent to that *************** *** 53,59 **** if self.bounce_info.has_key(email): del self.bounce_info[email] ! def RegisterBounce(self, email): report = "%s: %s - " % (self.real_name, email) bouncees = self.bounce_info.keys() this_dude = mm_utils.FindMatchingAddresses(email, bouncees) --- 53,59 ---- if self.bounce_info.has_key(email): del self.bounce_info[email] ! def RegisterBounce(self, email, msg): report = "%s: %s - " % (self.real_name, email) bouncees = self.bounce_info.keys() this_dude = mm_utils.FindMatchingAddresses(email, bouncees) *************** *** 84,90 **** self.minimum_post_count_before_bounce_action) and difference > self.minimum_removal_date * 24 * 60 * 60): self.LogMsg("bounce", report + "exceeded limits") ! self.HandleBouncingAddress(addr) return else: post_count = (self.minimum_post_count_before_bounce_action - --- 84,90 ---- self.minimum_post_count_before_bounce_action) and difference > self.minimum_removal_date * 24 * 60 * 60): self.LogMsg("bounce", report + "exceeded limits") ! self.HandleBouncingAddress(addr, msg) return else: post_count = (self.minimum_post_count_before_bounce_action - *************** *** 107,113 **** return if difference > self.minimum_removal_date * 24 * 60 * 60: self.LogMsg("bounce", "exceeded limits (D)") ! self.HandleBouncingAddress(addr) return self.LogMsg("bounce", "digester lucked out") --- 107,113 ---- return if difference > self.minimum_removal_date * 24 * 60 * 60: self.LogMsg("bounce", "exceeded limits (D)") ! self.HandleBouncingAddress(addr, msg) return self.LogMsg("bounce", "digester lucked out") *************** *** 117,164 **** self._internal_name, addr) ! def HandleBouncingAddress(self, addr): """Disable or remove addr according to bounce_action setting.""" if self.automatic_bounce_action == 0: return elif self.automatic_bounce_action == 1: ! succeeded = self.DisableBouncingAddress(addr) did = "disabled" - send = 1 elif self.automatic_bounce_action == 2: ! succeeded = self.DisableBouncingAddress(addr) did = "disabled" send = 0 elif self.automatic_bounce_action == 3: ! succeeded = self.RemoveBouncingAddress(addr) ! did = "removed" send = 1 if send: if succeeded != 1: negative="not " - recipient = mm_cfg.MAILMAN_OWNER else: negative="" ! recipient = self.GetAdminEmail() ! text = ("This is a mailman maillist administrator notice.\n" ! "\n\tMaillist:\t%s\n" ! "\tMember:\t\t%s\n" ! "\tAction:\t\tSubscription %s%s.\n" ! "\tReason:\t\tExcessive or fatal bounces.\n" ! % (self.real_name, addr, negative, did)) if succeeded != 1: ! text = text + "\tBUT:\t\t%s\n\n" % succeeded ! else: ! text = text + "\n" if did == "disabled" and succeeded == 1: ! text = text + ( ! "You can reenable their subscription by visiting " ! "their options page\n" ! "(via %s) and using your\n" ! "list admin password to authorize the option change.\n\n" ! % self.GetScriptURL('listinfo')) ! text = text + ("Questions? Contact the mailman site admin,\n%s" ! % mm_cfg.MAILMAN_OWNER) if negative: negative = string.upper(negative) self.SendTextToUser(subject = ("%s member %s %s%s due to bounces" --- 117,198 ---- self._internal_name, addr) ! def HandleBouncingAddress(self, addr, msg): """Disable or remove addr according to bounce_action setting.""" if self.automatic_bounce_action == 0: return elif self.automatic_bounce_action == 1: ! # Only send if call works ok. ! (succeeded, send) = self.DisableBouncingAddress(addr) did = "disabled" elif self.automatic_bounce_action == 2: ! (succeeded, send) = self.DisableBouncingAddress(addr) did = "disabled" + # Never send. send = 0 elif self.automatic_bounce_action == 3: ! (succeeded, send) = self.RemoveBouncingAddress(addr) ! # Always send. send = 1 + did = "removed" if send: if succeeded != 1: negative="not " else: negative="" ! recipient = self.GetAdminEmail() ! if addr in self.owner + [recipient]: ! # Whoops! This is a bounce of a bounce notice - do not ! # perpetuate the bounce loop! Log it prominently and be ! # satisfied with that. ! self.LogMsg("error", ! "%s: Bounce recipient loop" ! " encountered!\n\t%s\n\tBad admin recipient: %s", ! self._internal_name, ! "(Ie, bounce notification addr, itself, bounces.)", ! addr) ! return ! import mimetools ! boundary = mimetools.choose_boundary() ! text = [""] ! text.append("(This MIME message should be" ! " readable as plain text.)") ! text.append("") ! text.append("--" + boundary) ! text.append("Content-type: text/plain; charset=us-ascii") ! text.append("") ! text.append("This is a mailman mailing list bounce action notice:") ! text.append("") ! text.append("\tMaillist:\t%s" % self.real_name) ! text.append("\tMember:\t\t%s" % addr) ! text.append("\tAction:\t\tSubscription %s%s." % (negative, did)) ! text.append("\tReason:\t\tExcessive or fatal bounces.") if succeeded != 1: ! text.append("\tBUT:\t\t%s\n" % succeeded) ! text.append("") if did == "disabled" and succeeded == 1: ! text.append("You can reenable their subscription by visiting " ! "their options page") ! text.append("(via %s) and using your" ! % self.GetScriptURL('listinfo')) ! text.append( ! "list admin password to authorize the option change.") ! text.append("") ! text.append("The triggering bounce notice is attached below.") ! text.append("") ! text.append("Questions? Contact the mailman site admin,") ! text.append("\t" + mm_cfg.MAILMAN_OWNER) ! ! text.append("") ! text.append("--" + boundary) ! text.append("Content-type: text/plain; charset=us-ascii") ! text.append("") ! text.append(string.join(msg.headers, '')) ! text.append("") ! text.append(mm_utils.QuotePeriods(msg.body)) ! text.append("") ! text.append("--" + boundary + "--") ! if negative: negative = string.upper(negative) self.SendTextToUser(subject = ("%s member %s %s%s due to bounces" *************** *** 166,209 **** negative, did)), recipient = recipient, sender = mm_cfg.MAILMAN_OWNER, ! add_headers = ["Errors-To: %s" ! % mm_cfg.MAILMAN_OWNER], ! text = text) def DisableBouncingAddress(self, addr): if not self.IsMember(addr): reason = "User not found." self.LogMsg("bounce", "%s: NOT disabled %s: %s", self.real_name, addr, reason) ! return reason try: ! self.SetUserOption(addr, mm_cfg.DisableDelivery, 1) ! self.LogMsg("bounce", "%s: disabled %s", self.real_name, addr) ! self.Save() ! return 1 except mm_err.MMNoSuchUserError: self.LogMsg("bounce", "%s: NOT disabled %s: %s", self.real_name, addr, mm_err.MMNoSuchUserError) self.ClearBounceInfo(addr) self.Save() ! return mm_err.MMNoSuchUserError def RemoveBouncingAddress(self, addr): if not self.IsMember(addr): reason = "User not found." self.LogMsg("bounce", "%s: NOT removed %s: %s", self.real_name, addr, reason) ! return reason try: self.DeleteMember(addr, "bouncing addr") self.LogMsg("bounce", "%s: removed %s", self.real_name, addr) self.Save() ! return 1 except mm_err.MMNoSuchUserError: self.LogMsg("bounce", "%s: NOT removed %s: %s", self.real_name, addr, mm_err.MMNoSuchUserError) self.ClearBounceInfo(addr) self.Save() ! return mm_err.MMNoSuchUserError # Return 0 if we couldn't make any sense of it, 1 if we handled it. def ScanMessage(self, msg): --- 200,259 ---- negative, did)), recipient = recipient, sender = mm_cfg.MAILMAN_OWNER, ! add_headers = [ ! "Errors-To: %s" % mm_cfg.MAILMAN_OWNER, ! "MIME-version: 1.0", ! "Content-type: multipart/mixed;" ! ' boundary="%s"' % boundary], ! text = string.join(text, '\n')) def DisableBouncingAddress(self, addr): + """Disable delivery for bouncing user address. + + Returning success and notification status.""" if not self.IsMember(addr): reason = "User not found." self.LogMsg("bounce", "%s: NOT disabled %s: %s", self.real_name, addr, reason) ! return reason, 1 try: ! if self.GetUserOption(addr, mm_cfg.DisableDelivery): ! # No need to send out notification if they're already disabled. ! self.LogMsg("bounce", ! "%s: already disabled %s", self.real_name, addr) ! return 1, 0 ! else: ! self.SetUserOption(addr, mm_cfg.DisableDelivery, 1) ! self.LogMsg("bounce", ! "%s: disabled %s", self.real_name, addr) ! self.Save() ! return 1, 1 except mm_err.MMNoSuchUserError: self.LogMsg("bounce", "%s: NOT disabled %s: %s", self.real_name, addr, mm_err.MMNoSuchUserError) self.ClearBounceInfo(addr) self.Save() ! return mm_err.MMNoSuchUserError, 1 def RemoveBouncingAddress(self, addr): + """Unsubscribe user with bouncing address. + + Returning success and notification status.""" if not self.IsMember(addr): reason = "User not found." self.LogMsg("bounce", "%s: NOT removed %s: %s", self.real_name, addr, reason) ! return reason, 1 try: self.DeleteMember(addr, "bouncing addr") self.LogMsg("bounce", "%s: removed %s", self.real_name, addr) self.Save() ! return 1, 1 except mm_err.MMNoSuchUserError: self.LogMsg("bounce", "%s: NOT removed %s: %s", self.real_name, addr, mm_err.MMNoSuchUserError) self.ClearBounceInfo(addr) self.Save() ! return mm_err.MMNoSuchUserError, 1 # Return 0 if we couldn't make any sense of it, 1 if we handled it. def ScanMessage(self, msg): *************** *** 268,274 **** messy_pattern_6 = regex.compile('^[ \t]*[^ ]+: User unknown.*$') messy_pattern_7 = regex.compile('^[^ ]+ - User currently disabled.*$') ! message_groked = 0 for line in string.split(relevant_text, '\n'): for pattern, action in simple_bounce_pats: --- 318,324 ---- messy_pattern_6 = regex.compile('^[ \t]*[^ ]+: User unknown.*$') messy_pattern_7 = regex.compile('^[^ ]+ - User currently disabled.*$') ! message_grokked = 0 for line in string.split(relevant_text, '\n'): for pattern, action in simple_bounce_pats: *************** *** 276,331 **** email = self.ExtractBouncingAddr(line) if action == REMOVE: candidates = candidates + string.split(email,',') ! message_groked = 1 continue elif action == BOUNCE: emails = string.split(email,',') for email_addr in emails: ! self.RegisterBounce(email_addr) ! message_groked = 1 continue else: ! message_groked = 1 continue # Now for the special case messages that are harder to parse... if (messy_pattern_1.match(line) <> -1 or messy_pattern_2.match(line) <> -1): username = string.split(line)[1] ! self.RegisterBounce('%s@%s' % (username, remote_host)) ! message_groked = 1 continue if (messy_pattern_3.match(line) <> -1 or messy_pattern_4.match(line) <> -1 or messy_pattern_5.match(line) <> -1): username = string.split(line)[1] candidates.append('%s@%s' % (username, remote_host)) ! message_groked = 1 continue if messy_pattern_6.match(line) <> -1: username = string.split(string.strip(line))[0][:-1] candidates.append('%s@%s' % (username, remote_host)) ! message_groked = 1 continue if messy_pattern_7.match(line) <> -1: username = string.split(string.strip(line))[0] candidates.append('%s@%s' % (username, remote_host)) ! message_groked = 1 continue did = [] for i in candidates: el = string.find(i, "...") ! if el != -1: i = i[:el] if i not in did: ! self.HandleBouncingAddress(i) did.append(i) ! return message_groked def ExtractBouncingAddr(self, line): email = regsub.splitx(line, '[^ \t@<>]+@[^ \t@<>]+\.[^ \t<>.]+')[1] if email[0] == '<': - # Remove what's within the angles. return regsub.splitx(email[1:], ">")[0] else: return email --- 326,384 ---- email = self.ExtractBouncingAddr(line) if action == REMOVE: candidates = candidates + string.split(email,',') ! message_grokked = 1 continue elif action == BOUNCE: emails = string.split(email,',') for email_addr in emails: ! self.RegisterBounce(email_addr, msg) ! message_grokked = 1 continue else: ! message_grokked = 1 continue # Now for the special case messages that are harder to parse... if (messy_pattern_1.match(line) <> -1 or messy_pattern_2.match(line) <> -1): username = string.split(line)[1] ! self.RegisterBounce('%s@%s' % (username, remote_host), msg) ! message_grokked = 1 continue if (messy_pattern_3.match(line) <> -1 or messy_pattern_4.match(line) <> -1 or messy_pattern_5.match(line) <> -1): username = string.split(line)[1] candidates.append('%s@%s' % (username, remote_host)) ! message_grokked = 1 continue if messy_pattern_6.match(line) <> -1: username = string.split(string.strip(line))[0][:-1] candidates.append('%s@%s' % (username, remote_host)) ! message_grokked = 1 continue if messy_pattern_7.match(line) <> -1: username = string.split(string.strip(line))[0] candidates.append('%s@%s' % (username, remote_host)) ! message_grokked = 1 continue did = [] for i in candidates: el = string.find(i, "...") ! if el != -1: ! i = i[:el] ! if len(i) > 1 and i[0] == '<': ! # Use stuff after open angle and before (optional) close: ! i = regsub.splitx(i[1:], ">")[0] if i not in did: ! self.HandleBouncingAddress(i, msg) did.append(i) ! return message_grokked def ExtractBouncingAddr(self, line): email = regsub.splitx(line, '[^ \t@<>]+@[^ \t@<>]+\.[^ \t<>.]+')[1] if email[0] == '<': return regsub.splitx(email[1:], ">")[0] else: return email From klm@python.org Thu May 21 18:56:36 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 21 May 1998 13:56:36 -0400 (EDT) Subject: [Mailman-Developers] Bounce action policy - not distinguish "user not found"? Message-ID: <199805211756.NAA09630@glyph.CNRI.Reston.Va.US> Here's a bounce-action policy question, primarily for john, but also i think useful for general discussion and information. Currently, "user not found"/"user unknown" bounces are treated a fatal problems, invoking immediate (disable or unsubscribe, according to list conf) action. Most other random types of bounces are treated as potentially transient, and just added to the members tally. It was suggested to me that it might be better to just handle them all as potentially transient, and only take the action when the tally exceeds the threshhold. Offhand, i don't see the *need* to treat the "user not found" bounces specially, though i can imagine they're a bit less likely to be transient than, eg, "spool dir full" bounces. I'm inclined to think it would be simpler and more sensible to just lump all the bounces together. Thoughts? Ken From janne@avocado.pc.helsinki.fi Thu May 21 23:45:57 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 22 May 1998 01:45:57 +0300 Subject: [Mailman-Developers] Administrators would like to see hidden addresses Message-ID: An administrator would like to see all the subscribed addresses, including the hidden ones. This would be necessary to check whether they match some of the bounce messages which were not handled automatically. -- Janne From scott@chronis.pobox.com Fri May 22 00:01:41 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 21 May 1998 19:01:41 -0400 Subject: [Mailman-Developers] Administrators would like to see hidden addresses In-Reply-To: ; from Janne Sinkkonen on Fri, May 22, 1998 at 01:45:57AM +0300 References: Message-ID: <19980521190141.64768@chronis.icgroup.com> the admin cgi patch i posted a few days ago does this. in addition, administrators can edit the options or unsubscribe members en masse with this patch. scott On Fri, May 22, 1998 at 01:45:57AM +0300, Janne Sinkkonen wrote: | | An administrator would like to see all the subscribed addresses, | including the hidden ones. This would be necessary to check whether | they match some of the bounce messages which were not handled | automatically. | | -- | Janne | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers From janne@avocado.pc.helsinki.fi Fri May 22 00:04:35 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 22 May 1998 02:04:35 +0300 Subject: [Mailman-Developers] Administrators would like to see hidden addresses In-Reply-To: Scott's message of "Thu, 21 May 1998 19:01:41 -0400" References: <19980521190141.64768@chronis.icgroup.com> Message-ID: Scott writes: > the admin cgi patch i posted a few days ago does this. > in addition, administrators can edit the options or unsubscribe > members en masse with this patch. OK, thanks. :) Does anyone happen to have an up-to-date snapshot of mailman available for downloading? I'm lagging badly behind with the patches. -- Janne From klm@python.org Fri May 22 01:25:42 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 21 May 1998 20:25:42 -0400 (EDT) Subject: [Mailman-Developers] Administrators would like to see hidden addresses In-Reply-To: Message-ID: On 22 May 1998, Janne Sinkkonen wrote: > Scott writes: > > > the admin cgi patch i posted a few days ago does this. > > in addition, administrators can edit the options or unsubscribe > > members en masse with this patch. > > OK, thanks. :) > > Does anyone happen to have an up-to-date snapshot of mailman available > for downloading? I'm lagging badly behind with the patches. Now that it looks like i have all the patches, i'm (sorta) hoping to take some time this weekend or early next week to integrate them. However, i'm going to be going out of time in around a week, and time is tight, so no guarantees... Ken From klm@python.org Thu May 28 21:53:35 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 28 May 1998 16:53:35 -0400 (EDT) Subject: [Mailman-Developers] Re: Alternate hosts and sendmail Message-ID: <199805282053.QAA08478@glyph.CNRI.Reston.Va.US> It looks like having mailman do smtp directly is going to resolve the problem i was having getting mailman to do "virtual hosting". And that increases the priority of getting the direct smtp connection in - i have some people here at cnri who need the capability, not urgently, but it'd be nice sooner rather than later... The problem i'm referring to is that mailman does the right thing sending out messages as if they come from the domain for which the maillist is configured, but sendmail by default rewrites the addresses so the maillist address looks like the primary domain for which sendmail is configured. The consensus was that reconfiguring sendmail to avoid that would be dirty - it entails rewriting one of the rulesets, which is fairly-to-very intrusive. Well, it turns out that this problem does not apply to mail relayed by direct smtp connections, hooray. That means no special instructions will be necessary for sendmail (or other mda) configuration, hooray. Oh, and incidentally, i'm going to be away for the next week and a day - no new releases, though barry and john may have some nice stuff to spring on you in the interim. I'm gone! (Hooray.) Ken From klm@python.org Fri May 29 00:56:31 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 28 May 1998 19:56:31 -0400 (EDT) Subject: [Mailman-Developers] Some improvement suggestions Message-ID: <199805282356.TAA09183@glyph.CNRI.Reston.Va.US> Some of the other people here at cnri are using mailman, and they've hit up against one small change needed. I thought i'd mention it here, in case anyone gets the chance and inclination to address it. I also have one other item which i would really like to do, but the eventuality of actually doing it may be less near than i'd like (especially since i'm going away for a week, and there's all that catch-up time on return). The first item is the ability to edit the new-member welcome message. We offer list managers the ability to edit the html pages, and they get to see the way the magic refs (eg, ) are used on the template to see how they could use them for their own versions. Well, there is a number of standard mail messages, including particularly the welcome message, that they may want to edit. (IN FACT, two different groups, separately, came to me today with the need to edit the welcome message - for different reasons! A sure sign...) I think we would have to change the format substitutions, currently all the "%s" order dependent style, to be "%(real_name)s" dictionary substitutions, and otherwise do the same kind of thing we do for the html templates. (Ah, i see barry's done that in some recent checkins.) There is a bunch of messages, some which may benefit from this treatment, other might just clutter the list admins choices. Along these lines, there's something much more encompassing i'd like to do or see done. I'd like to extend the handling of list admin options and html templates so list-specific versions are only squirreled away in list-specific space if and when the list admin changes them from the default. This way, the central administrator could change the template or default values, and the new value would show for all the lists that haven't tailored their own versions. This would certainly be suitable for something like the html pages, where site style changes need propagation, and also for stuff like the forbidden posters. It may not be suitable for some others. This should be easy for the template files - just don't stash list-specific copies until an edit is made (and the edited copy differs), and use the default template if no list-specific one exists. It'd also be good to offer a way in the edit interface to revert to the default version, wiping out the list-specific copy. For the list admin options, we'd need a new field in the options data structure specifying the *name* of the default value, and likewise some way for the gui option for the list admin to explicitly revert to the default. Maybe there should be some way to have a locked-in list-specific setting, even if it doesn't differ from the _current_ site default. And then all the routines that use the list admin options will have to mediate through a routine (or options mediator object) that takes the name of the option and gets the default value when suitable... Well, i hope this is at least slightly understandable, if quick - i gotta run. C ya later! Ken From klm@python.org Fri May 29 01:12:39 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 28 May 1998 20:12:39 -0400 (EDT) Subject: [Mailman-Developers] oh yeah, one more teeny tiny needed change Message-ID: <199805290012.UAA09317@glyph.CNRI.Reston.Va.US> One more small suggestion i got today - list managers should be able to override the monthly password reminder for their list members. Some lists are infrequent, announcement only lists, and there can be enough info in the header/footer about where to find the place to get their passwords that the password reminder is gratuitous. At least, that's the opinion of someone who's running such a list... briefly, ken From dragondm@nexus.Integral.org Fri May 29 02:45:32 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Thu, 28 May 1998 20:45:32 -0500 (CDT) Subject: [Mailman-Developers] Smtp delivery. Message-ID: I noticed the message about direct smtp delivery being needed for virtual hosting.... As I mentioned before, I have a version on mailman 1.0b1 that I am running that already does this. I am going to give a good shot at integrating that feature into the current (1.0b3?) source this weekend. (I want to anyway, as I want to upgrade my server. ) BTW, I also mentioned some user database changes in my variant of mailman... I'm dropping those, as I don't like the way I implemented them. I'll just try to incorporate the other features I added to 1.0b1 (smtp delivery being the biggie) into the current tree, and go from there. -The Dragon De Monsyne From cklempay@jerkweed.ml.org Fri May 29 03:29:17 1998 From: cklempay@jerkweed.ml.org (Corbett Klempay) Date: Thu, 28 May 1998 22:29:17 -0400 (EDT) Subject: No subject Message-ID: (I'm not sure if this should really be on mailman-users or not instead, but it seems somewhat relevant to this list as well)... Does anyone have any quantitative (or maybe even non-quantitative) idea of Mailman's performance, especially compared to direct competitors such as Majordomo? Do we know of any very high volume lists being run on Mailman (or is this even a good idea?). I imagine that something like ezmlm would probably obliterate both Majordomo as well as Mailman as far as performance, but I don't see the average list needing the kind of fiendish performance ezmlm provides (don't me wrong; ezmlm has a lot to recommend it); most people could (and would) trade ezmlm's speed for Mailman's slick web interface. But what I'm wondering is do we know how big this performance tradeoff is? Also, I saw that there was a mention on the TODO list about adding support for other MTAs (such as qmail). Has anyone implemented this on their own yet? Would providing support for qmail affect (enhance) performance much? I'm trying to (seeing how smooth Mailman is) see what kind of potential user base there will be; I'm wondering at what point the list becomes too speed-critical to allow use of Mailman. Any comments? ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "There exists in India no politician daring enough to attempt to explain to the masses that cows can be eaten." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From dragondm@nexus.Integral.org Fri May 29 03:06:15 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Thu, 28 May 1998 21:06:15 -0500 (CDT) Subject: [Mailman-Developers] Some improvement suggestions In-Reply-To: <199805282356.TAA09183@glyph.CNRI.Reston.Va.US> Message-ID: On Thu, 28 May 1998, Ken Manheimer wrote: > Some of the other people here at cnri are using mailman, and they've > hit up against one small change needed. I thought i'd mention it > here, in case anyone gets the chance and inclination to address it. I > also have one other item which i would really like to do, but the > eventuality of actually doing it may be less near than i'd like > (especially since i'm going away for a week, and there's all that > catch-up time on return). > > The first item is the ability to edit the new-member welcome message. > We offer list managers the ability to edit the html pages, and they > get to see the way the magic refs (eg, ) are used on the > template to see how they could use them for their own versions. Well, > there is a number of standard mail messages, including particularly > the welcome message, that they may want to edit. (IN FACT, two > different groups, separately, came to me today with the need to edit > the welcome message - for different reasons! A sure sign...) > > I think we would have to change the format substitutions, currently > all the "%s" order dependent style, to be "%(real_name)s" dictionary > substitutions, and otherwise do the same kind of thing we do for the > html templates. (Ah, i see barry's done that in some recent > checkins.) There is a bunch of messages, some which may benefit from > this treatment, other might just clutter the list admins choices. Hmm.. Has anyone looked at doing the whole shot with Digicool's DocumentTemplate modules? The do both html and text, and their DTML is alot more flexible than mailan's current 'magic-tag' system. It seems a natural. > Along these lines, there's something much more encompassing i'd like > to do or see done. I'd like to extend the handling of list admin > options and html templates so list-specific versions are only > squirreled away in list-specific space if and when the list admin > changes them from the default. This way, the central administrator > could change the template or default values, and the new value would > show for all the lists that haven't tailored their own versions. This > would certainly be suitable for something like the html pages, where > site style changes need propagation, and also for stuff like the > forbidden posters. It may not be suitable for some others. > > This should be easy for the template files - just don't stash > list-specific copies until an edit is made (and the edited copy > differs), and use the default template if no list-specific one > exists. It'd also be good to offer a way in the edit interface to > revert to the default version, wiping out the list-specific copy. > > For the list admin options, we'd need a new field in the options data > structure specifying the *name* of the default value, and likewise > some way for the gui option for the list admin to explicitly revert to > the default. Maybe there should be some way to have a locked-in > list-specific setting, even if it doesn't differ from the _current_ > site default. And then all the routines that use the list admin > options will have to mediate through a routine (or options mediator > object) that takes the name of the option and gets the default value > when suitable... > Hmm... Suggestion... In my UserDB module (which I' not abandoning for various other reasons) that stored a database of User objects for each list, I had something like this. The Mailinglist object held certain 'default' settings for users, and the User object contained oly those settings that were changed. the User objects used the Aquisition mdule (part of the Extention Class system) to fetch the info from the list object if they didn't have it. IMHO, I realy think mailman needs a class of persistant Server object (one for each virtual server) to store server-specific info which really dosen't belong in the mailinglist object. Such an object could store defaults in the same way as my UserDB did . It could also have methods for things like getting the list of lists. PS Anyone thought of using mailman with Bobo? -The Dragon De Monsyne From viega@list.org Fri May 29 03:41:17 1998 From: viega@list.org (John Viega) Date: Thu, 28 May 1998 19:41:17 -0700 Subject: [Mailman-Developers] Re: your mail In-Reply-To: ; from Corbett Klempay on Thu, May 28, 1998 at 10:29:17PM -0400 References: Message-ID: <19980528194117.B18099@list.org> On Thu, May 28, 1998 at 10:29:17PM -0400, Corbett Klempay wrote: > Does anyone have any quantitative (or maybe even non-quantitative) idea of > Mailman's performance, especially compared to direct competitors such as > Majordomo? It's more efficient than Majordomo, especially when doing bulk mailing. > Do we know of any very high volume lists being run on Mailman How high volume? I know of some lists with 5000+ users, they don't seem to have any problems. The biggest one I run has been over 2000 strong. > (or is this even a good idea?). I imagine that something like ezmlm would > probably obliterate both Majordomo as well as Mailman as far as > performance, but I don't see the average list needing the kind of fiendish > performance ezmlm provides (don't me wrong; ezmlm has a lot to recommend > it); most people could (and would) trade ezmlm's speed for Mailman's slick > web interface. But what I'm wondering is do we know how big this > performance tradeoff is? You'd do a lot better performance wise trading sendmail for qmail than Mailman for ezmlm. Also, within a few months we're going to have an option to keep mailman as a persistant server, to avoid the costs involved of always starting up. > Also, I saw that there was a mention on the TODO list about adding support > for other MTAs (such as qmail). Has anyone implemented this on their own > yet? Would providing support for qmail affect (enhance) performance much? I'm working on this feature as we speak. Like I said, the underlying mail transport is more important than the MLM software in terms of how many messages you can push through a machine. > I'm trying to (seeing how smooth Mailman is) see what kind of potential > user base there will be; I'm wondering at what point the list becomes too > speed-critical to allow use of Mailman. Any comments? Well, I think that the current way the locking is done might cause some problems on lists that have large, large numbers of users AND get tons of users. I don't know how large those numbers are yet, but the big list I run has gotten 100 posts in an hour before, without problem. The locking is going to eventually become more fine-grained, so even for the largest lists, I don't think it'll be a problem. John From scott@chronis.pobox.com Fri May 29 04:00:06 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 28 May 1998 23:00:06 -0400 Subject: [Mailman-Developers] Re: your mail In-Reply-To: ; from Corbett Klempay on Thu, May 28, 1998 at 10:29:17PM -0400 References: Message-ID: <19980528230006.22326@chronis.icgroup.com> On Thu, May 28, 1998 at 10:29:17PM -0400, Corbett Klempay wrote: | Does anyone have any quantitative (or maybe even non-quantitative) idea of | Mailman's performance, especially compared to direct competitors such as | Majordomo? not that i know of. | Do we know of any very high volume lists being run on Mailman | (or is this even a good idea?). I The highest volume that i have heard of is ~250. Over the next number of weeks, i plan to give mailman a test run for some large lists. If i'm lucky i'll get some lists with thousands of subscribers. I think large lists with mailman is a good idea, and that mailman's programming is structured in a way that if it cannot handle very large lists as is, it will be easy to accomodate the needed changes. I am certain it will become quite scalable over the summer :) | imagine that something like ezmlm would | probably obliterate both Majordomo as well as Mailman as far as | performance, but I don't see the average list needing the kind of fiendish | performance ezmlm provides (don't me wrong; ezmlm has a lot to recommend | it); most people could (and would) trade ezmlm's speed for Mailman's slick | web interface. But what I'm wondering is do we know how big this | performance tradeoff is? As an administrator of a very high volume list site, i can assure you that the efficiency of the MTA is so much more important than the efficiency of the mailling list manager that the efficiency of the mailling list manager can *almost* be ignored. As i see it, mailman's efficiency is OK now, and can and will be made more and more efficient by implementing better handling of locking the lists, general code cleanup, and perhaps a client-server model. Without any hard data, in many ways it seems like mailman will outperform majordomo with most list operations in loaded circumstances simply because it deals with a lot less file io, which is one the main things that tends to slow down a host that is delivering lots of mail. Also, with the upcoming use of deliveries via smtp, it will be much easier to have the web interface for mailman on one machine and the outgoing deliveries happen on another machine. To my knowledge, most really large listservs do this, and it would be quite difficult to get to work with majordomo. I'm not familiar with ezmlm, but I know that no matter how efficient it is, it almost certainly won't make a noticable difference compared to differences produced by how an MTA handles mail, and if it's difficult to have delivery occur on a separate machine with ezmlm, then it can't scale as well as a mailling list manager that allows for that separation. | | Also, I saw that there was a mention on the TODO list about adding support | for other MTAs (such as qmail). Has anyone implemented this on their own | yet? Would providing support for qmail affect (enhance) performance much? the upcoming smtp changes will make any MTA work for delivery. i don't think anyone has made something that will manage aliases for qmail, though. | I'm trying to (seeing how smooth Mailman is) see what kind of potential | user base there will be; I'm wondering at what point the list becomes too | speed-critical to allow use of Mailman. Any comments? Once mailman is past it's beta stage, i'm willing to bet that it will handle large lists very well. In the mean time, there may well be some unforeseen glitches. scott From scott@chronis.pobox.com Fri May 29 04:06:15 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 28 May 1998 23:06:15 -0400 Subject: [Mailman-Developers] Re: your mail In-Reply-To: <19980528194117.B18099@list.org>; from John Viega on Thu, May 28, 1998 at 07:41:17PM -0700 References: <19980528194117.B18099@list.org> Message-ID: <19980528230615.23149@chronis.icgroup.com> On Thu, May 28, 1998 at 07:41:17PM -0700, John Viega wrote: | On Thu, May 28, 1998 at 10:29:17PM -0400, Corbett Klempay wrote: | > Does anyone have any quantitative (or maybe even non-quantitative) idea of | > Mailman's performance, especially compared to direct competitors such as | > Majordomo? | | It's more efficient than Majordomo, especially when doing bulk mailing. | | > Do we know of any very high volume lists being run on Mailman | | How high volume? I know of some lists with 5000+ users, they don't | seem to have any problems. The biggest one I run has been over 2000 | strong. Wow! | > (or is this even a good idea?). I imagine that something like ezmlm would | > probably obliterate both Majordomo as well as Mailman as far as | > performance, but I don't see the average list needing the kind of fiendish | > performance ezmlm provides (don't me wrong; ezmlm has a lot to recommend | > it); most people could (and would) trade ezmlm's speed for Mailman's slick | > web interface. But what I'm wondering is do we know how big this | > performance tradeoff is? | | You'd do a lot better performance wise trading sendmail for qmail than | Mailman for ezmlm. just wanted to suggest vmailer as well as qmail. personally, i like vmailer a lot more. it's about as efficient as qmail for mailling list delivery, a lot more efficient than qmail for 1 to 1 delivery, it has sane logging, and is more effectively a sendmail drop in replacement. i'm using it with mailman, and it required no changes whatsoever to the distribution to work. scott From klm@python.org Fri May 29 06:29:56 1998 From: klm@python.org (Ken Manheimer) Date: Fri, 29 May 1998 01:29:56 -0400 (EDT) Subject: [Mailman-Developers] Smtp delivery. In-Reply-To: Message-ID: On Thu, 28 May 1998, The Dragon De Monsyne wrote: > I noticed the message about direct smtp delivery being needed for > virtual hosting.... > > As I mentioned before, I have a version on mailman 1.0b1 that I am > running that already does this. I am going to give a good shot at > integrating that feature into the current (1.0b3?) source this weekend. > (I want to anyway, as I want to upgrade my server. ) It looks like john was doing this, today, and all that will come out with barry's big autoconf changes within the next few weeks - maybe sooner - probably as 1.0b4. So you may not want to spend more time on it. > BTW, I also mentioned some user database changes in my variant of > mailman... I'm dropping those, as I don't like the way I implemented them. > I'll just try to incorporate the other features I added to 1.0b1 (smtp > delivery being the biggie) into the current tree, and go from there. If you're interested in incorporating the other things, you should check with barry about getting a current snapshot before doing to much hacking... Ken From roy@endeavor.med.nyu.edu Fri May 29 12:39:53 1998 From: roy@endeavor.med.nyu.edu (Roy Smith) Date: Fri, 29 May 1998 07:39:53 -0400 Subject: [Mailman-Developers] Re: your mail Message-ID: <26162.3105416393@qwerky.med.nyu.edu> "John Viega" wrote: > It's more efficient than Majordomo, especially when doing bulk mailing. This may be a naive question, but how can that be? Majordomo doesn't actually get involved in mail delivery (at least the versions I've used). It just maintains an address file which is read by sendmail (or whatever MTA you're using) to do the actual delivery. So what does it mean to say that majordomo isn't efficient at bulk mailing? From viega@list.org Fri May 29 13:21:15 1998 From: viega@list.org (John Viega) Date: Fri, 29 May 1998 05:21:15 -0700 Subject: [Mailman-Developers] Re: your mail In-Reply-To: <26162.3105416393@qwerky.med.nyu.edu>; from Roy Smith on Fri, May 29, 1998 at 07:39:53AM -0400 References: <26162.3105416393@qwerky.med.nyu.edu> Message-ID: <19980529052115.A25666@list.org> The assumption being that you're using sendmail, I should have said that. What happens is, majordomo uses a single sendmail connection. On lists of 2000 people, it can take hours to deliver to everyone, because w/ a list that large, there are going to be lots of machines that are temporarily inaccessable, etc. Sendmail just marches down the list of addresses, sending each one. If one of them blocks, sendmail will stall delivery for everyone for a few minutes. Mailman by default forks off several versions of sendmail at once. You can set the number of sendmails to be whatever you like. For a mailing list of mine that used to take several hours to completely deliver a mail, Mailman reduced the time to about 2 minutes. Make sense? On Fri, May 29, 1998 at 07:38:57AM -0400, Roy Smith wrote: > "John Viega" wrote: > > It's more efficient than Majordomo, especially when doing bulk mailing. > > This may be a naive question, but how can that be? > > Majordomo doesn't actually get involved in mail delivery (at least the > versions I've used). It just maintains an address file which is read by > sendmail (or whatever MTA you're using) to do the actual delivery. So what > does it mean to say that majordomo isn't efficient at bulk mailing? > > Roy Smith > New York University School of Medicine > 550 First Avenue, New York, NY 10016 > > From viega@list.org Fri May 29 13:37:06 1998 From: viega@list.org (John Viega) Date: Fri, 29 May 1998 05:37:06 -0700 Subject: [Mailman-Developers] Smtp delivery. In-Reply-To: ; from Ken Manheimer on Fri, May 29, 1998 at 01:29:56AM -0400 References: Message-ID: <19980529053706.B25666@list.org> On Fri, May 29, 1998 at 01:29:56AM -0400, Ken Manheimer wrote: > On Thu, 28 May 1998, The Dragon De Monsyne wrote: > > > I noticed the message about direct smtp delivery being needed for > > virtual hosting.... > > > > As I mentioned before, I have a version on mailman 1.0b1 that I am > > running that already does this. I am going to give a good shot at > > integrating that feature into the current (1.0b3?) source this weekend. > > (I want to anyway, as I want to upgrade my server. ) > > It looks like john was doing this, today, and all that will come out > with barry's big autoconf changes within the next few weeks - maybe > sooner - probably as 1.0b4. So you may not want to spend more time on > it. Yeah, I got smtp delivery done last night, and it seems to be working well. > > BTW, I also mentioned some user database changes in my variant of > > mailman... I'm dropping those, as I don't like the way I implemented them. > > I'll just try to incorporate the other features I added to 1.0b1 (smtp > > delivery being the biggie) into the current tree, and go from there. > > If you're interested in incorporating the other things, you should check > with barry about getting a current snapshot before doing to much > hacking... Poor Barry :) I can provide you with a snapshot as well. In fact, let me try to merge patches that have recently been posted to this list into the distribution, and then I'll put up a snapshot of the current version just for the people on this mailing list. It would not only be good for people wanting to integrate more changes, but it will be good for testing out newer code... John From viega@list.org Fri May 29 13:49:28 1998 From: viega@list.org (John Viega) Date: Fri, 29 May 1998 05:49:28 -0700 Subject: [Mailman-Developers] Some improvement suggestions In-Reply-To: ; from The Dragon De Monsyne on Thu, May 28, 1998 at 09:06:15PM -0500 References: <199805282356.TAA09183@glyph.CNRI.Reston.Va.US> Message-ID: <19980529054928.C25666@list.org> On Thu, May 28, 1998 at 09:06:15PM -0500, The Dragon De Monsyne wrote: > Hmm.. Has anyone looked at doing the whole shot with Digicool's > DocumentTemplate modules? The do both html and text, and their DTML is > alot more flexible than mailan's current 'magic-tag' system. It seems a > natural. Sounds good. I don't think anyone is persuing that right now, if you want to do it.. > IMHO, I realy think mailman needs a class of persistant Server > object (one for each virtual server) to store server-specific info which > really dosen't belong in the mailinglist object. Such an object could > store defaults in the same way as my UserDB did . It could also have > methods for things like getting the list of lists. Hmm, on one level it does belong in the mailing list object. For example, you might have the default set up to be: www.mymachine.com for both web stuff and list stuff, and someone might want to change it to be www.mymachine.com for the web page, and list.mymachine.com for the list. Or the other way around. I've seen a few lists use completely different domain names for the mail address and the web address. It's definitely a property of the list, although storing defaults on a per-server level would be good. I think I'd prioritize it fairly low, though, since creating new lists is not a frequently done thing. If you want to work on it, though, I do think it'd be a fine addition. From dragondm@nexus.Integral.org Fri May 29 13:23:54 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Fri, 29 May 1998 07:23:54 -0500 (CDT) Subject: [Mailman-Developers] Smtp delivery. In-Reply-To: Message-ID: On Fri, 29 May 1998, Ken Manheimer wrote: > > I noticed the message about direct smtp delivery being needed for > > virtual hosting.... > > > > As I mentioned before, I have a version on mailman 1.0b1 that I am > > running that already does this. I am going to give a good shot at > > integrating that feature into the current (1.0b3?) source this weekend. > > (I want to anyway, as I want to upgrade my server. ) > > It looks like john was doing this, today, and all that will come out > with barry's big autoconf changes within the next few weeks - maybe > sooner - probably as 1.0b4. So you may not want to spend more time on > it. Hmm.... > > > BTW, I also mentioned some user database changes in my variant of > > mailman... I'm dropping those, as I don't like the way I implemented them. > > I'll just try to incorporate the other features I added to 1.0b1 (smtp > > delivery being the biggie) into the current tree, and go from there. > > If you're interested in incorporating the other things, you should check > with barry about getting a current snapshot before doing to much > hacking... Yes. Where/how do I get the current system? -The Dragon De Monsyne From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri May 29 17:47:13 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 29 May 1998 12:47:13 -0400 (EDT) Subject: [Mailman-Developers] Smtp delivery. References: Message-ID: <13678.58953.409728.12328@anthem.CNRI.Reston.Va.US> >>>>> "KM" == Ken Manheimer writes: KM> If you're interested in incorporating the other things, you KM> should check with barry about getting a current snapshot KM> before doing to much hacking... One of the things I would really like to do, is to get a read-only public CVS tree available, so that it will be easier for developers such as Scott to integrate their changes, and stay as (painfully) up-to-date as they want :-). -Barry From viega@list.org Sat May 30 11:41:04 1998 From: viega@list.org (John Viega) Date: Sat, 30 May 1998 03:41:04 -0700 Subject: [Mailman-Developers] Cookies Message-ID: <19980530034104.A3940@list.org> I installed Scott's patches for confirmation and admin logins (thank god for ediff-buffers). I have a couple of questions mainly for Scott, but I think other people might be interested in discussing them. First, I don't know what the expiration time for cookies is, but the cookie didn't go away when I shut down my browser. Do you think that's good behavior? I'd like to not be implicitly logged in if someone else starts up my browser. Also, I've seen some sites that log people off automatically after 15 mins of inactivity on that site. Do you think that's a good idea? Second, if you don't have cookies on, changes don't get made. You get sent back to the login screen, and when you log back in, everything is the same. Should cookies really be required? Something that could be done to offer similar functionality yet not require cookies would be to have an "enter your password" box after the initial login, and put the password in the proper field as default text. While that may not be incredibly secure, it's not much worse than sending a plaintext password via httpd the first time only (although the password will be in the page source). Also, perhaps there should be a way to explicitly log out? I can't get logged out, even by turning off cookies! John From scott@chronis.pobox.com Sat May 30 19:23:02 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 30 May 1998 14:23:02 -0400 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530034104.A3940@list.org>; from John Viega on Sat, May 30, 1998 at 03:41:04AM -0700 References: <19980530034104.A3940@list.org> Message-ID: <19980530142302.24877@chronis.icgroup.com> After i released the admin patches, i realized that the way the cookies were put together was causing some problems. i rewrote the function isAuthenticated to work more like the same function in private. this change addresses much of what you are saying. i haven't posted about this till now because i've been quite busy with other things and unsure about the best way to submit a patch to a patch to a patch ;) On Sat, May 30, 1998 at 03:41:04AM -0700, John Viega wrote: | I installed Scott's patches for confirmation and admin logins (thank | god for ediff-buffers). I have a couple of questions mainly for | Scott, but I think other people might be interested in discussing | them. | | First, I don't know what the expiration time for cookies is, but the | cookie didn't go away when I shut down my browser. Do you think | that's good behavior? no. | I'd like to not be implicitly logged in if | someone else starts up my browser. Also, I've seen some sites that | log people off automatically after 15 mins of inactivity on that site. | Do you think that's a good idea? The cookies will not allow anyone to submit changes after the timeout period (defaulting to 20 minutes). I'm not sure how to portably force people to be logged off in any other way. | | Second, if you don't have cookies on, changes don't get made. You get | sent back to the login screen, and when you log back in, everything is | the same. Should cookies really be required? With the changed isAuthenticated function, an admin can enter the password on each screen to make changes, but will still have to log into each section separately :(. | Something that could be | done to offer similar functionality yet not require cookies would be | to have an "enter your password" box after the initial login, and put | the password in the proper field as default text. While that may not | be incredibly secure, it's not much worse than sending a plaintext | password via httpd the first time only (although the password will be | in the page source). | | Also, perhaps there should be a way to explicitly log out? that sounds like a good idea. my rewrite of the isAuthenticated function in the admin cgi follows: scott SECRET="monty" def isAuthenticated(list, password=None, SECRET="SECRET"): import base64, md5 if password is not None: # explicit login try: list.ConfirmAdminPassword(password) except mm_err.MMBadPasswordError: AddErrorMessage(doc, 'Error: Incorrect admin password.') return 0 token = md5.new(SECRET + list_name + SECRET).digest() token = base64.encodestring(token) token = string.strip(token) c = Cookie.Cookie() cookie_key = list_name + "-admin" c[cookie_key] = token c[cookie_key]['expires'] = mm_cfg.ADMIN_COOKIE_LIFE path = list.GetScriptURL("admin") path = path[string.find(path, "://") + 3:] path = path[string.find(path, "/"):] c[cookie_key]["path"] = path print c # Output the cookie return 1 if os.environ.has_key('HTTP_COOKIE'): c = Cookie.Cookie( os.environ['HTTP_COOKIE'] ) if c.has_key(list_name + "-admin"): inp = base64.decodestring(c[list_name + "-admin"].value) check = md5.new(SECRET+list_name+SECRET).digest() if inp == check: return 1 else: return 0 return 0 From viega@list.org Sat May 30 21:28:29 1998 From: viega@list.org (John Viega) Date: Sat, 30 May 1998 13:28:29 -0700 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530142302.24877@chronis.icgroup.com>; from Scott on Sat, May 30, 1998 at 02:23:02PM -0400 References: <19980530034104.A3940@list.org> <19980530142302.24877@chronis.icgroup.com> Message-ID: <19980530132829.C4221@list.org> On Sat, May 30, 1998 at 02:23:02PM -0400, Scott wrote: > > | I'd like to not be implicitly logged in if > | someone else starts up my browser. Also, I've seen some sites that > | log people off automatically after 15 mins of inactivity on that site. > | Do you think that's a good idea? > > The cookies will not allow anyone to submit changes after the timeout > period (defaulting to 20 minutes). I'm not sure how to portably force > people to be logged off in any other way. Well, what I'm talking about is the same thing, but if you press "submit" after the cookie runs out, you'll get an error message saying, "we've already logged you out due to inactivity", instead of just giving the login screen. Also, I noticed that it looks like the edithtml pages don't share the same cookie (and, in fact, aren't using cookies at all at the moment...) John From scott@chronis.pobox.com Sat May 30 21:37:38 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 30 May 1998 16:37:38 -0400 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530132829.C4221@list.org>; from John Viega on Sat, May 30, 1998 at 01:28:29PM -0700 References: <19980530034104.A3940@list.org> <19980530142302.24877@chronis.icgroup.com> <19980530132829.C4221@list.org> Message-ID: <19980530163738.06030@chronis.icgroup.com> On Sat, May 30, 1998 at 01:28:29PM -0700, John Viega wrote: | On Sat, May 30, 1998 at 02:23:02PM -0400, Scott wrote: | > | > | I'd like to not be implicitly logged in if | > | someone else starts up my browser. Also, I've seen some sites that | > | log people off automatically after 15 mins of inactivity on that site. | > | Do you think that's a good idea? | > | > The cookies will not allow anyone to submit changes after the timeout | > period (defaulting to 20 minutes). I'm not sure how to portably force | > people to be logged off in any other way. | | Well, what I'm talking about is the same thing, but if you press | "submit" after the cookie runs out, you'll get an error message | saying, "we've already logged you out due to inactivity", instead of | just giving the login screen. how do you tell the difference between a cookie running out and no cookie being submitted in the first place? | Also, I noticed that it looks like the edithtml pages don't share the | same cookie (and, in fact, aren't using cookies at all at the moment...) true. it seems like they should more for the sake of uniformity than anything else, as there's no reason to hide publicly accessible html pages from people. this shouldn't be too hard to do with the existing isAuthenticated function from the admin cgi. i don't think i'll have time to that before i go away June 1-10, but i'm willing to change edithtml when i get back if no one's done it yet. scott From viega@list.org Sat May 30 21:42:42 1998 From: viega@list.org (John Viega) Date: Sat, 30 May 1998 13:42:42 -0700 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530163738.06030@chronis.icgroup.com>; from Scott on Sat, May 30, 1998 at 04:37:38PM -0400 References: <19980530034104.A3940@list.org> <19980530142302.24877@chronis.icgroup.com> <19980530132829.C4221@list.org> <19980530163738.06030@chronis.icgroup.com> Message-ID: <19980530134242.D4221@list.org> On Sat, May 30, 1998 at 04:37:38PM -0400, Scott wrote: > > how do you tell the difference between a cookie running out and no > cookie being submitted in the first place? I think by checking to see if there are extra CGI parameters passed? If so, you can assume they were still on your page. > | Also, I noticed that it looks like the edithtml pages don't share the > | same cookie (and, in fact, aren't using cookies at all at the moment...) > > true. it seems like they should more for the sake of uniformity than > anything else, as there's no reason to hide publicly accessible html > pages from people. this shouldn't be too hard to do with the existing > isAuthenticated function from the admin cgi. i don't think i'll have > time to that before i go away June 1-10, but i'm willing to change > edithtml when i get back if no one's done it yet. Would it be possible to keep 1 central auth checking function? I also noticed that admindb needs a similar mechanism. Also, when we add a site administrator's UI, we'll need to use the same functionality again... John From scott@chronis.pobox.com Sat May 30 21:50:58 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 30 May 1998 16:50:58 -0400 Subject: [Mailman-Developers] Cookies In-Reply-To: <19980530134242.D4221@list.org>; from John Viega on Sat, May 30, 1998 at 01:42:42PM -0700 References: <19980530034104.A3940@list.org> <19980530142302.24877@chronis.icgroup.com> <19980530132829.C4221@list.org> <19980530163738.06030@chronis.icgroup.com> <19980530134242.D4221@list.org> Message-ID: <19980530165058.09702@chronis.icgroup.com> On Sat, May 30, 1998 at 01:42:42PM -0700, John Viega wrote: | On Sat, May 30, 1998 at 04:37:38PM -0400, Scott wrote: | > | > how do you tell the difference between a cookie running out and no | > cookie being submitted in the first place? | | I think by checking to see if there are extra CGI parameters passed? | If so, you can assume they were still on your page. that's a good idea. | > | Also, I noticed that it looks like the edithtml pages don't share the | > | same cookie (and, in fact, aren't using cookies at all at the moment...) | > | > true. it seems like they should more for the sake of uniformity than | > anything else, as there's no reason to hide publicly accessible html | > pages from people. this shouldn't be too hard to do with the existing | > isAuthenticated function from the admin cgi. i don't think i'll have | > time to that before i go away June 1-10, but i'm willing to change | > edithtml when i get back if no one's done it yet. | | Would it be possible to keep 1 central auth checking function? I also | noticed that admindb needs a similar mechanism. Also, when we add a | site administrator's UI, we'll need to use the same functionality | again... i'm sure it would be possible, and thought that maybe such things should go in a general mailman cgi library, maybe in modules/mm_cgilib.py? i'm not sure where the best place for that is. scott From viega@list.org Sun May 31 03:35:46 1998 From: viega@list.org (John Viega) Date: Sat, 30 May 1998 19:35:46 -0700 Subject: [Mailman-Developers] Pipermail Message-ID: <19980530193546.A8889@list.org> Has anyone modified pipermail's hypermail emulation to work using only modules that are compiled in the default Python distribution? There's one dependency I have in mind: pipermail currently uses bsddb, which is not compiled into Python 1.5. I think gdbm is compiled in by default, but the two interfaces aren't compatible. My biggest goal w/ Mailman has always been to have everything people would ever want to do be easy to do. That's why I'm pretty reluctant to distribute without built-in archiving - one thing I hated about Majordomo was having to go out and download extras and then figure out how to integrate them to get useful functionality. I also believe that many people who will want to use Mailman will be using Python for the first time. The default Python installation is really easy to build. But as soon as you tell people, "Hey, now you have to recompile, and install crypt. Oh, and now you have to go download pipermail. And before you use that, you have to find bsddbm on the net somewhere, and install it", that's going to send people running back to the comfort of Perl and Majordomo, even though they'll be doing the same sorts of things. To that end, I've put in provisions for passwords to use md5 if crypt is not found. BTW, do you guys think md5 should be the default, and crypt an option for those who add it to their config? The only real advantage of crypt is that it is less space intensive. Also, the way I have things set up at the moment, if someone starts w/ the default distribution and later recompiles Python to include crypt, all the old passwords won't work, since they'll be in the md5 digest format. John From scott@chronis.pobox.com Sun May 31 20:12:04 1998 From: scott@chronis.pobox.com (Scott) Date: Sun, 31 May 1998 15:12:04 -0400 Subject: [Mailman-Developers] nasty admin Cookie bug (and fix) Message-ID: <19980531151204.04788@chronis.icgroup.com> there is a bug in the admin cgi changes that i posted. the block of code making general list changes fires from some brownsers when you log on and press enter to submit the data instead of clicking the submit button. the reason is that the code knows to process the login when the button is pressed because of it's name. that data is not submitted from some browsers when you submit the form by pressing enter instead of clicking on the button. since not much data is submitted, it wipes out a lot of the list information. bad bad bad! anyway, the fix is simple enough (against the previous patch and the substituted isAuthenticted function). scott chronis 3:01pm $ cvs diff -r1.4 admin Index: admin =================================================================== RCS file: /usr/local/cvsroot/mailman/cgi/admin,v retrieving revision 1.4 diff -r1.4 admin 689c690,692 < if category != 'members' and not cgi_info.has_key("request_login"): --- > if category != 'members' and \ > not cgi_info.has_key("request_login") and \ > len(cgi_info.keys()) > 1: