From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 1 00:48:08 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 31 Jan 2000 19:48:08 -0500 (EST) Subject: [Mailman-Developers] SourceForge Message-ID: <14486.11592.419220.75729@anthem.cnri.reston.va.us> What would you folks think about moving the CVS repository for Mailman over to SourceForge.net? They even /use/ Mailman over there. Our CVS repository is running on a pokey Sparc5 and getting others write access to the tree is a pain (not that anybody's clamoring for that). I like the idea of SourceForge and we had a bunch of people raving about it at the Python conference last week. Anybody have any negative experiences with it? -Barry From Nigel.Metheringham@vdata.co.uk Tue Feb 1 13:36:46 2000 From: Nigel.Metheringham@vdata.co.uk (Nigel Metheringham) Date: Tue, 01 Feb 2000 13:36:46 +0000 Subject: [Mailman-Developers] Bugs in weekly archiving - mailman 1.1? Message-ID: I have a problem that has shown up since I changed to mailman 1.1 (from 1.0). Coincidently the platform the system ran on was upgraded to RH 6.1 linux (this was all due to a hardware failure) - I think this upgraded python and glibc. Heavy traffic lists I archive on a weekly basis. At the start of each month it produces invalid datestamps, ie today Week-of-Mon-20000200 Week-of-Mon-200001-4 Week-of-Mon-199912-1 In the source, Mailman/Archiver/HyperArch.py there is the code:- elif self.ARCHIVE_PERIOD == 'week': datetuple = list(datetuple) datetuple[2] = datetuple[2] - datetuple[6] # subtract week day # # even if the the day of the month counter is negative, # we still get the right thing from strftime! -scott # return time.strftime("Week-of-Mon-%Y%m%d", tuple(datetuple)) I've test bedded this code under perl (which I can write rather better) and the system/library strftime() behaves as "scott" says - so putting in the -4th of Jan 2000 gives strftime output of 27th Dec 1999. I'm starting to wonder if python put a strftime implementation into its library functions which does not match these assumptions... although it does appear to link against one in the C libraries. All in all I'm a tad confused over what is causing this problem. I could work round this by changing that code to read datetuple=time.gmtime(date - (datetuple[6] * 86400)) which will always work - although its more costly on CPU. Its also working around a problem that should not be there :-) Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From jwing@pliantsystems.com Tue Feb 1 20:14:55 2000 From: jwing@pliantsystems.com (John Wingenbach) Date: Tue, 01 Feb 2000 15:14:55 -0500 Subject: [Mailman-Developers] script to modify arbitrary value in config.db Message-ID: <38973EBF.7C91C23D@pliantsystems.com> I created the following python script to modify an arbitrary value within the config.db file. However, it seems to not correctly work. The value gets changed albeit it doesn't register correctly in the web. I admit to not being a python scripter. :-) Where am I messed up? TIA, John #! /usr/bin/env /share/bin/python import marshal import sys def main(argv): file='/usr/local/mailman/lists/' + argv[1] + '/config.db' fp = open(file, 'r') dict = marshal.load(fp) fp.close(); oldval = dict[argv[2]] dict[argv[2]] = argv[3] print argv[2], ' was: >', oldval, '< now is: >', dict[argv[2]], '<' fp = open(file, 'w') marshal.dump(dict, fp) fp.close(); if __name__ == "__main__": raise SystemExit(main(sys.argv)) -- John C. Wingenbach Pliant Systems, Inc. Sr. Systems Administrator Work: (919) 405-4627 Fax: (919) 405-4544 From thomas@xs4all.net Wed Feb 2 13:00:42 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Feb 2000 14:00:42 +0100 Subject: [Mailman-Developers] race condition in locking ? Message-ID: <20000202140042.J19725@xs4all.nl> I've finally found the time to do some serious testing of mailman (that is, set up a mailman list and play with it ;) and I was hit by what looks like a race condition in the locking. I was sending a lot of small mails to the mailman-test list, using echo "foobar foobar baz baz baz" | mail -s "foobar" mailman-test@xs4all.nl and just manually hitting arrow-up, enter, arrow-up, enter, etc. I'd subscribed a bouncing email address and wanted to see what would happen ;) In any case, the machine mailman runs on is also used for some other things; majordomo and our 'batched-smtp' product. Both services generate very variable loads, majordomo because of some very large lists, and batched-smtp due to customers with very large mailqueues dialing in and starting a queuerun. I dont send the mail from the same machine, so the mails I sent probably arrived in one sendmail session, and then were executed by the queueing sendmail right after another. Some of the messages (around 10/15 per incident) I'd sent then bounced, most with the following traceback: Traceback (innermost last): File "/usr/local/mailman/scripts/mailowner", line 69, in ? main() File "/usr/local/mailman/scripts/mailowner", line 48, in main mlist = MailList.MailList(sys.argv[1]) File "/usr/local/mailman/Mailman/MailList.py", line 69, in __init__ self.Load() File "/usr/local/mailman/Mailman/MailList.py", line 824, in Load self.Lock() File "/usr/local/mailman/Mailman/MailList.py", line 1255, in Lock self.__lock.lock() File "/usr/local/mailman/Mailman/LockFile.py", line 209, in lock os.link(self.__lockfile, self.__tmpfname) OSError: [Errno 2] No such file or directory And a couple with the following traceback: Traceback (innermost last): File "/usr/local/mailman/scripts/post", line 86, in ? main() File "/usr/local/mailman/scripts/post", line 62, in main mlist.Post(msg) File "/usr/local/mailman/Mailman/MailList.py", line 1249, in Post self.Save() File "/usr/local/mailman/Mailman/MailList.py", line 814, in Save Utils.reraise() File "/usr/local/mailman/Mailman/MailList.py", line 811, in Save os.link(fname, fname_last) OSError: [Errno 17] File exists The first traceback is probably because one process does a kickstart, erasing the lockfile and starting a new one, while a lot of other processes are trying to link() to it. The second traceback looks like a lock was stolen from an active process, in other words a real locking failure. The lock-logfile suggests some inefficient locking, too; long pauses between unlocks and locks: (18502) mailman-test.lock waiting for claim (18492) mailman-test.lock unlocked (18513) mailman-test.lock waiting for claim (18584) mailman-test.lock waiting for claim Feb 01 18:53:08 2000 (18622) mailman-test.lock kickstarted (18425) mailman-test.lock waiting for claim (18504) mailman-test.lock waiting for claim (18456) mailman-test.lock waiting for claim (18440) mailman-test.lock waiting for claim (18622) mailman-test.lock waiting for claim (18467) mailman-test.lock waiting for claim (18584) mailman-test.lock waiting for claim (18443) mailman-test.lock waiting for claim (18500) mailman-test.lock waiting for claim (18502) mailman-test.lock waiting for claim (18513) mailman-test.lock waiting for claim (18526) mailman-test.lock waiting for claim (18504) mailman-test.lock waiting for claim (18440) mailman-test.lock waiting for claim (18425) mailman-test.lock waiting for claim (18456) mailman-test.lock waiting for claim (18622) mailman-test.lock waiting for claim (18443) mailman-test.lock waiting for claim (18513) mailman-test.lock waiting for claim (18467) mailman-test.lock waiting for claim (18584) mailman-test.lock waiting for claim (18500) mailman-test.lock waiting for claim (18502) mailman-test.lock waiting for claim (18526) mailman-test.lock waiting for claim (18440) mailman-test.lock waiting for claim (18504) mailman-test.lock waiting for claim (18425) mailman-test.lock waiting for claim (18456) mailman-test.lock waiting for claim (18622) mailman-test.lock waiting for claim (18443) mailman-test.lock got the lock (This is with the default sleep time of .25) This all led me to believe there was something wrong with locking, and I spent the mornings' traintrip from home to work (1.5 hours) reading LockFile.py and thinking about the locking. And I think the logic of LockFile.lock() is flawed :P lock() seems to rely on two things: os.stat() returning the right number of links for the lockfile (which is a fairly safe assumption) and the contents of the lockfile, which should give an indication as to who locked it and for how long. But there is at least one situation possible where one process thinks it has the lock, and another thinks the lock is faulty and does a forced kickstart: process 1 process 2 os.link(lockfile,tmpfile) os.stat(lockfile) == 2 os.link(lockfile, tmpfile) os.stat(lockfile) == 3 self.__write(): open(tmpfile, "w") self.__read() write(os.getpid(), etc) __kickstart(force=1) return I think I was able to reproduce this on my laptop by stressing it fairly hard (load of 15 and rising.) The testcode only acquired and released locks, with some small sleep inbetween, but I saw several double-locks, and one OSError on the os.link(). I also think I see a way to fix this problem, by checking the mtime of the file if the __read() fails. I think we can safely say that if the __read() fails AND the mtime is recent, someone else is either assuming they have the lock, or is kickstart()ing, in both of which cases we should step back and start over. Comments ? Am I trespassing on anyones' work here ? Is this part of mailman still being looked at ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Nigel.Metheringham@vdata.co.uk Wed Feb 2 18:10:20 2000 From: Nigel.Metheringham@vdata.co.uk (Nigel Metheringham) Date: Wed, 02 Feb 2000 18:10:20 +0000 Subject: [Mailman-Developers] Bugs in weekly archiving - mailman 1.1? In-Reply-To: Message from Nigel Metheringham of "Tue, 01 Feb 2000 13:36:46 GMT." Message-ID: This is a multipart MIME message. --==_Exmh_-12073758350 Content-Type: text/plain; charset=us-ascii [copied into mailman-users since the followup question is more appropriate there... please keep discussion onto users from now on] I've built a patch to fix the weekly archive bug I see in mailman 1.1 - this fixes the problem of abusing strftime and hoping it works (it does mostly but I have exceptions). Anyone know a neat way of rearchiving the misarchived messages (I can do this by manually pulling them out the mbox archive) and losing the broken week archives - ie I currently have a Week-of-Mon-20000200 which is no use to man nor beast, but needs more than just a subtle delete to get rid of... Nigel. --==_Exmh_-12073758350 Content-Type: application/x-patch ; name="mm.patch" Content-Description: mm.patch Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="mm.patch" LS0tIE1haWxtYW4vQXJjaGl2ZXIvSHlwZXJBcmNoLnB5Lm9sZAlXZWQgRmViICAyIDE3OjQy OjE1IDIwMDAKKysrIE1haWxtYW4vQXJjaGl2ZXIvSHlwZXJBcmNoLnB5CVdlZCBGZWIgIDIg MTc6NTE6MTUgMjAwMApAQCAtNjU3LDkgKzY1NywxNyBAQAogCSAgICBkYXRldHVwbGUgPSBs aXN0KGRhdGV0dXBsZSkKIAkgICAgZGF0ZXR1cGxlWzJdID0gZGF0ZXR1cGxlWzJdIC0gZGF0 ZXR1cGxlWzZdICMgc3VidHJhY3Qgd2VlayBkYXkKIAkgICAgIwotCSAgICAjIGV2ZW4gaWYg dGhlIHRoZSBkYXkgb2YgdGhlIG1vbnRoIGNvdW50ZXIgaXMgbmVnYXRpdmUsCi0JICAgICMg d2Ugc3RpbGwgZ2V0IHRoZSByaWdodCB0aGluZyBmcm9tIHN0cmZ0aW1lISAtc2NvdHQKKwkg ICAgIyBpZiB0aGUgdGhlIGRheSBvZiB0aGUgbW9udGggY291bnRlciBpcyBuZWdhdGl2ZQor CSAgICAjIHdlIGhhdmUgdG8gcmVjYWxjdWxhdGUgdGhlIHdob2xlIHRoaW5ncyBzaW5jZQor ICAgICAgICAgICAgIyBoYW5kbGluZyBjcm9zc2luZyBiYWNrIGJ5IG1vbnRoIG9yIGV2ZW4g eWVhcgorICAgICAgICAgICAgIyBpcyB0b28gaGFyZC4KKyAgICAgICAgICAgICMgVGhlcmUg aXMgdGhlb3JldGljYWxseSBhbiBpc3N1ZSB3aXRoIGNyb3NzaW5nCisgICAgICAgICAgICAj IGJhY2sgYSBEU1QgY2hhbmdlb3ZlciBib3VuZHJ5IGhlcmUgYXMgd2VsbCAtIHRoaXMKKyAg ICAgICAgICAgICMgc2hvdWxkIGJlIGhhbmRsZWQgYnkgZ210aW1lLCBidXQgSSBoYXZlIHNl ZW4KKyAgICAgICAgICAgICMgbmFzdHkgaW50ZXJhY3Rpb25zIHdpdGggc3RyZnRpbWUgYW5k IGxvY2FsZXMKIAkgICAgIworICAgICAgICAgICAgaWYgZGF0ZXR1cGxlWzJdIDw9IDA6Cisg ICAgICAgICAgICAgICAgZGF0ZXR1cGxlID0gdGltZS5nbXRpbWUoZGF0ZSAtIChkYXRldHVw bGVbNl0gKiA4NjQwMCkpCiAJICAgIHJldHVybiB0aW1lLnN0cmZ0aW1lKCJXZWVrLW9mLU1v bi0lWSVtJWQiLCB0dXBsZShkYXRldHVwbGUpKQogICAgICAgICAjIG1vbnRoLiAtZGRtCiAg CWVsc2U6Cg== --==_Exmh_-12073758350 Content-Type: text/plain; charset=us-ascii [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] --==_Exmh_-12073758350-- From thomas@xs4all.net Wed Feb 2 21:33:19 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Feb 2000 22:33:19 +0100 Subject: [Mailman-Developers] SourceForge In-Reply-To: <14486.11592.419220.75729@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Mon, Jan 31, 2000 at 07:48:08PM -0500 References: <14486.11592.419220.75729@anthem.cnri.reston.va.us> Message-ID: <20000202223319.A17313@xs4all.nl> On Mon, Jan 31, 2000 at 07:48:08PM -0500, Barry A. Warsaw wrote: > What would you folks think about moving the CVS repository for Mailman > over to SourceForge.net? They even /use/ Mailman over there. Our CVS > repository is running on a pokey Sparc5 and getting others write > access to the tree is a pain (not that anybody's clamoring for that). > I like the idea of SourceForge and we had a bunch of people raving > about it at the Python conference last week. Anybody have any > negative experiences with it? I haven't heard anything bad about SourceForge yet, but not that much goods either. I wasn't at IPC8 ;) But I'm planning to do some of the things on the mailman todo list (site-admin pages, for instance) and I'm already 'fixing' (from my point of view, see my other postings) the locking scheme, and I would like to know how to get them included in the CVS tree. I dont care wether it's CVS write access or simply a place to send cvs diffs, though ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Wed Feb 2 21:53:46 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Feb 2000 22:53:46 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000202140042.J19725@xs4all.nl>; from thomas@xs4all.net on Wed, Feb 02, 2000 at 02:00:42PM +0100 References: <20000202140042.J19725@xs4all.nl> Message-ID: <20000202225346.B17313@xs4all.nl> On Wed, Feb 02, 2000 at 02:00:42PM +0100, Thomas Wouters wrote: > This all led me to believe there was something wrong with locking, and I > spent the mornings' traintrip from home to work (1.5 hours) reading > LockFile.py and thinking about the locking. And I think the logic of > LockFile.lock() is flawed :P And today I figured out what exactly is flawed. The locking mechanism is backwards ! Maybe it's intended that way, it seems that the instructions for this type of locking is removed from 'the' linux open manpage (my rh6 boxes no longer have any talk about locking) so I can't double-check, but I doubt this is what it describes. We use a very similar locking scheme in our slightly adjusted mail.local (originally sendmails) which, if memory serves, was suggested by some manpage or other. But not Linux' open() manpage. In any case, we use it the other way 'round from mailman, and with great success. Mailman currently locks by creating a link from the lockfile to a private lockfile, and checking that the link-count is 2. If it isn't, it reads the file to see who is owning the lockfile, to check on their health etc. The problem is that, if a lot of processes are doing the same thing, you will very often have more than 2 links to the file. Doing some stresstesting with the default timeout values I saw as much as 25 concurrent links to the file, and because every mailman process uses the exact same sleep time, the chances of breaking out of it are not as large as they could be. The end result is a lot of processes trying to lock the file, which is unlocked the whole time, because the processes trying to grab it keep headbutting as they try to grab the lock. The way we do locking in mail.local (and this works at least C, on 4 different operating systems, over NFS ;) is that every process that tries to obtain the lock, creates a private lockfile, and then links from the private lockfile to the intended one. Each process can then very easily see wether they have the lock or not by stat()ing their _own_ lockfile and looking at the link count. Because link() will fail when the new file to link already exists, it is impossible to overwrite someone else's lock. And because the private lockfile is created in advance, it's also possible to write the lock-info mailman uses, into it _in_advance_, removing another race condition in the locking mechanism. (namely the code checking to see if the contents is valid, and if not, removing it.) I'm busy rewriting LockFile.py to use the new locking (actually, I've rewritten it, I've just not tested it yet ;) and I should be able to post it tomorrow... Unless anyone has objects to this locking method. Did I miss anything ? Is there anyone I should talk to before changing things, like the original authors ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Feb 2 22:20:37 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 2 Feb 2000 17:20:37 -0500 (EST) Subject: [Mailman-Developers] race condition in locking ? References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> Message-ID: <14488.44469.446995.168328@anthem.cnri.reston.va.us> >>>>> "TW" == Thomas Wouters writes: TW> I'm busy rewriting LockFile.py to use the new locking TW> (actually, I've rewritten it, I've just not tested it yet ;) TW> and I should be able to post it tomorrow... Unless anyone has TW> objects to this locking method. Did I miss anything ? Is there TW> anyone I should talk to before changing things, like the TW> original authors ? Thomas, I've only skimmed your emails but I think your analysis is basically correct, and the solution you propose here seems reasonable. This mailing list is the right place to discuss it, and although I've been too busy to respond lately, I /will/ see your message, and your changes. My suggestions would be these: be sure to look at the latest version of LockFile.py from the CVS repository. It doesn't fix your problems but it embodies the current semantics I want from the API. Second: if you submit substantial changes (as I suspect you will), we'll need to get FSF assignments from you. I'll send you the appropriate form when I see how big your changes are, but I wanted to give you a heads up. Thanks very much for looking into this. -Barry From thomas@xs4all.net Thu Feb 3 09:35:55 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 3 Feb 2000 10:35:55 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <14488.44469.446995.168328@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Wed, Feb 02, 2000 at 05:20:37PM -0500 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <14488.44469.446995.168328@anthem.cnri.reston.va.us> Message-ID: <20000203103555.C6378@xs4all.nl> --J2SCkAp4GZ/dPZZf Content-Type: text/plain; charset=us-ascii On Wed, Feb 02, 2000 at 05:20:37PM -0500, Barry A. Warsaw wrote: > I've only skimmed your emails but I think your analysis is basically > correct, and the solution you propose here seems reasonable. This > mailing list is the right place to discuss it, and although I've been > too busy to respond lately, I /will/ see your message, and your > changes. Okay, I figured you're a busy man, thanx for giving me a fast answer. > My suggestions would be these: be sure to look at the latest version > of LockFile.py from the CVS repository. It doesn't fix your problems > but it embodies the current semantics I want from the API. I was already using the CVS version, so no worries. > Second: if you submit substantial changes (as I suspect you will), > we'll need to get FSF assignments from you. I'll send you the > appropriate form when I see how big your changes are, but I wanted to > give you a heads up. Well, attached you'll find the first change, the cvs diff to LockFile.py. My apologies for this rather long email, but it's as much an explanation to this list about the changes I made, as a way for me to get this info out of my subconcious and into straight english. No need to read this all if you read the source and understand it, or if you dont care about locking ;) I kept the API the same with one teensy exception. With the new locking scheme, it's not possible to 'steal' a lock, only to break the lock and then enter the lock() cycle to try and grab it. So steal() might block indefinately (though this should only occur on very heavily loaded machines, with a lot of mailmen trying to grab the same lock.) I've also added some semi-random (more like 'not very regular') factor to the default sleep timeout, to slightly try to avoid running processes 'in stride'. In the previous locker, processes that were walking in stride would be guaranteed not to get the lock, and could only get out of it by irregularities in the python execution speed and/or OS speed. In the new locker collisions are much less harmful, but it's still nice to try and avoid running in stride. This version still has some debug info inserted, to test the timing of the locks and the like, and it could be speeded up a fair bit once we're satisfied that it works properly. Basically, how it works is like this: The locker creates a private lockfile (__tmpfname) containing pid, tmpfname and expected lifetime, and tries to make __lockfile a hard link to this file. This'll fail when __lockfile already exists, or when __tmpfname has been deleted (this shouldn't happen, though.) If the link() fails, the locker tries to read the contents of the lockfile, to find out who is locking it. If the read fails (the file does not exist, or the contents was of unexpected format) or the lock is too old, it tries to break the lock, using __break(). __break() checks __lockfile's CTIME, the inode change time, to see if it has changed recently -- which would indicate that the lock-owner is still alive, and that the lock contents might have changed since we last checked it. (CTIME is updated by writing to it, opening it for writing, and link()ing, but not by reading from it. We only ever write/link to our own lockfile, not the global one, so if there is write activity, it wasn't us. We do have to flush() or close() after write()ing, however, to make sure the changes get written out.) Note that __write() writes to the private lockfile, whereas __read() reads from the global one, and that there is a lot of redundancy in the lock-checking; ordinarily, the nlink-check should be enough to determine wether we have the lock or not. Also note that the locker no longer cleans up other mailmens lockfiles. If a mailman crashes and leaves its own lockfile, the file will not be deleted -- perhaps a default cronjob to clear old locks is in order, I'll think about that. In any case, old lockfiles shouldn't be a problem even if another mailman comes along on the same machine, with the same pid, trying to lock the same file, as it will simply use the file already there, and delete it when it's done. The other api functions (refresh, set_lifetime, unlock, locked) all should work exactly the same, only better -- in particular refresh() no longer exposes a race condition, thanks to the ctime check in __break(). And because the new locking scheme guarantees that the first attempt to get the lock after an unlock(), will succeed, the default sleep interval can be safely raised. I've set the default to 1 second, which is just above the average lock age on my listserver, given decent load. I've tested this locker quite heavily, mailbombing my listserver with 100+ simultaneous messages to mailman-test (which generated 100+ bounces sent back to mailman, because my bouncing email address is still on the list) and it hasn't barfed yet ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --J2SCkAp4GZ/dPZZf Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mailman.cvs.diff" Index: Mailman/LockFile.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/LockFile.py,v retrieving revision 1.13 diff -u -r1.13 LockFile.py --- LockFile.py 1999/12/14 04:22:05 1.13 +++ LockFile.py 2000/02/03 09:30:33 @@ -27,6 +27,8 @@ import errno #from stat import ST_NLINK ST_NLINK = 3 # faster +ST_CTIME = 9 +ST_INO = 1 # default intervals are both specified in seconds, and can be floating point @@ -36,7 +38,8 @@ # sleep before checking the lock's status, if we were not the winning claimant # the previous time around. DEFAULT_LOCK_LIFETIME = 15 -DEFAULT_SLEEP_INTERVAL = .25 +DEFAULT_SLEEP_INTERVAL = 1.0 +DEFAULT_INODE_GRACETIME = 2 from Mailman.Logging.StampedLogger import StampedLogger @@ -79,20 +82,21 @@ lifetime is the maximum length of time expected to keep this lock. This value is written into the lock file so that other claimants on - the lock know when it is safe to steal the lock, should the lock + the lock know when it is safe to break the lock, should the lock holder be wedged. - sleep_interval is how often to wake up and check the lock file + sleep_interval is roughly how often to wake up and check the lock file """ self.__lockfile = lockfile self.__lifetime = lifetime - self.__sleep_interval = sleep_interval + self.__sleep_interval = sleep_interval + (((os.getpid() % 20) - 11.5)/100.) self.__tmpfname = "%s.%s.%d" % (lockfile, socket.gethostname(), os.getpid()) self.__withlogging = withlogging - self.__kickstart() + self.__write() + self.__writelog("starting") def set_lifetime(self, lifetime): """Reset the lifetime of the lock. @@ -129,28 +133,6 @@ if self.locked(): self.unlock() - def __kickstart(self, force=0): - # forcing means to remove the original lock file, and create a new - # one. this might be necessary if the file contains bogus locker - # information such that the owner of the lock can't be determined - if force: - try: - os.unlink(self.__lockfile) - except IOError: - pass - if not os.path.exists(self.__lockfile): - try: - # make sure it's group writable - oldmask = os.umask(002) - try: - file = open(self.__lockfile, 'w+') - file.close() - finally: - os.umask(oldmask) - except IOError: - pass - self.__writelog('kickstarted') - def __write(self): # we expect to release our lock some time in the future. we want to # give other claimants some clue as to when we think we're going to be @@ -169,22 +151,38 @@ os.umask(oldmask) def __read(self): - # can raise ValueError in two situations: + # can raise ValueError in three situations: # - # either first element wasn't an integer (a valid pid), or we didn't - # get a sequence of the right size from the string.split. Either way, - # the data in the file is bogus, but this is caught and handled higher - # up. - fp = open(self.__tmpfname, 'r') + # - The lockfile does not exist (it's been released or broken) + # - We didn't get a sequence of the right size from split + # - the first 'word' is not an integer + # In each case we raise ValueError in response + try: + fp = open(self.__lockfile, 'r') + except IOError, err: + if err.errno == errno.ENOENT: + raise ValueError # invalid contents try: pid, winner, lockrelease = string.split(string.strip(fp.read())) finally: fp.close() return int(pid), winner, float(lockrelease) - # Note that no one new can grab the lock once we've opened our tmpfile - # until we close it, even if we don't have the lock. So checking the PID - # and stealing the lock are guaranteed to be atomic. + def __break(self): + try: + if os.stat(self.__lockfile)[ST_CTIME] > time.time() - DEFAULT_INODE_GRACETIME: + # The lockfile has changed very recently, so it might have + # changed since our caller evaluated the validity + return + except OSError, err: + if err.errno == errno.ENOENT: + return # Someone broke it already + raise + os.unlink(self.__lockfile) + + # Note that this function is mostly redundant, the try/except around + # os.link does almost all of what we do. Instead of lockfile-contents + # we could use lockfile CTIME, which would probably speed things up. def lock(self, timeout=0): """Blocks until the lock can be obtained. @@ -197,21 +195,42 @@ synchronized. """ - if timeout > 0: - timeout_time = time.time() + timeout - last_pid = -1 if self.locked(): self.__writelog('already locked') raise AlreadyLockedError - stolen = 0 + if timeout: + timeout_time = time.time() + timeout + last_pid = -1 + self.__write() # Make sure our own lockfile exists while 1: # create the hard link and test for exactly 2 links to the file - os.link(self.__lockfile, self.__tmpfname) + try: + os.link(self.__tmpfname, self.__lockfile) + except OSError, err: + if err.errno == errno.ENOENT: + # Our own lockfile has vanished from beneath us ? + self.__write() + # sleep anyway, so we dont hog the machine + self.__sleep() + continue + elif err.errno == errno.EEXIST: + # We failed to get the lock. If we worry about this locking + # scheme, we set a variable here and ascertain ourselves of + # its value later, when we think we have the lock. + pass + else: + raise + if os.stat(self.__tmpfname)[ST_NLINK] == 2: - # we have the lock (since there are no other links to the lock - # file), so we can piss on the hydrant + # we have the lock (since noone overwrote our link to the lock + # file), so we re-piss our hydrant, to refresh the lock timeout self.__write() - self.__writelog('got the lock') + now = time.time() + if not self.locked(): + self.__writelog('false positive on lockfile (%f)'%now) + self.__sleep() + continue + self.__writelog('got the lock (%f)'%now) break # we didn't get the lock this time. let's see if we timed out if timeout and timeout_time < time.time(): @@ -220,83 +239,34 @@ raise TimeOutError # someone else must have gotten the lock. let's find out who it # is. if there is some bogosity in the lock file's data then we - # will steal the lock. + # will try to steal the lock. try: pid, winner, lockrelease = self.__read() except ValueError: - os.unlink(self.__tmpfname) - self.__kickstart(force=1) + self.__break() + self.__sleep() continue - # If we've gotten to here, we should not be the winner, because - # otherwise, an AlreadyCalledLockError should have been raised - # above, and we should have never gotten into this loop. However, - # the following scenario can occur, and this is what the stolen - # flag takes care of: - # - # Say that processes A and B are already laying claim to the lock - # by creating link files, and say A actually has the lock (i.e., A - # is the winner). We are process C and we lay claim by creating a - # link file. All is cool, and we'll trip the pid <> last_pid test - # below, unlink our claim, sleep and try again. Second time - # through our loop, we again determine that A is the winner but - # because it and B are swapped out, we trip our lifetime test - # and figure we need to steal the lock. So we piss on the hydrant - # (write our info into the lock file), unlink A's link file and go - # around the loop again. However, because B is still laying - # claim, and we never knew it (since it wasn't the winner), we - # again have 3 links to the lock file the next time through this - # loop, and the assert will trip. - # - # The stolen flag alerts us that this has happened, but I still - # worry that our logic might be flawed here. - assert stolen or winner <> self.__tmpfname - # record the identity of the previous winner. lockrelease is the - # expected time that the winner will release the lock by. we - # don't want to steal it until this interval has passed, otherwise - # we could steal the lock out from underneath that process. + + # check for hanging processes, by remembering pid, and if it stays + # the same, checking lockrelease. if pid <> last_pid: last_pid = pid - # here's where we potentially steal the lock. if the pid in the - # lockfile hasn't changed by lockrelease (a fixed point in time), - # then we assume that the locker crashed + elif lockrelease < time.time(): - self.__write() # steal - self.__writelog('stolen!') - stolen = 1 - try: - os.unlink(winner) - except os.error: - # winner lockfile could be missing - pass - try: - os.unlink(self.__tmpfname) - except os.error, (code, msg): - # Let's say we stole the lock, but some other process's - # claim was never cleaned up, perhaps because it crashed - # before that could happen. The test for acquisition of - # the lock above will fail because there will be more than - # one hard link to the main lockfile. But we'll get here - # and winner==self.__tmpfname, so the unlink above will - # fail (we'll have deleted it twice). We could just steal - # the lock, but there's no reliable way to clean up the - # stale hard link, so we raise an exception instead and - # let the human operator take care of the problem. - if code == errno.ENOENT: - self.__log('stale lockfile found') - raise StaleLockFileError( - 'Stale lock file found linked to file: ' - +self.__lockfile+' (requires '+ - 'manual intervention)') - else: - raise + # We've waited long enough... + # Attempt to break the lock. We dont care much about the old + # winners __tmpfname, let it stay. + self.__break() + self.__sleep() continue - # okay, someone else has the lock, we didn't steal it, and our - # claim hasn't timed out yet. So let's wait a while for the owner - # of the lock to give it up. Unlink our claim to the lock and - # sleep for a while, then try again - os.unlink(self.__tmpfname) + # okay, someone else has the lock, we weren't able to steal it, + # and our claim hasn't timed out yet. So let's wait a while for + # the owner of the lock to give it up. self.__writelog('waiting for claim') - time.sleep(self.__sleep_interval) + self.__sleep() + + def __sleep(self): + time.sleep(self.__sleep_interval) # This could error if the lock is stolen. You must catch it. def unlock(self): @@ -308,24 +278,39 @@ """ if not self.locked(): raise NotLockedError + now = time.time() + os.unlink(self.__lockfile) os.unlink(self.__tmpfname) - self.__writelog('unlocked') + self.__writelog('unlocked (%f)'%now) def locked(self): """Returns 1 if we own the lock, 0 if we do not.""" if not os.path.exists(self.__tmpfname): return 0 + if os.stat(self.__tmpfname)[ST_NLINK] <> 2: + return 0 + # The above check for nlink should be enough to validate our claim on + # the lock. Nevertheless, we are paranoid. try: pid, winner, lockrelease = self.__read() except ValueError: - # the contents of the lock file was corrupted - os.unlink(self.__tmpfname) - self.__kickstart(force=1) - return 0 + # the contents of the lock file was corrupted, or the global lock + # is not a link to our lock. + if os.stat(self.__lockfile)[ST_INO] <> os.stat(self.__tmpfname)[ST_INO]: + self.__writelog("nlink of 2 but we don't own the lock ? strangeness") + os.unlink(self.__tmpfname) + self.__write() + return 0 + else: + # Okay, we really do have the lock, but the contents is messed up + self.__write() + return 1 return pid == os.getpid() # use with caution!!! def steal(self): """Explicitly steal the lock. USE WITH CAUTION!""" - self.__write() - self.__writelog('explicitly stolen') + self.__writelog('explicitly breaking') + self.__break() + self.lock() + --J2SCkAp4GZ/dPZZf-- From bbum@codefab.com Fri Feb 4 17:29:12 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Fri, 4 Feb 2000 12:29:12 -0500 Subject: [Mailman-Developers] integrating changes into distribution Message-ID: <200002041729.MAA29538@bjork.codefab.com> For the Mailman-decode-mime-file-via-webdav modifications, I have a relatively significant set of addtions and patches to Mailman. I would like to intregrate these into the core distribution. What is the best way to procede? Barry? b.bum From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Fri Feb 4 18:10:49 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 4 Feb 2000 13:10:49 -0500 (EST) Subject: [Mailman-Developers] integrating changes into distribution References: <200002041729.MAA29538@bjork.codefab.com> Message-ID: <14491.5673.280676.30360@anthem.cnri.reston.va.us> >>>>> "BB" == Bill Bumgarner writes: BB> For the Mailman-decode-mime-file-via-webdav modifications, I BB> have a relatively significant set of addtions and patches to BB> Mailman. BB> I would like to intregrate these into the core distribution. BB> What is the best way to procede? BB> Barry? I'd like to take a crack at integrating the i18n changes this weekend. If that goes smoothly, your stuff is next on my list. -barry From ricardo@rixhq.nu Fri Feb 4 19:56:30 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Fri, 4 Feb 2000 20:56:30 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000202225346.B17313@xs4all.nl>; from thomas@xs4all.net on Wed, Feb 02, 2000 at 10:53:46PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> Message-ID: <20000204205630.A4871@miss-janet.com> Hi, I haven't looked deeply into the MM locking code but i think this could also be the reason why MM (cvs) always crashes on my server when I approve about 10 posts at the same time... maybe it could be a timing thing which is more likely to happen if the load is quite high (which happens quite soon as my server isn't that powerfull :) ) Anyway i'm willing to test out any new changes to the locking code in my setup... Ricardo. -- From ricardo@rixhq.nu Fri Feb 4 20:22:43 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Fri, 4 Feb 2000 21:22:43 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000204205630.A4871@miss-janet.com>; from ricardo@rixhq.nu on Fri, Feb 04, 2000 at 08:56:30PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> Message-ID: <20000204212243.B4871@miss-janet.com> On Fri, Feb 04, 2000 at 08:56:30PM +0100, Ricardo Kustner wrote: > about 10 posts at the same time... maybe it could be a timing thing which is more > likely to happen if the load is quite high (which happens quite soon as my server > isn't that powerfull :) ) Anyway i'm willing to test out any new changes to the > locking code in my setup... well i patched my copy of MM with the diff Thomas had just posted, and then i tried to approve about 16 posts and I still got another assertion error :( maybe i just need to start saving some money for a more powerfull machine :) Ricardo. -- From thomas@xs4all.net Fri Feb 4 22:47:57 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 4 Feb 2000 23:47:57 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000204212243.B4871@miss-janet.com>; from ricardo@rixhq.nu on Fri, Feb 04, 2000 at 09:22:43PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> Message-ID: <20000204234757.A2826@xs4all.nl> On Fri, Feb 04, 2000 at 09:22:43PM +0100, Ricardo Kustner wrote: > On Fri, Feb 04, 2000 at 08:56:30PM +0100, Ricardo Kustner wrote: > > about 10 posts at the same time... maybe it could be a timing thing which is more > > likely to happen if the load is quite high (which happens quite soon as my server > > isn't that powerfull :) ) Anyway i'm willing to test out any new changes to the > > locking code in my setup... > > well i patched my copy of MM with the diff Thomas had just posted, and > then i tried to approve about 16 posts and I still got another assertion error :( > maybe i just need to start saving some money for a more powerfull machine :) Or perhaps you're experiencing an entirely differnet problem. Or a very related, but slightly different problem. Or perhaps this method of locking does not work on the type of machine, operating system and/or filesystem you are using. If you want, I can take a look at your AssertionError, see what caused it. If not, and it's a generic Mailman bug, I'll eventually hit it myself, anyway ;) Out of curiosity, the mailinglist you're running on mailman, that wouldn't happen to be the MissJanet mailing list that you used to run at XS4ALL, would it? ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From ricardo@rixhq.nu Sat Feb 5 00:15:07 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Sat, 5 Feb 2000 01:15:07 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000204234757.A2826@xs4all.nl>; from thomas@xs4all.net on Fri, Feb 04, 2000 at 11:47:57PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> Message-ID: <20000205011507.A405@miss-janet.com> Hi, On Fri, Feb 04, 2000 at 11:47:57PM +0100, Thomas Wouters wrote: > On Fri, Feb 04, 2000 at 09:22:43PM +0100, Ricardo Kustner wrote: > > well i patched my copy of MM with the diff Thomas had just posted, and > > then i tried to approve about 16 posts and I still got another assertion error :( > > > maybe i just need to start saving some money for a more powerfull machine :) > Or perhaps you're experiencing an entirely differnet problem. Or a very > related, but slightly different problem. Or perhaps this method of locking > does not work on the type of machine, operating system and/or filesystem you > are using. Linux 2.0.38, libc5 on a P100... not the newest stuff but it's stable :) usually i use newer kernels/libs for servers but I had to do so much work to optimize the apache/fastcgi/mysql stuff that I'm cautious with upgrading... I hope to hook up a second harddrive soon I can put the mail spool on a seperate disk which will hopefully reduce the load a bit... > If you want, I can take a look at your AssertionError, see what > caused it. If not, and it's a generic Mailman bug, I'll eventually hit it It always happens on line 60 in Mailman/ListAdmin.py ... assert self.Locked() and self.__filename from what i understand from python (haven't finnished O'reillys "Learning Python" yet :)), this is a debug statement and it bails out if !self.Locked == true ... so I guess the lock on the pending message db gets lost somehow (which could be because posts are being approved and the pending db being updated at the same time... but i'm only speculating here). > Out of curiosity, the mailinglist you're running on mailman, that wouldn't > happen to be the MissJanet mailing list that you used to run at XS4ALL, > would it? ;) yep... that was one of the first majordomo lists at xs4all i believe :) Ricardo. -- From ricardo@rixhq.nu Sat Feb 5 00:15:07 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Sat, 5 Feb 2000 01:15:07 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000204234757.A2826@xs4all.nl>; from thomas@xs4all.net on Fri, Feb 04, 2000 at 11:47:57PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> Message-ID: <20000205011507.A405@miss-janet.com> Hi, On Fri, Feb 04, 2000 at 11:47:57PM +0100, Thomas Wouters wrote: > On Fri, Feb 04, 2000 at 09:22:43PM +0100, Ricardo Kustner wrote: > > well i patched my copy of MM with the diff Thomas had just posted, and > > then i tried to approve about 16 posts and I still got another assertion error :( > > > maybe i just need to start saving some money for a more powerfull machine :) > Or perhaps you're experiencing an entirely differnet problem. Or a very > related, but slightly different problem. Or perhaps this method of locking > does not work on the type of machine, operating system and/or filesystem you > are using. Linux 2.0.38, libc5 on a P100... not the newest stuff but it's stable :) usually i use newer kernels/libs for servers but I had to do so much work to optimize the apache/fastcgi/mysql stuff that I'm cautious with upgrading... I hope to hook up a second harddrive soon I can put the mail spool on a seperate disk which will hopefully reduce the load a bit... > If you want, I can take a look at your AssertionError, see what > caused it. If not, and it's a generic Mailman bug, I'll eventually hit it It always happens on line 60 in Mailman/ListAdmin.py ... assert self.Locked() and self.__filename from what i understand from python (haven't finnished O'reillys "Learning Python" yet :)), this is a debug statement and it bails out if !self.Locked == true ... so I guess the lock on the pending message db gets lost somehow (which could be because posts are being approved and the pending db being updated at the same time... but i'm only speculating here). > Out of curiosity, the mailinglist you're running on mailman, that wouldn't > happen to be the MissJanet mailing list that you used to run at XS4ALL, > would it? ;) yep... that was one of the first majordomo lists at xs4all i believe :) Ricardo. -- From thomas@xs4all.net Sat Feb 5 15:43:18 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 5 Feb 2000 16:43:18 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000205011507.A405@miss-janet.com>; from ricardo@rixhq.nu on Sat, Feb 05, 2000 at 01:15:07AM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> Message-ID: <20000205164318.A15909@xs4all.nl> On Sat, Feb 05, 2000 at 01:15:07AM +0100, Ricardo Kustner wrote: > > > well i patched my copy of MM with the diff Thomas had just posted, and > > > then i tried to approve about 16 posts and I still got another assertion error :( > It always happens on line 60 in Mailman/ListAdmin.py ... > assert self.Locked() and self.__filename Can you post (or mail directly to me) the exact error message, with traceback, and what you do exactly to get that error ? I tried reproducing it here, by posting 100-some messages to a closed list, and then approving them all in one go, using the admin page, but everything went fine. It might require more of a local load to recreate, though, or maybe you're doing something subtly different. > from what i understand from python (haven't finnished O'reillys "Learning > Python" yet :)), this is a debug statement and it bails out if > !self.Locked == true ... Yes, assert is a debug statement, but both 'self.Locked()' and 'self.__filename' get tested. One of those two ends up false. You can try to split it up in two asserts: assert self.Locked() assert self.__filename just to see which part is failing. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Sat Feb 5 22:21:30 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 5 Feb 2000 23:21:30 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000205205653.A4105@miss-janet.com>; from ricardo@rixhq.nu on Sat, Feb 05, 2000 at 08:56:53PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> Message-ID: <20000205232130.D23471@xs4all.nl> On Sat, Feb 05, 2000 at 08:56:53PM +0100, Ricardo Kustner wrote (in private): > Traceback (innermost last): > File "/usr/local/mailman/scripts/driver", line 112, in run_main > main() > File "../Mailman/Cgi/admindb.py", line 123, in main > mlist.Save() > File "/usr/local/mailman/Mailman/MailList.py", line 819, in Save > self.SaveRequestsDb() > File "../Mailman/ListAdmin.py", line 90, in SaveRequestsDb > self.__closedb() > File "../Mailman/ListAdmin.py", line 74, in __closedb > assert self.Locked() > AssertionError: > if you need to see the logs/error or logs/lock files let me know... [ CC: the list, because of the questions at the bottom ] Hmmm. I think I found the problem, but I'm not entirely sure. It looks like the default MailList lock timeout is 60 seconds, and that that is too short for your machine. You should see some 'stolen!' or (in my LockFile version) 'broken!' messages, in your logs/locks file. In any case, you can easily try it out; in Mailman/MailList.py, on or around line 282, there should be a 'lifetime = 60', inside the constructor for the maillists' lockfile. Changing the '60' in, say, '600', should give you better mileage, at least until your machine gets so heavily loaded that a simple admin request takes ten full minutes to process ;) I'm not sure what the Right Fix is, however. MailList.MailList does not have an advertised way of setting the lock lifetime, nor of refresh()ing the lock. Maybe someone with more experience (barry ? any other active developers) can shed light here... Is the usual solution to raise the initial lock timeout, passing lock timeout in the constructor, adding the two functions and calling them where appropriate, or just using __lock directly ? BTW, whoever documented that part of the code, thank you ;) with the 'TBD' remark above the lockfile constructor this was a sucker to find. It really means a lot to have it so well documented. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Sun Feb 6 11:36:44 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 6 Feb 2000 12:36:44 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000205232130.D23471@xs4all.nl>; from thomas@xs4all.net on Sat, Feb 05, 2000 at 11:21:30PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> Message-ID: <20000206123644.C19265@xs4all.nl> On Sat, Feb 05, 2000 at 11:21:30PM +0100, Thomas Wouters wrote: > You should see some 'stolen!' or (in my LockFile version) 'broken!' > messages, in your logs/locks file. Er, apparently not. Somehow some logmessages got lost during the move from lock() to __break() ;P If you want to be sure, you can add something like this by hand: self.__writelog("stealing lock (from %s)"%winner) As for the locking problem, I think I see a better solution. A site-global 'lock lifetime multiplier' that the site admin can set, by which both the lifetime values of lockfiles and the lock timeout values given to LockFile.lock() are multiplied. By leaving the default to 1 we leave everything the same, but when a slow site is experiencing expired but still used locks, they can up the multiplier and get consistent behaviour all 'round -- no spurious locktimeout errors because lifetime is high but timeout isn't :P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Sun Feb 6 21:05:32 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 6 Feb 2000 22:05:32 +0100 Subject: [Mailman-Developers] locking. Message-ID: <20000206220532.D19265@xs4all.nl> I've found some small buglets in my locking patch, mostly causing leftover files and missing logging... if anyone is integrating my patches, drop me a line so I can send an updated version ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Sun Feb 6 21:17:49 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 6 Feb 2000 22:17:49 +0100 Subject: [Mailman-Developers] integrating changes into distribution In-Reply-To: <14491.5673.280676.30360@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Fri, Feb 04, 2000 at 01:10:49PM -0500 References: <200002041729.MAA29538@bjork.codefab.com> <14491.5673.280676.30360@anthem.cnri.reston.va.us> Message-ID: <20000206221749.E19265@xs4all.nl> On Fri, Feb 04, 2000 at 01:10:49PM -0500, Barry A. Warsaw wrote: > > >>>>> "BB" == Bill Bumgarner writes: > BB> For the Mailman-decode-mime-file-via-webdav modifications, I > BB> have a relatively significant set of addtions and patches to > BB> Mailman. > I'd like to take a crack at integrating the i18n changes this > weekend. If that goes smoothly, your stuff is next on my list. I'm curious about this... I've browsed through the decoding/webdav patches, and it seems explicitly webdav. I understand that Bill needed webdav, and has thus only worked on that, and I appreciate the effort (we might have customers who want it, eventually ;) but I'm wondering if the patch can't be split in two different elements; webdav archiving, and attachment-decoding, and making them seperately configurable. I do not care about installing and configuring webdav at the moment, our setup can't handle it yet, but the automatic decoding of whatever word-document or powerpoint presentation our (sometimes) braindead managers post to our internal all-employees mailinglists, and placing it simply as a seperate file in the mailing list archive is a _very_ attractive idea. Not just to me, the rest of the sysadmin departement went 'ooh' and 'ahhh' when I told them about this development ;) (Which made me feel quite good, I must add. I still feel I have to 'prove' python and mailman to them.) I have no problem with (re)writing that part of the code, of course, but I'd prefer to wait until Bill's code is in the CVS tree (so that I know the style/layout and general idea has been approved of ;) Regards, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From lee@ufojoe.com Mon Feb 7 01:02:00 2000 From: lee@ufojoe.com (Lee Weston) Date: Sun, 6 Feb 2000 20:02:00 -0500 Subject: [Mailman-Developers] RE: dont request passwords from web subscriber patch Message-ID: Fri, 7 Jan 2000 14:58:16 +0100 (MET) Fil fil@bok.net wrote: > >could the maintainers tell me if this patch is accepted or rejected ? >Patch name : dont request passwords from web subscriber I'm sorry to see no answer. I will be looking at adding it. However I seems to me to be only half an answer. The other half is allowing un-subscribe without password. If not you risk trapping silly buggers as if you were fly paper. An "only allow random password" mode would have the benifit of substiantially raising the bar for writting cgi to raid / spam a site, by making them parse email as well. ----------- ufojoe -------------------------------- 50 Canadian Folk Festivals http://www.interlog.com/~ufojoe/ From gorgo@sztaki.hu Mon Feb 7 12:13:56 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Mon, 07 Feb 2000 13:13:56 +0100 (MET) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) Message-ID: Hello, Any ideas what might cause this ? ---------- Forwarded message ---------- Date: Mon, 07 Feb 2000 03:01:23 +0000 (UTC) From: Lazarus Long To: submit@bugs.debian.org Subject: Bug#57223: mailman: gate_news problems Resent-Date: Mon, 07 Feb 2000 03:03:28 +0000 (GMT) Resent-From: Lazarus Long Resent-To: debian-bugs-dist@lists.debian.org Resent-cc: Gergely Madarasz Package: mailman Version: 1.1-2 Severity: important Subject: Cron [ -x /usr/bin/python -a -f /usr/lib/mailman/cron/gate_news ] && /usr/bin/python +/usr/lib/mailman/cron/gate_news X-Cron-Env: X-Cron-Env: X-Cron-Env: X-Cron-Env: Date: Mon, 7 Feb 2000 02:55:06 +0000 (UTC) Traceback (innermost last): File "/usr/lib/mailman/cron/gate_news", line 119, in ? main() File "/usr/lib/mailman/cron/gate_news", line 79, in main r,c,first,last,n = conn.group(mlist.linked_newsgroup) File "/usr/lib/python1.5/nntplib.py", line 208, in group resp = self.shortcmd('GROUP ' + name) File "/usr/lib/python1.5/nntplib.py", line 158, in shortcmd return self.getresp() File "/usr/lib/python1.5/nntplib.py", line 134, in getresp raise error_perm, resp nntplib.error_perm: 500 "GROUP" not implemented; try "help". -- System Information Debian Release: 2.2 Kernel Version: Linux phoenix 2.2.14 #1 Thu Jan 6 01:44:28 UTC 2000 i586 unknown Versions of the packages mailman depends on: ii apache 1.3.9-10 Versatile, high-performance HTTP server ii cron 3.0pl1-55 management of regular background processing ii libc6 2.1.2-13 GNU C Library: Shared libraries and Timezone data ii logrotate 3.2-11 Log rotation utility ii python-base 1.5.2-6 An interactive object-oriented scripting language. sendmail Not installed or no info ii postfix-tls 0.0.19991231pl02+0.51-1 A mail transport agent with RFC2487 encryption ^^^ (Provides virtual package mail-transport-agent) ii apache-ssl 1.3.9.10+1.37-1 Versatile, high-performance HTTP server with SSL support ^^^ (Provides virtual package httpd) From Harald.Meland@usit.uio.no Mon Feb 7 13:59:23 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 14:59:23 +0100 Subject: [Mailman-Developers] feature requests In-Reply-To: Christopher Kolar's message of "Fri, 1 Oct 1999 09:10:51 -0500" References: <99100109180002.09732@kolar.facnet.aurora.edu> Message-ID: Yes, the message this is a reply to is pretty old by now -- I'm quite some messages behind on -developers, not to mention -users :( I'll try to get up to date (and stay there :) now... [Christopher Kolar] > While beating on the documentation a few things came to mind. > > 1. Any chance that the monthly reminder can be used by a bounce > handler script as a membership probe (good for low-volume lists). > Right now I just get N bounce messages as mailman-owner but I do not > see that the list manager ever gets the information. With the currently released Mailman, where only the list objects themselves know which addresses are members, trying to implement something like that would probably scale pretty badly. However, here at uio.no, we've now been running a Mailman with a user database -- mainly for avoiding the pains of having separate passwords for separate mailing lists -- and I guess one could pull of something like what you suggest with the help of the user db. I'll try getting around to post some details about what (rather substantial) changes I've made to our Mailman within the next few days -- but for now, to -developers only, I guess. > 2. How about adding an "invite" feature like on some of the > commercial list providers. The invite would be non-binding, but a > simple reply would add the person to the list (I suppose that it > would use most of the same code that the sub-confirmation bits use). > See listbot and egroups if you do not know what I am talking about. This would just be a slightly differently worded "subscription confirmation" message and something to avoid "open subscribes" turning into "open invites", I guess. > 3. How about a chunk of html code that can be dropped into a web > page that would allow a person to just type in their e-mail and > click to make a sub request (also something that listbot/egroups > do). Mailman does, to my knowledge, not do any kind of Referer checking or somesuch, so the only problem is the (current) need to enter a subscription password when posting a sub request. This _could_ be hacked around in various (ugly) ways by generating this form (dynamically, to avoid everyone having the same password) with HIDDEN fields for passwords. > The request could be handled like a normal sub request with a > confirmation request being sent out to the e-mail address that was > entered. Is this already possible by hacking the listinfo page to > remove everything but the field for e-mail address and the submit > button? I deal with a lot of novice users and I would like to take > out a lot of the verbiage and the initial password setting. How do others feel about this? Should Mailman allow users subscribing without explicitly setting their member password (and instead have Mailman set a random password for them)? -- Harald From Harald.Meland@usit.uio.no Mon Feb 7 14:17:47 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 15:17:47 +0100 Subject: [Mailman-Developers] Re: Mailman and your code for online posting In-Reply-To: Steven Hazel's message of "Wed, 6 Oct 1999 22:19:38 -0500 (CDT)" References: Message-ID: [Steven Hazel] > I hope you don't mind if I cc my reply to the Mailman developers list. > > Is the code that you submitted to the developers list going to be > incorporated into a subsequent release of Mailman? > > I hope so. It got no response, so I'm not quite sure what's up. Just to let you have a response (finally :): I've marked the messages you sent as "look at these when there's time". As you can infer from the time between your posting them and this message, my time for Mailman has been rather sparse for some time :(, but I'm hoping that will improve now. -- Harald From thomas@xs4all.net Mon Feb 7 14:29:44 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 15:29:44 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 01:13:56PM +0100 References: Message-ID: <20000207152944.G19265@xs4all.nl> On Mon, Feb 07, 2000 at 01:13:56PM +0100, Gergely Madarasz wrote: > Any ideas what might cause this ? > nntplib.error_perm: 500 "GROUP" not implemented; try "help". You're connecting to a newsserver that does not have or perhaps does not allow the 'GROUP' command. I personally wasn't aware that there were such newsservers... But perhaps you are connected to a newsfeeder box, instead of a newsreader box ? Some ISPs split their newsservers in feeders, who only have enough of a newsserver to share news with other newsservers, serve as a gateway of sorts, and actual readers, which get their feed off the feeders, and who _do_ allow clients to connect, and offer most, if not all, NNTP commands. If you're fairly sure you are connected to the right newsserver, try connecting to that newsserver, from the machine running mailman, on the news port (119) and try the GROUP command yourself. Also see if you can check the newsservers' logfile, to see if it mentions anything else. If all else fails, you need to find a different newsserver, or shut off the news gateway. > ---------- Forwarded message ---------- > Package: mailman > Version: 1.1-2 > Severity: important > Traceback (innermost last): > File "/usr/lib/mailman/cron/gate_news", line 119, in ? > main() > File "/usr/lib/mailman/cron/gate_news", line 79, in main > r,c,first,last,n = conn.group(mlist.linked_newsgroup) > File "/usr/lib/python1.5/nntplib.py", line 208, in group > resp = self.shortcmd('GROUP ' + name) > File "/usr/lib/python1.5/nntplib.py", line 158, in shortcmd > return self.getresp() > File "/usr/lib/python1.5/nntplib.py", line 134, in getresp > raise error_perm, resp -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gorgo@sztaki.hu Mon Feb 7 14:41:55 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Mon, 07 Feb 2000 15:41:55 +0100 (MET) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207152944.G19265@xs4all.nl> Message-ID: On Mon, 7 Feb 2000, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 01:13:56PM +0100, Gergely Madarasz wrote: > > > Any ideas what might cause this ? > > > nntplib.error_perm: 500 "GROUP" not implemented; try "help". > > You're connecting to a newsserver that does not have or perhaps does not > allow the 'GROUP' command. I personally wasn't aware that there were such > newsservers... But perhaps you are connected to a newsfeeder box, instead of > a newsreader box ? Some ISPs split their newsservers in feeders, who only > have enough of a newsserver to share news with other newsservers, serve as a > gateway of sorts, and actual readers, which get their feed off the feeders, > and who _do_ allow clients to connect, and offer most, if not all, NNTP > commands. > > If you're fairly sure you are connected to the right newsserver, try > connecting to that newsserver, from the machine running mailman, on the news > port (119) and try the GROUP command yourself. Also see if you can check the > newsservers' logfile, to see if it mentions anything else. If all else > fails, you need to find a different newsserver, or shut off the news > gateway. Hmm... I found what might cause this... this if from the mailman list archives: http://www.python.org/pipermail/mailman-users/1999-October/002371.html so does this mean that there was no workaround made for this in mailman 1.1 ? Is there a patch somewhere perhaps ? -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From Harald.Meland@usit.uio.no Mon Feb 7 14:44:03 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 15:44:03 +0100 Subject: [Mailman-Developers] Re: [Mailman-Users] Can't change with admin pages In-Reply-To: Sean Reifschneider's message of "Wed, 13 Oct 1999 03:24:17 -0600" References: <19991013032417.O1005@tummy.com> Message-ID: [Sean Reifschneider] > On Mon, Oct 11, 1999 at 08:59:17PM +0200, Per Starback wrote: > > path = urlparse(self.web_page_url)[2] # '/mailman' > > Is that comment supposed to mean something? Only that the typical value of `path' after that line would be "/mailman", but might be different if the list has a "exotic" (which does not imply "invalid") `web_page_url'. > Should the above code be changed to something more like: > > path = urlparse(self.web_page_url)[2] # '/mailman' > if not path: path = self.web_page_url Hmm, I think it'd probably be better to fix problems with invalid `web_page_url's in MailList.CheckValues(), which is called every time the list is loaded (and which already contains code to ensure that `web_page_url' ends in a "/"). -- Harald From thomas@xs4all.net Mon Feb 7 15:29:30 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 16:29:30 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 03:41:55PM +0100 References: <20000207152944.G19265@xs4all.nl> Message-ID: <20000207162930.H19265@xs4all.nl> On Mon, Feb 07, 2000 at 03:41:55PM +0100, Gergely Madarasz wrote: > > > nntplib.error_perm: 500 "GROUP" not implemented; try "help". > > You're connecting to a newsserver that does not have or perhaps does not > > allow the 'GROUP' command. I personally wasn't aware that there were such > Hmm... I found what might cause this... this if from the mailman list > archives: > http://www.python.org/pipermail/mailman-users/1999-October/002371.html > so does this mean that there was no workaround made for this in mailman > 1.1 ? Is there a patch somewhere perhaps ? The problem is that the problem code isn't Mailman code ;) nntplib.py is in the standard python library, it's just included in the 'pythonlib' directory for older python distributions, that dont have nntplib.py. Apparently the patch (which isn't real clean, by the way, because a general except: clause can be very nasty) hasn't been sent to the python-list, or hasn't been accepted. But the problem does look like it needs fixing, i'll see if i can send a patch to the python-patches list. In the mean time you can apply the patch mentioned in that posting, to your own copy of nntplib, as long as you dont use nntplib in server mode anywhere (Not very likely, that last part) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gorgo@sztaki.hu Mon Feb 7 15:57:37 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Mon, 07 Feb 2000 16:57:37 +0100 (MET) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207162930.H19265@xs4all.nl> Message-ID: On Mon, 7 Feb 2000, Thomas Wouters wrote: > send a patch to the python-patches list. In the mean time you can apply the > patch mentioned in that posting, to your own copy of nntplib, as long as you > dont use nntplib in server mode anywhere (Not very likely, that last part) That is for the reporter of the problem... But I need to make a fix in the mailman .deb package for debian potato before potato is released... :) -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From thomas@xs4all.net Mon Feb 7 16:19:59 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 17:19:59 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 04:57:37PM +0100 References: <20000207162930.H19265@xs4all.nl> Message-ID: <20000207171959.K23471@xs4all.nl> --bg08WKrSYDhXBjb5 Content-Type: text/plain; charset=us-ascii On Mon, Feb 07, 2000 at 04:57:37PM +0100, Gergely Madarasz wrote: > On Mon, 7 Feb 2000, Thomas Wouters wrote: > > send a patch to the python-patches list. In the mean time you can apply the > > patch mentioned in that posting, to your own copy of nntplib, as long as you > > dont use nntplib in server mode anywhere (Not very likely, that last part) > That is for the reporter of the problem... But I need to make a fix in the > mailman .deb package for debian potato before potato is released... :) Ok, well, because the reader attempting the 'group' command, in cron/gate_news, isn't using authorization, we can just send the 'mode reader' command when we get the error 500. The attached patch should work, I think. It fixes the first call to 'group' instead of the constructor of nntplib.NNTP, so it should be usable with all nntplib version. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --bg08WKrSYDhXBjb5 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mailman.cvs.diff" Index: cron/gate_news =================================================================== RCS file: /projects/cvsroot/mailman/cron/gate_news,v retrieving revision 1.22 diff -u -r1.22 gate_news --- gate_news 1999/12/25 14:35:31 1.22 +++ gate_news 2000/02/07 16:12:40 @@ -125,7 +125,15 @@ sys.stderr.write('connect to nntp_host failed\n') sys.stderr.write(`e`) break - r,c,f,l,n = conn.group(mlist.linked_newsgroup) + + try: + r,c,f,l,n = conn.group(mlist.linked_newsgroup) + except nntplib.error_perm: + # We got an error 500. This could mean 'group' is not implemented, + # and that we need to set the connection to reader mode first. + conn.shortcmd('mode reader') + r,c,f,l,n = conn.group(mlist.linked_newsgroup) + if not updatewatermarks: # just post the specified messages and be done with it poll_newsgroup(mlist, conn, first, last+1) --bg08WKrSYDhXBjb5-- From pf@artcom-gmbh.de Mon Feb 7 16:52:31 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Mon, 7 Feb 2000 17:52:31 +0100 (MET) Subject: [Mailman-Developers] Quoted printable umlaut quick hack Message-ID: I'm running mailman 1.1 on our intranet web server. Since our lists are in German language, umlauts occure quite often in mail bodies and subjects: äöü ÄÖÜ ß. Depending on the used MUA these are often encoded as quoted printables and end up as ugly trash in the pipermail HTML archives: F=FCr instead of Für ... Unfortunately this dooes not only look ugly but also prevents HTdig (the search engine) to find any words with umlauts in the pipermail archives. So today I did some quick hacks to HyperArch.py which at least work for me. (Spend too much time on this detail already) My dirty patch against HyperArch.py is appended below. May be this will help somebody to do this right. Regards, Peter -- Peter Funk, Oldenburger Str.86, 27777 Ganderkesee, Tel: 04222 9502 70, Fax: -60 ---- 8< ---- 8< ---- cut here ---- 8< ---- schnipp ---- 8< ---- schnapp ---- --- /home/pf/freeware/mailman-1.1/Mailman/Archiver/HyperArch.py Sat Aug 21 07:13:23 1999 +++ HyperArch.py Mon Feb 7 17:36:07 2000 @@ -34,6 +34,7 @@ import sys import re, cgi, urllib, string import time, pickle, os, posixfile +import quopri, cStringIO import HyperDatabase import pipermail from Mailman import mm_cfg @@ -49,6 +50,7 @@ except ImportError: pass +umlaut_in_subj = re.compile(r"=\?iso-8859-1\?Q\?(.*)\?=", re.IGNORECASE) def html_quote(s): repls = ( ('&', '&'), @@ -228,6 +230,16 @@ self.subject=self.subject[i:] else: i=-1 if self.subject=="": self.subject='No subject' + # --Pefus quick Hack: + umls = umlaut_in_subj.search(self.subject) + if umls: + umlconvin = cStringIO.StringIO( + umlaut_in_subj.sub(r"\1", self.subject)) + umlconvout = cStringIO.StringIO() + quopri.decode(umlconvin, umlconvout) + umlconvout.seek(0) + self.subject = string.join(umlconvout.read(), "") + # -- if message.has_key('Date'): self.datestr=str(message['Date']) @@ -239,7 +251,12 @@ date, tzoffset=date[:9], date[-1] if not tzoffset: tzoffset = 0 - date=time.mktime(date)-tzoffset + try: + date=time.mktime(date)-tzoffset + except ValueError: + print "illegal date discovered:", date + date=time.time() + else: date=self.__last_article_time+1 @@ -276,8 +293,13 @@ # Read the message body self.body=[] message.rewindbody() + converterInput=cStringIO.StringIO( + string.join(message.fp.readlines(), "")) + converterOutput=cStringIO.StringIO() + quopri.decode(converterInput, converterOutput) + converterOutput.seek(0) while (1): - line=message.fp.readline() + line=converterOutput.readline() if line=="": break self.body.append(line) From gorgo@sztaki.hu Mon Feb 7 17:16:06 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Mon, 07 Feb 2000 18:16:06 +0100 (MET) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207171959.K23471@xs4all.nl> Message-ID: On Mon, 7 Feb 2000, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 04:57:37PM +0100, Gergely Madarasz wrote: > > On Mon, 7 Feb 2000, Thomas Wouters wrote: > > > > send a patch to the python-patches list. In the mean time you can apply the > > > patch mentioned in that posting, to your own copy of nntplib, as long as you > > > dont use nntplib in server mode anywhere (Not very likely, that last part) > > > That is for the reporter of the problem... But I need to make a fix in the > > mailman .deb package for debian potato before potato is released... :) > > Ok, well, because the reader attempting the 'group' command, in > cron/gate_news, isn't using authorization, we can just send the 'mode > reader' command when we get the error 500. The attached patch should work, I > think. It fixes the first call to 'group' instead of the constructor of > nntplib.NNTP, so it should be usable with all nntplib version. Thanks. Lazarus, please test this patch, and notify me if it worked for you :) -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From gerrit@nl.linux.org Mon Feb 7 16:56:53 2000 From: gerrit@nl.linux.org (Gerrit Holl) Date: Mon, 7 Feb 2000 17:56:53 +0100 Subject: [Mailman-Developers] Development reference manual? Message-ID: <20000207175653.A4084@stopcontact.palga.uucp> Hello, I wonder if there is any reference manual to all of Mailmans classes, functions and variables. Like www.python.org/doc/current/lib. I'd like such a feature, since I'm interested in developing Mailman. I've almost a year experience in Python, but I get confused my the Mailman core package. I've read the available documentaion but I get confused by some of the code. regards, Gerrit. P.S. Is the majority of people in here also Dutch, like on c.l.py? -- homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From bbum@codefab.com Mon Feb 7 17:33:37 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Mon, 7 Feb 2000 12:33:37 -0500 Subject: [Mailman-Developers] Re: Mailman-Developers digest, Vol 1 #442 - 12 msgs Message-ID: <200002071733.MAA18496@bjork.codefab.com> Agreed. I wish I had the time to fully generalize the code though I'm quite happy to hear that others think this is a good idea! As it stands, it is relatively close-- the Rewrite.py module has been extended such that the resulting rewritten message contains instances of WebDAVAttachment that contain the eventual URL (archival and retrieval) for the attachment and the decoded raw data of the attachment. I made the decision to call it WebDAVAttachment to imply the context within which it was written. However, the actual implementation of that particular class has very little that is specific to WebDAV other than some of the URL names. In hindsight, I really should have not prefixed the retrieval URL with 'dav'-- it has nothing to do with DAV, even given the current implementation. The ToArchiver Handler could clearly use some generalization! Actually, the actual process() method is relatively general, but the do_dav() method should likely be broken out. Applying a bit more thought to this, it strikes me that what should likely happen is one of several things: - enabling/disabling different archival methods should be separated from the configuration of that individual archival method. Certain archival configuration optiosn-- like monthly/weekly/etc (which the WebDAV stuff currently ignores. I apologize for that-- more below)-- should be left in a sort of "global" configuration area assuming that it is appropriate to apply the same options to more than one archive "style". - individual or broken out screens for the individual archival methods - broken out mechanism that does the actual "splitting" of the archive into the appropriate paths. Regardless of whether it is a filesystem or URL based archival mechanism, you pretty much *always* have a "root" location below which the entire archive resides. The current implementation does not share any code between the pipermail archiver and the WebDAV archiver. Part of this was because it was indicated that the pipermail stuff needed to be largely revisited and partially because I simply didn't have time to do the generalization I wanted. --- Thomas: If you want to contribute to the effort prior to Barry's work integrating back into the core repository, I can take on the task of integrating your patches into my source. Let me know and I can make sure the latest sources are available to you via a more direct means than the occasional tarball. The changes that you indicated are certainly the correct direction for this codebase-- it would likely make Barry's life easier in doing the final to-repository integration if the code were a bit more refined. Ultimately, it is likely Barry's call-- I have a limited amount of time available now that this stuff works, but I would like to invest that time in ensuring that the codebase is moving as rapidly as possible into being a supported part of the existing Mailman infrastructure. -- Apologies: These extensions were actually done as a part of the development of a proprietary solution for a client of CodeFab's. Early in the specification phase, it become abundantly clear that leveraging Mailman would gain us a whole bunch of functionality for free. Of course, the challenge was then convincing the client that the requirement for releasing said changes back to the community was a small price compared to the gain in feature set. This proved surprisingly easy. The client is a very positive minded, forward thinking group of folks that definitely understand and believe in things like Open Source and Good Engineering Practices. (If anyone is interested in working for such an entity in the NYC area-- with obvious ties to the Python community-- please let me know and I can forward your contact information to the appropriate individuals. They are a "pre-IPO" "dot com" with excellent "potential valuation". Yadda.). Our initial approach was to try and hire someone from this community to do the work. Both because we wanted someone more familiar with the codebase than me (1.2 expiremental via cvs was my first real exposure) and because I really wanted to focus on other aspects of the project. That didn't happen and I ended up doing all the work. Fine by me, I *love* programming in Python-- especially when working with a cool sourcebase. However, it did mean that there was a lot of pressure on me to make the code work as quickly as possible and minimize the amount of effort put forth to making the end result as generic as it really should be. To give the client credit, they understand and authorized me spending a bunch of time to do a minimal amount of work towards a design and implementation that would be palatable to the community. While they don't have an infinite amount of time/money to throw at this, they definitely recognized that an implementation that would be accepted by the community as rapidly as possible would benefit them, as well. So; I apologize for the state of some of the code. It is not as clean or as elegant as I would normally consider acceptable. At the same time, the work has been done, the end result actually works, and it is definitely a solution to a problem that has been very much at the forefront of a lot of folk's administrative environments. And, of course, getting that initial implementation out the door is always half the battle... thanks, b.bum /* From: Thomas Wouters wrote */ On Fri, Feb 04, 2000 at 01:10:49PM -0500, Barry A. Warsaw wrote: > > >>>>> "BB" == Bill Bumgarner writes: > BB> For the Mailman-decode-mime-file-via-webdav modifications, I > BB> have a relatively significant set of addtions and patches to > BB> Mailman. > I'd like to take a crack at integrating the i18n changes this > weekend. If that goes smoothly, your stuff is next on my list. /* * I'm curious about this... I've browsed through the decoding/webdav patches, * and it seems explicitly webdav. I understand that Bill needed webdav, and * has thus only worked on that, and I appreciate the effort (we might have * customers who want it, eventually ;) but I'm wondering if the patch can't be * split in two different elements; webdav archiving, and attachment-decoding, * and making them seperately configurable. * * I do not care about installing and configuring webdav at the moment, our * setup can't handle it yet, but the automatic decoding of whatever * word-document or powerpoint presentation our (sometimes) braindead managers * post to our internal all-employees mailinglists, and placing it simply as a * seperate file in the mailing list archive is a _very_ attractive idea. Not * just to me, the rest of the sysadmin departement went 'ooh' and 'ahhh' when * I told them about this development ;) (Which made me feel quite good, I must * add. I still feel I have to 'prove' python and mailman to them.) * * I have no problem with (re)writing that part of the code, of course, but I'd * prefer to wait until Bill's code is in the CVS tree (so that I know the * style/layout and general idea has been approved of ;) */ From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Mon Feb 7 17:49:37 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 7 Feb 2000 12:49:37 -0500 (EST) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> Message-ID: <14495.1457.828840.186393@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> Hmm... I found what might cause this... this if from the GM> mailman list archives: GM> http://www.python.org/pipermail/mailman-users/1999-October/002371.html GM> so does this mean that there was no workaround made for this GM> in mailman 1.1 ? Is there a patch somewhere perhaps ? Since this is a bug that appears to be best fixed in Python's nntplib.py, I will float the message to patches@python.org. If nobody objects I'll make the change for Python 1.6 and then back-copy the file into Mailman 1.2. The quick fix is to do the following: - copy Python 1.5.2's nntplib.py into Mailman/pythonlib - apply the patch - Change all occurrances of "import nntplib" to "from Mailman.pythonlib import nntplib" -Barry From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Mon Feb 7 17:53:17 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 7 Feb 2000 12:53:17 -0500 (EST) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> Message-ID: <14495.1677.982940.194724@anthem.cnri.reston.va.us> I'm cutting and pasting a message that originated on the mailman-users mailing list. I think this message is fairly self-explanatory. Can other NNTP experts look this over and see if it makes sense? I'll apply it if there are no objections. Note: the bare except is bothersome. For reference, the thread is here: http://www.python.org/pipermail/mailman-users/1999-October/002371.html -Barry -------------------- snip snip -------------------- On Sun, Oct 10, 1999 at 06:23:41PM -0700, Michael Ghens wrote: > Tried a news gateway test. This is the error: [...] > nntplib.error_perm: 500 "GROUP" not implemented; try "help". Were you running the mailman gateway on the same machine as the news server? (Or on another machine that is an NNTP peer of the news server?) Python's standard nntplib assumes that when it connects it will default to (NNRP) reader mode. If the connection ends up in NNTP/news feed mode, many commands, including "GROUP" will not be recognized. A quick hack is to add this to your nntplib.py: diff -u nntplib.py.dist nntplib.py --- nntplib.py.dist Tue Apr 28 17:43:35 1998 +++ nntplib.py Tue Jun 23 22:06:00 1998 @@ -71,6 +71,10 @@ self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + try: + self.welcome = self.shortcmd('mode reader') + except: + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': This makes nntplib use 'mode reader' for all connections. As Harald Meland pointed out, this should probably be controlled by adding another optional argument to nntplib.NNTP(), but if you only use the library with mailman or in other "client" applications this will suffice. Jim P.S. You could also do the 'mode reader' stanza in GatewayManager, but then you also need to repeat the 'authinfo' code there if your news server needs authentication, since that must follow the 'mode reader' command. -- Jim Tittsler, Tokyo From thomas@xs4all.net Mon Feb 7 18:08:33 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 19:08:33 +0100 Subject: [Mailman-Developers] Re: Mailman-Developers digest, Vol 1 #442 - 12 msgs In-Reply-To: <200002071733.MAA18496@bjork.codefab.com>; from bbum@codefab.com on Mon, Feb 07, 2000 at 12:33:37PM -0500 References: <200002071733.MAA18496@bjork.codefab.com> Message-ID: <20000207190833.K19265@xs4all.nl> On Mon, Feb 07, 2000 at 12:33:37PM -0500, Bill Bumgarner wrote: > Thomas: > > If you want to contribute to the effort prior to Barry's work > integrating back into the core repository, I can take on the task of > integrating your patches into my source. Let me know and I can make sure > the latest sources are available to you via a more direct means than the > occasional tarball. The changes that you indicated are certainly the > correct direction for this codebase-- it would likely make Barry's life > easier in doing the final to-repository integration if the code were a bit > more refined. > > Ultimately, it is likely Barry's call-- I have a limited amount of > time available now that this stuff works, but I would like to invest that > time in ensuring that the codebase is moving as rapidly as possible into > being a supported part of the existing Mailman infrastructure. Well, depending on howfar Barry has been able to integrate the patches yet, it might be easier for him to continue integration and for me to send patches relative to the then-CVS version -- unless you're still working on these patches, making changes ? My guess is that Barry will make minor adjustments while integrating, like changing tabs into spaces, and possibly merging it with other patches he has pending, like those i18n patches he mentioned, making additional patches by me based on your code a slight pain. Or if Barry has not touched the WebDAV patches yet, and people are willing to wait, i can take your patches, rewrite the bits that need rewriting, and re-submit them for integration. I'm not sure how fast i would have the other patch ready, as i'm still studying those parts of mailman (1.2 CVS is my first contact with mailman as well;) and if Barry or someone else wants the WebDAV changes into the CVS tree anytime soon, it's probably better to add them now and for me to post changes. Barry...? ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Mon Feb 7 18:17:13 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 19:17:13 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <14495.1677.982940.194724@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Mon, Feb 07, 2000 at 12:53:17PM -0500 References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> Message-ID: <20000207191713.L19265@xs4all.nl> On Mon, Feb 07, 2000 at 12:53:17PM -0500, Barry A. Warsaw wrote: > I'm cutting and pasting a message that originated on the mailman-users > mailing list. I think this message is fairly self-explanatory. Can > other NNTP experts look this over and see if it makes sense? I'll > apply it if there are no objections. > Note: the bare except is bothersome. I've discussed this with two of my colleagues, who do and have done a fair lot with INN, and the general concensus is that it should work just fine. Servers who dont need 'mode reader' but recognize it should ignore it, Servers who dont understand it should give a 500 error, 'error_perm', with 'unimplemented' or 'what ?' or some such as message text. Note, though, that i've already posted a small patch to mailman to fix this. Since the 'group' command is executed by gate_news, which doesn't need authentication, it's easy enough to try the group, and set the connection to readermode if it fails. Also note that I've already sent a patch to patches@python.org ;) with a pointer to the mailman-users discussion in question. http://www.python.org/pipermail/patches/2000-February/000015.html My patch uses an extra argument to NNTP.__init__, and avoids the bare except. To get the same behaviour as the patch below, sending 'mode reader' by default, the default value of readermode can be set to 1. I'm not sure if that's a wise thing, however, as some users of nntplib.NNTP might depend on the connection being able to do things that you cant do in readermode. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Harald.Meland@usit.uio.no Mon Feb 7 18:40:06 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 19:40:06 +0100 Subject: [Mailman-Developers] Feature suggestion In-Reply-To: Mats Wichmann's message of "Mon, 22 Nov 1999 11:03:10 -0700" References: <3.0.6.32.19991122110310.00e7cc50@laplaza.org> Message-ID: [Mats Wichmann] > At 01:55 PM 11/22/1999 -0000, Adrian Eyre wrote: > >> However, my idea is this. What if there was some sort of mechanism in > >> place that stored the file in a different type of archive, and instead > >> of mailing the file, it included a link to click on to pick up the file > >> instead. > > > >Good idea. Could it be generalised to allow all people on the mailing list > >to specify a maximum attachment size. All attachments below this are > >attached to the e-mail, and any larger are replaced with links. Or maybe > >as a total attachment size for each e-mail, rather than on an attachment > >by attachment basis. > > Sure seems like it would be a useful feature. I'm mostly stuck > on very slow lines, many times it's a long-distance call to get > that much. I'd like to be able to defer getting attachments - > pretty much ALL attachments - until I know at least I've got a > toll-free connection, even if it's a slow one. Maybe there's not > enough of us so "blessed" to be worth a change, though.... I can sympathize with the initial idea (replacing the attachment with a link for _all_ list members), but the "generalisation" I don't feel to good about. If individual list members are allowed to customize what parts of a list message they want, it means Mailman has to generate umpteen different versions of an incoming post, and pass the appropriate version on to the different sets of members. That would scale rather poorly... Of course, if anyone wants this bad enough to code it up, and makes it possible to turn off site-wide for admins so inclined, it might be included. As for the initial suggestion: First step would be to teach Mailman something about MIME messages, any takers are welcome :) -- Harald From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Mon Feb 7 19:16:52 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 7 Feb 2000 14:16:52 -0500 (EST) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207162930.H19265@xs4all.nl> <20000207171959.K23471@xs4all.nl> Message-ID: <14495.6692.360996.332479@anthem.cnri.reston.va.us> >>>>> "TW" == Thomas Wouters writes: TW> Ok, well, because the reader attempting the 'group' command, TW> in cron/gate_news, isn't using authorization, we can just send TW> the 'mode reader' command when we get the error 500. The TW> attached patch should work, I think. It fixes the first call TW> to 'group' instead of the constructor of nntplib.NNTP, so it TW> should be usable with all nntplib version. Thanks. I'll apply this patch (with a slightly rewritten comment) to the CVS snapshot. -Barry From ricardo@rixhq.nu Mon Feb 7 19:43:31 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Mon, 7 Feb 2000 20:43:31 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000205232130.D23471@xs4all.nl>; from thomas@xs4all.net on Sat, Feb 05, 2000 at 11:21:30PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> Message-ID: <20000207204331.A444@miss-janet.com> On Sat, Feb 05, 2000 at 11:21:30PM +0100, Thomas Wouters wrote: > On Sat, Feb 05, 2000 at 08:56:53PM +0100, Ricardo Kustner wrote (in private): > > AssertionError: > > if you need to see the logs/error or logs/lock files let me know... > [ CC: the list, because of the questions at the bottom ] > Hmmm. I think I found the problem, but I'm not entirely sure. It looks like > the default MailList lock timeout is 60 seconds, and that that is too short > for your machine. You should see some 'stolen!' or (in my LockFile version) > 'broken!' messages, in your logs/locks file. yes i did see "stolen!" in the logs just before the process was about to crash. > In any case, you can easily try it out; in Mailman/MailList.py, on or around > line 282, there should be a 'lifetime = 60', inside the constructor for the > maillists' lockfile. Changing the '60' in, say, '600', should give you thanks a lot :) I'm trying out the value '300'... just to be sure... > better mileage, at least until your machine gets so heavily loaded that a > simple admin request takes ten full minutes to process ;) with the earlier versions of MM, first I used exim and that made the load way to high (around 60) and often made it impossible to use... after I started using postfix, things went much better. > I'm not sure what the Right Fix is, however. MailList.MailList does not have I'll let you know how it works out after changing the lock timeout... Ricardo. -- From lazarus@overdue.dhis.org Mon Feb 7 20:09:18 2000 From: lazarus@overdue.dhis.org (Lazarus Long) Date: Mon, 7 Feb 2000 20:09:18 +0000 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 06:16:06PM +0100 References: <20000207171959.K23471@xs4all.nl> Message-ID: <20000207200918.A18352@overdue.dhis.net> On Mon, Feb 07, 2000 at 06:16:06PM +0100, Gergely Madarasz wrote: > > Thanks. > Lazarus, please test this patch, and notify me if it worked for you :) phoenix:/usr/lib/mailman/cron# patch --verbose --dry-run < mailman.cvs.diff Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |Index: cron/gate_news |=================================================================== |RCS file: /projects/cvsroot/mailman/cron/gate_news,v |retrieving revision 1.22 |diff -u -r1.22 gate_news |--- gate_news 1999/12/25 14:35:31 1.22 |+++ gate_news 2000/02/07 16:12:40 -------------------------- Patching file `gate_news' using Plan A... Hunk #1 FAILED at 125. 1 out of 1 hunk FAILED -- saving rejects to gate_news.rej done phoenix:/usr/lib/mailman/cron# No. :( What next? I know no python, fwiw. -- From Harald.Meland@usit.uio.no Mon Feb 7 20:11:04 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 21:11:04 +0100 Subject: [Mailman-Developers] Reply goes to list error In-Reply-To: Balazs Nagy's message of "Tue, 25 Jan 2000 01:07:39 +0100 (CET)" References: Message-ID: [Balazs Nagy] > On Tue, 18 Jan 2000, Balazs Nagy wrote: > > > Hi, > > > > I know that reply to the list address is obsolete and not supported but in > > my list there're a bunch of people who claim this feature. It worked in > > the last version but the CVS version doesn't use this option anymore. > > Here's the patch. > > I just cannot understand why nobody from thosw who have write access > did anything about the subject. Because Reply-To: munging is already part of CookHeaders.py, I guess. And that takes appropriate note of the "fastrack" attribute of the message object, too. -- Harald From Harald.Meland@usit.uio.no Mon Feb 7 20:17:50 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 21:17:50 +0100 Subject: [Mailman-Developers] Admin overview page should list number of pending admin requests per list In-Reply-To: "Barry A. Warsaw"'s message of "Wed, 19 Jan 2000 12:33:43 -0500 (EST)" References: <20000119175228.67F911FF03B@perens.com> <14469.62839.700229.701854@anthem.cnri.reston.va.us> Message-ID: [Barry A. Warsaw] > >>>>> "BP" == Bruce Perens writes: > > BP> Here's a feature request. I administrer about 10 lists. To see > BP> what the pending admin requests are, I either have to look in > BP> my mailbox for nudges from the server, or click on the "handle > BP> administrative requests" link for _every_ list. What I want > BP> is to be able to see this for all lists at a glance. > > As admin for about a dozen lists on python.org, I completely agree! But that means that _anyone_ can see whether there are held messages in a list -- which really is giving out more info than appropriate (at least in some cases). I think there should be a separate "userinfo" CGI script which, after proper autentication, shows what lists that user is 1) member of and 2) admin for. Putting the info you both crave for on that page would be Good. [ And just think how nicely it would integrate with a user database... OK, OK, I'll stop teasing RSN ;) ] -- Harald From lazarus@overdue.dhis.org Mon Feb 7 20:27:26 2000 From: lazarus@overdue.dhis.org (Lazarus Long) Date: Mon, 7 Feb 2000 20:27:26 +0000 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207171959.K23471@xs4all.nl>; from thomas@xs4all.net on Mon, Feb 07, 2000 at 05:19:59PM +0100 References: <20000207162930.H19265@xs4all.nl> <20000207171959.K23471@xs4all.nl> Message-ID: <20000207202726.B18542@overdue.dhis.net> On Mon, Feb 07, 2000 at 05:19:59PM +0100, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 04:57:37PM +0100, Gergely Madarasz wrote: > > On Mon, 7 Feb 2000, Thomas Wouters wrote: > > > > send a patch to the python-patches list. In the mean time you can apply the > > > patch mentioned in that posting, to your own copy of nntplib, as long as you > > > dont use nntplib in server mode anywhere (Not very likely, that last part) > > > That is for the reporter of the problem... But I need to make a fix in the > > mailman .deb package for debian potato before potato is released... :) > > Ok, well, because the reader attempting the 'group' command, in > cron/gate_news, isn't using authorization, we can just send the 'mode > reader' command when we get the error 500. The attached patch should work, I > think. It fixes the first call to 'group' instead of the constructor of > nntplib.NNTP, so it should be usable with all nntplib version. That patch would not apply here, as I believe I just posted to your list as well with my previous mail. I'm not sure if gate_news in the Debian packages differs from the CVS version or not however. Gergely? -- From ricardo@rixhq.nu Mon Feb 7 20:31:44 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Mon, 7 Feb 2000 21:31:44 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000207204331.A444@miss-janet.com>; from ricardo@rixhq.nu on Mon, Feb 07, 2000 at 08:43:31PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> <20000207204331.A444@miss-janet.com> Message-ID: <20000207213143.B444@miss-janet.com> On Mon, Feb 07, 2000 at 08:43:31PM +0100, Ricardo Kustner wrote: > yes i did see "stolen!" in the logs just before the process was about to crash. > > In any case, you can easily try it out; in Mailman/MailList.py, on or around > > line 282, there should be a 'lifetime = 60', inside the constructor for the > > maillists' lockfile. Changing the '60' in, say, '600', should give you > thanks a lot :) I'm trying out the value '300'... just to be sure... well I just tried it out... I approved 17 posts and didn't get an assertion error! though it took quite a while before the webpage was finnished loading (about 2 minutes) but the process load on the machine didn't get any higher than 2.5 I wonder though what happens if some impatient moderator decides not to wait before the page finnishes loading, switches to a differen webpage and therefor breaks the python cgi process... will some approved posts stay in the queue instead? I mentioned before that I think that it could be better if the cgi scripts don't do anything more than just "mark" message as being approved... cgi scripts should have a short life time and definately shouldn't be waiting for something too long... btw i used the original LockFile.py this time... Ricardo. -- From lazarus@overdue.dhis.org Mon Feb 7 20:55:35 2000 From: lazarus@overdue.dhis.org (Lazarus Long) Date: Mon, 7 Feb 2000 20:55:35 +0000 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 04:57:37PM +0100 References: <20000207162930.H19265@xs4all.nl> Message-ID: <20000207205535.C18542@overdue.dhis.net> On Mon, Feb 07, 2000 at 04:57:37PM +0100, Gergely Madarasz wrote: > > send a patch to the python-patches list. In the mean time you can apply the > > patch mentioned in that posting, to your own copy of nntplib, as long as you > > dont use nntplib in server mode anywhere (Not very likely, that last part) > > That is for the reporter of the problem... But I need to make a fix in the > mailman .deb package for debian potato before potato is released... :) Following up to what I've received here so far on this thread (some of which I appear to have missed, judging by quoted portions) I am running current standard versions of most of the involved packages here on a Debian system, including INN2 as the NNTP daemon, so I imagine standard commands are supported. In case specifics are useful to anyone, here is a list which I hope covers all potentially-involved packages based upon Debian package dependency information. I'll include those I suspect are irrelevant, in case I am wrong, in the interest of completeness. ||/ Name Version Description +++-===========================-===========================-====================================================================== ii task-news-server 2.2.2.2000.01.31-1 USENET News Server ii inn2 2.2.2.2000.01.31-1 News transport system `InterNetNews' by the ISC and Rich Salz ii libc6 2.1.2-13 GNU C Library: Shared libraries and Timezone data ii libgdbmg1 1.7.3-26.2 GNU dbm database routines (runtime version). [libc6 version] ii cron 3.0pl1-55 management of regular background processing ii postfix-tls 0.0.19991231pl02+0.51-1 A mail transport agent with RFC2487 encryption ii time 1.7-8 The GNU time command. ii inn2-inews 2.2.2.2000.01.31-1 NNTP client news injector, from InterNetNews (INN) ii grep 2.4-2 GNU grep, egrep and fgrep. ii perl-5.005 5.005.03-5.3 Larry Wall's Practical Extracting and Report Language. ii python-base 1.5.2-6 An interactive object-oriented scripting language. ii libncurses4 4.2-8 Shared libraries for terminal handling ii libreadlineg2 2.1-17 GNU readline and history libraries, run-time libraries. [libc6] ii mailman 1.1-2 Powerful, web based list processor ii logrotate 3.2-11 Log rotation utility ii apache 1.3.9-10 Versatile, high-performance HTTP server ii apache-ssl 1.3.9.10+1.37-1 Versatile, high-performance HTTP server with SSL support The original Debian bug report contained a similar list, but I would imagine it was not included on this list, since there was question of which NNTP commands were supported by the daemon in question. BTW, mangling python-base to prevent server mode connections will not be an option for me, nor would I imagine it to be acceptable for Debian-as-a-whole, although I don't speak for the Debian project in any way. Thanks in advance for your collective efforts, on behalf of all of the systems likely to be encountering this problem in the future (which should include most that install mailman on a Debian box.) -- From Harald.Meland@usit.uio.no Mon Feb 7 21:01:36 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 22:01:36 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: Thomas Wouters's message of "Thu, 3 Feb 2000 10:35:55 +0100" References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <14488.44469.446995.168328@anthem.cnri.reston.va.us> <20000203103555.C6378@xs4all.nl> Message-ID: [Thomas Wouters] > Well, attached you'll find the first change, the cvs diff to LockFile.py. Sorry for not having been around here earlier, but: I've already been running a non-backwards LockFile.py in our Mailman installation here at uio.no for over a month. My current CVS checkout is available ("live" :) under , with being of special interest in this discussion... > I kept the API the same with one teensy exception. With the new > locking scheme, it's not possible to 'steal' a lock, only to break > the lock and then enter the lock() cycle to try and grab it. Umm, I'm not sure that would go down very well with all of (current) Mailman, as it would mean you'd have to reload any data that might have changed on disk (marshals etc.) if someone else got the lock before you... I've been taking the approach that the parts of Mailman that actually call steal() have to know what they're doing (and grepping seemed to indicate that that was correct -- it's mostly used by forked-off children to steal locks held by the parent), and thus steal() is awfully blunt and just overwrites the current lock. > I've tested this locker quite heavily, mailbombing my listserver > with 100+ simultaneous messages to mailman-test (which generated > 100+ bounces sent back to mailman, because my bouncing email address > is still on the list) and it hasn't barfed yet ;) Looks good -- I implemented my version of this due to severe locking problems when our Mailman server nearly ran out of swap when various log mails started arriving in hordes just after midnight... Combined with the mm_cfg.EX_TEMPFAIL trick seen in this has kept our Sun SPARCstation 20 with 160MB RAM running our ~3.500 lists quite smoothly for the last month. I'll try having a look at what differences there are in the two versions of this tomorrow. Cheers, -- Harald From thomas@xs4all.net Mon Feb 7 21:44:58 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 22:44:58 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: ; from Harald.Meland@usit.uio.no on Mon, Feb 07, 2000 at 10:01:36PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <14488.44469.446995.168328@anthem.cnri.reston.va.us> <20000203103555.C6378@xs4all.nl> Message-ID: <20000207224457.T23471@xs4all.nl> --oj4kGyHlBMXGt3Le Content-Type: text/plain; charset=us-ascii On Mon, Feb 07, 2000 at 10:01:36PM +0100, Harald Meland wrote: > My current CVS checkout is available ("live" :) under http://www.uio.no/~hmeland/tmp/mailman-userdb/>, with http://www.uio.no/~hmeland/tmp/mailman-userdb/Mailman/LockFile.py> > being of special interest in this discussion... Ah, very cool. You'll find my current version of LockFile.py attached, for reference, as you asked. I think the link() solutions, as opposed to the rename() solution, is a better one, if you'll premit me. For one, you can check on lock ownership by statting your own lockfile, which should only be touched by you, instead of reading a possibly highly volatile lockfile. You can also doublecheck the lockfile inode, see if it is equal to your own lockfiles' inode, to confirm that you have the lock. > > I kept the API the same with one teensy exception. With the new > > locking scheme, it's not possible to 'steal' a lock, only to break > > the lock and then enter the lock() cycle to try and grab it. > Umm, I'm not sure that would go down very well with all of (current) > Mailman, as it would mean you'd have to reload any data that might > have changed on disk (marshals etc.) if someone else got the lock > before you... > I've been taking the approach that the parts of Mailman that actually > call steal() have to know what they're doing (and grepping seemed to > indicate that that was correct -- it's mostly used by forked-off > children to steal locks held by the parent), and thus steal() is > awfully blunt and just overwrites the current lock. Ah, well, I couldn't find any parts of Mailman in the CVS tree that acually _called_ steal, so I couldn't figure out what it should do in borderline cases. However, reading your LockFile.py made me realise how to steal the lock. LockFile.steal() now does its utmost best to steal the lock, only failing if someone else is busy stealing the lock from us. It returns 1 for success, 0 for failure, mostly because I wanted it to signify that in some way. There still are race conditions in there, however, and I doubt we'll ever get rid of them. You assume the parent knows what it is doing, as where I assume the parent _knew_ what it is doing, but can be wrong by now ;) Most of these cases are, granted, _very_ low probability. Well, let me know if you have any questions or comments regarding my implementation. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --oj4kGyHlBMXGt3Le Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="LockFile.py" # Copyright (C) 1998 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. """Portable (?) file locking with timeouts. This code should work with all versions of NFS. The algorithm was suggested by the GNU/Linux open() man page. Make sure no malicious people have access to link() to the lock file. """ import socket, os, time import string import errno #from stat import ST_NLINK ST_NLINK = 3 # faster ST_CTIME = 9 ST_INO = 1 # default intervals are both specified in seconds, and can be floating point # values. DEFAULT_HUNG_TIMEOUT specifies the default length of time that a # lock is expecting to hold the lock -- this can be set in the constructor, or # changed via a mutator. DEFAULT_SLEEP_INTERVAL is the amount of time to # sleep before checking the lock's status, if we were not the winning claimant # the previous time around. DEFAULT_LOCK_LIFETIME = 15 DEFAULT_SLEEP_INTERVAL = 1.0 DEFAULT_INODE_GRACETIME = 2 from Mailman.Logging.StampedLogger import StampedLogger _logfile = None # exceptions which can be raised class LockError(Exception): """Base class for all exceptions in this module.""" pass class AlreadyLockedError(LockError): """Raised when a lock is attempted on an already locked object.""" pass class NotLockedError(LockError): """Raised when an unlock is attempted on an objec that isn't locked.""" pass class TimeOutError(LockError): """Raised when a lock was attempted, but did not succeed in the given amount of time. """ pass class StaleLockFileError(LockError): """Raised when a stale hardlink lock file was found.""" pass class LockFile: """A portable way to lock resources by way of the file system.""" def __init__(self, lockfile, lifetime=DEFAULT_LOCK_LIFETIME, sleep_interval=DEFAULT_SLEEP_INTERVAL, withlogging=0): """Creates a lock file using the specified file. lifetime is the maximum length of time expected to keep this lock. This value is written into the lock file so that other claimants on the lock know when it is safe to break the lock, should the lock holder be wedged. sleep_interval is roughly how often to wake up and check the lock file """ self.__lockfile = lockfile self.__lifetime = lifetime self.__sleep_interval = sleep_interval + (((os.getpid() % 20) - 11.5)/100.) self.__tmpfname = "%s.%s.%d" % (lockfile, socket.gethostname(), os.getpid()) self.__withlogging = withlogging self.__writelog("starting") def set_lifetime(self, lifetime): """Reset the lifetime of the lock. Takes affect the next time the file is locked. """ self.__lifetime = lifetime def __writelog(self, msg): global _logfile if self.__withlogging: if not _logfile: _logfile = StampedLogger('locks', manual_reprime=1) head, tail = os.path.split(self.__lockfile) _logfile.write('%s %s\n' % (tail, msg)) def refresh(self, newlifetime=None): """Refresh the lock. This writes a new release time into the lock file. Use this if a process suddenly realizes it needs more time to do its work. With optional newlifetime, this resets the lock lifetime value too. NotLockedError is raised if we don't already own the lock. """ if not self.locked(): raise NotLockedError if newlifetime is not None: self.set_lifetime(newlifetime) self.__write() self.__writelog('lock lifetime refreshed for %d seconds' % self.__lifetime) def __del__(self): if self.locked(): self.unlock() def __write(self): # we expect to release our lock some time in the future. we want to # give other claimants some clue as to when we think we're going to be # done with it, so they don't try to steal it out from underneath us # unless we've actually been wedged. lockrelease = time.time() + self.__lifetime # make sure it's group writable oldmask = os.umask(002) try: fp = open(self.__tmpfname, 'w') fp.write('%d %s %f\n' % (os.getpid(), self.__tmpfname, lockrelease)) fp.close() finally: os.umask(oldmask) def __read(self): # can raise ValueError in three situations: # # - The lockfile does not exist (it's been released or broken) # - We didn't get a sequence of the right size from split # - the first 'word' is not an integer # In each case we raise ValueError in response try: fp = open(self.__lockfile, 'r') except IOError, err: if err.errno == errno.ENOENT: raise ValueError # invalid contents try: pid, winner, lockrelease = string.split(string.strip(fp.read())) finally: fp.close() return int(pid), winner, float(lockrelease) def __break(self): try: if os.stat(self.__lockfile)[ST_CTIME] > time.time() - DEFAULT_INODE_GRACETIME: # The lockfile has changed very recently, so it might have # changed since our caller evaluated the validity return except OSError, err: if err.errno == errno.ENOENT: return # Someone broke it already raise os.unlink(self.__lockfile) # Note that this function is mostly redundant, the try/except around # os.link does almost all of what we do. Instead of lockfile-contents # we could use lockfile CTIME, which would probably speed things up. def lock(self, timeout=0): """Blocks until the lock can be obtained. Raises a TimeOutError exception if a positive timeout value is given and that time elapses before the lock is obtained. This can possibly steal the lock from some other claimant, if the lock lifetime that was written to the file has been exceeded. Note that for this to work across machines, the clocks must be sufficiently synchronized. """ if self.locked(): self.__writelog('already locked') raise AlreadyLockedError if timeout: timeout_time = time.time() + timeout last_pid = -1 self.__write() # Make sure our own lockfile exists while 1: # create the hard link and test for exactly 2 links to the file try: os.link(self.__tmpfname, self.__lockfile) except OSError, err: if err.errno == errno.ENOENT: # Our own lockfile has vanished from beneath us ? self.__write() # sleep anyway, so we dont hog the machine self.__sleep() continue elif err.errno == errno.EEXIST: # We failed to get the lock. If we worry about this locking # scheme, we set a variable here and ascertain ourselves of # its value later, when we think we have the lock. pass else: raise if os.stat(self.__tmpfname)[ST_NLINK] == 2: # we have the lock (since noone overwrote our link to the lock # file), so we re-piss our hydrant, to refresh the lock timeout self.__write() now = time.time() if not self.locked(): self.__writelog('false positive on lockfile (%f)'%now) self.__sleep() continue self.__writelog('got the lock (%f)'%now) break # we didn't get the lock this time. let's see if we timed out if timeout and timeout_time < time.time(): os.unlink(self.__tmpfname) self.__writelog('timed out') raise TimeOutError # someone else must have gotten the lock. let's find out who it # is. if there is some bogosity in the lock file's data then we # will try to steal the lock. try: pid, winner, lockrelease = self.__read() except ValueError: self.__writelog('invalid lock contents; breaking lock') self.__break() self.__sleep() continue # check for hanging processes, by remembering pid, and if it stays # the same, checking lockrelease. if pid <> last_pid: last_pid = pid elif lockrelease < time.time(): # We've waited long enough... # Attempt to break the lock. We dont care much about the old # winners __tmpfname, let it stay. self.__writelog('lock timeout; breaking lock') self.__break() self.__sleep() continue # okay, someone else has the lock, we weren't able to steal it, # and our claim hasn't timed out yet. So let's wait a while for # the owner of the lock to give it up. self.__writelog('waiting for claim') self.__sleep() def __sleep(self): time.sleep(self.__sleep_interval) # This could error if the lock is stolen. You must catch it. def unlock(self): """Unlock the lock. If we don't already own the lock (either because of unbalanced unlock calls, or because the lock was stolen out from under us), raise a NotLockedError. """ if not self.locked(): raise NotLockedError now = time.time() os.unlink(self.__lockfile) os.unlink(self.__tmpfname) self.__writelog('unlocked (%f)'%now) def locked(self): """Returns 1 if we own the lock, 0 if we do not.""" if not os.path.exists(self.__tmpfname): return 0 if os.stat(self.__tmpfname)[ST_NLINK] <> 2: return 0 # The above check for nlink should be enough to validate our claim on # the lock. Nevertheless, we are paranoid. try: pid, winner, lockrelease = self.__read() except ValueError: # the contents of the lock file was corrupted, or the global lock # is not a link to our lock. if os.stat(self.__lockfile)[ST_INO] <> os.stat(self.__tmpfname)[ST_INO]: self.__writelog("nlink of 2 but we don't own the lock ? strangeness") os.unlink(self.__tmpfname) self.__write() return 0 else: # Okay, we really do have the lock, but the contents is messed up self.__write() return 1 return pid == os.getpid() # use with caution!!! def steal(self): """Explicitly steal the lock. USE WITH CAUTION!""" self.__writelog('explicitly breaking lock') if self.locked(): return 1 try: pid, winner, timeout = self.__read() os.rename(winner, self.__tmpfname) except ValueError: return 0 except OSError, err: if err.errno == errno.ENOENT: return 0 raise self.__write() return self.locked() --oj4kGyHlBMXGt3Le-- From thomas@xs4all.net Mon Feb 7 21:56:07 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 22:56:07 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207200918.A18352@overdue.dhis.net>; from lazarus@overdue.dhis.org on Mon, Feb 07, 2000 at 08:09:18PM +0000 References: <20000207171959.K23471@xs4all.nl> <20000207200918.A18352@overdue.dhis.net> Message-ID: <20000207225606.U23471@xs4all.nl> --3O1VwFp74L81IIeR Content-Type: text/plain; charset=us-ascii On Mon, Feb 07, 2000 at 08:09:18PM +0000, Lazarus Long wrote: > On Mon, Feb 07, 2000 at 06:16:06PM +0100, Gergely Madarasz wrote: > > Thanks. > > Lazarus, please test this patch, and notify me if it worked for you :) > Patching file `gate_news' using Plan A... > Hunk #1 FAILED at 125. > 1 out of 1 hunk FAILED -- saving rejects to gate_news.rej Ok, the patch doesn't work for mailman-1.1 gate_news ;P I've attached a version that should. Let me know if it still fails, or if you're using a different version of mailman. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --3O1VwFp74L81IIeR Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="gate_news.diff" --- cron/gate_news.orig Mon Feb 7 22:47:52 2000 +++ cron/gate_news Mon Feb 7 22:53:23 2000 @@ -76,7 +76,16 @@ # update the gate_watermarks file. we'll actually do the gating in a # child process conn = nntplib.NNTP(mlist.nntp_host) - r,c,first,last,n = conn.group(mlist.linked_newsgroup) + try: + r,c,first,last,n = conn.group(mlist.linked_newsgroup) + except nntplib.error_perm: + # We got an error 500, probably meaning that the 'group' command + # is not implemented. A likely scenario causing this is when + # running the Mailman gateway on the same machine as the news + # server; the server defaults to news feed mode. We need to set + # the connection to reader mode first. + conn.shortcmd('mode reader') + r,c,first,last,n = conn.group(mlist.linked_newsgroup) first = int(first) last = int(last) wm = watermarks.get(name, 0) --3O1VwFp74L81IIeR-- From thomas@xs4all.net Mon Feb 7 21:59:07 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 22:59:07 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000207213143.B444@miss-janet.com>; from ricardo@rixhq.nu on Mon, Feb 07, 2000 at 09:31:44PM +0100 References: <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> <20000207204331.A444@miss-janet.com> <20000207213143.B444@miss-janet.com> Message-ID: <20000207225907.M19265@xs4all.nl> On Mon, Feb 07, 2000 at 09:31:44PM +0100, Ricardo Kustner wrote: > I wonder though what happens if some impatient moderator decides not to > wait before the page finnishes loading, switches to a differen webpage and > therefor breaks the python cgi process... will some approved posts stay in > the queue instead? Try it out ! ;) It depends on the exact behaviour of both the webbrowser and the webserver. I haven't checked the Mailman code but i assume it either continues with its jobs until it tries to write output, usually at the end of the script, or it gets a signal and cleans up nicely. Postings shouldn't disappear, if that's what you mean. > I mentioned before that I think that it could be better if the cgi scripts > don't do anything more than just "mark" message as being approved... cgi > scripts should have a short life time and definately shouldn't be waiting > for something too long... I agree, and I think I've seen more comments and postings talking about it. Noone has implemented it yet, though, and the current method works too well for it to be a rela problem, I guess ;) > btw i used the original LockFile.py this time... Go ahead and try it with mine -- it should make the waits you have to endure a tad shorter. The problem with the old locking mechanism is that a lot of processes end up butting heads in a painful way when they try to grab the lock. In the modified one, the first process to try and get the lock after it is freed, actually gets the lock. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Tue Feb 8 01:05:27 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 8 Feb 2000 02:05:27 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval Message-ID: <20000208020526.N19265@xs4all.nl> --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Another minor patch adding what I need to mailman ;) I also noticed this on the jitterbug list, so I guess more people had wished or at least wondered for it ;P I'm posting it mostly to see if it generates comments, and for me to find a good excuse to sleep. It's functional, it works on my test-list, but some of it (most notably the english -- is there a good word for 'unsubscription request' ?) sucks. Adding this has been mostly an exercise to learn a bit about the Mailman insides, and to see how easily it can be done; fairly easily ;) Comments are welcome, including "d'oh, dont write that, i already wrote it !" The patchs misses the file 'templates/unsubauth.txt' by the way.. Just copy subauth.txt and change 'subscr' into 'unsubscr' ;P Sleepy-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mailman-unsub-approval.diff" ? templates/unsubauth.txt Index: Mailman/Bouncer.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Bouncer.py,v retrieving revision 1.37 diff -u -r1.37 Bouncer.py --- Bouncer.py 1999/12/16 17:11:04 1.37 +++ Bouncer.py 2000/02/08 00:57:47 @@ -299,7 +299,7 @@ self.real_name, addr, reason) return reason, 1 try: - self.DeleteMember(addr, "bouncing addr") + self.ApprovedDeleteMember(addr, "bouncing addr") self.LogMsg("bounce", "%s: removed %s", self.real_name, addr) self.Save() return 1, 1 Index: Mailman/Defaults.py.in =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Defaults.py.in,v retrieving revision 1.90 diff -u -r1.90 Defaults.py.in --- Defaults.py.in 1999/11/30 23:10:57 1.90 +++ Defaults.py.in 2000/02/08 00:57:47 @@ -344,4 +344,4 @@ from Version import VERSION # Data file version number -DATA_FILE_VERSION = 15 +DATA_FILE_VERSION = 16 Index: Mailman/HTMLFormatter.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/HTMLFormatter.py,v retrieving revision 1.46 diff -u -r1.46 HTMLFormatter.py --- HTMLFormatter.py 1999/11/24 21:15:21 1.46 +++ HTMLFormatter.py 2000/02/08 00:57:47 @@ -206,6 +206,11 @@ " request will be sent to the '%s' account for" " your address.)" % self.umbrella_member_suffix) + if self.unsubscribe_policy == 1: + msg = msg + ("

(Please note that this list does not allow" + " members to unsubscribe themselves without" + " administrator approval. Unsubscription requests" + " will get forwarded to the mailinglist administrator.)") return msg def FormatUndigestButton(self): Index: Mailman/ListAdmin.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/ListAdmin.py,v retrieving revision 1.27 diff -u -r1.27 ListAdmin.py --- ListAdmin.py 1999/11/24 21:13:59 1.27 +++ ListAdmin.py 2000/02/08 00:57:48 @@ -40,6 +40,7 @@ # Request types requiring admin approval HELDMSG = 1 SUBSCRIPTION = 2 +UNSUBSCRIPTION = 3 @@ -106,6 +107,9 @@ def GetSubscriptionIds(self): return self.__getmsgids(SUBSCRIPTION) + def GetUnsubscriptionIds(self): + return self.__getmsgids(UNSUBSCRIPTION) + def GetRecord(self, id): self.__opendb() type, data = self.__db[id] @@ -122,9 +126,11 @@ del self.__db[id] if rtype == HELDMSG: self.__handlepost(data, value, comment) - else: - assert rtype == SUBSCRIPTION + elif rtype == SUBSCRIPTION: self.__handlesubscription(data, value, comment) + else: + assert rtype == UNSUBSCRIPTION + self.__handleunsubscription(data, value, comment) def HoldMessage(self, msg, reason): # assure that the database is open for writing @@ -258,6 +264,53 @@ assert value == 1 self.ApprovedAddMember(addr, password, digest) + def HoldUnsubscription(self, addr, whence, admin_notif): + # assure that the database is open for writing + self.__opendb() + # get the next unique id + id = self.__request_id() + assert not self.__db.has_key(id) + # + # save the information to the request database. for held subscription + # entries, each record in the database will be one of the following + # format: + # + # the time the subscription request was received + # the subscriber's address + # the log message + # wether or not to send an unsubscription notify to the list admin + # + data = time.time(), addr, whence, admin_notif + self.__db[id] = (UNSUBSCRIPTION, data) + # + # TBD: this really shouldn't go here but I'm not sure where else is + # appropriate. + self.LogMsg('vette', '%s: held unsubscription request from %s' % + (self.real_name, addr)) + # possibly notify the administrator + if self.admin_immed_notify: + subject = 'New unsubscription request to list %s from %s' % ( + self.real_name, addr) + text = Utils.maketext( + 'unsubauth.txt', + {'username' : addr, + 'listname' : self.real_name, + 'hostname' : self.host_name, + 'admindb_url': self.GetAbsoluteScriptURL('admindb'), + }) + adminaddr = self.GetAdminEmail() + msg = Message.UserNotification(adminaddr, adminaddr, subject, text) + HandlerAPI.DeliverToUser(self, msg) + + def __handleunsubscription(self, record, value, comment): + stime, addr, whence, admin_notif = record + if value == 0: + # refused + self.__refuse('Unsubscription request', addr, comment) + else: + # subscribe + assert value == 1 + self.ApprovedDeleteMember(addr, whence, admin_notif) def __refuse(self, request, recip, comment, origmsg=None): adminaddr = self.GetAdminEmail() Index: Mailman/MailCommandHandler.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/MailCommandHandler.py,v retrieving revision 1.59 diff -u -r1.59 MailCommandHandler.py --- MailCommandHandler.py 1999/12/16 17:13:28 1.59 +++ MailCommandHandler.py 2000/02/08 00:57:48 @@ -500,6 +500,8 @@ self.ConfirmUserPassword(addr, args[0]) self.DeleteMember(addr, "mailcmd") self.AddToResponse("Succeeded.") + except Errors.MMNeedApproval: + self.AddToResponse("Unsubscribing needs admininistrator approval.\nYour request has been forwarded to the list administrator.") except Errors.MMListNotReady: self.AddError("List is not functional.") except (Errors.MMNoSuchUserError, Errors.MMNotAMemberError): Index: Mailman/MailList.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/MailList.py,v retrieving revision 1.151 diff -u -r1.151 MailList.py --- MailList.py 1999/12/27 22:54:55 1.151 +++ MailList.py 2000/02/08 00:57:49 @@ -339,6 +339,7 @@ self.welcome_msg = '' self.goodbye_msg = '' self.subscribe_policy = mm_cfg.DEFAULT_SUBSCRIBE_POLICY + self.unsubscribe_policy = mm_cfg.DEFAULT_UNSUBSCRIBE_POLICY self.private_roster = mm_cfg.DEFAULT_PRIVATE_ROSTER self.obscure_addresses = mm_cfg.DEFAULT_OBSCURE_ADDRESSES self.member_posting_only = mm_cfg.DEFAULT_MEMBER_POSTING_ONLY @@ -599,6 +600,17 @@ sub_cfentry, + ('unsubscribe_policy', mm_cfg.Radio, + ('none', 'require approval'), 0, + "What steps are required for unsubscription?
", + "none - anyone can unsubscribe using email or the webpage
" + "require approval - require list administrator approval" + " for unsubscriptions
" + "

Please leave the unsubscription 'open' unless you have a" + " very good reason to require approval. Leaving people on lists" + " against their will can easily be seen as spamming or harassment" + ), + "Membership exposure", ('private_roster', mm_cfg.Radio, @@ -1083,7 +1095,7 @@ HandlerAPI.DeliverToUser(self, msg) return result - def DeleteMember(self, name, whence=None, admin_notif=None): + def ApprovedDeleteMember(self, name, whence=None, admin_notif=None): self.IsListInitialized() # FindMatchingAddresses *should* never return more than 1 address. # However, should log this, just to make sure. @@ -1139,6 +1151,15 @@ whence = "" self.LogMsg("subscribe", "%s: deleted %s%s", self._internal_name, name, whence) + + def DeleteMember(self, name, whence=None, admin_notif=None): + self.IsListInitialized() + # FindMatchingAddresses *should* never return more than 1 address. + # However, should log this, just to make sure. + if self.unsubscribe_policy == 1: + self.HoldUnsubscription(name, whence, admin_notif) + raise Errors.MMNeedApproval + ApprovedDeleteMember(name, whence, admin_notif) def IsMember(self, address): return len(Utils.FindMatchingAddresses(address, self.members, Index: Mailman/Cgi/admin.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Cgi/admin.py,v retrieving revision 1.53 diff -u -r1.53 admin.py --- admin.py 1999/12/27 22:28:07 1.53 +++ admin.py 2000/02/08 00:57:50 @@ -887,7 +887,7 @@ for user in users: if not cgi_info.has_key('%s_subscribed' % (user)): try: - mlist.DeleteMember(user) + mlist.ApprovedDeleteMember(user) dirty = 1 except Errors.MMNoSuchUserError: unsubscribe_errors.append((user, 'Not subscribed')) Index: Mailman/Cgi/admindb.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Cgi/admindb.py,v retrieving revision 1.19 diff -u -r1.19 admindb.py --- admindb.py 1999/11/15 22:29:46 1.19 +++ admindb.py 2000/02/08 00:57:50 @@ -164,6 +164,19 @@ for id in subpendings: PrintAddMemberRequest(mlist, id, t) form.AddItem(t) + unsubpendings = mlist.GetUnsubscriptionIds() + if unsubpendings: + form.AddItem('

') + form.AddItem(Center(Header(2, 'Unsubscription Requests'))) + t = Table(border=2) + t.AddRow([ + Bold('Address'), + Bold('Your Decision'), + Bold('Reason for unsubscription refusal (optional)') + ]) + for id in unsubpendings: + PrintDeleteMemberRequest(mlist, id, t) + form.AddItem(t) # Post holds are now handled differently heldmsgs = mlist.GetHeldMessageIds() total = len(heldmsgs) @@ -183,6 +196,13 @@ time, addr, passwd, digest = mlist.GetRecord(id) table.AddRow([addr, RadioButtonArray(id, ('Refuse', 'Subscribe')), + TextBox('comment-%d' % id, size=30) + ]) + +def PrintDeleteMemberRequest(mlist, id, table): + time, addr, whence, admin_notif = mlist.GetRecord(id) + table.AddRow([addr, + RadioButtonArray(id, ('Refuse', 'Unsubscribe')), TextBox('comment-%d' % id, size=30) ]) Index: Mailman/Cgi/handle_opts.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Cgi/handle_opts.py,v retrieving revision 1.18 diff -u -r1.18 handle_opts.py --- handle_opts.py 2000/01/21 20:30:51 1.18 +++ handle_opts.py 2000/02/08 00:57:50 @@ -101,6 +101,12 @@ pw = form["upw"].value if mlist.ConfirmUserPassword(user, pw): mlist.DeleteMember(user, "web cmd") + except Errors.MMNeedApproval: + PrintResults(mlist, operation, doc, + "Unsubscribing from this list requires " + "administrator approval. Your request has been " + "forwarded. Please contact the list administrator " + "if you have any questions.") except Errors.MMListNotReady: PrintResults(mlist, operation, doc, "List is not functional.") except Errors.MMNoSuchUserError: Index: bin/remove_members =================================================================== RCS file: /projects/cvsroot/mailman/bin/remove_members,v retrieving revision 1.3 diff -u -r1.3 remove_members --- remove_members 1999/03/04 17:18:24 1.3 +++ remove_members 2000/02/08 00:57:50 @@ -100,7 +100,7 @@ for addr in addresses: try: - mlist.DeleteMember(addr) + mlist.ApprovedDeleteMember(addr) except Errors.MMNoSuchUserError: print "User `%s' not found." % addr Index: bin/sync_members =================================================================== RCS file: /projects/cvsroot/mailman/bin/sync_members,v retrieving revision 1.8 diff -u -r1.8 sync_members --- sync_members 1999/11/30 21:17:33 1.8 +++ sync_members 2000/02/08 00:57:50 @@ -244,7 +244,7 @@ for laddr, addr in addrs.items(): # should be a member, otherwise our test above is broken if not dryrun: - mlist.DeleteMember(addr, admin_notif=notifyadmin) + mlist.ApprovedDeleteMember(addr, admin_notif=notifyadmin) print 'Removed: %30s (%30s)' % (laddr, addr) mlist.Save() --ZPt4rx8FFjLCG7dd-- From jwt@dskk.co.jp Tue Feb 8 02:47:06 2000 From: jwt@dskk.co.jp (Jim Tittsler) Date: Tue, 8 Feb 2000 11:47:06 +0900 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207191713.L19265@xs4all.nl>; from thomas@xs4all.net on Mon, Feb 07, 2000 at 07:17:13PM +0100 References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> Message-ID: <20000208114706.C2908@mail.dskk.co.jp> --FCuugMFkClbJLl1L Content-Type: text/plain; charset=us-ascii On Mon, Feb 07, 2000 at 07:17:13PM +0100, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 12:53:17PM -0500, Barry A. Warsaw wrote: > > > I'm cutting and pasting a message that originated on the mailman-users > > mailing list. I think this message is fairly self-explanatory. Can > > other NNTP experts look this over and see if it makes sense? I'll > > apply it if there are no objections. > > Note: the bare except is bothersome. I confess I wasn't sure if a news server that didn't recognize the 'mode reader' could be counted on to return a 500 series error. It seemed logical, but I couldn't be sure. (All of the servers I have access to accept the 'mode reader'.) > Note, though, that i've already posted a small patch to mailman to fix this. > Since the 'group' command is executed by gate_news, which doesn't need > authentication, it's easy enough to try the group, and set the connection to > readermode if it fails. I dislike your proposed solution for two reasons: 1. It uses the nntplib's shortcmd outside of nntplib, even though it is documented as "internal". 2. It doesn't work when you *do* need authentication. The group command will claim 'No such group' for groups that are hidden by authentication. (One of my mailman run on such a machine.) You need to do the 'mode reader' before authenticating. I still think it better to change nntplib. > Also note that I've already sent a patch to patches@python.org ;) with a > pointer to the mailman-users discussion in question. > > http://www.python.org/pipermail/patches/2000-February/000015.html > > My patch uses an extra argument to NNTP.__init__, and avoids the bare > except. To get the same behaviour as the patch below, sending 'mode > reader' by default, the default value of readermode can be set to 1. I'm not > sure if that's a wise thing, however, as some users of nntplib.NNTP might > depend on the connection being able to do things that you cant do in > readermode. I sent a similar nntplib patch to mailman-users last week, although I defaulted to the current behavior and required the additional argument (or a user specified for authentication) to get the "newsreader" behavior. http://www.python.org/pipermail/mailman-users/2000-February/003935.html Jim --FCuugMFkClbJLl1L Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="nntplib.diff" --- /home/jwt/nntplib.py.orig Sat Sep 18 09:16:08 1999 +++ nntplib.py Sat Sep 18 11:39:00 1999 @@ -61,17 +61,24 @@ # Initialize an instance. Arguments: # - host: hostname to connect to # - port: port to connect to (default the standard NNTP port) + # - mode: if not None (or user is specified), issue 'mode reader' - def __init__(self, host, port = NNTP_PORT, user=None, password=None): + def __init__(self, host, port = NNTP_PORT, user=None, password=None, + mode=None): self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.sock.connect(self.host, self.port) self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + if user or mode: + try: + self.welcome = self.shortcmd('mode reader') + except: + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: @@ -466,9 +473,9 @@ # Minimal test function def _test(): - s = NNTP('news') + s = NNTP('news', mode='reader') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last resp, subs = s.xhdr('subject', first + '-' + last) --FCuugMFkClbJLl1L-- From Harald.Meland@usit.uio.no Tue Feb 8 02:58:47 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 08 Feb 2000 03:58:47 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: Thomas Wouters's message of "Mon, 7 Feb 2000 22:44:58 +0100" References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <14488.44469.446995.168328@anthem.cnri.reston.va.us> <20000203103555.C6378@xs4all.nl> <20000207224457.T23471@xs4all.nl> Message-ID: [Thomas Wouters] > On Mon, Feb 07, 2000 at 10:01:36PM +0100, Harald Meland wrote: > > > My current CVS checkout is available ("live" :) under > http://www.uio.no/~hmeland/tmp/mailman-userdb/>, with > http://www.uio.no/~hmeland/tmp/mailman-userdb/Mailman/LockFile.py> > > being of special interest in this discussion... > > Ah, very cool. You'll find my current version of LockFile.py > attached, for reference, as you asked. Thanks, I've skimmed through most of it now. > I think the link() solutions, as opposed to the rename() solution, > is a better one, if you'll premit me. Sure :) > For one, you can check on lock ownership by statting your own > lockfile, which should only be touched by you, instead of reading a > possibly highly volatile lockfile. You can also doublecheck the > lockfile inode, see if it is equal to your own lockfiles' inode, to > confirm that you have the lock. ... the downside being that if, e.g., anyone tries to read the info in the lockfile in the middle of a refresh(), they'll get a ValueError -- which your module seems to take care of, but it all looks a bit convoluted to my eyes. I especially dislike that lock() has to call locked() _after_ the check on ST_NLINK, because further down in lock() there's code that might break a lock it really shouldn't (if it weren't for the nonatomic-write-of-lock-contents issue). I also see some bugs that are fixed in my module but still present in yours, but those are rather irrelevant to this discussion (as they can easily be ported to your module if that's the one we'll go for). Does anyone but Thomas and me have any opinions on which locking scheme is preferrable? I guess I don't have any very strong feelings about the issue ;), so any sagely advise on the matter would be welcome. [ I've tried to model my version of the module as closely as possible after the way Exim does its lockfile locking, the idea being that Exim really should have been around long enough to have solved these things correctly. Reading the longish comment from ca. line 1206 and onwards in src/transports/appendfile.c of the Exim distribution proved very helpful to me. Of course, when it comes to lock stealing and other ugly (but needed, I guess) stuff, Exim couldn't provide much help. ] > Ah, well, I couldn't find any parts of Mailman in the CVS tree that > acually _called_ steal, so I couldn't figure out what it should do > in borderline cases. The only place I know of is cron/gate_news. > However, reading your LockFile.py made me realise how to steal the > lock. LockFile.steal() now does its utmost best to steal the lock, > only failing if someone else is busy stealing the lock from us. It > returns 1 for success, 0 for failure, mostly because I wanted it to > signify that in some way. ... but the caller should note that steal() returning 1 does not necessarily imply that it is the owner of the lock (anymore). The steal() stuff is bound to be full of race conditions, AFAICT -- so whether it returns anything doesn't really matter (IMHO), as the return code truly can't be trusted anyway. Cheers, -- Harald From Harald.Meland@usit.uio.no Tue Feb 8 03:06:15 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 08 Feb 2000 04:06:15 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: Thomas Wouters's message of "Mon, 7 Feb 2000 22:59:07 +0100" References: <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> <20000207204331.A444@miss-janet.com> <20000207213143.B444@miss-janet.com> <20000207225907.M19265@xs4all.nl> Message-ID: [Thomas Wouters] > On Mon, Feb 07, 2000 at 09:31:44PM +0100, Ricardo Kustner wrote: > > > I wonder though what happens if some impatient moderator decides not to > > wait before the page finnishes loading, switches to a differen webpage and > > therefor breaks the python cgi process... will some approved posts stay in > > the queue instead? > > Try it out ! ;) It depends on the exact behaviour of both the > webbrowser and the webserver. Yup. This might produce a SIGPIPE, and/or the corresponding IOError EPIPE, and I've experienced that these didn't always get caught -- an attempt at a fix is near the end of run_main in , but I'm by no means positive that it's the Right fix. -- Harald From thomas@xs4all.net Tue Feb 8 08:43:18 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 8 Feb 2000 09:43:18 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000208114706.C2908@mail.dskk.co.jp>; from jwt@dskk.co.jp on Tue, Feb 08, 2000 at 11:47:06AM +0900 References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> Message-ID: <20000208094318.Z23471@xs4all.nl> On Tue, Feb 08, 2000 at 11:47:06AM +0900, Jim Tittsler wrote: > On Mon, Feb 07, 2000 at 07:17:13PM +0100, Thomas Wouters wrote: > > Note, though, that i've already posted a small patch to mailman to fix this. > > Since the 'group' command is executed by gate_news, which doesn't need > > authentication, it's easy enough to try the group, and set the connection to > > readermode if it fails. > > I dislike your proposed solution for two reasons: > 1. It uses the nntplib's shortcmd outside of nntplib, even though it is > documented as "internal". > 2. It doesn't work when you *do* need authentication. The group command > will claim 'No such group' for groups that are hidden by authentication. > (One of my mailman run on such a machine.) You need to do the 'mode > reader' before authenticating. > I still think it better to change nntplib. So do I. However, Mailman does currently *not* use authentication there, so this is the simplest fix, including backwards to mailman 1.1. or earlier. I was not aware that shortcmd() is nntplib-private, though. I assumed it was public because it does not start with a _. If this is a problem, fixing nntplib and using a private copy might be the better solution. > > Also note that I've already sent a patch to patches@python.org ;) with a > > pointer to the mailman-users discussion in question. > > http://www.python.org/pipermail/patches/2000-February/000015.html > I sent a similar nntplib patch to mailman-users last week, although I Ah, sorry, I dont read mailman-users, apparently. I thought I did, though, Strange. I'm sorry if I seem to have stolen your idea, it was not intentional. Anyway, the right place for this patch is not Mailman, it is Python. nntplib is part of the standard library, nowadays, Mailman only has its own local copy for compatibility with older python distributions. I'm a bit fearful of setting readermode so often, though -- aren't there possibly cases where people want to provide authentication, but do not want readermode ? (I'm not concerned with newsservers that dont understand 'mode reader'. If they barf on that, they are highly likely to barf on everything else you try out, too. I am concerned with setting a newsconnection to reader when you dont want a reader -- though I'm not sure what would be the effect.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From julian7@kva.hu Tue Feb 8 15:34:56 2000 From: julian7@kva.hu (Balazs Nagy) Date: Tue, 8 Feb 2000 16:34:56 +0100 (CET) Subject: [Mailman-Developers] Reply goes to list error In-Reply-To: Message-ID: On 7 Feb 2000, Harald Meland wrote: > > On Tue, 18 Jan 2000, Balazs Nagy wrote: > > > > I just cannot understand why nobody from thosw who have write access > > did anything about the subject. > > Because Reply-To: munging is already part of CookHeaders.py, I guess. > And that takes appropriate note of the "fastrack" attribute of the > message object, too. CookHeaders doesn't do anything about the subject. It changes Reply-To and deletes From fields when the list is anonymous. There's an option (a deprecated option) to do it, but some ways it can be handy. It's a feature which is declared in the administration web page and it doesn't work. Why nobody want to fix it? -- Regards: Kevin (Balazs) From Ola.Flygt@msi.vxu.se Tue Feb 8 14:46:48 2000 From: Ola.Flygt@msi.vxu.se (Ola Flygt) Date: Tue, 8 Feb 2000 15:46:48 +0100 Subject: [Mailman-Developers] editing email addresses Message-ID: Hello At V=E4xj=F6 University we are using Mailman for a number of lists. We are now in the process of changing email addresses for a lot of users since we have a change in the organization. Is there a way to change all the users email addresses in the database instead of deleting and adding them again? Best Regards, Ola Flygt From Harald.Meland@usit.uio.no Tue Feb 8 15:25:18 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 08 Feb 2000 16:25:18 +0100 Subject: [Mailman-Developers] Reply goes to list error In-Reply-To: Balazs Nagy's message of "Tue, 8 Feb 2000 16:34:56 +0100 (CET)" References: Message-ID: [Balazs Nagy] > On 7 Feb 2000, Harald Meland wrote: > > > > On Tue, 18 Jan 2000, Balazs Nagy wrote: > > > > > > I just cannot understand why nobody from thosw who have write access > > > did anything about the subject. > > > > Because Reply-To: munging is already part of CookHeaders.py, I guess. > > And that takes appropriate note of the "fastrack" attribute of the > > message object, too. > > CookHeaders doesn't do anything about the subject. Ooops, sorry -- my version did, but I hadn't checked that in. Checked in now. -- Harald From gerrit@nl.linux.org Tue Feb 8 15:20:34 2000 From: gerrit@nl.linux.org (Gerrit Holl) Date: Tue, 8 Feb 2000 16:20:34 +0100 Subject: [Mailman-Developers] Two more TODO entries Message-ID: <20000208162034.A3621@stopcontact.palga.uucp> Hello, I have two TODO list entries: * I think a nice feature would be to set headers. You could set an option "setheader", "header" and "content", with a boolean option "override". I think it should be added to the TODO list. Maybe Barry (and others) think Reply-To sucks, but when it's not set it's certainly a nice feature to have it set. I've added this hack at my site: if not msg.get('Reply-To'): msg['Reply-To'] = mlist.GetListEmail() in Mailman.Handlers.Cookheaders.process Because currently *all* mailinglist use it. * A mail admin having an own page with all lists he is admin of, soe can approve messages and subscribsions from a single place. I'm wanting to contribute to the documentation. Has this ever been discussed yet? If not, I'm going to start with a document using the Linuxdoc DTD as soon as I understand all Mailman code (which'll take quite a long time, to be fair). regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From gerrit@nl.linux.org Tue Feb 8 14:30:43 2000 From: gerrit@nl.linux.org (Gerrit Holl) Date: Tue, 8 Feb 2000 15:30:43 +0100 Subject: [Mailman-Developers] Patch: raising exception when user can't change admin password Message-ID: <20000208153043.A3437@stopcontact.palga.uucp> --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Hello, I'm not sure wheter this is the right place to send patches. I sent a patch to mailman-cabal but it didn't seem to be the right place. Redirect me if I'm wrong. This patch, to SecurityManager.py, raises the MMAuthenticationError if an unauthorized user tries to change the site admin password. Currently, you just het an IOError. I find the currently unused MMAuthenticationError more clear. regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="SecurityManager.py.diff" --- SecurityManager.py Tue Feb 8 15:27:20 2000 +++ /tmp/SecurityManager.py Tue Feb 8 15:25:13 2000 @@ -36,7 +36,10 @@ class SecurityManager: def SetSiteAdminPassword(self, pw): - fp = Utils.open_ex(SITE_PW_FILE, 'w', perms=0640) + try: + fp = Utils.open_ex(SITE_PW_FILE, 'w', perms=0640) + except IOError: + raise Errors.MMAuthenticationError("no permission to change password") fp.write(Crypt.crypt(pw, Utils.GetRandomSeed())) fp.close() --Dxnq1zWXvFF0Q93v-- From bwarsaw@cnri.reston.va.us Tue Feb 8 17:08:09 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Tue, 8 Feb 2000 12:08:09 -0500 (EST) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> Message-ID: <14496.19833.341973.55133@anthem.cnri.reston.va.us> >>>>> "JT" == Jim Tittsler writes: JT> I still think it better to change nntplib. I agree. I'd still like to get rid of your bare except though; I'm guessing that error_perm is probably correct because of this: >>> import nntplib >>> x = nntplib.NNTP('ournews') >>> x.shortcmd('mode foobar') Traceback (innermost last): File "", line 1, in ? File "/depot/sundry/lib/python1.5/nntplib.py", line 158, in shortcmd return self.getresp() File "/depot/sundry/lib/python1.5/nntplib.py", line 134, in getresp raise error_perm, resp nntplib.error_perm: 500 Syntax error or bad command The Right Thing To Do is to recast the nntplib errors as exception classes and perhaps use it's base class in the except. I'd be willing to make this change, and merge in your patch, but you'll need to resend your patch with the standard email disclaimer appended. See http://www.python.org/patches/ for details. Would you be willing to do that? Thanks, -Barry From Gerrit Tue Feb 8 21:28:14 2000 From: Gerrit (Gerrit Holl) Date: Tue, 8 Feb 2000 22:28:14 +0100 Subject: [Mailman-Developers] When will mailman 1.2 be released? Message-ID: <20000208222814.A3736@stopcontact.palga.uucp> Hello, when do you think mailman 1.2 will be released? How stable is the current CVS tree, and how heavily is it tested? In what order of magnitude is the release frequency? regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From ricardo@rixhq.nu Tue Feb 8 21:48:51 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Tue, 8 Feb 2000 22:48:51 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000207225907.M19265@xs4all.nl>; from thomas@xs4all.net on Mon, Feb 07, 2000 at 10:59:07PM +0100 References: <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> <20000207204331.A444@miss-janet.com> <20000207213143.B444@miss-janet.com> <20000207225907.M19265@xs4all.nl> Message-ID: <20000208224851.B735@miss-janet.com> On Mon, Feb 07, 2000 at 10:59:07PM +0100, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 09:31:44PM +0100, Ricardo Kustner wrote: > > I wonder though what happens if some impatient moderator decides not to > > wait before the page finnishes loading, switches to a differen webpage and > > therefor breaks the python cgi process... will some approved posts stay in > > the queue instead? > Try it out ! ;) It depends on the exact behaviour of both the webbrowser and > the webserver. I haven't checked the Mailman code but i assume it either > continues with its jobs until it tries to write output, usually at the end > of the script, or it gets a signal and cleans up nicely. if you approve a bunch of posts it's a bit difficult work to figure out which posts arrive... especially since my server needs a bit of time to have send out all the posts... i never really timed how long it takes before all posts are send out (i don't care that much about that though... people need to wait anyway since the list is being moderated) > > > I mentioned before that I think that it could be better if the cgi scripts > > don't do anything more than just "mark" message as being approved... cgi > > scripts should have a short life time and definately shouldn't be waiting > > for something too long... > I agree, and I think I've seen more comments and postings talking about it. > Noone has implemented it yet, though, and the current method works too well > for it to be a rela problem, I guess ;) it's a big improvement that 1.2 now keeps the pending posts outside of the config.db, so i'm happy with that ... i have been one of the people begging for that :) > > btw i used the original LockFile.py this time... > Go ahead and try it with mine -- it should make the waits you have to endure could you send me your latest version (complete file is ok, and a bit easier for me :) ) thanks... Ricardo. -- From bwarsaw@cnri.reston.va.us Tue Feb 8 23:18:14 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Tue, 8 Feb 2000 18:18:14 -0500 (EST) Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> Message-ID: <14496.42038.761976.368014@anthem.cnri.reston.va.us> Okay, here's my proposed patch to the current CVS version of nntplib.py. It merges Jim's and Thomas's patches, and re-implements the module's exceptions as class-based (with backwards compatibility). I'm not sure what to do about the email disclaimers, but I'd say to be safe that both Jim and Thomas should re-submit their patches with the disclaimer attached (you can just to this to patches@python.org). Everyone else on patches@python.org: please let me know if you have any objections to this change. Otherwise, I'll commit it (and make the necessary documentation updates). -Barry -------------------- snip snip -------------------- Index: nntplib.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/nntplib.py,v retrieving revision 1.17 diff -c -r1.17 nntplib.py *** nntplib.py 2000/02/04 15:10:33 1.17 --- nntplib.py 2000/02/08 23:11:44 *************** *** 34,47 **** import string ! # Exception raised when an error or invalid response is received ! error_reply = 'nntplib.error_reply' # unexpected [123]xx reply ! error_temp = 'nntplib.error_temp' # 4xx errors ! error_perm = 'nntplib.error_perm' # 5xx errors ! error_proto = 'nntplib.error_proto' # response does not begin with [1-5] ! error_data = 'nntplib.error_data' # error in response data # Standard port used by NNTP servers NNTP_PORT = 119 --- 34,79 ---- import string ! ! # Exceptions raised when an error or invalid response is received ! class NNTPError(Exception): ! """Base class for all nntplib exceptions""" ! def __init__(self, *args): ! apply(Exception.__init__, (self,)+args) ! try: ! self.response = args[0] ! except IndexError: ! self.response = 'No response given' ! ! class NNTPReplyError(NNTPError): ! """Unexpected [123]xx reply""" ! pass ! ! class NNTPTemporaryError(NNTPError): ! """4xx errors""" ! pass ! ! class NNTPPermanentError(NNTPError): ! """5xx errors""" ! pass ! ! class NNTPProtocolError(NNTPError): ! """Response does not begin with [1-5]""" ! pass ! ! class NNTPDataError(NNTPError): ! """Error in response data""" ! pass ! ! # for backwards compatibility ! error_reply = NNTPReplyError ! error_temp = NNTPTemporaryError ! error_perm = NNTPPermanentError ! error_proto = NNTPProtocolError ! error_data = NNTPDataError + # Standard port used by NNTP servers NNTP_PORT = 119 *************** *** 54,68 **** CRLF = '\r\n' # The class itself - class NNTP: ! ! def __init__(self, host, port = NNTP_PORT, user=None, password=None): """Initialize an instance. Arguments: - host: hostname to connect to ! - port: port to connect to (default the standard NNTP port)""" ! self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) --- 86,108 ---- CRLF = '\r\n' + # The class itself class NNTP: ! def __init__(self, host, port = NNTP_PORT, user=None, password=None, ! readermode=None): """Initialize an instance. Arguments: - host: hostname to connect to ! - port: port to connect to (default the standard NNTP port) ! - user: username to authenticate with ! - password: password to use with username ! - readermode: if true, send 'mode reader' command after ! connecting. ! ! readermode is sometimes necessary if you are connecting to an ! NNTP server on your local machine and intend to call the ! group() method. ! """ self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) *************** *** 70,85 **** self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: ! raise error_reply, resp else: resp = self.shortcmd( 'authinfo pass '+password) if resp[:3] != '281': ! raise error_perm, resp def getwelcome(self): """Get the welcome message from the server --- 110,136 ---- self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + if readermode: + try: + self.welcome = self.shortcmd('mode reader') + except NNTPPermanentError: + # error 500, probably 'not implemented' + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: ! raise NNTPReplyError(resp) else: resp = self.shortcmd( 'authinfo pass '+password) if resp[:3] != '281': ! raise NNTPPermanentError(resp) ! ! # Get the welcome message from the server ! # (this is read and squirreled away by __init__()). ! # If the response code is 200, posting is allowed; ! # if it 201, posting is not allowed def getwelcome(self): """Get the welcome message from the server *************** *** 128,138 **** if self.debugging: print '*resp*', `resp` c = resp[:1] if c == '4': ! raise error_temp, resp if c == '5': ! raise error_perm, resp if c not in '123': ! raise error_proto, resp return resp def getlongresp(self): --- 179,189 ---- if self.debugging: print '*resp*', `resp` c = resp[:1] if c == '4': ! raise NNTPTemporaryError(resp) if c == '5': ! raise NNTPPermanentError(resp) if c not in '123': ! raise NNTPProtocolError(resp) return resp def getlongresp(self): *************** *** 140,146 **** Raise various errors if the response indicates an error.""" resp = self.getresp() if resp[:3] not in LONGRESP: ! raise error_reply, resp list = [] while 1: line = self.getline() --- 191,197 ---- Raise various errors if the response indicates an error.""" resp = self.getresp() if resp[:3] not in LONGRESP: ! raise NNTPReplyError(resp) list = [] while 1: line = self.getline() *************** *** 206,212 **** resp = self.shortcmd('GROUP ' + name) if resp[:3] <> '211': ! raise error_reply, resp words = string.split(resp) count = first = last = 0 n = len(words) --- 257,263 ---- resp = self.shortcmd('GROUP ' + name) if resp[:3] <> '211': ! raise NNTPReplyError(resp) words = string.split(resp) count = first = last = 0 n = len(words) *************** *** 230,236 **** def statparse(self, resp): """Internal: parse the response of a STAT, NEXT or LAST command.""" if resp[:2] <> '22': ! raise error_reply, resp words = string.split(resp) nr = 0 id = '' --- 281,287 ---- def statparse(self, resp): """Internal: parse the response of a STAT, NEXT or LAST command.""" if resp[:2] <> '22': ! raise NNTPReplyError(resp) words = string.split(resp) nr = 0 id = '' *************** *** 349,355 **** elem[6], elem[7])) except IndexError: ! raise error_data,line return resp,xover_lines def xgtitle(self, group): --- 400,406 ---- elem[6], elem[7])) except IndexError: ! raise NNTPDataError(line) return resp,xover_lines def xgtitle(self, group): *************** *** 377,387 **** resp = self.shortcmd("XPATH " + id) if resp[:3] <> '223': ! raise error_reply, resp try: [resp_num, path] = string.split(resp) except ValueError: ! raise error_reply, resp else: return resp, path --- 428,438 ---- resp = self.shortcmd("XPATH " + id) if resp[:3] <> '223': ! raise NNTPReplyError(resp) try: [resp_num, path] = string.split(resp) except ValueError: ! raise NNTPReplyError(resp) else: return resp, path *************** *** 395,408 **** resp = self.shortcmd("DATE") if resp[:3] <> '111': ! raise error_reply, resp elem = string.split(resp) if len(elem) != 2: ! raise error_data, resp date = elem[1][2:8] time = elem[1][-6:] if len(date) != 6 or len(time) != 6: ! raise error_data, resp return resp, date, time --- 446,459 ---- resp = self.shortcmd("DATE") if resp[:3] <> '111': ! raise NNTPReplyError(resp) elem = string.split(resp) if len(elem) != 2: ! raise NNTPDataError(resp) date = elem[1][2:8] time = elem[1][-6:] if len(date) != 6 or len(time) != 6: ! raise NNTPDataError(resp) return resp, date, time *************** *** 415,421 **** resp = self.shortcmd('POST') # Raises error_??? if posting is not allowed if resp[0] <> '3': ! raise error_reply, resp while 1: line = f.readline() if not line: --- 466,472 ---- resp = self.shortcmd('POST') # Raises error_??? if posting is not allowed if resp[0] <> '3': ! raise NNTPReplyError(resp) while 1: line = f.readline() if not line: *************** *** 439,445 **** resp = self.shortcmd('IHAVE ' + id) # Raises error_??? if the server already has it if resp[0] <> '3': ! raise error_reply, resp while 1: line = f.readline() if not line: --- 490,496 ---- resp = self.shortcmd('IHAVE ' + id) # Raises error_??? if the server already has it if resp[0] <> '3': ! raise NNTPReplyError(resp) while 1: line = f.readline() if not line: *************** *** 465,471 **** def _test(): """Minimal test function.""" ! s = NNTP('news') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last --- 516,522 ---- def _test(): """Minimal test function.""" ! s = NNTP('news', readermode='reader') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 8 23:22:50 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 8 Feb 2000 18:22:50 -0500 (EST) Subject: [Mailman-Developers] Reply goes to list error References: Message-ID: <14496.42314.426477.518945@anthem.cnri.reston.va.us> >>>>> "HM" == Harald Meland writes: HM> Ooops, sorry -- my version did, but I hadn't checked that in. HM> Checked in now. Actually, there's a bug in your version. When using the setitem interface (e.g. msg['reply-to'] = '...'), you don't want to set the value to include the key or the newline. I'm about to check in a fix. -Barry From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 8 23:27:48 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 8 Feb 2000 18:27:48 -0500 (EST) Subject: [Mailman-Developers] When will mailman 1.2 be released? References: <20000208222814.A3736@stopcontact.palga.uucp> Message-ID: <14496.42612.673774.185482@anthem.cnri.reston.va.us> >>>>> "GH" == Gerrit Holl writes: GH> when do you think mailman 1.2 will be released? I wish I could say. Frankly, I wish I could say whether the next release will be 1.2 or 2.0. I think if Harald's user database changes are significant enough, I might make it 2.0. GH> How stable is the current CVS tree, and how heavily is it GH> tested? Stable enough to run all the lists at Python.Org. If something's broken, I find out about it pretty dang quickly. No doubt there are minor bugs still lurking though. GH> In what order of magnitude is the release frequency? I'm not sure what you're asking, but I've been thinking that I should start getting beta releases out RSN. -Barry From Gerrit Wed Feb 9 07:13:23 2000 From: Gerrit (Gerrit Holl) Date: Wed, 9 Feb 2000 08:13:23 +0100 Subject: [Mailman-Developers] When will mailman 1.2 be released? In-Reply-To: <14496.42612.673774.185482@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Tue, Feb 08, 2000 at 06:27:48PM -0500 References: <20000208222814.A3736@stopcontact.palga.uucp> <14496.42612.673774.185482@anthem.cnri.reston.va.us> Message-ID: <20000209081323.A957@stopcontact.palga.uucp> Barry A. Warsaw wrote on 950030868: > > >>>>> "GH" == Gerrit Holl writes: > > GH> when do you think mailman 1.2 will be released? > > I wish I could say. Frankly, I wish I could say whether the next > release will be 1.2 or 2.0. I think if Harald's user database changes > are significant enough, I might make it 2.0. > > GH> How stable is the current CVS tree, and how heavily is it > GH> tested? > > Stable enough to run all the lists at Python.Org. If something's > broken, I find out about it pretty dang quickly. No doubt there are > minor bugs still lurking though. > > GH> In what order of magnitude is the release frequency? > > I'm not sure what you're asking, but I've been thinking that I should > start getting beta releases out RSN. Maybe I looked up the wrong words in my dictionairy :( I mean, is Mailman likely to be released every month like many other projects, or far less often, like Python? The last version was released in November last year, so it's not really "Release early, release often". What does 'RSN' mean? regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From brian@garage.co.jp Wed Feb 9 08:00:01 2000 From: brian@garage.co.jp (Brian Takashi Hooper) Date: Wed, 09 Feb 2000 17:00:01 +0900 Subject: [Mailman-Developers] I18N concerns for Japanese Message-ID: <38A11E81D4.8E5ABRIAN@smtp.garage.co.jp> Hi there - I just checked out the latest version of Mailman from CVS and am taking a look at it with an eye to making it work better with our environment here, which is Japanese. I think that Mailman would be very popular in Japan too with a localized interface and a few tweaks to ensure that mail goes out in the proper format. I haven't been on Mailman-dev for a long time yet, so forgive me if I'm rehashing an old topic (I checked through the archives as best I could, so I apologize in advance for redundancy): 1. Headers: To, Subject: The To: and Subject: lines in Japanese messages often contain Japanese; the proper encoding for this seems to be base-64 encoded ISO-2022 (JIS). Mail received from list members is of course no problem, but mail which the system sends out, if it contains Japanese, needs to be able to do this conversion. Looking at an example implementation (Mew, an emacs module which does Japanese mail), it looks like only those parts of the header which contain JIS escape sequences are encoded, that is, ASCII parts of the To: and Subject: lines are not encoded. I think this makes sense, since encoding the entirety of To: would obscure the recipient address, which might not be good for some mailers. I assume there are other standards out there for various languages to insert non-ascii characters in the To: and Subject: lines, probably which do similar tricks; are there any folks out there who have similar requirements or experiences? I think that my difficulty would be solved if it was possible to specify a filter which should be applied to outgoing message headers; this could by default be None, or a filter which just returns its string argument unchanged. This also should be linked to the specification of charset for the message. Content-type, Mime-Version: Typically, Japanese mail sets the Content-type and Mime headers like so: Content-type: text/plain; charset=ISO-2022-JP Mime-Version: 1.0 Looking at the Mailman source, it looked like charset=us-ascii is hardcoded for a few types of messages (ToDigest, bounce); as far as I can tell it's not set for outgoing admin messages (someone please catch me if I'm wrong on this). The message body of Japanese messages normally sent in ISO-2022 without bin-hex translation, but I don't think this has any implications vis-a-vis Mailman's current behavior. So, I guess what I'm suggesting is adding the ability to add custom filters to do special header processing based on locale... Opinions, additions, flames anyone? 2. Templates We have already done some localization of templates here, and will continue to do so until we've finished... in this case, maybe it would be nice to offer downloads of drop-in-replacement template folders for different languages somewhere. I volunteer our templates when they are done. Obviously there are other inline text messages which also require some mechanism for localization; maybe gettext? Are there any other localization issues which people have come across for other locales? --Brian Hooper Tokyo, Japan From thomas@xs4all.net Wed Feb 9 09:41:47 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Feb 2000 10:41:47 +0100 Subject: [Mailman-Developers] Archiver.py bug ? Message-ID: <20000209104147.A477@xs4all.nl> --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii I ran across this in my errors log: Traceback (innermost last): File "/usr/local/mailman/Mailman/Archiver/Archiver.py", line 212, in ArchiveMail self.__archive_to_mbox(msg) File "/usr/local/mailman/Mailman/Archiver/Archiver.py", line 168, in __archive_to_mbox post.SetHeader('Date', time.ctime(time.time())) post is a Mailman.Message.Message instance, which is a subclass of rfc822.Message... But neither of those objects define a SetHeader method, and the two calls to SetHeader in Archiver.py are the only calls in Mailman to that particular method. An old leftover, perhaps ? The attached patch should fix it. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mailman.diffd" Index: Mailman/Archiver/Archiver.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Archiver/Archiver.py,v retrieving revision 1.18 diff -u -r1.18 Archiver.py --- Archiver.py 1999/12/25 14:37:38 1.18 +++ Archiver.py 2000/02/09 09:25:12 @@ -163,7 +163,7 @@ if self.clobber_date: import time olddate = post.getheader('date') - post.SetHeader('Date', time.ctime(time.time())) + post['Date'] = time.ctime(time.time()) try: afn = self.ArchiveFileName() mbox = self.__archive_file(afn) @@ -176,7 +176,7 @@ reraise() if self.clobber_date: # Resurrect original date setting. - post.SetHeader('Date', olddate) + post['Date'] = olddate def ExternalArchive(self, ar, txt): d = SafeDict({'listname': self.internal_name()}) --TB36FDmn/VVEgNH/-- From gorgo@sztaki.hu Wed Feb 9 10:36:41 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 09 Feb 2000 11:36:41 +0100 (MET) Subject: [Mailman-Developers] Bug#57596: mailman: Here we go again... post: nntplib.error_perm : 500 "POST" not implemented; try "help". (fwd) Message-ID: Another day, another bugreport :/ Lazarus, can you tell me if this is after you applied the last patch suggested by Thomas, for gate_news ? Package: mailman Version: 1.1-2 Severity: important Feb 09 03:47:55 2000 post: Traceback (innermost last): post: File "/var/lib/mailman/scripts/post", line 73, in ? post: mlist.Post(msg, approved=fromusenet) post: File "/usr/lib/mailman/Mailman/MailList.py", line 1331, in Post post: self.SendMailToNewsGroup(msg) post: File "/usr/lib/mailman/Mailman/GatewayManager.py", line 204, in SendMailToNewsGroup post: con.post(msg) post: File "/usr/lib/python1.5/nntplib.py", line 417, in post post: resp = self.shortcmd('POST') post: File "/usr/lib/python1.5/nntplib.py", line 158, in shortcmd post: return self.getresp() post: File "/usr/lib/python1.5/nntplib.py", line 134, in getresp post: raise error_perm, resp You get the idea. Gatewaying seems completely broken. From gorgo@sztaki.hu Wed Feb 9 10:37:45 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 09 Feb 2000 11:37:45 +0100 (MET) Subject: [Mailman-Developers] Bug#57597: mailman: Different gate_news problems this time. (fwd) Message-ID: And a locking problem... is this serious ? ---------- Forwarded message ---------- Date: Wed, 09 Feb 2000 03:57:40 +0000 (UTC) From: Lazarus Long To: submit@bugs.debian.org Subject: Bug#57597: mailman: Different gate_news problems this time. Resent-Date: Wed, 09 Feb 2000 04:03:05 +0000 (GMT) Resent-From: Lazarus Long Resent-To: debian-bugs-dist@lists.debian.org Resent-cc: Gergely Madarasz Package: mailman Version: 1.1-2 Severity: important Subject: Cron [ -x /usr/bin/python -a -f /usr/lib/mailman/cron/gate_news ] && /usr/bin/python /usr/lib/mailman/cron/gate_news X-Cron-Env: X-Cron-Env: X-Cron-Env: X-Cron-Env: Date: Wed, 9 Feb 2000 03:50:15 +0000 (UTC) Exception Mailman.LockFile.NotLockedError: in ignored From Harald.Meland@usit.uio.no Wed Feb 9 10:58:58 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 09 Feb 2000 11:58:58 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval In-Reply-To: Thomas Wouters's message of "Tue, 8 Feb 2000 02:05:27 +0100" References: <20000208020526.N19265@xs4all.nl> Message-ID: [Thomas Wouters] > Another minor patch adding what I need to mailman ;) I also noticed > this on the jitterbug list, so I guess more people had wished or at > least wondered for it ;P Yup, that's correct. > I'm posting it mostly to see if it generates comments, and for me to > find a good excuse to sleep. :-) I've had a go at integrating your patch into my locally hacked Mailman (BTW, could anyone provide me with a clue as to how I can tell patch(1) to understand a multiple-directory CVS diff?) -- and noticed that you seemingly have forgotten to add a DEFAULT_UNSUBSCRIBE_POLICY to Defaults.py.in, and that the comments in some places bore clear signs of a bit hasty cut-and-paste work :) > is there a good word for 'unsubscription request' ? I'm probably not the best person to answer questions on English wording, but to me "unsubscription request" is OK. However, saying Your authorization is required for a mailing list unsubscription request approval: (quickly, three times in a row :) is quite a mouthful, so one might consider rephrasing that sentence somewhat ;) > Adding this has been mostly an exercise to learn a bit about the Mailman > insides, and to see how easily it can be done; fairly easily ;) Comments are > welcome, including "d'oh, dont write that, i already wrote it !" My immediate comment would be: How fast can we get this guy to sign the appropriate papers? :-D I'll try to get your stuff tested a bit here, too, and hopefully we can get around to committing it later. Thanks, -- Harald From Harald.Meland@usit.uio.no Wed Feb 9 11:08:56 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 09 Feb 2000 12:08:56 +0100 Subject: [Mailman-Developers] Reply goes to list error In-Reply-To: "Barry A. Warsaw"'s message of "Tue, 8 Feb 2000 18:22:50 -0500 (EST)" References: <14496.42314.426477.518945@anthem.cnri.reston.va.us> Message-ID: [Barry A. Warsaw] > >>>>> "HM" == Harald Meland writes: > > HM> Ooops, sorry -- my version did, but I hadn't checked that in. > HM> Checked in now. > > Actually, there's a bug in your version. When using the setitem > interface (e.g. msg['reply-to'] = '...'), you don't want to set the > value to include the key or the newline. I'm about to check in a fix. That's what I get for changing from the msg.headers.append() to the msg[header] style of doing these things just prior to checking it all in (because I realized I found the latter style to be "cleaner" in this context). I guess I'm now cured for the check-in-untested-code disease for a few more months... Sorry, -- Harald From rullfig@midway.uchicago.edu Wed Feb 9 16:23:36 2000 From: rullfig@midway.uchicago.edu (Roberto Ullfig) Date: Wed, 09 Feb 2000 10:23:36 -0600 Subject: [Mailman-Developers] new feature request for anon lists Message-ID: <38A19488.50F7BBA3@midway.uchicago.edu> Here are the relevant lines from MailList.py # if self.anonymous_list: # del msg['reply-to'] # del msg['sender'] # msg.SetHeader('From', self.GetAdminEmail()) # msg.SetHeader('Reply-to', self.GetListEmail()) if self.anonymous_list: del msg['sender'] msg.SetHeader('From', self.GetAdminEmail()) if self.reply_goes_to_list: del msg['reply-to'] msg.SetHeader('Reply-to', self.GetListEmail()) What I've done is use the reply_goes_to_list flag to control whether or not the Reply-To is munged instead of munging indiscriminately. It make a lot of sense to me that this is how it should work. Anyone see any problems with this? -- Roberto Ullfig : rullfig@uchicago.edu Systems Administrator Networking Services and Information Technologies University of Chicago From Nigel.Metheringham@vdata.co.uk Wed Feb 9 16:51:19 2000 From: Nigel.Metheringham@vdata.co.uk (Nigel Metheringham) Date: Wed, 09 Feb 2000 16:51:19 +0000 Subject: [Mailman-Developers] nightly_gzip - more than a little broken... Message-ID: I wondered why the nightly_gzip program was not working correctly on some of my lists - one (a high traffic list) has weekly based archiving, the other (an announce list) has quarterly based archiving. A very quick look at the code shows:- for f in allfiles: try: time.strptime(f, '%Y-%B.txt') except ValueError: continue that strptime seems to be ensuring that the archive mbox files have a year and month in their names before a .txt - and this only applies to monthly archives in practice. Would there be any problems with just changing this to be a regexp match for '\.txt$' instead? If you really want to check its an archive then look for a directory x with a file x.txt . Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From thomas@xs4all.net Wed Feb 9 17:31:07 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Feb 2000 18:31:07 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval In-Reply-To: ; from Harald.Meland@usit.uio.no on Wed, Feb 09, 2000 at 11:58:58AM +0100 References: <20000208020526.N19265@xs4all.nl> Message-ID: <20000209183107.B477@xs4all.nl> On Wed, Feb 09, 2000 at 11:58:58AM +0100, Harald Meland wrote: > I've had a go at integrating your patch into my locally hacked Mailman > (BTW, could anyone provide me with a clue as to how I can tell > patch(1) to understand a multiple-directory CVS diff?) -- and noticed > that you seemingly have forgotten to add a DEFAULT_UNSUBSCRIBE_POLICY > to Defaults.py.in, and that the comments in some places bore clear > signs of a bit hasty cut-and-paste work :) DISCLAIMER: It was late at night, i wanted to move a few of our lists to mailman the following day, and we have some peculiar requirements ;) We have basically two lists, an 'admin' list and a 'garbage' list. The 'admin' list is the entire company, and we dont want people to unsubscribe themselves without aproval. The garbage list is for discussions, jokes, anecdotes, etc. We want to allow people to subscribe and unsubscribe at will, IF they are on the 'admin' list. We didn't have this in Majordomo, so it's no big deal that they we have it now. It's on my wishlist tho ;) In the mean time, we set subscribe policy on the garbage list to 'approve'. We also want both these lists to be 'closed' (postings by members only) BUT a lot of our employees have multiple email adresses, and we want to allow them to post using all of them. So, in majordomo, we had a seperate file, included in _both_ lists as 'allow posts from this list of addresses', so we had to include only a single file. In Mailman, we'd have to paste the entire file (which is currently 495 entries long, and getting longer and longer ;) into the webform, and update them both. So instead I hacked the same behaviour into mailman -- include a seperate file. Not intended for general use, which is why I didn't post it here, and it has _no_ security, but it works ;) I'm thinking of changing it to make it allow postings from users on different lists as well, but I didn't want to spend my time figuring out if I can safely instantiate one or more MailList instances inside Hold.py's process(). Another hack I added this morning, right before I switched the first list (the 'garbage' list) to Mailman, is to allow people to read the (private) archives for the list _only_ when they connect from a specific network. This is a _really_ dirty hack and i'm ashamed now that I look back at it, but hey, it was 7:45am after a party and 5 hours of sleep. It's also embarassing because i realize now that there is a much better way. (_public_ archives, with a .htaccess in them. This is guaranteed to work, right ?) A colleague of mine found a bug, though (the hard way ;). MailList.MailList.DeleteMember() tries to call 'ApprovedDeleteMember()' instead of 'self.ApprovedDeleteMember()'. No other bugs yet, but noone tried to unsubscribe from a closed list yet, either. (other than me.) regarding patch: you need to do 'patch -p0'. See the manpage for 'patch'; there is a not-so-subtle difference between -p0 (or -p without a number, if it's the last option) and no -p option. And yes, the entire patch is mostly cut&paste work. I didn't think so much about wording and comments as I did about getting it working quickly. This patch is in no way what I would propose to be checked in ;P > Your authorization is required for a mailing list unsubscription > request approval: > > (quickly, three times in a row :) is quite a mouthful, so one might > consider rephrasing that sentence somewhat ;) Yeah, I agree. #TBD ;) > My immediate comment would be: > How fast can we get this guy to sign the appropriate papers? Me ? Are they the grep-able or the dead-tree kind of papers ? do I need to fax, or do you need to fax me something ? I have access to such things here at work, thankfully ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Wed Feb 9 17:36:49 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Feb 2000 18:36:49 +0100 Subject: [Mailman-Developers] Bug#57597: mailman: Different gate_news problems this time. (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Wed, Feb 09, 2000 at 11:37:45AM +0100 References: Message-ID: <20000209183649.C477@xs4all.nl> On Wed, Feb 09, 2000 at 11:37:45AM +0100, Gergely Madarasz wrote: > And a locking problem... is this serious ? > Exception Mailman.LockFile.NotLockedError: instance at 812bb50> in 8135e88> ignored Not really important, but peculiar. Did this happen with the default mailman 1.1 locking, or did you apply any patches ? In the default locking for mailman 1.1 this shouldn't happen, except perhaps if someone else stole the link right between the check in LockFile.__del__() and the check in LockFile.unlock(). In other words, a highly unlikely race condition. [BTW, kept CC list the same because i can't figure out who to include or not] -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From bwarsaw@cnri.reston.va.us Wed Feb 9 17:42:15 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 9 Feb 2000 12:42:15 -0500 (EST) Subject: [Mailman-Developers] When will mailman 1.2 be released? References: <20000208222814.A3736@stopcontact.palga.uucp> <14496.42612.673774.185482@anthem.cnri.reston.va.us> <20000209081323.A957@stopcontact.palga.uucp> Message-ID: <14497.42743.130965.646975@anthem.cnri.reston.va.us> >>>>> "GH" == Gerrit Holl writes: GH> I mean, is Mailman likely to be released every month like many GH> other projects, or far less often, like Python? The last GH> version was released in November last year, so it's not really GH> "Release early, release often". Unsure. I'm quite busy with real work, real life, and managing two big open source projects. I'll release when I can -- hopefully more often than Python, but probably not every month. You're always free to watch CVS, and I try very hard not to destabilize the CVS snapshot. GH> What does 'RSN' mean? Real Soon Now. -Barry From Harald.Meland@usit.uio.no Wed Feb 9 18:13:03 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 09 Feb 2000 19:13:03 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval In-Reply-To: Thomas Wouters's message of "Wed, 9 Feb 2000 18:31:07 +0100" References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> Message-ID: [Thomas Wouters] > So instead I hacked the same behaviour into mailman -- include a > seperate file. Not intended for general use, which is why I didn't > post it here, and it has _no_ security, but it works ;) Random thought: Would it be a good idea to add type thingies for some of the text boxes which have to take rather large amounts of text, or would it just clutter things? > A colleague of mine found a bug, though (the hard way ;). > MailList.MailList.DeleteMember() tries to call > 'ApprovedDeleteMember()' instead of > 'self.ApprovedDeleteMember()'. Thanks for letting me know, I've fixed this in my local copy too, now. > regarding patch: you need to do 'patch -p0'. Hmmmm, I already knew about -p0, and had tried that, but I get this (when standing in the toplevel Mailman directory, i.e. the one containing INSTALL etc.): octarine$ patch -p0 --dry-run < ~/mailman-unsub-approval.diff can't find file to patch at input line 9 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -------------------------- |? templates/unsubauth.txt |Index: Mailman/Bouncer.py |=================================================================== |RCS file: /projects/cvsroot/mailman/Mailman/Bouncer.py,v |retrieving revision 1.37 |diff -u -r1.37 Bouncer.py |--- Bouncer.py 1999/12/16 17:11:04 1.37 |+++ Bouncer.py 2000/02/08 00:57:47 -------------------------- File to patch: ^C etc. Thus my suspicion that patch isn't recognizing the Index: stuff, which I believe was put there by CVS. This is with GNU patch 2.5. > > My immediate comment would be: > > How fast can we get this guy to sign the appropriate papers? > > Me ? Are they the grep-able or the dead-tree kind of papers ? Well, that depends on whether anyone has finished work on that wetware grep yet ;) I'm talking about the stuff explained in -- I suspect that both you and a few others are (at least) bordering on "significant changes" to Mailman if most of the (very nice) patches you have submitted get checked in. > do I need to fax, or do you need to fax me something ? I have access > to such things here at work, thankfully ;) I don't have access to any GNU machines, so I won't be able to help you with this stuff. Barry is the official maintainer, so he should be able to sort this out (but please do give him some time to respond, in case he is heavily swamped at the moment). For what it's worth: When I did this, I received the appropriate templates[1] by email, printed them out, filled them in by hand, and then snail-mailed them (the originals) back across the Atlantic. [1] two, IIRC -- one (disclaim.future) to say my employer disclaims any ownership on the work I do on Mailman (signed by my boss), and one (assign.future) to say that I transfer copyright of my future work (either for all future, or for X years, IIRC) work on Mailman to the FSF (signed by me). -- Harald From claw@kanga.nu Wed Feb 9 18:16:03 2000 From: claw@kanga.nu (J C Lawrence) Date: Wed, 09 Feb 2000 10:16:03 -0800 Subject: [Mailman-Developers] When will mailman 1.2 be released? In-Reply-To: Message from Gerrit Holl of "Wed, 09 Feb 2000 08:13:23 +0100." <20000209081323.A957@stopcontact.palga.uucp> References: <20000208222814.A3736@stopcontact.palga.uucp> <14496.42612.673774.185482@anthem.cnri.reston.va.us> <20000209081323.A957@stopcontact.palga.uucp> Message-ID: <4598.950120163@kanga.nu> > I mean, is Mailman likely to be released every month like many > other projects, or far less often, like Python? The last version > was released in November last year, so it's not really "Release > early, release often". Hardly -- as all the sources are publicly available via CVS, every change is "released". In that context the "release process" really is just a labelling game, and nothing to really do with the function or development of the product itself. -- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From Gerrit Wed Feb 9 21:00:10 2000 From: Gerrit (Gerrit Holl) Date: Wed, 9 Feb 2000 22:00:10 +0100 Subject: [Mailman-Developers] I want to write docs! Message-ID: <20000209220010.A2699@stopcontact.palga.uucp> Hello, [see below for summary] I've downloaded Mailman about a week ago, and I'm very excited. I read all papers. I read many of the source code. I read all available documentation. And now, I want to write documentation. The TODO list says: * Documenation - a detailed feature list - a user's guide - a site-admin's guide - a list-admin's guide - more in-place documenation - A developer's guide with architecture and API info, etc. I want to add one: - more docstrings in the code I want to write all kind of documentation. However, there is no good system for it. I know nothing about setting up a system like the Python documentation system, or an XML DTD, or... I do know the DOCBOOK DTD, but I'm not able to get it to work. I don't know how to tell sgml2html where to search for all the files. I seem to to something wrong. I can't get it to work. But I certainly want to write documenation. Plain HTML, however, would be an awful thing, since we would need to convert it to the other system after we built such a system. Another thing: if I add docstrings to the code, what should I do with the changed code, with the patches? I can't find out where to send patches to be incorperated in mailman. Neither mailman-cabal, nor this list seems to be the right place :(. -----BEGIN SUMMARY BLOCK----- I want to write documentation, but I don't know how to do it in practice, e.g., in what format and where to send the docs. -----END SUMMARY BLOCK----- regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Feb 9 22:00:08 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 9 Feb 2000 17:00:08 -0500 (EST) Subject: [Mailman-Developers] I want to write docs! References: <20000209220010.A2699@stopcontact.palga.uucp> Message-ID: <14497.58216.799619.232133@anthem.cnri.reston.va.us> >>>>> "GH" == Gerrit Holl writes: GH> I've downloaded Mailman about a week ago, and I'm very GH> excited. Cool! GH> And now, I want to write documentation. There have been a few other people who have done some documenting. Unfortunately I haven't kept up with that status, so perhaps others can chime in. Because Mailman is GNU, texinfo is the standard documentation format. I'll point you at doc-sig@python.org (a Mailman list, 'natch) which is making all the decisions about Python documentation. It's not a list that I'm on. If whatever doc-sig chooses can be used to generate texinfo, then it would be fine to use that, but we need to distribute texinfo with Mailman. There's plenty of documentation about texinfo on www.gnu.org and it's pretty easy to write. You will, of course, need to copyright assign any documentation to the FSF, and I can send you the appropriate forms when you're ready. GH> Another thing: if I add docstrings to the code, what should I GH> do with the changed code, with the patches? I can't find out GH> where to send patches to be incorperated in mailman. Neither GH> mailman-cabal, nor this list seems to be the right place :(. Currently, mailman-developers is the right place to send your patches. If others think that'll clutter up this list too much, we can 1) create a mailman-patches mailing list; or 2) hasten the move to SourceForge and piggyback off of their tools (which I admittedly haven't investigated too deeply yet). I'm leary of creating yet-another-mailing list, so I'd be inclined to prefer using mailman-developers. Note that, you'll need different FSF copyright assignment forms for docstring patches. Again, I can send those to you when needed. -Barry From lazarus@overdue.dhis.net Wed Feb 9 23:07:51 2000 From: lazarus@overdue.dhis.net (Lazarus Long) Date: Wed, 9 Feb 2000 23:07:51 +0000 Subject: [Mailman-Developers] Re: Bug#57596: mailman: Here we go again... post: nntplib.error_perm : 500 "POST" not implemented; try "help". (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Wed, Feb 09, 2000 at 11:36:41AM +0100 References: Message-ID: <20000209230751.A9922@overdue.dhis.net> --H1spWtNR+x+ondvy Content-Type: multipart/mixed; boundary="y0ulUmNC+osPPQO6" --y0ulUmNC+osPPQO6 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable On Wed, Feb 09, 2000 at 11:36:41AM +0100, Gergely Madarasz wrote: >=20 > Another day, another bugreport :/ Sigh. I've gotten mail regarding this containing obscenities from another developer. I'd love to get this fixed rather than filing new bug reports. I'm sure we all would. > Lazarus, can you tell me if this is after you applied the last patch > suggested by Thomas, for gate_news ? The only patch i aplied follows: --- cron/gate_news.orig Mon Feb 7 22:47:52 2000 +++ cron/gate_news Mon Feb 7 22:53:23 2000 @@ -76,7 +76,16 @@ # update the gate_watermarks file. we'll actually do the gating i= n a # child process conn =3D nntplib.NNTP(mlist.nntp_host) - r,c,first,last,n =3D conn.group(mlist.linked_newsgroup) + try: + r,c,first,last,n =3D conn.group(mlist.linked_newsgroup) + except nntplib.error_perm: + # We got an error 500, probably meaning that the 'group' comma= nd + # is not implemented. A likely scenario causing this is when + # running the Mailman gateway on the same machine as the news + # server; the server defaults to news feed mode. We need to s= et + # the connection to reader mode first. + conn.shortcmd('mode reader') + r,c,first,last,n =3D conn.group(mlist.linked_newsgroup) first =3D int(first) last =3D int(last) wm =3D watermarks.get(name, 0) --=20 Please encrypt all mail whenever possible. The following Public Keys for Lazarus Long are available upon request: Type Bits/KeyID Fingerprint (GnuPG (GPG) is preferred.) GPG/ELG: 2048g/92F6493B 2C55 E967 278B 4E8B D25B F5F3 352B 9B0E 32C3 3BA4 GPG/DSA: 1024D/32C33BA4 (none for DSA keys) --y0ulUmNC+osPPQO6 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="gate_news.diff" --- cron/gate_news.orig Mon Feb 7 22:47:52 2000 +++ cron/gate_news Mon Feb 7 22:53:23 2000 @@ -76,7 +76,16 @@ # update the gate_watermarks file. we'll actually do the gating in a # child process conn = nntplib.NNTP(mlist.nntp_host) - r,c,first,last,n = conn.group(mlist.linked_newsgroup) + try: + r,c,first,last,n = conn.group(mlist.linked_newsgroup) + except nntplib.error_perm: + # We got an error 500, probably meaning that the 'group' command + # is not implemented. A likely scenario causing this is when + # running the Mailman gateway on the same machine as the news + # server; the server defaults to news feed mode. We need to set + # the connection to reader mode first. + conn.shortcmd('mode reader') + r,c,first,last,n = conn.group(mlist.linked_newsgroup) first = int(first) last = int(last) wm = watermarks.get(name, 0) --y0ulUmNC+osPPQO6-- --H1spWtNR+x+ondvy Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.1 (GNU/Linux) Comment: Now ready for primetime! http://www.gnupg.org iD8DBQE4ofNHNSubDjLDO6QRArOOAJ4safl/w/vzMIc3RD0aGqYCEybjlwCfTnTB G7meKaKRw4OCaMyYrA6uxxw= =SkQj -----END PGP SIGNATURE----- --H1spWtNR+x+ondvy-- From Gerrit Thu Feb 10 06:32:05 2000 From: Gerrit (Gerrit Holl) Date: Thu, 10 Feb 2000 07:32:05 +0100 Subject: [Mailman-Developers] I want to write docs! In-Reply-To: <14497.58216.799619.232133@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Wed, Feb 09, 2000 at 05:00:08PM -0500 References: <20000209220010.A2699@stopcontact.palga.uucp> <14497.58216.799619.232133@anthem.cnri.reston.va.us> Message-ID: <20000210073205.A617@stopcontact.palga.uucp> Barry A. Warsaw wrote on 950112008: > [texinfo] > You will, of course, need to copyright assign any documentation to the > FSF, and I can send you the appropriate forms when you're ready. That's totally fine. The copyright is absolutly not a problem. I totally trust the freedom of the FSF licenses :) > GH> Another thing: if I add docstrings to the code, what should I > GH> do with the changed code, with the patches? I can't find out > GH> where to send patches to be incorperated in mailman. Neither > GH> mailman-cabal, nor this list seems to be the right place :(. > > Currently, mailman-developers is the right place to send your patches. OK! > Note that, you'll need different FSF copyright assignment forms > for docstring patches. Again, I can send those to you when needed. And again, I'll totally leave that tou you and the FSF. I'll have a look on Texinfo this afternoon (I've got to get to school first). regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From Gergely Madarasz Thu Feb 10 13:01:44 2000 From: Gergely Madarasz (Gergely Madarasz) Date: Thu, 10 Feb 2000 14:01:44 +0100 (MET) Subject: [Mailman-Developers] fd leakage with run_queue Message-ID: Hello, It is quite often that the system load on a machine goes up so sendmail rejects connections.... so when run_queue runs, there are quite a few messages rejected in one process, meaning lots of logs... and then I get messages like this: Logging error: Traceback (innermost last): File "/usr/lib/mailman/Mailman/Logging/Logger.py", line 55, in __get_f IOError: [Errno 24] Too many open files: '/var/lib/mailman/logs/error' Original log message: [Errno 24] Too many open files: '/var/lib/mailman/logs/error' Feb 10 13:44:59 2000 processQueue: Processing of queue file /var/lib/mailman/data/mm_q.28769.1 failed: Feb 10 13:44:59 2000 processQueue: exceptions.IOError / [Errno 24] Too many open files: '/var/lib/mailman/data/mm_q.28769.1' Logging error: Traceback (innermost last): File "/usr/lib/mailman/Mailman/Logging/Logger.py", line 55, in __get_f IOError: [Errno 24] Too many open files: '/var/lib/mailman/logs/error' Original log message: [Errno 24] Too many open files: '/var/lib/mailman/logs/error' Feb 10 13:45:00 2000 processQueue: Processing of queue file /var/lib/mailman/data/mm_q.29101.1 failed: Feb 10 13:45:00 2000 processQueue: exceptions.IOError / [Errno 24] Too many open files: '/var/lib/mailman/data/mm_q.29101.1' It seems that for each log entry run_queue opens the logfile, and then doesn't close it. Is there a quick fix for this somewhere ? I'm running mailman 1.1 -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 10 13:16:30 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 10 Feb 2000 08:16:30 -0500 (EST) Subject: [Mailman-Developers] fd leakage with run_queue References: Message-ID: <14498.47662.909335.920840@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> It seems that for each log entry run_queue opens the logfile, GM> and then doesn't close it. Is there a quick fix for this GM> somewhere ? I'm running mailman 1.1 I seem to remember someone posting a patch for this, so check the archives. Otherwise, don't sweat it, run_queue is gone, gone, gone in the CVS version. -Barry From thomas@xs4all.net Thu Feb 10 16:31:14 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 10 Feb 2000 17:31:14 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval In-Reply-To: ; from Harald.Meland@usit.uio.no on Wed, Feb 09, 2000 at 07:13:03PM +0100 References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> Message-ID: <20000210173114.F477@xs4all.nl> On Wed, Feb 09, 2000 at 07:13:03PM +0100, Harald Meland wrote: > > So instead I hacked the same behaviour into mailman -- include a > > seperate file. Not intended for general use, which is why I didn't > > post it here, and it has _no_ security, but it works ;) > > Random thought: Would it be a good idea to add > > type thingies for some of the text boxes which have to take rather > large amounts of text, or would it just clutter things? I'm not sure. It might certainly be a good idea, even though in my case it wouldn't help (the problem was not the initial insertion, but rather keeping multiple lists in sync.) Clutterage is a problem in general, i think. Perhaps, if we want to give every textbox an upload field, a simple button next to the input field would suffice -- a button that would pop up a new window. (It might be even worth it to take a look at a frames interface, possibly ?) > > regarding patch: you need to do 'patch -p0'. > Hmmmm, I already knew about -p0, and had tried that, but I get this > (when standing in the toplevel Mailman directory, i.e. the one > containing INSTALL etc.): > > octarine$ patch -p0 --dry-run < ~/mailman-unsub-approval.diff > can't find file to patch at input line 9 > Perhaps you used the wrong -p or --strip option? > The text leading up to this was: > -------------------------- > |? templates/unsubauth.txt > |Index: Mailman/Bouncer.py > etc. Thus my suspicion that patch isn't recognizing the Index: stuff, > which I believe was put there by CVS. This is with GNU patch 2.5. Ok, found it. It's a bug in patch, or in patch's manpage: If there is an Index: line in the leading garbage and if either the old and new names are both absent or the POSIXLY_CORRECT environment variable is set, patch takes the name in the Index: line. Apparently, patch doesn't work like this; even if both the old and the new name are absent, it refuses to regard the Index line. I found a patch to patch that added a '--use-index-line' option, but it doesn't seem to be added to patch 2.5 yet. As a workaround, you can set the 'POSIXLY_CORRECT' environment variable. It works, I just tested it with this very same boutique. Err, patch. The only problem with that is that this makes patch behave completely posix-compliant: POSIXLY_CORRECT If set, patch conforms more strictly to the POSIX standard: it takes the first existing file from the list (old, new, index) when intuiting file names from diff headers, it does not remove files that are empty after patching, it does not ask whether to get files from RCS or SCCS, it requires that all options precede the files in the command line, and it does not backup files when there is a mismatch. I'll try to keep this in mind when sending patches, noting added/removed files, and keeping an eye on situations where both 'file.py' and 'foo/bar/file.py' exist. ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From adhir@forumone.com Thu Feb 10 18:14:43 2000 From: adhir@forumone.com (Alok Dhir) Date: Thu, 10 Feb 2000 13:14:43 -0500 Subject: [Mailman-Developers] When will mailman 1.2 be released? Message-ID: Sorry to be getting in this late - but what will Harald's user db changes do? Things I'd love to see: 1. Ability to search for members by email address 2. Ability to store membership info in a Mysql database It would also be great if one could extend the schema of the subscriber database to include additinal fields (name, address, etc), but this is a much lower priority than the other two... > -----Original Message----- > From: mailman-developers-admin@python.org > [mailto:mailman-developers-admin@python.org]On Behalf Of > Barry A. Warsaw > Sent: Tuesday, February 08, 2000 6:28 PM > To: Gerrit > Cc: mailman-developers@python.org > Subject: Re: [Mailman-Developers] When will mailman 1.2 be released? > > > > >>>>> "GH" == Gerrit Holl writes: > > GH> when do you think mailman 1.2 will be released? > > I wish I could say. Frankly, I wish I could say whether the next > release will be 1.2 or 2.0. I think if Harald's user database changes > are significant enough, I might make it 2.0. > > GH> How stable is the current CVS tree, and how heavily is it > GH> tested? > > Stable enough to run all the lists at Python.Org. If something's > broken, I find out about it pretty dang quickly. No doubt there are > minor bugs still lurking though. > > GH> In what order of magnitude is the release frequency? > > I'm not sure what you're asking, but I've been thinking that I should > start getting beta releases out RSN. > > -Barry > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers > From Gerrit Thu Feb 10 17:32:08 2000 From: Gerrit (Gerrit Holl) Date: Thu, 10 Feb 2000 18:32:08 +0100 Subject: [Mailman-Developers] Validating passwords Message-ID: <20000210183208.A7789@stopcontact.palga.uucp> Hello, I have an idea for a feature: The password the user supplies may not be too similar to the email adres. A minimal check could be (pseudo-code): if string.find(string.lower(email_adres), string.lower(password)): raise InvalidPassword("adress may not contain password") this could be an option set by the list owner. What do you think? regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From Gerrit Thu Feb 10 20:03:05 2000 From: Gerrit (Gerrit Holl) Date: Thu, 10 Feb 2000 21:03:05 +0100 Subject: [Mailman-Developers] PATCH: docstrings added Message-ID: <20000210210305.A8240@stopcontact.palga.uucp> --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Hello, I've added docstrings to *some* of the methods in MailList.py. More will probably follow. Can someone please apply this patch to the CVS tree, after checked the English and the content? regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="MailList.py.diff" --- MailList.py Thu Feb 10 20:57:53 2000 +++ /home/gerrit/MailList.py Thu Feb 10 20:54:18 2000 @@ -60,6 +60,19 @@ class MailList(MailCommandHandler, HTMLFormatter, Deliverer, ListAdmin, Archiver, Digester, SecurityManager, Bouncer, GatewayManager): + """MailList([name [,lock]]) -> instance + + This class represents a mailinglist. The constructor takes zero, one + or two arguments: if 'name' is given, the mailinglist with the name + 'name' is loaded. If the mailinglist isn't found, the exception + Errors.MMUnknownListError is raised. If the second argument is true, + the list is locked. This is true by default. If you are going to write + to the list, this should be true. It's a good habit to unlock the list + when you're ready, using the Unlock() method, in case something goes + wrong or your program runs a long time. You don't need to lock if you + are going to read list data only. + """ + def __init__(self, name=None, lock=1): if name and name not in Utils.list_names(): raise Errors.MMUnknownListError, 'list not found: %s' % name @@ -82,15 +95,27 @@ pass def GetMembers(self): - """returns a list of the members. (all lowercase)""" + """GetMembers() -> list + + Returns a list of all subscribed non-digest members, all in lowercase. + """ + return self.members.keys() def GetDigestMembers(self): - """returns a list of digest members. (all lowercase)""" + """GetDigestMembers() -> list + + Returns a list of all subscribed digest members, all in lowercase. + """ + return self.digest_members.keys() def GetDeliveryMembers(self): - """returns a list of the members with username case preserved.""" + """GetDeliveryMembers() -> list + + Returns a list of the non-digest members with username case preserved. + """ + res = [] for k, v in self.members.items(): if type(v) is StringType: @@ -100,7 +125,11 @@ return res def GetDigestDeliveryMembers(self): - """returns a list of the members with username case preserved.""" + """GetDigestDeliveryMembers() -> list + + Returns a list of the digest members with username case preserved. + """ + res = [] for k,v in self.digest_members.items(): if type(v) is StringType: @@ -110,9 +139,11 @@ return res def __AddMember(self, addr, digest): - """adds the appropriate data to the internal members dict. + """__AddMember(addr, digest) -> None - If the username has upercase letters in it, then the value + Adds the appropriate data to the internal members dict. + + If the username has uppercase letters in it, then the value in the members dict is the case preserved address, otherwise, the value is 0. """ @@ -128,18 +159,25 @@ self.members[string.lower(addr)] = addr def GetAdminEmail(self): + """GetAdminEmail() -> string + + Returns the email address (a string) of the list owner. + """ + return '%s-admin@%s' % (self._internal_name, self.host_name) def GetMemberAdminEmail(self, member): - """Usually the member addr, but modified for umbrella lists. + """GetMemberAdminEmail(member) + + Usually the member address, but modified for umbrella lists. Umbrella lists have other mailing lists as members, and so admin stuff like confirmation requests and passwords must not be sent to the member addresses - the sublists - but rather to the administrators of the sublists. This routine picks the right address, considering regular member address to be their own administrative addresses. - """ + if not self.umbrella_list: return member else: @@ -147,8 +185,11 @@ return "%s%s@%s" % (acct, self.umbrella_member_suffix, host) def GetUserSubscribedAddress(self, member): - """Return the member's case preserved address. + """GetUserSubscribedAddress(member) -> string + + Returns the member's case preserved address. """ + member = string.lower(member) cpuser = self.members.get(member) if type(cpuser) == IntType: @@ -163,24 +204,50 @@ return None def GetUserCanonicalAddress(self, member): - """Return the member's address lower cased.""" + """GetUserCanonicalAddress(member) -> string + + Return the member's address lower cased. + """ + cpuser = self.GetUserSubscribedAddress(member) if cpuser is not None: return string.lower(cpuser) return None def GetRequestEmail(self): + """GetRequestEmail(self) -> string + + Returns the address where users can send administrative requests. + """ + return '%s-request@%s' % (self._internal_name, self.host_name) def GetListEmail(self): + """GetListEmail() -> string + + Returns the email address of the mailinglist. + """ + return '%s@%s' % (self._internal_name, self.host_name) def GetListIdentifier(self): - """Return the unique (RFC draft-chandhok-listid-02) identifier.""" + """GetListIdentifier() -> string + + Returns the unique (RFC draft-chandhok-listid-02) identifier. + """ + return ("%s <%s.%s>" % (self.description, self._internal_name, self.host_name)) def GetScriptURL(self, scriptname, relative=0): + """GetScriptURL(scriptname[, relative]) -> string + + Returns the URL of the 'scriptname' page of the mailing list. If + the optional argument 'relative' is true (it's false by default), + the URL is relative to the directory in the PATH_INFO environment + variable. + """ + if relative: prefix = '../' * Utils.GetNestingLevel() if not prefix: @@ -197,6 +264,15 @@ self.internal_name()) def GetOptionsURL(self, addr, obscure=0, relative=0): + """GetOptionsURL(addr[ ,obscure][, relative]]) -> string + + Returns the URL of the options page of the email address 'addr'. + If 'obscure' is set, the email address is obscured using + Utils.ObscureEmail(addr). If 'relative' is set, the URL is + relative to the directory in the PATH_INFO environment + variable. + """ + addr = string.lower(addr) url = self.GetScriptURL('options', relative) if obscure: @@ -208,10 +284,22 @@ GetAbsoluteScriptURL = GetScriptURL def GetRelativeScriptURL(self, scriptname): + """GetRelativeScriptURL(scriptname) -> string + + Returns the URL of the 'scriptname' script, relative to the + directory in the PATH_INFO environment variable. + """ + return self.GetScriptURL(scriptname, relative=1) def GetUserOption(self, user, option): - """Return user's setting for option, defaulting to 0 if no settings.""" + """GetUserOption(user, option) -> boolean + + Returns wheter 'option' is set for the options of 'user'. If + 'user' is not a member of the mailing list, 0 is returned. + Note that 'user' is not a string, but an integer from the + mm_cfg module. + """ user = self.GetUserCanonicalAddress(user) if option == mm_cfg.Digests: return self.digest_members.has_key(user) @@ -220,6 +308,13 @@ return not not self.user_options[user] & option def SetUserOption(self, user, option, value, save_list=1): + """SetUserOption(user, option, value[, save_list]) -> None + + Sets the 'option' option of the member 'user' to 'value'. 'value' + is an integer from the mm_cfg module. Is 'save_list' is true (it is + by default), the list configuration is saved afterwards. + """ + user = self.GetUserCanonicalAddress(user) if not self.user_options.has_key(user): self.user_options[user] = 0 @@ -255,12 +350,12 @@ # self.passwords is, of course the password in plain text. def FindUser(self, email): - """Return the lowercased version of the subscribed email address. + """FindUser(email) -> string or None - If email is not subscribed, either as a regular member or digest - member, None is returned. If they are subscribed, the return value is - guaranteed to be lowercased. + If 'email' is a member of this list, the lowercased email address + is returned. Otherwise, None is returned. """ + # shortcut lcuser = self.GetUserCanonicalAddress(email) if lcuser is not None: @@ -274,7 +369,11 @@ return string.lower(matches[0]) def InitTempVars(self, name, lock): - """Set transient variables of this and inherited classes.""" + """InitTempVars(name, lock) -> None + + Set transient variables of this and inherited classes. + """ + self.__createlock_p = lock self.__lock = LockFile.LockFile( os.path.join(mm_cfg.LOCK_DIR, name or '') + '.lock', @@ -291,7 +390,13 @@ ListAdmin.InitTempVars(self) def InitVars(self, name=None, admin='', crypted_password=''): - """Assign default values - some will be overriden by stored state.""" + """InitVars([name][, admin][, crypted_password]) -> None + + Initialized the mailing list information. This should not + be used directly (FIXME: truth?), use the .Create method + to create a new list instead. + """ + # Non-configurable list info if name: self._internal_name = name @@ -363,6 +468,10 @@ self.msg_footer = mm_cfg.DEFAULT_MSG_FOOTER def GetConfigInfo(self): + """GetConfigInfo() -> dictionairy + + Returns the information about the list configuration options. + """ config_info = {} config_info['digest'] = Digester.GetConfigInfo(self) config_info['archive'] = Archiver.GetConfigInfo(self) @@ -739,6 +848,14 @@ return config_info def Create(self, name, admin, crypted_password): + '''Create(name, admin, crypted_password) -> None + + Creates a mailinglist called 'name', with 'admin' as the list + owner. If the list already exists, ValueError is raised. The + third argument is the _encrypted_ password (a password can be + encrypted using the Crypt.crypt() function). + ''' + if name in Utils.list_names(): raise ValueError, 'List %s already exists.' % name Utils.ValidateEmail(admin) @@ -766,6 +883,11 @@ os.umask(ou) def Save(self): + """Save() -> None + + Saves the current configuration of the list. + """ + # If more than one client is manipulating the database at once, we're # pretty hosed. That's a good reason to make this a daemon not a # program. --huq684BweRXVnRxX-- From Gerrit Thu Feb 10 20:14:52 2000 From: Gerrit (Gerrit Holl) Date: Thu, 10 Feb 2000 21:14:52 +0100 Subject: [Mailman-Developers] Frames (was: unsubscriptions requiring approval) In-Reply-To: <20000210173114.F477@xs4all.nl>; from thomas@xs4all.net on Thu, Feb 10, 2000 at 05:31:14PM +0100 References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> <20000210173114.F477@xs4all.nl> Message-ID: <20000210211452.A8811@stopcontact.palga.uucp> Thomas Wouters wrote on 950200274: ... > (It might be even worth it to take a look at a frames interface, possibly ?) Noooo! Please don't! Sorry, but I really, really hate frames for several reasons. * I want to be able to admin my site in lynx, since I often haven't booted X and want to administrate fast, * it's not possible to bookmark a frames page, you can't bookmark the combination of the frames, * in netscape, the images button doesn't work on frames, * the sidebar needs to use javascript, and javascript is hard to maintain, * it's often unclear which frame you selected, so scrolling is hard, * it's neither in the HTML 4.01, nor in the XHTML 1.0 dtd. There's a seperate DTD for it. I think we should conform the HTML to XHTML 1.0 anyway (comments?), * you can't have a quick look at the HTML source code (I use to do that very often). I'd rather administrate my lists by email than using frames! Okay, maybe I'm exaggerating, but please, no frames! regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 10 20:30:01 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 10 Feb 2000 15:30:01 -0500 (EST) Subject: [Mailman-Developers] Frames (was: unsubscriptions requiring approval) References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> <20000210173114.F477@xs4all.nl> <20000210211452.A8811@stopcontact.palga.uucp> Message-ID: <14499.8137.204319.580736@anthem.cnri.reston.va.us> >>>>> "GH" == Gerrit Holl writes: >> (It might be even worth it to take a look at a frames >> interface, possibly ?) | Noooo! Please don't! | Sorry, but I really, really hate frames for several reasons. I concur 100%. To get largely the same effect without the headaches, see what we do on Python.Org. -Barry From ricardo@miss-janet.com Thu Feb 10 20:39:26 2000 From: ricardo@miss-janet.com (Ricardo - Miss Janet . Fanclub) Date: Thu, 10 Feb 2000 21:39:26 +0100 Subject: [Mailman-Developers] When will mailman 1.2 be released? In-Reply-To: ; from adhir@forumone.com on Thu, Feb 10, 2000 at 01:14:43PM -0500 References: Message-ID: <20000210213926.A897@miss-janet.com> On Thu, Feb 10, 2000 at 01:14:43PM -0500, Alok Dhir wrote: > 1. Ability to search for members by email address > 2. Ability to store membership info in a Mysql database > It would also be great if one could extend the schema of the subscriber > database to include additinal fields (name, address, etc), but this is a > much lower priority than the other two... Actually I've been thinking of some way to integrate (My)SQL with Mailman. It would be great if you could choose to have all the configuration stored in any form of DB you want, instead of the python db (the name slipped me for a moment here)... but i guess that would mean a major rewrite of a lot of MM's fuctinality... or maybe the user db could be separated from config.db... does anybody have some thoughts on this too? I know a lot of people would like to be able to have a customizable userdatabase. We're going to use Mailman for a project soon, where we need to define extra fields too. Ricardo. From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 10 21:01:36 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 10 Feb 2000 16:01:36 -0500 (EST) Subject: [Mailman-Developers] When will mailman 1.2 be released? References: <20000210213926.A897@miss-janet.com> Message-ID: <14499.10032.385586.426796@anthem.cnri.reston.va.us> >>>>> "R" == Ricardo writes: R> Actually I've been thinking of some way to integrate (My)SQL R> with Mailman. It would be great if you could choose to have R> all the configuration stored in any form of DB you want, R> instead of the python db (the name slipped me for a moment R> here)... but i guess that would mean a major rewrite of a lot R> of MM's fuctinality... or maybe the user db could be separated R> from config.db... R> does anybody have some thoughts on this too? R> I know a lot of people would like to be able to have a R> customizable userdatabase. We're going to use Mailman for a R> project soon, where we need to define extra fields too. I believe that the Right Solution would be to design an API for the underlying user database. I want to support people who, e.g. already have user accounts through Zope or some other Web-app/portal software, so that they don't have to force users to log in twice. This would let us distribute a GPL compatible database back-end for stand-alone operation, but should also let us integrate with existing user databases. That's about as far as my own thinking has gone. -Barry From claw@kanga.nu Thu Feb 10 21:23:09 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 10 Feb 2000 13:23:09 -0800 Subject: [Mailman-Developers] Frames (was: unsubscriptions requiring approval) In-Reply-To: Message from Gerrit Holl of "Thu, 10 Feb 2000 21:14:52 +0100." <20000210211452.A8811@stopcontact.palga.uucp> References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> <20000210173114.F477@xs4all.nl> <20000210211452.A8811@stopcontact.palga.uucp> Message-ID: <12287.950217789@kanga.nu> On Thu, 10 Feb 2000 21:14:52 +0100 Gerrit Holl wrote: > I want to be able to admin my site in lynx, since I often haven't > booted X and want to administrate fast... While not disagreing with your view of frames, you might like to look at W3M as a replacement for Lynx. Its a nippy little text mode web browser that handles frames _and_ tables properly. -- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From Harald.Meland@usit.uio.no Thu Feb 10 23:28:35 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 11 Feb 2000 00:28:35 +0100 Subject: [Mailman-Developers] Features requested... References: <14384.53717.606487.837270@anthem.cnri.reston.va.us> <14384.58392.966859.375322@anthem.cnri.reston.va.us> Message-ID: [Barry A. Warsaw] > >>>>> "SH" == Steven Hazel writes: > > Me> The problem here is that we don't actually keep that > Me> information around in the database. This is bogus, and is > Me> something that I hope will get fixed when we have a real User > Me> database (don't ask when ;) > > SH> What are the current plans for this? I'd like to help. I > SH> should have some time over the coming holidays. > > Well, there are a number of questions (if it was an easy hack it'd > already be done :). This is my cue, I guess ;) > First, do we want to use a real OODB underneath (such as ZODB), or do > we cook our own? I tried using ZODB at first, but found that my try at that scaled very poorly (The FileStorage file was ~160MB even before there were 2000 entries in my database, and the time needed to open the database seemed to scale with the size of it...). Dunno if this was due to me being boneheaded when implementing stuff, or to actual problems with ZODB, but I kinda lost my spirit for it anyway :-\ So, I surfed to Parnassos, searched the vault there, and came up with a copy of bplustree.py. Implemented purely in Python, as opposed to ZODB, which relies on some C Zope modules. The disk space used per record seems very reasonable. Open time appears to be pretty much unrelated to file size. Record access time seems fast enough. One problem, though -- this module does not handle multiple simultanous writes, so we have to be careful with locking -- which *could* make the user db into a bottleneck. [ As we put Mailman into production with the ~3.500 lists we have here at uio.no, we discovered that the current locking code didn't work very well on a system that got so busy it started swapping -- so I rewrote the entire file locking module to be more robust, and added some code to have the "post" mail injection script exit with temporary error codes if it couldn't get a lock on the list within a reasonable amount of time -- leaving it to the MTA to try injecting the mail again later. Things have been running smooth ever since. ] A fresh entry (created to be added as member of LIST-A) in the resulting user db have these attributes: {'admin_for': {}, 'admin_pwd': None, 'aliases': {'ADDRESS@DOMAIN': 1}, 'member_of': {'LIST-A': 1}, 'member_pwd': 'PASSWORD', 'preferred_address': 'ADDRESS@DOMAIN'} Currently 'aliases' and 'preferred_address' isn't used, but the intention is to use them for associating several mail addresses with the same user account (probably by some means of email confirmation). 'member_pwd' is stored in cleartext, while 'admin_pwd' (when set) is stored as a MD5 digest. An "username" field has been added to all login pages -- currently we're running with this as a text entry field, but I'll probably install some changes to use a Message Header: Message Excerpt: Action:
Approve Reject Discard (eg, spam)
--h31gzZEtNLTqOjlF-- From Gerrit Mon Feb 14 15:50:37 2000 From: Gerrit (Gerrit Holl) Date: Mon, 14 Feb 2000 16:50:37 +0100 Subject: [Mailman-Developers] Mailman limitations In-Reply-To: ; from Harald.Meland@usit.uio.no on Sun, Feb 13, 2000 at 07:18:58PM +0100 References: <200002122023.MAA08929@ns2.uncanny.net> <20000212232702.K477@xs4all.nl> Message-ID: <20000214165037.A1849@stopcontact.palga.uucp> Harald Meland wrote on 950465938: > > I don't want to give anyone the idea that the direction of Mailman > development is not properly influenced by the input from the members > on this list -- after all, discussing Mailman development is what this > list is for. I've already sent some patches to this list, but none were implemented. For example, my latest patch, it added docstrings to many methods of MailList. I also posted a proposal for a feature to be added to the TODO list; no response. I don't feel very comfortable with it. Who have access to the CVS tree? The five people mentoined in the acknowledgements file? kind regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From Harald.Meland@usit.uio.no Mon Feb 14 16:57:54 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 14 Feb 2000 17:57:54 +0100 Subject: [Mailman-Developers] Mailman limitations In-Reply-To: Thomas Wouters's message of "Mon, 14 Feb 2000 00:38:07 +0100" References: <200002122023.MAA08929@ns2.uncanny.net> <20000212232702.K477@xs4all.nl> <20000214003807.Q477@xs4all.nl> Message-ID: [Thomas Wouters] > But Mailman is, in my opinion, nowhere near complicated enough to > make it easy to misconfigure it. Yet. With a few exceptions (read: weird special cases), I agree. > > Ideally, we'd like to support as big a set of (good :) features as any > > other mailing list administration package, but without alienating > > first-time Mailman list administrators through a veritable maze of > > list settings, all of them interconnected... Good documentation might > > solve part of this, but if there are (overly) complex features the > > corresponding documentation will probably have to be complex as well, > > meaning that the Mailman learning curve gets steeper. > > Exactly. And let me tell you that good documentation is not enough if the > first look the fresh administrator gets of his new list is as complicated as > the bridge of the USS Enterprise. That's what the "might solve _part of_ this" stuff was meant to imply... ;) > However, I dont think redesigning the adminpages to make it all fit AND > appear sane, is impossible. But, are "sane" and "user understandable" interchangeable? (Sorry, the bofh part of me took control there for a minute :). > I think it can be done with just a little bit of thought ;) > Currently, the admin page is, to be frank, overwhelming. Yup -- that's the reason I'm somewhat sceptical to accept patches for new "special case" features. > The moment you open the page, you get a large list of options to > select from. Some of these are useful and clear. Others, most, are > more-or-less obvious, but not of interest. And some are plain > weird. You do your best to browse them and edit them to your need, > and then you look up, and you see that you have *seven*more*pages* > to do ! AND three more pages, which may contain more links, of other > administrativia ! I guess this is a bit like what kind of stereo systems people prefer -- some people want the one with the most buttons (and flashing lights), while other people just want something that *works*. :-D Some time back, someone asked whether it would be possible to have different "styles" on the list admin pages -- i.e., the more techish admins would get the full (overwhelming) set of admin settings, while newbie admins would get a (much less confusing and) smaller set of the most commonly used list settings (possibly with the option of getting access to the full set of settings if they e.g. followed some "for expert list admins" link). Which set of options to present to the admin could either be settable per list or per admin (when we get the user database in place). There have also been a little talk of even more generic solutions to this -- e.g. implementing list configurations as inheritance hierarchies, so that a change to the "base" configuration would automatically affect all the lists that inherit the attributes that were changed but don't override them. However, it seems to me that every time we start discussing advanced web interface stuff, someone mentions integrating Mailman with Zope -- which it is said would buy us a lot of neat features practically for free -- and people go "Yeah" and "Cool" and "Sounds like a perfect match"... and then nobody actually _do_ anything about it all. :) Should Mailman be integrated with Zope? Would such a change mean that Mailman has to _rely_ on Zope being installed, or would it be feasible (and resonably easy) for Mailman to take advantage of Zope only if it's available (at run time? Or at install time?)? What advantages would such a change buy us? Are there things we want to do anyway, and which would _have_ to be changed to integrate with Zope, and thus could be started work on right away? Should all of this wait until after the next Mailman release is completed? ;) Is my ranting about this at all called for? ;) > What needs to be done mostly is just creating more but smaller pages, > especially for the current 'general options' and 'privacy options' pages, > plus for whatever options are added. To me, having a tangle with lots of different pages with few options on them isn't really much better than a tangle with lots of options on a few pages. The _real_ problem is to avoid the tangle. I agree that the structure of the current admin pages aren't optimal, though -- but restructuring the pages will only take us so far if we start adding specialcase code left and right. > (While I was writing this and thinking it up (in that order ;) I thought how > incredibly cool it would be to have some kind of rule-based message > acception (/refusal/holding/discarding/temp-fail/...) system, where each > rule could be header-match, number of postings to the list so far, load of > the machine matched against a priority for the list or user, time of day or > whatever. And where each rule would have it's own default reject/accept > messages, of course. Not very hard to implement in Python and the current > 'pipe' architecture, if I may be so kind. But for the life of me I can't > figure out how to make such a scheme configurable through an HTML page ;P) That's *exactly* the way I feel about that stuff. :) Of course, you could just let the admin edit the rules as pure text -- and then have the CGI script do the appropriate syntax checks and other sanity checks/rule validation before actually changing the list's configuration. Of course, that would mean we're _very_ close to significantly affecting the slope of Mailman's learning curve. > > > I can't say for certain what parts are currently being developped, > > > though, I'm relatively new on this list, and a lot of development > > > seems to happen behind the scenes. > > > I'm not quite sure what you mean by that, but just to clarify: I try > > to keep/redirect whatever discussions on Mailman development I get > > involved in to this list. However, I've "been away" for some time, > > busy doing Real Work -- part of which was to implement new Mailman > > features that we here at uio.no were in pressing need of. > > > I don't want to give anyone the idea that the direction of Mailman > > development is not properly influenced by the input from the members > > on this list -- after all, discussing Mailman development is what this > > list is for. > > I'm sorry, I didn't mean to imply that this list was defunct, or that > mailman development is dead, or that people are wasting their time waiting > for the developers, or any of that. Good -- and I tried to phrase the above in a way as not to suggest that I thought you did -- it was meant purely as an clarification, not as some kind of an accusation. > It's just that I dont see the lively discussions I'm used to I can relate with that -- but I can't really figure out why that's how things are here at mailman-developers... > Or maybe I hadn't realized how busy the cabal is -- having a fun > project (Mailman) highish on my priority list for once makes me > slightly impatient. You want us to believe that normally you're _not_ impatient? Yeah, right... ;-D > I shouldn't complain, anyway. Most of my patches have received good > commentary, and one even got applied ! Well, neither should we... just keep'em coming. Cheers, -- Harald From lindsey@ncsa.uiuc.edu Mon Feb 14 17:39:51 2000 From: lindsey@ncsa.uiuc.edu (Christopher Lindsey) Date: Mon, 14 Feb 2000 11:39:51 -0600 (CST) Subject: [Mailman-Developers] Mailman limitations In-Reply-To: from "Harald Meland" at Feb 14, 2000 05:57:54 PM Message-ID: <200002141739.LAA17610@glorfindel.ncsa.uiuc.edu> > Should Mailman be integrated with Zope? > > Would such a change mean that Mailman has to _rely_ on Zope being > installed, or would it be feasible (and resonably easy) for Mailman to > take advantage of Zope only if it's available (at run time? Or at > install time?)? The latter. Look at the mess that pipermail got Mailman into. It became an unsupported package, then people wanted to integrate something bigger and better but couldn't... Ultimately things changed to use built-in functionality (pipermail) but remain configurable enough to specify one's own external archiver... Mailman/Zope interoperability should probably follow the same guidelines. Chris From thomas@xs4all.net Mon Feb 14 21:10:19 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 14 Feb 2000 22:10:19 +0100 Subject: [Mailman-Developers] Mailman limitations In-Reply-To: ; from Harald.Meland@usit.uio.no on Mon, Feb 14, 2000 at 05:57:54PM +0100 References: <200002122023.MAA08929@ns2.uncanny.net> <20000212232702.K477@xs4all.nl> <20000214003807.Q477@xs4all.nl> Message-ID: <20000214221018.R477@xs4all.nl> On Mon, Feb 14, 2000 at 05:57:54PM +0100, Harald Meland wrote: > [Thomas Wouters] [me complaining about configuration pages] > Some time back, someone asked whether it would be possible to have > different "styles" on the list admin pages -- i.e., the more techish > admins would get the full (overwhelming) set of admin settings, while > newbie admins would get a (much less confusing and) smaller set of the > most commonly used list settings (possibly with the option of getting > access to the full set of settings if they e.g. followed some "for > expert list admins" link). Well, what would be pretty welcome, i think, is a clear division between 'You probably want to edit or at least browse these options', and 'you dont really need to change anything here, unless Mailman isn't working properly.' That's more or less what my incoherent blurp about adminpages was about. The expert-options dont actually need to be inaccessible... they just need to be flagged as 'dont bother with these', or some such. > Should Mailman be integrated with Zope? Eww, well, saying it like that, 'be integrated', my first reaction is 'no'. (Well, that's my first printable reaction ;) I dont run Zope, and i dont plan to run Zope anytime soon, mostly because it does something i, being a techy, have absolutely no need for. Maybe our webteam wants it, but i doubt it -- they're focusing on other things right now. I wouldn't object in any way against a sort-of 'driver' for Zope, a Zope-interface to Mailman, as long as you aren't supposed to use it, but i dobut it'll fly if it doesn' thave at least one, preferably more, dedicated maintainers. And the fact it hasn't been built yet, suggests that that dedication isn't there yet ;) > > (While I was writing this and thinking it up (in that order ;) I thought how > > incredibly cool it would be to have some kind of rule-based message > > acception (/refusal/holding/discarding/temp-fail/...) system, where each > > rule could be header-match, number of postings to the list so far, load of > > the machine matched against a priority for the list or user, time of day or > > whatever. And where each rule would have it's own default reject/accept > > messages, of course. Not very hard to implement in Python and the current > > 'pipe' architecture, if I may be so kind. But for the life of me I can't > > figure out how to make such a scheme configurable through an HTML page ;P) > > That's *exactly* the way I feel about that stuff. :) > Of course, you could just let the admin edit the rules as pure text -- > and then have the CGI script do the appropriate syntax checks and > other sanity checks/rule validation before actually changing the > list's configuration. > Of course, that would mean we're _very_ close to significantly > affecting the slope of Mailman's learning curve. No, because you can offer the current system as well. Maybe a few 'default' reject/hold rules that you can turn on and off. But you can't do it really flexible using HTML, so if you want the *real* power, you have to edit the textbox. And the textbox needs to have flashing red warning messages around it, of course. But it needn't be more difficult than Python. That's more or less what i had in mind... a 'Simple as hell, who needs a manual' section, and a 'easy as python, read the simple docs first' section ;) > > It's just that I dont see the lively discussions I'm used to > I can relate with that -- but I can't really figure out why that's how > things are here at mailman-developers... Because the developers are overworked, and the public too impatient ? :) Short-message-cuz-Fawlty-Towers-is-on-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From ckolar@admin.aurora.edu Mon Feb 14 22:57:55 2000 From: ckolar@admin.aurora.edu (Christopher Kolar) Date: Mon, 14 Feb 2000 16:57:55 -0600 Subject: [Mailman-Developers] list admin guide v0.2 now online Message-ID: <4.2.2.20000214165638.00aa6ac0@admin.aurora.edu> Hello everyone. It has been a while since I last updated the information, but I have finally gotten around to making revisions to my first attempt at MailMan documentation. The second versions of the list administrator's guide and quick reference are now online at: www.aurora.edu/~ckolar/mailman This document has been prepared for list owners: people who are not necessarily technical but who own a MailMan list. For various reasons I am locked into doing them in HTML for the time being, but will someday take it all and put it into TexInfo format. Please feel free to send me feedback. --chris -- /////\\\\\/////\\\\\ Christopher G. Kolar Director of Instructional Technology Aurora University, Aurora, Illinois ckolar@admin.aurora.edu -- www.aurora.edu/~ckolar [PGP Public Key ID: 0xC6492C72] From Harald.Meland@usit.uio.no Tue Feb 15 17:05:09 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 15 Feb 2000 18:05:09 +0100 Subject: [Mailman-Developers] Mailman limitations In-Reply-To: Gerrit Holl's message of "Mon, 14 Feb 2000 16:50:37 +0100" References: <200002122023.MAA08929@ns2.uncanny.net> <20000212232702.K477@xs4all.nl> <20000214165037.A1849@stopcontact.palga.uucp> Message-ID: [Gerrit Holl] > Harald Meland wrote on 950465938: > > > > I don't want to give anyone the idea that the direction of Mailman > > development is not properly influenced by the input from the members > > on this list -- after all, discussing Mailman development is what this > > list is for. > > I've already sent some patches to this list, but none were > implemented. For example, my latest patch, it added docstrings to > many methods of MailList. Sorry, I haven't gotten around to that -- and, as I've tried to indicate, my local tree is regrettably quite a bit out of sync with CVS right now :( I'm trying to use whatever time I can find to backport the parts of my local tree that can/should go into CVS _now_ -- and get up to date with this list. However, I have put your messages in the "look at this when I have time" category -- I'll try to become better at notifying people whenever I put stuff into that category. > Who have access to the CVS tree? The five people mentoined in the > acknowledgements file? AFAIK: Yes, only the "core developers" have CVS write access. I think there have been some attempts to make that situation better whenever it got clear that all the core developers were to busy with other stuff, but I don't think anything has come of these attempts yet. Hmmm, would SourceForge solve any of this? -- Harald From ricardo@rixhq.nu Tue Feb 15 21:57:11 2000 From: ricardo@rixhq.nu (Ricardo 'admindb' Kustner) Date: Tue, 15 Feb 2000 22:57:11 +0100 Subject: [Mailman-Developers] Message.Message Message-ID: <20000215225711.A4983@miss-janet.com> Hi, It seems like nobody looked at the changes i've posted for admindb.py cause i just found out that I made it more difficult than necessary to seperate the message body from the header :) This funcinality is already possible using MM/python libs : fp = open(os.path.join(mm_cfg.DATA_DIR, filename)) fullmsg = Message.Message(fp) msg_header = string.join(fullmsg.headers, '') msg_body = fullmsg.body fp.close() ...on the bright side, I learned a bit more about python and the MM code :) oh i just realized, that this way the ADMINDB_PAGE_TEXT_LIMIT isn't taken into account... :( Ricardo. From ckolar@admin.aurora.edu Wed Feb 16 16:07:42 2000 From: ckolar@admin.aurora.edu (Christopher Kolar) Date: Wed, 16 Feb 2000 10:07:42 -0600 Subject: [Mailman-Developers] sugg: disable notify when using remove_member Message-ID: <4.2.2.20000216100346.00b43770@admin.aurora.edu> Hi everyone. Is there any chance that a toggle could be added to the remove_member program to make it function like add_member with respect to notification? We have a lot of lists that get cleared out periodically, and we have a lot of users who leave us every semester, and the notification either generates unnecessary helpdesk calls from the confused or generates bounces from people whose accounts are no longer active. I am not asking for it to be added to the web interface, because I know that it can be abused, but it would be nice at the command line for some of the more heavy-duty automated things that we are putting together. Cheers, --chris -- /////\\\\\/////\\\\\ Christopher G. Kolar Director of Instructional Technology Aurora University, Aurora, Illinois ckolar@admin.aurora.edu -- www.aurora.edu/~ckolar [PGP Public Key ID: 0xC6492C72] From thomas@xs4all.net Wed Feb 16 17:20:16 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 16 Feb 2000 18:20:16 +0100 Subject: [Mailman-Developers] internationalization ? Message-ID: <20000216182016.U477@xs4all.nl> Barry mentioned something, a couple of weeks back, about spending a weekend trying to integrate i18n patches to mailman, to internationalize it. I assume those patches do some heavy-duty rewriting of various bits of the template code and such, so i'm a bit hesitant to work on those bits of code, though i have a few things left on my wishlist in those areas ;) Can anyone tell me what the status is of that code, wether it indeed affects large parts of mailman, or give me a pointer to the patches ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From ricardo@rixhq.nu Wed Feb 16 21:23:50 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Wed, 16 Feb 2000 22:23:50 +0100 Subject: [Mailman-Developers] internationalization ? In-Reply-To: <20000216182016.U477@xs4all.nl>; from thomas@xs4all.net on Wed, Feb 16, 2000 at 06:20:16PM +0100 References: <20000216182016.U477@xs4all.nl> Message-ID: <20000216222350.C13168@miss-janet.com> On Wed, Feb 16, 2000 at 06:20:16PM +0100, Thomas Wouters wrote: > Barry mentioned something, a couple of weeks back, about spending a weekend > trying to integrate i18n patches to mailman, to internationalize it. I > assume those patches do some heavy-duty rewriting of various bits of the > template code and such, so i'm a bit hesitant to work on those bits of code, > though i have a few things left on my wishlist in those areas ;) there's a special list for mailman i18n stuff... check out the info/archives at http://www.python.org/mailman/listinfo/mailman-i18n Ricardo. -- From jcrey@uma.es Thu Feb 17 08:08:40 2000 From: jcrey@uma.es (Juan Carlos Rey Anaya) Date: Thu, 17 Feb 2000 09:08:40 +0100 Subject: [Mailman-Developers] sugg: disable notify when using remove_member References: <4.2.2.20000216100346.00b43770@admin.aurora.edu> Message-ID: <38ABAC88.ECBB8935@uma.es> Christopher Kolar wrote: > > Hi everyone. Is there any chance that a toggle could be added to the > remove_member program to make it function like add_member with respect to > notification? Modify called function to: mlist.DeleteMember(addr, None, 0) in remove_members Cheers -- ___ / F \ [[[]]]] ( O O ) #----------------0000--(_)--0000---------------# | Juan Carlos Rey Anaya (jcrey@uma.es) | | Servicio Central de informática | | Universidad de Málaga - España | #----------------------------------------------# # Solo se que cada vez se menos :-| # #----------------------------------------------# From fil@bok.net Thu Feb 17 08:59:49 2000 From: fil@bok.net (Fil) Date: Thu, 17 Feb 2000 09:59:49 +0100 Subject: [Mailman-Developers] RE: dont request passwords from web subscriber patch In-Reply-To: ; from lee@ufojoe.com on Sun, Feb 06, 2000 at 08:02:00PM -0500 References: Message-ID: <20000217095949.A29324@orwell.bok.net> * Lee Weston (lee@ufojoe.com) écrivait : > Fri, 7 Jan 2000 14:58:16 +0100 (MET) > Fil fil@bok.net wrote: > > > >could the maintainers tell me if this patch is accepted or rejected ? > >Patch name : dont request passwords from web subscriber > > I'm sorry to see no answer. I will be looking at adding it. > An "only allow random password" mode would have the benifit of > substiantially raising the bar for writting cgi to raid / spam a site, by > making them parse email as well. That's right. From several messages in the preceding week it looks like there's interest in this kind of patch ; certainly the patch I have submited is not complete enough. So let's look at it as a feature request./ From jcrey@uma.es Thu Feb 17 09:32:16 2000 From: jcrey@uma.es (Juan Carlos Rey Anaya) Date: Thu, 17 Feb 2000 10:32:16 +0100 Subject: [Mailman-Developers] internationalization ? References: <20000216182016.U477@xs4all.nl> Message-ID: <38ABC020.4BD37692@uma.es> Thomas Wouters wrote: > > Can anyone tell me what the status is of that code, wether it indeed affects > large parts of mailman, or give me a pointer to the patches ? > The actual status is: I'm waiting for Barry to finish integrating my patches to actual Mailman. I sent patches on 11/Jan/2000 and new files. Modified sources are: /home/mailman/Mailman/Cgi/admin.py /home/mailman/Mailman/Cgi/admindb.py /home/mailman/Mailman/Cgi/edithtml.py /home/mailman/Mailman/Cgi/handle_opts.py /home/mailman/Mailman/Cgi/listinfo.py /home/mailman/Mailman/Cgi/options.py /home/mailman/Mailman/Cgi/private.py /home/mailman/Mailman/Cgi/roster.py /home/mailman/Mailman/Cgi/subscribe.py /home/mailman/Mailman/Archiver/Archiver.py /home/mailman/Mailman/Archiver/pipermail.py /home/mailman/Mailman/Bouncer.py /home/mailman/Mailman/Digester.py /home/mailman/Mailman/GatewayManager.py /home/mailman/Mailman/HTMLFormatter.py /home/mailman/Mailman/ListAdmin.py /home/mailman/Mailman/Deliverer.py /home/mailman/Mailman/MailCommandHandler.py /home/mailman/Mailman/MailList.py /home/mailman/Mailman/Utils.py /home/mailman/Mailman/htmlformat.py /home/mailman/Mailman/Defaults.py /home/mailman/Mailman/Handlers/Acknowledge.py /home/mailman/Mailman/Handlers/CookHeaders.py /home/mailman/Mailman/Handlers/HandlerAPI.py /home/mailman/Mailman/Handlers/Hold.py /home/mailman/Mailman/Handlers/ToDigest.py /home/mailman/Mailman/Handlers/ToUsenet.py and I created one directory hierarchy under /home/mailman/messages (all the stuff is new) and modified the structure of templates directory in subdirectories, each one for each suported language. I modified some minor changes I don't remeber now. We will have to wait :-) Cheers Ah! There is a functional demo version at: http://joker.sci.uma.es/mailman/listinfo if anybody wish to see admin messages translated, admin passwd is 'ejemplo' (behave well :-) -- ___ / F \ [[[]]]] ( O O ) #----------------0000--(_)--0000---------------# | Juan Carlos Rey Anaya (jcrey@uma.es) | | Servicio Central de informática | | Universidad de Málaga - España | #----------------------------------------------# # Solo se que cada vez se menos :-| # #----------------------------------------------# From ppetru@coltronix.com Thu Feb 17 15:03:21 2000 From: ppetru@coltronix.com (Petru Paler) Date: Thu, 17 Feb 2000 17:03:21 +0200 Subject: [Mailman-Developers] Adding a new archiver Message-ID: <20000217170321.A3457@coltronix.com> Hello, I want to add a custom archiver to Mailman to make it pump messages into Zope. Is there a standard way to do this or should I just go and hack the source ? Has anyone made Mailman and Zope talk to each other ? -Petru From claw@kanga.nu Fri Feb 18 07:46:13 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 17 Feb 2000 23:46:13 -0800 Subject: [Mailman-Developers] Adding a new archiver In-Reply-To: Message from Petru Paler of "Thu, 17 Feb 2000 17:03:21 +0200." <20000217170321.A3457@coltronix.com> References: <20000217170321.A3457@coltronix.com> Message-ID: <14483.950859973@kanga.nu> On Thu, 17 Feb 2000 17:03:21 +0200 Petru Paler wrote: > Hello, I want to add a custom archiver to Mailman to make it > pump messages into Zope. Is there a standard way to do this or > should I just go and hack the source ? Has anyone made Mailman and > Zope talk to each other ? Your best bet is probably to write a small external tool that accepts a message from stdin and then creates the appropriate object under Zope that contains the message. You can then just plug that into MailMan as an external archiver. BTW: If you do this, please drop a note to this and the zope mailing lists. I know several people who would be interested. -- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From Gerrit Fri Feb 18 14:31:52 2000 From: Gerrit (Gerrit Holl) Date: Fri, 18 Feb 2000 15:31:52 +0100 Subject: [Mailman-Developers] Adding a new archiver In-Reply-To: <20000217170321.A3457@coltronix.com>; from ppetru@coltronix.com on Thu, Feb 17, 2000 at 05:03:21PM +0200 References: <20000217170321.A3457@coltronix.com> Message-ID: <20000218153152.A3793@stopcontact.palga.uucp> Petru Paler wrote on 950803401: > Hello, > > I want to add a custom archiver to Mailman to make it pump messages into > Zope. Is there a standard way to do this or should I just go and hack the source ? Has anyone made Mailman and Zope talk to each other ? You can subscribe an address to the list that archives all messages it receives. Add some lines to MailList.Create to create a private member of the list. That's what the Sourceforge staff did. regards, Gerrit. -- cat: /home/gerrit/.signature: No such quote or joke From bbum@codefab.com Fri Feb 18 18:30:39 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Fri, 18 Feb 2000 13:30:39 -0500 Subject: [Mailman-Developers] external archiver Message-ID: <200002181830.NAA19438@bjork.codefab.com> I have written an archiver that uses WebDAV to archive messages [with or without decoded attachments] to a WebDAV enabled web server. I'm currently using Apache, but there is no reason why it shouldn't work "out of the box" with Zope (Zope has a WebDAV enabler). Email if you want patches/code... it works *very* well. We will be going into full production with it in the next week [www.bebusy.com]. b.bum From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Sat Feb 19 03:59:04 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 18 Feb 2000 22:59:04 -0500 (EST) Subject: [Mailman-Developers] New feature, auto-responding Message-ID: <14510.5384.592046.76136@anthem.cnri.reston.va.us> I've been a bad boy. I should have been killing bugs, or catching up on the list traffic, or working on the I18N or other new features. But I had a particularly annoying itch that I wanted see how easy it was to scratch. Turned out pretty easy - it took me about an hour to implement, and another hour or so to test. What I've added is an auto-responder. You can control whether auto-responses are send to mailing list posts and/or -admin emails (it makes no sense to auto-respond to -request emails, since that's already a bot). You can also set the auto-response texts and the grace period. Below is the check-in message for those of you not on mailman-checkins. Okay, now I promise to get to all the things I'm supposed to do. I'd like to be very close to releasing 1.2 by the first weekend in March. Enjoy, -Barry -------------------- snip snip -------------------- Autoresponder: New mixin class which contains attributes and configuration information for the new replybot functionality. Mailman can now send automatic responses to mailing list posts or -admin emails (-request autoreplies aren't necessary since that's a bot anyway). The following new configuration attributes control this feature: autorespond_postings - boolean which controls autoreplies to mailing list posts. autorespond_admin - same as above the -admin address. autoresponse_postings_text - the mailing list post autoreply text. This is %(string)s interpolated with a dictionary as described below. autoresponse_admin_text - same as above for the -admin address. autoresponse_graceperiod - the number of days between automatic responses to the same email author. This variable controls both autoresponses. If this is <= 0, then there is no grace period. The following non-configurable attributes store the state: postings_responses - a dictionary of email addresses (lower cased) to the time.time() that the grace period expires. admin_responses - same as above for the -admin address. autoresponse_postings_text and autoresponse_admin_text are string interpolated with the following dictionary: listname - the mailing list's "real_name" listurl - the list's "listinfo" url requestemail - the list's -request address adminemail - the list's -admin address We should probably add others. From bwarsaw@cnri.reston.va.us Sun Feb 20 15:30:08 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Sun, 20 Feb 2000 10:30:08 -0500 (EST) Subject: [Mailman-Developers] New feature, auto-responding References: <14510.5384.592046.76136@anthem.cnri.reston.va.us> <20000219115638.A4297@miss-janet.com> Message-ID: <14512.2176.760495.408684@anthem.cnri.reston.va.us> >>>>> "RK" == Ricardo Kustner writes: RK> this sounds like an interesting feature... it would be RK> extremely cool if MM could autorespond to messages containing RK> HTML encoded email: this way those people can get a response RK> that next time they time they shouldn't post HTML email, and RK> some short instructions how to change to plain text. Or am I RK> missing the point of the autoresponder usage here? I hacked this stuff in to handle a couple of MM lists we've got running on py.org, namely help@ and tutor@. Both of these have standard reply texts they send to posters, i.e. giving them a list of places to look while their email is processed. Maybe we could hook your idea into bbum's work? RK> I know there are several people on the develop list right now RK> who's fingers are itching to contribute to the MM code. Maybe RK> we could make a list of features we want to implement in 1.2, RK> and then see if anybody wants to pick up one or more of those RK> features. Personally I really want to help out too, even RK> though i'm still learning Python and the way MM is structured, RK> i'd like to expand my knowledge beyond admindb.py :) I think that's a great idea. One of the things that will help this, I believe, is moving the project over to SourceForge. I've started to set that up, but got waylaid because I think they assign project numbers from the Freshmeat database, and it looks like they linked us to MailMan, not Mailman (i.e. Endymion's completely unrelated product). I've got a couple of calls into SF, but am still waiting to hear about that. I think SF provides a lot of very useful tools for OSS project management, and I'd like to see us move there after 1.2. The trick to accepting actual code contributions is the copyright assignment requirements. As for the way MM is structured, I've done some cleaning up of that, but I think more still needs to be done. -Barry From ricardo@rixhq.nu Sat Feb 19 10:56:38 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Sat, 19 Feb 2000 11:56:38 +0100 Subject: [Mailman-Developers] New feature, auto-responding In-Reply-To: <14510.5384.592046.76136@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Fri, Feb 18, 2000 at 10:59:04PM -0500 References: <14510.5384.592046.76136@anthem.cnri.reston.va.us> Message-ID: <20000219115638.A4297@miss-janet.com> On Fri, Feb 18, 2000 at 10:59:04PM -0500, Barry A. Warsaw wrote: > What I've added is an auto-responder. You can control whether > auto-responses are send to mailing list posts and/or -admin emails (it > makes no sense to auto-respond to -request emails, since that's > already a bot). You can also set the auto-response texts and the > grace period. Below is the check-in message for those of you not on > mailman-checkins. this sounds like an interesting feature... it would be extremely cool if MM could autorespond to messages containing HTML encoded email: this way those people can get a response that next time they time they shouldn't post HTML email, and some short instructions how to change to plain text. Or am I missing the point of the autoresponder usage here? > Okay, now I promise to get to all the things I'm supposed to do. I'd > like to be very close to releasing 1.2 by the first weekend in March. I know there are several people on the develop list right now who's fingers are itching to contribute to the MM code. Maybe we could make a list of features we want to implement in 1.2, and then see if anybody wants to pick up one or more of those features. Personally I really want to help out too, even though i'm still learning Python and the way MM is structured, i'd like to expand my knowledge beyond admindb.py :) Ricardo. From lindsey@ncsa.uiuc.edu Mon Feb 21 17:12:03 2000 From: lindsey@ncsa.uiuc.edu (Christopher Lindsey) Date: Mon, 21 Feb 2000 11:12:03 -0600 (CST) Subject: [Mailman-Developers] Quick patch to fix '%' in sender Message-ID: <200002211712.LAA13941@glorfindel.ncsa.uiuc.edu> We were having problems with email addresses containing a '%'. Specifically, if the address were held for some reason the admindb interface would choke when it tried to process it. There's a (very) small patch below against the current CVS tree. Chris ---------------------------------------------------------------------- *** Mailman/ListAdmin.py Mon Feb 21 11:11:55 2000 --- Mailman/ListAdmin.py Mon Feb 21 11:11:55 2000 *************** *** 192,198 **** \tSubject: %(subject)s''' % { 'listname' : self.internal_name(), 'rejection': rejection, ! 'sender' : sender, 'subject' : strquote(subject), } if comment: --- 192,198 ---- \tSubject: %(subject)s''' % { 'listname' : self.internal_name(), 'rejection': rejection, ! 'sender' : strquote(sender), 'subject' : strquote(subject), } if comment: From pfaff@edge.cis.mcmaster.ca Mon Feb 21 18:40:44 2000 From: pfaff@edge.cis.mcmaster.ca (Todd Pfaff) Date: Mon, 21 Feb 2000 13:40:44 -0500 (EST) Subject: [Mailman-Developers] a more uniform interface to private and public archives Message-ID: for mailman-1.1, the installation documentation suggests to create, for example, an 'Alias' path for the web server to the mailman public archives and a 'ScriptAlias' cgi-bin path to the private archives. what i'd like to suggest is that we make the interface more uniform by eliminating the 'Alias' path and access both private and public archives via a single cgi-bin interface. if the archive is private we require authentication, if not we simply bypass the authentication. i've done this with my mailman installation by doing the following: - created a new version of the MailMan/Cgi/private.py program - in mm_cfg.py, set PUBLIC_ARCHIVE_URL = '/mailman/private' PRIVATE_ARCHIVE_URL = '/mailman/private' - these could then be collapsed into one ARCHIVE_URL - we could also replace PUBLIC_ARCHIVE_FILE_DIR = os.path.join(PREFIX, 'archives/public') PRIVATE_ARCHIVE_FILE_DIR = os.path.join(PREFIX, 'archives/private') with one ARCHIVE_FILE_DIR, and we could also get rid of the public and private subdirectories altogether. in the new private.py i check listobj.archive_private and if it's set to 1 i do the usual private authentication as before. if it's not 1, i set is_auth to 1 and fall through. that's it. very clean and simple. does anyone see any problems with this? i think it certainly makes things more clear and straightforward. -- Todd Pfaff \ Email: pfaff@mcmaster.ca Computing and Information Services \ Voice: (905) 525-9140 x22920 ABB 132 \ FAX: (905) 528-3773 McMaster University \ Hamilton, Ontario, Canada L8S 4M1 \ From secabeen@pobox.com Mon Feb 21 19:56:52 2000 From: secabeen@pobox.com (Ted Cabeen) Date: Mon, 21 Feb 2000 13:56:52 -0600 Subject: [Mailman-Developers] Membership management chunking requires two authentications.. Message-ID: <200002211956.NAA29500@entropy.uchicago.edu> Mailman: 1.1 Python: 1.5.2 OS: Solaris 5.7 I don't know if this has been fixed in the CVS version, but I thought I'd submit it anyways. There seems to be a small authentication bug with the chunking in the membership management page. The page will load up correctly the first time it is accessed with the first chunk size shown. However, when you request a different chunk, it requires a reauthentication and then gives the chunk=0 page instead of the requested chunk. If you then click on the alternate chunk it loads fine. However, the additional click and authentication is annoying. -- Ted Cabeen http://www.pobox.com/~secabeen secabeen@pobox.com Check Website or finger for PGP Public Key secabeen@midway.uchicago.edu "I have taken all knowledge to be my province." -F. Bacon cococabeen@aol.com "Human kind cannot bear very much reality."-T.S.Eliot 73126.626@compuserve.com From ricardo@rixhq.nu Mon Feb 21 21:15:18 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Mon, 21 Feb 2000 22:15:18 +0100 Subject: [Mailman-Developers] Membership management chunking requires two authentications.. In-Reply-To: <200002211956.NAA29500@entropy.uchicago.edu>; from secabeen@pobox.com on Mon, Feb 21, 2000 at 01:56:52PM -0600 References: <200002211956.NAA29500@entropy.uchicago.edu> Message-ID: <20000221221518.A17481@miss-janet.com> On Mon, Feb 21, 2000 at 01:56:52PM -0600, Ted Cabeen wrote: > I don't know if this has been fixed in the CVS version, but I thought I'd > submit it anyways. There seems to be a small authentication bug with the > chunking in the membership management page. The page will load up correctly > the first time it is accessed with the first chunk size shown. However, when > you request a different chunk, it requires a reauthentication and then gives > the chunk=0 page instead of the requested chunk. If you then click on the > alternate chunk it loads fine. However, the additional click and > authentication is annoying. I have/had the same problem, and i'm using the CVS version... I just had a closer look and now I know why it behaves this way in my case: our list is accessible at two hostnames which share the exact same website... the hostname that is linked at the membership chunk links is different than the hostname i sometimes use to access the list... so netscape thinks the cookie is for a different host... which makes me wonder why the chunk links on the admin page have a full http:// link while other links on it don't ? maybe you are experiencing the same problem? compare the hostname you use to access the admin page with the hostname that's being used in the links to the membership chunks... if they are different, that's the reason why you have to re-enter the password. Ricardo. -- From secabeen@pobox.com Mon Feb 21 21:26:48 2000 From: secabeen@pobox.com (Ted Cabeen) Date: Mon, 21 Feb 2000 15:26:48 -0600 Subject: [Mailman-Developers] Membership management chunking requires two authentications.. In-Reply-To: Your message of "Mon, 21 Feb 2000 22:15:18 +0100." <20000221221518.A17481@miss-janet.com> Message-ID: <200002212126.PAA00266@entropy.uchicago.edu> In message <20000221221518.A17481@miss-janet.com>, Ricardo Kustner writes: >On Mon, Feb 21, 2000 at 01:56:52PM -0600, Ted Cabeen wrote: >> I don't know if this has been fixed in the CVS version, but I thought I'd >> submit it anyways. There seems to be a small authentication bug with the >> chunking in the membership management page. The page will load up correctly > >> the first time it is accessed with the first chunk size shown. However, whe >n >> you request a different chunk, it requires a reauthentication and then gives > >> the chunk=0 page instead of the requested chunk. If you then click on the >> alternate chunk it loads fine. However, the additional click and >> authentication is annoying. > >I have/had the same problem, and i'm using the CVS version... > >I just had a closer look and now I know why it behaves this way in my case: >our list is accessible at two hostnames which share the exact same website... >the hostname that is linked at the membership chunk links is different >than the hostname i sometimes use to access the list... so netscape thinks >the cookie is for a different host... > >which makes me wonder why the chunk links on the admin page have a full >http:// link while other links on it don't ? > >maybe you are experiencing the same problem? compare the hostname you >use to access the admin page with the hostname that's being used in the >links to the membership chunks... if they are different, that's the >reason why you have to re-enter the password. You're right. However, that doesn't change the bug. The bug is as follows: When you try to access the admin pages with a hostname that you haven't used before, it asks you to reauthenticate for that hostname. This includes when you manually type in a shorter address for the machine(http://lists/ instead of http://lists.uchicago.edu/) That isn't really a bug. Reauthentication for different hostnames is understandable. However, when you are asking for a specific chunk of the membership list and you have to reauthenticate, it ignores your chunk request and always gives chunk 0. Is there a passing problem in the authentication module? -- Ted Cabeen http://www.pobox.com/~secabeen secabeen@pobox.com Check Website or finger for PGP Public Key secabeen@midway.uchicago.edu "I have taken all knowledge to be my province." -F. Bacon cococabeen@aol.com "Human kind cannot bear very much reality."-T.S.Eliot 73126.626@compuserve.com From ricardo@rixhq.nu Mon Feb 21 21:49:05 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Mon, 21 Feb 2000 22:49:05 +0100 Subject: [Mailman-Developers] Membership management chunking requires two authentications.. In-Reply-To: <200002212126.PAA00266@entropy.uchicago.edu>; from secabeen@pobox.com on Mon, Feb 21, 2000 at 03:26:48PM -0600 References: <20000221221518.A17481@miss-janet.com> <200002212126.PAA00266@entropy.uchicago.edu> Message-ID: <20000221224905.B17481@miss-janet.com> On Mon, Feb 21, 2000 at 03:26:48PM -0600, Ted Cabeen wrote: > different hostnames is understandable. However, when you are asking for a > specific chunk of the membership list and you have to reauthenticate, it > ignores your chunk request and always gives chunk 0. Is there a passing > problem in the authentication module? ah i see... well it's obvious the chunk gets losts, since the reauthenticate page gives you a form and it doesn't include the chunk (which was posted with a GET at first) after posting the entered password... a solution could be to include the chunk var into a hidden field in the (re)authenticate form... Ricardo. -- From secabeen@pobox.com Mon Feb 21 21:52:57 2000 From: secabeen@pobox.com (Ted Cabeen) Date: Mon, 21 Feb 2000 15:52:57 -0600 Subject: [Mailman-Developers] Wishlist items... Message-ID: <200002212152.PAA00571@entropy.uchicago.edu> Three wishlist items, one of which I've coded (but am not sure will work on CVS) and one I haven't. I'd like to submit these to the bug tracking system, but there doesn't seem to be an email submit address. Does anyone know what it is? First: Add an option in mm_cfg.py and Defaults.py to turn archiving on and off. I like to be able to have archiving off for the majority of my mailing lists unless the owner explicitly wants it. (I coded this one up in 5 minutes, but I'm still on 1.1, so I don't know if it will port. I'll send the diffs if anyone wants them) Second: Currently, the senddigests script sends a digest for every list that has digesting on and has any content to send. However, this means that on a medium traffic list, it's possible to send out a digest of one message, if the digest size was reached two messages ago. What I'd like to see is the digester only send out a time-based digest if no digest had been sent in some finite amount of time instead of every time the cron job is run. Third: Currently the digester never increments its volume number. There is no code in Mailman to increase the volume number. It it set to 1 in the InitVars function, and is never changed. Volume incrementing should probably be an option on the digest page. At the least, we should change the volume number yearly. Thanks. -- Ted Cabeen http://www.pobox.com/~secabeen secabeen@pobox.com Check Website or finger for PGP Public Key secabeen@midway.uchicago.edu "I have taken all knowledge to be my province." -F. Bacon cococabeen@aol.com "Human kind cannot bear very much reality."-T.S.Eliot 73126.626@compuserve.com From jeff@popmail.med.nyu.edu Tue Feb 22 17:47:23 2000 From: jeff@popmail.med.nyu.edu (Jeff Berliner) Date: Tue, 22 Feb 2000 12:47:23 -0500 Subject: [Mailman-Developers] small bug in bin/arch Message-ID: <1073583.3160212443@antwerp.med.nyu.edu> Sorry if this been fixed already in the 1.2 tree, but I can't access CVS from here. I searched the archives and didn't see anything. With a stock MM 1.1 install, line 77 of $prefix/bin/arch has an error. If in that block of code, a MMUnknownListError exception is raised, (for instance, if the config.db permissions are incorrect), arch actually dies with a NameError, and obviously doesn't report the correct problem. The solution is to add 'Errors.' to the name of the exception to catch. I guess this was just an oversight. *** ../src/mailman-1.1/bin/arch Sat Aug 21 14:21:44 1999 --- arch Tue Feb 22 10:13:33 2000 *************** *** 74,80 **** listname, mbox = sys.argv[1:] try: mlist = MailList(listname) ! except MMUnknownListError: usage(2, 'no such list: ' + listname) # lay claim to the archive's lock file. this is so no other post can --- 74,80 ---- listname, mbox = sys.argv[1:] try: mlist = MailList(listname) ! except Errors.MMUnknownListError: usage(2, 'no such list: ' + listname) # lay claim to the archive's lock file. this is so no other post can - Jeff -- Jeff Berliner jeff@popmail.med.nyu.edu Academic Computing, Phone: (212) 263-2051 New York University School of Medicine Fax: (212) 263-8542 From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 22 18:38:20 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 22 Feb 2000 13:38:20 -0500 (EST) Subject: [Mailman-Developers] small bug in bin/arch References: <1073583.3160212443@antwerp.med.nyu.edu> Message-ID: <14514.55196.6586.851416@anthem.cnri.reston.va.us> >>>>> "JB" == Jeff Berliner writes: JB> Sorry if this been fixed already in the 1.2 tree, but JB> I can't access CVS from here. I searched the archives and JB> didn't see anything. Remember, you can always access it via the Web. Check out cvs.python.org Anyway, I just applied this patch. Thanks! -Barry From ricardo@rixhq.nu Tue Feb 22 21:01:10 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Tue, 22 Feb 2000 22:01:10 +0100 Subject: [Mailman-Developers] nondelivery report to list Message-ID: <20000222220110.A19994@miss-janet.com> i just saw a non-delivery report (full mailbox) from a postmaster arrive on the list address... that's kinda odd... probably a really broken MTA? I haven't seen this happen before since i'm using Mailman... the MTA seems to be Microsoft Exchange: >Received: from mailincoming5.collegeclub.com ([10.10.5.85]) by mailoutgoing1.collegeclub.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) > id DZ3T7A17; Tue, 22 Feb 2000 02:50:44 -0800 >Received: from mail pickup service by mailincoming5.collegeclub.com with Microsoft SMTPSVC; > Tue, 22 Feb 2000 02:54:28 -0800 Ricardo. -- From gorgo@sztaki.hu Tue Feb 22 23:37:17 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 00:37:17 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway Message-ID: Hello, I've fixed now the MODE reader problems in the debian package of mailman (using the cvs nntplib.py in Mailman/pythonlib)... And now I was wondering... what happens if the newserver does not accept connections when mailman wants to gateway a post from mail to news... is there a queueing mechanism for nntp just like there is for smtp in case the MTA is down ? -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Feb 23 15:43:34 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 23 Feb 2000 10:43:34 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: Message-ID: <14516.38.89336.568349@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> I've fixed now the MODE reader problems in the debian package GM> of mailman (using the cvs nntplib.py in Mailman/pythonlib)... GM> And now I was wondering... what happens if the newserver does GM> not accept connections when mailman wants to gateway a post GM> from mail to news... is there a queueing mechanism for nntp GM> just like there is for smtp in case the MTA is down ? Nope. From gorgo@sztaki.hu Wed Feb 23 15:46:09 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 16:46:09 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <14516.38.89336.568349@anthem.cnri.reston.va.us> Message-ID: On Wed, 23 Feb 2000, Barry A. Warsaw wrote: > GM> I've fixed now the MODE reader problems in the debian package > GM> of mailman (using the cvs nntplib.py in Mailman/pythonlib)... > GM> And now I was wondering... what happens if the newserver does > GM> not accept connections when mailman wants to gateway a post > GM> from mail to news... is there a queueing mechanism for nntp > GM> just like there is for smtp in case the MTA is down ? > > Nope. Ahm... perhaps the queue should be expanded to handle the NNTP case too then ? :) We don't want mails get lost on their way to newsgroups... :) -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us Wed Feb 23 16:15:16 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 23 Feb 2000 11:15:16 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <14516.38.89336.568349@anthem.cnri.reston.va.us> Message-ID: <14516.1940.326856.375284@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> Ahm... perhaps the queue should be expanded to handle the NNTP GM> case too then ? :) We don't want mails get lost on their way GM> to newsgroups... :) Except that Mailman's mail queuing is gone too now in the current CVS snapshot. The 1.1 implementation caused a terrible performance hit so I ripped it out, figuring a well configured MTA should be able to do that job, and do it better. It could be added back, but I haven't found a need. If someone wants to re-implement this against the current architecture, then I agree that nntp queuing should be part of the picture. -Barry From gorgo@sztaki.hu Wed Feb 23 16:20:53 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 17:20:53 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <14516.1940.326856.375284@anthem.cnri.reston.va.us> Message-ID: On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote: > >>>>> "GM" == Gergely Madarasz writes: > > GM> Ahm... perhaps the queue should be expanded to handle the NNTP > GM> case too then ? :) We don't want mails get lost on their way > GM> to newsgroups... :) > > Except that Mailman's mail queuing is gone too now in the current CVS > snapshot. The 1.1 implementation caused a terrible performance hit so > I ripped it out, figuring a well configured MTA should be able to do > that job, and do it better. It could be added back, but I haven't > found a need. If someone wants to re-implement this against the > current architecture, then I agree that nntp queuing should be part of > the picture. Ah, indeed, now mailman calls the MTA directly, and no smtp on port 25, right ? Perhaps there is a way to do something like that with the news server, so no need to talk nntp... with inn rnews seems to be the program to call, I don't know about other nntpservers... -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us Wed Feb 23 18:06:17 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 23 Feb 2000 13:06:17 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <14516.1940.326856.375284@anthem.cnri.reston.va.us> Message-ID: <14516.8601.995835.332168@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> Ah, indeed, now mailman calls the MTA directly, and no smtp on GM> port 25, right ? Not quite. There are actually two interchangable modules which do delivery. Sendmail.py expects sendmail command line interface and uses os.popen() to invoke a subprocess. SMTPDirect.py opens port 25 and blasts the message to the MTA. The latter does no queuing because it is my observation that 1) assuming the SMTP is running correctly, and 2) you are not doing DSN, the SMTP delivery happens asynchronously so still only be notified of errors by the bounce mechanism. 1.1 had all the queuing stuff because with DSN enabled (for those MTAs that support this ESMTP feature), delivery was synchronous and Mailman had to deal with exceptions instead of letting the MTA do it. -Barry From gorgo@sztaki.hu Wed Feb 23 18:09:34 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 19:09:34 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <14516.8601.995835.332168@anthem.cnri.reston.va.us> Message-ID: On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote: > >>>>> "GM" == Gergely Madarasz writes: > > GM> Ah, indeed, now mailman calls the MTA directly, and no smtp on > GM> port 25, right ? > > Not quite. There are actually two interchangable modules which do > delivery. Sendmail.py expects sendmail command line interface and > uses os.popen() to invoke a subprocess. SMTPDirect.py opens port 25 > and blasts the message to the MTA. The latter does no queuing because > it is my observation that 1) assuming the SMTP is running correctly, > and 2) you are not doing DSN, the SMTP delivery happens asynchronously > so still only be notified of errors by the bounce mechanism. 1.1 had > all the queuing stuff because with DSN enabled (for those MTAs that > support this ESMTP feature), delivery was synchronous and Mailman had > to deal with exceptions instead of letting the MTA do it. But it happened quite often that mailman couldn't connect to port 25 because sendmail wasn't accepting connections due to high load (loads above 25 are usual on this machine...), so queueing was needed anyway... -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us Wed Feb 23 18:15:35 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 23 Feb 2000 13:15:35 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <14516.8601.995835.332168@anthem.cnri.reston.va.us> Message-ID: <14516.9159.596407.361870@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> But it happened quite often that mailman couldn't connect to GM> port 25 because sendmail wasn't accepting connections due to GM> high load (loads above 25 are usual on this machine...), so GM> queueing was needed anyway... One possible solution is to use a better MTA (i.e. not sendmail). Also, it's very possible that all that extra load was /caused/ by Mailman 1.1's queuing system. Some form of queuing may still be necessary, but the previous implementation did not work. I don't have time to implement a new system myself but if someone wants to work on it, let me know. -Barry From claw@kanga.nu Wed Feb 23 18:20:38 2000 From: claw@kanga.nu (J C Lawrence) Date: Wed, 23 Feb 2000 10:20:38 -0800 Subject: [Mailman-Developers] mail/news gateway In-Reply-To: Message from bwarsaw@cnri.reston.va.us of "Wed, 23 Feb 2000 13:15:35 EST." <14516.9159.596407.361870@anthem.cnri.reston.va.us> References: <14516.8601.995835.332168@anthem.cnri.reston.va.us> <14516.9159.596407.361870@anthem.cnri.reston.va.us> Message-ID: <10862.951330038@kanga.nu> On Wed, 23 Feb 2000 13:15:35 -0500 (EST) bwarsaw wrote: >>>>>> "GM" == Gergely Madarasz writes: GM> But it happened quite often that mailman couldn't connect to GM> port 25 because sendmail wasn't accepting connections due to GM> high load (loads above 25 are usual on this machine...), so GM> queueing was needed anyway... > One possible solution is to use a better MTA (i.e. not sendmail). > Also, it's very possible that all that extra load was /caused/ by > Mailman 1.1's queuing system. I potentially have the same problem -- the system occassionally gets very high load spikes, NOT due to MailMan, and Exim is configured to refuse connections when load gets too high. > Some form of queuing may still be necessary... If the new system also allows mail to be delivered to the local MTA by handingit off to a sendmail executable, there should be little to no problem. I forget the new design tho. -- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From gorgo@sztaki.hu Wed Feb 23 18:46:26 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 19:46:26 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <14516.9159.596407.361870@anthem.cnri.reston.va.us> Message-ID: On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote: > >>>>> "GM" == Gergely Madarasz writes: > > GM> But it happened quite often that mailman couldn't connect to > GM> port 25 because sendmail wasn't accepting connections due to > GM> high load (loads above 25 are usual on this machine...), so > GM> queueing was needed anyway... > > One possible solution is to use a better MTA (i.e. not sendmail). Sorry, but I can't accept this answer :) It seems quite normal for _any_ MTA to refuse connections when the load is high (protect the machine from even higher loads), and it is quite normal for machines to have high load... you don't want to say "don't use mailman on machines wich have load above 5", do you ? :) I use it, and it works nicely :) > Also, it's very possible that all that extra load was /caused/ by > Mailman 1.1's queuing system. Well... at least it took part of it. I'm really looking forward to the 1.2 release to test the performance :) > Some form of queuing may still be necessary, but the previous > implementation did not work. I don't have time to implement a new > system myself but if someone wants to work on it, let me know. I don't really see the need of a new smtp queueing system if there is the possibility of executing the MTA directly, so I'm not asking anyone to do it now :) I'm just saying that queueing was _needed_ before, and it _worked_ (I was one of those who requested this feature when 1.0b4 lost almost all my digests, because of the high load early morning :)) So there is no problem now :) And at the beginning of this thread I just warned that the same might be needed for the nntp server too, I didn't want to go into the smtp stuff now :) -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From Nigel.Metheringham@VData.co.uk Thu Feb 24 09:41:28 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Thu, 24 Feb 2000 09:41:28 +0000 Subject: [Mailman-Developers] mail/news gateway In-Reply-To: Message from J C Lawrence of "Wed, 23 Feb 2000 10:20:38 PST." <10862.951330038@kanga.nu> Message-ID: claw@kanga.nu said: > I potentially have the same problem -- the system occassionally gets > very high load spikes, NOT due to MailMan, and Exim is configured to > refuse connections when load gets too high. A solution to this is to add your mailman host (which would typically be 127.0.0.1) into smtp_reserve_hosts and then set the related parameters such as smtp_accept_reserve, smtp_load_reserve and the various *_load* options so that incoming SMTP is always accepted from mailman (although the messages may be queued rather than immediately delivered). The upside of this is that it puts less load on the system than additional invocations of exim from the command line, and all your load policy is pretty much in one place. Whichever method you use there may still be errors caused due to (for example) running out of spool space. [I'll try and work out a reasonable note on this for the mailman/exim howto information - alternatively I would just *love* it if someone would come up with some text for me :-) ] http://www.exim.org/howto/mailman.html Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From bwarsaw@cnri.reston.va.us Thu Feb 24 18:51:11 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Thu, 24 Feb 2000 13:51:11 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <10862.951330038@kanga.nu> Message-ID: <14517.32159.955431.115890@anthem.cnri.reston.va.us> >>>>> "NM" == Nigel Metheringham writes: NM> A solution to this is to add your mailman host (which would NM> typically be 127.0.0.1) into smtp_reserve_hosts and then set NM> the related parameters such as smtp_accept_reserve, NM> smtp_load_reserve and the various *_load* options so that NM> incoming SMTP is always accepted from mailman (although the NM> messages may be queued rather than immediately delivered). That would be fine. Anybody know of similar parameters for Postfix or other MTAs? It makes sense that MTAs should be able to treat localhost connections differently than remote connections. NM> The upside of this is that it puts less load on the system NM> than additional invocations of exim from the command line, and NM> all your load policy is pretty much in one place. Whichever NM> method you use there may still be errors caused due to (for NM> example) running out of spool space. And Mailman could have a bug, or Python could throw MemoryErrors, etc. Anyway, if Mailman can't successfully do the SMTP transaction, and it gets exceptions it doesn't know how to deal with, the exception should at least get logged. I know that doesn't make you feel great, and the situation should be improved. -Barry From ricardo@rixhq.nu Thu Feb 24 19:39:46 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Thu, 24 Feb 2000 20:39:46 +0100 Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <10862.951330038@kanga.nu>; from claw@kanga.nu on Wed, Feb 23, 2000 at 10:20:38AM -0800 References: <14516.8601.995835.332168@anthem.cnri.reston.va.us> <14516.9159.596407.361870@anthem.cnri.reston.va.us> <10862.951330038@kanga.nu> Message-ID: <20000224203946.A24852@miss-janet.com> On Wed, Feb 23, 2000 at 10:20:38AM -0800, J C Lawrence wrote: > On Wed, 23 Feb 2000 13:15:35 -0500 (EST) > bwarsaw wrote: > >>>>>> "GM" == Gergely Madarasz writes: > GM> But it happened quite often that mailman couldn't connect to > GM> port 25 because sendmail wasn't accepting connections due to > GM> high load (loads above 25 are usual on this machine...), so > GM> queueing was needed anyway... > > One possible solution is to use a better MTA (i.e. not sendmail). > > Also, it's very possible that all that extra load was /caused/ by > > Mailman 1.1's queuing system. > I potentially have the same problem -- the system occassionally gets > very high load spikes, NOT due to MailMan, and Exim is configured to > refuse connections when load gets too high. I got loads of up to 60 when i used exim with Mailman 1.1 ... and then slowly the system started to die ... this was happening when sending about 20 approved posts to 1500+ users. I configured exim not to send out anything when getting a higher load than 5, but then my fcgi scripts would die anyway during posting approvals... Since i'm using postfix with MM 1.2, loads don't get any higher than 2 or 3. So i'm very happy with the changes in 1.2 :) Ricardo. -- From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 24 20:16:10 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 24 Feb 2000 15:16:10 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <14516.8601.995835.332168@anthem.cnri.reston.va.us> <14516.9159.596407.361870@anthem.cnri.reston.va.us> <10862.951330038@kanga.nu> <20000224203946.A24852@miss-janet.com> Message-ID: <14517.37258.663839.652840@anthem.cnri.reston.va.us> >>>>> "RK" == Ricardo Kustner writes: RK> Since i'm using postfix with MM 1.2, loads don't get any RK> higher than 2 or 3. So i'm very happy with the changes in 1.2 RK> :) This is very good to hear! :) Who else is using any of the 1.2 CVS snapshots? -Barry From ricardo@rixhq.nu Thu Feb 24 21:14:14 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Thu, 24 Feb 2000 22:14:14 +0100 Subject: [Mailman-Developers] mailman development Message-ID: <20000224221414.C24852@miss-janet.com> Hi, I've been following mailman development since somewhere around april/may 1999. I was just reading back the archives and I remember that several times interesting discussions started on how it could be made easier for people to contribute to mailman development (cvs branches, bitkeeper and other things), but unfortunately there haven't been made much decissions about this so far (AFAIK)... I know the core developers are busy with other projects too, but the point of this email is that i'm hoping to bring this discussion back to life and that it results in some changes that will speed up development ... Also please have a look at this message Barry posted last year, and the thread that follows it... http://www.python.org/pipermail/mailman-developers/1999-October/001415.html Ricardo. -- From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 24 22:03:25 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 24 Feb 2000 17:03:25 -0500 (EST) Subject: [Mailman-Developers] mailman development References: <20000224221414.C24852@miss-janet.com> Message-ID: <14517.43693.320214.161856@anthem.cnri.reston.va.us> I'm still as busy as ever (maybe more so in fact!). I don't think BitKeeper is going to be the answer, but maybe moving Mailman to SourceForge will help. It's something I'm going to actively pursue after 1.2 is released. -Barry From jae@ilk.de Thu Feb 24 20:26:02 2000 From: jae@ilk.de (Juergen A. Erhard) Date: Thu, 24 Feb 2000 21:26:02 +0100 Subject: [Mailman-Developers] mailman, aliases and exim Message-ID: <24022000.5@sanctum.jae.ddns.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 There's a recipe for exim's configuration that makes it unnecessary to add aliases to /etc/aliases for every mailing list. list_transport: driver =3D pipe command =3D "/var/spool/smartlist/.bin/flist \ ${local_part}${local_part_suffix}" current_directory =3D /var/spool/smartlist home_directory =3D /var/spool/smartlist user =3D smartlist group =3D list Would/is something like this possible for mailman? This std aliases fragment would hint at needing to replace wrapper with a smart-wrapper or such that groks the list addresses (test, test-admin, etc) and calls wrapper as appropriate. ## test mailing list ## created: 27-Jan-2000 root test: "|/var/lib/mailman/mail/wrapper post test" test-admin: "|/var/lib/mailman/mail/wrapper mailowner test" test-request: "|/var/lib/mailman/mail/wrapper mailcmd test" owner-test: test-admin test-owner: test-admin Would this work? Comments? Bye, J PS: Need to get the Mailman source... as that wrapper ain't Python ;-) - -- = J=FCrgen A. Erhard eMail: jae@ilk.de phone: (GERMANY) 0721 27326 MARS: http://JuergenErhard.tripod.com/mars_index.html The GNU Project (http://www.gnu.org) Win32 has many known work arounds. For example, Linux. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.1 (GNU/Linux) Comment: Use Mailcrypt and GnuPG iEYEARECAAYFAji1k9kACgkQN0B+CS56qs2i1QCfdKXgC/3bvk9VCyDFWThx2Xus iWsAoIz+jUdl/12GImZTjTzlpKpeK4ha =3DJdHL -----END PGP SIGNATURE----- From jwt@dskk.co.jp Fri Feb 25 01:34:34 2000 From: jwt@dskk.co.jp (Jim Tittsler) Date: Fri, 25 Feb 2000 10:34:34 +0900 Subject: [Mailman-Developers] mailman, aliases and exim In-Reply-To: <24022000.5@sanctum.jae.ddns.org>; from jae@ilk.de on Thu, Feb 24, 2000 at 09:26:02PM +0100 References: <24022000.5@sanctum.jae.ddns.org> Message-ID: <20000225103434.A24396@mail.dskk.co.jp> On Thu, Feb 24, 2000 at 09:26:02PM +0100, Juergen A. Erhard wrote: > There's a recipe for exim's configuration that makes it unnecessary to > add aliases to /etc/aliases for every mailing list. > > list_transport: > driver = pipe > command = "/var/spool/smartlist/.bin/flist \ > ${local_part}${local_part_suffix}" [...] > Would/is something like this possible for mailman? This std aliases Yes, a similar scheme for mailman is described in the "HOWTO - Using exim and mailman together" that Nigel Metheringham put together: http://www.exim.org/howto/mailman.html From scottrus@raleigh.ibm.com Fri Feb 25 06:14:49 2000 From: scottrus@raleigh.ibm.com (Scott Russell) Date: Fri, 25 Feb 2000 01:14:49 -0500 Subject: [Mailman-Developers] sendmail exit 2 error on lists Message-ID: <20000225011449.F19733@bzimage.raleigh.ibm.com> All - I did make that migration from majordomo to Mailman 1.2 beta 1 (checked out of cvs on 02/23/00) and only ran into one issue. One of my users had the following email address while using majordomo: "Tim O'Flynn/DUB/xxxx" I used the bin/add_members script to import the list. This had no problems. The resulting email address in Mailman was: tim_o'flynn@xxxx.com The problem was sending mail to this list would result in the follwing error: post(32390): File "/home/mailman/scripts/post", line 86, in ? post(32390): main() post(32390): File "/home/mailman/scripts/post", line 62, in main post(32390): mlist.Post(msg) post(32390): File "/home/mailman/Mailman/MailList.py", line 1260, in Post post(32390): Mailman.Handlers.HandlerAPI.DeliverToList(self, msg) post(32390): File "/home/mailman/Mailman/Handlers/HandlerAPI.py", line 55, in DeliverToList post(32390): func(mlist, msg) post(32390): File "/home/mailman/Mailman/Handlers/Sendmail.py", line 88, in process post(32390): raise SendmailHandlerError(errcode) post(32390): Mailman.Handlers.Sendmail . SendmailHandlerError : 2 Also, a MAILER-DEAMON error was sent to the sender whenever they posted to the list. The error showed the above trace back and that a shell error had occured while processing the mail. Any thoughts on how to solve this? It should be easy to replicate. If someone writes a patch and it get's commited to cvs I'll update my copy and confirm the patch works. BTW, once again, I know I've said it before, THANK YOU FOR AN OUTSTANDING PRODUCT. Oh, I'm using postfix as my mail agaent, btw. Not sure if that mattered in the whole thing. -- Regards, Scott Russell (scottrus@raleigh.ibm.com) Linux Technology Center, System Admin Red Hat Certified Engineer From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Sat Feb 26 21:26:25 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Sat, 26 Feb 2000 16:26:25 -0500 (EST) Subject: [Mailman-Developers] sendmail exit 2 error on lists References: <20000225011449.F19733@bzimage.raleigh.ibm.com> Message-ID: <14520.17665.977202.913499@anthem.cnri.reston.va.us> >>>>> "SR" == Scott Russell writes: SR> I did make that migration from majordomo to Mailman 1.2 beta 1 SR> (checked out of cvs on 02/23/00) and only ran into one SR> issue. One of my users had the following email address while SR> using majordomo: SR> "Tim O'Flynn/DUB/xxxx" SR> I used the bin/add_members script to import the list. This had SR> no problems. The resulting email address in Mailman was: SR> tim_o'flynn@xxxx.com SR> The problem was sending mail to this list would result in the SR> follwing error: This is probably due to a shell escape problem with that single quote. The Sendmail.py delivery module uses os.popen() to invoke an external sendmail process, which goes through the shell. I'd suggest trying to switch to the SMTPDirect delivery module, which should avoid this problem. Add this line to your Mailman/mm_cfg.py file: DELIVERY_MODULE = 'SMTPDirect' You may need to change SMTPHOST and SMTPPORT; see Mailman/Defaults.py.in for details. I'm going to make SMTPDirect the default for the 1.2 release. -Barry From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 1 00:48:08 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 31 Jan 2000 19:48:08 -0500 (EST) Subject: [Mailman-Developers] SourceForge Message-ID: <14486.11592.419220.75729@anthem.cnri.reston.va.us> What would you folks think about moving the CVS repository for Mailman over to SourceForge.net? They even /use/ Mailman over there. Our CVS repository is running on a pokey Sparc5 and getting others write access to the tree is a pain (not that anybody's clamoring for that). I like the idea of SourceForge and we had a bunch of people raving about it at the Python conference last week. Anybody have any negative experiences with it? -Barry From Nigel.Metheringham@vdata.co.uk Tue Feb 1 13:36:46 2000 From: Nigel.Metheringham@vdata.co.uk (Nigel Metheringham) Date: Tue, 01 Feb 2000 13:36:46 +0000 Subject: [Mailman-Developers] Bugs in weekly archiving - mailman 1.1? Message-ID: I have a problem that has shown up since I changed to mailman 1.1 (from 1.0). Coincidently the platform the system ran on was upgraded to RH 6.1 linux (this was all due to a hardware failure) - I think this upgraded python and glibc. Heavy traffic lists I archive on a weekly basis. At the start of each month it produces invalid datestamps, ie today Week-of-Mon-20000200 Week-of-Mon-200001-4 Week-of-Mon-199912-1 In the source, Mailman/Archiver/HyperArch.py there is the code:- elif self.ARCHIVE_PERIOD == 'week': datetuple = list(datetuple) datetuple[2] = datetuple[2] - datetuple[6] # subtract week day # # even if the the day of the month counter is negative, # we still get the right thing from strftime! -scott # return time.strftime("Week-of-Mon-%Y%m%d", tuple(datetuple)) I've test bedded this code under perl (which I can write rather better) and the system/library strftime() behaves as "scott" says - so putting in the -4th of Jan 2000 gives strftime output of 27th Dec 1999. I'm starting to wonder if python put a strftime implementation into its library functions which does not match these assumptions... although it does appear to link against one in the C libraries. All in all I'm a tad confused over what is causing this problem. I could work round this by changing that code to read datetuple=time.gmtime(date - (datetuple[6] * 86400)) which will always work - although its more costly on CPU. Its also working around a problem that should not be there :-) Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From jwing@pliantsystems.com Tue Feb 1 20:14:55 2000 From: jwing@pliantsystems.com (John Wingenbach) Date: Tue, 01 Feb 2000 15:14:55 -0500 Subject: [Mailman-Developers] script to modify arbitrary value in config.db Message-ID: <38973EBF.7C91C23D@pliantsystems.com> I created the following python script to modify an arbitrary value within the config.db file. However, it seems to not correctly work. The value gets changed albeit it doesn't register correctly in the web. I admit to not being a python scripter. :-) Where am I messed up? TIA, John #! /usr/bin/env /share/bin/python import marshal import sys def main(argv): file='/usr/local/mailman/lists/' + argv[1] + '/config.db' fp = open(file, 'r') dict = marshal.load(fp) fp.close(); oldval = dict[argv[2]] dict[argv[2]] = argv[3] print argv[2], ' was: >', oldval, '< now is: >', dict[argv[2]], '<' fp = open(file, 'w') marshal.dump(dict, fp) fp.close(); if __name__ == "__main__": raise SystemExit(main(sys.argv)) -- John C. Wingenbach Pliant Systems, Inc. Sr. Systems Administrator Work: (919) 405-4627 Fax: (919) 405-4544 From thomas@xs4all.net Wed Feb 2 13:00:42 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Feb 2000 14:00:42 +0100 Subject: [Mailman-Developers] race condition in locking ? Message-ID: <20000202140042.J19725@xs4all.nl> I've finally found the time to do some serious testing of mailman (that is, set up a mailman list and play with it ;) and I was hit by what looks like a race condition in the locking. I was sending a lot of small mails to the mailman-test list, using echo "foobar foobar baz baz baz" | mail -s "foobar" mailman-test@xs4all.nl and just manually hitting arrow-up, enter, arrow-up, enter, etc. I'd subscribed a bouncing email address and wanted to see what would happen ;) In any case, the machine mailman runs on is also used for some other things; majordomo and our 'batched-smtp' product. Both services generate very variable loads, majordomo because of some very large lists, and batched-smtp due to customers with very large mailqueues dialing in and starting a queuerun. I dont send the mail from the same machine, so the mails I sent probably arrived in one sendmail session, and then were executed by the queueing sendmail right after another. Some of the messages (around 10/15 per incident) I'd sent then bounced, most with the following traceback: Traceback (innermost last): File "/usr/local/mailman/scripts/mailowner", line 69, in ? main() File "/usr/local/mailman/scripts/mailowner", line 48, in main mlist = MailList.MailList(sys.argv[1]) File "/usr/local/mailman/Mailman/MailList.py", line 69, in __init__ self.Load() File "/usr/local/mailman/Mailman/MailList.py", line 824, in Load self.Lock() File "/usr/local/mailman/Mailman/MailList.py", line 1255, in Lock self.__lock.lock() File "/usr/local/mailman/Mailman/LockFile.py", line 209, in lock os.link(self.__lockfile, self.__tmpfname) OSError: [Errno 2] No such file or directory And a couple with the following traceback: Traceback (innermost last): File "/usr/local/mailman/scripts/post", line 86, in ? main() File "/usr/local/mailman/scripts/post", line 62, in main mlist.Post(msg) File "/usr/local/mailman/Mailman/MailList.py", line 1249, in Post self.Save() File "/usr/local/mailman/Mailman/MailList.py", line 814, in Save Utils.reraise() File "/usr/local/mailman/Mailman/MailList.py", line 811, in Save os.link(fname, fname_last) OSError: [Errno 17] File exists The first traceback is probably because one process does a kickstart, erasing the lockfile and starting a new one, while a lot of other processes are trying to link() to it. The second traceback looks like a lock was stolen from an active process, in other words a real locking failure. The lock-logfile suggests some inefficient locking, too; long pauses between unlocks and locks: (18502) mailman-test.lock waiting for claim (18492) mailman-test.lock unlocked (18513) mailman-test.lock waiting for claim (18584) mailman-test.lock waiting for claim Feb 01 18:53:08 2000 (18622) mailman-test.lock kickstarted (18425) mailman-test.lock waiting for claim (18504) mailman-test.lock waiting for claim (18456) mailman-test.lock waiting for claim (18440) mailman-test.lock waiting for claim (18622) mailman-test.lock waiting for claim (18467) mailman-test.lock waiting for claim (18584) mailman-test.lock waiting for claim (18443) mailman-test.lock waiting for claim (18500) mailman-test.lock waiting for claim (18502) mailman-test.lock waiting for claim (18513) mailman-test.lock waiting for claim (18526) mailman-test.lock waiting for claim (18504) mailman-test.lock waiting for claim (18440) mailman-test.lock waiting for claim (18425) mailman-test.lock waiting for claim (18456) mailman-test.lock waiting for claim (18622) mailman-test.lock waiting for claim (18443) mailman-test.lock waiting for claim (18513) mailman-test.lock waiting for claim (18467) mailman-test.lock waiting for claim (18584) mailman-test.lock waiting for claim (18500) mailman-test.lock waiting for claim (18502) mailman-test.lock waiting for claim (18526) mailman-test.lock waiting for claim (18440) mailman-test.lock waiting for claim (18504) mailman-test.lock waiting for claim (18425) mailman-test.lock waiting for claim (18456) mailman-test.lock waiting for claim (18622) mailman-test.lock waiting for claim (18443) mailman-test.lock got the lock (This is with the default sleep time of .25) This all led me to believe there was something wrong with locking, and I spent the mornings' traintrip from home to work (1.5 hours) reading LockFile.py and thinking about the locking. And I think the logic of LockFile.lock() is flawed :P lock() seems to rely on two things: os.stat() returning the right number of links for the lockfile (which is a fairly safe assumption) and the contents of the lockfile, which should give an indication as to who locked it and for how long. But there is at least one situation possible where one process thinks it has the lock, and another thinks the lock is faulty and does a forced kickstart: process 1 process 2 os.link(lockfile,tmpfile) os.stat(lockfile) == 2 os.link(lockfile, tmpfile) os.stat(lockfile) == 3 self.__write(): open(tmpfile, "w") self.__read() write(os.getpid(), etc) __kickstart(force=1) return I think I was able to reproduce this on my laptop by stressing it fairly hard (load of 15 and rising.) The testcode only acquired and released locks, with some small sleep inbetween, but I saw several double-locks, and one OSError on the os.link(). I also think I see a way to fix this problem, by checking the mtime of the file if the __read() fails. I think we can safely say that if the __read() fails AND the mtime is recent, someone else is either assuming they have the lock, or is kickstart()ing, in both of which cases we should step back and start over. Comments ? Am I trespassing on anyones' work here ? Is this part of mailman still being looked at ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Nigel.Metheringham@vdata.co.uk Wed Feb 2 18:10:20 2000 From: Nigel.Metheringham@vdata.co.uk (Nigel Metheringham) Date: Wed, 02 Feb 2000 18:10:20 +0000 Subject: [Mailman-Developers] Bugs in weekly archiving - mailman 1.1? In-Reply-To: Message from Nigel Metheringham of "Tue, 01 Feb 2000 13:36:46 GMT." Message-ID: This is a multipart MIME message. --==_Exmh_-12073758350 Content-Type: text/plain; charset=us-ascii [copied into mailman-users since the followup question is more appropriate there... please keep discussion onto users from now on] I've built a patch to fix the weekly archive bug I see in mailman 1.1 - this fixes the problem of abusing strftime and hoping it works (it does mostly but I have exceptions). Anyone know a neat way of rearchiving the misarchived messages (I can do this by manually pulling them out the mbox archive) and losing the broken week archives - ie I currently have a Week-of-Mon-20000200 which is no use to man nor beast, but needs more than just a subtle delete to get rid of... Nigel. --==_Exmh_-12073758350 Content-Type: application/x-patch ; name="mm.patch" Content-Description: mm.patch Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="mm.patch" LS0tIE1haWxtYW4vQXJjaGl2ZXIvSHlwZXJBcmNoLnB5Lm9sZAlXZWQgRmViICAyIDE3OjQy OjE1IDIwMDAKKysrIE1haWxtYW4vQXJjaGl2ZXIvSHlwZXJBcmNoLnB5CVdlZCBGZWIgIDIg MTc6NTE6MTUgMjAwMApAQCAtNjU3LDkgKzY1NywxNyBAQAogCSAgICBkYXRldHVwbGUgPSBs aXN0KGRhdGV0dXBsZSkKIAkgICAgZGF0ZXR1cGxlWzJdID0gZGF0ZXR1cGxlWzJdIC0gZGF0 ZXR1cGxlWzZdICMgc3VidHJhY3Qgd2VlayBkYXkKIAkgICAgIwotCSAgICAjIGV2ZW4gaWYg dGhlIHRoZSBkYXkgb2YgdGhlIG1vbnRoIGNvdW50ZXIgaXMgbmVnYXRpdmUsCi0JICAgICMg d2Ugc3RpbGwgZ2V0IHRoZSByaWdodCB0aGluZyBmcm9tIHN0cmZ0aW1lISAtc2NvdHQKKwkg ICAgIyBpZiB0aGUgdGhlIGRheSBvZiB0aGUgbW9udGggY291bnRlciBpcyBuZWdhdGl2ZQor CSAgICAjIHdlIGhhdmUgdG8gcmVjYWxjdWxhdGUgdGhlIHdob2xlIHRoaW5ncyBzaW5jZQor ICAgICAgICAgICAgIyBoYW5kbGluZyBjcm9zc2luZyBiYWNrIGJ5IG1vbnRoIG9yIGV2ZW4g eWVhcgorICAgICAgICAgICAgIyBpcyB0b28gaGFyZC4KKyAgICAgICAgICAgICMgVGhlcmUg aXMgdGhlb3JldGljYWxseSBhbiBpc3N1ZSB3aXRoIGNyb3NzaW5nCisgICAgICAgICAgICAj IGJhY2sgYSBEU1QgY2hhbmdlb3ZlciBib3VuZHJ5IGhlcmUgYXMgd2VsbCAtIHRoaXMKKyAg ICAgICAgICAgICMgc2hvdWxkIGJlIGhhbmRsZWQgYnkgZ210aW1lLCBidXQgSSBoYXZlIHNl ZW4KKyAgICAgICAgICAgICMgbmFzdHkgaW50ZXJhY3Rpb25zIHdpdGggc3RyZnRpbWUgYW5k IGxvY2FsZXMKIAkgICAgIworICAgICAgICAgICAgaWYgZGF0ZXR1cGxlWzJdIDw9IDA6Cisg ICAgICAgICAgICAgICAgZGF0ZXR1cGxlID0gdGltZS5nbXRpbWUoZGF0ZSAtIChkYXRldHVw bGVbNl0gKiA4NjQwMCkpCiAJICAgIHJldHVybiB0aW1lLnN0cmZ0aW1lKCJXZWVrLW9mLU1v bi0lWSVtJWQiLCB0dXBsZShkYXRldHVwbGUpKQogICAgICAgICAjIG1vbnRoLiAtZGRtCiAg CWVsc2U6Cg== --==_Exmh_-12073758350 Content-Type: text/plain; charset=us-ascii [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] --==_Exmh_-12073758350-- From thomas@xs4all.net Wed Feb 2 21:33:19 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Feb 2000 22:33:19 +0100 Subject: [Mailman-Developers] SourceForge In-Reply-To: <14486.11592.419220.75729@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Mon, Jan 31, 2000 at 07:48:08PM -0500 References: <14486.11592.419220.75729@anthem.cnri.reston.va.us> Message-ID: <20000202223319.A17313@xs4all.nl> On Mon, Jan 31, 2000 at 07:48:08PM -0500, Barry A. Warsaw wrote: > What would you folks think about moving the CVS repository for Mailman > over to SourceForge.net? They even /use/ Mailman over there. Our CVS > repository is running on a pokey Sparc5 and getting others write > access to the tree is a pain (not that anybody's clamoring for that). > I like the idea of SourceForge and we had a bunch of people raving > about it at the Python conference last week. Anybody have any > negative experiences with it? I haven't heard anything bad about SourceForge yet, but not that much goods either. I wasn't at IPC8 ;) But I'm planning to do some of the things on the mailman todo list (site-admin pages, for instance) and I'm already 'fixing' (from my point of view, see my other postings) the locking scheme, and I would like to know how to get them included in the CVS tree. I dont care wether it's CVS write access or simply a place to send cvs diffs, though ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Wed Feb 2 21:53:46 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Feb 2000 22:53:46 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000202140042.J19725@xs4all.nl>; from thomas@xs4all.net on Wed, Feb 02, 2000 at 02:00:42PM +0100 References: <20000202140042.J19725@xs4all.nl> Message-ID: <20000202225346.B17313@xs4all.nl> On Wed, Feb 02, 2000 at 02:00:42PM +0100, Thomas Wouters wrote: > This all led me to believe there was something wrong with locking, and I > spent the mornings' traintrip from home to work (1.5 hours) reading > LockFile.py and thinking about the locking. And I think the logic of > LockFile.lock() is flawed :P And today I figured out what exactly is flawed. The locking mechanism is backwards ! Maybe it's intended that way, it seems that the instructions for this type of locking is removed from 'the' linux open manpage (my rh6 boxes no longer have any talk about locking) so I can't double-check, but I doubt this is what it describes. We use a very similar locking scheme in our slightly adjusted mail.local (originally sendmails) which, if memory serves, was suggested by some manpage or other. But not Linux' open() manpage. In any case, we use it the other way 'round from mailman, and with great success. Mailman currently locks by creating a link from the lockfile to a private lockfile, and checking that the link-count is 2. If it isn't, it reads the file to see who is owning the lockfile, to check on their health etc. The problem is that, if a lot of processes are doing the same thing, you will very often have more than 2 links to the file. Doing some stresstesting with the default timeout values I saw as much as 25 concurrent links to the file, and because every mailman process uses the exact same sleep time, the chances of breaking out of it are not as large as they could be. The end result is a lot of processes trying to lock the file, which is unlocked the whole time, because the processes trying to grab it keep headbutting as they try to grab the lock. The way we do locking in mail.local (and this works at least C, on 4 different operating systems, over NFS ;) is that every process that tries to obtain the lock, creates a private lockfile, and then links from the private lockfile to the intended one. Each process can then very easily see wether they have the lock or not by stat()ing their _own_ lockfile and looking at the link count. Because link() will fail when the new file to link already exists, it is impossible to overwrite someone else's lock. And because the private lockfile is created in advance, it's also possible to write the lock-info mailman uses, into it _in_advance_, removing another race condition in the locking mechanism. (namely the code checking to see if the contents is valid, and if not, removing it.) I'm busy rewriting LockFile.py to use the new locking (actually, I've rewritten it, I've just not tested it yet ;) and I should be able to post it tomorrow... Unless anyone has objects to this locking method. Did I miss anything ? Is there anyone I should talk to before changing things, like the original authors ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Feb 2 22:20:37 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 2 Feb 2000 17:20:37 -0500 (EST) Subject: [Mailman-Developers] race condition in locking ? References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> Message-ID: <14488.44469.446995.168328@anthem.cnri.reston.va.us> >>>>> "TW" == Thomas Wouters writes: TW> I'm busy rewriting LockFile.py to use the new locking TW> (actually, I've rewritten it, I've just not tested it yet ;) TW> and I should be able to post it tomorrow... Unless anyone has TW> objects to this locking method. Did I miss anything ? Is there TW> anyone I should talk to before changing things, like the TW> original authors ? Thomas, I've only skimmed your emails but I think your analysis is basically correct, and the solution you propose here seems reasonable. This mailing list is the right place to discuss it, and although I've been too busy to respond lately, I /will/ see your message, and your changes. My suggestions would be these: be sure to look at the latest version of LockFile.py from the CVS repository. It doesn't fix your problems but it embodies the current semantics I want from the API. Second: if you submit substantial changes (as I suspect you will), we'll need to get FSF assignments from you. I'll send you the appropriate form when I see how big your changes are, but I wanted to give you a heads up. Thanks very much for looking into this. -Barry From thomas@xs4all.net Thu Feb 3 09:35:55 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 3 Feb 2000 10:35:55 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <14488.44469.446995.168328@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Wed, Feb 02, 2000 at 05:20:37PM -0500 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <14488.44469.446995.168328@anthem.cnri.reston.va.us> Message-ID: <20000203103555.C6378@xs4all.nl> --J2SCkAp4GZ/dPZZf Content-Type: text/plain; charset=us-ascii On Wed, Feb 02, 2000 at 05:20:37PM -0500, Barry A. Warsaw wrote: > I've only skimmed your emails but I think your analysis is basically > correct, and the solution you propose here seems reasonable. This > mailing list is the right place to discuss it, and although I've been > too busy to respond lately, I /will/ see your message, and your > changes. Okay, I figured you're a busy man, thanx for giving me a fast answer. > My suggestions would be these: be sure to look at the latest version > of LockFile.py from the CVS repository. It doesn't fix your problems > but it embodies the current semantics I want from the API. I was already using the CVS version, so no worries. > Second: if you submit substantial changes (as I suspect you will), > we'll need to get FSF assignments from you. I'll send you the > appropriate form when I see how big your changes are, but I wanted to > give you a heads up. Well, attached you'll find the first change, the cvs diff to LockFile.py. My apologies for this rather long email, but it's as much an explanation to this list about the changes I made, as a way for me to get this info out of my subconcious and into straight english. No need to read this all if you read the source and understand it, or if you dont care about locking ;) I kept the API the same with one teensy exception. With the new locking scheme, it's not possible to 'steal' a lock, only to break the lock and then enter the lock() cycle to try and grab it. So steal() might block indefinately (though this should only occur on very heavily loaded machines, with a lot of mailmen trying to grab the same lock.) I've also added some semi-random (more like 'not very regular') factor to the default sleep timeout, to slightly try to avoid running processes 'in stride'. In the previous locker, processes that were walking in stride would be guaranteed not to get the lock, and could only get out of it by irregularities in the python execution speed and/or OS speed. In the new locker collisions are much less harmful, but it's still nice to try and avoid running in stride. This version still has some debug info inserted, to test the timing of the locks and the like, and it could be speeded up a fair bit once we're satisfied that it works properly. Basically, how it works is like this: The locker creates a private lockfile (__tmpfname) containing pid, tmpfname and expected lifetime, and tries to make __lockfile a hard link to this file. This'll fail when __lockfile already exists, or when __tmpfname has been deleted (this shouldn't happen, though.) If the link() fails, the locker tries to read the contents of the lockfile, to find out who is locking it. If the read fails (the file does not exist, or the contents was of unexpected format) or the lock is too old, it tries to break the lock, using __break(). __break() checks __lockfile's CTIME, the inode change time, to see if it has changed recently -- which would indicate that the lock-owner is still alive, and that the lock contents might have changed since we last checked it. (CTIME is updated by writing to it, opening it for writing, and link()ing, but not by reading from it. We only ever write/link to our own lockfile, not the global one, so if there is write activity, it wasn't us. We do have to flush() or close() after write()ing, however, to make sure the changes get written out.) Note that __write() writes to the private lockfile, whereas __read() reads from the global one, and that there is a lot of redundancy in the lock-checking; ordinarily, the nlink-check should be enough to determine wether we have the lock or not. Also note that the locker no longer cleans up other mailmens lockfiles. If a mailman crashes and leaves its own lockfile, the file will not be deleted -- perhaps a default cronjob to clear old locks is in order, I'll think about that. In any case, old lockfiles shouldn't be a problem even if another mailman comes along on the same machine, with the same pid, trying to lock the same file, as it will simply use the file already there, and delete it when it's done. The other api functions (refresh, set_lifetime, unlock, locked) all should work exactly the same, only better -- in particular refresh() no longer exposes a race condition, thanks to the ctime check in __break(). And because the new locking scheme guarantees that the first attempt to get the lock after an unlock(), will succeed, the default sleep interval can be safely raised. I've set the default to 1 second, which is just above the average lock age on my listserver, given decent load. I've tested this locker quite heavily, mailbombing my listserver with 100+ simultaneous messages to mailman-test (which generated 100+ bounces sent back to mailman, because my bouncing email address is still on the list) and it hasn't barfed yet ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --J2SCkAp4GZ/dPZZf Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mailman.cvs.diff" Index: Mailman/LockFile.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/LockFile.py,v retrieving revision 1.13 diff -u -r1.13 LockFile.py --- LockFile.py 1999/12/14 04:22:05 1.13 +++ LockFile.py 2000/02/03 09:30:33 @@ -27,6 +27,8 @@ import errno #from stat import ST_NLINK ST_NLINK = 3 # faster +ST_CTIME = 9 +ST_INO = 1 # default intervals are both specified in seconds, and can be floating point @@ -36,7 +38,8 @@ # sleep before checking the lock's status, if we were not the winning claimant # the previous time around. DEFAULT_LOCK_LIFETIME = 15 -DEFAULT_SLEEP_INTERVAL = .25 +DEFAULT_SLEEP_INTERVAL = 1.0 +DEFAULT_INODE_GRACETIME = 2 from Mailman.Logging.StampedLogger import StampedLogger @@ -79,20 +82,21 @@ lifetime is the maximum length of time expected to keep this lock. This value is written into the lock file so that other claimants on - the lock know when it is safe to steal the lock, should the lock + the lock know when it is safe to break the lock, should the lock holder be wedged. - sleep_interval is how often to wake up and check the lock file + sleep_interval is roughly how often to wake up and check the lock file """ self.__lockfile = lockfile self.__lifetime = lifetime - self.__sleep_interval = sleep_interval + self.__sleep_interval = sleep_interval + (((os.getpid() % 20) - 11.5)/100.) self.__tmpfname = "%s.%s.%d" % (lockfile, socket.gethostname(), os.getpid()) self.__withlogging = withlogging - self.__kickstart() + self.__write() + self.__writelog("starting") def set_lifetime(self, lifetime): """Reset the lifetime of the lock. @@ -129,28 +133,6 @@ if self.locked(): self.unlock() - def __kickstart(self, force=0): - # forcing means to remove the original lock file, and create a new - # one. this might be necessary if the file contains bogus locker - # information such that the owner of the lock can't be determined - if force: - try: - os.unlink(self.__lockfile) - except IOError: - pass - if not os.path.exists(self.__lockfile): - try: - # make sure it's group writable - oldmask = os.umask(002) - try: - file = open(self.__lockfile, 'w+') - file.close() - finally: - os.umask(oldmask) - except IOError: - pass - self.__writelog('kickstarted') - def __write(self): # we expect to release our lock some time in the future. we want to # give other claimants some clue as to when we think we're going to be @@ -169,22 +151,38 @@ os.umask(oldmask) def __read(self): - # can raise ValueError in two situations: + # can raise ValueError in three situations: # - # either first element wasn't an integer (a valid pid), or we didn't - # get a sequence of the right size from the string.split. Either way, - # the data in the file is bogus, but this is caught and handled higher - # up. - fp = open(self.__tmpfname, 'r') + # - The lockfile does not exist (it's been released or broken) + # - We didn't get a sequence of the right size from split + # - the first 'word' is not an integer + # In each case we raise ValueError in response + try: + fp = open(self.__lockfile, 'r') + except IOError, err: + if err.errno == errno.ENOENT: + raise ValueError # invalid contents try: pid, winner, lockrelease = string.split(string.strip(fp.read())) finally: fp.close() return int(pid), winner, float(lockrelease) - # Note that no one new can grab the lock once we've opened our tmpfile - # until we close it, even if we don't have the lock. So checking the PID - # and stealing the lock are guaranteed to be atomic. + def __break(self): + try: + if os.stat(self.__lockfile)[ST_CTIME] > time.time() - DEFAULT_INODE_GRACETIME: + # The lockfile has changed very recently, so it might have + # changed since our caller evaluated the validity + return + except OSError, err: + if err.errno == errno.ENOENT: + return # Someone broke it already + raise + os.unlink(self.__lockfile) + + # Note that this function is mostly redundant, the try/except around + # os.link does almost all of what we do. Instead of lockfile-contents + # we could use lockfile CTIME, which would probably speed things up. def lock(self, timeout=0): """Blocks until the lock can be obtained. @@ -197,21 +195,42 @@ synchronized. """ - if timeout > 0: - timeout_time = time.time() + timeout - last_pid = -1 if self.locked(): self.__writelog('already locked') raise AlreadyLockedError - stolen = 0 + if timeout: + timeout_time = time.time() + timeout + last_pid = -1 + self.__write() # Make sure our own lockfile exists while 1: # create the hard link and test for exactly 2 links to the file - os.link(self.__lockfile, self.__tmpfname) + try: + os.link(self.__tmpfname, self.__lockfile) + except OSError, err: + if err.errno == errno.ENOENT: + # Our own lockfile has vanished from beneath us ? + self.__write() + # sleep anyway, so we dont hog the machine + self.__sleep() + continue + elif err.errno == errno.EEXIST: + # We failed to get the lock. If we worry about this locking + # scheme, we set a variable here and ascertain ourselves of + # its value later, when we think we have the lock. + pass + else: + raise + if os.stat(self.__tmpfname)[ST_NLINK] == 2: - # we have the lock (since there are no other links to the lock - # file), so we can piss on the hydrant + # we have the lock (since noone overwrote our link to the lock + # file), so we re-piss our hydrant, to refresh the lock timeout self.__write() - self.__writelog('got the lock') + now = time.time() + if not self.locked(): + self.__writelog('false positive on lockfile (%f)'%now) + self.__sleep() + continue + self.__writelog('got the lock (%f)'%now) break # we didn't get the lock this time. let's see if we timed out if timeout and timeout_time < time.time(): @@ -220,83 +239,34 @@ raise TimeOutError # someone else must have gotten the lock. let's find out who it # is. if there is some bogosity in the lock file's data then we - # will steal the lock. + # will try to steal the lock. try: pid, winner, lockrelease = self.__read() except ValueError: - os.unlink(self.__tmpfname) - self.__kickstart(force=1) + self.__break() + self.__sleep() continue - # If we've gotten to here, we should not be the winner, because - # otherwise, an AlreadyCalledLockError should have been raised - # above, and we should have never gotten into this loop. However, - # the following scenario can occur, and this is what the stolen - # flag takes care of: - # - # Say that processes A and B are already laying claim to the lock - # by creating link files, and say A actually has the lock (i.e., A - # is the winner). We are process C and we lay claim by creating a - # link file. All is cool, and we'll trip the pid <> last_pid test - # below, unlink our claim, sleep and try again. Second time - # through our loop, we again determine that A is the winner but - # because it and B are swapped out, we trip our lifetime test - # and figure we need to steal the lock. So we piss on the hydrant - # (write our info into the lock file), unlink A's link file and go - # around the loop again. However, because B is still laying - # claim, and we never knew it (since it wasn't the winner), we - # again have 3 links to the lock file the next time through this - # loop, and the assert will trip. - # - # The stolen flag alerts us that this has happened, but I still - # worry that our logic might be flawed here. - assert stolen or winner <> self.__tmpfname - # record the identity of the previous winner. lockrelease is the - # expected time that the winner will release the lock by. we - # don't want to steal it until this interval has passed, otherwise - # we could steal the lock out from underneath that process. + + # check for hanging processes, by remembering pid, and if it stays + # the same, checking lockrelease. if pid <> last_pid: last_pid = pid - # here's where we potentially steal the lock. if the pid in the - # lockfile hasn't changed by lockrelease (a fixed point in time), - # then we assume that the locker crashed + elif lockrelease < time.time(): - self.__write() # steal - self.__writelog('stolen!') - stolen = 1 - try: - os.unlink(winner) - except os.error: - # winner lockfile could be missing - pass - try: - os.unlink(self.__tmpfname) - except os.error, (code, msg): - # Let's say we stole the lock, but some other process's - # claim was never cleaned up, perhaps because it crashed - # before that could happen. The test for acquisition of - # the lock above will fail because there will be more than - # one hard link to the main lockfile. But we'll get here - # and winner==self.__tmpfname, so the unlink above will - # fail (we'll have deleted it twice). We could just steal - # the lock, but there's no reliable way to clean up the - # stale hard link, so we raise an exception instead and - # let the human operator take care of the problem. - if code == errno.ENOENT: - self.__log('stale lockfile found') - raise StaleLockFileError( - 'Stale lock file found linked to file: ' - +self.__lockfile+' (requires '+ - 'manual intervention)') - else: - raise + # We've waited long enough... + # Attempt to break the lock. We dont care much about the old + # winners __tmpfname, let it stay. + self.__break() + self.__sleep() continue - # okay, someone else has the lock, we didn't steal it, and our - # claim hasn't timed out yet. So let's wait a while for the owner - # of the lock to give it up. Unlink our claim to the lock and - # sleep for a while, then try again - os.unlink(self.__tmpfname) + # okay, someone else has the lock, we weren't able to steal it, + # and our claim hasn't timed out yet. So let's wait a while for + # the owner of the lock to give it up. self.__writelog('waiting for claim') - time.sleep(self.__sleep_interval) + self.__sleep() + + def __sleep(self): + time.sleep(self.__sleep_interval) # This could error if the lock is stolen. You must catch it. def unlock(self): @@ -308,24 +278,39 @@ """ if not self.locked(): raise NotLockedError + now = time.time() + os.unlink(self.__lockfile) os.unlink(self.__tmpfname) - self.__writelog('unlocked') + self.__writelog('unlocked (%f)'%now) def locked(self): """Returns 1 if we own the lock, 0 if we do not.""" if not os.path.exists(self.__tmpfname): return 0 + if os.stat(self.__tmpfname)[ST_NLINK] <> 2: + return 0 + # The above check for nlink should be enough to validate our claim on + # the lock. Nevertheless, we are paranoid. try: pid, winner, lockrelease = self.__read() except ValueError: - # the contents of the lock file was corrupted - os.unlink(self.__tmpfname) - self.__kickstart(force=1) - return 0 + # the contents of the lock file was corrupted, or the global lock + # is not a link to our lock. + if os.stat(self.__lockfile)[ST_INO] <> os.stat(self.__tmpfname)[ST_INO]: + self.__writelog("nlink of 2 but we don't own the lock ? strangeness") + os.unlink(self.__tmpfname) + self.__write() + return 0 + else: + # Okay, we really do have the lock, but the contents is messed up + self.__write() + return 1 return pid == os.getpid() # use with caution!!! def steal(self): """Explicitly steal the lock. USE WITH CAUTION!""" - self.__write() - self.__writelog('explicitly stolen') + self.__writelog('explicitly breaking') + self.__break() + self.lock() + --J2SCkAp4GZ/dPZZf-- From bbum@codefab.com Fri Feb 4 17:29:12 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Fri, 4 Feb 2000 12:29:12 -0500 Subject: [Mailman-Developers] integrating changes into distribution Message-ID: <200002041729.MAA29538@bjork.codefab.com> For the Mailman-decode-mime-file-via-webdav modifications, I have a relatively significant set of addtions and patches to Mailman. I would like to intregrate these into the core distribution. What is the best way to procede? Barry? b.bum From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Fri Feb 4 18:10:49 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 4 Feb 2000 13:10:49 -0500 (EST) Subject: [Mailman-Developers] integrating changes into distribution References: <200002041729.MAA29538@bjork.codefab.com> Message-ID: <14491.5673.280676.30360@anthem.cnri.reston.va.us> >>>>> "BB" == Bill Bumgarner writes: BB> For the Mailman-decode-mime-file-via-webdav modifications, I BB> have a relatively significant set of addtions and patches to BB> Mailman. BB> I would like to intregrate these into the core distribution. BB> What is the best way to procede? BB> Barry? I'd like to take a crack at integrating the i18n changes this weekend. If that goes smoothly, your stuff is next on my list. -barry From ricardo@rixhq.nu Fri Feb 4 19:56:30 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Fri, 4 Feb 2000 20:56:30 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000202225346.B17313@xs4all.nl>; from thomas@xs4all.net on Wed, Feb 02, 2000 at 10:53:46PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> Message-ID: <20000204205630.A4871@miss-janet.com> Hi, I haven't looked deeply into the MM locking code but i think this could also be the reason why MM (cvs) always crashes on my server when I approve about 10 posts at the same time... maybe it could be a timing thing which is more likely to happen if the load is quite high (which happens quite soon as my server isn't that powerfull :) ) Anyway i'm willing to test out any new changes to the locking code in my setup... Ricardo. -- From ricardo@rixhq.nu Fri Feb 4 20:22:43 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Fri, 4 Feb 2000 21:22:43 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000204205630.A4871@miss-janet.com>; from ricardo@rixhq.nu on Fri, Feb 04, 2000 at 08:56:30PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> Message-ID: <20000204212243.B4871@miss-janet.com> On Fri, Feb 04, 2000 at 08:56:30PM +0100, Ricardo Kustner wrote: > about 10 posts at the same time... maybe it could be a timing thing which is more > likely to happen if the load is quite high (which happens quite soon as my server > isn't that powerfull :) ) Anyway i'm willing to test out any new changes to the > locking code in my setup... well i patched my copy of MM with the diff Thomas had just posted, and then i tried to approve about 16 posts and I still got another assertion error :( maybe i just need to start saving some money for a more powerfull machine :) Ricardo. -- From thomas@xs4all.net Fri Feb 4 22:47:57 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 4 Feb 2000 23:47:57 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000204212243.B4871@miss-janet.com>; from ricardo@rixhq.nu on Fri, Feb 04, 2000 at 09:22:43PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> Message-ID: <20000204234757.A2826@xs4all.nl> On Fri, Feb 04, 2000 at 09:22:43PM +0100, Ricardo Kustner wrote: > On Fri, Feb 04, 2000 at 08:56:30PM +0100, Ricardo Kustner wrote: > > about 10 posts at the same time... maybe it could be a timing thing which is more > > likely to happen if the load is quite high (which happens quite soon as my server > > isn't that powerfull :) ) Anyway i'm willing to test out any new changes to the > > locking code in my setup... > > well i patched my copy of MM with the diff Thomas had just posted, and > then i tried to approve about 16 posts and I still got another assertion error :( > maybe i just need to start saving some money for a more powerfull machine :) Or perhaps you're experiencing an entirely differnet problem. Or a very related, but slightly different problem. Or perhaps this method of locking does not work on the type of machine, operating system and/or filesystem you are using. If you want, I can take a look at your AssertionError, see what caused it. If not, and it's a generic Mailman bug, I'll eventually hit it myself, anyway ;) Out of curiosity, the mailinglist you're running on mailman, that wouldn't happen to be the MissJanet mailing list that you used to run at XS4ALL, would it? ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From ricardo@rixhq.nu Sat Feb 5 00:15:07 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Sat, 5 Feb 2000 01:15:07 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000204234757.A2826@xs4all.nl>; from thomas@xs4all.net on Fri, Feb 04, 2000 at 11:47:57PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> Message-ID: <20000205011507.A405@miss-janet.com> Hi, On Fri, Feb 04, 2000 at 11:47:57PM +0100, Thomas Wouters wrote: > On Fri, Feb 04, 2000 at 09:22:43PM +0100, Ricardo Kustner wrote: > > well i patched my copy of MM with the diff Thomas had just posted, and > > then i tried to approve about 16 posts and I still got another assertion error :( > > > maybe i just need to start saving some money for a more powerfull machine :) > Or perhaps you're experiencing an entirely differnet problem. Or a very > related, but slightly different problem. Or perhaps this method of locking > does not work on the type of machine, operating system and/or filesystem you > are using. Linux 2.0.38, libc5 on a P100... not the newest stuff but it's stable :) usually i use newer kernels/libs for servers but I had to do so much work to optimize the apache/fastcgi/mysql stuff that I'm cautious with upgrading... I hope to hook up a second harddrive soon I can put the mail spool on a seperate disk which will hopefully reduce the load a bit... > If you want, I can take a look at your AssertionError, see what > caused it. If not, and it's a generic Mailman bug, I'll eventually hit it It always happens on line 60 in Mailman/ListAdmin.py ... assert self.Locked() and self.__filename from what i understand from python (haven't finnished O'reillys "Learning Python" yet :)), this is a debug statement and it bails out if !self.Locked == true ... so I guess the lock on the pending message db gets lost somehow (which could be because posts are being approved and the pending db being updated at the same time... but i'm only speculating here). > Out of curiosity, the mailinglist you're running on mailman, that wouldn't > happen to be the MissJanet mailing list that you used to run at XS4ALL, > would it? ;) yep... that was one of the first majordomo lists at xs4all i believe :) Ricardo. -- From ricardo@rixhq.nu Sat Feb 5 00:15:07 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Sat, 5 Feb 2000 01:15:07 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000204234757.A2826@xs4all.nl>; from thomas@xs4all.net on Fri, Feb 04, 2000 at 11:47:57PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> Message-ID: <20000205011507.A405@miss-janet.com> Hi, On Fri, Feb 04, 2000 at 11:47:57PM +0100, Thomas Wouters wrote: > On Fri, Feb 04, 2000 at 09:22:43PM +0100, Ricardo Kustner wrote: > > well i patched my copy of MM with the diff Thomas had just posted, and > > then i tried to approve about 16 posts and I still got another assertion error :( > > > maybe i just need to start saving some money for a more powerfull machine :) > Or perhaps you're experiencing an entirely differnet problem. Or a very > related, but slightly different problem. Or perhaps this method of locking > does not work on the type of machine, operating system and/or filesystem you > are using. Linux 2.0.38, libc5 on a P100... not the newest stuff but it's stable :) usually i use newer kernels/libs for servers but I had to do so much work to optimize the apache/fastcgi/mysql stuff that I'm cautious with upgrading... I hope to hook up a second harddrive soon I can put the mail spool on a seperate disk which will hopefully reduce the load a bit... > If you want, I can take a look at your AssertionError, see what > caused it. If not, and it's a generic Mailman bug, I'll eventually hit it It always happens on line 60 in Mailman/ListAdmin.py ... assert self.Locked() and self.__filename from what i understand from python (haven't finnished O'reillys "Learning Python" yet :)), this is a debug statement and it bails out if !self.Locked == true ... so I guess the lock on the pending message db gets lost somehow (which could be because posts are being approved and the pending db being updated at the same time... but i'm only speculating here). > Out of curiosity, the mailinglist you're running on mailman, that wouldn't > happen to be the MissJanet mailing list that you used to run at XS4ALL, > would it? ;) yep... that was one of the first majordomo lists at xs4all i believe :) Ricardo. -- From thomas@xs4all.net Sat Feb 5 15:43:18 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 5 Feb 2000 16:43:18 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000205011507.A405@miss-janet.com>; from ricardo@rixhq.nu on Sat, Feb 05, 2000 at 01:15:07AM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> Message-ID: <20000205164318.A15909@xs4all.nl> On Sat, Feb 05, 2000 at 01:15:07AM +0100, Ricardo Kustner wrote: > > > well i patched my copy of MM with the diff Thomas had just posted, and > > > then i tried to approve about 16 posts and I still got another assertion error :( > It always happens on line 60 in Mailman/ListAdmin.py ... > assert self.Locked() and self.__filename Can you post (or mail directly to me) the exact error message, with traceback, and what you do exactly to get that error ? I tried reproducing it here, by posting 100-some messages to a closed list, and then approving them all in one go, using the admin page, but everything went fine. It might require more of a local load to recreate, though, or maybe you're doing something subtly different. > from what i understand from python (haven't finnished O'reillys "Learning > Python" yet :)), this is a debug statement and it bails out if > !self.Locked == true ... Yes, assert is a debug statement, but both 'self.Locked()' and 'self.__filename' get tested. One of those two ends up false. You can try to split it up in two asserts: assert self.Locked() assert self.__filename just to see which part is failing. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Sat Feb 5 22:21:30 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 5 Feb 2000 23:21:30 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000205205653.A4105@miss-janet.com>; from ricardo@rixhq.nu on Sat, Feb 05, 2000 at 08:56:53PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> Message-ID: <20000205232130.D23471@xs4all.nl> On Sat, Feb 05, 2000 at 08:56:53PM +0100, Ricardo Kustner wrote (in private): > Traceback (innermost last): > File "/usr/local/mailman/scripts/driver", line 112, in run_main > main() > File "../Mailman/Cgi/admindb.py", line 123, in main > mlist.Save() > File "/usr/local/mailman/Mailman/MailList.py", line 819, in Save > self.SaveRequestsDb() > File "../Mailman/ListAdmin.py", line 90, in SaveRequestsDb > self.__closedb() > File "../Mailman/ListAdmin.py", line 74, in __closedb > assert self.Locked() > AssertionError: > if you need to see the logs/error or logs/lock files let me know... [ CC: the list, because of the questions at the bottom ] Hmmm. I think I found the problem, but I'm not entirely sure. It looks like the default MailList lock timeout is 60 seconds, and that that is too short for your machine. You should see some 'stolen!' or (in my LockFile version) 'broken!' messages, in your logs/locks file. In any case, you can easily try it out; in Mailman/MailList.py, on or around line 282, there should be a 'lifetime = 60', inside the constructor for the maillists' lockfile. Changing the '60' in, say, '600', should give you better mileage, at least until your machine gets so heavily loaded that a simple admin request takes ten full minutes to process ;) I'm not sure what the Right Fix is, however. MailList.MailList does not have an advertised way of setting the lock lifetime, nor of refresh()ing the lock. Maybe someone with more experience (barry ? any other active developers) can shed light here... Is the usual solution to raise the initial lock timeout, passing lock timeout in the constructor, adding the two functions and calling them where appropriate, or just using __lock directly ? BTW, whoever documented that part of the code, thank you ;) with the 'TBD' remark above the lockfile constructor this was a sucker to find. It really means a lot to have it so well documented. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Sun Feb 6 11:36:44 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 6 Feb 2000 12:36:44 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000205232130.D23471@xs4all.nl>; from thomas@xs4all.net on Sat, Feb 05, 2000 at 11:21:30PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> Message-ID: <20000206123644.C19265@xs4all.nl> On Sat, Feb 05, 2000 at 11:21:30PM +0100, Thomas Wouters wrote: > You should see some 'stolen!' or (in my LockFile version) 'broken!' > messages, in your logs/locks file. Er, apparently not. Somehow some logmessages got lost during the move from lock() to __break() ;P If you want to be sure, you can add something like this by hand: self.__writelog("stealing lock (from %s)"%winner) As for the locking problem, I think I see a better solution. A site-global 'lock lifetime multiplier' that the site admin can set, by which both the lifetime values of lockfiles and the lock timeout values given to LockFile.lock() are multiplied. By leaving the default to 1 we leave everything the same, but when a slow site is experiencing expired but still used locks, they can up the multiplier and get consistent behaviour all 'round -- no spurious locktimeout errors because lifetime is high but timeout isn't :P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Sun Feb 6 21:05:32 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 6 Feb 2000 22:05:32 +0100 Subject: [Mailman-Developers] locking. Message-ID: <20000206220532.D19265@xs4all.nl> I've found some small buglets in my locking patch, mostly causing leftover files and missing logging... if anyone is integrating my patches, drop me a line so I can send an updated version ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Sun Feb 6 21:17:49 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 6 Feb 2000 22:17:49 +0100 Subject: [Mailman-Developers] integrating changes into distribution In-Reply-To: <14491.5673.280676.30360@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Fri, Feb 04, 2000 at 01:10:49PM -0500 References: <200002041729.MAA29538@bjork.codefab.com> <14491.5673.280676.30360@anthem.cnri.reston.va.us> Message-ID: <20000206221749.E19265@xs4all.nl> On Fri, Feb 04, 2000 at 01:10:49PM -0500, Barry A. Warsaw wrote: > > >>>>> "BB" == Bill Bumgarner writes: > BB> For the Mailman-decode-mime-file-via-webdav modifications, I > BB> have a relatively significant set of addtions and patches to > BB> Mailman. > I'd like to take a crack at integrating the i18n changes this > weekend. If that goes smoothly, your stuff is next on my list. I'm curious about this... I've browsed through the decoding/webdav patches, and it seems explicitly webdav. I understand that Bill needed webdav, and has thus only worked on that, and I appreciate the effort (we might have customers who want it, eventually ;) but I'm wondering if the patch can't be split in two different elements; webdav archiving, and attachment-decoding, and making them seperately configurable. I do not care about installing and configuring webdav at the moment, our setup can't handle it yet, but the automatic decoding of whatever word-document or powerpoint presentation our (sometimes) braindead managers post to our internal all-employees mailinglists, and placing it simply as a seperate file in the mailing list archive is a _very_ attractive idea. Not just to me, the rest of the sysadmin departement went 'ooh' and 'ahhh' when I told them about this development ;) (Which made me feel quite good, I must add. I still feel I have to 'prove' python and mailman to them.) I have no problem with (re)writing that part of the code, of course, but I'd prefer to wait until Bill's code is in the CVS tree (so that I know the style/layout and general idea has been approved of ;) Regards, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From lee@ufojoe.com Mon Feb 7 01:02:00 2000 From: lee@ufojoe.com (Lee Weston) Date: Sun, 6 Feb 2000 20:02:00 -0500 Subject: [Mailman-Developers] RE: dont request passwords from web subscriber patch Message-ID: Fri, 7 Jan 2000 14:58:16 +0100 (MET) Fil fil@bok.net wrote: > >could the maintainers tell me if this patch is accepted or rejected ? >Patch name : dont request passwords from web subscriber I'm sorry to see no answer. I will be looking at adding it. However I seems to me to be only half an answer. The other half is allowing un-subscribe without password. If not you risk trapping silly buggers as if you were fly paper. An "only allow random password" mode would have the benifit of substiantially raising the bar for writting cgi to raid / spam a site, by making them parse email as well. ----------- ufojoe -------------------------------- 50 Canadian Folk Festivals http://www.interlog.com/~ufojoe/ From gorgo@sztaki.hu Mon Feb 7 12:13:56 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Mon, 07 Feb 2000 13:13:56 +0100 (MET) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) Message-ID: Hello, Any ideas what might cause this ? ---------- Forwarded message ---------- Date: Mon, 07 Feb 2000 03:01:23 +0000 (UTC) From: Lazarus Long To: submit@bugs.debian.org Subject: Bug#57223: mailman: gate_news problems Resent-Date: Mon, 07 Feb 2000 03:03:28 +0000 (GMT) Resent-From: Lazarus Long Resent-To: debian-bugs-dist@lists.debian.org Resent-cc: Gergely Madarasz Package: mailman Version: 1.1-2 Severity: important Subject: Cron [ -x /usr/bin/python -a -f /usr/lib/mailman/cron/gate_news ] && /usr/bin/python +/usr/lib/mailman/cron/gate_news X-Cron-Env: X-Cron-Env: X-Cron-Env: X-Cron-Env: Date: Mon, 7 Feb 2000 02:55:06 +0000 (UTC) Traceback (innermost last): File "/usr/lib/mailman/cron/gate_news", line 119, in ? main() File "/usr/lib/mailman/cron/gate_news", line 79, in main r,c,first,last,n = conn.group(mlist.linked_newsgroup) File "/usr/lib/python1.5/nntplib.py", line 208, in group resp = self.shortcmd('GROUP ' + name) File "/usr/lib/python1.5/nntplib.py", line 158, in shortcmd return self.getresp() File "/usr/lib/python1.5/nntplib.py", line 134, in getresp raise error_perm, resp nntplib.error_perm: 500 "GROUP" not implemented; try "help". -- System Information Debian Release: 2.2 Kernel Version: Linux phoenix 2.2.14 #1 Thu Jan 6 01:44:28 UTC 2000 i586 unknown Versions of the packages mailman depends on: ii apache 1.3.9-10 Versatile, high-performance HTTP server ii cron 3.0pl1-55 management of regular background processing ii libc6 2.1.2-13 GNU C Library: Shared libraries and Timezone data ii logrotate 3.2-11 Log rotation utility ii python-base 1.5.2-6 An interactive object-oriented scripting language. sendmail Not installed or no info ii postfix-tls 0.0.19991231pl02+0.51-1 A mail transport agent with RFC2487 encryption ^^^ (Provides virtual package mail-transport-agent) ii apache-ssl 1.3.9.10+1.37-1 Versatile, high-performance HTTP server with SSL support ^^^ (Provides virtual package httpd) From Harald.Meland@usit.uio.no Mon Feb 7 13:59:23 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 14:59:23 +0100 Subject: [Mailman-Developers] feature requests In-Reply-To: Christopher Kolar's message of "Fri, 1 Oct 1999 09:10:51 -0500" References: <99100109180002.09732@kolar.facnet.aurora.edu> Message-ID: Yes, the message this is a reply to is pretty old by now -- I'm quite some messages behind on -developers, not to mention -users :( I'll try to get up to date (and stay there :) now... [Christopher Kolar] > While beating on the documentation a few things came to mind. > > 1. Any chance that the monthly reminder can be used by a bounce > handler script as a membership probe (good for low-volume lists). > Right now I just get N bounce messages as mailman-owner but I do not > see that the list manager ever gets the information. With the currently released Mailman, where only the list objects themselves know which addresses are members, trying to implement something like that would probably scale pretty badly. However, here at uio.no, we've now been running a Mailman with a user database -- mainly for avoiding the pains of having separate passwords for separate mailing lists -- and I guess one could pull of something like what you suggest with the help of the user db. I'll try getting around to post some details about what (rather substantial) changes I've made to our Mailman within the next few days -- but for now, to -developers only, I guess. > 2. How about adding an "invite" feature like on some of the > commercial list providers. The invite would be non-binding, but a > simple reply would add the person to the list (I suppose that it > would use most of the same code that the sub-confirmation bits use). > See listbot and egroups if you do not know what I am talking about. This would just be a slightly differently worded "subscription confirmation" message and something to avoid "open subscribes" turning into "open invites", I guess. > 3. How about a chunk of html code that can be dropped into a web > page that would allow a person to just type in their e-mail and > click to make a sub request (also something that listbot/egroups > do). Mailman does, to my knowledge, not do any kind of Referer checking or somesuch, so the only problem is the (current) need to enter a subscription password when posting a sub request. This _could_ be hacked around in various (ugly) ways by generating this form (dynamically, to avoid everyone having the same password) with HIDDEN fields for passwords. > The request could be handled like a normal sub request with a > confirmation request being sent out to the e-mail address that was > entered. Is this already possible by hacking the listinfo page to > remove everything but the field for e-mail address and the submit > button? I deal with a lot of novice users and I would like to take > out a lot of the verbiage and the initial password setting. How do others feel about this? Should Mailman allow users subscribing without explicitly setting their member password (and instead have Mailman set a random password for them)? -- Harald From Harald.Meland@usit.uio.no Mon Feb 7 14:17:47 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 15:17:47 +0100 Subject: [Mailman-Developers] Re: Mailman and your code for online posting In-Reply-To: Steven Hazel's message of "Wed, 6 Oct 1999 22:19:38 -0500 (CDT)" References: Message-ID: [Steven Hazel] > I hope you don't mind if I cc my reply to the Mailman developers list. > > Is the code that you submitted to the developers list going to be > incorporated into a subsequent release of Mailman? > > I hope so. It got no response, so I'm not quite sure what's up. Just to let you have a response (finally :): I've marked the messages you sent as "look at these when there's time". As you can infer from the time between your posting them and this message, my time for Mailman has been rather sparse for some time :(, but I'm hoping that will improve now. -- Harald From thomas@xs4all.net Mon Feb 7 14:29:44 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 15:29:44 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 01:13:56PM +0100 References: Message-ID: <20000207152944.G19265@xs4all.nl> On Mon, Feb 07, 2000 at 01:13:56PM +0100, Gergely Madarasz wrote: > Any ideas what might cause this ? > nntplib.error_perm: 500 "GROUP" not implemented; try "help". You're connecting to a newsserver that does not have or perhaps does not allow the 'GROUP' command. I personally wasn't aware that there were such newsservers... But perhaps you are connected to a newsfeeder box, instead of a newsreader box ? Some ISPs split their newsservers in feeders, who only have enough of a newsserver to share news with other newsservers, serve as a gateway of sorts, and actual readers, which get their feed off the feeders, and who _do_ allow clients to connect, and offer most, if not all, NNTP commands. If you're fairly sure you are connected to the right newsserver, try connecting to that newsserver, from the machine running mailman, on the news port (119) and try the GROUP command yourself. Also see if you can check the newsservers' logfile, to see if it mentions anything else. If all else fails, you need to find a different newsserver, or shut off the news gateway. > ---------- Forwarded message ---------- > Package: mailman > Version: 1.1-2 > Severity: important > Traceback (innermost last): > File "/usr/lib/mailman/cron/gate_news", line 119, in ? > main() > File "/usr/lib/mailman/cron/gate_news", line 79, in main > r,c,first,last,n = conn.group(mlist.linked_newsgroup) > File "/usr/lib/python1.5/nntplib.py", line 208, in group > resp = self.shortcmd('GROUP ' + name) > File "/usr/lib/python1.5/nntplib.py", line 158, in shortcmd > return self.getresp() > File "/usr/lib/python1.5/nntplib.py", line 134, in getresp > raise error_perm, resp -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gorgo@sztaki.hu Mon Feb 7 14:41:55 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Mon, 07 Feb 2000 15:41:55 +0100 (MET) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207152944.G19265@xs4all.nl> Message-ID: On Mon, 7 Feb 2000, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 01:13:56PM +0100, Gergely Madarasz wrote: > > > Any ideas what might cause this ? > > > nntplib.error_perm: 500 "GROUP" not implemented; try "help". > > You're connecting to a newsserver that does not have or perhaps does not > allow the 'GROUP' command. I personally wasn't aware that there were such > newsservers... But perhaps you are connected to a newsfeeder box, instead of > a newsreader box ? Some ISPs split their newsservers in feeders, who only > have enough of a newsserver to share news with other newsservers, serve as a > gateway of sorts, and actual readers, which get their feed off the feeders, > and who _do_ allow clients to connect, and offer most, if not all, NNTP > commands. > > If you're fairly sure you are connected to the right newsserver, try > connecting to that newsserver, from the machine running mailman, on the news > port (119) and try the GROUP command yourself. Also see if you can check the > newsservers' logfile, to see if it mentions anything else. If all else > fails, you need to find a different newsserver, or shut off the news > gateway. Hmm... I found what might cause this... this if from the mailman list archives: http://www.python.org/pipermail/mailman-users/1999-October/002371.html so does this mean that there was no workaround made for this in mailman 1.1 ? Is there a patch somewhere perhaps ? -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From Harald.Meland@usit.uio.no Mon Feb 7 14:44:03 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 15:44:03 +0100 Subject: [Mailman-Developers] Re: [Mailman-Users] Can't change with admin pages In-Reply-To: Sean Reifschneider's message of "Wed, 13 Oct 1999 03:24:17 -0600" References: <19991013032417.O1005@tummy.com> Message-ID: [Sean Reifschneider] > On Mon, Oct 11, 1999 at 08:59:17PM +0200, Per Starback wrote: > > path = urlparse(self.web_page_url)[2] # '/mailman' > > Is that comment supposed to mean something? Only that the typical value of `path' after that line would be "/mailman", but might be different if the list has a "exotic" (which does not imply "invalid") `web_page_url'. > Should the above code be changed to something more like: > > path = urlparse(self.web_page_url)[2] # '/mailman' > if not path: path = self.web_page_url Hmm, I think it'd probably be better to fix problems with invalid `web_page_url's in MailList.CheckValues(), which is called every time the list is loaded (and which already contains code to ensure that `web_page_url' ends in a "/"). -- Harald From thomas@xs4all.net Mon Feb 7 15:29:30 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 16:29:30 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 03:41:55PM +0100 References: <20000207152944.G19265@xs4all.nl> Message-ID: <20000207162930.H19265@xs4all.nl> On Mon, Feb 07, 2000 at 03:41:55PM +0100, Gergely Madarasz wrote: > > > nntplib.error_perm: 500 "GROUP" not implemented; try "help". > > You're connecting to a newsserver that does not have or perhaps does not > > allow the 'GROUP' command. I personally wasn't aware that there were such > Hmm... I found what might cause this... this if from the mailman list > archives: > http://www.python.org/pipermail/mailman-users/1999-October/002371.html > so does this mean that there was no workaround made for this in mailman > 1.1 ? Is there a patch somewhere perhaps ? The problem is that the problem code isn't Mailman code ;) nntplib.py is in the standard python library, it's just included in the 'pythonlib' directory for older python distributions, that dont have nntplib.py. Apparently the patch (which isn't real clean, by the way, because a general except: clause can be very nasty) hasn't been sent to the python-list, or hasn't been accepted. But the problem does look like it needs fixing, i'll see if i can send a patch to the python-patches list. In the mean time you can apply the patch mentioned in that posting, to your own copy of nntplib, as long as you dont use nntplib in server mode anywhere (Not very likely, that last part) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gorgo@sztaki.hu Mon Feb 7 15:57:37 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Mon, 07 Feb 2000 16:57:37 +0100 (MET) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207162930.H19265@xs4all.nl> Message-ID: On Mon, 7 Feb 2000, Thomas Wouters wrote: > send a patch to the python-patches list. In the mean time you can apply the > patch mentioned in that posting, to your own copy of nntplib, as long as you > dont use nntplib in server mode anywhere (Not very likely, that last part) That is for the reporter of the problem... But I need to make a fix in the mailman .deb package for debian potato before potato is released... :) -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From thomas@xs4all.net Mon Feb 7 16:19:59 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 17:19:59 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 04:57:37PM +0100 References: <20000207162930.H19265@xs4all.nl> Message-ID: <20000207171959.K23471@xs4all.nl> --bg08WKrSYDhXBjb5 Content-Type: text/plain; charset=us-ascii On Mon, Feb 07, 2000 at 04:57:37PM +0100, Gergely Madarasz wrote: > On Mon, 7 Feb 2000, Thomas Wouters wrote: > > send a patch to the python-patches list. In the mean time you can apply the > > patch mentioned in that posting, to your own copy of nntplib, as long as you > > dont use nntplib in server mode anywhere (Not very likely, that last part) > That is for the reporter of the problem... But I need to make a fix in the > mailman .deb package for debian potato before potato is released... :) Ok, well, because the reader attempting the 'group' command, in cron/gate_news, isn't using authorization, we can just send the 'mode reader' command when we get the error 500. The attached patch should work, I think. It fixes the first call to 'group' instead of the constructor of nntplib.NNTP, so it should be usable with all nntplib version. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --bg08WKrSYDhXBjb5 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mailman.cvs.diff" Index: cron/gate_news =================================================================== RCS file: /projects/cvsroot/mailman/cron/gate_news,v retrieving revision 1.22 diff -u -r1.22 gate_news --- gate_news 1999/12/25 14:35:31 1.22 +++ gate_news 2000/02/07 16:12:40 @@ -125,7 +125,15 @@ sys.stderr.write('connect to nntp_host failed\n') sys.stderr.write(`e`) break - r,c,f,l,n = conn.group(mlist.linked_newsgroup) + + try: + r,c,f,l,n = conn.group(mlist.linked_newsgroup) + except nntplib.error_perm: + # We got an error 500. This could mean 'group' is not implemented, + # and that we need to set the connection to reader mode first. + conn.shortcmd('mode reader') + r,c,f,l,n = conn.group(mlist.linked_newsgroup) + if not updatewatermarks: # just post the specified messages and be done with it poll_newsgroup(mlist, conn, first, last+1) --bg08WKrSYDhXBjb5-- From pf@artcom-gmbh.de Mon Feb 7 16:52:31 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Mon, 7 Feb 2000 17:52:31 +0100 (MET) Subject: [Mailman-Developers] Quoted printable umlaut quick hack Message-ID: I'm running mailman 1.1 on our intranet web server. Since our lists are in German language, umlauts occure quite often in mail bodies and subjects: äöü ÄÖÜ ß. Depending on the used MUA these are often encoded as quoted printables and end up as ugly trash in the pipermail HTML archives: F=FCr instead of Für ... Unfortunately this dooes not only look ugly but also prevents HTdig (the search engine) to find any words with umlauts in the pipermail archives. So today I did some quick hacks to HyperArch.py which at least work for me. (Spend too much time on this detail already) My dirty patch against HyperArch.py is appended below. May be this will help somebody to do this right. Regards, Peter -- Peter Funk, Oldenburger Str.86, 27777 Ganderkesee, Tel: 04222 9502 70, Fax: -60 ---- 8< ---- 8< ---- cut here ---- 8< ---- schnipp ---- 8< ---- schnapp ---- --- /home/pf/freeware/mailman-1.1/Mailman/Archiver/HyperArch.py Sat Aug 21 07:13:23 1999 +++ HyperArch.py Mon Feb 7 17:36:07 2000 @@ -34,6 +34,7 @@ import sys import re, cgi, urllib, string import time, pickle, os, posixfile +import quopri, cStringIO import HyperDatabase import pipermail from Mailman import mm_cfg @@ -49,6 +50,7 @@ except ImportError: pass +umlaut_in_subj = re.compile(r"=\?iso-8859-1\?Q\?(.*)\?=", re.IGNORECASE) def html_quote(s): repls = ( ('&', '&'), @@ -228,6 +230,16 @@ self.subject=self.subject[i:] else: i=-1 if self.subject=="": self.subject='No subject' + # --Pefus quick Hack: + umls = umlaut_in_subj.search(self.subject) + if umls: + umlconvin = cStringIO.StringIO( + umlaut_in_subj.sub(r"\1", self.subject)) + umlconvout = cStringIO.StringIO() + quopri.decode(umlconvin, umlconvout) + umlconvout.seek(0) + self.subject = string.join(umlconvout.read(), "") + # -- if message.has_key('Date'): self.datestr=str(message['Date']) @@ -239,7 +251,12 @@ date, tzoffset=date[:9], date[-1] if not tzoffset: tzoffset = 0 - date=time.mktime(date)-tzoffset + try: + date=time.mktime(date)-tzoffset + except ValueError: + print "illegal date discovered:", date + date=time.time() + else: date=self.__last_article_time+1 @@ -276,8 +293,13 @@ # Read the message body self.body=[] message.rewindbody() + converterInput=cStringIO.StringIO( + string.join(message.fp.readlines(), "")) + converterOutput=cStringIO.StringIO() + quopri.decode(converterInput, converterOutput) + converterOutput.seek(0) while (1): - line=message.fp.readline() + line=converterOutput.readline() if line=="": break self.body.append(line) From gorgo@sztaki.hu Mon Feb 7 17:16:06 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Mon, 07 Feb 2000 18:16:06 +0100 (MET) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207171959.K23471@xs4all.nl> Message-ID: On Mon, 7 Feb 2000, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 04:57:37PM +0100, Gergely Madarasz wrote: > > On Mon, 7 Feb 2000, Thomas Wouters wrote: > > > > send a patch to the python-patches list. In the mean time you can apply the > > > patch mentioned in that posting, to your own copy of nntplib, as long as you > > > dont use nntplib in server mode anywhere (Not very likely, that last part) > > > That is for the reporter of the problem... But I need to make a fix in the > > mailman .deb package for debian potato before potato is released... :) > > Ok, well, because the reader attempting the 'group' command, in > cron/gate_news, isn't using authorization, we can just send the 'mode > reader' command when we get the error 500. The attached patch should work, I > think. It fixes the first call to 'group' instead of the constructor of > nntplib.NNTP, so it should be usable with all nntplib version. Thanks. Lazarus, please test this patch, and notify me if it worked for you :) -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From gerrit@nl.linux.org Mon Feb 7 16:56:53 2000 From: gerrit@nl.linux.org (Gerrit Holl) Date: Mon, 7 Feb 2000 17:56:53 +0100 Subject: [Mailman-Developers] Development reference manual? Message-ID: <20000207175653.A4084@stopcontact.palga.uucp> Hello, I wonder if there is any reference manual to all of Mailmans classes, functions and variables. Like www.python.org/doc/current/lib. I'd like such a feature, since I'm interested in developing Mailman. I've almost a year experience in Python, but I get confused my the Mailman core package. I've read the available documentaion but I get confused by some of the code. regards, Gerrit. P.S. Is the majority of people in here also Dutch, like on c.l.py? -- homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From bbum@codefab.com Mon Feb 7 17:33:37 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Mon, 7 Feb 2000 12:33:37 -0500 Subject: [Mailman-Developers] Re: Mailman-Developers digest, Vol 1 #442 - 12 msgs Message-ID: <200002071733.MAA18496@bjork.codefab.com> Agreed. I wish I had the time to fully generalize the code though I'm quite happy to hear that others think this is a good idea! As it stands, it is relatively close-- the Rewrite.py module has been extended such that the resulting rewritten message contains instances of WebDAVAttachment that contain the eventual URL (archival and retrieval) for the attachment and the decoded raw data of the attachment. I made the decision to call it WebDAVAttachment to imply the context within which it was written. However, the actual implementation of that particular class has very little that is specific to WebDAV other than some of the URL names. In hindsight, I really should have not prefixed the retrieval URL with 'dav'-- it has nothing to do with DAV, even given the current implementation. The ToArchiver Handler could clearly use some generalization! Actually, the actual process() method is relatively general, but the do_dav() method should likely be broken out. Applying a bit more thought to this, it strikes me that what should likely happen is one of several things: - enabling/disabling different archival methods should be separated from the configuration of that individual archival method. Certain archival configuration optiosn-- like monthly/weekly/etc (which the WebDAV stuff currently ignores. I apologize for that-- more below)-- should be left in a sort of "global" configuration area assuming that it is appropriate to apply the same options to more than one archive "style". - individual or broken out screens for the individual archival methods - broken out mechanism that does the actual "splitting" of the archive into the appropriate paths. Regardless of whether it is a filesystem or URL based archival mechanism, you pretty much *always* have a "root" location below which the entire archive resides. The current implementation does not share any code between the pipermail archiver and the WebDAV archiver. Part of this was because it was indicated that the pipermail stuff needed to be largely revisited and partially because I simply didn't have time to do the generalization I wanted. --- Thomas: If you want to contribute to the effort prior to Barry's work integrating back into the core repository, I can take on the task of integrating your patches into my source. Let me know and I can make sure the latest sources are available to you via a more direct means than the occasional tarball. The changes that you indicated are certainly the correct direction for this codebase-- it would likely make Barry's life easier in doing the final to-repository integration if the code were a bit more refined. Ultimately, it is likely Barry's call-- I have a limited amount of time available now that this stuff works, but I would like to invest that time in ensuring that the codebase is moving as rapidly as possible into being a supported part of the existing Mailman infrastructure. -- Apologies: These extensions were actually done as a part of the development of a proprietary solution for a client of CodeFab's. Early in the specification phase, it become abundantly clear that leveraging Mailman would gain us a whole bunch of functionality for free. Of course, the challenge was then convincing the client that the requirement for releasing said changes back to the community was a small price compared to the gain in feature set. This proved surprisingly easy. The client is a very positive minded, forward thinking group of folks that definitely understand and believe in things like Open Source and Good Engineering Practices. (If anyone is interested in working for such an entity in the NYC area-- with obvious ties to the Python community-- please let me know and I can forward your contact information to the appropriate individuals. They are a "pre-IPO" "dot com" with excellent "potential valuation". Yadda.). Our initial approach was to try and hire someone from this community to do the work. Both because we wanted someone more familiar with the codebase than me (1.2 expiremental via cvs was my first real exposure) and because I really wanted to focus on other aspects of the project. That didn't happen and I ended up doing all the work. Fine by me, I *love* programming in Python-- especially when working with a cool sourcebase. However, it did mean that there was a lot of pressure on me to make the code work as quickly as possible and minimize the amount of effort put forth to making the end result as generic as it really should be. To give the client credit, they understand and authorized me spending a bunch of time to do a minimal amount of work towards a design and implementation that would be palatable to the community. While they don't have an infinite amount of time/money to throw at this, they definitely recognized that an implementation that would be accepted by the community as rapidly as possible would benefit them, as well. So; I apologize for the state of some of the code. It is not as clean or as elegant as I would normally consider acceptable. At the same time, the work has been done, the end result actually works, and it is definitely a solution to a problem that has been very much at the forefront of a lot of folk's administrative environments. And, of course, getting that initial implementation out the door is always half the battle... thanks, b.bum /* From: Thomas Wouters wrote */ On Fri, Feb 04, 2000 at 01:10:49PM -0500, Barry A. Warsaw wrote: > > >>>>> "BB" == Bill Bumgarner writes: > BB> For the Mailman-decode-mime-file-via-webdav modifications, I > BB> have a relatively significant set of addtions and patches to > BB> Mailman. > I'd like to take a crack at integrating the i18n changes this > weekend. If that goes smoothly, your stuff is next on my list. /* * I'm curious about this... I've browsed through the decoding/webdav patches, * and it seems explicitly webdav. I understand that Bill needed webdav, and * has thus only worked on that, and I appreciate the effort (we might have * customers who want it, eventually ;) but I'm wondering if the patch can't be * split in two different elements; webdav archiving, and attachment-decoding, * and making them seperately configurable. * * I do not care about installing and configuring webdav at the moment, our * setup can't handle it yet, but the automatic decoding of whatever * word-document or powerpoint presentation our (sometimes) braindead managers * post to our internal all-employees mailinglists, and placing it simply as a * seperate file in the mailing list archive is a _very_ attractive idea. Not * just to me, the rest of the sysadmin departement went 'ooh' and 'ahhh' when * I told them about this development ;) (Which made me feel quite good, I must * add. I still feel I have to 'prove' python and mailman to them.) * * I have no problem with (re)writing that part of the code, of course, but I'd * prefer to wait until Bill's code is in the CVS tree (so that I know the * style/layout and general idea has been approved of ;) */ From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Mon Feb 7 17:49:37 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 7 Feb 2000 12:49:37 -0500 (EST) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> Message-ID: <14495.1457.828840.186393@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> Hmm... I found what might cause this... this if from the GM> mailman list archives: GM> http://www.python.org/pipermail/mailman-users/1999-October/002371.html GM> so does this mean that there was no workaround made for this GM> in mailman 1.1 ? Is there a patch somewhere perhaps ? Since this is a bug that appears to be best fixed in Python's nntplib.py, I will float the message to patches@python.org. If nobody objects I'll make the change for Python 1.6 and then back-copy the file into Mailman 1.2. The quick fix is to do the following: - copy Python 1.5.2's nntplib.py into Mailman/pythonlib - apply the patch - Change all occurrances of "import nntplib" to "from Mailman.pythonlib import nntplib" -Barry From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Mon Feb 7 17:53:17 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 7 Feb 2000 12:53:17 -0500 (EST) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> Message-ID: <14495.1677.982940.194724@anthem.cnri.reston.va.us> I'm cutting and pasting a message that originated on the mailman-users mailing list. I think this message is fairly self-explanatory. Can other NNTP experts look this over and see if it makes sense? I'll apply it if there are no objections. Note: the bare except is bothersome. For reference, the thread is here: http://www.python.org/pipermail/mailman-users/1999-October/002371.html -Barry -------------------- snip snip -------------------- On Sun, Oct 10, 1999 at 06:23:41PM -0700, Michael Ghens wrote: > Tried a news gateway test. This is the error: [...] > nntplib.error_perm: 500 "GROUP" not implemented; try "help". Were you running the mailman gateway on the same machine as the news server? (Or on another machine that is an NNTP peer of the news server?) Python's standard nntplib assumes that when it connects it will default to (NNRP) reader mode. If the connection ends up in NNTP/news feed mode, many commands, including "GROUP" will not be recognized. A quick hack is to add this to your nntplib.py: diff -u nntplib.py.dist nntplib.py --- nntplib.py.dist Tue Apr 28 17:43:35 1998 +++ nntplib.py Tue Jun 23 22:06:00 1998 @@ -71,6 +71,10 @@ self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + try: + self.welcome = self.shortcmd('mode reader') + except: + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': This makes nntplib use 'mode reader' for all connections. As Harald Meland pointed out, this should probably be controlled by adding another optional argument to nntplib.NNTP(), but if you only use the library with mailman or in other "client" applications this will suffice. Jim P.S. You could also do the 'mode reader' stanza in GatewayManager, but then you also need to repeat the 'authinfo' code there if your news server needs authentication, since that must follow the 'mode reader' command. -- Jim Tittsler, Tokyo From thomas@xs4all.net Mon Feb 7 18:08:33 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 19:08:33 +0100 Subject: [Mailman-Developers] Re: Mailman-Developers digest, Vol 1 #442 - 12 msgs In-Reply-To: <200002071733.MAA18496@bjork.codefab.com>; from bbum@codefab.com on Mon, Feb 07, 2000 at 12:33:37PM -0500 References: <200002071733.MAA18496@bjork.codefab.com> Message-ID: <20000207190833.K19265@xs4all.nl> On Mon, Feb 07, 2000 at 12:33:37PM -0500, Bill Bumgarner wrote: > Thomas: > > If you want to contribute to the effort prior to Barry's work > integrating back into the core repository, I can take on the task of > integrating your patches into my source. Let me know and I can make sure > the latest sources are available to you via a more direct means than the > occasional tarball. The changes that you indicated are certainly the > correct direction for this codebase-- it would likely make Barry's life > easier in doing the final to-repository integration if the code were a bit > more refined. > > Ultimately, it is likely Barry's call-- I have a limited amount of > time available now that this stuff works, but I would like to invest that > time in ensuring that the codebase is moving as rapidly as possible into > being a supported part of the existing Mailman infrastructure. Well, depending on howfar Barry has been able to integrate the patches yet, it might be easier for him to continue integration and for me to send patches relative to the then-CVS version -- unless you're still working on these patches, making changes ? My guess is that Barry will make minor adjustments while integrating, like changing tabs into spaces, and possibly merging it with other patches he has pending, like those i18n patches he mentioned, making additional patches by me based on your code a slight pain. Or if Barry has not touched the WebDAV patches yet, and people are willing to wait, i can take your patches, rewrite the bits that need rewriting, and re-submit them for integration. I'm not sure how fast i would have the other patch ready, as i'm still studying those parts of mailman (1.2 CVS is my first contact with mailman as well;) and if Barry or someone else wants the WebDAV changes into the CVS tree anytime soon, it's probably better to add them now and for me to post changes. Barry...? ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Mon Feb 7 18:17:13 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 19:17:13 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <14495.1677.982940.194724@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Mon, Feb 07, 2000 at 12:53:17PM -0500 References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> Message-ID: <20000207191713.L19265@xs4all.nl> On Mon, Feb 07, 2000 at 12:53:17PM -0500, Barry A. Warsaw wrote: > I'm cutting and pasting a message that originated on the mailman-users > mailing list. I think this message is fairly self-explanatory. Can > other NNTP experts look this over and see if it makes sense? I'll > apply it if there are no objections. > Note: the bare except is bothersome. I've discussed this with two of my colleagues, who do and have done a fair lot with INN, and the general concensus is that it should work just fine. Servers who dont need 'mode reader' but recognize it should ignore it, Servers who dont understand it should give a 500 error, 'error_perm', with 'unimplemented' or 'what ?' or some such as message text. Note, though, that i've already posted a small patch to mailman to fix this. Since the 'group' command is executed by gate_news, which doesn't need authentication, it's easy enough to try the group, and set the connection to readermode if it fails. Also note that I've already sent a patch to patches@python.org ;) with a pointer to the mailman-users discussion in question. http://www.python.org/pipermail/patches/2000-February/000015.html My patch uses an extra argument to NNTP.__init__, and avoids the bare except. To get the same behaviour as the patch below, sending 'mode reader' by default, the default value of readermode can be set to 1. I'm not sure if that's a wise thing, however, as some users of nntplib.NNTP might depend on the connection being able to do things that you cant do in readermode. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Harald.Meland@usit.uio.no Mon Feb 7 18:40:06 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 19:40:06 +0100 Subject: [Mailman-Developers] Feature suggestion In-Reply-To: Mats Wichmann's message of "Mon, 22 Nov 1999 11:03:10 -0700" References: <3.0.6.32.19991122110310.00e7cc50@laplaza.org> Message-ID: [Mats Wichmann] > At 01:55 PM 11/22/1999 -0000, Adrian Eyre wrote: > >> However, my idea is this. What if there was some sort of mechanism in > >> place that stored the file in a different type of archive, and instead > >> of mailing the file, it included a link to click on to pick up the file > >> instead. > > > >Good idea. Could it be generalised to allow all people on the mailing list > >to specify a maximum attachment size. All attachments below this are > >attached to the e-mail, and any larger are replaced with links. Or maybe > >as a total attachment size for each e-mail, rather than on an attachment > >by attachment basis. > > Sure seems like it would be a useful feature. I'm mostly stuck > on very slow lines, many times it's a long-distance call to get > that much. I'd like to be able to defer getting attachments - > pretty much ALL attachments - until I know at least I've got a > toll-free connection, even if it's a slow one. Maybe there's not > enough of us so "blessed" to be worth a change, though.... I can sympathize with the initial idea (replacing the attachment with a link for _all_ list members), but the "generalisation" I don't feel to good about. If individual list members are allowed to customize what parts of a list message they want, it means Mailman has to generate umpteen different versions of an incoming post, and pass the appropriate version on to the different sets of members. That would scale rather poorly... Of course, if anyone wants this bad enough to code it up, and makes it possible to turn off site-wide for admins so inclined, it might be included. As for the initial suggestion: First step would be to teach Mailman something about MIME messages, any takers are welcome :) -- Harald From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Mon Feb 7 19:16:52 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 7 Feb 2000 14:16:52 -0500 (EST) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207162930.H19265@xs4all.nl> <20000207171959.K23471@xs4all.nl> Message-ID: <14495.6692.360996.332479@anthem.cnri.reston.va.us> >>>>> "TW" == Thomas Wouters writes: TW> Ok, well, because the reader attempting the 'group' command, TW> in cron/gate_news, isn't using authorization, we can just send TW> the 'mode reader' command when we get the error 500. The TW> attached patch should work, I think. It fixes the first call TW> to 'group' instead of the constructor of nntplib.NNTP, so it TW> should be usable with all nntplib version. Thanks. I'll apply this patch (with a slightly rewritten comment) to the CVS snapshot. -Barry From ricardo@rixhq.nu Mon Feb 7 19:43:31 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Mon, 7 Feb 2000 20:43:31 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000205232130.D23471@xs4all.nl>; from thomas@xs4all.net on Sat, Feb 05, 2000 at 11:21:30PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> Message-ID: <20000207204331.A444@miss-janet.com> On Sat, Feb 05, 2000 at 11:21:30PM +0100, Thomas Wouters wrote: > On Sat, Feb 05, 2000 at 08:56:53PM +0100, Ricardo Kustner wrote (in private): > > AssertionError: > > if you need to see the logs/error or logs/lock files let me know... > [ CC: the list, because of the questions at the bottom ] > Hmmm. I think I found the problem, but I'm not entirely sure. It looks like > the default MailList lock timeout is 60 seconds, and that that is too short > for your machine. You should see some 'stolen!' or (in my LockFile version) > 'broken!' messages, in your logs/locks file. yes i did see "stolen!" in the logs just before the process was about to crash. > In any case, you can easily try it out; in Mailman/MailList.py, on or around > line 282, there should be a 'lifetime = 60', inside the constructor for the > maillists' lockfile. Changing the '60' in, say, '600', should give you thanks a lot :) I'm trying out the value '300'... just to be sure... > better mileage, at least until your machine gets so heavily loaded that a > simple admin request takes ten full minutes to process ;) with the earlier versions of MM, first I used exim and that made the load way to high (around 60) and often made it impossible to use... after I started using postfix, things went much better. > I'm not sure what the Right Fix is, however. MailList.MailList does not have I'll let you know how it works out after changing the lock timeout... Ricardo. -- From lazarus@overdue.dhis.org Mon Feb 7 20:09:18 2000 From: lazarus@overdue.dhis.org (Lazarus Long) Date: Mon, 7 Feb 2000 20:09:18 +0000 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 06:16:06PM +0100 References: <20000207171959.K23471@xs4all.nl> Message-ID: <20000207200918.A18352@overdue.dhis.net> On Mon, Feb 07, 2000 at 06:16:06PM +0100, Gergely Madarasz wrote: > > Thanks. > Lazarus, please test this patch, and notify me if it worked for you :) phoenix:/usr/lib/mailman/cron# patch --verbose --dry-run < mailman.cvs.diff Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |Index: cron/gate_news |=================================================================== |RCS file: /projects/cvsroot/mailman/cron/gate_news,v |retrieving revision 1.22 |diff -u -r1.22 gate_news |--- gate_news 1999/12/25 14:35:31 1.22 |+++ gate_news 2000/02/07 16:12:40 -------------------------- Patching file `gate_news' using Plan A... Hunk #1 FAILED at 125. 1 out of 1 hunk FAILED -- saving rejects to gate_news.rej done phoenix:/usr/lib/mailman/cron# No. :( What next? I know no python, fwiw. -- From Harald.Meland@usit.uio.no Mon Feb 7 20:11:04 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 21:11:04 +0100 Subject: [Mailman-Developers] Reply goes to list error In-Reply-To: Balazs Nagy's message of "Tue, 25 Jan 2000 01:07:39 +0100 (CET)" References: Message-ID: [Balazs Nagy] > On Tue, 18 Jan 2000, Balazs Nagy wrote: > > > Hi, > > > > I know that reply to the list address is obsolete and not supported but in > > my list there're a bunch of people who claim this feature. It worked in > > the last version but the CVS version doesn't use this option anymore. > > Here's the patch. > > I just cannot understand why nobody from thosw who have write access > did anything about the subject. Because Reply-To: munging is already part of CookHeaders.py, I guess. And that takes appropriate note of the "fastrack" attribute of the message object, too. -- Harald From Harald.Meland@usit.uio.no Mon Feb 7 20:17:50 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 21:17:50 +0100 Subject: [Mailman-Developers] Admin overview page should list number of pending admin requests per list In-Reply-To: "Barry A. Warsaw"'s message of "Wed, 19 Jan 2000 12:33:43 -0500 (EST)" References: <20000119175228.67F911FF03B@perens.com> <14469.62839.700229.701854@anthem.cnri.reston.va.us> Message-ID: [Barry A. Warsaw] > >>>>> "BP" == Bruce Perens writes: > > BP> Here's a feature request. I administrer about 10 lists. To see > BP> what the pending admin requests are, I either have to look in > BP> my mailbox for nudges from the server, or click on the "handle > BP> administrative requests" link for _every_ list. What I want > BP> is to be able to see this for all lists at a glance. > > As admin for about a dozen lists on python.org, I completely agree! But that means that _anyone_ can see whether there are held messages in a list -- which really is giving out more info than appropriate (at least in some cases). I think there should be a separate "userinfo" CGI script which, after proper autentication, shows what lists that user is 1) member of and 2) admin for. Putting the info you both crave for on that page would be Good. [ And just think how nicely it would integrate with a user database... OK, OK, I'll stop teasing RSN ;) ] -- Harald From lazarus@overdue.dhis.org Mon Feb 7 20:27:26 2000 From: lazarus@overdue.dhis.org (Lazarus Long) Date: Mon, 7 Feb 2000 20:27:26 +0000 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207171959.K23471@xs4all.nl>; from thomas@xs4all.net on Mon, Feb 07, 2000 at 05:19:59PM +0100 References: <20000207162930.H19265@xs4all.nl> <20000207171959.K23471@xs4all.nl> Message-ID: <20000207202726.B18542@overdue.dhis.net> On Mon, Feb 07, 2000 at 05:19:59PM +0100, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 04:57:37PM +0100, Gergely Madarasz wrote: > > On Mon, 7 Feb 2000, Thomas Wouters wrote: > > > > send a patch to the python-patches list. In the mean time you can apply the > > > patch mentioned in that posting, to your own copy of nntplib, as long as you > > > dont use nntplib in server mode anywhere (Not very likely, that last part) > > > That is for the reporter of the problem... But I need to make a fix in the > > mailman .deb package for debian potato before potato is released... :) > > Ok, well, because the reader attempting the 'group' command, in > cron/gate_news, isn't using authorization, we can just send the 'mode > reader' command when we get the error 500. The attached patch should work, I > think. It fixes the first call to 'group' instead of the constructor of > nntplib.NNTP, so it should be usable with all nntplib version. That patch would not apply here, as I believe I just posted to your list as well with my previous mail. I'm not sure if gate_news in the Debian packages differs from the CVS version or not however. Gergely? -- From ricardo@rixhq.nu Mon Feb 7 20:31:44 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Mon, 7 Feb 2000 21:31:44 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000207204331.A444@miss-janet.com>; from ricardo@rixhq.nu on Mon, Feb 07, 2000 at 08:43:31PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> <20000207204331.A444@miss-janet.com> Message-ID: <20000207213143.B444@miss-janet.com> On Mon, Feb 07, 2000 at 08:43:31PM +0100, Ricardo Kustner wrote: > yes i did see "stolen!" in the logs just before the process was about to crash. > > In any case, you can easily try it out; in Mailman/MailList.py, on or around > > line 282, there should be a 'lifetime = 60', inside the constructor for the > > maillists' lockfile. Changing the '60' in, say, '600', should give you > thanks a lot :) I'm trying out the value '300'... just to be sure... well I just tried it out... I approved 17 posts and didn't get an assertion error! though it took quite a while before the webpage was finnished loading (about 2 minutes) but the process load on the machine didn't get any higher than 2.5 I wonder though what happens if some impatient moderator decides not to wait before the page finnishes loading, switches to a differen webpage and therefor breaks the python cgi process... will some approved posts stay in the queue instead? I mentioned before that I think that it could be better if the cgi scripts don't do anything more than just "mark" message as being approved... cgi scripts should have a short life time and definately shouldn't be waiting for something too long... btw i used the original LockFile.py this time... Ricardo. -- From lazarus@overdue.dhis.org Mon Feb 7 20:55:35 2000 From: lazarus@overdue.dhis.org (Lazarus Long) Date: Mon, 7 Feb 2000 20:55:35 +0000 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Mon, Feb 07, 2000 at 04:57:37PM +0100 References: <20000207162930.H19265@xs4all.nl> Message-ID: <20000207205535.C18542@overdue.dhis.net> On Mon, Feb 07, 2000 at 04:57:37PM +0100, Gergely Madarasz wrote: > > send a patch to the python-patches list. In the mean time you can apply the > > patch mentioned in that posting, to your own copy of nntplib, as long as you > > dont use nntplib in server mode anywhere (Not very likely, that last part) > > That is for the reporter of the problem... But I need to make a fix in the > mailman .deb package for debian potato before potato is released... :) Following up to what I've received here so far on this thread (some of which I appear to have missed, judging by quoted portions) I am running current standard versions of most of the involved packages here on a Debian system, including INN2 as the NNTP daemon, so I imagine standard commands are supported. In case specifics are useful to anyone, here is a list which I hope covers all potentially-involved packages based upon Debian package dependency information. I'll include those I suspect are irrelevant, in case I am wrong, in the interest of completeness. ||/ Name Version Description +++-===========================-===========================-====================================================================== ii task-news-server 2.2.2.2000.01.31-1 USENET News Server ii inn2 2.2.2.2000.01.31-1 News transport system `InterNetNews' by the ISC and Rich Salz ii libc6 2.1.2-13 GNU C Library: Shared libraries and Timezone data ii libgdbmg1 1.7.3-26.2 GNU dbm database routines (runtime version). [libc6 version] ii cron 3.0pl1-55 management of regular background processing ii postfix-tls 0.0.19991231pl02+0.51-1 A mail transport agent with RFC2487 encryption ii time 1.7-8 The GNU time command. ii inn2-inews 2.2.2.2000.01.31-1 NNTP client news injector, from InterNetNews (INN) ii grep 2.4-2 GNU grep, egrep and fgrep. ii perl-5.005 5.005.03-5.3 Larry Wall's Practical Extracting and Report Language. ii python-base 1.5.2-6 An interactive object-oriented scripting language. ii libncurses4 4.2-8 Shared libraries for terminal handling ii libreadlineg2 2.1-17 GNU readline and history libraries, run-time libraries. [libc6] ii mailman 1.1-2 Powerful, web based list processor ii logrotate 3.2-11 Log rotation utility ii apache 1.3.9-10 Versatile, high-performance HTTP server ii apache-ssl 1.3.9.10+1.37-1 Versatile, high-performance HTTP server with SSL support The original Debian bug report contained a similar list, but I would imagine it was not included on this list, since there was question of which NNTP commands were supported by the daemon in question. BTW, mangling python-base to prevent server mode connections will not be an option for me, nor would I imagine it to be acceptable for Debian-as-a-whole, although I don't speak for the Debian project in any way. Thanks in advance for your collective efforts, on behalf of all of the systems likely to be encountering this problem in the future (which should include most that install mailman on a Debian box.) -- From Harald.Meland@usit.uio.no Mon Feb 7 21:01:36 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 07 Feb 2000 22:01:36 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: Thomas Wouters's message of "Thu, 3 Feb 2000 10:35:55 +0100" References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <14488.44469.446995.168328@anthem.cnri.reston.va.us> <20000203103555.C6378@xs4all.nl> Message-ID: [Thomas Wouters] > Well, attached you'll find the first change, the cvs diff to LockFile.py. Sorry for not having been around here earlier, but: I've already been running a non-backwards LockFile.py in our Mailman installation here at uio.no for over a month. My current CVS checkout is available ("live" :) under , with being of special interest in this discussion... > I kept the API the same with one teensy exception. With the new > locking scheme, it's not possible to 'steal' a lock, only to break > the lock and then enter the lock() cycle to try and grab it. Umm, I'm not sure that would go down very well with all of (current) Mailman, as it would mean you'd have to reload any data that might have changed on disk (marshals etc.) if someone else got the lock before you... I've been taking the approach that the parts of Mailman that actually call steal() have to know what they're doing (and grepping seemed to indicate that that was correct -- it's mostly used by forked-off children to steal locks held by the parent), and thus steal() is awfully blunt and just overwrites the current lock. > I've tested this locker quite heavily, mailbombing my listserver > with 100+ simultaneous messages to mailman-test (which generated > 100+ bounces sent back to mailman, because my bouncing email address > is still on the list) and it hasn't barfed yet ;) Looks good -- I implemented my version of this due to severe locking problems when our Mailman server nearly ran out of swap when various log mails started arriving in hordes just after midnight... Combined with the mm_cfg.EX_TEMPFAIL trick seen in this has kept our Sun SPARCstation 20 with 160MB RAM running our ~3.500 lists quite smoothly for the last month. I'll try having a look at what differences there are in the two versions of this tomorrow. Cheers, -- Harald From thomas@xs4all.net Mon Feb 7 21:44:58 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 22:44:58 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: ; from Harald.Meland@usit.uio.no on Mon, Feb 07, 2000 at 10:01:36PM +0100 References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <14488.44469.446995.168328@anthem.cnri.reston.va.us> <20000203103555.C6378@xs4all.nl> Message-ID: <20000207224457.T23471@xs4all.nl> --oj4kGyHlBMXGt3Le Content-Type: text/plain; charset=us-ascii On Mon, Feb 07, 2000 at 10:01:36PM +0100, Harald Meland wrote: > My current CVS checkout is available ("live" :) under http://www.uio.no/~hmeland/tmp/mailman-userdb/>, with http://www.uio.no/~hmeland/tmp/mailman-userdb/Mailman/LockFile.py> > being of special interest in this discussion... Ah, very cool. You'll find my current version of LockFile.py attached, for reference, as you asked. I think the link() solutions, as opposed to the rename() solution, is a better one, if you'll premit me. For one, you can check on lock ownership by statting your own lockfile, which should only be touched by you, instead of reading a possibly highly volatile lockfile. You can also doublecheck the lockfile inode, see if it is equal to your own lockfiles' inode, to confirm that you have the lock. > > I kept the API the same with one teensy exception. With the new > > locking scheme, it's not possible to 'steal' a lock, only to break > > the lock and then enter the lock() cycle to try and grab it. > Umm, I'm not sure that would go down very well with all of (current) > Mailman, as it would mean you'd have to reload any data that might > have changed on disk (marshals etc.) if someone else got the lock > before you... > I've been taking the approach that the parts of Mailman that actually > call steal() have to know what they're doing (and grepping seemed to > indicate that that was correct -- it's mostly used by forked-off > children to steal locks held by the parent), and thus steal() is > awfully blunt and just overwrites the current lock. Ah, well, I couldn't find any parts of Mailman in the CVS tree that acually _called_ steal, so I couldn't figure out what it should do in borderline cases. However, reading your LockFile.py made me realise how to steal the lock. LockFile.steal() now does its utmost best to steal the lock, only failing if someone else is busy stealing the lock from us. It returns 1 for success, 0 for failure, mostly because I wanted it to signify that in some way. There still are race conditions in there, however, and I doubt we'll ever get rid of them. You assume the parent knows what it is doing, as where I assume the parent _knew_ what it is doing, but can be wrong by now ;) Most of these cases are, granted, _very_ low probability. Well, let me know if you have any questions or comments regarding my implementation. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --oj4kGyHlBMXGt3Le Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="LockFile.py" # Copyright (C) 1998 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. """Portable (?) file locking with timeouts. This code should work with all versions of NFS. The algorithm was suggested by the GNU/Linux open() man page. Make sure no malicious people have access to link() to the lock file. """ import socket, os, time import string import errno #from stat import ST_NLINK ST_NLINK = 3 # faster ST_CTIME = 9 ST_INO = 1 # default intervals are both specified in seconds, and can be floating point # values. DEFAULT_HUNG_TIMEOUT specifies the default length of time that a # lock is expecting to hold the lock -- this can be set in the constructor, or # changed via a mutator. DEFAULT_SLEEP_INTERVAL is the amount of time to # sleep before checking the lock's status, if we were not the winning claimant # the previous time around. DEFAULT_LOCK_LIFETIME = 15 DEFAULT_SLEEP_INTERVAL = 1.0 DEFAULT_INODE_GRACETIME = 2 from Mailman.Logging.StampedLogger import StampedLogger _logfile = None # exceptions which can be raised class LockError(Exception): """Base class for all exceptions in this module.""" pass class AlreadyLockedError(LockError): """Raised when a lock is attempted on an already locked object.""" pass class NotLockedError(LockError): """Raised when an unlock is attempted on an objec that isn't locked.""" pass class TimeOutError(LockError): """Raised when a lock was attempted, but did not succeed in the given amount of time. """ pass class StaleLockFileError(LockError): """Raised when a stale hardlink lock file was found.""" pass class LockFile: """A portable way to lock resources by way of the file system.""" def __init__(self, lockfile, lifetime=DEFAULT_LOCK_LIFETIME, sleep_interval=DEFAULT_SLEEP_INTERVAL, withlogging=0): """Creates a lock file using the specified file. lifetime is the maximum length of time expected to keep this lock. This value is written into the lock file so that other claimants on the lock know when it is safe to break the lock, should the lock holder be wedged. sleep_interval is roughly how often to wake up and check the lock file """ self.__lockfile = lockfile self.__lifetime = lifetime self.__sleep_interval = sleep_interval + (((os.getpid() % 20) - 11.5)/100.) self.__tmpfname = "%s.%s.%d" % (lockfile, socket.gethostname(), os.getpid()) self.__withlogging = withlogging self.__writelog("starting") def set_lifetime(self, lifetime): """Reset the lifetime of the lock. Takes affect the next time the file is locked. """ self.__lifetime = lifetime def __writelog(self, msg): global _logfile if self.__withlogging: if not _logfile: _logfile = StampedLogger('locks', manual_reprime=1) head, tail = os.path.split(self.__lockfile) _logfile.write('%s %s\n' % (tail, msg)) def refresh(self, newlifetime=None): """Refresh the lock. This writes a new release time into the lock file. Use this if a process suddenly realizes it needs more time to do its work. With optional newlifetime, this resets the lock lifetime value too. NotLockedError is raised if we don't already own the lock. """ if not self.locked(): raise NotLockedError if newlifetime is not None: self.set_lifetime(newlifetime) self.__write() self.__writelog('lock lifetime refreshed for %d seconds' % self.__lifetime) def __del__(self): if self.locked(): self.unlock() def __write(self): # we expect to release our lock some time in the future. we want to # give other claimants some clue as to when we think we're going to be # done with it, so they don't try to steal it out from underneath us # unless we've actually been wedged. lockrelease = time.time() + self.__lifetime # make sure it's group writable oldmask = os.umask(002) try: fp = open(self.__tmpfname, 'w') fp.write('%d %s %f\n' % (os.getpid(), self.__tmpfname, lockrelease)) fp.close() finally: os.umask(oldmask) def __read(self): # can raise ValueError in three situations: # # - The lockfile does not exist (it's been released or broken) # - We didn't get a sequence of the right size from split # - the first 'word' is not an integer # In each case we raise ValueError in response try: fp = open(self.__lockfile, 'r') except IOError, err: if err.errno == errno.ENOENT: raise ValueError # invalid contents try: pid, winner, lockrelease = string.split(string.strip(fp.read())) finally: fp.close() return int(pid), winner, float(lockrelease) def __break(self): try: if os.stat(self.__lockfile)[ST_CTIME] > time.time() - DEFAULT_INODE_GRACETIME: # The lockfile has changed very recently, so it might have # changed since our caller evaluated the validity return except OSError, err: if err.errno == errno.ENOENT: return # Someone broke it already raise os.unlink(self.__lockfile) # Note that this function is mostly redundant, the try/except around # os.link does almost all of what we do. Instead of lockfile-contents # we could use lockfile CTIME, which would probably speed things up. def lock(self, timeout=0): """Blocks until the lock can be obtained. Raises a TimeOutError exception if a positive timeout value is given and that time elapses before the lock is obtained. This can possibly steal the lock from some other claimant, if the lock lifetime that was written to the file has been exceeded. Note that for this to work across machines, the clocks must be sufficiently synchronized. """ if self.locked(): self.__writelog('already locked') raise AlreadyLockedError if timeout: timeout_time = time.time() + timeout last_pid = -1 self.__write() # Make sure our own lockfile exists while 1: # create the hard link and test for exactly 2 links to the file try: os.link(self.__tmpfname, self.__lockfile) except OSError, err: if err.errno == errno.ENOENT: # Our own lockfile has vanished from beneath us ? self.__write() # sleep anyway, so we dont hog the machine self.__sleep() continue elif err.errno == errno.EEXIST: # We failed to get the lock. If we worry about this locking # scheme, we set a variable here and ascertain ourselves of # its value later, when we think we have the lock. pass else: raise if os.stat(self.__tmpfname)[ST_NLINK] == 2: # we have the lock (since noone overwrote our link to the lock # file), so we re-piss our hydrant, to refresh the lock timeout self.__write() now = time.time() if not self.locked(): self.__writelog('false positive on lockfile (%f)'%now) self.__sleep() continue self.__writelog('got the lock (%f)'%now) break # we didn't get the lock this time. let's see if we timed out if timeout and timeout_time < time.time(): os.unlink(self.__tmpfname) self.__writelog('timed out') raise TimeOutError # someone else must have gotten the lock. let's find out who it # is. if there is some bogosity in the lock file's data then we # will try to steal the lock. try: pid, winner, lockrelease = self.__read() except ValueError: self.__writelog('invalid lock contents; breaking lock') self.__break() self.__sleep() continue # check for hanging processes, by remembering pid, and if it stays # the same, checking lockrelease. if pid <> last_pid: last_pid = pid elif lockrelease < time.time(): # We've waited long enough... # Attempt to break the lock. We dont care much about the old # winners __tmpfname, let it stay. self.__writelog('lock timeout; breaking lock') self.__break() self.__sleep() continue # okay, someone else has the lock, we weren't able to steal it, # and our claim hasn't timed out yet. So let's wait a while for # the owner of the lock to give it up. self.__writelog('waiting for claim') self.__sleep() def __sleep(self): time.sleep(self.__sleep_interval) # This could error if the lock is stolen. You must catch it. def unlock(self): """Unlock the lock. If we don't already own the lock (either because of unbalanced unlock calls, or because the lock was stolen out from under us), raise a NotLockedError. """ if not self.locked(): raise NotLockedError now = time.time() os.unlink(self.__lockfile) os.unlink(self.__tmpfname) self.__writelog('unlocked (%f)'%now) def locked(self): """Returns 1 if we own the lock, 0 if we do not.""" if not os.path.exists(self.__tmpfname): return 0 if os.stat(self.__tmpfname)[ST_NLINK] <> 2: return 0 # The above check for nlink should be enough to validate our claim on # the lock. Nevertheless, we are paranoid. try: pid, winner, lockrelease = self.__read() except ValueError: # the contents of the lock file was corrupted, or the global lock # is not a link to our lock. if os.stat(self.__lockfile)[ST_INO] <> os.stat(self.__tmpfname)[ST_INO]: self.__writelog("nlink of 2 but we don't own the lock ? strangeness") os.unlink(self.__tmpfname) self.__write() return 0 else: # Okay, we really do have the lock, but the contents is messed up self.__write() return 1 return pid == os.getpid() # use with caution!!! def steal(self): """Explicitly steal the lock. USE WITH CAUTION!""" self.__writelog('explicitly breaking lock') if self.locked(): return 1 try: pid, winner, timeout = self.__read() os.rename(winner, self.__tmpfname) except ValueError: return 0 except OSError, err: if err.errno == errno.ENOENT: return 0 raise self.__write() return self.locked() --oj4kGyHlBMXGt3Le-- From thomas@xs4all.net Mon Feb 7 21:56:07 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 22:56:07 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207200918.A18352@overdue.dhis.net>; from lazarus@overdue.dhis.org on Mon, Feb 07, 2000 at 08:09:18PM +0000 References: <20000207171959.K23471@xs4all.nl> <20000207200918.A18352@overdue.dhis.net> Message-ID: <20000207225606.U23471@xs4all.nl> --3O1VwFp74L81IIeR Content-Type: text/plain; charset=us-ascii On Mon, Feb 07, 2000 at 08:09:18PM +0000, Lazarus Long wrote: > On Mon, Feb 07, 2000 at 06:16:06PM +0100, Gergely Madarasz wrote: > > Thanks. > > Lazarus, please test this patch, and notify me if it worked for you :) > Patching file `gate_news' using Plan A... > Hunk #1 FAILED at 125. > 1 out of 1 hunk FAILED -- saving rejects to gate_news.rej Ok, the patch doesn't work for mailman-1.1 gate_news ;P I've attached a version that should. Let me know if it still fails, or if you're using a different version of mailman. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --3O1VwFp74L81IIeR Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="gate_news.diff" --- cron/gate_news.orig Mon Feb 7 22:47:52 2000 +++ cron/gate_news Mon Feb 7 22:53:23 2000 @@ -76,7 +76,16 @@ # update the gate_watermarks file. we'll actually do the gating in a # child process conn = nntplib.NNTP(mlist.nntp_host) - r,c,first,last,n = conn.group(mlist.linked_newsgroup) + try: + r,c,first,last,n = conn.group(mlist.linked_newsgroup) + except nntplib.error_perm: + # We got an error 500, probably meaning that the 'group' command + # is not implemented. A likely scenario causing this is when + # running the Mailman gateway on the same machine as the news + # server; the server defaults to news feed mode. We need to set + # the connection to reader mode first. + conn.shortcmd('mode reader') + r,c,first,last,n = conn.group(mlist.linked_newsgroup) first = int(first) last = int(last) wm = watermarks.get(name, 0) --3O1VwFp74L81IIeR-- From thomas@xs4all.net Mon Feb 7 21:59:07 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Feb 2000 22:59:07 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000207213143.B444@miss-janet.com>; from ricardo@rixhq.nu on Mon, Feb 07, 2000 at 09:31:44PM +0100 References: <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> <20000207204331.A444@miss-janet.com> <20000207213143.B444@miss-janet.com> Message-ID: <20000207225907.M19265@xs4all.nl> On Mon, Feb 07, 2000 at 09:31:44PM +0100, Ricardo Kustner wrote: > I wonder though what happens if some impatient moderator decides not to > wait before the page finnishes loading, switches to a differen webpage and > therefor breaks the python cgi process... will some approved posts stay in > the queue instead? Try it out ! ;) It depends on the exact behaviour of both the webbrowser and the webserver. I haven't checked the Mailman code but i assume it either continues with its jobs until it tries to write output, usually at the end of the script, or it gets a signal and cleans up nicely. Postings shouldn't disappear, if that's what you mean. > I mentioned before that I think that it could be better if the cgi scripts > don't do anything more than just "mark" message as being approved... cgi > scripts should have a short life time and definately shouldn't be waiting > for something too long... I agree, and I think I've seen more comments and postings talking about it. Noone has implemented it yet, though, and the current method works too well for it to be a rela problem, I guess ;) > btw i used the original LockFile.py this time... Go ahead and try it with mine -- it should make the waits you have to endure a tad shorter. The problem with the old locking mechanism is that a lot of processes end up butting heads in a painful way when they try to grab the lock. In the modified one, the first process to try and get the lock after it is freed, actually gets the lock. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Tue Feb 8 01:05:27 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 8 Feb 2000 02:05:27 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval Message-ID: <20000208020526.N19265@xs4all.nl> --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Another minor patch adding what I need to mailman ;) I also noticed this on the jitterbug list, so I guess more people had wished or at least wondered for it ;P I'm posting it mostly to see if it generates comments, and for me to find a good excuse to sleep. It's functional, it works on my test-list, but some of it (most notably the english -- is there a good word for 'unsubscription request' ?) sucks. Adding this has been mostly an exercise to learn a bit about the Mailman insides, and to see how easily it can be done; fairly easily ;) Comments are welcome, including "d'oh, dont write that, i already wrote it !" The patchs misses the file 'templates/unsubauth.txt' by the way.. Just copy subauth.txt and change 'subscr' into 'unsubscr' ;P Sleepy-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mailman-unsub-approval.diff" ? templates/unsubauth.txt Index: Mailman/Bouncer.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Bouncer.py,v retrieving revision 1.37 diff -u -r1.37 Bouncer.py --- Bouncer.py 1999/12/16 17:11:04 1.37 +++ Bouncer.py 2000/02/08 00:57:47 @@ -299,7 +299,7 @@ self.real_name, addr, reason) return reason, 1 try: - self.DeleteMember(addr, "bouncing addr") + self.ApprovedDeleteMember(addr, "bouncing addr") self.LogMsg("bounce", "%s: removed %s", self.real_name, addr) self.Save() return 1, 1 Index: Mailman/Defaults.py.in =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Defaults.py.in,v retrieving revision 1.90 diff -u -r1.90 Defaults.py.in --- Defaults.py.in 1999/11/30 23:10:57 1.90 +++ Defaults.py.in 2000/02/08 00:57:47 @@ -344,4 +344,4 @@ from Version import VERSION # Data file version number -DATA_FILE_VERSION = 15 +DATA_FILE_VERSION = 16 Index: Mailman/HTMLFormatter.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/HTMLFormatter.py,v retrieving revision 1.46 diff -u -r1.46 HTMLFormatter.py --- HTMLFormatter.py 1999/11/24 21:15:21 1.46 +++ HTMLFormatter.py 2000/02/08 00:57:47 @@ -206,6 +206,11 @@ " request will be sent to the '%s' account for" " your address.)" % self.umbrella_member_suffix) + if self.unsubscribe_policy == 1: + msg = msg + ("

(Please note that this list does not allow" + " members to unsubscribe themselves without" + " administrator approval. Unsubscription requests" + " will get forwarded to the mailinglist administrator.)") return msg def FormatUndigestButton(self): Index: Mailman/ListAdmin.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/ListAdmin.py,v retrieving revision 1.27 diff -u -r1.27 ListAdmin.py --- ListAdmin.py 1999/11/24 21:13:59 1.27 +++ ListAdmin.py 2000/02/08 00:57:48 @@ -40,6 +40,7 @@ # Request types requiring admin approval HELDMSG = 1 SUBSCRIPTION = 2 +UNSUBSCRIPTION = 3 @@ -106,6 +107,9 @@ def GetSubscriptionIds(self): return self.__getmsgids(SUBSCRIPTION) + def GetUnsubscriptionIds(self): + return self.__getmsgids(UNSUBSCRIPTION) + def GetRecord(self, id): self.__opendb() type, data = self.__db[id] @@ -122,9 +126,11 @@ del self.__db[id] if rtype == HELDMSG: self.__handlepost(data, value, comment) - else: - assert rtype == SUBSCRIPTION + elif rtype == SUBSCRIPTION: self.__handlesubscription(data, value, comment) + else: + assert rtype == UNSUBSCRIPTION + self.__handleunsubscription(data, value, comment) def HoldMessage(self, msg, reason): # assure that the database is open for writing @@ -258,6 +264,53 @@ assert value == 1 self.ApprovedAddMember(addr, password, digest) + def HoldUnsubscription(self, addr, whence, admin_notif): + # assure that the database is open for writing + self.__opendb() + # get the next unique id + id = self.__request_id() + assert not self.__db.has_key(id) + # + # save the information to the request database. for held subscription + # entries, each record in the database will be one of the following + # format: + # + # the time the subscription request was received + # the subscriber's address + # the log message + # wether or not to send an unsubscription notify to the list admin + # + data = time.time(), addr, whence, admin_notif + self.__db[id] = (UNSUBSCRIPTION, data) + # + # TBD: this really shouldn't go here but I'm not sure where else is + # appropriate. + self.LogMsg('vette', '%s: held unsubscription request from %s' % + (self.real_name, addr)) + # possibly notify the administrator + if self.admin_immed_notify: + subject = 'New unsubscription request to list %s from %s' % ( + self.real_name, addr) + text = Utils.maketext( + 'unsubauth.txt', + {'username' : addr, + 'listname' : self.real_name, + 'hostname' : self.host_name, + 'admindb_url': self.GetAbsoluteScriptURL('admindb'), + }) + adminaddr = self.GetAdminEmail() + msg = Message.UserNotification(adminaddr, adminaddr, subject, text) + HandlerAPI.DeliverToUser(self, msg) + + def __handleunsubscription(self, record, value, comment): + stime, addr, whence, admin_notif = record + if value == 0: + # refused + self.__refuse('Unsubscription request', addr, comment) + else: + # subscribe + assert value == 1 + self.ApprovedDeleteMember(addr, whence, admin_notif) def __refuse(self, request, recip, comment, origmsg=None): adminaddr = self.GetAdminEmail() Index: Mailman/MailCommandHandler.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/MailCommandHandler.py,v retrieving revision 1.59 diff -u -r1.59 MailCommandHandler.py --- MailCommandHandler.py 1999/12/16 17:13:28 1.59 +++ MailCommandHandler.py 2000/02/08 00:57:48 @@ -500,6 +500,8 @@ self.ConfirmUserPassword(addr, args[0]) self.DeleteMember(addr, "mailcmd") self.AddToResponse("Succeeded.") + except Errors.MMNeedApproval: + self.AddToResponse("Unsubscribing needs admininistrator approval.\nYour request has been forwarded to the list administrator.") except Errors.MMListNotReady: self.AddError("List is not functional.") except (Errors.MMNoSuchUserError, Errors.MMNotAMemberError): Index: Mailman/MailList.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/MailList.py,v retrieving revision 1.151 diff -u -r1.151 MailList.py --- MailList.py 1999/12/27 22:54:55 1.151 +++ MailList.py 2000/02/08 00:57:49 @@ -339,6 +339,7 @@ self.welcome_msg = '' self.goodbye_msg = '' self.subscribe_policy = mm_cfg.DEFAULT_SUBSCRIBE_POLICY + self.unsubscribe_policy = mm_cfg.DEFAULT_UNSUBSCRIBE_POLICY self.private_roster = mm_cfg.DEFAULT_PRIVATE_ROSTER self.obscure_addresses = mm_cfg.DEFAULT_OBSCURE_ADDRESSES self.member_posting_only = mm_cfg.DEFAULT_MEMBER_POSTING_ONLY @@ -599,6 +600,17 @@ sub_cfentry, + ('unsubscribe_policy', mm_cfg.Radio, + ('none', 'require approval'), 0, + "What steps are required for unsubscription?
", + "none - anyone can unsubscribe using email or the webpage
" + "require approval - require list administrator approval" + " for unsubscriptions
" + "

Please leave the unsubscription 'open' unless you have a" + " very good reason to require approval. Leaving people on lists" + " against their will can easily be seen as spamming or harassment" + ), + "Membership exposure", ('private_roster', mm_cfg.Radio, @@ -1083,7 +1095,7 @@ HandlerAPI.DeliverToUser(self, msg) return result - def DeleteMember(self, name, whence=None, admin_notif=None): + def ApprovedDeleteMember(self, name, whence=None, admin_notif=None): self.IsListInitialized() # FindMatchingAddresses *should* never return more than 1 address. # However, should log this, just to make sure. @@ -1139,6 +1151,15 @@ whence = "" self.LogMsg("subscribe", "%s: deleted %s%s", self._internal_name, name, whence) + + def DeleteMember(self, name, whence=None, admin_notif=None): + self.IsListInitialized() + # FindMatchingAddresses *should* never return more than 1 address. + # However, should log this, just to make sure. + if self.unsubscribe_policy == 1: + self.HoldUnsubscription(name, whence, admin_notif) + raise Errors.MMNeedApproval + ApprovedDeleteMember(name, whence, admin_notif) def IsMember(self, address): return len(Utils.FindMatchingAddresses(address, self.members, Index: Mailman/Cgi/admin.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Cgi/admin.py,v retrieving revision 1.53 diff -u -r1.53 admin.py --- admin.py 1999/12/27 22:28:07 1.53 +++ admin.py 2000/02/08 00:57:50 @@ -887,7 +887,7 @@ for user in users: if not cgi_info.has_key('%s_subscribed' % (user)): try: - mlist.DeleteMember(user) + mlist.ApprovedDeleteMember(user) dirty = 1 except Errors.MMNoSuchUserError: unsubscribe_errors.append((user, 'Not subscribed')) Index: Mailman/Cgi/admindb.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Cgi/admindb.py,v retrieving revision 1.19 diff -u -r1.19 admindb.py --- admindb.py 1999/11/15 22:29:46 1.19 +++ admindb.py 2000/02/08 00:57:50 @@ -164,6 +164,19 @@ for id in subpendings: PrintAddMemberRequest(mlist, id, t) form.AddItem(t) + unsubpendings = mlist.GetUnsubscriptionIds() + if unsubpendings: + form.AddItem('

') + form.AddItem(Center(Header(2, 'Unsubscription Requests'))) + t = Table(border=2) + t.AddRow([ + Bold('Address'), + Bold('Your Decision'), + Bold('Reason for unsubscription refusal (optional)') + ]) + for id in unsubpendings: + PrintDeleteMemberRequest(mlist, id, t) + form.AddItem(t) # Post holds are now handled differently heldmsgs = mlist.GetHeldMessageIds() total = len(heldmsgs) @@ -183,6 +196,13 @@ time, addr, passwd, digest = mlist.GetRecord(id) table.AddRow([addr, RadioButtonArray(id, ('Refuse', 'Subscribe')), + TextBox('comment-%d' % id, size=30) + ]) + +def PrintDeleteMemberRequest(mlist, id, table): + time, addr, whence, admin_notif = mlist.GetRecord(id) + table.AddRow([addr, + RadioButtonArray(id, ('Refuse', 'Unsubscribe')), TextBox('comment-%d' % id, size=30) ]) Index: Mailman/Cgi/handle_opts.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Cgi/handle_opts.py,v retrieving revision 1.18 diff -u -r1.18 handle_opts.py --- handle_opts.py 2000/01/21 20:30:51 1.18 +++ handle_opts.py 2000/02/08 00:57:50 @@ -101,6 +101,12 @@ pw = form["upw"].value if mlist.ConfirmUserPassword(user, pw): mlist.DeleteMember(user, "web cmd") + except Errors.MMNeedApproval: + PrintResults(mlist, operation, doc, + "Unsubscribing from this list requires " + "administrator approval. Your request has been " + "forwarded. Please contact the list administrator " + "if you have any questions.") except Errors.MMListNotReady: PrintResults(mlist, operation, doc, "List is not functional.") except Errors.MMNoSuchUserError: Index: bin/remove_members =================================================================== RCS file: /projects/cvsroot/mailman/bin/remove_members,v retrieving revision 1.3 diff -u -r1.3 remove_members --- remove_members 1999/03/04 17:18:24 1.3 +++ remove_members 2000/02/08 00:57:50 @@ -100,7 +100,7 @@ for addr in addresses: try: - mlist.DeleteMember(addr) + mlist.ApprovedDeleteMember(addr) except Errors.MMNoSuchUserError: print "User `%s' not found." % addr Index: bin/sync_members =================================================================== RCS file: /projects/cvsroot/mailman/bin/sync_members,v retrieving revision 1.8 diff -u -r1.8 sync_members --- sync_members 1999/11/30 21:17:33 1.8 +++ sync_members 2000/02/08 00:57:50 @@ -244,7 +244,7 @@ for laddr, addr in addrs.items(): # should be a member, otherwise our test above is broken if not dryrun: - mlist.DeleteMember(addr, admin_notif=notifyadmin) + mlist.ApprovedDeleteMember(addr, admin_notif=notifyadmin) print 'Removed: %30s (%30s)' % (laddr, addr) mlist.Save() --ZPt4rx8FFjLCG7dd-- From jwt@dskk.co.jp Tue Feb 8 02:47:06 2000 From: jwt@dskk.co.jp (Jim Tittsler) Date: Tue, 8 Feb 2000 11:47:06 +0900 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000207191713.L19265@xs4all.nl>; from thomas@xs4all.net on Mon, Feb 07, 2000 at 07:17:13PM +0100 References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> Message-ID: <20000208114706.C2908@mail.dskk.co.jp> --FCuugMFkClbJLl1L Content-Type: text/plain; charset=us-ascii On Mon, Feb 07, 2000 at 07:17:13PM +0100, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 12:53:17PM -0500, Barry A. Warsaw wrote: > > > I'm cutting and pasting a message that originated on the mailman-users > > mailing list. I think this message is fairly self-explanatory. Can > > other NNTP experts look this over and see if it makes sense? I'll > > apply it if there are no objections. > > Note: the bare except is bothersome. I confess I wasn't sure if a news server that didn't recognize the 'mode reader' could be counted on to return a 500 series error. It seemed logical, but I couldn't be sure. (All of the servers I have access to accept the 'mode reader'.) > Note, though, that i've already posted a small patch to mailman to fix this. > Since the 'group' command is executed by gate_news, which doesn't need > authentication, it's easy enough to try the group, and set the connection to > readermode if it fails. I dislike your proposed solution for two reasons: 1. It uses the nntplib's shortcmd outside of nntplib, even though it is documented as "internal". 2. It doesn't work when you *do* need authentication. The group command will claim 'No such group' for groups that are hidden by authentication. (One of my mailman run on such a machine.) You need to do the 'mode reader' before authenticating. I still think it better to change nntplib. > Also note that I've already sent a patch to patches@python.org ;) with a > pointer to the mailman-users discussion in question. > > http://www.python.org/pipermail/patches/2000-February/000015.html > > My patch uses an extra argument to NNTP.__init__, and avoids the bare > except. To get the same behaviour as the patch below, sending 'mode > reader' by default, the default value of readermode can be set to 1. I'm not > sure if that's a wise thing, however, as some users of nntplib.NNTP might > depend on the connection being able to do things that you cant do in > readermode. I sent a similar nntplib patch to mailman-users last week, although I defaulted to the current behavior and required the additional argument (or a user specified for authentication) to get the "newsreader" behavior. http://www.python.org/pipermail/mailman-users/2000-February/003935.html Jim --FCuugMFkClbJLl1L Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="nntplib.diff" --- /home/jwt/nntplib.py.orig Sat Sep 18 09:16:08 1999 +++ nntplib.py Sat Sep 18 11:39:00 1999 @@ -61,17 +61,24 @@ # Initialize an instance. Arguments: # - host: hostname to connect to # - port: port to connect to (default the standard NNTP port) + # - mode: if not None (or user is specified), issue 'mode reader' - def __init__(self, host, port = NNTP_PORT, user=None, password=None): + def __init__(self, host, port = NNTP_PORT, user=None, password=None, + mode=None): self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.sock.connect(self.host, self.port) self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + if user or mode: + try: + self.welcome = self.shortcmd('mode reader') + except: + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: @@ -466,9 +473,9 @@ # Minimal test function def _test(): - s = NNTP('news') + s = NNTP('news', mode='reader') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last resp, subs = s.xhdr('subject', first + '-' + last) --FCuugMFkClbJLl1L-- From Harald.Meland@usit.uio.no Tue Feb 8 02:58:47 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 08 Feb 2000 03:58:47 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: Thomas Wouters's message of "Mon, 7 Feb 2000 22:44:58 +0100" References: <20000202140042.J19725@xs4all.nl> <20000202225346.B17313@xs4all.nl> <14488.44469.446995.168328@anthem.cnri.reston.va.us> <20000203103555.C6378@xs4all.nl> <20000207224457.T23471@xs4all.nl> Message-ID: [Thomas Wouters] > On Mon, Feb 07, 2000 at 10:01:36PM +0100, Harald Meland wrote: > > > My current CVS checkout is available ("live" :) under > http://www.uio.no/~hmeland/tmp/mailman-userdb/>, with > http://www.uio.no/~hmeland/tmp/mailman-userdb/Mailman/LockFile.py> > > being of special interest in this discussion... > > Ah, very cool. You'll find my current version of LockFile.py > attached, for reference, as you asked. Thanks, I've skimmed through most of it now. > I think the link() solutions, as opposed to the rename() solution, > is a better one, if you'll premit me. Sure :) > For one, you can check on lock ownership by statting your own > lockfile, which should only be touched by you, instead of reading a > possibly highly volatile lockfile. You can also doublecheck the > lockfile inode, see if it is equal to your own lockfiles' inode, to > confirm that you have the lock. ... the downside being that if, e.g., anyone tries to read the info in the lockfile in the middle of a refresh(), they'll get a ValueError -- which your module seems to take care of, but it all looks a bit convoluted to my eyes. I especially dislike that lock() has to call locked() _after_ the check on ST_NLINK, because further down in lock() there's code that might break a lock it really shouldn't (if it weren't for the nonatomic-write-of-lock-contents issue). I also see some bugs that are fixed in my module but still present in yours, but those are rather irrelevant to this discussion (as they can easily be ported to your module if that's the one we'll go for). Does anyone but Thomas and me have any opinions on which locking scheme is preferrable? I guess I don't have any very strong feelings about the issue ;), so any sagely advise on the matter would be welcome. [ I've tried to model my version of the module as closely as possible after the way Exim does its lockfile locking, the idea being that Exim really should have been around long enough to have solved these things correctly. Reading the longish comment from ca. line 1206 and onwards in src/transports/appendfile.c of the Exim distribution proved very helpful to me. Of course, when it comes to lock stealing and other ugly (but needed, I guess) stuff, Exim couldn't provide much help. ] > Ah, well, I couldn't find any parts of Mailman in the CVS tree that > acually _called_ steal, so I couldn't figure out what it should do > in borderline cases. The only place I know of is cron/gate_news. > However, reading your LockFile.py made me realise how to steal the > lock. LockFile.steal() now does its utmost best to steal the lock, > only failing if someone else is busy stealing the lock from us. It > returns 1 for success, 0 for failure, mostly because I wanted it to > signify that in some way. ... but the caller should note that steal() returning 1 does not necessarily imply that it is the owner of the lock (anymore). The steal() stuff is bound to be full of race conditions, AFAICT -- so whether it returns anything doesn't really matter (IMHO), as the return code truly can't be trusted anyway. Cheers, -- Harald From Harald.Meland@usit.uio.no Tue Feb 8 03:06:15 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 08 Feb 2000 04:06:15 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: Thomas Wouters's message of "Mon, 7 Feb 2000 22:59:07 +0100" References: <20000202225346.B17313@xs4all.nl> <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> <20000207204331.A444@miss-janet.com> <20000207213143.B444@miss-janet.com> <20000207225907.M19265@xs4all.nl> Message-ID: [Thomas Wouters] > On Mon, Feb 07, 2000 at 09:31:44PM +0100, Ricardo Kustner wrote: > > > I wonder though what happens if some impatient moderator decides not to > > wait before the page finnishes loading, switches to a differen webpage and > > therefor breaks the python cgi process... will some approved posts stay in > > the queue instead? > > Try it out ! ;) It depends on the exact behaviour of both the > webbrowser and the webserver. Yup. This might produce a SIGPIPE, and/or the corresponding IOError EPIPE, and I've experienced that these didn't always get caught -- an attempt at a fix is near the end of run_main in , but I'm by no means positive that it's the Right fix. -- Harald From thomas@xs4all.net Tue Feb 8 08:43:18 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 8 Feb 2000 09:43:18 +0100 Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) In-Reply-To: <20000208114706.C2908@mail.dskk.co.jp>; from jwt@dskk.co.jp on Tue, Feb 08, 2000 at 11:47:06AM +0900 References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> Message-ID: <20000208094318.Z23471@xs4all.nl> On Tue, Feb 08, 2000 at 11:47:06AM +0900, Jim Tittsler wrote: > On Mon, Feb 07, 2000 at 07:17:13PM +0100, Thomas Wouters wrote: > > Note, though, that i've already posted a small patch to mailman to fix this. > > Since the 'group' command is executed by gate_news, which doesn't need > > authentication, it's easy enough to try the group, and set the connection to > > readermode if it fails. > > I dislike your proposed solution for two reasons: > 1. It uses the nntplib's shortcmd outside of nntplib, even though it is > documented as "internal". > 2. It doesn't work when you *do* need authentication. The group command > will claim 'No such group' for groups that are hidden by authentication. > (One of my mailman run on such a machine.) You need to do the 'mode > reader' before authenticating. > I still think it better to change nntplib. So do I. However, Mailman does currently *not* use authentication there, so this is the simplest fix, including backwards to mailman 1.1. or earlier. I was not aware that shortcmd() is nntplib-private, though. I assumed it was public because it does not start with a _. If this is a problem, fixing nntplib and using a private copy might be the better solution. > > Also note that I've already sent a patch to patches@python.org ;) with a > > pointer to the mailman-users discussion in question. > > http://www.python.org/pipermail/patches/2000-February/000015.html > I sent a similar nntplib patch to mailman-users last week, although I Ah, sorry, I dont read mailman-users, apparently. I thought I did, though, Strange. I'm sorry if I seem to have stolen your idea, it was not intentional. Anyway, the right place for this patch is not Mailman, it is Python. nntplib is part of the standard library, nowadays, Mailman only has its own local copy for compatibility with older python distributions. I'm a bit fearful of setting readermode so often, though -- aren't there possibly cases where people want to provide authentication, but do not want readermode ? (I'm not concerned with newsservers that dont understand 'mode reader'. If they barf on that, they are highly likely to barf on everything else you try out, too. I am concerned with setting a newsconnection to reader when you dont want a reader -- though I'm not sure what would be the effect.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From julian7@kva.hu Tue Feb 8 15:34:56 2000 From: julian7@kva.hu (Balazs Nagy) Date: Tue, 8 Feb 2000 16:34:56 +0100 (CET) Subject: [Mailman-Developers] Reply goes to list error In-Reply-To: Message-ID: On 7 Feb 2000, Harald Meland wrote: > > On Tue, 18 Jan 2000, Balazs Nagy wrote: > > > > I just cannot understand why nobody from thosw who have write access > > did anything about the subject. > > Because Reply-To: munging is already part of CookHeaders.py, I guess. > And that takes appropriate note of the "fastrack" attribute of the > message object, too. CookHeaders doesn't do anything about the subject. It changes Reply-To and deletes From fields when the list is anonymous. There's an option (a deprecated option) to do it, but some ways it can be handy. It's a feature which is declared in the administration web page and it doesn't work. Why nobody want to fix it? -- Regards: Kevin (Balazs) From Ola.Flygt@msi.vxu.se Tue Feb 8 14:46:48 2000 From: Ola.Flygt@msi.vxu.se (Ola Flygt) Date: Tue, 8 Feb 2000 15:46:48 +0100 Subject: [Mailman-Developers] editing email addresses Message-ID: Hello At V=E4xj=F6 University we are using Mailman for a number of lists. We are now in the process of changing email addresses for a lot of users since we have a change in the organization. Is there a way to change all the users email addresses in the database instead of deleting and adding them again? Best Regards, Ola Flygt From Harald.Meland@usit.uio.no Tue Feb 8 15:25:18 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 08 Feb 2000 16:25:18 +0100 Subject: [Mailman-Developers] Reply goes to list error In-Reply-To: Balazs Nagy's message of "Tue, 8 Feb 2000 16:34:56 +0100 (CET)" References: Message-ID: [Balazs Nagy] > On 7 Feb 2000, Harald Meland wrote: > > > > On Tue, 18 Jan 2000, Balazs Nagy wrote: > > > > > > I just cannot understand why nobody from thosw who have write access > > > did anything about the subject. > > > > Because Reply-To: munging is already part of CookHeaders.py, I guess. > > And that takes appropriate note of the "fastrack" attribute of the > > message object, too. > > CookHeaders doesn't do anything about the subject. Ooops, sorry -- my version did, but I hadn't checked that in. Checked in now. -- Harald From gerrit@nl.linux.org Tue Feb 8 15:20:34 2000 From: gerrit@nl.linux.org (Gerrit Holl) Date: Tue, 8 Feb 2000 16:20:34 +0100 Subject: [Mailman-Developers] Two more TODO entries Message-ID: <20000208162034.A3621@stopcontact.palga.uucp> Hello, I have two TODO list entries: * I think a nice feature would be to set headers. You could set an option "setheader", "header" and "content", with a boolean option "override". I think it should be added to the TODO list. Maybe Barry (and others) think Reply-To sucks, but when it's not set it's certainly a nice feature to have it set. I've added this hack at my site: if not msg.get('Reply-To'): msg['Reply-To'] = mlist.GetListEmail() in Mailman.Handlers.Cookheaders.process Because currently *all* mailinglist use it. * A mail admin having an own page with all lists he is admin of, soe can approve messages and subscribsions from a single place. I'm wanting to contribute to the documentation. Has this ever been discussed yet? If not, I'm going to start with a document using the Linuxdoc DTD as soon as I understand all Mailman code (which'll take quite a long time, to be fair). regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From gerrit@nl.linux.org Tue Feb 8 14:30:43 2000 From: gerrit@nl.linux.org (Gerrit Holl) Date: Tue, 8 Feb 2000 15:30:43 +0100 Subject: [Mailman-Developers] Patch: raising exception when user can't change admin password Message-ID: <20000208153043.A3437@stopcontact.palga.uucp> --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Hello, I'm not sure wheter this is the right place to send patches. I sent a patch to mailman-cabal but it didn't seem to be the right place. Redirect me if I'm wrong. This patch, to SecurityManager.py, raises the MMAuthenticationError if an unauthorized user tries to change the site admin password. Currently, you just het an IOError. I find the currently unused MMAuthenticationError more clear. regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="SecurityManager.py.diff" --- SecurityManager.py Tue Feb 8 15:27:20 2000 +++ /tmp/SecurityManager.py Tue Feb 8 15:25:13 2000 @@ -36,7 +36,10 @@ class SecurityManager: def SetSiteAdminPassword(self, pw): - fp = Utils.open_ex(SITE_PW_FILE, 'w', perms=0640) + try: + fp = Utils.open_ex(SITE_PW_FILE, 'w', perms=0640) + except IOError: + raise Errors.MMAuthenticationError("no permission to change password") fp.write(Crypt.crypt(pw, Utils.GetRandomSeed())) fp.close() --Dxnq1zWXvFF0Q93v-- From bwarsaw@cnri.reston.va.us Tue Feb 8 17:08:09 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Tue, 8 Feb 2000 12:08:09 -0500 (EST) Subject: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> Message-ID: <14496.19833.341973.55133@anthem.cnri.reston.va.us> >>>>> "JT" == Jim Tittsler writes: JT> I still think it better to change nntplib. I agree. I'd still like to get rid of your bare except though; I'm guessing that error_perm is probably correct because of this: >>> import nntplib >>> x = nntplib.NNTP('ournews') >>> x.shortcmd('mode foobar') Traceback (innermost last): File "", line 1, in ? File "/depot/sundry/lib/python1.5/nntplib.py", line 158, in shortcmd return self.getresp() File "/depot/sundry/lib/python1.5/nntplib.py", line 134, in getresp raise error_perm, resp nntplib.error_perm: 500 Syntax error or bad command The Right Thing To Do is to recast the nntplib errors as exception classes and perhaps use it's base class in the except. I'd be willing to make this change, and merge in your patch, but you'll need to resend your patch with the standard email disclaimer appended. See http://www.python.org/patches/ for details. Would you be willing to do that? Thanks, -Barry From Gerrit Tue Feb 8 21:28:14 2000 From: Gerrit (Gerrit Holl) Date: Tue, 8 Feb 2000 22:28:14 +0100 Subject: [Mailman-Developers] When will mailman 1.2 be released? Message-ID: <20000208222814.A3736@stopcontact.palga.uucp> Hello, when do you think mailman 1.2 will be released? How stable is the current CVS tree, and how heavily is it tested? In what order of magnitude is the release frequency? regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From ricardo@rixhq.nu Tue Feb 8 21:48:51 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Tue, 8 Feb 2000 22:48:51 +0100 Subject: [Mailman-Developers] race condition in locking ? In-Reply-To: <20000207225907.M19265@xs4all.nl>; from thomas@xs4all.net on Mon, Feb 07, 2000 at 10:59:07PM +0100 References: <20000204205630.A4871@miss-janet.com> <20000204212243.B4871@miss-janet.com> <20000204234757.A2826@xs4all.nl> <20000205011507.A405@miss-janet.com> <20000205164318.A15909@xs4all.nl> <20000205205653.A4105@miss-janet.com> <20000205232130.D23471@xs4all.nl> <20000207204331.A444@miss-janet.com> <20000207213143.B444@miss-janet.com> <20000207225907.M19265@xs4all.nl> Message-ID: <20000208224851.B735@miss-janet.com> On Mon, Feb 07, 2000 at 10:59:07PM +0100, Thomas Wouters wrote: > On Mon, Feb 07, 2000 at 09:31:44PM +0100, Ricardo Kustner wrote: > > I wonder though what happens if some impatient moderator decides not to > > wait before the page finnishes loading, switches to a differen webpage and > > therefor breaks the python cgi process... will some approved posts stay in > > the queue instead? > Try it out ! ;) It depends on the exact behaviour of both the webbrowser and > the webserver. I haven't checked the Mailman code but i assume it either > continues with its jobs until it tries to write output, usually at the end > of the script, or it gets a signal and cleans up nicely. if you approve a bunch of posts it's a bit difficult work to figure out which posts arrive... especially since my server needs a bit of time to have send out all the posts... i never really timed how long it takes before all posts are send out (i don't care that much about that though... people need to wait anyway since the list is being moderated) > > > I mentioned before that I think that it could be better if the cgi scripts > > don't do anything more than just "mark" message as being approved... cgi > > scripts should have a short life time and definately shouldn't be waiting > > for something too long... > I agree, and I think I've seen more comments and postings talking about it. > Noone has implemented it yet, though, and the current method works too well > for it to be a rela problem, I guess ;) it's a big improvement that 1.2 now keeps the pending posts outside of the config.db, so i'm happy with that ... i have been one of the people begging for that :) > > btw i used the original LockFile.py this time... > Go ahead and try it with mine -- it should make the waits you have to endure could you send me your latest version (complete file is ok, and a bit easier for me :) ) thanks... Ricardo. -- From bwarsaw@cnri.reston.va.us Tue Feb 8 23:18:14 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Tue, 8 Feb 2000 18:18:14 -0500 (EST) Subject: [Patches] Re: [Mailman-Developers] Bug#57223: mailman: gate_news problems (fwd) References: <20000207152944.G19265@xs4all.nl> <14495.1677.982940.194724@anthem.cnri.reston.va.us> <20000207191713.L19265@xs4all.nl> <20000208114706.C2908@mail.dskk.co.jp> Message-ID: <14496.42038.761976.368014@anthem.cnri.reston.va.us> Okay, here's my proposed patch to the current CVS version of nntplib.py. It merges Jim's and Thomas's patches, and re-implements the module's exceptions as class-based (with backwards compatibility). I'm not sure what to do about the email disclaimers, but I'd say to be safe that both Jim and Thomas should re-submit their patches with the disclaimer attached (you can just to this to patches@python.org). Everyone else on patches@python.org: please let me know if you have any objections to this change. Otherwise, I'll commit it (and make the necessary documentation updates). -Barry -------------------- snip snip -------------------- Index: nntplib.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/nntplib.py,v retrieving revision 1.17 diff -c -r1.17 nntplib.py *** nntplib.py 2000/02/04 15:10:33 1.17 --- nntplib.py 2000/02/08 23:11:44 *************** *** 34,47 **** import string ! # Exception raised when an error or invalid response is received ! error_reply = 'nntplib.error_reply' # unexpected [123]xx reply ! error_temp = 'nntplib.error_temp' # 4xx errors ! error_perm = 'nntplib.error_perm' # 5xx errors ! error_proto = 'nntplib.error_proto' # response does not begin with [1-5] ! error_data = 'nntplib.error_data' # error in response data # Standard port used by NNTP servers NNTP_PORT = 119 --- 34,79 ---- import string ! ! # Exceptions raised when an error or invalid response is received ! class NNTPError(Exception): ! """Base class for all nntplib exceptions""" ! def __init__(self, *args): ! apply(Exception.__init__, (self,)+args) ! try: ! self.response = args[0] ! except IndexError: ! self.response = 'No response given' ! ! class NNTPReplyError(NNTPError): ! """Unexpected [123]xx reply""" ! pass ! ! class NNTPTemporaryError(NNTPError): ! """4xx errors""" ! pass ! ! class NNTPPermanentError(NNTPError): ! """5xx errors""" ! pass ! ! class NNTPProtocolError(NNTPError): ! """Response does not begin with [1-5]""" ! pass ! ! class NNTPDataError(NNTPError): ! """Error in response data""" ! pass ! ! # for backwards compatibility ! error_reply = NNTPReplyError ! error_temp = NNTPTemporaryError ! error_perm = NNTPPermanentError ! error_proto = NNTPProtocolError ! error_data = NNTPDataError + # Standard port used by NNTP servers NNTP_PORT = 119 *************** *** 54,68 **** CRLF = '\r\n' # The class itself - class NNTP: ! ! def __init__(self, host, port = NNTP_PORT, user=None, password=None): """Initialize an instance. Arguments: - host: hostname to connect to ! - port: port to connect to (default the standard NNTP port)""" ! self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) --- 86,108 ---- CRLF = '\r\n' + # The class itself class NNTP: ! def __init__(self, host, port = NNTP_PORT, user=None, password=None, ! readermode=None): """Initialize an instance. Arguments: - host: hostname to connect to ! - port: port to connect to (default the standard NNTP port) ! - user: username to authenticate with ! - password: password to use with username ! - readermode: if true, send 'mode reader' command after ! connecting. ! ! readermode is sometimes necessary if you are connecting to an ! NNTP server on your local machine and intend to call the ! group() method. ! """ self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) *************** *** 70,85 **** self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: ! raise error_reply, resp else: resp = self.shortcmd( 'authinfo pass '+password) if resp[:3] != '281': ! raise error_perm, resp def getwelcome(self): """Get the welcome message from the server --- 110,136 ---- self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() + if readermode: + try: + self.welcome = self.shortcmd('mode reader') + except NNTPPermanentError: + # error 500, probably 'not implemented' + pass if user: resp = self.shortcmd('authinfo user '+user) if resp[:3] == '381': if not password: ! raise NNTPReplyError(resp) else: resp = self.shortcmd( 'authinfo pass '+password) if resp[:3] != '281': ! raise NNTPPermanentError(resp) ! ! # Get the welcome message from the server ! # (this is read and squirreled away by __init__()). ! # If the response code is 200, posting is allowed; ! # if it 201, posting is not allowed def getwelcome(self): """Get the welcome message from the server *************** *** 128,138 **** if self.debugging: print '*resp*', `resp` c = resp[:1] if c == '4': ! raise error_temp, resp if c == '5': ! raise error_perm, resp if c not in '123': ! raise error_proto, resp return resp def getlongresp(self): --- 179,189 ---- if self.debugging: print '*resp*', `resp` c = resp[:1] if c == '4': ! raise NNTPTemporaryError(resp) if c == '5': ! raise NNTPPermanentError(resp) if c not in '123': ! raise NNTPProtocolError(resp) return resp def getlongresp(self): *************** *** 140,146 **** Raise various errors if the response indicates an error.""" resp = self.getresp() if resp[:3] not in LONGRESP: ! raise error_reply, resp list = [] while 1: line = self.getline() --- 191,197 ---- Raise various errors if the response indicates an error.""" resp = self.getresp() if resp[:3] not in LONGRESP: ! raise NNTPReplyError(resp) list = [] while 1: line = self.getline() *************** *** 206,212 **** resp = self.shortcmd('GROUP ' + name) if resp[:3] <> '211': ! raise error_reply, resp words = string.split(resp) count = first = last = 0 n = len(words) --- 257,263 ---- resp = self.shortcmd('GROUP ' + name) if resp[:3] <> '211': ! raise NNTPReplyError(resp) words = string.split(resp) count = first = last = 0 n = len(words) *************** *** 230,236 **** def statparse(self, resp): """Internal: parse the response of a STAT, NEXT or LAST command.""" if resp[:2] <> '22': ! raise error_reply, resp words = string.split(resp) nr = 0 id = '' --- 281,287 ---- def statparse(self, resp): """Internal: parse the response of a STAT, NEXT or LAST command.""" if resp[:2] <> '22': ! raise NNTPReplyError(resp) words = string.split(resp) nr = 0 id = '' *************** *** 349,355 **** elem[6], elem[7])) except IndexError: ! raise error_data,line return resp,xover_lines def xgtitle(self, group): --- 400,406 ---- elem[6], elem[7])) except IndexError: ! raise NNTPDataError(line) return resp,xover_lines def xgtitle(self, group): *************** *** 377,387 **** resp = self.shortcmd("XPATH " + id) if resp[:3] <> '223': ! raise error_reply, resp try: [resp_num, path] = string.split(resp) except ValueError: ! raise error_reply, resp else: return resp, path --- 428,438 ---- resp = self.shortcmd("XPATH " + id) if resp[:3] <> '223': ! raise NNTPReplyError(resp) try: [resp_num, path] = string.split(resp) except ValueError: ! raise NNTPReplyError(resp) else: return resp, path *************** *** 395,408 **** resp = self.shortcmd("DATE") if resp[:3] <> '111': ! raise error_reply, resp elem = string.split(resp) if len(elem) != 2: ! raise error_data, resp date = elem[1][2:8] time = elem[1][-6:] if len(date) != 6 or len(time) != 6: ! raise error_data, resp return resp, date, time --- 446,459 ---- resp = self.shortcmd("DATE") if resp[:3] <> '111': ! raise NNTPReplyError(resp) elem = string.split(resp) if len(elem) != 2: ! raise NNTPDataError(resp) date = elem[1][2:8] time = elem[1][-6:] if len(date) != 6 or len(time) != 6: ! raise NNTPDataError(resp) return resp, date, time *************** *** 415,421 **** resp = self.shortcmd('POST') # Raises error_??? if posting is not allowed if resp[0] <> '3': ! raise error_reply, resp while 1: line = f.readline() if not line: --- 466,472 ---- resp = self.shortcmd('POST') # Raises error_??? if posting is not allowed if resp[0] <> '3': ! raise NNTPReplyError(resp) while 1: line = f.readline() if not line: *************** *** 439,445 **** resp = self.shortcmd('IHAVE ' + id) # Raises error_??? if the server already has it if resp[0] <> '3': ! raise error_reply, resp while 1: line = f.readline() if not line: --- 490,496 ---- resp = self.shortcmd('IHAVE ' + id) # Raises error_??? if the server already has it if resp[0] <> '3': ! raise NNTPReplyError(resp) while 1: line = f.readline() if not line: *************** *** 465,471 **** def _test(): """Minimal test function.""" ! s = NNTP('news') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last --- 516,522 ---- def _test(): """Minimal test function.""" ! s = NNTP('news', readermode='reader') resp, count, first, last, name = s.group('comp.lang.python') print resp print 'Group', name, 'has', count, 'articles, range', first, 'to', last From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 8 23:22:50 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 8 Feb 2000 18:22:50 -0500 (EST) Subject: [Mailman-Developers] Reply goes to list error References: Message-ID: <14496.42314.426477.518945@anthem.cnri.reston.va.us> >>>>> "HM" == Harald Meland writes: HM> Ooops, sorry -- my version did, but I hadn't checked that in. HM> Checked in now. Actually, there's a bug in your version. When using the setitem interface (e.g. msg['reply-to'] = '...'), you don't want to set the value to include the key or the newline. I'm about to check in a fix. -Barry From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 8 23:27:48 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 8 Feb 2000 18:27:48 -0500 (EST) Subject: [Mailman-Developers] When will mailman 1.2 be released? References: <20000208222814.A3736@stopcontact.palga.uucp> Message-ID: <14496.42612.673774.185482@anthem.cnri.reston.va.us> >>>>> "GH" == Gerrit Holl writes: GH> when do you think mailman 1.2 will be released? I wish I could say. Frankly, I wish I could say whether the next release will be 1.2 or 2.0. I think if Harald's user database changes are significant enough, I might make it 2.0. GH> How stable is the current CVS tree, and how heavily is it GH> tested? Stable enough to run all the lists at Python.Org. If something's broken, I find out about it pretty dang quickly. No doubt there are minor bugs still lurking though. GH> In what order of magnitude is the release frequency? I'm not sure what you're asking, but I've been thinking that I should start getting beta releases out RSN. -Barry From Gerrit Wed Feb 9 07:13:23 2000 From: Gerrit (Gerrit Holl) Date: Wed, 9 Feb 2000 08:13:23 +0100 Subject: [Mailman-Developers] When will mailman 1.2 be released? In-Reply-To: <14496.42612.673774.185482@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Tue, Feb 08, 2000 at 06:27:48PM -0500 References: <20000208222814.A3736@stopcontact.palga.uucp> <14496.42612.673774.185482@anthem.cnri.reston.va.us> Message-ID: <20000209081323.A957@stopcontact.palga.uucp> Barry A. Warsaw wrote on 950030868: > > >>>>> "GH" == Gerrit Holl writes: > > GH> when do you think mailman 1.2 will be released? > > I wish I could say. Frankly, I wish I could say whether the next > release will be 1.2 or 2.0. I think if Harald's user database changes > are significant enough, I might make it 2.0. > > GH> How stable is the current CVS tree, and how heavily is it > GH> tested? > > Stable enough to run all the lists at Python.Org. If something's > broken, I find out about it pretty dang quickly. No doubt there are > minor bugs still lurking though. > > GH> In what order of magnitude is the release frequency? > > I'm not sure what you're asking, but I've been thinking that I should > start getting beta releases out RSN. Maybe I looked up the wrong words in my dictionairy :( I mean, is Mailman likely to be released every month like many other projects, or far less often, like Python? The last version was released in November last year, so it's not really "Release early, release often". What does 'RSN' mean? regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From brian@garage.co.jp Wed Feb 9 08:00:01 2000 From: brian@garage.co.jp (Brian Takashi Hooper) Date: Wed, 09 Feb 2000 17:00:01 +0900 Subject: [Mailman-Developers] I18N concerns for Japanese Message-ID: <38A11E81D4.8E5ABRIAN@smtp.garage.co.jp> Hi there - I just checked out the latest version of Mailman from CVS and am taking a look at it with an eye to making it work better with our environment here, which is Japanese. I think that Mailman would be very popular in Japan too with a localized interface and a few tweaks to ensure that mail goes out in the proper format. I haven't been on Mailman-dev for a long time yet, so forgive me if I'm rehashing an old topic (I checked through the archives as best I could, so I apologize in advance for redundancy): 1. Headers: To, Subject: The To: and Subject: lines in Japanese messages often contain Japanese; the proper encoding for this seems to be base-64 encoded ISO-2022 (JIS). Mail received from list members is of course no problem, but mail which the system sends out, if it contains Japanese, needs to be able to do this conversion. Looking at an example implementation (Mew, an emacs module which does Japanese mail), it looks like only those parts of the header which contain JIS escape sequences are encoded, that is, ASCII parts of the To: and Subject: lines are not encoded. I think this makes sense, since encoding the entirety of To: would obscure the recipient address, which might not be good for some mailers. I assume there are other standards out there for various languages to insert non-ascii characters in the To: and Subject: lines, probably which do similar tricks; are there any folks out there who have similar requirements or experiences? I think that my difficulty would be solved if it was possible to specify a filter which should be applied to outgoing message headers; this could by default be None, or a filter which just returns its string argument unchanged. This also should be linked to the specification of charset for the message. Content-type, Mime-Version: Typically, Japanese mail sets the Content-type and Mime headers like so: Content-type: text/plain; charset=ISO-2022-JP Mime-Version: 1.0 Looking at the Mailman source, it looked like charset=us-ascii is hardcoded for a few types of messages (ToDigest, bounce); as far as I can tell it's not set for outgoing admin messages (someone please catch me if I'm wrong on this). The message body of Japanese messages normally sent in ISO-2022 without bin-hex translation, but I don't think this has any implications vis-a-vis Mailman's current behavior. So, I guess what I'm suggesting is adding the ability to add custom filters to do special header processing based on locale... Opinions, additions, flames anyone? 2. Templates We have already done some localization of templates here, and will continue to do so until we've finished... in this case, maybe it would be nice to offer downloads of drop-in-replacement template folders for different languages somewhere. I volunteer our templates when they are done. Obviously there are other inline text messages which also require some mechanism for localization; maybe gettext? Are there any other localization issues which people have come across for other locales? --Brian Hooper Tokyo, Japan From thomas@xs4all.net Wed Feb 9 09:41:47 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Feb 2000 10:41:47 +0100 Subject: [Mailman-Developers] Archiver.py bug ? Message-ID: <20000209104147.A477@xs4all.nl> --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii I ran across this in my errors log: Traceback (innermost last): File "/usr/local/mailman/Mailman/Archiver/Archiver.py", line 212, in ArchiveMail self.__archive_to_mbox(msg) File "/usr/local/mailman/Mailman/Archiver/Archiver.py", line 168, in __archive_to_mbox post.SetHeader('Date', time.ctime(time.time())) post is a Mailman.Message.Message instance, which is a subclass of rfc822.Message... But neither of those objects define a SetHeader method, and the two calls to SetHeader in Archiver.py are the only calls in Mailman to that particular method. An old leftover, perhaps ? The attached patch should fix it. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mailman.diffd" Index: Mailman/Archiver/Archiver.py =================================================================== RCS file: /projects/cvsroot/mailman/Mailman/Archiver/Archiver.py,v retrieving revision 1.18 diff -u -r1.18 Archiver.py --- Archiver.py 1999/12/25 14:37:38 1.18 +++ Archiver.py 2000/02/09 09:25:12 @@ -163,7 +163,7 @@ if self.clobber_date: import time olddate = post.getheader('date') - post.SetHeader('Date', time.ctime(time.time())) + post['Date'] = time.ctime(time.time()) try: afn = self.ArchiveFileName() mbox = self.__archive_file(afn) @@ -176,7 +176,7 @@ reraise() if self.clobber_date: # Resurrect original date setting. - post.SetHeader('Date', olddate) + post['Date'] = olddate def ExternalArchive(self, ar, txt): d = SafeDict({'listname': self.internal_name()}) --TB36FDmn/VVEgNH/-- From gorgo@sztaki.hu Wed Feb 9 10:36:41 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 09 Feb 2000 11:36:41 +0100 (MET) Subject: [Mailman-Developers] Bug#57596: mailman: Here we go again... post: nntplib.error_perm : 500 "POST" not implemented; try "help". (fwd) Message-ID: Another day, another bugreport :/ Lazarus, can you tell me if this is after you applied the last patch suggested by Thomas, for gate_news ? Package: mailman Version: 1.1-2 Severity: important Feb 09 03:47:55 2000 post: Traceback (innermost last): post: File "/var/lib/mailman/scripts/post", line 73, in ? post: mlist.Post(msg, approved=fromusenet) post: File "/usr/lib/mailman/Mailman/MailList.py", line 1331, in Post post: self.SendMailToNewsGroup(msg) post: File "/usr/lib/mailman/Mailman/GatewayManager.py", line 204, in SendMailToNewsGroup post: con.post(msg) post: File "/usr/lib/python1.5/nntplib.py", line 417, in post post: resp = self.shortcmd('POST') post: File "/usr/lib/python1.5/nntplib.py", line 158, in shortcmd post: return self.getresp() post: File "/usr/lib/python1.5/nntplib.py", line 134, in getresp post: raise error_perm, resp You get the idea. Gatewaying seems completely broken. From gorgo@sztaki.hu Wed Feb 9 10:37:45 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 09 Feb 2000 11:37:45 +0100 (MET) Subject: [Mailman-Developers] Bug#57597: mailman: Different gate_news problems this time. (fwd) Message-ID: And a locking problem... is this serious ? ---------- Forwarded message ---------- Date: Wed, 09 Feb 2000 03:57:40 +0000 (UTC) From: Lazarus Long To: submit@bugs.debian.org Subject: Bug#57597: mailman: Different gate_news problems this time. Resent-Date: Wed, 09 Feb 2000 04:03:05 +0000 (GMT) Resent-From: Lazarus Long Resent-To: debian-bugs-dist@lists.debian.org Resent-cc: Gergely Madarasz Package: mailman Version: 1.1-2 Severity: important Subject: Cron [ -x /usr/bin/python -a -f /usr/lib/mailman/cron/gate_news ] && /usr/bin/python /usr/lib/mailman/cron/gate_news X-Cron-Env: X-Cron-Env: X-Cron-Env: X-Cron-Env: Date: Wed, 9 Feb 2000 03:50:15 +0000 (UTC) Exception Mailman.LockFile.NotLockedError: in ignored From Harald.Meland@usit.uio.no Wed Feb 9 10:58:58 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 09 Feb 2000 11:58:58 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval In-Reply-To: Thomas Wouters's message of "Tue, 8 Feb 2000 02:05:27 +0100" References: <20000208020526.N19265@xs4all.nl> Message-ID: [Thomas Wouters] > Another minor patch adding what I need to mailman ;) I also noticed > this on the jitterbug list, so I guess more people had wished or at > least wondered for it ;P Yup, that's correct. > I'm posting it mostly to see if it generates comments, and for me to > find a good excuse to sleep. :-) I've had a go at integrating your patch into my locally hacked Mailman (BTW, could anyone provide me with a clue as to how I can tell patch(1) to understand a multiple-directory CVS diff?) -- and noticed that you seemingly have forgotten to add a DEFAULT_UNSUBSCRIBE_POLICY to Defaults.py.in, and that the comments in some places bore clear signs of a bit hasty cut-and-paste work :) > is there a good word for 'unsubscription request' ? I'm probably not the best person to answer questions on English wording, but to me "unsubscription request" is OK. However, saying Your authorization is required for a mailing list unsubscription request approval: (quickly, three times in a row :) is quite a mouthful, so one might consider rephrasing that sentence somewhat ;) > Adding this has been mostly an exercise to learn a bit about the Mailman > insides, and to see how easily it can be done; fairly easily ;) Comments are > welcome, including "d'oh, dont write that, i already wrote it !" My immediate comment would be: How fast can we get this guy to sign the appropriate papers? :-D I'll try to get your stuff tested a bit here, too, and hopefully we can get around to committing it later. Thanks, -- Harald From Harald.Meland@usit.uio.no Wed Feb 9 11:08:56 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 09 Feb 2000 12:08:56 +0100 Subject: [Mailman-Developers] Reply goes to list error In-Reply-To: "Barry A. Warsaw"'s message of "Tue, 8 Feb 2000 18:22:50 -0500 (EST)" References: <14496.42314.426477.518945@anthem.cnri.reston.va.us> Message-ID: [Barry A. Warsaw] > >>>>> "HM" == Harald Meland writes: > > HM> Ooops, sorry -- my version did, but I hadn't checked that in. > HM> Checked in now. > > Actually, there's a bug in your version. When using the setitem > interface (e.g. msg['reply-to'] = '...'), you don't want to set the > value to include the key or the newline. I'm about to check in a fix. That's what I get for changing from the msg.headers.append() to the msg[header] style of doing these things just prior to checking it all in (because I realized I found the latter style to be "cleaner" in this context). I guess I'm now cured for the check-in-untested-code disease for a few more months... Sorry, -- Harald From rullfig@midway.uchicago.edu Wed Feb 9 16:23:36 2000 From: rullfig@midway.uchicago.edu (Roberto Ullfig) Date: Wed, 09 Feb 2000 10:23:36 -0600 Subject: [Mailman-Developers] new feature request for anon lists Message-ID: <38A19488.50F7BBA3@midway.uchicago.edu> Here are the relevant lines from MailList.py # if self.anonymous_list: # del msg['reply-to'] # del msg['sender'] # msg.SetHeader('From', self.GetAdminEmail()) # msg.SetHeader('Reply-to', self.GetListEmail()) if self.anonymous_list: del msg['sender'] msg.SetHeader('From', self.GetAdminEmail()) if self.reply_goes_to_list: del msg['reply-to'] msg.SetHeader('Reply-to', self.GetListEmail()) What I've done is use the reply_goes_to_list flag to control whether or not the Reply-To is munged instead of munging indiscriminately. It make a lot of sense to me that this is how it should work. Anyone see any problems with this? -- Roberto Ullfig : rullfig@uchicago.edu Systems Administrator Networking Services and Information Technologies University of Chicago From Nigel.Metheringham@vdata.co.uk Wed Feb 9 16:51:19 2000 From: Nigel.Metheringham@vdata.co.uk (Nigel Metheringham) Date: Wed, 09 Feb 2000 16:51:19 +0000 Subject: [Mailman-Developers] nightly_gzip - more than a little broken... Message-ID: I wondered why the nightly_gzip program was not working correctly on some of my lists - one (a high traffic list) has weekly based archiving, the other (an announce list) has quarterly based archiving. A very quick look at the code shows:- for f in allfiles: try: time.strptime(f, '%Y-%B.txt') except ValueError: continue that strptime seems to be ensuring that the archive mbox files have a year and month in their names before a .txt - and this only applies to monthly archives in practice. Would there be any problems with just changing this to be a regexp match for '\.txt$' instead? If you really want to check its an archive then look for a directory x with a file x.txt . Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From thomas@xs4all.net Wed Feb 9 17:31:07 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Feb 2000 18:31:07 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval In-Reply-To: ; from Harald.Meland@usit.uio.no on Wed, Feb 09, 2000 at 11:58:58AM +0100 References: <20000208020526.N19265@xs4all.nl> Message-ID: <20000209183107.B477@xs4all.nl> On Wed, Feb 09, 2000 at 11:58:58AM +0100, Harald Meland wrote: > I've had a go at integrating your patch into my locally hacked Mailman > (BTW, could anyone provide me with a clue as to how I can tell > patch(1) to understand a multiple-directory CVS diff?) -- and noticed > that you seemingly have forgotten to add a DEFAULT_UNSUBSCRIBE_POLICY > to Defaults.py.in, and that the comments in some places bore clear > signs of a bit hasty cut-and-paste work :) DISCLAIMER: It was late at night, i wanted to move a few of our lists to mailman the following day, and we have some peculiar requirements ;) We have basically two lists, an 'admin' list and a 'garbage' list. The 'admin' list is the entire company, and we dont want people to unsubscribe themselves without aproval. The garbage list is for discussions, jokes, anecdotes, etc. We want to allow people to subscribe and unsubscribe at will, IF they are on the 'admin' list. We didn't have this in Majordomo, so it's no big deal that they we have it now. It's on my wishlist tho ;) In the mean time, we set subscribe policy on the garbage list to 'approve'. We also want both these lists to be 'closed' (postings by members only) BUT a lot of our employees have multiple email adresses, and we want to allow them to post using all of them. So, in majordomo, we had a seperate file, included in _both_ lists as 'allow posts from this list of addresses', so we had to include only a single file. In Mailman, we'd have to paste the entire file (which is currently 495 entries long, and getting longer and longer ;) into the webform, and update them both. So instead I hacked the same behaviour into mailman -- include a seperate file. Not intended for general use, which is why I didn't post it here, and it has _no_ security, but it works ;) I'm thinking of changing it to make it allow postings from users on different lists as well, but I didn't want to spend my time figuring out if I can safely instantiate one or more MailList instances inside Hold.py's process(). Another hack I added this morning, right before I switched the first list (the 'garbage' list) to Mailman, is to allow people to read the (private) archives for the list _only_ when they connect from a specific network. This is a _really_ dirty hack and i'm ashamed now that I look back at it, but hey, it was 7:45am after a party and 5 hours of sleep. It's also embarassing because i realize now that there is a much better way. (_public_ archives, with a .htaccess in them. This is guaranteed to work, right ?) A colleague of mine found a bug, though (the hard way ;). MailList.MailList.DeleteMember() tries to call 'ApprovedDeleteMember()' instead of 'self.ApprovedDeleteMember()'. No other bugs yet, but noone tried to unsubscribe from a closed list yet, either. (other than me.) regarding patch: you need to do 'patch -p0'. See the manpage for 'patch'; there is a not-so-subtle difference between -p0 (or -p without a number, if it's the last option) and no -p option. And yes, the entire patch is mostly cut&paste work. I didn't think so much about wording and comments as I did about getting it working quickly. This patch is in no way what I would propose to be checked in ;P > Your authorization is required for a mailing list unsubscription > request approval: > > (quickly, three times in a row :) is quite a mouthful, so one might > consider rephrasing that sentence somewhat ;) Yeah, I agree. #TBD ;) > My immediate comment would be: > How fast can we get this guy to sign the appropriate papers? Me ? Are they the grep-able or the dead-tree kind of papers ? do I need to fax, or do you need to fax me something ? I have access to such things here at work, thankfully ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Wed Feb 9 17:36:49 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Feb 2000 18:36:49 +0100 Subject: [Mailman-Developers] Bug#57597: mailman: Different gate_news problems this time. (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Wed, Feb 09, 2000 at 11:37:45AM +0100 References: Message-ID: <20000209183649.C477@xs4all.nl> On Wed, Feb 09, 2000 at 11:37:45AM +0100, Gergely Madarasz wrote: > And a locking problem... is this serious ? > Exception Mailman.LockFile.NotLockedError: instance at 812bb50> in 8135e88> ignored Not really important, but peculiar. Did this happen with the default mailman 1.1 locking, or did you apply any patches ? In the default locking for mailman 1.1 this shouldn't happen, except perhaps if someone else stole the link right between the check in LockFile.__del__() and the check in LockFile.unlock(). In other words, a highly unlikely race condition. [BTW, kept CC list the same because i can't figure out who to include or not] -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From bwarsaw@cnri.reston.va.us Wed Feb 9 17:42:15 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 9 Feb 2000 12:42:15 -0500 (EST) Subject: [Mailman-Developers] When will mailman 1.2 be released? References: <20000208222814.A3736@stopcontact.palga.uucp> <14496.42612.673774.185482@anthem.cnri.reston.va.us> <20000209081323.A957@stopcontact.palga.uucp> Message-ID: <14497.42743.130965.646975@anthem.cnri.reston.va.us> >>>>> "GH" == Gerrit Holl writes: GH> I mean, is Mailman likely to be released every month like many GH> other projects, or far less often, like Python? The last GH> version was released in November last year, so it's not really GH> "Release early, release often". Unsure. I'm quite busy with real work, real life, and managing two big open source projects. I'll release when I can -- hopefully more often than Python, but probably not every month. You're always free to watch CVS, and I try very hard not to destabilize the CVS snapshot. GH> What does 'RSN' mean? Real Soon Now. -Barry From Harald.Meland@usit.uio.no Wed Feb 9 18:13:03 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 09 Feb 2000 19:13:03 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval In-Reply-To: Thomas Wouters's message of "Wed, 9 Feb 2000 18:31:07 +0100" References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> Message-ID: [Thomas Wouters] > So instead I hacked the same behaviour into mailman -- include a > seperate file. Not intended for general use, which is why I didn't > post it here, and it has _no_ security, but it works ;) Random thought: Would it be a good idea to add type thingies for some of the text boxes which have to take rather large amounts of text, or would it just clutter things? > A colleague of mine found a bug, though (the hard way ;). > MailList.MailList.DeleteMember() tries to call > 'ApprovedDeleteMember()' instead of > 'self.ApprovedDeleteMember()'. Thanks for letting me know, I've fixed this in my local copy too, now. > regarding patch: you need to do 'patch -p0'. Hmmmm, I already knew about -p0, and had tried that, but I get this (when standing in the toplevel Mailman directory, i.e. the one containing INSTALL etc.): octarine$ patch -p0 --dry-run < ~/mailman-unsub-approval.diff can't find file to patch at input line 9 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -------------------------- |? templates/unsubauth.txt |Index: Mailman/Bouncer.py |=================================================================== |RCS file: /projects/cvsroot/mailman/Mailman/Bouncer.py,v |retrieving revision 1.37 |diff -u -r1.37 Bouncer.py |--- Bouncer.py 1999/12/16 17:11:04 1.37 |+++ Bouncer.py 2000/02/08 00:57:47 -------------------------- File to patch: ^C etc. Thus my suspicion that patch isn't recognizing the Index: stuff, which I believe was put there by CVS. This is with GNU patch 2.5. > > My immediate comment would be: > > How fast can we get this guy to sign the appropriate papers? > > Me ? Are they the grep-able or the dead-tree kind of papers ? Well, that depends on whether anyone has finished work on that wetware grep yet ;) I'm talking about the stuff explained in -- I suspect that both you and a few others are (at least) bordering on "significant changes" to Mailman if most of the (very nice) patches you have submitted get checked in. > do I need to fax, or do you need to fax me something ? I have access > to such things here at work, thankfully ;) I don't have access to any GNU machines, so I won't be able to help you with this stuff. Barry is the official maintainer, so he should be able to sort this out (but please do give him some time to respond, in case he is heavily swamped at the moment). For what it's worth: When I did this, I received the appropriate templates[1] by email, printed them out, filled them in by hand, and then snail-mailed them (the originals) back across the Atlantic. [1] two, IIRC -- one (disclaim.future) to say my employer disclaims any ownership on the work I do on Mailman (signed by my boss), and one (assign.future) to say that I transfer copyright of my future work (either for all future, or for X years, IIRC) work on Mailman to the FSF (signed by me). -- Harald From claw@kanga.nu Wed Feb 9 18:16:03 2000 From: claw@kanga.nu (J C Lawrence) Date: Wed, 09 Feb 2000 10:16:03 -0800 Subject: [Mailman-Developers] When will mailman 1.2 be released? In-Reply-To: Message from Gerrit Holl of "Wed, 09 Feb 2000 08:13:23 +0100." <20000209081323.A957@stopcontact.palga.uucp> References: <20000208222814.A3736@stopcontact.palga.uucp> <14496.42612.673774.185482@anthem.cnri.reston.va.us> <20000209081323.A957@stopcontact.palga.uucp> Message-ID: <4598.950120163@kanga.nu> > I mean, is Mailman likely to be released every month like many > other projects, or far less often, like Python? The last version > was released in November last year, so it's not really "Release > early, release often". Hardly -- as all the sources are publicly available via CVS, every change is "released". In that context the "release process" really is just a labelling game, and nothing to really do with the function or development of the product itself. -- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From Gerrit Wed Feb 9 21:00:10 2000 From: Gerrit (Gerrit Holl) Date: Wed, 9 Feb 2000 22:00:10 +0100 Subject: [Mailman-Developers] I want to write docs! Message-ID: <20000209220010.A2699@stopcontact.palga.uucp> Hello, [see below for summary] I've downloaded Mailman about a week ago, and I'm very excited. I read all papers. I read many of the source code. I read all available documentation. And now, I want to write documentation. The TODO list says: * Documenation - a detailed feature list - a user's guide - a site-admin's guide - a list-admin's guide - more in-place documenation - A developer's guide with architecture and API info, etc. I want to add one: - more docstrings in the code I want to write all kind of documentation. However, there is no good system for it. I know nothing about setting up a system like the Python documentation system, or an XML DTD, or... I do know the DOCBOOK DTD, but I'm not able to get it to work. I don't know how to tell sgml2html where to search for all the files. I seem to to something wrong. I can't get it to work. But I certainly want to write documenation. Plain HTML, however, would be an awful thing, since we would need to convert it to the other system after we built such a system. Another thing: if I add docstrings to the code, what should I do with the changed code, with the patches? I can't find out where to send patches to be incorperated in mailman. Neither mailman-cabal, nor this list seems to be the right place :(. -----BEGIN SUMMARY BLOCK----- I want to write documentation, but I don't know how to do it in practice, e.g., in what format and where to send the docs. -----END SUMMARY BLOCK----- regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Feb 9 22:00:08 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 9 Feb 2000 17:00:08 -0500 (EST) Subject: [Mailman-Developers] I want to write docs! References: <20000209220010.A2699@stopcontact.palga.uucp> Message-ID: <14497.58216.799619.232133@anthem.cnri.reston.va.us> >>>>> "GH" == Gerrit Holl writes: GH> I've downloaded Mailman about a week ago, and I'm very GH> excited. Cool! GH> And now, I want to write documentation. There have been a few other people who have done some documenting. Unfortunately I haven't kept up with that status, so perhaps others can chime in. Because Mailman is GNU, texinfo is the standard documentation format. I'll point you at doc-sig@python.org (a Mailman list, 'natch) which is making all the decisions about Python documentation. It's not a list that I'm on. If whatever doc-sig chooses can be used to generate texinfo, then it would be fine to use that, but we need to distribute texinfo with Mailman. There's plenty of documentation about texinfo on www.gnu.org and it's pretty easy to write. You will, of course, need to copyright assign any documentation to the FSF, and I can send you the appropriate forms when you're ready. GH> Another thing: if I add docstrings to the code, what should I GH> do with the changed code, with the patches? I can't find out GH> where to send patches to be incorperated in mailman. Neither GH> mailman-cabal, nor this list seems to be the right place :(. Currently, mailman-developers is the right place to send your patches. If others think that'll clutter up this list too much, we can 1) create a mailman-patches mailing list; or 2) hasten the move to SourceForge and piggyback off of their tools (which I admittedly haven't investigated too deeply yet). I'm leary of creating yet-another-mailing list, so I'd be inclined to prefer using mailman-developers. Note that, you'll need different FSF copyright assignment forms for docstring patches. Again, I can send those to you when needed. -Barry From lazarus@overdue.dhis.net Wed Feb 9 23:07:51 2000 From: lazarus@overdue.dhis.net (Lazarus Long) Date: Wed, 9 Feb 2000 23:07:51 +0000 Subject: [Mailman-Developers] Re: Bug#57596: mailman: Here we go again... post: nntplib.error_perm : 500 "POST" not implemented; try "help". (fwd) In-Reply-To: ; from gorgo@sztaki.hu on Wed, Feb 09, 2000 at 11:36:41AM +0100 References: Message-ID: <20000209230751.A9922@overdue.dhis.net> --H1spWtNR+x+ondvy Content-Type: multipart/mixed; boundary="y0ulUmNC+osPPQO6" --y0ulUmNC+osPPQO6 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable On Wed, Feb 09, 2000 at 11:36:41AM +0100, Gergely Madarasz wrote: >=20 > Another day, another bugreport :/ Sigh. I've gotten mail regarding this containing obscenities from another developer. I'd love to get this fixed rather than filing new bug reports. I'm sure we all would. > Lazarus, can you tell me if this is after you applied the last patch > suggested by Thomas, for gate_news ? The only patch i aplied follows: --- cron/gate_news.orig Mon Feb 7 22:47:52 2000 +++ cron/gate_news Mon Feb 7 22:53:23 2000 @@ -76,7 +76,16 @@ # update the gate_watermarks file. we'll actually do the gating i= n a # child process conn =3D nntplib.NNTP(mlist.nntp_host) - r,c,first,last,n =3D conn.group(mlist.linked_newsgroup) + try: + r,c,first,last,n =3D conn.group(mlist.linked_newsgroup) + except nntplib.error_perm: + # We got an error 500, probably meaning that the 'group' comma= nd + # is not implemented. A likely scenario causing this is when + # running the Mailman gateway on the same machine as the news + # server; the server defaults to news feed mode. We need to s= et + # the connection to reader mode first. + conn.shortcmd('mode reader') + r,c,first,last,n =3D conn.group(mlist.linked_newsgroup) first =3D int(first) last =3D int(last) wm =3D watermarks.get(name, 0) --=20 Please encrypt all mail whenever possible. The following Public Keys for Lazarus Long are available upon request: Type Bits/KeyID Fingerprint (GnuPG (GPG) is preferred.) GPG/ELG: 2048g/92F6493B 2C55 E967 278B 4E8B D25B F5F3 352B 9B0E 32C3 3BA4 GPG/DSA: 1024D/32C33BA4 (none for DSA keys) --y0ulUmNC+osPPQO6 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="gate_news.diff" --- cron/gate_news.orig Mon Feb 7 22:47:52 2000 +++ cron/gate_news Mon Feb 7 22:53:23 2000 @@ -76,7 +76,16 @@ # update the gate_watermarks file. we'll actually do the gating in a # child process conn = nntplib.NNTP(mlist.nntp_host) - r,c,first,last,n = conn.group(mlist.linked_newsgroup) + try: + r,c,first,last,n = conn.group(mlist.linked_newsgroup) + except nntplib.error_perm: + # We got an error 500, probably meaning that the 'group' command + # is not implemented. A likely scenario causing this is when + # running the Mailman gateway on the same machine as the news + # server; the server defaults to news feed mode. We need to set + # the connection to reader mode first. + conn.shortcmd('mode reader') + r,c,first,last,n = conn.group(mlist.linked_newsgroup) first = int(first) last = int(last) wm = watermarks.get(name, 0) --y0ulUmNC+osPPQO6-- --H1spWtNR+x+ondvy Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.1 (GNU/Linux) Comment: Now ready for primetime! http://www.gnupg.org iD8DBQE4ofNHNSubDjLDO6QRArOOAJ4safl/w/vzMIc3RD0aGqYCEybjlwCfTnTB G7meKaKRw4OCaMyYrA6uxxw= =SkQj -----END PGP SIGNATURE----- --H1spWtNR+x+ondvy-- From Gerrit Thu Feb 10 06:32:05 2000 From: Gerrit (Gerrit Holl) Date: Thu, 10 Feb 2000 07:32:05 +0100 Subject: [Mailman-Developers] I want to write docs! In-Reply-To: <14497.58216.799619.232133@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Wed, Feb 09, 2000 at 05:00:08PM -0500 References: <20000209220010.A2699@stopcontact.palga.uucp> <14497.58216.799619.232133@anthem.cnri.reston.va.us> Message-ID: <20000210073205.A617@stopcontact.palga.uucp> Barry A. Warsaw wrote on 950112008: > [texinfo] > You will, of course, need to copyright assign any documentation to the > FSF, and I can send you the appropriate forms when you're ready. That's totally fine. The copyright is absolutly not a problem. I totally trust the freedom of the FSF licenses :) > GH> Another thing: if I add docstrings to the code, what should I > GH> do with the changed code, with the patches? I can't find out > GH> where to send patches to be incorperated in mailman. Neither > GH> mailman-cabal, nor this list seems to be the right place :(. > > Currently, mailman-developers is the right place to send your patches. OK! > Note that, you'll need different FSF copyright assignment forms > for docstring patches. Again, I can send those to you when needed. And again, I'll totally leave that tou you and the FSF. I'll have a look on Texinfo this afternoon (I've got to get to school first). regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From Gergely Madarasz Thu Feb 10 13:01:44 2000 From: Gergely Madarasz (Gergely Madarasz) Date: Thu, 10 Feb 2000 14:01:44 +0100 (MET) Subject: [Mailman-Developers] fd leakage with run_queue Message-ID: Hello, It is quite often that the system load on a machine goes up so sendmail rejects connections.... so when run_queue runs, there are quite a few messages rejected in one process, meaning lots of logs... and then I get messages like this: Logging error: Traceback (innermost last): File "/usr/lib/mailman/Mailman/Logging/Logger.py", line 55, in __get_f IOError: [Errno 24] Too many open files: '/var/lib/mailman/logs/error' Original log message: [Errno 24] Too many open files: '/var/lib/mailman/logs/error' Feb 10 13:44:59 2000 processQueue: Processing of queue file /var/lib/mailman/data/mm_q.28769.1 failed: Feb 10 13:44:59 2000 processQueue: exceptions.IOError / [Errno 24] Too many open files: '/var/lib/mailman/data/mm_q.28769.1' Logging error: Traceback (innermost last): File "/usr/lib/mailman/Mailman/Logging/Logger.py", line 55, in __get_f IOError: [Errno 24] Too many open files: '/var/lib/mailman/logs/error' Original log message: [Errno 24] Too many open files: '/var/lib/mailman/logs/error' Feb 10 13:45:00 2000 processQueue: Processing of queue file /var/lib/mailman/data/mm_q.29101.1 failed: Feb 10 13:45:00 2000 processQueue: exceptions.IOError / [Errno 24] Too many open files: '/var/lib/mailman/data/mm_q.29101.1' It seems that for each log entry run_queue opens the logfile, and then doesn't close it. Is there a quick fix for this somewhere ? I'm running mailman 1.1 -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 10 13:16:30 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 10 Feb 2000 08:16:30 -0500 (EST) Subject: [Mailman-Developers] fd leakage with run_queue References: Message-ID: <14498.47662.909335.920840@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> It seems that for each log entry run_queue opens the logfile, GM> and then doesn't close it. Is there a quick fix for this GM> somewhere ? I'm running mailman 1.1 I seem to remember someone posting a patch for this, so check the archives. Otherwise, don't sweat it, run_queue is gone, gone, gone in the CVS version. -Barry From thomas@xs4all.net Thu Feb 10 16:31:14 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 10 Feb 2000 17:31:14 +0100 Subject: [Mailman-Developers] unsubscriptions requiring approval In-Reply-To: ; from Harald.Meland@usit.uio.no on Wed, Feb 09, 2000 at 07:13:03PM +0100 References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> Message-ID: <20000210173114.F477@xs4all.nl> On Wed, Feb 09, 2000 at 07:13:03PM +0100, Harald Meland wrote: > > So instead I hacked the same behaviour into mailman -- include a > > seperate file. Not intended for general use, which is why I didn't > > post it here, and it has _no_ security, but it works ;) > > Random thought: Would it be a good idea to add > > type thingies for some of the text boxes which have to take rather > large amounts of text, or would it just clutter things? I'm not sure. It might certainly be a good idea, even though in my case it wouldn't help (the problem was not the initial insertion, but rather keeping multiple lists in sync.) Clutterage is a problem in general, i think. Perhaps, if we want to give every textbox an upload field, a simple button next to the input field would suffice -- a button that would pop up a new window. (It might be even worth it to take a look at a frames interface, possibly ?) > > regarding patch: you need to do 'patch -p0'. > Hmmmm, I already knew about -p0, and had tried that, but I get this > (when standing in the toplevel Mailman directory, i.e. the one > containing INSTALL etc.): > > octarine$ patch -p0 --dry-run < ~/mailman-unsub-approval.diff > can't find file to patch at input line 9 > Perhaps you used the wrong -p or --strip option? > The text leading up to this was: > -------------------------- > |? templates/unsubauth.txt > |Index: Mailman/Bouncer.py > etc. Thus my suspicion that patch isn't recognizing the Index: stuff, > which I believe was put there by CVS. This is with GNU patch 2.5. Ok, found it. It's a bug in patch, or in patch's manpage: If there is an Index: line in the leading garbage and if either the old and new names are both absent or the POSIXLY_CORRECT environment variable is set, patch takes the name in the Index: line. Apparently, patch doesn't work like this; even if both the old and the new name are absent, it refuses to regard the Index line. I found a patch to patch that added a '--use-index-line' option, but it doesn't seem to be added to patch 2.5 yet. As a workaround, you can set the 'POSIXLY_CORRECT' environment variable. It works, I just tested it with this very same boutique. Err, patch. The only problem with that is that this makes patch behave completely posix-compliant: POSIXLY_CORRECT If set, patch conforms more strictly to the POSIX standard: it takes the first existing file from the list (old, new, index) when intuiting file names from diff headers, it does not remove files that are empty after patching, it does not ask whether to get files from RCS or SCCS, it requires that all options precede the files in the command line, and it does not backup files when there is a mismatch. I'll try to keep this in mind when sending patches, noting added/removed files, and keeping an eye on situations where both 'file.py' and 'foo/bar/file.py' exist. ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From adhir@forumone.com Thu Feb 10 18:14:43 2000 From: adhir@forumone.com (Alok Dhir) Date: Thu, 10 Feb 2000 13:14:43 -0500 Subject: [Mailman-Developers] When will mailman 1.2 be released? Message-ID: Sorry to be getting in this late - but what will Harald's user db changes do? Things I'd love to see: 1. Ability to search for members by email address 2. Ability to store membership info in a Mysql database It would also be great if one could extend the schema of the subscriber database to include additinal fields (name, address, etc), but this is a much lower priority than the other two... > -----Original Message----- > From: mailman-developers-admin@python.org > [mailto:mailman-developers-admin@python.org]On Behalf Of > Barry A. Warsaw > Sent: Tuesday, February 08, 2000 6:28 PM > To: Gerrit > Cc: mailman-developers@python.org > Subject: Re: [Mailman-Developers] When will mailman 1.2 be released? > > > > >>>>> "GH" == Gerrit Holl writes: > > GH> when do you think mailman 1.2 will be released? > > I wish I could say. Frankly, I wish I could say whether the next > release will be 1.2 or 2.0. I think if Harald's user database changes > are significant enough, I might make it 2.0. > > GH> How stable is the current CVS tree, and how heavily is it > GH> tested? > > Stable enough to run all the lists at Python.Org. If something's > broken, I find out about it pretty dang quickly. No doubt there are > minor bugs still lurking though. > > GH> In what order of magnitude is the release frequency? > > I'm not sure what you're asking, but I've been thinking that I should > start getting beta releases out RSN. > > -Barry > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers > From Gerrit Thu Feb 10 17:32:08 2000 From: Gerrit (Gerrit Holl) Date: Thu, 10 Feb 2000 18:32:08 +0100 Subject: [Mailman-Developers] Validating passwords Message-ID: <20000210183208.A7789@stopcontact.palga.uucp> Hello, I have an idea for a feature: The password the user supplies may not be too similar to the email adres. A minimal check could be (pseudo-code): if string.find(string.lower(email_adres), string.lower(password)): raise InvalidPassword("adress may not contain password") this could be an option set by the list owner. What do you think? regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From Gerrit Thu Feb 10 20:03:05 2000 From: Gerrit (Gerrit Holl) Date: Thu, 10 Feb 2000 21:03:05 +0100 Subject: [Mailman-Developers] PATCH: docstrings added Message-ID: <20000210210305.A8240@stopcontact.palga.uucp> --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Hello, I've added docstrings to *some* of the methods in MailList.py. More will probably follow. Can someone please apply this patch to the CVS tree, after checked the English and the content? regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="MailList.py.diff" --- MailList.py Thu Feb 10 20:57:53 2000 +++ /home/gerrit/MailList.py Thu Feb 10 20:54:18 2000 @@ -60,6 +60,19 @@ class MailList(MailCommandHandler, HTMLFormatter, Deliverer, ListAdmin, Archiver, Digester, SecurityManager, Bouncer, GatewayManager): + """MailList([name [,lock]]) -> instance + + This class represents a mailinglist. The constructor takes zero, one + or two arguments: if 'name' is given, the mailinglist with the name + 'name' is loaded. If the mailinglist isn't found, the exception + Errors.MMUnknownListError is raised. If the second argument is true, + the list is locked. This is true by default. If you are going to write + to the list, this should be true. It's a good habit to unlock the list + when you're ready, using the Unlock() method, in case something goes + wrong or your program runs a long time. You don't need to lock if you + are going to read list data only. + """ + def __init__(self, name=None, lock=1): if name and name not in Utils.list_names(): raise Errors.MMUnknownListError, 'list not found: %s' % name @@ -82,15 +95,27 @@ pass def GetMembers(self): - """returns a list of the members. (all lowercase)""" + """GetMembers() -> list + + Returns a list of all subscribed non-digest members, all in lowercase. + """ + return self.members.keys() def GetDigestMembers(self): - """returns a list of digest members. (all lowercase)""" + """GetDigestMembers() -> list + + Returns a list of all subscribed digest members, all in lowercase. + """ + return self.digest_members.keys() def GetDeliveryMembers(self): - """returns a list of the members with username case preserved.""" + """GetDeliveryMembers() -> list + + Returns a list of the non-digest members with username case preserved. + """ + res = [] for k, v in self.members.items(): if type(v) is StringType: @@ -100,7 +125,11 @@ return res def GetDigestDeliveryMembers(self): - """returns a list of the members with username case preserved.""" + """GetDigestDeliveryMembers() -> list + + Returns a list of the digest members with username case preserved. + """ + res = [] for k,v in self.digest_members.items(): if type(v) is StringType: @@ -110,9 +139,11 @@ return res def __AddMember(self, addr, digest): - """adds the appropriate data to the internal members dict. + """__AddMember(addr, digest) -> None - If the username has upercase letters in it, then the value + Adds the appropriate data to the internal members dict. + + If the username has uppercase letters in it, then the value in the members dict is the case preserved address, otherwise, the value is 0. """ @@ -128,18 +159,25 @@ self.members[string.lower(addr)] = addr def GetAdminEmail(self): + """GetAdminEmail() -> string + + Returns the email address (a string) of the list owner. + """ + return '%s-admin@%s' % (self._internal_name, self.host_name) def GetMemberAdminEmail(self, member): - """Usually the member addr, but modified for umbrella lists. + """GetMemberAdminEmail(member) + + Usually the member address, but modified for umbrella lists. Umbrella lists have other mailing lists as members, and so admin stuff like confirmation requests and passwords must not be sent to the member addresses - the sublists - but rather to the administrators of the sublists. This routine picks the right address, considering regular member address to be their own administrative addresses. - """ + if not self.umbrella_list: return member else: @@ -147,8 +185,11 @@ return "%s%s@%s" % (acct, self.umbrella_member_suffix, host) def GetUserSubscribedAddress(self, member): - """Return the member's case preserved address. + """GetUserSubscribedAddress(member) -> string + + Returns the member's case preserved address. """ + member = string.lower(member) cpuser = self.members.get(member) if type(cpuser) == IntType: @@ -163,24 +204,50 @@ return None def GetUserCanonicalAddress(self, member): - """Return the member's address lower cased.""" + """GetUserCanonicalAddress(member) -> string + + Return the member's address lower cased. + """ + cpuser = self.GetUserSubscribedAddress(member) if cpuser is not None: return string.lower(cpuser) return None def GetRequestEmail(self): + """GetRequestEmail(self) -> string + + Returns the address where users can send administrative requests. + """ + return '%s-request@%s' % (self._internal_name, self.host_name) def GetListEmail(self): + """GetListEmail() -> string + + Returns the email address of the mailinglist. + """ + return '%s@%s' % (self._internal_name, self.host_name) def GetListIdentifier(self): - """Return the unique (RFC draft-chandhok-listid-02) identifier.""" + """GetListIdentifier() -> string + + Returns the unique (RFC draft-chandhok-listid-02) identifier. + """ + return ("%s <%s.%s>" % (self.description, self._internal_name, self.host_name)) def GetScriptURL(self, scriptname, relative=0): + """GetScriptURL(scriptname[, relative]) -> string + + Returns the URL of the 'scriptname' page of the mailing list. If + the optional argument 'relative' is true (it's false by default), + the URL is relative to the directory in the PATH_INFO environment + variable. + """ + if relative: prefix = '../' * Utils.GetNestingLevel() if not prefix: @@ -197,6 +264,15 @@ self.internal_name()) def GetOptionsURL(self, addr, obscure=0, relative=0): + """GetOptionsURL(addr[ ,obscure][, relative]]) -> string + + Returns the URL of the options page of the email address 'addr'. + If 'obscure' is set, the email address is obscured using + Utils.ObscureEmail(addr). If 'relative' is set, the URL is + relative to the directory in the PATH_INFO environment + variable. + """ + addr = string.lower(addr) url = self.GetScriptURL('options', relative) if obscure: @@ -208,10 +284,22 @@ GetAbsoluteScriptURL = GetScriptURL def GetRelativeScriptURL(self, scriptname): + """GetRelativeScriptURL(scriptname) -> string + + Returns the URL of the 'scriptname' script, relative to the + directory in the PATH_INFO environment variable. + """ + return self.GetScriptURL(scriptname, relative=1) def GetUserOption(self, user, option): - """Return user's setting for option, defaulting to 0 if no settings.""" + """GetUserOption(user, option) -> boolean + + Returns wheter 'option' is set for the options of 'user'. If + 'user' is not a member of the mailing list, 0 is returned. + Note that 'user' is not a string, but an integer from the + mm_cfg module. + """ user = self.GetUserCanonicalAddress(user) if option == mm_cfg.Digests: return self.digest_members.has_key(user) @@ -220,6 +308,13 @@ return not not self.user_options[user] & option def SetUserOption(self, user, option, value, save_list=1): + """SetUserOption(user, option, value[, save_list]) -> None + + Sets the 'option' option of the member 'user' to 'value'. 'value' + is an integer from the mm_cfg module. Is 'save_list' is true (it is + by default), the list configuration is saved afterwards. + """ + user = self.GetUserCanonicalAddress(user) if not self.user_options.has_key(user): self.user_options[user] = 0 @@ -255,12 +350,12 @@ # self.passwords is, of course the password in plain text. def FindUser(self, email): - """Return the lowercased version of the subscribed email address. + """FindUser(email) -> string or None - If email is not subscribed, either as a regular member or digest - member, None is returned. If they are subscribed, the return value is - guaranteed to be lowercased. + If 'email' is a member of this list, the lowercased email address + is returned. Otherwise, None is returned. """ + # shortcut lcuser = self.GetUserCanonicalAddress(email) if lcuser is not None: @@ -274,7 +369,11 @@ return string.lower(matches[0]) def InitTempVars(self, name, lock): - """Set transient variables of this and inherited classes.""" + """InitTempVars(name, lock) -> None + + Set transient variables of this and inherited classes. + """ + self.__createlock_p = lock self.__lock = LockFile.LockFile( os.path.join(mm_cfg.LOCK_DIR, name or '') + '.lock', @@ -291,7 +390,13 @@ ListAdmin.InitTempVars(self) def InitVars(self, name=None, admin='', crypted_password=''): - """Assign default values - some will be overriden by stored state.""" + """InitVars([name][, admin][, crypted_password]) -> None + + Initialized the mailing list information. This should not + be used directly (FIXME: truth?), use the .Create method + to create a new list instead. + """ + # Non-configurable list info if name: self._internal_name = name @@ -363,6 +468,10 @@ self.msg_footer = mm_cfg.DEFAULT_MSG_FOOTER def GetConfigInfo(self): + """GetConfigInfo() -> dictionairy + + Returns the information about the list configuration options. + """ config_info = {} config_info['digest'] = Digester.GetConfigInfo(self) config_info['archive'] = Archiver.GetConfigInfo(self) @@ -739,6 +848,14 @@ return config_info def Create(self, name, admin, crypted_password): + '''Create(name, admin, crypted_password) -> None + + Creates a mailinglist called 'name', with 'admin' as the list + owner. If the list already exists, ValueError is raised. The + third argument is the _encrypted_ password (a password can be + encrypted using the Crypt.crypt() function). + ''' + if name in Utils.list_names(): raise ValueError, 'List %s already exists.' % name Utils.ValidateEmail(admin) @@ -766,6 +883,11 @@ os.umask(ou) def Save(self): + """Save() -> None + + Saves the current configuration of the list. + """ + # If more than one client is manipulating the database at once, we're # pretty hosed. That's a good reason to make this a daemon not a # program. --huq684BweRXVnRxX-- From Gerrit Thu Feb 10 20:14:52 2000 From: Gerrit (Gerrit Holl) Date: Thu, 10 Feb 2000 21:14:52 +0100 Subject: [Mailman-Developers] Frames (was: unsubscriptions requiring approval) In-Reply-To: <20000210173114.F477@xs4all.nl>; from thomas@xs4all.net on Thu, Feb 10, 2000 at 05:31:14PM +0100 References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> <20000210173114.F477@xs4all.nl> Message-ID: <20000210211452.A8811@stopcontact.palga.uucp> Thomas Wouters wrote on 950200274: ... > (It might be even worth it to take a look at a frames interface, possibly ?) Noooo! Please don't! Sorry, but I really, really hate frames for several reasons. * I want to be able to admin my site in lynx, since I often haven't booted X and want to administrate fast, * it's not possible to bookmark a frames page, you can't bookmark the combination of the frames, * in netscape, the images button doesn't work on frames, * the sidebar needs to use javascript, and javascript is hard to maintain, * it's often unclear which frame you selected, so scrolling is hard, * it's neither in the HTML 4.01, nor in the XHTML 1.0 dtd. There's a seperate DTD for it. I think we should conform the HTML to XHTML 1.0 anyway (comments?), * you can't have a quick look at the HTML source code (I use to do that very often). I'd rather administrate my lists by email than using frames! Okay, maybe I'm exaggerating, but please, no frames! regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 10 20:30:01 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 10 Feb 2000 15:30:01 -0500 (EST) Subject: [Mailman-Developers] Frames (was: unsubscriptions requiring approval) References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> <20000210173114.F477@xs4all.nl> <20000210211452.A8811@stopcontact.palga.uucp> Message-ID: <14499.8137.204319.580736@anthem.cnri.reston.va.us> >>>>> "GH" == Gerrit Holl writes: >> (It might be even worth it to take a look at a frames >> interface, possibly ?) | Noooo! Please don't! | Sorry, but I really, really hate frames for several reasons. I concur 100%. To get largely the same effect without the headaches, see what we do on Python.Org. -Barry From ricardo@miss-janet.com Thu Feb 10 20:39:26 2000 From: ricardo@miss-janet.com (Ricardo - Miss Janet . Fanclub) Date: Thu, 10 Feb 2000 21:39:26 +0100 Subject: [Mailman-Developers] When will mailman 1.2 be released? In-Reply-To: ; from adhir@forumone.com on Thu, Feb 10, 2000 at 01:14:43PM -0500 References: Message-ID: <20000210213926.A897@miss-janet.com> On Thu, Feb 10, 2000 at 01:14:43PM -0500, Alok Dhir wrote: > 1. Ability to search for members by email address > 2. Ability to store membership info in a Mysql database > It would also be great if one could extend the schema of the subscriber > database to include additinal fields (name, address, etc), but this is a > much lower priority than the other two... Actually I've been thinking of some way to integrate (My)SQL with Mailman. It would be great if you could choose to have all the configuration stored in any form of DB you want, instead of the python db (the name slipped me for a moment here)... but i guess that would mean a major rewrite of a lot of MM's fuctinality... or maybe the user db could be separated from config.db... does anybody have some thoughts on this too? I know a lot of people would like to be able to have a customizable userdatabase. We're going to use Mailman for a project soon, where we need to define extra fields too. Ricardo. From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 10 21:01:36 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 10 Feb 2000 16:01:36 -0500 (EST) Subject: [Mailman-Developers] When will mailman 1.2 be released? References: <20000210213926.A897@miss-janet.com> Message-ID: <14499.10032.385586.426796@anthem.cnri.reston.va.us> >>>>> "R" == Ricardo writes: R> Actually I've been thinking of some way to integrate (My)SQL R> with Mailman. It would be great if you could choose to have R> all the configuration stored in any form of DB you want, R> instead of the python db (the name slipped me for a moment R> here)... but i guess that would mean a major rewrite of a lot R> of MM's fuctinality... or maybe the user db could be separated R> from config.db... R> does anybody have some thoughts on this too? R> I know a lot of people would like to be able to have a R> customizable userdatabase. We're going to use Mailman for a R> project soon, where we need to define extra fields too. I believe that the Right Solution would be to design an API for the underlying user database. I want to support people who, e.g. already have user accounts through Zope or some other Web-app/portal software, so that they don't have to force users to log in twice. This would let us distribute a GPL compatible database back-end for stand-alone operation, but should also let us integrate with existing user databases. That's about as far as my own thinking has gone. -Barry From claw@kanga.nu Thu Feb 10 21:23:09 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 10 Feb 2000 13:23:09 -0800 Subject: [Mailman-Developers] Frames (was: unsubscriptions requiring approval) In-Reply-To: Message from Gerrit Holl of "Thu, 10 Feb 2000 21:14:52 +0100." <20000210211452.A8811@stopcontact.palga.uucp> References: <20000208020526.N19265@xs4all.nl> <20000209183107.B477@xs4all.nl> <20000210173114.F477@xs4all.nl> <20000210211452.A8811@stopcontact.palga.uucp> Message-ID: <12287.950217789@kanga.nu> On Thu, 10 Feb 2000 21:14:52 +0100 Gerrit Holl wrote: > I want to be able to admin my site in lynx, since I often haven't > booted X and want to administrate fast... While not disagreing with your view of frames, you might like to look at W3M as a replacement for Lynx. Its a nippy little text mode web browser that handles frames _and_ tables properly. -- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From Harald.Meland@usit.uio.no Thu Feb 10 23:28:35 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 11 Feb 2000 00:28:35 +0100 Subject: [Mailman-Developers] Features requested... References: <14384.53717.606487.837270@anthem.cnri.reston.va.us> <14384.58392.966859.375322@anthem.cnri.reston.va.us> Message-ID: [Barry A. Warsaw] > >>>>> "SH" == Steven Hazel writes: > > Me> The problem here is that we don't actually keep that > Me> information around in the database. This is bogus, and is > Me> something that I hope will get fixed when we have a real User > Me> database (don't ask when ;) > > SH> What are the current plans for this? I'd like to help. I > SH> should have some time over the coming holidays. > > Well, there are a number of questions (if it was an easy hack it'd > already be done :). This is my cue, I guess ;) > First, do we want to use a real OODB underneath (such as ZODB), or do > we cook our own? I tried using ZODB at first, but found that my try at that scaled very poorly (The FileStorage file was ~160MB even before there were 2000 entries in my database, and the time needed to open the database seemed to scale with the size of it...). Dunno if this was due to me being boneheaded when implementing stuff, or to actual problems with ZODB, but I kinda lost my spirit for it anyway :-\ So, I surfed to Parnassos, searched the vault there, and came up with a copy of bplustree.py. Implemented purely in Python, as opposed to ZODB, which relies on some C Zope modules. The disk space used per record seems very reasonable. Open time appears to be pretty much unrelated to file size. Record access time seems fast enough. One problem, though -- this module does not handle multiple simultanous writes, so we have to be careful with locking -- which *could* make the user db into a bottleneck. [ As we put Mailman into production with the ~3.500 lists we have here at uio.no, we discovered that the current locking code didn't work very well on a system that got so busy it started swapping -- so I rewrote the entire file locking module to be more robust, and added some code to have the "post" mail injection script exit with temporary error codes if it couldn't get a lock on the list within a reasonable amount of time -- leaving it to the MTA to try injecting the mail again later. Things have been running smooth ever since. ] A fresh entry (created to be added as member of LIST-A) in the resulting user db have these attributes: {'admin_for': {}, 'admin_pwd': None, 'aliases': {'ADDRESS@DOMAIN': 1}, 'member_of': {'LIST-A': 1}, 'member_pwd': 'PASSWORD', 'preferred_address': 'ADDRESS@DOMAIN'} Currently 'aliases' and 'preferred_address' isn't used, but the intention is to use them for associating several mail addresses with the same user account (probably by some means of email confirmation). 'member_pwd' is stored in cleartext, while 'admin_pwd' (when set) is stored as a MD5 digest. An "username" field has been added to all login pages -- currently we're running with this as a text entry field, but I'll probably install some changes to use a Message Header: Message Excerpt: Action:
Approve Reject Discard (eg, spam)
--h31gzZEtNLTqOjlF-- From Gerrit Mon Feb 14 15:50:37 2000 From: Gerrit (Gerrit Holl) Date: Mon, 14 Feb 2000 16:50:37 +0100 Subject: [Mailman-Developers] Mailman limitations In-Reply-To: ; from Harald.Meland@usit.uio.no on Sun, Feb 13, 2000 at 07:18:58PM +0100 References: <200002122023.MAA08929@ns2.uncanny.net> <20000212232702.K477@xs4all.nl> Message-ID: <20000214165037.A1849@stopcontact.palga.uucp> Harald Meland wrote on 950465938: > > I don't want to give anyone the idea that the direction of Mailman > development is not properly influenced by the input from the members > on this list -- after all, discussing Mailman development is what this > list is for. I've already sent some patches to this list, but none were implemented. For example, my latest patch, it added docstrings to many methods of MailList. I also posted a proposal for a feature to be added to the TODO list; no response. I don't feel very comfortable with it. Who have access to the CVS tree? The five people mentoined in the acknowledgements file? kind regards, Gerrit. -- Homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth From Harald.Meland@usit.uio.no Mon Feb 14 16:57:54 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 14 Feb 2000 17:57:54 +0100 Subject: [Mailman-Developers] Mailman limitations In-Reply-To: Thomas Wouters's message of "Mon, 14 Feb 2000 00:38:07 +0100" References: <200002122023.MAA08929@ns2.uncanny.net> <20000212232702.K477@xs4all.nl> <20000214003807.Q477@xs4all.nl> Message-ID: [Thomas Wouters] > But Mailman is, in my opinion, nowhere near complicated enough to > make it easy to misconfigure it. Yet. With a few exceptions (read: weird special cases), I agree. > > Ideally, we'd like to support as big a set of (good :) features as any > > other mailing list administration package, but without alienating > > first-time Mailman list administrators through a veritable maze of > > list settings, all of them interconnected... Good documentation might > > solve part of this, but if there are (overly) complex features the > > corresponding documentation will probably have to be complex as well, > > meaning that the Mailman learning curve gets steeper. > > Exactly. And let me tell you that good documentation is not enough if the > first look the fresh administrator gets of his new list is as complicated as > the bridge of the USS Enterprise. That's what the "might solve _part of_ this" stuff was meant to imply... ;) > However, I dont think redesigning the adminpages to make it all fit AND > appear sane, is impossible. But, are "sane" and "user understandable" interchangeable? (Sorry, the bofh part of me took control there for a minute :). > I think it can be done with just a little bit of thought ;) > Currently, the admin page is, to be frank, overwhelming. Yup -- that's the reason I'm somewhat sceptical to accept patches for new "special case" features. > The moment you open the page, you get a large list of options to > select from. Some of these are useful and clear. Others, most, are > more-or-less obvious, but not of interest. And some are plain > weird. You do your best to browse them and edit them to your need, > and then you look up, and you see that you have *seven*more*pages* > to do ! AND three more pages, which may contain more links, of other > administrativia ! I guess this is a bit like what kind of stereo systems people prefer -- some people want the one with the most buttons (and flashing lights), while other people just want something that *works*. :-D Some time back, someone asked whether it would be possible to have different "styles" on the list admin pages -- i.e., the more techish admins would get the full (overwhelming) set of admin settings, while newbie admins would get a (much less confusing and) smaller set of the most commonly used list settings (possibly with the option of getting access to the full set of settings if they e.g. followed some "for expert list admins" link). Which set of options to present to the admin could either be settable per list or per admin (when we get the user database in place). There have also been a little talk of even more generic solutions to this -- e.g. implementing list configurations as inheritance hierarchies, so that a change to the "base" configuration would automatically affect all the lists that inherit the attributes that were changed but don't override them. However, it seems to me that every time we start discussing advanced web interface stuff, someone mentions integrating Mailman with Zope -- which it is said would buy us a lot of neat features practically for free -- and people go "Yeah" and "Cool" and "Sounds like a perfect match"... and then nobody actually _do_ anything about it all. :) Should Mailman be integrated with Zope? Would such a change mean that Mailman has to _rely_ on Zope being installed, or would it be feasible (and resonably easy) for Mailman to take advantage of Zope only if it's available (at run time? Or at install time?)? What advantages would such a change buy us? Are there things we want to do anyway, and which would _have_ to be changed to integrate with Zope, and thus could be started work on right away? Should all of this wait until after the next Mailman release is completed? ;) Is my ranting about this at all called for? ;) > What needs to be done mostly is just creating more but smaller pages, > especially for the current 'general options' and 'privacy options' pages, > plus for whatever options are added. To me, having a tangle with lots of different pages with few options on them isn't really much better than a tangle with lots of options on a few pages. The _real_ problem is to avoid the tangle. I agree that the structure of the current admin pages aren't optimal, though -- but restructuring the pages will only take us so far if we start adding specialcase code left and right. > (While I was writing this and thinking it up (in that order ;) I thought how > incredibly cool it would be to have some kind of rule-based message > acception (/refusal/holding/discarding/temp-fail/...) system, where each > rule could be header-match, number of postings to the list so far, load of > the machine matched against a priority for the list or user, time of day or > whatever. And where each rule would have it's own default reject/accept > messages, of course. Not very hard to implement in Python and the current > 'pipe' architecture, if I may be so kind. But for the life of me I can't > figure out how to make such a scheme configurable through an HTML page ;P) That's *exactly* the way I feel about that stuff. :) Of course, you could just let the admin edit the rules as pure text -- and then have the CGI script do the appropriate syntax checks and other sanity checks/rule validation before actually changing the list's configuration. Of course, that would mean we're _very_ close to significantly affecting the slope of Mailman's learning curve. > > > I can't say for certain what parts are currently being developped, > > > though, I'm relatively new on this list, and a lot of development > > > seems to happen behind the scenes. > > > I'm not quite sure what you mean by that, but just to clarify: I try > > to keep/redirect whatever discussions on Mailman development I get > > involved in to this list. However, I've "been away" for some time, > > busy doing Real Work -- part of which was to implement new Mailman > > features that we here at uio.no were in pressing need of. > > > I don't want to give anyone the idea that the direction of Mailman > > development is not properly influenced by the input from the members > > on this list -- after all, discussing Mailman development is what this > > list is for. > > I'm sorry, I didn't mean to imply that this list was defunct, or that > mailman development is dead, or that people are wasting their time waiting > for the developers, or any of that. Good -- and I tried to phrase the above in a way as not to suggest that I thought you did -- it was meant purely as an clarification, not as some kind of an accusation. > It's just that I dont see the lively discussions I'm used to I can relate with that -- but I can't really figure out why that's how things are here at mailman-developers... > Or maybe I hadn't realized how busy the cabal is -- having a fun > project (Mailman) highish on my priority list for once makes me > slightly impatient. You want us to believe that normally you're _not_ impatient? Yeah, right... ;-D > I shouldn't complain, anyway. Most of my patches have received good > commentary, and one even got applied ! Well, neither should we... just keep'em coming. Cheers, -- Harald From lindsey@ncsa.uiuc.edu Mon Feb 14 17:39:51 2000 From: lindsey@ncsa.uiuc.edu (Christopher Lindsey) Date: Mon, 14 Feb 2000 11:39:51 -0600 (CST) Subject: [Mailman-Developers] Mailman limitations In-Reply-To: from "Harald Meland" at Feb 14, 2000 05:57:54 PM Message-ID: <200002141739.LAA17610@glorfindel.ncsa.uiuc.edu> > Should Mailman be integrated with Zope? > > Would such a change mean that Mailman has to _rely_ on Zope being > installed, or would it be feasible (and resonably easy) for Mailman to > take advantage of Zope only if it's available (at run time? Or at > install time?)? The latter. Look at the mess that pipermail got Mailman into. It became an unsupported package, then people wanted to integrate something bigger and better but couldn't... Ultimately things changed to use built-in functionality (pipermail) but remain configurable enough to specify one's own external archiver... Mailman/Zope interoperability should probably follow the same guidelines. Chris From thomas@xs4all.net Mon Feb 14 21:10:19 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 14 Feb 2000 22:10:19 +0100 Subject: [Mailman-Developers] Mailman limitations In-Reply-To: ; from Harald.Meland@usit.uio.no on Mon, Feb 14, 2000 at 05:57:54PM +0100 References: <200002122023.MAA08929@ns2.uncanny.net> <20000212232702.K477@xs4all.nl> <20000214003807.Q477@xs4all.nl> Message-ID: <20000214221018.R477@xs4all.nl> On Mon, Feb 14, 2000 at 05:57:54PM +0100, Harald Meland wrote: > [Thomas Wouters] [me complaining about configuration pages] > Some time back, someone asked whether it would be possible to have > different "styles" on the list admin pages -- i.e., the more techish > admins would get the full (overwhelming) set of admin settings, while > newbie admins would get a (much less confusing and) smaller set of the > most commonly used list settings (possibly with the option of getting > access to the full set of settings if they e.g. followed some "for > expert list admins" link). Well, what would be pretty welcome, i think, is a clear division between 'You probably want to edit or at least browse these options', and 'you dont really need to change anything here, unless Mailman isn't working properly.' That's more or less what my incoherent blurp about adminpages was about. The expert-options dont actually need to be inaccessible... they just need to be flagged as 'dont bother with these', or some such. > Should Mailman be integrated with Zope? Eww, well, saying it like that, 'be integrated', my first reaction is 'no'. (Well, that's my first printable reaction ;) I dont run Zope, and i dont plan to run Zope anytime soon, mostly because it does something i, being a techy, have absolutely no need for. Maybe our webteam wants it, but i doubt it -- they're focusing on other things right now. I wouldn't object in any way against a sort-of 'driver' for Zope, a Zope-interface to Mailman, as long as you aren't supposed to use it, but i dobut it'll fly if it doesn' thave at least one, preferably more, dedicated maintainers. And the fact it hasn't been built yet, suggests that that dedication isn't there yet ;) > > (While I was writing this and thinking it up (in that order ;) I thought how > > incredibly cool it would be to have some kind of rule-based message > > acception (/refusal/holding/discarding/temp-fail/...) system, where each > > rule could be header-match, number of postings to the list so far, load of > > the machine matched against a priority for the list or user, time of day or > > whatever. And where each rule would have it's own default reject/accept > > messages, of course. Not very hard to implement in Python and the current > > 'pipe' architecture, if I may be so kind. But for the life of me I can't > > figure out how to make such a scheme configurable through an HTML page ;P) > > That's *exactly* the way I feel about that stuff. :) > Of course, you could just let the admin edit the rules as pure text -- > and then have the CGI script do the appropriate syntax checks and > other sanity checks/rule validation before actually changing the > list's configuration. > Of course, that would mean we're _very_ close to significantly > affecting the slope of Mailman's learning curve. No, because you can offer the current system as well. Maybe a few 'default' reject/hold rules that you can turn on and off. But you can't do it really flexible using HTML, so if you want the *real* power, you have to edit the textbox. And the textbox needs to have flashing red warning messages around it, of course. But it needn't be more difficult than Python. That's more or less what i had in mind... a 'Simple as hell, who needs a manual' section, and a 'easy as python, read the simple docs first' section ;) > > It's just that I dont see the lively discussions I'm used to > I can relate with that -- but I can't really figure out why that's how > things are here at mailman-developers... Because the developers are overworked, and the public too impatient ? :) Short-message-cuz-Fawlty-Towers-is-on-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From ckolar@admin.aurora.edu Mon Feb 14 22:57:55 2000 From: ckolar@admin.aurora.edu (Christopher Kolar) Date: Mon, 14 Feb 2000 16:57:55 -0600 Subject: [Mailman-Developers] list admin guide v0.2 now online Message-ID: <4.2.2.20000214165638.00aa6ac0@admin.aurora.edu> Hello everyone. It has been a while since I last updated the information, but I have finally gotten around to making revisions to my first attempt at MailMan documentation. The second versions of the list administrator's guide and quick reference are now online at: www.aurora.edu/~ckolar/mailman This document has been prepared for list owners: people who are not necessarily technical but who own a MailMan list. For various reasons I am locked into doing them in HTML for the time being, but will someday take it all and put it into TexInfo format. Please feel free to send me feedback. --chris -- /////\\\\\/////\\\\\ Christopher G. Kolar Director of Instructional Technology Aurora University, Aurora, Illinois ckolar@admin.aurora.edu -- www.aurora.edu/~ckolar [PGP Public Key ID: 0xC6492C72] From Harald.Meland@usit.uio.no Tue Feb 15 17:05:09 2000 From: Harald.Meland@usit.uio.no (Harald Meland) Date: 15 Feb 2000 18:05:09 +0100 Subject: [Mailman-Developers] Mailman limitations In-Reply-To: Gerrit Holl's message of "Mon, 14 Feb 2000 16:50:37 +0100" References: <200002122023.MAA08929@ns2.uncanny.net> <20000212232702.K477@xs4all.nl> <20000214165037.A1849@stopcontact.palga.uucp> Message-ID: [Gerrit Holl] > Harald Meland wrote on 950465938: > > > > I don't want to give anyone the idea that the direction of Mailman > > development is not properly influenced by the input from the members > > on this list -- after all, discussing Mailman development is what this > > list is for. > > I've already sent some patches to this list, but none were > implemented. For example, my latest patch, it added docstrings to > many methods of MailList. Sorry, I haven't gotten around to that -- and, as I've tried to indicate, my local tree is regrettably quite a bit out of sync with CVS right now :( I'm trying to use whatever time I can find to backport the parts of my local tree that can/should go into CVS _now_ -- and get up to date with this list. However, I have put your messages in the "look at this when I have time" category -- I'll try to become better at notifying people whenever I put stuff into that category. > Who have access to the CVS tree? The five people mentoined in the > acknowledgements file? AFAIK: Yes, only the "core developers" have CVS write access. I think there have been some attempts to make that situation better whenever it got clear that all the core developers were to busy with other stuff, but I don't think anything has come of these attempts yet. Hmmm, would SourceForge solve any of this? -- Harald From ricardo@rixhq.nu Tue Feb 15 21:57:11 2000 From: ricardo@rixhq.nu (Ricardo 'admindb' Kustner) Date: Tue, 15 Feb 2000 22:57:11 +0100 Subject: [Mailman-Developers] Message.Message Message-ID: <20000215225711.A4983@miss-janet.com> Hi, It seems like nobody looked at the changes i've posted for admindb.py cause i just found out that I made it more difficult than necessary to seperate the message body from the header :) This funcinality is already possible using MM/python libs : fp = open(os.path.join(mm_cfg.DATA_DIR, filename)) fullmsg = Message.Message(fp) msg_header = string.join(fullmsg.headers, '') msg_body = fullmsg.body fp.close() ...on the bright side, I learned a bit more about python and the MM code :) oh i just realized, that this way the ADMINDB_PAGE_TEXT_LIMIT isn't taken into account... :( Ricardo. From ckolar@admin.aurora.edu Wed Feb 16 16:07:42 2000 From: ckolar@admin.aurora.edu (Christopher Kolar) Date: Wed, 16 Feb 2000 10:07:42 -0600 Subject: [Mailman-Developers] sugg: disable notify when using remove_member Message-ID: <4.2.2.20000216100346.00b43770@admin.aurora.edu> Hi everyone. Is there any chance that a toggle could be added to the remove_member program to make it function like add_member with respect to notification? We have a lot of lists that get cleared out periodically, and we have a lot of users who leave us every semester, and the notification either generates unnecessary helpdesk calls from the confused or generates bounces from people whose accounts are no longer active. I am not asking for it to be added to the web interface, because I know that it can be abused, but it would be nice at the command line for some of the more heavy-duty automated things that we are putting together. Cheers, --chris -- /////\\\\\/////\\\\\ Christopher G. Kolar Director of Instructional Technology Aurora University, Aurora, Illinois ckolar@admin.aurora.edu -- www.aurora.edu/~ckolar [PGP Public Key ID: 0xC6492C72] From thomas@xs4all.net Wed Feb 16 17:20:16 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 16 Feb 2000 18:20:16 +0100 Subject: [Mailman-Developers] internationalization ? Message-ID: <20000216182016.U477@xs4all.nl> Barry mentioned something, a couple of weeks back, about spending a weekend trying to integrate i18n patches to mailman, to internationalize it. I assume those patches do some heavy-duty rewriting of various bits of the template code and such, so i'm a bit hesitant to work on those bits of code, though i have a few things left on my wishlist in those areas ;) Can anyone tell me what the status is of that code, wether it indeed affects large parts of mailman, or give me a pointer to the patches ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From ricardo@rixhq.nu Wed Feb 16 21:23:50 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Wed, 16 Feb 2000 22:23:50 +0100 Subject: [Mailman-Developers] internationalization ? In-Reply-To: <20000216182016.U477@xs4all.nl>; from thomas@xs4all.net on Wed, Feb 16, 2000 at 06:20:16PM +0100 References: <20000216182016.U477@xs4all.nl> Message-ID: <20000216222350.C13168@miss-janet.com> On Wed, Feb 16, 2000 at 06:20:16PM +0100, Thomas Wouters wrote: > Barry mentioned something, a couple of weeks back, about spending a weekend > trying to integrate i18n patches to mailman, to internationalize it. I > assume those patches do some heavy-duty rewriting of various bits of the > template code and such, so i'm a bit hesitant to work on those bits of code, > though i have a few things left on my wishlist in those areas ;) there's a special list for mailman i18n stuff... check out the info/archives at http://www.python.org/mailman/listinfo/mailman-i18n Ricardo. -- From jcrey@uma.es Thu Feb 17 08:08:40 2000 From: jcrey@uma.es (Juan Carlos Rey Anaya) Date: Thu, 17 Feb 2000 09:08:40 +0100 Subject: [Mailman-Developers] sugg: disable notify when using remove_member References: <4.2.2.20000216100346.00b43770@admin.aurora.edu> Message-ID: <38ABAC88.ECBB8935@uma.es> Christopher Kolar wrote: > > Hi everyone. Is there any chance that a toggle could be added to the > remove_member program to make it function like add_member with respect to > notification? Modify called function to: mlist.DeleteMember(addr, None, 0) in remove_members Cheers -- ___ / F \ [[[]]]] ( O O ) #----------------0000--(_)--0000---------------# | Juan Carlos Rey Anaya (jcrey@uma.es) | | Servicio Central de informática | | Universidad de Málaga - España | #----------------------------------------------# # Solo se que cada vez se menos :-| # #----------------------------------------------# From fil@bok.net Thu Feb 17 08:59:49 2000 From: fil@bok.net (Fil) Date: Thu, 17 Feb 2000 09:59:49 +0100 Subject: [Mailman-Developers] RE: dont request passwords from web subscriber patch In-Reply-To: ; from lee@ufojoe.com on Sun, Feb 06, 2000 at 08:02:00PM -0500 References: Message-ID: <20000217095949.A29324@orwell.bok.net> * Lee Weston (lee@ufojoe.com) écrivait : > Fri, 7 Jan 2000 14:58:16 +0100 (MET) > Fil fil@bok.net wrote: > > > >could the maintainers tell me if this patch is accepted or rejected ? > >Patch name : dont request passwords from web subscriber > > I'm sorry to see no answer. I will be looking at adding it. > An "only allow random password" mode would have the benifit of > substiantially raising the bar for writting cgi to raid / spam a site, by > making them parse email as well. That's right. From several messages in the preceding week it looks like there's interest in this kind of patch ; certainly the patch I have submited is not complete enough. So let's look at it as a feature request./ From jcrey@uma.es Thu Feb 17 09:32:16 2000 From: jcrey@uma.es (Juan Carlos Rey Anaya) Date: Thu, 17 Feb 2000 10:32:16 +0100 Subject: [Mailman-Developers] internationalization ? References: <20000216182016.U477@xs4all.nl> Message-ID: <38ABC020.4BD37692@uma.es> Thomas Wouters wrote: > > Can anyone tell me what the status is of that code, wether it indeed affects > large parts of mailman, or give me a pointer to the patches ? > The actual status is: I'm waiting for Barry to finish integrating my patches to actual Mailman. I sent patches on 11/Jan/2000 and new files. Modified sources are: /home/mailman/Mailman/Cgi/admin.py /home/mailman/Mailman/Cgi/admindb.py /home/mailman/Mailman/Cgi/edithtml.py /home/mailman/Mailman/Cgi/handle_opts.py /home/mailman/Mailman/Cgi/listinfo.py /home/mailman/Mailman/Cgi/options.py /home/mailman/Mailman/Cgi/private.py /home/mailman/Mailman/Cgi/roster.py /home/mailman/Mailman/Cgi/subscribe.py /home/mailman/Mailman/Archiver/Archiver.py /home/mailman/Mailman/Archiver/pipermail.py /home/mailman/Mailman/Bouncer.py /home/mailman/Mailman/Digester.py /home/mailman/Mailman/GatewayManager.py /home/mailman/Mailman/HTMLFormatter.py /home/mailman/Mailman/ListAdmin.py /home/mailman/Mailman/Deliverer.py /home/mailman/Mailman/MailCommandHandler.py /home/mailman/Mailman/MailList.py /home/mailman/Mailman/Utils.py /home/mailman/Mailman/htmlformat.py /home/mailman/Mailman/Defaults.py /home/mailman/Mailman/Handlers/Acknowledge.py /home/mailman/Mailman/Handlers/CookHeaders.py /home/mailman/Mailman/Handlers/HandlerAPI.py /home/mailman/Mailman/Handlers/Hold.py /home/mailman/Mailman/Handlers/ToDigest.py /home/mailman/Mailman/Handlers/ToUsenet.py and I created one directory hierarchy under /home/mailman/messages (all the stuff is new) and modified the structure of templates directory in subdirectories, each one for each suported language. I modified some minor changes I don't remeber now. We will have to wait :-) Cheers Ah! There is a functional demo version at: http://joker.sci.uma.es/mailman/listinfo if anybody wish to see admin messages translated, admin passwd is 'ejemplo' (behave well :-) -- ___ / F \ [[[]]]] ( O O ) #----------------0000--(_)--0000---------------# | Juan Carlos Rey Anaya (jcrey@uma.es) | | Servicio Central de informática | | Universidad de Málaga - España | #----------------------------------------------# # Solo se que cada vez se menos :-| # #----------------------------------------------# From ppetru@coltronix.com Thu Feb 17 15:03:21 2000 From: ppetru@coltronix.com (Petru Paler) Date: Thu, 17 Feb 2000 17:03:21 +0200 Subject: [Mailman-Developers] Adding a new archiver Message-ID: <20000217170321.A3457@coltronix.com> Hello, I want to add a custom archiver to Mailman to make it pump messages into Zope. Is there a standard way to do this or should I just go and hack the source ? Has anyone made Mailman and Zope talk to each other ? -Petru From claw@kanga.nu Fri Feb 18 07:46:13 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 17 Feb 2000 23:46:13 -0800 Subject: [Mailman-Developers] Adding a new archiver In-Reply-To: Message from Petru Paler of "Thu, 17 Feb 2000 17:03:21 +0200." <20000217170321.A3457@coltronix.com> References: <20000217170321.A3457@coltronix.com> Message-ID: <14483.950859973@kanga.nu> On Thu, 17 Feb 2000 17:03:21 +0200 Petru Paler wrote: > Hello, I want to add a custom archiver to Mailman to make it > pump messages into Zope. Is there a standard way to do this or > should I just go and hack the source ? Has anyone made Mailman and > Zope talk to each other ? Your best bet is probably to write a small external tool that accepts a message from stdin and then creates the appropriate object under Zope that contains the message. You can then just plug that into MailMan as an external archiver. BTW: If you do this, please drop a note to this and the zope mailing lists. I know several people who would be interested. -- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From Gerrit Fri Feb 18 14:31:52 2000 From: Gerrit (Gerrit Holl) Date: Fri, 18 Feb 2000 15:31:52 +0100 Subject: [Mailman-Developers] Adding a new archiver In-Reply-To: <20000217170321.A3457@coltronix.com>; from ppetru@coltronix.com on Thu, Feb 17, 2000 at 05:03:21PM +0200 References: <20000217170321.A3457@coltronix.com> Message-ID: <20000218153152.A3793@stopcontact.palga.uucp> Petru Paler wrote on 950803401: > Hello, > > I want to add a custom archiver to Mailman to make it pump messages into > Zope. Is there a standard way to do this or should I just go and hack the source ? Has anyone made Mailman and Zope talk to each other ? You can subscribe an address to the list that archives all messages it receives. Add some lines to MailList.Create to create a private member of the list. That's what the Sourceforge staff did. regards, Gerrit. -- cat: /home/gerrit/.signature: No such quote or joke From bbum@codefab.com Fri Feb 18 18:30:39 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Fri, 18 Feb 2000 13:30:39 -0500 Subject: [Mailman-Developers] external archiver Message-ID: <200002181830.NAA19438@bjork.codefab.com> I have written an archiver that uses WebDAV to archive messages [with or without decoded attachments] to a WebDAV enabled web server. I'm currently using Apache, but there is no reason why it shouldn't work "out of the box" with Zope (Zope has a WebDAV enabler). Email if you want patches/code... it works *very* well. We will be going into full production with it in the next week [www.bebusy.com]. b.bum From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Sat Feb 19 03:59:04 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 18 Feb 2000 22:59:04 -0500 (EST) Subject: [Mailman-Developers] New feature, auto-responding Message-ID: <14510.5384.592046.76136@anthem.cnri.reston.va.us> I've been a bad boy. I should have been killing bugs, or catching up on the list traffic, or working on the I18N or other new features. But I had a particularly annoying itch that I wanted see how easy it was to scratch. Turned out pretty easy - it took me about an hour to implement, and another hour or so to test. What I've added is an auto-responder. You can control whether auto-responses are send to mailing list posts and/or -admin emails (it makes no sense to auto-respond to -request emails, since that's already a bot). You can also set the auto-response texts and the grace period. Below is the check-in message for those of you not on mailman-checkins. Okay, now I promise to get to all the things I'm supposed to do. I'd like to be very close to releasing 1.2 by the first weekend in March. Enjoy, -Barry -------------------- snip snip -------------------- Autoresponder: New mixin class which contains attributes and configuration information for the new replybot functionality. Mailman can now send automatic responses to mailing list posts or -admin emails (-request autoreplies aren't necessary since that's a bot anyway). The following new configuration attributes control this feature: autorespond_postings - boolean which controls autoreplies to mailing list posts. autorespond_admin - same as above the -admin address. autoresponse_postings_text - the mailing list post autoreply text. This is %(string)s interpolated with a dictionary as described below. autoresponse_admin_text - same as above for the -admin address. autoresponse_graceperiod - the number of days between automatic responses to the same email author. This variable controls both autoresponses. If this is <= 0, then there is no grace period. The following non-configurable attributes store the state: postings_responses - a dictionary of email addresses (lower cased) to the time.time() that the grace period expires. admin_responses - same as above for the -admin address. autoresponse_postings_text and autoresponse_admin_text are string interpolated with the following dictionary: listname - the mailing list's "real_name" listurl - the list's "listinfo" url requestemail - the list's -request address adminemail - the list's -admin address We should probably add others. From bwarsaw@cnri.reston.va.us Sun Feb 20 15:30:08 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Sun, 20 Feb 2000 10:30:08 -0500 (EST) Subject: [Mailman-Developers] New feature, auto-responding References: <14510.5384.592046.76136@anthem.cnri.reston.va.us> <20000219115638.A4297@miss-janet.com> Message-ID: <14512.2176.760495.408684@anthem.cnri.reston.va.us> >>>>> "RK" == Ricardo Kustner writes: RK> this sounds like an interesting feature... it would be RK> extremely cool if MM could autorespond to messages containing RK> HTML encoded email: this way those people can get a response RK> that next time they time they shouldn't post HTML email, and RK> some short instructions how to change to plain text. Or am I RK> missing the point of the autoresponder usage here? I hacked this stuff in to handle a couple of MM lists we've got running on py.org, namely help@ and tutor@. Both of these have standard reply texts they send to posters, i.e. giving them a list of places to look while their email is processed. Maybe we could hook your idea into bbum's work? RK> I know there are several people on the develop list right now RK> who's fingers are itching to contribute to the MM code. Maybe RK> we could make a list of features we want to implement in 1.2, RK> and then see if anybody wants to pick up one or more of those RK> features. Personally I really want to help out too, even RK> though i'm still learning Python and the way MM is structured, RK> i'd like to expand my knowledge beyond admindb.py :) I think that's a great idea. One of the things that will help this, I believe, is moving the project over to SourceForge. I've started to set that up, but got waylaid because I think they assign project numbers from the Freshmeat database, and it looks like they linked us to MailMan, not Mailman (i.e. Endymion's completely unrelated product). I've got a couple of calls into SF, but am still waiting to hear about that. I think SF provides a lot of very useful tools for OSS project management, and I'd like to see us move there after 1.2. The trick to accepting actual code contributions is the copyright assignment requirements. As for the way MM is structured, I've done some cleaning up of that, but I think more still needs to be done. -Barry From ricardo@rixhq.nu Sat Feb 19 10:56:38 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Sat, 19 Feb 2000 11:56:38 +0100 Subject: [Mailman-Developers] New feature, auto-responding In-Reply-To: <14510.5384.592046.76136@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Fri, Feb 18, 2000 at 10:59:04PM -0500 References: <14510.5384.592046.76136@anthem.cnri.reston.va.us> Message-ID: <20000219115638.A4297@miss-janet.com> On Fri, Feb 18, 2000 at 10:59:04PM -0500, Barry A. Warsaw wrote: > What I've added is an auto-responder. You can control whether > auto-responses are send to mailing list posts and/or -admin emails (it > makes no sense to auto-respond to -request emails, since that's > already a bot). You can also set the auto-response texts and the > grace period. Below is the check-in message for those of you not on > mailman-checkins. this sounds like an interesting feature... it would be extremely cool if MM could autorespond to messages containing HTML encoded email: this way those people can get a response that next time they time they shouldn't post HTML email, and some short instructions how to change to plain text. Or am I missing the point of the autoresponder usage here? > Okay, now I promise to get to all the things I'm supposed to do. I'd > like to be very close to releasing 1.2 by the first weekend in March. I know there are several people on the develop list right now who's fingers are itching to contribute to the MM code. Maybe we could make a list of features we want to implement in 1.2, and then see if anybody wants to pick up one or more of those features. Personally I really want to help out too, even though i'm still learning Python and the way MM is structured, i'd like to expand my knowledge beyond admindb.py :) Ricardo. From lindsey@ncsa.uiuc.edu Mon Feb 21 17:12:03 2000 From: lindsey@ncsa.uiuc.edu (Christopher Lindsey) Date: Mon, 21 Feb 2000 11:12:03 -0600 (CST) Subject: [Mailman-Developers] Quick patch to fix '%' in sender Message-ID: <200002211712.LAA13941@glorfindel.ncsa.uiuc.edu> We were having problems with email addresses containing a '%'. Specifically, if the address were held for some reason the admindb interface would choke when it tried to process it. There's a (very) small patch below against the current CVS tree. Chris ---------------------------------------------------------------------- *** Mailman/ListAdmin.py Mon Feb 21 11:11:55 2000 --- Mailman/ListAdmin.py Mon Feb 21 11:11:55 2000 *************** *** 192,198 **** \tSubject: %(subject)s''' % { 'listname' : self.internal_name(), 'rejection': rejection, ! 'sender' : sender, 'subject' : strquote(subject), } if comment: --- 192,198 ---- \tSubject: %(subject)s''' % { 'listname' : self.internal_name(), 'rejection': rejection, ! 'sender' : strquote(sender), 'subject' : strquote(subject), } if comment: From pfaff@edge.cis.mcmaster.ca Mon Feb 21 18:40:44 2000 From: pfaff@edge.cis.mcmaster.ca (Todd Pfaff) Date: Mon, 21 Feb 2000 13:40:44 -0500 (EST) Subject: [Mailman-Developers] a more uniform interface to private and public archives Message-ID: for mailman-1.1, the installation documentation suggests to create, for example, an 'Alias' path for the web server to the mailman public archives and a 'ScriptAlias' cgi-bin path to the private archives. what i'd like to suggest is that we make the interface more uniform by eliminating the 'Alias' path and access both private and public archives via a single cgi-bin interface. if the archive is private we require authentication, if not we simply bypass the authentication. i've done this with my mailman installation by doing the following: - created a new version of the MailMan/Cgi/private.py program - in mm_cfg.py, set PUBLIC_ARCHIVE_URL = '/mailman/private' PRIVATE_ARCHIVE_URL = '/mailman/private' - these could then be collapsed into one ARCHIVE_URL - we could also replace PUBLIC_ARCHIVE_FILE_DIR = os.path.join(PREFIX, 'archives/public') PRIVATE_ARCHIVE_FILE_DIR = os.path.join(PREFIX, 'archives/private') with one ARCHIVE_FILE_DIR, and we could also get rid of the public and private subdirectories altogether. in the new private.py i check listobj.archive_private and if it's set to 1 i do the usual private authentication as before. if it's not 1, i set is_auth to 1 and fall through. that's it. very clean and simple. does anyone see any problems with this? i think it certainly makes things more clear and straightforward. -- Todd Pfaff \ Email: pfaff@mcmaster.ca Computing and Information Services \ Voice: (905) 525-9140 x22920 ABB 132 \ FAX: (905) 528-3773 McMaster University \ Hamilton, Ontario, Canada L8S 4M1 \ From secabeen@pobox.com Mon Feb 21 19:56:52 2000 From: secabeen@pobox.com (Ted Cabeen) Date: Mon, 21 Feb 2000 13:56:52 -0600 Subject: [Mailman-Developers] Membership management chunking requires two authentications.. Message-ID: <200002211956.NAA29500@entropy.uchicago.edu> Mailman: 1.1 Python: 1.5.2 OS: Solaris 5.7 I don't know if this has been fixed in the CVS version, but I thought I'd submit it anyways. There seems to be a small authentication bug with the chunking in the membership management page. The page will load up correctly the first time it is accessed with the first chunk size shown. However, when you request a different chunk, it requires a reauthentication and then gives the chunk=0 page instead of the requested chunk. If you then click on the alternate chunk it loads fine. However, the additional click and authentication is annoying. -- Ted Cabeen http://www.pobox.com/~secabeen secabeen@pobox.com Check Website or finger for PGP Public Key secabeen@midway.uchicago.edu "I have taken all knowledge to be my province." -F. Bacon cococabeen@aol.com "Human kind cannot bear very much reality."-T.S.Eliot 73126.626@compuserve.com From ricardo@rixhq.nu Mon Feb 21 21:15:18 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Mon, 21 Feb 2000 22:15:18 +0100 Subject: [Mailman-Developers] Membership management chunking requires two authentications.. In-Reply-To: <200002211956.NAA29500@entropy.uchicago.edu>; from secabeen@pobox.com on Mon, Feb 21, 2000 at 01:56:52PM -0600 References: <200002211956.NAA29500@entropy.uchicago.edu> Message-ID: <20000221221518.A17481@miss-janet.com> On Mon, Feb 21, 2000 at 01:56:52PM -0600, Ted Cabeen wrote: > I don't know if this has been fixed in the CVS version, but I thought I'd > submit it anyways. There seems to be a small authentication bug with the > chunking in the membership management page. The page will load up correctly > the first time it is accessed with the first chunk size shown. However, when > you request a different chunk, it requires a reauthentication and then gives > the chunk=0 page instead of the requested chunk. If you then click on the > alternate chunk it loads fine. However, the additional click and > authentication is annoying. I have/had the same problem, and i'm using the CVS version... I just had a closer look and now I know why it behaves this way in my case: our list is accessible at two hostnames which share the exact same website... the hostname that is linked at the membership chunk links is different than the hostname i sometimes use to access the list... so netscape thinks the cookie is for a different host... which makes me wonder why the chunk links on the admin page have a full http:// link while other links on it don't ? maybe you are experiencing the same problem? compare the hostname you use to access the admin page with the hostname that's being used in the links to the membership chunks... if they are different, that's the reason why you have to re-enter the password. Ricardo. -- From secabeen@pobox.com Mon Feb 21 21:26:48 2000 From: secabeen@pobox.com (Ted Cabeen) Date: Mon, 21 Feb 2000 15:26:48 -0600 Subject: [Mailman-Developers] Membership management chunking requires two authentications.. In-Reply-To: Your message of "Mon, 21 Feb 2000 22:15:18 +0100." <20000221221518.A17481@miss-janet.com> Message-ID: <200002212126.PAA00266@entropy.uchicago.edu> In message <20000221221518.A17481@miss-janet.com>, Ricardo Kustner writes: >On Mon, Feb 21, 2000 at 01:56:52PM -0600, Ted Cabeen wrote: >> I don't know if this has been fixed in the CVS version, but I thought I'd >> submit it anyways. There seems to be a small authentication bug with the >> chunking in the membership management page. The page will load up correctly > >> the first time it is accessed with the first chunk size shown. However, whe >n >> you request a different chunk, it requires a reauthentication and then gives > >> the chunk=0 page instead of the requested chunk. If you then click on the >> alternate chunk it loads fine. However, the additional click and >> authentication is annoying. > >I have/had the same problem, and i'm using the CVS version... > >I just had a closer look and now I know why it behaves this way in my case: >our list is accessible at two hostnames which share the exact same website... >the hostname that is linked at the membership chunk links is different >than the hostname i sometimes use to access the list... so netscape thinks >the cookie is for a different host... > >which makes me wonder why the chunk links on the admin page have a full >http:// link while other links on it don't ? > >maybe you are experiencing the same problem? compare the hostname you >use to access the admin page with the hostname that's being used in the >links to the membership chunks... if they are different, that's the >reason why you have to re-enter the password. You're right. However, that doesn't change the bug. The bug is as follows: When you try to access the admin pages with a hostname that you haven't used before, it asks you to reauthenticate for that hostname. This includes when you manually type in a shorter address for the machine(http://lists/ instead of http://lists.uchicago.edu/) That isn't really a bug. Reauthentication for different hostnames is understandable. However, when you are asking for a specific chunk of the membership list and you have to reauthenticate, it ignores your chunk request and always gives chunk 0. Is there a passing problem in the authentication module? -- Ted Cabeen http://www.pobox.com/~secabeen secabeen@pobox.com Check Website or finger for PGP Public Key secabeen@midway.uchicago.edu "I have taken all knowledge to be my province." -F. Bacon cococabeen@aol.com "Human kind cannot bear very much reality."-T.S.Eliot 73126.626@compuserve.com From ricardo@rixhq.nu Mon Feb 21 21:49:05 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Mon, 21 Feb 2000 22:49:05 +0100 Subject: [Mailman-Developers] Membership management chunking requires two authentications.. In-Reply-To: <200002212126.PAA00266@entropy.uchicago.edu>; from secabeen@pobox.com on Mon, Feb 21, 2000 at 03:26:48PM -0600 References: <20000221221518.A17481@miss-janet.com> <200002212126.PAA00266@entropy.uchicago.edu> Message-ID: <20000221224905.B17481@miss-janet.com> On Mon, Feb 21, 2000 at 03:26:48PM -0600, Ted Cabeen wrote: > different hostnames is understandable. However, when you are asking for a > specific chunk of the membership list and you have to reauthenticate, it > ignores your chunk request and always gives chunk 0. Is there a passing > problem in the authentication module? ah i see... well it's obvious the chunk gets losts, since the reauthenticate page gives you a form and it doesn't include the chunk (which was posted with a GET at first) after posting the entered password... a solution could be to include the chunk var into a hidden field in the (re)authenticate form... Ricardo. -- From secabeen@pobox.com Mon Feb 21 21:52:57 2000 From: secabeen@pobox.com (Ted Cabeen) Date: Mon, 21 Feb 2000 15:52:57 -0600 Subject: [Mailman-Developers] Wishlist items... Message-ID: <200002212152.PAA00571@entropy.uchicago.edu> Three wishlist items, one of which I've coded (but am not sure will work on CVS) and one I haven't. I'd like to submit these to the bug tracking system, but there doesn't seem to be an email submit address. Does anyone know what it is? First: Add an option in mm_cfg.py and Defaults.py to turn archiving on and off. I like to be able to have archiving off for the majority of my mailing lists unless the owner explicitly wants it. (I coded this one up in 5 minutes, but I'm still on 1.1, so I don't know if it will port. I'll send the diffs if anyone wants them) Second: Currently, the senddigests script sends a digest for every list that has digesting on and has any content to send. However, this means that on a medium traffic list, it's possible to send out a digest of one message, if the digest size was reached two messages ago. What I'd like to see is the digester only send out a time-based digest if no digest had been sent in some finite amount of time instead of every time the cron job is run. Third: Currently the digester never increments its volume number. There is no code in Mailman to increase the volume number. It it set to 1 in the InitVars function, and is never changed. Volume incrementing should probably be an option on the digest page. At the least, we should change the volume number yearly. Thanks. -- Ted Cabeen http://www.pobox.com/~secabeen secabeen@pobox.com Check Website or finger for PGP Public Key secabeen@midway.uchicago.edu "I have taken all knowledge to be my province." -F. Bacon cococabeen@aol.com "Human kind cannot bear very much reality."-T.S.Eliot 73126.626@compuserve.com From jeff@popmail.med.nyu.edu Tue Feb 22 17:47:23 2000 From: jeff@popmail.med.nyu.edu (Jeff Berliner) Date: Tue, 22 Feb 2000 12:47:23 -0500 Subject: [Mailman-Developers] small bug in bin/arch Message-ID: <1073583.3160212443@antwerp.med.nyu.edu> Sorry if this been fixed already in the 1.2 tree, but I can't access CVS from here. I searched the archives and didn't see anything. With a stock MM 1.1 install, line 77 of $prefix/bin/arch has an error. If in that block of code, a MMUnknownListError exception is raised, (for instance, if the config.db permissions are incorrect), arch actually dies with a NameError, and obviously doesn't report the correct problem. The solution is to add 'Errors.' to the name of the exception to catch. I guess this was just an oversight. *** ../src/mailman-1.1/bin/arch Sat Aug 21 14:21:44 1999 --- arch Tue Feb 22 10:13:33 2000 *************** *** 74,80 **** listname, mbox = sys.argv[1:] try: mlist = MailList(listname) ! except MMUnknownListError: usage(2, 'no such list: ' + listname) # lay claim to the archive's lock file. this is so no other post can --- 74,80 ---- listname, mbox = sys.argv[1:] try: mlist = MailList(listname) ! except Errors.MMUnknownListError: usage(2, 'no such list: ' + listname) # lay claim to the archive's lock file. this is so no other post can - Jeff -- Jeff Berliner jeff@popmail.med.nyu.edu Academic Computing, Phone: (212) 263-2051 New York University School of Medicine Fax: (212) 263-8542 From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 22 18:38:20 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 22 Feb 2000 13:38:20 -0500 (EST) Subject: [Mailman-Developers] small bug in bin/arch References: <1073583.3160212443@antwerp.med.nyu.edu> Message-ID: <14514.55196.6586.851416@anthem.cnri.reston.va.us> >>>>> "JB" == Jeff Berliner writes: JB> Sorry if this been fixed already in the 1.2 tree, but JB> I can't access CVS from here. I searched the archives and JB> didn't see anything. Remember, you can always access it via the Web. Check out cvs.python.org Anyway, I just applied this patch. Thanks! -Barry From ricardo@rixhq.nu Tue Feb 22 21:01:10 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Tue, 22 Feb 2000 22:01:10 +0100 Subject: [Mailman-Developers] nondelivery report to list Message-ID: <20000222220110.A19994@miss-janet.com> i just saw a non-delivery report (full mailbox) from a postmaster arrive on the list address... that's kinda odd... probably a really broken MTA? I haven't seen this happen before since i'm using Mailman... the MTA seems to be Microsoft Exchange: >Received: from mailincoming5.collegeclub.com ([10.10.5.85]) by mailoutgoing1.collegeclub.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) > id DZ3T7A17; Tue, 22 Feb 2000 02:50:44 -0800 >Received: from mail pickup service by mailincoming5.collegeclub.com with Microsoft SMTPSVC; > Tue, 22 Feb 2000 02:54:28 -0800 Ricardo. -- From gorgo@sztaki.hu Tue Feb 22 23:37:17 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 00:37:17 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway Message-ID: Hello, I've fixed now the MODE reader problems in the debian package of mailman (using the cvs nntplib.py in Mailman/pythonlib)... And now I was wondering... what happens if the newserver does not accept connections when mailman wants to gateway a post from mail to news... is there a queueing mechanism for nntp just like there is for smtp in case the MTA is down ? -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Feb 23 15:43:34 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 23 Feb 2000 10:43:34 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: Message-ID: <14516.38.89336.568349@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> I've fixed now the MODE reader problems in the debian package GM> of mailman (using the cvs nntplib.py in Mailman/pythonlib)... GM> And now I was wondering... what happens if the newserver does GM> not accept connections when mailman wants to gateway a post GM> from mail to news... is there a queueing mechanism for nntp GM> just like there is for smtp in case the MTA is down ? Nope. From gorgo@sztaki.hu Wed Feb 23 15:46:09 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 16:46:09 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <14516.38.89336.568349@anthem.cnri.reston.va.us> Message-ID: On Wed, 23 Feb 2000, Barry A. Warsaw wrote: > GM> I've fixed now the MODE reader problems in the debian package > GM> of mailman (using the cvs nntplib.py in Mailman/pythonlib)... > GM> And now I was wondering... what happens if the newserver does > GM> not accept connections when mailman wants to gateway a post > GM> from mail to news... is there a queueing mechanism for nntp > GM> just like there is for smtp in case the MTA is down ? > > Nope. Ahm... perhaps the queue should be expanded to handle the NNTP case too then ? :) We don't want mails get lost on their way to newsgroups... :) -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us Wed Feb 23 16:15:16 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 23 Feb 2000 11:15:16 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <14516.38.89336.568349@anthem.cnri.reston.va.us> Message-ID: <14516.1940.326856.375284@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> Ahm... perhaps the queue should be expanded to handle the NNTP GM> case too then ? :) We don't want mails get lost on their way GM> to newsgroups... :) Except that Mailman's mail queuing is gone too now in the current CVS snapshot. The 1.1 implementation caused a terrible performance hit so I ripped it out, figuring a well configured MTA should be able to do that job, and do it better. It could be added back, but I haven't found a need. If someone wants to re-implement this against the current architecture, then I agree that nntp queuing should be part of the picture. -Barry From gorgo@sztaki.hu Wed Feb 23 16:20:53 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 17:20:53 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <14516.1940.326856.375284@anthem.cnri.reston.va.us> Message-ID: On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote: > >>>>> "GM" == Gergely Madarasz writes: > > GM> Ahm... perhaps the queue should be expanded to handle the NNTP > GM> case too then ? :) We don't want mails get lost on their way > GM> to newsgroups... :) > > Except that Mailman's mail queuing is gone too now in the current CVS > snapshot. The 1.1 implementation caused a terrible performance hit so > I ripped it out, figuring a well configured MTA should be able to do > that job, and do it better. It could be added back, but I haven't > found a need. If someone wants to re-implement this against the > current architecture, then I agree that nntp queuing should be part of > the picture. Ah, indeed, now mailman calls the MTA directly, and no smtp on port 25, right ? Perhaps there is a way to do something like that with the news server, so no need to talk nntp... with inn rnews seems to be the program to call, I don't know about other nntpservers... -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us Wed Feb 23 18:06:17 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 23 Feb 2000 13:06:17 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <14516.1940.326856.375284@anthem.cnri.reston.va.us> Message-ID: <14516.8601.995835.332168@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> Ah, indeed, now mailman calls the MTA directly, and no smtp on GM> port 25, right ? Not quite. There are actually two interchangable modules which do delivery. Sendmail.py expects sendmail command line interface and uses os.popen() to invoke a subprocess. SMTPDirect.py opens port 25 and blasts the message to the MTA. The latter does no queuing because it is my observation that 1) assuming the SMTP is running correctly, and 2) you are not doing DSN, the SMTP delivery happens asynchronously so still only be notified of errors by the bounce mechanism. 1.1 had all the queuing stuff because with DSN enabled (for those MTAs that support this ESMTP feature), delivery was synchronous and Mailman had to deal with exceptions instead of letting the MTA do it. -Barry From gorgo@sztaki.hu Wed Feb 23 18:09:34 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 19:09:34 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <14516.8601.995835.332168@anthem.cnri.reston.va.us> Message-ID: On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote: > >>>>> "GM" == Gergely Madarasz writes: > > GM> Ah, indeed, now mailman calls the MTA directly, and no smtp on > GM> port 25, right ? > > Not quite. There are actually two interchangable modules which do > delivery. Sendmail.py expects sendmail command line interface and > uses os.popen() to invoke a subprocess. SMTPDirect.py opens port 25 > and blasts the message to the MTA. The latter does no queuing because > it is my observation that 1) assuming the SMTP is running correctly, > and 2) you are not doing DSN, the SMTP delivery happens asynchronously > so still only be notified of errors by the bounce mechanism. 1.1 had > all the queuing stuff because with DSN enabled (for those MTAs that > support this ESMTP feature), delivery was synchronous and Mailman had > to deal with exceptions instead of letting the MTA do it. But it happened quite often that mailman couldn't connect to port 25 because sendmail wasn't accepting connections due to high load (loads above 25 are usual on this machine...), so queueing was needed anyway... -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From bwarsaw@cnri.reston.va.us Wed Feb 23 18:15:35 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 23 Feb 2000 13:15:35 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <14516.8601.995835.332168@anthem.cnri.reston.va.us> Message-ID: <14516.9159.596407.361870@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> But it happened quite often that mailman couldn't connect to GM> port 25 because sendmail wasn't accepting connections due to GM> high load (loads above 25 are usual on this machine...), so GM> queueing was needed anyway... One possible solution is to use a better MTA (i.e. not sendmail). Also, it's very possible that all that extra load was /caused/ by Mailman 1.1's queuing system. Some form of queuing may still be necessary, but the previous implementation did not work. I don't have time to implement a new system myself but if someone wants to work on it, let me know. -Barry From claw@kanga.nu Wed Feb 23 18:20:38 2000 From: claw@kanga.nu (J C Lawrence) Date: Wed, 23 Feb 2000 10:20:38 -0800 Subject: [Mailman-Developers] mail/news gateway In-Reply-To: Message from bwarsaw@cnri.reston.va.us of "Wed, 23 Feb 2000 13:15:35 EST." <14516.9159.596407.361870@anthem.cnri.reston.va.us> References: <14516.8601.995835.332168@anthem.cnri.reston.va.us> <14516.9159.596407.361870@anthem.cnri.reston.va.us> Message-ID: <10862.951330038@kanga.nu> On Wed, 23 Feb 2000 13:15:35 -0500 (EST) bwarsaw wrote: >>>>>> "GM" == Gergely Madarasz writes: GM> But it happened quite often that mailman couldn't connect to GM> port 25 because sendmail wasn't accepting connections due to GM> high load (loads above 25 are usual on this machine...), so GM> queueing was needed anyway... > One possible solution is to use a better MTA (i.e. not sendmail). > Also, it's very possible that all that extra load was /caused/ by > Mailman 1.1's queuing system. I potentially have the same problem -- the system occassionally gets very high load spikes, NOT due to MailMan, and Exim is configured to refuse connections when load gets too high. > Some form of queuing may still be necessary... If the new system also allows mail to be delivered to the local MTA by handingit off to a sendmail executable, there should be little to no problem. I forget the new design tho. -- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From gorgo@sztaki.hu Wed Feb 23 18:46:26 2000 From: gorgo@sztaki.hu (Gergely Madarasz) Date: Wed, 23 Feb 2000 19:46:26 +0100 (MET) Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <14516.9159.596407.361870@anthem.cnri.reston.va.us> Message-ID: On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote: > >>>>> "GM" == Gergely Madarasz writes: > > GM> But it happened quite often that mailman couldn't connect to > GM> port 25 because sendmail wasn't accepting connections due to > GM> high load (loads above 25 are usual on this machine...), so > GM> queueing was needed anyway... > > One possible solution is to use a better MTA (i.e. not sendmail). Sorry, but I can't accept this answer :) It seems quite normal for _any_ MTA to refuse connections when the load is high (protect the machine from even higher loads), and it is quite normal for machines to have high load... you don't want to say "don't use mailman on machines wich have load above 5", do you ? :) I use it, and it works nicely :) > Also, it's very possible that all that extra load was /caused/ by > Mailman 1.1's queuing system. Well... at least it took part of it. I'm really looking forward to the 1.2 release to test the performance :) > Some form of queuing may still be necessary, but the previous > implementation did not work. I don't have time to implement a new > system myself but if someone wants to work on it, let me know. I don't really see the need of a new smtp queueing system if there is the possibility of executing the MTA directly, so I'm not asking anyone to do it now :) I'm just saying that queueing was _needed_ before, and it _worked_ (I was one of those who requested this feature when 1.0b4 lost almost all my digests, because of the high load early morning :)) So there is no problem now :) And at the beginning of this thread I just warned that the same might be needed for the nntp server too, I didn't want to go into the smtp stuff now :) -- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/ From Nigel.Metheringham@VData.co.uk Thu Feb 24 09:41:28 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Thu, 24 Feb 2000 09:41:28 +0000 Subject: [Mailman-Developers] mail/news gateway In-Reply-To: Message from J C Lawrence of "Wed, 23 Feb 2000 10:20:38 PST." <10862.951330038@kanga.nu> Message-ID: claw@kanga.nu said: > I potentially have the same problem -- the system occassionally gets > very high load spikes, NOT due to MailMan, and Exim is configured to > refuse connections when load gets too high. A solution to this is to add your mailman host (which would typically be 127.0.0.1) into smtp_reserve_hosts and then set the related parameters such as smtp_accept_reserve, smtp_load_reserve and the various *_load* options so that incoming SMTP is always accepted from mailman (although the messages may be queued rather than immediately delivered). The upside of this is that it puts less load on the system than additional invocations of exim from the command line, and all your load policy is pretty much in one place. Whichever method you use there may still be errors caused due to (for example) running out of spool space. [I'll try and work out a reasonable note on this for the mailman/exim howto information - alternatively I would just *love* it if someone would come up with some text for me :-) ] http://www.exim.org/howto/mailman.html Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From bwarsaw@cnri.reston.va.us Thu Feb 24 18:51:11 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Thu, 24 Feb 2000 13:51:11 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <10862.951330038@kanga.nu> Message-ID: <14517.32159.955431.115890@anthem.cnri.reston.va.us> >>>>> "NM" == Nigel Metheringham writes: NM> A solution to this is to add your mailman host (which would NM> typically be 127.0.0.1) into smtp_reserve_hosts and then set NM> the related parameters such as smtp_accept_reserve, NM> smtp_load_reserve and the various *_load* options so that NM> incoming SMTP is always accepted from mailman (although the NM> messages may be queued rather than immediately delivered). That would be fine. Anybody know of similar parameters for Postfix or other MTAs? It makes sense that MTAs should be able to treat localhost connections differently than remote connections. NM> The upside of this is that it puts less load on the system NM> than additional invocations of exim from the command line, and NM> all your load policy is pretty much in one place. Whichever NM> method you use there may still be errors caused due to (for NM> example) running out of spool space. And Mailman could have a bug, or Python could throw MemoryErrors, etc. Anyway, if Mailman can't successfully do the SMTP transaction, and it gets exceptions it doesn't know how to deal with, the exception should at least get logged. I know that doesn't make you feel great, and the situation should be improved. -Barry From ricardo@rixhq.nu Thu Feb 24 19:39:46 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Thu, 24 Feb 2000 20:39:46 +0100 Subject: [Mailman-Developers] mail/news gateway In-Reply-To: <10862.951330038@kanga.nu>; from claw@kanga.nu on Wed, Feb 23, 2000 at 10:20:38AM -0800 References: <14516.8601.995835.332168@anthem.cnri.reston.va.us> <14516.9159.596407.361870@anthem.cnri.reston.va.us> <10862.951330038@kanga.nu> Message-ID: <20000224203946.A24852@miss-janet.com> On Wed, Feb 23, 2000 at 10:20:38AM -0800, J C Lawrence wrote: > On Wed, 23 Feb 2000 13:15:35 -0500 (EST) > bwarsaw wrote: > >>>>>> "GM" == Gergely Madarasz writes: > GM> But it happened quite often that mailman couldn't connect to > GM> port 25 because sendmail wasn't accepting connections due to > GM> high load (loads above 25 are usual on this machine...), so > GM> queueing was needed anyway... > > One possible solution is to use a better MTA (i.e. not sendmail). > > Also, it's very possible that all that extra load was /caused/ by > > Mailman 1.1's queuing system. > I potentially have the same problem -- the system occassionally gets > very high load spikes, NOT due to MailMan, and Exim is configured to > refuse connections when load gets too high. I got loads of up to 60 when i used exim with Mailman 1.1 ... and then slowly the system started to die ... this was happening when sending about 20 approved posts to 1500+ users. I configured exim not to send out anything when getting a higher load than 5, but then my fcgi scripts would die anyway during posting approvals... Since i'm using postfix with MM 1.2, loads don't get any higher than 2 or 3. So i'm very happy with the changes in 1.2 :) Ricardo. -- From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 24 20:16:10 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 24 Feb 2000 15:16:10 -0500 (EST) Subject: [Mailman-Developers] mail/news gateway References: <14516.8601.995835.332168@anthem.cnri.reston.va.us> <14516.9159.596407.361870@anthem.cnri.reston.va.us> <10862.951330038@kanga.nu> <20000224203946.A24852@miss-janet.com> Message-ID: <14517.37258.663839.652840@anthem.cnri.reston.va.us> >>>>> "RK" == Ricardo Kustner writes: RK> Since i'm using postfix with MM 1.2, loads don't get any RK> higher than 2 or 3. So i'm very happy with the changes in 1.2 RK> :) This is very good to hear! :) Who else is using any of the 1.2 CVS snapshots? -Barry From ricardo@rixhq.nu Thu Feb 24 21:14:14 2000 From: ricardo@rixhq.nu (Ricardo Kustner) Date: Thu, 24 Feb 2000 22:14:14 +0100 Subject: [Mailman-Developers] mailman development Message-ID: <20000224221414.C24852@miss-janet.com> Hi, I've been following mailman development since somewhere around april/may 1999. I was just reading back the archives and I remember that several times interesting discussions started on how it could be made easier for people to contribute to mailman development (cvs branches, bitkeeper and other things), but unfortunately there haven't been made much decissions about this so far (AFAIK)... I know the core developers are busy with other projects too, but the point of this email is that i'm hoping to bring this discussion back to life and that it results in some changes that will speed up development ... Also please have a look at this message Barry posted last year, and the thread that follows it... http://www.python.org/pipermail/mailman-developers/1999-October/001415.html Ricardo. -- From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 24 22:03:25 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 24 Feb 2000 17:03:25 -0500 (EST) Subject: [Mailman-Developers] mailman development References: <20000224221414.C24852@miss-janet.com> Message-ID: <14517.43693.320214.161856@anthem.cnri.reston.va.us> I'm still as busy as ever (maybe more so in fact!). I don't think BitKeeper is going to be the answer, but maybe moving Mailman to SourceForge will help. It's something I'm going to actively pursue after 1.2 is released. -Barry From jae@ilk.de Thu Feb 24 20:26:02 2000 From: jae@ilk.de (Juergen A. Erhard) Date: Thu, 24 Feb 2000 21:26:02 +0100 Subject: [Mailman-Developers] mailman, aliases and exim Message-ID: <24022000.5@sanctum.jae.ddns.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 There's a recipe for exim's configuration that makes it unnecessary to add aliases to /etc/aliases for every mailing list. list_transport: driver =3D pipe command =3D "/var/spool/smartlist/.bin/flist \ ${local_part}${local_part_suffix}" current_directory =3D /var/spool/smartlist home_directory =3D /var/spool/smartlist user =3D smartlist group =3D list Would/is something like this possible for mailman? This std aliases fragment would hint at needing to replace wrapper with a smart-wrapper or such that groks the list addresses (test, test-admin, etc) and calls wrapper as appropriate. ## test mailing list ## created: 27-Jan-2000 root test: "|/var/lib/mailman/mail/wrapper post test" test-admin: "|/var/lib/mailman/mail/wrapper mailowner test" test-request: "|/var/lib/mailman/mail/wrapper mailcmd test" owner-test: test-admin test-owner: test-admin Would this work? Comments? Bye, J PS: Need to get the Mailman source... as that wrapper ain't Python ;-) - -- = J=FCrgen A. Erhard eMail: jae@ilk.de phone: (GERMANY) 0721 27326 MARS: http://JuergenErhard.tripod.com/mars_index.html The GNU Project (http://www.gnu.org) Win32 has many known work arounds. For example, Linux. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.1 (GNU/Linux) Comment: Use Mailcrypt and GnuPG iEYEARECAAYFAji1k9kACgkQN0B+CS56qs2i1QCfdKXgC/3bvk9VCyDFWThx2Xus iWsAoIz+jUdl/12GImZTjTzlpKpeK4ha =3DJdHL -----END PGP SIGNATURE----- From jwt@dskk.co.jp Fri Feb 25 01:34:34 2000 From: jwt@dskk.co.jp (Jim Tittsler) Date: Fri, 25 Feb 2000 10:34:34 +0900 Subject: [Mailman-Developers] mailman, aliases and exim In-Reply-To: <24022000.5@sanctum.jae.ddns.org>; from jae@ilk.de on Thu, Feb 24, 2000 at 09:26:02PM +0100 References: <24022000.5@sanctum.jae.ddns.org> Message-ID: <20000225103434.A24396@mail.dskk.co.jp> On Thu, Feb 24, 2000 at 09:26:02PM +0100, Juergen A. Erhard wrote: > There's a recipe for exim's configuration that makes it unnecessary to > add aliases to /etc/aliases for every mailing list. > > list_transport: > driver = pipe > command = "/var/spool/smartlist/.bin/flist \ > ${local_part}${local_part_suffix}" [...] > Would/is something like this possible for mailman? This std aliases Yes, a similar scheme for mailman is described in the "HOWTO - Using exim and mailman together" that Nigel Metheringham put together: http://www.exim.org/howto/mailman.html From scottrus@raleigh.ibm.com Fri Feb 25 06:14:49 2000 From: scottrus@raleigh.ibm.com (Scott Russell) Date: Fri, 25 Feb 2000 01:14:49 -0500 Subject: [Mailman-Developers] sendmail exit 2 error on lists Message-ID: <20000225011449.F19733@bzimage.raleigh.ibm.com> All - I did make that migration from majordomo to Mailman 1.2 beta 1 (checked out of cvs on 02/23/00) and only ran into one issue. One of my users had the following email address while using majordomo: "Tim O'Flynn/DUB/xxxx" I used the bin/add_members script to import the list. This had no problems. The resulting email address in Mailman was: tim_o'flynn@xxxx.com The problem was sending mail to this list would result in the follwing error: post(32390): File "/home/mailman/scripts/post", line 86, in ? post(32390): main() post(32390): File "/home/mailman/scripts/post", line 62, in main post(32390): mlist.Post(msg) post(32390): File "/home/mailman/Mailman/MailList.py", line 1260, in Post post(32390): Mailman.Handlers.HandlerAPI.DeliverToList(self, msg) post(32390): File "/home/mailman/Mailman/Handlers/HandlerAPI.py", line 55, in DeliverToList post(32390): func(mlist, msg) post(32390): File "/home/mailman/Mailman/Handlers/Sendmail.py", line 88, in process post(32390): raise SendmailHandlerError(errcode) post(32390): Mailman.Handlers.Sendmail . SendmailHandlerError : 2 Also, a MAILER-DEAMON error was sent to the sender whenever they posted to the list. The error showed the above trace back and that a shell error had occured while processing the mail. Any thoughts on how to solve this? It should be easy to replicate. If someone writes a patch and it get's commited to cvs I'll update my copy and confirm the patch works. BTW, once again, I know I've said it before, THANK YOU FOR AN OUTSTANDING PRODUCT. Oh, I'm using postfix as my mail agaent, btw. Not sure if that mattered in the whole thing. -- Regards, Scott Russell (scottrus@raleigh.ibm.com) Linux Technology Center, System Admin Red Hat Certified Engineer From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Sat Feb 26 21:26:25 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Sat, 26 Feb 2000 16:26:25 -0500 (EST) Subject: [Mailman-Developers] sendmail exit 2 error on lists References: <20000225011449.F19733@bzimage.raleigh.ibm.com> Message-ID: <14520.17665.977202.913499@anthem.cnri.reston.va.us> >>>>> "SR" == Scott Russell writes: SR> I did make that migration from majordomo to Mailman 1.2 beta 1 SR> (checked out of cvs on 02/23/00) and only ran into one SR> issue. One of my users had the following email address while SR> using majordomo: SR> "Tim O'Flynn/DUB/xxxx" SR> I used the bin/add_members script to import the list. This had SR> no problems. The resulting email address in Mailman was: SR> tim_o'flynn@xxxx.com SR> The problem was sending mail to this list would result in the SR> follwing error: This is probably due to a shell escape problem with that single quote. The Sendmail.py delivery module uses os.popen() to invoke an external sendmail process, which goes through the shell. I'd suggest trying to switch to the SMTPDirect delivery module, which should avoid this problem. Add this line to your Mailman/mm_cfg.py file: DELIVERY_MODULE = 'SMTPDirect' You may need to change SMTPHOST and SMTPPORT; see Mailman/Defaults.py.in for details. I'm going to make SMTPDirect the default for the 1.2 release. -Barry