From barry@wooz.org Wed Nov 1 00:43:13 2000 From: barry@wooz.org (barry@wooz.org) Date: Tue, 31 Oct 2000 19:43:13 -0500 (EST) Subject: [Mailman-Developers] My admins speak out... References: <200010312311.PAA19920@utopia.west.sun.com> Message-ID: <14847.26401.287186.613646@anthem.concentric.net> >>>>> "DM" == Dan Mick writes: >> if getattr(mlist, 'auto_reject', 0): raise DiscardMessage DM> Very nice; had no idea that one little exception is all it DM> takes. That only works for pipeline Handlers modules, because they're wrapped in the try/except inside do_pipeline(). DM> Is there anything similarly-simple to generate a generic DM> bounce message and discard, or is that all "call some reply DM> method and then raise DiscardMessage"? Correct. But generating an outgoing message is fairly easy. See the Message.UserNotification class and HandlerAPI.DeliverToUser() (for response to a single user). -Barry From barry@wooz.org Wed Nov 1 01:16:36 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Tue, 31 Oct 2000 20:16:36 -0500 (EST) Subject: [Mailman-Developers] [Mailman-Users] forwarded message from Guido van Rossum (fwd) References: <20001030192005.F12812@xs4all.nl> Message-ID: <14847.28404.962845.410706@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> Anyone else notice something weird about the mail below ? It TW> was sent by Guido to python-dev, where Barry picked it up from TW> and sent it to both mailman-developers and mailman-users. I am TW> only subscribed to mailman-developers, and I got it TW> twice. Are you on mailman-announce? I sent it there too. TW> What's more, the second mail seems to have gone to TW> mailman-developers *through* mailman-users. Notice the extra TW> list-sig at the obttom of the mail, and extra [listname] in TW> the subject. TW> Bug in Mailman ? ;-P Hrm. My simple test posts to multiple lists don't seem to have a problem. -Barry From barry@wooz.org Wed Nov 1 03:41:30 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Tue, 31 Oct 2000 22:41:30 -0500 (EST) Subject: [Mailman-Developers] Get listed. :) Message-ID: <14847.37098.537126.455758@anthem.concentric.net> I'm putting together the on-line documentation for the 2.0 release and I'd like to include a list of sites that run Mailman. I've got Python.Org and lists.apple.com (if that's cool with you Chuq ;). Please send any others you'd like to include to me directly. Thanks, -Barry From barry@wooz.org Wed Nov 1 03:41:30 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Tue, 31 Oct 2000 22:41:30 -0500 (EST) Subject: [Mailman-Developers] [Mailman-Users] Get listed. :) Message-ID: <14847.37098.537126.455758@anthem.concentric.net> I'm putting together the on-line documentation for the 2.0 release and I'd like to include a list of sites that run Mailman. I've got Python.Org and lists.apple.com (if that's cool with you Chuq ;). Please send any others you'd like to include to me directly. Thanks, -Barry ------------------------------------------------------ Mailman-Users maillist - Mailman-Users@python.org http://www.python.org/mailman/listinfo/mailman-users From Johnny Fuerst; Governor" ; from barry@wooz.org on Tue, Oct 31, 2000 at 10:41:30PM -0500 References: <14847.37098.537126.455758@anthem.concentric.net> Message-ID: <20001031220220.A1038@cathat.net> Hey Barry, Can I list my site, if right now, I am only running 1.1? Sincerely, Johnny On Tue, Oct 31, 2000 at 10:41:30PM -0500, Barry A. Warsaw wrote: > > I'm putting together the on-line documentation for the 2.0 release and > I'd like to include a list of sites that run Mailman. I've got > Python.Org and lists.apple.com (if that's cool with you Chuq ;). > Please send any others you'd like to include to me directly. > > Thanks, > -Barry > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users -- On Behalf Of: Johnny Fuerst, Governor ................ governor@cathat.net Opinions stated in this message are mine and mine alone, and do not necessarily represent those of my ISP/employer! .period. ------------------------------------------------------------ 'I Desire Compassion, and not a Sacrifice.' 'La Vida E Bella' ____________________________ Random Quote: If you eat a live frog in the morning, nothing worse will happen to either of you for the rest of the day. johnny@cathat.net From chuqui@plaidworks.com Wed Nov 1 04:21:14 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 31 Oct 2000 20:21:14 -0800 Subject: [Mailman-Developers] Get listed. :) In-Reply-To: <14847.37098.537126.455758@anthem.concentric.net> References: <14847.37098.537126.455758@anthem.concentric.net> Message-ID: At 10:41 PM -0500 10/31/00, Barry A. Warsaw wrote: >I've got >Python.Org and lists.apple.com (if that's cool with you Chuq ;). >Please send any others you'd like to include to me directly. It's cool with Apple. Also, fi you would, www.hockeyfanz.com, the Other Me. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From chuqui@plaidworks.com Wed Nov 1 04:21:14 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 31 Oct 2000 20:21:14 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Get listed. :) In-Reply-To: <14847.37098.537126.455758@anthem.concentric.net> References: <14847.37098.537126.455758@anthem.concentric.net> Message-ID: At 10:41 PM -0500 10/31/00, Barry A. Warsaw wrote: >I've got >Python.Org and lists.apple.com (if that's cool with you Chuq ;). >Please send any others you'd like to include to me directly. It's cool with Apple. Also, fi you would, www.hockeyfanz.com, the Other Me. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. ------------------------------------------------------ Mailman-Users maillist - Mailman-Users@python.org http://www.python.org/mailman/listinfo/mailman-users From chuqui@plaidworks.com Wed Nov 1 04:31:24 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 31 Oct 2000 20:31:24 -0800 Subject: [Mailman-Developers] mailman-developers problem. Message-ID: Barry, we have a problem with mailman-developers. I responded to your mail, and my response went to -users and -developers, and I got THREE copies. I got one from -users. I got one from -developers. and then I got a third copy of the -users mail, resent to -developers. looks like someone on the list is doing something stupid: To: catholic.org@catholicnet.org.uk Received: from dinsdale.python.org (dinsdale.cnri.reston.va.us [132.151.1.21]) by webmail6.catholic.org (8.10.1/8.10.1) with SMTP id eA14OAQ18572 for ; Wed, 1 Nov 2000 04:24:11 GMT Received: from dinsdale.python.org (localhost [127.0.0.1]) by dinsdale.python.org (Postfix) with ESMTP id 0C2981CE17; Tue, 31 Oct 2000 23:24:11 -0500 (EST) the third copy is routing through that address. Can you please shoot this person? the catholic.org address seems to be hosed. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From barry@wooz.org Wed Nov 1 04:39:58 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Tue, 31 Oct 2000 23:39:58 -0500 (EST) Subject: [Mailman-Developers] mailman-developers problem. References: Message-ID: <14847.40606.557263.702033@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> Barry, we have a problem with mailman-developers. I responded CVR> to your mail, and my response went to -users and -developers, CVR> and I got THREE copies. CVR> I got one from -users. I got one from -developers. and then I CVR> got a third copy of the -users mail, resent to -developers. CVR> looks like someone on the list is doing something stupid: | To: catholic.org@catholicnet.org.uk Received: from | dinsdale.python.org (dinsdale.cnri.reston.va.us | [132.151.1.21]) | by webmail6.catholic.org (8.10.1/8.10.1) with SMTP id | eA14OAQ18572 for ; Wed, 1 Nov 2000 04:24:11 | GMT | Received: from dinsdale.python.org (localhost [127.0.0.1]) | by dinsdale.python.org (Postfix) with ESMTP | id 0C2981CE17; Tue, 31 Oct 2000 23:24:11 -0500 (EST) CVR> the third copy is routing through that address. Can you CVR> please shoot this person? the catholic.org address seems to CVR> be hosed. Thomas noticed this earlier and thought it might be a Mailman bug. I couldn't reproduce it ('natch :), and I deleted the messages so couldn't look at them closely. I got a dup on my last posting though. Now why would he do that? :/ Anyway, I've disabled that account. Thanks, -Barry From chuqui@plaidworks.com Wed Nov 1 05:56:40 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 31 Oct 2000 21:56:40 -0800 Subject: [Mailman-Developers] mailman-developers problem. In-Reply-To: <14847.40606.557263.702033@anthem.concentric.net> References: <14847.40606.557263.702033@anthem.concentric.net> Message-ID: At 11:39 PM -0500 10/31/00, Barry A. Warsaw wrote: > >Now why would he do that? :/ Anyway, I've disabled that account. most commonly, it's an attempt at a vacation bot that failed miserably. less commonly, it's simply a broken mailer. And once in a while, I've seen someone do that because they think it'll make us unsubscribe them -- now why they don't simply unsubscribe, I dunno, but people are illogical, unlike computers. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From Nigel.Metheringham@VData.co.uk Wed Nov 1 10:10:47 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Wed, 01 Nov 2000 10:10:47 +0000 Subject: [Mailman-Developers] Slight oddity on ubsubscription Message-ID: Here's a message I got from one of my subscribers - makes a reasonable point [I have edited the address in there to prevent it being exposed]. Nigel. So, I see that an old form of my address is subscribed to the announce list. I look for a way to change the address, don't see it, so unsubscribe. On the unsubscription results page, I see: -----------------------------< cut here >------------------------------- Exim-announce Unsubscribe Results You have been unsubscribed. Continue to edit your personal options. -----------------------------< cut here >------------------------------- The last four words there are a link; following that brings me to a page stating: -----------------------------< cut here >------------------------------- Error exim-announce: No such member 'removed@to.protect.guilty'. -----------------------------< cut here >------------------------------- Perhaps someone who's unsubscribed shouldn't be prompted to continue editting? Ah well -- I followed it because I was worried that cruft might have been left behind. I've at least had some reassurance about that. :^) -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From roger@infomed.sld.cu Wed Nov 1 19:32:10 2000 From: roger@infomed.sld.cu (Roger Pena) Date: Wed, 1 Nov 2000 14:32:10 -0500 (CST) Subject: [Mailman-Developers] Get listed. :) In-Reply-To: <14847.37098.537126.455758@anthem.concentric.net> Message-ID: On Tue, 31 Oct 2000, Barry A. Warsaw wrote: > > I'm putting together the on-line documentation for the 2.0 release and > I'd like to include a list of sites that run Mailman. I've got > Python.Org and lists.apple.com (if that's cool with you Chuq ;). > Please send any others you'd like to include to me directly. if you like you can include, redhat inc samba surceforge linux.cu :-) i sure that other important projects are using mailman as the mail list server thanks roger > > Thanks, > -Barry > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers > From barry@wooz.org Thu Nov 2 23:08:51 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Thu, 2 Nov 2000 18:08:51 -0500 (EST) Subject: [Mailman-Developers] Fix (I think) for duplicates problem Message-ID: <14849.62467.911275.369342@anthem.concentric.net> The last followup to bug #117015 by deejster provided what I think is the critical clue to the strange duplication problem. Given that recipe I was indeed able to reproduce the bug: - a message gets held for whatever reason - the message is approved, but there are smtp errors during delivery - the message gets re-queued, but with a file name composed of the old filebase, not the new filebase. - fwap! you've now got a duplicate of the original message Here's a fix for this problem. It removes the filebase value from the original msgdata, so that this will be recalculated when the file is re-queued after approval. For those of you seeing duplicates, please apply this patch and let me know if it fixes your dup problem. This is important enough to spin a second release candidate. -Barry Index: ListAdmin.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/ListAdmin.py,v retrieving revision 1.46 diff -u -r1.46 ListAdmin.py --- ListAdmin.py 2000/10/10 06:33:31 1.46 +++ ListAdmin.py 2000/11/02 23:05:57 @@ -208,6 +208,12 @@ return LOST msg = Message.Message(fp) msgdata['approved'] = 1 + # Calculate a new filebase for the approved message, otherwise + # delivery errors will cause duplicates. + try: + del msgdata['filebase'] + except KeyError: + pass # Queue the file for delivery by qrunner. Trying to deliver the # message directly here can lead to a huge delay in web # turnaround. From NeilK@ActiveState.com Thu Nov 2 23:48:08 2000 From: NeilK@ActiveState.com (Neil K) Date: Thu, 2 Nov 2000 15:48:08 -0800 Subject: [Mailman-Developers] RE: [Mailman-Users] Fix (I think) for duplicates problem In-Reply-To: <14849.62467.911275.369342@anthem.concentric.net> Message-ID: Thanks Barry. We figured out our problem, which I think is different from what you are talking about. Details: - our aborted upgrade to mailman 2.0rc1 left behind a few files. - somehow these files interfered with normal exception handling. - when mailman decides a message must be held for moderation, it sends the user a "held for moderation" message and raises an special exception. - but this exception was no longer handled appropriately, so it got passed all the way up to qmail. - qmail therefore thought the delivery was a failure, and would schedule another attempt. And so on. Deleting the leftover files from 2.0rc1 seems to have fixed it. This was certainly a very bizarre interaction... I'm not sure if a patch to mailman is appropriate. Perhaps the action of sending the 'held for moderation', or similar, email, should be deferred as late as possible. That way if there are problems, at least it won't be sent out an infinite number of times. Or alternatively, close the connection to the MTA sooner. -- Neil Kandalgaonkar Web Application Developer, ActiveState From gietl@gietl.com Fri Nov 3 14:07:44 2000 From: gietl@gietl.com (Andreas Gietl) Date: Fri, 03 Nov 2000 15:07:44 +0100 Subject: [Mailman-Developers] forever pending subscriptions Message-ID: <3A02C6B0.2D82FC79@gietl.com> Hi. I'm using mailman on a lot of different machines with the Exim MTA in version 3.16 and everything worked fine with beta2. In the last weeks i tried a few times to upgrade to beta6 or rc1. But this always ends up in a very strange behavior. Since i am not the only one who reported this to the mailman-users-list and got no answer i also send this mail to the developers list, because i think it may be a bug. The Following happens: After upgrading to rc1 without any errors from make install mailman still works fine, but it does no subscriptions any longer. A user who signs up in the listinfo page gets his confirmation mail and after he/she confirmed the message (and the MTA delivered it w/o error) nothing happens. The subscribe log still shows the user as pending and nothing happens. Any suggestion would be appreciated. thx andreas -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From barry@wooz.org Fri Nov 3 18:19:41 2000 From: barry@wooz.org (barry@wooz.org) Date: Fri, 3 Nov 2000 13:19:41 -0500 (EST) Subject: [Mailman-Developers] RE: [Mailman-Users] Fix (I think) for duplicates problem References: <14849.62467.911275.369342@anthem.concentric.net> Message-ID: <14851.445.927341.896264@anthem.concentric.net> >>>>> "NK" == Neil K writes: NK> We figured out our problem, which I think is different from NK> what you are talking about. Oh, whoops! I missed this in my inbox. Looks like you've largely solved the problem, so ignore my last message. NK> - our aborted upgrade to mailman 2.0rc1 left behind a few NK> files. I'd still /really/ like to know what problems you all had with rc1. It's a release candidate, so if there are bugs, better to catch them now than before 2.0 final goes out! NK> This was certainly a very bizarre interaction... I'm not sure NK> if a patch to mailman is appropriate. Probably not. Rolling back is a bad idea (and definitely "unsupported")! -Barry From bernhard@intevation.de Fri Nov 3 18:43:03 2000 From: bernhard@intevation.de (Bernhard Reiter) Date: Fri, 3 Nov 2000 19:43:03 +0100 Subject: [Mailman-Developers] Direct Python access tutorial wanted Message-ID: <20001103194303.H25116@cheops.usf.Uni-Osnabrueck.DE> --df+09Je9rNq3P+GE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Mailman developers, just noticed that the minimal instructions for list server administrators to mass manipulate mailinglists directly have been vanished from the mailman README file in the jump to version 2. (last cvs revision containg the advice is README rev1.45) As I considered it quite useful my suggestion is to add it again to the mailman documentation. Or is it not possible or advisable to do=20 this kind of list surgery anymore? =09 Regards, Bernhard ps: please cc relevant replies to me as I am not subscribe to the list --=20 Professional Service around Free Software (intevation.net) = =20 The FreeGIS Project (freegis.org) Association for a Free Informational Infrastructure (ffii.org) --df+09Je9rNq3P+GE Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.3 (SunOS) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjoDBzYACgkQh9ag3dpKERbKiACfV8WyHlNZ2MGLD/DxNQfijt24 GxgAnineH6Fqo6IUUzILNuuG8z7PUVZ9 =xvuO -----END PGP SIGNATURE----- --df+09Je9rNq3P+GE-- From Dan.Mick@west.sun.com Fri Nov 3 20:35:37 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Fri, 03 Nov 2000 12:35:37 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> Message-ID: <3A032199.67D1CA2E@west.sun.com> I just tested this again yesterday, and it worked for me. Andreas Gietl wrote: > > Hi. > > I'm using mailman on a lot of different machines with the Exim MTA in > version 3.16 and everything worked fine with beta2. > > In the last weeks i tried a few times to upgrade to beta6 or rc1. > But this always ends up in a very strange behavior. Since i am not the > only one who reported this to the mailman-users-list and got no answer i > also send this mail to the developers list, because i think it may be a > bug. > > The Following happens: > > After upgrading to rc1 without any errors from make install mailman > still works fine, but it does no subscriptions any longer. > A user who signs up in the listinfo page gets his confirmation mail and > after he/she confirmed the message (and the MTA delivered it w/o error) > nothing happens. The subscribe log still shows the user as pending and > nothing happens. > > Any suggestion would be appreciated. > > thx > > andreas > -- > andreas gietl > gietl internet services > fon +49 9402 2551 > fax +49 9402 2604 > mobile +49 171 60 70 008 > gietl@gietl.com > > ############################################ > # Das Handbuch sagt, das Programm benötige # > # Windows 95 oder besser. Also habe ich # > # Linux installiert! # > ############################################ > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users From barry@wooz.org Fri Nov 3 21:15:50 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Fri, 3 Nov 2000 16:15:50 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> Message-ID: <14851.11014.397538.19202@anthem.concentric.net> >>>>> "AG" == Andreas Gietl writes: AG> After upgrading to rc1 without any errors from make install AG> mailman still works fine, but it does no subscriptions any AG> longer. A user who signs up in the listinfo page gets his AG> confirmation mail and after he/she confirmed the message (and AG> the MTA delivered it w/o error) nothing happens. The subscribe AG> log still shows the user as pending and nothing happens. >>>>> "DM" == Dan Mick writes: DM> I just tested this again yesterday, and it worked for me. Me too. When you do a "bin/dumpdb data/pending_subscriptions.db" do you still see the pending subscription in there? Do you still see the confirmation message in qfiles? If so, are there any obvious errors in logs/error? Does it have the expected Subject: header (i.e. at the end of the line, "request xxxxxx" where those x's are the numbers). -Barry From mtran@bmedesign.com Fri Nov 3 23:31:17 2000 From: mtran@bmedesign.com (mike tran) Date: Fri, 03 Nov 2000 16:31:17 -0700 Subject: [Mailman-Developers] mailman installation error Message-ID: <3A034AC5.4191FD96@bmedesign.com> After I did ./configure --with-cgi-gid=nobody --with-mail-gid=mailman and make install. Then I ran check_perms. It keep telling me that it cannot find the module "paths" in line 19 of check_perms. Any suggestions is appricated. I am using RH 6.2 and the latest Apache with python 1.5.2 From claw@kanga.nu Sat Nov 4 02:13:32 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 03 Nov 2000 18:13:32 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail Message-ID: <7154.973304012@kanga.nu> <> On Fri, 3 Nov 2000 17:01:01 -0800 Marc MERLIN wrote: > On Fri, Nov 03, 2000 at 04:56:33PM -0800, J C Lawrence wrote: >> Yup, that's what I mean by pass-thru. > Ok, I wasn't sure. No worries. >> of having Exim do that a loooong time back but I don't know if >> anything ever happened there (there are obvious timeout and >> performance issues when delivering to slow/unresponsive sites >> (think about behavioural changes mid-negotiation)). > I see. So the message gets spooled as a backup and erased right > afterwards if the current process was able to deliver it I > suppose. Quite. Note that this leaves with the same disk IO that not doing pass-thru does. The only time pass-thru gains you anything is when you can hand the message off very close to as fast as, or faster than you can receive it. In many cases I could see this being a gain given free queue runners and minimal queue runner negotiation overhead or complexity. >> To deal with the physical IO problems, the very large volume mail >> server companies I've dealt with and talked to (eg my last >> client, Critical Path) use solid state disks for their mail >> queues (in CP's case under a modified QMail, soon to be replaced >> by NPLex). Its a bloody expensive solution to be sure, but its >> also a fast one. > It can come down to that, but that's throwing money at hardware > instead of fixing mailman's deficiencies. True. Mailman is the real bottleneck here. A simple, and computationally and code-wise cheap solution would be to split the queues per list. Simply have each list have a private outbound queue area (this may be done already, I haven't looked at v2 in detail and am short of time right now). Then rather than threading, simply have per-list queue locks. The qrunner then forks N children (configurable) which proceed to attempt to empty each queue in order, each one locking its queue as it gets it (such that the others pass over it) etc. Thus deliveries to the MTA are parallelised. Its a fairly light weight code change (private queues and a child forking qrunner) but it skips all the overhead of threading and fancy locking schemse. Further, as the number of posts per list queue is likely to be low, the iteration rate, and therefore the lock period (for other posts to join a given list's queue will be small. If Mailman v2 currently uses private queue (dunno) then your likely problem is twofold: the thrash rate of walking 10K private queues plus the MTA receipt problem. This problem is mostly unique to the very large number of lists your run. A possibly better tack there (given a reasonbly high frequency qrunner polling rate) would probably be to store the queue entries in hash trees (hash by list or hash by post) and then go from there. Adjust the hash for say N buckets (where N is the total number of parallel qrunners you want). This has the advantage of minimising disk and directory thrashing. The disadvantage is that the code impact would be correspondingly large. > On Fri, Nov 03, 2000 at 04:57:11PM -0800, J C Lawrence wrote: >> > Yeah, that's just what I was saying here. ...etc >> >> BTW: Did you inbtend this to be off list? > Yeah, it's not mailman related, it's exim stuff, thus off topic > :-) This conversely whould probably go back to the list. Mind if I cross it? -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From gietl@gietl.com Sat Nov 4 12:58:10 2000 From: gietl@gietl.com (Andreas Gietl) Date: Sat, 04 Nov 2000 13:58:10 +0100 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> Message-ID: <3A0407E2.42ABB0AF@gietl.com> "Barry A. Warsaw" wrote: > > >>>>> "AG" == Andreas Gietl writes: > > AG> After upgrading to rc1 without any errors from make install > AG> mailman still works fine, but it does no subscriptions any > AG> longer. A user who signs up in the listinfo page gets his > AG> confirmation mail and after he/she confirmed the message (and > AG> the MTA delivered it w/o error) nothing happens. The subscribe > AG> log still shows the user as pending and nothing happens. > > >>>>> "DM" == Dan Mick writes: > > DM> I just tested this again yesterday, and it worked for me. > > Me too. > > When you do a "bin/dumpdb data/pending_subscriptions.db" do you still > see the pending subscription in there? Yeah, the subscription is still in there! Do you still see the > confirmation message in qfiles? Yeah, the confirmation message is in there! If so, are there any obvious errors > in logs/error? nothing. i of course checked that! Does it have the expected Subject: header (i.e. at the > end of the line, "request xxxxxx" where those x's are the numbers). > Yeah. it does have. Do you need any further information? > -Barry > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From barry@wooz.org Sat Nov 4 15:30:58 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Sat, 4 Nov 2000 10:30:58 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail References: <20001103113539.S22363@marc.merlins.org> Message-ID: <14852.11186.967453.490195@anthem.concentric.net> >>>>> "MM" == Marc MERLIN writes: MM> [I am not Ccing mailman-developers as this is not encouraged, MM> but if someone on both lists thinks it should be forwarded MM> there, please feel free] I'm going to answer some of the other questions as best I can, and then I propose to move this discussion to mailman-developers so we can design a proper fix over there. MM> The problem is due to qrunner being single threaded by default MM> and having a global lock. Because some mailing lists have MM> subscribers in domains where DNS is slow and unreliable, the MM> MTA will hang on those rcpt to until DNS resolves or timeouts, MM> and qrunner won't be done in time. After that, it's all MM> downhill from there, more mail queues up, qrunner falls even MM> further behind, etc, etc... So one of the problems is that the handoff between Mailman and the MTA is synchronous with some aspect of the MTA's delivery to the remote site, namely dns lookup. One of the first things you need to do is break this synchrony, either by improving the dns lookup on the MTA side or putting the MTA in asynchronous mode for local message acceptance. Basically you want Mailman to just say "here, don't process this yet, just drop it in your outgoing queue and deal with it later." This might be related to DSN (delivery status notification); there's two DSN RFC (don't have the numbers handy right now), one that talks about the mime bounce format and another that talks about esmtp extensions for synchronous notification of delivery failures. You /don't/ want that! From the followups, it sounds like Exim can be configured to take local delivery asynchronously, and I believe that that is how Postfix works by default. Dunno about Qmail or Sendmail, but I have to believe it's possible to put them in those modes. Now, to sketch out how I think Mailman ought to work for 2.1, and it would not be too hard to whip something up with the 2.0 architecture (ob plug: we might be able to arrange some consulting gigs with Digital Creations if necessary). First, I'd write a long-running process based around asynchat/asyncore that was essentially our own bulk mailer. The async* modules are standard Python modules which make possible select-based high performance servers. They aren't multithreaded, and do not need to be because these servers are primarily i/o bound. When i/o blocks on one channel blocks, another one picks up and works for a while. So this new process, let's call it `bulkmail'. Bulkmail would have one (probably) unix socket open to take new outgoing messages from qrunner. It'd probably write them to disk as a backup so failures don't drop messages. I'm thinking it would then sort recipients based on domains, and then it would start resolving MX records, caching the results. There'd be bins for each MX containing pointers to the messages that need to be delivered to that MX. As more messages came in for that MX, they'd be dropped at the end of the bin. Once a connetion to the MX is established, bulkmail would then just start delivering messages to it until the bin was emptied. Any i/o blocks in any of the processes will allow async* to switch to a different delivery channel. We may need to do some explicit channel management to make sure some are not starved. We'd have to have a watchdog to make sure bulkmail is running, and I'm sure there are other issues to work out, but I've gotta run. I think this will work better than the current SMTPDirect threading stuff. Thoughts? -Barry From barry@wooz.org Sat Nov 4 15:35:23 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Sat, 4 Nov 2000 10:35:23 -0500 (EST) Subject: [Mailman-Developers] mailman installation error References: <3A034AC5.4191FD96@bmedesign.com> Message-ID: <14852.11451.138206.842858@anthem.concentric.net> >>>>> "mt" == mike tran writes: mt> After I did ./configure --with-cgi-gid=nobody mt> --with-mail-gid=mailman and make install. Then I ran mt> check_perms. It keep telling me that it cannot find the mt> module "paths" in line 19 of check_perms. Any suggestions is mt> appricated. I am using RH 6.2 and the latest Apache with mt> python 1.5.2 Look in /home/mailman/bin. If there's no paths.py file in there, your installation failed somehow. -Barry From barry@wooz.org Sat Nov 4 15:45:53 2000 From: barry@wooz.org (barry@wooz.org) Date: Sat, 4 Nov 2000 10:45:53 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> Message-ID: <14852.12081.57421.807051@anthem.concentric.net> >>>>> "AG" == Andreas Gietl writes: >> Do you still see the >> confirmation message in qfiles? AG> Yeah, the confirmation message is in there! >> If so, are there any obvious errors >> in logs/error? AG> nothing. i of course checked that! >> Does it have the expected Subject: header (i.e. at the >> end of the line, "request xxxxxx" where those x's are the >> numbers). AG> Yeah. it does have. AG> Do you need any further information? Yes, do a bin/dumpdb qfiles/.db and look for the `pipeline' entry. That should show us where things are getting stuck. -Barry From claw@kanga.nu Sat Nov 4 17:06:39 2000 From: claw@kanga.nu (J C Lawrence) Date: Sat, 04 Nov 2000 09:06:39 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: Message from barry@wooz.org (Barry A. Warsaw) of "Sat, 04 Nov 2000 10:30:58 EST." <14852.11186.967453.490195@anthem.concentric.net> References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> Message-ID: <20435.973357599@kanga.nu> On Sat, 4 Nov 2000 10:30:58 -0500 (EST) Barry A Warsaw wrote: > So this new process, let's call it `bulkmail'. Bulkmail would > have one (probably) unix socket open to take new outgoing messages > from qrunner. It'd probably write them to disk as a backup so > failures don't drop messages. I'm thinking it would then sort > recipients based on domains, and then it would start resolving MX > records, caching the results. There'd be bins for each MX > containing pointers to the messages that need to be delivered to > that MX. As more messages came in for that MX, they'd be dropped > at the end of the bin. > Once a connetion to the MX is established, bulkmail would then > just start delivering messages to it until the bin was emptied. > Any i/o blocks in any of the processes will allow async* to switch > to a different delivery channel. We may need to do some explicit > channel management to make sure some are not starved. Ouch. I really don't like this idea. For one, it make the processing of oubound mail from a list server unique to the rest of the mail system both on the local host, and the local 'net (consider smarthosts, intermittent connectivity, local domain based mail routing, firewalling etc). It also places needs for a long running process which then needs to be resistant and tolerant to unexpected/frequent shutdowns (laptops, home machines, etc). As discussed previously amongst Chuq, Nigel and I, the needs of large list server systems are rather different from the normal home hobbyest requirements, but are not compleatly alien. However, the needs of very large list installations (cf ListServ, Egroups, or SourceForge) are rather different yet again. I'm not convinced of the value in beating on Mailman to support the (comparitively rare if high profile) very large installations when the current (much larger and more common Mailman-wise) mid-size realm still needs attention. Certainly, such changes should not detract from Mailman's current level of suitability for smaller installations. Now that said, more intelligent handling of Mailman's outbound queue and its hand off to the local or a remote smarthost-like MTA could stand considerable improvement. I've posted a couple ideas which would require relatively small code changes and which should improve the situation. However, both ideas were deliberately attempting to minimise code impact. What would be a really good approach without concern for code impact? I suspect a modified form of the hash tree for queue storage (cf QMail's implementation minus the silly (for this use) inode specifics) with a slightly perverted form of your (Barry's) long running bulkmailer to process that hash queue. I'd tend to make the bulkmailer actually an intermittently running item to help support for intermittently connected nodes. Say something like: Cron launches the bulkmailer. The bulkmailer forks N children processing the queue. The bulkmailer exits upon an empty queue. Should cron launch a new bulkmailer when the prevvious incarnation hasn't exited yet, the new instance merely exits immediately. Locking for the above is fairly simple. Standard IPCs can be used for the instance collision checks. Locking on the hash queues could be a bit intereting from a portability and performance vantage given the fact that the list side will be attemptiong to deliver into the same tree at the same time that deliveries are happening (no more lock collisions please) -- which pretty much requires that locking be on the queue-entry level rather than the hash bucket level. Not rocket science, just a bit finnicky. Will this handle SourceForge? Probably. Certainly given a local MTA that does no checking on mail from localhost (and possibly which immeidately respools out to a dedicated smarthost) it would significantly improve the current state. Enough? Dunno. I'd need more specific data on their current setup, metrics etc. -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From forsberg@lysator.liu.se Sat Nov 4 17:27:15 2000 From: forsberg@lysator.liu.se (Erik Forsberg) Date: Sat, 4 Nov 2000 18:27:15 +0100 (CET) Subject: [Mailman-Developers] Patch #102268 Message-ID: <14852.18163.486910.710242@j218.ryd.student.liu.se> Hi! I submitted a patch (at sourceforge) earlier today, #102268. It's basically a two-line-change to MailMan/Archiver/HyperArch.py in 2.0rc1 that makes the Content-Transfer-Encoding all lowercase. Hopefully I produced a working patch-file.. In my humble opinion that patch is small and uncomplicated enough to fit into the release version, but I'll leave that decision to the Cabal. Anyway, the reason I wrote the patch was that some mails in a list I have didn't decode the QP the right way because the Content-Transfer-Encoding was QUOTED-PRINTABLE (all uppercase). The person writing the mails is using dtmail. So far, the patch comment is quite the same as this mail. The reason I write this mail is that I actually found some text in RFC 1521 that supports my patch fully. Here: ---snip--- The Content-Transfer-Encoding field is designed to specify an invertible mapping between the "native" representation of a type of data and a representation that can be readily exchanged using 7 bit mail transport protocols, such as those defined by RFC 821 (SMTP). This field has not been defined by any previous standard. The field's value is a single token specifying the type of encoding, as enumerated below. Formally: encoding := "Content-Transfer-Encoding" ":" mechanism mechanism := "7bit" ; case-insensitive / "quoted-printable" / "base64" / "8bit" / "binary" / x-token These values are not case sensitive. That is, Base64 and BASE64 and bAsE64 are all equivalent. An encoding type of 7BIT requires that the body is already in a seven-bit mail-ready representation. This is the default value -- that is, "Content-Transfer-Encoding: 7BIT" is assumed if the Content-Transfer-Encoding header field is not present. ---snap--- Note the the words "not case sensitive" :-). And guess if I was confused by the fact that the archive creates new files every time you run the 'arch' utility. After testing my code, I reloaded the message in my browser, and nothing happended. Very confusing until I understood that there was a new file with the message in it.. I guess the archiver behaves that way so search engines won't get confused if you remove a message or something? \EF -- Erik Forsberg http://www.lysator.liu.se/~forsberg/ PGP fingerprint = 37 1B D7 9E 97 8C EF 39 0E DE 08 E8 99 0C 5E A9 From chuqui@plaidworks.com Sat Nov 4 18:29:15 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Sat, 4 Nov 2000 10:29:15 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: <20435.973357599@kanga.nu> References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <20435.973357599@kanga.nu> Message-ID: > > Once a connetion to the MX is established, bulkmail would then >> just start delivering messages to it until the bin was emptied. >> Any i/o blocks in any of the processes will allow async* to switch >> to a different delivery channel. We may need to do some explicit >> channel management to make sure some are not starved. > >Ouch. I really don't like this idea. Neither do I. This is actually something that I've looked at long and hard in my non-mailman server work. After a fair amount of work and research, I finally came to the conclusion that you are MUCH better off letting the MTA do the MTA's work, and letting the MLM do the MLM work, and once you make the decision that the MLM has to *also* become an MTA, you're doing down a road you don't want to travel. Sendmail, for instance, has many years experience optimizing delivery as an MTA. It's a complex, nasty business with lots of subtleties. If you're building a list manager, how much work would you need to do to get a private delivery system that's as well tuned and efficient as sendmail already is? ditto all of the other MTAs. I've built some prototype systems to test this. Even though (in theory) you're adding a layer of delivery and other overhead, it's very difficult to come even close to the performance a tuned MTA can give you -- and you're writing a lot of code to do it. One of the systems I've been investigating, for instance, would do 100% customized mail driven by a template document and pulling data out of a database -- with a design parameter of up to 10 million deliveries. the goal is at least 500K deliveries an hour, preferably double that. Right now, on a system with a sendmail 8.9 base and a non-optimized delivery tool, I'm doing 400-450K/hour. I expect to see a nice addition when I move to sendmail 8.10.x in a week or so. this is on a Sun E250, FWIW, with the sendmail queues living in a ram disk. Good sized hardware, but not particularly big or fast hardware. Instead of reinventing the MTA wheel, I think we're much better off coming up with an MTA -> MLM interface that's very flexible and highly configurable (most especially in how to deliver and how much to parallelize the infeed to the MLM), and then focus on how to tune the MTA and MLM through documentation. Splitting the inbound and outbound queue would be my first thing here, and probably split bounces into a third queue. That's a pretty quick, easy optimization that makes sure the end user sees fast response without being bogged down by deliveries, and that's a huge perception issue. Then focus on parallelizing the delivery from mailman into the MTA, and make that configurable so each admin can tune it to their system and needs. >As discussed previously amongst Chuq, Nigel and I, the needs of >large list server systems are rather different from the normal home >hobbyest requirements, but are not compleatly alien. However, the >needs of very large list installations (cf ListServ, Egroups, or >SourceForge) are rather different yet again. This is a basic reality -- things don't scale. Or worse, they scale for a while, and then you need to switch paradigms. I found that one out the hard way. If someone wants a rhetoric on how to scale mail list servers infinitely, I'd be happy to explain how, since I've had to develop an architecture to do so. the nice thing is, it can be done without exceptional engineering hassles -- but it's not just adding another daemon or a faster CPU (although those are solutions for parts of it, just not ALKWAYs the solutions) > I'm not convinced of >the value in beating on Mailman to support the (comparitively rare >if high profile) very large installations when the current (much >larger and more common Mailman-wise) mid-size realm still needs >attention. Certainly, such changes should not detract from >Mailman's current level of suitability for smaller installations. I think we can build a Mailman that does this, at least for, oh, 95% of the universe out there, and the other 5% are going to have custom solutions anyway (or should!). What we don't want to do is screw up Mailman for the "typical" user to make it work for the big site; but we also don't want Mailman to get a reputation as a "small server only" system, because it'll cause people to reject it in implementations. Fortunately, I don't think you need to do that. It just needs some tweaking. >support for intermittently connected nodes. Say something like: > > Cron launches the bulkmailer. > The bulkmailer forks N children processing the queue. > The bulkmailer exits upon an empty queue. > Should cron launch a new bulkmailer when the prevvious incarnation > hasn't exited yet, the new instance merely exits immediately. > >Locking for the above is fairly simple. Standard IPCs can be used >for the instance collision checks. Locking on the hash queues could >be a bit intereting from a portability and performance vantage given >the fact that the list side will be attemptiong to deliver into the >same tree at the same time that deliveries are happening (no more >lock collisions please) -- which pretty much requires that locking >be on the queue-entry level rather than the hash bucket level. Not >rocket science, just a bit finnicky. > >Will this handle SourceForge? Probably. On reasonable hardware, definitely. That's basically how my current custom system works. right now, the number of parallel infeeds from mailman is 1. I'm willing to bet the delivery MTA is basically idle and bored. By moving to parallel infeeds, you can stoke the MTA up to speed, and the trick is for each site to figure out what number of parallel infeeds will work keeping the queues full for the MTA to stay busy without overloading them and causing the MTA to thrash. That's simply a case of tuning and modelling. And simply allowing "N" infeed threads to the MLM will solve Sourceforge's problem and pretty much everyone else's, without having to get into the MTA business, where the best we can really hope is to be "as good" as the real MTA. so my recommendation is: 1) split the current qfiles into three queues: inbound, outbound and bounce 2) parallelize the outbound queue into "N" configurable delivery threads. 3) work on documentation on how to tune this for maximum perforamnce with major MTAs, and how to tune MTAs for maxiumum performance. That's a set of pretty easy updates, no technological miracles or black boxes, and solves all but the worst problems someone running Mailman is likely to see. And for sites this *doesn't* solve, it's either because they're doing the 5 pounds in a 1 pound bag thing, or they probably need to start hiring people like us to custom build someething. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From Dan.Mick@west.sun.com Sat Nov 4 23:36:18 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Sat, 04 Nov 2000 15:36:18 -0800 Subject: [Mailman-Developers] Patch #102268 References: <14852.18163.486910.710242@j218.ryd.student.liu.se> Message-ID: <3A049D72.22801714@west.sun.com> Erik Forsberg wrote: > > Hi! > > I submitted a patch (at sourceforge) earlier today, #102268. It's basically a > two-line-change to MailMan/Archiver/HyperArch.py in 2.0rc1 that makes > the Content-Transfer-Encoding all lowercase. Hopefully I produced a > working patch-file.. > > In my humble opinion that patch is small and uncomplicated enough to > fit into the release version, but I'll leave that decision to the > Cabal. > > Anyway, the reason I wrote the patch was that some mails in a list I > have didn't decode the QP the right way because the > Content-Transfer-Encoding was QUOTED-PRINTABLE (all uppercase). The > person writing the mails is using dtmail. So far, the patch comment is > quite the same as this mail. > > The reason I write this mail is that I actually found some text in RFC > 1521 that supports my patch fully. Here: > > ---snip--- > The Content-Transfer-Encoding field is designed to specify an > invertible mapping between the "native" representation of a type of > data and a representation that can be readily exchanged using 7 bit > mail transport protocols, such as those defined by RFC 821 (SMTP). > This field has not been defined by any previous standard. The field's > value is a single token specifying the type of encoding, as > enumerated below. Formally: > > encoding := "Content-Transfer-Encoding" ":" mechanism > > mechanism := "7bit" ; case-insensitive > / "quoted-printable" > / "base64" > / "8bit" > / "binary" > / x-token > > These values are not case sensitive. That is, Base64 and BASE64 and > bAsE64 are all equivalent. An encoding type of 7BIT requires that > the body is already in a seven-bit mail-ready representation. This > is the default value -- that is, "Content-Transfer-Encoding: 7BIT" is > assumed if the Content-Transfer-Encoding header field is not > present. > ---snap--- > > Note the the words "not case sensitive" :-). So, that means that a header which reads QUOTED-PRINTABLE is completely RFC-compliant, and if there's a mailreader or other utility that barfs on it, it's that mailreader's bug, not Mailman's. Or am I missing something? > And guess if I was confused by the fact that the archive creates new > files every time you run the 'arch' utility. After testing my code, I > reloaded the message in my browser, and nothing happended. Very > confusing until I understood that there was a new file with the > message in it.. > > I guess the archiver behaves that way so search engines won't get > confused if you remove a message or something? No, arch is there to rebuild archives from .mbox files; it's not there to add messages. Hence its comment: Use this command to rebuild the archives for a mailing list. From forsberg@lysator.liu.se Sat Nov 4 23:40:49 2000 From: forsberg@lysator.liu.se (Erik Forsberg) Date: Sun, 5 Nov 2000 00:40:49 +0100 (CET) Subject: [Mailman-Developers] Patch #102268 In-Reply-To: <3A049D72.22801714@west.sun.com> References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <3A049D72.22801714@west.sun.com> Message-ID: <14852.40577.83440.45618@j218.ryd.student.liu.se> > >So, that means that a header which reads QUOTED-PRINTABLE is completely >RFC-compliant, and if there's a mailreader or other utility that barfs on >it, it's that mailreader's bug, not Mailman's. > >Or am I missing something? Yes. Mailman didn't archive correctly because it didn't understand the encoding. Here's the code that caused the problem: (Mailman/Archiver/HyperArch.py) if self.charset is None or self.cenc != "quoted-printable": return null_to_space(string.join(body, "")) now what happends if self.cenc is "QUOTED-PRINTABLE"? The code that knows how to decode QP is never executed and the mail looks very ugly in the Web archive. My patch lowercases self.cenc where it's value is set. >> I guess the archiver behaves that way so search engines won't get >> confused if you remove a message or something? > >No, arch is there to rebuild archives from .mbox files; it's not there to add >messages. Hence its comment: > > Use this command to rebuild the archives for a mailing list. Try using arch on a archive that's already existing... \EF -- Erik Forsberg http://www.lysator.liu.se/~forsberg/ PGP fingerprint = 37 1B D7 9E 97 8C EF 39 0E DE 08 E8 99 0C 5E A9 From Dan.Mick@west.sun.com Sat Nov 4 23:47:24 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Sat, 04 Nov 2000 15:47:24 -0800 Subject: [Mailman-Developers] Patch #102268 References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <3A049D72.22801714@west.sun.com> <14852.40577.83440.45618@j218.ryd.student.liu.se> Message-ID: <3A04A00C.A4896940@west.sun.com> Erik Forsberg wrote: > Yes. Mailman didn't archive correctly because it didn't understand the > encoding. Here's the code that caused the problem: > My patch lowercases self.cenc where it's value is set. Ah, I see. I didn't read the patch, and misread your description of it. Sorry. > >> I guess the archiver behaves that way so search engines won't get > >> confused if you remove a message or something? > > > >No, arch is there to rebuild archives from .mbox files; it's not there to add > >messages. Hence its comment: > > > > Use this command to rebuild the archives for a mailing list. > > Try using arch on a archive that's already existing... Right, but that's "augment the archives", which is not what arch tries to do. If you want to rebuild, delete or move the existing archives, and rebuild, and you'll get the result you expect. From ocschwar@MIT.EDU Sun Nov 5 22:55:37 2000 From: ocschwar@MIT.EDU (The guy named after an Om Kalthoum song) Date: Sun, 05 Nov 2000 17:55:37 -0500 Subject: [Mailman-Developers] Mailman and GPG. Message-ID: <200011052255.RAA14775@alice-whacker.mit.edu> Greetings! I am wondering if anyone has begun an implementation of Gnu Provacy Guard email handling in Mailman, that would work along these lines: 1. mailman has its own keypair, and a keyring of the public keys of all mailing list subscribers. 2. A poster to a list will use GPG to encript his posting using mailman's public key, and then mailman sends out individually encripted messages or digests to each subscriber. Something like this existed (PGPdomo), but it vanished, and putting this functionality back in Majordomo is a Lovecraftian task. I may be itnerested in doing this as an exercise to learn Python. Has anyone done something like this? Also, is there a Python interface to GPG? Regards, -- Omri Schwarz --from somewhere on the net: float o=0.075,h=1.5,T,r,O,l,I;int _,L=80,s=3200;main(){for(;s%L|| (h-=o,T= -2),s;4 -(r=O*O)<(l=I*I)|++ _==L&&write(1,(--s%L?_ Are the mailman developers at all concerned by http://www.google.com/search?q=mailman-owner+reminder+password http://x66.deja.com/=dnc/getdoc.xp?AN=641175690 This is probably especially a problem with lists that were converted from another MLM, where there was no explicit "one address, one reader" assumption at the time the archive was subscribed. [note: 1. I'm not subscribed to the mailman mailing lists, nor do I use mailman except as a list subscriber and the maintainer of the rec.music.gaffa news2mail gateway; please cc me on any direct responses. 2. I looked through the archives and saw many discussions of passwords, but did not find this issue addressed; my apologies if I missed it. 3. Yes, the rec.music.gaffa gateway filters mailman admin messages, and has done so since a few hours after the first reminder appeared in the newsgroup. 4. No, I don't think this is a huge security issue, but it certainly does have some potential for minor mischief. 5. I may mention this (in passing) in a submission to comp.risks soon. ] -dan From claw@kanga.nu Mon Nov 6 07:23:35 2000 From: claw@kanga.nu (J C Lawrence) Date: Sun, 05 Nov 2000 23:23:35 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: Message from Chuq Von Rospach of "Sat, 04 Nov 2000 10:29:15 PST." References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <20435.973357599@kanga.nu> Message-ID: <4754.973495415@kanga.nu> On Sat, 4 Nov 2000 10:29:15 -0800 Chuq Von Rospach wrote: >> > Once a connetion to the MX is established, bulkmail would then >>> just start delivering messages to it until the bin was emptied. >>> Any i/o blocks in any of the processes will allow async* to >>> switch to a different delivery channel. We may need to do some >>> explicit channel management to make sure some are not starved. >> Ouch. I really don't like this idea. > Neither do I. This is actually something that I've looked at long > and hard in my non-mailman server work. After a fair amount of > work and research, I finally came to the conclusion that you are > MUCH better off letting the MTA do the MTA's work, and letting the > MLM do the MLM work, and once you make the decision that the MLM > has to *also* become an MTA, you're doing down a road you don't > want to travel. Agreed. There's also the simple aspect of the fact that optimising for the very large case (which needs the MLM-sepcific MTA supports ala ListServ) is near on pessimal for the small and medium cases. Spiffy domain based routing and near-target explosion just don't make a whole lot of sense for lists with a few hundred/thousand members. Asides from which, the optimisations for mail delivery to a massive number of targets and mass mail delivery to a comparitively constrained target set can be rather different. That's ugly territorry. We don't need to go there yet. > Instead of reinventing the MTA wheel, I think we're much better > off coming up with an MTA -> MLM interface that's very flexible > and highly configurable (most especially in how to deliver and how > much to parallelize the infeed to the MLM), and then focus on how > to tune the MTA and MLM through documentation. Agreed. There's not a whole lot of tuning you can do on SMTP hand-offs to the MTA (which is necessary as a feature to support non-localhost MTAs which is in turn needed by many sites), but some MTAs have divers command line options that can be useful in an MLM delivery world (peeking quickly at the Exim docs). > Splitting the inbound and outbound queue would be my first thing > here, and probably split bounces into a third queue. That's a > pretty quick, easy optimization that makes sure the end user sees > fast response without being bogged down by deliveries, and that's > a huge perception issue. Then focus on parallelizing the delivery > from mailman into the MTA, and make that configurable so each > admin can tune it to their system and needs. Without spending any time collecting metrics, it does look like the low hanging fruit is in handling the MLM->MTA handoff. Given that the distribution of posts across lists is often uneven (and unpredictable), and that the optimal parallelism (N) is fairly fixed for a given machine, any pattern which supported keeping N pipes stuffed throwing things at the MTA (with minimal locking) would seem fair. As for splitting the queues, yep, hadn't thought about that aspect, but that's an obvious contention area for an active list. >> As discussed previously amongst Chuq, Nigel and I, the needs of >> large list server systems are rather different from the normal >> home hobbyest requirements, but are not compleatly alien. >> However, the needs of very large list installations (cf ListServ, >> Egroups, or SourceForge) are rather different yet again. > This is a basic reality -- things don't scale. Or worse, they > scale for a while, and then you need to switch paradigms. I found > that one out the hard way. If someone wants a rhetoric on how to > scale mail list servers infinitely, I'd be happy to explain how, > since I've had to develop an architecture to do so. the nice thing > is, it can be done without exceptional engineering hassles -- but > it's not just adding another daemon or a faster CPU (although > those are solutions for parts of it, just not ALKWAYs the > solutions) We should really have lunch some time. I'm currently working down off San Tomas and Saratoga (startup, oddly enough next door to BeOpen). I'm wrapping up this contract this month and looking for next one, so time is a little tight here, but I'd love to get together for a chat. You over at the main Apple campus? >> I'm not convinced of the value in beating on Mailman to support >> the (comparitively rare if high profile) very large installations >> when the current (much larger and more common Mailman-wise) >> mid-size realm still needs attention. Certainly, such changes >> should not detract from Mailman's current level of suitability >> for smaller installations. > I think we can build a Mailman that does this, at least for, oh, > 95% of the universe out there, and the other 5% are going to have > custom solutions anyway (or should!). What we don't want to do is > screw up Mailman for the "typical" user to make it work for the > big site; but we also don't want Mailman to get a reputation as a > "small server only" system, because it'll cause people to reject > it in implementations. Fortunately, I don't think you need to do > that. It just needs some tweaking. We must stop agreeing like this. Its getting embarassing. > And for sites this *doesn't* solve, it's either because they're > doing the 5 pounds in a 1 pound bag thing, or they probably need > to start hiring people like us to custom build someething. Ungghhhh. -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Mon Nov 6 07:39:37 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Sun, 5 Nov 2000 23:39:37 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: <4754.973495415@kanga.nu> References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <20435.973357599@kanga.nu> <4754.973495415@kanga.nu> Message-ID: At 11:23 PM -0800 11/5/00, J C Lawrence wrote: > >Agreed. There's also the simple aspect of the fact that optimising >for the very large case (which needs the MLM-sepcific MTA supports >ala ListServ) is near on pessimal for the small and medium cases. >Spiffy domain based routing and near-target explosion just don't >make a whole lot of sense for lists with a few hundred/thousand >members. Asides from which, the optimisations for mail delivery to >a massive number of targets and mass mail delivery to a >comparitively constrained target set can be rather different. Nope. This is a job for -- ta da -- an API with a plug in architecture. Don't try to solve the huge list probelm, but solve the 90% problem (what works for the first 90% of the sites out there), and make it easy to plug in a different delivery beast so that other 10% can simply write a new module and swap it in (and hopefully send it back to the mailman project...) > >We should really have lunch some time. I'm currently working down >off San Tomas and Saratoga (startup, oddly enough next door to >BeOpen). I'm wrapping up this contract this month and looking for >next one, so time is a little tight here, but I'd love to get >together for a chat. You over at the main Apple campus? Effectively, yes. I'm down by the data center, which is hidden a bit off campus. > > And for sites this *doesn't* solve, it's either because they're >> doing the 5 pounds in a 1 pound bag thing, or they probably need >> to start hiring people like us to custom build someething. > >Ungghhhh. heh. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From thomas@xs4all.net Mon Nov 6 11:01:36 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 6 Nov 2000 12:01:36 +0100 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <200011052255.RAA14775@alice-whacker.mit.edu>; from ocschwar@MIT.EDU on Sun, Nov 05, 2000 at 05:55:37PM -0500 References: <200011052255.RAA14775@alice-whacker.mit.edu> Message-ID: <20001106120132.F27208@xs4all.nl> On Sun, Nov 05, 2000 at 05:55:37PM -0500, The guy named after an Om Kalthoum song wrote: [ Mailman using GPG to decrypt and re-crypt messages ] > I may be itnerested in doing this as an exercise to learn > Python. Has anyone done something like this? I don't think so. It's the first I heard about it, in any case. Note that "it isn't that simple" ;) You have to think about Archives. Do you want to enable them over SSL only ? SSL with client certificates ? Do you just want to disable them ? What about news gatewaying ? Just disable it for 'secure' groups, or just sign the postings ? > Also, is there a Python interface to GPG? I think I saw something like it, but I can't fid it now. A quick google search shows some hits on 'python' in the gnupg-devel list, so maybe there is the right place to look ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From jam@jamux.com Mon Nov 6 13:33:07 2000 From: jam@jamux.com (John A. Martin) Date: Mon, 06 Nov 2000 08:33:07 -0500 Subject: [Mailman-Developers] rc1 Possible confusion in Defaults.py comments Message-ID: <20001106133307.349FE48031@athene.jamux.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Using Mailman version 2.0rc1, in Defaults.py, under Bounce processing defaults, one sees: # 0 means do nothing # 1 means disable and send admin a report, # 2 means nuke'em (remove) and send admin a report, # 3 means nuke 'em and don't report (whee:) DEFAULT_AUTOMATIC_BOUNCE_ACTION = 1 and on the Bounce Options web page:
Do nothing Disable and notify me Disable and DON'T notify me Remove and notify me
(Ie. from left to right, (0) do nothing, (1) disable and notify, (2) disable and DON'T notify, and (3) Remove and notify. Changing the value of DEFAULT_AUTOMATIC_BOUNCE_ACTION in mm_cfg.py moves the radio button on the web page from left to right for 0, 1, 2, 3 as expected. However, the comments in Defaults.py do not seem to agree with the descriptions of options 2 and 3 on the web page. I have not checked the actual behavior of these options. jam -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: OpenPGP encrypted mail preferred. See iEYEARECAAYFAjoGsw4ACgkQUEvv1b/iXy+WSgCdEeZSCGMLA2xyt58Lcs3HyAkb BPgAoIvUH5uNQXPP2RWAHXOnbBQULOrH =dPoa -----END PGP SIGNATURE----- From jarrell@vt.edu Mon Nov 6 15:34:18 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Mon, 06 Nov 2000 10:34:18 -0500 Subject: [Mailman-Developers] mailman installation error In-Reply-To: <14852.11451.138206.842858@anthem.concentric.net> References: <3A034AC5.4191FD96@bmedesign.com> Message-ID: <5.0.0.25.2.20001106103101.03094c30@vtserf.cc.vt.edu> At 10:35 AM 11/4/00 -0500, Barry A. Warsaw wrote: >>>>>> "mt" == mike tran writes: > > mt> After I did ./configure --with-cgi-gid=nobody > mt> --with-mail-gid=mailman and make install. Then I ran > mt> check_perms. It keep telling me that it cannot find the > mt> module "paths" in line 19 of check_perms. Any suggestions is > mt> appricated. I am using RH 6.2 and the latest Apache with > mt> python 1.5.2 > >Look in /home/mailman/bin. If there's no paths.py file in there, your >installation failed somehow. Actually, this probably is a doc problem. I've dealt with this several times on the users list, and in all but one case (where their python was hosed) it was people not clearly following the directions. They're supposed to install, then run check_perms out of ~mailman/bin. Where there's a paths.py(c). However, what confuses a lot of people who only skim, is that in ${srcdir}/mailman/bin there's a check_perms. (The distribution copy). Which, of course, doesn't have a paths.py installed yet, and thus, doesn't work, but looks like it ought to, particularly to a complete newcomer who doesn't understand how things go together. (Heck, it took *me* a couple of days to figure out whatinhell they were doing wrong.) There probably needs to be a mention in the install file to *not* run check_perms until *after* you install, and to run it from the *installed* base. Or rename the install copy to check_perms.dist, and install it as check_perms... From jarrell@vt.edu Mon Nov 6 15:37:05 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Mon, 06 Nov 2000 10:37:05 -0500 Subject: [Mailman-Developers] Patch #102268 In-Reply-To: <14852.40577.83440.45618@j218.ryd.student.liu.se> References: <3A049D72.22801714@west.sun.com> <14852.18163.486910.710242@j218.ryd.student.liu.se> <3A049D72.22801714@west.sun.com> Message-ID: <5.0.0.25.2.20001106103556.030bfb00@vtserf.cc.vt.edu> At 12:40 AM 11/5/00 +0100, you wrote: >Try using arch on a archive that's already existing... Dan's point is you're not supposed to; it's an admin tool for rebuilding archives from scratch. You're supposed to wipe the archive before using it. If you don't, the results aren't pretty. From Nigel.Metheringham@VData.co.uk Mon Nov 6 16:21:45 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Mon, 06 Nov 2000 16:21:45 +0000 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: Message from Chuq Von Rospach of "Sun, 05 Nov 2000 23:39:37 PST." Message-ID: [Please can we try and kill these postings that are sent to both mailman-users & mailman-developers] Having been watching the floods at home in York for the last few days I'll just come in at this point chuqui@plaidworks.com said: > Nope. This is a job for -- ta da -- an API with a plug in > architecture. Don't try to solve the huge list probelm, but solve the > 90% problem (what works for the first 90% of the sites out there), > and make it easy to plug in a different delivery beast so that other > 10% can simply write a new module and swap it in (and hopefully send > it back to the mailman project...) [Loud applause] Mailman is a MLM, its not an MTA. Don't think about making it an MTA. Just leave enough hooks for people to trash their systems effectively. For the original specifics, I'll talk with the sourceforge people about an appropriate exim config for their problem. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From gietl@gietl.com Mon Nov 6 16:33:48 2000 From: gietl@gietl.com (Andreas Gietl) Date: Mon, 06 Nov 2000 17:33:48 +0100 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> Message-ID: <3A06DD6C.D4F23706@gietl.com> any further ideas? i'm sorry, but there is no pipeline entry: that's the db-file: { 'filebase': 'ccfe93988bf681ad271ad049fa54dcd7260a60ac', 'listname': 'bittecomgietltest2', 'torequest': 1, 'version': 2} and this is the .msg file: >From gietl@gietl.com Sat Nov 04 13:52:46 2000 Received: from [195.227.84.2] (helo=d1.x-mailer.de) by d18.x-mailer.de with esmtp (Exim 3.16 #1) id 13s2oQ-0003e1-00 for bittecomgietltest2-request@bitte.com; Sat, 04 Nov 2000 13:52:46 +0100 Received: from p3ee039ab.dip.t-dialin.net ([62.224.57.171] helo=gietl.com) by d1.x-mailer.de with esmtp (Exim 3.16 #1) id 13s2n9-0004iW-00 for bittecomgietltest2-request@bitte.com; Sat, 04 Nov 2000 13:51:27 +0100 Message-ID: <3A0406EC.6C4B8583@gietl.com> Date: Sat, 04 Nov 2000 13:54:04 +0100 From: Andreas Gietl X-Mailer: Mozilla 4.73 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: bittecomgietltest2-request@bitte.com Subject: Re: Bittecomgietltest2 -- confirmation of subscription -- request 569170 References: Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit bittecomgietltest2-request@bitte.com wrote: > > Bittecomgietltest2 -- confirmation of subscription -- request 569170 > > We have received a request from p3ee039ab.dip.t-dialin.net for > subscription of your email address, , to the > bittecomgietltest2@bitte.com mailing list. To confirm the request, > please send a message to bittecomgietltest2-request@bitte.com, and > either: > > - maintain the subject line as is (the reply's additional "Re:" is > ok), > > - or include the following line - and only the following line - in the > message body: > > confirm 569170 > > (Simply sending a 'reply' to this message should work from most email > interfaces, since that usually leaves the subject line in the right > form.) > > If you do not wish to subscribe to this list, please simply disregard > this message. Send questions to bittecomgietltest2-admin@bitte.com. barry@wooz.org wrote: > > >>>>> "AG" == Andreas Gietl writes: > > >> Do you still see the > >> confirmation message in qfiles? > > AG> Yeah, the confirmation message is in there! > > >> If so, are there any obvious errors > >> in logs/error? > > AG> nothing. i of course checked that! > > >> Does it have the expected Subject: header (i.e. at the > >> end of the line, "request xxxxxx" where those x's are the > >> numbers). > > AG> Yeah. it does have. > AG> Do you need any further information? > > Yes, do a bin/dumpdb qfiles/.db and look for the > `pipeline' entry. > > That should show us where things are getting stuck. > -Barry > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From gietl@gietl.com Mon Nov 6 18:11:42 2000 From: gietl@gietl.com (Andreas Gietl) Date: Mon, 06 Nov 2000 19:11:42 +0100 Subject: [Mailman-Developers] what happens after writing to qfiles? Message-ID: <3A06F45E.EBBCD656@gietl.com> Hi, since a few weeks i am exploring mails not proceeded by mailman after upgrades to rc1 or even on servers with a new fresh installation. Since it seems you could not help me till today i want to ask you what happens after mailman wrote the two files to qfiles when it gets a confirmation message? How are they proceeded? concretly: what happens after this is done? [pid 32132] open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.db", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 32132] umask(0) = 022 [pid 32132] open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.db", O_WRONLY|O_CREAT|O_TRUNC, 0664) = 5 [pid 32132] fcntl(5, F_GETFL) = 0x1 (flags O_WRONLY) [pid 32132] fstat(5, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0 [pid 32132] mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4014a000 [pid 32132] _llseek(5, 0, [0], SEEK_CUR) = 0 [pid 32132] umask(022) = 0 [pid 32132] write(5, "{s\7\0\0\0versioni\2\0\0\0s\t\0\0\0torequest"..., 124) = 124 [pid 32132] close(5) = 0 [pid 32132] munmap(0x4014a000, 4096) = 0 [pid 32132] umask(0) = 022 [pid 32132] open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.msg", O_WRONLY|O_CREAT|O_TRUNC, 0664) = 5 thx andreas -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From gietl@gietl.com Mon Nov 6 18:25:24 2000 From: gietl@gietl.com (Andreas Gietl) Date: Mon, 06 Nov 2000 19:25:24 +0100 Subject: [Mailman-Developers] what happens after writing to qfiles? References: <3A06F45E.EBBCD656@gietl.com> Message-ID: <3A06F794.51ADB723@gietl.com> Are the qfiles just processed by qrunner? Andreas Gietl wrote: > > Hi, > > since a few weeks i am exploring mails not proceeded by mailman after > upgrades to rc1 or even on servers with a new fresh installation. > > Since it seems you could not help me till today i want to ask you what > happens after mailman wrote the two files to qfiles when it gets a > confirmation message? How are they proceeded? > > concretly: what happens after this is done? > [pid 32132] > open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.db", > O_RDONLY) = -1 ENOENT (No such file or directory) > [pid 32132] umask(0) = 022 > [pid 32132] > open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.db", > O_WRONLY|O_CREAT|O_TRUNC, 0664) = 5 > [pid 32132] fcntl(5, F_GETFL) = 0x1 (flags O_WRONLY) > [pid 32132] fstat(5, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0 > [pid 32132] mmap(NULL, 4096, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4014a000 > [pid 32132] _llseek(5, 0, [0], SEEK_CUR) = 0 > [pid 32132] umask(022) = 0 > [pid 32132] write(5, "{s\7\0\0\0versioni\2\0\0\0s\t\0\0\0torequest"..., > 124) = 124 > [pid 32132] close(5) = 0 > [pid 32132] munmap(0x4014a000, 4096) = 0 > [pid 32132] umask(0) = 022 > [pid 32132] > open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.msg", > O_WRONLY|O_CREAT|O_TRUNC, 0664) = 5 > > thx > > andreas > -- > andreas gietl > gietl internet services > fon +49 9402 2551 > fax +49 9402 2604 > mobile +49 171 60 70 008 > gietl@gietl.com > > ############################################ > # Das Handbuch sagt, das Programm benötige # > # Windows 95 oder besser. Also habe ich # > # Linux installiert! # > ############################################ > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From claw@kanga.nu Mon Nov 6 18:28:49 2000 From: claw@kanga.nu (J C Lawrence) Date: Mon, 06 Nov 2000 10:28:49 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Message from "The guy named after an Om Kalthoum song" of "Sun, 05 Nov 2000 17:55:37 EST." <200011052255.RAA14775@alice-whacker.mit.edu> References: <200011052255.RAA14775@alice-whacker.mit.edu> Message-ID: <31582.973535329@kanga.nu> On Sun, 05 Nov 2000 17:55:37 -0500 The guy named after an Om Kalthoum song wrote: > Greetings! I am wondering if anyone has begun an implementation > of Gnu Provacy Guard email handling in Mailman, that would work > along these lines: I'm actually engaged (very slowly) in implementing automatically crypted mail support at the MTA level for a friend (he wants Sendmail support, I'm starting with Exim). There actually is little to no reason to add Mailman to the mess -- all the (de)crypt activities can occur compleatly outside of Mailman's operations. The 50,000 foot view for what I'm doing: Mail arrives at the MTA (localhost or SMTP). The MTA detects a flag string in the message or message x-headers. Depending on which strings were used, the MTA either: -- signs the mail with the author's key -- signs the mail and crypts it with the recipient's key The author's key is stored locally. All other keys are required to be available via public keyservers. Extending this to a mailing list actually isn't very difficult: Mail arrives at the list. Instead of being handed off to the list exploder, it passes thru a filter first which looks for blocks crypted with the list's key. If a crypted block is found, it is decrypted, surrounded by flag strings, and a flag x-header is added to the message. The message is then handed to the list exploder. The list exploder explodes the message and hands the bits back to the MTA. The MTA upon delivery of *ANY* message, looks for the X-header. If present it then crypts the flagged block with the envelope's key. If this is so simple, why am I not done yet? -- correct MIME support isn't entirely trivial -- I'm trying to think of ways to make it auditable *AND* scalable -- I've been distracted -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Mon Nov 6 18:37:52 2000 From: claw@kanga.nu (J C Lawrence) Date: Mon, 06 Nov 2000 10:37:52 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Message from J C Lawrence of "Mon, 06 Nov 2000 10:28:49 PST." <31582.973535329@kanga.nu> References: <200011052255.RAA14775@alice-whacker.mit.edu> <31582.973535329@kanga.nu> Message-ID: <32397.973535872@kanga.nu> On Mon, 06 Nov 2000 10:28:49 -0800 J C Lawrence wrote: > On Sun, 05 Nov 2000 17:55:37 -0500 The guy named after an Om > Kalthoum song wrote: >> Greetings! I am wondering if anyone has begun an implementation >> of Gnu Provacy Guard email handling in Mailman, that would work >> along these lines: > I'm actually engaged (very slowly) in implementing automatically > crypted mail support at the MTA level for a friend (he wants > Sendmail support, I'm starting with Exim). There actually is > little to no reason to add Mailman to the mess -- all the > (de)crypt activities can occur compleatly outside of Mailman's > operations. ... Some other attempts are referenced here: https://emu.kanga.nu/bin/view/Main/EncryptedMailingLists -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Mon Nov 6 19:03:08 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 11:03:08 -0800 Subject: [Mailman-Developers] passwords in third party web archives, newsgroups In-Reply-To: <3A05F488.57CFC441@mail.lns.cornell.edu> References: <3A05F488.57CFC441@mail.lns.cornell.edu> Message-ID: At 7:00 PM -0500 11/5/00, Dan Riley wrote: >Are the mailman developers at all concerned by > >http://www.google.com/search?q=mailman-owner+reminder+password >http://x66.deja.com/=dnc/getdoc.xp?AN=641175690 > >This is probably especially a problem with lists that were converted >from another MLM, It's an interesting issue. Mailman includes the X-No-Archive header in these messages, so anyone who's archiving them anyway isn't following the protocol. I'm not sure it's a major issue, and it's an end-user mis-behavior at that -- but it's still somewhat troubling that it gets into archives and search engines. I'm not sure what mailman can do to prevent end-users from shooting themselves in the foot here, though. the passwords are a trivial issue to me. the REAL issue is you have mail lists putting up archives that are being put into global search engines -- and those archives are full of unprotected email addresses just waiting for the spam harvester bots. Compared to that, the passwords are nothing -- like dealing iwth a hangnail on a foot with gangrene. and if the lists want their archives to be wide open like that, there's not a damn thing Mailman can do to save them from themselves. But as long as there are easily harvested email addresses in the search engines, the passwords simply aren't something I'm going to worry about. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From ocschwar@MIT.EDU Mon Nov 6 19:54:38 2000 From: ocschwar@MIT.EDU (Omri Schwarz) Date: Mon, 06 Nov 2000 14:54:38 -0500 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Your message of "Mon, 06 Nov 2000 12:01:36 +0100." <20001106120132.F27208@xs4all.nl> Message-ID: <200011061954.OAA23482@hodge-podge.mit.edu> > On Sun, Nov 05, 2000 at 05:55:37PM -0500, The guy named after an Om Kalthoum song wrote: > > [ Mailman using GPG to decrypt and re-crypt messages ] > > > I may be itnerested in doing this as an exercise to learn > > Python. Has anyone done something like this? > > I don't think so. It's the first I heard about it, in any case. Note that > "it isn't that simple" ;) You have to think about Archives. Do you want to > enable them over SSL only ? SSL with client certificates ? Do you just want > to disable them ? What about news gatewaying ? Just disable it for 'secure' > groups, or just sign the postings ? The motivation I have behind asking (which can quickly drift off-topic for this list) is that the main reason behind the failure of widespread email encryption is human factors. Therefore, the right amount of social engineering will be the driving force in getting people to encrypt email. If a mailing list exploder like what I described is available, people will learn not to 1. share TMI type information on any other kind of mailing list, or 2. share proprietary discussions on any other kind of mailing list. So, a list like this will 1. have no Web archiving, 2. no news gatewaying, and 3. rapidly expiring mailing list keypairs, Just In Case (TM). I'm asking this on the Mailman forum because Mailman would be easier to GPG-enable than Majordomo (just as eating ice cream is more pleasant than root canal..), and because apart from that, I am not picky on how this should be done, hence would be willing to fork Mailman to warp it for this end. -- ------- Omri Schwarz ocschwar@mit.edu "Fair enough: anyone who believes that the laws of physics are | "Prof drop!" mere social conventions is invited to try transgressing | "Thud." those conventions from the windows of my apartment. | "Ow! (I live on the twenty-first floor.)" Alan Sokal - Physicist | My Spleen!" From chuqui@plaidworks.com Mon Nov 6 20:06:13 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 12:06:13 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <200011061954.OAA23482@hodge-podge.mit.edu> References: <200011061954.OAA23482@hodge-podge.mit.edu> Message-ID: At 2:54 PM -0500 11/6/00, Omri Schwarz wrote: >The motivation I have behind asking >(which can quickly drift off-topic for this list) >is that the main reason behind the failure of >widespread email encryption is human factors. >Therefore, the right amount of social engineering >will be the driving force in getting people to encrypt email. I agree with this -- but I disagree that the mailing list is where the focus needs to go. The focus needs to go in fixing mail client interfaces to properly integrate this stuff and make it easy to use. Until that happens, it doesn't matter what else is done, because users won't use it. IMHO, of course. but working on the MLM side is the cart before the horse. Get the clients using encryption rationally, and the server side will follow naturally. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From Dan.Mick@west.sun.com Mon Nov 6 21:34:13 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Mon, 06 Nov 2000 13:34:13 -0800 Subject: [Mailman-Developers] mailman installation error References: <3A034AC5.4191FD96@bmedesign.com> <5.0.0.25.2.20001106103101.03094c30@vtserf.cc.vt.edu> Message-ID: <3A0723D5.C7398BC7@west.sun.com> Ron Jarrell wrote: > > At 10:35 AM 11/4/00 -0500, Barry A. Warsaw wrote: > > >>>>>> "mt" == mike tran writes: > > > > mt> After I did ./configure --with-cgi-gid=nobody > > mt> --with-mail-gid=mailman and make install. Then I ran > > mt> check_perms. It keep telling me that it cannot find the > > mt> module "paths" in line 19 of check_perms. Any suggestions is > > mt> appricated. I am using RH 6.2 and the latest Apache with > > mt> python 1.5.2 > > > >Look in /home/mailman/bin. If there's no paths.py file in there, your > >installation failed somehow. > > Actually, this probably is a doc problem. I've dealt with this several times on > the users list, and in all but one case (where their python was hosed) it was > people not clearly following the directions. They're supposed to install, > then run check_perms out of ~mailman/bin. Where there's a paths.py(c). > > However, what confuses a lot of people who only skim, is that in > ${srcdir}/mailman/bin there's a check_perms. (The distribution copy). > Which, of course, doesn't have a paths.py installed yet, and thus, doesn't > work, but looks like it ought to, particularly to a complete newcomer who > doesn't understand how things go together. (Heck, it took *me* a couple of > days to figure out whatinhell they were doing wrong.) > > There probably needs to be a mention in the install file to *not* run check_perms > until *after* you install, and to run it from the *installed* base. Or rename the > install copy to check_perms.dist, and install it as check_perms... It's hard to imagine how to make this more clear, though; it *is* step 3, which is numbered, and after the "make install" step, and it *does* say - cd to $prefix - Run bin/check_perms I mean, the next thing is to add things around that point saying "no, really, we really mean exactly what we say here, and not what you think you're reading, so read this carefully, please.." but you'd think that would be obvious from the INSTALL file being the one concise thing you have to read and follow to install Mailman... From jarrell@vt.edu Mon Nov 6 22:27:15 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Mon, 06 Nov 2000 17:27:15 -0500 Subject: [Mailman-Developers] mailman installation error In-Reply-To: <3A0723D5.C7398BC7@west.sun.com> References: <3A034AC5.4191FD96@bmedesign.com> <5.0.0.25.2.20001106103101.03094c30@vtserf.cc.vt.edu> Message-ID: <5.0.0.25.2.20001106172510.0824f330@vtserf.cc.vt.edu> At 01:34 PM 11/6/00 -0800, you wrote: >Ron Jarrell wrote: >> >> At 10:35 AM 11/4/00 -0500, Barry A. Warsaw wrote: >> >> >>>>>> "mt" == mike tran writes: >> > >> > mt> After I did ./configure --with-cgi-gid=nobody >> > mt> --with-mail-gid=mailman and make install. Then I ran >> > mt> check_perms. It keep telling me that it cannot find the >> > mt> module "paths" in line 19 of check_perms. Any suggestions is >> > mt> appricated. I am using RH 6.2 and the latest Apache with >> > mt> python 1.5.2 >> > >> >Look in /home/mailman/bin. If there's no paths.py file in there, your >> >installation failed somehow. >> >> Actually, this probably is a doc problem. I've dealt with this several times on >> the users list, and in all but one case (where their python was hosed) it was >> people not clearly following the directions. They're supposed to install, >> then run check_perms out of ~mailman/bin. Where there's a paths.py(c). >> >> However, what confuses a lot of people who only skim, is that in >> ${srcdir}/mailman/bin there's a check_perms. (The distribution copy). >> Which, of course, doesn't have a paths.py installed yet, and thus, doesn't >> work, but looks like it ought to, particularly to a complete newcomer who >> doesn't understand how things go together. (Heck, it took *me* a couple of >> days to figure out whatinhell they were doing wrong.) >> >> There probably needs to be a mention in the install file to *not* run check_perms >> until *after* you install, and to run it from the *installed* base. Or rename the >> install copy to check_perms.dist, and install it as check_perms... > >It's hard to imagine how to make this more clear, though; it *is* step 3, which >is numbered, and after the "make install" step, and it *does* say > > - cd to $prefix > > - Run bin/check_perms > >I mean, the next thing is to add things around that point saying "no, really, we really >mean exactly what we say here, and not what you think you're reading, so read this >carefully, please.." but you'd think that would be obvious from the INSTALL file >being the one concise thing you have to read and follow to install Mailman... Yea, I know. *I* didn't have have any problems following it the first time, back when I thought python was a big assed snake :-). But given the number of people who've I've talked to that *this* was the problem (it's up to about 10 now) there must be a huge number out there scratching their heads going "this software sucks". These are likely the same people who use the blowdryer in the tub, cause the tag said something about "use blowdryer" and "near water"... :-) From claw@kanga.nu Tue Nov 7 04:55:55 2000 From: claw@kanga.nu (J C Lawrence) Date: Mon, 06 Nov 2000 20:55:55 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Message from Omri Schwarz of "Mon, 06 Nov 2000 14:54:38 EST." <200011061954.OAA23482@hodge-podge.mit.edu> References: <200011061954.OAA23482@hodge-podge.mit.edu> Message-ID: <24458.973572955@kanga.nu> On Mon, 06 Nov 2000 14:54:38 -0500 Omri Schwarz wrote: > The motivation I have behind asking (which can quickly drift > off-topic for this list) is that the main reason behind the > failure of widespread email encryption is human factors. True. More simply, given that most email is of a casual nature, there is little to no return on invested effort for casual users. They fail so see any benefit from crypting or signing their "Funny joke" messages. > Therefore, the right amount of social engineering will be the > driving force in getting people to encrypt email. You first need to create an awareness with them of the problem you wish to solve with encryption. Nobody, and that encludes me, is going to go thru the bother of genning keys, getting them signed, auditing and tracking them, and generally attempting to be responsible here unless I've got some jolly good reason to, unless I've got some problem that going thru all that hassle solves. > If a mailing list exploder like what I described is available, > people will learn not to 1. share TMI type information on any > other kind of mailing list, or 2. share proprietary discussions on > any other kind of mailing list. Uhh, yeah. > So, a list like this will 1. have no Web archiving, 2. no news > gatewaying, and 3. rapidly expiring mailing list keypairs, Just In > Case (TM). This depends on what you are attempting to protect and why. In the case of trade secret protections, web archiving may be a significant plus if you can also audit and control access to those archives (S/Key etc). There is no one model fits all. > I'm asking this on the Mailman forum because Mailman would be > easier to GPG-enable than Majordomo (just as eating ice cream is > more pleasant than root canal..), and because apart from that, I > am not picky on how this should be done, hence would be willing to > fork Mailman to warp it for this end. I'd argue that the crypted list problem is actually orthogonal to the MLM software used. The MLM never needs to be involved. You can involve it if you really want to, but there's not much benefit to doing so. -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From barry@wooz.org Tue Nov 7 04:59:18 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Mon, 6 Nov 2000 23:59:18 -0500 (EST) Subject: [Mailman-Developers] Problem with qrunner and too much incoming mail References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> Message-ID: <14855.35878.37533.497288@anthem.concentric.net> [Removed mailman-users from the recips.] Okay, well that went over like a lead balloon... :) [Aside: it's time to start capturing these notes in a more useful place than the mail archives. I've started a ZWiki page which will serve as "design central" for Mailman. I did this partly to have experience with Wikis, which everybody at my new employer simply raves about. Please visit http://www.zope.org/Members/bwarsaw/MailmanDesignNotes and remember, you too can edit and contribute to these pages! Don't make me the bottleneck. You can learn the little bit you need to know about Wikis from that URL too. Think collaborative web pages.] I completely agree that we want a pluggable architecture for MLM->MTA handoff. We /almost/ have that now with the delivery pipeline, but my mistake was in making the delivery module part of that pipeline. What Mailman's pipeline ought to do is the prep-work on the message only: spam and privacy filtering, setting headers, updating per-list counters, appending to digests, etc. Anything that does not require writing list-specific data could be pulled out of the pipeline. I'm thinking about specifically about nntp posting and the mta-handoff. An API for the handoff is A Good Thing, and of course given that, there's no reason why someone looking for a project couldn't write an external, outgoing-only whizzymailer along the lines I outlined. Based on results that others are getting writing pure-Python servers for other protocols, I think you might be able to get some fairly impressive performance there, especially because this would be outgoing only (it doesn't need to handle any incoming smtp connections). But I definitely didn't envision this to be the /only/ or even the /primary/ way for mail to be delivered, just another option. One of the things that such an approach would give us, is the ability to do more direct bounce detection and handling, eliminating some of the error prone bounce message parsing. E.g. our whizzymailer would know the details of Mailman so when it got errors during the smtp transaction, it could update the db's directly. This isn't as likely to happen when we handoff to a localhost MTA, unless they support DSN and we run them synchronously (which clobbers the current architecture, as we're seeing). Any API we come up with for MLM->MTA handoff should give us the benefits of dsn without the problems. I.e. it should be two-way. Some combination of API and better architecture is probably necessary. >>>>> "JCL" == J C Lawrence writes: JCL> What would be a really good approach without concern for code JCL> impact? I suspect a modified form of the hash tree for queue JCL> storage (cf QMail's implementation minus the silly (for this JCL> use) inode specifics) with a slightly perverted form of your JCL> (Barry's) long running bulkmailer to process that hash queue. Let's flesh that out a little. What does data Qmail hash on? Would the hash tree be in-memory? Would there be any disk persistence in case of system failure? Each message currently has two parts: message content and metadata. Would both be stored in the hash tree? That might get expensive for really big messages. Maybe the message content should be stored in a file and the metadata in the hash tree. Then again, since most messages don't live for very long in the queue, maybe the elimination of the disk i/o is worth a little instability or larger memory footprint. JCL> I'd tend to make the bulkmailer actually an intermittently JCL> running item to help support for intermittently connected JCL> nodes. Say something like: JCL> Cron launches the bulkmailer. The bulkmailer forks N JCL> children processing the queue. The bulkmailer exits upon an JCL> empty queue. Should cron launch a new bulkmailer when the JCL> prevvious incarnation hasn't exited yet, the new instance JCL> merely exits immediately. Forking is pretty heavyweight, and threading has its problems too. One of the things I like about the select-and-continuations-based servers is that for i/o bound tasks, they aren't very difficult to code efficiently. Cron could be used to watchdog the process though. >>>>> "CVR" == Chuq Von Rospach writes: CVR> Instead of reinventing the MTA wheel, I think we're much CVR> better off coming up with an MTA -> MLM interface that's very CVR> flexible and highly configurable (most especially in how to CVR> deliver and how much to parallelize the infeed to the MLM), CVR> and then focus on how to tune the MTA and MLM through CVR> documentation. CVR> Splitting the inbound and outbound queue would be my first CVR> thing here, and probably split bounces into a third CVR> queue. Great idea. Each queue has it's own requirements, e.g. there's definitely been complaints about the minimum 1-minute delay outgoing messages. CVR> That's a pretty quick, easy optimization that makes CVR> sure the end user sees fast response without being bogged CVR> down by deliveries, and that's a huge perception issue. Then CVR> focus on parallelizing the delivery from mailman into the CVR> MTA, and make that configurable so each admin can tune it to CVR> their system and needs. Agreed. I also want that feedback for list-bound messages so that Mailman can be notified directly from the MTA about certain types of delivery failures. I still worry about bottlenecks in synchronous mode, even with a high degree of parallelism and shallow buckets. Thinking out loud: what if the API had two channels, mlm->mta and mta->mlm, let's call them outbound and inbound respectively. The outbound channel needs to contain the message text (or a disk file, ownership of which is passed to the mta), a list of recipients, and an set of metadata to associate with the message. Metadata may include: the list name, the list of error codes to report back to us, a VERP flag, and possibly other opaque data. Incoming is limited only to error reporting, e.g. a list of failed addrs and their error codes, and the metadata reflected back for that message. CVR> If someone wants a rhetoric on how to scale mail list servers CVR> infinitely, I'd be happy to explain how, since I've had to CVR> develop an architecture to do so. If you write it up, I'll add it to the documentation. At the very least, let's add it to the ZWiki. CVR> I think we can build a Mailman that does this, at least for, CVR> oh, 95% of the universe out there, and the other 5% are going CVR> to have custom solutions anyway (or should!). What we don't CVR> want to do is screw up Mailman for the "typical" user to make CVR> it work for the big site; but we also don't want Mailman to CVR> get a reputation as a "small server only" system, because CVR> it'll cause people to reject it in CVR> implementations. Fortunately, I don't think you need to do CVR> that. It just needs some tweaking. Completely agree. CVR> On reasonable hardware, definitely. That's basically how my CVR> current custom system works. right now, the number of CVR> parallel infeeds from mailman is 1. I'm willing to bet the CVR> delivery MTA is basically idle and bored. Have you played at all with the threaded delivery in SMTPDirect? Admittedly it's not integrated correctly with the rest of Mailman, but I'm still curious if the notion is salvagable. -Barry From chuqui@plaidworks.com Tue Nov 7 05:29:01 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 21:29:01 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <24458.973572955@kanga.nu> References: <200011061954.OAA23482@hodge-podge.mit.edu> <24458.973572955@kanga.nu> Message-ID: >I'd argue that the crypted list problem is actually orthogonal to >the MLM software used. The MLM never needs to be involved. You can >involve it if you really want to, but there's not much benefit to >doing so. Except in the case of authentification of the user. It's a critical need there -- I'd kill for the ability to be able to verify subscribers via S/MIME or some other non-spoofable form. then they could post from anywhere, as long as their authentification worked. This general ability - to validate an incoming e-mail, not just for MLM -- would be a killer app for the anti-spam folks.Anything without a valid signature, you dump. But, ask me to add this support to something like Mailman, and I'll say no. Why? Because until the clients support it cleanly and easily and it's on its way towards general acceptance in the user base, it's wasted effort. Right now, encryption is way too nerdy, too complicated, and the typical user doens't see the need. Spend time writing support into a MLM, and it won't be used. Try to make it mandator, and you'll kill the MLM. If the List-* headers are something the MLM's have to lead the pack in, encryption has to be led out of the mail clients, because that's where 95% of the utility will be. any encrpytion support in the MLM proper would be as an add-on to the support in the client, and it makes no sense to do it until the clients do it (and set the standard for interfacing to it). An d that's a long way off from what I see. When I can reliably expect to find (and use) an embedded digital signature on an e-mail, then it's time to look at adding it to Mailman. Until then -- if you want encryption, go lobby the mail clients to add the functionaliy and make it usable and easy enough that people actually use it. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From ocschwar@MIT.EDU Tue Nov 7 05:54:41 2000 From: ocschwar@MIT.EDU (Omri Schwarz) Date: Tue, 07 Nov 2000 00:54:41 -0500 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Your message of "Mon, 06 Nov 2000 20:55:55 PST." <24458.973572955@kanga.nu> Message-ID: <200011070554.AAA25089@hodge-podge.mit.edu> > On Mon, 06 Nov 2000 14:54:38 -0500 > Omri Schwarz wrote: > > > I'm asking this on the Mailman forum because Mailman would be > > easier to GPG-enable than Majordomo (just as eating ice cream is > > more pleasant than root canal..), and because apart from that, I > > am not picky on how this should be done, hence would be willing to > > fork Mailman to warp it for this end. > > I'd argue that the crypted list problem is actually orthogonal to > the MLM software used. The MLM never needs to be involved. You can > involve it if you really want to, but there's not much benefit to > doing so. Both your solution and mine do the same thing on the human failings angle: they allow a mail server admin to set up a list that does encryption for everyone, so that people learn that some things are best not discussed in plaintext. (Said mail server admin, now enabled with this solution, can also go fascist on people who don't comply with said policy. Enough mail server admins deciding to go this route, and you may see an effort on the part of users and MUA writers to improve things on that end.) As to which solution is better on the software side, you're probably right. I'll have to have a closer look at the Mailman code. But you're sort of committed to something, as am I, so there's hope. Now, for Mr. Von Rospach: >This general ability - to validate an incoming e-mail, not just for >MLM -- would be a killer app for the anti-spam folks.Anything without >a valid signature, you dump. >But, ask me to add this support to something like Mailman, and I'll >say no. Why? Because until the clients support it cleanly and easily >and it's on its way towards general acceptance in the user base, it's >wasted effort. GPG version chauvinism is a must for such a project. PGP-GPG and intra-PGP version incompatibilities are a pain. In turn, that kills the MUAs. However, I don't believe good GPG handling in the MUAs is the necessary-and-sufficient part to bring this about. (My likely-wrong opinion here.) So, to summarize: Python-GPG interface exists somewhere, and not much happening in the Mail server or MLM group software side. So the niche is here and I'm ready to give it a try. Thank you for your attention, y'all. -- Omri Schwarz Some people have told me they don't think a fat penguin really embodies the grace of Linux, which just tells me they have never seen a angry penguin charging at them in excess of 100mph. They'd be a lot more careful about what they say if they had. -- Linus Torvalds From chuqui@plaidworks.com Tue Nov 7 07:00:32 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 23:00:32 -0800 Subject: [Mailman-Developers] Problem with qrunner and too much incoming mail In-Reply-To: <14855.35878.37533.497288@anthem.concentric.net> References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <14855.35878.37533.497288@anthem.concentric.net> Message-ID: At 11:59 PM -0500 11/6/00, Barry A. Warsaw wrote: >experience with Wikis, which everybody at my new employer simply raves >about. Please visit > > http://www.zope.org/Members/bwarsaw/MailmanDesignNotes got that bookmarked. Looks interesting. >mistake was in making the delivery module part of that pipeline. What >Mailman's pipeline ought to do is the prep-work on the message only: That's pretty much what I do now on my big mother machine. There's a web page for posting, and it spawns a script that creates a message file (with full headers already finished), then sucks the subscriber list out of an SQL database, generates a set of commandfiles and subscriber lists, and throws them in a queueing system. it's configurable on the fly (FWIW, I use QPS for queueing, I didn't write one. Gnu Queue was my first choice, but it has problems on solaris I didn't have time to debug...) >spam and privacy filtering, setting headers, updating per-list >counters, appending to digests, etc. Anything that does not require >writing list-specific data could be pulled out of the pipeline. I'm >thinking about specifically about nntp posting and the mta-handoff. I'm beginning to think that "mailman 3.0" may end up being NOTHING but APIs and enough glue to interface them. that way, *any* piece can be swapped out for something equivalent if we want to, and we can strongly isolate interactions and keep each code-base simple. A subscription API, a spam API, a digest API, etc, etc, etc. It's not at all that simple in practive, but in theory, you ultimately have a set of Python classes that define a MLM... > E.g. our whizzymailer would >know the details of Mailman so when it got errors during the smtp >transaction, it could update the db's directly. This isn't as likely >to happen when we handoff to a localhost MTA, unless they support DSN >and we run them synchronously (which clobbers the current >architecture, as we're seeing). this is nice in theory, but again, you start BEING an MTA, and the subtleties are going to whack you up side the head. I looked at this a while back, and decided that was a place I didn't want to go. I really think we need to be careful about optimization by subverting the MTA, because down that path lies sendmail -- which does everything known to mankind, but nobody can figure out the documentation... Let the MTA be an MTA, and simply hand stuff off and process the returns. That gives you the ability to build tools that leverage other people's strength's, whether it's Postfix or smartbounce. Otherwise, you're asking for a LOT of work that you don't think you're going to need to do yet. I didn't realize that until I *did* start designing sendmail out of my system and saw the results - slower, uglier and I got to maintain the code myself... > Would there be any disk persistence in >case of system failure? There has to be, but one thing sendmail did with 8.10 to boost performance was to figure out what needed to be on disk for recovery, and keeping the rest in memory. For their purposes, that was the df and qf files, and they do all of the locking and status files via what's effectively an in-sendmail RAM-based pseudo disk for all of the other files, like lock files. so stuff relevant to the current delivery attempt is in RAM, the stuff you need ot decide whether ot deliver or how to deliver is on disk. In a crash, you only lose data that's not relevant after a crash. the big problem I see on sendmail 8.9 is inode locking, especially when it's updating the /var/spool/mqueue directory inode. sendmail 8.10 goes a long way towards fixing that with the /var/spool/mqueue/* setup -- you can imagine the fun of 400 sendmails all trying to update their queue files in the same directory inode. (wince). So before we get into fancy hashing systems, let's see how we do with the basics -- split in/out/bounce into separate qfiles, split content/metadata/status/lock into separate subdirectories, and if necessary, allow multiple directories to further split the directory contention. In fact, a really simply way to parallelize Mailman would be to allow multiple qfiles, and every time qrunner is spawned from cron, it creates one instance per directory. that way, you distribute the load out evenly and can rearrange it as you need by adding or removing directories. No funky config file issues. >Then again, since most messages don't live for very long in the queue, >maybe the elimination of the disk i/o is worth a little instability or >larger memory footprint. Before you make that decision, we need to know whether the I/O is actually significant, and which pieces of the I/O can safely be held off. But in reality, I'll bet you won't find a Mailman site where the mailman directories are I/O bound in an significant way. We shouldn't try to optimize things that aren't the bottlenecks.... >Forking is pretty heavyweight, and threading has its problems too. But for what we're talking about, the fork overhead is pretty trivial. Forking is bad for lots of little, short-lived things. Forking is good for relatively few, long-lived things. Given what the processing cost of delivering 500 pieces of email will be, the overhead of the fork is non-existant. If we were forking a process per address, I'd worry about it. Forking a process per message isn't. Reality is somewhere in the middle -- but the trick is to find the slow parts and speed them up, rather than just try to speed up various things we guess might be slow. > CVR> Splitting the inbound and outbound queue would be my first > CVR> thing here, and probably split bounces into a third > CVR> queue. > >Great idea. Each queue has it's own requirements, e.g. there's >definitely been complaints about the minimum 1-minute delay outgoing >messages. the outbound queue is a perfect place for a daemon to sit, and make sure there are always up to "N" messages being processed (we might want to amend that so only one message is in process at a time for a given list. Hmm, does it make sense to split the outqueue into subdirs (see above) by list name? the outqueue daemon could then round-robin the lists, to prevent a busy list from stuffing the other lists into a corner... >Agreed. I also want that feedback for list-bound messages so that >Mailman can be notified directly from the MTA about certain types of >delivery failures. I wouldn't worry about this. the programming complexity makes this a false economy. It sounds nice in theory, but I wouldn't make it a design goal until we get other stuff in place -- if then. Bounces are a pain in the neck, but not that nasty to deal with, and the places where simply background processing bounces falls down, this isn't really likely to help, because it's the guy three forwards away from the subscribed address, behind a firewall, on a Notes server, who changed his name when he got married four months ago... What you're really proposing, Barry, is to have to implement TWO bounce processing systems. One for stuff where the delivery attempt fails locally, and a second one for stuff where the mail is delivered to an agent that then sends a reject back. and that latter includes all of the major ISPs (especially AOL and MSN), most major corporations (including Apple), and basically every large site with firewalls and mail relays through them. So you're doubling your work writing bounce processing code, and it buys you very, very little. And the real trouble cases won't be helped at all, because they'r the ones that won't get nailed until they come back through those 4 relays with the addresses munched and headers stripped. > I still worry about bottlenecks in synchronous >mode, even with a high degree of parallelism and shallow buckets. From the point of view of Mailman, I doubt it's really an issue. I think you're still thinking MTA here. Mailman is NOT an MTA. You don't want to write an MTA. you don't want to think like an MTA writer. (see this swinging watch? you are getting sleepy, sleepy... you do not want to write sendmail. you do not want to integrate sendmail into Mailman. you are not eric allman. sendmail.cf files give you hives...) >Thinking out loud: what if the API had two channels, mlm->mta and >mta->mlm, let's call them outbound and inbound respectively. I'd do three APIs, actually. DeliverMail, IncomingMail, BounceMail. they might hand off to each other (forinstance IncomingMail would recognize a bounce, and forward it ot BounceMail), but they're really independent operations. I worry that trying to build a single API would start throwing in compromises or overloading concepts. And iwth three APIs, they can be developed independently by different people -- and swapped out independntly. With a single API, that's tougher. > CVR> If someone wants a rhetoric on how to scale mail list servers > CVR> infinitely, I'd be happy to explain how, since I've had to > CVR> develop an architecture to do so. > >If you write it up, I'll add it to the documentation. At the very >least, let's add it to the ZWiki. Will do, once I get a chance. I've put it on my todo list. >Have you played at all with the threaded delivery in SMTPDirect? >Admittedly it's not integrated correctly with the rest of Mailman, but >I'm still curious if the notion is salvagable. A little. I really think the fork model needs to be used, because the thread locks don't seem to allow enough processing independence. The delivery stuff is going to spend most of its time in I/O wait with kernel locks, and needs to respond quickly when the I/O is available. I'm afraid that using threads for this defeats that purpose, because the block has to be reported to Python, which then has to get around to activating that thread, and by the time you do, you've lost the performance edge, especially when you're talking about a number of these fighting for the single CPU resource. This is a case where the fork overhead is trivial compared to the job overhead, and you really want these independent and down in the kernel, since you're dealing with stuff the kernel is best suited to resolve. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From chuqui@plaidworks.com Tue Nov 7 07:07:40 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 23:07:40 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <200011070554.AAA25089@hodge-podge.mit.edu> References: <200011070554.AAA25089@hodge-podge.mit.edu> Message-ID: At 12:54 AM -0500 11/7/00, Omri Schwarz wrote: >Both your solution and mine do the same thing on the human >failings angle: they allow a mail server admin to set up a list >that does encryption for everyone, so that people learn that >some things are best not discussed in plaintext. no, it really doesn't, because the message is sent to the MLM in plaintext, so it has no security at all. If you depend on the MLM to do the encryption, you might as well not encrypt, bceause anyone sniffing packets will have the data no proble. what you're doing is setting up a sense of *false* security, but you're in fact leaving things wide open. It has to be encrypted leaving the client, or it's not secure. >GPG version chauvinism is a must for such a project. why? you want encryption endemic. Which implies abiliy to handle anyone's public key and do something reasonable with it, not just one. Otherwise, you're balkanized, and that defeats the purpose again. >In turn, that kills the MUAs. However, >I don't believe good GPG handling in the MUAs >is the necessary-and-sufficient part to bring this about. If the MUAs don't support encryption, then how will users decrypt something the MLM encrypted? And if the MUA does support encryption -- the MLM doens't have to. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From ocschwar@MIT.EDU Tue Nov 7 07:13:39 2000 From: ocschwar@MIT.EDU (Omri Schwarz) Date: Tue, 07 Nov 2000 02:13:39 -0500 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Your message of "Mon, 06 Nov 2000 23:07:40 PST." Message-ID: <200011070713.CAA25343@hodge-podge.mit.edu> > At 12:54 AM -0500 11/7/00, Omri Schwarz wrote: > > >Both your solution and mine do the same thing on the human > >failings angle: they allow a mail server admin to set up a list > >that does encryption for everyone, so that people learn that > >some things are best not discussed in plaintext. > > no, it really doesn't, because the message is sent to the MLM in > plaintext, so it has no security at all. If you depend on the MLM to > do the encryption, you might as well not encrypt, bceause anyone > sniffing packets will have the data no proble. what you're doing is > setting up a sense of *false* security, but you're in fact leaving > things wide open. It has to be encrypted leaving the client, or it's > not secure. Unless I misunderstood, in both cases a program on the server decripts incoming mail and then re-encrypts, but that in once case the Sendmail/Qmail program does this while I want the MLM to do it. Setting up an encription-required rule for a list should be easy in either case. > >GPG version chauvinism is a must for such a project. > > why? you want encryption endemic. Which implies abiliy to handle > anyone's public key and do something reasonable with it, not just > one. Otherwise, you're balkanized, and that defeats the purpose again. > > >In turn, that kills the MUAs. However, > >I don't believe good GPG handling in the MUAs > >is the necessary-and-sufficient part to bring this about. > > If the MUAs don't support encryption, then how will users decrypt > something the MLM encrypted? And if the MUA does support encryption > -- the MLM doens't have to. > MUAs that support encryption do exist. Unfortunately, they cater mostly to Unix gurus. From chuqui@plaidworks.com Tue Nov 7 07:17:22 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 23:17:22 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <200011070713.CAA25343@hodge-podge.mit.edu> References: <200011070713.CAA25343@hodge-podge.mit.edu> Message-ID: At 2:13 AM -0500 11/7/00, Omri Schwarz wrote: > >MUAs that support encryption do exist. >Unfortunately, they cater mostly to Unix gurus. Until you can convince the general public to encrypt mail, then you'll never get real encryption support in systems (much as we need it)... right now, encryption stuff sings only to the choir, not the church members.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From ocschwar@MIT.EDU Tue Nov 7 07:38:40 2000 From: ocschwar@MIT.EDU (Omri Schwarz) Date: Tue, 07 Nov 2000 02:38:40 -0500 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Your message of "Mon, 06 Nov 2000 23:17:22 PST." Message-ID: <200011070738.CAA25453@hodge-podge.mit.edu> > At 2:13 AM -0500 11/7/00, Omri Schwarz wrote: > > > >MUAs that support encryption do exist. > >Unfortunately, they cater mostly to Unix gurus. > > Until you can convince the general public to encrypt mail, then > you'll never get real encryption support in systems (much as we need > it)... right now, encryption stuff sings only to the choir, not the > church members.... We'll restart this discussion when and if I have something more than vapor to show y'all. From Dan.Mick@west.sun.com Tue Nov 7 20:21:57 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Tue, 07 Nov 2000 12:21:57 -0800 Subject: [Mailman-Developers] Problem with qrunner and too much incoming mail References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <14855.35878.37533.497288@anthem.concentric.net> Message-ID: <3A086465.A57EEA92@west.sun.com> > CVR> On reasonable hardware, definitely. That's basically how my > CVR> current custom system works. right now, the number of > CVR> parallel infeeds from mailman is 1. I'm willing to bet the > CVR> delivery MTA is basically idle and bored. > > Have you played at all with the threaded delivery in SMTPDirect? > Admittedly it's not integrated correctly with the rest of Mailman, but > I'm still curious if the notion is salvagable. I also really wonder whether anyone besides me and the author of the patch are using the "sort by domain and chunkify" patch to SMTPDirect, too. I've been using it for six months and I think I like it; it's a way of distributing the load a bit: do the obvious-easy-simple stuff and let the MTA do the rest. Wonder if it would make a difference for those with enough mail to be hitting bottlenecks? From barry@wooz.org Wed Nov 8 19:42:34 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Wed, 8 Nov 2000 14:42:34 -0500 (EST) Subject: [Mailman-Developers] Mailman and GPG. References: <200011052255.RAA14775@alice-whacker.mit.edu> Message-ID: <14857.44202.507836.574952@anthem.concentric.net> Omri> I may be itnerested in doing this as an exercise to learn Omri> Python. I'm not one to discourage anybody's motivation for learning Python, so go for it. I think Chuq and JC have outlined the concerns and issues nicely, so I won't add much else other than to say that I /hope/ you'd be able to add the bits you'd need without forking Mailman. There should be enough pluggable architecture to add what you need as an extension without major architectural changes. If not, then post your ideas here and we can discuss them. Having said that, my own personal opinion is that this isn't very useful for a general tool. IMO, encryption and security will never passed the Grandma Test: could it be easy enough to understand and do correctly that your grandmother would use it? Encryption is complicated and security protocols are even harder to understand and implement right, let alone do so in a way that Normal People can grasp. I'm pessimistic that a pervasively encrypted environment will ever really exist, or will even be deemed necessary by the fast armies of grandmas on the 'net. (unlike web and email, which my son's grandma can do without much trouble) Let us know how it goes though. It definitely tickles my geek-libertarian funny bone! :) -Barry From chuqui@plaidworks.com Wed Nov 8 20:05:53 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 8 Nov 2000 12:05:53 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <14857.44202.507836.574952@anthem.concentric.net> References: <200011052255.RAA14775@alice-whacker.mit.edu> <14857.44202.507836.574952@anthem.concentric.net> Message-ID: At 2:42 PM -0500 11/8/00, Barry A. Warsaw wrote: >IMO, encryption and security will never >passed the Grandma Test: could it be easy enough to understand and do >correctly that your grandmother would use it? this is a place I disagree with Barry -- I think it will, but not until it's as easy to use as the web and email is. Which means the client authors need to decide it's necessary to integrate into the client tools, and serious enough about it to integrate in a non-geeky way (i.e., Eudora's PGP plug-ins ain't it). That means serious user interface design and integration. And I think it will -- but not soon. Why? the U.S. recently made digital signatures legally binding. that means encryption. And once people start needing (or wanting) digital signatures, that'll drive the integration of encryption. Until that happens, though, it'll continue to be a niche technology. A crucial one, but not one you can easily explain to mom and dad. This is a job for the client tools. It can be and should be done. But I don't see it happening soon, and I see government's globally fighting it every step of the way... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From jam@jamux.com Wed Nov 8 21:40:29 2000 From: jam@jamux.com (John A. Martin) Date: Wed, 08 Nov 2000 16:40:29 -0500 Subject: [Mailman-Developers] rc1 Kinky header on welcome message Message-ID: <20001108214029.467A248031@athene.jamux.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Using Mailman version 2.0rc1, the "confirmation of subscription -- request" and the first "welcome" message for a list contain the following header fields (others fields OK) - -------------- cut here ---->8 ---< head List-Subscribe: <../../listinfo/test-2>, List-Id: List-Unsubscribe: <../../listinfo/test-2>, List-Archive: - ---- 8<------- cut here ----------> tail while other mails, including welcome messages for subsequent subscribers seem OK as follows. - -------------- cut here ---->8 ---< head List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: - ---- 8<------- cut here ----------> tail My python skills are too infantile to see where it is that something seems to fail to get initialized. jam -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: OpenPGP encrypted mail preferred. See iEYEARECAAYFAjoJyDEACgkQUEvv1b/iXy8slQCdEwN7Hc151sdaM2RG1UrDYqyb 1HkAoJu0lvuR/eqiTQp9sMWVY81wo6cp =Z5x0 -----END PGP SIGNATURE----- From barry@wooz.org Wed Nov 8 22:21:42 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Wed, 8 Nov 2000 17:21:42 -0500 (EST) Subject: [Mailman-Developers] rc1 Kinky header on welcome message References: <20001108214029.467A248031@athene.jamux.com> Message-ID: <14857.53750.617120.581090@anthem.concentric.net> I believe this is fixed in rc2 (coming real soon now). From jarrell@vt.edu Wed Nov 8 23:22:42 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Wed, 08 Nov 2000 18:22:42 -0500 Subject: [Mailman-Developers] listinfo page link problem Message-ID: <5.0.0.25.2.20001108182108.04bd8c30@vtserf.cc.vt.edu> Interesting... Anyone else seeing this? On my nodename/mailman/listinfo page, where it says "List admins go the the administrative interface" that link points at nodename/admin Not mailman/admin... I'm on my way out the door, so I can't look at the code to see what's up, but I thought I'd mention it because otherwise I'll forget :-). From Dan Mick Thu Nov 9 00:04:32 2000 From: Dan Mick (Dan Mick) Date: Wed, 8 Nov 2000 16:04:32 -0800 (PST) Subject: [Mailman-Developers] listinfo page link problem Message-ID: <200011090003.QAA04094@utopia.west.sun.com> > Interesting... Anyone else seeing this? Not me. > On my > > nodename/mailman/listinfo > > page, where it says "List admins go the the administrative interface" that link points at > nodename/admin > > Not mailman/admin... Is your DEFAULT_URL set with a trailing /? From barry@wooz.org Thu Nov 9 01:59:27 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Wed, 8 Nov 2000 20:59:27 -0500 (EST) Subject: [Mailman-Developers] rc1 Possible confusion in Defaults.py comments References: <20001106133307.349FE48031@athene.jamux.com> Message-ID: <14858.1279.817235.627843@anthem.concentric.net> >>>>> "jam" == John A Martin writes: jam> Using Mailman version 2.0rc1, in Defaults.py, under Bounce jam> processing defaults, one sees: | # 0 means do nothing | # 1 means disable and send admin a report, | # 2 means nuke'em (remove) and send admin a report, | # 3 means nuke 'em and don't report (whee:) | DEFAULT_AUTOMATIC_BOUNCE_ACTION = 1 jam> and on the Bounce Options web page: jam> (Ie. from left to right, (0) do nothing, (1) disable and jam> notify, (2) disable and DON'T notify, and (3) Remove and jam> notify. Here's what the actual code does (see HandleBouncingAddress() in Mailiman/Bouncer.py): 0 = do nothing 1 = disable and notify 2 = disable and don't notify 3 = remove and notify So it looks like the web page is right but the comment in Defaults.py.in are wrong. I'll fix those. -Barry From barry@wooz.org Thu Nov 9 02:29:32 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Wed, 8 Nov 2000 21:29:32 -0500 (EST) Subject: [Mailman-Developers] Slight oddity on ubsubscription References: Message-ID: <14858.3084.417234.540058@anthem.concentric.net> >>>>> "NM" == Nigel Metheringham writes: NM> Here's a message I got from one of my subscribers - makes a NM> reasonable point [I have edited the address in there to NM> prevent it being exposed]. | So, I see that an old form of my address is subscribed to the | announce list. I look for a way to change the address, don't | see it, so unsubscribe. | The last four words there are a link; following that brings me | to a page stating: | exim-announce: No such member 'removed@to.protect.guilty'. | Perhaps someone who's unsubscribed shouldn't be prompted to | continue editting? I agree, this is bogus. I've filed a bug report on it, but probably will not fix it for 2.0. -Barry From jarrell@vt.edu Thu Nov 9 17:55:50 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Thu, 09 Nov 2000 12:55:50 -0500 Subject: [Mailman-Developers] listinfo page link problem In-Reply-To: <200011090003.QAA04094@utopia.west.sun.com> Message-ID: <5.0.0.25.2.20001109125510.04b40790@vtserf.cc.vt.edu> At 04:04 PM 11/8/00 -0800, Dan Mick wrote: >> page, where it says "List admins go the the administrative interface" that >link points at >> nodename/admin >> >> Not mailman/admin... > >Is your DEFAULT_URL set with a trailing /? *Thwack*. Well... Feh. Apparently I fixed that on all machines except, of course, *that one*... Sigh. From mrlist@ActiveState.com Thu Nov 9 18:09:49 2000 From: mrlist@ActiveState.com (Eric Wang) Date: Thu, 09 Nov 2000 10:09:49 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions In-Reply-To: <3A06DD6C.D4F23706@gietl.com> References: <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> Message-ID: <20001109100851.E230.MRLIST@activestate.com> Has this problem been solved ? If so , please tell me how, thanks! On Mon, 06 Nov 2000 17:33:48 +0100 Andreas Gietl wrote: > any further ideas? > > i'm sorry, but there is no pipeline entry: > > that's the db-file: > { 'filebase': 'ccfe93988bf681ad271ad049fa54dcd7260a60ac', > 'listname': 'bittecomgietltest2', > 'torequest': 1, > 'version': 2} > > and this is the .msg file: > > >From gietl@gietl.com Sat Nov 04 13:52:46 2000 > Received: from [195.227.84.2] (helo=d1.x-mailer.de) > by d18.x-mailer.de with esmtp (Exim 3.16 #1) > id 13s2oQ-0003e1-00 > for bittecomgietltest2-request@bitte.com; Sat, 04 Nov 2000 > 13:52:46 +0100 > Received: from p3ee039ab.dip.t-dialin.net ([62.224.57.171] > helo=gietl.com) > by d1.x-mailer.de with esmtp (Exim 3.16 #1) > id 13s2n9-0004iW-00 > for bittecomgietltest2-request@bitte.com; Sat, 04 Nov 2000 > 13:51:27 +0100 > Message-ID: <3A0406EC.6C4B8583@gietl.com> > Date: Sat, 04 Nov 2000 13:54:04 +0100 > From: Andreas Gietl > X-Mailer: Mozilla 4.73 [en] (WinNT; I) > X-Accept-Language: en > MIME-Version: 1.0 > To: bittecomgietltest2-request@bitte.com > Subject: Re: Bittecomgietltest2 -- confirmation of subscription -- > request 569170 > References: > Content-Type: text/plain; charset=iso-8859-1 > Content-Transfer-Encoding: 8bit > > > > bittecomgietltest2-request@bitte.com wrote: > > > > Bittecomgietltest2 -- confirmation of subscription -- request 569170 > > > > We have received a request from p3ee039ab.dip.t-dialin.net for > > subscription of your email address, , to the > > bittecomgietltest2@bitte.com mailing list. To confirm the request, > > please send a message to bittecomgietltest2-request@bitte.com, and > > either: > > > > - maintain the subject line as is (the reply's additional "Re:" is > > ok), > > > > - or include the following line - and only the following line - in the > > message body: > > > > confirm 569170 > > > > (Simply sending a 'reply' to this message should work from most email > > interfaces, since that usually leaves the subject line in the right > > form.) > > > > If you do not wish to subscribe to this list, please simply disregard > > this message. Send questions to bittecomgietltest2-admin@bitte.com. > > barry@wooz.org wrote: > > > > >>>>> "AG" == Andreas Gietl writes: > > > > >> Do you still see the > > >> confirmation message in qfiles? > > > > AG> Yeah, the confirmation message is in there! > > > > >> If so, are there any obvious errors > > >> in logs/error? > > > > AG> nothing. i of course checked that! > > > > >> Does it have the expected Subject: header (i.e. at the > > >> end of the line, "request xxxxxx" where those x's are the > > >> numbers). > > > > AG> Yeah. it does have. > > AG> Do you need any further information? > > > > Yes, do a bin/dumpdb qfiles/.db and look for the > > `pipeline' entry. > > > > That should show us where things are getting stuck. > > -Barry > > > > ------------------------------------------------------ > > Mailman-Users maillist - Mailman-Users@python.org > > http://www.python.org/mailman/listinfo/mailman-users > > -- > andreas gietl > gietl internet services > fon +49 9402 2551 > fax +49 9402 2604 > mobile +49 171 60 70 008 > gietl@gietl.com > > ############################################ > # Das Handbuch sagt, das Programm ben~{vt~}ige # > # Windows 95 oder besser. Also habe ich # > # Linux installiert! # > ############################################ > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users From barry@wooz.org Thu Nov 9 19:13:30 2000 From: barry@wooz.org (barry@wooz.org) Date: Thu, 9 Nov 2000 14:13:30 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> Message-ID: <14858.63322.557304.241298@anthem.concentric.net> Trying to come back to this problem. It looks like Eric Wang is also seeing problems, so there must be /something/ going on. I can't figure out what though, so perhaps one of you guys can give me ssh access to the host in question? >>>>> "AG" == Andreas Gietl writes: AG> i'm sorry, but there is no pipeline entry: | that's the db-file: | { 'filebase': 'ccfe93988bf681ad271ad049fa54dcd7260a60ac', | 'listname': 'bittecomgietltest2', | 'torequest': 1, | 'version': 2} Okay, this looks pretty good, assuming the file is qfiles/ccfe93988bf681ad271ad049fa54dcd7260a60ac.db We don't get a `pipeline' key in the dictionary because, by virtue of the `torequest' key, qrunner sends this message through the email command parser instead of the delivery pipeline. So far so good. AG> and this is the .msg file: [...most headers deleted...] | To: bittecomgietltest2-request@bitte.com | Subject: Re: Bittecomgietltest2 -- confirmation of subscription -- | request 569170 | References: | Content-Type: text/plain; charset=iso-8859-1 | Content-Transfer-Encoding: 8bit I notice that the Subject: line is wrapped, but not in an rfc 822 valid way. I'm assuming that something got munged in the forwarding of the message to this list, and that the text `request 569170' is actually at the end of the Subject: line. In my own tests, if the message looked exactly as you sent it, with the line being wrapped and `request' showing in column zero, the mailcmd parser will generate an error message back to the sender. But you're not seeing that. If the `request' line were wrapped a la rfc 822, Mailman should be able to grok out the command just fine. But you're not seeing that either. Another log file to look at: doe you see anything in logs/bounce? I don't expect you to -- just wondering. We're at the point of using debug statements now, and I don't want to bore mailman-users with that. If I can't get ssh access to your installation, let's carry on the debugging conversation on mailman-developers only. -Barry From chuqui@plaidworks.com Thu Nov 9 19:38:02 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Thu, 9 Nov 2000 11:38:02 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions In-Reply-To: <14858.63322.557304.241298@anthem.concentric.net> References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> Message-ID: At 2:13 PM -0500 11/9/00, barry@wooz.org wrote: > | Subject: Re: Bittecomgietltest2 -- confirmation of subscription -- > | request 569170 I had one of these this week. the Subject line was wrapped 'request\n\t' and mailman didn't handle it. To my knowledge subject should never wrap anyway, and when I looked in my inbox archives, it was the users mail client that had done it. It looked like a client issue, but it gave us the clue to help him work around it. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From dans@audifans.com Thu Nov 9 19:51:20 2000 From: dans@audifans.com (Dan Simoes) Date: Thu, 09 Nov 2000 14:51:20 -0500 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> Message-ID: <3A0B0038.CD725B31@audifans.com> barry@wooz.org wrote: > > Trying to come back to this problem. It looks like Eric Wang is also > seeing problems, so there must be /something/ going on. I can't > figure out what though, so perhaps one of you guys can give me ssh > access to the host in question? I've also reported the same problems. After much woe, I am back at beta6, which seems to have fixed my web interface problems and does not present the confirmation issue. I would love to help, but at this point I have over 2000 pissed off users, and I can't risk another upgrade to test. I may do so in the future, but only if I have a solid way of backing out. I am also strongly considering a return to majordomo. Thanks, | Dan | From barry@wooz.org Thu Nov 9 20:38:20 2000 From: barry@wooz.org (barry@wooz.org) Date: Thu, 9 Nov 2000 15:38:20 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> <3A0B0038.CD725B31@audifans.com> Message-ID: <14859.2876.127155.739471@anthem.concentric.net> >>>>> "DS" == Dan Simoes writes: >> Trying to come back to this problem. It looks like Eric Wang >> is also seeing problems, so there must be /something/ going on. >> I can't figure out what though, so perhaps one of you guys can >> give me ssh access to the host in question? DS> I've also reported the same problems. After much woe, I am DS> back at beta6, which seems to have fixed my web interface DS> problems and does not present the confirmation issue. We tracked Andreas' problem down to the qrunner cron job not being installed, a common problem for upgraders from what I've seen. Please first be absolutely sure this isn't your problem! You can test this by cd'ing into /home/mailman and running % python -S cron/qrunner if that unclogs things, your cron isn't set up right. If not, then we'll need to look at things more closely. To start with, you can send me the output of "bin/dumpdb qfiles/whatever.db" and the associated .msg file. Also, look in the log files, especially logs/error and see if there's anything relevant. DS> I would love to help, but at this point I have over 2000 DS> pissed off users, and I can't risk another upgrade to test. I DS> may do so in the future, but only if I have a solid way of DS> backing out. Well, you may have to set up a test list in a different installation until you're confident that 2.0rc1 works for you. DS> I am also strongly considering a return to majordomo. I hope that won't be necessary, and I'm willing to help debug the problem. -Barry From barry@wooz.org Thu Nov 9 20:42:49 2000 From: barry@wooz.org (barry@wooz.org) Date: Thu, 9 Nov 2000 15:42:49 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> Message-ID: <14859.3145.597151.979023@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> I had one of these this week. the Subject line was wrapped CVR> 'request\n\t' and mailman didn't handle it. To my CVR> knowledge subject should never wrap anyway, and when I looked CVR> in my inbox archives, it was the users mail client that had CVR> done it. It looked like a client issue, but it gave us the CVR> clue to help him work around it. Hmm. I just tested this (wrapping before and after `request' with \n\t) and it works for me. The rfc822 module /should/ be able to handle this and return a Subject with the continuation lines. -Barry From Dan.Mick@west.sun.com Thu Nov 9 21:10:16 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Thu, 09 Nov 2000 13:10:16 -0800 Subject: [Mailman-Developers] listinfo page link problem References: <5.0.0.25.2.20001109125510.04b40790@vtserf.cc.vt.edu> Message-ID: <3A0B12B8.97E3B928@west.sun.com> Ron Jarrell wrote: > > At 04:04 PM 11/8/00 -0800, Dan Mick wrote: > >> page, where it says "List admins go the the administrative interface" that > >link points at > >> nodename/admin > >> > >> Not mailman/admin... > > > >Is your DEFAULT_URL set with a trailing /? > > *Thwack*. Well... Feh. Apparently I fixed that on all machines except, of course, *that one*... > Sigh. Barry, might this be worth checking for, and warning about in error or somewhere? From Dan.Mick@west.sun.com Thu Nov 9 21:24:53 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Thu, 09 Nov 2000 13:24:53 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> <3A0B0038.CD725B31@audifans.com> <14859.2876.127155.739471@anthem.concentric.net> Message-ID: <3A0B1625.A270241@west.sun.com> barry@wooz.org wrote: > We tracked Andreas' problem down to the qrunner cron job not being > installed Well, there's a new one. Not mentioned anywhere in the installation docs, either. Glad we had to get Barry to debug this one. From vikas98@hotmail.com Fri Nov 10 07:54:34 2000 From: vikas98@hotmail.com (Vikas Gupta) Date: Fri, 10 Nov 2000 07:54:34 GMT Subject: [Mailman-Developers] [please heklp!!]cannot list public lists from listinfo , can from admin Message-ID: I know this has been asked on this list before but I think I have a new twist which I cant figure out. I have created a public list, which is listed fine under /admin, but does not appear under /listinfo. I get the : "There currently are no publicly-advertised mailman mailing lists" message. I believe I have folloed the INSTALL instrctions correctly for mailman and apache. Can anyone help me out? Thanks in advance, Vikas _________________________________________________________________________ Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com. Share information about yourself, create your own public profile at http://profiles.msn.com. From Nigel.Metheringham@VData.co.uk Fri Nov 10 10:07:35 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Fri, 10 Nov 2000 10:07:35 +0000 Subject: [Mailman-Developers] [please heklp!!]cannot list public lists from listinfo , can from admin In-Reply-To: Message from "Vikas Gupta" of "Fri, 10 Nov 2000 07:54:34 GMT." Message-ID: [I think this is a mailman-users issue - *please* can we try and get the list splits appropriate - mailman-developers is not just a means of getting more clued (or less low-clued) responses for a problem because you think its urgent] vikas98@hotmail.com said: > I know this has been asked on this list before but I think I have a > new twist which I cant figure out. > I have created a public list, which is listed fine under /admin, but > does not appear under /listinfo. I get the : > "There currently are no publicly-advertised mailman mailing lists" > message. It appears that Mailman only displays lists that match the URL prefix of your web server for mailman/listinfo whereas it displays all lists known to that installation no matter what the URL prefix for mailman/admin For an example of this see the exim.org box which runs lists for both exim.org and wylug.org.uk http://www.exim.org/mailman/listinfo/ - lists defined within exim.org http://list.wylug.org.uk/mailman/listinfo/ - lists on wylug.org.uk http://www.exim.org/mailman/admin/ - all lists on that machine So I guess you need to go to the admin pages for your lists and modify the "Base URL for Mailman web interface. " at the bottom of the "General Options" page. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From dc-mmdev@dragonwerks.net Fri Nov 10 09:46:47 2000 From: dc-mmdev@dragonwerks.net (dc-mmdev@dragonwerks.net) Date: 10 Nov 2000 02:46:47 -0700 Subject: from listinfo , can from adminRe: [Mailman-Developers] [please heklp!!]cannot list public lists In-Reply-To: Message-ID: <20001110024647.1.10248.qmail@dragonwerks.net> On Fri, 10 Nov 2000, Nigel Metheringham wrote: > It appears that Mailman only displays lists that match the URL prefix > of your web server for mailman/listinfo whereas it displays all lists > known to that installation no matter what the URL prefix for > mailman/admin > > So I guess you need to go to the admin pages for your lists and modify > the "Base URL for Mailman web interface. " at the bottom of the > "General Options" page. > This only works if you don't have any wierd capitization in your DNS names, if you do setting the base url won't help, however "VIRTUAL_HOST_OVERVIEW = 0" will fix it. -- DanielC From barry@digicool.com Fri Nov 10 23:45:35 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 18:45:35 -0500 (EST) Subject: [Mailman-Developers] Patch #102268 References: <14852.18163.486910.710242@j218.ryd.student.liu.se> Message-ID: <14860.34975.617122.912354@anthem.concentric.net> >>>>> "EF" == Erik Forsberg writes: EF> I submitted a patch (at sourceforge) earlier today, EF> #102268. It's basically a two-line-change to EF> MailMan/Archiver/HyperArch.py in 2.0rc1 that makes the EF> Content-Transfer-Encoding all lowercase. Hopefully I produced EF> a working patch-file.. You did, I just had to do it differently. :) See the updated patch on SF, or grab the CVS snapshot. -Barry From barry@digicool.com Fri Nov 10 23:47:08 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 18:47:08 -0500 (EST) Subject: [Mailman-Developers] Patch #102268 References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <3A049D72.22801714@west.sun.com> <14852.40577.83440.45618@j218.ryd.student.liu.se> <3A04A00C.A4896940@west.sun.com> Message-ID: <14860.35068.507104.805480@anthem.concentric.net> >>>>> "DM" == Dan Mick writes: DM> Right, but that's "augment the archives", which is not what DM> arch tries to do. If you want to rebuild, delete or move the DM> existing archives, and rebuild, and you'll get the result you DM> expect. Which I actually think is bogus. bin/arch should have an option to wipe the old archive and rebuild from scratch from the .mbox file. Not for 2.0 though. -Barry From barry@digicool.com Sat Nov 11 01:56:39 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 20:56:39 -0500 (EST) Subject: [Mailman-Developers] Direct Python access tutorial wanted References: <20001103194303.H25116@cheops.usf.Uni-Osnabrueck.DE> Message-ID: <14860.42839.447275.999218@anthem.concentric.net> >>>>> "BR" == Bernhard Reiter writes: BR> Hi Mailman developers, BR> just noticed that the minimal instructions for list server BR> administrators to mass manipulate mailinglists directly have BR> been vanished from the mailman README file in the jump to BR> version 2. (last cvs revision containg the advice is README BR> rev1.45) BR> As I considered it quite useful my suggestion is to add it BR> again to the mailman documentation. BR> Or is it not possible or advisable to do BR> this kind of list surgery anymore? | Regards, | Bernhard BR> ps: please cc relevant replies to me as I am not subscribe to BR> the list Bernard, In Mailman 2.0, the bin/withlist script is provided as a better framework for doing these kinds of manipulations. -Barry From barry@digicool.com Sat Nov 11 02:01:57 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 21:01:57 -0500 (EST) Subject: [Mailman-Developers] passwords in third party web archives, newsgroups References: <3A05F488.57CFC441@mail.lns.cornell.edu> Message-ID: <14860.43157.667139.522237@anthem.concentric.net> >>>>> "DR" == Dan Riley writes: DR> Are the mailman developers at all concerned by DR> http://www.google.com/search?q=mailman-owner+reminder+password DR> http://x66.deja.com/=dnc/getdoc.xp?AN=641175690 Yes, but it's too late to do anything about this in Mailman 2.0. Individual users should be able to disable password reminders to their address. Such archiver "false-users" can then just have this turned off. I'm glad there's a workaround for now! -Barry From barry@wooz.org Sat Nov 11 02:11:12 2000 From: barry@wooz.org (barry@wooz.org) Date: Fri, 10 Nov 2000 21:11:12 -0500 (EST) Subject: [Mailman-Developers] mailman installation error References: <3A034AC5.4191FD96@bmedesign.com> <5.0.0.25.2.20001106103101.03094c30@vtserf.cc.vt.edu> <3A0723D5.C7398BC7@west.sun.com> Message-ID: <14860.43712.22670.651590@anthem.concentric.net> >>>>> "DM" == Dan Mick writes: DM> It's hard to imagine how to make this more clear, though; it DM> *is* step 3, which is numbered, and after the "make install" DM> step, and it *does* say DM> - cd to $prefix DM> - Run bin/check_perms How about this? [in-source-directory]% bin/check_perms Could not import paths! This probably means that you are trying to run check_perms from the source directory. You must run this from the installation directory instead. Traceback (most recent call last): File "bin/check_perms", line 38, in ? import paths ImportError: No module named paths -Barry From barry@digicool.com Sat Nov 11 04:24:05 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 23:24:05 -0500 (EST) Subject: [Mailman-Developers] Announcing Mailman 2.0 release candidate 2 Message-ID: <14860.51685.368139.841987@anthem.concentric.net> This is it. Mailman 2.0 release candidate 2 is now available from SourceForge at http://sourceforge.net/project/showfiles.php?group_id=103 www.list.org and www.gnu.org should be updated soon. Excerpts from the NEWS file are given below. Of primary importance in this release is the fix of the last known mail duplication bug, and updated on-line documentation. You can view the documentation at http://mailman.sourceforge.net Please let me know if you find any errors in the docs. Unless something's royally screwed, those are the only changes I'll make before 2.0 final. Plan for that one week from today: Friday November 17th. Enjoy, -Barry -------------------- snip snip -------------------- 2.0 release candidate 2 (10-Nov-2000) - Documentation updates: start at admin/www/index.html - bin/withlist accepts additional command line arguments when used with the --run flag; bin/mmsitepass and bin/newlist accept -h/--help flags - bin/newlist has a -o/--output flag to append /etc/aliases suggestions to a specified file - SourceForge bugs fixed: 116615 (README.BSD update), 117015 (duplicate messages on moderated posts), 117548 (exception in HyperArch.py), 117682 (typos), 121185 (vsnprintf signature), 121591 and 122017 (bogus link after web unsubscribe), 121811 (`subscribe' in Subject: doesn't get archived) - SourceForge patches applied: 101812 (securelinux_fix.py contrib), 102097 (fix for bug 117548), 102211 (additional args for withlist), 102268 (case insensitive Content-Transfer-Encoding:) From ocschwar@MIT.EDU Sat Nov 11 05:40:44 2000 From: ocschwar@MIT.EDU (The guy named after an Om Kalthoum song) Date: Sat, 11 Nov 2000 00:40:44 EST Subject: [Mailman-Developers] Mea Culpa. Message-ID: <200011110540.AAA17093@lola-granola.mit.edu> When I first set out on this I was already planning on setting up a server with an encrypted mailing list in order to invite some friends of mine to try out the social dimension of it. So, I quickly concocted a gross misunderstanding in my mind about how mail transfer agents and mailing list managers divide up their duties. Thank you for correcting me on that. I also was a little optimistic on the idea that crypto email fora being the final element to establish pervasive crypto. A look at cypherpunks shows I'm dead wrong. What I want to do was attempted in the past and the code just died from obscurity. But, let's say I still set out on this: On the MTA side (I'd probably diff Postfix) I would have to enable the MTA to know to divert mail coming in to certain lists over to a side spool, activate the crypto-exploder, and then spool it to outgoing. Then comes writing the crypto-exploder, which would be a simple Perl or Python script invoking relevant the GPG and MTA modules. On the MLM side, all that really is necessary is for Mailman to be able to collect and revoke public keys(/etc/pki?), and deliver its own public key to those who request it. A host-owned (rather than user-owned) key ring has been discouraged in theory, since it would prolong the life-span of a revoked public key. Any server that used one would need to check in with a keyserver on a cronly basis. Regardless of the MTA issue, a GPG-enabled Mailman would be convenient. You could automatically process signed transaction request emails, and have the admin manually process unsigned ones, for example. So if I do it I hope you'll accept the plug in. Regards, -- Omri Schwarz I get serious letters from university students, asking questions for a project they are doing - these are not much different from those I get from school-children (written in green crayon), except the writing is a little worse. -- Terry Pratchett, Warwick Uni (10.11.94) From John Block Sat Nov 11 11:04:46 2000 From: John Block (John Block) Date: Sat, 11 Nov 2000 11:04:46 +0000 Subject: [Mailman-Developers] installation instruction suggestions In-Reply-To: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: Hello Barry > Please let me know if you find any errors in the docs. The odd line needs putting in to help the less able Set Ftp to Binary tar zxvf mailman-xxxx.tgz (I had to get the zxvf from support) Under configure options, it is not clear what to do. Is it tap in the command configure --option1 --option2 or is it edit the configure file, changing the lines... If it is edit the configure file, then putting these variable apart in an editable section and giving commented out examples would help. Thanks, John Unless > something's royally screwed, those are the only changes I'll make > before 2.0 final. Plan for that one week from today: Friday November > 17th. > > Enjoy, > -Barry > > -------------------- snip snip -------------------- > 2.0 release candidate 2 (10-Nov-2000) > > - Documentation updates: start at admin/www/index.html > > - bin/withlist accepts additional command line arguments when used > with the --run flag; bin/mmsitepass and bin/newlist accept > -h/--help flags > > - bin/newlist has a -o/--output flag to append /etc/aliases > suggestions to a specified file > > - SourceForge bugs fixed: > 116615 (README.BSD update), 117015 (duplicate messages on > moderated posts), 117548 (exception in HyperArch.py), 117682 > (typos), 121185 (vsnprintf signature), 121591 and 122017 > (bogus link after web unsubscribe), 121811 (`subscribe' in > Subject: doesn't get archived) > > - SourceForge patches applied: > 101812 (securelinux_fix.py contrib), 102097 (fix for bug > 117548), 102211 (additional args for withlist), 102268 (case > insensitive Content-Transfer-Encoding:) > > > _______________________________________________ > Mailman-announce mailing list > Mailman-announce@python.org > http://www.python.org/mailman/listinfo/mailman-announce > Regards From John Block Sat Nov 11 11:23:04 2000 From: John Block (John Block) Date: Sat, 11 Nov 2000 11:23:04 +0000 Subject: [Mailman-Developers] installation instruction suggestions In-Reply-To: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: Hello Barry > Please let me know if you find any errors in the docs. The odd line needs putting in to help the less able Set Ftp to Binary tar zxvf mailman-xxxx.tgz (I had to get the zxvf from support) Under configure options, it is not clear what to do. Is it tap in the command configure --option1 --option2 or is it edit the configure file, changing the lines... If it is edit the configure file, then putting these variable apart in an editable section and giving commented out examples would help. Thanks, John From ge204@eng.cam.ac.uk Sat Nov 11 18:00:24 2000 From: ge204@eng.cam.ac.uk (Gunnar Evermann) Date: 11 Nov 2000 18:00:24 +0000 Subject: [Mailman-Developers] Announcing Mailman 2.0 release candidate 2 In-Reply-To: barry@digicool.com's message of "Fri, 10 Nov 2000 23:24:05 -0500 (EST)" References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > This is it. Mailman 2.0 release candidate 2 is now available from > SourceForge at ok, I've just installed this from scratch. > - SourceForge patches applied: > 102268 (case insensitive Content-Transfer-Encoding:) This fix is wrong AFAICT. If an article doesn't contain a C-T-E header then I get the following from running the archiver list@casablanca:/var/lib/mailman$ bin/arch mailman-test-list-a Traceback (innermost last): File "bin/arch", line 129, in ? main() File "bin/arch", line 118, in main archiver.processUnixMailbox(fp, Article) File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 521, in processUnixMailbox a = articleClass(m, self.sequence) File "/var/lib/mailman/Mailman/Archiver/HyperArch.py", line 224, in __init__ self.cenc = string.lower(cenc) TypeError: read-only character buffer, None I just started learning Python, so I'll leave the (trivial) fix to you, Barry. Just wanted to let you know before you release this :-) Gunnar From bernhard@intevation.de Sat Nov 11 19:40:26 2000 From: bernhard@intevation.de (Bernhard Reiter) Date: Sat, 11 Nov 2000 20:40:26 +0100 Subject: [Mailman-Developers] Direct Python access tutorial wanted In-Reply-To: <14860.42839.447275.999218@anthem.concentric.net>; from barry@digicool.com on Fri, Nov 10, 2000 at 08:56:39PM -0500 References: <20001103194303.H25116@cheops.usf.Uni-Osnabrueck.DE> <14860.42839.447275.999218@anthem.concentric.net> Message-ID: <20001111204026.B24404@cheops.usf.Uni-Osnabrueck.DE> --NDin8bjvE/0mNLFQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 10, 2000 at 08:56:39PM -0500, Barry A. Warsaw wrote: > >>>>> "BR" =3D=3D Bernhard Reiter writes: >=20 > BR> Hi Mailman developers, >=20 > BR> just noticed that the minimal instructions for list server > BR> administrators to mass manipulate mailinglists directly have > BR> been vanished=20 > In Mailman 2.0, the bin/withlist script is provided as a better > framework for doing these kinds of manipulations. Good to know! It should be mentioned in the same place in the README file. (or did I read over it? :>) I found out that the old method also works. Why remove the description for low level hacking, if it is still valid? Thanks, Bernhard --=20 Professional Service around Free Software (intevation.net) = =20 The FreeGIS Project (freegis.org) Association for a Free Informational Infrastructure (ffii.org) --NDin8bjvE/0mNLFQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.3 (SunOS) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjoNoKoACgkQh9ag3dpKERZUIgCfT3xG1Lm73m4K5jxSKmIjT1Cz v3YAn2CP6DRNDCEoJhG4qOlva+CW2DJU =Gpxf -----END PGP SIGNATURE----- --NDin8bjvE/0mNLFQ-- From forsberg@lysator.liu.se Sun Nov 12 01:29:24 2000 From: forsberg@lysator.liu.se (Erik Forsberg) Date: Sun, 12 Nov 2000 02:29:24 +0100 (CET) Subject: [Mailman-Developers] Patch #102268 In-Reply-To: <14860.34975.617122.912354@anthem.concentric.net> References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <14860.34975.617122.912354@anthem.concentric.net> Message-ID: <14861.62068.946729.503549@j218.ryd.student.liu.se> Barry A. Warsaw writes: >You did, I just had to do it differently. :) But of course :-) >See the updated patch on, or grab the CVS snapshot. I would do it a bit different, for two reasons. 1) What happends if there is no Content-Transfer-Encoding header in the mail? Well.. cenc will get the "value" None. What happends if you do a string.lower on None? >>> string.lower(None) Traceback (innermost last): File "", line 1, in ? TypeError: read-only character buffer, None So guess why my 'if' where there.. And yes, I had a mail where there were no such header. Later in the code, there is a check whether self.cenc is None, so it should hold that value. 2) There is a perhaps more readable way of setting ctype to 'text/plain' if there were no such header: `getheader(name[, default])' Like `getrawheader(NAME)', but strip leading and trailing whitespace. Internal whitespace is not stripped. The optional DEFAULT argument can be used to specify a different default to be Mostly a cosmetic change, though. The current code looks a bit too much like Perl to me ;). Revised patch follows (Note: I edited the patch by hand. Will verify and upload to sourceforge sometime tomorrow^H^H^H^H^H^Hday (when it's not 02:27 in the morning..)): Index: HyperArch.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/Archiver/HyperArch.py,v retrieving revision 1.44 diff -u -r1.44 HyperArch.py --- HyperArch.py 2000/11/10 15:24:53 1.44 +++ HyperArch.py 2000/11/10 23:44:17 @@ -216,9 +216,12 @@ if mm_cfg.ARCHIVER_OBSCURES_EMAILADDRS: self.email = re.sub('@', ' at ', self.email) - # snag the content-type - self.ctype = message.getheader('Content-Type') or "text/plain" - self.cenc = message.getheader('Content-Transfer-Encoding') + # Snag the content-* headers. RFC 1521 states that their values are + # case insensitive. + ctype = message.getheader('Content-Type', "text/plain") + self.cenc = message.getheader('Content-Transfer-Encoding') + self.ctype = string.lower(ctype) + if None != self.cenc: + self.cenc = string.lower(self.cenc) self.decoded = {} mo = rx_charset.search(self.ctype) if mo: \EF -- Erik Forsberg http://www.lysator.liu.se/~forsberg/ PGP fingerprint = 37 1B D7 9E 97 8C EF 39 0E DE 08 E8 99 0C 5E A9 From gconnor@nekodojo.org Sun Nov 12 10:11:30 2000 From: gconnor@nekodojo.org (Greg Connor) Date: Sun, 12 Nov 2000 02:11:30 -0800 Subject: [Mailman-Developers] Re: [Mailman-Announce] Announcing Mailman 2.0 release candidate 2 References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: <3A0E6CD2.7CEBD4C@nekodojo.org> "Barry A. Warsaw" wrote: > > This is it. Mailman 2.0 release candidate 2 is now available from > SourceForge at Thanks for the heads-up, and thanks for a great product. I installed 2.0rc2, and I had a problem with lists that had administrative requests pending. I had to remove all the pending requests before "update" would run successfully. I know it's probably a bit late to fix, but I wanted to let you know... I posted to mailman-users with details and a log of my install. Good luck! -- Greg Connor From forsberg@lysator.liu.se Sun Nov 12 19:23:32 2000 From: forsberg@lysator.liu.se (Erik Forsberg) Date: Sun, 12 Nov 2000 20:23:32 +0100 (CET) Subject: [Mailman-Developers] Patch #102268 In-Reply-To: <14861.62068.946729.503549@j218.ryd.student.liu.se> References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <14860.34975.617122.912354@anthem.concentric.net> <14861.62068.946729.503549@j218.ryd.student.liu.se> Message-ID: <14862.60980.140238.458331@j218.ryd.student.liu.se> > >Revised patch follows (Note: I edited the patch by hand. Will verify >and upload to sourceforge sometime tomorrow^H^H^H^H^H^Hday (when it's >not 02:27 in the morning..)): And as usual, when it's 2 o'clock in the morning nothing valuable is produced. The patch I sent didn't work 100%. I've now resubmitted the patch to sourceforge. It's a little bit different too - It doesn't use a if statement, instead it sets cenc to "" if there was no such header. The check later in the code will work as good if self.cenc is set to "" as if it were set to None. I've tested the code in production. Note: I submitted two patches, since I made a very silly mistake at first. It's even worse that now it's not 2 in the morning, since now I have nothing to blame :-). \EF -- Erik Forsberg http://www.lysator.liu.se/~forsberg/ PGP fingerprint = 37 1B D7 9E 97 8C EF 39 0E DE 08 E8 99 0C 5E A9 From vizisz@freemail.hu Mon Nov 13 09:38:03 2000 From: vizisz@freemail.hu (Vizi Szilard) Date: Mon, 13 Nov 2000 10:38:03 +0100 Subject: [Mailman-Developers] Suggestions 4 Mailman Message-ID: <3A0FB67B.919D5121@freemail.hu> Hello! I am using Mailman for 5 small mailing lists, and I have some advice. I have not read through the developers archive yet, so forgive me if I say something that someone already said. It would be better if the monthly reminder contains the current settings for both the user and the list. Like this: -----------------------reminder-------------------------------- YOUR Listname PASSWORD IS: password YOUR Listname SETTINGS ARE: [ack] [digest] [plain] [nomail] [norcv] [hide] For explanations see the mail called HELP. The Listname SETTINGS ARE: [public/private/hidden] [maxmessage size=xxx kB] [archive is public/privat (authorization needed] [subscribers list is public/privat/not avaliable] admin address is: name@some.where Any comments, question should be send to the admin. -----------------------reminder-------------------------------- Okay, I know that this can be achived with an "options" command, but the dummy users never use the commands. Even a few of them do not understand the differences between the listname and the listname-request email addresses. And the Listname settings can help the users how big could be a message, eg. why can not be seen the listname at the web interface (because it is hidden), and who is managing the mailinglist. And here is my second problem. Can be Mailman a bit more intelligent? If I have a user who makes the same mistakes, Mailman can solve the problem without any admin interaction. eg. the user posts a message but the address is in the BCC header, not in the To header. Mailman processes the message like the address is in the To header. Or there is a user who always post "funny" pictures, sometimes he/she posts a bigger picture, than the list setting allows. Mailman processes this message for this user only, other users with big messages should wait for the admin interaction. a.s.o Thats all, I think. Thanx. Szilard Vizi From mats@laplaza.org Mon Nov 13 19:48:12 2000 From: mats@laplaza.org (Mats Wichmann) Date: Mon, 13 Nov 2000 12:48:12 -0700 Subject: [Mailman-Developers] Re: [Mailman-Users] installation instruction suggestions In-Reply-To: References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: <5.0.0.25.1.20001113124615.00a61eb0@mail.laplaza.org> >The odd line needs putting in to help the less able > >Set Ftp to Binary Just musing... is there actually any benefit to EVER having ftp not set to binary? Why do we still have to fight this battle? From ralph@inputplus.demon.co.uk Mon Nov 13 23:46:04 2000 From: ralph@inputplus.demon.co.uk (Ralph Corderoy) Date: Mon, 13 Nov 2000 23:46:04 +0000 Subject: [Mailman-Developers] Re: [Mailman-Users] installation instruction suggestions In-Reply-To: Message from Mats Wichmann of "Mon, 13 Nov 2000 12:48:12 MST." <5.0.0.25.1.20001113124615.00a61eb0@mail.laplaza.org> Message-ID: <200011132346.XAA04047@inputplus.demon.co.uk> Hi, > > The odd line needs putting in to help the less able > > > > Set Ftp to Binary > > Just musing... is there actually any benefit to EVER having ftp not > set to binary? Why do we still have to fight this battle? Transferring text files between systems with different formats for text files can cause corrupt files if the transfer mode isn't text. Transferring binary files between systems with different formats for text files causes corrupt files if the transfer mode is text. Ralph. From barry@digicool.com Tue Nov 14 04:51:00 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Mon, 13 Nov 2000 23:51:00 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Announce] Announcing Mailman 2.0 release candidate 2 References: <14860.51685.368139.841987@anthem.concentric.net> <3A0E6CD2.7CEBD4C@nekodojo.org> Message-ID: <14864.50356.152336.517891@anthem.concentric.net> >>>>> "GC" == Greg Connor writes: GC> Thanks for the heads-up, and thanks for a great product. You're welcome! GC> I installed 2.0rc2, and I had a problem with lists that had GC> administrative requests pending. I had to remove all the GC> pending requests before "update" would run successfully. I GC> know it's probably a bit late to fix, but I wanted to let you GC> know... I posted to mailman-users with details and a log of GC> my install. Yup. There's a buglet in CheckVersion() when there are pending requests. Some older versions used a different requests database format, and Mailman's auto-update feature tries to repopulate using the new database format. This can only happen when the list is locked, but the new update script doesn't lock the list right away (to avoid hangs while updating). Here's a patch to fix the problem. -Barry -------------------- snip snip -------------------- Index: MailList.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/MailList.py,v retrieving revision 1.187 retrieving revision 1.188 diff -u -r1.187 -r1.188 --- MailList.py 2000/11/10 17:56:55 1.187 +++ MailList.py 2000/11/14 04:44:01 1.188 @@ -913,9 +913,10 @@ else: self.InitVars() # Init any new variables, self.Load(check_version = 0) # then reload the file - from versions import Update - Update(self, stored_state) - self.data_version = mm_cfg.DATA_FILE_VERSION + if self.Locked(): + from versions import Update + Update(self, stored_state) + self.data_version = mm_cfg.DATA_FILE_VERSION if self.Locked(): self.Save() From barry@digicool.com Tue Nov 14 04:51:35 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Mon, 13 Nov 2000 23:51:35 -0500 (EST) Subject: [Mailman-Developers] Announcing Mailman 2.0 release candidate 2 References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: <14864.50391.634350.998016@anthem.concentric.net> >>>>> "GE" == Gunnar Evermann writes: GE> I just started learning Python, so I'll leave the (trivial) GE> fix to you, Barry. Just wanted to let you know before you GE> release this :-) Thanks much! See my previously posted fix for this buglet. -Barry From midnight@the-oasis.net Tue Nov 14 05:25:28 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 00:25:28 -0500 Subject: [Mailman-Developers] Reverting question Message-ID: <3A108678.29865.5F12883@localhost> Am I likely to encounter problems going from 2.0rc2 to 2.0b6? Did any data formats change? -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From barry@digicool.com Tue Nov 14 05:30:45 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 14 Nov 2000 00:30:45 -0500 (EST) Subject: [Mailman-Developers] installation instruction suggestions References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: <14864.52741.977088.723846@anthem.concentric.net> >>>>> "JB" == John Block writes: JB> The odd line needs putting in to help the less able JB> Set Ftp to Binary JB> tar zxvf mailman-xxxx.tgz JB> (I had to get the zxvf from support) JB> Under configure options, it is not clear what to do. JB> Is it tap in the command JB> configure --option1 --option2 JB> or is it edit the configure file, changing the lines... JB> If it is edit the configure file, then putting these variable JB> apart in an editable section and giving commented out examples JB> would help. No, you don't need to edit the configure file, just include the options on the command line. Thanks for the feedback; I've added some extra text to the documents to clarify things. I sometimes forget that not everybody is a grisled and curmudgeonly ex-sysadmin like myself. :) dinosaurish-ly y'rs, -Barry From barry@digicool.com Tue Nov 14 05:32:46 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 14 Nov 2000 00:32:46 -0500 (EST) Subject: [Mailman-Developers] Reverting question References: <3A108678.29865.5F12883@localhost> Message-ID: <14864.52862.465404.35651@anthem.concentric.net> >>>>> "PB" == Phil Barnett writes: PB> Am I likely to encounter problems going from 2.0rc2 to 2.0b6? Do you mean downgrading? Don't do that! If you really mean from 2.0b6 to 2.0rc2 then you should be fine. Any data formats that need updating will get updated automatically. -Barry From midnight@the-oasis.net Tue Nov 14 05:37:27 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 00:37:27 -0500 Subject: [Mailman-Users] Re: [Mailman-Developers] Announcing Mailman 2.0 release candidate 2 In-Reply-To: <14864.51912.327124.393642@anthem.concentric.net> Message-ID: <3A108947.2828.5FC21DD@localhost> On 14 Nov 2000, at 0:16, Barry A. Warsaw wrote: > > >>>>> "PB" == Phil Barnett writes: > > PB> I upgraded from 2.0b6 from 2.0rc2 Friday night. A person wrote > PB> me and said the replyto: is no longer pointed to the list. I > PB> checked several messages and they were, but some were not. Any > PB> message I send to the list has a replyto of myself instead of > PB> to the list. > > Please remember that if the original message has a Reply-To: already, > Mailman will not overwrite that, even if "explicit reply-to" is set. > So, did the original mesasge have a Reply-To already? > > I just tested this using rc2. With reply_goes_to_list set to > "explicit address" and reply_to_address set to some non-blank value, > any message without a Reply-To: gets one set to that value. Any > message that already has a Reply-To: set is unchanged. That's > expected behavior (i.e. works for me!) Expected behavour for whom? For all the users on my lists, it has them screaming. They want to know why Mailman is broken after the upgrade. 2.0b6 did not exhibit this behaviour. How can I make 2.0rc2 work like b6 did? I think it's rather impossible to get all list users on the planet to remove their replyto so they can reply to the list on a list that replyto list is set. Expecially so since setting replyto is generally a global setting in most MUA's. Why was this changed? Is there a way to make Mailman to have this be selectable behaviour for it to work both ways? Not being able to force replyto to the list is broken behaviour in my users eyes, and, they are the ones who really count. My open source developers list is ready to switch back to egroups over this. That can't be good news. -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From midnight@the-oasis.net Tue Nov 14 05:43:23 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 00:43:23 -0500 Subject: [Mailman-Developers] Reverting question In-Reply-To: <14864.52862.465404.35651@anthem.concentric.net> Message-ID: <3A108AAB.25143.6018DAE@localhost> On 14 Nov 2000, at 0:32, Barry A. Warsaw wrote: > > >>>>> "PB" == Phil Barnett writes: > > PB> Am I likely to encounter problems going from 2.0rc2 to 2.0b6? > > Do you mean downgrading? Don't do that! If you really mean from > 2.0b6 to 2.0rc2 then you should be fine. Any data formats that need > updating will get updated automatically. I specifically mean downgrading. I have a hornets nest on my hands and I'm about to lose my lists back to egroups. This lack of reply to the list 'updated feature' has them HEATED. Perhaps I'm not clear. If I lose them to egroups, there is no point to me running Mailman anymore. This 'upgraded feature' is suicidal for Mailman on my machine. -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From simon@uow.edu.au Tue Nov 14 05:57:49 2000 From: simon@uow.edu.au (Simon Coggins) Date: Tue, 14 Nov 2000 16:57:49 +1100 Subject: [Mailman-Developers] Reverting question In-Reply-To: <3A108AAB.25143.6018DAE@localhost>; from midnight@the-oasis.net on Tue, Nov 14, 2000 at 12:43:23AM -0500 References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> Message-ID: <20001114165748.B3475@uow.edu.au> I have to agree here. Not being able to override the reply-to: fields on lists is a *MAJOr* problem. I've also had this problem with my lists. And I was hoping for a 'fix' but it looks like it's intended behaviour?. I think if thats going to be the case there should be another option. 'Force list replyto' . Regards Simon On Tue, Nov 14, 2000 at 12:43:23AM -0500, Phil Barnett wrote: > On 14 Nov 2000, at 0:32, Barry A. Warsaw wrote: > > > > > >>>>> "PB" == Phil Barnett writes: > > > > PB> Am I likely to encounter problems going from 2.0rc2 to 2.0b6? > > > > Do you mean downgrading? Don't do that! If you really mean from > > 2.0b6 to 2.0rc2 then you should be fine. Any data formats that need > > updating will get updated automatically. > > I specifically mean downgrading. I have a hornets nest on my > hands and I'm about to lose my lists back to egroups. > > This lack of reply to the list 'updated feature' has them HEATED. > > Perhaps I'm not clear. If I lose them to egroups, there is no point to > me running Mailman anymore. > > This 'upgraded feature' is suicidal for Mailman on my machine. > > > -- > Phil Barnett mailto:midnight@the-oasis.net > WWW http://www.the-oasis.net/ > FTP Site ftp://ftp.the-oasis.net > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers -- Simon Coggins (SAGE-AU Member) Email: simon@uow.edu.au Network and System Management Officer Phone: +61-2-4221-3775 Information Technology Systems (ITS) Mobile: 0408 115861 University of Wollongong, 2522, Australia Fax: +61-2-4229-1985 From barry@digicool.com Tue Nov 14 06:07:01 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 14 Nov 2000 01:07:01 -0500 (EST) Subject: [Mailman-Developers] Reverting question References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> Message-ID: <14864.54917.27112.665096@anthem.concentric.net> >>>>> "PB" == Phil Barnett writes: PB> I specifically mean downgrading. I have a hornets nest on my PB> hands and I'm about to lose my lists back to egroups. You don't want to downgrade. See my untested patch in a different followup. PB> This lack of reply to the list 'updated feature' has them PB> HEATED. I'm surprised. I would think that munging reply-to is only useful for lists of people who don't know how to drive their MUAs. Is the problem just that they got used to one way of doing things, and now there's a different way? That I can sympathize with. Is it that suddenly, some people are getting two copies of messages because people don't know how to trim their headers? Yeah, that sucks. Or is it something else? -Barry From midnight@the-oasis.net Tue Nov 14 06:31:00 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 01:31:00 -0500 Subject: [Mailman-Developers] Reverting question In-Reply-To: <14864.54917.27112.665096@anthem.concentric.net> Message-ID: <3A1095D4.14770.62D26CF@localhost> On 14 Nov 2000, at 1:07, Barry A. Warsaw wrote: > I'm surprised. I would think that munging reply-to is only useful for > lists of people who don't know how to drive their MUAs. Is the > problem just that they got used to one way of doing things, and now > there's a different way? That I can sympathize with. Is it that > suddenly, some people are getting two copies of messages because > people don't know how to trim their headers? Yeah, that sucks. > > Or is it something else? I use Pegasus. My replyto is global. If I unset my replyto, it's removed from ALL of my mail. I don't really want to be going to my configuration page and blanking it for some mail and turning it back on for other mail. First, it's a hassle. Second, I'll undoubtedly do it wrong at some point. I want a replyto in my mail. Every mail list that I have participated in overrides replyto when replyto is set to the list. To my fellow list users who have been using lists that do this for the last three years, not having it work that way is disconcerting to say the least. egroups and opensource.org are two examples of list providers that override replyto. But, I certainly understand the desire to have it work the way you have it set now. I just think the behaviour should be selectable. It doesn't have to be all or none. Thanks for the patch! Now, if I could create a new list, I'd be back on track... -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From John Block Tue Nov 14 10:25:00 2000 From: John Block (John Block) Date: Tue, 14 Nov 2000 10:25:00 +0000 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: <14864.54917.27112.665096@anthem.concentric.net> Message-ID: Hello Barry > I'm surprised. I would think that munging reply-to is only useful for > lists of people who don't know how to drive their MUAs. MUA? My mail software gives a range of reply options: to all recipients, to sender, to recipients, so there is still choice. Anyhow, I just tried to reply to the question about searchable archives and only got the poster's address. My use for mailman will be to let users help each other and be publically supported and build up an archive of messages to create a searchable brainfile people can query for solutions. If, in many cases, help is only routed to the original poster and not publically, then this plan turns to dust. John From johnblock@storeshop.com Tue Nov 14 10:18:20 2000 From: johnblock@storeshop.com (John Block) Date: Tue, 14 Nov 2000 10:18:20 +0000 Subject: [Mailman-Developers] Re: Searchable Archives In-Reply-To: <3A10E552.CCB12417@nleaudio.com> Message-ID: Hello Bob Fortunatly I'm skilled enough to beable to cut and paste addresses so you will get this publically. > #2. Is there any, ANY way to make the archives searchable? (Under Linux) My plan is to have pipermail which is included in the tarball create html pages. I'm then plannning to get the http://www.thunderstone.com search engine installed to search the html Thunderstone seems to offer alternative search terms based on relationships in the data. A feature which has has got me excited) John From ge204@eng.cam.ac.uk Tue Nov 14 12:59:54 2000 From: ge204@eng.cam.ac.uk (Gunnar Evermann) Date: 14 Nov 2000 12:59:54 +0000 Subject: [Mailman-Developers] Future of pipermail? Message-ID: What are the current plans about the html archiver in mailman? The major shortcomings of pipermail I see are: - doesn't decode MIME - doesn't handle mixed charsets - no search engine - somewhat convoluted code. I run a mailing list with a lot of subscribers from Asia who use MS Outlook, invariably posting (English) messages spuriously tagged as CJK charsets (but only using the ASCII subset) and gratuitously encoded in base64. Therefore I am motivated to find a solution to handling MIME. :-( I have hacked base64 support into pipermail and am currently looking at supporting MIME (probably using mimecntl.py), but this will just be short-term fixes. I know I could just switch to MHonarc or whatever but I like the idea of having a nicely structured, modular archiver tightly integrated with mailman. Are there people working on on pipermail or a replacement of it? Gunnar From chuqui@plaidworks.com Tue Nov 14 15:43:33 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 07:43:33 -0800 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: References: Message-ID: At 10:25 AM +0000 11/14/00, John Block wrote: > > I'm surprised. I would think that munging reply-to is only useful for >> lists of people who don't know how to drive their MUAs. > >MUA? Mail User Agent (aka client) Other jargon: MLM (mail list manager) MTA (mail transfer agent) In general I'm against coercing Reply-To. Strongly. But there are times when it's the right/necessary thing to do, so you want that option. In general yous houldn't use it, but sometimes you need it. One classic case is a mailing list under NDA discussing a beta software release, where all e-mail has to be logged and evaluated, and the *wrong* thing to do is reply privately to a person, because the beta team needs a copy. In that case, reply-to gets coerced to the list, and that reply-to needs to be dominant (i.e., an existing reply-to from a user *can't* take precedent). -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From barry@digicool.com Tue Nov 14 16:40:45 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 14 Nov 2000 11:40:45 -0500 (EST) Subject: [Mailman-Developers] Re: Reverting question References: Message-ID: <14865.27405.857124.313441@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> In general I'm against coercing Reply-To. Strongly. Me too. CVR> But there are times when it's the right/necessary thing to CVR> do, so you want that option. In general yous houldn't use it, CVR> but sometimes you need it. One classic case is a mailing list CVR> under NDA discussing a beta software release, where all CVR> e-mail has to be logged and evaluated, and the *wrong* thing CVR> to do is reply privately to a person, because the beta team CVR> needs a copy. In that case, reply-to gets coerced to the CVR> list, and that reply-to needs to be dominant (i.e., an CVR> existing reply-to from a user *can't* take precedent). That's the key thing that I don't like about Reply-To munging: it makes it much more difficult to do private replies. Maybe it's just the lists I manage, but my users are just as heated about /not/ doing reply-to munging as apparently other users on other lists are about doing it. The primary complaint is that people accidently send private responses to the whole list. This happens because they've trained themselves to know the difference between replies and followups, and have MUAs that separate those functions. The only grumblings occur when people don't trim their CC headers (like I've done here) and folks start getting duplicates. That can be handled in other ways (e.g. the list /could/ suppress deliveries to list members that it sees explicitly in the recipients list, although as we've discussed before, that has some potential for abuse). I've slept on this one and I'm prepared to change things for 2.0 final so that if Reply-To munging is turned on, it'll override any existing Reply-To field in the original message. The deciding factor for me was realizing how difficult it was to send private replies with munging turned on, so it might as well be equally difficult for every poster. I'm concerned about making this change so late in the game, but I'm willing to do it, /if/ I get enough positive feedback asap. I'm attaching a diff against the current CVS tree. Please try it and let me know. If I get enough positive responses with few or no negative responses, I'll add this ("enough" being defined completely subjectively :). -Barry -------------------- snip snip -------------------- Index: Mailman/Defaults.py.in =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/Defaults.py.in,v retrieving revision 1.124 diff -u -r1.124 Defaults.py.in --- Mailman/Defaults.py.in 2000/11/09 02:00:30 1.124 +++ Mailman/Defaults.py.in 2000/11/14 16:38:16 @@ -310,7 +310,15 @@ from: list@listme.com from: .*@uplinkpro.com """ -# Replies to posts inherently directed to list or original sender? + +# Mailman can be configured to "munge" Reply-To: headers for any passing +# messages. One the one hand, there are a lot of good reasons not to munge +# Reply-To: but on the other, people really seem to want this feature. See +# the help for reply_goes_to_list in the web UI for links discussing the +# issue. +# 0 - Reply-To: not munged +# 1 - Reply-To: set back to the list +# 2 - Reply-To: set to an explicit value (reply_to_address) DEFAULT_REPLY_GOES_TO_LIST = 0 # SUBSCRIBE POLICY Index: Mailman/MailList.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/MailList.py,v retrieving revision 1.188 diff -u -r1.188 MailList.py --- Mailman/MailList.py 2000/11/14 04:44:01 1.188 +++ Mailman/MailList.py 2000/11/14 16:38:18 @@ -424,13 +424,21 @@ is strongly recommended for most mailing lists.''', # Details for reply_goes_to_list - """There are many reasons not to introduce headers like -Reply-To: into other people's messages. One is that some posters -depend on their own Reply-To: settings to convey their valid return -address. See + """This option controls what Mailman does to the +Reply-To: header in messages flowing through this mailing list. When +set to Poster, no Reply-To: header is added by Mailman, +although if one is present in the original message, it is not stripped. +Setting this value to either This list or Explicit address +causes Mailman to insert a specific Reply-To: header in all messages, +overriding in the original message if necessary. + +

There are many reasons not to introduce or override the Reply-To: +header. One is that some posters depend on their own Reply-To: +settings to convey their valid return address. Another is that modifying +Reply-To: makes it much more difficult to send private replies. See `Reply-To' Munging -Considered Harmful for a general discussion of this issue. See -Reply-To +Considered Harmful for a general discussion of this issue. See Reply-To Munging Considered Useful for a dissenting opinion.

Some mailing lists have restricted posting privileges, with a parallel list Index: Mailman/Handlers/CookHeaders.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/Handlers/CookHeaders.py,v retrieving revision 1.17 diff -u -r1.17 CookHeaders.py --- Mailman/Handlers/CookHeaders.py 2000/10/27 18:55:21 1.17 +++ Mailman/Handlers/CookHeaders.py 2000/11/14 16:38:19 @@ -76,11 +76,8 @@ msg['Precedence'] = 'bulk' # # Reply-To: munging. Do not do this if the message is "fast tracked", - # meaning it is internally crafted and delivered to a specific user, - # or if there is already a reply-to set. If the user has set - # one we assume they have a good reason for it, and we don't - # second guess them. - if not fasttrack and not msg.get('reply-to'): + # meaning it is internally crafted and delivered to a specific user. + if not fasttrack: # Set Reply-To: header to point back to this list if mlist.reply_goes_to_list == 1: msg['Reply-To'] = mlist.GetListEmail() From ray_frush@agilent.com Tue Nov 14 17:22:42 2000 From: ray_frush@agilent.com (Ray Frush) Date: Tue, 14 Nov 2000 10:22:42 -0700 Subject: [Mailman-Developers] Re: Mailman-Users digest, Vol 1 #870 - 2 msgs References: <20001114170146.9D8861D16F@dinsdale.python.org> Message-ID: <3A1174E2.F0236153@ale.ftc.agilent.com> > I've slept on this one and I'm prepared to change things for 2.0 final > so that if Reply-To munging is turned on, it'll override any existing > Reply-To field in the original message. The deciding factor for me > was realizing how difficult it was to send private replies with > munging turned on, so it might as well be equally difficult for every > poster. If Reply-to munging is enabled, I would expect that it would really munge. That is, if you turn on the feature, it will ALWAYS exhibit the behaviour. In the current version, Reply-To: munging appears broken because it appears to work part of the time. Count this as a vote to enable the much debated Reply-to munging that will override any client (MUA) Reply-to settings. -- Ray Frush "Either you are part of the solution, T:970.288.6223 or part of the precipitate." -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- Agilent Technologies IT | Technical Computing | Fort Collins Site From chuqui@plaidworks.com Tue Nov 14 17:49:53 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 09:49:53 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Re: Reverting question In-Reply-To: <14865.27405.857124.313441@anthem.concentric.net> References: <14865.27405.857124.313441@anthem.concentric.net> Message-ID: At 11:40 AM -0500 11/14/00, Barry A. Warsaw wrote: >That's the key thing that I don't like about Reply-To munging: it >makes it much more difficult to do private replies. Maybe it's just >the lists I manage, but my users are just as heated about /not/ doing >reply-to munging as apparently other users on other lists are about >doing it. It's a religious issue to some level, but I've done any number of studies (at least half a dozen) on it with my users over the years. I've never found *any* list or user population where the majority wants reply-to coerced. What you really end up with, if you survey the *entire* user base, is a noisy minority that wants it, and that minority is under 20-25% of those that have opinions. So what ends up happening is that because they're the ones making noise, the list gets set that way. But if you survey the entire user base and get a non-biased sample, they're the minority viewpoint (the same is true of Subject line tweaks, like the [Mailman-users] flag. Only with those, I've never found a list where even 5% of the users wanted it, and there's usually a solid majority that hate the damn things.... But again, you tend to run into noisy minorites that push their preferences, and unless you take the time to go out and run formal list surveys (which are time consuming), it's hard to tell whether it is a squeaky wheel or whether it's a list consensus... >The only grumblings occur when people don't >trim their CC headers (like I've done here) and folks start getting >duplicates. Another vocal minority. Less than 1% of a typical subscriber base cares about this. Those that do tend to be very sensitive to it and vocal, but whacking the server for that group isn't a smart idea (IMHO), since there are client ways of doing it, like message-ID trapping. > That can be handled in other ways (e.g. the list /could/ >suppress deliveries to list members that it sees explicitly in the >recipients list, although as we've discussed before, that has some >potential for abuse). True, but it's what I'd like to see happen down the road, at least as a configuration option for the server admin. If you know they're getting it direct, why send them a second copy? or maybe tag this to a users "metoo" flag? If they've told the server not to send them copies of their own posts, assume they also don't want duplicates? That might be the easiest way, and leave it by default off, but then users can set it if they care. >I've slept on this one and I'm prepared to change things for 2.0 final >so that if Reply-To munging is turned on, it'll override any existing >Reply-To field in the original message. The deciding factor for me >was realizing how difficult it was to send private replies with >munging turned on, so it might as well be equally difficult for every >poster. I think that's the right thing to do, and it sets you in line with how other MLMs work, so it better matches user expectations, I think... It's a touch issue, because there are personal preferences, because Reply-to is used for different things (not always correctly), and because it just plain old isn't all that well defined as a header, despite it's long-standing usage... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From jwblist@olympus.net Tue Nov 14 20:33:16 2000 From: jwblist@olympus.net (John W Baxter) Date: Tue, 14 Nov 2000 12:33:16 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <14864.54917.27112.665096@anthem.concentric.net> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <14864.54917.27112.665096@anthem.concentric.net> Message-ID: At 1:07 -0500 11/14/00, Barry A. Warsaw wrote: >I'm surprised. I would think that munging reply-to is only useful for >lists of people who don't know how to drive their MUAs. Is the >problem just that they got used to one way of doing things, and now >there's a different way? That I can sympathize with. Is it that >suddenly, some people are getting two copies of messages because >people don't know how to trim their headers? Yeah, that sucks. I'm not on such a list (we're expecting to switch to Mailman when it settles down). But I would be perturbed if on a single list sometimes the Reply button sets up a reply to the list and sometimes it sets up a private reply. [I can--and do--deal with both kinds of list, and both are a nuisance some of the time.] The idea that Reply's result on a single list depends on who* sent the message seems like a faulty solution to the unsolvable problem that neither reply-goes-to-the-list nor reply-goes-to-the-sender is right. [Right would take mindreading: reply goes where I expect *this* reply to go.] *Or, even worse, where a given person was were when the message was sent. --John -- John Baxter jwblist@olympus.net Port Ludlow, WA, USA From johnblock@storeshop.com Tue Nov 14 16:48:05 2000 From: johnblock@storeshop.com (John Block) Date: Tue, 14 Nov 2000 16:48:05 +0000 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: Message-ID: Hello Chuq Thank you for clearing up the TLAs. > In general I'm against coercing Reply-To. Strongly. Who is coercing? By just having the sender as a reply option, the software is stopping the none technical from replying to the list. The more aware are caused irritation as they can't simply hit their "r" button. If you can't reply publically easily, for a lot of people the motivations to reply are removed. There is no name in print on the screen, no look how clever I am, no visible penny in the help bank so i can get help in return > But there are times when it's the right/necessary thing to do, so you > want that option. In general yous houldn't use it, but sometimes you > need it. In other words it should be a togglable feature. One classic case is a mailing list under NDA discussing a > beta software release, where all e-mail has to be logged and > evaluated, That should be the case on any help list, so people can search archives before needlessly bothering everyone else with something which has been solved already. Thanks, John From midnight@the-oasis.net Wed Nov 15 01:43:35 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 20:43:35 -0500 Subject: [Mailman-Developers] Default path for archives wrong in rc2 Message-ID: <3A11A3F7.14204.A4C5F54@localhost> I know that we got caught up in the replyto munging issue, but did anyone see that when I switched to rc2, my archives went offline and I can't add a new user because the archive is not where it's expected to be now because of a /bin/ in the message? I can post the problems specifically again if necessary, but the path problem was posted a couple of days ago. Since my rc2 upgrade from b6, I can't create new lists and my archives are offline. http://www.matrixlist.com/mailman/listinfo is broken and the data didnt' move. -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From marc_news@valinux.com Wed Nov 15 01:50:12 2000 From: marc_news@valinux.com (Marc MERLIN) Date: Tue, 14 Nov 2000 17:50:12 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <20001114165748.B3475@uow.edu.au>; from simon@uow.edu.au on Tue, Nov 14, 2000 at 04:57:49PM +1100 References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> Message-ID: <20001114175012.T27544@marc.merlins.org> On Tue, Nov 14, 2000 at 04:57:49PM +1100, Simon Coggins wrote: > I have to agree here. Not being able to override the reply-to: fields on > lists is a *MAJOr* problem. I've also had this problem with my lists. And I > was hoping for a 'fix' but it looks like it's intended behaviour?. The irony of this coming back over and over again... http://www.unicom.com/pw/reply-to-harmful.html 1) Don't set a reply-to in your lists (see above link) 2) If you don't want to take that advise, you certainly shouldn't overwrite a users' reply to with the list's 3) If you insist in doing the above 2, you can patch the code Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From chuqui@plaidworks.com Wed Nov 15 02:13:53 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 18:13:53 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <20001114175012.T27544@marc.merlins.org> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> Message-ID: At 5:50 PM -0800 11/14/00, Marc MERLIN wrote: >2) If you don't want to take that advise, you certainly shouldn't overwrite > a users' reply to with the list's I'll disagree. If you feel you must coerce reply-to, coerce it unconditionally. You've already decided the list's needs outweigh the individual's needs, or you woudln't be coercing in the first place Once you make that decision, whether or not the user ALSO uses a reply-to is meaningless, and trying to take that into consideration only confuses things, because then the list has its reply-to coerced -- sometimes, and you have to know how to read mail headers ot figure out when. Not good for the typical user, who may never figure out why something isn't working randomly. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From Dan Mick Wed Nov 15 02:48:04 2000 From: Dan Mick (Dan Mick) Date: Tue, 14 Nov 2000 18:48:04 -0800 (PST) Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question Message-ID: <200011150246.SAA26058@utopia.west.sun.com> > At 5:50 PM -0800 11/14/00, Marc MERLIN wrote: > >2) If you don't want to take that advise, you certainly shouldn't overwrite > > a users' reply to with the list's > > I'll disagree. If you feel you must coerce reply-to, coerce it > unconditionally. You've already decided the list's needs outweigh the > individual's needs, or you woudln't be coercing in the first place > Once you make that decision, whether or not the user ALSO uses a > reply-to is meaningless, and trying to take that into consideration > only confuses things, because then the list has its reply-to coerced > -- sometimes, and you have to know how to read mail headers ot figure > out when. Not good for the typical user, who may never figure out why > something isn't working randomly. I have to agree with Chuq; all or nothing seems to be the only sane policy. (and I'm a big Reply-To hater.) From midnight@the-oasis.net Wed Nov 15 03:20:23 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 22:20:23 -0500 Subject: [Mailman-Developers] Info on newlist bug Message-ID: <3A11BAA7.22965.AA4FDB3@localhost> Here is the traceback for the failure. Now that I think about it, it's probably that the bin/newlist executable is looking at the 0 parameter instead of the 1 parameter. For example, on the command line bin/newlist test bin/newlist is parameter 0 test is parameter 1 Notice that the expected directory should have been: '/home/mailman/archives/private/test.mbox' but was instead: '/home/mailman/archives/private/bin/newlist.mbox' I've been looking in the code to see where this might be going wrong, but haven't gotten anywhere. -----------traceback------------- [root@taz2 mailman]# bin/newlist test Enter the email of the person running the list: midnight@the- oasis.net Initial bin/newlist password: Traceback (innermost last): File "bin/newlist", line 213, in ? main(sys.argv) File "bin/newlist", line 163, in main mlist.Create(listname, owner_mail, pw) File "/home/mailman/Mailman/MailList.py", line 775, in Create self.InitVars(name, admin, crypted_password) File "/home/mailman/Mailman/MailList.py", line 335, in InitVars Archiver.InitVars(self) # has configurable stuff File "/home/mailman/Mailman/Archiver/Archiver.py", line 108, in InitVars Utils.mkdir(self.private_archive_file_dir) File "/home/mailman/Mailman/Utils.py", line 568, in mkdir os.mkdir(dir, mode) OSError: [Errno 2] No such file or directory: '/home/mailman/archives/private/bin/newlist.mbox' -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From lovelace@wayfarer.org Wed Nov 15 04:42:53 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Tue, 14 Nov 2000 23:42:53 -0500 Subject: [Mailman-Developers] Re: Reverting question References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> Message-ID: <3A12144D.64C33371@wayfarer.org> Marc MERLIN wrote: > > The irony of this coming back over and over again... > http://www.unicom.com/pw/reply-to-harmful.html One good URL deserves another (straight from the "reply_goes_to_list Option" documentation): http://www.metasystema.org/essays/reply-to-useful.mhtml There are arguments on both sides for this issue. I personally think it depends a lot on what the people on your list prefer. I run a large non-technical mailing list where we tried making the "Reply-To" point to the list for a while. Unfortunately, it resulted in someone accidentally posting a very unflattering message about someone else on the list. Since, this is a non-technical list (actually it's about as non-technical as you can get since it's a list for medieval recreationist list), I can't assume that people will automatically understand the ramifications of the "Reply-To" header so I decided that for my list it would be better to have the "Reply-To" not point to the list. For other lists, however, I think it could be advantageous. Because of that, I think that letting the list administrator decide, rather than the software, is the best method. I also think, though, that sometimes setting the "Reply-To" and sometimes not, depending on whether or not it has already been set has the potential to confuse a lot of people. If you're going to provide *that* functionality, you should make sure it is *extremely* well documented. Tanner Lovelace P.S. I'm kind of curious as to how many people use Mailman with qmail. Are there any statistics on this? I haven't been able to find any. -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? From lovelace@wayfarer.org Wed Nov 15 05:00:15 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Wed, 15 Nov 2000 00:00:15 -0500 Subject: [Mailman-Developers] Re: Info on newlist bug References: <3A11BAA7.22965.AA4FDB3@localhost> Message-ID: <3A12185E.5B84C50B@wayfarer.org> Phil Barnett wrote: > > Here is the traceback for the failure. Now that I think about it, it's > probably that the bin/newlist executable is looking at the 0 > parameter instead of the 1 parameter. > At the beginning of main there is this line of code that parses the arguments. opts, args = getopt.getopt(sys.argv[1:], 'ho:', ['help', 'output=']) getopt takes the passed in arguments (sys.argv[1:]) and removes the ones it takes care of leaving the rest in the "args" variable. Unfortunately, later on when reading the command line arguments they are read from the original argv instead of the modified args. Attached is a patch that fixes this problem. Tanner Lovelace --- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? --- newlist.orig Tue Nov 14 23:48:13 2000 +++ newlist Tue Nov 14 23:55:47 2000 @@ -125,7 +125,7 @@ usage(0) if len(args) > 0: - listname = argv[0] + listname = args[0] else: listname = raw_input("Enter the name of the list: ") listname = string.lower(listname) @@ -137,13 +137,13 @@ usage(1, 'List already exists: ' + listname) if len(args) > 1: - owner_mail = argv[1] + owner_mail = args[1] else: owner_mail = raw_input( "Enter the email of the person running the list: ") if len(args) > 2: - list_pw = argv[2] + list_pw = args[2] else: list_pw = getpass.getpass("Initial %s password: " % listname) # List passwords cannot be empty From lovelace@wayfarer.org Wed Nov 15 05:30:11 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Wed, 15 Nov 2000 00:30:11 -0500 Subject: [Mailman-Developers] Problem archiving mail sent from pine (and other MUAs) Message-ID: <3A121F63.1BF3B141@wayfarer.org> I seem to have found an error where pipermail fails to process mail sent from pine (and possibly other mailers). The message gets added to the .mbox file but nothing else. In the logs/error file I get the following message: Nov 15 00:15:01 2000 qrunner(32366): Traceback (innermost last): Nov 15 00:15:01 2000 qrunner(32366): File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 221, in ArchiveMail Nov 15 00:15:01 2000 qrunner(32366): File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 521, in processUnixMailbox Nov 15 00:15:01 2000 qrunner(32366): File "/usr/lib/mailman/Mailman/Archiver/HyperArch.py", line 224, in __init__ Nov 15 00:15:01 2000 qrunner(32366): TypeError: read-only character buffer, None Nov 15 00:15:01 2000 (32366) CORRUPT ARCHIVE FOR LIST: test Looking at the file HyperArch.py at line 224 I find the following: # Snag the content-* headers. RFC 1521 states that their values are # case insensitive. ctype = message.getheader('Content-Type') or "text/plain" cenc = message.getheader('Content-Transfer-Encoding') self.ctype = string.lower(ctype) self.cenc = string.lower(cenc) (line 224 is the last one.) So, apparently it is failing when it tries to lowercase the "Content-Transfer-Encoding" header string. So, I checked, and sure enough, pine (at least on the two systems I had access to) does not set this header. I then looked at RFC 1521 to see if this header is required. From the RFC (p.14) I quote: These values are not case sensitive. That is, Base64 and BASE64 and bAsE64 are all equivalent. An encoding type of 7BIT requires that the body is already in a seven-bit mail-ready representation. This is the default value -- that is, "Content-Transfer-Encoding: 7BIT" is assumed if the Content-Transfer-Encoding header field is not present. So, the "Contenet-Transfer-Encoding" header is not mandatory and pipermail should not crash when it isn't there. Since I don't actually know python, :-) can anyone suggest an easy fix? It doesn't seem like it would be that hard to fix. I believe this is the same problem that Joshua Burley first reported. Tanner Lovelace P.S. Should I submit this as a bug to sourceforge? -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? From chuqui@plaidworks.com Wed Nov 15 05:22:49 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 21:22:49 -0800 Subject: [Mailman-Developers] [Mailman-Users] Re: Reverting question In-Reply-To: <3A12144D.64C33371@wayfarer.org> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A12144D.64C33371@wayfarer.org> Message-ID: At 11:42 PM -0500 11/14/00, Tanner Lovelace wrote: >There are arguments on both sides for this issue. I >personally think it depends a lot on what the people >on your list prefer. But you need to be wary of making the mistake of "who's making noise" being "what the people on the list prefer". Becaues I've found, if I survey my lists, that the ones making the noise *don't* represent the overall preferences of the list, but are instead a noisy minority. Just because someone's complaining doesn't mean they're right.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From claw@kanga.nu Wed Nov 15 05:44:48 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 14 Nov 2000 21:44:48 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Re: Reverting question In-Reply-To: Message from Chuq Von Rospach of "Tue, 14 Nov 2000 21:22:49 PST." References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A12144D.64C33371@wayfarer.org> Message-ID: <26367.974267088@kanga.nu> On Tue, 14 Nov 2000 21:22:49 -0800 Chuq Von Rospach wrote: > At 11:42 PM -0500 11/14/00, Tanner Lovelace wrote: >> There are arguments on both sides for this issue. I personally >> think it depends a lot on what the people on your list prefer. > But you need to be wary of making the mistake of "who's making > noise" being "what the people on the list prefer". Becaues I've > found, if I survey my lists, that the ones making the noise > *don't* represent the overall preferences of the list, but are > instead a noisy minority. Just because someone's complaining > doesn't mean they're right.... I've found the same statistics. That said, I've deliberately decided to comply with the noisy moinority view for the worst of reasons: I value the contributions of certain members of that minority highly (they occupy useful/divergent views or represent intellectually significant factions of the list topic), and, quite simply, the majority do not disagree enough to act on it (leave the list). An ugly trade-off, but one that (seems to) work. -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From midnight@the-oasis.net Wed Nov 15 05:50:09 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Wed, 15 Nov 2000 00:50:09 -0500 Subject: [Mailman-Developers] Re: Info on newlist bug In-Reply-To: <3A12185E.5B84C50B@wayfarer.org> Message-ID: <3A11DDC1.23494.381810@localhost> On 15 Nov 2000, at 0:00, Tanner Lovelace wrote: > Phil Barnett wrote: > > > > Here is the traceback for the failure. Now that I think about it, > > it's probably that the bin/newlist executable is looking at the 0 > > parameter instead of the 1 parameter. > > > > At the beginning of main there is this line of code > that parses the arguments. > > opts, args = getopt.getopt(sys.argv[1:], 'ho:', > ['help', 'output=']) > > getopt takes the passed in arguments (sys.argv[1:]) > and removes the ones it takes care of leaving the > rest in the "args" variable. Unfortunately, later > on when reading the command line arguments they > are read from the original argv instead of the > modified args. Attached is a patch that fixes > this problem. Great, that fixed newlist. Thanks. At the same time newlist stopped working, my archives and web interfaces all stopped working. I wonder what else is hiding... How do we start troubleshooting this? http://www.matrixlist.com/mailman/listinfo -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From lovelace@wayfarer.org Wed Nov 15 06:32:59 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Wed, 15 Nov 2000 01:32:59 -0500 Subject: [Mailman-Developers] Re: Info on newlist bug References: <3A11DDC1.23494.381810@localhost> Message-ID: <3A122E1B.2546FD1B@wayfarer.org> Phil Barnett wrote: > > Great, that fixed newlist. > On further investigation, there was already a patch on sourceforge for this bug. The patch on sourceforge also modifies one more line that mine didn't that should be changed. The patch can be found at: https://sourceforge.net/patch/download.php?id=102370 Other references: https://sourceforge.net/patch/?func=detailpatch&patch_id=102370&group_id=103 https://sourceforge.net/bugs/?func=detailbug&bug_id=122333&group_id=103 Tanner Lovelace -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? From mailman-users@python.org Wed Nov 15 06:34:41 2000 From: mailman-users@python.org (Marc MERLIN) Date: Tue, 14 Nov 2000 22:34:41 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <3A11F004.13A1BCDF@nleaudio.com>; from bob@nleaudio.com on Tue, Nov 14, 2000 at 09:08:04PM -0500 References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> Message-ID: <20001114223441.G9163@marc.merlins.org> [Reply-To set to mailman-users@python.org] On Tue, Nov 14, 2000 at 06:13:53PM -0800, Chuq Von Rospach wrote: > At 5:50 PM -0800 11/14/00, Marc MERLIN wrote: > >2) If you don't want to take that advise, you certainly shouldn't overwrite > > a users' reply to with the list's > > I'll disagree. If you feel you must coerce reply-to, coerce it > unconditionally. You've already decided the list's needs outweigh the A few others mentionned that. The problem here, is that by putting a reply-to, you are already trying to prevent me from replying directly to the poster (well, not quite since mutt lets me tell it to ignore all those reply-tos), but by munging reply tos unconditionally, you are completely preventing me from replying to someone who has a non valid from address and set a good reply-to, or someone who'd like me to reply to him at another address. Doing a reply-to rewrite also prevents me from Ccing a second list and setting a reply-to there to move the discussion. That said, I could see why others would prefer unconditionally overwriting the reply-to. Well, in both cases, you lose, which is yet another reason why lists should not set a reply-to :-) I think I agree with JC in saying that short of removing the reply to misfeature altogether, unconditional rewriting should probably be an option On Wed, Nov 15, 2000 at 12:27:14PM +0800, Gregory Leblanc wrote: > Fine, that's all well and good, munging reply to is harmful. Sometimes > the harm that it causes is less than the harm that it prevents. I've yet to have a single person show me what harm not munging reply-to creates... It's all about users that have been trained to do the wrong thing and some of them who then believe it's the only way to go. > This is a good policy in most cases, but there ARE some where it doesn't > make sense. Yep, the only one I know is onelist which tries to force as many traffic back to the list in order to spam you with more messages which each contain a small add they make money on. Ok, more seriously, the only downside of group replies is that for MUAs that don't support list-reply, like mutt does, the sender gets two copies of the answer (which are easy to get rid off by removing duplicate messages that contain the same message id) > I'm sorry, but this is silly. If you're going to write reply-to headers > via your MLM, you need to write them ALL the time, otherwise you end up > with strange and un-predictable behaviour. Keep the reply-to header Point taken. The other option is wrong too though, for the reason I explained. But I guess if you choose to shoot yourself in the foot, it'd be nice to be able to chose which gun you're going to use (i.e. make the unconditional rewriting an option) > certainly, but if headers are sending mail back to the list, the user's > reply-to really doesn't make much difference, unless you need to send a > private reply. If you're sending a private reply, you can get it out of > the x-reply-to header. Great, I need to patch my MUA some more to attempt to deal with all this misconfiguration nonsense. On Tue, Nov 14, 2000 at 11:42:53PM -0500, Tanner Lovelace wrote: > Marc MERLIN wrote: > > > > The irony of this coming back over and over again... > > http://www.unicom.com/pw/reply-to-harmful.html > > One good URL deserves another (straight from the > "reply_goes_to_list Option" documentation): > > http://www.metasystema.org/essays/reply-to-useful.mhtml Yeah, except that the author is full of shit. Ok, I'll bite. - "Reply-To gives the respondant an option which would not otherwise exist: namely the ability to reply only to the list." false, see list-reply function in good MUAs (mutt is one) - "If you use a reasonable mailer, Reply-To munging does provide new functionality, namely the ability to reply only to the list" No comment... (the author completely ignores list-reply, and says that MUAs should all be patched to allow overriding reply-tos so that they can continue to be misused) - "Reply-To munging adds additional functionality, it actually increases freedom of choice" *cough* bullshit *cough* For one, it prevents a thread across two lists and the redirection to one list or a third list. - some mailers apparently do not have a separate reply to all function Ok, 1) I have yet to see one 2) in the point above, the author was saying that any MUA that can't override a reply-to is not a reasonable mailer and shouldn't be used. The irony... - "For discussion type lists, I would estimate that ninety percent of the time, people want to reply to the list" Yeah, so? That's why you have several reply functions, where one does not require more work than the other (the author mentions having to type addresses by hand, I'm not sure why) - "Some administrators claim that munging Reply-To headers is harmful because it surprises people, and can cause damage when things go awry" Damn straight. If you reply to the list by mistake due to reply-to munging, you look like an idiot, or worse. If you reply to the sender only by mistake, no harm is done, the mistake can be fixed. - "It's What People Want" Yeah, there are misguided people out there, and they are often vocal. Users typically know nothing about mail standards or what header sender vs envelope sender means. I don't think there opinion is that relevant :-) Ok, more seriously, users do matter but if they're trained the wrong way when they start, some get very resistant to change, regardless of whether it's a good thing or not. We did get a bit off topic here, I apologize to all the people who already know about all this, but since we have lots of listmasters here, I'm hoping a few with switch, and hopefully more will configure their lists the right way in the first place (that's when it's easy, changing is hard) > There are arguments on both sides for this issue. I personally think it > depends a lot on what the people on your list prefer. I run a large > non-technical mailing list where we tried making the "Reply-To" point to > the list for a while. Unfortunately, it resulted in someone accidentally > posting a very unflattering message about someone else on the list. Yep, this happens all the time. Many of us have seen this. > the "Reply-To" not point to the list. For other lists, however, I think > it could be advantageous. Because of that, I think that letting the list > administrator decide, rather than the software, is the best method. While I'm very much against reply-to munging (duh!), I'm for freedom of choice. I do think that mailman should offer a reply-to option, but with a very big disclaimer and maybe this discussion (or another one) so that inexperienced listmasters realize what they're doing. > I also think, though, that sometimes setting the "Reply-To" and sometimes > not, depending on whether or not it has already been set has the > potential to confuse a lot of people. If you're going to provide *that* > functionality, you should make sure it is *extremely* well documented. Agreed. Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From claw@kanga.nu Wed Nov 15 06:57:21 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 14 Nov 2000 22:57:21 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: Message from Marc MERLIN of "Tue, 14 Nov 2000 22:34:41 PST." <20001114223441.G9163@marc.merlins.org> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> Message-ID: <32332.974271441@kanga.nu> On Tue, 14 Nov 2000 22:34:41 -0800 Marc MERLIN wrote: > Ok, more seriously, the only downside of group replies is that for > MUAs that don't support list-reply, like mutt does, the sender > gets two copies of the answer (which are easy to get rid off by > removing duplicate messages that contain the same message id) On *-ix platforms which have procmail/formail. Windows, MacOS, and BeOS users, to name but a few, have slightly more difficulty on this area. Sad, but true. >> http://www.metasystema.org/essays/reply-to-useful.mhtml > Yeah, except that the author is full of shit. Key points: This argument is a FAQ I've yet to see a single case where someone has been persuaded by the arguments of the other side once the argument has begun. Now, can we get on with some more productive discussions, like how vi users are scum-sucking brainless cretins and emacs users are the obviously intelligent few who can see and use true brilliance? > I do think that mailman should offer a reply-to option, but with a > very big disclaimer and maybe this discussion (or another one) so > that inexperienced listmasters realize what they're doing. I note that this is already done. > Microsoft is to operating systems & security Now, about them silly sendmail users... -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From mailman-users@python.org Wed Nov 15 07:39:43 2000 From: mailman-users@python.org (Marc MERLIN) Date: Tue, 14 Nov 2000 23:39:43 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <32332.974271441@kanga.nu>; from claw@kanga.nu on Tue, Nov 14, 2000 at 10:57:21PM -0800 References: <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> <32332.974271441@kanga.nu> Message-ID: <20001114233943.B21148@marc.merlins.org> On Tue, Nov 14, 2000 at 10:57:21PM -0800, J C Lawrence wrote: > I've yet to see a single case where someone has been persuaded by > the arguments of the other side once the argument has begun. I've been on several lists that have been converted away from reply-to munging. I've never been on any list that was converted to reply-to munging. 'nuf said. > > I do think that mailman should offer a reply-to option, but with a > > very big disclaimer and maybe this discussion (or another one) so > > that inexperienced listmasters realize what they're doing. > > I note that this is already done. I'm basically replying to the list to apologize on this issue. Indeed mailman already gives plenty of warning about this, I just never saw the configuration help page > > Microsoft is to operating systems & security > > Now, about them silly sendmail users... Troll :-) Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From chuqui@plaidworks.com Wed Nov 15 07:31:27 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 23:31:27 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <32332.974271441@kanga.nu> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> <32332.974271441@kanga.nu> Message-ID: At 10:57 PM -0800 11/14/00, J C Lawrence wrote: > I've yet to see a single case where someone has been persuaded by > the arguments of the other side once the argument has begun. isn't this where someone's supposed to call the other a fascist and make a spurious Hitler reference? >Now, can we get on with some more productive discussions, like how >vi users are scum-sucking brainless cretins and emacs users are the >obviously intelligent few who can see and use true brilliance? What's the difference between an emacs user and a vi user? Give each a file and a set of changes. the vi user sits down and hacks in the changes, and goes to lunch. the emacs user sits down and writes a macro that'll make the changes while he's at lunch. At the end, they both have the changed file, but the emacs user has a macro he'll put in his library and never use again. > > Microsoft is to operating systems & security First, get an OS that doesn't hate you... >Now, about them silly sendmail users... (sticks out tongue) -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From chuqui@plaidworks.com Wed Nov 15 07:47:51 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 23:47:51 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <20001114233943.B21148@marc.merlins.org> References: <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> <32332.974271441@kanga.nu> <20001114233943.B21148@marc.merlins.org> Message-ID: At 11:39 PM -0800 11/14/00, Marc MERLIN wrote: >I've been on several lists that have been converted away from reply-to >munging. >I've never been on any list that was converted to reply-to munging. >'nuf said. I've actually experimented on my users. I've had lists that started with no reply-to, added it for a while, then removed it. And monitored preferences and problems. All my discussion lists are not coerced -- that's what people ended up preferring, and the problems are significantly lessened (we won't even get into vacation bots that get reply-toed onto the list and through thousands of messages onto the list before the admin wakes up....) > > Now, about them silly sendmail users... > >Troll :-) You rang? -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From barry@digicool.com Wed Nov 15 12:49:49 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 15 Nov 2000 07:49:49 -0500 (EST) Subject: [Mailman-Developers] Re: Info on newlist bug References: <3A12185E.5B84C50B@wayfarer.org> <3A11DDC1.23494.381810@localhost> Message-ID: <14866.34413.539257.537295@anthem.concentric.net> Below is the bin/newlist patch I just checked in. >>>>> "PB" == Phil Barnett writes: PB> At the same time newlist stopped working, my archives and web PB> interfaces all stopped working. I wonder what else is PB> hiding... Take a look at the current CVS snapshot. I think those bugs (well, at least the archives one) should be fixed. -Barry -------------------- snip snip -------------------- Index: newlist =================================================================== RCS file: /cvsroot/mailman/mailman/bin/newlist,v retrieving revision 1.35 retrieving revision 1.36 diff -u -r1.35 -r1.36 --- newlist 2000/11/10 18:42:46 1.35 +++ newlist 2000/11/15 12:49:18 1.36 @@ -18,10 +18,16 @@ """Create a new, unpopulated mailing list. -Usage: %(PROGRAM)s +Usage: %(PROGRAM)s [options] listname listadmin-addr admin-password Options: + -q + --quiet + Normally the administrator is notified by email (after a prompt) that + their list has been created. This option suppresses that + notification. + -o file --output=file Append the alias setting recommendations to file, in addition to @@ -110,22 +116,25 @@ -def main(argv): +def main(): try: - opts, args = getopt.getopt(sys.argv[1:], 'ho:', - ['help', 'output=']) + opts, args = getopt.getopt(sys.argv[1:], 'ho:q', + ['help', 'output=', 'quiet']) except getopt.error, msg: usage(1, msg) appendfile = None + quiet = 0 for opt, arg in opts: if opt in ('-o', '--output'): appendfile = arg if opt in ('-h', '--help'): usage(0) + if opt in ('-q', '--quiet'): + quiet = 1 if len(args) > 0: - listname = argv[0] + listname = args[0] else: listname = raw_input("Enter the name of the list: ") listname = string.lower(listname) @@ -137,13 +146,13 @@ usage(1, 'List already exists: ' + listname) if len(args) > 1: - owner_mail = argv[1] + owner_mail = args[1] else: owner_mail = raw_input( "Enter the email of the person running the list: ") if len(args) > 2: - list_pw = argv[2] + list_pw = args[2] else: list_pw = getpass.getpass("Initial %s password: " % listname) # List passwords cannot be empty @@ -186,28 +195,29 @@ fp.write('\n') fp.close() - if len(argv) < 5: + if not quiet: print ("Hit enter to continue with %s owner notification..." % listname), sys.stdin.readline() - # send the notice to the list owner - text = Utils.maketext( - 'newlist.txt', - {'listname' : listname, - 'password' : list_pw, - 'admin_url' : mlist.GetScriptURL('admin', absolute=1), - 'listinfo_url': mlist.GetScriptURL('listinfo', absolute=1), - 'requestaddr' : "%s-request@%s" % (listname, mlist.host_name), - 'hostname' : mlist.host_name, - }) - msg = Message.UserNotification(owner_mail, - 'mailman-owner@' + mlist.host_name, - 'Your new mailing list: ' + listname, - text) - HandlerAPI.DeliverToUser(mlist, msg) + # send the notice to the list owner + text = Utils.maketext( + 'newlist.txt', + {'listname' : listname, + 'password' : list_pw, + 'admin_url' : mlist.GetScriptURL('admin', absolute=1), + 'listinfo_url': mlist.GetScriptURL('listinfo', absolute=1), + 'requestaddr' : "%s-request@%s" % (listname, mlist.host_name), + 'hostname' : mlist.host_name, + }) + msg = Message.UserNotification( + owner_mail, + 'mailman-owner@' + mlist.host_name, + 'Your new mailing list: ' + listname, + text) + HandlerAPI.DeliverToUser(mlist, msg) finally: mlist.Unlock() if __name__ == '__main__': - main(sys.argv) + main() From claw@kanga.nu Wed Nov 15 21:18:59 2000 From: claw@kanga.nu (J C Lawrence) Date: Wed, 15 Nov 2000 13:18:59 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? Message-ID: <12203.974323139@kanga.nu> I'm looking to setup a test Mailman site under Postfix (need to do some performance modelling of Exim against Postfix under Mailman for a given membership list). I can get a basic setup working no problem -- now I'm looking to duplicate the neater aspects I already have under Exim, such as losing the Mailman aliases file, etc. Anybody already done this? -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From gerald@impressive.net Wed Nov 15 22:39:27 2000 From: gerald@impressive.net (Gerald Oskoboiny) Date: Wed, 15 Nov 2000 17:39:27 -0500 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: <14865.27405.857124.313441@anthem.concentric.net>; from barry@digicool.com on Tue, Nov 14, 2000 at 11:40:45AM -0500 References: <14865.27405.857124.313441@anthem.concentric.net> Message-ID: <20001115173927.E11322@impressive.net> On Tue, Nov 14, 2000 at 11:40:45AM -0500, Barry A. Warsaw wrote: > >>>>> "CVR" == Chuq Von Rospach writes: > > CVR> In general I'm against coercing Reply-To. Strongly. > > Me too. Me too. > The primary complaint is that people accidently send private responses > to the whole list. This happens because they've trained themselves to > know the difference between replies and followups, and have MUAs that > separate those functions. Yes, and that's definitely a Good Thing: it should be a very conscious choice whether to send a private reply or to send mail to possibly hundreds or thousands of people. > The only grumblings occur when people don't > trim their CC headers (like I've done here) and folks start getting > duplicates. That can be handled in other ways (e.g. the list /could/ > suppress deliveries to list members that it sees explicitly in the > recipients list, although as we've discussed before, that has some > potential for abuse). Another solution to this is the Mail-Followup-To header: (not widely supported yet) http://cr.yp.to/proto/replyto.html (related info for mutt users: http://larve.net/people/hugo/2000/07/ml-mutt ) -- Gerald Oskoboiny http://impressive.net/people/gerald/ From barry@digicool.com Thu Nov 16 04:34:31 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 15 Nov 2000 23:34:31 -0500 (EST) Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> Message-ID: <14867.25559.227185.305800@anthem.concentric.net> >>>>> "MM" == Marc MERLIN writes: MM> While I'm very much against reply-to munging (duh!), I'm for MM> freedom of choice. I do think that mailman should offer a MM> reply-to option, but with a very big disclaimer and maybe this MM> discussion (or another one) so that inexperienced listmasters MM> realize what they're doing. Marc, Very nice rebuttal to the rebuttal to the reply-to-considered-harmful article. You should write that up nicely and put it on the web some place. I'd love to point to it in the details page. BTW, I've applied the unconditional Reply-To: munging patch. -Barry From chuqui@plaidworks.com Thu Nov 16 05:07:22 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 15 Nov 2000 21:07:22 -0800 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: <20001115173927.E11322@impressive.net> References: <14865.27405.857124.313441@anthem.concentric.net> <20001115173927.E11322@impressive.net> Message-ID: At 5:39 PM -0500 11/15/00, Gerald Oskoboiny wrote: >Another solution to this is the Mail-Followup-To header: >(not widely supported yet) > > http://cr.yp.to/proto/replyto.html No, it's actually stillborn... dan may still be pushing it, but I don't think anyone else is. Since the new RFC supports list-post, we really don't NEED Mail-Followup-To. Instead, an MUA can look for the existance of List-Post and use it to enable reply-to-list. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From gerald@impressive.net Thu Nov 16 05:48:57 2000 From: gerald@impressive.net (Gerald Oskoboiny) Date: Thu, 16 Nov 2000 00:48:57 -0500 Subject: Mail-Followup-To (was Re: [Mailman-Developers] Re: Reverting question) In-Reply-To: ; from chuqui@plaidworks.com on Wed, Nov 15, 2000 at 09:07:22PM -0800 References: <14865.27405.857124.313441@anthem.concentric.net> <20001115173927.E11322@impressive.net> Message-ID: <20001116004857.F11322@impressive.net> On Wed, Nov 15, 2000 at 09:07:22PM -0800, Chuq Von Rospach wrote: > At 5:39 PM -0500 11/15/00, Gerald Oskoboiny wrote: > > >Another solution to this is the Mail-Followup-To header: > >(not widely supported yet) > > > > http://cr.yp.to/proto/replyto.html > > No, it's actually stillborn... That's a matter of opinion :) > dan may still be pushing it, but I don't think anyone else is. There's me, and a few other people... (and you, once I convince you ;) I just pointed to Dan's page since that's the best writeup of it that I have found. > Since the new RFC supports list-post, we really don't NEED > Mail-Followup-To. Instead, an MUA can look for the existance of > List-Post and use it to enable reply-to-list. That doesn't work for crossposted threads (like this one), or for non-subscribers who want to be Cc'd on a given thread. -- Gerald Oskoboiny http://impressive.net/people/gerald/ From barry@digicool.com Thu Nov 16 13:54:16 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 08:54:16 -0500 (EST) Subject: [Mailman-Developers] Re: Reverting question References: <14865.27405.857124.313441@anthem.concentric.net> <20001115173927.E11322@impressive.net> Message-ID: <14867.59144.179977.605777@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> Since the new RFC supports list-post, we really don't NEED CVR> Mail-Followup-To. Instead, an MUA can look for the existance CVR> of List-Post and use it to enable reply-to-list. Chuq, are you referring to draft-ietf-drums-msg-fmt-09.txt of 11-Sep-2000? There's no mention of List-Post in there that I can find. -Barry From Nigel.Metheringham@VData.co.uk Thu Nov 16 17:19:52 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Thu, 16 Nov 2000 17:19:52 +0000 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Thu, 16 Nov 2000 08:54:16 EST." <14867.59144.179977.605777@anthem.concentric.net> Message-ID: For those with too little to do and read, there is another discussion on exactly this sort of topic going on over in the exmh lists (exmh is a MUA). See http://www.xray.mpe.mpg.de/mailing-lists/exmh/ specifically the thread over October and November entitled "state of the world" and also "Reply magic for lists" Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From lohner@debian.org Thu Nov 16 17:32:53 2000 From: lohner@debian.org (Nils Lohner) Date: Thu, 16 Nov 2000 18:32:53 +0100 Subject: [Mailman-Developers] mailman 2.0 beta 5 problem Message-ID: <200011161732.SAA05076@topaze.ecf.teradyne.com> I've got a problem with mailman. I installed the debian package ii mailman 2.0beta5-1 Powerful, web based list processor and after setting up a list tried the menus. I've got several problems and questions. - in the apache error log: client denied by server configuration: /usr/doc/mailman/images/mailman.jpg I've tried allowing everything to man /usr/doc/ work as in the Readme.debian (in the .deb) but no luck. - when I submit my changes to something it tries to contact www.localhost.com This seems to be exclusively a cgi problem. - when at http://131.101.98.199/mailman/admin the link for "General list information can be found at the mailing list overview page." points up one level (i.e. http://131.101.98.199/listinfo) I'll dig into this more tomorrow... help appreciated. Thanks, Nils. From barry@digicool.com Thu Nov 16 18:16:52 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 13:16:52 -0500 (EST) Subject: [Mailman-Developers] Re: Reverting question References: <14865.27405.857124.313441@anthem.concentric.net> <20001115173927.E11322@impressive.net> Message-ID: <14868.9364.657224.536341@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> Since the new RFC supports list-post, we really don't NEED CVR> Mail-Followup-To. Instead, an MUA can look for the existance CVR> of List-Post and use it to enable reply-to-list. Duh. List-Post: is in RFC 2369 and of course Mailman already inserts this header in messages. Sad what happens to short term memory when you get older. :) BTW, I think this is the right bit of information that an MUA would need to support a "post-to-list" option. Seems to me all the information is there without Reply-To munging for the user agent to support whatever followup/reply operation it wants. Reply-To munging should be considered a hack that's (questionably) necessary until the MUAs catch on. -Barry From barry@digicool.com Thu Nov 16 20:12:05 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 15:12:05 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? References: <12203.974323139@kanga.nu> Message-ID: <14868.16277.437055.764467@anthem.concentric.net> >>>>> "JCL" == J C Lawrence writes: JCL> I'm looking to setup a test Mailman site under Postfix (need JCL> to do some performance modelling of Exim against Postfix JCL> under Mailman for a given membership list). I can get a JCL> basic setup working no problem -- now I'm looking to JCL> duplicate the neater aspects I already have under Exim, such JCL> as losing the Mailman aliases file, etc. Anybody already JCL> done this? Nope, but I agree, this is one of the cooler features of Exim. -Barry From claw@kanga.nu Thu Nov 16 20:22:59 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 16 Nov 2000 12:22:59 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Thu, 16 Nov 2000 15:12:05 EST." <14868.16277.437055.764467@anthem.concentric.net> References: <12203.974323139@kanga.nu> <14868.16277.437055.764467@anthem.concentric.net> Message-ID: <9116.974406179@kanga.nu> On Thu, 16 Nov 2000 15:12:05 -0500 Barry A Warsaw wrote: >>>>>> "JCL" == J C Lawrence writes: JCL> I'm looking to setup a test Mailman site under Postfix (need to JCL> do some performance modelling of Exim against Postfix under JCL> Mailman for a given membership list). I can get a basic setup JCL> working no problem -- now I'm looking to duplicate the neater JCL> aspects I already have under Exim, such as losing the Mailman JCL> aliases file, etc. Anybody already done this? > Nope, but I agree, this is one of the cooler features of Exim. Yup, it sure is. That said I'm finding myself a little dissapointed in the configurability of Postfix in such regards. There's a whole lot of simple clevers you can do with Exim's filtering intelligence that just doesn't seem to be possible with Postfix. Another simple example is Exim's delivery time filters: http://www.exim.org/exim-html-3.10/doc/html/spec_14.html#SEC388 Very handy that. I've no idea yet how I'd do such under Postfix. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From barry@digicool.com Thu Nov 16 20:30:32 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 15:30:32 -0500 Subject: [Mailman-Developers] Future of pipermail? References: Message-ID: <14868.17384.847069.558207@anthem.concentric.net> >>>>> "GE" == Gunnar Evermann writes: GE> What are the current plans about the html archiver in mailman? GE> The major shortcomings of pipermail I see are: | - doesn't decode MIME | - doesn't handle mixed charsets | - no search engine | - somewhat convoluted code. GE> I run a mailing list with a lot of subscribers from Asia who GE> use MS Outlook, invariably posting (English) messages GE> spuriously tagged as CJK charsets (but only using the ASCII GE> subset) and gratuitously encoded in base64. Therefore I am GE> motivated to find a solution to handling MIME. :-( GE> I have hacked base64 support into pipermail and am currently GE> looking at supporting MIME (probably using mimecntl.py), but GE> this will just be short-term fixes. GE> I know I could just switch to MHonarc or whatever but I like GE> the idea of having a nicely structured, modular archiver GE> tightly integrated with mailman. Then why are you using Pipermail. :) Anyway, yes, we'd like to rewrite the archiver from the ground up, but nobody's working on that yet. I suspect either Jeremy or I will tackle it at some point. In the meantime, if you have useful patches, upload them to SourceForge so they don't get lost. -Barry From otaylor@redhat.com Thu Nov 16 21:09:38 2000 From: otaylor@redhat.com (Owen Taylor) Date: 16 Nov 2000 16:09:38 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: J C Lawrence's message of "Wed, 15 Nov 2000 13:18:59 -0800" References: <12203.974323139@kanga.nu> Message-ID: --=-=-= J C Lawrence writes: > I'm looking to setup a test Mailman site under Postfix (need to do > some performance modelling of Exim against Postfix under Mailman for > a given membership list). I can get a basic setup working no > problem -- now I'm looking to duplicate the neater aspects I already > have under Exim, such as losing the Mailman aliases file, etc. > Anybody already done this? Well, you can't lose the aliases file, but you can make it pretty darn invisible. From postfix's main.cf: alias_maps = hash:/etc/aliases,hash:/home/mailman/aliases And the following patch automatically updates the file. Regards, Owen --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=mailman-2.0beta5-postfix.patch --- mailman-2.0beta5/bin/newlist.postfix Sat Aug 5 18:24:11 2000 +++ mailman-2.0beta5/bin/newlist Sat Aug 5 18:26:08 2000 @@ -56,6 +56,14 @@ %(owner2)s %(listname)s-admin ''' +ALIASFILETEMPLATE = ''' +## %(listname)s mailing list +## created: %(date)s %(user)s +%(list)s "|%(wrapper)s post %(listname)s" +%(admin)s "|%(wrapper)s mailowner %(listname)s" +%(request)s "|%(wrapper)s mailcmd %(listname)s" +%(owner2)s %(listname)s-admin +''' def getusername(): @@ -117,7 +125,11 @@ except Errors.MMListAlreadyExistsError: usage(1, 'List already exists: ' + listname) - print ALIASTEMPLATE % { + aliaspath = os.path.join(mm_cfg.PREFIX, 'aliases') + print "Appending aliases to " + aliaspath + + aliasfile = open(aliaspath, "a") + aliasfile.write (ALIASFILETEMPLATE % { 'listname': listname, 'list' : "%-24s" % (listname + ":"), 'wrapper' : '%s/wrapper' % mm_cfg.WRAPPER_DIR, @@ -126,8 +138,12 @@ 'owner2' : "%-24s" % ("%s-owner:" % listname), 'date' : time.strftime('%d-%b-%Y', time.localtime(time.time())), 'user' : getusername(), - } + }) + aliasfile.close() + print "Rehashing alias database" + os.system ("/usr/sbin/postalias hash:" + aliaspath) + if len(argv) < 5: print ("Hit enter to continue with %s owner notification..." % listname), --=-=-=-- From barry@digicool.com Thu Nov 16 21:22:20 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 16:22:20 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? References: <12203.974323139@kanga.nu> Message-ID: <14868.20492.287182.956825@anthem.concentric.net> >>>>> "OT" == Owen Taylor writes: OT> Well, you can't lose the aliases file, but you can make it OT> pretty darn invisible. >> From postfix's main.cf: OT> alias_maps = hash:/etc/aliases,hash:/home/mailman/aliases I do this. OT> And the following patch automatically updates the file. A couple of notes. First, the version of newlist that's going to be in rc3 (some time today) has a -o/--output switch that does the appending to a specified file, somewhat like the way your patch does (although the file name is taken from the command line and a few other minor differences). But the real problem is a permissions problem. Say you run postalias as yourself, because you installed Mailman. Or say you installed it as user `mailman', and that's the user you run postalias as. Then the resulting aliases.db file will be owned by you (or `mailman') and Postfix will try to deliver email destined for those addresses as your (or mailman's) gid, but not as the gid you compiled into the wrapper script. Thus I found that I had to run newaliases (a.k.a. postalias) as root for the gids to be correct. Not an insurmountable problem, but it's a pain. It would be nice if Postfix could be configured to automatically run newaliases if necessary (maybe there already is such an option?). -Barry From barry@digicool.com Thu Nov 16 22:35:03 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 17:35:03 -0500 Subject: [Mailman-Developers] Announcing Mailman 2.0 release candidate 3 Message-ID: <14868.24855.54781.693947@anthem.concentric.net> Mailman 2.0 release candidate 3 is now available from SourceForge at http://sourceforge.net/project/showfiles.php?group_id=103 www.list.org should be updated soon (I'm not going to bother the gnu.org guys again until 2.0 final is released). Excerpts from the NEWS file are given below. This should quash the few known 2.0rc2 buglets. Of course the on-line documentation is available at: http://mailman.sourceforge.net New goal for 2.0 final: Wed 22-Nov-2000 Enjoy, -Barry -------------------- snip snip -------------------- 2.0 release candidate 3 (16-Nov-2000) - By popular demand, Reply-To: munging policy is now to always override any Reply-To: header in the original message, if reply_goes_to_list is set to "This list" or "Explicit Address" - bin/newlist given -q/--quiet flag instead of the positional argument - Hopefully last fix to DEFAULT_URL not ending in a slash sensitivity - 2.0rc2 buglets fixed: o newlist argument parsing o updating with unlocked lists o HyperArch.py traceback when there's no Content-Transfer-Encoding: header - SourceForge bugs fixed: 122358 (qmail-to-mailman.py listname case folding) - SourceForge patches applied: 102373 (qmail-to-mailman.py listname case folding) From otaylor@redhat.com Thu Nov 16 23:11:21 2000 From: otaylor@redhat.com (Owen Taylor) Date: 16 Nov 2000 18:11:21 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: barry@digicool.com's message of "Thu, 16 Nov 2000 16:22:20 -0500" References: <12203.974323139@kanga.nu> <14868.20492.287182.956825@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > >>>>> "OT" == Owen Taylor writes: > > OT> Well, you can't lose the aliases file, but you can make it > OT> pretty darn invisible. > > >> From postfix's main.cf: > > OT> alias_maps = hash:/etc/aliases,hash:/home/mailman/aliases > > I do this. > > OT> And the following patch automatically updates the file. > > A couple of notes. First, the version of newlist that's going to be > in rc3 (some time today) has a -o/--output switch that does the > appending to a specified file, somewhat like the way your patch does > (although the file name is taken from the command line and a few > other minor differences). Hmmm, it seems to me that that should be in the config system somehow - so everybody running newalias doesn't need to know to use the flag. (I can set up a wrapper script, but that isn't so nice.) > But the real problem is a permissions problem. Say you run postalias > as yourself, because you installed Mailman. Or say you installed it > as user `mailman', and that's the user you run postalias as. Then the > resulting aliases.db file will be owned by you (or `mailman') and > Postfix will try to deliver email destined for those addresses as your > (or mailman's) gid, but not as the gid you compiled into the wrapper > script. > > Thus I found that I had to run newaliases (a.k.a. postalias) as root > for the gids to be correct. Well, I have the mailman gid as the gid compiled into the wrapper script, so all mailman processesing occurs with the mailman uid/gid. I guess this doesn't work in general, though it works well for our setup. > Not an insurmountable problem, but it's a pain. It would be nice if > Postfix could be configured to automatically run newaliases if > necessary (maybe there already is such an option?). Well, it wouldn't very secure if a user could put something into an alias file, and then postfix would compile that into an alias.db that is executed with root permissions... Basically, from a security point of view, the person composing the alias file can put arbitrary commands into the alias file, so the execution of commands in the alias file has to be done with the permissions of the owner of the file. Regards, Owen From andrewm@connect.com.au Fri Nov 17 00:04:39 2000 From: andrewm@connect.com.au (Andrew McNamara) Date: Fri, 17 Nov 2000 11:04:39 +1100 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Your message of "Thu, 16 Nov 2000 15:12:05 CDT." <14868.16277.437055.764467@anthem.concentric.net> Message-ID: <20001117000439.80F3B2870E@wawura.off.connect.com.au> > JCL> I'm looking to setup a test Mailman site under Postfix (need > JCL> to do some performance modelling of Exim against Postfix > JCL> under Mailman for a given membership list). I can get a > JCL> basic setup working no problem -- now I'm looking to > JCL> duplicate the neater aspects I already have under Exim, such > JCL> as losing the Mailman aliases file, etc. Anybody already > JCL> done this? > >Nope, but I agree, this is one of the cooler features of Exim. Can you give a brief precis of how Exim presents this to the admin? I might be able to suggest an equivilent feature, or at least inject the idea into Postfix's development. --- Andrew McNamara (System Architect) connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111 From claw@kanga.nu Fri Nov 17 00:08:15 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 16 Nov 2000 16:08:15 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from Andrew McNamara of "Fri, 17 Nov 2000 11:04:39 +1100." <20001117000439.80F3B2870E@wawura.off.connect.com.au> References: <20001117000439.80F3B2870E@wawura.off.connect.com.au> Message-ID: <29159.974419695@kanga.nu> On Fri, 17 Nov 2000 11:04:39 +1100 Andrew McNamara wrote: > Can you give a brief precis of how Exim presents this to the > admin? I might be able to suggest an equivilent feature, or at > least inject the idea into Postfix's development. See http://www.exim.org/howto/mailman.html -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From andrewm@connect.com.au Fri Nov 17 00:29:31 2000 From: andrewm@connect.com.au (Andrew McNamara) Date: Fri, 17 Nov 2000 11:29:31 +1100 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Your message of "Thu, 16 Nov 2000 12:22:59 -0800." <9116.974406179@kanga.nu> Message-ID: <20001117002931.CBE5A2870A@wawura.off.connect.com.au> >Yup, it sure is. That said I'm finding myself a little dissapointed >in the configurability of Postfix in such regards. There's a whole >lot of simple clevers you can do with Exim's filtering intelligence >that just doesn't seem to be possible with Postfix. Another simple >example is Exim's delivery time filters: > > http://www.exim.org/exim-html-3.10/doc/html/spec_14.html#SEC388 > >Very handy that. I've no idea yet how I'd do such under Postfix. If you are using one of the recent Postfix snapshots, it has this functionality. It's documented in the file FILTER_README in the top level directory. I realise that a snapshot isn't appropriate for some sites (however, Wietse's definition of a snapshot is simply a release with features that may not be carried through - the code receives his usual extensive testing before release). --- Andrew McNamara (System Architect) connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111 From fhorch@ecoaccess.org Fri Nov 17 01:22:02 2000 From: fhorch@ecoaccess.org (Fred Wilson Horch) Date: Thu, 16 Nov 2000 20:22:02 -0500 Subject: [Mailman-Developers] Running Mailman CGI under Zope ZServer Message-ID: <3A14883A.D1A8B45@ecoaccess.org> Hello out there, I'm looking for someone with some more Zope "Zen" to help me figure out how to add support for legacy CGI scripts (specifically Mailman 1.1) to Zope. To recap, I know and understand how to get Apache to play nicely with Zope. What I'm trying to do is remove the need for Apache by allowing ZServer to serve CGI scripts itself. The motivation is to allow me to run Mailman 1.1 on my Zope site without having to have Apache (or any other web server) configured and running. First, am I really the first person to try running CGI scripts from ZServer? I have found some hints here and there of people doing somewhat similar things, but I haven't yet found a product for easily adding legacy CGI scripts to a Zope site. It seems most people run Zope behind Apache. Is ZServer really slow or buggy or something? Second, is Mailman going to be integrated with Zope? It seems like a natural fit. It would be nice to move away from the pipermail archiving and use the Zope object database for archiving messages instead. Third, thanks to code from Eric Walstad and Chris Withers (author of PathHandler), I have hacked up an external method that sort of works. It attempts to add to Zope the functionality of ScriptAlias in Apache. But I'm running into some problems with the HTTP header. The attached code simply ignores the HTTP header sent by the Mailman CGI script, letting Zope do its thing. But this becomes a problem when Mailman wants to set cookies for authentication. Can anyone help me figure out how to convince Zope to let the CGI script send both the HTTP header and message body? Thanks, Fred Here's my external method. It is called by a Path Handler object. Some notes: 1) REQUEST['PATH_INFO'] seems to be brain dead 2) the path_to_handle key is inserted by the PathHandler product 3) this is written for Linux; I'm not sure how Windows handles os.popen 4) I don't understand why sometimes this function gets called with one argument and sometimes with two -- I always just ignore the second argument import os, os.path, string def testing(self, second=None): if self.REQUEST.has_key('path_to_handle'): script = self.REQUEST['path_to_handle'][0] if script: script_path = '/home/mailman/cgi-bin/%s' % script if os.path.isfile(script_path): ENV = '' if len(self.REQUEST['path_to_handle']) > 1: ENV = ENV + 'PATH_INFO=/%s' % string.join(self.REQUEST['path_to_handle'][1:],'/') elif self.REQUEST['PATH_INFO'][-1] == '/': ENV = ENV + 'PATH_INFO=/' ENV = ENV + ' HTTP_HOST=%s' % self.REQUEST['HTTP_HOST'] ENV = ENV + ' SERVER_NAME=%s' % self.REQUEST['SERVER_NAME'] f = os.popen("/usr/bin/env %s %s" % \ (ENV, script_path)) header = 1 while header: if (f.readline() == '\n'): header = 0 output = f.read() # grab the output of the CGI here status = f.close() else: return '%s: CGI script not found.' % script else: return 'No CGI script specified.' else: return 'No path to handle.' return output + '

' + str(self.REQUEST.keys()) + '
' + str(self.REQUEST) From claw@kanga.nu Fri Nov 17 01:33:21 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 16 Nov 2000 17:33:21 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Thu, 16 Nov 2000 16:22:20 EST." <14868.20492.287182.956825@anthem.concentric.net> References: <12203.974323139@kanga.nu> <14868.20492.287182.956825@anthem.concentric.net> Message-ID: <4622.974424801@kanga.nu> On Thu, 16 Nov 2000 16:22:20 -0500 Barry A Warsaw wrote: > A couple of notes. First, the version of newlist that's going to > be in rc3 (some time today) has a -o/--output switch that does the > appending to a specified file, somewhat like the way your patch > does (although the file name is taken from the command line and a > few other minor differences). I'm currently toying about to see if I can't persuade some of the alias re-writing and procmail to play to get the sme hiding level as Exim. The most obvious attack is via the $luser, but I'm not real happy with that. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Fri Nov 17 01:34:15 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 16 Nov 2000 17:34:15 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from Owen Taylor of "16 Nov 2000 16:09:38 EST." References: <12203.974323139@kanga.nu> Message-ID: <4724.974424855@kanga.nu> On 16 Nov 2000 16:09:38 -0500 Owen Taylor wrote: > Well, you can't lose the aliases file, but you can make it pretty > darn invisible. Yeah, I'd gotten that far. I'd really like to get entirely rid of the Mailman alias file however. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From andrewm@connect.com.au Fri Nov 17 01:56:56 2000 From: andrewm@connect.com.au (Andrew McNamara) Date: Fri, 17 Nov 2000 12:56:56 +1100 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Your message of "Thu, 16 Nov 2000 16:08:15 -0800." <29159.974419695@kanga.nu> Message-ID: <20001117015656.DC2E328706@wawura.off.connect.com.au> >> Can you give a brief precis of how Exim presents this to the >> admin? I might be able to suggest an equivilent feature, or at >> least inject the idea into Postfix's development. > >See http://www.exim.org/howto/mailman.html Okay - the closest I can come to this off the top of my head is to define "fallback_transport" to point to a "pipe" transport that called Mailman. Mailman would then receive all unknown local destinations, and would have to recognise owner- and -request style addresses also. It would also need to return EX_NOUSER (exit 67). Not as pretty, but should work okay. --- Andrew McNamara (System Architect) connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111 From jeremy@alum.mit.edu Fri Nov 17 02:41:32 2000 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 16 Nov 2000 21:41:32 -0500 (EST) Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: <710385309@toto.iv> Message-ID: <14868.39644.236453.34834@bitdiddle.concentric.net> >>>>> "BAW" == Barry A Warsaw writes: >>>>> "GE" == Gunnar Evermann writes: GE> What are the current plans about the html archiver in mailman? GE> The major shortcomings of pipermail I see are: | - doesn't decode MIME | - doesn't handle mixed charsets | - no search engine | - somewhat convoluted code. The last is the killer, because it makes the first three that much harder to accomplish. I tried some refactoring for the 2.0 release, but I'm not convinced there is much worth saving. BAW> Anyway, yes, we'd like to rewrite the archiver from the ground BAW> up, but nobody's working on that yet. I suspect either Jeremy BAW> or I will tackle it at some point. It's not near the top of my priorities at the moment, but I would like to work on a new archiver. Perhaps we should start sketching some design issues in Barry's Mailman wiki. I would also like to see separation of the presentation part of the archive from the code, so that it is easier to customize the presentation. Jeremy From chuqui@plaidworks.com Fri Nov 17 04:42:27 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Thu, 16 Nov 2000 20:42:27 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: <14868.39644.236453.34834@bitdiddle.concentric.net> References: <14868.39644.236453.34834@bitdiddle.concentric.net> Message-ID: At 9:41 PM -0500 11/16/00, Jeremy Hylton wrote: > >It's not near the top of my priorities at the moment, but I would like >to work on a new archiver. Perhaps we should start sketching some >design issues in Barry's Mailman wiki. Does mailman have to an an integrated archiver? Isn't this another variant of the "integrated MTA" discussion? I'd prefer, honestly, splitting out pipermail (or it's successor) into a separately tracked project, and building a standard interface that will work with it and with other archivers, like Mhonarc. That way, a separate team can be built to work on Pipermail, and releases of each piece don't have to be wedged into the same schedule, and you simplify the complexity of each project to improve focus.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Fri Nov 17 04:47:47 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Thu, 16 Nov 2000 20:47:47 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: <20001117015656.DC2E328706@wawura.off.connect.com.au> References: <20001117015656.DC2E328706@wawura.off.connect.com.au> Message-ID: At 12:56 PM +1100 11/17/00, Andrew McNamara wrote: >Okay - the closest I can come to this off the top of my head is to >define "fallback_transport" to point to a "pipe" transport that called >Mailman. > >Mailman would then receive all unknown local destinations, and would >have to recognise owner- and -request style addresses also. It would >also need to return EX_NOUSER (exit 67). Not as pretty, but should work >okay. How about, as an alternative, adding an LDAP api to Mailman (or some other interface that could be attached to something like LDAP). that way, you could write a really simple thing like wrapper that could dynamically look up whether a list exists and then do something useful with it. So instead of trying to add this functionality to Mailman, you add an interface that allows a special tool to be written that adds taht functionality -- and allows for other things (one thing I *really* want is subscription validation for external uses, like authentification into my archives. this kills both birds with one stone and a couple of simple glue procs... Which leads me to something I've talked to Barry about, which is an open source authentification system that everything can plug into, including Mailman. So we can finally stop rewriting user databases from scratch every time we write a tool, and can integrate everything into a single system for a site... But that's down the road -- but think in terms of whether a feature like this stands alone, or whether a generalization of it can be used to solve multiple options..... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From barry@digicool.com Fri Nov 17 05:34:43 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 17 Nov 2000 00:34:43 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? References: <29159.974419695@kanga.nu> <20001117015656.DC2E328706@wawura.off.connect.com.au> <12203.974323139@kanga.nu> <14868.20492.287182.956825@anthem.concentric.net> <4622.974424801@kanga.nu> Message-ID: <14868.50035.50158.985566@anthem.concentric.net> >>>>> "JCL" == J C Lawrence writes: JCL> I'm currently toying about to see if I can't persuade some of JCL> the alias re-writing and procmail to play to get the sme JCL> hiding level as Exim. The most obvious attack is via the JCL> $luser, but I'm not real happy with that. >>>>> "AM" == Andrew McNamara writes: AM> Mailman would then receive all unknown local destinations, and AM> would have to recognise owner- and -request style addresses AM> also. It would also need to return EX_NOUSER (exit 67). Not as AM> pretty, but should work okay. Bling! You guys just put the puzzle together for me. What follows is a one hour hack, but it works well enough that I'm fairly certain it could be fleshed out into a real solution. First, add these configuration options to Postfix's main.cf: luser_relay = mm+$user@wooz.org recipient_delimiter = + (changing of course wooz.org to whatever your own domain is). Now all unknown recipients will be forwarded to mm+recipient@wooz.org, but they'll /really/ be sent to the `mm' alias. Which is defined as: mm: "|/home/mailman/mail/wrapper auto" This will send all such email through the `auto' script (given below) after traveling as normal through the wrapper safety net. You'll need this patch to src/mail-wrapper.c: -------------------- snip snip -------------------- Index: mail-wrapper.c =================================================================== RCS file: /cvsroot/mailman/mailman/src/mail-wrapper.c,v retrieving revision 1.16 diff -u -r1.16 mail-wrapper.c --- mail-wrapper.c 2000/08/02 03:23:25 1.16 +++ mail-wrapper.c 2000/11/17 05:19:41 @@ -36,6 +36,7 @@ "post", "mailcmd", "mailowner", + "auto", NULL /* Sentinel, don't remove */ }; -------------------- snip snip -------------------- and this patch to scripts/Makefile.in: -------------------- snip snip -------------------- Index: Makefile.in =================================================================== RCS file: /cvsroot/mailman/mailman/scripts/Makefile.in,v retrieving revision 1.5 diff -u -r1.5 Makefile.in --- Makefile.in 2000/03/21 06:26:37 1.5 +++ Makefile.in 2000/11/17 05:20:37 @@ -40,7 +40,7 @@ SHELL= /bin/sh -SCRIPTS= answer_majordomo_mail mailcmd mailowner post driver +SCRIPTS= answer_majordomo_mail mailcmd mailowner post driver auto # Modes for directories and executables created by the install # process. Default to group-writable directories but -------------------- snip snip -------------------- The new auto script is the heart of the beast. Briefly what it does is look at the recipients list (To:, Cc:, Resent-To:, and Resent-Cc: headers) to see if it has a valid Mailman list destination. It gathers this from all the list objects on the system and should be virtual host aware. Next it matches the recipients its found in the message against the list of possible valid Mailman destinations. If if finds no matches, it exits with EX_NOUSER. If it does find a match, it enqueues the message in a manner similar to the post, mailcmd, and mailowner scripts. qrunner would handle those enqueued messages without ever knowing the difference. Here are some gotchas, which I believe can either be worked out or lived with: - Say I send a message to bogus@wooz.org. This message will travel through the auto script and I've verified that you get a user unknown bounce, but the bounce message says that mm+bogus@wooz.org is the bouncing address, not bogus@wooz.org, which is what I'd like to see. - On a system with a lot of lists, auto either should just a trampoline into a long running daemon, or it should try to cache more information. Trying to open and read every config.db file for every received message is probably way overkill. - I'm not sure that auto as it's currently written will handle cross-posting correctly, since enqueuing the message multiple times (say for list1@, list2@, and list3@) will still result in just one .db and .msg file for the received message. Shouldn't be too hard to craft a different sha hash for each separately. - This won't live very nicely with other $luser_relay's if you've got them. It would be nice if Postfix had a slightly more elaborate hookable framework here, where I could say: "let auto try to handle the address, but if it can't, pass it on to the next hook in the sequence." - The recipient_delimiter bit is probably extraneous. So anyway, time for sleep. Have fun! If this can be fleshed out and completed, it can be added to 2.1. -Barry P.S. auto below requires Python 2.0. Taste of things to come. :) -------------------- snip snip -------------------- #! /usr/bin/env python # # Copyright (C) 1998,1999,2000 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. import sys import paths from Mailman import mm_cfg from Mailman import Utils from Mailman import MailList from Mailman import Message from Mailman.Logging.Utils import LogStdErr from Mailman.Logging.Syslog import syslog # Error code if it's really not a Mailman list addr destination EX_NOUSER = 67 LogStdErr('auto', 'auto') def fqdn_listname(listname, hostname): return ('%s@%s' % (listname, hostname)).lower() def main(): syslog('auto', 'Running the auto deliver script') # valid destinations valids = {} lists = {} for listname in Utils.list_names(): mlist = MailList.MailList(listname, lock=0) lists[listname] = mlist valids[fqdn_listname(listname, mlist.host_name)] = 1 syslog('auto', 'valids: %s' % valids) # Get the message from standard input msg = Message.Message(sys.stdin) addrs = [] for header in ('to', 'cc', 'resent-to', 'resent-cc'): addrs.extend(msg.getaddrlist(header)) syslog('auto', 'found these addresses: %s' % addrs) # Parse the headers. Keys are 'post', 'request', 'owner' destinations = {} for name, addr in addrs: localpart, hostname = addr.split('@', 1) try: listname, subdest = localpart.split('-', 1) except ValueError: listname = localpart subdest = 'post' if subdest in ('post', 'request', 'owner', 'admin') and \ valids.has_key(fqdn_listname(listname, hostname)): destinations.setdefault(subdest, []).append(listname) if not destinations: # Tell Postfix we found no valid list destinations syslog('auto', 'No Mailman destination found') return EX_NOUSER # TBD: This may not work for cross-posted messages for subdest, listnames in destinations.items(): for listname in listnames: syslog('auto', 'Enqueuing to %s-%s' % (listname, subdest)) mlist = lists[listname] if subdest == 'post': msg.Enqueue(mlist, tolist=1) elif subdest == 'request': msg.Enqueue(mlist, torequest=1) elif subdest in ('owner', 'admin'): msg.Enqueue(mlist, toadmin=1) return 0 if __name__ == '__main__': code = main() sys.exit(code) From Nigel.Metheringham@VData.co.uk Fri Nov 17 09:20:55 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Fri, 17 Nov 2000 09:20:55 +0000 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Thu, 16 Nov 2000 20:42:27 PST." Message-ID: chuqui@plaidworks.com said: > Does mailman have to an an integrated archiver? Isn't this another > variant of the "integrated MTA" discussion? Maybe. There are times when I would really love to have archive message id or URL references in messages for example - ie when a particularly great pearl of wisdom comes from you folks which I want to reference on another list it would be great if I could easily grab an archive reference from the original message (as opposed to going to the archives and hunting out the original message then copying the URL) to send on. > I'd prefer, honestly, splitting out pipermail (or it's successor) > into a separately tracked project, and building a standard interface > that will work with it and with other archivers, like Mhonarc. That > way, a separate team can be built to work on Pipermail, and releases > of each piece don't have to be wedged into the same schedule, and you > simplify the complexity of each project to improve focus.... However my general gut feel does strongly agree with Chuq here. I would also be keen on the idea of a MIME reprocessor front-ended onto Mailman - sort of like demime but more controllable, and particularly with the ability to strip unwanted MIME from the message but save the parts onto the MLM server, adding URLs to the processed message to allow the extracted parts to be retrieved. This is not really the archiver function, but has some ideas in common. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From Nigel.Metheringham@VData.co.uk Fri Nov 17 09:28:12 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Fri, 17 Nov 2000 09:28:12 +0000 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Fri, 17 Nov 2000 00:34:43 EST." <14868.50035.50158.985566@anthem.concentric.net> Message-ID: barry@digicool.com said: > The new auto script is the heart of the beast. Briefly what it does > is look at the recipients list (To:, Cc:, Resent-To:, and Resent-Cc: > headers) to see if it has a valid Mailman list destination. It > gathers this from all the list objects on the system and should be > virtual host aware. Parsing out of headers could be rather interesting... what about a message copied to 2 lists - its not going to get replicated is it? Can you just pass the target address one the command line to the auto script - makes life a *lot* easier. Alternatively, all the exim stuff does is a file existence test on a transformed address - sure that ought to be relatively easy to postfix. Nigel. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From chuqui@plaidworks.com Fri Nov 17 15:22:41 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Fri, 17 Nov 2000 07:22:41 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: References: Message-ID: At 9:20 AM +0000 11/17/00, Nigel Metheringham wrote: >chuqui@plaidworks.com said: >> Does mailman have to an an integrated archiver? Isn't this another >> variant of the "integrated MTA" discussion? > >Maybe. There are times when I would really love to have archive >message id or URL references in messages for example But does that imply it has to be integrated into mailman? Or just available. The only reason I see for it to be fully integrated is if we want to add a header pointing to the article in the archive -- that would require archive processing during delivery. Other than that, it can be post -processed, so it wouldn't need to be part of hte main MLM code, it simply has to interface with it. That'd allow both pieces to be managed separately. > > >I would also be keen on the idea of a MIME reprocessor front-ended onto >Mailman - sort of like demime but more controllable, I'm toying with that, actually. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From steffen@wflin3.asta.uni-wuppertal.de Fri Nov 17 17:33:47 2000 From: steffen@wflin3.asta.uni-wuppertal.de (Steffen Bardolatzi) Date: Fri, 17 Nov 2000 18:33:47 +0100 Subject: [Mailman-Developers] "Date:"-bug already solved in CR2? Message-ID: <20001117183347.F14904@wflin3.asta.uni-wuppertal.de> I recently hit this bug: Nov 10 16:42:12 2000 (5913) Delivery exception: illegal argument type for built- Nov 10 16:42:12 2000 (5913) Traceback (innermost last): File "/var/mailman/Mailman/Handlers/HandlerAPI.py", line 82, in do_pipeline func(mlist, msg, msgdata) File "/var/mailman/Mailman/Handlers/ToArchive.py", line 50, in process msg['Date'] = date File "/var/mailman/Mailman/pythonlib/rfc822.py", line 393, in __setitem__ text = name + ": " + value TypeError: illegal argument type for built-in operation (plus a corresponding message with the message-id was sent to the user mailman --> had to use grep to find the corrupted message file). ... which is caused by a missing Date-header an e-mail (in this case I simply took an mbox-file and "feeded" it via the MTA into mailman to create an archive). As as side effect the CPU was used by a python resulting in 0-10% idele time on our server - as for e-mail with a message-id with a complete message-id (that is with a domain); this took *until* I deleted these e-mail in the qfiles-directory [these ones with a message ID like 6686876ghgk (e.g., without domain) didn´t block the message queue]. Of course all other messages were blocked in the way mentioned above ... ==> in our office the system actually was used for that job with 90-100% ... ... hope this going to be fixed in 2.0 Final (still running Beta 6 to wait for Final 2.0). Sorry for the delay, I have been rather busy. Thanks in advance. ----- End forwarded message ----- From barry@digicool.com Fri Nov 17 20:36:06 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 17 Nov 2000 15:36:06 -0500 Subject: [Mailman-Developers] Future of pipermail? References: <14868.39644.236453.34834@bitdiddle.concentric.net> Message-ID: <14869.38582.796232.612354@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> Does mailman have to an an integrated archiver? Isn't this CVR> another variant of the "integrated MTA" discussion? CVR> I'd prefer, honestly, splitting out pipermail (or it's CVR> successor) into a separately tracked project, and building a CVR> standard interface that will work with it and with other CVR> archivers, like Mhonarc. That way, a separate team can be CVR> built to work on Pipermail, and releases of each piece don't CVR> have to be wedged into the same schedule, and you simplify CVR> the complexity of each project to improve focus.... This could probably work. I do want an archiver that I can bundle with releases (let's say final releases primarily) because it's just so much easier for newbies to get started if they have everything they need. They already have the burden of having to download and install Python (and maybe a web server and an MTA), so reducing the number of components you need to get started is a good thing. For all it's flaws, Pipermail provides an essential service with no extra work (it'd be nice to also have a bundled search solution). OTOH, I think we can handle it similar to the way Python does it's rpms or windows installers and add-ons like readline and Tk. We find a stable version of the archiver and bundle it in the big releases, but beta users may have to do more work. >>>>> "NM" == Nigel Metheringham writes: NM> Maybe. There are times when I would really love to have NM> archive message id or URL references in messages for example - CVR> But does that imply it has to be integrated into mailman? Or CVR> just available. CVR> The only reason I see for it to be fully integrated is if we CVR> want to add a header pointing to the article in the archive CVR> -- that would require archive processing during CVR> delivery. Maybe not, if we define an API that Mailman can ask the archiver, "give me a url for message-id XYZ". -Barry From claw@kanga.nu Fri Nov 17 20:45:15 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 17 Nov 2000 12:45:15 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from Andrew McNamara of "Fri, 17 Nov 2000 11:29:31 +1100." <20001117002931.CBE5A2870A@wawura.off.connect.com.au> References: <20001117002931.CBE5A2870A@wawura.off.connect.com.au> Message-ID: <9939.974493915@kanga.nu> On Fri, 17 Nov 2000 11:29:31 +1100 Andrew McNamara wrote: > If you are using one of the recent Postfix snapshots, it has this > functionality. It's documented in the file FILTER_README in the > top level directory. I realise that a snapshot isn't appropriate > for some sites (however, Wietse's definition of a snapshot is > simply a release with features that may not be carried through - > the code receives his usual extensive testing before release). Wonderful -- I'm poking at it as I type. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Fri Nov 17 20:52:13 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 17 Nov 2000 12:52:13 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Thu, 16 Nov 2000 20:42:27 PST." References: <14868.39644.236453.34834@bitdiddle.concentric.net> Message-ID: <10378.974494333@kanga.nu> On Thu, 16 Nov 2000 20:42:27 -0800 Chuq Von Rospach wrote: > At 9:41 PM -0500 11/16/00, Jeremy Hylton wrote: >> It's not near the top of my priorities at the moment, but I >> would like to work on a new archiver. Perhaps we should start >> sketching some design issues in Barry's Mailman wiki. > Does mailman have to an an integrated archiver? Isn't this another > variant of the "integrated MTA" discussion? For inexperienced/new admins and sites there's significant advantage to the all-in-one approach. You and I may not be fond of it, but we also aren't members of that audience (which is a significant one for Mailman in terms of population). I'm actually much of the mind that the current state of the archiver is rather good when viewed from a Jamie Zawinski "Worse is Better" perspective. It fulfills a minimal need, and provides (reasonably, could be better) clear pointers to where to go if you want to handle archiving better. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From marc_news@valinux.com Fri Nov 17 21:05:21 2000 From: marc_news@valinux.com (Marc MERLIN) Date: Fri, 17 Nov 2000 13:05:21 -0800 Subject: [Mailman-Developers] Doing load balancing with mailman Message-ID: <20001117130521.V9808@marc.merlins.org> So I didn't hear anything back on mailman-users, would anyone here be more inspired with those questions? Thanks Marc ----- Forwarded message from Marc MERLIN ----- [I didn't Cc mailman-developers, but if I should ask programing specific questions there, let me know] While a correctly tuned mailman+exim config should be able to handle sourceforge's lists just fine, I've been encouraged to look into a failover/load balancing solution :-) So, I've seen two options so far: Option #1: Lists are physically split between let's say 2 mail servers. All mail goes to listname@lists.sourceforge.net, which is MXed the two mail servers. I know how to configure exim so that it rewrites the envelope To to resend the mail to the other list server if listname is not local to that mail server. So far so good. The problem, is that for web access, I then have to show URLs with http://lists1.sourceforge.net/lists/listinfo/listname and http://lists2.sourceforge.net/lists/listinfo/listname Ok, that can be fixed with some clever apache rewriting/proxying, but it'd be a bit iffy. As far as failover is concerned, I can rsync the list configs on a periodic basis and with some symlink magic, I could have one mail server mostly take over the other mail server's lists if it were to disappear. This requires mail header rewriting, http URL rewriting, plus other magic, and it's not very "automatic". Option #2: Hold on, this is scary :-) /var/local/mailman is NFS mounted on the mail servers. /var/local/mailman/qfiles is a symlink to local disk qrunner is modified to set a lockfile in qfiles and not data (it should hopefully be a simple patch) cron jobs are run on only one mail server, with the exception of qrunner. both mail servers offer web access (same URL with load balancing) to the same lists. Voila :-) Did I forget anything? Is there any reason why this wouldn't work? (mailman's locking looks NFS safe to me, but I don't know if having locks created for possibl the same lists in the mailman/locks directory, by two different machines would be a problem for some reason that would escape me right now). Is there anything else besides qrunner that I'd need to patch? Thanks Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key ------------------------------------------------------ Mailman-Users maillist - Mailman-Users@python.org http://www.python.org/mailman/listinfo/mailman-users ----- End forwarded message ----- From ge204@eng.cam.ac.uk Fri Nov 17 21:40:14 2000 From: ge204@eng.cam.ac.uk (Gunnar Evermann) Date: 17 Nov 2000 21:40:14 +0000 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: barry@digicool.com's message of "Fri, 17 Nov 2000 15:36:06 -0500" References: <14868.39644.236453.34834@bitdiddle.concentric.net> <14869.38582.796232.612354@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > I do want an archiver that I can bundle with releases (let's say final > releases primarily) because it's just so much easier for newbies to > get started if they have everything they need. yes, I think this is vital. When I needed to setup a couple of mailing list this is what convinced me to install Mailman. I think this appeals does not only to "newbies". Even experienced admins prefer not to spend ages fiddling half a dozen components just to get a mailing list working. Gunnar From marc_news@valinux.com Fri Nov 17 23:16:14 2000 From: marc_news@valinux.com (Marc MERLIN) Date: Fri, 17 Nov 2000 15:16:14 -0800 Subject: [Mailman-Developers] Problem with web_page_url Message-ID: <20001117151613.D17368@marc.merlins.org> So I noticed that if web_page_url points to a bad value, I can get to the admin page, but I cannot change it back to a good value as the submit link posts the changes to a bad URL. Basically, one of the list admins on one of my boxes thought he would get cute and changed http://lists.scruz.org/mailman/ to http://www.scruz.org/ I think mailman got specifically unhappy with the fact that '/mailman/' was missing at the end. The only way I was able to recover from this was to change the value back in the config.db with withlist. I'm not too sure how to fix that. Can mailman check the new value to make sure it works before allowing a user to change it to something bad? Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From barry@digicool.com Sat Nov 18 03:07:10 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 17 Nov 2000 22:07:10 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? References: <29159.974419695@kanga.nu> <20001117015656.DC2E328706@wawura.off.connect.com.au> <12203.974323139@kanga.nu> <14868.20492.287182.956825@anthem.concentric.net> <4622.974424801@kanga.nu> <14868.50035.50158.985566@anthem.concentric.net> Message-ID: <14869.62046.125012.324360@anthem.concentric.net> Here's a slightly better version of auto, which works with cross-posting. The key insight was that Postfix calls auto once for each mailing list recipient, with the Delivered-To: header set to the mailing list address. So this version sucks out the Delivered-To:'s and parses the necessary information out of there. Turns out the recipient_delimiter is necessary in order to get the name of the mailing list in the Delivered-To: header (with the `mm' prefix, which is stripped off). Because of this, each sha hash will be unique so they shouldn't step on each other's toes. The (potential) performance issue isn't addressed here, nor is the bogus address bounce notifications. The former will need a little bit of extra help from other parts of Mailman, and the latter may take some changes in Postfix. But IMO, it's not a bad hack! Enjoy, -Barry -------------------- snip snip -------------------- #! /usr/bin/env python # # Copyright (C) 1998,1999,2000 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. import sys import paths from Mailman import mm_cfg from Mailman import Utils from Mailman import MailList from Mailman import Message from Mailman.Logging.Utils import LogStdErr from Mailman.Logging.Syslog import syslog # Error code if it's really not a Mailman list addr destination EX_NOUSER = 67 LogStdErr('auto', 'auto') def fqdn_listname(listname, hostname): return ('%s@%s' % (listname, hostname)).lower() def main(): syslog('auto', 'Running the auto deliver script') # valid destinations valids = {} lists = {} for listname in Utils.list_names(): mlist = MailList.MailList(listname, lock=0) lists[listname] = mlist valids[fqdn_listname(listname, mlist.host_name)] = 1 # Get the message from standard input msg = Message.Message(sys.stdin) addrs = [addr for name, addr in msg.getaddrlist('delivered-to')] syslog('auto', 'found these addresses: %s' % addrs) # Parse the headers. Keys are 'post', 'request', 'owner' destinations = {} for addr in addrs: localpart, hostname = addr.split('@', 1) try: listname, subdest = localpart.split('-', 1) except ValueError: listname = localpart subdest = 'post' try: prefix, listname = listname.split('+') except ValueError: prefix = None if subdest in ('post', 'request', 'owner', 'admin') and \ valids.has_key(fqdn_listname(listname, hostname)) and \ prefix == 'mm': destinations.setdefault(subdest, []).append(listname) if not destinations: # Tell Postfix we found no valid list destinations syslog('auto', 'No Mailman destination found') return EX_NOUSER syslog('auto', 'destinations: %s' % destinations) # TBD: This may not work for cross-posted messages for subdest, listnames in destinations.items(): for listname in listnames: syslog('auto', 'Enqueuing to %s-%s' % (listname, subdest)) mlist = lists[listname] if subdest == 'post': msg.Enqueue(mlist, tolist=1) elif subdest == 'request': msg.Enqueue(mlist, torequest=1) elif subdest in ('owner', 'admin'): msg.Enqueue(mlist, toadmin=1) # success return 0 if __name__ == '__main__': code = main() sys.exit(code) From claw@kanga.nu Sat Nov 18 03:24:05 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 17 Nov 2000 19:24:05 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: Message from Nigel Metheringham of "Fri, 17 Nov 2000 09:20:55 GMT." References: Message-ID: <12250.974517845@kanga.nu> On Fri, 17 Nov 2000 09:20:55 +0000 Nigel Metheringham wrote: > chuqui@plaidworks.com said: >> Does mailman have to an an integrated archiver? Isn't this >> another variant of the "integrated MTA" discussion? > Maybe. There are times when I would really love to have archive > message id or URL references in messages for example - ie when a > particularly great pearl of wisdom comes from you folks which I > want to reference on another list it would be great if I could > easily grab an archive reference from the original message (as > opposed to going to the archives and hunting out the original > message then copying the URL) to send on. One of my users (Dominic Edison IIRC) hacked a CGI to do this with MHonArc based archives. Hand it a MessageID and it hands you the archive page for that message. You can find it in the tasks at Kanga.Nu (I have yet to enable it). -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From andyheath2@lineone.net Sat Nov 18 08:53:06 2000 From: andyheath2@lineone.net (Andy Heath) Date: Sat, 18 Nov 2000 08:53:06 +0000 Subject: [Mailman-Developers] interface mailman list passwords python cgi Message-ID: <3A164372.2B43C505@lineone.net> I don't know whether I should send this to mailman users or developers so apologies if not appropriate. I need to write some cgi applications with python that interface to mailman 1.0 email id's and passwords - access only to list members, that sort of thing. I'd appreciate advice on how to go about this. I find some "config.db" files which obviously is some sort of database containing the info. Is there some easy way I can get into this info ? Later I will need to move the solution to working with mailman 2.0 All advice welcome Andy -- ------------------------------------------------------------ Andy Heath Home:+44 (0)114 2885738 a.k.heath@shu.ac.uk From claw@kanga.nu Sat Nov 18 16:38:44 2000 From: claw@kanga.nu (J C Lawrence) Date: Sat, 18 Nov 2000 08:38:44 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] interface mailman list passwords python cgi In-Reply-To: Message from Andy Heath of "Sat, 18 Nov 2000 08:53:06 GMT." <3A164372.2B43C505@lineone.net> References: <3A164372.2B43C505@lineone.net> Message-ID: <14507.974565524@kanga.nu> On Sat, 18 Nov 2000 08:53:06 +0000 Andy Heath wrote: > I need to write some cgi applications with python that interface > to mailman 1.0 email id's and passwords - access only to list > members, that sort of thing. Study ~bin/withlist. > I'd appreciate advice on how to go about this. I find some > "config.db" files which obviously is some sort of database > containing the info. Is there some easy way I can get into this > info ? config.db is a python pickle -- essentially a streamed python structure. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From gconnor@nekodojo.org Sun Nov 19 06:59:03 2000 From: gconnor@nekodojo.org (Greg Connor) Date: Sat, 18 Nov 2000 22:59:03 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Bug in 2.0rc2 adding newlist References: <3A106ECF.1857.594BDCE@localhost> Message-ID: <3A177A37.46F87CDD@nekodojo.org> Phil Barnett wrote: > > Running 2.0rc2. > > I was trying to add a list with bin/newlist and got this: > ... > File "/home/mailman/Mailman/Utils.py", line 568, in mkdir > os.mkdir(dir, mode) > OSError: [Errno 2] No such file or directory: > '/home/mailman/archives/private/bin/newlist.mbox' > I got this too. Looks like a buglet in 2.0rc2. However, I noticed that running bin/newlist foo fails, but just running bin/newlist and then typing the new listname when prompted, works. When you put the name on the command line, it looks like it picks up "bin/newlist" as the name instead. -- Greg Connor From Antti.Siiskonen@tut.fi Sun Nov 19 11:35:19 2000 From: Antti.Siiskonen@tut.fi (Antti Siiskonen) Date: 19 Nov 2000 13:35:19 +0200 Subject: [Mailman-Developers] how to limit posting to a domain? Message-ID: Is there a way to allow posting only for those within a certain domain? It seems to me that .. 'Addresses of members accepted for posting to this list without implicit approval requirement.' .. does not take regexps. Is there a way around this or is this feature just missing from Mailman? -- //Antti From lovelace@wayfarer.org Sun Nov 19 14:36:26 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Sun, 19 Nov 2000 09:36:26 -0500 Subject: [Mailman-Developers] Re: Bug in 2.0rc2 adding newlist References: <3A106ECF.1857.594BDCE@localhost> <3A177A37.46F87CDD@nekodojo.org> Message-ID: <3A17E56A.691DD65E@wayfarer.org> Greg Connor wrote: > > Phil Barnett wrote: > > > > Running 2.0rc2. > > > > I was trying to add a list with bin/newlist and got this: > > > ... > > File "/home/mailman/Mailman/Utils.py", line 568, in mkdir > > os.mkdir(dir, mode) > > OSError: [Errno 2] No such file or directory: > > '/home/mailman/archives/private/bin/newlist.mbox' > > > > I got this too. Looks like a buglet in 2.0rc2. However, I noticed that > running > bin/newlist foo > fails, but just running > bin/newlist > and then typing the new listname when prompted, works. When you put the > name on the command line, it looks like it picks up "bin/newlist" as the > name instead. > I believe this has been fixed in 2.0rc3. argv was being used instead of args. -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? From claw@kanga.nu Sun Nov 19 16:00:24 2000 From: claw@kanga.nu (J C Lawrence) Date: Sun, 19 Nov 2000 08:00:24 -0800 Subject: [Mailman-Developers] how to limit posting to a domain? In-Reply-To: Message from Antti Siiskonen of "19 Nov 2000 13:35:19 +0200." References: Message-ID: <16831.974649624@kanga.nu> On 19 Nov 2000 13:35:19 +0200 Antti Siiskonen wrote: > Is there a way to allow posting only for those within a certain > domain? Not currently, and not for v2.0. This is commonly requested, especially by corporates. I'd expect to see it in v2.1. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From ken@kyler.com Sun Nov 19 16:25:07 2000 From: ken@kyler.com (Ken Kyler) Date: Sun, 19 Nov 2000 11:25:07 -0500 Subject: [Mailman-Developers] how to limit posting to a domain? In-Reply-To: <16831.974649624@kanga.nu> Message-ID: > > Is there a way to allow posting only for those within a certain > > domain? > > Not currently, and not for v2.0. This is commonly requested, > especially by corporates. I'd expect to see it in v2.1. Is it safe to assume the reverse will be true? That whole domains can be prohibited from posting? Ken From claw@kanga.nu Sun Nov 19 16:33:40 2000 From: claw@kanga.nu (J C Lawrence) Date: Sun, 19 Nov 2000 08:33:40 -0800 Subject: [Mailman-Developers] how to limit posting to a domain? In-Reply-To: Message from "Ken Kyler" of "Sun, 19 Nov 2000 11:25:07 EST." References: Message-ID: <21419.974651620@kanga.nu> On Sun, 19 Nov 2000 11:25:07 -0500 Ken Kyler wrote: > Is it safe to assume the reverse will be true? That whole domains > can be prohibited from posting? You can write mailman filters that will hold such posts for moderation, but Malman does not have the ability to ignore such. That said, this is likely trivial to do at either the MTA level, or as a pre-filter before the Mailman scripts: Just check the envelope and don't apss it through if it doesn't match. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From Nigel.Metheringham@VData.co.uk Mon Nov 20 11:14:50 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Mon, 20 Nov 2000 11:14:50 +0000 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Fri, 17 Nov 2000 15:36:06 EST." <14869.38582.796232.612354@anthem.concentric.net> Message-ID: Continuing the thoughts on integration or separation of archiver functions... heres a flow of my ideal wrt to archiving and MIME processing of list messages:- - Message received by mailman - Policy decisions applied to message (ie accept, bounce, moderate) depending on things such as sender/recipient/list policy/MIME etc - Message archived. MIME attachments/portions split into separate retrievable sections. The archived MIME sections become available to latter processes as retrieval URLs. Overall Message ref also available to all subsequent processes (ie as additional header with this message archive URL) - Send out process set. This would allow, for example:- - some recipients to receive original MIME encoded message - some recipients to receive simplified message with limited MIME (maybe just text/plain) plus URLs - some receive text/plain version as a digest The send out messages would depend on a combination of list and personal policy - ie some people could be on a MIME accepting list, but prefer simplified mail. Or the list could mandate no transmitted MIME. The original policy set could also just block MIME or large MIME. However this is just one way things could work... I just get this feeling that archiving and MIME stripping can be closely related... Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From chuqui@plaidworks.com Tue Nov 21 05:41:51 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 20 Nov 2000 21:41:51 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: <14869.38582.796232.612354@anthem.concentric.net> References: <14868.39644.236453.34834@bitdiddle.concentric.net> <14869.38582.796232.612354@anthem.concentric.net> Message-ID: sorry for the delay. we went to Portland for our annual fall trip, and to be honest, I didn't turn on a computer the whole trip (and the world didn't end, either. amazing...) >I do want an archiver that I can bundle with releases (let's say final >releases primarily) because it's just so much easier for newbies to >get started if they have everything they need. fair point, one I've sort of ignored. In theory, I'd prefer a situation where we simply included a "turnkey" set of instructions for a preferred archiver, but in practice -- that theory doesn't work as well... > They already have the >burden of having to download and install Python (and maybe a web >server and an MTA), so reducing the number of components you need to >get started is a good thing. true. Or maybe the answer is some kind of meta-setup beast like some folks use for Apache? Or does that just add complexity without solving anything? I'm brainstorming -- not pretending to answers, but asking questions that'll help us understand where the answers ought to be... > >Maybe not, if we define an API that Mailman can ask the archiver, >"give me a url for message-id XYZ". man, I can be such a luddite some daays. I insist on thinking about APIs as hand-off interfaces, not bi-directional. Of course data can go both ways, although if you want to plug in an external archiver, the API has to be smart enough to return some kind of not-supported value (or preferably, a flag that can be queried during web-page creation as to whether or not ot show the option....) -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From barry@digicool.com Tue Nov 21 16:52:12 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 11:52:12 -0500 Subject: [Mailman-Developers] Future of pipermail? References: <14869.38582.796232.612354@anthem.concentric.net> Message-ID: <14874.43068.477150.391834@anthem.concentric.net> >>>>> "NM" == Nigel Metheringham writes: NM> Continuing the thoughts on integration or separation of NM> archiver functions... heres a flow of my ideal wrt to NM> archiving and MIME processing of list messages:- Good points Nigel. I've added some of these to the FutureOfPipermail Wiki. NM> However this is just one way things could work... I just get NM> this feeling that archiving and MIME stripping can be closely NM> related... Very possibly so. Having a general repository for messages and their MIME parts might be a good thing. I'm a little worried that we might be giving up independence and plugability, since I don't know whether any of the other 3rd party archivers support such features. -Barry From bbum@codefab.com Tue Nov 21 17:11:00 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Tue, 21 Nov 2000 12:11:00 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? Message-ID: <200011211711.eALHB1322170@bjork.codefab.com> WebDAV works really, really well as an archiver. And it would be particularly cool if the archive already had the MIME pieces exploded out. Even cooler would be if the messages sent to the list were rewritten such that all of the MIME pieces-- typically large, on some lists-- were actually URLs to the archived pieces on teh WebDAV server. If it had an interface that sent an HTTP message to some random webserver/appserver/cgi-bin letting it know a new message was submitted so that server could decide an appropriate time to rebuild an index file, that would allow the presentation of the archive to be offloaded from Mailman. Since WebDAV will be included with every copy of Apache 2.0 by default, there really isn't that much additional "moving pieces" load on the configuration. Mailman already requires diddling the web server configuration-- copy/paste a few more lines isn't that big of a deal. Anyone want to take this on? Oh, wait a minute, that's right... it has already been done! ftp://ftp.codefab.com/pub/unsupported/mailman_20000821_1123.tar.gz It is based on a version of Mailman from sometime around May or so... so, it needs to be updated to the latest version from the repository. However, it DOES work and has been used in production systems [who will remain nameless to protect this community from me venting] for quite some time. The company that hired me to build this actually paid a bunch of $$$ for it to be built-- as such, I actually had the freedom to thiink about doing it right and not just hacking it on as fast as possible. I don't know if I did do it right, but Barry provided quite a bit of guidance and I don't think it is a totally offensive set of changes... b.bum From chuqui@plaidworks.com Tue Nov 21 18:00:53 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 10:00:53 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <200011211711.eALHB1322170@bjork.codefab.com> References: <200011211711.eALHB1322170@bjork.codefab.com> Message-ID: At 12:11 PM -0500 11/21/00, Bill Bumgarner wrote: >WebDAV works really, really well as an archiver. webDAV? Oh, man. I guess I need to go look at another piece of technology. Wanna give those of us who have no clue what this is the 30 second executive overview? We can't, however, assume users are running Apache, even if it's the overwhelming choice... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Tue Nov 21 17:59:07 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 09:59:07 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: <14874.43068.477150.391834@anthem.concentric.net> References: <14869.38582.796232.612354@anthem.concentric.net> <14874.43068.477150.391834@anthem.concentric.net> Message-ID: At 11:52 AM -0500 11/21/00, Barry A. Warsaw wrote: >Very possibly so. Having a general repository for messages and their >MIME parts might be a good thing. I'm a little worried that we might >be giving up independence and plugability, since I don't know whether >any of the other 3rd party archivers support such features. some do. Maybe the answer is to allow the API to configure how the archiver wants it, so we can have mailman strip it to a text-only part if the archiver wants it that way. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From bbum@codefab.com Tue Nov 21 19:51:24 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Tue, 21 Nov 2000 14:51:24 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <200011211711.eALHB1322170@bjork.codefab.com> Message-ID: <200011211951.eALJpO325197@bjork.codefab.com> Chuq, et. al., See www.webdav.org for more information. WebDAV is a standard that extends, but does not modify, HTTP to allow for several additional operations against a web server. There are two levels to the specification. Class 0 will be described before-- it is complete and production quality. Class One is in the final stages of discussion and, as such, is not anywhere near having a reference implementation (but I will discuss briefly because it is very cool): - supports PUT of resources [files] onto the server (i.e. upload a file to server) - creation of collections (mkdir) - renaming/moving/deleting/copying of resources or collections - assigning properties to resources or collections. A set of properties is basically a chunk of XML associated with the collection or resource. For the MailMan WebDAV extensions, I use properties to store the mime-type and any extended headers associated with the attachment in question. - searching the properties database for particular values/attributes - locking; locking is limited to write locks because read locks would have required modification of the HTTP spec. Can have either shared or exclusive write locks. However, it is trivial to throw together a cgi-bin/ that allows read access to a resource if and only if the client sends a long the appropriate shared write lock token. This is used in the Mailman WebDAV extensions to support lmited access to the archive of messages. - locks can have a timeout. As such, it is trivial to generate an URL that allows access to a particular piece of the archive for, say, only the next 30 minutes. --- Most importantly, WebDAV is truly a standard and a very widely accepted standard, at that. - Apple ships a WebDAV client with Mac OS X. The builds of Apache with Mac OS X and Mac OS X Server both include WebDAV modules that are disabled by default. - Microsoft has full blown WebDAV support in Office and Windows 2000. In Office, you can open a document served by a WebDAV server and subsequently hit ctrl-s to save (or save as) as if the document were on a local fileserver. - IBM's WebSphere fully supports WebDAV - Zope fully supports WebDAV - GoLive CyberStudio and other HTML authoring packages either ship with WebDAV support or have announced that they will soon. So, WebDAV is, by no means, an apache only solution! Because it is a simple extension to HTTP-- and class 0 is relatively trivial in nature-- WebDAV capabilities can be provided via cgi-bin/ without a problem. --- The new class of DAV functionality is aimed at full support for version control. This includes basic revisioning as well as tags, maps, workareas, multiple users, etc... This revision to the spec is in the final stages of discussion and Greg Stein-- author of mod_dav-- is actively working with the group to create an implementation of the spec. --- In terms of Mailman, there is no real reason why creating the archive should be something that isn't completely abstracted within Mailman. Archival is basically three tasks: - storing data - storing meta-infrormation - creating some kind of index/view Decoding of attachments and rewriting the messages is mostly just an extension of the above concepts. As such, the Mailman archival interface could be an abstract interface of which we provide two concrete implementations; direct-to-filesystem and WebDAV. The actual process of creating an index/view isn't really a Mailman problem-- obivously, it is useful to ship with an out-of-the-box solution. b.bum From: Chuq Von Rospach Date: 2000-11-21 10:00:53 -0800 To: bbum@codefab.com, mailman-developers@python.org Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <200011211711.eALHB1322170@bjork.codefab.com> At 12:11 PM -0500 11/21/00, Bill Bumgarner wrote: >WebDAV works really, really well as an archiver. webDAV? Oh, man. I guess I need to go look at another piece of technology. Wanna give those of us who have no clue what this is the 30 second executive overview? We can't, however, assume users are running Apache, even if it's the overwhelming choice... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From Dan.Mick@west.sun.com Tue Nov 21 21:00:30 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Tue, 21 Nov 2000 13:00:30 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011211951.eALJpO325197@bjork.codefab.com> Message-ID: <3A1AE26E.D82BBA53@west.sun.com> WebDAV: stands for "Web-based Distributed Authoring and Versioning", which sort of neatly sums up what Bill was saying about its capabilities. (It's new to me, too. There seem to be two new web-database "environments" a week these days.) From bbum@codefab.com Tue Nov 21 21:04:31 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Tue, 21 Nov 2000 16:04:31 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? Message-ID: <200011212104.eALL4W327658@bjork.codefab.com> Except that WebDAV isn't really new and it is a ratified standard that basically everyone is adopting! (but, yes, I should have included the acronym in the first place!) The key with WebDAV is that use therein means that you can point WebDAV enabled Mailman at Zope, IBM's websphere, Apache+mod_dav, or any of the other WebDAV enabled HTTP servers out there and it should "just work" (assuming they did the implementation right). There is nothing proprietary about the solution and choosing it as a foundation for archival/management of messages/attachments opens doors (instead of closing doors). b.bum From: Dan Mick Date: 2000-11-21 13:00:30 -0800 To: bbum@codefab.com Subject: Re: [Mailman-Developers] Re: Future of pipermail? CC: Chuq Von Rospach , mailman-developers@python.org X-Accept-Language: en X-Mailer: Mozilla 4.75 [en] (Win98; U) WebDAV: stands for "Web-based Distributed Authoring and Versioning", which sort of neatly sums up what Bill was saying about its capabilities. (It's new to me, too. There seem to be two new web-database "environments" a week these days.) From Dan Mick Tue Nov 21 21:55:42 2000 From: Dan Mick (Dan Mick) Date: Tue, 21 Nov 2000 13:55:42 -0800 (PST) Subject: [Mailman-Developers] Re: Future of pipermail? Message-ID: <200011212154.NAA19723@utopia.west.sun.com> > Except that WebDAV isn't really new and it is a ratified standard that > basically everyone is adopting! Ah, well, new is in the timespan of the beholder. To me, Web-based anything is "relatively new".... > (but, yes, I should have included the acronym in the first place!) > > The key with WebDAV is that use therein means that you can point WebDAV > enabled Mailman at Zope, IBM's websphere, Apache+mod_dav, or any of the other > WebDAV enabled HTTP servers out there and it should "just work" (assuming they > did the implementation right). There is nothing proprietary about the > solution and choosing it as a foundation for archival/management of > messages/attachments opens doors (instead of closing doors). Yeah, I understand the point. But understand that virtually everyone claims ubiquity long before they actually have it, too... I'll research some more about the web servers I use. From barry@digicool.com Tue Nov 21 22:33:52 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 17:33:52 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011211711.eALHB1322170@bjork.codefab.com> <200011211951.eALJpO325197@bjork.codefab.com> Message-ID: <14874.63568.856071.405653@anthem.concentric.net> >>>>> "BB" == Bill Bumgarner writes: BB> This revision to the spec is in the final stages of discussion BB> and Greg Stein-- author of mod_dav-- is actively working with BB> the group to create an implementation of the spec. I'll just note briefly that Greg is a long time Pythonista and Mailmaner (in fact he owns the list.org domain), so WebDAV support in Python ought to be very good. I want to read and think more, but I've been interested in WebDAV as a technology for a while now. -Barry From chuqui@plaidworks.com Tue Nov 21 22:58:14 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 14:58:14 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <14874.63568.856071.405653@anthem.concentric.net> References: <200011211711.eALHB1322170@bjork.codefab.com> <200011211951.eALJpO325197@bjork.codefab.com> <14874.63568.856071.405653@anthem.concentric.net> Message-ID: At 5:33 PM -0500 11/21/00, Barry A. Warsaw wrote: >I'll just note briefly that Greg is a long time Pythonista excuse me if I quietly twich at the terminology (does anyone know where this -ista stuff came from? My first run-in iwth it was Guy Kawasaki and his "evangelistas", but was it used before that? >I want to read and think more, but I've been interested in WebDAV as a >technology for a while now. I'm starting to see serious pushes (and I'll self-admit to be one here!) towards serious, hard-core integration of technologies. Which is a double-edged sword of the worst and best kind. I've talked to Barry about building ways to allow for site-wide authentication issues, for instance, and now we're seeing WebDAV (which I find interesting, and given it's going to be part of Apache 2.0, it's got to be considered heir presumptive in this technology psace now, over things like, oh, Zope. And I was looking over a couple of other things today (MetaDot to name one.. ) and going "hmm. you know, that stuff's got potential, too...) There's HUGE synergy potential here. We certainly shouldn't avoid those things, but the more we start tying into (and/or depending on) other technologies, the more we lock users into a single technology suite, the more we start depending on specific technology suites the more we lock uers into our view of how things work, and the less flexibility we give them in building their sites. And, to some degree, the less ability we have to adapt to future changes in the net universe ourselves. None of which is a reason to not do these things -- but it's reason to be wary. We really need to understand how these things interact and what the side effects are, including things as basic as "does this mean it'll never run on Windows NT again?" or "did we just set it up so they have to run apache?" I think there's a lot here we want to do (and need to do) but -- not for the sake of doing it. WebDAV looks fascinating. WebDAV -- as mailman's default archiver -- looks to me to be overkill at a quick evaluation. Yes, it'll be in there by default for apache 2.0, once Apache 2.0 ships and is out for a while and everyone upgrades. that's, what, 2 years out before 50% of the sites are running apache 2.x? Easily. Until then, what do we do? And what about non-apache sites? what's this do to mailman across all supported platforms? I dunno -- but there's a lot of complexity here we need to make sure we deal with in these issues. And to some degree, I think it's another excuse for me to pull out the "lots of separately tracked modules with API's" schtick, because we can build a Mailman API and a WebDAV plug-in for it(along with Mhonarc and Pipermail plug-ins) and over time, as WebDAV becomes endemic, switch the priority of development in its direction, without locking into it. In two years, there'll be other stuff beyond WebDAV, too, I'll bet. I know, I know... it's all details. Details are boring (grin). but necessary... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From ken@kyler.com Tue Nov 21 23:16:59 2000 From: ken@kyler.com (Ken Kyler) Date: Tue, 21 Nov 2000 18:16:59 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message-ID: > There's HUGE synergy potential here. We certainly shouldn't avoid > those things, but the more we start tying into (and/or depending on) > other technologies, the more we lock users into a single technology > suite, the more we start depending on specific technology suites the > more we lock uers into our view of how things work, and the less > flexibility we give them in building their sites. And, to some > degree, the less ability we have to adapt to future changes in the > net universe ourselves. I'd like to suggest that instead of integrating other technologies, that making Mailman open to them would be better. Such as offering the alternative to use Webdav, or making APIs for PHP, Perl, etc to interact with it. This is a classic COTS integration issue. If you depend too heavily on a product, it is possible for a future release of that product to break yours and the other guys don't care because you're a small fry in the pond. Been there myself and ended up gutting a product I was working on. My apologies for butting in - been lurking a long time and this is an interesting thread. Ken Kyler From barry@digicool.com Tue Nov 21 23:21:29 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 18:21:29 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011211711.eALHB1322170@bjork.codefab.com> <200011211951.eALJpO325197@bjork.codefab.com> <14874.63568.856071.405653@anthem.concentric.net> Message-ID: <14875.889.807156.688940@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> excuse me if I quietly twich at the terminology (does anyone CVR> know where this -ista stuff came from? My first run-in iwth CVR> it was Guy Kawasaki and his "evangelistas", but was it used CVR> before that? Dunno, and I'm not sure who invented "Pythonista" or "Perlie" or whatever. I actually like the term "Pythoneer" because it's reminiscent of "Pioneer" and "Engineer". Maybe we should just call them "Bruces" (to avoid confusion :). CVR> I'm starting to see serious pushes (and I'll self-admit to be CVR> one here!) towards serious, hard-core integration of CVR> technologies. Which is a double-edged sword of the worst and CVR> best kind. I think the trick is to balance what is necessary and expedient for Mailman with long term flexibility and integration with standards. Given the size of the todo list, where we can leverage existing technology we probably want opt that way instead of inventing our own. But the flexibility is important because different sites have so many different requirements. That's why I like the idea of moving toward a core set of classes and scripts that provide the basic Mailman functionality, building interfaces and hooks so that people can do the customization they want. CVR> There's HUGE synergy potential here. We certainly shouldn't CVR> avoid those things, but the more we start tying into (and/or CVR> depending on) other technologies, the more we lock users into CVR> a single technology suite, the more we start depending on CVR> specific technology suites the more we lock uers into our CVR> view of how things work, and the less flexibility we give CVR> them in building their sites. And, to some degree, the less CVR> ability we have to adapt to future changes in the net CVR> universe ourselves. CVR> None of which is a reason to not do these things -- but it's CVR> reason to be wary. We really need to understand how these CVR> things interact and what the side effects are, including CVR> things as basic as "does this mean it'll never run on Windows CVR> NT again?" or "did we just set it up so they have to run CVR> apache?" I agree. CVR> WebDAV looks fascinating. WebDAV -- as mailman's default CVR> archiver -- looks to me to be overkill at a quick CVR> evaluation. Yes, it'll be in there by default for apache 2.0, CVR> once Apache 2.0 ships and is out for a while and everyone CVR> upgrades. that's, what, 2 years out before 50% of the sites CVR> are running apache 2.x? Easily. Until then, what do we do? CVR> And what about non-apache sites? what's this do to mailman CVR> across all supported platforms? CVR> I dunno -- but there's a lot of complexity here we need to CVR> make sure we deal with in these issues. And to some degree, I CVR> think it's another excuse for me to pull out the "lots of CVR> separately tracked modules with API's" schtick, because we CVR> can build a Mailman API and a WebDAV plug-in for it(along CVR> with Mhonarc and Pipermail plug-ins) and over time, as WebDAV CVR> becomes endemic, switch the priority of development in its CVR> direction, without locking into it. In two years, there'll be CVR> other stuff beyond WebDAV, too, I'll bet. Agreed again. CVR> I know, I know... it's all details. Details are boring CVR> (grin). but necessary... Indeed. There's also the "worse is better" mantra, or put in other words: "All software sucks, let's just try to suck less". :) -Barry From barry@digicool.com Tue Nov 21 23:34:23 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 18:34:23 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: Message-ID: <14875.1663.422580.420642@anthem.concentric.net> >>>>> "KK" == Ken Kyler writes: KK> I'd like to suggest that instead of integrating other KK> technologies, that making Mailman open to them would be KK> better. Such as offering the alternative to use Webdav, or KK> making APIs for PHP, Perl, etc to interact with it. KK> This is a classic COTS integration issue. If you depend too KK> heavily on a product, it is possible for a future release of KK> that product to break yours and the other guys don't care KK> because you're a small fry in the pond. Been there myself and KK> ended up gutting a product I was working on. Yes, but I think there's a difference in that we're going to be relying on open standards, not closed COTS technology. Mailman, of course, is already built on a number of standards: RFC 822, SMTP, NNTP, HTTP, CGI, etc., etc. Any of these could go away or be replaced or whatever, but we're doing a good thing by relying on these standards. Of course, you have to pick the right ones, and (IMO) build an architecture that lets you more easily move to new standards when they arise and become useful. KK> My apologies for butting in - been lurking a long time and KK> this is an interesting thread. The more the merrier! -Barry From barry@digicool.com Tue Nov 21 23:39:36 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 18:39:36 -0500 Subject: [Mailman-Developers] Future of pipermail? References: <14868.39644.236453.34834@bitdiddle.concentric.net> <14869.38582.796232.612354@anthem.concentric.net> Message-ID: <14875.1976.757143.922146@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> true. Or maybe the answer is some kind of meta-setup beast CVR> like some folks use for Apache? Or does that just add CVR> complexity without solving anything? There's something in Python called `distutils' which is a way to distribute and build extensions to Python. I don't think distutils currently works well for applications, but that might be a direction to explore. Distutils will likely be the underlying technology for a Pythonic CPAN like thing that everyone's been pining for but few have had the time to pursue. One nice thing about distutils is that it makes the building of extension modules completely trivial. I have a couple of nice (optional) performance improvements written in C for Mailman 2.1 which are easily built and installed with a distutils `setup.py' script. -Barry From ken@kyler.com Tue Nov 21 23:55:57 2000 From: ken@kyler.com (Ken Kyler) Date: Tue, 21 Nov 2000 18:55:57 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <14875.1663.422580.420642@anthem.concentric.net> Message-ID: > standards. Of course, you have to pick the right ones, and (IMO) > build an architecture that lets you more easily move to new standards > when they arise and become useful. How true - what I would give for omniscience. Keep up the good work guys! If I ever finish grad school I'll help. Ken From chuqui@plaidworks.com Tue Nov 21 23:57:40 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 15:57:40 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: References: Message-ID: At 6:55 PM -0500 11/21/00, Ken Kyler wrote: >Keep up the good work guys! If I ever finish grad school I'll help. hey, this ain't HP, guy! we don't care about pieces of paper! (grin) -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From tanner@real-time.com Wed Nov 22 00:04:07 2000 From: tanner@real-time.com (Bob Tanner) Date: Tue, 21 Nov 2000 18:04:07 -0600 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck Message-ID: <20001121180407.A20735@real-time.com> I have to say I am very disappointed at mailman's performance with sendmail. Especially qrunner and how it works. I am running sendmail with 10 queues, but it looks like mailman only understands and utilizes 1 queue. It also looks like qrunner is single threaded AND multiple instances cannot be running at the same time. This lends itself to a very slow delivery system. If a message is not deliverable for whatever reason, mailman puts files into a directory called qfiles. Then onces every minute a process called qrunner is invoked to walk the qfiles directory trying to re-deliver the messages. What sucks is qrunner just tries to deliver each message one at a time. If message 5 takes forever on the rcpt to: command (for example), no other messages deliveries are attempted because qrunner sits on message 5 until it times out. I have turned the rcpt to timeout down to 2mins, but I am still constantly running 1000+ message in the qfiles directory. Any ideas on speeding things up? Will mailman support multiple "qfiles" directories in the future? -- Bob Tanner | Phone : (612)943-8700 http://www.mn-linux.org | Fax : (612)943-8500 Key fingerprint = 6C E9 51 4F D5 3E 4C 66 62 A9 10 E5 35 85 39 D9 From barry@digicool.com Wed Nov 22 00:04:10 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 19:04:10 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <14875.1663.422580.420642@anthem.concentric.net> Message-ID: <14875.3450.340146.28315@anthem.concentric.net> >>>>> "KK" == Ken Kyler writes: >> standards. Of course, you have to pick the right ones, and >> (IMO) build an architecture that lets you more easily move to >> new standards when they arise and become useful. KK> How true - what I would give for omniscience. Don't forget Guido's time machine! When he's away, we like to borrow the keys and take 'er for a spin. One thing I've learned though: NEVER touch the purple lever, no matter how loudly it pleads with you. And the three-pole beaver switch only works if your Pythonic spirit is pure. KK> Keep up the good work guys! If I ever finish grad school I'll KK> help. Thanks! -Barry From barry@digicool.com Wed Nov 22 00:23:14 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 19:23:14 -0500 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck References: <20001121180407.A20735@real-time.com> Message-ID: <14875.4594.972300.212749@anthem.concentric.net> >>>>> "BT" == Bob Tanner writes: BT> I have turned the rcpt to timeout down to 2mins, but I am BT> still constantly running 1000+ message in the qfiles BT> directory. BT> Any ideas on speeding things up? See the FAQ question #2: http://www.list.org/faq.html From claw@kanga.nu Wed Nov 22 00:29:20 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 16:29:20 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Bill Bumgarner of "Tue, 21 Nov 2000 16:04:31 EST." <200011212104.eALL4W327658@bjork.codefab.com> References: <200011212104.eALL4W327658@bjork.codefab.com> Message-ID: <26896.974852960@kanga.nu> On Tue, 21 Nov 2000 16:04:31 -0500 Bill Bumgarner wrote: > The key with WebDAV is that use therein means that you can point > WebDAV enabled Mailman at Zope, IBM's websphere, Apache+mod_dav, > or any of the other WebDAV enabled HTTP servers out there and it > should "just work" (assuming they did the implementation right). > There is nothing proprietary about the solution and choosing it as > a foundation for archival/management of messages/attachments opens > doors (instead of closing doors). For me WebDAV raises concerns centering around authentication and access security. That said, I fail to see either the use or value in having what is essentially (if I understand it correctly) a posting tool (ability to post content to a site) *UNLESS* the archiver is running on a different machine than Mailman (in which case a networked filesystem would seem a safer/lighter approach). -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From lovelace@wayfarer.org Sun Nov 19 14:36:26 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Sun, 19 Nov 2000 09:36:26 -0500 Subject: [Mailman-Developers] [Mailman-Users] Re: Bug in 2.0rc2 adding newlist References: <3A106ECF.1857.594BDCE@localhost> <3A177A37.46F87CDD@nekodojo.org> Message-ID: <3A17E56A.691DD65E@wayfarer.org> Greg Connor wrote: > > Phil Barnett wrote: > > > > Running 2.0rc2. > > > > I was trying to add a list with bin/newlist and got this: > > > ... > > File "/home/mailman/Mailman/Utils.py", line 568, in mkdir > > os.mkdir(dir, mode) > > OSError: [Errno 2] No such file or directory: > > '/home/mailman/archives/private/bin/newlist.mbox' > > > > I got this too. Looks like a buglet in 2.0rc2. However, I noticed that > running > bin/newlist foo > fails, but just running > bin/newlist > and then typing the new listname when prompted, works. When you put the > name on the command line, it looks like it picks up "bin/newlist" as the > name instead. > I believe this has been fixed in 2.0rc3. argv was being used instead of args. -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? ------------------------------------------------------ Mailman-Users maillist - Mailman-Users@python.org http://www.python.org/mailman/listinfo/mailman-users From claw@kanga.nu Wed Nov 22 01:53:33 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 17:53:33 -0800 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: Message from Bob Tanner of "Tue, 21 Nov 2000 18:04:07 CST." <20001121180407.A20735@real-time.com> References: <20001121180407.A20735@real-time.com> Message-ID: <6080.974858013@kanga.nu> On Tue, 21 Nov 2000 18:04:07 -0600 Bob Tanner wrote: > I have to say I am very disappointed at mailman's performance with > sendmail. Especially qrunner and how it works. This is not a sendmail problem. Questionably it is a Mailman problem in that qrunner as currently architected is not terribly scalable. > I am running sendmail with 10 queues, but it looks like mailman > only understands and utilizes 1 queue. It also looks like qrunner > is single threaded AND multiple instances cannot be running at the > same time. True. > This lends itself to a very slow delivery system. Only if your MTA is configured to make it so. Example: Delivering 100 messages (1K members with 10 RCPT_TOs per message) via qrunner to Exim or Postfix takes just under 2 seconds here. Admittedly this is a fast machine. Additionally the local MTA is configured to not do name lookups on local deliveries, and, of course, I'm running a cacheing name server locally. > What sucks is qrunner just tries to deliver each message one at a > time. If message 5 takes forever on the rcpt to: command (for > example), no other messages deliveries are attempted because > qrunner sits on message 5 until it times out. Translation: You are doing name lookups on messages delivered via SMTP from the localhost. There's not much reason for this. Turn it off. > Will mailman support multiple "qfiles" directories in the future? Not for v2.0. After, yes. No designs are finalised. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From Dan Mick Wed Nov 22 02:14:35 2000 From: Dan Mick (Dan Mick) Date: Tue, 21 Nov 2000 18:14:35 -0800 (PST) Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck Message-ID: <200011220213.SAA03503@utopia.west.sun.com> > > What sucks is qrunner just tries to deliver each message one at a > > time. If message 5 takes forever on the rcpt to: command (for > > example), no other messages deliveries are attempted because > > qrunner sits on message 5 until it times out. > > Translation: You are doing name lookups on messages delivered via > SMTP from the localhost. There's not much reason for this. Turn it > off. Just out of curiosity: anyone know how to do this in sendmail? Is this the 'delay_checks' feature? I'm sure I haven't configured it, but my list isn't big enough that I care about such delays. If I were to want to change it, though, it's not at all clear to me how I would, and since it's become such a refrain....anyone know? From chuqui@plaidworks.com Wed Nov 22 02:01:34 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 18:01:34 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <14875.3450.340146.28315@anthem.concentric.net> References: <14875.1663.422580.420642@anthem.concentric.net> <14875.3450.340146.28315@anthem.concentric.net> Message-ID: At 7:04 PM -0500 11/21/00, Barry A. Warsaw wrote: > One thing I've learned though: >NEVER touch the purple lever, no matter how loudly it pleads with you. Only one thing I can say about this -- if the *lever* is pleading, you need to switch to Decaf. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 02:46:33 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 18:46:33 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <26896.974852960@kanga.nu> References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> Message-ID: At 4:29 PM -0800 11/21/00, J C Lawrence wrote: >For me WebDAV raises concerns centering around authentication and >access security. Authentication is a big bugaboo in general, which Barry and I have hashed around a bit. More on that someday, maybe. > That said, I fail to see either the use or value >in having what is essentially (if I understand it correctly) a >posting tool (ability to post content to a site) *UNLESS* the >archiver is running on a different machine than Mailman (in which >case a networked filesystem would seem a safer/lighter approach). Well.... it comes down to two key terms: The "C" word and the "I" word. Convergence and Integration. There are fewer and fewer sites that "run mailing lists" -- and more and more sites that have mailing lists as part of a larger site/strategy. all of these technologies are converging as people figure out they can be used together to complement each other -- and that convergence is creating a severe need to integrate them to keep them usable (and preferably, easy to use) for the end user. Take my sites as examples --- www.hockeyfanz.com. Apache web server, mwforum for web discussion (nad my list directories), mailman for lists, htdig for search engine (soon), mhonarc for archives, ircu for real time (soon). The forum has a separate authentication from the lists, and the archives have their own password (although I've figured out how to write an apache authentication module as a hack, so I can use mailman info instead -- someday). the forums have a different search engine than the archives will. So to fully use the system, there are two places to register/login, a third password for archives (each list has its own password, remember), and two search engines, depending on whether you want list to search the forum or mail lists. And this is after I've worked my butt off to find stuff that integrates as well as I can. It's about as close to state of the art as possible (IMHO), in terms of the technologies I need. The biggest problem I have? user confusion -- can you blame them? I understand all of these pieces, but why should they? Answer: they shouldn't. I have a site. My user should have a way to register and use it. Which piece registers what way shouldn't matter based on phase of the moon.... So for me, integration and convergence is a Big Issue. I have Ideas On How To Do this, as Barry knows, but nothing we've taken public yet. But this is the next big phase of this stuff. Not making it work, making it work TOGETHER. And as we figure out how to make it work together, we can't throw out the people who just want to run a couple of mail lists. And *that* is the problem with Zope, or WebDAV, or name your favorite integration tool - because the more you depend on that stuff, the harder it is to just use the tool itself. It has to be a floor wax and a dessert topping. Which is very possible. Ca be done, but not easy. But worth doing. All of these things are converging, and to do so succeesfully, we have to integrate. But we need to integrate in ways that are modular with standard interfaces, so that we don't require someone install 200K lines of code to get a stupid mail list running for their bridge club.... So tools like Zope, or WebDAV, or Mhonarc, or even Pipermail are important things to consider, support and perhaps use, but we have to be hugely careful about requiring the use of things, while enabling them to be used. The fewer things you force someone to use, the better off we are: but the more things wee *can* work with, the better off we'll be. (and the other cool feature is that if we avoid reinventing stuff, but instead write interfaces to them, we leverage off a lot more work of a lot more people, because writing an glue interface to mhonarc is a LOT faster and easier than writing Mhonarc. writing a glue interface to HtDig is a LOT faster and easier thanw riting HtDig. And if someone doesn't want Mhonarc or HtDig, they can simply write their own interface to their own tool, and (hopefully) submit it back to the project so we can all benefit....THAT is, IMHO, the strongest argument against integrating stuff like Pipermail into the project itself -- it requires owrk that can go into core functionality instead, and it also discourages people from building or interfacing other technologies, because we've "defined" the default. That's why I think running Pipermail as a separate project, and writing the interface glue so that it's supported by Mailman "out of the box" is a better idea than rnning them as the same project -- they're separate code pieces with a common interface that really odn't have anything in common, and I think it discourages integration of other pieces. There are things that are core functionality for mailman: subscriber services, delivering mail. There are core technologies that Mailman needs, like archives and search engines (if you want to have integrated archives, you have to have search capabilities), but services like that are probably better handled via interface glues. Tehy don't need to ship with the core code, but instead, you ship the interface and instructions on getting it installed and running. it can still be the 'official' or 'default' tools, but they don't need to be IN the distribution, especially if they're things people might or might not use. Actually, *everything* ought to be defined with a formal API that can be replaced with a different plug-in if someone wants to (at least as a long-term goal), but some stuff is appropriate to be *in* the distribution, and some interfaced to by the distribution. And which is which is part of the architecting process. But given that we're no longer in a "alone in the world" environment, we have to stop thinking in terms of doing it all ourselves. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 02:53:16 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 18:53:16 -0800 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: <20001121180407.A20735@real-time.com> References: <20001121180407.A20735@real-time.com> Message-ID: At 6:04 PM -0600 11/21/00, Bob Tanner wrote: >I have to say I am very disappointed at mailman's performance with sendmail. it works pretty well for me. I've got it running a rather large setup (50ish lists, 40,000+ subscribers, 300-400 messages posted a day) without problem. You need to go into your system, and set SMTP_MAX_RCPTS = 10 in your mm_cfg.py. That'll cause mailman to split mail delivery into batches of ten, and parallelize the hell out of your list delivery. The big problem you're seeing isn't qrunner, but that each message is sent out as a single batch to sendmail, so sendmail can't parallelize. that means delivery is as slow as the slowest machines in the queue ahead of you. Very, very bad. But with splits into smaller batches, sendamil can do smart queue management again. >Will mailman support multiple "qfiles" directories in the future? yes. That'll be improved as well down the road, but that's not really your problem here, unless you're trying to run on underpowered hardware. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 02:55:13 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 18:55:13 -0800 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: <200011220213.SAA03503@utopia.west.sun.com> References: <200011220213.SAA03503@utopia.west.sun.com> Message-ID: At 6:14 PM -0800 11/21/00, Dan Mick wrote: > > Translation: You are doing name lookups on messages delivered via > > SMTP from the localhost. There's not much reason for this. Turn it > > off. > >Just out of curiosity: anyone know how to do this in sendmail? someone (I think it was JC...) told me a while back to enable accept_unresolvable_domains as a feature in your .cf. I haven't tested it yet, though. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From bbum@codefab.com Wed Nov 22 04:23:45 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Tue, 21 Nov 2000 23:23:45 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? Message-ID: <200011220423.eAM4Njb16105@s1.optonline.net> Let me describe a bit about what I envisioned when going down the = Mailman+WebDAV implementation path. Maybe it is reasonable, maybe not. I agree with Chuq, Ken and Barry regarding various concerns with = Mailman. That the integration of technologies as a part of the required = baseline makes the tool *less* flexible and *less* easily accesible; = that security is a major concern; and that the future of Mailman should = be about refining/abstracting what we have, not glomming new bits onto = it. With that said, let me discuss some of the issues I faced and some of = the thinking that resulted. This is in no particular order and is not a = criticism of anyone's coding contributions! My code is *not*Not*NOT* = perfect and it all should be reviewed, as well!! - archival of messages is a lot more than just writing the bodies to a = web server and then generating some kind of automatic TOC/index. MIME = based email content is not only a reality, but it is often the default. = Mailman needs to more gracefully handle mime-types; automatic filters = would be one solution [very desirable for things like a pure discussion = list] and automatic decoding/filing/handling of attachments is another = solution [and very desirable for things like project discussion lists, = etc]. - message rewriting is a really powerful concept and should be an = abstract and first class part of the Mailman core processing pipeline. = When I implemented the decode-and-file-to-WebDAV solution, the first = challenge was to build a handler that sat in the delivery pipeline and = did some basic decode-and-rewrite-based-on-mailing-list-configuration as = the message passed through the pipe. Once decoded/rewritten, then = filing/processing/filtering/etc can all be abstracted *away* from the = actual decode/rewrite process. - Python's built in MIME type handling code sucks. It is great if you = want to *compose* MIME content, it sucks if you want to rip apart = existing mime content and have any kind of random access. As such, I = turned to mimecntl.py by . It was quite useful and enabled me to get the job done, but = does not have the most obvious of APIs. Did it improve in 2.0? I = don't think so, but have not looked closely. - the current solution for creating an archive is lacking-- that doesn't = mean it doesn't work. It works and it works very well, but it is *not* = modular, does *not* follow the design patterns of the rest of the tool = and makes it *very* difficult to slap in replacement models for archival = (has this changed since, say, may? If it has, I'll have to retract this = until I look again). Before WebDAV or anything else can be considered, = the actual archival process needs to be abstracted away from the hack = that is in there now. - gatewaying messages to an HTTP server as they are processed is an = extremely powerful addition [and very easy to do]. However, it is = useful to *also* add the ability to automatically requeue messages for = blastting at the HTTP server if the server returns certain error codes. = This is done in the codebase I posted an ftp:// to earlier. This allows = for *extremely* flexible integration of Mailman w/other infrastructure. - WebDAV does *not* address security. Nor does it contribute to = "security issues". Any web based archive of data is only as secure as = the Web server that allows access to the content. Given that the Web is = now the *standard* way of presenting things like Mailing list archives, = this is a security issue mailman faces *regardless* of the use of WebDAV = as the means of storing the archive. WebDAV is merely an extension to = the existing HTTP protocol. However, WebDAV does allow for = interesting potential security implementations; URLs that automatically = time out after X minutes/hours/days are trivial to implement. - for the archival of plain text messages, WebDAV is overkill [as Chuq = mentions]. However, as soon as you move to archiving mime attachments, = it quickly becomes extremely advantageous to archive various properties = with the archived message pieces. Example; if my modified Mailman = receives a message from a Macintosh with an attachment, it stores the = Creator/Type information [and other extended headers] as a set of = properties associated with the Resource on the attachment server. This = allows for... - ....restoring decoded attachments and reencoding back to their = original state with their original headers is an extremely cool feature. = With this [in the modified version of Mailman+WebDAV], it is quite = easy to send a request to Mailman that causes it to reconstitute a = single attachment in all of its original encoded glory with its original = headers based entirely on the decoded information found on the WebDAV = server. This message can then be automatically sent to the user and = will be treated just like the original attachment in the original = message. - if we are to manage the complexity associated with the integration of = numerous technologies, it is only going to happen through well refined = and highly modular APIs.... b.bum From chuqui@plaidworks.com Wed Nov 22 04:52:26 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 20:52:26 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <200011220423.eAM4Njb16105@s1.optonline.net> References: <200011220423.eAM4Njb16105@s1.optonline.net> Message-ID: At 11:23 PM -0500 11/21/00, Bill Bumgarner wrote: >- archival of messages is a lot more than just writing the bodies to >a web server and then generating some kind of automatic TOC/index. agreed completely. I'd take it a step further and say it probably shouldn't generate indexes at all, but that indexes should be generated when a user wants to access the archives, dynamically. That's probably the single major weakness of mhonarc. > MIME based email content is not only a reality, but it is often the default. and since AOL 6.0 for windows no longer has any provision for text-only operation, you now have no choice but to deal with MIME in some way. they no lnoger HAVE the option to turn MIME off. So the server has to figure it out. And IMHO, from my research, we're currently making the switch to style-capable e-mail in the mainstream. A year from now -- lists that don't support it reasonably (and I don't believe "strip it to the text part and throw the rest away" will qualify as "reasonable" -- it's a transition stopgap) wll be looked on negatively. People want enclosures and HTML, because the tools to deal with them are maturing. Looking forward, it's going t be standard, and that means all of the tools need to deal with them properly, and taht includes displaying them, transporting them, and protecting users from them as needed. > Mailman needs to more gracefully handle mime-types; automatic >filters would be one solution [very desirable for things like a pure >discussion list] and automatic decoding/filing/handling of >attachments is another solution [and very desirable for things like >project discussion lists, etc]. agreed, and it's been talked about here. We need to properly support it, plus we also need the ability to filter adn edit it -- to strip .exe files, for instance, to store HTML and graphics on the server to show properly in the archives, to generate what I call TOC-digests (simply a list of pointers into the archives), and other stuff. >- message rewriting is a really powerful concept and should be an >abstract and first class part of the Mailman core processing >pipeline. It's something I'm doing pre-design work on now, in fact. >- the current solution for creating an archive is lacking-- that >doesn't mean it doesn't work. It's very much an older-technology base -- where it's falling behind a rapidly changing technology. A year ago, pipermail was great. Now, it's not. Fortunately, I think we're close to the end of the rapid evolution of e-mail, or at least where we can take a breath, figure out reality and catch up a bit. But it's been really interesting trying to NOTICE all of the changes, much less manage them... > It works and it works very well, but it is *not* modular, does >*not* follow the design patterns of the rest of the tool and makes >it *very* difficult to slap in replacement models for archival (has >this changed since, say, may? A concept I've been sort of beating into the ground, but if there's a weakness in open source software, it tends to be written by programmers used to the "I want this, therefore I shall write it", and interfacing stuff to other pieces of similar stuff is low on the priority list, or missing. And the future of the software world are those damn C and I words I keep using... this is not a criticism, but a recognition that the paradigms are changing. I'm seeing these concepts adopted into open source as well as commercial software (probably faster in much of the open source world now, in fact) -- but the way we wrote stuff a year ago isn't how we'll write stuff in a year. We're seeing what Apple wanted to do with OpenDoc actually starting to happen, IMHO. Pluggable extensibility. Object-oriented systems, not object oriented software. >- for the archival of plain text messages, WebDAV is overkill [as >Chuq mentions]. However, as soon as you move to archiving mime >attachments, it quickly becomes extremely advantageous to archive >various properties with the archived message pieces. but you can do that with a lot less overhead in MySQL by doing a focussed database. In fact, you could program a system to do this via DBI that'd work in any DBI-capable environment, so users could roll their own based on what they've already adopted. unless WebDAV gives us enough extra capabilities to be worthy of the specialization, my argument is (and will be) we program to a more general API (like DBI), so that we work in many environments, and if someone wants, they can program a DBI->WebDAV interface to attach to it. This way, we get DB, MySQL, PostGres, Oracle, ODBC, yadayada more or less for free, giving us functionality across multiple environments that users can tailor. If we program just to WebDAV, we don't get that. So it's choosing what the appropriate interfaces are that's as important as having interfaces. you don't program to a technology unless you have to -- you program to an interface that enables technologies. (image: this is chuqui. this is a dead horse. This is chuqui holding a whip...) >- ....restoring decoded attachments and reencoding back to their >original state with their original headers is an extremely cool >feature. Truly. And if we can support BLOBs in DBI, well, we don't have to write anything to disk and can generate an entire message out of a DBI database -- portable to any decent database. >- if we are to manage the complexity associated with the integration >of numerous technologies, it is only going to happen through well >refined and highly modular APIs.... agreed. and to make ti clear, I'm not arguing against WebDAV. I'm arguing that for something like this, you define the interface and see if you can build it in a way that you don't JUST get WebDAV, but support at a more abstracted level that gets you a range of supported technologies (and future capability for that yet discovered) for an incrementally greater amount of work. the trick is to find the right abstractions and the proper technology layer to attach that to. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From barry@digicool.com Wed Nov 22 05:02:26 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 22 Nov 2000 00:02:26 -0500 Subject: [Mailman-Developers] Towards 2.1 Message-ID: <14875.21346.357211.669599@anthem.concentric.net> There have been lots of great discussion on mailman-developers about the directions Mailman should be taking. This is all very cool stuff (and I want it to continue), but I view it as being more for Mailman 3.0, for which I have no time frame at the moment. Now that 2.0 final is imminent[*], I'd also like to open up some discussion about what we want to see in 2.1. My hard commitment, long overdue, is to integrate Juan Carlos's and Victoriano's I18N work. Juan Carlos has sent me patches, which I intend to start unpacking and looking at, probably after the US Thanksgiving holiday weekend. I'd also like to start using some of the nicer Python 2.0 features to clean up the code in places. Other than that, I'd like to get people's feedback on the really crucial stuff that needs to get fixed for 2.1. I want to keep to a bare minimum anything that requires re-architecting, pushing that into 3.0 after more discussions. I'd like to try to get a 2.1 beta out by the end of the year, with 2.1 final some time in January. I don't know if that's realistic or not. From the TODO list, things I can imagine tackling for 2.1 include: - Plain text digests should conform to RFC 1153. - If a message has MIME parts and the header/footer is going to be added, the message should be transformed into a mulitpart/mixed with the header and footer added as text/plain parts. (note that this will require improving Python's standard MIME message handling. b.bum astutely observes that it basically sucks right now and fixing this dovetails with work i want to do for Python 2.1 as well). - Allow "urgent" postings to all members by the list admin which bypasses normal digest delivery. - A button that will bundled and deliver a digest Right Now. - Allow the list-admin to require approvals for unsubs - Allow the user to be excluded from postings if they're getting them in the to: or cc: headers. Be smarter about filtering out duplicate deliveries. - Allow users to subscribe without selecting a password and have Mailman create a password for them. - Allow the site admin to define list styles or themes, and list admins to choose one of the canned styles to apply to their list. - Full creation, deletion, renaming, etc. of lists through the web (and email?), including fixing aliases file updates. - add_members should have a switch to disable admin notifications - Allow individuals to turn off password reminders - Don't use the first public mailing list as the `originator' of password reminders. - Provide an email interface to all administrative commands - Add -join and -remove addresses for easy subscription, unsubscription - For email subscribes, keep an audit of where requests are coming from, and send the original request headers in the confirmation message. Helps track down subscribe bombs. - Support the `which' command. - archive link should do *something* reasonable before the first message has been posted to the list. We should also do /something/ about qrunner, but again, I want to keep radical changes to a minimum. Stability is crucial, but performance counts too. Maybe Chuq's idea of splitting the queue into 3 parts can be done without much disruption. I don't think /everything/ can be done for 2.1, and maybe other people have different priorities. I'm going to keep this list up-to-date on the Mailman2.1 Wiki: http://www.zope.org/Members/bwarsaw/MailmanDesignNotes/MailmanTwoDotOne Feel free to contribute. -Barry [*] The astute observer will notice the tarball on SourceForge already. I'm waiting for the mirror sites to update, but will announce tomorrow even if they don't. From bbum@codefab.com Wed Nov 22 05:49:46 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Wed, 22 Nov 2000 00:49:46 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011220423.eAM4Njb16105@s1.optonline.net> Message-ID: <3A1B5E7C.6E6E71AD@codefab.com> Chuq Von Rospach wrote: > At 11:23 PM -0500 11/21/00, Bill Bumgarner wrote: > > >- archival of messages is a lot more than just writing the bodies to > >a web server and then generating some kind of automatic TOC/index. > > agreed completely. I'd take it a step further and say it probably > shouldn't generate indexes at all, but that indexes should be > generated when a user wants to access the archives, dynamically. > That's probably the single major weakness of mhonarc. I'd take it a step beyond THAT and say that this is really almost a per-list issue. I have mailing lists that: - are professional in nature and I wouldn't mind even PAYING for a realtime solution that was very user friendly - low bandwidth lists that on demand indexing would not be an issue - high bandwidth free lists that once-a-night indexing would be the ideal. [bunch of useful stuff that I agree with deleted] > > > >- for the archival of plain text messages, WebDAV is overkill [as > >Chuq mentions]. However, as soon as you move to archiving mime > >attachments, it quickly becomes extremely advantageous to archive > >various properties with the archived message pieces. > > but you can do that with a lot less overhead in MySQL by doing a > focussed database. In fact, you could program a system to do this via > DBI that'd work in any DBI-capable environment, so users could roll > their own based on what they've already adopted. unless WebDAV gives > us enough extra capabilities to be worthy of the specialization, my > argument is (and will be) we program to a more general API (like > DBI), so that we work in many environments, and if someone wants, > they can program a DBI->WebDAV interface to attach to it. This way, > we get DB, MySQL, PostGres, Oracle, ODBC, yadayada more or less for > free, giving us functionality across multiple environments that users > can tailor. If we program just to WebDAV, we don't get that. This is where i disagree *very* strongly-- maybe not with the implementation choice [DBI], but with the reasoning behind it. I don't think archival should be treated as a database centric operation. The concept of archival falls very naturally into a static hierarchy of collections/directories containing resoruces/files with a bit of additional meta information associated with some resources. This is exactly the kind of information archive that a web server is *designed* to optimally serve. Adding extra layers here or abstracting to a DBI really doesn't buy us much. Alone, a basic filesystem served webserver gives us: - effecient access to archives - basic per-site, per-list authentication - [with little addition] unified access/passwords between lists, etc... - almost *zero* overhead with *very little* implementation cost WebDAV adds the ability to do advanced locking, easy meta information storage, etc... but-- most importantly-- does not take away the very effecient presentation of data naturally present within a filesystem of stuff served by a web server. As well, a filesystem centric storage/presentation solution-- webdav or raw filesystem-- solves *most* peoples archiving problems *most* of the time. I feel *very strongly* that the archival solution-- whether it be raw messages or decoded messages-- should be centric to storing files in directories and serving files from directories. The second reason I feel strongly that moving to a DBI based interface wouldn't present that much of an advantage is that most people that need to actually store the data in a database are going to have their own requirements surrounding decoding, storage, indexing, and presentation of said database related content. There are few *real* standards in terms of the storage of multimedia [MIME] content into a database environment and, as such, the developer will likely have to rip the data out of whatever our implementation prefers and into their own storage subsystem. In my experience [storing email into a database was actually a problem we had to solve-- this is the implementation we successfully/effectively used], it is far more convenient to provide an HTTP [replaced with a MODULAR in a real implementation] gateway that delivers the processed, but still relatively raw, messages to some other subsystem for subsequent parsing and storage. In our case, we used HTTP to deliver inbound messages to a WebObjects application that parsed the message into EOs [enterprise objects] and persisted those via the various APIs included with WebObjects. Another way of looking at this is that as soon as most developers are going to want to work with the data in the context of a true database, most developers are also going to want to actually use their tool-du-jour [WebObjects, ASP, EJB, PHP, Zope, etc...] to process that information. Taking a two pronged aproach to archival-- we provide a simple [and modular] filesystem-esque approach to store the data in a more traditional manner-- be it directly to the filesystem or via a WebDAV adaptor (since WebDAV is very filesystem like, just w/HTTP as the protocol of choice) and an equally as simple modular gateway that allows the developer/administrator to easily configure the system such that the data is delivered to their server of choice via the protocol of chioice-- will likely reduce the complexity of our implementation and increase the attractiveness in that our codebase is that much simpler and more approachable. This is *not* to say that the DBI approach isn't the right way to go; if a generic DBI->filesystem, DBI->WebDAV, DBI->DB capable API were put together and was relatively hidden from the user and casual developer, it might be a huge win. > > So it's choosing what the appropriate interfaces are that's as > important as having interfaces. you don't program to a technology > unless you have to -- you program to an interface that enables > technologies. (image: this is chuqui. this is a dead horse. This is > chuqui holding a whip...) And bbum following with a club.... :-) Agreed. > > >- ....restoring decoded attachments and reencoding back to their > >original state with their original headers is an extremely cool > >feature. > > Truly. And if we can support BLOBs in DBI, well, we don't have to > write anything to disk and can generate an entire message out of a > DBI database -- portable to any decent database. But an order of magnitude less effecient than downloading the BLOB off of disk via a webserver! Generic access with simple access control is what *most* users/administrators want *most* of the time. More complex/abstracted/portable access is less of a requirement and *a lot* of the people with such requirements also have other issues-- real or imagined-- that dictate that they really just want Mailman to hand the stuff to them as quickly/easily as possible and be done with it. > >- if we are to manage the complexity associated with the integration > >of numerous technologies, it is only going to happen through well > >refined and highly modular APIs.... > > agreed. and to make ti clear, I'm not arguing against WebDAV. I'm > arguing that for something like this, you define the interface and > see if you can build it in a way that you don't JUST get WebDAV, but > support at a more abstracted level that gets you a range of supported > technologies (and future capability for that yet discovered) for an > incrementally greater amount of work. the trick is to find the right > abstractions and the proper technology layer to attach that to. Totally-- and I hope no one thought I was advocating WebDAV as the end all, be all, only solution! I feel strongly that abstraction is key, but that we should also provide decent, production quality, implementations of solutions to the very same set of problems for which we build the gneric abstracted/modularized APIs. If Mailman is not fully functional "out of the box", then people will ignore it. However, if it isn't also flexible enough to be integrated into their weird environments (because every server on the web has weirdness), they'll bitch and moan until they find something else that doesn't solve their problem to B&M about.... b.bum From marouni@earlham.edu Wed Nov 22 05:53:06 2000 From: marouni@earlham.edu (Nick Marouf) Date: Wed, 22 Nov 2000 00:53:06 -0500 Subject: [Mailman-Developers] Towards 2.1 References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: <3A1B5F41.FEE1A62B@earlham.edu> Hi Barry, A fellow student of mine is working on getting full name support for mailman addresses. Is there a way we can get that in with the 2.1 release? I will keep you updated on his progress and send in a patch once that is done. Thanks for all the great work. Nick "Barry A. Warsaw" wrote: > There have been lots of great discussion on mailman-developers about > the directions Mailman should be taking. This is all very cool stuff > (and I want it to continue), but I view it as being more for Mailman > 3.0, for which I have no time frame at the moment. > > Now that 2.0 final is imminent[*], I'd also like to open up some > discussion about what we want to see in 2.1. My hard commitment, long > overdue, is to integrate Juan Carlos's and Victoriano's I18N work. > Juan Carlos has sent me patches, which I intend to start unpacking and > looking at, probably after the US Thanksgiving holiday weekend. I'd > also like to start using some of the nicer Python 2.0 features to > clean up the code in places. > > Other than that, I'd like to get people's feedback on the really > crucial stuff that needs to get fixed for 2.1. I want to keep to a > bare minimum anything that requires re-architecting, pushing that into > 3.0 after more discussions. I'd like to try to get a 2.1 beta out by > the end of the year, with 2.1 final some time in January. I don't > know if that's realistic or not. > > >From the TODO list, things I can imagine tackling for 2.1 include: > > - Plain text digests should conform to RFC 1153. > - If a message has MIME parts and the header/footer is going to be > added, the message should be transformed into a mulitpart/mixed > with the header and footer added as text/plain parts. > > (note that this will require improving Python's standard MIME > message handling. b.bum astutely observes that it > basically sucks right now and fixing this dovetails with work i > want to do for Python 2.1 as well). > > - Allow "urgent" postings to all members by the list admin which > bypasses normal digest delivery. > - A button that will bundled and deliver a digest Right Now. > - Allow the list-admin to require approvals for unsubs > - Allow the user to be excluded from postings if they're getting > them in the to: or cc: headers. Be smarter about filtering out > duplicate deliveries. > - Allow users to subscribe without selecting a password and have > Mailman create a password for them. > - Allow the site admin to define list styles or themes, and list > admins to choose one of the canned styles to apply to their > list. > - Full creation, deletion, renaming, etc. of lists through the web > (and email?), including fixing aliases file updates. > - add_members should have a switch to disable admin notifications > - Allow individuals to turn off password reminders > - Don't use the first public mailing list as the `originator' of > password reminders. > - Provide an email interface to all administrative commands > - Add -join and -remove addresses for easy subscription, > unsubscription > - For email subscribes, keep an audit of where requests are coming > from, and send the original request headers in the confirmation > message. Helps track down subscribe bombs. > - Support the `which' command. > - archive link should do *something* reasonable before the first > message has been posted to the list. > > We should also do /something/ about qrunner, but again, I want to > keep radical changes to a minimum. Stability is crucial, but > performance counts too. Maybe Chuq's idea of splitting the queue > into 3 parts can be done without much disruption. > > I don't think /everything/ can be done for 2.1, and maybe other people > have different priorities. I'm going to keep this list up-to-date on > the Mailman2.1 Wiki: > > http://www.zope.org/Members/bwarsaw/MailmanDesignNotes/MailmanTwoDotOne > > Feel free to contribute. > > -Barry > > [*] The astute observer will notice the tarball on SourceForge > already. I'm waiting for the mirror sites to update, but will > announce tomorrow even if they don't. > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers -- Nicholas Marouf http://www.ramallahonline.com From claw@kanga.nu Wed Nov 22 06:04:14 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:04:14 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 20:52:26 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> Message-ID: <7649.974873054@kanga.nu> On Tue, 21 Nov 2000 20:52:26 -0800 Chuq Von Rospach wrote: > At 11:23 PM -0500 11/21/00, Bill Bumgarner wrote: >> - archival of messages is a lot more than just writing the bodies >> to a web server and then generating some kind of automatic >> TOC/index. > agreed completely. I'd take it a step further and say it probably > shouldn't generate indexes at all, but that indexes should be > generated when a user wants to access the archives, dynamically. > That's probably the single major weakness of mhonarc. I have half a design and some basic code roughed out to have the PHP+MHonArc setup I used on Kanga.Nu instead inject the deconstructed data into an SQL DB with the message presentation code rather than reading from the PHP variable assignements sucking everything out of the DB etc etc etc. It wouldn't be very difficult to do and MHonArc has excellent MIME parsing and handling features to act in support of this. >> MIME based email content is not only a reality, but it is often >> the default. > and since AOL 6.0 for windows no longer has any provision for > text-only operation, you now have no choice but to deal with MIME > in some way. they no lnoger HAVE the option to turn MIME off. So > the server has to figure it out. Ahh. That explains why I've been rejectting so many postings from AOL members on my lists lately (I don't allow HTML as an aid to promoting high signal lists). -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 06:08:33 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 22:08:33 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <3A1B5E7C.6E6E71AD@codefab.com> References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> Message-ID: At 12:49 AM -0500 11/22/00, Bill Bumgarner wrote: >This is where i disagree *very* strongly-- maybe not with the implementation >choice [DBI], but with the reasoning behind it. > >I don't think archival should be treated as a database centric operation. The >concept of archival falls very naturally into a static hierarchy of >collections/directories containing resoruces/files with a bit of >additional meta >information associated with some resources. This is exactly the kind of >information archive that a web server is *designed* to optimally >serve. Adding >extra layers here or abstracting to a DBI really doesn't buy us much. um, except you seem to be defining a database-centric operation using a filesystem as a data store. It's still database-centric. >Alone, a basic filesystem served webserver gives us: > > - effecient access to archives > > - basic per-site, per-list authentication > > - [with little addition] unified access/passwords between lists, etc... > > - almost *zero* overhead with *very little* implementation cost which are all as true with a database based system. Except I don't agree withyou on the overhead and implementation costs on your end. Ask anyone who manages a decent-sized NNTP system -- filesystem centric storage systems don't scale to large data sets well at all. Databases do.... >I feel *very strongly* that the archival solution-- whether it be >raw messages or >decoded messages-- should be centric to storing files in directories >and serving >files from directories. And I disagree, since you're tying to a single technology that works for some cases, but isn't a general solution, and limits other options. For instance, it's fairly easy to implement a lot of searching via DBI. it's a lot harder using a filesystem store. >The second reason I feel strongly that moving to a DBI based >interface wouldn't >present that much of an advantage is that most people that need to >actually store >the data in a database are going to have their own requirements surrounding >decoding, storage, indexing, and presentation of said database >related content. >There are few *real* standards in terms of the storage of multimedia [MIME] >content into a database environment and, as such, the developer will >likely have >to rip the data out of whatever our implementation prefers and into their own >storage subsystem. I don't see this as an issue. You have to define an architecture either way. >This is *not* to say that the DBI approach isn't the right way to go; if a >generic DBI->filesystem, DBI->WebDAV, DBI->DB capable API were put >together and >was relatively hidden from the user and casual developer, it might >be a huge win. which is exactly what I'm arguing. And DBI is a well-known interface that ias easily accessible to anyone who wants to write an interface -- if the architecture is done right. And it's portable off of Unix, which is an issue for Mailman. > > Truly. And if we can support BLOBs in DBI, well, we don't have to >> write anything to disk and can generate an entire message out of a >> DBI database -- portable to any decent database. > >But an order of magnitude less effecient than downloading the BLOB >off of disk via >a webserver! Not necessarily. And is the efficiency important? A lot of time is wasted in computer development optimizing the wrong things.... >Generic access with simple access control is what *most* users/administrators >want *most* of the time. More complex/abstracted/portable access >is less of a >requirement and *a lot* of the people with such requirements also have other >issues-- real or imagined-- that dictate that they really just want Mailman to >hand the stuff to them as quickly/easily as possible and be done with it. that's changing, rapidly. Generic, text-only e-mail was what most users/administrators wanted a year ago, too. If you implement to what they WANTED, by the time it's written, they won't want it any more. You have to figure out what's possible and what they're going to want when you're done writing it. you don't want to write obsolete stuff because you implement what they wanted last time. >I feel strongly that abstraction is key, but that we should also >provide decent, >production quality, implementations of solutions to the very same >set of problems >for which we build the gneric abstracted/modularized APIs. agreed. >If Mailman is not fully functional "out of the box", then people >will ignore it. >However, if it isn't also flexible enough to be integrated into their weird >environments (because every server on the web has weirdness), >they'll bitch and >moan until they find something else that doesn't solve their problem to B&M >about.... it has to be functional out of the box, but we have to make sure we define "functional" properly, too. You can't be missing features but doing the "kitchen sink" think just in case is wrong, too. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From tanner@real-time.com Wed Nov 22 06:13:14 2000 From: tanner@real-time.com (Bob Tanner) Date: Wed, 22 Nov 2000 00:13:14 -0600 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: ; from chuqui@plaidworks.com on Tue, Nov 21, 2000 at 06:53:16PM -0800 References: <20001121180407.A20735@real-time.com> Message-ID: <20001122001314.B3147@real-time.com> Quoting Chuq Von Rospach (chuqui@plaidworks.com): > You need to go into your system, and set > > SMTP_MAX_RCPTS = 10 I had it down to 50, but I'll give 10 a try. Thanks. I think this should be an FAQ. FAQ #2 states to turn off synchronous DNS resolution. Anyone know how to do this via sendmail's .mc files? -- Bob Tanner | Phone : (612)943-8700 http://www.mn-linux.org | Fax : (612)943-8500 Key fingerprint = 6C E9 51 4F D5 3E 4C 66 62 A9 10 E5 35 85 39 D9 From chuqui@plaidworks.com Wed Nov 22 06:13:12 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 22:13:12 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <7649.974873054@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> Message-ID: >At 10:04 PM -0800 11/21/00, J C Lawrence wrote: >> and since AOL 6.0 for windows no longer has any provision for >> text-only operation, you now have no choice but to deal with MIME >> in some way. they no lnoger HAVE the option to turn MIME off. So >> the server has to figure it out. > >Ahh. That explains why I've been rejectting so many postings from >AOL members on my lists lately (I don't allow HTML as an aid to >promoting high signal lists). yup. and you can't turn it off. There's no option. You HAVE to deal with multipart-alternative and pull the right pieces out, or their dead in the water. Any list that can't handle MIME e-mail is incompatible with AOL 6 for windows now... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From claw@kanga.nu Wed Nov 22 06:31:44 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:31:44 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Bill Bumgarner of "Wed, 22 Nov 2000 00:49:46 EST." <3A1B5E7C.6E6E71AD@codefab.com> References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> Message-ID: <11426.974874704@kanga.nu> On Wed, 22 Nov 2000 00:49:46 -0500 Bill Bumgarner wrote: > I don't think archival should be treated as a database centric > operation. The concept of archival falls very naturally into a > static hierarchy of collections/directories containing > resoruces/files with a bit of additional meta information > associated with some resources. This is exactly the kind of > information archive that a web server is *designed* to optimally > serve. Adding extra layers here or abstracting to a DBI really > doesn't buy us much. While true in general, this ignores the benefits and ease of dynamically generated reports on archives. Now, perhaps I should talk a little about what I'm spending my copious spare time on lately (other than attempting to chase heisenbugs that crash machines for unknowable reasons). Among other things I'm attempting to meld the basic concepts of WikiWiki, mail archives, and advogato/slashdot-style commentary systems into a form where the basic tenets of Wiki noun definition are retained such that every individual comment and archive page has a dynamically generated/assigned WikiName and there is a moderately sound karma/trust net running as a meta layer atop this as an abstracted report layer. The fact that further abstractions can be made, and that rule-based reports/abstractions can be created as dynamically filled out WikiPages on the other side of this is just, well, really nice. I realise that's pretty turgid. Without spending considerable time and verbiage I don't want to spend right now I'm not sure I can do better. If you're interested in details, and I don't mean just casually, please ask away. But, getting back to the point, to do this properly (ie so that its scalable and reasonably system efficient), especially given the fact that any given data item (be it an archived message, a comment, a report, or whatever) has at least presentation modes each of which has very different surrounding capabilities, the apparancy is that I'm going to have to shove everything into a set of SQL tables. The current half-implementation is a mix of SQL and static files (mostly wrapped around my current PHPLib template based PHP4+MHonArc based archiving setup). > WebDAV adds the ability to do advanced locking, easy meta > information storage, etc... but-- most importantly-- does not take > away the very effecient presentation of data naturally present > within a filesystem of stuff served by a web server. I'm actually considering WebDav slightly longer term as a potential Wiki-ish posting method. I'm not at all convinced its a good way to go, but I'm willing to look into it. > As well, a filesystem centric storage/presentation solution-- > webdav or raw filesystem-- solves *most* peoples archiving > problems *most* of the time. Very true. > I feel *very strongly* that the archival solution-- whether it be > raw messages or decoded messages-- should be centric to storing > files in directories and serving files from directories. In the general case, I agree. > This is *not* to say that the DBI approach isn't the right way to > go; if a generic DBI->filesystem, DBI->WebDAV, DBI->DB capable API > were put together and was relatively hidden from the user and > casual developer, it might be a huge win. I'd like to see something trivial: The archiver deconstructs messages into a stasndardised and well documented templatised form on to the file system. A standard CGI-like toolset is then provided to reconstruct those fragments back into a message (please resend this message to me, I didn't get it the first time), into an HTML page (web archives), etc. This is done by having standard small scripts which do >whatever< by assembling simple call sequences to the toolset. End users may redefine this by re-writing said scripts, acessing the deconstructions or reconstructions as they wise, to then do anything they want with the resulting archives. This could be SQL inserts, report generation, or black voodoo, it really doesn't matter, and more importantly, Mailman, and Mailman's archiver doesn't need to know about it, only the user does. The other nice thing is that we don't do it by defining APIs (which then reuires programmers to do useful things with), but by defining small useful atomic tools that may be assembled/ordered in interesting fashions wihtou having to subject the users to Python, or programming much beyond writing shell scripts and editing canned templates. > But an order of magnitude less effecient than downloading the BLOB > off of disk via a webserver! Greenspun > Generic access with simple access control is what *most* > users/administrators want *most* of the time. More > complex/abstracted/portable access is less of a requirement and *a > lot* of the people with such requirements also have other issues-- > real or imagined-- that dictate that they really just want Mailman > to hand the stuff to them as quickly/easily as possible and be > done with it. Precisely. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Wed Nov 22 06:39:21 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:39:21 -0800 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: Message from Bob Tanner of "Wed, 22 Nov 2000 00:13:14 CST." <20001122001314.B3147@real-time.com> References: <20001121180407.A20735@real-time.com> <20001122001314.B3147@real-time.com> Message-ID: <12681.974875161@kanga.nu> On Wed, 22 Nov 2000 00:13:14 -0600 Bob Tanner wrote: > Quoting Chuq Von Rospach (chuqui@plaidworks.com): >> You need to go into your system, and set >> >> SMTP_MAX_RCPTS = 10 > I had it down to 50, but I'll give 10 a try. Thanks. FWLIW I usually use either 5 or 10. Depending on your MTA (Historically I've been using Exim, recently moved to Postfix for non-MTA related reasons) and the distribution of your membership lists across domains this can significantly speed queue emptying speed. BTW: I haven't checked and I don't have anything in my spools to check. Does Mailman currently pre-sort the RCPT_TO list by domain? BTW I recently wrote the following in another forum on Postix and Exim: ---- I've spent the last week or so playing with both fairly extensively. They're sweet systems. More interestingly the performance curves for both are remarkably similar once your mail loads become large (ie your spool is always active (mail to deliver)). The different mostly lies in the curve up to continual activity, and the curve as they approach saturation (mail is arriving more quickly than they can get rid of it on a near continual basis). Loosely, Postfix' initial curve is very steep, but they both reach similar delivery rates for the continually active point. I've had some problems simulating sustained saturation (test networks et al), but Exim seems to be friendlier to the localhost in such circumstances. Exim appears to implicitly assume that such queue congestion is temporary and that given enough time things will slow down enough so it can empty the queue. As such its less interested in "get that damned mail out of here now" than it is in, "be nice to the local machine and get the mail out when you can". There are obvious problems with this if your loads stay at or above capacity for extended periods, but equally, its rather nice from an admin perspective for the local machine. Postfix conversely seems to operate on the "get that mail out of here now", and can be rather brutal on the local host during that process. Postfix under sustained saturation makes for a very heavily loaded system. Depending on what else you are doing there, this may be a problem, or not. It is this nature however which gives it the steep attack on delivery rates as versus Exim. The fact that their sustained delivery rates are so similar at the end of that curve is the really interesting bit however. ---- Unfortunatly I didn't have time to keep careful metrics, and my process wasn't really rigorous enough for that either a I was just trying to get some basic parameters on their behaviour. > I think this should be an FAQ. > FAQ #2 states to turn off synchronous DNS resolution. Anyone know > how to do this via sendmail's .mc files? Sorry, no idea. I gave up sendmail 5 years ago. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Wed Nov 22 06:41:56 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:41:56 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 22:13:12 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> Message-ID: <13342.974875316@kanga.nu> On Tue, 21 Nov 2000 22:13:12 -0800 Chuq Von Rospach wrote: >> At 10:04 PM -0800 11/21/00, J C Lawrence wrote: >> Ahh. That explains why I've been rejecting so many postings from >> AOL members on my lists lately (I don't allow HTML as an aid to >> promoting high signal lists). > yup. and you can't turn it off. There's no option. You HAVE to > deal with multipart-alternative and pull the right pieces out, or > their dead in the water. Any list that can't handle MIME e-mail is > incompatible with AOL 6 for windows now... Sad. I already don't allow posts from services that append advertising to their messages (eg Hotmail, Yahoo, etc). I guess I'll have to explicitly add AOL to that list. Annoying. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 06:42:02 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 22:42:02 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <11426.974874704@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> <11426.974874704@kanga.nu> Message-ID: At 10:31 PM -0800 11/21/00, J C Lawrence wrote: > >I'd like to see something trivial: > > The archiver deconstructs messages into a stasndardised and well > documented templatised form on to the file system. XML. Think XML. then it can be repurposed by using different definitions without having to convert from form to form based on context. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 06:43:40 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 22:43:40 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <13342.974875316@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> Message-ID: > >Sad. I already don't allow posts from services that append >advertising to their messages (eg Hotmail, Yahoo, etc). I guess >I'll have to explicitly add AOL to that list. Annoying. or strip to the pieces you find acceptable, using something like demime. Which I hope to be able to get into 2.1 in some form.... Rather than reject -- adapt. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From claw@kanga.nu Wed Nov 22 06:47:46 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:47:46 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 22:42:02 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> <11426.974874704@kanga.nu> Message-ID: <14150.974875666@kanga.nu> On Tue, 21 Nov 2000 22:42:02 -0800 Chuq Von Rospach wrote: > At 10:31 PM -0800 11/21/00, J C Lawrence wrote: >> The archiver deconstructs messages into a stasndardised and well >> documented templatised form on to the file system. > XML. Think XML. then it can be repurposed by using different > definitions without having to convert from form to form based on > context. If you bug my offices or meeting rooms one more time I am going have to come on over there and shove those damned microphones right up... <> -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Wed Nov 22 06:53:10 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:53:10 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 18:46:33 PST." References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> Message-ID: <14763.974875990@kanga.nu> On Tue, 21 Nov 2000 18:46:33 -0800 Chuq Von Rospach wrote: > At 4:29 PM -0800 11/21/00, J C Lawrence wrote: >> For me WebDAV raises concerns centering around authentication and >> access security. > Authentication is a big bugaboo in general, which Barry and I have > hashed around a bit. More on that someday, maybe. FWLIW I see authentication visavis Mailman as a two level problem: list activities (command confirmations) access control The former can be handled with ad hod dynamically generated tokens much as subscribe confirms are handled now. I've posted some notes on good implementations on this previously (I liked the bit about auto-genning an URL that did the command confirmation). The latter just needs to be abstracted to a small script which accepts two command line parameters: UserID and Password. The user can then replace that script with anything he pleases, thus authenticating agsinst he pleases be it SQL, LDAP, or lunar weather sensors. Ditto BTW holds tru for handling membership lists: just have a tool which when run returns the list of members. Simple command line options then spec returning account details, configs, etc. A little over head for text parsing, but not a whole lot (ObNote XML is a reasonable communications format). Simple, easy to extrapolate, nice efficient piped IO, etc. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Wed Nov 22 06:58:22 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:58:22 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 22:43:40 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> Message-ID: <15324.974876302@kanga.nu> On Tue, 21 Nov 2000 22:43:40 -0800 Chuq Von Rospach wrote: >> Sad. I already don't allow posts from services that append >> advertising to their messages (eg Hotmail, Yahoo, etc). I guess >> I'll have to explicitly add AOL to that list. Annoying. > or strip to the pieces you find acceptable, using something like > demime. Which I hope to be able to get into 2.1 in some form.... > Rather than reject -- adapt. I've spent considerable time thinking about this and believe it is to the lists' advantage in this case to hold firm. My core promise is signal. In the final analysis it is the only promise I make and keep for the lists: there will be signal, always. To achieve that I need fairly repsectable norming forces. To achieve that rites of participation, minor barriers to entry, identifying characteristics, etc are all cultural assistants to establishing what is in essence a club (as in professional or gentlemans) atmosphere and culture. Avoiding HTML, or in this case AOL, would seem a minor and reasonable fillip on this. Aside: Less than 5% of my members are from AOL. Less than 1% of my semi-regular posters are from AOL. The impact here (for me) is quite small. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 07:03:10 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 23:03:10 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <14763.974875990@kanga.nu> References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> Message-ID: At 10:53 PM -0800 11/21/00, J C Lawrence wrote: > > Authentication is a big bugaboo in general, which Barry and I have >> hashed around a bit. More on that someday, maybe. > >FWLIW I see authentication visavis Mailman as a two level problem: > > list activities (command confirmations) > access control Okay, I hope Barry doesn't mind, but since the talk has started, here's something I sent him a while back that we've sat on to get 2.0 out the door. But it's a first cut at my idea where some of this ought to go... When you see this, you'll start to really understand where I'm coming from in this whole API/module thing -- everything in the system is a module of some sort, that communicates via an API -- which means that in theory, aynone can replace any piece of functionality by simply replacing the module. It also means that each module can be tracked and developed separately, since all it needs to worry about is conforming to the published API. The trick is going to be architecting the modules, interfaces and APIs, and deciding one which ones are central to mailman, which ones peripheral and which ones are optional. But when we're done, it doesn't matter if you use DB or a filesystem database model, as long as the glue inteface exists. And it doesn't matter if you use the current mailman subscription interface or download your data from a corporate database running Oracle and throw out the entire subscription system -- because Mailman no longer cares. So the whole archiving issue is a subset of a larger issue, which is defining and standardizing all of Mailman and breaking it up into pieces that meld into a cohesive whole. --- Date: Wed, 11 Oct 2000 23:30:28 -0700 Subject: Re: Planning Mailman 2.1 At 12:15 AM -0400 10/12/00, Barry A. Warsaw wrote: > > CVR> Actually, I want to talk to you about this -- I have an idea > CVR> that you'll love, or hate, and no middle ground. It'd be a > CVR> "mailman 3.0" thing, and it'll turn it on its ear. But the > CVR> possibilities are stupendous. I just haven't brought it up > CVR> because I know you've been busy. > >(barry's comments deleted...) Okay, here's the other rock. What I want to propose is a new major project -- an open, extensible membership database system. This would completely replace the existing mailman subscriber system. Here's what it is -- it's a web-based (and, perhaps, optional e-mail) interface to a member database. That database would manage and authenticate members for a given environment (site, subset of a site, whatever we plug into it). Front end is web. Back end is DBI, and I'd want to do initial support with MySQL, but in theory, want to allow it to use and DBI-compatbile system). So far, no biggie. But here's where it gets different. The membership database system uses a plug-in architecture of some sort and defined APIs to allow you to write an access module for anything that needs a membership database. That will allow you to define a tab on the membership page to manage whatever it is you need managed, by using the common pool of data (user name, email addr, etc, etc), plus keeping custom data for each module. End result, you go to a membership page, register, and once registered, can access any system configured in -- whether it's mailman, user forums, a personal home page, web logs, whatever. For the user, there's a single consistent interface, single account, single password, and no confusion. for the project (like mailman), it lets the membership database worry about that stuff and focus on doing what it does best -- delivering mail. I've come up with this while trying to put together a multi-system community setup. lists, web boards, archives, real time stuff. And doing research into communities about other things people might want, whether it's public member directory pages, archive access controls, even things like a community calendar. Trying to integrate everything is, um, a bitch. Everyone wants to control their own data, when lots of that data is in common. So, I thought, build a centralized system for managing that common data, and allow it to be customized by compatible systems that interface to it. for mailman, you rip out all of the user and admin stuff, and instead write an access module that defines what mailman needs for subscriptions, how to query mailman for info (like what lists to manage), and how to get subscriber lists so you can deliver to them. A second module handles defining admin access, because if we do this right, those are simply tabs on the member pages as well. It's conceptually very simple -- but implementing it's going to get gnarly. it's something that will require a lot of architecting (basically, we're going to need an interface like Apache's DSO), and to do ti right, we'll have to define a number of services to implement with it to make sure the API works. MLM is one, public member directories is another, and web forums is an obvious third (whether we evangelize some technology in or go try to build one we'll have to see). Oh, and it needs to be language independent, although it would be written in Python. Sound interesting? kinda like a membership ZOPE. I think simply the idea of a system that allows people to stop building yet-another-member-database is a big win, but if we can start integrating stuff as well, it gives sites real opportunity for synergies down the road. It needs a lot of fleshing out, and it's not a short term project, which is why I think it's something to do parallel with the Mailman 3 timeframe. It's something I'm more than happy to spearhead and architect, but at the same time, it needs some serious partners and backing. It's something that if you toss it onto freshmeat and wait for people to find it, it'll die. But if we can find a few key technologies to work with it and make it happen and support it, could really go places. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 07:09:55 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 23:09:55 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <15324.974876302@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> Message-ID: At 10:58 PM -0800 11/21/00, J C Lawrence wrote: >I've spent considerable time thinking about this and believe it is >to the lists' advantage in this case to hold firm. you'll lose. AOL is the first to make this mandatory for their users, but users want these features. And besides, AOL is the 800 pound gorilla, and it's eating its lunch on your fence. Do you really think they're going to change back before your fence collapses under it? AOL is too large to tell to go to hell if you're running general purpose lists. If you have a specialist audience, maybe -- but on my lists, AOL is generally 12-30% of my membership. That's a lot of cutting off of noses to spite the face. >Aside: Less than 5% of my members are from AOL. Less than 1% of my >semi-regular posters are from AOL. The impact here (for me) is >quite small. lucky you. you're not typical then, not given the size of AOL. But -- it's a losing battle, JC. AOL is the first. It won't be the only. My numbers show me that not only do ~90% of mail users have support for MIME email (stylized text, html, etc.), but most of them want to use it. And they don't like having to go through gyrations to avoid it -- and most have no clue how, and don't want to. So I think it's important that the server end deal with this rationally, because otherwise, you're fighting a constant battle of teaching users to placate the server, one user at a time. instead of teaching the server to deal with the users once. It turns into either a brick wall for users to bang their heads against and then give up and leave you -- or a huge support battle for you. The war's over. there's a huge difference between keeping your web site accessible to Lynx users (a good thing!) and demanding users use lynx to access your site (a bad thing!). And now, not dealing with this rationally on the server is like telling everyone to use Lynx to read your web pages. It works if you have the right niche audience and/or have content they can't live without. But few people are in that kind of sellers market.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From claw@kanga.nu Wed Nov 22 07:22:06 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 23:22:06 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 23:09:55 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> Message-ID: <18729.974877726@kanga.nu> On Tue, 21 Nov 2000 23:09:55 -0800 Chuq Von Rospach wrote: > At 10:58 PM -0800 11/21/00, J C Lawrence wrote: >> I've spent considerable time thinking about this and believe it >> is to the lists' advantage in this case to hold firm. > you'll lose. AOL is the first to make this mandatory for their > users, but users want these features. And besides, AOL is the 800 > pound gorilla, and it's eating its lunch on your fence. Do you > really think they're going to change back before your fence > collapses under it? No, but I also realise that AOL actually is't sitting on my fence. They are sitting on other people's fence to be sure but their presence in my neighborhood is small enough to be noise. >> Aside: Less than 5% of my members are from AOL. Less than 1% of >> my semi-regular posters are from AOL. The impact here (for me) >> is quite small. > lucky you. you're not typical then, not given the size of AOL. Precisely. > But -- it's a losing battle, JC. AOL is the first. It won't be the > only. My numbers show me that not only do ~90% of mail users have > support for MIME email (stylized text, html, etc.), but most of > them want to use it. And they don't like having to go through > gyrations to avoid it -- and most have no clue how, and don't want > to. While I agree (with some distaste), those gyrations have shown themselves to have positive value for me. FWIW I also enforce quoting style (new text below quote), quoting formats (must have leading quote characters on quoted lines), correct attributions for all quoted text, etc, which are leading directly against AOL, MSN et al, but which have also helped define the value I offer. It a per-case specific question, not suitable for the general case. > t few people are in that kind of sellers market.... Too true. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From bbum@codefab.com Wed Nov 22 07:23:28 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Wed, 22 Nov 2000 02:23:28 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> Message-ID: <3A1B7471.5F16AAFA@codefab.com> Chuq Von Rospach wrote: > >Alone, a basic filesystem served webserver gives us: > > > > - effecient access to archives > > > > - basic per-site, per-list authentication > > > > - [with little addition] unified access/passwords between lists, etc... > > > > - almost *zero* overhead with *very little* implementation cost > > which are all as true with a database based system. Except I don't > agree withyou on the overhead and implementation costs on your end. > Ask anyone who manages a decent-sized NNTP system -- filesystem > centric storage systems don't scale to large data sets well at all. > Databases do.... Yes-- all as true with a database system... but require the addition of something other than just a vanilla HTTP server to implement the "get the data" part. Everything on the list above save for the "with little addition" item can be done w/an out of the box apache. Obviously, this does not include the administrative part-- the piece that does per list/per-site configuration requires some additional work regardless of a database or filesystem (or WebDAV) backing store. Another advantage to a filesystem based archival arrangement is that it is *extremely* easy to write a random shell script or two that prunes data from crontab, rebuilds indices, moves stuff around, reformats things, archives off to archived archives, etc... Yes-- of course-- all of these operations can be done with a database backing store, as well, but it is significantly more difficult to develop such tools and install them into the system. Likely, this is somewhat of a perception issue, but most administrators will not hesitate to toss together a shell script to manage an archive of stuff, but will think twice before diddling a database. Very large and very expensive databases scale very well-- MySQL does not. I agree that for truly huge, high traffic sites, moving away from a pure filesystem approach-- moving everything into, say, an ES10000 running one of the magawhompus Oracle/Sybase license-- would be the way to go. But I don't think that is what 90% of the Mailman users are going to be using the system for and requiring-- or even encouraging the use of-- a database as a backing store for messages will not add value to those people. Considering most of the usage of an archive of messages.... - write operations are infrequent, modifications pretty much non existant - retrieval tends to be extremely sporadic and is generally *not* evenly divided across the archive-- a relative few messages receive most of the hits - there are extremely limited ways of viewing the data; by author, date, thread, subject.... with MOST views focused on thread. - indices are periodically updated ... I still believe that a webserver-reading-files-from-filesystem is going to be loads more effecient than a webserver-reading-data-from-client-server-connection-to-app-adaptor-reading-data-from-client-server-connection-to-multiuser-database. It is the same reason why publishing systems don't often put the images in the database [instead, dropping URLs or other symbolic references into the database]-- modern operating systems [Linux, FreeBSD, OSX, Solaris] are very good about caching data off of the disk in memory until that memory is needed for something else. A webserver sitting on such a system will typically map anything the client asks for into memory and not reread it off the disk until something else bumps it out of memory. Even if the web server is cycling children, the underlying filesystem cache isn't going to push mapped files out of memory unless that memory is needed for something else. Certainly, solution providers like Oracle and Sybase rolled their own webservers that connect relativley directly to the database. I don't htink very many Mailman installations woould use this and, if they do, the cost of adding a generic "accept an http msg w/an XML body describing an email and I'll deal w/archiving it in the fashion most appropriate to my implementation needs" is minor compared to the other costs likely inherent to such a project. > > >I feel *very strongly* that the archival solution-- whether it be > >raw messages or > >decoded messages-- should be centric to storing files in directories > >and serving > >files from directories. > > And I disagree, since you're tying to a single technology that works > for some cases, but isn't a general solution, and limits other > options. For instance, it's fairly easy to implement a lot of > searching via DBI. it's a lot harder using a filesystem store. Yes-- if you are doing textual searches *directly* against the filesystem, filesystems are a lose. But a database is not a whole lot better unless you blow the relatively major [compared to most Mailman deployment budgets] $$$ on something like Sybase's or Oracle's free text searching solutions. Regardless of where the data is stored, searches would typically have to be backed by some kind of a indexing store-- be it in a dabase, a btree type solution [Faircom's C-Tree comes to mind], or any of the myriad of "search an index" solutions available. Building that index from a database or from a filesystem isn't really that much different in terms of difficulty though a number of folks would find the problem of walking a filesystem more approachable than walking a database. > >This is *not* to say that the DBI approach isn't the right way to go; if a > >generic DBI->filesystem, DBI->WebDAV, DBI->DB capable API were put > >together and > >was relatively hidden from the user and casual developer, it might > >be a huge win. > > which is exactly what I'm arguing. And DBI is a well-known interface > that ias easily accessible to anyone who wants to write an interface > -- if the architecture is done right. And it's portable off of Unix, > which is an issue for Mailman. I think we agree on the architectural direction of Mailman, but disagree on the relative merits of different backing stores. As such, the above discussion is very interesting, but largely moot. Internally, it makes a lot of sense to use DBI as the abstraction layer between Mailman and the thing that persists BLOBs, messages, and meta information. The underlying implementation is mostly irelevant. With that said, I do feel strongly that an abstraction layer does little good without concrete implementations of the adaptors underneath. As such, I volunteer myself to write the adaptor to WebDAV and I'll tentatively volunteer Chuq to write the adaptor that speaks to a database backing store. Our fierce sense of technical pride and intellectual competition will guarantee that Mailman version X.X will ship with first class ipmlementations of both adaptors. :-) > > > > Truly. And if we can support BLOBs in DBI, well, we don't have to > >> write anything to disk and can generate an entire message out of a > >> DBI database -- portable to any decent database. > > > >But an order of magnitude less effecient than downloading the BLOB > >off of disk via > >a webserver! > > Not necessarily. And is the efficiency important? A lot of time is > wasted in computer development optimizing the wrong things.... Agreed. Never optimize until you know where performance is going south... > >I feel strongly that abstraction is key, but that we should also > >provide decent, > >production quality, implementations of solutions to the very same > >set of problems > >for which we build the gneric abstracted/modularized APIs. > > agreed. > > >If Mailman is not fully functional "out of the box", then people > >will ignore it. > >However, if it isn't also flexible enough to be integrated into their weird > >environments (because every server on the web has weirdness), > >they'll bitch and > >moan until they find something else that doesn't solve their problem to B&M > >about.... > > it has to be functional out of the box, but we have to make sure we > define "functional" properly, too. You can't be missing features but > doing the "kitchen sink" think just in case is wrong, too. Agreed. b.bum From bbum@codefab.com Wed Nov 22 07:26:26 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Wed, 22 Nov 2000 02:26:26 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> Message-ID: <3A1B7523.1C3F9AFA@codefab.com> Yum. This would have saved us weeksandweeksandweeks of development on the system we built that had Mailman as a critical component. Anything that is *using* Mailman as just-another-moving-part-among-many faces the memebership management problem and it is nasty,nasty,nasty. b.bum Chuq Von Rospach wrote: > At 10:53 PM -0800 11/21/00, J C Lawrence wrote: > > > Authentication is a big bugaboo in general, which Barry and I have > >> hashed around a bit. More on that someday, maybe. > > > >FWLIW I see authentication visavis Mailman as a two level problem: > > > > list activities (command confirmations) > > access control > > Okay, I hope Barry doesn't mind, but since the talk has started, > here's something I sent him a while back that we've sat on to get 2.0 > out the door. But it's a first cut at my idea where some of this > ought to go... When you see this, you'll start to really understand > where I'm coming from in this whole API/module thing -- everything in > the system is a module of some sort, that communicates via an API -- > which means that in theory, aynone can replace any piece of > functionality by simply replacing the module. It also means that each > module can be tracked and developed separately, since all it needs to > worry about is conforming to the published API. > > The trick is going to be architecting the modules, interfaces and > APIs, and deciding one which ones are central to mailman, which ones > peripheral and which ones are optional. But when we're done, it > doesn't matter if you use DB or a filesystem database model, as long > as the glue inteface exists. And it doesn't matter if you use the > current mailman subscription interface or download your data from a > corporate database running Oracle and throw out the entire > subscription system -- because Mailman no longer cares. So the whole > archiving issue is a subset of a larger issue, which is defining and > standardizing all of Mailman and breaking it up into pieces that meld > into a cohesive whole. > > --- > Date: Wed, 11 Oct 2000 23:30:28 -0700 > Subject: Re: Planning Mailman 2.1 > .... system description deleted .... From bbum@codefab.com Wed Nov 22 07:30:30 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Wed, 22 Nov 2000 02:30:30 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> Message-ID: <3A1B7617.2836857C@codefab.com> Agreed. I would much rather see Mailman have an option to configure a list such that it is text-only-bounce-mime-straight-to-/dev/null for those situations that warrant it (and to remain New Communist Community Compliant). Similarly, I would also like to see a per-user option that would allow a mailing list subscriber to say "try and filter anything to ASCII and never, ever send me an attachment". b.bum Chuq Von Rospach wrote: > At 10:58 PM -0800 11/21/00, J C Lawrence wrote: > > >I've spent considerable time thinking about this and believe it is > >to the lists' advantage in this case to hold firm. > > you'll lose. AOL is the first to make this mandatory for their users, > but users want these features. And besides, AOL is the 800 pound > gorilla, and it's eating its lunch on your fence. Do you really think > they're going to change back before your fence collapses under it? > ..... war is over, etc [it's in ther archive... read it, it is good] .... From claw@kanga.nu Wed Nov 22 07:36:19 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 23:36:19 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Bill Bumgarner of "Wed, 22 Nov 2000 02:30:30 EST." <3A1B7617.2836857C@codefab.com> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> <3A1B7617.2836857C@codefab.com> Message-ID: <20380.974878579@kanga.nu> On Wed, 22 Nov 2000 02:30:30 -0500 Bill Bumgarner wrote: > Agreed. I would much rather see Mailman have an option to > configure a list such that it is > text-only-bounce-mime-straight-to-/dev/null for those situations > that warrant it (and to remain New Communist Community Compliant). Mailman needs the ability to define basic filters/rules for what is acceptable for being considered as a list posting. Sometimes that's MIME. Sometimes it is the source domain. Sometimes its an LDAP tag relted to a header field, or something else entirely. > Similarly, I would also like to see a per-user option that would > allow a mailing list subscriber to say "try and filter anything to > ASCII and never, ever send me an attachment". While pursueing the LServe game we need to not lose sight of the little guys running a couple lists for their club of 20-some model train enthusiasts. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 07:54:04 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 23:54:04 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <3A1B7471.5F16AAFA@codefab.com> References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> <3A1B7471.5F16AAFA@codefab.com> Message-ID: At 2:23 AM -0500 11/22/00, Bill Bumgarner wrote: >Yes-- all as true with a database system... but require the addition >of something >other than just a vanilla HTTP server to implement the "get the data" part. So does running mailman. So does running a search engine on the archives. We've already committed to adding a bunch of stuff to vanilla HTTP server (heck, what started this whole discussion, WebDAV,is adding something ot a vanilla HTTP server This is a strawman -- we can't do anything close to what we need with a vanilla HTTP server, so putting down one thing because it doesn't meet that goal is wrong. NOTHING useful meets that goal) >Everything on the list above save for the "with little addition" >item can be done >w/an out of the box apache. um, arguable. > > Obviously, this does not include the administrative >part-- the piece that does per list/per-site configuration requires some >additional work regardless of a database or filesystem (or WebDAV) >backing store. oh -- everyting, well, except, um... (giggle) >Another advantage to a filesystem based archival arrangement is that it is >*extremely* easy to write a random shell script or two that prunes data from >crontab, rebuilds indices, moves stuff around, reformats things, >archives off to >archived archives, etc... have you ever worked with NNTP? Because you're reinventing something the NNTP people have spent years designing out of their systems, because it has horrible scaling capabilities and is horrible resource inefficient. Some of us were arguing that this was a bad design model over a decade ago, and it's been proven by NNTP very nicely. > Yes-- of course-- all of these operations can be done >with a database backing store, as well, but it is significantly more >difficult to >develop such tools and install them into the system. It is? have you done it? I have... (www.apple.com/signmeup). It's not as bad as you think. > Likely, this is somewhat of >a perception issue, but most administrators will not hesitate to >toss together a >shell script to manage an archive of stuff, but will think twice >before diddling a >database. yes, that's a eprception issue, and it's also a strawman. I don't buy that for a second. Numbers, please. you can't throw a strawman like this out without backing data. >Very large and very expensive databases scale very well-- MySQL does not. sure it does. But even more important -- if I outgrow MySQL and need to throw in a really big muther database like Oracle, I can fairly easily. If your filesystem store system is outgrown and you need to add capacity to scale it, how do you do it? you rearchitect from scratch, probably going to a database-centric design. > I >agree that for truly huge, high traffic sites, moving away from a >pure filesystem >approach-- moving everything into, say, an ES10000 running one of >the magawhompus >Oracle/Sybase license-- would be the way to go. Oh, please. I'm running big megawhompus stuff on E250's and E450s (for my news servers), and MySQL no problemo. My big muther lsit server is MySQL on an E250, handles 28,000 database updates a day on average, sends out a few million emails a week, and spends most of its time idle (the only huge CPU sink I have is bounce processing, but then it take stime to process 300 megabyte bounce files) > But I don't think that is what >90% of the Mailman users are going to be using the system for and >requiring-- or >even encouraging the use of-- a database as a backing store for >messages will not >add value to those people. nor is that what I'm proposing. I don't know why you're so database averse, to be honest. but I think it's a personal aversion on your part, not a legitimate teechincal issue. >Considering most of the usage of an archive of messages.... > > - write operations are infrequent, modifications pretty much non existant > > - retrieval tends to be extremely sporadic and is generally *not* evenly >divided across the archive-- a relative few messages receive most of the hits > > - there are extremely limited ways of viewing the data; by author, date, >thread, subject.... with MOST views focused on thread. > > - indices are periodically updated > >... I still believe that a webserver-reading-files-from-filesystem >is going to be >loads more effecient than a >webserver-reading-data-from-client-server-connection-to-app-adaptor-reading-data-from-client-server-connection-to-multiuser-database. Sorry, but my reseach doesn't agree. And I'm not sure I agree with your idea of what goes on in archives, but I don't have numbers to back myself up on that. It also ignores ancillary advantages of databasing this stuff -- like the easy addition of content searching and the ability to write really good, customized search capabilities. In your way, you have to build technology (or borrow it, like HtDig) to get that, so anything you might possibly save resource or development wise in your model gets eaten by trying to do searching right -- and one thing I *have* found from my users is that archives without good search tools are pretty useless to them. So I consider archive/search a single key integrated module, even if the technologies are seeparate, and databasing stuff allows me to build a lot of search power into the system, where your setup doesn't -- you have to go and do it the hard way (and I've done that, and it sucks...) > >Yes-- if you are doing textual searches *directly* against the filesystem, >filesystems are a lose. But a database is not a whole lot better >unless you blow >the relatively major Not true. >Regardless of where the data is stored, searches would typically >have to be backed >by some kind of a indexing store-- be it in a dabase, a btree type solution but one of the nice things about databases is they're written to build indexes for you -- and the guys who write those indexing routines are experts. so you leverage their strengths. >solutions available. Building that index from a database or from a >filesystem >isn't really that much different in terms of difficulty though a >number of folks >would find the problem of walking a filesystem more approachable >than walking a >database. I've done both. I disagree. filesystem-centric systems are system intensive resource hogs that are fine for small to medium installations but scale poorly, and which require re-architecting when you outgrow them. Databse centric ones might be a little more work up front, might be somewhat overkill for tiny sites, but even for small ones, tend to break even, and scale upward basically infinitely because you can swap in bigger horses based on your budget -- but you don't need those horses. You can do really good stuff with open source tools. All you need is some good database design. not even great database design. >With that said, I do feel strongly that an abstraction layer does little good >without concrete implementations of the adaptors underneath. As such, I >volunteer myself to write the adaptor to WebDAV and I'll tentatively volunteer >Chuq to write the adaptor that speaks to a database backing store. Actually, chuq's planning on architecting large parts of mailman 3.0, assuming Barry gives the Okay. but I volunteered for that weeks ago, and have been whacking on concepts on and off since. And since a lot of what I hope to do in mailman wll be leveraged off work I've already done or am planning to do for Other Things I Can't Admit To, a lot of it is beyond "I think this will work, maybe" in thought... Since 1994, I've architected and implemented about half a dozen production e-mail systems, from really tiny things based on common tools (first was Listproc, then majordomo, now Mailman, for the three generations of my 'generic' servers), to really bastard big things to corporate email systems... And in the next six months, I'm rewriting my big muther almost from scratch to take it to the 25,000,000 subscriber capability and double delivery speed (again), and then we're probably redoing the internal corporate (which supports ~15,000 lists) to handle list lookup and delivery/authentication on demand via LDAP to the corporate databases (right now, we snapshot/download/munge the data...). So a lot of where I think Mailman ought to go is taking pieces of some of these boxes (with my bosses kind permission) and making it part of Mailman... And trust me, it'll be a long time before even my biggest email system needs Oracle or an ES10000. you can do wonders with a stack of Ultra 5s slaved to a decent sized box like an E250 (in fact, in many cases, taht's a lot better). I apologize if this sounds like I'm pulling rank, but -- you keep saying that things can't be done that I know are wrong, because I already have in some form or another, or that I've already done the design (and/or prototype for and know how it'll work. chuq -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 07:59:47 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 23:59:47 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <3A1B7523.1C3F9AFA@codefab.com> References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> <3A1B7523.1C3F9AFA@codefab.com> Message-ID: At 2:26 AM -0500 11/22/00, Bill Bumgarner wrote: >This would have saved us weeksandweeksandweeks of development on the >system we built that had Mailman as a critical component. > >Anything that is *using* Mailman as just-another-moving-part-among-many >faces the memebership management problem and it is nasty,nasty,nasty. If we end up doing the authentication widget, I want it to ship with a set of modules and/or interfaces. That would include one that does personal home pages, surveys, an interface to mailman, an interface to a web forum, an LDAP interface of some sort for external directory lookup and authentication purposes, and a few other toys. With it and a few key tools, you can easily do 90% of Yahoo, minus the big muther sets of data, but when it's done, you're very close to a yahoo portal with clubs, IRC, egroups and some other tools all rolled into a single cohesive beast -- and it'll scale, and it will be extensible through a standard interface. Assuming, of course, what I want to do works and I don't botch the architecture... (and what I find scary is I don't see *anything* that is technically a huge stretch. it's all engineering, not magic... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 08:04:55 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 22 Nov 2000 00:04:55 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <18729.974877726@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> <18729.974877726@kanga.nu> Message-ID: At 11:22 PM -0800 11/21/00, J C Lawrence wrote: >While I agree (with some distaste), those gyrations have shown >themselves to have positive value for me. short term. I had the same general attitude for a while, and did pretty much teh same thing, and found that over time, it basically killed my site through isolation and stagnation. I served a given population very well, but found that all I did was drive off fresh blood, and the existing population stopped growing, stagnated and started shrinking as old members moved on and weren't replaced. Which, if it sounds like USENET today, well, oyu're right. USENET is doing the same thing, but on a hugely massive scale. I'm still working to recover what I feel is an essential vitality in my lists and get back into growth mode. But we're so far off anything relevant to Mailman issues it's insane, so I won't push this one further on these lists, but audience management is tricky, and even things you think are working my bite you in the butt when you least expect it. Once you set the expectation of exclusionism on your site, it's a real bitch to change back.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 08:11:48 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 22 Nov 2000 00:11:48 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <20380.974878579@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> <3A1B7617.2836857C@codefab.com> <20380.974878579@kanga.nu> Message-ID: At 11:36 PM -0800 11/21/00, J C Lawrence wrote: >While pursueing the LServe game we need to not lose sight of the >little guys running a couple lists for their club of 20-some model >train enthusiasts. If we even want to be in teh LServe game -- but this nis one of the neat things of the pure plug-in universe. if someone wants a big muther listserve, plug in the fancy module. If they want simple subscription systems, plug in the easy subscriber setup. you have to be very careful not to try to build A THING that is everythign for everyone, but you sure can build a framework that contains the capability to build things that allow you to build a system catered to each system's needs. And maybe we don't ship all of those variations initially (or ever!), but if someone wants it, we've at elast built a system where they can write it for themselves (and us). Lots of sites won't want my muther subscriber system from hell module, but there's no reason at all we can't take mailman's existing system and turn it into a "simple" subscriber system. Or write one that 100% mimics majordomo doing e-mail only. Adn in fact, for something like this, having two or three variations to test the API would be great. And it'd help us prove the architecture and keep us honest, so the different modules don't cheat by looking over each other's shoulders... If folks want to see a successful (so far) project taht's doing this, I think you need look no further than Gnome, and its underlying X windows basis. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From thomas@xs4all.net Wed Nov 22 09:58:33 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 22 Nov 2000 10:58:33 +0100 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: ; from ken@kyler.com on Tue, Nov 21, 2000 at 06:55:57PM -0500 References: <14875.1663.422580.420642@anthem.concentric.net> Message-ID: <20001122105832.F540@xs4all.nl> On Tue, Nov 21, 2000 at 06:55:57PM -0500, Ken Kyler wrote: > Keep up the good work guys! If I ever finish grad school I'll help. Shht, don't tell Barry, but I never finished grad school (or any form of highschool, for that matter) and he still put some of my fixes into Mailman, and adapted others ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From barry@digicool.com Wed Nov 22 14:17:44 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 22 Nov 2000 09:17:44 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> <3A1B7617.2836857C@codefab.com> <20380.974878579@kanga.nu> Message-ID: <14875.54664.457063.83250@anthem.concentric.net> Jeez, try to get one good night of sleep and look what happens! :) From bkuhn@gnu.org Wed Nov 22 16:27:02 2000 From: bkuhn@gnu.org (Bradley M. Kuhn) Date: Wed, 22 Nov 2000 11:27:02 -0500 Subject: [Mailman-Developers] Errors in mailman that I cannot figure out Message-ID: <20001122112702.F2833@ebb.org> --FeAIMMcddNRN4P4/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I am writing on behalf of the GNU project. We have an discovered some sort of bug in mailman. The .../logs/errors file is getting: Nov 22 11:24:01 2000 qrunner(29262): Traceback (innermost last): Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 278, in ? Nov 22 11:24:01 2000 qrunner(29262): kids = main(lock) Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 247, in main Nov 22 11:24:01 2000 qrunner(29262): keepqueued = dispose_message(mlist, msg, msgdata) Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 121, in dispose_message Nov 22 11:24:01 2000 qrunner(29262): if BouncerAPI.ScanMessages(mlist, mimemsg): Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/BouncerAPI.py", line 59, in ScanMessages Nov 22 11:24:01 2000 qrunner(29262): addrs = func(msg) Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/Catchall.py", line 132, in process Nov 22 11:24:01 2000 qrunner(29262): username = string.split(line)[1] Nov 22 11:24:01 2000 qrunner(29262): IndexError : list index out of range This was happening with 2.0beta6, so I upgraded to see if it's a bug fixed by 2.0, but it isn't. I think this may be related to a particular file in the queue being formatted oddly, but the result is the entire queue is held up. I am looking a bit deeper into the code right now, but I'd appreciate help. (Mailing lists at gnu.org are currently down becasue of this bug). I will send more information as I discover it. --FeAIMMcddNRN4P4/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6G/PW53XjJNtBs4cRAqiFAKCsUa2uMjiXeW8omDeQLXz49JBsOQCgu2Tv qrpYvXRrvBwhYoxcN3OrxB8= =aCnS -----END PGP SIGNATURE----- --FeAIMMcddNRN4P4/-- From barry@digicool.com Wed Nov 22 17:33:16 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 22 Nov 2000 12:33:16 -0500 Subject: [Mailman-Developers] [ANNOUNCE] Mailman 2.0 Message-ID: <14876.860.607056.18676@anthem.concentric.net> I'm please to announce the 2.0 final release of Mailman, the GNU Mailing List Manager. Mailman is released under the GNU General Public License (GPL). Mailman is software to help manage electronic mail discussion lists, much like Majordomo or Smartmail. Mailman gives each mailing list a unique web page and allows users to subscribe, unsubscribe, and change their account options over the web. Even the list manager can administer his or her list entirely via the web. Mailman has most of the features that people want in a mailing list management system, including built-in archiving, mail-to-news gateways, spam filters, bounce detection, digest delivery, and so on. Mailman is compatible with most web servers, web browsers, and mail servers. It should run on any Unix-like operating system. Mailman requires Python 1.5.2 or better. To install Mailman from source, you will need a C compiler. For more information on Mailman, please see the following mirrors: http://www.list.org http://mailman.sourceforge.net http://www.gnu.org/software/mailman/mailman.html (the latter is not yet up to date) There are email lists (managed by Mailman, of course!) for both Mailman users and developers. See the web sites above for details. My thanks to everybody who has helped with this release, especially those named in the ACKNOWLEDGMENTS file. My thanks also to the SourceForge crew for their open source project management site. Couldn't have done it without them. Enjoy, -Barry From bkuhn@gnu.org Wed Nov 22 17:46:20 2000 From: bkuhn@gnu.org (Bradley M. Kuhn) Date: Wed, 22 Nov 2000 12:46:20 -0500 Subject: [Mailman-Developers] Fixed problem at gnu.org, patch included (was Re: Errors in mailman that I cannot figure out) In-Reply-To: <20001122112702.F2833@ebb.org>; from bkuhn@gnu.org on Wed, Nov 22, 2000 at 11:27:02AM -0500 References: <20001122112702.F2833@ebb.org> Message-ID: <20001122124619.K2833@ebb.org> --8tUgZ4IE8L4vmMyh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Bradley M. Kuhn wrote: > I am writing on behalf of the GNU project. We have an discovered some sort > of bug in mailman. The .../logs/errors file is getting: > > Nov 22 11:24:01 2000 qrunner(29262): Traceback (innermost last): > Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 278, in ? > Nov 22 11:24:01 2000 qrunner(29262): kids = main(lock) > Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 247, in main > Nov 22 11:24:01 2000 qrunner(29262): keepqueued = dispose_message(mlist, msg, msgdata) > Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 121, in dispose_message > Nov 22 11:24:01 2000 qrunner(29262): if BouncerAPI.ScanMessages(mlist, mimemsg): > Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/BouncerAPI.py", line 59, in ScanMessages > Nov 22 11:24:01 2000 qrunner(29262): addrs = func(msg) > Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/Catchall.py", line 132, in process > Nov 22 11:24:01 2000 qrunner(29262): username = string.split(line)[1] > Nov 22 11:24:01 2000 qrunner(29262): IndexError : list index out of range I have fixed this bug. It had to do with regexes that weren't precise in Catchall.py. Here's a patch. I also made a ChangeLog, because you all don't appear to have one yet! ############################################################################### # Fixed bug with Catchall regexs # # To apply this patch: # STEP 1: Chdir to the source directory. # STEP 2: Run the 'applypatch' program with this patch file as input. # # If you do not have 'applypatch', it is part of the 'makepatch' package # that you can fetch from the Comprehensive Perl Archive Network: # http://www.perl.com/CPAN/authors/Johan_Vromans/makepatch-x.y.tar.gz # In the above URL, 'x' should be 2 or higher. # # To apply this patch without the use of 'applypatch': # STEP 1: Chdir to the source directory. # If you have a decent Bourne-type shell: # STEP 2: Run the shell with this file as input. # If you don't have such a shell, you may need to manually create # the files as shown below. # STEP 3: Run the 'patch' program with this file as input. # # These are the commands needed to create/delete files/directories: # touch 'ChangeLog' chmod 0600 'ChangeLog' # # This command terminates the shell and need not be executed manually. exit # #### End of Preamble #### #### Patch data follows #### diff -u /dev/null 'mailman/ChangeLog' Index: ./ChangeLog *** ./ChangeLog Wed Dec 31 19:00:00 1969 --- ./ChangeLog Wed Nov 22 12:01:37 2000 *************** *** 0 **** --- 1,4 ---- + 2000-11-22 Bradley M. Kuhn + + * Mailman/Bouncers/Catchall.py (process): changed some of the + "messy" regular expressions so errors don't occur diff -u 'mailman-pristine/Mailman/Bouncers/Catchall.py' 'mailman/Mailman/Bouncers/Catchall.py' Index: ./Mailman/Bouncers/Catchall.py --- ./Mailman/Bouncers/Catchall.py Sun Aug 6 22:34:33 2000 +++ ./Mailman/Bouncers/Catchall.py Wed Nov 22 12:02:52 2000 @@ -102,9 +102,12 @@ (regex.compile('.*%s: User unknown.*' % email_regexp), REMOVE), (regex.compile('.*%s\.\.\. User unknown' % email_regexp), REMOVE)) # patterns we can't directly extract the email (special case these) - messy_pattern_1 = regex.compile('^Recipient .*$') - messy_pattern_2 = regex.compile('^Addressee: .*$') - messy_pattern_3 = regex.compile('^User .* not listed.*$') + # Make sure that these patterns will not cause the special cases below + # to fail. In other words, be sure that these regexs ensure that split's + # and other operations done below will always work. + messy_pattern_1 = regex.compile('^Recipient[ \t]+[^ \t]+[ \t]*$') + messy_pattern_2 = regex.compile('^Addressee:[ \t]*[^ \t]+[ \t]*$') + messy_pattern_3 = regex.compile('^User [^ \t]+ not listed.*$') messy_pattern_4 = regex.compile('^550 [^ ]+\.\.\. User unknown.*$') messy_pattern_5 = regex.compile('^User [^ ]+ is not defined.*$') messy_pattern_6 = regex.compile('^[ \t]*[^ ]+: User unknown.*$') #### End of Patch data #### #### ApplyPatch data follows #### # Data version : 1.0 # Date generated : Wed Nov 22 12:25:09 2000 # Generated by : makepatch 2.00 # Recurse directories : Yes # Excluded files : (\A|.*/)CVS(/.*|\Z) # (\A|.*/)RCS(/.*|\Z) # ,v\Z # (\A|.*/)SCCS(/.*|\Z) # (\A|.*/)[sp]\..+\Z # c 'ChangeLog' 0 974912497 0100600 # p 'Mailman/Bouncers/Catchall.py' 7873 974912572 0100600 #### End of ApplyPatch data #### #### End of Patch kit [created: Wed Nov 22 12:25:09 2000] #### #### Checksum: 78 3275 60184 #### --8tUgZ4IE8L4vmMyh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6HAZr53XjJNtBs4cRAmgTAJ4zQeFCa3xpnFK31auM+IMRkymMIgCdF5Hj aBqPjNlRvGvBnb3Dwmz6thk= =WgRu -----END PGP SIGNATURE----- --8tUgZ4IE8L4vmMyh-- From claw@kanga.nu Wed Nov 22 20:11:40 2000 From: claw@kanga.nu (J C Lawrence) Date: Wed, 22 Nov 2000 12:11:40 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 23:59:47 PST." References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> <3A1B7523.1C3F9AFA@codefab.com> Message-ID: <25256.974923900@kanga.nu> On Tue, 21 Nov 2000 23:59:47 -0800 Chuq Von Rospach wrote: > At 2:26 AM -0500 11/22/00, Bill Bumgarner wrote: >> Anything that is *using* Mailman as >> just-another-moving-part-among-many faces the memebership >> management problem and it is nasty,nasty,nasty. > If we end up doing the authentication widget, I want it to ship > with a set of modules and/or interfaces. That would include one > that does personal home pages, surveys, an interface to mailman, > an interface to a web forum, an LDAP interface of some sort for > external directory lookup and authentication purposes, and a few > other toys. I have slight allergic reactions here. Yes, I agree that a generic modular authentication model would be extremely useful (and I'd kill for a sufficiently flexible one), and it would be good if Mailman were able to plug into such a thing were it to be available on the local system, I don't see this as a Mailman specific problem, or one that should really be considered part of or rolled into the Mailman design process outside of "we should be able to integrate with one of those if its available, otherwise we punt to our LCD implementation". > With it and a few key tools, you can easily do 90% of Yahoo, minus > the big muther sets of data, but when it's done, you're very close > to a yahoo portal with clubs, IRC, egroups and some other tools > all rolled into a single cohesive beast -- and it'll scale, and it > will be extensible through a standard interface. Which, while a nice problem, is not the problem I belive Mailman is trying to solve. Mailman should of course play nicely with such solutions (yes, plural), but I see considerable loss to be realised if we tie it to a specific implementation or model (eg Zope), and abstracting at just the API level (as versus at the process level) is going to tend to lead you in that direction. > (and what I find scary is I don't see *anything* that is > technically a huge stretch. it's all engineering, not magic... Too true (written as one who has just re-engineered his own auth models). There's no rocket science here, just the normal stack of security model and privacy concerns. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 20:29:08 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 22 Nov 2000 12:29:08 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <25256.974923900@kanga.nu> References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> <3A1B7523.1C3F9AFA@codefab.com> <25256.974923900@kanga.nu> Message-ID: At 12:11 PM -0800 11/22/00, J C Lawrence wrote: >I have slight allergic reactions here. Yes, I agree that a generic >modular authentication model would be extremely useful (and I'd kill >for a sufficiently flexible one), and it would be good if Mailman >were able to plug into such a thing were it to be available on the >local system, I don't see this as a Mailman specific problem, or one >that should really be considered part of or rolled into the Mailman >design process outside of "we should be able to integrate with one >of those if its available, otherwise we punt to our LCD >implementation". that's why I propose this as a separate project -- and why the first thing we'd need to do is define the API, then port the existing subscription system to the API. The end result would be that if you want to use the existing system, you would, but the hooks would be there to move to a more extensive system. I don't for a minute think we want that more extensive system to be the default for mailman, not for any significant amount of time, if ever (any more than I think mailman ought to require Zope, ought to require external templating modules, ought to require...) So from the point of view of Mailman, the work would be to define and architect the API and get the existing system working with it -- and then work on an external project that'd interface to that API, not replace the existing system. >Which, while a nice problem, is not the problem I belive Mailman is >trying to solve. yup. But it's part of this larger convergence and integration problem. The integration tools need to be created, but I'm not proposing they be mandatory. >Too true (written as one who has just re-engineered his own auth >models). There's no rocket science here, just the normal stack of >security model and privacy concerns. ah, who cares about security and privacy, anyway? But to be honest -- if we can build a single, GOOD authentication widget and get those issues right, doesn't that make it nicer for everyone else, since they can simply write to use the widge and not reinvent the wheel endlessly? -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From darrell@grumblesmurf.net Wed Nov 22 23:04:26 2000 From: darrell@grumblesmurf.net (Darrell Fuhriman) Date: 22 Nov 2000 15:04:26 -0800 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: barry@digicool.com's message of "Wed, 22 Nov 2000 00:02:26 -0500" References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > >From the TODO list, things I can imagine tackling for 2.1 include: I would like to see a couple things added. 1) the patch I wrote to sort the list and break it up in chunks when passed to the mailer -- FWIW, I've been using it since I wrote it without problems, surely others have, too. Like I said earlier, at worst, it's an innocuous feature, but in general it does benefit delivery performance. 2) In pipermail, I'd like to be able to define HTML tags that can be wrapped around non-important parts of the particular message. Specifically, this is to enable you to exclude certain parts of the message from indexing, such as navigation information. (I'm thinking specifically of the tags. That is, of course, a rather obscure feature. :) Darrell From ckolar@admin.aurora.edu Wed Nov 22 23:52:53 2000 From: ckolar@admin.aurora.edu (Christopher Kolar) Date: Wed, 22 Nov 2000 17:52:53 -0600 Subject: [Mailman-Developers] v2 Management Documents, GFDL Message-ID: <5.0.1.4.2.20001122175114.0205df30@admin.aurora.edu> I am pleased to announce that v2 of the Mailman list manager's documentation is available at www.aurora.edu/~ckolar/mailman. Aurora University has generously agreed to release these documents under the GNU Free Documentation License (GFDL), details are on the site. --chris From Dan Mick Thu Nov 23 02:46:56 2000 From: Dan Mick (Dan Mick) Date: Wed, 22 Nov 2000 18:46:56 -0800 (PST) Subject: [Mailman-Developers] Towards 2.1 Message-ID: <200011230245.SAA09389@utopia.west.sun.com> > barry@digicool.com (Barry A. Warsaw) writes: > > > >From the TODO list, things I can imagine tackling for 2.1 include: > > I would like to see a couple things added. > > 1) the patch I wrote to sort the list and break it up in chunks > when passed to the mailer -- FWIW, I've been using it since I > wrote it without problems, surely others have, too. Like I said > earlier, at worst, it's an innocuous feature, but in general it > does benefit delivery performance. I've been using it, and I think I love it. I think it ought to be optionally added, too. A pleasant side-effect of the domain sorting is when Hotmail gets screwed up (once or twice a week), all the hotmail mail bounces at the same time. :-} From Nigel.Metheringham@VData.co.uk Thu Nov 23 09:45:12 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Thu, 23 Nov 2000 09:45:12 +0000 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: Message from Darrell Fuhriman of "22 Nov 2000 15:04:26 PST." Message-ID: darrell@grumblesmurf.net said: > 2) In pipermail, I'd like to be able to define HTML tags that can be > wrapped around non-important parts of the particular message. > Specifically, this is to enable you to exclude certain parts of the > message from indexing, such as navigation information. (I'm thinking > specifically of the tags. > That is, of course, a rather obscure feature. :) There is already a patch to do this - its marked as being for htdig, but all it actually does is inject definable indexer on/off statements into the generated HTML. It does assume that your indexer parses are utilises the indexer tags that already appear in pipermail HTML. See https://sourceforge.net/patch/?func=detailpatch&patch_id=102422&group_ id=103 or if that doesn't work, Patch 102422 Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From carsten.dot.geckeler@gmx.dot.de Thu Nov 23 15:38:26 2000 From: carsten.dot.geckeler@gmx.dot.de (Carsten Geckeler) Date: Thu, 23 Nov 2000 16:38:26 +0100 (MET) Subject: [Mailman-Developers] Re: [ANNOUNCE] Mailman 2.0 In-Reply-To: <14876.860.607056.18676@anthem.concentric.net> Message-ID: This is bad. -- Carsten Geckeler: carsten dot geckeler at gmx dot de From carsten.dot.geckeler@gmx.dot.de Thu Nov 23 21:07:51 2000 From: carsten.dot.geckeler@gmx.dot.de (Carsten Geckeler) Date: Thu, 23 Nov 2000 22:07:51 +0100 (MET) Subject: [Mailman-Developers] Re: [ANNOUNCE] Mailman 2.0 In-Reply-To: Message-ID: Sorry, sorry, sorry! My reply was just accidentially send. I was testing my mail filter for masking the From: address. It just slipped accidentially into the send queue. So my mail was not an expression of my opinion about Mailman. Will not happen again. Cheers, Carsten -- Carsten Geckeler: carsten dot geckeler at gmx dot de From lkarrer@trash.net Thu Nov 23 22:29:10 2000 From: lkarrer@trash.net (Lukas Karrer) Date: Thu, 23 Nov 2000 23:29:10 +0100 (MET) Subject: [Mailman-Developers] Re: [Mailman-Announce] [ANNOUNCE] Mailman 2.0 In-Reply-To: <14876.860.607056.18676@anthem.concentric.net> Message-ID: On Wed, 22 Nov 2000, Barry A. Warsaw wrote: Hi Mailman developpers.. I just seamlessly migrated a couple of lists to version 2... JUST PERFECT how everything is well documented and works as it should!! THANKS ALOT for your effort! Keep on writing good Software Lukas +---------------------------------------------------------------------+ Lukas Karrer Email: lkarrer@trash.net Sysadmin WWW: http://www.trash.net/ +---------------------------------------------------------------------+ Check out the stinkiest site on the web! Get a sniff from www.trash.net t r a s h . n e t - free UNIX Shellaccounts for Switzerland From bbum@codefab.com Fri Nov 24 05:01:29 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Fri, 24 Nov 2000 00:01:29 -0500 (EST) Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message-ID: How about PAM-- pluggable authentication modules? It seems to have become relatively standard on LInux and BSD and provides a flexible and portable API that provides for versatile plug-n-play authentication across platforms. I recently used it with Apache and ProFTPD to easily configure both to use the same user/passwd database-- and neither to use the systems existing user/pass system. PAM modules exist for berkeley DB and MySQL, among various random standard authentication procedures such as Kerberos and /etc/passwd. It would not be hard to provide a module that makes the system accessible from Python and the applicability would range a lot further than Mailman [make it immediately attractive to others]. (It may be the case that a PAM python module already exists? Haven't looked...) Even if the implementation is not appropriate, the architecture is certainly interesting and will likely contain ideas that are pertinent to building such an authentication system as Chuq described. b.bum From jeff@ollie.clive.ia.us Fri Nov 24 07:04:29 2000 From: jeff@ollie.clive.ia.us (Jeffrey C. Ollie) Date: Fri, 24 Nov 2000 01:04:29 -0600 Subject: [Mailman-Developers] Interface to Archivers Message-ID: <20001124010429.A14295@ollie.clive.ia.us> --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline It would seem to me that the discussion so far regarding the "future of pipermail" is going too far in the direction of discussing "what would the most keenest archiver look like" rather than discussing "what should the interface between the Mailman and an archiver look like" and "what should the default archiver look like". So to try and steer the discussion in this direction, here I go sticking my foot in my mouth: What Should the Interface Between Mailman and an Archiver Look Like: def submit(mlist, msg) archiver = __import__('Mailman.Archivers.' + mm_cfg.ARCHIVE_MODULE) submit = getattr(getattr(getattr(mod, 'Archivers'), mm_cfg.ARCHIVE_MODULE), 'submit') urls = submit(mlist, msg) return urls The 'urls' returned from the archive module will be a data structure that includes a URL that points to the archived message as a whole and optionally URLs that point to individual MIME-decoded individual parts. If the archiver includes URLs for individual parts Mailman could optionally reformat the message to remove attachments and replace them with a pointer to the archive. So then we write any number of modules that interface to popular archivers, plus we include a simple default archiver (IMHO, in the same tarball as the Mailman). This way, the details of the archiver are completely hidden from Mailman and adding support for new archivers is very simple. I suppose that one could even add support for different archivers for every list. What Should the Default Archiver Look Like: It'll look a lot like pipermail, in that it stores archived messages in files on a filesystem. Indexes and tables of contents will be rebuilt as needed and saved in static files. It'll look different from pipermail in that MIME message parts will be decoded and be accessible as individual files. The reason for going with files stored on a filesystem is that it's very simple. No database to deploy and no fancy web server is required. More complex schemes are of course possible by writing the appropriate plug-in. Jeff --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6HhL9mCSL+rEqeRsRAvoMAJ9ZmUkm3+XFtRfjfnibP2E9AIQh4QCgmeAo NNxpNdKGCU5dbeDMpSa+ca4= =DC7Y -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24-- From claw@kanga.nu Fri Nov 24 07:45:26 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 23 Nov 2000 23:45:26 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Bill Bumgarner of "Fri, 24 Nov 2000 00:01:29 EST." References: Message-ID: <24944.975051926@kanga.nu> On Fri, 24 Nov 2000 00:01:29 -0500 (EST) Bill Bumgarner wrote: > How about PAM-- pluggable authentication modules? It seems to > have become relatively standard on LInux and BSD and provides a > flexible and portable API that provides for versatile plug-n-play > authentication across platforms. Authentication data in the sense of identification (you know the UID and the password) is the simplest problem, and is probably so small as to be ignorable. The problem is not just UID/PW combos, but access rights, settings, configurations, meta data, etc. > Even if the implementation is not appropriate, the architecture is > certainly interesting and will likely contain ideas that are > pertinent to building such an authentication system as Chuq > described. True. As happens I'm more tempted by LDAP. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Fri Nov 24 07:54:23 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Thu, 23 Nov 2000 23:54:23 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <24944.975051926@kanga.nu> References: <24944.975051926@kanga.nu> Message-ID: At 11:45 PM -0800 11/23/00, J C Lawrence wrote: >True. As happens I'm more tempted by LDAP. I tend to think that the mailman <-> authentication interface is going to be gnarly enough that it'll be a custom API. But I also think that having PAM and LDAP access to the data will also important to have for access in other ways -- PAM solves my problem for archive access in apache wonderfully, and either PAM or LDAP would make it easier to evolve my web forums into using a sitewide account/password setup, which is something I really want down the road. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From claw@kanga.nu Fri Nov 24 16:36:18 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 24 Nov 2000 08:36:18 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Thu, 23 Nov 2000 23:54:23 PST." References: <24944.975051926@kanga.nu> Message-ID: <31572.975083778@kanga.nu> On Thu, 23 Nov 2000 23:54:23 -0800 Chuq Von Rospach wrote: > At 11:45 PM -0800 11/23/00, J C Lawrence wrote: >> True. As happens I'm more tempted by LDAP. > I tend to think that the mailman <-> authentication interface is > going to be gnarly enough that it'll be a custom API. But I also > think that having PAM and LDAP access to the data will also > important to have for access in other ways -- PAM solves my > problem for archive access in apache wonderfully, and either PAM > or LDAP would make it easier to evolve my web forums into using a > sitewide account/password setup, which is something I really want > down the road. I don't see them as exclusive: http://www.padl.com/pam_ldap.html ---- The pam_ldap module provides the means for Solaris and Linux workstations to authenticate against LDAP directories, and to change their passwords in the directory. ---- -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Fri Nov 24 18:31:29 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 24 Nov 2000 10:31:29 -0800 Subject: [Mailman-Developers] Editing of messages at the admin page Message-ID: <16058.975090689@kanga.nu> I'm toying with the idea of adding raw message editing capabilities to the web moderation interface. The interface design would be as follows: ADMINDB_PAGE_TEXT_LIMIT would accept a special value of "0" meaning "entire message". This would have the side effect of also displaying all headers. If the config is set to display entire messages and the moderator edits the message in the relevant TEXTAREA, then the following will happen: A custom header will be inserteted in the message headers: X-Moderator: This message was edited by the list moderator A similar comment will be prepended at the top of the message. (I'm still argueing with myself on this one as I'd like it but some wouldn't) The edited form of the message will be broadcast. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From Ricardo Kustner" References: <16058.975090689@kanga.nu> Message-ID: <200011242255.XAA08079@smtp9.xs4all.nl> On Fri, 24 Nov 2000 10:31:29 -0800, J C Lawrence said: > I'm toying with the idea of adding raw message editing capabilities > to the web moderation interface. The interface design would be as > follows: > ADMINDB_PAGE_TEXT_LIMIT would accept a special value of "0" > meaning "entire message". This would have the side effect of also > displaying all headers. > The edited form of the message will be broadcast. Although I like this idea, I personally think it would be more friendly to add an extra comment field that will be added to the top or bottom of the message... something like "[ Administrators note: blah blah blah ]"... If you edit the content of a persons message nobody will know what exactly you editted which can lead to questionable situations... Ricardo. -- http://rixhq.nu "You think that's air you're breathing?" -- Morpheus From claw@kanga.nu Sat Nov 25 00:26:57 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 24 Nov 2000 16:26:57 -0800 Subject: [Mailman-Developers] Editing of messages at the admin page In-Reply-To: Message from "Ricardo Kustner" of "24 Nov 2000 23:55:20." <200011242255.XAA08079@smtp9.xs4all.nl> References: <16058.975090689@kanga.nu> <200011242255.XAA08079@smtp9.xs4all.nl> Message-ID: <31526.975112017@kanga.nu> On 24 Nov 2000 23:55:20 CET Ricardo Kustner wrote: > Although I like this idea, I personally think it would be more > friendly to add an extra comment field that will be added to the > top or bottom of the message... something like "[ Administrators > note: blah blah blah ]"... If you edit the content of a persons > message nobody will know what exactly you editted which can lead > to questionable situations... While I have a minor history of editing user's submissions to the list, in each case I've prefaced the resultant message with something akin to: EdNote: The following lightly edited for eg: http://www.kanga.nu/archives/MUD-Dev-L/2000Q4/msg00205.php I like this practice for the reasons you state. It is for similar reasons that I preface messages posted via my archives with text indicating both the source and invalidity of the From: address: http://www.kanga.nu/archives/IRead-L/2000Q4/msg00000.php The problem with doing this automatically upon edits is handling MIME in an end-user friendly fashion. The intent of the added header etc is partially substitute for an editorial note, and to make up for those occassions where the moderator edits and then fogets to annotate his changes. I don't want to embed diffs of the changes (which would remove the value of the edit in the case of flammage or inappropriate content), but I feel the system should at least flag the fact of the edit in some automatic fashion. This all said, the possibility of abuse is always present. Mailman can't prevent it, and I don't believe should be in the busines of controlling content at that level (tehre are applications where its unwelcome and unecessary). -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From tismer@tismer.com Sat Nov 25 15:03:09 2000 From: tismer@tismer.com (Christian Tismer) Date: Sat, 25 Nov 2000 17:03:09 +0200 Subject: [Mailman-Developers] Minor flaw in archive date handling Message-ID: <3A1FD4AD.5270BBFC@tismer.com> Hi folks, I had some trouble getting the stackless mailing list repaired, after one message was repeated thousands of times. This was a message from October, but interestingly, the thousands of messages appeared in both the October and the November archives. Looking forther at one instance of the thousands of copies, I recognized that the message had no "Date:" entry, just a Resent-Date, for some unknown reason. After clearing the whole pipermail archive and recreating it with the bin/arch command, the message again appeared in the November archive. I took one of the fields with valid date enties and added it as "Date:" field by hand. Then the archiving worked as expected. My question: When a message happens to have no date, messages appear to be archived into the wrong archive file. There are a couple of other date fields available, like the usual "from" - line on top of the message, fields like Resent-Date or dates from the message passing through other servers. Wouldn't it make sense to do a more exhaustive search for a proper date, instead of accepting a missing date? cheers - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com From tismer@tismer.com Sat Nov 25 15:45:44 2000 From: tismer@tismer.com (Christian Tismer) Date: Sat, 25 Nov 2000 17:45:44 +0200 Subject: [Mailman-Developers] Minor flaw in archive date handling References: <3A1FD4AD.5270BBFC@tismer.com> Message-ID: <3A1FDEA8.CD987FB0@tismer.com> Christian Tismer wrote: I wrote: > My question: When a message happens to have no date, messages > appear to be archived into the wrong archive file. There > are a couple of other date fields available, like the usual > "from" - line on top of the message, fields like Resent-Date > or dates from the message passing through other servers. > > Wouldn't it make sense to do a more exhaustive search for > a proper date, instead of accepting a missing date? Changing the message from "one-should-do-such-and-such" style to "I-would-do-a-patch,-just-tell-me-how" style: I'm asking if such a change makes sense, and where to do the adjustment: In the process where the messages are collected in the .mbox file (Mailman change), or later, when the archives are generated (pipermail-change) ? regards - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com From tneff@bigfoot.com Sat Nov 25 17:10:04 2000 From: tneff@bigfoot.com (Tom Neff) Date: Sat, 25 Nov 2000 12:10:04 -0500 Subject: [Mailman-Developers] Re: editing messages In-Reply-To: <20001125170103.089071CD6A@dinsdale.python.org> Message-ID: <196628133.975154204@[192.168.0.2]> I patched 1.1 a year ago to allow editing a message before approval. I didn't bother with any pantywaist headers or disclaimers, although I suppose making one optionally available would be a joy-joy for the San Angeles Email Police. Mellow greetings and be well! *** admindb.py.orig Sun Jun 13 04:10:09 1999 --- admindb.py Mon Sep 18 14:27:48 2000 *************** *** 196,202 **** # We already handled this request. continue comment_key = 'comment-%d' % request_id ! if form.has_key(comment_key): list.HandleRequest(request, v, form[comment_key].value) else: list.HandleRequest(request, v) --- 196,207 ---- # We already handled this request. continue comment_key = 'comment-%d' % request_id ! contents_key = 'contents-%d' % request_id ! if form.has_key(contents_key) and form.has_key(comment_key): ! list.HandleRequest(request, v, form[comment_key].value, contents=form[contents_key].value) ! elif form.has_key(contents_key): ! list.HandleRequest(request, v, None, contents=form[contents_key].value) ! elif form.has_key(comment_key): list.HandleRequest(request, v, form[comment_key].value) else: list.HandleRequest(request, v) *************** *** 241,247 **** FontSize("+1", Bold('Contents:')) ]) form.AddItem(t) ! form.AddItem(Preformatted(val[2][1])) form.AddItem('

') --- 246,254 ---- FontSize("+1", Bold('Contents:')) ]) form.AddItem(t) ! # form.AddItem(Preformatted(val[2][1])) ! form.AddItem(TextArea("contents-%d" % val[0], rows=30, cols=85, ! text=val[2][1])) form.AddItem('

') From brian@gweep.bc.ca Sun Nov 26 14:20:05 2000 From: brian@gweep.bc.ca (Brian Edmonds) Date: 26 Nov 2000 06:20:05 -0800 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: barry@digicool.com's message of "Wed, 22 Nov 2000 00:02:26 -0500" References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > - Plain text digests should conform to RFC 1153. Yes please. Gnus supports 1153 digests, but doesn't recognize Mailman digests unless I switch to MIME format. > - Allow "urgent" postings to all members by the list admin which > bypasses normal digest delivery. I would like to be able to do this as more than just list admin. I've got at least three different lists on my system that have an announce, regular and digest options. The announce has separate subscribers and also goes immediately to all regular and digest subscribers. Currently those lists are still running under Majordomo, as I have no idea how to make them do the same thing with Mailman. And in the case of one of the lists it's not even an announce list, just a "regular" address that anyone can post to for topically urgent items. > - Allow the user to be excluded from postings if they're getting > them in the to: or cc: headers. Be smarter about filtering out > duplicate deliveries. This would be an interesting option, but I don't think I'd turn it on for most lists. > - Don't use the first public mailing list as the `originator' of > password reminders. Very much yes please. :) > - Provide an email interface to all administrative commands Ditto. > - For email subscribes, keep an audit of where requests are coming > from, and send the original request headers in the confirmation > message. Helps track down subscribe bombs. That sounds like a good idea. I've always liked mlms that do this. > - Support the `which' command. I had a user asking how to do this just yesterday. :) Brian. From chuqui@plaidworks.com Sun Nov 26 17:01:11 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Sun, 26 Nov 2000 09:01:11 -0800 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: At 6:20 AM -0800 11/26/00, Brian Edmonds wrote: >barry@digicool.com (Barry A. Warsaw) writes: >> - Plain text digests should conform to RFC 1153. > >Yes please. Gnus supports 1153 digests, but doesn't recognize Mailman >digests unless I switch to MIME format. which -- if you're munging digests -- you should anyway, since that's a big reaso FOR MIME digest format. Tehy're designed to be parseable. But that doesn't ignore that digests need to be tweaked, but if you're doing stuff to digests (like bursting), you should use MIME digests any time tehy're available. >I would like to be able to do this as more than just list admin. I've >got at least three different lists on my system that have an announce, >regular and digest options. The announce has separate subscribers and >also goes immediately to all regular and digest subscribers. Currently >those lists are still running under Majordomo, as I have no idea how to >make them do the same thing with Mailman. The "easy" answer is to disable digests for announce lists. Since they tend to be very low volume anyway, digest versions are generally pretty useless. >And in the case of one of the lists it's not even an announce list, just >a "regular" address that anyone can post to for topically urgent items. ditto here. It's pretty useless to read a digest of topical/timely items, because the concept of digests is to hold it for later. Defeats teh purpose, so turn them off. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From brian@gweep.bc.ca Sun Nov 26 17:18:22 2000 From: brian@gweep.bc.ca (Brian Edmonds) Date: 26 Nov 2000 09:18:22 -0800 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: Chuq Von Rospach's message of "Sun, 26 Nov 2000 09:01:11 -0800" References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: Chuq Von Rospach writes: > if you're doing stuff to digests (like bursting), you should use MIME > digests any time tehy're available. Sure, in theory, but I can burst 1153 digests for reading purposes just fine. The full headers in the MIME digests can be nice on occasion, but for 99% of my needs, 1153 is just fine an is both smaller and faster for my client to parse. > >The announce has separate subscribers and also goes immediately to > >all regular and digest subscribers. Currently those lists are still > >running under Majordomo, as I have no idea how to make them do the > >same thing with Mailman. > The "easy" answer is to disable digests for announce lists. Since they > tend to be very low volume anyway, digest versions are generally pretty > useless. Um, no. Posts to the announce list go to the announce subscribers, the regular subscribers and directly to the digest subscribers. I could maintain a list of additional announce recipients which would be the regular and digest subscribers, but that would be an extra mechanism on top of Mailman, and prone to getting out of sync. Majordomo lets me set things up this way, which is pretty much the only thing keeping me from moving all my lists to Mailman. I'd also like to be able to do admin regexp matching on the body, but this may already be present in 2.0. I'm still running beta2. Likewise configurable holding or immediate rejection on a per regexp basis. Brian. From tneff@bigfoot.com Mon Nov 27 17:20:51 2000 From: tneff@bigfoot.com (Tom Neff) Date: Mon, 27 Nov 2000 12:20:51 -0500 Subject: [Mailman-Developers] Re: MIME "digests" vs. real Digests In-Reply-To: <20001127170103.7A9AD1CDC9@dinsdale.python.org> Message-ID: <370075433.975327651@NY110-19-024B.bloomberg.com> Generating proper 1153 Digests is very important. MIME "Digests" are not true digests in the original sense of providing a readable daily omnibus of list traffic, they are a hack concocted by people who were thrilled with MIME and thought digests were just another way of concatenating messages and files. They don't save much space and they don't provide the same service; most modern mailers capable of "dealing" with them are equally capable of routing individual messages to folders based on headers, which is arguably the better solution for people who don't care about bandwidth or disk space to begin with. From chuqui@plaidworks.com Mon Nov 27 22:01:22 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 27 Nov 2000 14:01:22 -0800 Subject: [Mailman-Developers] fyi -- Ted Montgomery Message-ID: Ted Montgomery's lists of over and under-rated players on each team. Note his leaf's picks. Not that I'm making an editorial jab at anyone on THIS list. nope. Nope me. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Mon Nov 27 22:13:23 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 27 Nov 2000 14:13:23 -0800 Subject: [Mailman-Developers] fyi -- Ted Montgomery In-Reply-To: References: Message-ID: whoops. Sorry about that, folks. Right message, wrong list. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From barry@digicool.com Mon Nov 27 22:27:38 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Mon, 27 Nov 2000 17:27:38 -0500 Subject: [Mailman-Developers] fyi -- Ted Montgomery References: Message-ID: <14882.57306.927142.149194@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> whoops. Sorry about that, folks. Right message, wrong list. And I thought you were jabbing at me with this: Teams with the brightest future in the East: Tampa Bay and the Islanders. Teams with the bleakest future in the East: Carolina and Washington. go-caps!-ly y'rs, -Barry :) From jeb@ocha.net Tue Nov 28 00:51:30 2000 From: jeb@ocha.net (Jeb Bateman) Date: Mon, 27 Nov 2000 16:51:30 -0800 Subject: [Mailman-Developers] Bug fix? (was: [ANNOUNCE] Mailman 2.0) In-Reply-To: <14876.860.607056.18676@anthem.concentric.net>; from barry@digicool.com on Wed, Nov 22, 2000 at 12:33:16PM -0500 References: <14876.860.607056.18676@anthem.concentric.net> Message-ID: <20001127165129.W13647@freequotes.com> Hello, On Wed, Nov 22, 2000 at 12:33:16PM -0500, Barry A. Warsaw wrote: > I'm please to announce the 2.0 final release of Mailman I just installed 2.0 final over 2.0beta2, and all went well, except it generates broken URL paths (missing / between mailman/admin). The following snippet is from the owner notification message for a new list, and all the paths are broken (ie. mailmanlistinfo instead of mailman/listinfo) in the message, and in the headers... --- You can configure your mailing list at the following web page: http://qube.freequotes.com/mailmanadmin/bc-discussion --- Does anyone have a quick pointer where to fix this in the source? I'll look for it myself, but a pointer would still help... (Btw, Please Cc me, as I am not on this developers list.) Thanks, -jeb -- Jeb Bateman Internet Consultant 455 California Ave. Reno, NV 89509 Personal home page: http://jeb.ocha.net From marouni@earlham.edu Tue Nov 28 01:08:44 2000 From: marouni@earlham.edu (Nicholas Marouf) Date: Mon, 27 Nov 2000 20:08:44 -0500 Subject: [Mailman-Developers] Bug fix? (was: [ANNOUNCE] Mailman 2.0) References: <14876.860.607056.18676@anthem.concentric.net> <20001127165129.W13647@freequotes.com> Message-ID: <3A23059C.2F84F871@earlham.edu> Hi Jeb, I got around this by adding a slash at the end of url below in $prefix/Mailman/mm_cfg.py DEFAULT_URL = 'http://www.earlham.edu/mailman/' that fixed the problem. Nick Jeb Bateman wrote: > > Hello, > > On Wed, Nov 22, 2000 at 12:33:16PM -0500, Barry A. Warsaw wrote: > > I'm please to announce the 2.0 final release of Mailman > > I just installed 2.0 final over 2.0beta2, and all went well, except it > generates broken URL paths (missing / between mailman/admin). The > following snippet is from the owner notification message for a new > list, and all the paths are broken (ie. mailmanlistinfo instead of > mailman/listinfo) in the message, and in the headers... > > --- > You can configure your mailing list at the following web page: > > http://qube.freequotes.com/mailmanadmin/bc-discussion > --- > > Does anyone have a quick pointer where to fix this in the source? > I'll look for it myself, but a pointer would still help... (Btw, > Please Cc me, as I am not on this developers list.) > Thanks, > -jeb > > -- > Jeb Bateman > Internet Consultant > 455 California Ave. Reno, NV 89509 > Personal home page: http://jeb.ocha.net > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers -- Nicholas Marouf http://www.ramallahonline.com From heinrich@wh9.tu-dresden.de Tue Nov 28 11:26:44 2000 From: heinrich@wh9.tu-dresden.de (Heinrich Langos) Date: Tue, 28 Nov 2000 12:26:44 +0100 Subject: [Mailman-Developers] pipermail: subjects with "Re[2]:" are not recognized as reply Message-ID: <20001128122644.A15544@wh9.tu-dresden.de> hi there. seems like the "The Bat" mailer inserts a counter into the "Re:" subject line so that "Re[2]:blah blah" or "Re[3]: more blah blah" show up. these are not cought by Mailman/Archiver/HyperArch.py and therefore are not sorted into the thread that they belong to. this is the line in Mailman/Archiver/HyperArch.py that seems to be responsible for it. -------------------- # Subject lines preceded with 'Re:' REpat = re.compile( r"\s*RE\s*:\s*", re.IGNORECASE) -------------------- i don't know python but if the regex is like perl a change like this should do the job: ----------------- REpat = re.compile( r"\s*RE\s*(\[\d*\])??\s*:\s*", re.IGNORECASE) ----------------- as i said i don't have a clue about python and i don't want to test it right now since i have an active list and i don't want to miss any mails in the archive ... i would appreciate if somebody could check it out and tell me if it works. and tell me if i have to recompile those HyperArch.pyc and HyperArch.pyo files. cheers and thanx for that nice program. -heinrich ps: i run mailman 1.1 but i checked the CVS archive and the regex used for REpat seems to be the same. pps: i submitted this as a bug in sourceforge as well .. so if you fix it you can close it there too. ppps: i am not subscribed to mailman-developers@python.org so please include me in CC:'s -- Heinrich Langos pgp: http://wh9.tu-dresden.de/~heinrich/pub_pgp_key.asc _________________________________________________________________ |o| The reason we come up with new versions is not to fix bugs. |o| |o| It's absolutely not. It's the stupidest reason to buy a new |o| |o| version I ever heard. -- Bill Gates, Microsoft Corporation |o| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From Nigel.Metheringham@VData.co.uk Tue Nov 28 13:17:04 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Tue, 28 Nov 2000 13:17:04 +0000 Subject: [Mailman-Developers] pipermail: subjects with "Re[2]:" are not recognized as reply In-Reply-To: Message from Heinrich Langos of "Tue, 28 Nov 2000 12:26:44 +0100." <20001128122644.A15544@wh9.tu-dresden.de> Message-ID: heinrich@wh9.tu-dresden.de said: > seems like the "The Bat" mailer inserts a counter into the "Re:" > subject line so that "Re[2]:blah blah" or "Re[3]: more blah blah" show > up. these are not cought by Mailman/Archiver/HyperArch.py and > therefore are not sorted into the thread that they belong to. What goes around, comes around. This came up in 94/95 or so with the nn newsreader. After an outcry and much quoting of RFCs nn backed down and stopped doing it. I suggest someone deal with the author of "The bat" in the same way. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From heinrich@wh9.tu-dresden.de Tue Nov 28 14:37:07 2000 From: heinrich@wh9.tu-dresden.de (Heinrich Langos) Date: Tue, 28 Nov 2000 15:37:07 +0100 Subject: [Mailman-Developers] pipermail: subjects with "Re[2]:" are not recognized as reply In-Reply-To: ; from Nigel.Metheringham@VData.co.uk on Tue, Nov 28, 2000 at 01:17:04PM +0000 References: Message-ID: <20001128153707.C15544@wh9.tu-dresden.de> On Tue, Nov 28, 2000 at 01:17:04PM +0000, Nigel Metheringham wrote: > > heinrich@wh9.tu-dresden.de said: > > seems like the "The Bat" mailer inserts a counter into the "Re:" > > subject line so that "Re[2]:blah blah" or "Re[3]: more blah blah" show > > up. these are not cought by Mailman/Archiver/HyperArch.py and > > therefore are not sorted into the thread that they belong to. > > What goes around, comes around. This came up in 94/95 or so with the > nn newsreader. After an outcry and much quoting of RFCs nn backed down > and stopped doing it. I suggest someone deal with the author of "The > bat" in the same way. > > Nigel. i've found a way for The Bat users to disable that irritating behavior. http://www.mail-archive.com/tbudl@thebat.dutaint.com/msg16021.html could you point me to an archive of that discussion? preferably with pointers to the RFCs that this behavior violates? i would like to contact the authors but not without some background information. thanx in advance -heinrich -- Heinrich Langos pgp: http://wh9.tu-dresden.de/~heinrich/pub_pgp_key.asc _________________________________________________________________ |o| The reason we come up with new versions is not to fix bugs. |o| |o| It's absolutely not. It's the stupidest reason to buy a new |o| |o| version I ever heard. -- Bill Gates, Microsoft Corporation |o| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From claw@kanga.nu Tue Nov 28 18:19:04 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 28 Nov 2000 10:19:04 -0800 Subject: [Mailman-Developers] pipermail: subjects with "Re[2]:" are not recognized as reply In-Reply-To: Message from Heinrich Langos of "Tue, 28 Nov 2000 12:26:44 +0100." <20001128122644.A15544@wh9.tu-dresden.de> References: <20001128122644.A15544@wh9.tu-dresden.de> Message-ID: <23723.975435544@kanga.nu> On Tue, 28 Nov 2000 12:26:44 +0100 Heinrich Langos wrote: > seems like the "The Bat" mailer inserts a counter into the "Re:" > subject line so that "Re[2]:blah blah" or "Re[3]: more blah blah" > show up. these are not cought by Mailman/Archiver/HyperArch.py > and therefore are not sorted into the thread that they belong to. LotusNotes does (or did, or can do, I'm not sure which) the same thing. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From exim-users@exim.org Tue Nov 28 23:42:27 2000 From: exim-users@exim.org (Jeffrey C. Ollie) Date: Tue, 28 Nov 2000 17:42:27 -0600 Subject: [Mailman-Developers] Python embedded in Exim Message-ID: <20001128174227.A24208@ollie.clive.ia.us> --aM3YZ0Iwxop3KEKx Content-Type: multipart/mixed; boundary="FL5UXtIhxfXey3p5" Content-Disposition: inline --FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Well, I had an epiphany over the US Thanksgiving holiday weekend (or maybe it was a brain aneurism) so I've spent some late nights and some time stolen from work and come up with a first pass at embedding Python in Exim. I imagine that embedding Python in Exim will be interesting to those folks writing virus scanners or for VERY tight integration of Mailman with Exim. You can also do all of those things that you can do with the Perl string expansions. Note: this code is VERY lightly tested. It compiles on my home hacked-up RedHat Linux system and works on a couple of simple tests that I've done, but no real e-mails have passed through this code yet. I'm tossing this out for public comment on the design and implementation... It's been a while since I've done a lot of C coding. Unfortunately, you must embed Python 2.0. This is because previous versions of Python used a hacked copy of PCRE as it's regular expression engine that conflicts with the copy of PCRE in Exim. Starting in Python 2.0 the default regular expression is a completely new one called SRE that supports Unicode. The PCRE-based regular expression engine is still included in Python 2.0 but I get around that by creating a private copy of the Python library and delete the offending object modules. You could do that in versions of Python prior to 2.0 but a lot of the standard library depends on regular expressions I didn't think that there was much point in trying to make the embedding work with 1.5.2 or 1.6. Well, on to the patch itself. The patch is made against Exim v3.20. After you've patched the source code, you need to set four variables in the Local/Makefile: EXIM_PYTHON=python.o PYTHON_INC=-I/usr/local/include/python2.0 PYTHON_LIB=/usr/local/lib/python2.0/config/libpython2.0.a PYTHON_EXTRA_LIBS=-lpthread -ldl -lutil Then build Exim as usual. There are three runtime directives that control the embedding, all of which are optional: python_at_start boolean - force startup of Python interpreter python_module_paths colon separated list of paths to append to sys.path python_initial_import colon separated list of modules to import at interpreter initialization time There are also two new command line switches -ys and -yd that will force an immediate startup of Python or delay startup, overriding python_at_start. Then you use it: ${python{.}{}...} You must specify the module name and the function name. The named module will be imported automatically. And currently you must specify a function that returns a string (I'll look into adding accessing attributes, methods of classes, and converting non-strings into strings). There can be up to eight arguments. Each argument will be expanded by Exim before it's passed to the Python interpreter. Your python code has access to a module called "exim", which currently defines two things: exim.expand_string(s) which calls back to Exim to expand a string and exim.error which is an exception object used by expand_string One simple example: ${python{string.upper}{$local_part}} Error reporting isn't the best right now, I'm going to look into formatting exceptions and tracebacks for more meaningful error messages. Well, I think that's the gist of it. I'll work on patches to the documentation when this thing becomes more stable. As I said before I've only lightly tested this... beware! Comments, criticisms, suggestions, flames welcome... Jeff --FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="exim-python.patch" Content-Transfer-Encoding: quoted-printable Index: exim/OS/Makefile-Base diff -u exim/OS/Makefile-Base:1.1.1.1 exim/OS/Makefile-Base:1.1.1.1.2.3 --- exim/OS/Makefile-Base:1.1.1.1 Mon Nov 27 08:18:15 2000 +++ exim/OS/Makefile-Base Tue Nov 28 16:49:25 2000 @@ -192,6 +192,16 @@ @chmod a+x ../util/convert4r3 @echo ">>> convert4r3 script built in util directory"; echo "" =20 +libpython.a: $(PYTHON_LIB) + if [ -n $(PYTHON_LIB) ] ;\ + then \ + cp $(PYTHON_LIB) libpython.a ;\ + ar d libpython.a pcremodule.o pypcre.o ;\ + ranlib libpython.a ;\ + else \ + ar cq libpython.a ;\ + ranlib libpython.a ;\ + fi =20 # Targets for final binaries; the main one has a build number which is # updated each time. We don't bother with that for the auxiliaries. @@ -201,11 +211,12 @@ header.o host.o log.o match.o moan.o os.o parse.o queue.o \ readconf.o retry.o rewrite.o \ route.o search.o smtp_in.o smtp_out.o spool_in.o spool_out.o \ - store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PE= RL) + store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PE= RL) \ + $(EXIM_PYTHON) =20 exim: libident/libident.a pcre/libpcre.a lookups/lookups.a auths/auths.a= \ directors/directors.a routers/routers.a transports/transports.a \ - $(OBJ_EXIM) version.c + $(OBJ_EXIM) version.c libpython.a awk '{ print ($$1+1) }' cnumber.h > cnumber.temp /bin/rm -f cnumber.h; mv cnumber.temp cnumber.h $(CC) -c $(CFLAGS) $(INCLUDE) $(IPV6_INCLUDE) $(TLS_INCLUDE) version.c @@ -215,7 +226,8 @@ routers/routers.a transports/transports.a lookups/lookups.a \ auths/auths.a \ $(LIBS) $(LIBS_EXIM) $(IPV6_LIBS) $(EXTRALIBS) $(EXTRALIBS_EXIM) \ - $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) $(TLS_LIBS) + $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) libpython.a $(PYTHON= _EXTRA_LIBS) \ + $(TLS_LIBS) $(EXIM_CHMOD) @echo " " @echo ">>> exim binary built" @@ -316,6 +328,11 @@ =20 perl.o: $(HDRS) perl.c $(PERL_CC) $(PERL_CCOPTS) $(CFLAGS) $(INCLUDE) -c perl.c + +# Build instructions for python.o for when EXIM_PYTHON is set + +python.o: $(HDRS) python.c + $(CC) -c $(CFLAGS) $(INCLUDE) $(PYTHON_INC) -I. python.c =20 # Dependencies for the "ordinary" exim modules =20 Index: exim/scripts/MakeLinks diff -u exim/scripts/MakeLinks:1.1.1.1 exim/scripts/MakeLinks:1.1.1.1.2.1 --- exim/scripts/MakeLinks:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/scripts/MakeLinks Mon Nov 27 08:27:46 2000 @@ -189,6 +189,7 @@ ln -s ../src/moan.c moan.c ln -s ../src/parse.c parse.c ln -s ../src/perl.c perl.c +ln -s ../src/python.c python.c ln -s ../src/queue.c queue.c ln -s ../src/readconf.c readconf.c ln -s ../src/retry.c retry.c Index: exim/src/EDITME diff -u exim/src/EDITME:1.1.1.2 exim/src/EDITME:1.1.1.2.2.4 --- exim/src/EDITME:1.1.1.2 Mon Nov 27 08:18:54 2000 +++ exim/src/EDITME Tue Nov 28 16:20:19 2000 @@ -260,6 +260,10 @@ # PERL_CCOPTS=3D # PERL_LIBS=3D =20 +# EXIM_PYTHON=3Dpython.o +# PYTHON_INC=3D-I/usr/local/include/python2.0 +# PYTHON_LIB=3D/usr/local/lib/python2.0/config/libpython2.0.a +# PYTHON_EXTRA_LIBS=3D-lpthread -ldl -lutil =20 # This parameter sets the maximum length of the header portion of a message # that Exim is prepared to process. The default setting is one megabyte. T= here Index: exim/src/config.h.defaults diff -u exim/src/config.h.defaults:1.1.1.1 exim/src/config.h.defaults:1.1.1= .1.2.1 --- exim/src/config.h.defaults:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/src/config.h.defaults Mon Nov 27 08:43:04 2000 @@ -36,6 +36,8 @@ =20 #define EXIM_PERL =20 +#define EXIM_PYTHON + #define HAVE_SA_LEN #define HEADER_MAXSIZE (1024*1024) =20 Index: exim/src/exim.c diff -u exim/src/exim.c:1.1.1.2 exim/src/exim.c:1.1.1.2.2.4 --- exim/src/exim.c:1.1.1.2 Mon Nov 27 08:18:54 2000 +++ exim/src/exim.c Tue Nov 28 00:46:52 2000 @@ -351,6 +351,9 @@ #ifdef EXIM_PERL int perl_start_option =3D 0; #endif +#ifdef EXIM_PYTHON +int python_start_option =3D 0; +#endif int recipients_arg =3D argc; int sender_address_domain =3D 0; int test_retry_arg =3D -1; @@ -583,6 +586,17 @@ simply recorded for checking and handling afterwards. Do a high-level swit= ch on the second character (the one after '-'), to save some effort. */ =20 +#ifdef EXIM_PYTHON +opt_python_original_argc =3D argc; +opt_python_original_argv =3D store_get(sizeof(char *) * (argc + 1)); +opt_python_original_argv[argc] =3D NULL; + +for (i =3D 0; i < argc; i++) + { + opt_python_original_argv[i] =3D string_copy(argv[i]); + } +#endif + for (i =3D 1; i < argc; i++) { BOOL badarg =3D FALSE; @@ -1521,6 +1535,17 @@ break; #endif =20 + /* -ys: force Python startup; -yd force delayed Python startup */ + + #ifdef EXIM_PYTHON + case 'y': + if (*argrest =3D=3D 's' && argrest[1] =3D=3D 0) python_start_option = =3D 1; + else + if (*argrest =3D=3D 'd' && argrest[1] =3D=3D 0) python_start_option = =3D -1; + else badarg =3D TRUE; + break; + #endif + =20 case 'q': =20 @@ -1967,6 +1992,27 @@ opt_perl_started =3D TRUE; } #endif /* EXIM_PERL */ + +/* Start up Python interpreter if Python support is configured and +there is a and the configuration or the command line specifies +initializing starting. Note that the global variables are actually +called opt_python_xxx. */ + +#ifdef EXIM_PYTHON +if (python_start_option !=3D 0) + opt_python_at_start =3D (python_start_option > 0); +if (opt_python_at_start) + { + char *errstr; + DEBUG(9) debug_printf("Starting Python interpreter\n"); + errstr =3D init_python(); + if (errstr !=3D NULL) + { + fprintf(stderr, "exim: error during Python startup: %s\n", errstr); + return EXIT_FAILURE; + } + } +#endif /* EXIM_PYTHON */ =20 /* Log the arguments of the call if the configuration file said so. This is a debugging feature for finding out what arguments certain MUAs actually u= se. Index: exim/src/expand.c diff -u exim/src/expand.c:1.1.1.2 exim/src/expand.c:1.1.1.2.2.3 --- exim/src/expand.c:1.1.1.2 Mon Nov 27 08:18:54 2000 +++ exim/src/expand.c Tue Nov 28 00:23:10 2000 @@ -1318,7 +1318,7 @@ =20 Operators: domain extracts domain from an address - escape escapes non-printing characters + escape escapes non-printing characters=20 expand expands the string one more time hash__ hash the string, making one that is of length n, using m (default 26) characters from hashcodes @@ -1865,6 +1865,100 @@ continue; } #endif /* EXIM_PERL */ + + /* If Perl support is configured, handle calling embedded perl subroutin= es, + unless locked out at this time. Syntax is ${perl{sub}} or ${perl{sub}{ar= g}} + or ${perl{sub}{arg1}{arg2}} or up to a maximum of EXIM_PERL_MAX_ARGS + arguments (defined below). */ + + #ifdef EXIM_PYTHON + #define EXIM_PYTHON_MAX_ARGS 8 + + if (strcmp(name, "python") =3D=3D 0) + { + int i =3D 0; + char *sub_name; + char *sub_arg[EXIM_PYTHON_MAX_ARGS + 1]; + char *new_yield; + + if (expand_forbid_python) + { + expand_string_message =3D "Python calls are not permitted"; + goto EXPAND_FAILED; + } + + while (isspace((uschar)*s)) s++; + if (*s !=3D '{') goto EXPAND_FAILED_CURLY; + sub_name =3D expand_string_internal(s+1, TRUE, &s, FALSE); + if (sub_name =3D=3D NULL) goto EXPAND_FAILED; + while (isspace((uschar)*s)) s++; + if (*s++ !=3D '}') goto EXPAND_FAILED_CURLY; + + while (isspace((uschar)*s)) s++; + + while (*s =3D=3D '{') + { + if (i =3D=3D EXIM_PYTHON_MAX_ARGS) + { + expand_string_message =3D + string_sprintf("Too many arguments passed to Python subroutine \= "%s\" " + "(max is %d)", sub_name, EXIM_PYTHON_MAX_ARGS); + goto EXPAND_FAILED; + } + sub_arg[i] =3D expand_string_internal(s+1, TRUE, &s, FALSE); + if (sub_arg[i++] =3D=3D NULL) goto EXPAND_FAILED; + while (isspace((uschar)*s)) s++; + if (*s++ !=3D '}') goto EXPAND_FAILED_CURLY; + while (isspace((uschar)*s)) s++; + } + + if (*s++ !=3D '}') goto EXPAND_FAILED_CURLY; + sub_arg[i] =3D 0; + + /* Start the interpreter if necessary */ + + if (!opt_python_started) + { + char *initerror; + DEBUG(9) debug_printf("Starting Python interpreter\n"); + initerror =3D init_python(); + if (initerror !=3D NULL) + { + expand_string_message =3D + string_sprintf("error during Python startup: %s\n", initerror); + goto EXPAND_FAILED; + } + } + + /* Call the function */ + + new_yield =3D call_python_cat(yield, &size, &ptr, &expand_string_messa= ge, + sub_name, sub_arg); + + /* NULL yield indicates failure; if the message pointer has been set to + NULL, the yield was undef, indicating a forced failure. Otherwise the + message will indicate some kind of Perl error. */ + + if (new_yield =3D=3D NULL) + { + if (expand_string_message =3D=3D NULL) + { + expand_string_message =3D + string_sprintf("Python subroutine \"%s\" returned undef to force= " + "failure", sub_name); + expand_string_forcedfail =3D TRUE; + } + goto EXPAND_FAILED; + } + + /* Yield succeeded. Ensure forcedfail is unset, just in case it got + set during a callback from Python. */ + + expand_string_forcedfail =3D FALSE; + yield =3D new_yield; + continue; + } + #endif /* EXIM_PYTHON */ =20 /* Handle character translation for "tr" */ =20 Index: exim/src/functions.h diff -u exim/src/functions.h:1.1.1.2 exim/src/functions.h:1.1.1.2.2.1 --- exim/src/functions.h:1.1.1.2 Mon Nov 27 08:18:54 2000 +++ exim/src/functions.h Mon Nov 27 10:13:25 2000 @@ -16,6 +16,12 @@ extern char *init_perl(char *); #endif =20 +#ifdef EXIM_PYTHON +extern char *call_python_cat(char *, int *, int *, char **, char *, char *= *); +extern void cleanup_python(void); +extern char *init_python(void); +#endif + #ifdef SUPPORT_TLS extern BOOL tls_client_start(int, host_item *, address_item *, char *, char *, char *, char *, char *, int); Index: exim/src/globals.c diff -u exim/src/globals.c:1.1.1.1 exim/src/globals.c:1.1.1.1.2.2 --- exim/src/globals.c:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/src/globals.c Mon Nov 27 11:10:15 2000 @@ -35,6 +35,15 @@ BOOL opt_perl_started =3D FALSE; #endif =20 +#ifdef EXIM_PYTHON +BOOL opt_python_at_start =3D FALSE; +char *opt_python_module_paths =3D NULL; +char *opt_python_initial_import =3D NULL; +BOOL opt_python_started =3D FALSE; +int opt_python_original_argc =3D 0; +char **opt_python_original_argv =3D NULL; +#endif + #ifdef HAVE_AUTH BOOL auth_always_advertise =3D TRUE; char *auth_hosts =3D NULL; @@ -310,6 +319,7 @@ BOOL expand_forbid_exists =3D FALSE; BOOL expand_forbid_lookup =3D FALSE; BOOL expand_forbid_perl =3D FALSE; +BOOL expand_forbid_python =3D FALSE; int expand_nlength[EXPAND_MAXN+1]; int expand_nmax =3D -1; char *expand_nstring[EXPAND_MAXN+1]; Index: exim/src/globals.h diff -u exim/src/globals.h:1.1.1.1 exim/src/globals.h:1.1.1.1.2.2 --- exim/src/globals.h:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/src/globals.h Mon Nov 27 11:10:15 2000 @@ -21,6 +21,15 @@ extern BOOL opt_perl_started; /* Set once interpreter started */ #endif =20 +#ifdef EXIM_PYTHON +extern BOOL opt_python_at_start; /* start Python at startup */ +extern char *opt_python_module_paths; /* list of paths to append to sys= .path */ +extern char *opt_python_initial_import; /* list of modules to import at i= nterpreter initialization time */ +extern BOOL opt_python_started; /* set once Python interpreter st= arted */ +extern int opt_python_original_argc; +extern char **opt_python_original_argv; +#endif + #ifdef HAVE_AUTH extern BOOL auth_always_advertise; /* If FALSE, advertise only when nee= ded */ extern char *auth_hosts; /* These must authenticate */ @@ -208,6 +217,7 @@ extern BOOL expand_forbid_exists; /* Lock out exists checking */ extern BOOL expand_forbid_lookup; /* Lock out lookups */ extern BOOL expand_forbid_perl; /* Lock out Perl calls */ +extern BOOL expand_forbid_python; /* Lock out Python calls */ extern int expand_nlength[]; /* Lengths of numbered strings */ extern int expand_nmax; /* Max numerical value */ extern char *expand_nstring[]; /* Numbered strings */ Index: exim/src/python.c diff -u /dev/null exim/src/python.c:1.1.2.11 --- /dev/null Tue Nov 28 16:50:15 2000 +++ exim/src/python.c Tue Nov 28 16:22:03 2000 @@ -0,0 +1,291 @@ +#include "exim.h" + +#include "Python.h" + +static void init_exim_module(void); + +char *init_python(void) +{ + if (opt_python_started) + { + return NULL; + } + + Py_SetProgramName(opt_python_original_argv[0]); + =20 + Py_Initialize(); + =20 + PySys_SetArgv(opt_python_original_argc, opt_python_original_argv); + + init_exim_module(); + + if (opt_python_module_paths) + { + char *listptr =3D opt_python_module_paths; + int separator =3D 0; + char buffer[256]; + PyObject *path; + PyObject *sys_module; + PyObject *sys_dict; + PyObject *sys_path; + PyObject *name; + + name =3D PyString_FromString("sys"); + if (name =3D=3D NULL) + { + PyErr_Clear(); + return "problem creating string \"sys\""; + } + + sys_module =3D PyImport_Import(name); + Py_DECREF(name); + + if (sys_module =3D=3D NULL) + { + PyErr_Clear(); + return "problem trying to import sys"; + } + + sys_dict =3D PyModule_GetDict(sys_module); + if (sys_dict =3D=3D NULL) + { + PyErr_Clear(); + return "problem trying get dictionary from module sys"; + } + + sys_path =3D PyDict_GetItemString(sys_dict, "path"); + if (sys_path =3D=3D NULL) + { + PyErr_Clear(); + return "problem trying to get sys.path"; + } + + while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)= )) + { + path =3D PyString_FromString(buffer); + if (path =3D=3D NULL) + { + PyErr_Clear(); + return string_sprintf("problem while allocating Python string for \"%= s\"", buffer); + } + + if(!PyList_Append(sys_path, path)) + { + PyErr_Clear(); + return string_sprintf("problem while insering string \"%s\"into sys.p= ath", buffer); + } + } + } + + if (opt_python_initial_import) + { + char *listptr =3D opt_python_initial_import; + int separator =3D 0; + char buffer[256]; + PyObject *module; + PyObject *name; + + while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)= )) + { + name =3D PyString_FromString(buffer); + if (name =3D=3D NULL) + { + PyErr_Clear(); + return string_sprintf("problem creating string \"%s\"", buffer); + } + + module =3D PyImport_Import(name); + Py_DECREF(name); + + if (module =3D=3D NULL) + { + PyErr_Clear(); + return string_sprintf("problem importing module %s", buffer); + } + } + } + + =20 + opt_python_started =3D TRUE; + return NULL; +} + +void cleanup_python(void) +{ + Py_Finalize(); +} + +static PyObject *find_function(char *name, char **errstrp) +{ + PyObject *module_name; + PyObject *function_name; + PyObject *module; + PyObject *dictionary; + PyObject *result; + + char *ptr; + + ptr =3D strrchr(name, '.'); + =20 + if (ptr =3D=3D NULL) + { + *errstrp =3D string_sprintf("function name must include module: ."); + return NULL; + } + + function_name =3D PyString_FromString(ptr + 1); + + if (function_name =3D=3D NULL) + { + PyErr_Clear(); + *errstrp =3D string_sprintf("error trying to allocate function name"= ); + return NULL; + } + + module_name =3D PyString_FromStringAndSize(name, ptr - name); + + if (function_name =3D=3D NULL) + { + PyErr_Clear(); + Py_DECREF(function_name); + *errstrp =3D string_sprintf("error trying to allocate module name"); + return NULL; + } + + module =3D PyImport_Import(module_name); + + Py_DECREF(module_name); + + if (module =3D=3D NULL) + { + PyErr_Clear(); + Py_DECREF(function_name); + *errstrp =3D string_sprintf("error trying to import module while try= ing to find %s", name); + return NULL; + } + + dictionary =3D PyModule_GetDict(module); + + result =3D PyDict_GetItem(dictionary, function_name); + + Py_DECREF(function_name); + + if (!PyCallable_Check(result)) + { + PyErr_Clear(); + *errstrp =3D string_sprintf("%s is not a callable object", name); + return NULL; + } + + return result; +} + +char *call_python_cat(char *yield, int *sizep, int *ptrp, char **errstrp, + char *name, char **arg) +{ + PyObject *function; + PyObject *arguments; + PyObject *result; + int index; + int count=3D0; + char **p; + char *buffer; + int length; + + function =3D find_function(name, errstrp); + + if (function =3D=3D NULL) + return NULL; + + p =3D arg; + + while(*p) + { + count++; + p++; + } + + arguments =3D PyTuple_New(count); + + index =3D 0; + while(index < count) + { + + PyTuple_SetItem(arguments, index, PyString_FromString(arg[index])); + index++; + } + + result =3D PyObject_CallObject(function, arguments); + + Py_DECREF(arguments); + + if (result =3D=3D NULL) + { + PyErr_Clear(); + *errstrp =3D string_sprintf("error in Python function %s", name); + return NULL; + } + + if(!PyString_Check(result)) + { + Py_DECREF(result); + *errstrp =3D string_sprintf("Python function %s returned non-string"= , name); + return NULL; + } + + if(PyString_AsStringAndSize(result, &buffer, &length)) + { + PyErr_Clear(); + Py_DECREF(result); + *errstrp =3D string_sprintf("could not extract the results of Python= function %s", name); + return NULL; + } + + yield =3D string_cat(yield, sizep, ptrp, buffer, length); + + Py_DECREF(result); + + return yield; +} + +static PyObject *exim_error =3D NULL; + +static PyObject *exim_expand_string(PyObject *self, PyObject *args) +{ + char *i; + char *o; + + if(!PyArg_ParseTuple(args, "s", i)) + return NULL; + =20 + o =3D expand_string(i); + =20 + if (o =3D=3D NULL) + { + PyErr_SetString(exim_error, expand_string_message); + return NULL; + } + =20 + return PyString_FromString(o); +} + +static PyMethodDef exim_methods[] =3D { + {"expand_string", exim_expand_string, 1}, + {NULL, NULL} /* sentinel */ +}; + +static void init_exim_module(void) +{ + PyObject *m; + PyObject *d; + PyObject *o; + + m =3D PyImport_AddModule("exim"); + d =3D PyModule_GetDict(m); + + Py_InitModule("exim", exim_methods); + + exim_error =3D PyErr_NewException("exim.error", NULL, NULL); + PyDict_SetItemString(d, "error", exim_error); +} + Index: exim/src/readconf.c diff -u exim/src/readconf.c:1.1.1.1 exim/src/readconf.c:1.1.1.1.2.2 --- exim/src/readconf.c:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/src/readconf.c Mon Nov 27 10:14:17 2000 @@ -160,6 +160,11 @@ { "perl_at_start", opt_bool, &opt_perl_at_start }, { "perl_startup", opt_stringptr, &opt_perl_startup }, #endif +#ifdef EXIM_PYTHON + { "python_at_start", opt_bool, &opt_python_at_start }, + { "python_module_paths", opt_stringptr, &opt_python_module_paths = }, + { "python_initial_import", opt_stringptr, &opt_python_initial_impor= t }, +#endif #ifdef LOOKUP_PGSQL { "pgsql_servers", opt_stringptr, &pgsql_servers }, #endif --FL5UXtIhxfXey3p5-- --aM3YZ0Iwxop3KEKx Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6JELjmCSL+rEqeRsRArY9AJ9iw1RL79TcwrKoRxLCFGSvJ/hPqgCglwRc JfzWMV388u0Hohv9S1xDxcE= =430v -----END PGP SIGNATURE----- --aM3YZ0Iwxop3KEKx-- From jam@jamux.com Wed Nov 29 15:57:19 2000 From: jam@jamux.com (John A. Martin) Date: Wed, 29 Nov 2000 10:57:19 -0500 Subject: [Mailman-Developers] Possible cookie problem Message-ID: <20001129155720.0BECF4800B@athene.jamux.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I ain't got no clue. I access with no problem, but I use a different "HTTP_USER_AGENT". :-) - - -------------- cut here ---->8 ---< head logs/error Nov 29 07:35:21 2000 admin(26929): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ admin(26929): [----- Mailman Version: 2.0 -----] admin(26929): [----- Traceback ------] admin(26929): Traceback (innermost last): admin(26929): File "/home/mailman/scripts/driver", line 96, in run_main admin(26929): main() admin(26929): File "/home/mailman/Mailman/Cgi/admin.py", line 82, in main admin(26929): Auth.authenticate(mlist, cgidata) admin(26929): File "/home/mailman/Mailman/Cgi/Auth.py", line 72, in authenticate admin(26929): isauthed = mlist.WebAuthenticate(password=adminpw, cookie='admin') admin(26929): File "/home/mailman/Mailman/SecurityManager.py", line 68, in WebAuthenticate admin(26929): return self.CheckCookie(key) admin(26929): File "/home/mailman/Mailman/SecurityManager.py", line 117, in CheckCookie admin(26929): c = Cookie.Cookie(cookiedata) admin(26929): File "/home/mailman/Mailman/Cookie.py", line 509, in __init__ admin(26929): if input: self.load(input) admin(26929): File "/home/mailman/Mailman/Cookie.py", line 546, in load admin(26929): self.__ParseString(rawdata) admin(26929): File "/home/mailman/Mailman/Cookie.py", line 573, in __ParseString admin(26929): M.set(K, apply(self.net_setfunc, (V,)), V) admin(26929): File "/home/mailman/Mailman/Cookie.py", line 421, in set admin(26929): raise CookieError("Attempt to set a reserved key: %s" % key) admin(26929): CookieError: Attempt to set a reserved key: path admin(26929): [----- Python Information -----] admin(26929): sys.version = 1.5.2 (#1, Feb 1 2000, 16:32:16) [GCC egcs-2.91.66 19990314/Linux (egcs- admin(26929): sys.executable = /usr/bin/python admin(26929): sys.prefix = /usr admin(26929): sys.exec_prefix= /usr admin(26929): sys.path = /usr admin(26929): sys.platform = linux-i386 admin(26929): [----- Environment Variables -----] admin(26929): DOCUMENT_ROOT: /home/listproc/httpd/html/ admin(26929): SERVER_ADDR: 216.0.124.17 admin(26929): SERVER_PORT: 80 admin(26929): PATH_TRANSLATED: /home/listproc/httpd/html/hague-jur-commercial-law/ admin(26929): GATEWAY_INTERFACE: CGI/1.1 admin(26929): HTTP_USER_AGENT: Microsoft Internet Explorer/4.40.426 (Windows 95) admin(26929): REMOTE_ADDR: 24.19.174.74 admin(26929): SERVER_NAME: lists.essential.org admin(26929): SCRIPT_FILENAME: /home/mailman/cgi-bin/admin admin(26929): HTTP_ACCEPT: www/source, text/html, video/mpeg, image/jpeg, image/x-tiff, image/x-rgb, image/x-xbm, image/gif, */*, application/postscript admin(26929): REQUEST_URI: /mailman/admin/hague-jur-commercial-law/ admin(26929): QUERY_STRING: admin(26929): SERVER_PROTOCOL: HTTP/1.0 admin(26929): PATH_INFO: /hague-jur-commercial-law/ admin(26929): HTTP_HOST: lists.essential.org admin(26929): REQUEST_METHOD: GET admin(26929): SERVER_SIGNATURE:

Apache/1.3.14 Server at lists.essential.org Port 80
admin(26929): SCRIPT_NAME: /mailman/admin admin(26929): SERVER_ADMIN: root@localhost admin(26929): SERVER_SOFTWARE: Apache/1.3.14 (Unix) (Red-Hat/Linux) mod_perl/1.23 admin(26929): PYTHONPATH: /home/mailman admin(26929): HTTP_COOKIE: UID=7E6266E13A24F713; path=/; domain=.excite.com; expires=Wednesday, 31-Dec-2010 12:00:00 GMT admin(26929): REMOTE_PORT: 1791 - - ---- 8<------- cut here ----------> tail logs/error jam -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: OpenPGP encrypted mail preferred. See iEYEARECAAYFAjolJwsACgkQUEvv1b/iXy/rAgCgkqY4ep4vv6ViO6hjjZvOFqqU IeIAnRdzPPGUFha5TCY8r6PLm+y5JO5n =4QsZ -----END PGP SIGNATURE----- From Dan Mick Thu Nov 30 01:16:14 2000 From: Dan Mick (Dan Mick) Date: Wed, 29 Nov 2000 17:16:14 -0800 (PST) Subject: [Mailman-Developers] offtopic: CVS checkout to `pwd` Message-ID: <200011300114.RAA24615@utopia.west.sun.com> This is a question about CVS. I'm constantly creating a directory that I want Mailman to go into, for example /local/src/mailman-pristine, and then realizing that I can't make CVS actually put Mailman *into* that directory. I have to use "checkout mailman" to get it, and that insists on creating a 'mailman' directory. If I move everything from mailman to ./, things work just fine from then on, but isn't there any way to make the checkout command use the current dir? The obvious option, "checkout -d ", gives me an error if I use '.' or an absolute pathname: protocol error: is not absolute Can anyone else do this, or is this just part of Life with CVS? From barry@wooz.org Wed Nov 1 00:43:13 2000 From: barry@wooz.org (barry@wooz.org) Date: Tue, 31 Oct 2000 19:43:13 -0500 (EST) Subject: [Mailman-Developers] My admins speak out... References: <200010312311.PAA19920@utopia.west.sun.com> Message-ID: <14847.26401.287186.613646@anthem.concentric.net> >>>>> "DM" == Dan Mick writes: >> if getattr(mlist, 'auto_reject', 0): raise DiscardMessage DM> Very nice; had no idea that one little exception is all it DM> takes. That only works for pipeline Handlers modules, because they're wrapped in the try/except inside do_pipeline(). DM> Is there anything similarly-simple to generate a generic DM> bounce message and discard, or is that all "call some reply DM> method and then raise DiscardMessage"? Correct. But generating an outgoing message is fairly easy. See the Message.UserNotification class and HandlerAPI.DeliverToUser() (for response to a single user). -Barry From barry@wooz.org Wed Nov 1 01:16:36 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Tue, 31 Oct 2000 20:16:36 -0500 (EST) Subject: [Mailman-Developers] [Mailman-Users] forwarded message from Guido van Rossum (fwd) References: <20001030192005.F12812@xs4all.nl> Message-ID: <14847.28404.962845.410706@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> Anyone else notice something weird about the mail below ? It TW> was sent by Guido to python-dev, where Barry picked it up from TW> and sent it to both mailman-developers and mailman-users. I am TW> only subscribed to mailman-developers, and I got it TW> twice. Are you on mailman-announce? I sent it there too. TW> What's more, the second mail seems to have gone to TW> mailman-developers *through* mailman-users. Notice the extra TW> list-sig at the obttom of the mail, and extra [listname] in TW> the subject. TW> Bug in Mailman ? ;-P Hrm. My simple test posts to multiple lists don't seem to have a problem. -Barry From barry@wooz.org Wed Nov 1 03:41:30 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Tue, 31 Oct 2000 22:41:30 -0500 (EST) Subject: [Mailman-Developers] Get listed. :) Message-ID: <14847.37098.537126.455758@anthem.concentric.net> I'm putting together the on-line documentation for the 2.0 release and I'd like to include a list of sites that run Mailman. I've got Python.Org and lists.apple.com (if that's cool with you Chuq ;). Please send any others you'd like to include to me directly. Thanks, -Barry From barry@wooz.org Wed Nov 1 03:41:30 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Tue, 31 Oct 2000 22:41:30 -0500 (EST) Subject: [Mailman-Developers] [Mailman-Users] Get listed. :) Message-ID: <14847.37098.537126.455758@anthem.concentric.net> I'm putting together the on-line documentation for the 2.0 release and I'd like to include a list of sites that run Mailman. I've got Python.Org and lists.apple.com (if that's cool with you Chuq ;). Please send any others you'd like to include to me directly. Thanks, -Barry ------------------------------------------------------ Mailman-Users maillist - Mailman-Users@python.org http://www.python.org/mailman/listinfo/mailman-users From Johnny Fuerst; Governor" ; from barry@wooz.org on Tue, Oct 31, 2000 at 10:41:30PM -0500 References: <14847.37098.537126.455758@anthem.concentric.net> Message-ID: <20001031220220.A1038@cathat.net> Hey Barry, Can I list my site, if right now, I am only running 1.1? Sincerely, Johnny On Tue, Oct 31, 2000 at 10:41:30PM -0500, Barry A. Warsaw wrote: > > I'm putting together the on-line documentation for the 2.0 release and > I'd like to include a list of sites that run Mailman. I've got > Python.Org and lists.apple.com (if that's cool with you Chuq ;). > Please send any others you'd like to include to me directly. > > Thanks, > -Barry > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users -- On Behalf Of: Johnny Fuerst, Governor ................ governor@cathat.net Opinions stated in this message are mine and mine alone, and do not necessarily represent those of my ISP/employer! .period. ------------------------------------------------------------ 'I Desire Compassion, and not a Sacrifice.' 'La Vida E Bella' ____________________________ Random Quote: If you eat a live frog in the morning, nothing worse will happen to either of you for the rest of the day. johnny@cathat.net From chuqui@plaidworks.com Wed Nov 1 04:21:14 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 31 Oct 2000 20:21:14 -0800 Subject: [Mailman-Developers] Get listed. :) In-Reply-To: <14847.37098.537126.455758@anthem.concentric.net> References: <14847.37098.537126.455758@anthem.concentric.net> Message-ID: At 10:41 PM -0500 10/31/00, Barry A. Warsaw wrote: >I've got >Python.Org and lists.apple.com (if that's cool with you Chuq ;). >Please send any others you'd like to include to me directly. It's cool with Apple. Also, fi you would, www.hockeyfanz.com, the Other Me. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From chuqui@plaidworks.com Wed Nov 1 04:21:14 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 31 Oct 2000 20:21:14 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Get listed. :) In-Reply-To: <14847.37098.537126.455758@anthem.concentric.net> References: <14847.37098.537126.455758@anthem.concentric.net> Message-ID: At 10:41 PM -0500 10/31/00, Barry A. Warsaw wrote: >I've got >Python.Org and lists.apple.com (if that's cool with you Chuq ;). >Please send any others you'd like to include to me directly. It's cool with Apple. Also, fi you would, www.hockeyfanz.com, the Other Me. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. ------------------------------------------------------ Mailman-Users maillist - Mailman-Users@python.org http://www.python.org/mailman/listinfo/mailman-users From chuqui@plaidworks.com Wed Nov 1 04:31:24 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 31 Oct 2000 20:31:24 -0800 Subject: [Mailman-Developers] mailman-developers problem. Message-ID: Barry, we have a problem with mailman-developers. I responded to your mail, and my response went to -users and -developers, and I got THREE copies. I got one from -users. I got one from -developers. and then I got a third copy of the -users mail, resent to -developers. looks like someone on the list is doing something stupid: To: catholic.org@catholicnet.org.uk Received: from dinsdale.python.org (dinsdale.cnri.reston.va.us [132.151.1.21]) by webmail6.catholic.org (8.10.1/8.10.1) with SMTP id eA14OAQ18572 for ; Wed, 1 Nov 2000 04:24:11 GMT Received: from dinsdale.python.org (localhost [127.0.0.1]) by dinsdale.python.org (Postfix) with ESMTP id 0C2981CE17; Tue, 31 Oct 2000 23:24:11 -0500 (EST) the third copy is routing through that address. Can you please shoot this person? the catholic.org address seems to be hosed. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From barry@wooz.org Wed Nov 1 04:39:58 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Tue, 31 Oct 2000 23:39:58 -0500 (EST) Subject: [Mailman-Developers] mailman-developers problem. References: Message-ID: <14847.40606.557263.702033@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> Barry, we have a problem with mailman-developers. I responded CVR> to your mail, and my response went to -users and -developers, CVR> and I got THREE copies. CVR> I got one from -users. I got one from -developers. and then I CVR> got a third copy of the -users mail, resent to -developers. CVR> looks like someone on the list is doing something stupid: | To: catholic.org@catholicnet.org.uk Received: from | dinsdale.python.org (dinsdale.cnri.reston.va.us | [132.151.1.21]) | by webmail6.catholic.org (8.10.1/8.10.1) with SMTP id | eA14OAQ18572 for ; Wed, 1 Nov 2000 04:24:11 | GMT | Received: from dinsdale.python.org (localhost [127.0.0.1]) | by dinsdale.python.org (Postfix) with ESMTP | id 0C2981CE17; Tue, 31 Oct 2000 23:24:11 -0500 (EST) CVR> the third copy is routing through that address. Can you CVR> please shoot this person? the catholic.org address seems to CVR> be hosed. Thomas noticed this earlier and thought it might be a Mailman bug. I couldn't reproduce it ('natch :), and I deleted the messages so couldn't look at them closely. I got a dup on my last posting though. Now why would he do that? :/ Anyway, I've disabled that account. Thanks, -Barry From chuqui@plaidworks.com Wed Nov 1 05:56:40 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 31 Oct 2000 21:56:40 -0800 Subject: [Mailman-Developers] mailman-developers problem. In-Reply-To: <14847.40606.557263.702033@anthem.concentric.net> References: <14847.40606.557263.702033@anthem.concentric.net> Message-ID: At 11:39 PM -0500 10/31/00, Barry A. Warsaw wrote: > >Now why would he do that? :/ Anyway, I've disabled that account. most commonly, it's an attempt at a vacation bot that failed miserably. less commonly, it's simply a broken mailer. And once in a while, I've seen someone do that because they think it'll make us unsubscribe them -- now why they don't simply unsubscribe, I dunno, but people are illogical, unlike computers. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From Nigel.Metheringham@VData.co.uk Wed Nov 1 10:10:47 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Wed, 01 Nov 2000 10:10:47 +0000 Subject: [Mailman-Developers] Slight oddity on ubsubscription Message-ID: Here's a message I got from one of my subscribers - makes a reasonable point [I have edited the address in there to prevent it being exposed]. Nigel. So, I see that an old form of my address is subscribed to the announce list. I look for a way to change the address, don't see it, so unsubscribe. On the unsubscription results page, I see: -----------------------------< cut here >------------------------------- Exim-announce Unsubscribe Results You have been unsubscribed. Continue to edit your personal options. -----------------------------< cut here >------------------------------- The last four words there are a link; following that brings me to a page stating: -----------------------------< cut here >------------------------------- Error exim-announce: No such member 'removed@to.protect.guilty'. -----------------------------< cut here >------------------------------- Perhaps someone who's unsubscribed shouldn't be prompted to continue editting? Ah well -- I followed it because I was worried that cruft might have been left behind. I've at least had some reassurance about that. :^) -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From roger@infomed.sld.cu Wed Nov 1 19:32:10 2000 From: roger@infomed.sld.cu (Roger Pena) Date: Wed, 1 Nov 2000 14:32:10 -0500 (CST) Subject: [Mailman-Developers] Get listed. :) In-Reply-To: <14847.37098.537126.455758@anthem.concentric.net> Message-ID: On Tue, 31 Oct 2000, Barry A. Warsaw wrote: > > I'm putting together the on-line documentation for the 2.0 release and > I'd like to include a list of sites that run Mailman. I've got > Python.Org and lists.apple.com (if that's cool with you Chuq ;). > Please send any others you'd like to include to me directly. if you like you can include, redhat inc samba surceforge linux.cu :-) i sure that other important projects are using mailman as the mail list server thanks roger > > Thanks, > -Barry > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers > From barry@wooz.org Thu Nov 2 23:08:51 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Thu, 2 Nov 2000 18:08:51 -0500 (EST) Subject: [Mailman-Developers] Fix (I think) for duplicates problem Message-ID: <14849.62467.911275.369342@anthem.concentric.net> The last followup to bug #117015 by deejster provided what I think is the critical clue to the strange duplication problem. Given that recipe I was indeed able to reproduce the bug: - a message gets held for whatever reason - the message is approved, but there are smtp errors during delivery - the message gets re-queued, but with a file name composed of the old filebase, not the new filebase. - fwap! you've now got a duplicate of the original message Here's a fix for this problem. It removes the filebase value from the original msgdata, so that this will be recalculated when the file is re-queued after approval. For those of you seeing duplicates, please apply this patch and let me know if it fixes your dup problem. This is important enough to spin a second release candidate. -Barry Index: ListAdmin.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/ListAdmin.py,v retrieving revision 1.46 diff -u -r1.46 ListAdmin.py --- ListAdmin.py 2000/10/10 06:33:31 1.46 +++ ListAdmin.py 2000/11/02 23:05:57 @@ -208,6 +208,12 @@ return LOST msg = Message.Message(fp) msgdata['approved'] = 1 + # Calculate a new filebase for the approved message, otherwise + # delivery errors will cause duplicates. + try: + del msgdata['filebase'] + except KeyError: + pass # Queue the file for delivery by qrunner. Trying to deliver the # message directly here can lead to a huge delay in web # turnaround. From NeilK@ActiveState.com Thu Nov 2 23:48:08 2000 From: NeilK@ActiveState.com (Neil K) Date: Thu, 2 Nov 2000 15:48:08 -0800 Subject: [Mailman-Developers] RE: [Mailman-Users] Fix (I think) for duplicates problem In-Reply-To: <14849.62467.911275.369342@anthem.concentric.net> Message-ID: Thanks Barry. We figured out our problem, which I think is different from what you are talking about. Details: - our aborted upgrade to mailman 2.0rc1 left behind a few files. - somehow these files interfered with normal exception handling. - when mailman decides a message must be held for moderation, it sends the user a "held for moderation" message and raises an special exception. - but this exception was no longer handled appropriately, so it got passed all the way up to qmail. - qmail therefore thought the delivery was a failure, and would schedule another attempt. And so on. Deleting the leftover files from 2.0rc1 seems to have fixed it. This was certainly a very bizarre interaction... I'm not sure if a patch to mailman is appropriate. Perhaps the action of sending the 'held for moderation', or similar, email, should be deferred as late as possible. That way if there are problems, at least it won't be sent out an infinite number of times. Or alternatively, close the connection to the MTA sooner. -- Neil Kandalgaonkar Web Application Developer, ActiveState From gietl@gietl.com Fri Nov 3 14:07:44 2000 From: gietl@gietl.com (Andreas Gietl) Date: Fri, 03 Nov 2000 15:07:44 +0100 Subject: [Mailman-Developers] forever pending subscriptions Message-ID: <3A02C6B0.2D82FC79@gietl.com> Hi. I'm using mailman on a lot of different machines with the Exim MTA in version 3.16 and everything worked fine with beta2. In the last weeks i tried a few times to upgrade to beta6 or rc1. But this always ends up in a very strange behavior. Since i am not the only one who reported this to the mailman-users-list and got no answer i also send this mail to the developers list, because i think it may be a bug. The Following happens: After upgrading to rc1 without any errors from make install mailman still works fine, but it does no subscriptions any longer. A user who signs up in the listinfo page gets his confirmation mail and after he/she confirmed the message (and the MTA delivered it w/o error) nothing happens. The subscribe log still shows the user as pending and nothing happens. Any suggestion would be appreciated. thx andreas -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From barry@wooz.org Fri Nov 3 18:19:41 2000 From: barry@wooz.org (barry@wooz.org) Date: Fri, 3 Nov 2000 13:19:41 -0500 (EST) Subject: [Mailman-Developers] RE: [Mailman-Users] Fix (I think) for duplicates problem References: <14849.62467.911275.369342@anthem.concentric.net> Message-ID: <14851.445.927341.896264@anthem.concentric.net> >>>>> "NK" == Neil K writes: NK> We figured out our problem, which I think is different from NK> what you are talking about. Oh, whoops! I missed this in my inbox. Looks like you've largely solved the problem, so ignore my last message. NK> - our aborted upgrade to mailman 2.0rc1 left behind a few NK> files. I'd still /really/ like to know what problems you all had with rc1. It's a release candidate, so if there are bugs, better to catch them now than before 2.0 final goes out! NK> This was certainly a very bizarre interaction... I'm not sure NK> if a patch to mailman is appropriate. Probably not. Rolling back is a bad idea (and definitely "unsupported")! -Barry From bernhard@intevation.de Fri Nov 3 18:43:03 2000 From: bernhard@intevation.de (Bernhard Reiter) Date: Fri, 3 Nov 2000 19:43:03 +0100 Subject: [Mailman-Developers] Direct Python access tutorial wanted Message-ID: <20001103194303.H25116@cheops.usf.Uni-Osnabrueck.DE> --df+09Je9rNq3P+GE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Mailman developers, just noticed that the minimal instructions for list server administrators to mass manipulate mailinglists directly have been vanished from the mailman README file in the jump to version 2. (last cvs revision containg the advice is README rev1.45) As I considered it quite useful my suggestion is to add it again to the mailman documentation. Or is it not possible or advisable to do=20 this kind of list surgery anymore? =09 Regards, Bernhard ps: please cc relevant replies to me as I am not subscribe to the list --=20 Professional Service around Free Software (intevation.net) = =20 The FreeGIS Project (freegis.org) Association for a Free Informational Infrastructure (ffii.org) --df+09Je9rNq3P+GE Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.3 (SunOS) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjoDBzYACgkQh9ag3dpKERbKiACfV8WyHlNZ2MGLD/DxNQfijt24 GxgAnineH6Fqo6IUUzILNuuG8z7PUVZ9 =xvuO -----END PGP SIGNATURE----- --df+09Je9rNq3P+GE-- From Dan.Mick@west.sun.com Fri Nov 3 20:35:37 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Fri, 03 Nov 2000 12:35:37 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> Message-ID: <3A032199.67D1CA2E@west.sun.com> I just tested this again yesterday, and it worked for me. Andreas Gietl wrote: > > Hi. > > I'm using mailman on a lot of different machines with the Exim MTA in > version 3.16 and everything worked fine with beta2. > > In the last weeks i tried a few times to upgrade to beta6 or rc1. > But this always ends up in a very strange behavior. Since i am not the > only one who reported this to the mailman-users-list and got no answer i > also send this mail to the developers list, because i think it may be a > bug. > > The Following happens: > > After upgrading to rc1 without any errors from make install mailman > still works fine, but it does no subscriptions any longer. > A user who signs up in the listinfo page gets his confirmation mail and > after he/she confirmed the message (and the MTA delivered it w/o error) > nothing happens. The subscribe log still shows the user as pending and > nothing happens. > > Any suggestion would be appreciated. > > thx > > andreas > -- > andreas gietl > gietl internet services > fon +49 9402 2551 > fax +49 9402 2604 > mobile +49 171 60 70 008 > gietl@gietl.com > > ############################################ > # Das Handbuch sagt, das Programm benötige # > # Windows 95 oder besser. Also habe ich # > # Linux installiert! # > ############################################ > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users From barry@wooz.org Fri Nov 3 21:15:50 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Fri, 3 Nov 2000 16:15:50 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> Message-ID: <14851.11014.397538.19202@anthem.concentric.net> >>>>> "AG" == Andreas Gietl writes: AG> After upgrading to rc1 without any errors from make install AG> mailman still works fine, but it does no subscriptions any AG> longer. A user who signs up in the listinfo page gets his AG> confirmation mail and after he/she confirmed the message (and AG> the MTA delivered it w/o error) nothing happens. The subscribe AG> log still shows the user as pending and nothing happens. >>>>> "DM" == Dan Mick writes: DM> I just tested this again yesterday, and it worked for me. Me too. When you do a "bin/dumpdb data/pending_subscriptions.db" do you still see the pending subscription in there? Do you still see the confirmation message in qfiles? If so, are there any obvious errors in logs/error? Does it have the expected Subject: header (i.e. at the end of the line, "request xxxxxx" where those x's are the numbers). -Barry From mtran@bmedesign.com Fri Nov 3 23:31:17 2000 From: mtran@bmedesign.com (mike tran) Date: Fri, 03 Nov 2000 16:31:17 -0700 Subject: [Mailman-Developers] mailman installation error Message-ID: <3A034AC5.4191FD96@bmedesign.com> After I did ./configure --with-cgi-gid=nobody --with-mail-gid=mailman and make install. Then I ran check_perms. It keep telling me that it cannot find the module "paths" in line 19 of check_perms. Any suggestions is appricated. I am using RH 6.2 and the latest Apache with python 1.5.2 From claw@kanga.nu Sat Nov 4 02:13:32 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 03 Nov 2000 18:13:32 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail Message-ID: <7154.973304012@kanga.nu> <> On Fri, 3 Nov 2000 17:01:01 -0800 Marc MERLIN wrote: > On Fri, Nov 03, 2000 at 04:56:33PM -0800, J C Lawrence wrote: >> Yup, that's what I mean by pass-thru. > Ok, I wasn't sure. No worries. >> of having Exim do that a loooong time back but I don't know if >> anything ever happened there (there are obvious timeout and >> performance issues when delivering to slow/unresponsive sites >> (think about behavioural changes mid-negotiation)). > I see. So the message gets spooled as a backup and erased right > afterwards if the current process was able to deliver it I > suppose. Quite. Note that this leaves with the same disk IO that not doing pass-thru does. The only time pass-thru gains you anything is when you can hand the message off very close to as fast as, or faster than you can receive it. In many cases I could see this being a gain given free queue runners and minimal queue runner negotiation overhead or complexity. >> To deal with the physical IO problems, the very large volume mail >> server companies I've dealt with and talked to (eg my last >> client, Critical Path) use solid state disks for their mail >> queues (in CP's case under a modified QMail, soon to be replaced >> by NPLex). Its a bloody expensive solution to be sure, but its >> also a fast one. > It can come down to that, but that's throwing money at hardware > instead of fixing mailman's deficiencies. True. Mailman is the real bottleneck here. A simple, and computationally and code-wise cheap solution would be to split the queues per list. Simply have each list have a private outbound queue area (this may be done already, I haven't looked at v2 in detail and am short of time right now). Then rather than threading, simply have per-list queue locks. The qrunner then forks N children (configurable) which proceed to attempt to empty each queue in order, each one locking its queue as it gets it (such that the others pass over it) etc. Thus deliveries to the MTA are parallelised. Its a fairly light weight code change (private queues and a child forking qrunner) but it skips all the overhead of threading and fancy locking schemse. Further, as the number of posts per list queue is likely to be low, the iteration rate, and therefore the lock period (for other posts to join a given list's queue will be small. If Mailman v2 currently uses private queue (dunno) then your likely problem is twofold: the thrash rate of walking 10K private queues plus the MTA receipt problem. This problem is mostly unique to the very large number of lists your run. A possibly better tack there (given a reasonbly high frequency qrunner polling rate) would probably be to store the queue entries in hash trees (hash by list or hash by post) and then go from there. Adjust the hash for say N buckets (where N is the total number of parallel qrunners you want). This has the advantage of minimising disk and directory thrashing. The disadvantage is that the code impact would be correspondingly large. > On Fri, Nov 03, 2000 at 04:57:11PM -0800, J C Lawrence wrote: >> > Yeah, that's just what I was saying here. ...etc >> >> BTW: Did you inbtend this to be off list? > Yeah, it's not mailman related, it's exim stuff, thus off topic > :-) This conversely whould probably go back to the list. Mind if I cross it? -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From gietl@gietl.com Sat Nov 4 12:58:10 2000 From: gietl@gietl.com (Andreas Gietl) Date: Sat, 04 Nov 2000 13:58:10 +0100 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> Message-ID: <3A0407E2.42ABB0AF@gietl.com> "Barry A. Warsaw" wrote: > > >>>>> "AG" == Andreas Gietl writes: > > AG> After upgrading to rc1 without any errors from make install > AG> mailman still works fine, but it does no subscriptions any > AG> longer. A user who signs up in the listinfo page gets his > AG> confirmation mail and after he/she confirmed the message (and > AG> the MTA delivered it w/o error) nothing happens. The subscribe > AG> log still shows the user as pending and nothing happens. > > >>>>> "DM" == Dan Mick writes: > > DM> I just tested this again yesterday, and it worked for me. > > Me too. > > When you do a "bin/dumpdb data/pending_subscriptions.db" do you still > see the pending subscription in there? Yeah, the subscription is still in there! Do you still see the > confirmation message in qfiles? Yeah, the confirmation message is in there! If so, are there any obvious errors > in logs/error? nothing. i of course checked that! Does it have the expected Subject: header (i.e. at the > end of the line, "request xxxxxx" where those x's are the numbers). > Yeah. it does have. Do you need any further information? > -Barry > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From barry@wooz.org Sat Nov 4 15:30:58 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Sat, 4 Nov 2000 10:30:58 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail References: <20001103113539.S22363@marc.merlins.org> Message-ID: <14852.11186.967453.490195@anthem.concentric.net> >>>>> "MM" == Marc MERLIN writes: MM> [I am not Ccing mailman-developers as this is not encouraged, MM> but if someone on both lists thinks it should be forwarded MM> there, please feel free] I'm going to answer some of the other questions as best I can, and then I propose to move this discussion to mailman-developers so we can design a proper fix over there. MM> The problem is due to qrunner being single threaded by default MM> and having a global lock. Because some mailing lists have MM> subscribers in domains where DNS is slow and unreliable, the MM> MTA will hang on those rcpt to until DNS resolves or timeouts, MM> and qrunner won't be done in time. After that, it's all MM> downhill from there, more mail queues up, qrunner falls even MM> further behind, etc, etc... So one of the problems is that the handoff between Mailman and the MTA is synchronous with some aspect of the MTA's delivery to the remote site, namely dns lookup. One of the first things you need to do is break this synchrony, either by improving the dns lookup on the MTA side or putting the MTA in asynchronous mode for local message acceptance. Basically you want Mailman to just say "here, don't process this yet, just drop it in your outgoing queue and deal with it later." This might be related to DSN (delivery status notification); there's two DSN RFC (don't have the numbers handy right now), one that talks about the mime bounce format and another that talks about esmtp extensions for synchronous notification of delivery failures. You /don't/ want that! From the followups, it sounds like Exim can be configured to take local delivery asynchronously, and I believe that that is how Postfix works by default. Dunno about Qmail or Sendmail, but I have to believe it's possible to put them in those modes. Now, to sketch out how I think Mailman ought to work for 2.1, and it would not be too hard to whip something up with the 2.0 architecture (ob plug: we might be able to arrange some consulting gigs with Digital Creations if necessary). First, I'd write a long-running process based around asynchat/asyncore that was essentially our own bulk mailer. The async* modules are standard Python modules which make possible select-based high performance servers. They aren't multithreaded, and do not need to be because these servers are primarily i/o bound. When i/o blocks on one channel blocks, another one picks up and works for a while. So this new process, let's call it `bulkmail'. Bulkmail would have one (probably) unix socket open to take new outgoing messages from qrunner. It'd probably write them to disk as a backup so failures don't drop messages. I'm thinking it would then sort recipients based on domains, and then it would start resolving MX records, caching the results. There'd be bins for each MX containing pointers to the messages that need to be delivered to that MX. As more messages came in for that MX, they'd be dropped at the end of the bin. Once a connetion to the MX is established, bulkmail would then just start delivering messages to it until the bin was emptied. Any i/o blocks in any of the processes will allow async* to switch to a different delivery channel. We may need to do some explicit channel management to make sure some are not starved. We'd have to have a watchdog to make sure bulkmail is running, and I'm sure there are other issues to work out, but I've gotta run. I think this will work better than the current SMTPDirect threading stuff. Thoughts? -Barry From barry@wooz.org Sat Nov 4 15:35:23 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Sat, 4 Nov 2000 10:35:23 -0500 (EST) Subject: [Mailman-Developers] mailman installation error References: <3A034AC5.4191FD96@bmedesign.com> Message-ID: <14852.11451.138206.842858@anthem.concentric.net> >>>>> "mt" == mike tran writes: mt> After I did ./configure --with-cgi-gid=nobody mt> --with-mail-gid=mailman and make install. Then I ran mt> check_perms. It keep telling me that it cannot find the mt> module "paths" in line 19 of check_perms. Any suggestions is mt> appricated. I am using RH 6.2 and the latest Apache with mt> python 1.5.2 Look in /home/mailman/bin. If there's no paths.py file in there, your installation failed somehow. -Barry From barry@wooz.org Sat Nov 4 15:45:53 2000 From: barry@wooz.org (barry@wooz.org) Date: Sat, 4 Nov 2000 10:45:53 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> Message-ID: <14852.12081.57421.807051@anthem.concentric.net> >>>>> "AG" == Andreas Gietl writes: >> Do you still see the >> confirmation message in qfiles? AG> Yeah, the confirmation message is in there! >> If so, are there any obvious errors >> in logs/error? AG> nothing. i of course checked that! >> Does it have the expected Subject: header (i.e. at the >> end of the line, "request xxxxxx" where those x's are the >> numbers). AG> Yeah. it does have. AG> Do you need any further information? Yes, do a bin/dumpdb qfiles/.db and look for the `pipeline' entry. That should show us where things are getting stuck. -Barry From claw@kanga.nu Sat Nov 4 17:06:39 2000 From: claw@kanga.nu (J C Lawrence) Date: Sat, 04 Nov 2000 09:06:39 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: Message from barry@wooz.org (Barry A. Warsaw) of "Sat, 04 Nov 2000 10:30:58 EST." <14852.11186.967453.490195@anthem.concentric.net> References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> Message-ID: <20435.973357599@kanga.nu> On Sat, 4 Nov 2000 10:30:58 -0500 (EST) Barry A Warsaw wrote: > So this new process, let's call it `bulkmail'. Bulkmail would > have one (probably) unix socket open to take new outgoing messages > from qrunner. It'd probably write them to disk as a backup so > failures don't drop messages. I'm thinking it would then sort > recipients based on domains, and then it would start resolving MX > records, caching the results. There'd be bins for each MX > containing pointers to the messages that need to be delivered to > that MX. As more messages came in for that MX, they'd be dropped > at the end of the bin. > Once a connetion to the MX is established, bulkmail would then > just start delivering messages to it until the bin was emptied. > Any i/o blocks in any of the processes will allow async* to switch > to a different delivery channel. We may need to do some explicit > channel management to make sure some are not starved. Ouch. I really don't like this idea. For one, it make the processing of oubound mail from a list server unique to the rest of the mail system both on the local host, and the local 'net (consider smarthosts, intermittent connectivity, local domain based mail routing, firewalling etc). It also places needs for a long running process which then needs to be resistant and tolerant to unexpected/frequent shutdowns (laptops, home machines, etc). As discussed previously amongst Chuq, Nigel and I, the needs of large list server systems are rather different from the normal home hobbyest requirements, but are not compleatly alien. However, the needs of very large list installations (cf ListServ, Egroups, or SourceForge) are rather different yet again. I'm not convinced of the value in beating on Mailman to support the (comparitively rare if high profile) very large installations when the current (much larger and more common Mailman-wise) mid-size realm still needs attention. Certainly, such changes should not detract from Mailman's current level of suitability for smaller installations. Now that said, more intelligent handling of Mailman's outbound queue and its hand off to the local or a remote smarthost-like MTA could stand considerable improvement. I've posted a couple ideas which would require relatively small code changes and which should improve the situation. However, both ideas were deliberately attempting to minimise code impact. What would be a really good approach without concern for code impact? I suspect a modified form of the hash tree for queue storage (cf QMail's implementation minus the silly (for this use) inode specifics) with a slightly perverted form of your (Barry's) long running bulkmailer to process that hash queue. I'd tend to make the bulkmailer actually an intermittently running item to help support for intermittently connected nodes. Say something like: Cron launches the bulkmailer. The bulkmailer forks N children processing the queue. The bulkmailer exits upon an empty queue. Should cron launch a new bulkmailer when the prevvious incarnation hasn't exited yet, the new instance merely exits immediately. Locking for the above is fairly simple. Standard IPCs can be used for the instance collision checks. Locking on the hash queues could be a bit intereting from a portability and performance vantage given the fact that the list side will be attemptiong to deliver into the same tree at the same time that deliveries are happening (no more lock collisions please) -- which pretty much requires that locking be on the queue-entry level rather than the hash bucket level. Not rocket science, just a bit finnicky. Will this handle SourceForge? Probably. Certainly given a local MTA that does no checking on mail from localhost (and possibly which immeidately respools out to a dedicated smarthost) it would significantly improve the current state. Enough? Dunno. I'd need more specific data on their current setup, metrics etc. -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From forsberg@lysator.liu.se Sat Nov 4 17:27:15 2000 From: forsberg@lysator.liu.se (Erik Forsberg) Date: Sat, 4 Nov 2000 18:27:15 +0100 (CET) Subject: [Mailman-Developers] Patch #102268 Message-ID: <14852.18163.486910.710242@j218.ryd.student.liu.se> Hi! I submitted a patch (at sourceforge) earlier today, #102268. It's basically a two-line-change to MailMan/Archiver/HyperArch.py in 2.0rc1 that makes the Content-Transfer-Encoding all lowercase. Hopefully I produced a working patch-file.. In my humble opinion that patch is small and uncomplicated enough to fit into the release version, but I'll leave that decision to the Cabal. Anyway, the reason I wrote the patch was that some mails in a list I have didn't decode the QP the right way because the Content-Transfer-Encoding was QUOTED-PRINTABLE (all uppercase). The person writing the mails is using dtmail. So far, the patch comment is quite the same as this mail. The reason I write this mail is that I actually found some text in RFC 1521 that supports my patch fully. Here: ---snip--- The Content-Transfer-Encoding field is designed to specify an invertible mapping between the "native" representation of a type of data and a representation that can be readily exchanged using 7 bit mail transport protocols, such as those defined by RFC 821 (SMTP). This field has not been defined by any previous standard. The field's value is a single token specifying the type of encoding, as enumerated below. Formally: encoding := "Content-Transfer-Encoding" ":" mechanism mechanism := "7bit" ; case-insensitive / "quoted-printable" / "base64" / "8bit" / "binary" / x-token These values are not case sensitive. That is, Base64 and BASE64 and bAsE64 are all equivalent. An encoding type of 7BIT requires that the body is already in a seven-bit mail-ready representation. This is the default value -- that is, "Content-Transfer-Encoding: 7BIT" is assumed if the Content-Transfer-Encoding header field is not present. ---snap--- Note the the words "not case sensitive" :-). And guess if I was confused by the fact that the archive creates new files every time you run the 'arch' utility. After testing my code, I reloaded the message in my browser, and nothing happended. Very confusing until I understood that there was a new file with the message in it.. I guess the archiver behaves that way so search engines won't get confused if you remove a message or something? \EF -- Erik Forsberg http://www.lysator.liu.se/~forsberg/ PGP fingerprint = 37 1B D7 9E 97 8C EF 39 0E DE 08 E8 99 0C 5E A9 From chuqui@plaidworks.com Sat Nov 4 18:29:15 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Sat, 4 Nov 2000 10:29:15 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: <20435.973357599@kanga.nu> References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <20435.973357599@kanga.nu> Message-ID: > > Once a connetion to the MX is established, bulkmail would then >> just start delivering messages to it until the bin was emptied. >> Any i/o blocks in any of the processes will allow async* to switch >> to a different delivery channel. We may need to do some explicit >> channel management to make sure some are not starved. > >Ouch. I really don't like this idea. Neither do I. This is actually something that I've looked at long and hard in my non-mailman server work. After a fair amount of work and research, I finally came to the conclusion that you are MUCH better off letting the MTA do the MTA's work, and letting the MLM do the MLM work, and once you make the decision that the MLM has to *also* become an MTA, you're doing down a road you don't want to travel. Sendmail, for instance, has many years experience optimizing delivery as an MTA. It's a complex, nasty business with lots of subtleties. If you're building a list manager, how much work would you need to do to get a private delivery system that's as well tuned and efficient as sendmail already is? ditto all of the other MTAs. I've built some prototype systems to test this. Even though (in theory) you're adding a layer of delivery and other overhead, it's very difficult to come even close to the performance a tuned MTA can give you -- and you're writing a lot of code to do it. One of the systems I've been investigating, for instance, would do 100% customized mail driven by a template document and pulling data out of a database -- with a design parameter of up to 10 million deliveries. the goal is at least 500K deliveries an hour, preferably double that. Right now, on a system with a sendmail 8.9 base and a non-optimized delivery tool, I'm doing 400-450K/hour. I expect to see a nice addition when I move to sendmail 8.10.x in a week or so. this is on a Sun E250, FWIW, with the sendmail queues living in a ram disk. Good sized hardware, but not particularly big or fast hardware. Instead of reinventing the MTA wheel, I think we're much better off coming up with an MTA -> MLM interface that's very flexible and highly configurable (most especially in how to deliver and how much to parallelize the infeed to the MLM), and then focus on how to tune the MTA and MLM through documentation. Splitting the inbound and outbound queue would be my first thing here, and probably split bounces into a third queue. That's a pretty quick, easy optimization that makes sure the end user sees fast response without being bogged down by deliveries, and that's a huge perception issue. Then focus on parallelizing the delivery from mailman into the MTA, and make that configurable so each admin can tune it to their system and needs. >As discussed previously amongst Chuq, Nigel and I, the needs of >large list server systems are rather different from the normal home >hobbyest requirements, but are not compleatly alien. However, the >needs of very large list installations (cf ListServ, Egroups, or >SourceForge) are rather different yet again. This is a basic reality -- things don't scale. Or worse, they scale for a while, and then you need to switch paradigms. I found that one out the hard way. If someone wants a rhetoric on how to scale mail list servers infinitely, I'd be happy to explain how, since I've had to develop an architecture to do so. the nice thing is, it can be done without exceptional engineering hassles -- but it's not just adding another daemon or a faster CPU (although those are solutions for parts of it, just not ALKWAYs the solutions) > I'm not convinced of >the value in beating on Mailman to support the (comparitively rare >if high profile) very large installations when the current (much >larger and more common Mailman-wise) mid-size realm still needs >attention. Certainly, such changes should not detract from >Mailman's current level of suitability for smaller installations. I think we can build a Mailman that does this, at least for, oh, 95% of the universe out there, and the other 5% are going to have custom solutions anyway (or should!). What we don't want to do is screw up Mailman for the "typical" user to make it work for the big site; but we also don't want Mailman to get a reputation as a "small server only" system, because it'll cause people to reject it in implementations. Fortunately, I don't think you need to do that. It just needs some tweaking. >support for intermittently connected nodes. Say something like: > > Cron launches the bulkmailer. > The bulkmailer forks N children processing the queue. > The bulkmailer exits upon an empty queue. > Should cron launch a new bulkmailer when the prevvious incarnation > hasn't exited yet, the new instance merely exits immediately. > >Locking for the above is fairly simple. Standard IPCs can be used >for the instance collision checks. Locking on the hash queues could >be a bit intereting from a portability and performance vantage given >the fact that the list side will be attemptiong to deliver into the >same tree at the same time that deliveries are happening (no more >lock collisions please) -- which pretty much requires that locking >be on the queue-entry level rather than the hash bucket level. Not >rocket science, just a bit finnicky. > >Will this handle SourceForge? Probably. On reasonable hardware, definitely. That's basically how my current custom system works. right now, the number of parallel infeeds from mailman is 1. I'm willing to bet the delivery MTA is basically idle and bored. By moving to parallel infeeds, you can stoke the MTA up to speed, and the trick is for each site to figure out what number of parallel infeeds will work keeping the queues full for the MTA to stay busy without overloading them and causing the MTA to thrash. That's simply a case of tuning and modelling. And simply allowing "N" infeed threads to the MLM will solve Sourceforge's problem and pretty much everyone else's, without having to get into the MTA business, where the best we can really hope is to be "as good" as the real MTA. so my recommendation is: 1) split the current qfiles into three queues: inbound, outbound and bounce 2) parallelize the outbound queue into "N" configurable delivery threads. 3) work on documentation on how to tune this for maximum perforamnce with major MTAs, and how to tune MTAs for maxiumum performance. That's a set of pretty easy updates, no technological miracles or black boxes, and solves all but the worst problems someone running Mailman is likely to see. And for sites this *doesn't* solve, it's either because they're doing the 5 pounds in a 1 pound bag thing, or they probably need to start hiring people like us to custom build someething. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From Dan.Mick@west.sun.com Sat Nov 4 23:36:18 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Sat, 04 Nov 2000 15:36:18 -0800 Subject: [Mailman-Developers] Patch #102268 References: <14852.18163.486910.710242@j218.ryd.student.liu.se> Message-ID: <3A049D72.22801714@west.sun.com> Erik Forsberg wrote: > > Hi! > > I submitted a patch (at sourceforge) earlier today, #102268. It's basically a > two-line-change to MailMan/Archiver/HyperArch.py in 2.0rc1 that makes > the Content-Transfer-Encoding all lowercase. Hopefully I produced a > working patch-file.. > > In my humble opinion that patch is small and uncomplicated enough to > fit into the release version, but I'll leave that decision to the > Cabal. > > Anyway, the reason I wrote the patch was that some mails in a list I > have didn't decode the QP the right way because the > Content-Transfer-Encoding was QUOTED-PRINTABLE (all uppercase). The > person writing the mails is using dtmail. So far, the patch comment is > quite the same as this mail. > > The reason I write this mail is that I actually found some text in RFC > 1521 that supports my patch fully. Here: > > ---snip--- > The Content-Transfer-Encoding field is designed to specify an > invertible mapping between the "native" representation of a type of > data and a representation that can be readily exchanged using 7 bit > mail transport protocols, such as those defined by RFC 821 (SMTP). > This field has not been defined by any previous standard. The field's > value is a single token specifying the type of encoding, as > enumerated below. Formally: > > encoding := "Content-Transfer-Encoding" ":" mechanism > > mechanism := "7bit" ; case-insensitive > / "quoted-printable" > / "base64" > / "8bit" > / "binary" > / x-token > > These values are not case sensitive. That is, Base64 and BASE64 and > bAsE64 are all equivalent. An encoding type of 7BIT requires that > the body is already in a seven-bit mail-ready representation. This > is the default value -- that is, "Content-Transfer-Encoding: 7BIT" is > assumed if the Content-Transfer-Encoding header field is not > present. > ---snap--- > > Note the the words "not case sensitive" :-). So, that means that a header which reads QUOTED-PRINTABLE is completely RFC-compliant, and if there's a mailreader or other utility that barfs on it, it's that mailreader's bug, not Mailman's. Or am I missing something? > And guess if I was confused by the fact that the archive creates new > files every time you run the 'arch' utility. After testing my code, I > reloaded the message in my browser, and nothing happended. Very > confusing until I understood that there was a new file with the > message in it.. > > I guess the archiver behaves that way so search engines won't get > confused if you remove a message or something? No, arch is there to rebuild archives from .mbox files; it's not there to add messages. Hence its comment: Use this command to rebuild the archives for a mailing list. From forsberg@lysator.liu.se Sat Nov 4 23:40:49 2000 From: forsberg@lysator.liu.se (Erik Forsberg) Date: Sun, 5 Nov 2000 00:40:49 +0100 (CET) Subject: [Mailman-Developers] Patch #102268 In-Reply-To: <3A049D72.22801714@west.sun.com> References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <3A049D72.22801714@west.sun.com> Message-ID: <14852.40577.83440.45618@j218.ryd.student.liu.se> > >So, that means that a header which reads QUOTED-PRINTABLE is completely >RFC-compliant, and if there's a mailreader or other utility that barfs on >it, it's that mailreader's bug, not Mailman's. > >Or am I missing something? Yes. Mailman didn't archive correctly because it didn't understand the encoding. Here's the code that caused the problem: (Mailman/Archiver/HyperArch.py) if self.charset is None or self.cenc != "quoted-printable": return null_to_space(string.join(body, "")) now what happends if self.cenc is "QUOTED-PRINTABLE"? The code that knows how to decode QP is never executed and the mail looks very ugly in the Web archive. My patch lowercases self.cenc where it's value is set. >> I guess the archiver behaves that way so search engines won't get >> confused if you remove a message or something? > >No, arch is there to rebuild archives from .mbox files; it's not there to add >messages. Hence its comment: > > Use this command to rebuild the archives for a mailing list. Try using arch on a archive that's already existing... \EF -- Erik Forsberg http://www.lysator.liu.se/~forsberg/ PGP fingerprint = 37 1B D7 9E 97 8C EF 39 0E DE 08 E8 99 0C 5E A9 From Dan.Mick@west.sun.com Sat Nov 4 23:47:24 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Sat, 04 Nov 2000 15:47:24 -0800 Subject: [Mailman-Developers] Patch #102268 References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <3A049D72.22801714@west.sun.com> <14852.40577.83440.45618@j218.ryd.student.liu.se> Message-ID: <3A04A00C.A4896940@west.sun.com> Erik Forsberg wrote: > Yes. Mailman didn't archive correctly because it didn't understand the > encoding. Here's the code that caused the problem: > My patch lowercases self.cenc where it's value is set. Ah, I see. I didn't read the patch, and misread your description of it. Sorry. > >> I guess the archiver behaves that way so search engines won't get > >> confused if you remove a message or something? > > > >No, arch is there to rebuild archives from .mbox files; it's not there to add > >messages. Hence its comment: > > > > Use this command to rebuild the archives for a mailing list. > > Try using arch on a archive that's already existing... Right, but that's "augment the archives", which is not what arch tries to do. If you want to rebuild, delete or move the existing archives, and rebuild, and you'll get the result you expect. From ocschwar@MIT.EDU Sun Nov 5 22:55:37 2000 From: ocschwar@MIT.EDU (The guy named after an Om Kalthoum song) Date: Sun, 05 Nov 2000 17:55:37 -0500 Subject: [Mailman-Developers] Mailman and GPG. Message-ID: <200011052255.RAA14775@alice-whacker.mit.edu> Greetings! I am wondering if anyone has begun an implementation of Gnu Provacy Guard email handling in Mailman, that would work along these lines: 1. mailman has its own keypair, and a keyring of the public keys of all mailing list subscribers. 2. A poster to a list will use GPG to encript his posting using mailman's public key, and then mailman sends out individually encripted messages or digests to each subscriber. Something like this existed (PGPdomo), but it vanished, and putting this functionality back in Majordomo is a Lovecraftian task. I may be itnerested in doing this as an exercise to learn Python. Has anyone done something like this? Also, is there a Python interface to GPG? Regards, -- Omri Schwarz --from somewhere on the net: float o=0.075,h=1.5,T,r,O,l,I;int _,L=80,s=3200;main(){for(;s%L|| (h-=o,T= -2),s;4 -(r=O*O)<(l=I*I)|++ _==L&&write(1,(--s%L?_ Are the mailman developers at all concerned by http://www.google.com/search?q=mailman-owner+reminder+password http://x66.deja.com/=dnc/getdoc.xp?AN=641175690 This is probably especially a problem with lists that were converted from another MLM, where there was no explicit "one address, one reader" assumption at the time the archive was subscribed. [note: 1. I'm not subscribed to the mailman mailing lists, nor do I use mailman except as a list subscriber and the maintainer of the rec.music.gaffa news2mail gateway; please cc me on any direct responses. 2. I looked through the archives and saw many discussions of passwords, but did not find this issue addressed; my apologies if I missed it. 3. Yes, the rec.music.gaffa gateway filters mailman admin messages, and has done so since a few hours after the first reminder appeared in the newsgroup. 4. No, I don't think this is a huge security issue, but it certainly does have some potential for minor mischief. 5. I may mention this (in passing) in a submission to comp.risks soon. ] -dan From claw@kanga.nu Mon Nov 6 07:23:35 2000 From: claw@kanga.nu (J C Lawrence) Date: Sun, 05 Nov 2000 23:23:35 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: Message from Chuq Von Rospach of "Sat, 04 Nov 2000 10:29:15 PST." References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <20435.973357599@kanga.nu> Message-ID: <4754.973495415@kanga.nu> On Sat, 4 Nov 2000 10:29:15 -0800 Chuq Von Rospach wrote: >> > Once a connetion to the MX is established, bulkmail would then >>> just start delivering messages to it until the bin was emptied. >>> Any i/o blocks in any of the processes will allow async* to >>> switch to a different delivery channel. We may need to do some >>> explicit channel management to make sure some are not starved. >> Ouch. I really don't like this idea. > Neither do I. This is actually something that I've looked at long > and hard in my non-mailman server work. After a fair amount of > work and research, I finally came to the conclusion that you are > MUCH better off letting the MTA do the MTA's work, and letting the > MLM do the MLM work, and once you make the decision that the MLM > has to *also* become an MTA, you're doing down a road you don't > want to travel. Agreed. There's also the simple aspect of the fact that optimising for the very large case (which needs the MLM-sepcific MTA supports ala ListServ) is near on pessimal for the small and medium cases. Spiffy domain based routing and near-target explosion just don't make a whole lot of sense for lists with a few hundred/thousand members. Asides from which, the optimisations for mail delivery to a massive number of targets and mass mail delivery to a comparitively constrained target set can be rather different. That's ugly territorry. We don't need to go there yet. > Instead of reinventing the MTA wheel, I think we're much better > off coming up with an MTA -> MLM interface that's very flexible > and highly configurable (most especially in how to deliver and how > much to parallelize the infeed to the MLM), and then focus on how > to tune the MTA and MLM through documentation. Agreed. There's not a whole lot of tuning you can do on SMTP hand-offs to the MTA (which is necessary as a feature to support non-localhost MTAs which is in turn needed by many sites), but some MTAs have divers command line options that can be useful in an MLM delivery world (peeking quickly at the Exim docs). > Splitting the inbound and outbound queue would be my first thing > here, and probably split bounces into a third queue. That's a > pretty quick, easy optimization that makes sure the end user sees > fast response without being bogged down by deliveries, and that's > a huge perception issue. Then focus on parallelizing the delivery > from mailman into the MTA, and make that configurable so each > admin can tune it to their system and needs. Without spending any time collecting metrics, it does look like the low hanging fruit is in handling the MLM->MTA handoff. Given that the distribution of posts across lists is often uneven (and unpredictable), and that the optimal parallelism (N) is fairly fixed for a given machine, any pattern which supported keeping N pipes stuffed throwing things at the MTA (with minimal locking) would seem fair. As for splitting the queues, yep, hadn't thought about that aspect, but that's an obvious contention area for an active list. >> As discussed previously amongst Chuq, Nigel and I, the needs of >> large list server systems are rather different from the normal >> home hobbyest requirements, but are not compleatly alien. >> However, the needs of very large list installations (cf ListServ, >> Egroups, or SourceForge) are rather different yet again. > This is a basic reality -- things don't scale. Or worse, they > scale for a while, and then you need to switch paradigms. I found > that one out the hard way. If someone wants a rhetoric on how to > scale mail list servers infinitely, I'd be happy to explain how, > since I've had to develop an architecture to do so. the nice thing > is, it can be done without exceptional engineering hassles -- but > it's not just adding another daemon or a faster CPU (although > those are solutions for parts of it, just not ALKWAYs the > solutions) We should really have lunch some time. I'm currently working down off San Tomas and Saratoga (startup, oddly enough next door to BeOpen). I'm wrapping up this contract this month and looking for next one, so time is a little tight here, but I'd love to get together for a chat. You over at the main Apple campus? >> I'm not convinced of the value in beating on Mailman to support >> the (comparitively rare if high profile) very large installations >> when the current (much larger and more common Mailman-wise) >> mid-size realm still needs attention. Certainly, such changes >> should not detract from Mailman's current level of suitability >> for smaller installations. > I think we can build a Mailman that does this, at least for, oh, > 95% of the universe out there, and the other 5% are going to have > custom solutions anyway (or should!). What we don't want to do is > screw up Mailman for the "typical" user to make it work for the > big site; but we also don't want Mailman to get a reputation as a > "small server only" system, because it'll cause people to reject > it in implementations. Fortunately, I don't think you need to do > that. It just needs some tweaking. We must stop agreeing like this. Its getting embarassing. > And for sites this *doesn't* solve, it's either because they're > doing the 5 pounds in a 1 pound bag thing, or they probably need > to start hiring people like us to custom build someething. Ungghhhh. -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Mon Nov 6 07:39:37 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Sun, 5 Nov 2000 23:39:37 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: <4754.973495415@kanga.nu> References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <20435.973357599@kanga.nu> <4754.973495415@kanga.nu> Message-ID: At 11:23 PM -0800 11/5/00, J C Lawrence wrote: > >Agreed. There's also the simple aspect of the fact that optimising >for the very large case (which needs the MLM-sepcific MTA supports >ala ListServ) is near on pessimal for the small and medium cases. >Spiffy domain based routing and near-target explosion just don't >make a whole lot of sense for lists with a few hundred/thousand >members. Asides from which, the optimisations for mail delivery to >a massive number of targets and mass mail delivery to a >comparitively constrained target set can be rather different. Nope. This is a job for -- ta da -- an API with a plug in architecture. Don't try to solve the huge list probelm, but solve the 90% problem (what works for the first 90% of the sites out there), and make it easy to plug in a different delivery beast so that other 10% can simply write a new module and swap it in (and hopefully send it back to the mailman project...) > >We should really have lunch some time. I'm currently working down >off San Tomas and Saratoga (startup, oddly enough next door to >BeOpen). I'm wrapping up this contract this month and looking for >next one, so time is a little tight here, but I'd love to get >together for a chat. You over at the main Apple campus? Effectively, yes. I'm down by the data center, which is hidden a bit off campus. > > And for sites this *doesn't* solve, it's either because they're >> doing the 5 pounds in a 1 pound bag thing, or they probably need >> to start hiring people like us to custom build someething. > >Ungghhhh. heh. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From thomas@xs4all.net Mon Nov 6 11:01:36 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 6 Nov 2000 12:01:36 +0100 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <200011052255.RAA14775@alice-whacker.mit.edu>; from ocschwar@MIT.EDU on Sun, Nov 05, 2000 at 05:55:37PM -0500 References: <200011052255.RAA14775@alice-whacker.mit.edu> Message-ID: <20001106120132.F27208@xs4all.nl> On Sun, Nov 05, 2000 at 05:55:37PM -0500, The guy named after an Om Kalthoum song wrote: [ Mailman using GPG to decrypt and re-crypt messages ] > I may be itnerested in doing this as an exercise to learn > Python. Has anyone done something like this? I don't think so. It's the first I heard about it, in any case. Note that "it isn't that simple" ;) You have to think about Archives. Do you want to enable them over SSL only ? SSL with client certificates ? Do you just want to disable them ? What about news gatewaying ? Just disable it for 'secure' groups, or just sign the postings ? > Also, is there a Python interface to GPG? I think I saw something like it, but I can't fid it now. A quick google search shows some hits on 'python' in the gnupg-devel list, so maybe there is the right place to look ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From jam@jamux.com Mon Nov 6 13:33:07 2000 From: jam@jamux.com (John A. Martin) Date: Mon, 06 Nov 2000 08:33:07 -0500 Subject: [Mailman-Developers] rc1 Possible confusion in Defaults.py comments Message-ID: <20001106133307.349FE48031@athene.jamux.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Using Mailman version 2.0rc1, in Defaults.py, under Bounce processing defaults, one sees: # 0 means do nothing # 1 means disable and send admin a report, # 2 means nuke'em (remove) and send admin a report, # 3 means nuke 'em and don't report (whee:) DEFAULT_AUTOMATIC_BOUNCE_ACTION = 1 and on the Bounce Options web page:
Do nothing Disable and notify me Disable and DON'T notify me Remove and notify me
(Ie. from left to right, (0) do nothing, (1) disable and notify, (2) disable and DON'T notify, and (3) Remove and notify. Changing the value of DEFAULT_AUTOMATIC_BOUNCE_ACTION in mm_cfg.py moves the radio button on the web page from left to right for 0, 1, 2, 3 as expected. However, the comments in Defaults.py do not seem to agree with the descriptions of options 2 and 3 on the web page. I have not checked the actual behavior of these options. jam -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: OpenPGP encrypted mail preferred. See iEYEARECAAYFAjoGsw4ACgkQUEvv1b/iXy+WSgCdEeZSCGMLA2xyt58Lcs3HyAkb BPgAoIvUH5uNQXPP2RWAHXOnbBQULOrH =dPoa -----END PGP SIGNATURE----- From jarrell@vt.edu Mon Nov 6 15:34:18 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Mon, 06 Nov 2000 10:34:18 -0500 Subject: [Mailman-Developers] mailman installation error In-Reply-To: <14852.11451.138206.842858@anthem.concentric.net> References: <3A034AC5.4191FD96@bmedesign.com> Message-ID: <5.0.0.25.2.20001106103101.03094c30@vtserf.cc.vt.edu> At 10:35 AM 11/4/00 -0500, Barry A. Warsaw wrote: >>>>>> "mt" == mike tran writes: > > mt> After I did ./configure --with-cgi-gid=nobody > mt> --with-mail-gid=mailman and make install. Then I ran > mt> check_perms. It keep telling me that it cannot find the > mt> module "paths" in line 19 of check_perms. Any suggestions is > mt> appricated. I am using RH 6.2 and the latest Apache with > mt> python 1.5.2 > >Look in /home/mailman/bin. If there's no paths.py file in there, your >installation failed somehow. Actually, this probably is a doc problem. I've dealt with this several times on the users list, and in all but one case (where their python was hosed) it was people not clearly following the directions. They're supposed to install, then run check_perms out of ~mailman/bin. Where there's a paths.py(c). However, what confuses a lot of people who only skim, is that in ${srcdir}/mailman/bin there's a check_perms. (The distribution copy). Which, of course, doesn't have a paths.py installed yet, and thus, doesn't work, but looks like it ought to, particularly to a complete newcomer who doesn't understand how things go together. (Heck, it took *me* a couple of days to figure out whatinhell they were doing wrong.) There probably needs to be a mention in the install file to *not* run check_perms until *after* you install, and to run it from the *installed* base. Or rename the install copy to check_perms.dist, and install it as check_perms... From jarrell@vt.edu Mon Nov 6 15:37:05 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Mon, 06 Nov 2000 10:37:05 -0500 Subject: [Mailman-Developers] Patch #102268 In-Reply-To: <14852.40577.83440.45618@j218.ryd.student.liu.se> References: <3A049D72.22801714@west.sun.com> <14852.18163.486910.710242@j218.ryd.student.liu.se> <3A049D72.22801714@west.sun.com> Message-ID: <5.0.0.25.2.20001106103556.030bfb00@vtserf.cc.vt.edu> At 12:40 AM 11/5/00 +0100, you wrote: >Try using arch on a archive that's already existing... Dan's point is you're not supposed to; it's an admin tool for rebuilding archives from scratch. You're supposed to wipe the archive before using it. If you don't, the results aren't pretty. From Nigel.Metheringham@VData.co.uk Mon Nov 6 16:21:45 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Mon, 06 Nov 2000 16:21:45 +0000 Subject: [Mailman-Developers] Re: [Mailman-Users] Problem with qrunner and too much incoming mail In-Reply-To: Message from Chuq Von Rospach of "Sun, 05 Nov 2000 23:39:37 PST." Message-ID: [Please can we try and kill these postings that are sent to both mailman-users & mailman-developers] Having been watching the floods at home in York for the last few days I'll just come in at this point chuqui@plaidworks.com said: > Nope. This is a job for -- ta da -- an API with a plug in > architecture. Don't try to solve the huge list probelm, but solve the > 90% problem (what works for the first 90% of the sites out there), > and make it easy to plug in a different delivery beast so that other > 10% can simply write a new module and swap it in (and hopefully send > it back to the mailman project...) [Loud applause] Mailman is a MLM, its not an MTA. Don't think about making it an MTA. Just leave enough hooks for people to trash their systems effectively. For the original specifics, I'll talk with the sourceforge people about an appropriate exim config for their problem. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From gietl@gietl.com Mon Nov 6 16:33:48 2000 From: gietl@gietl.com (Andreas Gietl) Date: Mon, 06 Nov 2000 17:33:48 +0100 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> Message-ID: <3A06DD6C.D4F23706@gietl.com> any further ideas? i'm sorry, but there is no pipeline entry: that's the db-file: { 'filebase': 'ccfe93988bf681ad271ad049fa54dcd7260a60ac', 'listname': 'bittecomgietltest2', 'torequest': 1, 'version': 2} and this is the .msg file: >From gietl@gietl.com Sat Nov 04 13:52:46 2000 Received: from [195.227.84.2] (helo=d1.x-mailer.de) by d18.x-mailer.de with esmtp (Exim 3.16 #1) id 13s2oQ-0003e1-00 for bittecomgietltest2-request@bitte.com; Sat, 04 Nov 2000 13:52:46 +0100 Received: from p3ee039ab.dip.t-dialin.net ([62.224.57.171] helo=gietl.com) by d1.x-mailer.de with esmtp (Exim 3.16 #1) id 13s2n9-0004iW-00 for bittecomgietltest2-request@bitte.com; Sat, 04 Nov 2000 13:51:27 +0100 Message-ID: <3A0406EC.6C4B8583@gietl.com> Date: Sat, 04 Nov 2000 13:54:04 +0100 From: Andreas Gietl X-Mailer: Mozilla 4.73 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: bittecomgietltest2-request@bitte.com Subject: Re: Bittecomgietltest2 -- confirmation of subscription -- request 569170 References: Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit bittecomgietltest2-request@bitte.com wrote: > > Bittecomgietltest2 -- confirmation of subscription -- request 569170 > > We have received a request from p3ee039ab.dip.t-dialin.net for > subscription of your email address, , to the > bittecomgietltest2@bitte.com mailing list. To confirm the request, > please send a message to bittecomgietltest2-request@bitte.com, and > either: > > - maintain the subject line as is (the reply's additional "Re:" is > ok), > > - or include the following line - and only the following line - in the > message body: > > confirm 569170 > > (Simply sending a 'reply' to this message should work from most email > interfaces, since that usually leaves the subject line in the right > form.) > > If you do not wish to subscribe to this list, please simply disregard > this message. Send questions to bittecomgietltest2-admin@bitte.com. barry@wooz.org wrote: > > >>>>> "AG" == Andreas Gietl writes: > > >> Do you still see the > >> confirmation message in qfiles? > > AG> Yeah, the confirmation message is in there! > > >> If so, are there any obvious errors > >> in logs/error? > > AG> nothing. i of course checked that! > > >> Does it have the expected Subject: header (i.e. at the > >> end of the line, "request xxxxxx" where those x's are the > >> numbers). > > AG> Yeah. it does have. > AG> Do you need any further information? > > Yes, do a bin/dumpdb qfiles/.db and look for the > `pipeline' entry. > > That should show us where things are getting stuck. > -Barry > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From gietl@gietl.com Mon Nov 6 18:11:42 2000 From: gietl@gietl.com (Andreas Gietl) Date: Mon, 06 Nov 2000 19:11:42 +0100 Subject: [Mailman-Developers] what happens after writing to qfiles? Message-ID: <3A06F45E.EBBCD656@gietl.com> Hi, since a few weeks i am exploring mails not proceeded by mailman after upgrades to rc1 or even on servers with a new fresh installation. Since it seems you could not help me till today i want to ask you what happens after mailman wrote the two files to qfiles when it gets a confirmation message? How are they proceeded? concretly: what happens after this is done? [pid 32132] open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.db", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 32132] umask(0) = 022 [pid 32132] open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.db", O_WRONLY|O_CREAT|O_TRUNC, 0664) = 5 [pid 32132] fcntl(5, F_GETFL) = 0x1 (flags O_WRONLY) [pid 32132] fstat(5, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0 [pid 32132] mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4014a000 [pid 32132] _llseek(5, 0, [0], SEEK_CUR) = 0 [pid 32132] umask(022) = 0 [pid 32132] write(5, "{s\7\0\0\0versioni\2\0\0\0s\t\0\0\0torequest"..., 124) = 124 [pid 32132] close(5) = 0 [pid 32132] munmap(0x4014a000, 4096) = 0 [pid 32132] umask(0) = 022 [pid 32132] open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.msg", O_WRONLY|O_CREAT|O_TRUNC, 0664) = 5 thx andreas -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From gietl@gietl.com Mon Nov 6 18:25:24 2000 From: gietl@gietl.com (Andreas Gietl) Date: Mon, 06 Nov 2000 19:25:24 +0100 Subject: [Mailman-Developers] what happens after writing to qfiles? References: <3A06F45E.EBBCD656@gietl.com> Message-ID: <3A06F794.51ADB723@gietl.com> Are the qfiles just processed by qrunner? Andreas Gietl wrote: > > Hi, > > since a few weeks i am exploring mails not proceeded by mailman after > upgrades to rc1 or even on servers with a new fresh installation. > > Since it seems you could not help me till today i want to ask you what > happens after mailman wrote the two files to qfiles when it gets a > confirmation message? How are they proceeded? > > concretly: what happens after this is done? > [pid 32132] > open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.db", > O_RDONLY) = -1 ENOENT (No such file or directory) > [pid 32132] umask(0) = 022 > [pid 32132] > open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.db", > O_WRONLY|O_CREAT|O_TRUNC, 0664) = 5 > [pid 32132] fcntl(5, F_GETFL) = 0x1 (flags O_WRONLY) > [pid 32132] fstat(5, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0 > [pid 32132] mmap(NULL, 4096, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4014a000 > [pid 32132] _llseek(5, 0, [0], SEEK_CUR) = 0 > [pid 32132] umask(022) = 0 > [pid 32132] write(5, "{s\7\0\0\0versioni\2\0\0\0s\t\0\0\0torequest"..., > 124) = 124 > [pid 32132] close(5) = 0 > [pid 32132] munmap(0x4014a000, 4096) = 0 > [pid 32132] umask(0) = 022 > [pid 32132] > open("/home/mailman/qfiles/5d8a73f7dfa649d60b947426c439a34c2b17f4b9.msg", > O_WRONLY|O_CREAT|O_TRUNC, 0664) = 5 > > thx > > andreas > -- > andreas gietl > gietl internet services > fon +49 9402 2551 > fax +49 9402 2604 > mobile +49 171 60 70 008 > gietl@gietl.com > > ############################################ > # Das Handbuch sagt, das Programm benötige # > # Windows 95 oder besser. Also habe ich # > # Linux installiert! # > ############################################ > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers -- andreas gietl gietl internet services fon +49 9402 2551 fax +49 9402 2604 mobile +49 171 60 70 008 gietl@gietl.com ############################################ # Das Handbuch sagt, das Programm benötige # # Windows 95 oder besser. Also habe ich # # Linux installiert! # ############################################ From claw@kanga.nu Mon Nov 6 18:28:49 2000 From: claw@kanga.nu (J C Lawrence) Date: Mon, 06 Nov 2000 10:28:49 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Message from "The guy named after an Om Kalthoum song" of "Sun, 05 Nov 2000 17:55:37 EST." <200011052255.RAA14775@alice-whacker.mit.edu> References: <200011052255.RAA14775@alice-whacker.mit.edu> Message-ID: <31582.973535329@kanga.nu> On Sun, 05 Nov 2000 17:55:37 -0500 The guy named after an Om Kalthoum song wrote: > Greetings! I am wondering if anyone has begun an implementation > of Gnu Provacy Guard email handling in Mailman, that would work > along these lines: I'm actually engaged (very slowly) in implementing automatically crypted mail support at the MTA level for a friend (he wants Sendmail support, I'm starting with Exim). There actually is little to no reason to add Mailman to the mess -- all the (de)crypt activities can occur compleatly outside of Mailman's operations. The 50,000 foot view for what I'm doing: Mail arrives at the MTA (localhost or SMTP). The MTA detects a flag string in the message or message x-headers. Depending on which strings were used, the MTA either: -- signs the mail with the author's key -- signs the mail and crypts it with the recipient's key The author's key is stored locally. All other keys are required to be available via public keyservers. Extending this to a mailing list actually isn't very difficult: Mail arrives at the list. Instead of being handed off to the list exploder, it passes thru a filter first which looks for blocks crypted with the list's key. If a crypted block is found, it is decrypted, surrounded by flag strings, and a flag x-header is added to the message. The message is then handed to the list exploder. The list exploder explodes the message and hands the bits back to the MTA. The MTA upon delivery of *ANY* message, looks for the X-header. If present it then crypts the flagged block with the envelope's key. If this is so simple, why am I not done yet? -- correct MIME support isn't entirely trivial -- I'm trying to think of ways to make it auditable *AND* scalable -- I've been distracted -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Mon Nov 6 18:37:52 2000 From: claw@kanga.nu (J C Lawrence) Date: Mon, 06 Nov 2000 10:37:52 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Message from J C Lawrence of "Mon, 06 Nov 2000 10:28:49 PST." <31582.973535329@kanga.nu> References: <200011052255.RAA14775@alice-whacker.mit.edu> <31582.973535329@kanga.nu> Message-ID: <32397.973535872@kanga.nu> On Mon, 06 Nov 2000 10:28:49 -0800 J C Lawrence wrote: > On Sun, 05 Nov 2000 17:55:37 -0500 The guy named after an Om > Kalthoum song wrote: >> Greetings! I am wondering if anyone has begun an implementation >> of Gnu Provacy Guard email handling in Mailman, that would work >> along these lines: > I'm actually engaged (very slowly) in implementing automatically > crypted mail support at the MTA level for a friend (he wants > Sendmail support, I'm starting with Exim). There actually is > little to no reason to add Mailman to the mess -- all the > (de)crypt activities can occur compleatly outside of Mailman's > operations. ... Some other attempts are referenced here: https://emu.kanga.nu/bin/view/Main/EncryptedMailingLists -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Mon Nov 6 19:03:08 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 11:03:08 -0800 Subject: [Mailman-Developers] passwords in third party web archives, newsgroups In-Reply-To: <3A05F488.57CFC441@mail.lns.cornell.edu> References: <3A05F488.57CFC441@mail.lns.cornell.edu> Message-ID: At 7:00 PM -0500 11/5/00, Dan Riley wrote: >Are the mailman developers at all concerned by > >http://www.google.com/search?q=mailman-owner+reminder+password >http://x66.deja.com/=dnc/getdoc.xp?AN=641175690 > >This is probably especially a problem with lists that were converted >from another MLM, It's an interesting issue. Mailman includes the X-No-Archive header in these messages, so anyone who's archiving them anyway isn't following the protocol. I'm not sure it's a major issue, and it's an end-user mis-behavior at that -- but it's still somewhat troubling that it gets into archives and search engines. I'm not sure what mailman can do to prevent end-users from shooting themselves in the foot here, though. the passwords are a trivial issue to me. the REAL issue is you have mail lists putting up archives that are being put into global search engines -- and those archives are full of unprotected email addresses just waiting for the spam harvester bots. Compared to that, the passwords are nothing -- like dealing iwth a hangnail on a foot with gangrene. and if the lists want their archives to be wide open like that, there's not a damn thing Mailman can do to save them from themselves. But as long as there are easily harvested email addresses in the search engines, the passwords simply aren't something I'm going to worry about. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From ocschwar@MIT.EDU Mon Nov 6 19:54:38 2000 From: ocschwar@MIT.EDU (Omri Schwarz) Date: Mon, 06 Nov 2000 14:54:38 -0500 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Your message of "Mon, 06 Nov 2000 12:01:36 +0100." <20001106120132.F27208@xs4all.nl> Message-ID: <200011061954.OAA23482@hodge-podge.mit.edu> > On Sun, Nov 05, 2000 at 05:55:37PM -0500, The guy named after an Om Kalthoum song wrote: > > [ Mailman using GPG to decrypt and re-crypt messages ] > > > I may be itnerested in doing this as an exercise to learn > > Python. Has anyone done something like this? > > I don't think so. It's the first I heard about it, in any case. Note that > "it isn't that simple" ;) You have to think about Archives. Do you want to > enable them over SSL only ? SSL with client certificates ? Do you just want > to disable them ? What about news gatewaying ? Just disable it for 'secure' > groups, or just sign the postings ? The motivation I have behind asking (which can quickly drift off-topic for this list) is that the main reason behind the failure of widespread email encryption is human factors. Therefore, the right amount of social engineering will be the driving force in getting people to encrypt email. If a mailing list exploder like what I described is available, people will learn not to 1. share TMI type information on any other kind of mailing list, or 2. share proprietary discussions on any other kind of mailing list. So, a list like this will 1. have no Web archiving, 2. no news gatewaying, and 3. rapidly expiring mailing list keypairs, Just In Case (TM). I'm asking this on the Mailman forum because Mailman would be easier to GPG-enable than Majordomo (just as eating ice cream is more pleasant than root canal..), and because apart from that, I am not picky on how this should be done, hence would be willing to fork Mailman to warp it for this end. -- ------- Omri Schwarz ocschwar@mit.edu "Fair enough: anyone who believes that the laws of physics are | "Prof drop!" mere social conventions is invited to try transgressing | "Thud." those conventions from the windows of my apartment. | "Ow! (I live on the twenty-first floor.)" Alan Sokal - Physicist | My Spleen!" From chuqui@plaidworks.com Mon Nov 6 20:06:13 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 12:06:13 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <200011061954.OAA23482@hodge-podge.mit.edu> References: <200011061954.OAA23482@hodge-podge.mit.edu> Message-ID: At 2:54 PM -0500 11/6/00, Omri Schwarz wrote: >The motivation I have behind asking >(which can quickly drift off-topic for this list) >is that the main reason behind the failure of >widespread email encryption is human factors. >Therefore, the right amount of social engineering >will be the driving force in getting people to encrypt email. I agree with this -- but I disagree that the mailing list is where the focus needs to go. The focus needs to go in fixing mail client interfaces to properly integrate this stuff and make it easy to use. Until that happens, it doesn't matter what else is done, because users won't use it. IMHO, of course. but working on the MLM side is the cart before the horse. Get the clients using encryption rationally, and the server side will follow naturally. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From Dan.Mick@west.sun.com Mon Nov 6 21:34:13 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Mon, 06 Nov 2000 13:34:13 -0800 Subject: [Mailman-Developers] mailman installation error References: <3A034AC5.4191FD96@bmedesign.com> <5.0.0.25.2.20001106103101.03094c30@vtserf.cc.vt.edu> Message-ID: <3A0723D5.C7398BC7@west.sun.com> Ron Jarrell wrote: > > At 10:35 AM 11/4/00 -0500, Barry A. Warsaw wrote: > > >>>>>> "mt" == mike tran writes: > > > > mt> After I did ./configure --with-cgi-gid=nobody > > mt> --with-mail-gid=mailman and make install. Then I ran > > mt> check_perms. It keep telling me that it cannot find the > > mt> module "paths" in line 19 of check_perms. Any suggestions is > > mt> appricated. I am using RH 6.2 and the latest Apache with > > mt> python 1.5.2 > > > >Look in /home/mailman/bin. If there's no paths.py file in there, your > >installation failed somehow. > > Actually, this probably is a doc problem. I've dealt with this several times on > the users list, and in all but one case (where their python was hosed) it was > people not clearly following the directions. They're supposed to install, > then run check_perms out of ~mailman/bin. Where there's a paths.py(c). > > However, what confuses a lot of people who only skim, is that in > ${srcdir}/mailman/bin there's a check_perms. (The distribution copy). > Which, of course, doesn't have a paths.py installed yet, and thus, doesn't > work, but looks like it ought to, particularly to a complete newcomer who > doesn't understand how things go together. (Heck, it took *me* a couple of > days to figure out whatinhell they were doing wrong.) > > There probably needs to be a mention in the install file to *not* run check_perms > until *after* you install, and to run it from the *installed* base. Or rename the > install copy to check_perms.dist, and install it as check_perms... It's hard to imagine how to make this more clear, though; it *is* step 3, which is numbered, and after the "make install" step, and it *does* say - cd to $prefix - Run bin/check_perms I mean, the next thing is to add things around that point saying "no, really, we really mean exactly what we say here, and not what you think you're reading, so read this carefully, please.." but you'd think that would be obvious from the INSTALL file being the one concise thing you have to read and follow to install Mailman... From jarrell@vt.edu Mon Nov 6 22:27:15 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Mon, 06 Nov 2000 17:27:15 -0500 Subject: [Mailman-Developers] mailman installation error In-Reply-To: <3A0723D5.C7398BC7@west.sun.com> References: <3A034AC5.4191FD96@bmedesign.com> <5.0.0.25.2.20001106103101.03094c30@vtserf.cc.vt.edu> Message-ID: <5.0.0.25.2.20001106172510.0824f330@vtserf.cc.vt.edu> At 01:34 PM 11/6/00 -0800, you wrote: >Ron Jarrell wrote: >> >> At 10:35 AM 11/4/00 -0500, Barry A. Warsaw wrote: >> >> >>>>>> "mt" == mike tran writes: >> > >> > mt> After I did ./configure --with-cgi-gid=nobody >> > mt> --with-mail-gid=mailman and make install. Then I ran >> > mt> check_perms. It keep telling me that it cannot find the >> > mt> module "paths" in line 19 of check_perms. Any suggestions is >> > mt> appricated. I am using RH 6.2 and the latest Apache with >> > mt> python 1.5.2 >> > >> >Look in /home/mailman/bin. If there's no paths.py file in there, your >> >installation failed somehow. >> >> Actually, this probably is a doc problem. I've dealt with this several times on >> the users list, and in all but one case (where their python was hosed) it was >> people not clearly following the directions. They're supposed to install, >> then run check_perms out of ~mailman/bin. Where there's a paths.py(c). >> >> However, what confuses a lot of people who only skim, is that in >> ${srcdir}/mailman/bin there's a check_perms. (The distribution copy). >> Which, of course, doesn't have a paths.py installed yet, and thus, doesn't >> work, but looks like it ought to, particularly to a complete newcomer who >> doesn't understand how things go together. (Heck, it took *me* a couple of >> days to figure out whatinhell they were doing wrong.) >> >> There probably needs to be a mention in the install file to *not* run check_perms >> until *after* you install, and to run it from the *installed* base. Or rename the >> install copy to check_perms.dist, and install it as check_perms... > >It's hard to imagine how to make this more clear, though; it *is* step 3, which >is numbered, and after the "make install" step, and it *does* say > > - cd to $prefix > > - Run bin/check_perms > >I mean, the next thing is to add things around that point saying "no, really, we really >mean exactly what we say here, and not what you think you're reading, so read this >carefully, please.." but you'd think that would be obvious from the INSTALL file >being the one concise thing you have to read and follow to install Mailman... Yea, I know. *I* didn't have have any problems following it the first time, back when I thought python was a big assed snake :-). But given the number of people who've I've talked to that *this* was the problem (it's up to about 10 now) there must be a huge number out there scratching their heads going "this software sucks". These are likely the same people who use the blowdryer in the tub, cause the tag said something about "use blowdryer" and "near water"... :-) From claw@kanga.nu Tue Nov 7 04:55:55 2000 From: claw@kanga.nu (J C Lawrence) Date: Mon, 06 Nov 2000 20:55:55 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Message from Omri Schwarz of "Mon, 06 Nov 2000 14:54:38 EST." <200011061954.OAA23482@hodge-podge.mit.edu> References: <200011061954.OAA23482@hodge-podge.mit.edu> Message-ID: <24458.973572955@kanga.nu> On Mon, 06 Nov 2000 14:54:38 -0500 Omri Schwarz wrote: > The motivation I have behind asking (which can quickly drift > off-topic for this list) is that the main reason behind the > failure of widespread email encryption is human factors. True. More simply, given that most email is of a casual nature, there is little to no return on invested effort for casual users. They fail so see any benefit from crypting or signing their "Funny joke" messages. > Therefore, the right amount of social engineering will be the > driving force in getting people to encrypt email. You first need to create an awareness with them of the problem you wish to solve with encryption. Nobody, and that encludes me, is going to go thru the bother of genning keys, getting them signed, auditing and tracking them, and generally attempting to be responsible here unless I've got some jolly good reason to, unless I've got some problem that going thru all that hassle solves. > If a mailing list exploder like what I described is available, > people will learn not to 1. share TMI type information on any > other kind of mailing list, or 2. share proprietary discussions on > any other kind of mailing list. Uhh, yeah. > So, a list like this will 1. have no Web archiving, 2. no news > gatewaying, and 3. rapidly expiring mailing list keypairs, Just In > Case (TM). This depends on what you are attempting to protect and why. In the case of trade secret protections, web archiving may be a significant plus if you can also audit and control access to those archives (S/Key etc). There is no one model fits all. > I'm asking this on the Mailman forum because Mailman would be > easier to GPG-enable than Majordomo (just as eating ice cream is > more pleasant than root canal..), and because apart from that, I > am not picky on how this should be done, hence would be willing to > fork Mailman to warp it for this end. I'd argue that the crypted list problem is actually orthogonal to the MLM software used. The MLM never needs to be involved. You can involve it if you really want to, but there's not much benefit to doing so. -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From barry@wooz.org Tue Nov 7 04:59:18 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Mon, 6 Nov 2000 23:59:18 -0500 (EST) Subject: [Mailman-Developers] Problem with qrunner and too much incoming mail References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> Message-ID: <14855.35878.37533.497288@anthem.concentric.net> [Removed mailman-users from the recips.] Okay, well that went over like a lead balloon... :) [Aside: it's time to start capturing these notes in a more useful place than the mail archives. I've started a ZWiki page which will serve as "design central" for Mailman. I did this partly to have experience with Wikis, which everybody at my new employer simply raves about. Please visit http://www.zope.org/Members/bwarsaw/MailmanDesignNotes and remember, you too can edit and contribute to these pages! Don't make me the bottleneck. You can learn the little bit you need to know about Wikis from that URL too. Think collaborative web pages.] I completely agree that we want a pluggable architecture for MLM->MTA handoff. We /almost/ have that now with the delivery pipeline, but my mistake was in making the delivery module part of that pipeline. What Mailman's pipeline ought to do is the prep-work on the message only: spam and privacy filtering, setting headers, updating per-list counters, appending to digests, etc. Anything that does not require writing list-specific data could be pulled out of the pipeline. I'm thinking about specifically about nntp posting and the mta-handoff. An API for the handoff is A Good Thing, and of course given that, there's no reason why someone looking for a project couldn't write an external, outgoing-only whizzymailer along the lines I outlined. Based on results that others are getting writing pure-Python servers for other protocols, I think you might be able to get some fairly impressive performance there, especially because this would be outgoing only (it doesn't need to handle any incoming smtp connections). But I definitely didn't envision this to be the /only/ or even the /primary/ way for mail to be delivered, just another option. One of the things that such an approach would give us, is the ability to do more direct bounce detection and handling, eliminating some of the error prone bounce message parsing. E.g. our whizzymailer would know the details of Mailman so when it got errors during the smtp transaction, it could update the db's directly. This isn't as likely to happen when we handoff to a localhost MTA, unless they support DSN and we run them synchronously (which clobbers the current architecture, as we're seeing). Any API we come up with for MLM->MTA handoff should give us the benefits of dsn without the problems. I.e. it should be two-way. Some combination of API and better architecture is probably necessary. >>>>> "JCL" == J C Lawrence writes: JCL> What would be a really good approach without concern for code JCL> impact? I suspect a modified form of the hash tree for queue JCL> storage (cf QMail's implementation minus the silly (for this JCL> use) inode specifics) with a slightly perverted form of your JCL> (Barry's) long running bulkmailer to process that hash queue. Let's flesh that out a little. What does data Qmail hash on? Would the hash tree be in-memory? Would there be any disk persistence in case of system failure? Each message currently has two parts: message content and metadata. Would both be stored in the hash tree? That might get expensive for really big messages. Maybe the message content should be stored in a file and the metadata in the hash tree. Then again, since most messages don't live for very long in the queue, maybe the elimination of the disk i/o is worth a little instability or larger memory footprint. JCL> I'd tend to make the bulkmailer actually an intermittently JCL> running item to help support for intermittently connected JCL> nodes. Say something like: JCL> Cron launches the bulkmailer. The bulkmailer forks N JCL> children processing the queue. The bulkmailer exits upon an JCL> empty queue. Should cron launch a new bulkmailer when the JCL> prevvious incarnation hasn't exited yet, the new instance JCL> merely exits immediately. Forking is pretty heavyweight, and threading has its problems too. One of the things I like about the select-and-continuations-based servers is that for i/o bound tasks, they aren't very difficult to code efficiently. Cron could be used to watchdog the process though. >>>>> "CVR" == Chuq Von Rospach writes: CVR> Instead of reinventing the MTA wheel, I think we're much CVR> better off coming up with an MTA -> MLM interface that's very CVR> flexible and highly configurable (most especially in how to CVR> deliver and how much to parallelize the infeed to the MLM), CVR> and then focus on how to tune the MTA and MLM through CVR> documentation. CVR> Splitting the inbound and outbound queue would be my first CVR> thing here, and probably split bounces into a third CVR> queue. Great idea. Each queue has it's own requirements, e.g. there's definitely been complaints about the minimum 1-minute delay outgoing messages. CVR> That's a pretty quick, easy optimization that makes CVR> sure the end user sees fast response without being bogged CVR> down by deliveries, and that's a huge perception issue. Then CVR> focus on parallelizing the delivery from mailman into the CVR> MTA, and make that configurable so each admin can tune it to CVR> their system and needs. Agreed. I also want that feedback for list-bound messages so that Mailman can be notified directly from the MTA about certain types of delivery failures. I still worry about bottlenecks in synchronous mode, even with a high degree of parallelism and shallow buckets. Thinking out loud: what if the API had two channels, mlm->mta and mta->mlm, let's call them outbound and inbound respectively. The outbound channel needs to contain the message text (or a disk file, ownership of which is passed to the mta), a list of recipients, and an set of metadata to associate with the message. Metadata may include: the list name, the list of error codes to report back to us, a VERP flag, and possibly other opaque data. Incoming is limited only to error reporting, e.g. a list of failed addrs and their error codes, and the metadata reflected back for that message. CVR> If someone wants a rhetoric on how to scale mail list servers CVR> infinitely, I'd be happy to explain how, since I've had to CVR> develop an architecture to do so. If you write it up, I'll add it to the documentation. At the very least, let's add it to the ZWiki. CVR> I think we can build a Mailman that does this, at least for, CVR> oh, 95% of the universe out there, and the other 5% are going CVR> to have custom solutions anyway (or should!). What we don't CVR> want to do is screw up Mailman for the "typical" user to make CVR> it work for the big site; but we also don't want Mailman to CVR> get a reputation as a "small server only" system, because CVR> it'll cause people to reject it in CVR> implementations. Fortunately, I don't think you need to do CVR> that. It just needs some tweaking. Completely agree. CVR> On reasonable hardware, definitely. That's basically how my CVR> current custom system works. right now, the number of CVR> parallel infeeds from mailman is 1. I'm willing to bet the CVR> delivery MTA is basically idle and bored. Have you played at all with the threaded delivery in SMTPDirect? Admittedly it's not integrated correctly with the rest of Mailman, but I'm still curious if the notion is salvagable. -Barry From chuqui@plaidworks.com Tue Nov 7 05:29:01 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 21:29:01 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <24458.973572955@kanga.nu> References: <200011061954.OAA23482@hodge-podge.mit.edu> <24458.973572955@kanga.nu> Message-ID: >I'd argue that the crypted list problem is actually orthogonal to >the MLM software used. The MLM never needs to be involved. You can >involve it if you really want to, but there's not much benefit to >doing so. Except in the case of authentification of the user. It's a critical need there -- I'd kill for the ability to be able to verify subscribers via S/MIME or some other non-spoofable form. then they could post from anywhere, as long as their authentification worked. This general ability - to validate an incoming e-mail, not just for MLM -- would be a killer app for the anti-spam folks.Anything without a valid signature, you dump. But, ask me to add this support to something like Mailman, and I'll say no. Why? Because until the clients support it cleanly and easily and it's on its way towards general acceptance in the user base, it's wasted effort. Right now, encryption is way too nerdy, too complicated, and the typical user doens't see the need. Spend time writing support into a MLM, and it won't be used. Try to make it mandator, and you'll kill the MLM. If the List-* headers are something the MLM's have to lead the pack in, encryption has to be led out of the mail clients, because that's where 95% of the utility will be. any encrpytion support in the MLM proper would be as an add-on to the support in the client, and it makes no sense to do it until the clients do it (and set the standard for interfacing to it). An d that's a long way off from what I see. When I can reliably expect to find (and use) an embedded digital signature on an e-mail, then it's time to look at adding it to Mailman. Until then -- if you want encryption, go lobby the mail clients to add the functionaliy and make it usable and easy enough that people actually use it. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From ocschwar@MIT.EDU Tue Nov 7 05:54:41 2000 From: ocschwar@MIT.EDU (Omri Schwarz) Date: Tue, 07 Nov 2000 00:54:41 -0500 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Your message of "Mon, 06 Nov 2000 20:55:55 PST." <24458.973572955@kanga.nu> Message-ID: <200011070554.AAA25089@hodge-podge.mit.edu> > On Mon, 06 Nov 2000 14:54:38 -0500 > Omri Schwarz wrote: > > > I'm asking this on the Mailman forum because Mailman would be > > easier to GPG-enable than Majordomo (just as eating ice cream is > > more pleasant than root canal..), and because apart from that, I > > am not picky on how this should be done, hence would be willing to > > fork Mailman to warp it for this end. > > I'd argue that the crypted list problem is actually orthogonal to > the MLM software used. The MLM never needs to be involved. You can > involve it if you really want to, but there's not much benefit to > doing so. Both your solution and mine do the same thing on the human failings angle: they allow a mail server admin to set up a list that does encryption for everyone, so that people learn that some things are best not discussed in plaintext. (Said mail server admin, now enabled with this solution, can also go fascist on people who don't comply with said policy. Enough mail server admins deciding to go this route, and you may see an effort on the part of users and MUA writers to improve things on that end.) As to which solution is better on the software side, you're probably right. I'll have to have a closer look at the Mailman code. But you're sort of committed to something, as am I, so there's hope. Now, for Mr. Von Rospach: >This general ability - to validate an incoming e-mail, not just for >MLM -- would be a killer app for the anti-spam folks.Anything without >a valid signature, you dump. >But, ask me to add this support to something like Mailman, and I'll >say no. Why? Because until the clients support it cleanly and easily >and it's on its way towards general acceptance in the user base, it's >wasted effort. GPG version chauvinism is a must for such a project. PGP-GPG and intra-PGP version incompatibilities are a pain. In turn, that kills the MUAs. However, I don't believe good GPG handling in the MUAs is the necessary-and-sufficient part to bring this about. (My likely-wrong opinion here.) So, to summarize: Python-GPG interface exists somewhere, and not much happening in the Mail server or MLM group software side. So the niche is here and I'm ready to give it a try. Thank you for your attention, y'all. -- Omri Schwarz Some people have told me they don't think a fat penguin really embodies the grace of Linux, which just tells me they have never seen a angry penguin charging at them in excess of 100mph. They'd be a lot more careful about what they say if they had. -- Linus Torvalds From chuqui@plaidworks.com Tue Nov 7 07:00:32 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 23:00:32 -0800 Subject: [Mailman-Developers] Problem with qrunner and too much incoming mail In-Reply-To: <14855.35878.37533.497288@anthem.concentric.net> References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <14855.35878.37533.497288@anthem.concentric.net> Message-ID: At 11:59 PM -0500 11/6/00, Barry A. Warsaw wrote: >experience with Wikis, which everybody at my new employer simply raves >about. Please visit > > http://www.zope.org/Members/bwarsaw/MailmanDesignNotes got that bookmarked. Looks interesting. >mistake was in making the delivery module part of that pipeline. What >Mailman's pipeline ought to do is the prep-work on the message only: That's pretty much what I do now on my big mother machine. There's a web page for posting, and it spawns a script that creates a message file (with full headers already finished), then sucks the subscriber list out of an SQL database, generates a set of commandfiles and subscriber lists, and throws them in a queueing system. it's configurable on the fly (FWIW, I use QPS for queueing, I didn't write one. Gnu Queue was my first choice, but it has problems on solaris I didn't have time to debug...) >spam and privacy filtering, setting headers, updating per-list >counters, appending to digests, etc. Anything that does not require >writing list-specific data could be pulled out of the pipeline. I'm >thinking about specifically about nntp posting and the mta-handoff. I'm beginning to think that "mailman 3.0" may end up being NOTHING but APIs and enough glue to interface them. that way, *any* piece can be swapped out for something equivalent if we want to, and we can strongly isolate interactions and keep each code-base simple. A subscription API, a spam API, a digest API, etc, etc, etc. It's not at all that simple in practive, but in theory, you ultimately have a set of Python classes that define a MLM... > E.g. our whizzymailer would >know the details of Mailman so when it got errors during the smtp >transaction, it could update the db's directly. This isn't as likely >to happen when we handoff to a localhost MTA, unless they support DSN >and we run them synchronously (which clobbers the current >architecture, as we're seeing). this is nice in theory, but again, you start BEING an MTA, and the subtleties are going to whack you up side the head. I looked at this a while back, and decided that was a place I didn't want to go. I really think we need to be careful about optimization by subverting the MTA, because down that path lies sendmail -- which does everything known to mankind, but nobody can figure out the documentation... Let the MTA be an MTA, and simply hand stuff off and process the returns. That gives you the ability to build tools that leverage other people's strength's, whether it's Postfix or smartbounce. Otherwise, you're asking for a LOT of work that you don't think you're going to need to do yet. I didn't realize that until I *did* start designing sendmail out of my system and saw the results - slower, uglier and I got to maintain the code myself... > Would there be any disk persistence in >case of system failure? There has to be, but one thing sendmail did with 8.10 to boost performance was to figure out what needed to be on disk for recovery, and keeping the rest in memory. For their purposes, that was the df and qf files, and they do all of the locking and status files via what's effectively an in-sendmail RAM-based pseudo disk for all of the other files, like lock files. so stuff relevant to the current delivery attempt is in RAM, the stuff you need ot decide whether ot deliver or how to deliver is on disk. In a crash, you only lose data that's not relevant after a crash. the big problem I see on sendmail 8.9 is inode locking, especially when it's updating the /var/spool/mqueue directory inode. sendmail 8.10 goes a long way towards fixing that with the /var/spool/mqueue/* setup -- you can imagine the fun of 400 sendmails all trying to update their queue files in the same directory inode. (wince). So before we get into fancy hashing systems, let's see how we do with the basics -- split in/out/bounce into separate qfiles, split content/metadata/status/lock into separate subdirectories, and if necessary, allow multiple directories to further split the directory contention. In fact, a really simply way to parallelize Mailman would be to allow multiple qfiles, and every time qrunner is spawned from cron, it creates one instance per directory. that way, you distribute the load out evenly and can rearrange it as you need by adding or removing directories. No funky config file issues. >Then again, since most messages don't live for very long in the queue, >maybe the elimination of the disk i/o is worth a little instability or >larger memory footprint. Before you make that decision, we need to know whether the I/O is actually significant, and which pieces of the I/O can safely be held off. But in reality, I'll bet you won't find a Mailman site where the mailman directories are I/O bound in an significant way. We shouldn't try to optimize things that aren't the bottlenecks.... >Forking is pretty heavyweight, and threading has its problems too. But for what we're talking about, the fork overhead is pretty trivial. Forking is bad for lots of little, short-lived things. Forking is good for relatively few, long-lived things. Given what the processing cost of delivering 500 pieces of email will be, the overhead of the fork is non-existant. If we were forking a process per address, I'd worry about it. Forking a process per message isn't. Reality is somewhere in the middle -- but the trick is to find the slow parts and speed them up, rather than just try to speed up various things we guess might be slow. > CVR> Splitting the inbound and outbound queue would be my first > CVR> thing here, and probably split bounces into a third > CVR> queue. > >Great idea. Each queue has it's own requirements, e.g. there's >definitely been complaints about the minimum 1-minute delay outgoing >messages. the outbound queue is a perfect place for a daemon to sit, and make sure there are always up to "N" messages being processed (we might want to amend that so only one message is in process at a time for a given list. Hmm, does it make sense to split the outqueue into subdirs (see above) by list name? the outqueue daemon could then round-robin the lists, to prevent a busy list from stuffing the other lists into a corner... >Agreed. I also want that feedback for list-bound messages so that >Mailman can be notified directly from the MTA about certain types of >delivery failures. I wouldn't worry about this. the programming complexity makes this a false economy. It sounds nice in theory, but I wouldn't make it a design goal until we get other stuff in place -- if then. Bounces are a pain in the neck, but not that nasty to deal with, and the places where simply background processing bounces falls down, this isn't really likely to help, because it's the guy three forwards away from the subscribed address, behind a firewall, on a Notes server, who changed his name when he got married four months ago... What you're really proposing, Barry, is to have to implement TWO bounce processing systems. One for stuff where the delivery attempt fails locally, and a second one for stuff where the mail is delivered to an agent that then sends a reject back. and that latter includes all of the major ISPs (especially AOL and MSN), most major corporations (including Apple), and basically every large site with firewalls and mail relays through them. So you're doubling your work writing bounce processing code, and it buys you very, very little. And the real trouble cases won't be helped at all, because they'r the ones that won't get nailed until they come back through those 4 relays with the addresses munched and headers stripped. > I still worry about bottlenecks in synchronous >mode, even with a high degree of parallelism and shallow buckets. From the point of view of Mailman, I doubt it's really an issue. I think you're still thinking MTA here. Mailman is NOT an MTA. You don't want to write an MTA. you don't want to think like an MTA writer. (see this swinging watch? you are getting sleepy, sleepy... you do not want to write sendmail. you do not want to integrate sendmail into Mailman. you are not eric allman. sendmail.cf files give you hives...) >Thinking out loud: what if the API had two channels, mlm->mta and >mta->mlm, let's call them outbound and inbound respectively. I'd do three APIs, actually. DeliverMail, IncomingMail, BounceMail. they might hand off to each other (forinstance IncomingMail would recognize a bounce, and forward it ot BounceMail), but they're really independent operations. I worry that trying to build a single API would start throwing in compromises or overloading concepts. And iwth three APIs, they can be developed independently by different people -- and swapped out independntly. With a single API, that's tougher. > CVR> If someone wants a rhetoric on how to scale mail list servers > CVR> infinitely, I'd be happy to explain how, since I've had to > CVR> develop an architecture to do so. > >If you write it up, I'll add it to the documentation. At the very >least, let's add it to the ZWiki. Will do, once I get a chance. I've put it on my todo list. >Have you played at all with the threaded delivery in SMTPDirect? >Admittedly it's not integrated correctly with the rest of Mailman, but >I'm still curious if the notion is salvagable. A little. I really think the fork model needs to be used, because the thread locks don't seem to allow enough processing independence. The delivery stuff is going to spend most of its time in I/O wait with kernel locks, and needs to respond quickly when the I/O is available. I'm afraid that using threads for this defeats that purpose, because the block has to be reported to Python, which then has to get around to activating that thread, and by the time you do, you've lost the performance edge, especially when you're talking about a number of these fighting for the single CPU resource. This is a case where the fork overhead is trivial compared to the job overhead, and you really want these independent and down in the kernel, since you're dealing with stuff the kernel is best suited to resolve. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From chuqui@plaidworks.com Tue Nov 7 07:07:40 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 23:07:40 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <200011070554.AAA25089@hodge-podge.mit.edu> References: <200011070554.AAA25089@hodge-podge.mit.edu> Message-ID: At 12:54 AM -0500 11/7/00, Omri Schwarz wrote: >Both your solution and mine do the same thing on the human >failings angle: they allow a mail server admin to set up a list >that does encryption for everyone, so that people learn that >some things are best not discussed in plaintext. no, it really doesn't, because the message is sent to the MLM in plaintext, so it has no security at all. If you depend on the MLM to do the encryption, you might as well not encrypt, bceause anyone sniffing packets will have the data no proble. what you're doing is setting up a sense of *false* security, but you're in fact leaving things wide open. It has to be encrypted leaving the client, or it's not secure. >GPG version chauvinism is a must for such a project. why? you want encryption endemic. Which implies abiliy to handle anyone's public key and do something reasonable with it, not just one. Otherwise, you're balkanized, and that defeats the purpose again. >In turn, that kills the MUAs. However, >I don't believe good GPG handling in the MUAs >is the necessary-and-sufficient part to bring this about. If the MUAs don't support encryption, then how will users decrypt something the MLM encrypted? And if the MUA does support encryption -- the MLM doens't have to. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From ocschwar@MIT.EDU Tue Nov 7 07:13:39 2000 From: ocschwar@MIT.EDU (Omri Schwarz) Date: Tue, 07 Nov 2000 02:13:39 -0500 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Your message of "Mon, 06 Nov 2000 23:07:40 PST." Message-ID: <200011070713.CAA25343@hodge-podge.mit.edu> > At 12:54 AM -0500 11/7/00, Omri Schwarz wrote: > > >Both your solution and mine do the same thing on the human > >failings angle: they allow a mail server admin to set up a list > >that does encryption for everyone, so that people learn that > >some things are best not discussed in plaintext. > > no, it really doesn't, because the message is sent to the MLM in > plaintext, so it has no security at all. If you depend on the MLM to > do the encryption, you might as well not encrypt, bceause anyone > sniffing packets will have the data no proble. what you're doing is > setting up a sense of *false* security, but you're in fact leaving > things wide open. It has to be encrypted leaving the client, or it's > not secure. Unless I misunderstood, in both cases a program on the server decripts incoming mail and then re-encrypts, but that in once case the Sendmail/Qmail program does this while I want the MLM to do it. Setting up an encription-required rule for a list should be easy in either case. > >GPG version chauvinism is a must for such a project. > > why? you want encryption endemic. Which implies abiliy to handle > anyone's public key and do something reasonable with it, not just > one. Otherwise, you're balkanized, and that defeats the purpose again. > > >In turn, that kills the MUAs. However, > >I don't believe good GPG handling in the MUAs > >is the necessary-and-sufficient part to bring this about. > > If the MUAs don't support encryption, then how will users decrypt > something the MLM encrypted? And if the MUA does support encryption > -- the MLM doens't have to. > MUAs that support encryption do exist. Unfortunately, they cater mostly to Unix gurus. From chuqui@plaidworks.com Tue Nov 7 07:17:22 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 6 Nov 2000 23:17:22 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <200011070713.CAA25343@hodge-podge.mit.edu> References: <200011070713.CAA25343@hodge-podge.mit.edu> Message-ID: At 2:13 AM -0500 11/7/00, Omri Schwarz wrote: > >MUAs that support encryption do exist. >Unfortunately, they cater mostly to Unix gurus. Until you can convince the general public to encrypt mail, then you'll never get real encryption support in systems (much as we need it)... right now, encryption stuff sings only to the choir, not the church members.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From ocschwar@MIT.EDU Tue Nov 7 07:38:40 2000 From: ocschwar@MIT.EDU (Omri Schwarz) Date: Tue, 07 Nov 2000 02:38:40 -0500 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: Your message of "Mon, 06 Nov 2000 23:17:22 PST." Message-ID: <200011070738.CAA25453@hodge-podge.mit.edu> > At 2:13 AM -0500 11/7/00, Omri Schwarz wrote: > > > >MUAs that support encryption do exist. > >Unfortunately, they cater mostly to Unix gurus. > > Until you can convince the general public to encrypt mail, then > you'll never get real encryption support in systems (much as we need > it)... right now, encryption stuff sings only to the choir, not the > church members.... We'll restart this discussion when and if I have something more than vapor to show y'all. From Dan.Mick@west.sun.com Tue Nov 7 20:21:57 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Tue, 07 Nov 2000 12:21:57 -0800 Subject: [Mailman-Developers] Problem with qrunner and too much incoming mail References: <20001103113539.S22363@marc.merlins.org> <14852.11186.967453.490195@anthem.concentric.net> <14855.35878.37533.497288@anthem.concentric.net> Message-ID: <3A086465.A57EEA92@west.sun.com> > CVR> On reasonable hardware, definitely. That's basically how my > CVR> current custom system works. right now, the number of > CVR> parallel infeeds from mailman is 1. I'm willing to bet the > CVR> delivery MTA is basically idle and bored. > > Have you played at all with the threaded delivery in SMTPDirect? > Admittedly it's not integrated correctly with the rest of Mailman, but > I'm still curious if the notion is salvagable. I also really wonder whether anyone besides me and the author of the patch are using the "sort by domain and chunkify" patch to SMTPDirect, too. I've been using it for six months and I think I like it; it's a way of distributing the load a bit: do the obvious-easy-simple stuff and let the MTA do the rest. Wonder if it would make a difference for those with enough mail to be hitting bottlenecks? From barry@wooz.org Wed Nov 8 19:42:34 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Wed, 8 Nov 2000 14:42:34 -0500 (EST) Subject: [Mailman-Developers] Mailman and GPG. References: <200011052255.RAA14775@alice-whacker.mit.edu> Message-ID: <14857.44202.507836.574952@anthem.concentric.net> Omri> I may be itnerested in doing this as an exercise to learn Omri> Python. I'm not one to discourage anybody's motivation for learning Python, so go for it. I think Chuq and JC have outlined the concerns and issues nicely, so I won't add much else other than to say that I /hope/ you'd be able to add the bits you'd need without forking Mailman. There should be enough pluggable architecture to add what you need as an extension without major architectural changes. If not, then post your ideas here and we can discuss them. Having said that, my own personal opinion is that this isn't very useful for a general tool. IMO, encryption and security will never passed the Grandma Test: could it be easy enough to understand and do correctly that your grandmother would use it? Encryption is complicated and security protocols are even harder to understand and implement right, let alone do so in a way that Normal People can grasp. I'm pessimistic that a pervasively encrypted environment will ever really exist, or will even be deemed necessary by the fast armies of grandmas on the 'net. (unlike web and email, which my son's grandma can do without much trouble) Let us know how it goes though. It definitely tickles my geek-libertarian funny bone! :) -Barry From chuqui@plaidworks.com Wed Nov 8 20:05:53 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 8 Nov 2000 12:05:53 -0800 Subject: [Mailman-Developers] Mailman and GPG. In-Reply-To: <14857.44202.507836.574952@anthem.concentric.net> References: <200011052255.RAA14775@alice-whacker.mit.edu> <14857.44202.507836.574952@anthem.concentric.net> Message-ID: At 2:42 PM -0500 11/8/00, Barry A. Warsaw wrote: >IMO, encryption and security will never >passed the Grandma Test: could it be easy enough to understand and do >correctly that your grandmother would use it? this is a place I disagree with Barry -- I think it will, but not until it's as easy to use as the web and email is. Which means the client authors need to decide it's necessary to integrate into the client tools, and serious enough about it to integrate in a non-geeky way (i.e., Eudora's PGP plug-ins ain't it). That means serious user interface design and integration. And I think it will -- but not soon. Why? the U.S. recently made digital signatures legally binding. that means encryption. And once people start needing (or wanting) digital signatures, that'll drive the integration of encryption. Until that happens, though, it'll continue to be a niche technology. A crucial one, but not one you can easily explain to mom and dad. This is a job for the client tools. It can be and should be done. But I don't see it happening soon, and I see government's globally fighting it every step of the way... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From jam@jamux.com Wed Nov 8 21:40:29 2000 From: jam@jamux.com (John A. Martin) Date: Wed, 08 Nov 2000 16:40:29 -0500 Subject: [Mailman-Developers] rc1 Kinky header on welcome message Message-ID: <20001108214029.467A248031@athene.jamux.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Using Mailman version 2.0rc1, the "confirmation of subscription -- request" and the first "welcome" message for a list contain the following header fields (others fields OK) - -------------- cut here ---->8 ---< head List-Subscribe: <../../listinfo/test-2>, List-Id: List-Unsubscribe: <../../listinfo/test-2>, List-Archive: - ---- 8<------- cut here ----------> tail while other mails, including welcome messages for subsequent subscribers seem OK as follows. - -------------- cut here ---->8 ---< head List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: - ---- 8<------- cut here ----------> tail My python skills are too infantile to see where it is that something seems to fail to get initialized. jam -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: OpenPGP encrypted mail preferred. See iEYEARECAAYFAjoJyDEACgkQUEvv1b/iXy8slQCdEwN7Hc151sdaM2RG1UrDYqyb 1HkAoJu0lvuR/eqiTQp9sMWVY81wo6cp =Z5x0 -----END PGP SIGNATURE----- From barry@wooz.org Wed Nov 8 22:21:42 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Wed, 8 Nov 2000 17:21:42 -0500 (EST) Subject: [Mailman-Developers] rc1 Kinky header on welcome message References: <20001108214029.467A248031@athene.jamux.com> Message-ID: <14857.53750.617120.581090@anthem.concentric.net> I believe this is fixed in rc2 (coming real soon now). From jarrell@vt.edu Wed Nov 8 23:22:42 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Wed, 08 Nov 2000 18:22:42 -0500 Subject: [Mailman-Developers] listinfo page link problem Message-ID: <5.0.0.25.2.20001108182108.04bd8c30@vtserf.cc.vt.edu> Interesting... Anyone else seeing this? On my nodename/mailman/listinfo page, where it says "List admins go the the administrative interface" that link points at nodename/admin Not mailman/admin... I'm on my way out the door, so I can't look at the code to see what's up, but I thought I'd mention it because otherwise I'll forget :-). From Dan Mick Thu Nov 9 00:04:32 2000 From: Dan Mick (Dan Mick) Date: Wed, 8 Nov 2000 16:04:32 -0800 (PST) Subject: [Mailman-Developers] listinfo page link problem Message-ID: <200011090003.QAA04094@utopia.west.sun.com> > Interesting... Anyone else seeing this? Not me. > On my > > nodename/mailman/listinfo > > page, where it says "List admins go the the administrative interface" that link points at > nodename/admin > > Not mailman/admin... Is your DEFAULT_URL set with a trailing /? From barry@wooz.org Thu Nov 9 01:59:27 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Wed, 8 Nov 2000 20:59:27 -0500 (EST) Subject: [Mailman-Developers] rc1 Possible confusion in Defaults.py comments References: <20001106133307.349FE48031@athene.jamux.com> Message-ID: <14858.1279.817235.627843@anthem.concentric.net> >>>>> "jam" == John A Martin writes: jam> Using Mailman version 2.0rc1, in Defaults.py, under Bounce jam> processing defaults, one sees: | # 0 means do nothing | # 1 means disable and send admin a report, | # 2 means nuke'em (remove) and send admin a report, | # 3 means nuke 'em and don't report (whee:) | DEFAULT_AUTOMATIC_BOUNCE_ACTION = 1 jam> and on the Bounce Options web page: jam> (Ie. from left to right, (0) do nothing, (1) disable and jam> notify, (2) disable and DON'T notify, and (3) Remove and jam> notify. Here's what the actual code does (see HandleBouncingAddress() in Mailiman/Bouncer.py): 0 = do nothing 1 = disable and notify 2 = disable and don't notify 3 = remove and notify So it looks like the web page is right but the comment in Defaults.py.in are wrong. I'll fix those. -Barry From barry@wooz.org Thu Nov 9 02:29:32 2000 From: barry@wooz.org (Barry A. Warsaw) Date: Wed, 8 Nov 2000 21:29:32 -0500 (EST) Subject: [Mailman-Developers] Slight oddity on ubsubscription References: Message-ID: <14858.3084.417234.540058@anthem.concentric.net> >>>>> "NM" == Nigel Metheringham writes: NM> Here's a message I got from one of my subscribers - makes a NM> reasonable point [I have edited the address in there to NM> prevent it being exposed]. | So, I see that an old form of my address is subscribed to the | announce list. I look for a way to change the address, don't | see it, so unsubscribe. | The last four words there are a link; following that brings me | to a page stating: | exim-announce: No such member 'removed@to.protect.guilty'. | Perhaps someone who's unsubscribed shouldn't be prompted to | continue editting? I agree, this is bogus. I've filed a bug report on it, but probably will not fix it for 2.0. -Barry From jarrell@vt.edu Thu Nov 9 17:55:50 2000 From: jarrell@vt.edu (Ron Jarrell) Date: Thu, 09 Nov 2000 12:55:50 -0500 Subject: [Mailman-Developers] listinfo page link problem In-Reply-To: <200011090003.QAA04094@utopia.west.sun.com> Message-ID: <5.0.0.25.2.20001109125510.04b40790@vtserf.cc.vt.edu> At 04:04 PM 11/8/00 -0800, Dan Mick wrote: >> page, where it says "List admins go the the administrative interface" that >link points at >> nodename/admin >> >> Not mailman/admin... > >Is your DEFAULT_URL set with a trailing /? *Thwack*. Well... Feh. Apparently I fixed that on all machines except, of course, *that one*... Sigh. From mrlist@ActiveState.com Thu Nov 9 18:09:49 2000 From: mrlist@ActiveState.com (Eric Wang) Date: Thu, 09 Nov 2000 10:09:49 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions In-Reply-To: <3A06DD6C.D4F23706@gietl.com> References: <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> Message-ID: <20001109100851.E230.MRLIST@activestate.com> Has this problem been solved ? If so , please tell me how, thanks! On Mon, 06 Nov 2000 17:33:48 +0100 Andreas Gietl wrote: > any further ideas? > > i'm sorry, but there is no pipeline entry: > > that's the db-file: > { 'filebase': 'ccfe93988bf681ad271ad049fa54dcd7260a60ac', > 'listname': 'bittecomgietltest2', > 'torequest': 1, > 'version': 2} > > and this is the .msg file: > > >From gietl@gietl.com Sat Nov 04 13:52:46 2000 > Received: from [195.227.84.2] (helo=d1.x-mailer.de) > by d18.x-mailer.de with esmtp (Exim 3.16 #1) > id 13s2oQ-0003e1-00 > for bittecomgietltest2-request@bitte.com; Sat, 04 Nov 2000 > 13:52:46 +0100 > Received: from p3ee039ab.dip.t-dialin.net ([62.224.57.171] > helo=gietl.com) > by d1.x-mailer.de with esmtp (Exim 3.16 #1) > id 13s2n9-0004iW-00 > for bittecomgietltest2-request@bitte.com; Sat, 04 Nov 2000 > 13:51:27 +0100 > Message-ID: <3A0406EC.6C4B8583@gietl.com> > Date: Sat, 04 Nov 2000 13:54:04 +0100 > From: Andreas Gietl > X-Mailer: Mozilla 4.73 [en] (WinNT; I) > X-Accept-Language: en > MIME-Version: 1.0 > To: bittecomgietltest2-request@bitte.com > Subject: Re: Bittecomgietltest2 -- confirmation of subscription -- > request 569170 > References: > Content-Type: text/plain; charset=iso-8859-1 > Content-Transfer-Encoding: 8bit > > > > bittecomgietltest2-request@bitte.com wrote: > > > > Bittecomgietltest2 -- confirmation of subscription -- request 569170 > > > > We have received a request from p3ee039ab.dip.t-dialin.net for > > subscription of your email address, , to the > > bittecomgietltest2@bitte.com mailing list. To confirm the request, > > please send a message to bittecomgietltest2-request@bitte.com, and > > either: > > > > - maintain the subject line as is (the reply's additional "Re:" is > > ok), > > > > - or include the following line - and only the following line - in the > > message body: > > > > confirm 569170 > > > > (Simply sending a 'reply' to this message should work from most email > > interfaces, since that usually leaves the subject line in the right > > form.) > > > > If you do not wish to subscribe to this list, please simply disregard > > this message. Send questions to bittecomgietltest2-admin@bitte.com. > > barry@wooz.org wrote: > > > > >>>>> "AG" == Andreas Gietl writes: > > > > >> Do you still see the > > >> confirmation message in qfiles? > > > > AG> Yeah, the confirmation message is in there! > > > > >> If so, are there any obvious errors > > >> in logs/error? > > > > AG> nothing. i of course checked that! > > > > >> Does it have the expected Subject: header (i.e. at the > > >> end of the line, "request xxxxxx" where those x's are the > > >> numbers). > > > > AG> Yeah. it does have. > > AG> Do you need any further information? > > > > Yes, do a bin/dumpdb qfiles/.db and look for the > > `pipeline' entry. > > > > That should show us where things are getting stuck. > > -Barry > > > > ------------------------------------------------------ > > Mailman-Users maillist - Mailman-Users@python.org > > http://www.python.org/mailman/listinfo/mailman-users > > -- > andreas gietl > gietl internet services > fon +49 9402 2551 > fax +49 9402 2604 > mobile +49 171 60 70 008 > gietl@gietl.com > > ############################################ > # Das Handbuch sagt, das Programm ben~{vt~}ige # > # Windows 95 oder besser. Also habe ich # > # Linux installiert! # > ############################################ > > ------------------------------------------------------ > Mailman-Users maillist - Mailman-Users@python.org > http://www.python.org/mailman/listinfo/mailman-users From barry@wooz.org Thu Nov 9 19:13:30 2000 From: barry@wooz.org (barry@wooz.org) Date: Thu, 9 Nov 2000 14:13:30 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> Message-ID: <14858.63322.557304.241298@anthem.concentric.net> Trying to come back to this problem. It looks like Eric Wang is also seeing problems, so there must be /something/ going on. I can't figure out what though, so perhaps one of you guys can give me ssh access to the host in question? >>>>> "AG" == Andreas Gietl writes: AG> i'm sorry, but there is no pipeline entry: | that's the db-file: | { 'filebase': 'ccfe93988bf681ad271ad049fa54dcd7260a60ac', | 'listname': 'bittecomgietltest2', | 'torequest': 1, | 'version': 2} Okay, this looks pretty good, assuming the file is qfiles/ccfe93988bf681ad271ad049fa54dcd7260a60ac.db We don't get a `pipeline' key in the dictionary because, by virtue of the `torequest' key, qrunner sends this message through the email command parser instead of the delivery pipeline. So far so good. AG> and this is the .msg file: [...most headers deleted...] | To: bittecomgietltest2-request@bitte.com | Subject: Re: Bittecomgietltest2 -- confirmation of subscription -- | request 569170 | References: | Content-Type: text/plain; charset=iso-8859-1 | Content-Transfer-Encoding: 8bit I notice that the Subject: line is wrapped, but not in an rfc 822 valid way. I'm assuming that something got munged in the forwarding of the message to this list, and that the text `request 569170' is actually at the end of the Subject: line. In my own tests, if the message looked exactly as you sent it, with the line being wrapped and `request' showing in column zero, the mailcmd parser will generate an error message back to the sender. But you're not seeing that. If the `request' line were wrapped a la rfc 822, Mailman should be able to grok out the command just fine. But you're not seeing that either. Another log file to look at: doe you see anything in logs/bounce? I don't expect you to -- just wondering. We're at the point of using debug statements now, and I don't want to bore mailman-users with that. If I can't get ssh access to your installation, let's carry on the debugging conversation on mailman-developers only. -Barry From chuqui@plaidworks.com Thu Nov 9 19:38:02 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Thu, 9 Nov 2000 11:38:02 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions In-Reply-To: <14858.63322.557304.241298@anthem.concentric.net> References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> Message-ID: At 2:13 PM -0500 11/9/00, barry@wooz.org wrote: > | Subject: Re: Bittecomgietltest2 -- confirmation of subscription -- > | request 569170 I had one of these this week. the Subject line was wrapped 'request\n\t' and mailman didn't handle it. To my knowledge subject should never wrap anyway, and when I looked in my inbox archives, it was the users mail client that had done it. It looked like a client issue, but it gave us the clue to help him work around it. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From dans@audifans.com Thu Nov 9 19:51:20 2000 From: dans@audifans.com (Dan Simoes) Date: Thu, 09 Nov 2000 14:51:20 -0500 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> Message-ID: <3A0B0038.CD725B31@audifans.com> barry@wooz.org wrote: > > Trying to come back to this problem. It looks like Eric Wang is also > seeing problems, so there must be /something/ going on. I can't > figure out what though, so perhaps one of you guys can give me ssh > access to the host in question? I've also reported the same problems. After much woe, I am back at beta6, which seems to have fixed my web interface problems and does not present the confirmation issue. I would love to help, but at this point I have over 2000 pissed off users, and I can't risk another upgrade to test. I may do so in the future, but only if I have a solid way of backing out. I am also strongly considering a return to majordomo. Thanks, | Dan | From barry@wooz.org Thu Nov 9 20:38:20 2000 From: barry@wooz.org (barry@wooz.org) Date: Thu, 9 Nov 2000 15:38:20 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> <3A0B0038.CD725B31@audifans.com> Message-ID: <14859.2876.127155.739471@anthem.concentric.net> >>>>> "DS" == Dan Simoes writes: >> Trying to come back to this problem. It looks like Eric Wang >> is also seeing problems, so there must be /something/ going on. >> I can't figure out what though, so perhaps one of you guys can >> give me ssh access to the host in question? DS> I've also reported the same problems. After much woe, I am DS> back at beta6, which seems to have fixed my web interface DS> problems and does not present the confirmation issue. We tracked Andreas' problem down to the qrunner cron job not being installed, a common problem for upgraders from what I've seen. Please first be absolutely sure this isn't your problem! You can test this by cd'ing into /home/mailman and running % python -S cron/qrunner if that unclogs things, your cron isn't set up right. If not, then we'll need to look at things more closely. To start with, you can send me the output of "bin/dumpdb qfiles/whatever.db" and the associated .msg file. Also, look in the log files, especially logs/error and see if there's anything relevant. DS> I would love to help, but at this point I have over 2000 DS> pissed off users, and I can't risk another upgrade to test. I DS> may do so in the future, but only if I have a solid way of DS> backing out. Well, you may have to set up a test list in a different installation until you're confident that 2.0rc1 works for you. DS> I am also strongly considering a return to majordomo. I hope that won't be necessary, and I'm willing to help debug the problem. -Barry From barry@wooz.org Thu Nov 9 20:42:49 2000 From: barry@wooz.org (barry@wooz.org) Date: Thu, 9 Nov 2000 15:42:49 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> Message-ID: <14859.3145.597151.979023@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> I had one of these this week. the Subject line was wrapped CVR> 'request\n\t' and mailman didn't handle it. To my CVR> knowledge subject should never wrap anyway, and when I looked CVR> in my inbox archives, it was the users mail client that had CVR> done it. It looked like a client issue, but it gave us the CVR> clue to help him work around it. Hmm. I just tested this (wrapping before and after `request' with \n\t) and it works for me. The rfc822 module /should/ be able to handle this and return a Subject with the continuation lines. -Barry From Dan.Mick@west.sun.com Thu Nov 9 21:10:16 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Thu, 09 Nov 2000 13:10:16 -0800 Subject: [Mailman-Developers] listinfo page link problem References: <5.0.0.25.2.20001109125510.04b40790@vtserf.cc.vt.edu> Message-ID: <3A0B12B8.97E3B928@west.sun.com> Ron Jarrell wrote: > > At 04:04 PM 11/8/00 -0800, Dan Mick wrote: > >> page, where it says "List admins go the the administrative interface" that > >link points at > >> nodename/admin > >> > >> Not mailman/admin... > > > >Is your DEFAULT_URL set with a trailing /? > > *Thwack*. Well... Feh. Apparently I fixed that on all machines except, of course, *that one*... > Sigh. Barry, might this be worth checking for, and warning about in error or somewhere? From Dan.Mick@west.sun.com Thu Nov 9 21:24:53 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Thu, 09 Nov 2000 13:24:53 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] forever pending subscriptions References: <3A02C6B0.2D82FC79@gietl.com> <3A032199.67D1CA2E@west.sun.com> <14851.11014.397538.19202@anthem.concentric.net> <3A0407E2.42ABB0AF@gietl.com> <14852.12081.57421.807051@anthem.concentric.net> <3A06DD6C.D4F23706@gietl.com> <14858.63322.557304.241298@anthem.concentric.net> <3A0B0038.CD725B31@audifans.com> <14859.2876.127155.739471@anthem.concentric.net> Message-ID: <3A0B1625.A270241@west.sun.com> barry@wooz.org wrote: > We tracked Andreas' problem down to the qrunner cron job not being > installed Well, there's a new one. Not mentioned anywhere in the installation docs, either. Glad we had to get Barry to debug this one. From vikas98@hotmail.com Fri Nov 10 07:54:34 2000 From: vikas98@hotmail.com (Vikas Gupta) Date: Fri, 10 Nov 2000 07:54:34 GMT Subject: [Mailman-Developers] [please heklp!!]cannot list public lists from listinfo , can from admin Message-ID: I know this has been asked on this list before but I think I have a new twist which I cant figure out. I have created a public list, which is listed fine under /admin, but does not appear under /listinfo. I get the : "There currently are no publicly-advertised mailman mailing lists" message. I believe I have folloed the INSTALL instrctions correctly for mailman and apache. Can anyone help me out? Thanks in advance, Vikas _________________________________________________________________________ Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com. Share information about yourself, create your own public profile at http://profiles.msn.com. From Nigel.Metheringham@VData.co.uk Fri Nov 10 10:07:35 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Fri, 10 Nov 2000 10:07:35 +0000 Subject: [Mailman-Developers] [please heklp!!]cannot list public lists from listinfo , can from admin In-Reply-To: Message from "Vikas Gupta" of "Fri, 10 Nov 2000 07:54:34 GMT." Message-ID: [I think this is a mailman-users issue - *please* can we try and get the list splits appropriate - mailman-developers is not just a means of getting more clued (or less low-clued) responses for a problem because you think its urgent] vikas98@hotmail.com said: > I know this has been asked on this list before but I think I have a > new twist which I cant figure out. > I have created a public list, which is listed fine under /admin, but > does not appear under /listinfo. I get the : > "There currently are no publicly-advertised mailman mailing lists" > message. It appears that Mailman only displays lists that match the URL prefix of your web server for mailman/listinfo whereas it displays all lists known to that installation no matter what the URL prefix for mailman/admin For an example of this see the exim.org box which runs lists for both exim.org and wylug.org.uk http://www.exim.org/mailman/listinfo/ - lists defined within exim.org http://list.wylug.org.uk/mailman/listinfo/ - lists on wylug.org.uk http://www.exim.org/mailman/admin/ - all lists on that machine So I guess you need to go to the admin pages for your lists and modify the "Base URL for Mailman web interface. " at the bottom of the "General Options" page. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From dc-mmdev@dragonwerks.net Fri Nov 10 09:46:47 2000 From: dc-mmdev@dragonwerks.net (dc-mmdev@dragonwerks.net) Date: 10 Nov 2000 02:46:47 -0700 Subject: from listinfo , can from adminRe: [Mailman-Developers] [please heklp!!]cannot list public lists In-Reply-To: Message-ID: <20001110024647.1.10248.qmail@dragonwerks.net> On Fri, 10 Nov 2000, Nigel Metheringham wrote: > It appears that Mailman only displays lists that match the URL prefix > of your web server for mailman/listinfo whereas it displays all lists > known to that installation no matter what the URL prefix for > mailman/admin > > So I guess you need to go to the admin pages for your lists and modify > the "Base URL for Mailman web interface. " at the bottom of the > "General Options" page. > This only works if you don't have any wierd capitization in your DNS names, if you do setting the base url won't help, however "VIRTUAL_HOST_OVERVIEW = 0" will fix it. -- DanielC From barry@digicool.com Fri Nov 10 23:45:35 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 18:45:35 -0500 (EST) Subject: [Mailman-Developers] Patch #102268 References: <14852.18163.486910.710242@j218.ryd.student.liu.se> Message-ID: <14860.34975.617122.912354@anthem.concentric.net> >>>>> "EF" == Erik Forsberg writes: EF> I submitted a patch (at sourceforge) earlier today, EF> #102268. It's basically a two-line-change to EF> MailMan/Archiver/HyperArch.py in 2.0rc1 that makes the EF> Content-Transfer-Encoding all lowercase. Hopefully I produced EF> a working patch-file.. You did, I just had to do it differently. :) See the updated patch on SF, or grab the CVS snapshot. -Barry From barry@digicool.com Fri Nov 10 23:47:08 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 18:47:08 -0500 (EST) Subject: [Mailman-Developers] Patch #102268 References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <3A049D72.22801714@west.sun.com> <14852.40577.83440.45618@j218.ryd.student.liu.se> <3A04A00C.A4896940@west.sun.com> Message-ID: <14860.35068.507104.805480@anthem.concentric.net> >>>>> "DM" == Dan Mick writes: DM> Right, but that's "augment the archives", which is not what DM> arch tries to do. If you want to rebuild, delete or move the DM> existing archives, and rebuild, and you'll get the result you DM> expect. Which I actually think is bogus. bin/arch should have an option to wipe the old archive and rebuild from scratch from the .mbox file. Not for 2.0 though. -Barry From barry@digicool.com Sat Nov 11 01:56:39 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 20:56:39 -0500 (EST) Subject: [Mailman-Developers] Direct Python access tutorial wanted References: <20001103194303.H25116@cheops.usf.Uni-Osnabrueck.DE> Message-ID: <14860.42839.447275.999218@anthem.concentric.net> >>>>> "BR" == Bernhard Reiter writes: BR> Hi Mailman developers, BR> just noticed that the minimal instructions for list server BR> administrators to mass manipulate mailinglists directly have BR> been vanished from the mailman README file in the jump to BR> version 2. (last cvs revision containg the advice is README BR> rev1.45) BR> As I considered it quite useful my suggestion is to add it BR> again to the mailman documentation. BR> Or is it not possible or advisable to do BR> this kind of list surgery anymore? | Regards, | Bernhard BR> ps: please cc relevant replies to me as I am not subscribe to BR> the list Bernard, In Mailman 2.0, the bin/withlist script is provided as a better framework for doing these kinds of manipulations. -Barry From barry@digicool.com Sat Nov 11 02:01:57 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 21:01:57 -0500 (EST) Subject: [Mailman-Developers] passwords in third party web archives, newsgroups References: <3A05F488.57CFC441@mail.lns.cornell.edu> Message-ID: <14860.43157.667139.522237@anthem.concentric.net> >>>>> "DR" == Dan Riley writes: DR> Are the mailman developers at all concerned by DR> http://www.google.com/search?q=mailman-owner+reminder+password DR> http://x66.deja.com/=dnc/getdoc.xp?AN=641175690 Yes, but it's too late to do anything about this in Mailman 2.0. Individual users should be able to disable password reminders to their address. Such archiver "false-users" can then just have this turned off. I'm glad there's a workaround for now! -Barry From barry@wooz.org Sat Nov 11 02:11:12 2000 From: barry@wooz.org (barry@wooz.org) Date: Fri, 10 Nov 2000 21:11:12 -0500 (EST) Subject: [Mailman-Developers] mailman installation error References: <3A034AC5.4191FD96@bmedesign.com> <5.0.0.25.2.20001106103101.03094c30@vtserf.cc.vt.edu> <3A0723D5.C7398BC7@west.sun.com> Message-ID: <14860.43712.22670.651590@anthem.concentric.net> >>>>> "DM" == Dan Mick writes: DM> It's hard to imagine how to make this more clear, though; it DM> *is* step 3, which is numbered, and after the "make install" DM> step, and it *does* say DM> - cd to $prefix DM> - Run bin/check_perms How about this? [in-source-directory]% bin/check_perms Could not import paths! This probably means that you are trying to run check_perms from the source directory. You must run this from the installation directory instead. Traceback (most recent call last): File "bin/check_perms", line 38, in ? import paths ImportError: No module named paths -Barry From barry@digicool.com Sat Nov 11 04:24:05 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 10 Nov 2000 23:24:05 -0500 (EST) Subject: [Mailman-Developers] Announcing Mailman 2.0 release candidate 2 Message-ID: <14860.51685.368139.841987@anthem.concentric.net> This is it. Mailman 2.0 release candidate 2 is now available from SourceForge at http://sourceforge.net/project/showfiles.php?group_id=103 www.list.org and www.gnu.org should be updated soon. Excerpts from the NEWS file are given below. Of primary importance in this release is the fix of the last known mail duplication bug, and updated on-line documentation. You can view the documentation at http://mailman.sourceforge.net Please let me know if you find any errors in the docs. Unless something's royally screwed, those are the only changes I'll make before 2.0 final. Plan for that one week from today: Friday November 17th. Enjoy, -Barry -------------------- snip snip -------------------- 2.0 release candidate 2 (10-Nov-2000) - Documentation updates: start at admin/www/index.html - bin/withlist accepts additional command line arguments when used with the --run flag; bin/mmsitepass and bin/newlist accept -h/--help flags - bin/newlist has a -o/--output flag to append /etc/aliases suggestions to a specified file - SourceForge bugs fixed: 116615 (README.BSD update), 117015 (duplicate messages on moderated posts), 117548 (exception in HyperArch.py), 117682 (typos), 121185 (vsnprintf signature), 121591 and 122017 (bogus link after web unsubscribe), 121811 (`subscribe' in Subject: doesn't get archived) - SourceForge patches applied: 101812 (securelinux_fix.py contrib), 102097 (fix for bug 117548), 102211 (additional args for withlist), 102268 (case insensitive Content-Transfer-Encoding:) From ocschwar@MIT.EDU Sat Nov 11 05:40:44 2000 From: ocschwar@MIT.EDU (The guy named after an Om Kalthoum song) Date: Sat, 11 Nov 2000 00:40:44 EST Subject: [Mailman-Developers] Mea Culpa. Message-ID: <200011110540.AAA17093@lola-granola.mit.edu> When I first set out on this I was already planning on setting up a server with an encrypted mailing list in order to invite some friends of mine to try out the social dimension of it. So, I quickly concocted a gross misunderstanding in my mind about how mail transfer agents and mailing list managers divide up their duties. Thank you for correcting me on that. I also was a little optimistic on the idea that crypto email fora being the final element to establish pervasive crypto. A look at cypherpunks shows I'm dead wrong. What I want to do was attempted in the past and the code just died from obscurity. But, let's say I still set out on this: On the MTA side (I'd probably diff Postfix) I would have to enable the MTA to know to divert mail coming in to certain lists over to a side spool, activate the crypto-exploder, and then spool it to outgoing. Then comes writing the crypto-exploder, which would be a simple Perl or Python script invoking relevant the GPG and MTA modules. On the MLM side, all that really is necessary is for Mailman to be able to collect and revoke public keys(/etc/pki?), and deliver its own public key to those who request it. A host-owned (rather than user-owned) key ring has been discouraged in theory, since it would prolong the life-span of a revoked public key. Any server that used one would need to check in with a keyserver on a cronly basis. Regardless of the MTA issue, a GPG-enabled Mailman would be convenient. You could automatically process signed transaction request emails, and have the admin manually process unsigned ones, for example. So if I do it I hope you'll accept the plug in. Regards, -- Omri Schwarz I get serious letters from university students, asking questions for a project they are doing - these are not much different from those I get from school-children (written in green crayon), except the writing is a little worse. -- Terry Pratchett, Warwick Uni (10.11.94) From John Block Sat Nov 11 11:04:46 2000 From: John Block (John Block) Date: Sat, 11 Nov 2000 11:04:46 +0000 Subject: [Mailman-Developers] installation instruction suggestions In-Reply-To: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: Hello Barry > Please let me know if you find any errors in the docs. The odd line needs putting in to help the less able Set Ftp to Binary tar zxvf mailman-xxxx.tgz (I had to get the zxvf from support) Under configure options, it is not clear what to do. Is it tap in the command configure --option1 --option2 or is it edit the configure file, changing the lines... If it is edit the configure file, then putting these variable apart in an editable section and giving commented out examples would help. Thanks, John Unless > something's royally screwed, those are the only changes I'll make > before 2.0 final. Plan for that one week from today: Friday November > 17th. > > Enjoy, > -Barry > > -------------------- snip snip -------------------- > 2.0 release candidate 2 (10-Nov-2000) > > - Documentation updates: start at admin/www/index.html > > - bin/withlist accepts additional command line arguments when used > with the --run flag; bin/mmsitepass and bin/newlist accept > -h/--help flags > > - bin/newlist has a -o/--output flag to append /etc/aliases > suggestions to a specified file > > - SourceForge bugs fixed: > 116615 (README.BSD update), 117015 (duplicate messages on > moderated posts), 117548 (exception in HyperArch.py), 117682 > (typos), 121185 (vsnprintf signature), 121591 and 122017 > (bogus link after web unsubscribe), 121811 (`subscribe' in > Subject: doesn't get archived) > > - SourceForge patches applied: > 101812 (securelinux_fix.py contrib), 102097 (fix for bug > 117548), 102211 (additional args for withlist), 102268 (case > insensitive Content-Transfer-Encoding:) > > > _______________________________________________ > Mailman-announce mailing list > Mailman-announce@python.org > http://www.python.org/mailman/listinfo/mailman-announce > Regards From John Block Sat Nov 11 11:23:04 2000 From: John Block (John Block) Date: Sat, 11 Nov 2000 11:23:04 +0000 Subject: [Mailman-Developers] installation instruction suggestions In-Reply-To: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: Hello Barry > Please let me know if you find any errors in the docs. The odd line needs putting in to help the less able Set Ftp to Binary tar zxvf mailman-xxxx.tgz (I had to get the zxvf from support) Under configure options, it is not clear what to do. Is it tap in the command configure --option1 --option2 or is it edit the configure file, changing the lines... If it is edit the configure file, then putting these variable apart in an editable section and giving commented out examples would help. Thanks, John From ge204@eng.cam.ac.uk Sat Nov 11 18:00:24 2000 From: ge204@eng.cam.ac.uk (Gunnar Evermann) Date: 11 Nov 2000 18:00:24 +0000 Subject: [Mailman-Developers] Announcing Mailman 2.0 release candidate 2 In-Reply-To: barry@digicool.com's message of "Fri, 10 Nov 2000 23:24:05 -0500 (EST)" References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > This is it. Mailman 2.0 release candidate 2 is now available from > SourceForge at ok, I've just installed this from scratch. > - SourceForge patches applied: > 102268 (case insensitive Content-Transfer-Encoding:) This fix is wrong AFAICT. If an article doesn't contain a C-T-E header then I get the following from running the archiver list@casablanca:/var/lib/mailman$ bin/arch mailman-test-list-a Traceback (innermost last): File "bin/arch", line 129, in ? main() File "bin/arch", line 118, in main archiver.processUnixMailbox(fp, Article) File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 521, in processUnixMailbox a = articleClass(m, self.sequence) File "/var/lib/mailman/Mailman/Archiver/HyperArch.py", line 224, in __init__ self.cenc = string.lower(cenc) TypeError: read-only character buffer, None I just started learning Python, so I'll leave the (trivial) fix to you, Barry. Just wanted to let you know before you release this :-) Gunnar From bernhard@intevation.de Sat Nov 11 19:40:26 2000 From: bernhard@intevation.de (Bernhard Reiter) Date: Sat, 11 Nov 2000 20:40:26 +0100 Subject: [Mailman-Developers] Direct Python access tutorial wanted In-Reply-To: <14860.42839.447275.999218@anthem.concentric.net>; from barry@digicool.com on Fri, Nov 10, 2000 at 08:56:39PM -0500 References: <20001103194303.H25116@cheops.usf.Uni-Osnabrueck.DE> <14860.42839.447275.999218@anthem.concentric.net> Message-ID: <20001111204026.B24404@cheops.usf.Uni-Osnabrueck.DE> --NDin8bjvE/0mNLFQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 10, 2000 at 08:56:39PM -0500, Barry A. Warsaw wrote: > >>>>> "BR" =3D=3D Bernhard Reiter writes: >=20 > BR> Hi Mailman developers, >=20 > BR> just noticed that the minimal instructions for list server > BR> administrators to mass manipulate mailinglists directly have > BR> been vanished=20 > In Mailman 2.0, the bin/withlist script is provided as a better > framework for doing these kinds of manipulations. Good to know! It should be mentioned in the same place in the README file. (or did I read over it? :>) I found out that the old method also works. Why remove the description for low level hacking, if it is still valid? Thanks, Bernhard --=20 Professional Service around Free Software (intevation.net) = =20 The FreeGIS Project (freegis.org) Association for a Free Informational Infrastructure (ffii.org) --NDin8bjvE/0mNLFQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.3 (SunOS) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjoNoKoACgkQh9ag3dpKERZUIgCfT3xG1Lm73m4K5jxSKmIjT1Cz v3YAn2CP6DRNDCEoJhG4qOlva+CW2DJU =Gpxf -----END PGP SIGNATURE----- --NDin8bjvE/0mNLFQ-- From forsberg@lysator.liu.se Sun Nov 12 01:29:24 2000 From: forsberg@lysator.liu.se (Erik Forsberg) Date: Sun, 12 Nov 2000 02:29:24 +0100 (CET) Subject: [Mailman-Developers] Patch #102268 In-Reply-To: <14860.34975.617122.912354@anthem.concentric.net> References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <14860.34975.617122.912354@anthem.concentric.net> Message-ID: <14861.62068.946729.503549@j218.ryd.student.liu.se> Barry A. Warsaw writes: >You did, I just had to do it differently. :) But of course :-) >See the updated patch on, or grab the CVS snapshot. I would do it a bit different, for two reasons. 1) What happends if there is no Content-Transfer-Encoding header in the mail? Well.. cenc will get the "value" None. What happends if you do a string.lower on None? >>> string.lower(None) Traceback (innermost last): File "", line 1, in ? TypeError: read-only character buffer, None So guess why my 'if' where there.. And yes, I had a mail where there were no such header. Later in the code, there is a check whether self.cenc is None, so it should hold that value. 2) There is a perhaps more readable way of setting ctype to 'text/plain' if there were no such header: `getheader(name[, default])' Like `getrawheader(NAME)', but strip leading and trailing whitespace. Internal whitespace is not stripped. The optional DEFAULT argument can be used to specify a different default to be Mostly a cosmetic change, though. The current code looks a bit too much like Perl to me ;). Revised patch follows (Note: I edited the patch by hand. Will verify and upload to sourceforge sometime tomorrow^H^H^H^H^H^Hday (when it's not 02:27 in the morning..)): Index: HyperArch.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/Archiver/HyperArch.py,v retrieving revision 1.44 diff -u -r1.44 HyperArch.py --- HyperArch.py 2000/11/10 15:24:53 1.44 +++ HyperArch.py 2000/11/10 23:44:17 @@ -216,9 +216,12 @@ if mm_cfg.ARCHIVER_OBSCURES_EMAILADDRS: self.email = re.sub('@', ' at ', self.email) - # snag the content-type - self.ctype = message.getheader('Content-Type') or "text/plain" - self.cenc = message.getheader('Content-Transfer-Encoding') + # Snag the content-* headers. RFC 1521 states that their values are + # case insensitive. + ctype = message.getheader('Content-Type', "text/plain") + self.cenc = message.getheader('Content-Transfer-Encoding') + self.ctype = string.lower(ctype) + if None != self.cenc: + self.cenc = string.lower(self.cenc) self.decoded = {} mo = rx_charset.search(self.ctype) if mo: \EF -- Erik Forsberg http://www.lysator.liu.se/~forsberg/ PGP fingerprint = 37 1B D7 9E 97 8C EF 39 0E DE 08 E8 99 0C 5E A9 From gconnor@nekodojo.org Sun Nov 12 10:11:30 2000 From: gconnor@nekodojo.org (Greg Connor) Date: Sun, 12 Nov 2000 02:11:30 -0800 Subject: [Mailman-Developers] Re: [Mailman-Announce] Announcing Mailman 2.0 release candidate 2 References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: <3A0E6CD2.7CEBD4C@nekodojo.org> "Barry A. Warsaw" wrote: > > This is it. Mailman 2.0 release candidate 2 is now available from > SourceForge at Thanks for the heads-up, and thanks for a great product. I installed 2.0rc2, and I had a problem with lists that had administrative requests pending. I had to remove all the pending requests before "update" would run successfully. I know it's probably a bit late to fix, but I wanted to let you know... I posted to mailman-users with details and a log of my install. Good luck! -- Greg Connor From forsberg@lysator.liu.se Sun Nov 12 19:23:32 2000 From: forsberg@lysator.liu.se (Erik Forsberg) Date: Sun, 12 Nov 2000 20:23:32 +0100 (CET) Subject: [Mailman-Developers] Patch #102268 In-Reply-To: <14861.62068.946729.503549@j218.ryd.student.liu.se> References: <14852.18163.486910.710242@j218.ryd.student.liu.se> <14860.34975.617122.912354@anthem.concentric.net> <14861.62068.946729.503549@j218.ryd.student.liu.se> Message-ID: <14862.60980.140238.458331@j218.ryd.student.liu.se> > >Revised patch follows (Note: I edited the patch by hand. Will verify >and upload to sourceforge sometime tomorrow^H^H^H^H^H^Hday (when it's >not 02:27 in the morning..)): And as usual, when it's 2 o'clock in the morning nothing valuable is produced. The patch I sent didn't work 100%. I've now resubmitted the patch to sourceforge. It's a little bit different too - It doesn't use a if statement, instead it sets cenc to "" if there was no such header. The check later in the code will work as good if self.cenc is set to "" as if it were set to None. I've tested the code in production. Note: I submitted two patches, since I made a very silly mistake at first. It's even worse that now it's not 2 in the morning, since now I have nothing to blame :-). \EF -- Erik Forsberg http://www.lysator.liu.se/~forsberg/ PGP fingerprint = 37 1B D7 9E 97 8C EF 39 0E DE 08 E8 99 0C 5E A9 From vizisz@freemail.hu Mon Nov 13 09:38:03 2000 From: vizisz@freemail.hu (Vizi Szilard) Date: Mon, 13 Nov 2000 10:38:03 +0100 Subject: [Mailman-Developers] Suggestions 4 Mailman Message-ID: <3A0FB67B.919D5121@freemail.hu> Hello! I am using Mailman for 5 small mailing lists, and I have some advice. I have not read through the developers archive yet, so forgive me if I say something that someone already said. It would be better if the monthly reminder contains the current settings for both the user and the list. Like this: -----------------------reminder-------------------------------- YOUR Listname PASSWORD IS: password YOUR Listname SETTINGS ARE: [ack] [digest] [plain] [nomail] [norcv] [hide] For explanations see the mail called HELP. The Listname SETTINGS ARE: [public/private/hidden] [maxmessage size=xxx kB] [archive is public/privat (authorization needed] [subscribers list is public/privat/not avaliable] admin address is: name@some.where Any comments, question should be send to the admin. -----------------------reminder-------------------------------- Okay, I know that this can be achived with an "options" command, but the dummy users never use the commands. Even a few of them do not understand the differences between the listname and the listname-request email addresses. And the Listname settings can help the users how big could be a message, eg. why can not be seen the listname at the web interface (because it is hidden), and who is managing the mailinglist. And here is my second problem. Can be Mailman a bit more intelligent? If I have a user who makes the same mistakes, Mailman can solve the problem without any admin interaction. eg. the user posts a message but the address is in the BCC header, not in the To header. Mailman processes the message like the address is in the To header. Or there is a user who always post "funny" pictures, sometimes he/she posts a bigger picture, than the list setting allows. Mailman processes this message for this user only, other users with big messages should wait for the admin interaction. a.s.o Thats all, I think. Thanx. Szilard Vizi From mats@laplaza.org Mon Nov 13 19:48:12 2000 From: mats@laplaza.org (Mats Wichmann) Date: Mon, 13 Nov 2000 12:48:12 -0700 Subject: [Mailman-Developers] Re: [Mailman-Users] installation instruction suggestions In-Reply-To: References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: <5.0.0.25.1.20001113124615.00a61eb0@mail.laplaza.org> >The odd line needs putting in to help the less able > >Set Ftp to Binary Just musing... is there actually any benefit to EVER having ftp not set to binary? Why do we still have to fight this battle? From ralph@inputplus.demon.co.uk Mon Nov 13 23:46:04 2000 From: ralph@inputplus.demon.co.uk (Ralph Corderoy) Date: Mon, 13 Nov 2000 23:46:04 +0000 Subject: [Mailman-Developers] Re: [Mailman-Users] installation instruction suggestions In-Reply-To: Message from Mats Wichmann of "Mon, 13 Nov 2000 12:48:12 MST." <5.0.0.25.1.20001113124615.00a61eb0@mail.laplaza.org> Message-ID: <200011132346.XAA04047@inputplus.demon.co.uk> Hi, > > The odd line needs putting in to help the less able > > > > Set Ftp to Binary > > Just musing... is there actually any benefit to EVER having ftp not > set to binary? Why do we still have to fight this battle? Transferring text files between systems with different formats for text files can cause corrupt files if the transfer mode isn't text. Transferring binary files between systems with different formats for text files causes corrupt files if the transfer mode is text. Ralph. From barry@digicool.com Tue Nov 14 04:51:00 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Mon, 13 Nov 2000 23:51:00 -0500 (EST) Subject: [Mailman-Developers] Re: [Mailman-Announce] Announcing Mailman 2.0 release candidate 2 References: <14860.51685.368139.841987@anthem.concentric.net> <3A0E6CD2.7CEBD4C@nekodojo.org> Message-ID: <14864.50356.152336.517891@anthem.concentric.net> >>>>> "GC" == Greg Connor writes: GC> Thanks for the heads-up, and thanks for a great product. You're welcome! GC> I installed 2.0rc2, and I had a problem with lists that had GC> administrative requests pending. I had to remove all the GC> pending requests before "update" would run successfully. I GC> know it's probably a bit late to fix, but I wanted to let you GC> know... I posted to mailman-users with details and a log of GC> my install. Yup. There's a buglet in CheckVersion() when there are pending requests. Some older versions used a different requests database format, and Mailman's auto-update feature tries to repopulate using the new database format. This can only happen when the list is locked, but the new update script doesn't lock the list right away (to avoid hangs while updating). Here's a patch to fix the problem. -Barry -------------------- snip snip -------------------- Index: MailList.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/MailList.py,v retrieving revision 1.187 retrieving revision 1.188 diff -u -r1.187 -r1.188 --- MailList.py 2000/11/10 17:56:55 1.187 +++ MailList.py 2000/11/14 04:44:01 1.188 @@ -913,9 +913,10 @@ else: self.InitVars() # Init any new variables, self.Load(check_version = 0) # then reload the file - from versions import Update - Update(self, stored_state) - self.data_version = mm_cfg.DATA_FILE_VERSION + if self.Locked(): + from versions import Update + Update(self, stored_state) + self.data_version = mm_cfg.DATA_FILE_VERSION if self.Locked(): self.Save() From barry@digicool.com Tue Nov 14 04:51:35 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Mon, 13 Nov 2000 23:51:35 -0500 (EST) Subject: [Mailman-Developers] Announcing Mailman 2.0 release candidate 2 References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: <14864.50391.634350.998016@anthem.concentric.net> >>>>> "GE" == Gunnar Evermann writes: GE> I just started learning Python, so I'll leave the (trivial) GE> fix to you, Barry. Just wanted to let you know before you GE> release this :-) Thanks much! See my previously posted fix for this buglet. -Barry From midnight@the-oasis.net Tue Nov 14 05:25:28 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 00:25:28 -0500 Subject: [Mailman-Developers] Reverting question Message-ID: <3A108678.29865.5F12883@localhost> Am I likely to encounter problems going from 2.0rc2 to 2.0b6? Did any data formats change? -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From barry@digicool.com Tue Nov 14 05:30:45 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 14 Nov 2000 00:30:45 -0500 (EST) Subject: [Mailman-Developers] installation instruction suggestions References: <14860.51685.368139.841987@anthem.concentric.net> Message-ID: <14864.52741.977088.723846@anthem.concentric.net> >>>>> "JB" == John Block writes: JB> The odd line needs putting in to help the less able JB> Set Ftp to Binary JB> tar zxvf mailman-xxxx.tgz JB> (I had to get the zxvf from support) JB> Under configure options, it is not clear what to do. JB> Is it tap in the command JB> configure --option1 --option2 JB> or is it edit the configure file, changing the lines... JB> If it is edit the configure file, then putting these variable JB> apart in an editable section and giving commented out examples JB> would help. No, you don't need to edit the configure file, just include the options on the command line. Thanks for the feedback; I've added some extra text to the documents to clarify things. I sometimes forget that not everybody is a grisled and curmudgeonly ex-sysadmin like myself. :) dinosaurish-ly y'rs, -Barry From barry@digicool.com Tue Nov 14 05:32:46 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 14 Nov 2000 00:32:46 -0500 (EST) Subject: [Mailman-Developers] Reverting question References: <3A108678.29865.5F12883@localhost> Message-ID: <14864.52862.465404.35651@anthem.concentric.net> >>>>> "PB" == Phil Barnett writes: PB> Am I likely to encounter problems going from 2.0rc2 to 2.0b6? Do you mean downgrading? Don't do that! If you really mean from 2.0b6 to 2.0rc2 then you should be fine. Any data formats that need updating will get updated automatically. -Barry From midnight@the-oasis.net Tue Nov 14 05:37:27 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 00:37:27 -0500 Subject: [Mailman-Users] Re: [Mailman-Developers] Announcing Mailman 2.0 release candidate 2 In-Reply-To: <14864.51912.327124.393642@anthem.concentric.net> Message-ID: <3A108947.2828.5FC21DD@localhost> On 14 Nov 2000, at 0:16, Barry A. Warsaw wrote: > > >>>>> "PB" == Phil Barnett writes: > > PB> I upgraded from 2.0b6 from 2.0rc2 Friday night. A person wrote > PB> me and said the replyto: is no longer pointed to the list. I > PB> checked several messages and they were, but some were not. Any > PB> message I send to the list has a replyto of myself instead of > PB> to the list. > > Please remember that if the original message has a Reply-To: already, > Mailman will not overwrite that, even if "explicit reply-to" is set. > So, did the original mesasge have a Reply-To already? > > I just tested this using rc2. With reply_goes_to_list set to > "explicit address" and reply_to_address set to some non-blank value, > any message without a Reply-To: gets one set to that value. Any > message that already has a Reply-To: set is unchanged. That's > expected behavior (i.e. works for me!) Expected behavour for whom? For all the users on my lists, it has them screaming. They want to know why Mailman is broken after the upgrade. 2.0b6 did not exhibit this behaviour. How can I make 2.0rc2 work like b6 did? I think it's rather impossible to get all list users on the planet to remove their replyto so they can reply to the list on a list that replyto list is set. Expecially so since setting replyto is generally a global setting in most MUA's. Why was this changed? Is there a way to make Mailman to have this be selectable behaviour for it to work both ways? Not being able to force replyto to the list is broken behaviour in my users eyes, and, they are the ones who really count. My open source developers list is ready to switch back to egroups over this. That can't be good news. -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From midnight@the-oasis.net Tue Nov 14 05:43:23 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 00:43:23 -0500 Subject: [Mailman-Developers] Reverting question In-Reply-To: <14864.52862.465404.35651@anthem.concentric.net> Message-ID: <3A108AAB.25143.6018DAE@localhost> On 14 Nov 2000, at 0:32, Barry A. Warsaw wrote: > > >>>>> "PB" == Phil Barnett writes: > > PB> Am I likely to encounter problems going from 2.0rc2 to 2.0b6? > > Do you mean downgrading? Don't do that! If you really mean from > 2.0b6 to 2.0rc2 then you should be fine. Any data formats that need > updating will get updated automatically. I specifically mean downgrading. I have a hornets nest on my hands and I'm about to lose my lists back to egroups. This lack of reply to the list 'updated feature' has them HEATED. Perhaps I'm not clear. If I lose them to egroups, there is no point to me running Mailman anymore. This 'upgraded feature' is suicidal for Mailman on my machine. -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From simon@uow.edu.au Tue Nov 14 05:57:49 2000 From: simon@uow.edu.au (Simon Coggins) Date: Tue, 14 Nov 2000 16:57:49 +1100 Subject: [Mailman-Developers] Reverting question In-Reply-To: <3A108AAB.25143.6018DAE@localhost>; from midnight@the-oasis.net on Tue, Nov 14, 2000 at 12:43:23AM -0500 References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> Message-ID: <20001114165748.B3475@uow.edu.au> I have to agree here. Not being able to override the reply-to: fields on lists is a *MAJOr* problem. I've also had this problem with my lists. And I was hoping for a 'fix' but it looks like it's intended behaviour?. I think if thats going to be the case there should be another option. 'Force list replyto' . Regards Simon On Tue, Nov 14, 2000 at 12:43:23AM -0500, Phil Barnett wrote: > On 14 Nov 2000, at 0:32, Barry A. Warsaw wrote: > > > > > >>>>> "PB" == Phil Barnett writes: > > > > PB> Am I likely to encounter problems going from 2.0rc2 to 2.0b6? > > > > Do you mean downgrading? Don't do that! If you really mean from > > 2.0b6 to 2.0rc2 then you should be fine. Any data formats that need > > updating will get updated automatically. > > I specifically mean downgrading. I have a hornets nest on my > hands and I'm about to lose my lists back to egroups. > > This lack of reply to the list 'updated feature' has them HEATED. > > Perhaps I'm not clear. If I lose them to egroups, there is no point to > me running Mailman anymore. > > This 'upgraded feature' is suicidal for Mailman on my machine. > > > -- > Phil Barnett mailto:midnight@the-oasis.net > WWW http://www.the-oasis.net/ > FTP Site ftp://ftp.the-oasis.net > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers -- Simon Coggins (SAGE-AU Member) Email: simon@uow.edu.au Network and System Management Officer Phone: +61-2-4221-3775 Information Technology Systems (ITS) Mobile: 0408 115861 University of Wollongong, 2522, Australia Fax: +61-2-4229-1985 From barry@digicool.com Tue Nov 14 06:07:01 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 14 Nov 2000 01:07:01 -0500 (EST) Subject: [Mailman-Developers] Reverting question References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> Message-ID: <14864.54917.27112.665096@anthem.concentric.net> >>>>> "PB" == Phil Barnett writes: PB> I specifically mean downgrading. I have a hornets nest on my PB> hands and I'm about to lose my lists back to egroups. You don't want to downgrade. See my untested patch in a different followup. PB> This lack of reply to the list 'updated feature' has them PB> HEATED. I'm surprised. I would think that munging reply-to is only useful for lists of people who don't know how to drive their MUAs. Is the problem just that they got used to one way of doing things, and now there's a different way? That I can sympathize with. Is it that suddenly, some people are getting two copies of messages because people don't know how to trim their headers? Yeah, that sucks. Or is it something else? -Barry From midnight@the-oasis.net Tue Nov 14 06:31:00 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 01:31:00 -0500 Subject: [Mailman-Developers] Reverting question In-Reply-To: <14864.54917.27112.665096@anthem.concentric.net> Message-ID: <3A1095D4.14770.62D26CF@localhost> On 14 Nov 2000, at 1:07, Barry A. Warsaw wrote: > I'm surprised. I would think that munging reply-to is only useful for > lists of people who don't know how to drive their MUAs. Is the > problem just that they got used to one way of doing things, and now > there's a different way? That I can sympathize with. Is it that > suddenly, some people are getting two copies of messages because > people don't know how to trim their headers? Yeah, that sucks. > > Or is it something else? I use Pegasus. My replyto is global. If I unset my replyto, it's removed from ALL of my mail. I don't really want to be going to my configuration page and blanking it for some mail and turning it back on for other mail. First, it's a hassle. Second, I'll undoubtedly do it wrong at some point. I want a replyto in my mail. Every mail list that I have participated in overrides replyto when replyto is set to the list. To my fellow list users who have been using lists that do this for the last three years, not having it work that way is disconcerting to say the least. egroups and opensource.org are two examples of list providers that override replyto. But, I certainly understand the desire to have it work the way you have it set now. I just think the behaviour should be selectable. It doesn't have to be all or none. Thanks for the patch! Now, if I could create a new list, I'd be back on track... -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From John Block Tue Nov 14 10:25:00 2000 From: John Block (John Block) Date: Tue, 14 Nov 2000 10:25:00 +0000 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: <14864.54917.27112.665096@anthem.concentric.net> Message-ID: Hello Barry > I'm surprised. I would think that munging reply-to is only useful for > lists of people who don't know how to drive their MUAs. MUA? My mail software gives a range of reply options: to all recipients, to sender, to recipients, so there is still choice. Anyhow, I just tried to reply to the question about searchable archives and only got the poster's address. My use for mailman will be to let users help each other and be publically supported and build up an archive of messages to create a searchable brainfile people can query for solutions. If, in many cases, help is only routed to the original poster and not publically, then this plan turns to dust. John From johnblock@storeshop.com Tue Nov 14 10:18:20 2000 From: johnblock@storeshop.com (John Block) Date: Tue, 14 Nov 2000 10:18:20 +0000 Subject: [Mailman-Developers] Re: Searchable Archives In-Reply-To: <3A10E552.CCB12417@nleaudio.com> Message-ID: Hello Bob Fortunatly I'm skilled enough to beable to cut and paste addresses so you will get this publically. > #2. Is there any, ANY way to make the archives searchable? (Under Linux) My plan is to have pipermail which is included in the tarball create html pages. I'm then plannning to get the http://www.thunderstone.com search engine installed to search the html Thunderstone seems to offer alternative search terms based on relationships in the data. A feature which has has got me excited) John From ge204@eng.cam.ac.uk Tue Nov 14 12:59:54 2000 From: ge204@eng.cam.ac.uk (Gunnar Evermann) Date: 14 Nov 2000 12:59:54 +0000 Subject: [Mailman-Developers] Future of pipermail? Message-ID: What are the current plans about the html archiver in mailman? The major shortcomings of pipermail I see are: - doesn't decode MIME - doesn't handle mixed charsets - no search engine - somewhat convoluted code. I run a mailing list with a lot of subscribers from Asia who use MS Outlook, invariably posting (English) messages spuriously tagged as CJK charsets (but only using the ASCII subset) and gratuitously encoded in base64. Therefore I am motivated to find a solution to handling MIME. :-( I have hacked base64 support into pipermail and am currently looking at supporting MIME (probably using mimecntl.py), but this will just be short-term fixes. I know I could just switch to MHonarc or whatever but I like the idea of having a nicely structured, modular archiver tightly integrated with mailman. Are there people working on on pipermail or a replacement of it? Gunnar From chuqui@plaidworks.com Tue Nov 14 15:43:33 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 07:43:33 -0800 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: References: Message-ID: At 10:25 AM +0000 11/14/00, John Block wrote: > > I'm surprised. I would think that munging reply-to is only useful for >> lists of people who don't know how to drive their MUAs. > >MUA? Mail User Agent (aka client) Other jargon: MLM (mail list manager) MTA (mail transfer agent) In general I'm against coercing Reply-To. Strongly. But there are times when it's the right/necessary thing to do, so you want that option. In general yous houldn't use it, but sometimes you need it. One classic case is a mailing list under NDA discussing a beta software release, where all e-mail has to be logged and evaluated, and the *wrong* thing to do is reply privately to a person, because the beta team needs a copy. In that case, reply-to gets coerced to the list, and that reply-to needs to be dominant (i.e., an existing reply-to from a user *can't* take precedent). -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From barry@digicool.com Tue Nov 14 16:40:45 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 14 Nov 2000 11:40:45 -0500 (EST) Subject: [Mailman-Developers] Re: Reverting question References: Message-ID: <14865.27405.857124.313441@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> In general I'm against coercing Reply-To. Strongly. Me too. CVR> But there are times when it's the right/necessary thing to CVR> do, so you want that option. In general yous houldn't use it, CVR> but sometimes you need it. One classic case is a mailing list CVR> under NDA discussing a beta software release, where all CVR> e-mail has to be logged and evaluated, and the *wrong* thing CVR> to do is reply privately to a person, because the beta team CVR> needs a copy. In that case, reply-to gets coerced to the CVR> list, and that reply-to needs to be dominant (i.e., an CVR> existing reply-to from a user *can't* take precedent). That's the key thing that I don't like about Reply-To munging: it makes it much more difficult to do private replies. Maybe it's just the lists I manage, but my users are just as heated about /not/ doing reply-to munging as apparently other users on other lists are about doing it. The primary complaint is that people accidently send private responses to the whole list. This happens because they've trained themselves to know the difference between replies and followups, and have MUAs that separate those functions. The only grumblings occur when people don't trim their CC headers (like I've done here) and folks start getting duplicates. That can be handled in other ways (e.g. the list /could/ suppress deliveries to list members that it sees explicitly in the recipients list, although as we've discussed before, that has some potential for abuse). I've slept on this one and I'm prepared to change things for 2.0 final so that if Reply-To munging is turned on, it'll override any existing Reply-To field in the original message. The deciding factor for me was realizing how difficult it was to send private replies with munging turned on, so it might as well be equally difficult for every poster. I'm concerned about making this change so late in the game, but I'm willing to do it, /if/ I get enough positive feedback asap. I'm attaching a diff against the current CVS tree. Please try it and let me know. If I get enough positive responses with few or no negative responses, I'll add this ("enough" being defined completely subjectively :). -Barry -------------------- snip snip -------------------- Index: Mailman/Defaults.py.in =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/Defaults.py.in,v retrieving revision 1.124 diff -u -r1.124 Defaults.py.in --- Mailman/Defaults.py.in 2000/11/09 02:00:30 1.124 +++ Mailman/Defaults.py.in 2000/11/14 16:38:16 @@ -310,7 +310,15 @@ from: list@listme.com from: .*@uplinkpro.com """ -# Replies to posts inherently directed to list or original sender? + +# Mailman can be configured to "munge" Reply-To: headers for any passing +# messages. One the one hand, there are a lot of good reasons not to munge +# Reply-To: but on the other, people really seem to want this feature. See +# the help for reply_goes_to_list in the web UI for links discussing the +# issue. +# 0 - Reply-To: not munged +# 1 - Reply-To: set back to the list +# 2 - Reply-To: set to an explicit value (reply_to_address) DEFAULT_REPLY_GOES_TO_LIST = 0 # SUBSCRIBE POLICY Index: Mailman/MailList.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/MailList.py,v retrieving revision 1.188 diff -u -r1.188 MailList.py --- Mailman/MailList.py 2000/11/14 04:44:01 1.188 +++ Mailman/MailList.py 2000/11/14 16:38:18 @@ -424,13 +424,21 @@ is strongly recommended for most mailing lists.''', # Details for reply_goes_to_list - """There are many reasons not to introduce headers like -Reply-To: into other people's messages. One is that some posters -depend on their own Reply-To: settings to convey their valid return -address. See + """This option controls what Mailman does to the +Reply-To: header in messages flowing through this mailing list. When +set to Poster, no Reply-To: header is added by Mailman, +although if one is present in the original message, it is not stripped. +Setting this value to either This list or Explicit address +causes Mailman to insert a specific Reply-To: header in all messages, +overriding in the original message if necessary. + +

There are many reasons not to introduce or override the Reply-To: +header. One is that some posters depend on their own Reply-To: +settings to convey their valid return address. Another is that modifying +Reply-To: makes it much more difficult to send private replies. See `Reply-To' Munging -Considered Harmful for a general discussion of this issue. See -Reply-To +Considered Harmful for a general discussion of this issue. See Reply-To Munging Considered Useful for a dissenting opinion.

Some mailing lists have restricted posting privileges, with a parallel list Index: Mailman/Handlers/CookHeaders.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/Handlers/CookHeaders.py,v retrieving revision 1.17 diff -u -r1.17 CookHeaders.py --- Mailman/Handlers/CookHeaders.py 2000/10/27 18:55:21 1.17 +++ Mailman/Handlers/CookHeaders.py 2000/11/14 16:38:19 @@ -76,11 +76,8 @@ msg['Precedence'] = 'bulk' # # Reply-To: munging. Do not do this if the message is "fast tracked", - # meaning it is internally crafted and delivered to a specific user, - # or if there is already a reply-to set. If the user has set - # one we assume they have a good reason for it, and we don't - # second guess them. - if not fasttrack and not msg.get('reply-to'): + # meaning it is internally crafted and delivered to a specific user. + if not fasttrack: # Set Reply-To: header to point back to this list if mlist.reply_goes_to_list == 1: msg['Reply-To'] = mlist.GetListEmail() From ray_frush@agilent.com Tue Nov 14 17:22:42 2000 From: ray_frush@agilent.com (Ray Frush) Date: Tue, 14 Nov 2000 10:22:42 -0700 Subject: [Mailman-Developers] Re: Mailman-Users digest, Vol 1 #870 - 2 msgs References: <20001114170146.9D8861D16F@dinsdale.python.org> Message-ID: <3A1174E2.F0236153@ale.ftc.agilent.com> > I've slept on this one and I'm prepared to change things for 2.0 final > so that if Reply-To munging is turned on, it'll override any existing > Reply-To field in the original message. The deciding factor for me > was realizing how difficult it was to send private replies with > munging turned on, so it might as well be equally difficult for every > poster. If Reply-to munging is enabled, I would expect that it would really munge. That is, if you turn on the feature, it will ALWAYS exhibit the behaviour. In the current version, Reply-To: munging appears broken because it appears to work part of the time. Count this as a vote to enable the much debated Reply-to munging that will override any client (MUA) Reply-to settings. -- Ray Frush "Either you are part of the solution, T:970.288.6223 or part of the precipitate." -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- Agilent Technologies IT | Technical Computing | Fort Collins Site From chuqui@plaidworks.com Tue Nov 14 17:49:53 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 09:49:53 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Re: Reverting question In-Reply-To: <14865.27405.857124.313441@anthem.concentric.net> References: <14865.27405.857124.313441@anthem.concentric.net> Message-ID: At 11:40 AM -0500 11/14/00, Barry A. Warsaw wrote: >That's the key thing that I don't like about Reply-To munging: it >makes it much more difficult to do private replies. Maybe it's just >the lists I manage, but my users are just as heated about /not/ doing >reply-to munging as apparently other users on other lists are about >doing it. It's a religious issue to some level, but I've done any number of studies (at least half a dozen) on it with my users over the years. I've never found *any* list or user population where the majority wants reply-to coerced. What you really end up with, if you survey the *entire* user base, is a noisy minority that wants it, and that minority is under 20-25% of those that have opinions. So what ends up happening is that because they're the ones making noise, the list gets set that way. But if you survey the entire user base and get a non-biased sample, they're the minority viewpoint (the same is true of Subject line tweaks, like the [Mailman-users] flag. Only with those, I've never found a list where even 5% of the users wanted it, and there's usually a solid majority that hate the damn things.... But again, you tend to run into noisy minorites that push their preferences, and unless you take the time to go out and run formal list surveys (which are time consuming), it's hard to tell whether it is a squeaky wheel or whether it's a list consensus... >The only grumblings occur when people don't >trim their CC headers (like I've done here) and folks start getting >duplicates. Another vocal minority. Less than 1% of a typical subscriber base cares about this. Those that do tend to be very sensitive to it and vocal, but whacking the server for that group isn't a smart idea (IMHO), since there are client ways of doing it, like message-ID trapping. > That can be handled in other ways (e.g. the list /could/ >suppress deliveries to list members that it sees explicitly in the >recipients list, although as we've discussed before, that has some >potential for abuse). True, but it's what I'd like to see happen down the road, at least as a configuration option for the server admin. If you know they're getting it direct, why send them a second copy? or maybe tag this to a users "metoo" flag? If they've told the server not to send them copies of their own posts, assume they also don't want duplicates? That might be the easiest way, and leave it by default off, but then users can set it if they care. >I've slept on this one and I'm prepared to change things for 2.0 final >so that if Reply-To munging is turned on, it'll override any existing >Reply-To field in the original message. The deciding factor for me >was realizing how difficult it was to send private replies with >munging turned on, so it might as well be equally difficult for every >poster. I think that's the right thing to do, and it sets you in line with how other MLMs work, so it better matches user expectations, I think... It's a touch issue, because there are personal preferences, because Reply-to is used for different things (not always correctly), and because it just plain old isn't all that well defined as a header, despite it's long-standing usage... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From jwblist@olympus.net Tue Nov 14 20:33:16 2000 From: jwblist@olympus.net (John W Baxter) Date: Tue, 14 Nov 2000 12:33:16 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <14864.54917.27112.665096@anthem.concentric.net> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <14864.54917.27112.665096@anthem.concentric.net> Message-ID: At 1:07 -0500 11/14/00, Barry A. Warsaw wrote: >I'm surprised. I would think that munging reply-to is only useful for >lists of people who don't know how to drive their MUAs. Is the >problem just that they got used to one way of doing things, and now >there's a different way? That I can sympathize with. Is it that >suddenly, some people are getting two copies of messages because >people don't know how to trim their headers? Yeah, that sucks. I'm not on such a list (we're expecting to switch to Mailman when it settles down). But I would be perturbed if on a single list sometimes the Reply button sets up a reply to the list and sometimes it sets up a private reply. [I can--and do--deal with both kinds of list, and both are a nuisance some of the time.] The idea that Reply's result on a single list depends on who* sent the message seems like a faulty solution to the unsolvable problem that neither reply-goes-to-the-list nor reply-goes-to-the-sender is right. [Right would take mindreading: reply goes where I expect *this* reply to go.] *Or, even worse, where a given person was were when the message was sent. --John -- John Baxter jwblist@olympus.net Port Ludlow, WA, USA From johnblock@storeshop.com Tue Nov 14 16:48:05 2000 From: johnblock@storeshop.com (John Block) Date: Tue, 14 Nov 2000 16:48:05 +0000 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: Message-ID: Hello Chuq Thank you for clearing up the TLAs. > In general I'm against coercing Reply-To. Strongly. Who is coercing? By just having the sender as a reply option, the software is stopping the none technical from replying to the list. The more aware are caused irritation as they can't simply hit their "r" button. If you can't reply publically easily, for a lot of people the motivations to reply are removed. There is no name in print on the screen, no look how clever I am, no visible penny in the help bank so i can get help in return > But there are times when it's the right/necessary thing to do, so you > want that option. In general yous houldn't use it, but sometimes you > need it. In other words it should be a togglable feature. One classic case is a mailing list under NDA discussing a > beta software release, where all e-mail has to be logged and > evaluated, That should be the case on any help list, so people can search archives before needlessly bothering everyone else with something which has been solved already. Thanks, John From midnight@the-oasis.net Wed Nov 15 01:43:35 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 20:43:35 -0500 Subject: [Mailman-Developers] Default path for archives wrong in rc2 Message-ID: <3A11A3F7.14204.A4C5F54@localhost> I know that we got caught up in the replyto munging issue, but did anyone see that when I switched to rc2, my archives went offline and I can't add a new user because the archive is not where it's expected to be now because of a /bin/ in the message? I can post the problems specifically again if necessary, but the path problem was posted a couple of days ago. Since my rc2 upgrade from b6, I can't create new lists and my archives are offline. http://www.matrixlist.com/mailman/listinfo is broken and the data didnt' move. -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From marc_news@valinux.com Wed Nov 15 01:50:12 2000 From: marc_news@valinux.com (Marc MERLIN) Date: Tue, 14 Nov 2000 17:50:12 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <20001114165748.B3475@uow.edu.au>; from simon@uow.edu.au on Tue, Nov 14, 2000 at 04:57:49PM +1100 References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> Message-ID: <20001114175012.T27544@marc.merlins.org> On Tue, Nov 14, 2000 at 04:57:49PM +1100, Simon Coggins wrote: > I have to agree here. Not being able to override the reply-to: fields on > lists is a *MAJOr* problem. I've also had this problem with my lists. And I > was hoping for a 'fix' but it looks like it's intended behaviour?. The irony of this coming back over and over again... http://www.unicom.com/pw/reply-to-harmful.html 1) Don't set a reply-to in your lists (see above link) 2) If you don't want to take that advise, you certainly shouldn't overwrite a users' reply to with the list's 3) If you insist in doing the above 2, you can patch the code Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From chuqui@plaidworks.com Wed Nov 15 02:13:53 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 18:13:53 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <20001114175012.T27544@marc.merlins.org> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> Message-ID: At 5:50 PM -0800 11/14/00, Marc MERLIN wrote: >2) If you don't want to take that advise, you certainly shouldn't overwrite > a users' reply to with the list's I'll disagree. If you feel you must coerce reply-to, coerce it unconditionally. You've already decided the list's needs outweigh the individual's needs, or you woudln't be coercing in the first place Once you make that decision, whether or not the user ALSO uses a reply-to is meaningless, and trying to take that into consideration only confuses things, because then the list has its reply-to coerced -- sometimes, and you have to know how to read mail headers ot figure out when. Not good for the typical user, who may never figure out why something isn't working randomly. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From Dan Mick Wed Nov 15 02:48:04 2000 From: Dan Mick (Dan Mick) Date: Tue, 14 Nov 2000 18:48:04 -0800 (PST) Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question Message-ID: <200011150246.SAA26058@utopia.west.sun.com> > At 5:50 PM -0800 11/14/00, Marc MERLIN wrote: > >2) If you don't want to take that advise, you certainly shouldn't overwrite > > a users' reply to with the list's > > I'll disagree. If you feel you must coerce reply-to, coerce it > unconditionally. You've already decided the list's needs outweigh the > individual's needs, or you woudln't be coercing in the first place > Once you make that decision, whether or not the user ALSO uses a > reply-to is meaningless, and trying to take that into consideration > only confuses things, because then the list has its reply-to coerced > -- sometimes, and you have to know how to read mail headers ot figure > out when. Not good for the typical user, who may never figure out why > something isn't working randomly. I have to agree with Chuq; all or nothing seems to be the only sane policy. (and I'm a big Reply-To hater.) From midnight@the-oasis.net Wed Nov 15 03:20:23 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Tue, 14 Nov 2000 22:20:23 -0500 Subject: [Mailman-Developers] Info on newlist bug Message-ID: <3A11BAA7.22965.AA4FDB3@localhost> Here is the traceback for the failure. Now that I think about it, it's probably that the bin/newlist executable is looking at the 0 parameter instead of the 1 parameter. For example, on the command line bin/newlist test bin/newlist is parameter 0 test is parameter 1 Notice that the expected directory should have been: '/home/mailman/archives/private/test.mbox' but was instead: '/home/mailman/archives/private/bin/newlist.mbox' I've been looking in the code to see where this might be going wrong, but haven't gotten anywhere. -----------traceback------------- [root@taz2 mailman]# bin/newlist test Enter the email of the person running the list: midnight@the- oasis.net Initial bin/newlist password: Traceback (innermost last): File "bin/newlist", line 213, in ? main(sys.argv) File "bin/newlist", line 163, in main mlist.Create(listname, owner_mail, pw) File "/home/mailman/Mailman/MailList.py", line 775, in Create self.InitVars(name, admin, crypted_password) File "/home/mailman/Mailman/MailList.py", line 335, in InitVars Archiver.InitVars(self) # has configurable stuff File "/home/mailman/Mailman/Archiver/Archiver.py", line 108, in InitVars Utils.mkdir(self.private_archive_file_dir) File "/home/mailman/Mailman/Utils.py", line 568, in mkdir os.mkdir(dir, mode) OSError: [Errno 2] No such file or directory: '/home/mailman/archives/private/bin/newlist.mbox' -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From lovelace@wayfarer.org Wed Nov 15 04:42:53 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Tue, 14 Nov 2000 23:42:53 -0500 Subject: [Mailman-Developers] Re: Reverting question References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> Message-ID: <3A12144D.64C33371@wayfarer.org> Marc MERLIN wrote: > > The irony of this coming back over and over again... > http://www.unicom.com/pw/reply-to-harmful.html One good URL deserves another (straight from the "reply_goes_to_list Option" documentation): http://www.metasystema.org/essays/reply-to-useful.mhtml There are arguments on both sides for this issue. I personally think it depends a lot on what the people on your list prefer. I run a large non-technical mailing list where we tried making the "Reply-To" point to the list for a while. Unfortunately, it resulted in someone accidentally posting a very unflattering message about someone else on the list. Since, this is a non-technical list (actually it's about as non-technical as you can get since it's a list for medieval recreationist list), I can't assume that people will automatically understand the ramifications of the "Reply-To" header so I decided that for my list it would be better to have the "Reply-To" not point to the list. For other lists, however, I think it could be advantageous. Because of that, I think that letting the list administrator decide, rather than the software, is the best method. I also think, though, that sometimes setting the "Reply-To" and sometimes not, depending on whether or not it has already been set has the potential to confuse a lot of people. If you're going to provide *that* functionality, you should make sure it is *extremely* well documented. Tanner Lovelace P.S. I'm kind of curious as to how many people use Mailman with qmail. Are there any statistics on this? I haven't been able to find any. -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? From lovelace@wayfarer.org Wed Nov 15 05:00:15 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Wed, 15 Nov 2000 00:00:15 -0500 Subject: [Mailman-Developers] Re: Info on newlist bug References: <3A11BAA7.22965.AA4FDB3@localhost> Message-ID: <3A12185E.5B84C50B@wayfarer.org> Phil Barnett wrote: > > Here is the traceback for the failure. Now that I think about it, it's > probably that the bin/newlist executable is looking at the 0 > parameter instead of the 1 parameter. > At the beginning of main there is this line of code that parses the arguments. opts, args = getopt.getopt(sys.argv[1:], 'ho:', ['help', 'output=']) getopt takes the passed in arguments (sys.argv[1:]) and removes the ones it takes care of leaving the rest in the "args" variable. Unfortunately, later on when reading the command line arguments they are read from the original argv instead of the modified args. Attached is a patch that fixes this problem. Tanner Lovelace --- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? --- newlist.orig Tue Nov 14 23:48:13 2000 +++ newlist Tue Nov 14 23:55:47 2000 @@ -125,7 +125,7 @@ usage(0) if len(args) > 0: - listname = argv[0] + listname = args[0] else: listname = raw_input("Enter the name of the list: ") listname = string.lower(listname) @@ -137,13 +137,13 @@ usage(1, 'List already exists: ' + listname) if len(args) > 1: - owner_mail = argv[1] + owner_mail = args[1] else: owner_mail = raw_input( "Enter the email of the person running the list: ") if len(args) > 2: - list_pw = argv[2] + list_pw = args[2] else: list_pw = getpass.getpass("Initial %s password: " % listname) # List passwords cannot be empty From lovelace@wayfarer.org Wed Nov 15 05:30:11 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Wed, 15 Nov 2000 00:30:11 -0500 Subject: [Mailman-Developers] Problem archiving mail sent from pine (and other MUAs) Message-ID: <3A121F63.1BF3B141@wayfarer.org> I seem to have found an error where pipermail fails to process mail sent from pine (and possibly other mailers). The message gets added to the .mbox file but nothing else. In the logs/error file I get the following message: Nov 15 00:15:01 2000 qrunner(32366): Traceback (innermost last): Nov 15 00:15:01 2000 qrunner(32366): File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 221, in ArchiveMail Nov 15 00:15:01 2000 qrunner(32366): File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 521, in processUnixMailbox Nov 15 00:15:01 2000 qrunner(32366): File "/usr/lib/mailman/Mailman/Archiver/HyperArch.py", line 224, in __init__ Nov 15 00:15:01 2000 qrunner(32366): TypeError: read-only character buffer, None Nov 15 00:15:01 2000 (32366) CORRUPT ARCHIVE FOR LIST: test Looking at the file HyperArch.py at line 224 I find the following: # Snag the content-* headers. RFC 1521 states that their values are # case insensitive. ctype = message.getheader('Content-Type') or "text/plain" cenc = message.getheader('Content-Transfer-Encoding') self.ctype = string.lower(ctype) self.cenc = string.lower(cenc) (line 224 is the last one.) So, apparently it is failing when it tries to lowercase the "Content-Transfer-Encoding" header string. So, I checked, and sure enough, pine (at least on the two systems I had access to) does not set this header. I then looked at RFC 1521 to see if this header is required. From the RFC (p.14) I quote: These values are not case sensitive. That is, Base64 and BASE64 and bAsE64 are all equivalent. An encoding type of 7BIT requires that the body is already in a seven-bit mail-ready representation. This is the default value -- that is, "Content-Transfer-Encoding: 7BIT" is assumed if the Content-Transfer-Encoding header field is not present. So, the "Contenet-Transfer-Encoding" header is not mandatory and pipermail should not crash when it isn't there. Since I don't actually know python, :-) can anyone suggest an easy fix? It doesn't seem like it would be that hard to fix. I believe this is the same problem that Joshua Burley first reported. Tanner Lovelace P.S. Should I submit this as a bug to sourceforge? -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? From chuqui@plaidworks.com Wed Nov 15 05:22:49 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 21:22:49 -0800 Subject: [Mailman-Developers] [Mailman-Users] Re: Reverting question In-Reply-To: <3A12144D.64C33371@wayfarer.org> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A12144D.64C33371@wayfarer.org> Message-ID: At 11:42 PM -0500 11/14/00, Tanner Lovelace wrote: >There are arguments on both sides for this issue. I >personally think it depends a lot on what the people >on your list prefer. But you need to be wary of making the mistake of "who's making noise" being "what the people on the list prefer". Becaues I've found, if I survey my lists, that the ones making the noise *don't* represent the overall preferences of the list, but are instead a noisy minority. Just because someone's complaining doesn't mean they're right.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From claw@kanga.nu Wed Nov 15 05:44:48 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 14 Nov 2000 21:44:48 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Re: Reverting question In-Reply-To: Message from Chuq Von Rospach of "Tue, 14 Nov 2000 21:22:49 PST." References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A12144D.64C33371@wayfarer.org> Message-ID: <26367.974267088@kanga.nu> On Tue, 14 Nov 2000 21:22:49 -0800 Chuq Von Rospach wrote: > At 11:42 PM -0500 11/14/00, Tanner Lovelace wrote: >> There are arguments on both sides for this issue. I personally >> think it depends a lot on what the people on your list prefer. > But you need to be wary of making the mistake of "who's making > noise" being "what the people on the list prefer". Becaues I've > found, if I survey my lists, that the ones making the noise > *don't* represent the overall preferences of the list, but are > instead a noisy minority. Just because someone's complaining > doesn't mean they're right.... I've found the same statistics. That said, I've deliberately decided to comply with the noisy moinority view for the worst of reasons: I value the contributions of certain members of that minority highly (they occupy useful/divergent views or represent intellectually significant factions of the list topic), and, quite simply, the majority do not disagree enough to act on it (leave the list). An ugly trade-off, but one that (seems to) work. -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From midnight@the-oasis.net Wed Nov 15 05:50:09 2000 From: midnight@the-oasis.net (Phil Barnett) Date: Wed, 15 Nov 2000 00:50:09 -0500 Subject: [Mailman-Developers] Re: Info on newlist bug In-Reply-To: <3A12185E.5B84C50B@wayfarer.org> Message-ID: <3A11DDC1.23494.381810@localhost> On 15 Nov 2000, at 0:00, Tanner Lovelace wrote: > Phil Barnett wrote: > > > > Here is the traceback for the failure. Now that I think about it, > > it's probably that the bin/newlist executable is looking at the 0 > > parameter instead of the 1 parameter. > > > > At the beginning of main there is this line of code > that parses the arguments. > > opts, args = getopt.getopt(sys.argv[1:], 'ho:', > ['help', 'output=']) > > getopt takes the passed in arguments (sys.argv[1:]) > and removes the ones it takes care of leaving the > rest in the "args" variable. Unfortunately, later > on when reading the command line arguments they > are read from the original argv instead of the > modified args. Attached is a patch that fixes > this problem. Great, that fixed newlist. Thanks. At the same time newlist stopped working, my archives and web interfaces all stopped working. I wonder what else is hiding... How do we start troubleshooting this? http://www.matrixlist.com/mailman/listinfo -- Phil Barnett mailto:midnight@the-oasis.net WWW http://www.the-oasis.net/ FTP Site ftp://ftp.the-oasis.net From lovelace@wayfarer.org Wed Nov 15 06:32:59 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Wed, 15 Nov 2000 01:32:59 -0500 Subject: [Mailman-Developers] Re: Info on newlist bug References: <3A11DDC1.23494.381810@localhost> Message-ID: <3A122E1B.2546FD1B@wayfarer.org> Phil Barnett wrote: > > Great, that fixed newlist. > On further investigation, there was already a patch on sourceforge for this bug. The patch on sourceforge also modifies one more line that mine didn't that should be changed. The patch can be found at: https://sourceforge.net/patch/download.php?id=102370 Other references: https://sourceforge.net/patch/?func=detailpatch&patch_id=102370&group_id=103 https://sourceforge.net/bugs/?func=detailbug&bug_id=122333&group_id=103 Tanner Lovelace -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? From mailman-users@python.org Wed Nov 15 06:34:41 2000 From: mailman-users@python.org (Marc MERLIN) Date: Tue, 14 Nov 2000 22:34:41 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <3A11F004.13A1BCDF@nleaudio.com>; from bob@nleaudio.com on Tue, Nov 14, 2000 at 09:08:04PM -0500 References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> Message-ID: <20001114223441.G9163@marc.merlins.org> [Reply-To set to mailman-users@python.org] On Tue, Nov 14, 2000 at 06:13:53PM -0800, Chuq Von Rospach wrote: > At 5:50 PM -0800 11/14/00, Marc MERLIN wrote: > >2) If you don't want to take that advise, you certainly shouldn't overwrite > > a users' reply to with the list's > > I'll disagree. If you feel you must coerce reply-to, coerce it > unconditionally. You've already decided the list's needs outweigh the A few others mentionned that. The problem here, is that by putting a reply-to, you are already trying to prevent me from replying directly to the poster (well, not quite since mutt lets me tell it to ignore all those reply-tos), but by munging reply tos unconditionally, you are completely preventing me from replying to someone who has a non valid from address and set a good reply-to, or someone who'd like me to reply to him at another address. Doing a reply-to rewrite also prevents me from Ccing a second list and setting a reply-to there to move the discussion. That said, I could see why others would prefer unconditionally overwriting the reply-to. Well, in both cases, you lose, which is yet another reason why lists should not set a reply-to :-) I think I agree with JC in saying that short of removing the reply to misfeature altogether, unconditional rewriting should probably be an option On Wed, Nov 15, 2000 at 12:27:14PM +0800, Gregory Leblanc wrote: > Fine, that's all well and good, munging reply to is harmful. Sometimes > the harm that it causes is less than the harm that it prevents. I've yet to have a single person show me what harm not munging reply-to creates... It's all about users that have been trained to do the wrong thing and some of them who then believe it's the only way to go. > This is a good policy in most cases, but there ARE some where it doesn't > make sense. Yep, the only one I know is onelist which tries to force as many traffic back to the list in order to spam you with more messages which each contain a small add they make money on. Ok, more seriously, the only downside of group replies is that for MUAs that don't support list-reply, like mutt does, the sender gets two copies of the answer (which are easy to get rid off by removing duplicate messages that contain the same message id) > I'm sorry, but this is silly. If you're going to write reply-to headers > via your MLM, you need to write them ALL the time, otherwise you end up > with strange and un-predictable behaviour. Keep the reply-to header Point taken. The other option is wrong too though, for the reason I explained. But I guess if you choose to shoot yourself in the foot, it'd be nice to be able to chose which gun you're going to use (i.e. make the unconditional rewriting an option) > certainly, but if headers are sending mail back to the list, the user's > reply-to really doesn't make much difference, unless you need to send a > private reply. If you're sending a private reply, you can get it out of > the x-reply-to header. Great, I need to patch my MUA some more to attempt to deal with all this misconfiguration nonsense. On Tue, Nov 14, 2000 at 11:42:53PM -0500, Tanner Lovelace wrote: > Marc MERLIN wrote: > > > > The irony of this coming back over and over again... > > http://www.unicom.com/pw/reply-to-harmful.html > > One good URL deserves another (straight from the > "reply_goes_to_list Option" documentation): > > http://www.metasystema.org/essays/reply-to-useful.mhtml Yeah, except that the author is full of shit. Ok, I'll bite. - "Reply-To gives the respondant an option which would not otherwise exist: namely the ability to reply only to the list." false, see list-reply function in good MUAs (mutt is one) - "If you use a reasonable mailer, Reply-To munging does provide new functionality, namely the ability to reply only to the list" No comment... (the author completely ignores list-reply, and says that MUAs should all be patched to allow overriding reply-tos so that they can continue to be misused) - "Reply-To munging adds additional functionality, it actually increases freedom of choice" *cough* bullshit *cough* For one, it prevents a thread across two lists and the redirection to one list or a third list. - some mailers apparently do not have a separate reply to all function Ok, 1) I have yet to see one 2) in the point above, the author was saying that any MUA that can't override a reply-to is not a reasonable mailer and shouldn't be used. The irony... - "For discussion type lists, I would estimate that ninety percent of the time, people want to reply to the list" Yeah, so? That's why you have several reply functions, where one does not require more work than the other (the author mentions having to type addresses by hand, I'm not sure why) - "Some administrators claim that munging Reply-To headers is harmful because it surprises people, and can cause damage when things go awry" Damn straight. If you reply to the list by mistake due to reply-to munging, you look like an idiot, or worse. If you reply to the sender only by mistake, no harm is done, the mistake can be fixed. - "It's What People Want" Yeah, there are misguided people out there, and they are often vocal. Users typically know nothing about mail standards or what header sender vs envelope sender means. I don't think there opinion is that relevant :-) Ok, more seriously, users do matter but if they're trained the wrong way when they start, some get very resistant to change, regardless of whether it's a good thing or not. We did get a bit off topic here, I apologize to all the people who already know about all this, but since we have lots of listmasters here, I'm hoping a few with switch, and hopefully more will configure their lists the right way in the first place (that's when it's easy, changing is hard) > There are arguments on both sides for this issue. I personally think it > depends a lot on what the people on your list prefer. I run a large > non-technical mailing list where we tried making the "Reply-To" point to > the list for a while. Unfortunately, it resulted in someone accidentally > posting a very unflattering message about someone else on the list. Yep, this happens all the time. Many of us have seen this. > the "Reply-To" not point to the list. For other lists, however, I think > it could be advantageous. Because of that, I think that letting the list > administrator decide, rather than the software, is the best method. While I'm very much against reply-to munging (duh!), I'm for freedom of choice. I do think that mailman should offer a reply-to option, but with a very big disclaimer and maybe this discussion (or another one) so that inexperienced listmasters realize what they're doing. > I also think, though, that sometimes setting the "Reply-To" and sometimes > not, depending on whether or not it has already been set has the > potential to confuse a lot of people. If you're going to provide *that* > functionality, you should make sure it is *extremely* well documented. Agreed. Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From claw@kanga.nu Wed Nov 15 06:57:21 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 14 Nov 2000 22:57:21 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: Message from Marc MERLIN of "Tue, 14 Nov 2000 22:34:41 PST." <20001114223441.G9163@marc.merlins.org> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> Message-ID: <32332.974271441@kanga.nu> On Tue, 14 Nov 2000 22:34:41 -0800 Marc MERLIN wrote: > Ok, more seriously, the only downside of group replies is that for > MUAs that don't support list-reply, like mutt does, the sender > gets two copies of the answer (which are easy to get rid off by > removing duplicate messages that contain the same message id) On *-ix platforms which have procmail/formail. Windows, MacOS, and BeOS users, to name but a few, have slightly more difficulty on this area. Sad, but true. >> http://www.metasystema.org/essays/reply-to-useful.mhtml > Yeah, except that the author is full of shit. Key points: This argument is a FAQ I've yet to see a single case where someone has been persuaded by the arguments of the other side once the argument has begun. Now, can we get on with some more productive discussions, like how vi users are scum-sucking brainless cretins and emacs users are the obviously intelligent few who can see and use true brilliance? > I do think that mailman should offer a reply-to option, but with a > very big disclaimer and maybe this discussion (or another one) so > that inexperienced listmasters realize what they're doing. I note that this is already done. > Microsoft is to operating systems & security Now, about them silly sendmail users... -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From mailman-users@python.org Wed Nov 15 07:39:43 2000 From: mailman-users@python.org (Marc MERLIN) Date: Tue, 14 Nov 2000 23:39:43 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <32332.974271441@kanga.nu>; from claw@kanga.nu on Tue, Nov 14, 2000 at 10:57:21PM -0800 References: <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> <32332.974271441@kanga.nu> Message-ID: <20001114233943.B21148@marc.merlins.org> On Tue, Nov 14, 2000 at 10:57:21PM -0800, J C Lawrence wrote: > I've yet to see a single case where someone has been persuaded by > the arguments of the other side once the argument has begun. I've been on several lists that have been converted away from reply-to munging. I've never been on any list that was converted to reply-to munging. 'nuf said. > > I do think that mailman should offer a reply-to option, but with a > > very big disclaimer and maybe this discussion (or another one) so > > that inexperienced listmasters realize what they're doing. > > I note that this is already done. I'm basically replying to the list to apologize on this issue. Indeed mailman already gives plenty of warning about this, I just never saw the configuration help page > > Microsoft is to operating systems & security > > Now, about them silly sendmail users... Troll :-) Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From chuqui@plaidworks.com Wed Nov 15 07:31:27 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 23:31:27 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <32332.974271441@kanga.nu> References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> <32332.974271441@kanga.nu> Message-ID: At 10:57 PM -0800 11/14/00, J C Lawrence wrote: > I've yet to see a single case where someone has been persuaded by > the arguments of the other side once the argument has begun. isn't this where someone's supposed to call the other a fascist and make a spurious Hitler reference? >Now, can we get on with some more productive discussions, like how >vi users are scum-sucking brainless cretins and emacs users are the >obviously intelligent few who can see and use true brilliance? What's the difference between an emacs user and a vi user? Give each a file and a set of changes. the vi user sits down and hacks in the changes, and goes to lunch. the emacs user sits down and writes a macro that'll make the changes while he's at lunch. At the end, they both have the changed file, but the emacs user has a macro he'll put in his library and never use again. > > Microsoft is to operating systems & security First, get an OS that doesn't hate you... >Now, about them silly sendmail users... (sticks out tongue) -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From chuqui@plaidworks.com Wed Nov 15 07:47:51 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 14 Nov 2000 23:47:51 -0800 Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question In-Reply-To: <20001114233943.B21148@marc.merlins.org> References: <20001114175012.T27544@marc.merlins.org> <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> <32332.974271441@kanga.nu> <20001114233943.B21148@marc.merlins.org> Message-ID: At 11:39 PM -0800 11/14/00, Marc MERLIN wrote: >I've been on several lists that have been converted away from reply-to >munging. >I've never been on any list that was converted to reply-to munging. >'nuf said. I've actually experimented on my users. I've had lists that started with no reply-to, added it for a while, then removed it. And monitored preferences and problems. All my discussion lists are not coerced -- that's what people ended up preferring, and the problems are significantly lessened (we won't even get into vacation bots that get reply-toed onto the list and through thousands of messages onto the list before the admin wakes up....) > > Now, about them silly sendmail users... > >Troll :-) You rang? -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From barry@digicool.com Wed Nov 15 12:49:49 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 15 Nov 2000 07:49:49 -0500 (EST) Subject: [Mailman-Developers] Re: Info on newlist bug References: <3A12185E.5B84C50B@wayfarer.org> <3A11DDC1.23494.381810@localhost> Message-ID: <14866.34413.539257.537295@anthem.concentric.net> Below is the bin/newlist patch I just checked in. >>>>> "PB" == Phil Barnett writes: PB> At the same time newlist stopped working, my archives and web PB> interfaces all stopped working. I wonder what else is PB> hiding... Take a look at the current CVS snapshot. I think those bugs (well, at least the archives one) should be fixed. -Barry -------------------- snip snip -------------------- Index: newlist =================================================================== RCS file: /cvsroot/mailman/mailman/bin/newlist,v retrieving revision 1.35 retrieving revision 1.36 diff -u -r1.35 -r1.36 --- newlist 2000/11/10 18:42:46 1.35 +++ newlist 2000/11/15 12:49:18 1.36 @@ -18,10 +18,16 @@ """Create a new, unpopulated mailing list. -Usage: %(PROGRAM)s +Usage: %(PROGRAM)s [options] listname listadmin-addr admin-password Options: + -q + --quiet + Normally the administrator is notified by email (after a prompt) that + their list has been created. This option suppresses that + notification. + -o file --output=file Append the alias setting recommendations to file, in addition to @@ -110,22 +116,25 @@ -def main(argv): +def main(): try: - opts, args = getopt.getopt(sys.argv[1:], 'ho:', - ['help', 'output=']) + opts, args = getopt.getopt(sys.argv[1:], 'ho:q', + ['help', 'output=', 'quiet']) except getopt.error, msg: usage(1, msg) appendfile = None + quiet = 0 for opt, arg in opts: if opt in ('-o', '--output'): appendfile = arg if opt in ('-h', '--help'): usage(0) + if opt in ('-q', '--quiet'): + quiet = 1 if len(args) > 0: - listname = argv[0] + listname = args[0] else: listname = raw_input("Enter the name of the list: ") listname = string.lower(listname) @@ -137,13 +146,13 @@ usage(1, 'List already exists: ' + listname) if len(args) > 1: - owner_mail = argv[1] + owner_mail = args[1] else: owner_mail = raw_input( "Enter the email of the person running the list: ") if len(args) > 2: - list_pw = argv[2] + list_pw = args[2] else: list_pw = getpass.getpass("Initial %s password: " % listname) # List passwords cannot be empty @@ -186,28 +195,29 @@ fp.write('\n') fp.close() - if len(argv) < 5: + if not quiet: print ("Hit enter to continue with %s owner notification..." % listname), sys.stdin.readline() - # send the notice to the list owner - text = Utils.maketext( - 'newlist.txt', - {'listname' : listname, - 'password' : list_pw, - 'admin_url' : mlist.GetScriptURL('admin', absolute=1), - 'listinfo_url': mlist.GetScriptURL('listinfo', absolute=1), - 'requestaddr' : "%s-request@%s" % (listname, mlist.host_name), - 'hostname' : mlist.host_name, - }) - msg = Message.UserNotification(owner_mail, - 'mailman-owner@' + mlist.host_name, - 'Your new mailing list: ' + listname, - text) - HandlerAPI.DeliverToUser(mlist, msg) + # send the notice to the list owner + text = Utils.maketext( + 'newlist.txt', + {'listname' : listname, + 'password' : list_pw, + 'admin_url' : mlist.GetScriptURL('admin', absolute=1), + 'listinfo_url': mlist.GetScriptURL('listinfo', absolute=1), + 'requestaddr' : "%s-request@%s" % (listname, mlist.host_name), + 'hostname' : mlist.host_name, + }) + msg = Message.UserNotification( + owner_mail, + 'mailman-owner@' + mlist.host_name, + 'Your new mailing list: ' + listname, + text) + HandlerAPI.DeliverToUser(mlist, msg) finally: mlist.Unlock() if __name__ == '__main__': - main(sys.argv) + main() From claw@kanga.nu Wed Nov 15 21:18:59 2000 From: claw@kanga.nu (J C Lawrence) Date: Wed, 15 Nov 2000 13:18:59 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? Message-ID: <12203.974323139@kanga.nu> I'm looking to setup a test Mailman site under Postfix (need to do some performance modelling of Exim against Postfix under Mailman for a given membership list). I can get a basic setup working no problem -- now I'm looking to duplicate the neater aspects I already have under Exim, such as losing the Mailman aliases file, etc. Anybody already done this? -- J C Lawrence Home: claw@kanga.nu ---------(*) Other: coder@kanga.nu http://www.kanga.nu/~claw/ Keys etc: finger claw@kanga.nu --=| A man is as sane as he is dangerous to his environment |=-- From gerald@impressive.net Wed Nov 15 22:39:27 2000 From: gerald@impressive.net (Gerald Oskoboiny) Date: Wed, 15 Nov 2000 17:39:27 -0500 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: <14865.27405.857124.313441@anthem.concentric.net>; from barry@digicool.com on Tue, Nov 14, 2000 at 11:40:45AM -0500 References: <14865.27405.857124.313441@anthem.concentric.net> Message-ID: <20001115173927.E11322@impressive.net> On Tue, Nov 14, 2000 at 11:40:45AM -0500, Barry A. Warsaw wrote: > >>>>> "CVR" == Chuq Von Rospach writes: > > CVR> In general I'm against coercing Reply-To. Strongly. > > Me too. Me too. > The primary complaint is that people accidently send private responses > to the whole list. This happens because they've trained themselves to > know the difference between replies and followups, and have MUAs that > separate those functions. Yes, and that's definitely a Good Thing: it should be a very conscious choice whether to send a private reply or to send mail to possibly hundreds or thousands of people. > The only grumblings occur when people don't > trim their CC headers (like I've done here) and folks start getting > duplicates. That can be handled in other ways (e.g. the list /could/ > suppress deliveries to list members that it sees explicitly in the > recipients list, although as we've discussed before, that has some > potential for abuse). Another solution to this is the Mail-Followup-To header: (not widely supported yet) http://cr.yp.to/proto/replyto.html (related info for mutt users: http://larve.net/people/hugo/2000/07/ml-mutt ) -- Gerald Oskoboiny http://impressive.net/people/gerald/ From barry@digicool.com Thu Nov 16 04:34:31 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 15 Nov 2000 23:34:31 -0500 (EST) Subject: [Mailman-Users] Re: [Mailman-Developers] Reverting question References: <14864.52862.465404.35651@anthem.concentric.net> <3A108AAB.25143.6018DAE@localhost> <20001114165748.B3475@uow.edu.au> <20001114175012.T27544@marc.merlins.org> <3A11F004.13A1BCDF@nleaudio.com> <20001114223441.G9163@marc.merlins.org> Message-ID: <14867.25559.227185.305800@anthem.concentric.net> >>>>> "MM" == Marc MERLIN writes: MM> While I'm very much against reply-to munging (duh!), I'm for MM> freedom of choice. I do think that mailman should offer a MM> reply-to option, but with a very big disclaimer and maybe this MM> discussion (or another one) so that inexperienced listmasters MM> realize what they're doing. Marc, Very nice rebuttal to the rebuttal to the reply-to-considered-harmful article. You should write that up nicely and put it on the web some place. I'd love to point to it in the details page. BTW, I've applied the unconditional Reply-To: munging patch. -Barry From chuqui@plaidworks.com Thu Nov 16 05:07:22 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 15 Nov 2000 21:07:22 -0800 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: <20001115173927.E11322@impressive.net> References: <14865.27405.857124.313441@anthem.concentric.net> <20001115173927.E11322@impressive.net> Message-ID: At 5:39 PM -0500 11/15/00, Gerald Oskoboiny wrote: >Another solution to this is the Mail-Followup-To header: >(not widely supported yet) > > http://cr.yp.to/proto/replyto.html No, it's actually stillborn... dan may still be pushing it, but I don't think anyone else is. Since the new RFC supports list-post, we really don't NEED Mail-Followup-To. Instead, an MUA can look for the existance of List-Post and use it to enable reply-to-list. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) Be just, and fear not. From gerald@impressive.net Thu Nov 16 05:48:57 2000 From: gerald@impressive.net (Gerald Oskoboiny) Date: Thu, 16 Nov 2000 00:48:57 -0500 Subject: Mail-Followup-To (was Re: [Mailman-Developers] Re: Reverting question) In-Reply-To: ; from chuqui@plaidworks.com on Wed, Nov 15, 2000 at 09:07:22PM -0800 References: <14865.27405.857124.313441@anthem.concentric.net> <20001115173927.E11322@impressive.net> Message-ID: <20001116004857.F11322@impressive.net> On Wed, Nov 15, 2000 at 09:07:22PM -0800, Chuq Von Rospach wrote: > At 5:39 PM -0500 11/15/00, Gerald Oskoboiny wrote: > > >Another solution to this is the Mail-Followup-To header: > >(not widely supported yet) > > > > http://cr.yp.to/proto/replyto.html > > No, it's actually stillborn... That's a matter of opinion :) > dan may still be pushing it, but I don't think anyone else is. There's me, and a few other people... (and you, once I convince you ;) I just pointed to Dan's page since that's the best writeup of it that I have found. > Since the new RFC supports list-post, we really don't NEED > Mail-Followup-To. Instead, an MUA can look for the existance of > List-Post and use it to enable reply-to-list. That doesn't work for crossposted threads (like this one), or for non-subscribers who want to be Cc'd on a given thread. -- Gerald Oskoboiny http://impressive.net/people/gerald/ From barry@digicool.com Thu Nov 16 13:54:16 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 08:54:16 -0500 (EST) Subject: [Mailman-Developers] Re: Reverting question References: <14865.27405.857124.313441@anthem.concentric.net> <20001115173927.E11322@impressive.net> Message-ID: <14867.59144.179977.605777@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> Since the new RFC supports list-post, we really don't NEED CVR> Mail-Followup-To. Instead, an MUA can look for the existance CVR> of List-Post and use it to enable reply-to-list. Chuq, are you referring to draft-ietf-drums-msg-fmt-09.txt of 11-Sep-2000? There's no mention of List-Post in there that I can find. -Barry From Nigel.Metheringham@VData.co.uk Thu Nov 16 17:19:52 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Thu, 16 Nov 2000 17:19:52 +0000 Subject: [Mailman-Developers] Re: Reverting question In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Thu, 16 Nov 2000 08:54:16 EST." <14867.59144.179977.605777@anthem.concentric.net> Message-ID: For those with too little to do and read, there is another discussion on exactly this sort of topic going on over in the exmh lists (exmh is a MUA). See http://www.xray.mpe.mpg.de/mailing-lists/exmh/ specifically the thread over October and November entitled "state of the world" and also "Reply magic for lists" Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From lohner@debian.org Thu Nov 16 17:32:53 2000 From: lohner@debian.org (Nils Lohner) Date: Thu, 16 Nov 2000 18:32:53 +0100 Subject: [Mailman-Developers] mailman 2.0 beta 5 problem Message-ID: <200011161732.SAA05076@topaze.ecf.teradyne.com> I've got a problem with mailman. I installed the debian package ii mailman 2.0beta5-1 Powerful, web based list processor and after setting up a list tried the menus. I've got several problems and questions. - in the apache error log: client denied by server configuration: /usr/doc/mailman/images/mailman.jpg I've tried allowing everything to man /usr/doc/ work as in the Readme.debian (in the .deb) but no luck. - when I submit my changes to something it tries to contact www.localhost.com This seems to be exclusively a cgi problem. - when at http://131.101.98.199/mailman/admin the link for "General list information can be found at the mailing list overview page." points up one level (i.e. http://131.101.98.199/listinfo) I'll dig into this more tomorrow... help appreciated. Thanks, Nils. From barry@digicool.com Thu Nov 16 18:16:52 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 13:16:52 -0500 (EST) Subject: [Mailman-Developers] Re: Reverting question References: <14865.27405.857124.313441@anthem.concentric.net> <20001115173927.E11322@impressive.net> Message-ID: <14868.9364.657224.536341@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> Since the new RFC supports list-post, we really don't NEED CVR> Mail-Followup-To. Instead, an MUA can look for the existance CVR> of List-Post and use it to enable reply-to-list. Duh. List-Post: is in RFC 2369 and of course Mailman already inserts this header in messages. Sad what happens to short term memory when you get older. :) BTW, I think this is the right bit of information that an MUA would need to support a "post-to-list" option. Seems to me all the information is there without Reply-To munging for the user agent to support whatever followup/reply operation it wants. Reply-To munging should be considered a hack that's (questionably) necessary until the MUAs catch on. -Barry From barry@digicool.com Thu Nov 16 20:12:05 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 15:12:05 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? References: <12203.974323139@kanga.nu> Message-ID: <14868.16277.437055.764467@anthem.concentric.net> >>>>> "JCL" == J C Lawrence writes: JCL> I'm looking to setup a test Mailman site under Postfix (need JCL> to do some performance modelling of Exim against Postfix JCL> under Mailman for a given membership list). I can get a JCL> basic setup working no problem -- now I'm looking to JCL> duplicate the neater aspects I already have under Exim, such JCL> as losing the Mailman aliases file, etc. Anybody already JCL> done this? Nope, but I agree, this is one of the cooler features of Exim. -Barry From claw@kanga.nu Thu Nov 16 20:22:59 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 16 Nov 2000 12:22:59 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Thu, 16 Nov 2000 15:12:05 EST." <14868.16277.437055.764467@anthem.concentric.net> References: <12203.974323139@kanga.nu> <14868.16277.437055.764467@anthem.concentric.net> Message-ID: <9116.974406179@kanga.nu> On Thu, 16 Nov 2000 15:12:05 -0500 Barry A Warsaw wrote: >>>>>> "JCL" == J C Lawrence writes: JCL> I'm looking to setup a test Mailman site under Postfix (need to JCL> do some performance modelling of Exim against Postfix under JCL> Mailman for a given membership list). I can get a basic setup JCL> working no problem -- now I'm looking to duplicate the neater JCL> aspects I already have under Exim, such as losing the Mailman JCL> aliases file, etc. Anybody already done this? > Nope, but I agree, this is one of the cooler features of Exim. Yup, it sure is. That said I'm finding myself a little dissapointed in the configurability of Postfix in such regards. There's a whole lot of simple clevers you can do with Exim's filtering intelligence that just doesn't seem to be possible with Postfix. Another simple example is Exim's delivery time filters: http://www.exim.org/exim-html-3.10/doc/html/spec_14.html#SEC388 Very handy that. I've no idea yet how I'd do such under Postfix. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From barry@digicool.com Thu Nov 16 20:30:32 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 15:30:32 -0500 Subject: [Mailman-Developers] Future of pipermail? References: Message-ID: <14868.17384.847069.558207@anthem.concentric.net> >>>>> "GE" == Gunnar Evermann writes: GE> What are the current plans about the html archiver in mailman? GE> The major shortcomings of pipermail I see are: | - doesn't decode MIME | - doesn't handle mixed charsets | - no search engine | - somewhat convoluted code. GE> I run a mailing list with a lot of subscribers from Asia who GE> use MS Outlook, invariably posting (English) messages GE> spuriously tagged as CJK charsets (but only using the ASCII GE> subset) and gratuitously encoded in base64. Therefore I am GE> motivated to find a solution to handling MIME. :-( GE> I have hacked base64 support into pipermail and am currently GE> looking at supporting MIME (probably using mimecntl.py), but GE> this will just be short-term fixes. GE> I know I could just switch to MHonarc or whatever but I like GE> the idea of having a nicely structured, modular archiver GE> tightly integrated with mailman. Then why are you using Pipermail. :) Anyway, yes, we'd like to rewrite the archiver from the ground up, but nobody's working on that yet. I suspect either Jeremy or I will tackle it at some point. In the meantime, if you have useful patches, upload them to SourceForge so they don't get lost. -Barry From otaylor@redhat.com Thu Nov 16 21:09:38 2000 From: otaylor@redhat.com (Owen Taylor) Date: 16 Nov 2000 16:09:38 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: J C Lawrence's message of "Wed, 15 Nov 2000 13:18:59 -0800" References: <12203.974323139@kanga.nu> Message-ID: --=-=-= J C Lawrence writes: > I'm looking to setup a test Mailman site under Postfix (need to do > some performance modelling of Exim against Postfix under Mailman for > a given membership list). I can get a basic setup working no > problem -- now I'm looking to duplicate the neater aspects I already > have under Exim, such as losing the Mailman aliases file, etc. > Anybody already done this? Well, you can't lose the aliases file, but you can make it pretty darn invisible. From postfix's main.cf: alias_maps = hash:/etc/aliases,hash:/home/mailman/aliases And the following patch automatically updates the file. Regards, Owen --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=mailman-2.0beta5-postfix.patch --- mailman-2.0beta5/bin/newlist.postfix Sat Aug 5 18:24:11 2000 +++ mailman-2.0beta5/bin/newlist Sat Aug 5 18:26:08 2000 @@ -56,6 +56,14 @@ %(owner2)s %(listname)s-admin ''' +ALIASFILETEMPLATE = ''' +## %(listname)s mailing list +## created: %(date)s %(user)s +%(list)s "|%(wrapper)s post %(listname)s" +%(admin)s "|%(wrapper)s mailowner %(listname)s" +%(request)s "|%(wrapper)s mailcmd %(listname)s" +%(owner2)s %(listname)s-admin +''' def getusername(): @@ -117,7 +125,11 @@ except Errors.MMListAlreadyExistsError: usage(1, 'List already exists: ' + listname) - print ALIASTEMPLATE % { + aliaspath = os.path.join(mm_cfg.PREFIX, 'aliases') + print "Appending aliases to " + aliaspath + + aliasfile = open(aliaspath, "a") + aliasfile.write (ALIASFILETEMPLATE % { 'listname': listname, 'list' : "%-24s" % (listname + ":"), 'wrapper' : '%s/wrapper' % mm_cfg.WRAPPER_DIR, @@ -126,8 +138,12 @@ 'owner2' : "%-24s" % ("%s-owner:" % listname), 'date' : time.strftime('%d-%b-%Y', time.localtime(time.time())), 'user' : getusername(), - } + }) + aliasfile.close() + print "Rehashing alias database" + os.system ("/usr/sbin/postalias hash:" + aliaspath) + if len(argv) < 5: print ("Hit enter to continue with %s owner notification..." % listname), --=-=-=-- From barry@digicool.com Thu Nov 16 21:22:20 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 16:22:20 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? References: <12203.974323139@kanga.nu> Message-ID: <14868.20492.287182.956825@anthem.concentric.net> >>>>> "OT" == Owen Taylor writes: OT> Well, you can't lose the aliases file, but you can make it OT> pretty darn invisible. >> From postfix's main.cf: OT> alias_maps = hash:/etc/aliases,hash:/home/mailman/aliases I do this. OT> And the following patch automatically updates the file. A couple of notes. First, the version of newlist that's going to be in rc3 (some time today) has a -o/--output switch that does the appending to a specified file, somewhat like the way your patch does (although the file name is taken from the command line and a few other minor differences). But the real problem is a permissions problem. Say you run postalias as yourself, because you installed Mailman. Or say you installed it as user `mailman', and that's the user you run postalias as. Then the resulting aliases.db file will be owned by you (or `mailman') and Postfix will try to deliver email destined for those addresses as your (or mailman's) gid, but not as the gid you compiled into the wrapper script. Thus I found that I had to run newaliases (a.k.a. postalias) as root for the gids to be correct. Not an insurmountable problem, but it's a pain. It would be nice if Postfix could be configured to automatically run newaliases if necessary (maybe there already is such an option?). -Barry From barry@digicool.com Thu Nov 16 22:35:03 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 16 Nov 2000 17:35:03 -0500 Subject: [Mailman-Developers] Announcing Mailman 2.0 release candidate 3 Message-ID: <14868.24855.54781.693947@anthem.concentric.net> Mailman 2.0 release candidate 3 is now available from SourceForge at http://sourceforge.net/project/showfiles.php?group_id=103 www.list.org should be updated soon (I'm not going to bother the gnu.org guys again until 2.0 final is released). Excerpts from the NEWS file are given below. This should quash the few known 2.0rc2 buglets. Of course the on-line documentation is available at: http://mailman.sourceforge.net New goal for 2.0 final: Wed 22-Nov-2000 Enjoy, -Barry -------------------- snip snip -------------------- 2.0 release candidate 3 (16-Nov-2000) - By popular demand, Reply-To: munging policy is now to always override any Reply-To: header in the original message, if reply_goes_to_list is set to "This list" or "Explicit Address" - bin/newlist given -q/--quiet flag instead of the positional argument - Hopefully last fix to DEFAULT_URL not ending in a slash sensitivity - 2.0rc2 buglets fixed: o newlist argument parsing o updating with unlocked lists o HyperArch.py traceback when there's no Content-Transfer-Encoding: header - SourceForge bugs fixed: 122358 (qmail-to-mailman.py listname case folding) - SourceForge patches applied: 102373 (qmail-to-mailman.py listname case folding) From otaylor@redhat.com Thu Nov 16 23:11:21 2000 From: otaylor@redhat.com (Owen Taylor) Date: 16 Nov 2000 18:11:21 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: barry@digicool.com's message of "Thu, 16 Nov 2000 16:22:20 -0500" References: <12203.974323139@kanga.nu> <14868.20492.287182.956825@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > >>>>> "OT" == Owen Taylor writes: > > OT> Well, you can't lose the aliases file, but you can make it > OT> pretty darn invisible. > > >> From postfix's main.cf: > > OT> alias_maps = hash:/etc/aliases,hash:/home/mailman/aliases > > I do this. > > OT> And the following patch automatically updates the file. > > A couple of notes. First, the version of newlist that's going to be > in rc3 (some time today) has a -o/--output switch that does the > appending to a specified file, somewhat like the way your patch does > (although the file name is taken from the command line and a few > other minor differences). Hmmm, it seems to me that that should be in the config system somehow - so everybody running newalias doesn't need to know to use the flag. (I can set up a wrapper script, but that isn't so nice.) > But the real problem is a permissions problem. Say you run postalias > as yourself, because you installed Mailman. Or say you installed it > as user `mailman', and that's the user you run postalias as. Then the > resulting aliases.db file will be owned by you (or `mailman') and > Postfix will try to deliver email destined for those addresses as your > (or mailman's) gid, but not as the gid you compiled into the wrapper > script. > > Thus I found that I had to run newaliases (a.k.a. postalias) as root > for the gids to be correct. Well, I have the mailman gid as the gid compiled into the wrapper script, so all mailman processesing occurs with the mailman uid/gid. I guess this doesn't work in general, though it works well for our setup. > Not an insurmountable problem, but it's a pain. It would be nice if > Postfix could be configured to automatically run newaliases if > necessary (maybe there already is such an option?). Well, it wouldn't very secure if a user could put something into an alias file, and then postfix would compile that into an alias.db that is executed with root permissions... Basically, from a security point of view, the person composing the alias file can put arbitrary commands into the alias file, so the execution of commands in the alias file has to be done with the permissions of the owner of the file. Regards, Owen From andrewm@connect.com.au Fri Nov 17 00:04:39 2000 From: andrewm@connect.com.au (Andrew McNamara) Date: Fri, 17 Nov 2000 11:04:39 +1100 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Your message of "Thu, 16 Nov 2000 15:12:05 CDT." <14868.16277.437055.764467@anthem.concentric.net> Message-ID: <20001117000439.80F3B2870E@wawura.off.connect.com.au> > JCL> I'm looking to setup a test Mailman site under Postfix (need > JCL> to do some performance modelling of Exim against Postfix > JCL> under Mailman for a given membership list). I can get a > JCL> basic setup working no problem -- now I'm looking to > JCL> duplicate the neater aspects I already have under Exim, such > JCL> as losing the Mailman aliases file, etc. Anybody already > JCL> done this? > >Nope, but I agree, this is one of the cooler features of Exim. Can you give a brief precis of how Exim presents this to the admin? I might be able to suggest an equivilent feature, or at least inject the idea into Postfix's development. --- Andrew McNamara (System Architect) connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111 From claw@kanga.nu Fri Nov 17 00:08:15 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 16 Nov 2000 16:08:15 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from Andrew McNamara of "Fri, 17 Nov 2000 11:04:39 +1100." <20001117000439.80F3B2870E@wawura.off.connect.com.au> References: <20001117000439.80F3B2870E@wawura.off.connect.com.au> Message-ID: <29159.974419695@kanga.nu> On Fri, 17 Nov 2000 11:04:39 +1100 Andrew McNamara wrote: > Can you give a brief precis of how Exim presents this to the > admin? I might be able to suggest an equivilent feature, or at > least inject the idea into Postfix's development. See http://www.exim.org/howto/mailman.html -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From andrewm@connect.com.au Fri Nov 17 00:29:31 2000 From: andrewm@connect.com.au (Andrew McNamara) Date: Fri, 17 Nov 2000 11:29:31 +1100 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Your message of "Thu, 16 Nov 2000 12:22:59 -0800." <9116.974406179@kanga.nu> Message-ID: <20001117002931.CBE5A2870A@wawura.off.connect.com.au> >Yup, it sure is. That said I'm finding myself a little dissapointed >in the configurability of Postfix in such regards. There's a whole >lot of simple clevers you can do with Exim's filtering intelligence >that just doesn't seem to be possible with Postfix. Another simple >example is Exim's delivery time filters: > > http://www.exim.org/exim-html-3.10/doc/html/spec_14.html#SEC388 > >Very handy that. I've no idea yet how I'd do such under Postfix. If you are using one of the recent Postfix snapshots, it has this functionality. It's documented in the file FILTER_README in the top level directory. I realise that a snapshot isn't appropriate for some sites (however, Wietse's definition of a snapshot is simply a release with features that may not be carried through - the code receives his usual extensive testing before release). --- Andrew McNamara (System Architect) connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111 From fhorch@ecoaccess.org Fri Nov 17 01:22:02 2000 From: fhorch@ecoaccess.org (Fred Wilson Horch) Date: Thu, 16 Nov 2000 20:22:02 -0500 Subject: [Mailman-Developers] Running Mailman CGI under Zope ZServer Message-ID: <3A14883A.D1A8B45@ecoaccess.org> Hello out there, I'm looking for someone with some more Zope "Zen" to help me figure out how to add support for legacy CGI scripts (specifically Mailman 1.1) to Zope. To recap, I know and understand how to get Apache to play nicely with Zope. What I'm trying to do is remove the need for Apache by allowing ZServer to serve CGI scripts itself. The motivation is to allow me to run Mailman 1.1 on my Zope site without having to have Apache (or any other web server) configured and running. First, am I really the first person to try running CGI scripts from ZServer? I have found some hints here and there of people doing somewhat similar things, but I haven't yet found a product for easily adding legacy CGI scripts to a Zope site. It seems most people run Zope behind Apache. Is ZServer really slow or buggy or something? Second, is Mailman going to be integrated with Zope? It seems like a natural fit. It would be nice to move away from the pipermail archiving and use the Zope object database for archiving messages instead. Third, thanks to code from Eric Walstad and Chris Withers (author of PathHandler), I have hacked up an external method that sort of works. It attempts to add to Zope the functionality of ScriptAlias in Apache. But I'm running into some problems with the HTTP header. The attached code simply ignores the HTTP header sent by the Mailman CGI script, letting Zope do its thing. But this becomes a problem when Mailman wants to set cookies for authentication. Can anyone help me figure out how to convince Zope to let the CGI script send both the HTTP header and message body? Thanks, Fred Here's my external method. It is called by a Path Handler object. Some notes: 1) REQUEST['PATH_INFO'] seems to be brain dead 2) the path_to_handle key is inserted by the PathHandler product 3) this is written for Linux; I'm not sure how Windows handles os.popen 4) I don't understand why sometimes this function gets called with one argument and sometimes with two -- I always just ignore the second argument import os, os.path, string def testing(self, second=None): if self.REQUEST.has_key('path_to_handle'): script = self.REQUEST['path_to_handle'][0] if script: script_path = '/home/mailman/cgi-bin/%s' % script if os.path.isfile(script_path): ENV = '' if len(self.REQUEST['path_to_handle']) > 1: ENV = ENV + 'PATH_INFO=/%s' % string.join(self.REQUEST['path_to_handle'][1:],'/') elif self.REQUEST['PATH_INFO'][-1] == '/': ENV = ENV + 'PATH_INFO=/' ENV = ENV + ' HTTP_HOST=%s' % self.REQUEST['HTTP_HOST'] ENV = ENV + ' SERVER_NAME=%s' % self.REQUEST['SERVER_NAME'] f = os.popen("/usr/bin/env %s %s" % \ (ENV, script_path)) header = 1 while header: if (f.readline() == '\n'): header = 0 output = f.read() # grab the output of the CGI here status = f.close() else: return '%s: CGI script not found.' % script else: return 'No CGI script specified.' else: return 'No path to handle.' return output + '

' + str(self.REQUEST.keys()) + '
' + str(self.REQUEST) From claw@kanga.nu Fri Nov 17 01:33:21 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 16 Nov 2000 17:33:21 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Thu, 16 Nov 2000 16:22:20 EST." <14868.20492.287182.956825@anthem.concentric.net> References: <12203.974323139@kanga.nu> <14868.20492.287182.956825@anthem.concentric.net> Message-ID: <4622.974424801@kanga.nu> On Thu, 16 Nov 2000 16:22:20 -0500 Barry A Warsaw wrote: > A couple of notes. First, the version of newlist that's going to > be in rc3 (some time today) has a -o/--output switch that does the > appending to a specified file, somewhat like the way your patch > does (although the file name is taken from the command line and a > few other minor differences). I'm currently toying about to see if I can't persuade some of the alias re-writing and procmail to play to get the sme hiding level as Exim. The most obvious attack is via the $luser, but I'm not real happy with that. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Fri Nov 17 01:34:15 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 16 Nov 2000 17:34:15 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from Owen Taylor of "16 Nov 2000 16:09:38 EST." References: <12203.974323139@kanga.nu> Message-ID: <4724.974424855@kanga.nu> On 16 Nov 2000 16:09:38 -0500 Owen Taylor wrote: > Well, you can't lose the aliases file, but you can make it pretty > darn invisible. Yeah, I'd gotten that far. I'd really like to get entirely rid of the Mailman alias file however. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From andrewm@connect.com.au Fri Nov 17 01:56:56 2000 From: andrewm@connect.com.au (Andrew McNamara) Date: Fri, 17 Nov 2000 12:56:56 +1100 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Your message of "Thu, 16 Nov 2000 16:08:15 -0800." <29159.974419695@kanga.nu> Message-ID: <20001117015656.DC2E328706@wawura.off.connect.com.au> >> Can you give a brief precis of how Exim presents this to the >> admin? I might be able to suggest an equivilent feature, or at >> least inject the idea into Postfix's development. > >See http://www.exim.org/howto/mailman.html Okay - the closest I can come to this off the top of my head is to define "fallback_transport" to point to a "pipe" transport that called Mailman. Mailman would then receive all unknown local destinations, and would have to recognise owner- and -request style addresses also. It would also need to return EX_NOUSER (exit 67). Not as pretty, but should work okay. --- Andrew McNamara (System Architect) connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111 From jeremy@alum.mit.edu Fri Nov 17 02:41:32 2000 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 16 Nov 2000 21:41:32 -0500 (EST) Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: <710385309@toto.iv> Message-ID: <14868.39644.236453.34834@bitdiddle.concentric.net> >>>>> "BAW" == Barry A Warsaw writes: >>>>> "GE" == Gunnar Evermann writes: GE> What are the current plans about the html archiver in mailman? GE> The major shortcomings of pipermail I see are: | - doesn't decode MIME | - doesn't handle mixed charsets | - no search engine | - somewhat convoluted code. The last is the killer, because it makes the first three that much harder to accomplish. I tried some refactoring for the 2.0 release, but I'm not convinced there is much worth saving. BAW> Anyway, yes, we'd like to rewrite the archiver from the ground BAW> up, but nobody's working on that yet. I suspect either Jeremy BAW> or I will tackle it at some point. It's not near the top of my priorities at the moment, but I would like to work on a new archiver. Perhaps we should start sketching some design issues in Barry's Mailman wiki. I would also like to see separation of the presentation part of the archive from the code, so that it is easier to customize the presentation. Jeremy From chuqui@plaidworks.com Fri Nov 17 04:42:27 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Thu, 16 Nov 2000 20:42:27 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: <14868.39644.236453.34834@bitdiddle.concentric.net> References: <14868.39644.236453.34834@bitdiddle.concentric.net> Message-ID: At 9:41 PM -0500 11/16/00, Jeremy Hylton wrote: > >It's not near the top of my priorities at the moment, but I would like >to work on a new archiver. Perhaps we should start sketching some >design issues in Barry's Mailman wiki. Does mailman have to an an integrated archiver? Isn't this another variant of the "integrated MTA" discussion? I'd prefer, honestly, splitting out pipermail (or it's successor) into a separately tracked project, and building a standard interface that will work with it and with other archivers, like Mhonarc. That way, a separate team can be built to work on Pipermail, and releases of each piece don't have to be wedged into the same schedule, and you simplify the complexity of each project to improve focus.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Fri Nov 17 04:47:47 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Thu, 16 Nov 2000 20:47:47 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: <20001117015656.DC2E328706@wawura.off.connect.com.au> References: <20001117015656.DC2E328706@wawura.off.connect.com.au> Message-ID: At 12:56 PM +1100 11/17/00, Andrew McNamara wrote: >Okay - the closest I can come to this off the top of my head is to >define "fallback_transport" to point to a "pipe" transport that called >Mailman. > >Mailman would then receive all unknown local destinations, and would >have to recognise owner- and -request style addresses also. It would >also need to return EX_NOUSER (exit 67). Not as pretty, but should work >okay. How about, as an alternative, adding an LDAP api to Mailman (or some other interface that could be attached to something like LDAP). that way, you could write a really simple thing like wrapper that could dynamically look up whether a list exists and then do something useful with it. So instead of trying to add this functionality to Mailman, you add an interface that allows a special tool to be written that adds taht functionality -- and allows for other things (one thing I *really* want is subscription validation for external uses, like authentification into my archives. this kills both birds with one stone and a couple of simple glue procs... Which leads me to something I've talked to Barry about, which is an open source authentification system that everything can plug into, including Mailman. So we can finally stop rewriting user databases from scratch every time we write a tool, and can integrate everything into a single system for a site... But that's down the road -- but think in terms of whether a feature like this stands alone, or whether a generalization of it can be used to solve multiple options..... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From barry@digicool.com Fri Nov 17 05:34:43 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 17 Nov 2000 00:34:43 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? References: <29159.974419695@kanga.nu> <20001117015656.DC2E328706@wawura.off.connect.com.au> <12203.974323139@kanga.nu> <14868.20492.287182.956825@anthem.concentric.net> <4622.974424801@kanga.nu> Message-ID: <14868.50035.50158.985566@anthem.concentric.net> >>>>> "JCL" == J C Lawrence writes: JCL> I'm currently toying about to see if I can't persuade some of JCL> the alias re-writing and procmail to play to get the sme JCL> hiding level as Exim. The most obvious attack is via the JCL> $luser, but I'm not real happy with that. >>>>> "AM" == Andrew McNamara writes: AM> Mailman would then receive all unknown local destinations, and AM> would have to recognise owner- and -request style addresses AM> also. It would also need to return EX_NOUSER (exit 67). Not as AM> pretty, but should work okay. Bling! You guys just put the puzzle together for me. What follows is a one hour hack, but it works well enough that I'm fairly certain it could be fleshed out into a real solution. First, add these configuration options to Postfix's main.cf: luser_relay = mm+$user@wooz.org recipient_delimiter = + (changing of course wooz.org to whatever your own domain is). Now all unknown recipients will be forwarded to mm+recipient@wooz.org, but they'll /really/ be sent to the `mm' alias. Which is defined as: mm: "|/home/mailman/mail/wrapper auto" This will send all such email through the `auto' script (given below) after traveling as normal through the wrapper safety net. You'll need this patch to src/mail-wrapper.c: -------------------- snip snip -------------------- Index: mail-wrapper.c =================================================================== RCS file: /cvsroot/mailman/mailman/src/mail-wrapper.c,v retrieving revision 1.16 diff -u -r1.16 mail-wrapper.c --- mail-wrapper.c 2000/08/02 03:23:25 1.16 +++ mail-wrapper.c 2000/11/17 05:19:41 @@ -36,6 +36,7 @@ "post", "mailcmd", "mailowner", + "auto", NULL /* Sentinel, don't remove */ }; -------------------- snip snip -------------------- and this patch to scripts/Makefile.in: -------------------- snip snip -------------------- Index: Makefile.in =================================================================== RCS file: /cvsroot/mailman/mailman/scripts/Makefile.in,v retrieving revision 1.5 diff -u -r1.5 Makefile.in --- Makefile.in 2000/03/21 06:26:37 1.5 +++ Makefile.in 2000/11/17 05:20:37 @@ -40,7 +40,7 @@ SHELL= /bin/sh -SCRIPTS= answer_majordomo_mail mailcmd mailowner post driver +SCRIPTS= answer_majordomo_mail mailcmd mailowner post driver auto # Modes for directories and executables created by the install # process. Default to group-writable directories but -------------------- snip snip -------------------- The new auto script is the heart of the beast. Briefly what it does is look at the recipients list (To:, Cc:, Resent-To:, and Resent-Cc: headers) to see if it has a valid Mailman list destination. It gathers this from all the list objects on the system and should be virtual host aware. Next it matches the recipients its found in the message against the list of possible valid Mailman destinations. If if finds no matches, it exits with EX_NOUSER. If it does find a match, it enqueues the message in a manner similar to the post, mailcmd, and mailowner scripts. qrunner would handle those enqueued messages without ever knowing the difference. Here are some gotchas, which I believe can either be worked out or lived with: - Say I send a message to bogus@wooz.org. This message will travel through the auto script and I've verified that you get a user unknown bounce, but the bounce message says that mm+bogus@wooz.org is the bouncing address, not bogus@wooz.org, which is what I'd like to see. - On a system with a lot of lists, auto either should just a trampoline into a long running daemon, or it should try to cache more information. Trying to open and read every config.db file for every received message is probably way overkill. - I'm not sure that auto as it's currently written will handle cross-posting correctly, since enqueuing the message multiple times (say for list1@, list2@, and list3@) will still result in just one .db and .msg file for the received message. Shouldn't be too hard to craft a different sha hash for each separately. - This won't live very nicely with other $luser_relay's if you've got them. It would be nice if Postfix had a slightly more elaborate hookable framework here, where I could say: "let auto try to handle the address, but if it can't, pass it on to the next hook in the sequence." - The recipient_delimiter bit is probably extraneous. So anyway, time for sleep. Have fun! If this can be fleshed out and completed, it can be added to 2.1. -Barry P.S. auto below requires Python 2.0. Taste of things to come. :) -------------------- snip snip -------------------- #! /usr/bin/env python # # Copyright (C) 1998,1999,2000 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. import sys import paths from Mailman import mm_cfg from Mailman import Utils from Mailman import MailList from Mailman import Message from Mailman.Logging.Utils import LogStdErr from Mailman.Logging.Syslog import syslog # Error code if it's really not a Mailman list addr destination EX_NOUSER = 67 LogStdErr('auto', 'auto') def fqdn_listname(listname, hostname): return ('%s@%s' % (listname, hostname)).lower() def main(): syslog('auto', 'Running the auto deliver script') # valid destinations valids = {} lists = {} for listname in Utils.list_names(): mlist = MailList.MailList(listname, lock=0) lists[listname] = mlist valids[fqdn_listname(listname, mlist.host_name)] = 1 syslog('auto', 'valids: %s' % valids) # Get the message from standard input msg = Message.Message(sys.stdin) addrs = [] for header in ('to', 'cc', 'resent-to', 'resent-cc'): addrs.extend(msg.getaddrlist(header)) syslog('auto', 'found these addresses: %s' % addrs) # Parse the headers. Keys are 'post', 'request', 'owner' destinations = {} for name, addr in addrs: localpart, hostname = addr.split('@', 1) try: listname, subdest = localpart.split('-', 1) except ValueError: listname = localpart subdest = 'post' if subdest in ('post', 'request', 'owner', 'admin') and \ valids.has_key(fqdn_listname(listname, hostname)): destinations.setdefault(subdest, []).append(listname) if not destinations: # Tell Postfix we found no valid list destinations syslog('auto', 'No Mailman destination found') return EX_NOUSER # TBD: This may not work for cross-posted messages for subdest, listnames in destinations.items(): for listname in listnames: syslog('auto', 'Enqueuing to %s-%s' % (listname, subdest)) mlist = lists[listname] if subdest == 'post': msg.Enqueue(mlist, tolist=1) elif subdest == 'request': msg.Enqueue(mlist, torequest=1) elif subdest in ('owner', 'admin'): msg.Enqueue(mlist, toadmin=1) return 0 if __name__ == '__main__': code = main() sys.exit(code) From Nigel.Metheringham@VData.co.uk Fri Nov 17 09:20:55 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Fri, 17 Nov 2000 09:20:55 +0000 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Thu, 16 Nov 2000 20:42:27 PST." Message-ID: chuqui@plaidworks.com said: > Does mailman have to an an integrated archiver? Isn't this another > variant of the "integrated MTA" discussion? Maybe. There are times when I would really love to have archive message id or URL references in messages for example - ie when a particularly great pearl of wisdom comes from you folks which I want to reference on another list it would be great if I could easily grab an archive reference from the original message (as opposed to going to the archives and hunting out the original message then copying the URL) to send on. > I'd prefer, honestly, splitting out pipermail (or it's successor) > into a separately tracked project, and building a standard interface > that will work with it and with other archivers, like Mhonarc. That > way, a separate team can be built to work on Pipermail, and releases > of each piece don't have to be wedged into the same schedule, and you > simplify the complexity of each project to improve focus.... However my general gut feel does strongly agree with Chuq here. I would also be keen on the idea of a MIME reprocessor front-ended onto Mailman - sort of like demime but more controllable, and particularly with the ability to strip unwanted MIME from the message but save the parts onto the MLM server, adding URLs to the processed message to allow the extracted parts to be retrieved. This is not really the archiver function, but has some ideas in common. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From Nigel.Metheringham@VData.co.uk Fri Nov 17 09:28:12 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Fri, 17 Nov 2000 09:28:12 +0000 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Fri, 17 Nov 2000 00:34:43 EST." <14868.50035.50158.985566@anthem.concentric.net> Message-ID: barry@digicool.com said: > The new auto script is the heart of the beast. Briefly what it does > is look at the recipients list (To:, Cc:, Resent-To:, and Resent-Cc: > headers) to see if it has a valid Mailman list destination. It > gathers this from all the list objects on the system and should be > virtual host aware. Parsing out of headers could be rather interesting... what about a message copied to 2 lists - its not going to get replicated is it? Can you just pass the target address one the command line to the auto script - makes life a *lot* easier. Alternatively, all the exim stuff does is a file existence test on a transformed address - sure that ought to be relatively easy to postfix. Nigel. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From chuqui@plaidworks.com Fri Nov 17 15:22:41 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Fri, 17 Nov 2000 07:22:41 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: References: Message-ID: At 9:20 AM +0000 11/17/00, Nigel Metheringham wrote: >chuqui@plaidworks.com said: >> Does mailman have to an an integrated archiver? Isn't this another >> variant of the "integrated MTA" discussion? > >Maybe. There are times when I would really love to have archive >message id or URL references in messages for example But does that imply it has to be integrated into mailman? Or just available. The only reason I see for it to be fully integrated is if we want to add a header pointing to the article in the archive -- that would require archive processing during delivery. Other than that, it can be post -processed, so it wouldn't need to be part of hte main MLM code, it simply has to interface with it. That'd allow both pieces to be managed separately. > > >I would also be keen on the idea of a MIME reprocessor front-ended onto >Mailman - sort of like demime but more controllable, I'm toying with that, actually. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From steffen@wflin3.asta.uni-wuppertal.de Fri Nov 17 17:33:47 2000 From: steffen@wflin3.asta.uni-wuppertal.de (Steffen Bardolatzi) Date: Fri, 17 Nov 2000 18:33:47 +0100 Subject: [Mailman-Developers] "Date:"-bug already solved in CR2? Message-ID: <20001117183347.F14904@wflin3.asta.uni-wuppertal.de> I recently hit this bug: Nov 10 16:42:12 2000 (5913) Delivery exception: illegal argument type for built- Nov 10 16:42:12 2000 (5913) Traceback (innermost last): File "/var/mailman/Mailman/Handlers/HandlerAPI.py", line 82, in do_pipeline func(mlist, msg, msgdata) File "/var/mailman/Mailman/Handlers/ToArchive.py", line 50, in process msg['Date'] = date File "/var/mailman/Mailman/pythonlib/rfc822.py", line 393, in __setitem__ text = name + ": " + value TypeError: illegal argument type for built-in operation (plus a corresponding message with the message-id was sent to the user mailman --> had to use grep to find the corrupted message file). ... which is caused by a missing Date-header an e-mail (in this case I simply took an mbox-file and "feeded" it via the MTA into mailman to create an archive). As as side effect the CPU was used by a python resulting in 0-10% idele time on our server - as for e-mail with a message-id with a complete message-id (that is with a domain); this took *until* I deleted these e-mail in the qfiles-directory [these ones with a message ID like 6686876ghgk (e.g., without domain) didn´t block the message queue]. Of course all other messages were blocked in the way mentioned above ... ==> in our office the system actually was used for that job with 90-100% ... ... hope this going to be fixed in 2.0 Final (still running Beta 6 to wait for Final 2.0). Sorry for the delay, I have been rather busy. Thanks in advance. ----- End forwarded message ----- From barry@digicool.com Fri Nov 17 20:36:06 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 17 Nov 2000 15:36:06 -0500 Subject: [Mailman-Developers] Future of pipermail? References: <14868.39644.236453.34834@bitdiddle.concentric.net> Message-ID: <14869.38582.796232.612354@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> Does mailman have to an an integrated archiver? Isn't this CVR> another variant of the "integrated MTA" discussion? CVR> I'd prefer, honestly, splitting out pipermail (or it's CVR> successor) into a separately tracked project, and building a CVR> standard interface that will work with it and with other CVR> archivers, like Mhonarc. That way, a separate team can be CVR> built to work on Pipermail, and releases of each piece don't CVR> have to be wedged into the same schedule, and you simplify CVR> the complexity of each project to improve focus.... This could probably work. I do want an archiver that I can bundle with releases (let's say final releases primarily) because it's just so much easier for newbies to get started if they have everything they need. They already have the burden of having to download and install Python (and maybe a web server and an MTA), so reducing the number of components you need to get started is a good thing. For all it's flaws, Pipermail provides an essential service with no extra work (it'd be nice to also have a bundled search solution). OTOH, I think we can handle it similar to the way Python does it's rpms or windows installers and add-ons like readline and Tk. We find a stable version of the archiver and bundle it in the big releases, but beta users may have to do more work. >>>>> "NM" == Nigel Metheringham writes: NM> Maybe. There are times when I would really love to have NM> archive message id or URL references in messages for example - CVR> But does that imply it has to be integrated into mailman? Or CVR> just available. CVR> The only reason I see for it to be fully integrated is if we CVR> want to add a header pointing to the article in the archive CVR> -- that would require archive processing during CVR> delivery. Maybe not, if we define an API that Mailman can ask the archiver, "give me a url for message-id XYZ". -Barry From claw@kanga.nu Fri Nov 17 20:45:15 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 17 Nov 2000 12:45:15 -0800 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? In-Reply-To: Message from Andrew McNamara of "Fri, 17 Nov 2000 11:29:31 +1100." <20001117002931.CBE5A2870A@wawura.off.connect.com.au> References: <20001117002931.CBE5A2870A@wawura.off.connect.com.au> Message-ID: <9939.974493915@kanga.nu> On Fri, 17 Nov 2000 11:29:31 +1100 Andrew McNamara wrote: > If you are using one of the recent Postfix snapshots, it has this > functionality. It's documented in the file FILTER_README in the > top level directory. I realise that a snapshot isn't appropriate > for some sites (however, Wietse's definition of a snapshot is > simply a release with features that may not be carried through - > the code receives his usual extensive testing before release). Wonderful -- I'm poking at it as I type. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Fri Nov 17 20:52:13 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 17 Nov 2000 12:52:13 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Thu, 16 Nov 2000 20:42:27 PST." References: <14868.39644.236453.34834@bitdiddle.concentric.net> Message-ID: <10378.974494333@kanga.nu> On Thu, 16 Nov 2000 20:42:27 -0800 Chuq Von Rospach wrote: > At 9:41 PM -0500 11/16/00, Jeremy Hylton wrote: >> It's not near the top of my priorities at the moment, but I >> would like to work on a new archiver. Perhaps we should start >> sketching some design issues in Barry's Mailman wiki. > Does mailman have to an an integrated archiver? Isn't this another > variant of the "integrated MTA" discussion? For inexperienced/new admins and sites there's significant advantage to the all-in-one approach. You and I may not be fond of it, but we also aren't members of that audience (which is a significant one for Mailman in terms of population). I'm actually much of the mind that the current state of the archiver is rather good when viewed from a Jamie Zawinski "Worse is Better" perspective. It fulfills a minimal need, and provides (reasonably, could be better) clear pointers to where to go if you want to handle archiving better. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From marc_news@valinux.com Fri Nov 17 21:05:21 2000 From: marc_news@valinux.com (Marc MERLIN) Date: Fri, 17 Nov 2000 13:05:21 -0800 Subject: [Mailman-Developers] Doing load balancing with mailman Message-ID: <20001117130521.V9808@marc.merlins.org> So I didn't hear anything back on mailman-users, would anyone here be more inspired with those questions? Thanks Marc ----- Forwarded message from Marc MERLIN ----- [I didn't Cc mailman-developers, but if I should ask programing specific questions there, let me know] While a correctly tuned mailman+exim config should be able to handle sourceforge's lists just fine, I've been encouraged to look into a failover/load balancing solution :-) So, I've seen two options so far: Option #1: Lists are physically split between let's say 2 mail servers. All mail goes to listname@lists.sourceforge.net, which is MXed the two mail servers. I know how to configure exim so that it rewrites the envelope To to resend the mail to the other list server if listname is not local to that mail server. So far so good. The problem, is that for web access, I then have to show URLs with http://lists1.sourceforge.net/lists/listinfo/listname and http://lists2.sourceforge.net/lists/listinfo/listname Ok, that can be fixed with some clever apache rewriting/proxying, but it'd be a bit iffy. As far as failover is concerned, I can rsync the list configs on a periodic basis and with some symlink magic, I could have one mail server mostly take over the other mail server's lists if it were to disappear. This requires mail header rewriting, http URL rewriting, plus other magic, and it's not very "automatic". Option #2: Hold on, this is scary :-) /var/local/mailman is NFS mounted on the mail servers. /var/local/mailman/qfiles is a symlink to local disk qrunner is modified to set a lockfile in qfiles and not data (it should hopefully be a simple patch) cron jobs are run on only one mail server, with the exception of qrunner. both mail servers offer web access (same URL with load balancing) to the same lists. Voila :-) Did I forget anything? Is there any reason why this wouldn't work? (mailman's locking looks NFS safe to me, but I don't know if having locks created for possibl the same lists in the mailman/locks directory, by two different machines would be a problem for some reason that would escape me right now). Is there anything else besides qrunner that I'd need to patch? Thanks Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key ------------------------------------------------------ Mailman-Users maillist - Mailman-Users@python.org http://www.python.org/mailman/listinfo/mailman-users ----- End forwarded message ----- From ge204@eng.cam.ac.uk Fri Nov 17 21:40:14 2000 From: ge204@eng.cam.ac.uk (Gunnar Evermann) Date: 17 Nov 2000 21:40:14 +0000 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: barry@digicool.com's message of "Fri, 17 Nov 2000 15:36:06 -0500" References: <14868.39644.236453.34834@bitdiddle.concentric.net> <14869.38582.796232.612354@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > I do want an archiver that I can bundle with releases (let's say final > releases primarily) because it's just so much easier for newbies to > get started if they have everything they need. yes, I think this is vital. When I needed to setup a couple of mailing list this is what convinced me to install Mailman. I think this appeals does not only to "newbies". Even experienced admins prefer not to spend ages fiddling half a dozen components just to get a mailing list working. Gunnar From marc_news@valinux.com Fri Nov 17 23:16:14 2000 From: marc_news@valinux.com (Marc MERLIN) Date: Fri, 17 Nov 2000 15:16:14 -0800 Subject: [Mailman-Developers] Problem with web_page_url Message-ID: <20001117151613.D17368@marc.merlins.org> So I noticed that if web_page_url points to a bad value, I can get to the admin page, but I cannot change it back to a good value as the submit link posts the changes to a bad URL. Basically, one of the list admins on one of my boxes thought he would get cute and changed http://lists.scruz.org/mailman/ to http://www.scruz.org/ I think mailman got specifically unhappy with the fact that '/mailman/' was missing at the end. The only way I was able to recover from this was to change the value back in the config.db with withlist. I'm not too sure how to fix that. Can mailman check the new value to make sure it works before allowing a user to change it to something bad? Marc -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f@merlins.org for PGP key From barry@digicool.com Sat Nov 18 03:07:10 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Fri, 17 Nov 2000 22:07:10 -0500 Subject: [Mailman-Developers] Postfix, Mailman, no aliases file, neat setup? References: <29159.974419695@kanga.nu> <20001117015656.DC2E328706@wawura.off.connect.com.au> <12203.974323139@kanga.nu> <14868.20492.287182.956825@anthem.concentric.net> <4622.974424801@kanga.nu> <14868.50035.50158.985566@anthem.concentric.net> Message-ID: <14869.62046.125012.324360@anthem.concentric.net> Here's a slightly better version of auto, which works with cross-posting. The key insight was that Postfix calls auto once for each mailing list recipient, with the Delivered-To: header set to the mailing list address. So this version sucks out the Delivered-To:'s and parses the necessary information out of there. Turns out the recipient_delimiter is necessary in order to get the name of the mailing list in the Delivered-To: header (with the `mm' prefix, which is stripped off). Because of this, each sha hash will be unique so they shouldn't step on each other's toes. The (potential) performance issue isn't addressed here, nor is the bogus address bounce notifications. The former will need a little bit of extra help from other parts of Mailman, and the latter may take some changes in Postfix. But IMO, it's not a bad hack! Enjoy, -Barry -------------------- snip snip -------------------- #! /usr/bin/env python # # Copyright (C) 1998,1999,2000 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. import sys import paths from Mailman import mm_cfg from Mailman import Utils from Mailman import MailList from Mailman import Message from Mailman.Logging.Utils import LogStdErr from Mailman.Logging.Syslog import syslog # Error code if it's really not a Mailman list addr destination EX_NOUSER = 67 LogStdErr('auto', 'auto') def fqdn_listname(listname, hostname): return ('%s@%s' % (listname, hostname)).lower() def main(): syslog('auto', 'Running the auto deliver script') # valid destinations valids = {} lists = {} for listname in Utils.list_names(): mlist = MailList.MailList(listname, lock=0) lists[listname] = mlist valids[fqdn_listname(listname, mlist.host_name)] = 1 # Get the message from standard input msg = Message.Message(sys.stdin) addrs = [addr for name, addr in msg.getaddrlist('delivered-to')] syslog('auto', 'found these addresses: %s' % addrs) # Parse the headers. Keys are 'post', 'request', 'owner' destinations = {} for addr in addrs: localpart, hostname = addr.split('@', 1) try: listname, subdest = localpart.split('-', 1) except ValueError: listname = localpart subdest = 'post' try: prefix, listname = listname.split('+') except ValueError: prefix = None if subdest in ('post', 'request', 'owner', 'admin') and \ valids.has_key(fqdn_listname(listname, hostname)) and \ prefix == 'mm': destinations.setdefault(subdest, []).append(listname) if not destinations: # Tell Postfix we found no valid list destinations syslog('auto', 'No Mailman destination found') return EX_NOUSER syslog('auto', 'destinations: %s' % destinations) # TBD: This may not work for cross-posted messages for subdest, listnames in destinations.items(): for listname in listnames: syslog('auto', 'Enqueuing to %s-%s' % (listname, subdest)) mlist = lists[listname] if subdest == 'post': msg.Enqueue(mlist, tolist=1) elif subdest == 'request': msg.Enqueue(mlist, torequest=1) elif subdest in ('owner', 'admin'): msg.Enqueue(mlist, toadmin=1) # success return 0 if __name__ == '__main__': code = main() sys.exit(code) From claw@kanga.nu Sat Nov 18 03:24:05 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 17 Nov 2000 19:24:05 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: Message from Nigel Metheringham of "Fri, 17 Nov 2000 09:20:55 GMT." References: Message-ID: <12250.974517845@kanga.nu> On Fri, 17 Nov 2000 09:20:55 +0000 Nigel Metheringham wrote: > chuqui@plaidworks.com said: >> Does mailman have to an an integrated archiver? Isn't this >> another variant of the "integrated MTA" discussion? > Maybe. There are times when I would really love to have archive > message id or URL references in messages for example - ie when a > particularly great pearl of wisdom comes from you folks which I > want to reference on another list it would be great if I could > easily grab an archive reference from the original message (as > opposed to going to the archives and hunting out the original > message then copying the URL) to send on. One of my users (Dominic Edison IIRC) hacked a CGI to do this with MHonArc based archives. Hand it a MessageID and it hands you the archive page for that message. You can find it in the tasks at Kanga.Nu (I have yet to enable it). -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From andyheath2@lineone.net Sat Nov 18 08:53:06 2000 From: andyheath2@lineone.net (Andy Heath) Date: Sat, 18 Nov 2000 08:53:06 +0000 Subject: [Mailman-Developers] interface mailman list passwords python cgi Message-ID: <3A164372.2B43C505@lineone.net> I don't know whether I should send this to mailman users or developers so apologies if not appropriate. I need to write some cgi applications with python that interface to mailman 1.0 email id's and passwords - access only to list members, that sort of thing. I'd appreciate advice on how to go about this. I find some "config.db" files which obviously is some sort of database containing the info. Is there some easy way I can get into this info ? Later I will need to move the solution to working with mailman 2.0 All advice welcome Andy -- ------------------------------------------------------------ Andy Heath Home:+44 (0)114 2885738 a.k.heath@shu.ac.uk From claw@kanga.nu Sat Nov 18 16:38:44 2000 From: claw@kanga.nu (J C Lawrence) Date: Sat, 18 Nov 2000 08:38:44 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] interface mailman list passwords python cgi In-Reply-To: Message from Andy Heath of "Sat, 18 Nov 2000 08:53:06 GMT." <3A164372.2B43C505@lineone.net> References: <3A164372.2B43C505@lineone.net> Message-ID: <14507.974565524@kanga.nu> On Sat, 18 Nov 2000 08:53:06 +0000 Andy Heath wrote: > I need to write some cgi applications with python that interface > to mailman 1.0 email id's and passwords - access only to list > members, that sort of thing. Study ~bin/withlist. > I'd appreciate advice on how to go about this. I find some > "config.db" files which obviously is some sort of database > containing the info. Is there some easy way I can get into this > info ? config.db is a python pickle -- essentially a streamed python structure. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From gconnor@nekodojo.org Sun Nov 19 06:59:03 2000 From: gconnor@nekodojo.org (Greg Connor) Date: Sat, 18 Nov 2000 22:59:03 -0800 Subject: [Mailman-Developers] Re: [Mailman-Users] Bug in 2.0rc2 adding newlist References: <3A106ECF.1857.594BDCE@localhost> Message-ID: <3A177A37.46F87CDD@nekodojo.org> Phil Barnett wrote: > > Running 2.0rc2. > > I was trying to add a list with bin/newlist and got this: > ... > File "/home/mailman/Mailman/Utils.py", line 568, in mkdir > os.mkdir(dir, mode) > OSError: [Errno 2] No such file or directory: > '/home/mailman/archives/private/bin/newlist.mbox' > I got this too. Looks like a buglet in 2.0rc2. However, I noticed that running bin/newlist foo fails, but just running bin/newlist and then typing the new listname when prompted, works. When you put the name on the command line, it looks like it picks up "bin/newlist" as the name instead. -- Greg Connor From Antti.Siiskonen@tut.fi Sun Nov 19 11:35:19 2000 From: Antti.Siiskonen@tut.fi (Antti Siiskonen) Date: 19 Nov 2000 13:35:19 +0200 Subject: [Mailman-Developers] how to limit posting to a domain? Message-ID: Is there a way to allow posting only for those within a certain domain? It seems to me that .. 'Addresses of members accepted for posting to this list without implicit approval requirement.' .. does not take regexps. Is there a way around this or is this feature just missing from Mailman? -- //Antti From lovelace@wayfarer.org Sun Nov 19 14:36:26 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Sun, 19 Nov 2000 09:36:26 -0500 Subject: [Mailman-Developers] Re: Bug in 2.0rc2 adding newlist References: <3A106ECF.1857.594BDCE@localhost> <3A177A37.46F87CDD@nekodojo.org> Message-ID: <3A17E56A.691DD65E@wayfarer.org> Greg Connor wrote: > > Phil Barnett wrote: > > > > Running 2.0rc2. > > > > I was trying to add a list with bin/newlist and got this: > > > ... > > File "/home/mailman/Mailman/Utils.py", line 568, in mkdir > > os.mkdir(dir, mode) > > OSError: [Errno 2] No such file or directory: > > '/home/mailman/archives/private/bin/newlist.mbox' > > > > I got this too. Looks like a buglet in 2.0rc2. However, I noticed that > running > bin/newlist foo > fails, but just running > bin/newlist > and then typing the new listname when prompted, works. When you put the > name on the command line, it looks like it picks up "bin/newlist" as the > name instead. > I believe this has been fixed in 2.0rc3. argv was being used instead of args. -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? From claw@kanga.nu Sun Nov 19 16:00:24 2000 From: claw@kanga.nu (J C Lawrence) Date: Sun, 19 Nov 2000 08:00:24 -0800 Subject: [Mailman-Developers] how to limit posting to a domain? In-Reply-To: Message from Antti Siiskonen of "19 Nov 2000 13:35:19 +0200." References: Message-ID: <16831.974649624@kanga.nu> On 19 Nov 2000 13:35:19 +0200 Antti Siiskonen wrote: > Is there a way to allow posting only for those within a certain > domain? Not currently, and not for v2.0. This is commonly requested, especially by corporates. I'd expect to see it in v2.1. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From ken@kyler.com Sun Nov 19 16:25:07 2000 From: ken@kyler.com (Ken Kyler) Date: Sun, 19 Nov 2000 11:25:07 -0500 Subject: [Mailman-Developers] how to limit posting to a domain? In-Reply-To: <16831.974649624@kanga.nu> Message-ID: > > Is there a way to allow posting only for those within a certain > > domain? > > Not currently, and not for v2.0. This is commonly requested, > especially by corporates. I'd expect to see it in v2.1. Is it safe to assume the reverse will be true? That whole domains can be prohibited from posting? Ken From claw@kanga.nu Sun Nov 19 16:33:40 2000 From: claw@kanga.nu (J C Lawrence) Date: Sun, 19 Nov 2000 08:33:40 -0800 Subject: [Mailman-Developers] how to limit posting to a domain? In-Reply-To: Message from "Ken Kyler" of "Sun, 19 Nov 2000 11:25:07 EST." References: Message-ID: <21419.974651620@kanga.nu> On Sun, 19 Nov 2000 11:25:07 -0500 Ken Kyler wrote: > Is it safe to assume the reverse will be true? That whole domains > can be prohibited from posting? You can write mailman filters that will hold such posts for moderation, but Malman does not have the ability to ignore such. That said, this is likely trivial to do at either the MTA level, or as a pre-filter before the Mailman scripts: Just check the envelope and don't apss it through if it doesn't match. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From Nigel.Metheringham@VData.co.uk Mon Nov 20 11:14:50 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Mon, 20 Nov 2000 11:14:50 +0000 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) of "Fri, 17 Nov 2000 15:36:06 EST." <14869.38582.796232.612354@anthem.concentric.net> Message-ID: Continuing the thoughts on integration or separation of archiver functions... heres a flow of my ideal wrt to archiving and MIME processing of list messages:- - Message received by mailman - Policy decisions applied to message (ie accept, bounce, moderate) depending on things such as sender/recipient/list policy/MIME etc - Message archived. MIME attachments/portions split into separate retrievable sections. The archived MIME sections become available to latter processes as retrieval URLs. Overall Message ref also available to all subsequent processes (ie as additional header with this message archive URL) - Send out process set. This would allow, for example:- - some recipients to receive original MIME encoded message - some recipients to receive simplified message with limited MIME (maybe just text/plain) plus URLs - some receive text/plain version as a digest The send out messages would depend on a combination of list and personal policy - ie some people could be on a MIME accepting list, but prefer simplified mail. Or the list could mandate no transmitted MIME. The original policy set could also just block MIME or large MIME. However this is just one way things could work... I just get this feeling that archiving and MIME stripping can be closely related... Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From chuqui@plaidworks.com Tue Nov 21 05:41:51 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 20 Nov 2000 21:41:51 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: <14869.38582.796232.612354@anthem.concentric.net> References: <14868.39644.236453.34834@bitdiddle.concentric.net> <14869.38582.796232.612354@anthem.concentric.net> Message-ID: sorry for the delay. we went to Portland for our annual fall trip, and to be honest, I didn't turn on a computer the whole trip (and the world didn't end, either. amazing...) >I do want an archiver that I can bundle with releases (let's say final >releases primarily) because it's just so much easier for newbies to >get started if they have everything they need. fair point, one I've sort of ignored. In theory, I'd prefer a situation where we simply included a "turnkey" set of instructions for a preferred archiver, but in practice -- that theory doesn't work as well... > They already have the >burden of having to download and install Python (and maybe a web >server and an MTA), so reducing the number of components you need to >get started is a good thing. true. Or maybe the answer is some kind of meta-setup beast like some folks use for Apache? Or does that just add complexity without solving anything? I'm brainstorming -- not pretending to answers, but asking questions that'll help us understand where the answers ought to be... > >Maybe not, if we define an API that Mailman can ask the archiver, >"give me a url for message-id XYZ". man, I can be such a luddite some daays. I insist on thinking about APIs as hand-off interfaces, not bi-directional. Of course data can go both ways, although if you want to plug in an external archiver, the API has to be smart enough to return some kind of not-supported value (or preferably, a flag that can be queried during web-page creation as to whether or not ot show the option....) -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From barry@digicool.com Tue Nov 21 16:52:12 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 11:52:12 -0500 Subject: [Mailman-Developers] Future of pipermail? References: <14869.38582.796232.612354@anthem.concentric.net> Message-ID: <14874.43068.477150.391834@anthem.concentric.net> >>>>> "NM" == Nigel Metheringham writes: NM> Continuing the thoughts on integration or separation of NM> archiver functions... heres a flow of my ideal wrt to NM> archiving and MIME processing of list messages:- Good points Nigel. I've added some of these to the FutureOfPipermail Wiki. NM> However this is just one way things could work... I just get NM> this feeling that archiving and MIME stripping can be closely NM> related... Very possibly so. Having a general repository for messages and their MIME parts might be a good thing. I'm a little worried that we might be giving up independence and plugability, since I don't know whether any of the other 3rd party archivers support such features. -Barry From bbum@codefab.com Tue Nov 21 17:11:00 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Tue, 21 Nov 2000 12:11:00 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? Message-ID: <200011211711.eALHB1322170@bjork.codefab.com> WebDAV works really, really well as an archiver. And it would be particularly cool if the archive already had the MIME pieces exploded out. Even cooler would be if the messages sent to the list were rewritten such that all of the MIME pieces-- typically large, on some lists-- were actually URLs to the archived pieces on teh WebDAV server. If it had an interface that sent an HTTP message to some random webserver/appserver/cgi-bin letting it know a new message was submitted so that server could decide an appropriate time to rebuild an index file, that would allow the presentation of the archive to be offloaded from Mailman. Since WebDAV will be included with every copy of Apache 2.0 by default, there really isn't that much additional "moving pieces" load on the configuration. Mailman already requires diddling the web server configuration-- copy/paste a few more lines isn't that big of a deal. Anyone want to take this on? Oh, wait a minute, that's right... it has already been done! ftp://ftp.codefab.com/pub/unsupported/mailman_20000821_1123.tar.gz It is based on a version of Mailman from sometime around May or so... so, it needs to be updated to the latest version from the repository. However, it DOES work and has been used in production systems [who will remain nameless to protect this community from me venting] for quite some time. The company that hired me to build this actually paid a bunch of $$$ for it to be built-- as such, I actually had the freedom to thiink about doing it right and not just hacking it on as fast as possible. I don't know if I did do it right, but Barry provided quite a bit of guidance and I don't think it is a totally offensive set of changes... b.bum From chuqui@plaidworks.com Tue Nov 21 18:00:53 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 10:00:53 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <200011211711.eALHB1322170@bjork.codefab.com> References: <200011211711.eALHB1322170@bjork.codefab.com> Message-ID: At 12:11 PM -0500 11/21/00, Bill Bumgarner wrote: >WebDAV works really, really well as an archiver. webDAV? Oh, man. I guess I need to go look at another piece of technology. Wanna give those of us who have no clue what this is the 30 second executive overview? We can't, however, assume users are running Apache, even if it's the overwhelming choice... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Tue Nov 21 17:59:07 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 09:59:07 -0800 Subject: [Mailman-Developers] Future of pipermail? In-Reply-To: <14874.43068.477150.391834@anthem.concentric.net> References: <14869.38582.796232.612354@anthem.concentric.net> <14874.43068.477150.391834@anthem.concentric.net> Message-ID: At 11:52 AM -0500 11/21/00, Barry A. Warsaw wrote: >Very possibly so. Having a general repository for messages and their >MIME parts might be a good thing. I'm a little worried that we might >be giving up independence and plugability, since I don't know whether >any of the other 3rd party archivers support such features. some do. Maybe the answer is to allow the API to configure how the archiver wants it, so we can have mailman strip it to a text-only part if the archiver wants it that way. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From bbum@codefab.com Tue Nov 21 19:51:24 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Tue, 21 Nov 2000 14:51:24 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <200011211711.eALHB1322170@bjork.codefab.com> Message-ID: <200011211951.eALJpO325197@bjork.codefab.com> Chuq, et. al., See www.webdav.org for more information. WebDAV is a standard that extends, but does not modify, HTTP to allow for several additional operations against a web server. There are two levels to the specification. Class 0 will be described before-- it is complete and production quality. Class One is in the final stages of discussion and, as such, is not anywhere near having a reference implementation (but I will discuss briefly because it is very cool): - supports PUT of resources [files] onto the server (i.e. upload a file to server) - creation of collections (mkdir) - renaming/moving/deleting/copying of resources or collections - assigning properties to resources or collections. A set of properties is basically a chunk of XML associated with the collection or resource. For the MailMan WebDAV extensions, I use properties to store the mime-type and any extended headers associated with the attachment in question. - searching the properties database for particular values/attributes - locking; locking is limited to write locks because read locks would have required modification of the HTTP spec. Can have either shared or exclusive write locks. However, it is trivial to throw together a cgi-bin/ that allows read access to a resource if and only if the client sends a long the appropriate shared write lock token. This is used in the Mailman WebDAV extensions to support lmited access to the archive of messages. - locks can have a timeout. As such, it is trivial to generate an URL that allows access to a particular piece of the archive for, say, only the next 30 minutes. --- Most importantly, WebDAV is truly a standard and a very widely accepted standard, at that. - Apple ships a WebDAV client with Mac OS X. The builds of Apache with Mac OS X and Mac OS X Server both include WebDAV modules that are disabled by default. - Microsoft has full blown WebDAV support in Office and Windows 2000. In Office, you can open a document served by a WebDAV server and subsequently hit ctrl-s to save (or save as) as if the document were on a local fileserver. - IBM's WebSphere fully supports WebDAV - Zope fully supports WebDAV - GoLive CyberStudio and other HTML authoring packages either ship with WebDAV support or have announced that they will soon. So, WebDAV is, by no means, an apache only solution! Because it is a simple extension to HTTP-- and class 0 is relatively trivial in nature-- WebDAV capabilities can be provided via cgi-bin/ without a problem. --- The new class of DAV functionality is aimed at full support for version control. This includes basic revisioning as well as tags, maps, workareas, multiple users, etc... This revision to the spec is in the final stages of discussion and Greg Stein-- author of mod_dav-- is actively working with the group to create an implementation of the spec. --- In terms of Mailman, there is no real reason why creating the archive should be something that isn't completely abstracted within Mailman. Archival is basically three tasks: - storing data - storing meta-infrormation - creating some kind of index/view Decoding of attachments and rewriting the messages is mostly just an extension of the above concepts. As such, the Mailman archival interface could be an abstract interface of which we provide two concrete implementations; direct-to-filesystem and WebDAV. The actual process of creating an index/view isn't really a Mailman problem-- obivously, it is useful to ship with an out-of-the-box solution. b.bum From: Chuq Von Rospach Date: 2000-11-21 10:00:53 -0800 To: bbum@codefab.com, mailman-developers@python.org Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <200011211711.eALHB1322170@bjork.codefab.com> At 12:11 PM -0500 11/21/00, Bill Bumgarner wrote: >WebDAV works really, really well as an archiver. webDAV? Oh, man. I guess I need to go look at another piece of technology. Wanna give those of us who have no clue what this is the 30 second executive overview? We can't, however, assume users are running Apache, even if it's the overwhelming choice... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From Dan.Mick@west.sun.com Tue Nov 21 21:00:30 2000 From: Dan.Mick@west.sun.com (Dan Mick) Date: Tue, 21 Nov 2000 13:00:30 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011211951.eALJpO325197@bjork.codefab.com> Message-ID: <3A1AE26E.D82BBA53@west.sun.com> WebDAV: stands for "Web-based Distributed Authoring and Versioning", which sort of neatly sums up what Bill was saying about its capabilities. (It's new to me, too. There seem to be two new web-database "environments" a week these days.) From bbum@codefab.com Tue Nov 21 21:04:31 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Tue, 21 Nov 2000 16:04:31 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? Message-ID: <200011212104.eALL4W327658@bjork.codefab.com> Except that WebDAV isn't really new and it is a ratified standard that basically everyone is adopting! (but, yes, I should have included the acronym in the first place!) The key with WebDAV is that use therein means that you can point WebDAV enabled Mailman at Zope, IBM's websphere, Apache+mod_dav, or any of the other WebDAV enabled HTTP servers out there and it should "just work" (assuming they did the implementation right). There is nothing proprietary about the solution and choosing it as a foundation for archival/management of messages/attachments opens doors (instead of closing doors). b.bum From: Dan Mick Date: 2000-11-21 13:00:30 -0800 To: bbum@codefab.com Subject: Re: [Mailman-Developers] Re: Future of pipermail? CC: Chuq Von Rospach , mailman-developers@python.org X-Accept-Language: en X-Mailer: Mozilla 4.75 [en] (Win98; U) WebDAV: stands for "Web-based Distributed Authoring and Versioning", which sort of neatly sums up what Bill was saying about its capabilities. (It's new to me, too. There seem to be two new web-database "environments" a week these days.) From Dan Mick Tue Nov 21 21:55:42 2000 From: Dan Mick (Dan Mick) Date: Tue, 21 Nov 2000 13:55:42 -0800 (PST) Subject: [Mailman-Developers] Re: Future of pipermail? Message-ID: <200011212154.NAA19723@utopia.west.sun.com> > Except that WebDAV isn't really new and it is a ratified standard that > basically everyone is adopting! Ah, well, new is in the timespan of the beholder. To me, Web-based anything is "relatively new".... > (but, yes, I should have included the acronym in the first place!) > > The key with WebDAV is that use therein means that you can point WebDAV > enabled Mailman at Zope, IBM's websphere, Apache+mod_dav, or any of the other > WebDAV enabled HTTP servers out there and it should "just work" (assuming they > did the implementation right). There is nothing proprietary about the > solution and choosing it as a foundation for archival/management of > messages/attachments opens doors (instead of closing doors). Yeah, I understand the point. But understand that virtually everyone claims ubiquity long before they actually have it, too... I'll research some more about the web servers I use. From barry@digicool.com Tue Nov 21 22:33:52 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 17:33:52 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011211711.eALHB1322170@bjork.codefab.com> <200011211951.eALJpO325197@bjork.codefab.com> Message-ID: <14874.63568.856071.405653@anthem.concentric.net> >>>>> "BB" == Bill Bumgarner writes: BB> This revision to the spec is in the final stages of discussion BB> and Greg Stein-- author of mod_dav-- is actively working with BB> the group to create an implementation of the spec. I'll just note briefly that Greg is a long time Pythonista and Mailmaner (in fact he owns the list.org domain), so WebDAV support in Python ought to be very good. I want to read and think more, but I've been interested in WebDAV as a technology for a while now. -Barry From chuqui@plaidworks.com Tue Nov 21 22:58:14 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 14:58:14 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <14874.63568.856071.405653@anthem.concentric.net> References: <200011211711.eALHB1322170@bjork.codefab.com> <200011211951.eALJpO325197@bjork.codefab.com> <14874.63568.856071.405653@anthem.concentric.net> Message-ID: At 5:33 PM -0500 11/21/00, Barry A. Warsaw wrote: >I'll just note briefly that Greg is a long time Pythonista excuse me if I quietly twich at the terminology (does anyone know where this -ista stuff came from? My first run-in iwth it was Guy Kawasaki and his "evangelistas", but was it used before that? >I want to read and think more, but I've been interested in WebDAV as a >technology for a while now. I'm starting to see serious pushes (and I'll self-admit to be one here!) towards serious, hard-core integration of technologies. Which is a double-edged sword of the worst and best kind. I've talked to Barry about building ways to allow for site-wide authentication issues, for instance, and now we're seeing WebDAV (which I find interesting, and given it's going to be part of Apache 2.0, it's got to be considered heir presumptive in this technology psace now, over things like, oh, Zope. And I was looking over a couple of other things today (MetaDot to name one.. ) and going "hmm. you know, that stuff's got potential, too...) There's HUGE synergy potential here. We certainly shouldn't avoid those things, but the more we start tying into (and/or depending on) other technologies, the more we lock users into a single technology suite, the more we start depending on specific technology suites the more we lock uers into our view of how things work, and the less flexibility we give them in building their sites. And, to some degree, the less ability we have to adapt to future changes in the net universe ourselves. None of which is a reason to not do these things -- but it's reason to be wary. We really need to understand how these things interact and what the side effects are, including things as basic as "does this mean it'll never run on Windows NT again?" or "did we just set it up so they have to run apache?" I think there's a lot here we want to do (and need to do) but -- not for the sake of doing it. WebDAV looks fascinating. WebDAV -- as mailman's default archiver -- looks to me to be overkill at a quick evaluation. Yes, it'll be in there by default for apache 2.0, once Apache 2.0 ships and is out for a while and everyone upgrades. that's, what, 2 years out before 50% of the sites are running apache 2.x? Easily. Until then, what do we do? And what about non-apache sites? what's this do to mailman across all supported platforms? I dunno -- but there's a lot of complexity here we need to make sure we deal with in these issues. And to some degree, I think it's another excuse for me to pull out the "lots of separately tracked modules with API's" schtick, because we can build a Mailman API and a WebDAV plug-in for it(along with Mhonarc and Pipermail plug-ins) and over time, as WebDAV becomes endemic, switch the priority of development in its direction, without locking into it. In two years, there'll be other stuff beyond WebDAV, too, I'll bet. I know, I know... it's all details. Details are boring (grin). but necessary... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From ken@kyler.com Tue Nov 21 23:16:59 2000 From: ken@kyler.com (Ken Kyler) Date: Tue, 21 Nov 2000 18:16:59 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message-ID: > There's HUGE synergy potential here. We certainly shouldn't avoid > those things, but the more we start tying into (and/or depending on) > other technologies, the more we lock users into a single technology > suite, the more we start depending on specific technology suites the > more we lock uers into our view of how things work, and the less > flexibility we give them in building their sites. And, to some > degree, the less ability we have to adapt to future changes in the > net universe ourselves. I'd like to suggest that instead of integrating other technologies, that making Mailman open to them would be better. Such as offering the alternative to use Webdav, or making APIs for PHP, Perl, etc to interact with it. This is a classic COTS integration issue. If you depend too heavily on a product, it is possible for a future release of that product to break yours and the other guys don't care because you're a small fry in the pond. Been there myself and ended up gutting a product I was working on. My apologies for butting in - been lurking a long time and this is an interesting thread. Ken Kyler From barry@digicool.com Tue Nov 21 23:21:29 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 18:21:29 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011211711.eALHB1322170@bjork.codefab.com> <200011211951.eALJpO325197@bjork.codefab.com> <14874.63568.856071.405653@anthem.concentric.net> Message-ID: <14875.889.807156.688940@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> excuse me if I quietly twich at the terminology (does anyone CVR> know where this -ista stuff came from? My first run-in iwth CVR> it was Guy Kawasaki and his "evangelistas", but was it used CVR> before that? Dunno, and I'm not sure who invented "Pythonista" or "Perlie" or whatever. I actually like the term "Pythoneer" because it's reminiscent of "Pioneer" and "Engineer". Maybe we should just call them "Bruces" (to avoid confusion :). CVR> I'm starting to see serious pushes (and I'll self-admit to be CVR> one here!) towards serious, hard-core integration of CVR> technologies. Which is a double-edged sword of the worst and CVR> best kind. I think the trick is to balance what is necessary and expedient for Mailman with long term flexibility and integration with standards. Given the size of the todo list, where we can leverage existing technology we probably want opt that way instead of inventing our own. But the flexibility is important because different sites have so many different requirements. That's why I like the idea of moving toward a core set of classes and scripts that provide the basic Mailman functionality, building interfaces and hooks so that people can do the customization they want. CVR> There's HUGE synergy potential here. We certainly shouldn't CVR> avoid those things, but the more we start tying into (and/or CVR> depending on) other technologies, the more we lock users into CVR> a single technology suite, the more we start depending on CVR> specific technology suites the more we lock uers into our CVR> view of how things work, and the less flexibility we give CVR> them in building their sites. And, to some degree, the less CVR> ability we have to adapt to future changes in the net CVR> universe ourselves. CVR> None of which is a reason to not do these things -- but it's CVR> reason to be wary. We really need to understand how these CVR> things interact and what the side effects are, including CVR> things as basic as "does this mean it'll never run on Windows CVR> NT again?" or "did we just set it up so they have to run CVR> apache?" I agree. CVR> WebDAV looks fascinating. WebDAV -- as mailman's default CVR> archiver -- looks to me to be overkill at a quick CVR> evaluation. Yes, it'll be in there by default for apache 2.0, CVR> once Apache 2.0 ships and is out for a while and everyone CVR> upgrades. that's, what, 2 years out before 50% of the sites CVR> are running apache 2.x? Easily. Until then, what do we do? CVR> And what about non-apache sites? what's this do to mailman CVR> across all supported platforms? CVR> I dunno -- but there's a lot of complexity here we need to CVR> make sure we deal with in these issues. And to some degree, I CVR> think it's another excuse for me to pull out the "lots of CVR> separately tracked modules with API's" schtick, because we CVR> can build a Mailman API and a WebDAV plug-in for it(along CVR> with Mhonarc and Pipermail plug-ins) and over time, as WebDAV CVR> becomes endemic, switch the priority of development in its CVR> direction, without locking into it. In two years, there'll be CVR> other stuff beyond WebDAV, too, I'll bet. Agreed again. CVR> I know, I know... it's all details. Details are boring CVR> (grin). but necessary... Indeed. There's also the "worse is better" mantra, or put in other words: "All software sucks, let's just try to suck less". :) -Barry From barry@digicool.com Tue Nov 21 23:34:23 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 18:34:23 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: Message-ID: <14875.1663.422580.420642@anthem.concentric.net> >>>>> "KK" == Ken Kyler writes: KK> I'd like to suggest that instead of integrating other KK> technologies, that making Mailman open to them would be KK> better. Such as offering the alternative to use Webdav, or KK> making APIs for PHP, Perl, etc to interact with it. KK> This is a classic COTS integration issue. If you depend too KK> heavily on a product, it is possible for a future release of KK> that product to break yours and the other guys don't care KK> because you're a small fry in the pond. Been there myself and KK> ended up gutting a product I was working on. Yes, but I think there's a difference in that we're going to be relying on open standards, not closed COTS technology. Mailman, of course, is already built on a number of standards: RFC 822, SMTP, NNTP, HTTP, CGI, etc., etc. Any of these could go away or be replaced or whatever, but we're doing a good thing by relying on these standards. Of course, you have to pick the right ones, and (IMO) build an architecture that lets you more easily move to new standards when they arise and become useful. KK> My apologies for butting in - been lurking a long time and KK> this is an interesting thread. The more the merrier! -Barry From barry@digicool.com Tue Nov 21 23:39:36 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 18:39:36 -0500 Subject: [Mailman-Developers] Future of pipermail? References: <14868.39644.236453.34834@bitdiddle.concentric.net> <14869.38582.796232.612354@anthem.concentric.net> Message-ID: <14875.1976.757143.922146@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> true. Or maybe the answer is some kind of meta-setup beast CVR> like some folks use for Apache? Or does that just add CVR> complexity without solving anything? There's something in Python called `distutils' which is a way to distribute and build extensions to Python. I don't think distutils currently works well for applications, but that might be a direction to explore. Distutils will likely be the underlying technology for a Pythonic CPAN like thing that everyone's been pining for but few have had the time to pursue. One nice thing about distutils is that it makes the building of extension modules completely trivial. I have a couple of nice (optional) performance improvements written in C for Mailman 2.1 which are easily built and installed with a distutils `setup.py' script. -Barry From ken@kyler.com Tue Nov 21 23:55:57 2000 From: ken@kyler.com (Ken Kyler) Date: Tue, 21 Nov 2000 18:55:57 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <14875.1663.422580.420642@anthem.concentric.net> Message-ID: > standards. Of course, you have to pick the right ones, and (IMO) > build an architecture that lets you more easily move to new standards > when they arise and become useful. How true - what I would give for omniscience. Keep up the good work guys! If I ever finish grad school I'll help. Ken From chuqui@plaidworks.com Tue Nov 21 23:57:40 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 15:57:40 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: References: Message-ID: At 6:55 PM -0500 11/21/00, Ken Kyler wrote: >Keep up the good work guys! If I ever finish grad school I'll help. hey, this ain't HP, guy! we don't care about pieces of paper! (grin) -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From tanner@real-time.com Wed Nov 22 00:04:07 2000 From: tanner@real-time.com (Bob Tanner) Date: Tue, 21 Nov 2000 18:04:07 -0600 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck Message-ID: <20001121180407.A20735@real-time.com> I have to say I am very disappointed at mailman's performance with sendmail. Especially qrunner and how it works. I am running sendmail with 10 queues, but it looks like mailman only understands and utilizes 1 queue. It also looks like qrunner is single threaded AND multiple instances cannot be running at the same time. This lends itself to a very slow delivery system. If a message is not deliverable for whatever reason, mailman puts files into a directory called qfiles. Then onces every minute a process called qrunner is invoked to walk the qfiles directory trying to re-deliver the messages. What sucks is qrunner just tries to deliver each message one at a time. If message 5 takes forever on the rcpt to: command (for example), no other messages deliveries are attempted because qrunner sits on message 5 until it times out. I have turned the rcpt to timeout down to 2mins, but I am still constantly running 1000+ message in the qfiles directory. Any ideas on speeding things up? Will mailman support multiple "qfiles" directories in the future? -- Bob Tanner | Phone : (612)943-8700 http://www.mn-linux.org | Fax : (612)943-8500 Key fingerprint = 6C E9 51 4F D5 3E 4C 66 62 A9 10 E5 35 85 39 D9 From barry@digicool.com Wed Nov 22 00:04:10 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 19:04:10 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <14875.1663.422580.420642@anthem.concentric.net> Message-ID: <14875.3450.340146.28315@anthem.concentric.net> >>>>> "KK" == Ken Kyler writes: >> standards. Of course, you have to pick the right ones, and >> (IMO) build an architecture that lets you more easily move to >> new standards when they arise and become useful. KK> How true - what I would give for omniscience. Don't forget Guido's time machine! When he's away, we like to borrow the keys and take 'er for a spin. One thing I've learned though: NEVER touch the purple lever, no matter how loudly it pleads with you. And the three-pole beaver switch only works if your Pythonic spirit is pure. KK> Keep up the good work guys! If I ever finish grad school I'll KK> help. Thanks! -Barry From barry@digicool.com Wed Nov 22 00:23:14 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 21 Nov 2000 19:23:14 -0500 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck References: <20001121180407.A20735@real-time.com> Message-ID: <14875.4594.972300.212749@anthem.concentric.net> >>>>> "BT" == Bob Tanner writes: BT> I have turned the rcpt to timeout down to 2mins, but I am BT> still constantly running 1000+ message in the qfiles BT> directory. BT> Any ideas on speeding things up? See the FAQ question #2: http://www.list.org/faq.html From claw@kanga.nu Wed Nov 22 00:29:20 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 16:29:20 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Bill Bumgarner of "Tue, 21 Nov 2000 16:04:31 EST." <200011212104.eALL4W327658@bjork.codefab.com> References: <200011212104.eALL4W327658@bjork.codefab.com> Message-ID: <26896.974852960@kanga.nu> On Tue, 21 Nov 2000 16:04:31 -0500 Bill Bumgarner wrote: > The key with WebDAV is that use therein means that you can point > WebDAV enabled Mailman at Zope, IBM's websphere, Apache+mod_dav, > or any of the other WebDAV enabled HTTP servers out there and it > should "just work" (assuming they did the implementation right). > There is nothing proprietary about the solution and choosing it as > a foundation for archival/management of messages/attachments opens > doors (instead of closing doors). For me WebDAV raises concerns centering around authentication and access security. That said, I fail to see either the use or value in having what is essentially (if I understand it correctly) a posting tool (ability to post content to a site) *UNLESS* the archiver is running on a different machine than Mailman (in which case a networked filesystem would seem a safer/lighter approach). -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From lovelace@wayfarer.org Sun Nov 19 14:36:26 2000 From: lovelace@wayfarer.org (Tanner Lovelace) Date: Sun, 19 Nov 2000 09:36:26 -0500 Subject: [Mailman-Developers] [Mailman-Users] Re: Bug in 2.0rc2 adding newlist References: <3A106ECF.1857.594BDCE@localhost> <3A177A37.46F87CDD@nekodojo.org> Message-ID: <3A17E56A.691DD65E@wayfarer.org> Greg Connor wrote: > > Phil Barnett wrote: > > > > Running 2.0rc2. > > > > I was trying to add a list with bin/newlist and got this: > > > ... > > File "/home/mailman/Mailman/Utils.py", line 568, in mkdir > > os.mkdir(dir, mode) > > OSError: [Errno 2] No such file or directory: > > '/home/mailman/archives/private/bin/newlist.mbox' > > > > I got this too. Looks like a buglet in 2.0rc2. However, I noticed that > running > bin/newlist foo > fails, but just running > bin/newlist > and then typing the new listname when prompted, works. When you put the > name on the command line, it looks like it picks up "bin/newlist" as the > name instead. > I believe this has been fixed in 2.0rc3. argv was being used instead of args. -- Tanner Lovelace lovelace@wayfarer.org http://wtl.wayfarer.org/ Cthulu for President. Why settle for the lesser evil? ------------------------------------------------------ Mailman-Users maillist - Mailman-Users@python.org http://www.python.org/mailman/listinfo/mailman-users From claw@kanga.nu Wed Nov 22 01:53:33 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 17:53:33 -0800 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: Message from Bob Tanner of "Tue, 21 Nov 2000 18:04:07 CST." <20001121180407.A20735@real-time.com> References: <20001121180407.A20735@real-time.com> Message-ID: <6080.974858013@kanga.nu> On Tue, 21 Nov 2000 18:04:07 -0600 Bob Tanner wrote: > I have to say I am very disappointed at mailman's performance with > sendmail. Especially qrunner and how it works. This is not a sendmail problem. Questionably it is a Mailman problem in that qrunner as currently architected is not terribly scalable. > I am running sendmail with 10 queues, but it looks like mailman > only understands and utilizes 1 queue. It also looks like qrunner > is single threaded AND multiple instances cannot be running at the > same time. True. > This lends itself to a very slow delivery system. Only if your MTA is configured to make it so. Example: Delivering 100 messages (1K members with 10 RCPT_TOs per message) via qrunner to Exim or Postfix takes just under 2 seconds here. Admittedly this is a fast machine. Additionally the local MTA is configured to not do name lookups on local deliveries, and, of course, I'm running a cacheing name server locally. > What sucks is qrunner just tries to deliver each message one at a > time. If message 5 takes forever on the rcpt to: command (for > example), no other messages deliveries are attempted because > qrunner sits on message 5 until it times out. Translation: You are doing name lookups on messages delivered via SMTP from the localhost. There's not much reason for this. Turn it off. > Will mailman support multiple "qfiles" directories in the future? Not for v2.0. After, yes. No designs are finalised. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From Dan Mick Wed Nov 22 02:14:35 2000 From: Dan Mick (Dan Mick) Date: Tue, 21 Nov 2000 18:14:35 -0800 (PST) Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck Message-ID: <200011220213.SAA03503@utopia.west.sun.com> > > What sucks is qrunner just tries to deliver each message one at a > > time. If message 5 takes forever on the rcpt to: command (for > > example), no other messages deliveries are attempted because > > qrunner sits on message 5 until it times out. > > Translation: You are doing name lookups on messages delivered via > SMTP from the localhost. There's not much reason for this. Turn it > off. Just out of curiosity: anyone know how to do this in sendmail? Is this the 'delay_checks' feature? I'm sure I haven't configured it, but my list isn't big enough that I care about such delays. If I were to want to change it, though, it's not at all clear to me how I would, and since it's become such a refrain....anyone know? From chuqui@plaidworks.com Wed Nov 22 02:01:34 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 18:01:34 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <14875.3450.340146.28315@anthem.concentric.net> References: <14875.1663.422580.420642@anthem.concentric.net> <14875.3450.340146.28315@anthem.concentric.net> Message-ID: At 7:04 PM -0500 11/21/00, Barry A. Warsaw wrote: > One thing I've learned though: >NEVER touch the purple lever, no matter how loudly it pleads with you. Only one thing I can say about this -- if the *lever* is pleading, you need to switch to Decaf. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 02:46:33 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 18:46:33 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <26896.974852960@kanga.nu> References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> Message-ID: At 4:29 PM -0800 11/21/00, J C Lawrence wrote: >For me WebDAV raises concerns centering around authentication and >access security. Authentication is a big bugaboo in general, which Barry and I have hashed around a bit. More on that someday, maybe. > That said, I fail to see either the use or value >in having what is essentially (if I understand it correctly) a >posting tool (ability to post content to a site) *UNLESS* the >archiver is running on a different machine than Mailman (in which >case a networked filesystem would seem a safer/lighter approach). Well.... it comes down to two key terms: The "C" word and the "I" word. Convergence and Integration. There are fewer and fewer sites that "run mailing lists" -- and more and more sites that have mailing lists as part of a larger site/strategy. all of these technologies are converging as people figure out they can be used together to complement each other -- and that convergence is creating a severe need to integrate them to keep them usable (and preferably, easy to use) for the end user. Take my sites as examples --- www.hockeyfanz.com. Apache web server, mwforum for web discussion (nad my list directories), mailman for lists, htdig for search engine (soon), mhonarc for archives, ircu for real time (soon). The forum has a separate authentication from the lists, and the archives have their own password (although I've figured out how to write an apache authentication module as a hack, so I can use mailman info instead -- someday). the forums have a different search engine than the archives will. So to fully use the system, there are two places to register/login, a third password for archives (each list has its own password, remember), and two search engines, depending on whether you want list to search the forum or mail lists. And this is after I've worked my butt off to find stuff that integrates as well as I can. It's about as close to state of the art as possible (IMHO), in terms of the technologies I need. The biggest problem I have? user confusion -- can you blame them? I understand all of these pieces, but why should they? Answer: they shouldn't. I have a site. My user should have a way to register and use it. Which piece registers what way shouldn't matter based on phase of the moon.... So for me, integration and convergence is a Big Issue. I have Ideas On How To Do this, as Barry knows, but nothing we've taken public yet. But this is the next big phase of this stuff. Not making it work, making it work TOGETHER. And as we figure out how to make it work together, we can't throw out the people who just want to run a couple of mail lists. And *that* is the problem with Zope, or WebDAV, or name your favorite integration tool - because the more you depend on that stuff, the harder it is to just use the tool itself. It has to be a floor wax and a dessert topping. Which is very possible. Ca be done, but not easy. But worth doing. All of these things are converging, and to do so succeesfully, we have to integrate. But we need to integrate in ways that are modular with standard interfaces, so that we don't require someone install 200K lines of code to get a stupid mail list running for their bridge club.... So tools like Zope, or WebDAV, or Mhonarc, or even Pipermail are important things to consider, support and perhaps use, but we have to be hugely careful about requiring the use of things, while enabling them to be used. The fewer things you force someone to use, the better off we are: but the more things wee *can* work with, the better off we'll be. (and the other cool feature is that if we avoid reinventing stuff, but instead write interfaces to them, we leverage off a lot more work of a lot more people, because writing an glue interface to mhonarc is a LOT faster and easier than writing Mhonarc. writing a glue interface to HtDig is a LOT faster and easier thanw riting HtDig. And if someone doesn't want Mhonarc or HtDig, they can simply write their own interface to their own tool, and (hopefully) submit it back to the project so we can all benefit....THAT is, IMHO, the strongest argument against integrating stuff like Pipermail into the project itself -- it requires owrk that can go into core functionality instead, and it also discourages people from building or interfacing other technologies, because we've "defined" the default. That's why I think running Pipermail as a separate project, and writing the interface glue so that it's supported by Mailman "out of the box" is a better idea than rnning them as the same project -- they're separate code pieces with a common interface that really odn't have anything in common, and I think it discourages integration of other pieces. There are things that are core functionality for mailman: subscriber services, delivering mail. There are core technologies that Mailman needs, like archives and search engines (if you want to have integrated archives, you have to have search capabilities), but services like that are probably better handled via interface glues. Tehy don't need to ship with the core code, but instead, you ship the interface and instructions on getting it installed and running. it can still be the 'official' or 'default' tools, but they don't need to be IN the distribution, especially if they're things people might or might not use. Actually, *everything* ought to be defined with a formal API that can be replaced with a different plug-in if someone wants to (at least as a long-term goal), but some stuff is appropriate to be *in* the distribution, and some interfaced to by the distribution. And which is which is part of the architecting process. But given that we're no longer in a "alone in the world" environment, we have to stop thinking in terms of doing it all ourselves. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 02:53:16 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 18:53:16 -0800 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: <20001121180407.A20735@real-time.com> References: <20001121180407.A20735@real-time.com> Message-ID: At 6:04 PM -0600 11/21/00, Bob Tanner wrote: >I have to say I am very disappointed at mailman's performance with sendmail. it works pretty well for me. I've got it running a rather large setup (50ish lists, 40,000+ subscribers, 300-400 messages posted a day) without problem. You need to go into your system, and set SMTP_MAX_RCPTS = 10 in your mm_cfg.py. That'll cause mailman to split mail delivery into batches of ten, and parallelize the hell out of your list delivery. The big problem you're seeing isn't qrunner, but that each message is sent out as a single batch to sendmail, so sendmail can't parallelize. that means delivery is as slow as the slowest machines in the queue ahead of you. Very, very bad. But with splits into smaller batches, sendamil can do smart queue management again. >Will mailman support multiple "qfiles" directories in the future? yes. That'll be improved as well down the road, but that's not really your problem here, unless you're trying to run on underpowered hardware. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 02:55:13 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 18:55:13 -0800 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: <200011220213.SAA03503@utopia.west.sun.com> References: <200011220213.SAA03503@utopia.west.sun.com> Message-ID: At 6:14 PM -0800 11/21/00, Dan Mick wrote: > > Translation: You are doing name lookups on messages delivered via > > SMTP from the localhost. There's not much reason for this. Turn it > > off. > >Just out of curiosity: anyone know how to do this in sendmail? someone (I think it was JC...) told me a while back to enable accept_unresolvable_domains as a feature in your .cf. I haven't tested it yet, though. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From bbum@codefab.com Wed Nov 22 04:23:45 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Tue, 21 Nov 2000 23:23:45 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? Message-ID: <200011220423.eAM4Njb16105@s1.optonline.net> Let me describe a bit about what I envisioned when going down the = Mailman+WebDAV implementation path. Maybe it is reasonable, maybe not. I agree with Chuq, Ken and Barry regarding various concerns with = Mailman. That the integration of technologies as a part of the required = baseline makes the tool *less* flexible and *less* easily accesible; = that security is a major concern; and that the future of Mailman should = be about refining/abstracting what we have, not glomming new bits onto = it. With that said, let me discuss some of the issues I faced and some of = the thinking that resulted. This is in no particular order and is not a = criticism of anyone's coding contributions! My code is *not*Not*NOT* = perfect and it all should be reviewed, as well!! - archival of messages is a lot more than just writing the bodies to a = web server and then generating some kind of automatic TOC/index. MIME = based email content is not only a reality, but it is often the default. = Mailman needs to more gracefully handle mime-types; automatic filters = would be one solution [very desirable for things like a pure discussion = list] and automatic decoding/filing/handling of attachments is another = solution [and very desirable for things like project discussion lists, = etc]. - message rewriting is a really powerful concept and should be an = abstract and first class part of the Mailman core processing pipeline. = When I implemented the decode-and-file-to-WebDAV solution, the first = challenge was to build a handler that sat in the delivery pipeline and = did some basic decode-and-rewrite-based-on-mailing-list-configuration as = the message passed through the pipe. Once decoded/rewritten, then = filing/processing/filtering/etc can all be abstracted *away* from the = actual decode/rewrite process. - Python's built in MIME type handling code sucks. It is great if you = want to *compose* MIME content, it sucks if you want to rip apart = existing mime content and have any kind of random access. As such, I = turned to mimecntl.py by . It was quite useful and enabled me to get the job done, but = does not have the most obvious of APIs. Did it improve in 2.0? I = don't think so, but have not looked closely. - the current solution for creating an archive is lacking-- that doesn't = mean it doesn't work. It works and it works very well, but it is *not* = modular, does *not* follow the design patterns of the rest of the tool = and makes it *very* difficult to slap in replacement models for archival = (has this changed since, say, may? If it has, I'll have to retract this = until I look again). Before WebDAV or anything else can be considered, = the actual archival process needs to be abstracted away from the hack = that is in there now. - gatewaying messages to an HTTP server as they are processed is an = extremely powerful addition [and very easy to do]. However, it is = useful to *also* add the ability to automatically requeue messages for = blastting at the HTTP server if the server returns certain error codes. = This is done in the codebase I posted an ftp:// to earlier. This allows = for *extremely* flexible integration of Mailman w/other infrastructure. - WebDAV does *not* address security. Nor does it contribute to = "security issues". Any web based archive of data is only as secure as = the Web server that allows access to the content. Given that the Web is = now the *standard* way of presenting things like Mailing list archives, = this is a security issue mailman faces *regardless* of the use of WebDAV = as the means of storing the archive. WebDAV is merely an extension to = the existing HTTP protocol. However, WebDAV does allow for = interesting potential security implementations; URLs that automatically = time out after X minutes/hours/days are trivial to implement. - for the archival of plain text messages, WebDAV is overkill [as Chuq = mentions]. However, as soon as you move to archiving mime attachments, = it quickly becomes extremely advantageous to archive various properties = with the archived message pieces. Example; if my modified Mailman = receives a message from a Macintosh with an attachment, it stores the = Creator/Type information [and other extended headers] as a set of = properties associated with the Resource on the attachment server. This = allows for... - ....restoring decoded attachments and reencoding back to their = original state with their original headers is an extremely cool feature. = With this [in the modified version of Mailman+WebDAV], it is quite = easy to send a request to Mailman that causes it to reconstitute a = single attachment in all of its original encoded glory with its original = headers based entirely on the decoded information found on the WebDAV = server. This message can then be automatically sent to the user and = will be treated just like the original attachment in the original = message. - if we are to manage the complexity associated with the integration of = numerous technologies, it is only going to happen through well refined = and highly modular APIs.... b.bum From chuqui@plaidworks.com Wed Nov 22 04:52:26 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 20:52:26 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <200011220423.eAM4Njb16105@s1.optonline.net> References: <200011220423.eAM4Njb16105@s1.optonline.net> Message-ID: At 11:23 PM -0500 11/21/00, Bill Bumgarner wrote: >- archival of messages is a lot more than just writing the bodies to >a web server and then generating some kind of automatic TOC/index. agreed completely. I'd take it a step further and say it probably shouldn't generate indexes at all, but that indexes should be generated when a user wants to access the archives, dynamically. That's probably the single major weakness of mhonarc. > MIME based email content is not only a reality, but it is often the default. and since AOL 6.0 for windows no longer has any provision for text-only operation, you now have no choice but to deal with MIME in some way. they no lnoger HAVE the option to turn MIME off. So the server has to figure it out. And IMHO, from my research, we're currently making the switch to style-capable e-mail in the mainstream. A year from now -- lists that don't support it reasonably (and I don't believe "strip it to the text part and throw the rest away" will qualify as "reasonable" -- it's a transition stopgap) wll be looked on negatively. People want enclosures and HTML, because the tools to deal with them are maturing. Looking forward, it's going t be standard, and that means all of the tools need to deal with them properly, and taht includes displaying them, transporting them, and protecting users from them as needed. > Mailman needs to more gracefully handle mime-types; automatic >filters would be one solution [very desirable for things like a pure >discussion list] and automatic decoding/filing/handling of >attachments is another solution [and very desirable for things like >project discussion lists, etc]. agreed, and it's been talked about here. We need to properly support it, plus we also need the ability to filter adn edit it -- to strip .exe files, for instance, to store HTML and graphics on the server to show properly in the archives, to generate what I call TOC-digests (simply a list of pointers into the archives), and other stuff. >- message rewriting is a really powerful concept and should be an >abstract and first class part of the Mailman core processing >pipeline. It's something I'm doing pre-design work on now, in fact. >- the current solution for creating an archive is lacking-- that >doesn't mean it doesn't work. It's very much an older-technology base -- where it's falling behind a rapidly changing technology. A year ago, pipermail was great. Now, it's not. Fortunately, I think we're close to the end of the rapid evolution of e-mail, or at least where we can take a breath, figure out reality and catch up a bit. But it's been really interesting trying to NOTICE all of the changes, much less manage them... > It works and it works very well, but it is *not* modular, does >*not* follow the design patterns of the rest of the tool and makes >it *very* difficult to slap in replacement models for archival (has >this changed since, say, may? A concept I've been sort of beating into the ground, but if there's a weakness in open source software, it tends to be written by programmers used to the "I want this, therefore I shall write it", and interfacing stuff to other pieces of similar stuff is low on the priority list, or missing. And the future of the software world are those damn C and I words I keep using... this is not a criticism, but a recognition that the paradigms are changing. I'm seeing these concepts adopted into open source as well as commercial software (probably faster in much of the open source world now, in fact) -- but the way we wrote stuff a year ago isn't how we'll write stuff in a year. We're seeing what Apple wanted to do with OpenDoc actually starting to happen, IMHO. Pluggable extensibility. Object-oriented systems, not object oriented software. >- for the archival of plain text messages, WebDAV is overkill [as >Chuq mentions]. However, as soon as you move to archiving mime >attachments, it quickly becomes extremely advantageous to archive >various properties with the archived message pieces. but you can do that with a lot less overhead in MySQL by doing a focussed database. In fact, you could program a system to do this via DBI that'd work in any DBI-capable environment, so users could roll their own based on what they've already adopted. unless WebDAV gives us enough extra capabilities to be worthy of the specialization, my argument is (and will be) we program to a more general API (like DBI), so that we work in many environments, and if someone wants, they can program a DBI->WebDAV interface to attach to it. This way, we get DB, MySQL, PostGres, Oracle, ODBC, yadayada more or less for free, giving us functionality across multiple environments that users can tailor. If we program just to WebDAV, we don't get that. So it's choosing what the appropriate interfaces are that's as important as having interfaces. you don't program to a technology unless you have to -- you program to an interface that enables technologies. (image: this is chuqui. this is a dead horse. This is chuqui holding a whip...) >- ....restoring decoded attachments and reencoding back to their >original state with their original headers is an extremely cool >feature. Truly. And if we can support BLOBs in DBI, well, we don't have to write anything to disk and can generate an entire message out of a DBI database -- portable to any decent database. >- if we are to manage the complexity associated with the integration >of numerous technologies, it is only going to happen through well >refined and highly modular APIs.... agreed. and to make ti clear, I'm not arguing against WebDAV. I'm arguing that for something like this, you define the interface and see if you can build it in a way that you don't JUST get WebDAV, but support at a more abstracted level that gets you a range of supported technologies (and future capability for that yet discovered) for an incrementally greater amount of work. the trick is to find the right abstractions and the proper technology layer to attach that to. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From barry@digicool.com Wed Nov 22 05:02:26 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 22 Nov 2000 00:02:26 -0500 Subject: [Mailman-Developers] Towards 2.1 Message-ID: <14875.21346.357211.669599@anthem.concentric.net> There have been lots of great discussion on mailman-developers about the directions Mailman should be taking. This is all very cool stuff (and I want it to continue), but I view it as being more for Mailman 3.0, for which I have no time frame at the moment. Now that 2.0 final is imminent[*], I'd also like to open up some discussion about what we want to see in 2.1. My hard commitment, long overdue, is to integrate Juan Carlos's and Victoriano's I18N work. Juan Carlos has sent me patches, which I intend to start unpacking and looking at, probably after the US Thanksgiving holiday weekend. I'd also like to start using some of the nicer Python 2.0 features to clean up the code in places. Other than that, I'd like to get people's feedback on the really crucial stuff that needs to get fixed for 2.1. I want to keep to a bare minimum anything that requires re-architecting, pushing that into 3.0 after more discussions. I'd like to try to get a 2.1 beta out by the end of the year, with 2.1 final some time in January. I don't know if that's realistic or not. From the TODO list, things I can imagine tackling for 2.1 include: - Plain text digests should conform to RFC 1153. - If a message has MIME parts and the header/footer is going to be added, the message should be transformed into a mulitpart/mixed with the header and footer added as text/plain parts. (note that this will require improving Python's standard MIME message handling. b.bum astutely observes that it basically sucks right now and fixing this dovetails with work i want to do for Python 2.1 as well). - Allow "urgent" postings to all members by the list admin which bypasses normal digest delivery. - A button that will bundled and deliver a digest Right Now. - Allow the list-admin to require approvals for unsubs - Allow the user to be excluded from postings if they're getting them in the to: or cc: headers. Be smarter about filtering out duplicate deliveries. - Allow users to subscribe without selecting a password and have Mailman create a password for them. - Allow the site admin to define list styles or themes, and list admins to choose one of the canned styles to apply to their list. - Full creation, deletion, renaming, etc. of lists through the web (and email?), including fixing aliases file updates. - add_members should have a switch to disable admin notifications - Allow individuals to turn off password reminders - Don't use the first public mailing list as the `originator' of password reminders. - Provide an email interface to all administrative commands - Add -join and -remove addresses for easy subscription, unsubscription - For email subscribes, keep an audit of where requests are coming from, and send the original request headers in the confirmation message. Helps track down subscribe bombs. - Support the `which' command. - archive link should do *something* reasonable before the first message has been posted to the list. We should also do /something/ about qrunner, but again, I want to keep radical changes to a minimum. Stability is crucial, but performance counts too. Maybe Chuq's idea of splitting the queue into 3 parts can be done without much disruption. I don't think /everything/ can be done for 2.1, and maybe other people have different priorities. I'm going to keep this list up-to-date on the Mailman2.1 Wiki: http://www.zope.org/Members/bwarsaw/MailmanDesignNotes/MailmanTwoDotOne Feel free to contribute. -Barry [*] The astute observer will notice the tarball on SourceForge already. I'm waiting for the mirror sites to update, but will announce tomorrow even if they don't. From bbum@codefab.com Wed Nov 22 05:49:46 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Wed, 22 Nov 2000 00:49:46 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011220423.eAM4Njb16105@s1.optonline.net> Message-ID: <3A1B5E7C.6E6E71AD@codefab.com> Chuq Von Rospach wrote: > At 11:23 PM -0500 11/21/00, Bill Bumgarner wrote: > > >- archival of messages is a lot more than just writing the bodies to > >a web server and then generating some kind of automatic TOC/index. > > agreed completely. I'd take it a step further and say it probably > shouldn't generate indexes at all, but that indexes should be > generated when a user wants to access the archives, dynamically. > That's probably the single major weakness of mhonarc. I'd take it a step beyond THAT and say that this is really almost a per-list issue. I have mailing lists that: - are professional in nature and I wouldn't mind even PAYING for a realtime solution that was very user friendly - low bandwidth lists that on demand indexing would not be an issue - high bandwidth free lists that once-a-night indexing would be the ideal. [bunch of useful stuff that I agree with deleted] > > > >- for the archival of plain text messages, WebDAV is overkill [as > >Chuq mentions]. However, as soon as you move to archiving mime > >attachments, it quickly becomes extremely advantageous to archive > >various properties with the archived message pieces. > > but you can do that with a lot less overhead in MySQL by doing a > focussed database. In fact, you could program a system to do this via > DBI that'd work in any DBI-capable environment, so users could roll > their own based on what they've already adopted. unless WebDAV gives > us enough extra capabilities to be worthy of the specialization, my > argument is (and will be) we program to a more general API (like > DBI), so that we work in many environments, and if someone wants, > they can program a DBI->WebDAV interface to attach to it. This way, > we get DB, MySQL, PostGres, Oracle, ODBC, yadayada more or less for > free, giving us functionality across multiple environments that users > can tailor. If we program just to WebDAV, we don't get that. This is where i disagree *very* strongly-- maybe not with the implementation choice [DBI], but with the reasoning behind it. I don't think archival should be treated as a database centric operation. The concept of archival falls very naturally into a static hierarchy of collections/directories containing resoruces/files with a bit of additional meta information associated with some resources. This is exactly the kind of information archive that a web server is *designed* to optimally serve. Adding extra layers here or abstracting to a DBI really doesn't buy us much. Alone, a basic filesystem served webserver gives us: - effecient access to archives - basic per-site, per-list authentication - [with little addition] unified access/passwords between lists, etc... - almost *zero* overhead with *very little* implementation cost WebDAV adds the ability to do advanced locking, easy meta information storage, etc... but-- most importantly-- does not take away the very effecient presentation of data naturally present within a filesystem of stuff served by a web server. As well, a filesystem centric storage/presentation solution-- webdav or raw filesystem-- solves *most* peoples archiving problems *most* of the time. I feel *very strongly* that the archival solution-- whether it be raw messages or decoded messages-- should be centric to storing files in directories and serving files from directories. The second reason I feel strongly that moving to a DBI based interface wouldn't present that much of an advantage is that most people that need to actually store the data in a database are going to have their own requirements surrounding decoding, storage, indexing, and presentation of said database related content. There are few *real* standards in terms of the storage of multimedia [MIME] content into a database environment and, as such, the developer will likely have to rip the data out of whatever our implementation prefers and into their own storage subsystem. In my experience [storing email into a database was actually a problem we had to solve-- this is the implementation we successfully/effectively used], it is far more convenient to provide an HTTP [replaced with a MODULAR in a real implementation] gateway that delivers the processed, but still relatively raw, messages to some other subsystem for subsequent parsing and storage. In our case, we used HTTP to deliver inbound messages to a WebObjects application that parsed the message into EOs [enterprise objects] and persisted those via the various APIs included with WebObjects. Another way of looking at this is that as soon as most developers are going to want to work with the data in the context of a true database, most developers are also going to want to actually use their tool-du-jour [WebObjects, ASP, EJB, PHP, Zope, etc...] to process that information. Taking a two pronged aproach to archival-- we provide a simple [and modular] filesystem-esque approach to store the data in a more traditional manner-- be it directly to the filesystem or via a WebDAV adaptor (since WebDAV is very filesystem like, just w/HTTP as the protocol of choice) and an equally as simple modular gateway that allows the developer/administrator to easily configure the system such that the data is delivered to their server of choice via the protocol of chioice-- will likely reduce the complexity of our implementation and increase the attractiveness in that our codebase is that much simpler and more approachable. This is *not* to say that the DBI approach isn't the right way to go; if a generic DBI->filesystem, DBI->WebDAV, DBI->DB capable API were put together and was relatively hidden from the user and casual developer, it might be a huge win. > > So it's choosing what the appropriate interfaces are that's as > important as having interfaces. you don't program to a technology > unless you have to -- you program to an interface that enables > technologies. (image: this is chuqui. this is a dead horse. This is > chuqui holding a whip...) And bbum following with a club.... :-) Agreed. > > >- ....restoring decoded attachments and reencoding back to their > >original state with their original headers is an extremely cool > >feature. > > Truly. And if we can support BLOBs in DBI, well, we don't have to > write anything to disk and can generate an entire message out of a > DBI database -- portable to any decent database. But an order of magnitude less effecient than downloading the BLOB off of disk via a webserver! Generic access with simple access control is what *most* users/administrators want *most* of the time. More complex/abstracted/portable access is less of a requirement and *a lot* of the people with such requirements also have other issues-- real or imagined-- that dictate that they really just want Mailman to hand the stuff to them as quickly/easily as possible and be done with it. > >- if we are to manage the complexity associated with the integration > >of numerous technologies, it is only going to happen through well > >refined and highly modular APIs.... > > agreed. and to make ti clear, I'm not arguing against WebDAV. I'm > arguing that for something like this, you define the interface and > see if you can build it in a way that you don't JUST get WebDAV, but > support at a more abstracted level that gets you a range of supported > technologies (and future capability for that yet discovered) for an > incrementally greater amount of work. the trick is to find the right > abstractions and the proper technology layer to attach that to. Totally-- and I hope no one thought I was advocating WebDAV as the end all, be all, only solution! I feel strongly that abstraction is key, but that we should also provide decent, production quality, implementations of solutions to the very same set of problems for which we build the gneric abstracted/modularized APIs. If Mailman is not fully functional "out of the box", then people will ignore it. However, if it isn't also flexible enough to be integrated into their weird environments (because every server on the web has weirdness), they'll bitch and moan until they find something else that doesn't solve their problem to B&M about.... b.bum From marouni@earlham.edu Wed Nov 22 05:53:06 2000 From: marouni@earlham.edu (Nick Marouf) Date: Wed, 22 Nov 2000 00:53:06 -0500 Subject: [Mailman-Developers] Towards 2.1 References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: <3A1B5F41.FEE1A62B@earlham.edu> Hi Barry, A fellow student of mine is working on getting full name support for mailman addresses. Is there a way we can get that in with the 2.1 release? I will keep you updated on his progress and send in a patch once that is done. Thanks for all the great work. Nick "Barry A. Warsaw" wrote: > There have been lots of great discussion on mailman-developers about > the directions Mailman should be taking. This is all very cool stuff > (and I want it to continue), but I view it as being more for Mailman > 3.0, for which I have no time frame at the moment. > > Now that 2.0 final is imminent[*], I'd also like to open up some > discussion about what we want to see in 2.1. My hard commitment, long > overdue, is to integrate Juan Carlos's and Victoriano's I18N work. > Juan Carlos has sent me patches, which I intend to start unpacking and > looking at, probably after the US Thanksgiving holiday weekend. I'd > also like to start using some of the nicer Python 2.0 features to > clean up the code in places. > > Other than that, I'd like to get people's feedback on the really > crucial stuff that needs to get fixed for 2.1. I want to keep to a > bare minimum anything that requires re-architecting, pushing that into > 3.0 after more discussions. I'd like to try to get a 2.1 beta out by > the end of the year, with 2.1 final some time in January. I don't > know if that's realistic or not. > > >From the TODO list, things I can imagine tackling for 2.1 include: > > - Plain text digests should conform to RFC 1153. > - If a message has MIME parts and the header/footer is going to be > added, the message should be transformed into a mulitpart/mixed > with the header and footer added as text/plain parts. > > (note that this will require improving Python's standard MIME > message handling. b.bum astutely observes that it > basically sucks right now and fixing this dovetails with work i > want to do for Python 2.1 as well). > > - Allow "urgent" postings to all members by the list admin which > bypasses normal digest delivery. > - A button that will bundled and deliver a digest Right Now. > - Allow the list-admin to require approvals for unsubs > - Allow the user to be excluded from postings if they're getting > them in the to: or cc: headers. Be smarter about filtering out > duplicate deliveries. > - Allow users to subscribe without selecting a password and have > Mailman create a password for them. > - Allow the site admin to define list styles or themes, and list > admins to choose one of the canned styles to apply to their > list. > - Full creation, deletion, renaming, etc. of lists through the web > (and email?), including fixing aliases file updates. > - add_members should have a switch to disable admin notifications > - Allow individuals to turn off password reminders > - Don't use the first public mailing list as the `originator' of > password reminders. > - Provide an email interface to all administrative commands > - Add -join and -remove addresses for easy subscription, > unsubscription > - For email subscribes, keep an audit of where requests are coming > from, and send the original request headers in the confirmation > message. Helps track down subscribe bombs. > - Support the `which' command. > - archive link should do *something* reasonable before the first > message has been posted to the list. > > We should also do /something/ about qrunner, but again, I want to > keep radical changes to a minimum. Stability is crucial, but > performance counts too. Maybe Chuq's idea of splitting the queue > into 3 parts can be done without much disruption. > > I don't think /everything/ can be done for 2.1, and maybe other people > have different priorities. I'm going to keep this list up-to-date on > the Mailman2.1 Wiki: > > http://www.zope.org/Members/bwarsaw/MailmanDesignNotes/MailmanTwoDotOne > > Feel free to contribute. > > -Barry > > [*] The astute observer will notice the tarball on SourceForge > already. I'm waiting for the mirror sites to update, but will > announce tomorrow even if they don't. > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers -- Nicholas Marouf http://www.ramallahonline.com From claw@kanga.nu Wed Nov 22 06:04:14 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:04:14 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 20:52:26 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> Message-ID: <7649.974873054@kanga.nu> On Tue, 21 Nov 2000 20:52:26 -0800 Chuq Von Rospach wrote: > At 11:23 PM -0500 11/21/00, Bill Bumgarner wrote: >> - archival of messages is a lot more than just writing the bodies >> to a web server and then generating some kind of automatic >> TOC/index. > agreed completely. I'd take it a step further and say it probably > shouldn't generate indexes at all, but that indexes should be > generated when a user wants to access the archives, dynamically. > That's probably the single major weakness of mhonarc. I have half a design and some basic code roughed out to have the PHP+MHonArc setup I used on Kanga.Nu instead inject the deconstructed data into an SQL DB with the message presentation code rather than reading from the PHP variable assignements sucking everything out of the DB etc etc etc. It wouldn't be very difficult to do and MHonArc has excellent MIME parsing and handling features to act in support of this. >> MIME based email content is not only a reality, but it is often >> the default. > and since AOL 6.0 for windows no longer has any provision for > text-only operation, you now have no choice but to deal with MIME > in some way. they no lnoger HAVE the option to turn MIME off. So > the server has to figure it out. Ahh. That explains why I've been rejectting so many postings from AOL members on my lists lately (I don't allow HTML as an aid to promoting high signal lists). -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 06:08:33 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 22:08:33 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <3A1B5E7C.6E6E71AD@codefab.com> References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> Message-ID: At 12:49 AM -0500 11/22/00, Bill Bumgarner wrote: >This is where i disagree *very* strongly-- maybe not with the implementation >choice [DBI], but with the reasoning behind it. > >I don't think archival should be treated as a database centric operation. The >concept of archival falls very naturally into a static hierarchy of >collections/directories containing resoruces/files with a bit of >additional meta >information associated with some resources. This is exactly the kind of >information archive that a web server is *designed* to optimally >serve. Adding >extra layers here or abstracting to a DBI really doesn't buy us much. um, except you seem to be defining a database-centric operation using a filesystem as a data store. It's still database-centric. >Alone, a basic filesystem served webserver gives us: > > - effecient access to archives > > - basic per-site, per-list authentication > > - [with little addition] unified access/passwords between lists, etc... > > - almost *zero* overhead with *very little* implementation cost which are all as true with a database based system. Except I don't agree withyou on the overhead and implementation costs on your end. Ask anyone who manages a decent-sized NNTP system -- filesystem centric storage systems don't scale to large data sets well at all. Databases do.... >I feel *very strongly* that the archival solution-- whether it be >raw messages or >decoded messages-- should be centric to storing files in directories >and serving >files from directories. And I disagree, since you're tying to a single technology that works for some cases, but isn't a general solution, and limits other options. For instance, it's fairly easy to implement a lot of searching via DBI. it's a lot harder using a filesystem store. >The second reason I feel strongly that moving to a DBI based >interface wouldn't >present that much of an advantage is that most people that need to >actually store >the data in a database are going to have their own requirements surrounding >decoding, storage, indexing, and presentation of said database >related content. >There are few *real* standards in terms of the storage of multimedia [MIME] >content into a database environment and, as such, the developer will >likely have >to rip the data out of whatever our implementation prefers and into their own >storage subsystem. I don't see this as an issue. You have to define an architecture either way. >This is *not* to say that the DBI approach isn't the right way to go; if a >generic DBI->filesystem, DBI->WebDAV, DBI->DB capable API were put >together and >was relatively hidden from the user and casual developer, it might >be a huge win. which is exactly what I'm arguing. And DBI is a well-known interface that ias easily accessible to anyone who wants to write an interface -- if the architecture is done right. And it's portable off of Unix, which is an issue for Mailman. > > Truly. And if we can support BLOBs in DBI, well, we don't have to >> write anything to disk and can generate an entire message out of a >> DBI database -- portable to any decent database. > >But an order of magnitude less effecient than downloading the BLOB >off of disk via >a webserver! Not necessarily. And is the efficiency important? A lot of time is wasted in computer development optimizing the wrong things.... >Generic access with simple access control is what *most* users/administrators >want *most* of the time. More complex/abstracted/portable access >is less of a >requirement and *a lot* of the people with such requirements also have other >issues-- real or imagined-- that dictate that they really just want Mailman to >hand the stuff to them as quickly/easily as possible and be done with it. that's changing, rapidly. Generic, text-only e-mail was what most users/administrators wanted a year ago, too. If you implement to what they WANTED, by the time it's written, they won't want it any more. You have to figure out what's possible and what they're going to want when you're done writing it. you don't want to write obsolete stuff because you implement what they wanted last time. >I feel strongly that abstraction is key, but that we should also >provide decent, >production quality, implementations of solutions to the very same >set of problems >for which we build the gneric abstracted/modularized APIs. agreed. >If Mailman is not fully functional "out of the box", then people >will ignore it. >However, if it isn't also flexible enough to be integrated into their weird >environments (because every server on the web has weirdness), >they'll bitch and >moan until they find something else that doesn't solve their problem to B&M >about.... it has to be functional out of the box, but we have to make sure we define "functional" properly, too. You can't be missing features but doing the "kitchen sink" think just in case is wrong, too. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From tanner@real-time.com Wed Nov 22 06:13:14 2000 From: tanner@real-time.com (Bob Tanner) Date: Wed, 22 Nov 2000 00:13:14 -0600 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: ; from chuqui@plaidworks.com on Tue, Nov 21, 2000 at 06:53:16PM -0800 References: <20001121180407.A20735@real-time.com> Message-ID: <20001122001314.B3147@real-time.com> Quoting Chuq Von Rospach (chuqui@plaidworks.com): > You need to go into your system, and set > > SMTP_MAX_RCPTS = 10 I had it down to 50, but I'll give 10 a try. Thanks. I think this should be an FAQ. FAQ #2 states to turn off synchronous DNS resolution. Anyone know how to do this via sendmail's .mc files? -- Bob Tanner | Phone : (612)943-8700 http://www.mn-linux.org | Fax : (612)943-8500 Key fingerprint = 6C E9 51 4F D5 3E 4C 66 62 A9 10 E5 35 85 39 D9 From chuqui@plaidworks.com Wed Nov 22 06:13:12 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 22:13:12 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <7649.974873054@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> Message-ID: >At 10:04 PM -0800 11/21/00, J C Lawrence wrote: >> and since AOL 6.0 for windows no longer has any provision for >> text-only operation, you now have no choice but to deal with MIME >> in some way. they no lnoger HAVE the option to turn MIME off. So >> the server has to figure it out. > >Ahh. That explains why I've been rejectting so many postings from >AOL members on my lists lately (I don't allow HTML as an aid to >promoting high signal lists). yup. and you can't turn it off. There's no option. You HAVE to deal with multipart-alternative and pull the right pieces out, or their dead in the water. Any list that can't handle MIME e-mail is incompatible with AOL 6 for windows now... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From claw@kanga.nu Wed Nov 22 06:31:44 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:31:44 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Bill Bumgarner of "Wed, 22 Nov 2000 00:49:46 EST." <3A1B5E7C.6E6E71AD@codefab.com> References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> Message-ID: <11426.974874704@kanga.nu> On Wed, 22 Nov 2000 00:49:46 -0500 Bill Bumgarner wrote: > I don't think archival should be treated as a database centric > operation. The concept of archival falls very naturally into a > static hierarchy of collections/directories containing > resoruces/files with a bit of additional meta information > associated with some resources. This is exactly the kind of > information archive that a web server is *designed* to optimally > serve. Adding extra layers here or abstracting to a DBI really > doesn't buy us much. While true in general, this ignores the benefits and ease of dynamically generated reports on archives. Now, perhaps I should talk a little about what I'm spending my copious spare time on lately (other than attempting to chase heisenbugs that crash machines for unknowable reasons). Among other things I'm attempting to meld the basic concepts of WikiWiki, mail archives, and advogato/slashdot-style commentary systems into a form where the basic tenets of Wiki noun definition are retained such that every individual comment and archive page has a dynamically generated/assigned WikiName and there is a moderately sound karma/trust net running as a meta layer atop this as an abstracted report layer. The fact that further abstractions can be made, and that rule-based reports/abstractions can be created as dynamically filled out WikiPages on the other side of this is just, well, really nice. I realise that's pretty turgid. Without spending considerable time and verbiage I don't want to spend right now I'm not sure I can do better. If you're interested in details, and I don't mean just casually, please ask away. But, getting back to the point, to do this properly (ie so that its scalable and reasonably system efficient), especially given the fact that any given data item (be it an archived message, a comment, a report, or whatever) has at least presentation modes each of which has very different surrounding capabilities, the apparancy is that I'm going to have to shove everything into a set of SQL tables. The current half-implementation is a mix of SQL and static files (mostly wrapped around my current PHPLib template based PHP4+MHonArc based archiving setup). > WebDAV adds the ability to do advanced locking, easy meta > information storage, etc... but-- most importantly-- does not take > away the very effecient presentation of data naturally present > within a filesystem of stuff served by a web server. I'm actually considering WebDav slightly longer term as a potential Wiki-ish posting method. I'm not at all convinced its a good way to go, but I'm willing to look into it. > As well, a filesystem centric storage/presentation solution-- > webdav or raw filesystem-- solves *most* peoples archiving > problems *most* of the time. Very true. > I feel *very strongly* that the archival solution-- whether it be > raw messages or decoded messages-- should be centric to storing > files in directories and serving files from directories. In the general case, I agree. > This is *not* to say that the DBI approach isn't the right way to > go; if a generic DBI->filesystem, DBI->WebDAV, DBI->DB capable API > were put together and was relatively hidden from the user and > casual developer, it might be a huge win. I'd like to see something trivial: The archiver deconstructs messages into a stasndardised and well documented templatised form on to the file system. A standard CGI-like toolset is then provided to reconstruct those fragments back into a message (please resend this message to me, I didn't get it the first time), into an HTML page (web archives), etc. This is done by having standard small scripts which do >whatever< by assembling simple call sequences to the toolset. End users may redefine this by re-writing said scripts, acessing the deconstructions or reconstructions as they wise, to then do anything they want with the resulting archives. This could be SQL inserts, report generation, or black voodoo, it really doesn't matter, and more importantly, Mailman, and Mailman's archiver doesn't need to know about it, only the user does. The other nice thing is that we don't do it by defining APIs (which then reuires programmers to do useful things with), but by defining small useful atomic tools that may be assembled/ordered in interesting fashions wihtou having to subject the users to Python, or programming much beyond writing shell scripts and editing canned templates. > But an order of magnitude less effecient than downloading the BLOB > off of disk via a webserver! Greenspun > Generic access with simple access control is what *most* > users/administrators want *most* of the time. More > complex/abstracted/portable access is less of a requirement and *a > lot* of the people with such requirements also have other issues-- > real or imagined-- that dictate that they really just want Mailman > to hand the stuff to them as quickly/easily as possible and be > done with it. Precisely. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Wed Nov 22 06:39:21 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:39:21 -0800 Subject: [Mailman-Developers] Mailman with sendmail = performance bottleneck In-Reply-To: Message from Bob Tanner of "Wed, 22 Nov 2000 00:13:14 CST." <20001122001314.B3147@real-time.com> References: <20001121180407.A20735@real-time.com> <20001122001314.B3147@real-time.com> Message-ID: <12681.974875161@kanga.nu> On Wed, 22 Nov 2000 00:13:14 -0600 Bob Tanner wrote: > Quoting Chuq Von Rospach (chuqui@plaidworks.com): >> You need to go into your system, and set >> >> SMTP_MAX_RCPTS = 10 > I had it down to 50, but I'll give 10 a try. Thanks. FWLIW I usually use either 5 or 10. Depending on your MTA (Historically I've been using Exim, recently moved to Postfix for non-MTA related reasons) and the distribution of your membership lists across domains this can significantly speed queue emptying speed. BTW: I haven't checked and I don't have anything in my spools to check. Does Mailman currently pre-sort the RCPT_TO list by domain? BTW I recently wrote the following in another forum on Postix and Exim: ---- I've spent the last week or so playing with both fairly extensively. They're sweet systems. More interestingly the performance curves for both are remarkably similar once your mail loads become large (ie your spool is always active (mail to deliver)). The different mostly lies in the curve up to continual activity, and the curve as they approach saturation (mail is arriving more quickly than they can get rid of it on a near continual basis). Loosely, Postfix' initial curve is very steep, but they both reach similar delivery rates for the continually active point. I've had some problems simulating sustained saturation (test networks et al), but Exim seems to be friendlier to the localhost in such circumstances. Exim appears to implicitly assume that such queue congestion is temporary and that given enough time things will slow down enough so it can empty the queue. As such its less interested in "get that damned mail out of here now" than it is in, "be nice to the local machine and get the mail out when you can". There are obvious problems with this if your loads stay at or above capacity for extended periods, but equally, its rather nice from an admin perspective for the local machine. Postfix conversely seems to operate on the "get that mail out of here now", and can be rather brutal on the local host during that process. Postfix under sustained saturation makes for a very heavily loaded system. Depending on what else you are doing there, this may be a problem, or not. It is this nature however which gives it the steep attack on delivery rates as versus Exim. The fact that their sustained delivery rates are so similar at the end of that curve is the really interesting bit however. ---- Unfortunatly I didn't have time to keep careful metrics, and my process wasn't really rigorous enough for that either a I was just trying to get some basic parameters on their behaviour. > I think this should be an FAQ. > FAQ #2 states to turn off synchronous DNS resolution. Anyone know > how to do this via sendmail's .mc files? Sorry, no idea. I gave up sendmail 5 years ago. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Wed Nov 22 06:41:56 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:41:56 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 22:13:12 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> Message-ID: <13342.974875316@kanga.nu> On Tue, 21 Nov 2000 22:13:12 -0800 Chuq Von Rospach wrote: >> At 10:04 PM -0800 11/21/00, J C Lawrence wrote: >> Ahh. That explains why I've been rejecting so many postings from >> AOL members on my lists lately (I don't allow HTML as an aid to >> promoting high signal lists). > yup. and you can't turn it off. There's no option. You HAVE to > deal with multipart-alternative and pull the right pieces out, or > their dead in the water. Any list that can't handle MIME e-mail is > incompatible with AOL 6 for windows now... Sad. I already don't allow posts from services that append advertising to their messages (eg Hotmail, Yahoo, etc). I guess I'll have to explicitly add AOL to that list. Annoying. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 06:42:02 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 22:42:02 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <11426.974874704@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> <11426.974874704@kanga.nu> Message-ID: At 10:31 PM -0800 11/21/00, J C Lawrence wrote: > >I'd like to see something trivial: > > The archiver deconstructs messages into a stasndardised and well > documented templatised form on to the file system. XML. Think XML. then it can be repurposed by using different definitions without having to convert from form to form based on context. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 06:43:40 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 22:43:40 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <13342.974875316@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> Message-ID: > >Sad. I already don't allow posts from services that append >advertising to their messages (eg Hotmail, Yahoo, etc). I guess >I'll have to explicitly add AOL to that list. Annoying. or strip to the pieces you find acceptable, using something like demime. Which I hope to be able to get into 2.1 in some form.... Rather than reject -- adapt. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From claw@kanga.nu Wed Nov 22 06:47:46 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:47:46 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 22:42:02 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> <11426.974874704@kanga.nu> Message-ID: <14150.974875666@kanga.nu> On Tue, 21 Nov 2000 22:42:02 -0800 Chuq Von Rospach wrote: > At 10:31 PM -0800 11/21/00, J C Lawrence wrote: >> The archiver deconstructs messages into a stasndardised and well >> documented templatised form on to the file system. > XML. Think XML. then it can be repurposed by using different > definitions without having to convert from form to form based on > context. If you bug my offices or meeting rooms one more time I am going have to come on over there and shove those damned microphones right up... <> -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Wed Nov 22 06:53:10 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:53:10 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 18:46:33 PST." References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> Message-ID: <14763.974875990@kanga.nu> On Tue, 21 Nov 2000 18:46:33 -0800 Chuq Von Rospach wrote: > At 4:29 PM -0800 11/21/00, J C Lawrence wrote: >> For me WebDAV raises concerns centering around authentication and >> access security. > Authentication is a big bugaboo in general, which Barry and I have > hashed around a bit. More on that someday, maybe. FWLIW I see authentication visavis Mailman as a two level problem: list activities (command confirmations) access control The former can be handled with ad hod dynamically generated tokens much as subscribe confirms are handled now. I've posted some notes on good implementations on this previously (I liked the bit about auto-genning an URL that did the command confirmation). The latter just needs to be abstracted to a small script which accepts two command line parameters: UserID and Password. The user can then replace that script with anything he pleases, thus authenticating agsinst he pleases be it SQL, LDAP, or lunar weather sensors. Ditto BTW holds tru for handling membership lists: just have a tool which when run returns the list of members. Simple command line options then spec returning account details, configs, etc. A little over head for text parsing, but not a whole lot (ObNote XML is a reasonable communications format). Simple, easy to extrapolate, nice efficient piped IO, etc. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Wed Nov 22 06:58:22 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 22:58:22 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 22:43:40 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> Message-ID: <15324.974876302@kanga.nu> On Tue, 21 Nov 2000 22:43:40 -0800 Chuq Von Rospach wrote: >> Sad. I already don't allow posts from services that append >> advertising to their messages (eg Hotmail, Yahoo, etc). I guess >> I'll have to explicitly add AOL to that list. Annoying. > or strip to the pieces you find acceptable, using something like > demime. Which I hope to be able to get into 2.1 in some form.... > Rather than reject -- adapt. I've spent considerable time thinking about this and believe it is to the lists' advantage in this case to hold firm. My core promise is signal. In the final analysis it is the only promise I make and keep for the lists: there will be signal, always. To achieve that I need fairly repsectable norming forces. To achieve that rites of participation, minor barriers to entry, identifying characteristics, etc are all cultural assistants to establishing what is in essence a club (as in professional or gentlemans) atmosphere and culture. Avoiding HTML, or in this case AOL, would seem a minor and reasonable fillip on this. Aside: Less than 5% of my members are from AOL. Less than 1% of my semi-regular posters are from AOL. The impact here (for me) is quite small. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 07:03:10 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 23:03:10 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <14763.974875990@kanga.nu> References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> Message-ID: At 10:53 PM -0800 11/21/00, J C Lawrence wrote: > > Authentication is a big bugaboo in general, which Barry and I have >> hashed around a bit. More on that someday, maybe. > >FWLIW I see authentication visavis Mailman as a two level problem: > > list activities (command confirmations) > access control Okay, I hope Barry doesn't mind, but since the talk has started, here's something I sent him a while back that we've sat on to get 2.0 out the door. But it's a first cut at my idea where some of this ought to go... When you see this, you'll start to really understand where I'm coming from in this whole API/module thing -- everything in the system is a module of some sort, that communicates via an API -- which means that in theory, aynone can replace any piece of functionality by simply replacing the module. It also means that each module can be tracked and developed separately, since all it needs to worry about is conforming to the published API. The trick is going to be architecting the modules, interfaces and APIs, and deciding one which ones are central to mailman, which ones peripheral and which ones are optional. But when we're done, it doesn't matter if you use DB or a filesystem database model, as long as the glue inteface exists. And it doesn't matter if you use the current mailman subscription interface or download your data from a corporate database running Oracle and throw out the entire subscription system -- because Mailman no longer cares. So the whole archiving issue is a subset of a larger issue, which is defining and standardizing all of Mailman and breaking it up into pieces that meld into a cohesive whole. --- Date: Wed, 11 Oct 2000 23:30:28 -0700 Subject: Re: Planning Mailman 2.1 At 12:15 AM -0400 10/12/00, Barry A. Warsaw wrote: > > CVR> Actually, I want to talk to you about this -- I have an idea > CVR> that you'll love, or hate, and no middle ground. It'd be a > CVR> "mailman 3.0" thing, and it'll turn it on its ear. But the > CVR> possibilities are stupendous. I just haven't brought it up > CVR> because I know you've been busy. > >(barry's comments deleted...) Okay, here's the other rock. What I want to propose is a new major project -- an open, extensible membership database system. This would completely replace the existing mailman subscriber system. Here's what it is -- it's a web-based (and, perhaps, optional e-mail) interface to a member database. That database would manage and authenticate members for a given environment (site, subset of a site, whatever we plug into it). Front end is web. Back end is DBI, and I'd want to do initial support with MySQL, but in theory, want to allow it to use and DBI-compatbile system). So far, no biggie. But here's where it gets different. The membership database system uses a plug-in architecture of some sort and defined APIs to allow you to write an access module for anything that needs a membership database. That will allow you to define a tab on the membership page to manage whatever it is you need managed, by using the common pool of data (user name, email addr, etc, etc), plus keeping custom data for each module. End result, you go to a membership page, register, and once registered, can access any system configured in -- whether it's mailman, user forums, a personal home page, web logs, whatever. For the user, there's a single consistent interface, single account, single password, and no confusion. for the project (like mailman), it lets the membership database worry about that stuff and focus on doing what it does best -- delivering mail. I've come up with this while trying to put together a multi-system community setup. lists, web boards, archives, real time stuff. And doing research into communities about other things people might want, whether it's public member directory pages, archive access controls, even things like a community calendar. Trying to integrate everything is, um, a bitch. Everyone wants to control their own data, when lots of that data is in common. So, I thought, build a centralized system for managing that common data, and allow it to be customized by compatible systems that interface to it. for mailman, you rip out all of the user and admin stuff, and instead write an access module that defines what mailman needs for subscriptions, how to query mailman for info (like what lists to manage), and how to get subscriber lists so you can deliver to them. A second module handles defining admin access, because if we do this right, those are simply tabs on the member pages as well. It's conceptually very simple -- but implementing it's going to get gnarly. it's something that will require a lot of architecting (basically, we're going to need an interface like Apache's DSO), and to do ti right, we'll have to define a number of services to implement with it to make sure the API works. MLM is one, public member directories is another, and web forums is an obvious third (whether we evangelize some technology in or go try to build one we'll have to see). Oh, and it needs to be language independent, although it would be written in Python. Sound interesting? kinda like a membership ZOPE. I think simply the idea of a system that allows people to stop building yet-another-member-database is a big win, but if we can start integrating stuff as well, it gives sites real opportunity for synergies down the road. It needs a lot of fleshing out, and it's not a short term project, which is why I think it's something to do parallel with the Mailman 3 timeframe. It's something I'm more than happy to spearhead and architect, but at the same time, it needs some serious partners and backing. It's something that if you toss it onto freshmeat and wait for people to find it, it'll die. But if we can find a few key technologies to work with it and make it happen and support it, could really go places. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 07:09:55 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 23:09:55 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <15324.974876302@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> Message-ID: At 10:58 PM -0800 11/21/00, J C Lawrence wrote: >I've spent considerable time thinking about this and believe it is >to the lists' advantage in this case to hold firm. you'll lose. AOL is the first to make this mandatory for their users, but users want these features. And besides, AOL is the 800 pound gorilla, and it's eating its lunch on your fence. Do you really think they're going to change back before your fence collapses under it? AOL is too large to tell to go to hell if you're running general purpose lists. If you have a specialist audience, maybe -- but on my lists, AOL is generally 12-30% of my membership. That's a lot of cutting off of noses to spite the face. >Aside: Less than 5% of my members are from AOL. Less than 1% of my >semi-regular posters are from AOL. The impact here (for me) is >quite small. lucky you. you're not typical then, not given the size of AOL. But -- it's a losing battle, JC. AOL is the first. It won't be the only. My numbers show me that not only do ~90% of mail users have support for MIME email (stylized text, html, etc.), but most of them want to use it. And they don't like having to go through gyrations to avoid it -- and most have no clue how, and don't want to. So I think it's important that the server end deal with this rationally, because otherwise, you're fighting a constant battle of teaching users to placate the server, one user at a time. instead of teaching the server to deal with the users once. It turns into either a brick wall for users to bang their heads against and then give up and leave you -- or a huge support battle for you. The war's over. there's a huge difference between keeping your web site accessible to Lynx users (a good thing!) and demanding users use lynx to access your site (a bad thing!). And now, not dealing with this rationally on the server is like telling everyone to use Lynx to read your web pages. It works if you have the right niche audience and/or have content they can't live without. But few people are in that kind of sellers market.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From claw@kanga.nu Wed Nov 22 07:22:06 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 23:22:06 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 23:09:55 PST." References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> Message-ID: <18729.974877726@kanga.nu> On Tue, 21 Nov 2000 23:09:55 -0800 Chuq Von Rospach wrote: > At 10:58 PM -0800 11/21/00, J C Lawrence wrote: >> I've spent considerable time thinking about this and believe it >> is to the lists' advantage in this case to hold firm. > you'll lose. AOL is the first to make this mandatory for their > users, but users want these features. And besides, AOL is the 800 > pound gorilla, and it's eating its lunch on your fence. Do you > really think they're going to change back before your fence > collapses under it? No, but I also realise that AOL actually is't sitting on my fence. They are sitting on other people's fence to be sure but their presence in my neighborhood is small enough to be noise. >> Aside: Less than 5% of my members are from AOL. Less than 1% of >> my semi-regular posters are from AOL. The impact here (for me) >> is quite small. > lucky you. you're not typical then, not given the size of AOL. Precisely. > But -- it's a losing battle, JC. AOL is the first. It won't be the > only. My numbers show me that not only do ~90% of mail users have > support for MIME email (stylized text, html, etc.), but most of > them want to use it. And they don't like having to go through > gyrations to avoid it -- and most have no clue how, and don't want > to. While I agree (with some distaste), those gyrations have shown themselves to have positive value for me. FWIW I also enforce quoting style (new text below quote), quoting formats (must have leading quote characters on quoted lines), correct attributions for all quoted text, etc, which are leading directly against AOL, MSN et al, but which have also helped define the value I offer. It a per-case specific question, not suitable for the general case. > t few people are in that kind of sellers market.... Too true. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From bbum@codefab.com Wed Nov 22 07:23:28 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Wed, 22 Nov 2000 02:23:28 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> Message-ID: <3A1B7471.5F16AAFA@codefab.com> Chuq Von Rospach wrote: > >Alone, a basic filesystem served webserver gives us: > > > > - effecient access to archives > > > > - basic per-site, per-list authentication > > > > - [with little addition] unified access/passwords between lists, etc... > > > > - almost *zero* overhead with *very little* implementation cost > > which are all as true with a database based system. Except I don't > agree withyou on the overhead and implementation costs on your end. > Ask anyone who manages a decent-sized NNTP system -- filesystem > centric storage systems don't scale to large data sets well at all. > Databases do.... Yes-- all as true with a database system... but require the addition of something other than just a vanilla HTTP server to implement the "get the data" part. Everything on the list above save for the "with little addition" item can be done w/an out of the box apache. Obviously, this does not include the administrative part-- the piece that does per list/per-site configuration requires some additional work regardless of a database or filesystem (or WebDAV) backing store. Another advantage to a filesystem based archival arrangement is that it is *extremely* easy to write a random shell script or two that prunes data from crontab, rebuilds indices, moves stuff around, reformats things, archives off to archived archives, etc... Yes-- of course-- all of these operations can be done with a database backing store, as well, but it is significantly more difficult to develop such tools and install them into the system. Likely, this is somewhat of a perception issue, but most administrators will not hesitate to toss together a shell script to manage an archive of stuff, but will think twice before diddling a database. Very large and very expensive databases scale very well-- MySQL does not. I agree that for truly huge, high traffic sites, moving away from a pure filesystem approach-- moving everything into, say, an ES10000 running one of the magawhompus Oracle/Sybase license-- would be the way to go. But I don't think that is what 90% of the Mailman users are going to be using the system for and requiring-- or even encouraging the use of-- a database as a backing store for messages will not add value to those people. Considering most of the usage of an archive of messages.... - write operations are infrequent, modifications pretty much non existant - retrieval tends to be extremely sporadic and is generally *not* evenly divided across the archive-- a relative few messages receive most of the hits - there are extremely limited ways of viewing the data; by author, date, thread, subject.... with MOST views focused on thread. - indices are periodically updated ... I still believe that a webserver-reading-files-from-filesystem is going to be loads more effecient than a webserver-reading-data-from-client-server-connection-to-app-adaptor-reading-data-from-client-server-connection-to-multiuser-database. It is the same reason why publishing systems don't often put the images in the database [instead, dropping URLs or other symbolic references into the database]-- modern operating systems [Linux, FreeBSD, OSX, Solaris] are very good about caching data off of the disk in memory until that memory is needed for something else. A webserver sitting on such a system will typically map anything the client asks for into memory and not reread it off the disk until something else bumps it out of memory. Even if the web server is cycling children, the underlying filesystem cache isn't going to push mapped files out of memory unless that memory is needed for something else. Certainly, solution providers like Oracle and Sybase rolled their own webservers that connect relativley directly to the database. I don't htink very many Mailman installations woould use this and, if they do, the cost of adding a generic "accept an http msg w/an XML body describing an email and I'll deal w/archiving it in the fashion most appropriate to my implementation needs" is minor compared to the other costs likely inherent to such a project. > > >I feel *very strongly* that the archival solution-- whether it be > >raw messages or > >decoded messages-- should be centric to storing files in directories > >and serving > >files from directories. > > And I disagree, since you're tying to a single technology that works > for some cases, but isn't a general solution, and limits other > options. For instance, it's fairly easy to implement a lot of > searching via DBI. it's a lot harder using a filesystem store. Yes-- if you are doing textual searches *directly* against the filesystem, filesystems are a lose. But a database is not a whole lot better unless you blow the relatively major [compared to most Mailman deployment budgets] $$$ on something like Sybase's or Oracle's free text searching solutions. Regardless of where the data is stored, searches would typically have to be backed by some kind of a indexing store-- be it in a dabase, a btree type solution [Faircom's C-Tree comes to mind], or any of the myriad of "search an index" solutions available. Building that index from a database or from a filesystem isn't really that much different in terms of difficulty though a number of folks would find the problem of walking a filesystem more approachable than walking a database. > >This is *not* to say that the DBI approach isn't the right way to go; if a > >generic DBI->filesystem, DBI->WebDAV, DBI->DB capable API were put > >together and > >was relatively hidden from the user and casual developer, it might > >be a huge win. > > which is exactly what I'm arguing. And DBI is a well-known interface > that ias easily accessible to anyone who wants to write an interface > -- if the architecture is done right. And it's portable off of Unix, > which is an issue for Mailman. I think we agree on the architectural direction of Mailman, but disagree on the relative merits of different backing stores. As such, the above discussion is very interesting, but largely moot. Internally, it makes a lot of sense to use DBI as the abstraction layer between Mailman and the thing that persists BLOBs, messages, and meta information. The underlying implementation is mostly irelevant. With that said, I do feel strongly that an abstraction layer does little good without concrete implementations of the adaptors underneath. As such, I volunteer myself to write the adaptor to WebDAV and I'll tentatively volunteer Chuq to write the adaptor that speaks to a database backing store. Our fierce sense of technical pride and intellectual competition will guarantee that Mailman version X.X will ship with first class ipmlementations of both adaptors. :-) > > > > Truly. And if we can support BLOBs in DBI, well, we don't have to > >> write anything to disk and can generate an entire message out of a > >> DBI database -- portable to any decent database. > > > >But an order of magnitude less effecient than downloading the BLOB > >off of disk via > >a webserver! > > Not necessarily. And is the efficiency important? A lot of time is > wasted in computer development optimizing the wrong things.... Agreed. Never optimize until you know where performance is going south... > >I feel strongly that abstraction is key, but that we should also > >provide decent, > >production quality, implementations of solutions to the very same > >set of problems > >for which we build the gneric abstracted/modularized APIs. > > agreed. > > >If Mailman is not fully functional "out of the box", then people > >will ignore it. > >However, if it isn't also flexible enough to be integrated into their weird > >environments (because every server on the web has weirdness), > >they'll bitch and > >moan until they find something else that doesn't solve their problem to B&M > >about.... > > it has to be functional out of the box, but we have to make sure we > define "functional" properly, too. You can't be missing features but > doing the "kitchen sink" think just in case is wrong, too. Agreed. b.bum From bbum@codefab.com Wed Nov 22 07:26:26 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Wed, 22 Nov 2000 02:26:26 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> Message-ID: <3A1B7523.1C3F9AFA@codefab.com> Yum. This would have saved us weeksandweeksandweeks of development on the system we built that had Mailman as a critical component. Anything that is *using* Mailman as just-another-moving-part-among-many faces the memebership management problem and it is nasty,nasty,nasty. b.bum Chuq Von Rospach wrote: > At 10:53 PM -0800 11/21/00, J C Lawrence wrote: > > > Authentication is a big bugaboo in general, which Barry and I have > >> hashed around a bit. More on that someday, maybe. > > > >FWLIW I see authentication visavis Mailman as a two level problem: > > > > list activities (command confirmations) > > access control > > Okay, I hope Barry doesn't mind, but since the talk has started, > here's something I sent him a while back that we've sat on to get 2.0 > out the door. But it's a first cut at my idea where some of this > ought to go... When you see this, you'll start to really understand > where I'm coming from in this whole API/module thing -- everything in > the system is a module of some sort, that communicates via an API -- > which means that in theory, aynone can replace any piece of > functionality by simply replacing the module. It also means that each > module can be tracked and developed separately, since all it needs to > worry about is conforming to the published API. > > The trick is going to be architecting the modules, interfaces and > APIs, and deciding one which ones are central to mailman, which ones > peripheral and which ones are optional. But when we're done, it > doesn't matter if you use DB or a filesystem database model, as long > as the glue inteface exists. And it doesn't matter if you use the > current mailman subscription interface or download your data from a > corporate database running Oracle and throw out the entire > subscription system -- because Mailman no longer cares. So the whole > archiving issue is a subset of a larger issue, which is defining and > standardizing all of Mailman and breaking it up into pieces that meld > into a cohesive whole. > > --- > Date: Wed, 11 Oct 2000 23:30:28 -0700 > Subject: Re: Planning Mailman 2.1 > .... system description deleted .... From bbum@codefab.com Wed Nov 22 07:30:30 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Wed, 22 Nov 2000 02:30:30 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> Message-ID: <3A1B7617.2836857C@codefab.com> Agreed. I would much rather see Mailman have an option to configure a list such that it is text-only-bounce-mime-straight-to-/dev/null for those situations that warrant it (and to remain New Communist Community Compliant). Similarly, I would also like to see a per-user option that would allow a mailing list subscriber to say "try and filter anything to ASCII and never, ever send me an attachment". b.bum Chuq Von Rospach wrote: > At 10:58 PM -0800 11/21/00, J C Lawrence wrote: > > >I've spent considerable time thinking about this and believe it is > >to the lists' advantage in this case to hold firm. > > you'll lose. AOL is the first to make this mandatory for their users, > but users want these features. And besides, AOL is the 800 pound > gorilla, and it's eating its lunch on your fence. Do you really think > they're going to change back before your fence collapses under it? > ..... war is over, etc [it's in ther archive... read it, it is good] .... From claw@kanga.nu Wed Nov 22 07:36:19 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 21 Nov 2000 23:36:19 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Bill Bumgarner of "Wed, 22 Nov 2000 02:30:30 EST." <3A1B7617.2836857C@codefab.com> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> <3A1B7617.2836857C@codefab.com> Message-ID: <20380.974878579@kanga.nu> On Wed, 22 Nov 2000 02:30:30 -0500 Bill Bumgarner wrote: > Agreed. I would much rather see Mailman have an option to > configure a list such that it is > text-only-bounce-mime-straight-to-/dev/null for those situations > that warrant it (and to remain New Communist Community Compliant). Mailman needs the ability to define basic filters/rules for what is acceptable for being considered as a list posting. Sometimes that's MIME. Sometimes it is the source domain. Sometimes its an LDAP tag relted to a header field, or something else entirely. > Similarly, I would also like to see a per-user option that would > allow a mailing list subscriber to say "try and filter anything to > ASCII and never, ever send me an attachment". While pursueing the LServe game we need to not lose sight of the little guys running a couple lists for their club of 20-some model train enthusiasts. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 07:54:04 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 23:54:04 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <3A1B7471.5F16AAFA@codefab.com> References: <200011220423.eAM4Njb16105@s1.optonline.net> <3A1B5E7C.6E6E71AD@codefab.com> <3A1B7471.5F16AAFA@codefab.com> Message-ID: At 2:23 AM -0500 11/22/00, Bill Bumgarner wrote: >Yes-- all as true with a database system... but require the addition >of something >other than just a vanilla HTTP server to implement the "get the data" part. So does running mailman. So does running a search engine on the archives. We've already committed to adding a bunch of stuff to vanilla HTTP server (heck, what started this whole discussion, WebDAV,is adding something ot a vanilla HTTP server This is a strawman -- we can't do anything close to what we need with a vanilla HTTP server, so putting down one thing because it doesn't meet that goal is wrong. NOTHING useful meets that goal) >Everything on the list above save for the "with little addition" >item can be done >w/an out of the box apache. um, arguable. > > Obviously, this does not include the administrative >part-- the piece that does per list/per-site configuration requires some >additional work regardless of a database or filesystem (or WebDAV) >backing store. oh -- everyting, well, except, um... (giggle) >Another advantage to a filesystem based archival arrangement is that it is >*extremely* easy to write a random shell script or two that prunes data from >crontab, rebuilds indices, moves stuff around, reformats things, >archives off to >archived archives, etc... have you ever worked with NNTP? Because you're reinventing something the NNTP people have spent years designing out of their systems, because it has horrible scaling capabilities and is horrible resource inefficient. Some of us were arguing that this was a bad design model over a decade ago, and it's been proven by NNTP very nicely. > Yes-- of course-- all of these operations can be done >with a database backing store, as well, but it is significantly more >difficult to >develop such tools and install them into the system. It is? have you done it? I have... (www.apple.com/signmeup). It's not as bad as you think. > Likely, this is somewhat of >a perception issue, but most administrators will not hesitate to >toss together a >shell script to manage an archive of stuff, but will think twice >before diddling a >database. yes, that's a eprception issue, and it's also a strawman. I don't buy that for a second. Numbers, please. you can't throw a strawman like this out without backing data. >Very large and very expensive databases scale very well-- MySQL does not. sure it does. But even more important -- if I outgrow MySQL and need to throw in a really big muther database like Oracle, I can fairly easily. If your filesystem store system is outgrown and you need to add capacity to scale it, how do you do it? you rearchitect from scratch, probably going to a database-centric design. > I >agree that for truly huge, high traffic sites, moving away from a >pure filesystem >approach-- moving everything into, say, an ES10000 running one of >the magawhompus >Oracle/Sybase license-- would be the way to go. Oh, please. I'm running big megawhompus stuff on E250's and E450s (for my news servers), and MySQL no problemo. My big muther lsit server is MySQL on an E250, handles 28,000 database updates a day on average, sends out a few million emails a week, and spends most of its time idle (the only huge CPU sink I have is bounce processing, but then it take stime to process 300 megabyte bounce files) > But I don't think that is what >90% of the Mailman users are going to be using the system for and >requiring-- or >even encouraging the use of-- a database as a backing store for >messages will not >add value to those people. nor is that what I'm proposing. I don't know why you're so database averse, to be honest. but I think it's a personal aversion on your part, not a legitimate teechincal issue. >Considering most of the usage of an archive of messages.... > > - write operations are infrequent, modifications pretty much non existant > > - retrieval tends to be extremely sporadic and is generally *not* evenly >divided across the archive-- a relative few messages receive most of the hits > > - there are extremely limited ways of viewing the data; by author, date, >thread, subject.... with MOST views focused on thread. > > - indices are periodically updated > >... I still believe that a webserver-reading-files-from-filesystem >is going to be >loads more effecient than a >webserver-reading-data-from-client-server-connection-to-app-adaptor-reading-data-from-client-server-connection-to-multiuser-database. Sorry, but my reseach doesn't agree. And I'm not sure I agree with your idea of what goes on in archives, but I don't have numbers to back myself up on that. It also ignores ancillary advantages of databasing this stuff -- like the easy addition of content searching and the ability to write really good, customized search capabilities. In your way, you have to build technology (or borrow it, like HtDig) to get that, so anything you might possibly save resource or development wise in your model gets eaten by trying to do searching right -- and one thing I *have* found from my users is that archives without good search tools are pretty useless to them. So I consider archive/search a single key integrated module, even if the technologies are seeparate, and databasing stuff allows me to build a lot of search power into the system, where your setup doesn't -- you have to go and do it the hard way (and I've done that, and it sucks...) > >Yes-- if you are doing textual searches *directly* against the filesystem, >filesystems are a lose. But a database is not a whole lot better >unless you blow >the relatively major Not true. >Regardless of where the data is stored, searches would typically >have to be backed >by some kind of a indexing store-- be it in a dabase, a btree type solution but one of the nice things about databases is they're written to build indexes for you -- and the guys who write those indexing routines are experts. so you leverage their strengths. >solutions available. Building that index from a database or from a >filesystem >isn't really that much different in terms of difficulty though a >number of folks >would find the problem of walking a filesystem more approachable >than walking a >database. I've done both. I disagree. filesystem-centric systems are system intensive resource hogs that are fine for small to medium installations but scale poorly, and which require re-architecting when you outgrow them. Databse centric ones might be a little more work up front, might be somewhat overkill for tiny sites, but even for small ones, tend to break even, and scale upward basically infinitely because you can swap in bigger horses based on your budget -- but you don't need those horses. You can do really good stuff with open source tools. All you need is some good database design. not even great database design. >With that said, I do feel strongly that an abstraction layer does little good >without concrete implementations of the adaptors underneath. As such, I >volunteer myself to write the adaptor to WebDAV and I'll tentatively volunteer >Chuq to write the adaptor that speaks to a database backing store. Actually, chuq's planning on architecting large parts of mailman 3.0, assuming Barry gives the Okay. but I volunteered for that weeks ago, and have been whacking on concepts on and off since. And since a lot of what I hope to do in mailman wll be leveraged off work I've already done or am planning to do for Other Things I Can't Admit To, a lot of it is beyond "I think this will work, maybe" in thought... Since 1994, I've architected and implemented about half a dozen production e-mail systems, from really tiny things based on common tools (first was Listproc, then majordomo, now Mailman, for the three generations of my 'generic' servers), to really bastard big things to corporate email systems... And in the next six months, I'm rewriting my big muther almost from scratch to take it to the 25,000,000 subscriber capability and double delivery speed (again), and then we're probably redoing the internal corporate (which supports ~15,000 lists) to handle list lookup and delivery/authentication on demand via LDAP to the corporate databases (right now, we snapshot/download/munge the data...). So a lot of where I think Mailman ought to go is taking pieces of some of these boxes (with my bosses kind permission) and making it part of Mailman... And trust me, it'll be a long time before even my biggest email system needs Oracle or an ES10000. you can do wonders with a stack of Ultra 5s slaved to a decent sized box like an E250 (in fact, in many cases, taht's a lot better). I apologize if this sounds like I'm pulling rank, but -- you keep saying that things can't be done that I know are wrong, because I already have in some form or another, or that I've already done the design (and/or prototype for and know how it'll work. chuq -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 07:59:47 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Tue, 21 Nov 2000 23:59:47 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <3A1B7523.1C3F9AFA@codefab.com> References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> <3A1B7523.1C3F9AFA@codefab.com> Message-ID: At 2:26 AM -0500 11/22/00, Bill Bumgarner wrote: >This would have saved us weeksandweeksandweeks of development on the >system we built that had Mailman as a critical component. > >Anything that is *using* Mailman as just-another-moving-part-among-many >faces the memebership management problem and it is nasty,nasty,nasty. If we end up doing the authentication widget, I want it to ship with a set of modules and/or interfaces. That would include one that does personal home pages, surveys, an interface to mailman, an interface to a web forum, an LDAP interface of some sort for external directory lookup and authentication purposes, and a few other toys. With it and a few key tools, you can easily do 90% of Yahoo, minus the big muther sets of data, but when it's done, you're very close to a yahoo portal with clubs, IRC, egroups and some other tools all rolled into a single cohesive beast -- and it'll scale, and it will be extensible through a standard interface. Assuming, of course, what I want to do works and I don't botch the architecture... (and what I find scary is I don't see *anything* that is technically a huge stretch. it's all engineering, not magic... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 08:04:55 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 22 Nov 2000 00:04:55 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <18729.974877726@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> <18729.974877726@kanga.nu> Message-ID: At 11:22 PM -0800 11/21/00, J C Lawrence wrote: >While I agree (with some distaste), those gyrations have shown >themselves to have positive value for me. short term. I had the same general attitude for a while, and did pretty much teh same thing, and found that over time, it basically killed my site through isolation and stagnation. I served a given population very well, but found that all I did was drive off fresh blood, and the existing population stopped growing, stagnated and started shrinking as old members moved on and weren't replaced. Which, if it sounds like USENET today, well, oyu're right. USENET is doing the same thing, but on a hugely massive scale. I'm still working to recover what I feel is an essential vitality in my lists and get back into growth mode. But we're so far off anything relevant to Mailman issues it's insane, so I won't push this one further on these lists, but audience management is tricky, and even things you think are working my bite you in the butt when you least expect it. Once you set the expectation of exclusionism on your site, it's a real bitch to change back.... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Wed Nov 22 08:11:48 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 22 Nov 2000 00:11:48 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <20380.974878579@kanga.nu> References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> <3A1B7617.2836857C@codefab.com> <20380.974878579@kanga.nu> Message-ID: At 11:36 PM -0800 11/21/00, J C Lawrence wrote: >While pursueing the LServe game we need to not lose sight of the >little guys running a couple lists for their club of 20-some model >train enthusiasts. If we even want to be in teh LServe game -- but this nis one of the neat things of the pure plug-in universe. if someone wants a big muther listserve, plug in the fancy module. If they want simple subscription systems, plug in the easy subscriber setup. you have to be very careful not to try to build A THING that is everythign for everyone, but you sure can build a framework that contains the capability to build things that allow you to build a system catered to each system's needs. And maybe we don't ship all of those variations initially (or ever!), but if someone wants it, we've at elast built a system where they can write it for themselves (and us). Lots of sites won't want my muther subscriber system from hell module, but there's no reason at all we can't take mailman's existing system and turn it into a "simple" subscriber system. Or write one that 100% mimics majordomo doing e-mail only. Adn in fact, for something like this, having two or three variations to test the API would be great. And it'd help us prove the architecture and keep us honest, so the different modules don't cheat by looking over each other's shoulders... If folks want to see a successful (so far) project taht's doing this, I think you need look no further than Gnome, and its underlying X windows basis. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From thomas@xs4all.net Wed Nov 22 09:58:33 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 22 Nov 2000 10:58:33 +0100 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: ; from ken@kyler.com on Tue, Nov 21, 2000 at 06:55:57PM -0500 References: <14875.1663.422580.420642@anthem.concentric.net> Message-ID: <20001122105832.F540@xs4all.nl> On Tue, Nov 21, 2000 at 06:55:57PM -0500, Ken Kyler wrote: > Keep up the good work guys! If I ever finish grad school I'll help. Shht, don't tell Barry, but I never finished grad school (or any form of highschool, for that matter) and he still put some of my fixes into Mailman, and adapted others ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From barry@digicool.com Wed Nov 22 14:17:44 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 22 Nov 2000 09:17:44 -0500 Subject: [Mailman-Developers] Re: Future of pipermail? References: <200011220423.eAM4Njb16105@s1.optonline.net> <7649.974873054@kanga.nu> <13342.974875316@kanga.nu> <15324.974876302@kanga.nu> <3A1B7617.2836857C@codefab.com> <20380.974878579@kanga.nu> Message-ID: <14875.54664.457063.83250@anthem.concentric.net> Jeez, try to get one good night of sleep and look what happens! :) From bkuhn@gnu.org Wed Nov 22 16:27:02 2000 From: bkuhn@gnu.org (Bradley M. Kuhn) Date: Wed, 22 Nov 2000 11:27:02 -0500 Subject: [Mailman-Developers] Errors in mailman that I cannot figure out Message-ID: <20001122112702.F2833@ebb.org> --FeAIMMcddNRN4P4/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I am writing on behalf of the GNU project. We have an discovered some sort of bug in mailman. The .../logs/errors file is getting: Nov 22 11:24:01 2000 qrunner(29262): Traceback (innermost last): Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 278, in ? Nov 22 11:24:01 2000 qrunner(29262): kids = main(lock) Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 247, in main Nov 22 11:24:01 2000 qrunner(29262): keepqueued = dispose_message(mlist, msg, msgdata) Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 121, in dispose_message Nov 22 11:24:01 2000 qrunner(29262): if BouncerAPI.ScanMessages(mlist, mimemsg): Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/BouncerAPI.py", line 59, in ScanMessages Nov 22 11:24:01 2000 qrunner(29262): addrs = func(msg) Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/Catchall.py", line 132, in process Nov 22 11:24:01 2000 qrunner(29262): username = string.split(line)[1] Nov 22 11:24:01 2000 qrunner(29262): IndexError : list index out of range This was happening with 2.0beta6, so I upgraded to see if it's a bug fixed by 2.0, but it isn't. I think this may be related to a particular file in the queue being formatted oddly, but the result is the entire queue is held up. I am looking a bit deeper into the code right now, but I'd appreciate help. (Mailing lists at gnu.org are currently down becasue of this bug). I will send more information as I discover it. --FeAIMMcddNRN4P4/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6G/PW53XjJNtBs4cRAqiFAKCsUa2uMjiXeW8omDeQLXz49JBsOQCgu2Tv qrpYvXRrvBwhYoxcN3OrxB8= =aCnS -----END PGP SIGNATURE----- --FeAIMMcddNRN4P4/-- From barry@digicool.com Wed Nov 22 17:33:16 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 22 Nov 2000 12:33:16 -0500 Subject: [Mailman-Developers] [ANNOUNCE] Mailman 2.0 Message-ID: <14876.860.607056.18676@anthem.concentric.net> I'm please to announce the 2.0 final release of Mailman, the GNU Mailing List Manager. Mailman is released under the GNU General Public License (GPL). Mailman is software to help manage electronic mail discussion lists, much like Majordomo or Smartmail. Mailman gives each mailing list a unique web page and allows users to subscribe, unsubscribe, and change their account options over the web. Even the list manager can administer his or her list entirely via the web. Mailman has most of the features that people want in a mailing list management system, including built-in archiving, mail-to-news gateways, spam filters, bounce detection, digest delivery, and so on. Mailman is compatible with most web servers, web browsers, and mail servers. It should run on any Unix-like operating system. Mailman requires Python 1.5.2 or better. To install Mailman from source, you will need a C compiler. For more information on Mailman, please see the following mirrors: http://www.list.org http://mailman.sourceforge.net http://www.gnu.org/software/mailman/mailman.html (the latter is not yet up to date) There are email lists (managed by Mailman, of course!) for both Mailman users and developers. See the web sites above for details. My thanks to everybody who has helped with this release, especially those named in the ACKNOWLEDGMENTS file. My thanks also to the SourceForge crew for their open source project management site. Couldn't have done it without them. Enjoy, -Barry From bkuhn@gnu.org Wed Nov 22 17:46:20 2000 From: bkuhn@gnu.org (Bradley M. Kuhn) Date: Wed, 22 Nov 2000 12:46:20 -0500 Subject: [Mailman-Developers] Fixed problem at gnu.org, patch included (was Re: Errors in mailman that I cannot figure out) In-Reply-To: <20001122112702.F2833@ebb.org>; from bkuhn@gnu.org on Wed, Nov 22, 2000 at 11:27:02AM -0500 References: <20001122112702.F2833@ebb.org> Message-ID: <20001122124619.K2833@ebb.org> --8tUgZ4IE8L4vmMyh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Bradley M. Kuhn wrote: > I am writing on behalf of the GNU project. We have an discovered some sort > of bug in mailman. The .../logs/errors file is getting: > > Nov 22 11:24:01 2000 qrunner(29262): Traceback (innermost last): > Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 278, in ? > Nov 22 11:24:01 2000 qrunner(29262): kids = main(lock) > Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 247, in main > Nov 22 11:24:01 2000 qrunner(29262): keepqueued = dispose_message(mlist, msg, msgdata) > Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 121, in dispose_message > Nov 22 11:24:01 2000 qrunner(29262): if BouncerAPI.ScanMessages(mlist, mimemsg): > Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/BouncerAPI.py", line 59, in ScanMessages > Nov 22 11:24:01 2000 qrunner(29262): addrs = func(msg) > Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/Catchall.py", line 132, in process > Nov 22 11:24:01 2000 qrunner(29262): username = string.split(line)[1] > Nov 22 11:24:01 2000 qrunner(29262): IndexError : list index out of range I have fixed this bug. It had to do with regexes that weren't precise in Catchall.py. Here's a patch. I also made a ChangeLog, because you all don't appear to have one yet! ############################################################################### # Fixed bug with Catchall regexs # # To apply this patch: # STEP 1: Chdir to the source directory. # STEP 2: Run the 'applypatch' program with this patch file as input. # # If you do not have 'applypatch', it is part of the 'makepatch' package # that you can fetch from the Comprehensive Perl Archive Network: # http://www.perl.com/CPAN/authors/Johan_Vromans/makepatch-x.y.tar.gz # In the above URL, 'x' should be 2 or higher. # # To apply this patch without the use of 'applypatch': # STEP 1: Chdir to the source directory. # If you have a decent Bourne-type shell: # STEP 2: Run the shell with this file as input. # If you don't have such a shell, you may need to manually create # the files as shown below. # STEP 3: Run the 'patch' program with this file as input. # # These are the commands needed to create/delete files/directories: # touch 'ChangeLog' chmod 0600 'ChangeLog' # # This command terminates the shell and need not be executed manually. exit # #### End of Preamble #### #### Patch data follows #### diff -u /dev/null 'mailman/ChangeLog' Index: ./ChangeLog *** ./ChangeLog Wed Dec 31 19:00:00 1969 --- ./ChangeLog Wed Nov 22 12:01:37 2000 *************** *** 0 **** --- 1,4 ---- + 2000-11-22 Bradley M. Kuhn + + * Mailman/Bouncers/Catchall.py (process): changed some of the + "messy" regular expressions so errors don't occur diff -u 'mailman-pristine/Mailman/Bouncers/Catchall.py' 'mailman/Mailman/Bouncers/Catchall.py' Index: ./Mailman/Bouncers/Catchall.py --- ./Mailman/Bouncers/Catchall.py Sun Aug 6 22:34:33 2000 +++ ./Mailman/Bouncers/Catchall.py Wed Nov 22 12:02:52 2000 @@ -102,9 +102,12 @@ (regex.compile('.*%s: User unknown.*' % email_regexp), REMOVE), (regex.compile('.*%s\.\.\. User unknown' % email_regexp), REMOVE)) # patterns we can't directly extract the email (special case these) - messy_pattern_1 = regex.compile('^Recipient .*$') - messy_pattern_2 = regex.compile('^Addressee: .*$') - messy_pattern_3 = regex.compile('^User .* not listed.*$') + # Make sure that these patterns will not cause the special cases below + # to fail. In other words, be sure that these regexs ensure that split's + # and other operations done below will always work. + messy_pattern_1 = regex.compile('^Recipient[ \t]+[^ \t]+[ \t]*$') + messy_pattern_2 = regex.compile('^Addressee:[ \t]*[^ \t]+[ \t]*$') + messy_pattern_3 = regex.compile('^User [^ \t]+ not listed.*$') messy_pattern_4 = regex.compile('^550 [^ ]+\.\.\. User unknown.*$') messy_pattern_5 = regex.compile('^User [^ ]+ is not defined.*$') messy_pattern_6 = regex.compile('^[ \t]*[^ ]+: User unknown.*$') #### End of Patch data #### #### ApplyPatch data follows #### # Data version : 1.0 # Date generated : Wed Nov 22 12:25:09 2000 # Generated by : makepatch 2.00 # Recurse directories : Yes # Excluded files : (\A|.*/)CVS(/.*|\Z) # (\A|.*/)RCS(/.*|\Z) # ,v\Z # (\A|.*/)SCCS(/.*|\Z) # (\A|.*/)[sp]\..+\Z # c 'ChangeLog' 0 974912497 0100600 # p 'Mailman/Bouncers/Catchall.py' 7873 974912572 0100600 #### End of ApplyPatch data #### #### End of Patch kit [created: Wed Nov 22 12:25:09 2000] #### #### Checksum: 78 3275 60184 #### --8tUgZ4IE8L4vmMyh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6HAZr53XjJNtBs4cRAmgTAJ4zQeFCa3xpnFK31auM+IMRkymMIgCdF5Hj aBqPjNlRvGvBnb3Dwmz6thk= =WgRu -----END PGP SIGNATURE----- --8tUgZ4IE8L4vmMyh-- From claw@kanga.nu Wed Nov 22 20:11:40 2000 From: claw@kanga.nu (J C Lawrence) Date: Wed, 22 Nov 2000 12:11:40 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Tue, 21 Nov 2000 23:59:47 PST." References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> <3A1B7523.1C3F9AFA@codefab.com> Message-ID: <25256.974923900@kanga.nu> On Tue, 21 Nov 2000 23:59:47 -0800 Chuq Von Rospach wrote: > At 2:26 AM -0500 11/22/00, Bill Bumgarner wrote: >> Anything that is *using* Mailman as >> just-another-moving-part-among-many faces the memebership >> management problem and it is nasty,nasty,nasty. > If we end up doing the authentication widget, I want it to ship > with a set of modules and/or interfaces. That would include one > that does personal home pages, surveys, an interface to mailman, > an interface to a web forum, an LDAP interface of some sort for > external directory lookup and authentication purposes, and a few > other toys. I have slight allergic reactions here. Yes, I agree that a generic modular authentication model would be extremely useful (and I'd kill for a sufficiently flexible one), and it would be good if Mailman were able to plug into such a thing were it to be available on the local system, I don't see this as a Mailman specific problem, or one that should really be considered part of or rolled into the Mailman design process outside of "we should be able to integrate with one of those if its available, otherwise we punt to our LCD implementation". > With it and a few key tools, you can easily do 90% of Yahoo, minus > the big muther sets of data, but when it's done, you're very close > to a yahoo portal with clubs, IRC, egroups and some other tools > all rolled into a single cohesive beast -- and it'll scale, and it > will be extensible through a standard interface. Which, while a nice problem, is not the problem I belive Mailman is trying to solve. Mailman should of course play nicely with such solutions (yes, plural), but I see considerable loss to be realised if we tie it to a specific implementation or model (eg Zope), and abstracting at just the API level (as versus at the process level) is going to tend to lead you in that direction. > (and what I find scary is I don't see *anything* that is > technically a huge stretch. it's all engineering, not magic... Too true (written as one who has just re-engineered his own auth models). There's no rocket science here, just the normal stack of security model and privacy concerns. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Wed Nov 22 20:29:08 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Wed, 22 Nov 2000 12:29:08 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <25256.974923900@kanga.nu> References: <200011212104.eALL4W327658@bjork.codefab.com> <26896.974852960@kanga.nu> <14763.974875990@kanga.nu> <3A1B7523.1C3F9AFA@codefab.com> <25256.974923900@kanga.nu> Message-ID: At 12:11 PM -0800 11/22/00, J C Lawrence wrote: >I have slight allergic reactions here. Yes, I agree that a generic >modular authentication model would be extremely useful (and I'd kill >for a sufficiently flexible one), and it would be good if Mailman >were able to plug into such a thing were it to be available on the >local system, I don't see this as a Mailman specific problem, or one >that should really be considered part of or rolled into the Mailman >design process outside of "we should be able to integrate with one >of those if its available, otherwise we punt to our LCD >implementation". that's why I propose this as a separate project -- and why the first thing we'd need to do is define the API, then port the existing subscription system to the API. The end result would be that if you want to use the existing system, you would, but the hooks would be there to move to a more extensive system. I don't for a minute think we want that more extensive system to be the default for mailman, not for any significant amount of time, if ever (any more than I think mailman ought to require Zope, ought to require external templating modules, ought to require...) So from the point of view of Mailman, the work would be to define and architect the API and get the existing system working with it -- and then work on an external project that'd interface to that API, not replace the existing system. >Which, while a nice problem, is not the problem I belive Mailman is >trying to solve. yup. But it's part of this larger convergence and integration problem. The integration tools need to be created, but I'm not proposing they be mandatory. >Too true (written as one who has just re-engineered his own auth >models). There's no rocket science here, just the normal stack of >security model and privacy concerns. ah, who cares about security and privacy, anyway? But to be honest -- if we can build a single, GOOD authentication widget and get those issues right, doesn't that make it nicer for everyone else, since they can simply write to use the widge and not reinvent the wheel endlessly? -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From darrell@grumblesmurf.net Wed Nov 22 23:04:26 2000 From: darrell@grumblesmurf.net (Darrell Fuhriman) Date: 22 Nov 2000 15:04:26 -0800 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: barry@digicool.com's message of "Wed, 22 Nov 2000 00:02:26 -0500" References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > >From the TODO list, things I can imagine tackling for 2.1 include: I would like to see a couple things added. 1) the patch I wrote to sort the list and break it up in chunks when passed to the mailer -- FWIW, I've been using it since I wrote it without problems, surely others have, too. Like I said earlier, at worst, it's an innocuous feature, but in general it does benefit delivery performance. 2) In pipermail, I'd like to be able to define HTML tags that can be wrapped around non-important parts of the particular message. Specifically, this is to enable you to exclude certain parts of the message from indexing, such as navigation information. (I'm thinking specifically of the tags. That is, of course, a rather obscure feature. :) Darrell From ckolar@admin.aurora.edu Wed Nov 22 23:52:53 2000 From: ckolar@admin.aurora.edu (Christopher Kolar) Date: Wed, 22 Nov 2000 17:52:53 -0600 Subject: [Mailman-Developers] v2 Management Documents, GFDL Message-ID: <5.0.1.4.2.20001122175114.0205df30@admin.aurora.edu> I am pleased to announce that v2 of the Mailman list manager's documentation is available at www.aurora.edu/~ckolar/mailman. Aurora University has generously agreed to release these documents under the GNU Free Documentation License (GFDL), details are on the site. --chris From Dan Mick Thu Nov 23 02:46:56 2000 From: Dan Mick (Dan Mick) Date: Wed, 22 Nov 2000 18:46:56 -0800 (PST) Subject: [Mailman-Developers] Towards 2.1 Message-ID: <200011230245.SAA09389@utopia.west.sun.com> > barry@digicool.com (Barry A. Warsaw) writes: > > > >From the TODO list, things I can imagine tackling for 2.1 include: > > I would like to see a couple things added. > > 1) the patch I wrote to sort the list and break it up in chunks > when passed to the mailer -- FWIW, I've been using it since I > wrote it without problems, surely others have, too. Like I said > earlier, at worst, it's an innocuous feature, but in general it > does benefit delivery performance. I've been using it, and I think I love it. I think it ought to be optionally added, too. A pleasant side-effect of the domain sorting is when Hotmail gets screwed up (once or twice a week), all the hotmail mail bounces at the same time. :-} From Nigel.Metheringham@VData.co.uk Thu Nov 23 09:45:12 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Thu, 23 Nov 2000 09:45:12 +0000 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: Message from Darrell Fuhriman of "22 Nov 2000 15:04:26 PST." Message-ID: darrell@grumblesmurf.net said: > 2) In pipermail, I'd like to be able to define HTML tags that can be > wrapped around non-important parts of the particular message. > Specifically, this is to enable you to exclude certain parts of the > message from indexing, such as navigation information. (I'm thinking > specifically of the tags. > That is, of course, a rather obscure feature. :) There is already a patch to do this - its marked as being for htdig, but all it actually does is inject definable indexer on/off statements into the generated HTML. It does assume that your indexer parses are utilises the indexer tags that already appear in pipermail HTML. See https://sourceforge.net/patch/?func=detailpatch&patch_id=102422&group_ id=103 or if that doesn't work, Patch 102422 Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From carsten.dot.geckeler@gmx.dot.de Thu Nov 23 15:38:26 2000 From: carsten.dot.geckeler@gmx.dot.de (Carsten Geckeler) Date: Thu, 23 Nov 2000 16:38:26 +0100 (MET) Subject: [Mailman-Developers] Re: [ANNOUNCE] Mailman 2.0 In-Reply-To: <14876.860.607056.18676@anthem.concentric.net> Message-ID: This is bad. -- Carsten Geckeler: carsten dot geckeler at gmx dot de From carsten.dot.geckeler@gmx.dot.de Thu Nov 23 21:07:51 2000 From: carsten.dot.geckeler@gmx.dot.de (Carsten Geckeler) Date: Thu, 23 Nov 2000 22:07:51 +0100 (MET) Subject: [Mailman-Developers] Re: [ANNOUNCE] Mailman 2.0 In-Reply-To: Message-ID: Sorry, sorry, sorry! My reply was just accidentially send. I was testing my mail filter for masking the From: address. It just slipped accidentially into the send queue. So my mail was not an expression of my opinion about Mailman. Will not happen again. Cheers, Carsten -- Carsten Geckeler: carsten dot geckeler at gmx dot de From lkarrer@trash.net Thu Nov 23 22:29:10 2000 From: lkarrer@trash.net (Lukas Karrer) Date: Thu, 23 Nov 2000 23:29:10 +0100 (MET) Subject: [Mailman-Developers] Re: [Mailman-Announce] [ANNOUNCE] Mailman 2.0 In-Reply-To: <14876.860.607056.18676@anthem.concentric.net> Message-ID: On Wed, 22 Nov 2000, Barry A. Warsaw wrote: Hi Mailman developpers.. I just seamlessly migrated a couple of lists to version 2... JUST PERFECT how everything is well documented and works as it should!! THANKS ALOT for your effort! Keep on writing good Software Lukas +---------------------------------------------------------------------+ Lukas Karrer Email: lkarrer@trash.net Sysadmin WWW: http://www.trash.net/ +---------------------------------------------------------------------+ Check out the stinkiest site on the web! Get a sniff from www.trash.net t r a s h . n e t - free UNIX Shellaccounts for Switzerland From bbum@codefab.com Fri Nov 24 05:01:29 2000 From: bbum@codefab.com (Bill Bumgarner) Date: Fri, 24 Nov 2000 00:01:29 -0500 (EST) Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message-ID: How about PAM-- pluggable authentication modules? It seems to have become relatively standard on LInux and BSD and provides a flexible and portable API that provides for versatile plug-n-play authentication across platforms. I recently used it with Apache and ProFTPD to easily configure both to use the same user/passwd database-- and neither to use the systems existing user/pass system. PAM modules exist for berkeley DB and MySQL, among various random standard authentication procedures such as Kerberos and /etc/passwd. It would not be hard to provide a module that makes the system accessible from Python and the applicability would range a lot further than Mailman [make it immediately attractive to others]. (It may be the case that a PAM python module already exists? Haven't looked...) Even if the implementation is not appropriate, the architecture is certainly interesting and will likely contain ideas that are pertinent to building such an authentication system as Chuq described. b.bum From jeff@ollie.clive.ia.us Fri Nov 24 07:04:29 2000 From: jeff@ollie.clive.ia.us (Jeffrey C. Ollie) Date: Fri, 24 Nov 2000 01:04:29 -0600 Subject: [Mailman-Developers] Interface to Archivers Message-ID: <20001124010429.A14295@ollie.clive.ia.us> --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline It would seem to me that the discussion so far regarding the "future of pipermail" is going too far in the direction of discussing "what would the most keenest archiver look like" rather than discussing "what should the interface between the Mailman and an archiver look like" and "what should the default archiver look like". So to try and steer the discussion in this direction, here I go sticking my foot in my mouth: What Should the Interface Between Mailman and an Archiver Look Like: def submit(mlist, msg) archiver = __import__('Mailman.Archivers.' + mm_cfg.ARCHIVE_MODULE) submit = getattr(getattr(getattr(mod, 'Archivers'), mm_cfg.ARCHIVE_MODULE), 'submit') urls = submit(mlist, msg) return urls The 'urls' returned from the archive module will be a data structure that includes a URL that points to the archived message as a whole and optionally URLs that point to individual MIME-decoded individual parts. If the archiver includes URLs for individual parts Mailman could optionally reformat the message to remove attachments and replace them with a pointer to the archive. So then we write any number of modules that interface to popular archivers, plus we include a simple default archiver (IMHO, in the same tarball as the Mailman). This way, the details of the archiver are completely hidden from Mailman and adding support for new archivers is very simple. I suppose that one could even add support for different archivers for every list. What Should the Default Archiver Look Like: It'll look a lot like pipermail, in that it stores archived messages in files on a filesystem. Indexes and tables of contents will be rebuilt as needed and saved in static files. It'll look different from pipermail in that MIME message parts will be decoded and be accessible as individual files. The reason for going with files stored on a filesystem is that it's very simple. No database to deploy and no fancy web server is required. More complex schemes are of course possible by writing the appropriate plug-in. Jeff --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6HhL9mCSL+rEqeRsRAvoMAJ9ZmUkm3+XFtRfjfnibP2E9AIQh4QCgmeAo NNxpNdKGCU5dbeDMpSa+ca4= =DC7Y -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24-- From claw@kanga.nu Fri Nov 24 07:45:26 2000 From: claw@kanga.nu (J C Lawrence) Date: Thu, 23 Nov 2000 23:45:26 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Bill Bumgarner of "Fri, 24 Nov 2000 00:01:29 EST." References: Message-ID: <24944.975051926@kanga.nu> On Fri, 24 Nov 2000 00:01:29 -0500 (EST) Bill Bumgarner wrote: > How about PAM-- pluggable authentication modules? It seems to > have become relatively standard on LInux and BSD and provides a > flexible and portable API that provides for versatile plug-n-play > authentication across platforms. Authentication data in the sense of identification (you know the UID and the password) is the simplest problem, and is probably so small as to be ignorable. The problem is not just UID/PW combos, but access rights, settings, configurations, meta data, etc. > Even if the implementation is not appropriate, the architecture is > certainly interesting and will likely contain ideas that are > pertinent to building such an authentication system as Chuq > described. True. As happens I'm more tempted by LDAP. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From chuqui@plaidworks.com Fri Nov 24 07:54:23 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Thu, 23 Nov 2000 23:54:23 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: <24944.975051926@kanga.nu> References: <24944.975051926@kanga.nu> Message-ID: At 11:45 PM -0800 11/23/00, J C Lawrence wrote: >True. As happens I'm more tempted by LDAP. I tend to think that the mailman <-> authentication interface is going to be gnarly enough that it'll be a custom API. But I also think that having PAM and LDAP access to the data will also important to have for access in other ways -- PAM solves my problem for archive access in apache wonderfully, and either PAM or LDAP would make it easier to evolve my web forums into using a sitewide account/password setup, which is something I really want down the road. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From claw@kanga.nu Fri Nov 24 16:36:18 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 24 Nov 2000 08:36:18 -0800 Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: Message from Chuq Von Rospach of "Thu, 23 Nov 2000 23:54:23 PST." References: <24944.975051926@kanga.nu> Message-ID: <31572.975083778@kanga.nu> On Thu, 23 Nov 2000 23:54:23 -0800 Chuq Von Rospach wrote: > At 11:45 PM -0800 11/23/00, J C Lawrence wrote: >> True. As happens I'm more tempted by LDAP. > I tend to think that the mailman <-> authentication interface is > going to be gnarly enough that it'll be a custom API. But I also > think that having PAM and LDAP access to the data will also > important to have for access in other ways -- PAM solves my > problem for archive access in apache wonderfully, and either PAM > or LDAP would make it easier to evolve my web forums into using a > sitewide account/password setup, which is something I really want > down the road. I don't see them as exclusive: http://www.padl.com/pam_ldap.html ---- The pam_ldap module provides the means for Solaris and Linux workstations to authenticate against LDAP directories, and to change their passwords in the directory. ---- -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From claw@kanga.nu Fri Nov 24 18:31:29 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 24 Nov 2000 10:31:29 -0800 Subject: [Mailman-Developers] Editing of messages at the admin page Message-ID: <16058.975090689@kanga.nu> I'm toying with the idea of adding raw message editing capabilities to the web moderation interface. The interface design would be as follows: ADMINDB_PAGE_TEXT_LIMIT would accept a special value of "0" meaning "entire message". This would have the side effect of also displaying all headers. If the config is set to display entire messages and the moderator edits the message in the relevant TEXTAREA, then the following will happen: A custom header will be inserteted in the message headers: X-Moderator: This message was edited by the list moderator A similar comment will be prepended at the top of the message. (I'm still argueing with myself on this one as I'd like it but some wouldn't) The edited form of the message will be broadcast. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From Ricardo Kustner" References: <16058.975090689@kanga.nu> Message-ID: <200011242255.XAA08079@smtp9.xs4all.nl> On Fri, 24 Nov 2000 10:31:29 -0800, J C Lawrence said: > I'm toying with the idea of adding raw message editing capabilities > to the web moderation interface. The interface design would be as > follows: > ADMINDB_PAGE_TEXT_LIMIT would accept a special value of "0" > meaning "entire message". This would have the side effect of also > displaying all headers. > The edited form of the message will be broadcast. Although I like this idea, I personally think it would be more friendly to add an extra comment field that will be added to the top or bottom of the message... something like "[ Administrators note: blah blah blah ]"... If you edit the content of a persons message nobody will know what exactly you editted which can lead to questionable situations... Ricardo. -- http://rixhq.nu "You think that's air you're breathing?" -- Morpheus From claw@kanga.nu Sat Nov 25 00:26:57 2000 From: claw@kanga.nu (J C Lawrence) Date: Fri, 24 Nov 2000 16:26:57 -0800 Subject: [Mailman-Developers] Editing of messages at the admin page In-Reply-To: Message from "Ricardo Kustner" of "24 Nov 2000 23:55:20." <200011242255.XAA08079@smtp9.xs4all.nl> References: <16058.975090689@kanga.nu> <200011242255.XAA08079@smtp9.xs4all.nl> Message-ID: <31526.975112017@kanga.nu> On 24 Nov 2000 23:55:20 CET Ricardo Kustner wrote: > Although I like this idea, I personally think it would be more > friendly to add an extra comment field that will be added to the > top or bottom of the message... something like "[ Administrators > note: blah blah blah ]"... If you edit the content of a persons > message nobody will know what exactly you editted which can lead > to questionable situations... While I have a minor history of editing user's submissions to the list, in each case I've prefaced the resultant message with something akin to: EdNote: The following lightly edited for eg: http://www.kanga.nu/archives/MUD-Dev-L/2000Q4/msg00205.php I like this practice for the reasons you state. It is for similar reasons that I preface messages posted via my archives with text indicating both the source and invalidity of the From: address: http://www.kanga.nu/archives/IRead-L/2000Q4/msg00000.php The problem with doing this automatically upon edits is handling MIME in an end-user friendly fashion. The intent of the added header etc is partially substitute for an editorial note, and to make up for those occassions where the moderator edits and then fogets to annotate his changes. I don't want to embed diffs of the changes (which would remove the value of the edit in the case of flammage or inappropriate content), but I feel the system should at least flag the fact of the edit in some automatic fashion. This all said, the possibility of abuse is always present. Mailman can't prevent it, and I don't believe should be in the busines of controlling content at that level (tehre are applications where its unwelcome and unecessary). -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From tismer@tismer.com Sat Nov 25 15:03:09 2000 From: tismer@tismer.com (Christian Tismer) Date: Sat, 25 Nov 2000 17:03:09 +0200 Subject: [Mailman-Developers] Minor flaw in archive date handling Message-ID: <3A1FD4AD.5270BBFC@tismer.com> Hi folks, I had some trouble getting the stackless mailing list repaired, after one message was repeated thousands of times. This was a message from October, but interestingly, the thousands of messages appeared in both the October and the November archives. Looking forther at one instance of the thousands of copies, I recognized that the message had no "Date:" entry, just a Resent-Date, for some unknown reason. After clearing the whole pipermail archive and recreating it with the bin/arch command, the message again appeared in the November archive. I took one of the fields with valid date enties and added it as "Date:" field by hand. Then the archiving worked as expected. My question: When a message happens to have no date, messages appear to be archived into the wrong archive file. There are a couple of other date fields available, like the usual "from" - line on top of the message, fields like Resent-Date or dates from the message passing through other servers. Wouldn't it make sense to do a more exhaustive search for a proper date, instead of accepting a missing date? cheers - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com From tismer@tismer.com Sat Nov 25 15:45:44 2000 From: tismer@tismer.com (Christian Tismer) Date: Sat, 25 Nov 2000 17:45:44 +0200 Subject: [Mailman-Developers] Minor flaw in archive date handling References: <3A1FD4AD.5270BBFC@tismer.com> Message-ID: <3A1FDEA8.CD987FB0@tismer.com> Christian Tismer wrote: I wrote: > My question: When a message happens to have no date, messages > appear to be archived into the wrong archive file. There > are a couple of other date fields available, like the usual > "from" - line on top of the message, fields like Resent-Date > or dates from the message passing through other servers. > > Wouldn't it make sense to do a more exhaustive search for > a proper date, instead of accepting a missing date? Changing the message from "one-should-do-such-and-such" style to "I-would-do-a-patch,-just-tell-me-how" style: I'm asking if such a change makes sense, and where to do the adjustment: In the process where the messages are collected in the .mbox file (Mailman change), or later, when the archives are generated (pipermail-change) ? regards - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com From tneff@bigfoot.com Sat Nov 25 17:10:04 2000 From: tneff@bigfoot.com (Tom Neff) Date: Sat, 25 Nov 2000 12:10:04 -0500 Subject: [Mailman-Developers] Re: editing messages In-Reply-To: <20001125170103.089071CD6A@dinsdale.python.org> Message-ID: <196628133.975154204@[192.168.0.2]> I patched 1.1 a year ago to allow editing a message before approval. I didn't bother with any pantywaist headers or disclaimers, although I suppose making one optionally available would be a joy-joy for the San Angeles Email Police. Mellow greetings and be well! *** admindb.py.orig Sun Jun 13 04:10:09 1999 --- admindb.py Mon Sep 18 14:27:48 2000 *************** *** 196,202 **** # We already handled this request. continue comment_key = 'comment-%d' % request_id ! if form.has_key(comment_key): list.HandleRequest(request, v, form[comment_key].value) else: list.HandleRequest(request, v) --- 196,207 ---- # We already handled this request. continue comment_key = 'comment-%d' % request_id ! contents_key = 'contents-%d' % request_id ! if form.has_key(contents_key) and form.has_key(comment_key): ! list.HandleRequest(request, v, form[comment_key].value, contents=form[contents_key].value) ! elif form.has_key(contents_key): ! list.HandleRequest(request, v, None, contents=form[contents_key].value) ! elif form.has_key(comment_key): list.HandleRequest(request, v, form[comment_key].value) else: list.HandleRequest(request, v) *************** *** 241,247 **** FontSize("+1", Bold('Contents:')) ]) form.AddItem(t) ! form.AddItem(Preformatted(val[2][1])) form.AddItem('

') --- 246,254 ---- FontSize("+1", Bold('Contents:')) ]) form.AddItem(t) ! # form.AddItem(Preformatted(val[2][1])) ! form.AddItem(TextArea("contents-%d" % val[0], rows=30, cols=85, ! text=val[2][1])) form.AddItem('

') From brian@gweep.bc.ca Sun Nov 26 14:20:05 2000 From: brian@gweep.bc.ca (Brian Edmonds) Date: 26 Nov 2000 06:20:05 -0800 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: barry@digicool.com's message of "Wed, 22 Nov 2000 00:02:26 -0500" References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: barry@digicool.com (Barry A. Warsaw) writes: > - Plain text digests should conform to RFC 1153. Yes please. Gnus supports 1153 digests, but doesn't recognize Mailman digests unless I switch to MIME format. > - Allow "urgent" postings to all members by the list admin which > bypasses normal digest delivery. I would like to be able to do this as more than just list admin. I've got at least three different lists on my system that have an announce, regular and digest options. The announce has separate subscribers and also goes immediately to all regular and digest subscribers. Currently those lists are still running under Majordomo, as I have no idea how to make them do the same thing with Mailman. And in the case of one of the lists it's not even an announce list, just a "regular" address that anyone can post to for topically urgent items. > - Allow the user to be excluded from postings if they're getting > them in the to: or cc: headers. Be smarter about filtering out > duplicate deliveries. This would be an interesting option, but I don't think I'd turn it on for most lists. > - Don't use the first public mailing list as the `originator' of > password reminders. Very much yes please. :) > - Provide an email interface to all administrative commands Ditto. > - For email subscribes, keep an audit of where requests are coming > from, and send the original request headers in the confirmation > message. Helps track down subscribe bombs. That sounds like a good idea. I've always liked mlms that do this. > - Support the `which' command. I had a user asking how to do this just yesterday. :) Brian. From chuqui@plaidworks.com Sun Nov 26 17:01:11 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Sun, 26 Nov 2000 09:01:11 -0800 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: At 6:20 AM -0800 11/26/00, Brian Edmonds wrote: >barry@digicool.com (Barry A. Warsaw) writes: >> - Plain text digests should conform to RFC 1153. > >Yes please. Gnus supports 1153 digests, but doesn't recognize Mailman >digests unless I switch to MIME format. which -- if you're munging digests -- you should anyway, since that's a big reaso FOR MIME digest format. Tehy're designed to be parseable. But that doesn't ignore that digests need to be tweaked, but if you're doing stuff to digests (like bursting), you should use MIME digests any time tehy're available. >I would like to be able to do this as more than just list admin. I've >got at least three different lists on my system that have an announce, >regular and digest options. The announce has separate subscribers and >also goes immediately to all regular and digest subscribers. Currently >those lists are still running under Majordomo, as I have no idea how to >make them do the same thing with Mailman. The "easy" answer is to disable digests for announce lists. Since they tend to be very low volume anyway, digest versions are generally pretty useless. >And in the case of one of the lists it's not even an announce list, just >a "regular" address that anyone can post to for topically urgent items. ditto here. It's pretty useless to read a digest of topical/timely items, because the concept of digests is to hold it for later. Defeats teh purpose, so turn them off. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From brian@gweep.bc.ca Sun Nov 26 17:18:22 2000 From: brian@gweep.bc.ca (Brian Edmonds) Date: 26 Nov 2000 09:18:22 -0800 Subject: [Mailman-Developers] Towards 2.1 In-Reply-To: Chuq Von Rospach's message of "Sun, 26 Nov 2000 09:01:11 -0800" References: <14875.21346.357211.669599@anthem.concentric.net> Message-ID: Chuq Von Rospach writes: > if you're doing stuff to digests (like bursting), you should use MIME > digests any time tehy're available. Sure, in theory, but I can burst 1153 digests for reading purposes just fine. The full headers in the MIME digests can be nice on occasion, but for 99% of my needs, 1153 is just fine an is both smaller and faster for my client to parse. > >The announce has separate subscribers and also goes immediately to > >all regular and digest subscribers. Currently those lists are still > >running under Majordomo, as I have no idea how to make them do the > >same thing with Mailman. > The "easy" answer is to disable digests for announce lists. Since they > tend to be very low volume anyway, digest versions are generally pretty > useless. Um, no. Posts to the announce list go to the announce subscribers, the regular subscribers and directly to the digest subscribers. I could maintain a list of additional announce recipients which would be the regular and digest subscribers, but that would be an extra mechanism on top of Mailman, and prone to getting out of sync. Majordomo lets me set things up this way, which is pretty much the only thing keeping me from moving all my lists to Mailman. I'd also like to be able to do admin regexp matching on the body, but this may already be present in 2.0. I'm still running beta2. Likewise configurable holding or immediate rejection on a per regexp basis. Brian. From tneff@bigfoot.com Mon Nov 27 17:20:51 2000 From: tneff@bigfoot.com (Tom Neff) Date: Mon, 27 Nov 2000 12:20:51 -0500 Subject: [Mailman-Developers] Re: MIME "digests" vs. real Digests In-Reply-To: <20001127170103.7A9AD1CDC9@dinsdale.python.org> Message-ID: <370075433.975327651@NY110-19-024B.bloomberg.com> Generating proper 1153 Digests is very important. MIME "Digests" are not true digests in the original sense of providing a readable daily omnibus of list traffic, they are a hack concocted by people who were thrilled with MIME and thought digests were just another way of concatenating messages and files. They don't save much space and they don't provide the same service; most modern mailers capable of "dealing" with them are equally capable of routing individual messages to folders based on headers, which is arguably the better solution for people who don't care about bandwidth or disk space to begin with. From chuqui@plaidworks.com Mon Nov 27 22:01:22 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 27 Nov 2000 14:01:22 -0800 Subject: [Mailman-Developers] fyi -- Ted Montgomery Message-ID: Ted Montgomery's lists of over and under-rated players on each team. Note his leaf's picks. Not that I'm making an editorial jab at anyone on THIS list. nope. Nope me. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From chuqui@plaidworks.com Mon Nov 27 22:13:23 2000 From: chuqui@plaidworks.com (Chuq Von Rospach) Date: Mon, 27 Nov 2000 14:13:23 -0800 Subject: [Mailman-Developers] fyi -- Ted Montgomery In-Reply-To: References: Message-ID: whoops. Sorry about that, folks. Right message, wrong list. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy From barry@digicool.com Mon Nov 27 22:27:38 2000 From: barry@digicool.com (Barry A. Warsaw) Date: Mon, 27 Nov 2000 17:27:38 -0500 Subject: [Mailman-Developers] fyi -- Ted Montgomery References: Message-ID: <14882.57306.927142.149194@anthem.concentric.net> >>>>> "CVR" == Chuq Von Rospach writes: CVR> whoops. Sorry about that, folks. Right message, wrong list. And I thought you were jabbing at me with this: Teams with the brightest future in the East: Tampa Bay and the Islanders. Teams with the bleakest future in the East: Carolina and Washington. go-caps!-ly y'rs, -Barry :) From jeb@ocha.net Tue Nov 28 00:51:30 2000 From: jeb@ocha.net (Jeb Bateman) Date: Mon, 27 Nov 2000 16:51:30 -0800 Subject: [Mailman-Developers] Bug fix? (was: [ANNOUNCE] Mailman 2.0) In-Reply-To: <14876.860.607056.18676@anthem.concentric.net>; from barry@digicool.com on Wed, Nov 22, 2000 at 12:33:16PM -0500 References: <14876.860.607056.18676@anthem.concentric.net> Message-ID: <20001127165129.W13647@freequotes.com> Hello, On Wed, Nov 22, 2000 at 12:33:16PM -0500, Barry A. Warsaw wrote: > I'm please to announce the 2.0 final release of Mailman I just installed 2.0 final over 2.0beta2, and all went well, except it generates broken URL paths (missing / between mailman/admin). The following snippet is from the owner notification message for a new list, and all the paths are broken (ie. mailmanlistinfo instead of mailman/listinfo) in the message, and in the headers... --- You can configure your mailing list at the following web page: http://qube.freequotes.com/mailmanadmin/bc-discussion --- Does anyone have a quick pointer where to fix this in the source? I'll look for it myself, but a pointer would still help... (Btw, Please Cc me, as I am not on this developers list.) Thanks, -jeb -- Jeb Bateman Internet Consultant 455 California Ave. Reno, NV 89509 Personal home page: http://jeb.ocha.net From marouni@earlham.edu Tue Nov 28 01:08:44 2000 From: marouni@earlham.edu (Nicholas Marouf) Date: Mon, 27 Nov 2000 20:08:44 -0500 Subject: [Mailman-Developers] Bug fix? (was: [ANNOUNCE] Mailman 2.0) References: <14876.860.607056.18676@anthem.concentric.net> <20001127165129.W13647@freequotes.com> Message-ID: <3A23059C.2F84F871@earlham.edu> Hi Jeb, I got around this by adding a slash at the end of url below in $prefix/Mailman/mm_cfg.py DEFAULT_URL = 'http://www.earlham.edu/mailman/' that fixed the problem. Nick Jeb Bateman wrote: > > Hello, > > On Wed, Nov 22, 2000 at 12:33:16PM -0500, Barry A. Warsaw wrote: > > I'm please to announce the 2.0 final release of Mailman > > I just installed 2.0 final over 2.0beta2, and all went well, except it > generates broken URL paths (missing / between mailman/admin). The > following snippet is from the owner notification message for a new > list, and all the paths are broken (ie. mailmanlistinfo instead of > mailman/listinfo) in the message, and in the headers... > > --- > You can configure your mailing list at the following web page: > > http://qube.freequotes.com/mailmanadmin/bc-discussion > --- > > Does anyone have a quick pointer where to fix this in the source? > I'll look for it myself, but a pointer would still help... (Btw, > Please Cc me, as I am not on this developers list.) > Thanks, > -jeb > > -- > Jeb Bateman > Internet Consultant > 455 California Ave. Reno, NV 89509 > Personal home page: http://jeb.ocha.net > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers -- Nicholas Marouf http://www.ramallahonline.com From heinrich@wh9.tu-dresden.de Tue Nov 28 11:26:44 2000 From: heinrich@wh9.tu-dresden.de (Heinrich Langos) Date: Tue, 28 Nov 2000 12:26:44 +0100 Subject: [Mailman-Developers] pipermail: subjects with "Re[2]:" are not recognized as reply Message-ID: <20001128122644.A15544@wh9.tu-dresden.de> hi there. seems like the "The Bat" mailer inserts a counter into the "Re:" subject line so that "Re[2]:blah blah" or "Re[3]: more blah blah" show up. these are not cought by Mailman/Archiver/HyperArch.py and therefore are not sorted into the thread that they belong to. this is the line in Mailman/Archiver/HyperArch.py that seems to be responsible for it. -------------------- # Subject lines preceded with 'Re:' REpat = re.compile( r"\s*RE\s*:\s*", re.IGNORECASE) -------------------- i don't know python but if the regex is like perl a change like this should do the job: ----------------- REpat = re.compile( r"\s*RE\s*(\[\d*\])??\s*:\s*", re.IGNORECASE) ----------------- as i said i don't have a clue about python and i don't want to test it right now since i have an active list and i don't want to miss any mails in the archive ... i would appreciate if somebody could check it out and tell me if it works. and tell me if i have to recompile those HyperArch.pyc and HyperArch.pyo files. cheers and thanx for that nice program. -heinrich ps: i run mailman 1.1 but i checked the CVS archive and the regex used for REpat seems to be the same. pps: i submitted this as a bug in sourceforge as well .. so if you fix it you can close it there too. ppps: i am not subscribed to mailman-developers@python.org so please include me in CC:'s -- Heinrich Langos pgp: http://wh9.tu-dresden.de/~heinrich/pub_pgp_key.asc _________________________________________________________________ |o| The reason we come up with new versions is not to fix bugs. |o| |o| It's absolutely not. It's the stupidest reason to buy a new |o| |o| version I ever heard. -- Bill Gates, Microsoft Corporation |o| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From Nigel.Metheringham@VData.co.uk Tue Nov 28 13:17:04 2000 From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham) Date: Tue, 28 Nov 2000 13:17:04 +0000 Subject: [Mailman-Developers] pipermail: subjects with "Re[2]:" are not recognized as reply In-Reply-To: Message from Heinrich Langos of "Tue, 28 Nov 2000 12:26:44 +0100." <20001128122644.A15544@wh9.tu-dresden.de> Message-ID: heinrich@wh9.tu-dresden.de said: > seems like the "The Bat" mailer inserts a counter into the "Re:" > subject line so that "Re[2]:blah blah" or "Re[3]: more blah blah" show > up. these are not cought by Mailman/Archiver/HyperArch.py and > therefore are not sorted into the thread that they belong to. What goes around, comes around. This came up in 94/95 or so with the nn newsreader. After an outcry and much quoting of RFCs nn backed down and stopped doing it. I suggest someone deal with the author of "The bat" in the same way. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham@VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] From heinrich@wh9.tu-dresden.de Tue Nov 28 14:37:07 2000 From: heinrich@wh9.tu-dresden.de (Heinrich Langos) Date: Tue, 28 Nov 2000 15:37:07 +0100 Subject: [Mailman-Developers] pipermail: subjects with "Re[2]:" are not recognized as reply In-Reply-To: ; from Nigel.Metheringham@VData.co.uk on Tue, Nov 28, 2000 at 01:17:04PM +0000 References: Message-ID: <20001128153707.C15544@wh9.tu-dresden.de> On Tue, Nov 28, 2000 at 01:17:04PM +0000, Nigel Metheringham wrote: > > heinrich@wh9.tu-dresden.de said: > > seems like the "The Bat" mailer inserts a counter into the "Re:" > > subject line so that "Re[2]:blah blah" or "Re[3]: more blah blah" show > > up. these are not cought by Mailman/Archiver/HyperArch.py and > > therefore are not sorted into the thread that they belong to. > > What goes around, comes around. This came up in 94/95 or so with the > nn newsreader. After an outcry and much quoting of RFCs nn backed down > and stopped doing it. I suggest someone deal with the author of "The > bat" in the same way. > > Nigel. i've found a way for The Bat users to disable that irritating behavior. http://www.mail-archive.com/tbudl@thebat.dutaint.com/msg16021.html could you point me to an archive of that discussion? preferably with pointers to the RFCs that this behavior violates? i would like to contact the authors but not without some background information. thanx in advance -heinrich -- Heinrich Langos pgp: http://wh9.tu-dresden.de/~heinrich/pub_pgp_key.asc _________________________________________________________________ |o| The reason we come up with new versions is not to fix bugs. |o| |o| It's absolutely not. It's the stupidest reason to buy a new |o| |o| version I ever heard. -- Bill Gates, Microsoft Corporation |o| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From claw@kanga.nu Tue Nov 28 18:19:04 2000 From: claw@kanga.nu (J C Lawrence) Date: Tue, 28 Nov 2000 10:19:04 -0800 Subject: [Mailman-Developers] pipermail: subjects with "Re[2]:" are not recognized as reply In-Reply-To: Message from Heinrich Langos of "Tue, 28 Nov 2000 12:26:44 +0100." <20001128122644.A15544@wh9.tu-dresden.de> References: <20001128122644.A15544@wh9.tu-dresden.de> Message-ID: <23723.975435544@kanga.nu> On Tue, 28 Nov 2000 12:26:44 +0100 Heinrich Langos wrote: > seems like the "The Bat" mailer inserts a counter into the "Re:" > subject line so that "Re[2]:blah blah" or "Re[3]: more blah blah" > show up. these are not cought by Mailman/Archiver/HyperArch.py > and therefore are not sorted into the thread that they belong to. LotusNotes does (or did, or can do, I'm not sure which) the same thing. -- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- From exim-users@exim.org Tue Nov 28 23:42:27 2000 From: exim-users@exim.org (Jeffrey C. Ollie) Date: Tue, 28 Nov 2000 17:42:27 -0600 Subject: [Mailman-Developers] Python embedded in Exim Message-ID: <20001128174227.A24208@ollie.clive.ia.us> --aM3YZ0Iwxop3KEKx Content-Type: multipart/mixed; boundary="FL5UXtIhxfXey3p5" Content-Disposition: inline --FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Well, I had an epiphany over the US Thanksgiving holiday weekend (or maybe it was a brain aneurism) so I've spent some late nights and some time stolen from work and come up with a first pass at embedding Python in Exim. I imagine that embedding Python in Exim will be interesting to those folks writing virus scanners or for VERY tight integration of Mailman with Exim. You can also do all of those things that you can do with the Perl string expansions. Note: this code is VERY lightly tested. It compiles on my home hacked-up RedHat Linux system and works on a couple of simple tests that I've done, but no real e-mails have passed through this code yet. I'm tossing this out for public comment on the design and implementation... It's been a while since I've done a lot of C coding. Unfortunately, you must embed Python 2.0. This is because previous versions of Python used a hacked copy of PCRE as it's regular expression engine that conflicts with the copy of PCRE in Exim. Starting in Python 2.0 the default regular expression is a completely new one called SRE that supports Unicode. The PCRE-based regular expression engine is still included in Python 2.0 but I get around that by creating a private copy of the Python library and delete the offending object modules. You could do that in versions of Python prior to 2.0 but a lot of the standard library depends on regular expressions I didn't think that there was much point in trying to make the embedding work with 1.5.2 or 1.6. Well, on to the patch itself. The patch is made against Exim v3.20. After you've patched the source code, you need to set four variables in the Local/Makefile: EXIM_PYTHON=python.o PYTHON_INC=-I/usr/local/include/python2.0 PYTHON_LIB=/usr/local/lib/python2.0/config/libpython2.0.a PYTHON_EXTRA_LIBS=-lpthread -ldl -lutil Then build Exim as usual. There are three runtime directives that control the embedding, all of which are optional: python_at_start boolean - force startup of Python interpreter python_module_paths colon separated list of paths to append to sys.path python_initial_import colon separated list of modules to import at interpreter initialization time There are also two new command line switches -ys and -yd that will force an immediate startup of Python or delay startup, overriding python_at_start. Then you use it: ${python{.}{}...} You must specify the module name and the function name. The named module will be imported automatically. And currently you must specify a function that returns a string (I'll look into adding accessing attributes, methods of classes, and converting non-strings into strings). There can be up to eight arguments. Each argument will be expanded by Exim before it's passed to the Python interpreter. Your python code has access to a module called "exim", which currently defines two things: exim.expand_string(s) which calls back to Exim to expand a string and exim.error which is an exception object used by expand_string One simple example: ${python{string.upper}{$local_part}} Error reporting isn't the best right now, I'm going to look into formatting exceptions and tracebacks for more meaningful error messages. Well, I think that's the gist of it. I'll work on patches to the documentation when this thing becomes more stable. As I said before I've only lightly tested this... beware! Comments, criticisms, suggestions, flames welcome... Jeff --FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="exim-python.patch" Content-Transfer-Encoding: quoted-printable Index: exim/OS/Makefile-Base diff -u exim/OS/Makefile-Base:1.1.1.1 exim/OS/Makefile-Base:1.1.1.1.2.3 --- exim/OS/Makefile-Base:1.1.1.1 Mon Nov 27 08:18:15 2000 +++ exim/OS/Makefile-Base Tue Nov 28 16:49:25 2000 @@ -192,6 +192,16 @@ @chmod a+x ../util/convert4r3 @echo ">>> convert4r3 script built in util directory"; echo "" =20 +libpython.a: $(PYTHON_LIB) + if [ -n $(PYTHON_LIB) ] ;\ + then \ + cp $(PYTHON_LIB) libpython.a ;\ + ar d libpython.a pcremodule.o pypcre.o ;\ + ranlib libpython.a ;\ + else \ + ar cq libpython.a ;\ + ranlib libpython.a ;\ + fi =20 # Targets for final binaries; the main one has a build number which is # updated each time. We don't bother with that for the auxiliaries. @@ -201,11 +211,12 @@ header.o host.o log.o match.o moan.o os.o parse.o queue.o \ readconf.o retry.o rewrite.o \ route.o search.o smtp_in.o smtp_out.o spool_in.o spool_out.o \ - store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PE= RL) + store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PE= RL) \ + $(EXIM_PYTHON) =20 exim: libident/libident.a pcre/libpcre.a lookups/lookups.a auths/auths.a= \ directors/directors.a routers/routers.a transports/transports.a \ - $(OBJ_EXIM) version.c + $(OBJ_EXIM) version.c libpython.a awk '{ print ($$1+1) }' cnumber.h > cnumber.temp /bin/rm -f cnumber.h; mv cnumber.temp cnumber.h $(CC) -c $(CFLAGS) $(INCLUDE) $(IPV6_INCLUDE) $(TLS_INCLUDE) version.c @@ -215,7 +226,8 @@ routers/routers.a transports/transports.a lookups/lookups.a \ auths/auths.a \ $(LIBS) $(LIBS_EXIM) $(IPV6_LIBS) $(EXTRALIBS) $(EXTRALIBS_EXIM) \ - $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) $(TLS_LIBS) + $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) libpython.a $(PYTHON= _EXTRA_LIBS) \ + $(TLS_LIBS) $(EXIM_CHMOD) @echo " " @echo ">>> exim binary built" @@ -316,6 +328,11 @@ =20 perl.o: $(HDRS) perl.c $(PERL_CC) $(PERL_CCOPTS) $(CFLAGS) $(INCLUDE) -c perl.c + +# Build instructions for python.o for when EXIM_PYTHON is set + +python.o: $(HDRS) python.c + $(CC) -c $(CFLAGS) $(INCLUDE) $(PYTHON_INC) -I. python.c =20 # Dependencies for the "ordinary" exim modules =20 Index: exim/scripts/MakeLinks diff -u exim/scripts/MakeLinks:1.1.1.1 exim/scripts/MakeLinks:1.1.1.1.2.1 --- exim/scripts/MakeLinks:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/scripts/MakeLinks Mon Nov 27 08:27:46 2000 @@ -189,6 +189,7 @@ ln -s ../src/moan.c moan.c ln -s ../src/parse.c parse.c ln -s ../src/perl.c perl.c +ln -s ../src/python.c python.c ln -s ../src/queue.c queue.c ln -s ../src/readconf.c readconf.c ln -s ../src/retry.c retry.c Index: exim/src/EDITME diff -u exim/src/EDITME:1.1.1.2 exim/src/EDITME:1.1.1.2.2.4 --- exim/src/EDITME:1.1.1.2 Mon Nov 27 08:18:54 2000 +++ exim/src/EDITME Tue Nov 28 16:20:19 2000 @@ -260,6 +260,10 @@ # PERL_CCOPTS=3D # PERL_LIBS=3D =20 +# EXIM_PYTHON=3Dpython.o +# PYTHON_INC=3D-I/usr/local/include/python2.0 +# PYTHON_LIB=3D/usr/local/lib/python2.0/config/libpython2.0.a +# PYTHON_EXTRA_LIBS=3D-lpthread -ldl -lutil =20 # This parameter sets the maximum length of the header portion of a message # that Exim is prepared to process. The default setting is one megabyte. T= here Index: exim/src/config.h.defaults diff -u exim/src/config.h.defaults:1.1.1.1 exim/src/config.h.defaults:1.1.1= .1.2.1 --- exim/src/config.h.defaults:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/src/config.h.defaults Mon Nov 27 08:43:04 2000 @@ -36,6 +36,8 @@ =20 #define EXIM_PERL =20 +#define EXIM_PYTHON + #define HAVE_SA_LEN #define HEADER_MAXSIZE (1024*1024) =20 Index: exim/src/exim.c diff -u exim/src/exim.c:1.1.1.2 exim/src/exim.c:1.1.1.2.2.4 --- exim/src/exim.c:1.1.1.2 Mon Nov 27 08:18:54 2000 +++ exim/src/exim.c Tue Nov 28 00:46:52 2000 @@ -351,6 +351,9 @@ #ifdef EXIM_PERL int perl_start_option =3D 0; #endif +#ifdef EXIM_PYTHON +int python_start_option =3D 0; +#endif int recipients_arg =3D argc; int sender_address_domain =3D 0; int test_retry_arg =3D -1; @@ -583,6 +586,17 @@ simply recorded for checking and handling afterwards. Do a high-level swit= ch on the second character (the one after '-'), to save some effort. */ =20 +#ifdef EXIM_PYTHON +opt_python_original_argc =3D argc; +opt_python_original_argv =3D store_get(sizeof(char *) * (argc + 1)); +opt_python_original_argv[argc] =3D NULL; + +for (i =3D 0; i < argc; i++) + { + opt_python_original_argv[i] =3D string_copy(argv[i]); + } +#endif + for (i =3D 1; i < argc; i++) { BOOL badarg =3D FALSE; @@ -1521,6 +1535,17 @@ break; #endif =20 + /* -ys: force Python startup; -yd force delayed Python startup */ + + #ifdef EXIM_PYTHON + case 'y': + if (*argrest =3D=3D 's' && argrest[1] =3D=3D 0) python_start_option = =3D 1; + else + if (*argrest =3D=3D 'd' && argrest[1] =3D=3D 0) python_start_option = =3D -1; + else badarg =3D TRUE; + break; + #endif + =20 case 'q': =20 @@ -1967,6 +1992,27 @@ opt_perl_started =3D TRUE; } #endif /* EXIM_PERL */ + +/* Start up Python interpreter if Python support is configured and +there is a and the configuration or the command line specifies +initializing starting. Note that the global variables are actually +called opt_python_xxx. */ + +#ifdef EXIM_PYTHON +if (python_start_option !=3D 0) + opt_python_at_start =3D (python_start_option > 0); +if (opt_python_at_start) + { + char *errstr; + DEBUG(9) debug_printf("Starting Python interpreter\n"); + errstr =3D init_python(); + if (errstr !=3D NULL) + { + fprintf(stderr, "exim: error during Python startup: %s\n", errstr); + return EXIT_FAILURE; + } + } +#endif /* EXIM_PYTHON */ =20 /* Log the arguments of the call if the configuration file said so. This is a debugging feature for finding out what arguments certain MUAs actually u= se. Index: exim/src/expand.c diff -u exim/src/expand.c:1.1.1.2 exim/src/expand.c:1.1.1.2.2.3 --- exim/src/expand.c:1.1.1.2 Mon Nov 27 08:18:54 2000 +++ exim/src/expand.c Tue Nov 28 00:23:10 2000 @@ -1318,7 +1318,7 @@ =20 Operators: domain extracts domain from an address - escape escapes non-printing characters + escape escapes non-printing characters=20 expand expands the string one more time hash__ hash the string, making one that is of length n, using m (default 26) characters from hashcodes @@ -1865,6 +1865,100 @@ continue; } #endif /* EXIM_PERL */ + + /* If Perl support is configured, handle calling embedded perl subroutin= es, + unless locked out at this time. Syntax is ${perl{sub}} or ${perl{sub}{ar= g}} + or ${perl{sub}{arg1}{arg2}} or up to a maximum of EXIM_PERL_MAX_ARGS + arguments (defined below). */ + + #ifdef EXIM_PYTHON + #define EXIM_PYTHON_MAX_ARGS 8 + + if (strcmp(name, "python") =3D=3D 0) + { + int i =3D 0; + char *sub_name; + char *sub_arg[EXIM_PYTHON_MAX_ARGS + 1]; + char *new_yield; + + if (expand_forbid_python) + { + expand_string_message =3D "Python calls are not permitted"; + goto EXPAND_FAILED; + } + + while (isspace((uschar)*s)) s++; + if (*s !=3D '{') goto EXPAND_FAILED_CURLY; + sub_name =3D expand_string_internal(s+1, TRUE, &s, FALSE); + if (sub_name =3D=3D NULL) goto EXPAND_FAILED; + while (isspace((uschar)*s)) s++; + if (*s++ !=3D '}') goto EXPAND_FAILED_CURLY; + + while (isspace((uschar)*s)) s++; + + while (*s =3D=3D '{') + { + if (i =3D=3D EXIM_PYTHON_MAX_ARGS) + { + expand_string_message =3D + string_sprintf("Too many arguments passed to Python subroutine \= "%s\" " + "(max is %d)", sub_name, EXIM_PYTHON_MAX_ARGS); + goto EXPAND_FAILED; + } + sub_arg[i] =3D expand_string_internal(s+1, TRUE, &s, FALSE); + if (sub_arg[i++] =3D=3D NULL) goto EXPAND_FAILED; + while (isspace((uschar)*s)) s++; + if (*s++ !=3D '}') goto EXPAND_FAILED_CURLY; + while (isspace((uschar)*s)) s++; + } + + if (*s++ !=3D '}') goto EXPAND_FAILED_CURLY; + sub_arg[i] =3D 0; + + /* Start the interpreter if necessary */ + + if (!opt_python_started) + { + char *initerror; + DEBUG(9) debug_printf("Starting Python interpreter\n"); + initerror =3D init_python(); + if (initerror !=3D NULL) + { + expand_string_message =3D + string_sprintf("error during Python startup: %s\n", initerror); + goto EXPAND_FAILED; + } + } + + /* Call the function */ + + new_yield =3D call_python_cat(yield, &size, &ptr, &expand_string_messa= ge, + sub_name, sub_arg); + + /* NULL yield indicates failure; if the message pointer has been set to + NULL, the yield was undef, indicating a forced failure. Otherwise the + message will indicate some kind of Perl error. */ + + if (new_yield =3D=3D NULL) + { + if (expand_string_message =3D=3D NULL) + { + expand_string_message =3D + string_sprintf("Python subroutine \"%s\" returned undef to force= " + "failure", sub_name); + expand_string_forcedfail =3D TRUE; + } + goto EXPAND_FAILED; + } + + /* Yield succeeded. Ensure forcedfail is unset, just in case it got + set during a callback from Python. */ + + expand_string_forcedfail =3D FALSE; + yield =3D new_yield; + continue; + } + #endif /* EXIM_PYTHON */ =20 /* Handle character translation for "tr" */ =20 Index: exim/src/functions.h diff -u exim/src/functions.h:1.1.1.2 exim/src/functions.h:1.1.1.2.2.1 --- exim/src/functions.h:1.1.1.2 Mon Nov 27 08:18:54 2000 +++ exim/src/functions.h Mon Nov 27 10:13:25 2000 @@ -16,6 +16,12 @@ extern char *init_perl(char *); #endif =20 +#ifdef EXIM_PYTHON +extern char *call_python_cat(char *, int *, int *, char **, char *, char *= *); +extern void cleanup_python(void); +extern char *init_python(void); +#endif + #ifdef SUPPORT_TLS extern BOOL tls_client_start(int, host_item *, address_item *, char *, char *, char *, char *, char *, int); Index: exim/src/globals.c diff -u exim/src/globals.c:1.1.1.1 exim/src/globals.c:1.1.1.1.2.2 --- exim/src/globals.c:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/src/globals.c Mon Nov 27 11:10:15 2000 @@ -35,6 +35,15 @@ BOOL opt_perl_started =3D FALSE; #endif =20 +#ifdef EXIM_PYTHON +BOOL opt_python_at_start =3D FALSE; +char *opt_python_module_paths =3D NULL; +char *opt_python_initial_import =3D NULL; +BOOL opt_python_started =3D FALSE; +int opt_python_original_argc =3D 0; +char **opt_python_original_argv =3D NULL; +#endif + #ifdef HAVE_AUTH BOOL auth_always_advertise =3D TRUE; char *auth_hosts =3D NULL; @@ -310,6 +319,7 @@ BOOL expand_forbid_exists =3D FALSE; BOOL expand_forbid_lookup =3D FALSE; BOOL expand_forbid_perl =3D FALSE; +BOOL expand_forbid_python =3D FALSE; int expand_nlength[EXPAND_MAXN+1]; int expand_nmax =3D -1; char *expand_nstring[EXPAND_MAXN+1]; Index: exim/src/globals.h diff -u exim/src/globals.h:1.1.1.1 exim/src/globals.h:1.1.1.1.2.2 --- exim/src/globals.h:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/src/globals.h Mon Nov 27 11:10:15 2000 @@ -21,6 +21,15 @@ extern BOOL opt_perl_started; /* Set once interpreter started */ #endif =20 +#ifdef EXIM_PYTHON +extern BOOL opt_python_at_start; /* start Python at startup */ +extern char *opt_python_module_paths; /* list of paths to append to sys= .path */ +extern char *opt_python_initial_import; /* list of modules to import at i= nterpreter initialization time */ +extern BOOL opt_python_started; /* set once Python interpreter st= arted */ +extern int opt_python_original_argc; +extern char **opt_python_original_argv; +#endif + #ifdef HAVE_AUTH extern BOOL auth_always_advertise; /* If FALSE, advertise only when nee= ded */ extern char *auth_hosts; /* These must authenticate */ @@ -208,6 +217,7 @@ extern BOOL expand_forbid_exists; /* Lock out exists checking */ extern BOOL expand_forbid_lookup; /* Lock out lookups */ extern BOOL expand_forbid_perl; /* Lock out Perl calls */ +extern BOOL expand_forbid_python; /* Lock out Python calls */ extern int expand_nlength[]; /* Lengths of numbered strings */ extern int expand_nmax; /* Max numerical value */ extern char *expand_nstring[]; /* Numbered strings */ Index: exim/src/python.c diff -u /dev/null exim/src/python.c:1.1.2.11 --- /dev/null Tue Nov 28 16:50:15 2000 +++ exim/src/python.c Tue Nov 28 16:22:03 2000 @@ -0,0 +1,291 @@ +#include "exim.h" + +#include "Python.h" + +static void init_exim_module(void); + +char *init_python(void) +{ + if (opt_python_started) + { + return NULL; + } + + Py_SetProgramName(opt_python_original_argv[0]); + =20 + Py_Initialize(); + =20 + PySys_SetArgv(opt_python_original_argc, opt_python_original_argv); + + init_exim_module(); + + if (opt_python_module_paths) + { + char *listptr =3D opt_python_module_paths; + int separator =3D 0; + char buffer[256]; + PyObject *path; + PyObject *sys_module; + PyObject *sys_dict; + PyObject *sys_path; + PyObject *name; + + name =3D PyString_FromString("sys"); + if (name =3D=3D NULL) + { + PyErr_Clear(); + return "problem creating string \"sys\""; + } + + sys_module =3D PyImport_Import(name); + Py_DECREF(name); + + if (sys_module =3D=3D NULL) + { + PyErr_Clear(); + return "problem trying to import sys"; + } + + sys_dict =3D PyModule_GetDict(sys_module); + if (sys_dict =3D=3D NULL) + { + PyErr_Clear(); + return "problem trying get dictionary from module sys"; + } + + sys_path =3D PyDict_GetItemString(sys_dict, "path"); + if (sys_path =3D=3D NULL) + { + PyErr_Clear(); + return "problem trying to get sys.path"; + } + + while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)= )) + { + path =3D PyString_FromString(buffer); + if (path =3D=3D NULL) + { + PyErr_Clear(); + return string_sprintf("problem while allocating Python string for \"%= s\"", buffer); + } + + if(!PyList_Append(sys_path, path)) + { + PyErr_Clear(); + return string_sprintf("problem while insering string \"%s\"into sys.p= ath", buffer); + } + } + } + + if (opt_python_initial_import) + { + char *listptr =3D opt_python_initial_import; + int separator =3D 0; + char buffer[256]; + PyObject *module; + PyObject *name; + + while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)= )) + { + name =3D PyString_FromString(buffer); + if (name =3D=3D NULL) + { + PyErr_Clear(); + return string_sprintf("problem creating string \"%s\"", buffer); + } + + module =3D PyImport_Import(name); + Py_DECREF(name); + + if (module =3D=3D NULL) + { + PyErr_Clear(); + return string_sprintf("problem importing module %s", buffer); + } + } + } + + =20 + opt_python_started =3D TRUE; + return NULL; +} + +void cleanup_python(void) +{ + Py_Finalize(); +} + +static PyObject *find_function(char *name, char **errstrp) +{ + PyObject *module_name; + PyObject *function_name; + PyObject *module; + PyObject *dictionary; + PyObject *result; + + char *ptr; + + ptr =3D strrchr(name, '.'); + =20 + if (ptr =3D=3D NULL) + { + *errstrp =3D string_sprintf("function name must include module: ."); + return NULL; + } + + function_name =3D PyString_FromString(ptr + 1); + + if (function_name =3D=3D NULL) + { + PyErr_Clear(); + *errstrp =3D string_sprintf("error trying to allocate function name"= ); + return NULL; + } + + module_name =3D PyString_FromStringAndSize(name, ptr - name); + + if (function_name =3D=3D NULL) + { + PyErr_Clear(); + Py_DECREF(function_name); + *errstrp =3D string_sprintf("error trying to allocate module name"); + return NULL; + } + + module =3D PyImport_Import(module_name); + + Py_DECREF(module_name); + + if (module =3D=3D NULL) + { + PyErr_Clear(); + Py_DECREF(function_name); + *errstrp =3D string_sprintf("error trying to import module while try= ing to find %s", name); + return NULL; + } + + dictionary =3D PyModule_GetDict(module); + + result =3D PyDict_GetItem(dictionary, function_name); + + Py_DECREF(function_name); + + if (!PyCallable_Check(result)) + { + PyErr_Clear(); + *errstrp =3D string_sprintf("%s is not a callable object", name); + return NULL; + } + + return result; +} + +char *call_python_cat(char *yield, int *sizep, int *ptrp, char **errstrp, + char *name, char **arg) +{ + PyObject *function; + PyObject *arguments; + PyObject *result; + int index; + int count=3D0; + char **p; + char *buffer; + int length; + + function =3D find_function(name, errstrp); + + if (function =3D=3D NULL) + return NULL; + + p =3D arg; + + while(*p) + { + count++; + p++; + } + + arguments =3D PyTuple_New(count); + + index =3D 0; + while(index < count) + { + + PyTuple_SetItem(arguments, index, PyString_FromString(arg[index])); + index++; + } + + result =3D PyObject_CallObject(function, arguments); + + Py_DECREF(arguments); + + if (result =3D=3D NULL) + { + PyErr_Clear(); + *errstrp =3D string_sprintf("error in Python function %s", name); + return NULL; + } + + if(!PyString_Check(result)) + { + Py_DECREF(result); + *errstrp =3D string_sprintf("Python function %s returned non-string"= , name); + return NULL; + } + + if(PyString_AsStringAndSize(result, &buffer, &length)) + { + PyErr_Clear(); + Py_DECREF(result); + *errstrp =3D string_sprintf("could not extract the results of Python= function %s", name); + return NULL; + } + + yield =3D string_cat(yield, sizep, ptrp, buffer, length); + + Py_DECREF(result); + + return yield; +} + +static PyObject *exim_error =3D NULL; + +static PyObject *exim_expand_string(PyObject *self, PyObject *args) +{ + char *i; + char *o; + + if(!PyArg_ParseTuple(args, "s", i)) + return NULL; + =20 + o =3D expand_string(i); + =20 + if (o =3D=3D NULL) + { + PyErr_SetString(exim_error, expand_string_message); + return NULL; + } + =20 + return PyString_FromString(o); +} + +static PyMethodDef exim_methods[] =3D { + {"expand_string", exim_expand_string, 1}, + {NULL, NULL} /* sentinel */ +}; + +static void init_exim_module(void) +{ + PyObject *m; + PyObject *d; + PyObject *o; + + m =3D PyImport_AddModule("exim"); + d =3D PyModule_GetDict(m); + + Py_InitModule("exim", exim_methods); + + exim_error =3D PyErr_NewException("exim.error", NULL, NULL); + PyDict_SetItemString(d, "error", exim_error); +} + Index: exim/src/readconf.c diff -u exim/src/readconf.c:1.1.1.1 exim/src/readconf.c:1.1.1.1.2.2 --- exim/src/readconf.c:1.1.1.1 Mon Nov 27 08:18:16 2000 +++ exim/src/readconf.c Mon Nov 27 10:14:17 2000 @@ -160,6 +160,11 @@ { "perl_at_start", opt_bool, &opt_perl_at_start }, { "perl_startup", opt_stringptr, &opt_perl_startup }, #endif +#ifdef EXIM_PYTHON + { "python_at_start", opt_bool, &opt_python_at_start }, + { "python_module_paths", opt_stringptr, &opt_python_module_paths = }, + { "python_initial_import", opt_stringptr, &opt_python_initial_impor= t }, +#endif #ifdef LOOKUP_PGSQL { "pgsql_servers", opt_stringptr, &pgsql_servers }, #endif --FL5UXtIhxfXey3p5-- --aM3YZ0Iwxop3KEKx Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6JELjmCSL+rEqeRsRArY9AJ9iw1RL79TcwrKoRxLCFGSvJ/hPqgCglwRc JfzWMV388u0Hohv9S1xDxcE= =430v -----END PGP SIGNATURE----- --aM3YZ0Iwxop3KEKx-- From jam@jamux.com Wed Nov 29 15:57:19 2000 From: jam@jamux.com (John A. Martin) Date: Wed, 29 Nov 2000 10:57:19 -0500 Subject: [Mailman-Developers] Possible cookie problem Message-ID: <20001129155720.0BECF4800B@athene.jamux.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I ain't got no clue. I access with no problem, but I use a different "HTTP_USER_AGENT". :-) - - -------------- cut here ---->8 ---< head logs/error Nov 29 07:35:21 2000 admin(26929): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ admin(26929): [----- Mailman Version: 2.0 -----] admin(26929): [----- Traceback ------] admin(26929): Traceback (innermost last): admin(26929): File "/home/mailman/scripts/driver", line 96, in run_main admin(26929): main() admin(26929): File "/home/mailman/Mailman/Cgi/admin.py", line 82, in main admin(26929): Auth.authenticate(mlist, cgidata) admin(26929): File "/home/mailman/Mailman/Cgi/Auth.py", line 72, in authenticate admin(26929): isauthed = mlist.WebAuthenticate(password=adminpw, cookie='admin') admin(26929): File "/home/mailman/Mailman/SecurityManager.py", line 68, in WebAuthenticate admin(26929): return self.CheckCookie(key) admin(26929): File "/home/mailman/Mailman/SecurityManager.py", line 117, in CheckCookie admin(26929): c = Cookie.Cookie(cookiedata) admin(26929): File "/home/mailman/Mailman/Cookie.py", line 509, in __init__ admin(26929): if input: self.load(input) admin(26929): File "/home/mailman/Mailman/Cookie.py", line 546, in load admin(26929): self.__ParseString(rawdata) admin(26929): File "/home/mailman/Mailman/Cookie.py", line 573, in __ParseString admin(26929): M.set(K, apply(self.net_setfunc, (V,)), V) admin(26929): File "/home/mailman/Mailman/Cookie.py", line 421, in set admin(26929): raise CookieError("Attempt to set a reserved key: %s" % key) admin(26929): CookieError: Attempt to set a reserved key: path admin(26929): [----- Python Information -----] admin(26929): sys.version = 1.5.2 (#1, Feb 1 2000, 16:32:16) [GCC egcs-2.91.66 19990314/Linux (egcs- admin(26929): sys.executable = /usr/bin/python admin(26929): sys.prefix = /usr admin(26929): sys.exec_prefix= /usr admin(26929): sys.path = /usr admin(26929): sys.platform = linux-i386 admin(26929): [----- Environment Variables -----] admin(26929): DOCUMENT_ROOT: /home/listproc/httpd/html/ admin(26929): SERVER_ADDR: 216.0.124.17 admin(26929): SERVER_PORT: 80 admin(26929): PATH_TRANSLATED: /home/listproc/httpd/html/hague-jur-commercial-law/ admin(26929): GATEWAY_INTERFACE: CGI/1.1 admin(26929): HTTP_USER_AGENT: Microsoft Internet Explorer/4.40.426 (Windows 95) admin(26929): REMOTE_ADDR: 24.19.174.74 admin(26929): SERVER_NAME: lists.essential.org admin(26929): SCRIPT_FILENAME: /home/mailman/cgi-bin/admin admin(26929): HTTP_ACCEPT: www/source, text/html, video/mpeg, image/jpeg, image/x-tiff, image/x-rgb, image/x-xbm, image/gif, */*, application/postscript admin(26929): REQUEST_URI: /mailman/admin/hague-jur-commercial-law/ admin(26929): QUERY_STRING: admin(26929): SERVER_PROTOCOL: HTTP/1.0 admin(26929): PATH_INFO: /hague-jur-commercial-law/ admin(26929): HTTP_HOST: lists.essential.org admin(26929): REQUEST_METHOD: GET admin(26929): SERVER_SIGNATURE:

Apache/1.3.14 Server at lists.essential.org Port 80
admin(26929): SCRIPT_NAME: /mailman/admin admin(26929): SERVER_ADMIN: root@localhost admin(26929): SERVER_SOFTWARE: Apache/1.3.14 (Unix) (Red-Hat/Linux) mod_perl/1.23 admin(26929): PYTHONPATH: /home/mailman admin(26929): HTTP_COOKIE: UID=7E6266E13A24F713; path=/; domain=.excite.com; expires=Wednesday, 31-Dec-2010 12:00:00 GMT admin(26929): REMOTE_PORT: 1791 - - ---- 8<------- cut here ----------> tail logs/error jam -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: OpenPGP encrypted mail preferred. See iEYEARECAAYFAjolJwsACgkQUEvv1b/iXy/rAgCgkqY4ep4vv6ViO6hjjZvOFqqU IeIAnRdzPPGUFha5TCY8r6PLm+y5JO5n =4QsZ -----END PGP SIGNATURE----- From Dan Mick Thu Nov 30 01:16:14 2000 From: Dan Mick (Dan Mick) Date: Wed, 29 Nov 2000 17:16:14 -0800 (PST) Subject: [Mailman-Developers] offtopic: CVS checkout to `pwd` Message-ID: <200011300114.RAA24615@utopia.west.sun.com> This is a question about CVS. I'm constantly creating a directory that I want Mailman to go into, for example /local/src/mailman-pristine, and then realizing that I can't make CVS actually put Mailman *into* that directory. I have to use "checkout mailman" to get it, and that insists on creating a 'mailman' directory. If I move everything from mailman to ./, things work just fine from then on, but isn't there any way to make the checkout command use the current dir? The obvious option, "checkout -d ", gives me an error if I use '.' or an absolute pathname: protocol error: is not absolute Can anyone else do this, or is this just part of Life with CVS?