From r.barrett at ftel.co.uk  Fri Aug  1 01:03:55 2003
From: r.barrett at ftel.co.uk (Richard Barrett)
Date: Thu Jul 31 19:04:08 2003
Subject: [Mailman-Developers] a template "arrrgh...."
In-Reply-To: <E19iL7A-0006Rg-00@mail.islandnet.com>
Message-ID: <42656828-C3AB-11D7-88C4-000A957C9A50@ftel.co.uk>


On Thursday, July 31, 2003, at 10:37  pm, Ron Brogden wrote:

> On July 31, 2003 01:58 pm, you wrote:
>> There is a known bug in MM 2.1.2 (and earlier 2.1.x) for which a patch
>> is available. This patch has been folded into the CVS and should thus
>> appear in the next release of MM (2.1.3?) but for the moment it is
>> applicable to 2.1.2; see:
>> http://sourceforge.net/tracker/
>> index.php?func=detail&aid=730769&group_id=103&atid=100103
>
> Hello Richard.  Thanks for the tip but now I have a new problem (ain't 
> that
> always the case).
>
> =)
>
> I tried installing the patch but it seemed to have messed up the 
> template
> parsing (I was suddenly left with unparsed tags such as "%(meta)s"
> everywhere.  I quickly reverted back to the unmodified code but now it
> appears that this data has been physically cached somewhere so now I 
> have the
> problem of not being able to clear out the stale cache.
>
> Any idea where this cache is stored?  My archives will shortly be 
> useless if
> I cannot catch this quickly (and a rebuild is nasty, nasty as this is 
> a busy
> server).
>

mailmanctl restart after making the code change. The cache is in 
process memory and is freshly created on demand within each process 
each time the mailman daemons or scripts are started. You will also 
need to rebuild existing mail archives using bin/arch to get the 
revised templates to affect existing HTML archive pages.

> Arrrgh.
>
> =)
>
> Cheers


From r.barrett at openinfo.co.uk  Fri Aug  1 01:52:55 2003
From: r.barrett at openinfo.co.uk (Richard Barrett)
Date: Thu Jul 31 19:53:09 2003
Subject: [Mailman-Developers] more template shenanigans
In-Reply-To: <E19iMy9-0007Rk-00@mail.islandnet.com>
Message-ID: <1B0FECCF-C3B2-11D7-88C4-000A957C9A50@openinfo.co.uk>


On Friday, August 1, 2003, at 12:36  am, Ron Brogden wrote:

> On July 31, 2003 04:03 pm, you wrote:
>> mailmanctl restart after making the code change. The cache is in
>> process memory and is freshly created on demand within each process
>> each time the mailman daemons or scripts are started. You will also
>> need to rebuild existing mail archives using bin/arch to get the
>> revised templates to affect existing HTML archive pages.
>
> Howdy.  Ok, I now understand a bit more as to what is going on.   I had
> forgotten that Mailman is now effectively resident and that these 
> details
> would be sitting internally in the queue runner process.  When stuff 
> lingered
> around I figured there was a disk cache involved somewhere buried in 
> the
> code. My bad.
>
> I think that the previous "fix" actually did work but I inadvertently 
> came
> across a separate issue which obscured the real problem.  The mangled
> template was a separate issue altogether (removing all language files 
> would
> have made this a moot point anyway).
>
> Here goes. . .
>
> I have edited archtoc.html (as well as the other templates) to give the
> archive a unified look.  After doing so, the archive templates no 
> longer get
> parsed properly (i.e. none of the placeholders get replaced).  Looking 
> at the
> source, the only immediate candidate is the "%" symbol in some of the 
> table
> tags (i.e. "width=100%") but I am not clear yet on how the conversions 
> are
> done.  I know that a dictionary is passed to maketext() which is used 
> but
> what character exactly is screwing up the tag parsing I am unsure.
>

To get a single % use %%. For example, running python from the command 
line this fails:

 >>> t = '%(fred)s:%(bert)s x="100%"'
 >>> v = {'fred': 'abc', 'bert': 'xyz'}
 >>> t % v
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
TypeError: not enough arguments for format string

but this succeeds:

 >>> t = '%(fred)s:%(bert)s x="100%%"'
 >>> v = {'fred': 'abc', 'bert': 'xyz'}
 >>> t % v
'abc:xyz x="100%"'

> I have now restored the stock templates while I troubleshoot but does 
> anyone
> have any tips for trouble shooting a template that no longer parses?
>
> Thanks again for any help you can provide.
>
> Cheers,
>
> Ron
>
>
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers@python.org
> http://mail.python.org/mailman/listinfo/mailman-developers
>
>
-----------------------------------------------------------------------
Richard Barrett                               http://www.openinfo.co.uk


From brad.knowles at skynet.be  Fri Aug  1 06:38:46 2003
From: brad.knowles at skynet.be (Brad Knowles)
Date: Thu Jul 31 23:39:00 2003
Subject: [Mailman-Developers] Fwd: [Mailman-Users] 
	Bi-directional news 	gateway?
In-Reply-To: <a06001271bb4ea9c600ac@[10.0.1.2]>
References: <a0600124cbb4d9d160799@[10.0.1.2]>
	<1059582699.30752.89.camel@yyz> <a06001271bb4ea9c600ac@[10.0.1.2]>
Message-ID: <a06001209bb4f90d01a06@[10.0.1.2]>

At 1:12 PM +0200 2003/07/31, Brad Knowles wrote:

>>   Yes.  cron/gate_news is the script you need to run to pull stuff off the
>>   news server.
>
>  	Okay, next question -- how do I auto-approve all posts coming
>  from the newsgroup, while still applying the Privacy Options, Sender
>  Filters to messages being sent via e-mail?

	One more question -- we've got a subject prefix that we're 
putting on all mail messages coming into our mailing lists.  But this 
sort of thing really isn't appropriate on the newsgroup.  Is there 
any way we can apply the subject prefix after the incoming mail 
message is posted to the newsgroup?

-- 
Brad Knowles, <brad.knowles@skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

From marco_henriques at yahoo.com.br  Fri Aug  1 18:36:58 2003
From: marco_henriques at yahoo.com.br (Marco Antonio Reis Henriques)
Date: Fri Aug  1 16:37:10 2003
Subject: [Mailman-Developers] Possible suggestion to Mailman
Message-ID: <0dc301c3586c$ab972ef0$38053d0a@MARHNTB>

Hi All,

    Recently we was faced with an embarrasing situation. We created a list
with a list administrator and option "Send e-mail to inform list owner..."
checked. After, we included more three administrators, but these
administrators didn't receive a message to inform the password for list
administrator. I think this is useful to inform another list administrators
from the list.
    What you think ?!?

Cheers,

Marco


From jam at jamux.com  Sat Aug  2 12:11:27 2003
From: jam at jamux.com (John A. Martin)
Date: Sat Aug  2 11:11:36 2003
Subject: [Mailman-Developers] Attempt to invoke directly as script 
Message-ID: <87smokksbk.fsf@athene.jamux.com>

Log entries like the following appear with some regularity. 

,----[ Excerpt from /var/log/apache-ssl ]
[Fri Aug  1 15:54:12 2003] [error] [client 128.242.232.130] attempt to 
  invoke directory as script: /usr/lib/cgi-bin/mailman  
`----

I've poked around a bit but haven't seen how to tickle the bug.

This is with the Debian package mailman-2.1.2-7.


-- System Information 
Debian Release: 3.0 
Kernel Version: Linux lists 2.4.20-bf2.4 #1 Wed Dec 25 13:17:08 UTC 2002 i686 GN\
U/Linux 
 
Versions of the packages mailman depends on: 
ii  cron           3.0pl1-72      management of regular background processing 
ii  debconf        1.0.32         Debian configuration management system 
ii  libc6          2.3.1-17       GNU C Library: Shared libraries and Timezone 
ii  logrotate      3.5.9-8        Log rotation utility 
ii  pwgen          2.03-1         Automatic Password generation 
ii  python         2.2.3-3        An interactive high-level object-oriented la 
ii  ucf            0.18           Update Configuration File: preserves user ch 
exim    Not installed or no info 
ii  postfix        2.0.13-2003070 A high-performance mail transport agent 
        ^^^ (Provides virtual package mail-transport-agent) 
apache  Not installed or no info 
ii  apache-ssl     1.3.27.0-2     Versatile, high-performance HTTP server with 
        ^^^ (Provides virtual package httpd) 
 
        jam

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 154 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20030802/1b1eae03/attachment.bin
From marc at perkel.com  Sat Aug  2 10:54:45 2003
From: marc at perkel.com (Marc Perkel)
Date: Sat Aug  2 12:55:02 2003
Subject: [Mailman-Developers] Mailman Looping
Message-ID: <3F2BECD5.8060708@perkel.com>

Mailman started looping - using up a lot of processor and resources. I 
ran an strace and this is what it's doing. What do I have messed up?


munmap(0x401bc000, 4096)                = 0
unlink("/etc/mailman/qfiles/out/1059668916.586032+8c61dda9317d5ea99d9c330ae73482eaede0d693.db") 
= 0
open("/etc/mailman/qfiles/out/1059668916.586032+8c61dda9317d5ea99d9c330ae73482eaede0d693.pck", 

O_RDONLY|O_LARGEFILE) = 6
fstat64(6, {st_mode=S_IFREG|0660, st_size=1608, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 
0) = 0x401bc000
read(6, "(cMailman.Message\nUserNotificati"..., 4096) = 1608
unlink("/etc/mailman/qfiles/out/1059668916.586032+8c61dda9317d5ea99d9c330ae73482eaede0d693.pck") 
= 0
close(6)                                = 0
munmap(0x401bc000, 4096)                = 0
stat64("/usr/mailman/messages/en_US.ISO8859-1/LC_MESSAGES/mailman.mo", 
0xbfffee80) = -1 ENOENT (No such file
or directory)
stat64("/usr/mailman/messages/en_US/LC_MESSAGES/mailman.mo", 0xbfffee80) 
= -1 ENOENT (No such file or
directory)
stat64("/usr/mailman/messages/en.ISO8859-1/LC_MESSAGES/mailman.mo", 
0xbfffee80) = -1 ENOENT (No such file or
directory)
stat64("/usr/mailman/messages/en/LC_MESSAGES/mailman.mo", 0xbfffee80) = 
-1 ENOENT (No such file or directory)
gettimeofday({1059841437, 482524}, NULL) = 0
gettimeofday({1059841437, 482615}, NULL) = 0
umask(07)                               = 0
open("/etc/mailman/qfiles/out/1059668916.586032+3fe9305e11f8dcd259ddd30b1fc9f6e7cdb340fb.pck", 

O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 6
umask(0)                                = 07
fstat64(6, {st_mode=S_IFREG|0660, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 
0) = 0x401bc000
write(6, "(cMailman.Message\nUserNotificati"..., 1608) = 1608
close(6)                                = 0
munmap(0x401bc000, 4096)                = 0
umask(07)                               = 0
open("/etc/mailman/qfiles/out/1059668916.586032+3fe9305e11f8dcd259ddd30b1fc9f6e7cdb340fb.db.tmp", 

O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 6
umask(0)                                = 07
fstat64(6, {st_mode=S_IFREG|0660, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 
0) = 0x401bc000
write(6, "{s\4\0\0\0langs\2\0\0\0ens\10\0\0\0listnames\n"..., 426) = 426
close(6)                                = 0
munmap(0x401bc000, 4096)                = 0




From chuqui at plaidworks.com  Tue Aug  5 22:38:45 2003
From: chuqui at plaidworks.com (Chuq Von Rospach)
Date: Wed Aug  6 00:39:41 2003
Subject: [Mailman-Developers] digest problems (anyone seen this?)
Message-ID: <DD35BC84-C7C7-11D7-8561-0003934516A8@plaidworks.com>


Hey, all. Anyone else run into this? I'm getting corrupted digests.

> Today's Topics:
>
>    1. Re: Trade Rumor! (CA94NHL@aol.com)
>    2. Re: Trade Rumor! (Yvonne Smead)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 5 Aug 2003 17:03:32 EDT
>

[stuff deleted]

>
> Message: 2
> Date: Tue, 5 Aug 2003 14:21:44 -0700
> From: xxx
> Subject: Re: Trade Rumor!
>

[stuff deleted]

>
> Sharks aren't gonna let go of Marco either with the about-to-happen new
> influx of Germans!From simutis@tsoft.com Tue Aug  5 17:08:27 2003
> Received: from mail0.rawbw.com (mail0.rawbw.com [198.144.192.41])
> 	by plaidworks.com (8.12.9/8.12.9) with ESMTP id h7608Cvj012823
> 	for <sharks@plaidworks.com>; Tue, 5 Aug 2003 17:08:13 -0700 (PDT)
> Received: from tsoft.com (m208-174.dsl.rawbw.com [198.144.208.174])
> 	by mail0.rawbw.com (8.11.6p2/8.11.6) with ESMTP id h7608AD87787
> 	for <sharks@plaidworks.com>; Tue, 5 Aug 2003 17:08:10 -0700 (PDT)
> Date: Tue, 5 Aug 2003 17:09:05 -0700
> Content-Type: text/plain; charset=WINDOWS-1252; format=flowed
> Mime-Version: 1.0 (Apple Message framework v552)
> From: xxxx
> To: sharks@plaidworks.com
> In-Reply-To: <200308052035.h75KZEde009556@plaidworks.com>
> Message-Id: <31757AF5-C7A2-11D7-B5A7-00039390D592@tsoft.com>
> X-Mailer: Apple Mail (2.552)
> Content-Transfer-Encoding: 8bit
> X-MIME-Autoconverted: from quoted-printable to 8bit by plaidworks.com 
> id
> 	h7608Cvj012823
> Subject: Re: migratory moose
> X-BeenThere: sharks@plaidworks.com
>

It seems that some messages aren't written to the digest mbox properly 
(I *think* the routine assumes the message ends with a blank line, and 
if it doesn't, the digester can't handle the mbox properly -- but I 
haven't proven that yet). It's always the same small set of users 
e-mail that causes this, so it's clearly something about their 
messages, but since I can only see stuff after it's converted to mbox, 
it's hard to tell (but whatever procmail does for local delivery fixes 
this, while Mailman doesn't)

Any thoughts? I've been told by users this has been going on since I 
upgraded to 2.1, but in all honesty, I just noticed...



From Daniel.Buchmann at bibsys.no  Wed Aug  6 08:54:55 2003
From: Daniel.Buchmann at bibsys.no (Daniel Buchmann)
Date: Wed Aug  6 03:54:57 2003
Subject: [Mailman-Developers] Mailman's CGI doesn't seem to handle the
	PROPFIND HTTP Request Method
Message-ID: <1060156457.2829.66.camel@FORNAX.bibsys.no>

Forwarding a bug from the Debian bug tracking system:
http://bugs.debian.org/cgi-bin/bugreport.cgi?archive=no&bug=202821

Anyone else seen this? What wrong here/who's fault is this?

<QUOTE>
When browsing Mailman's administrative pages, Galeon does some PROPFIND
HTTP (WebDAV ?) requests. It seems that MailMan then tries to answer
to these requests instead of ignoring them, and fails :

--- Mailman's traceback ---
Jul 25 10:09:29 2003 admin(18164):
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
admin(18164): [----- Mailman Version: 2.1.2 -----]
admin(18164): [----- Traceback ------]
admin(18164): Traceback (most recent call last):
admin(18164):   File "/var/lib/mailman/scripts/driver", line 87, in
run_main
admin(18164):     main()
admin(18164):   File "/usr/lib/mailman/Mailman/Cgi/admin.py", line 82,
in main
admin(18164):     cgidata.getvalue('adminpw', '')):
admin(18164):   File "/usr/lib/python2.2/cgi.py", line 558, in getvalue
admin(18164):     if self.has_key(key):
admin(18164):   File "/usr/lib/python2.2/cgi.py", line 601, in has_key
admin(18164):     raise TypeError, "not indexable"
admin(18164): TypeError: not indexable
admin(18164): [----- Python Information -----]
admin(18164): sys.version     =   2.2.3+ (#1, Jul  5 2003, 11:04:18)
[GCC 3.3.1 20030626 (Debian prerelease)]
admin(18164): sys.executable  =   /usr/bin/python
admin(18164): sys.prefix      =   /usr
admin(18164): sys.exec_prefix =   /usr
admin(18164): sys.path        =   /usr
admin(18164): sys.platform    =   linux2
admin(18164): [----- Environment Variables -----]
admin(18164):   HTTP_ACCEPT: */*
admin(18164):   PYTHONPATH: /var/lib/mailman
admin(18164):   SERVER_SOFTWARE: Apache/1.3.27 (Unix) Debian GNU/Linux
PHP/4.1.2
admin(18164):   SCRIPT_FILENAME: /usr/lib/cgi-bin/mailman/admin
admin(18164):   SERVER_ADMIN: root@www.librelogiciel.com
admin(18164):   SCRIPT_NAME: /mailman/admin
admin(18164):   SERVER_SIGNATURE: 
admin(18164):   REQUEST_METHOD: PROPFIND
...
--- Mailman's traceback ---
</QUOTE>

-- 
Daniel Buchmann <avalon@users.sourceforge.net>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 232 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20030806/692b4099/attachment.bin
From bernhard at intevation.de  Tue Aug  5 20:17:34 2003
From: bernhard at intevation.de (Bernhard Reiter)
Date: Wed Aug  6 08:21:58 2003
Subject: [Mailman-Developers] Patch for List-ID bug (665732)
Message-ID: <20030805171734.GA25266@intevation.de>

I've just added a patch fixing a priority 7 bug to the tracker.
https://sourceforge.net/tracker/index.php?func=detail&aid=665732&group_id=103&atid=100103

What wasn't entirely clear to me is 
if I'd better add that patch to the patch tracker and just point to it.
Which of the two methods is more likely to be seen 
by the people with CVS write access?

Also is somebody notified with the current tracker setup,
if a fix for an import change comes in?

Thanks,
	Bernhard
ps.: Please send me a carbon copy, I'm not subscribed to the list.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20030805/9ea0376a/attachment.bin
From chuqui at plaidworks.com  Wed Aug  6 16:20:55 2003
From: chuqui at plaidworks.com (Chuq Von Rospach)
Date: Wed Aug  6 18:22:03 2003
Subject: [Mailman-Developers] digest problems (anyone seen this?)
In-Reply-To: <DF858D51-C83B-11D7-AA44-000393AFA48A@efn.org>
Message-ID: <3F87A550-C85C-11D7-8561-0003934516A8@plaidworks.com>


> I was getting a similar problem where lists would stop altogether 
> because of illegal characters in purportedly base64 encoded 
> attachments.
>

I checked for mime problems, but that's not it. I'm stripping to 
text/plain, 8859-1. No funky stuff.

Looking in the digest.mbox files, it looks like the messages causing 
problems don't have a terminating \n on the message:

> -- 
>
> Sharks are the best from the east to the west!!
> Go Sharks!!!
> Check out the latest news, etc. @ 
> <http://www.gingerose.info>plaidworks.com 162#
>

So it looks like the digesting code needs to be taught to either 
recognize this and add it if missing, or (easier) simply throw out an 
extra \n when appending a message to the digest.mbox file. that's 
harmless in the mbox format...

Off to whack the code...


From brad.knowles at skynet.be  Thu Aug  7 01:33:43 2003
From: brad.knowles at skynet.be (Brad Knowles)
Date: Wed Aug  6 18:35:26 2003
Subject: [Mailman-Developers] digest problems (anyone seen
 this?)
In-Reply-To: <3F87A550-C85C-11D7-8561-0003934516A8@plaidworks.com>
References: <3F87A550-C85C-11D7-8561-0003934516A8@plaidworks.com>
Message-ID: <a0600120dbb57328d0e75@[10.0.1.4]>

At 3:20 PM -0700 2003/08/06, Chuq Von Rospach wrote:

>  Looking in the digest.mbox files, it looks like the messages causing
>  problems don't have a terminating \n on the message:

	Blech.  Any idea what the offending MUAs/client platforms are?

-- 
Brad Knowles, <brad.knowles@skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

From chuqui at plaidworks.com  Wed Aug  6 16:48:12 2003
From: chuqui at plaidworks.com (Chuq Von Rospach)
Date: Wed Aug  6 18:48:51 2003
Subject: [Mailman-Developers] digest problems (anyone seen this?)
In-Reply-To: <a0600120dbb57328d0e75@[10.0.1.4]>
Message-ID: <0EFD6D0F-C860-11D7-8561-0003934516A8@plaidworks.com>


On Wednesday, August 6, 2003, at 03:33  PM, Brad Knowles wrote:

>>  Looking in the digest.mbox files, it looks like the messages causing
>>  problems don't have a terminating \n on the message:
>
> 	Blech.  Any idea what the offending MUAs/client platforms are?
>

hate to say it, but I think it's mailman. I'm getting the message 
multipart/alternative, with a text and html part. that's being stripped 
just to the text part, and that text part seems to be messed up.

here's the original:

> Content-Type: multipart/alternative; 
> boundary="============_-1151915027==_ma====
> ========"
>
> --============_-1151915027==_ma============
> Content-Type: text/plain; charset="us-ascii" ; format="flowed"
>
> <http://www.sjsharks.com/sharks/news/20030806-1060196976.htm>
> --
>
> Sharks are the best from the east to the west!!
> Go Sharks!!!
> Check out the latest news, etc. @ <http://www.gingerose.info>
> --============_-1151915027==_ma============
> Content-Type: text/html; charset="us-ascii"
>

Not there's no blank line between the end of the message and the MIME 
separator. Most MUA's add a blank line, but that's legal, and it ought 
to work. But when Mailman's done stripping, you end up with:

>
> <http://www.sjsharks.com/sharks/news/20030806-1060196976.htm>
> --
>
> Sharks are the best from the east to the west!!
> Go Sharks!!!
> Check out the latest news, etc. @ <http://www.gingerose.info><EOF>
>

So it seems to be Mailman making an unsafe assumption about how the 
MIME is constructed, not a problem with an incoming message.

Barry? What's your thought here?


From barry at python.org  Thu Aug  7 00:25:07 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Aug  6 19:25:09 2003
Subject: [Mailman-Developers] digest problems (anyone seen this?)
In-Reply-To: <0EFD6D0F-C860-11D7-8561-0003934516A8@plaidworks.com>
References: <0EFD6D0F-C860-11D7-8561-0003934516A8@plaidworks.com>
Message-ID: <1060212271.9621.22.camel@anthem>

On Wed, 2003-08-06 at 18:48, Chuq Von Rospach wrote:

> So it seems to be Mailman making an unsafe assumption about how the 
> MIME is constructed, not a problem with an incoming message.
> 
> Barry? What's your thought here?

If it's really the digest.mbox file that's busted here (and you can
check by looking at lists/<yourlist>/digest.mbox before the next digest
is sent out), then it's likely something in the scrubber.  Try to find
the matching offending message in your archive's .mbox file and send me
a copy of that.  I'll run it through my own scrubber and see what
happens.

If the scrubbing leaves the text/plain payload without a trailing
newline then indeed it will get into the digest.mbox file that way, and
that will screw up the digest parsing later.  In that case, then the
following (untested) patch might do the trick.

If this patch works, then I'll probably apply it as a fallback, but I'll
want to make sure the scrubber is fixed too, otherwise it's possible
that your mbox archive is also broken (though you wouldn't notice it
until you regenerated your pipermail archives).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ToDigest.py-patch.txt
Type: text/x-patch
Size: 967 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20030806/be3adca6/ToDigest.py-patch.bin
From chuqui at plaidworks.com  Wed Aug  6 17:36:36 2003
From: chuqui at plaidworks.com (Chuq Von Rospach)
Date: Wed Aug  6 19:36:51 2003
Subject: [Mailman-Developers] digest problems (anyone seen this?)
In-Reply-To: <1060212271.9621.22.camel@anthem>
Message-ID: <D2096E71-C866-11D7-8561-0003934516A8@plaidworks.com>


On Wednesday, August 6, 2003, at 04:24  PM, Barry Warsaw wrote:

> If it's really the digest.mbox file that's busted here (and you can
> check by looking at lists/<yourlist>/digest.mbox before the next digest
> is sent out), then it's likely something in the scrubber.

It's messed up in digest.mbox, so it's the scrubber.

>  Try to find
> the matching offending message in your archive's .mbox file and send me
> a copy of that.  I'll run it through my own scrubber and see what
> happens.

coming via separate email.

> If this patch works, then I'll probably apply it as a fallback, but 
> I'll
> want to make sure the scrubber is fixed too, otherwise it's possible
> that your mbox archive is also broken (though you wouldn't notice it
> until you regenerated your pipermail archives).
>

I don't use pipermail, I have my own half-built solution based on 
MHonarc... (grin/sigh)


From barry at python.org  Thu Aug  7 04:44:33 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Aug  6 23:44:35 2003
Subject: [Mailman-Developers] digest problems (anyone seen this?)
In-Reply-To: <1060212271.9621.22.camel@anthem>
References: <0EFD6D0F-C860-11D7-8561-0003934516A8@plaidworks.com>
	<1060212271.9621.22.camel@anthem>
Message-ID: <1060227836.9621.25.camel@anthem>

On Wed, 2003-08-06 at 19:24, Barry Warsaw wrote:

> If this patch works, then I'll probably apply it as a fallback, but I'll
> want to make sure the scrubber is fixed too, otherwise it's possible
> that your mbox archive is also broken (though you wouldn't notice it
> until you regenerated your pipermail archives).

This dawned on me at practice.  The previous patch was probably no good,
but this one might be <wink>.  I'll test it out tomorrow night.

-Barry


-------------- next part --------------
A non-text attachment was scrubbed...
Name: ToDigest.py-patch.txt
Type: text/x-patch
Size: 971 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20030807/dc9b399a/ToDigest.py-patch.bin
From tfheen at raw.no  Fri Aug  8 11:05:03 2003
From: tfheen at raw.no (Tollef Fog Heen)
Date: Fri Aug  8 04:05:14 2003
Subject: [Mailman-Developers] Bug in Dutch translation
Message-ID: <87y8y4pob4.fsf@yiwaz.raw.no>


(see http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=204196 for the
Debian bug report)

There's an error in the dutch translation where the keyword used is
the wrong one.  This causes Mailman to shunt large (> 40KB by default)
mails on Dutch mailing lists.

--- mailman-2.1.2.orig/messages/nl/LC_MESSAGES/mailman.po
+++ mailman-2.1.2/messages/nl/LC_MESSAGES/mailman.po
@@ -7406,7 +7406,7 @@
 "%(kb)d KB in size."
 msgstr ""
 "Uw bericht was te groot.Het mag maar\n"
-"%(limit)d KB groot zijn."
+"%(kb)d KB groot zijn."
 
 #: Mailman/Handlers/Hold.py:110
 msgid "Posting to a moderated newsgroup"

-- 
Tollef Fog Heen                                                        ,''`.
UNIX is user friendly, it's just picky about who its friends are      : :' :
                                                                      `. `' 
                                                                        `-  

From r.barrett at openinfo.co.uk  Tue Aug 12 12:41:24 2003
From: r.barrett at openinfo.co.uk (Richard Barrett)
Date: Tue Aug 12 06:42:25 2003
Subject: [Mailman-Developers] Differences in URL hostname computed for
	private and public archives
Message-ID: <84F3CD5C-CCB1-11D7-9EAE-000A957C9A50@openinfo.co.uk>

I am referring to Mailman 2.1.2 in what follows.

When computing an absolute URL pointing at a list's archives, the 
computation of the hostname in the URL is performed differently 
depending on whether the list archives are private or public.

I cannot spot an obvious reason for why this should be so. I am hoping 
some kind reader will enlighten me.

My analysis, in summary:

1. The same function GetBaseArchiveURL() in 
$prefix/Mailman/Archive/Archiver.py is called to deliver the list's 
archive URL for both public and private list archives.

2. Private list archives call subsidiary functions which leads to the 
FQDN extracted from the list's web_page_url attribute being used to 
form the required URL.

3. Public list archives use the list's host_name attribute and use that 
perform lookup in an inversion of the mm.cfg.VIRTUAL_HOSTS dictionary 
to obtain a URL_FQDN to form the required URL. If the lookup fails then 
the DEFAULT_URL_HOST is used.

Why the difference in approach?

A consequence of this approach is that if several different virtual 
hosts use the same EMAIL_FQDN, which means the VIRTUAL_HOSTS dictionary 
is a many-one mapping for those virtual hosts' URL_FQDNs, their public 
list archive base URLs are arbitrarily assigned to just one of those 
URL_FQDNs.

This method of computing the URL pointing at a list's archives also 
seems to interact negatively with a weak aspect of the cPanel 
(http://http://cpanel.net/) implementation of MM 2.1.2

cPanel does not appear to maintain (my guess because I do not have 
access to the source code of their MM related stuff) the 
mm_cfg.VIRTUAL_HOSTS dictionary and seems to setup the web_page_url and 
host_name attributes of list created via cPanel without reference to 
VIRTUAL_HOSTS dictionary. But, as a consequence, when list archives are 
public the URLs of links to them on the /mailman/listinfo/<listname> 
and /mailman/admin/<listname> pages use the hosting server's hostname 
rather than the hosted domain name.

While it seems that the CPanel implementation of Mailman is partially 
broken, the URLs generated for MM's CGI prgarams happen to work OK 
because they are formed from the web_page_url list attribute.


-----------------------------------------------------------------------
Richard Barrett                               http://www.openinfo.co.uk


From davidb at chelsea.net  Wed Aug 13 18:55:31 2003
From: davidb at chelsea.net (David Birnbaum)
Date: Wed Aug 13 17:55:34 2003
Subject: [Mailman-Developers] Mailman - list corruption
In-Reply-To: <14526.29365.432370.248009@anthem.cnri.reston.va.us>
References: <200003020353.TAA19329@utopia.West.Sun.COM>
	<Pine.GSO.4.05.10003020658120.14002-100000@corona.chelsea.net>
	<14526.29365.432370.248009@anthem.cnri.reston.va.us>
Message-ID: <Pine.GSO.4.52.0308131718140.17060@corona.chelsea.net>

Howdy,

I just had a list corruption today - twice - with no smoking gun in the
log files that indicated the cause of the problem, just that the list
database became corrupted (see error messages below).  It's a list of
about 22,000 people at the moment.

I was able to dump the list members and list configuration, delete the
list, and recreate it.  However, it became corrupted again a few hours
later, although not as badly.  In the first corruption, the list was
unusable by the web interface.  The second time, it "fixed" itself, in
that the web interface started working again.  I investigated further, and
I discovered a lot of broken locks (which seems odd, given a lifetime of
five hours by default):

Aug 13 08:44:14 2003 (911) thebody.lock lifetime has expired, breaking
Aug 13 11:14:07 2003 (911) thebody.lock lifetime has expired, breaking
Aug 13 12:40:41 2003 (911) thebody.lock lifetime has expired, breaking
Aug 13 15:11:50 2003 (911) thebody.lock lifetime has expired, breaking
Aug 13 15:58:56 2003 (911) thebody.lock lifetime has expired, breaking
Aug 13 16:26:56 2003 (911) thebody.lock lifetime has expired, breaking

Not sure if this has anything to do with it, but it's the only unusual
thing.  I also noted that the dates on the locks are often in the future;
perhaps this is by design, it's certainly very strange.  The locks are
appearing/disappearing appropriately, as far as I can tell, and nothing
strange is showing up otherwise.

I've stopped and restarted the master mailman process, and dumped/restored
the list again, and all appears well.  Any suggestions on where to start?
I'm running the latest mailman 2.1.2, on Solaris 2.8, with Python 2.2.2.
Haven't had any problems with the machine or memory that have showed up in
the logs.

Thanks in advance,

David.

-----

Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.pck
Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.pck.last
Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.db
[Errno 2] No such file or directory: '/home/mailman/lists/thelist/config.db'
Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.db.last
[Errno 2] No such file or directory: '/home/mailman/lists/thelist/config.db.last'
Aug 13 17:01:40 2003 (22680) All thelist fallbacks were corrupt, giving up
Aug 13 17:01:40 2003 admin(22680): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
admin(22680): [----- Mailman Version: 2.1.2 -----]
admin(22680): [----- Traceback ------]
admin(22680): Traceback (most recent call last):
admin(22680):   File "/home/mailman/scripts/driver", line 87, in run_main
admin(22680):     main()
admin(22680):   File "/home/mailman/Mailman/Cgi/admin.py", line 162, in main
admin(22680):     mlist.Lock()
admin(22680):   File "/home/mailman/Mailman/MailList.py", line 159, in Lock
admin(22680):     self.Load()
admin(22680):   File "/home/mailman/Mailman/MailList.py", line 598, in Load
admin(22680):     raise Errors.MMCorruptListDatabaseError, e
admin(22680): MMCorruptListDatabaseError: [Errno 2] No such file or directory: '/home/mailman/lists/
thelist/config.db.last'
admin(22680): [----- Python Information -----]
admin(22680): sys.version     =   2.2.2 (#1, Feb  4 2003, 14:15:12)
[GCC 3.2.1]
admin(22680): sys.executable  =   /usr/local/bin/python
admin(22680): sys.prefix      =   /opt/python/2.2.2
admin(22680): sys.exec_prefix =   /opt/python/2.2.2
admin(22680): sys.path        =   /opt/python/2.2.2
admin(22680): sys.platform    =   sunos5
admin(22680): [----- Environment Variables -----]
admin(22680):   HTTP_COOKIE: thelist+admin=28020000006988523a3f7328000000633165626261613432313637316
66631633863613437326636386435353365623233326666623938
admin(22680):   SERVER_SOFTWARE: Apache/1.3.26 (Unix) mod_ssl/2.8.10 OpenSSL/0.9.6c
admin(22680):   PYTHONPATH: /home/mailman
admin(22680):   SCRIPT_FILENAME: /home/mailman/cgi-bin/admin.cgi
admin(22680):   SERVER_ADMIN: webmaster@chelsea.net
admin(22680):   SCRIPT_NAME: /cgi-bin/admin.cgi
admin(22680):   REQUEST_METHOD: GET
admin(22680):   HTTP_HOST: mailman.chelsea.net
admin(22680):   PATH_INFO: /thelist
admin(22680):   SERVER_PROTOCOL: HTTP/1.1
admin(22680):   QUERY_STRING:
admin(22680):   TZ: US/Eastern
admin(22680):   REQUEST_URI: /cgi-bin/admin.cgi/thelist
admin(22680):   HTTP_ACCEPT: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/msword
, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/x-shockwave-flash, */*
admin(22680):   HTTP_USER_AGENT: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)
admin(22680):   HTTP_CONNECTION: Keep-Alive
admin(22680):   SERVER_NAME: mailman.chelsea.net
admin(22680):   REMOTE_ADDR: 209.212.72.130
admin(22680):   REMOTE_PORT: 5086
admin(22680):   HTTP_ACCEPT_LANGUAGE: en-us
admin(22680):   PATH_TRANSLATED: /home/mailman/htdocs/thelist
admin(22680):   SERVER_PORT: 80
admin(22680):   GATEWAY_INTERFACE: CGI/1.1
admin(22680):   HTTP_ACCEPT_ENCODING: gzip, deflate
admin(22680):   SERVER_ADDR: 209.212.66.37

From davidb at pins.net  Wed Aug 13 20:02:53 2003
From: davidb at pins.net (David Birnbaum)
Date: Wed Aug 13 19:02:56 2003
Subject: [Mailman-Developers] Re: Mailman - list corruption - problem found
In-Reply-To: <Pine.GSO.4.52.0308131718140.17060@corona.chelsea.net>
References: <200003020353.TAA19329@utopia.West.Sun.COM>
	<Pine.GSO.4.05.10003020658120.14002-100000@corona.chelsea.net>
	<14526.29365.432370.248009@anthem.cnri.reston.va.us>
	<Pine.GSO.4.52.0308131718140.17060@corona.chelsea.net>
Message-ID: <Pine.GSO.4.52.0308131859580.17060@corona.chelsea.net>

Folks,

I found the problem, after grabbing a truss of the web process.  It turns
out the resident process size became 30M, which turned out to be the
RLimitMEM on our web server by default.  It was sporadic because the list
was *just* under the limit, and it would only fail if it couldn't lock the
list quickly enough (minor memory leak there, perhaps?)

Since python grabs the Err#12 ENOMEM, it would be nice to propagate that
one back if possible!

Sorry for the false alarm!

Side question, though - why *ARE* the locks made "in the future"?

Thanks,

David.

-----

On Wed, 13 Aug 2003, David Birnbaum wrote:

> Howdy,
>
> I just had a list corruption today - twice - with no smoking gun in the
> log files that indicated the cause of the problem, just that the list
> database became corrupted (see error messages below).  It's a list of
> about 22,000 people at the moment.
>
> I was able to dump the list members and list configuration, delete the
> list, and recreate it.  However, it became corrupted again a few hours
> later, although not as badly.  In the first corruption, the list was
> unusable by the web interface.  The second time, it "fixed" itself, in
> that the web interface started working again.  I investigated further, and
> I discovered a lot of broken locks (which seems odd, given a lifetime of
> five hours by default):
>
> Aug 13 08:44:14 2003 (911) thebody.lock lifetime has expired, breaking
> Aug 13 11:14:07 2003 (911) thebody.lock lifetime has expired, breaking
> Aug 13 12:40:41 2003 (911) thebody.lock lifetime has expired, breaking
> Aug 13 15:11:50 2003 (911) thebody.lock lifetime has expired, breaking
> Aug 13 15:58:56 2003 (911) thebody.lock lifetime has expired, breaking
> Aug 13 16:26:56 2003 (911) thebody.lock lifetime has expired, breaking
>
> Not sure if this has anything to do with it, but it's the only unusual
> thing.  I also noted that the dates on the locks are often in the future;
> perhaps this is by design, it's certainly very strange.  The locks are
> appearing/disappearing appropriately, as far as I can tell, and nothing
> strange is showing up otherwise.
>
> I've stopped and restarted the master mailman process, and dumped/restored
> the list again, and all appears well.  Any suggestions on where to start?
> I'm running the latest mailman 2.1.2, on Solaris 2.8, with Python 2.2.2.
> Haven't had any problems with the machine or memory that have showed up in
> the logs.
>
> Thanks in advance,
>
> David.
>
> -----
>
> Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.pck
> Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.pck.last
> Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.db
> [Errno 2] No such file or directory: '/home/mailman/lists/thelist/config.db'
> Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.db.last
> [Errno 2] No such file or directory: '/home/mailman/lists/thelist/config.db.last'
> Aug 13 17:01:40 2003 (22680) All thelist fallbacks were corrupt, giving up
> Aug 13 17:01:40 2003 admin(22680): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
> admin(22680): [----- Mailman Version: 2.1.2 -----]
> admin(22680): [----- Traceback ------]
> admin(22680): Traceback (most recent call last):
> admin(22680):   File "/home/mailman/scripts/driver", line 87, in run_main
> admin(22680):     main()
> admin(22680):   File "/home/mailman/Mailman/Cgi/admin.py", line 162, in main
> admin(22680):     mlist.Lock()
> admin(22680):   File "/home/mailman/Mailman/MailList.py", line 159, in Lock
> admin(22680):     self.Load()
> admin(22680):   File "/home/mailman/Mailman/MailList.py", line 598, in Load
> admin(22680):     raise Errors.MMCorruptListDatabaseError, e
> admin(22680): MMCorruptListDatabaseError: [Errno 2] No such file or directory: '/home/mailman/lists/
> thelist/config.db.last'
> admin(22680): [----- Python Information -----]
> admin(22680): sys.version     =   2.2.2 (#1, Feb  4 2003, 14:15:12)
> [GCC 3.2.1]
> admin(22680): sys.executable  =   /usr/local/bin/python
> admin(22680): sys.prefix      =   /opt/python/2.2.2
> admin(22680): sys.exec_prefix =   /opt/python/2.2.2
> admin(22680): sys.path        =   /opt/python/2.2.2
> admin(22680): sys.platform    =   sunos5
> admin(22680): [----- Environment Variables -----]
> admin(22680):   HTTP_COOKIE: thelist+admin=28020000006988523a3f7328000000633165626261613432313637316
> 66631633863613437326636386435353365623233326666623938
> admin(22680):   SERVER_SOFTWARE: Apache/1.3.26 (Unix) mod_ssl/2.8.10 OpenSSL/0.9.6c
> admin(22680):   PYTHONPATH: /home/mailman
> admin(22680):   SCRIPT_FILENAME: /home/mailman/cgi-bin/admin.cgi
> admin(22680):   SERVER_ADMIN: webmaster@chelsea.net
> admin(22680):   SCRIPT_NAME: /cgi-bin/admin.cgi
> admin(22680):   REQUEST_METHOD: GET
> admin(22680):   HTTP_HOST: mailman.chelsea.net
> admin(22680):   PATH_INFO: /thelist
> admin(22680):   SERVER_PROTOCOL: HTTP/1.1
> admin(22680):   QUERY_STRING:
> admin(22680):   TZ: US/Eastern
> admin(22680):   REQUEST_URI: /cgi-bin/admin.cgi/thelist
> admin(22680):   HTTP_ACCEPT: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/msword
> , application/vnd.ms-excel, application/vnd.ms-powerpoint, application/x-shockwave-flash, */*
> admin(22680):   HTTP_USER_AGENT: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)
> admin(22680):   HTTP_CONNECTION: Keep-Alive
> admin(22680):   SERVER_NAME: mailman.chelsea.net
> admin(22680):   REMOTE_ADDR: 209.212.72.130
> admin(22680):   REMOTE_PORT: 5086
> admin(22680):   HTTP_ACCEPT_LANGUAGE: en-us
> admin(22680):   PATH_TRANSLATED: /home/mailman/htdocs/thelist
> admin(22680):   SERVER_PORT: 80
> admin(22680):   GATEWAY_INTERFACE: CGI/1.1
> admin(22680):   HTTP_ACCEPT_ENCODING: gzip, deflate
> admin(22680):   SERVER_ADDR: 209.212.66.37
>

From barry at python.org  Thu Aug 14 05:07:05 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Aug 14 00:07:07 2003
Subject: [Mailman-Developers] Mailman - list corruption
In-Reply-To: <Pine.GSO.4.52.0308131718140.17060@corona.chelsea.net>
References: <200003020353.TAA19329@utopia.West.Sun.COM>
	<Pine.GSO.4.05.10003020658120.14002-100000@corona.chelsea.net>
	<14526.29365.432370.248009@anthem.cnri.reston.va.us>
	<Pine.GSO.4.52.0308131718140.17060@corona.chelsea.net>
Message-ID: <1060833988.14628.20.camel@anthem>

I'm not ignoring you guys. ;)  Well, actually I am, but I wish I didn't
have to.  I'm slammed to the gills with work stuff these days.  I had
hope to have some free evenings this week to catch up, but no dice.

Chuq, I have a working patch for you that I still haven't had time to
check in.  I'm also nearly finished with a patch that should make bounce
processing much less painful.  Hopefully, I'll have a few hours soon to
finish that up and then release a MM2.1.3.

Sigh.  I'll be back soon.
-Barry



From barry at python.org  Thu Aug 14 17:50:44 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Aug 14 12:50:46 2003
Subject: [Mailman-Developers] Bug in Dutch translation
In-Reply-To: <87y8y4pob4.fsf@yiwaz.raw.no>
References: <87y8y4pob4.fsf@yiwaz.raw.no>
Message-ID: <1060470638.1047.3.camel@geddy>

On Fri, 2003-08-08 at 04:05, Tollef Fog Heen wrote:
> (see http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=204196 for the
> Debian bug report)
> 
> There's an error in the dutch translation where the keyword used is
> the wrong one.  This causes Mailman to shunt large (> 40KB by default)
> mails on Dutch mailing lists.
> 
> --- mailman-2.1.2.orig/messages/nl/LC_MESSAGES/mailman.po
> +++ mailman-2.1.2/messages/nl/LC_MESSAGES/mailman.po
> @@ -7406,7 +7406,7 @@
>  "%(kb)d KB in size."
>  msgstr ""
>  "Uw bericht was te groot.Het mag maar\n"
> -"%(limit)d KB groot zijn."
> +"%(kb)d KB groot zijn."
>  
>  #: Mailman/Handlers/Hold.py:110
>  msgid "Posting to a moderated newsgroup"

Thanks.  Applied.
-Barry



From tkikuchi at is.kochi-u.ac.jp  Fri Aug 15 10:57:09 2003
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Thu Aug 14 20:57:22 2003
Subject: [Mailman-Developers] Differences in URL hostname computed for
	private and public archives
In-Reply-To: <84F3CD5C-CCB1-11D7-9EAE-000A957C9A50@openinfo.co.uk>
References: <84F3CD5C-CCB1-11D7-9EAE-000A957C9A50@openinfo.co.uk>
Message-ID: <3F3C2FE5.5000607@is.kochi-u.ac.jp>

Hi,

I have just uploaded a patch for this issue.
http://sourceforge.net/tracker/index.php?func=detail&aid=789015&group_id=103&atid=300103


Richard Barrett wrote:

> I am referring to Mailman 2.1.2 in what follows.
> 
> When computing an absolute URL pointing at a list's archives, the 
> computation of the hostname in the URL is performed differently 
> depending on whether the list archives are private or public.
> 
> I cannot spot an obvious reason for why this should be so. I am hoping 
> some kind reader will enlighten me.
> 
> My analysis, in summary:
> 
> 1. The same function GetBaseArchiveURL() in 
> $prefix/Mailman/Archive/Archiver.py is called to deliver the list's 
> archive URL for both public and private list archives.
> 
> 2. Private list archives call subsidiary functions which leads to the 
> FQDN extracted from the list's web_page_url attribute being used to form 
> the required URL.
> 
> 3. Public list archives use the list's host_name attribute and use that 
> perform lookup in an inversion of the mm.cfg.VIRTUAL_HOSTS dictionary to 
> obtain a URL_FQDN to form the required URL. If the lookup fails then the 
> DEFAULT_URL_HOST is used.
> 
> Why the difference in approach?
> 
> A consequence of this approach is that if several different virtual 
> hosts use the same EMAIL_FQDN, which means the VIRTUAL_HOSTS dictionary 
> is a many-one mapping for those virtual hosts' URL_FQDNs, their public 
> list archive base URLs are arbitrarily assigned to just one of those 
> URL_FQDNs.
> 
> This method of computing the URL pointing at a list's archives also 
> seems to interact negatively with a weak aspect of the cPanel 
> (http://http://cpanel.net/) implementation of MM 2.1.2
> 
> cPanel does not appear to maintain (my guess because I do not have 
> access to the source code of their MM related stuff) the 
> mm_cfg.VIRTUAL_HOSTS dictionary and seems to setup the web_page_url and 
> host_name attributes of list created via cPanel without reference to 
> VIRTUAL_HOSTS dictionary. But, as a consequence, when list archives are 
> public the URLs of links to them on the /mailman/listinfo/<listname> and 
> /mailman/admin/<listname> pages use the hosting server's hostname rather 
> than the hosted domain name.
> 
> While it seems that the CPanel implementation of Mailman is partially 
> broken, the URLs generated for MM's CGI prgarams happen to work OK 
> because they are formed from the web_page_url list attribute.
> 
> 
> -----------------------------------------------------------------------
> Richard Barrett                               http://www.openinfo.co.uk
> 
> 
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers@python.org
> http://mail.python.org/mailman/listinfo/mailman-developers
> 
> 

-- 
Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/


From chris at jellybaby.net  Fri Aug 15 12:03:25 2003
From: chris at jellybaby.net (Chris Boulter)
Date: Fri Aug 15 06:03:30 2003
Subject: [Mailman-Developers] Build oddity when mm_cfg.py exists
Message-ID: <20030815100325.GA61887@jellybaby.net>

Hi,

Here's an odd little problem I've encountered and a workaround for it.

I've seen an error message when attempting to 'make install' when
Mailman/mm_cfg.py exists within the directory where I've downloaded Mailman.
Someone else posted about this error message back in January (see
http://mail.python.org/pipermail/mailman-users/2003-January/025466.html
) but didn't get a reply, so I'm posting this in case it's useful.

I've downloaded Mailman to
   ~mailmandeploy/mailman-2.1.2
and inside that directory I've hacked about with it, creating
   ~mailmandeploy/mailman-2.1.2/Mailman/mm_cfg.py

Now when I configure && make && make install, I get:

   Traceback (most recent call last):
     File "bin/update", line 48, in ?
       from Mailman import MailList
     File "/usr/local/mailman/Mailman/MailList.py", line 49, in ?
       from Mailman.Archiver import Archiver
     File "/usr/local/mailman/Mailman/Archiver/__init__.py", line 17, in ?
       from Archiver import *
     File "/usr/local/mailman/Mailman/Archiver/Archiver.py", line 36, in ?
       from Mailman.i18n import _
     File "/usr/local/mailman/Mailman/i18n.py", line 52, in ?
       set_language()
     File "/usr/local/mailman/Mailman/i18n.py", line 34, in set_language
       _translation = gettext.translation('mailman', mm_cfg.MESSAGES_DIR,
   AttributeError: 'module' object has no attribute 'MESSAGES_DIR'
   make: *** [update] Error 1

The other odd thing is that even if I delete
   ~mailmandeploy/mailman-2.1.2/Mailman/mm_cfg.py
and make install again, I get the same error. It seems that something is
written into the installation directory on the first unsuccessful
installation.

I doubt if this is the normal way of working - most people would install
first then configure (and that's what I'm going to do as a workaround).
Still, if it looks like there's anything of interest in this I can play some
more and find out more detail. Here's the contents of my mm_cfg.py:

   # Are archives public or private by default?
   # 0=public, 1=private
   DEFAULT_ARCHIVE_PRIVATE = 1
   
   # Definitely don't want password reminders, since we use single sign-on
   # Nor do we want welcome messages, for the same reason
   DEFAULT_SEND_REMINDERS = 0
   DEFAULT_SEND_WELCOME_MSG = 0
   DEFAULT_SEND_GOODBYE_MSG = 0
   
   # Since archives are private, no need to obscure email addrs
   ARCHIVER_OBSCURES_EMAILADDRS = 0

Cheers,
Chris

-- 
Write friendlier email : http://www.math.fu-berlin.de/~guckes/mail/edit.html
    A. Because it breaks the logical order of conversation.
    Q. Why is top posting bad?

From chris at jellybaby.net  Fri Aug 15 13:00:05 2003
From: chris at jellybaby.net (Chris Boulter)
Date: Fri Aug 15 07:00:08 2003
Subject: [Mailman-Developers] Build oddity when mm_cfg.py exists
In-Reply-To: <20030815100325.GA61887@jellybaby.net>
References: <20030815100325.GA61887@jellybaby.net>
Message-ID: <20030815110005.GB61887@jellybaby.net>

On Fri 2003-08-15 11:03:25 +0100, Chris Boulter wrote:
> Here's an odd little problem I've encountered and a workaround for it.

Oops. I think all this confusion might have been caused by my mm_cfg.py
missing

    from Defaults import *

Restoring that to the start of the file looks promising, anyway.

-- 
Write friendlier email : http://www.math.fu-berlin.de/~guckes/mail/edit.html
    A. Because it breaks the logical order of conversation.
    Q. Why is top posting bad?

From mrbill at mrbill.net  Fri Aug 15 10:58:39 2003
From: mrbill at mrbill.net (Bill Bradford)
Date: Fri Aug 15 10:58:43 2003
Subject: [Mailman-Developers] Problem with MM after power outage
Message-ID: <20030815145839.GM460@mrbill.net>

After power was restored and the machine came up, Mailman (2.1.2) is now
spitting out errors:

Traceback (most recent call last):
  File "/usr/local/mailman/bin/qrunner", line 270, in ?
    main()
  File "/usr/local/mailman/bin/qrunner", line 230, in main
    qrunner.run()
  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 59, in run
    filecnt = self._oneloop()
  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 88, in _oneloop
    msg, msgdata = self._switchboard.dequeue(filebase)
  File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 144, in dequeue
    data = self._ext_read(dbfile)
  File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 246, in _ext_read
    dict = marshal.load(fp)
ValueError: bad marshal data

Any suggestions?  Worked fine before power got yanked...

Bill

-- 
bill bradford
mrbill@mrbill.net
austin, texas

From mrbill at mrbill.net  Fri Aug 15 12:52:44 2003
From: mrbill at mrbill.net (Bill Bradford)
Date: Fri Aug 15 12:52:48 2003
Subject: [Mailman-Developers] Problem with MM after power outage
In-Reply-To: <20030815145839.GM460@mrbill.net>
References: <20030815145839.GM460@mrbill.net>
Message-ID: <20030815165244.GZ460@mrbill.net>

Looks like the problem resolved itself after a few minutes.

Bill

On Fri, Aug 15, 2003 at 09:58:39AM -0500, Bill Bradford wrote:
> After power was restored and the machine came up, Mailman (2.1.2) is now
> spitting out errors:
> Traceback (most recent call last):
>   File "/usr/local/mailman/bin/qrunner", line 270, in ?
>     main()
>   File "/usr/local/mailman/bin/qrunner", line 230, in main
>     qrunner.run()
>   File "/usr/local/mailman/Mailman/Queue/Runner.py", line 59, in run
>     filecnt = self._oneloop()
>   File "/usr/local/mailman/Mailman/Queue/Runner.py", line 88, in _oneloop
>     msg, msgdata = self._switchboard.dequeue(filebase)
>   File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 144, in dequeue
>     data = self._ext_read(dbfile)
>   File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 246, in _ext_read
>     dict = marshal.load(fp)
> ValueError: bad marshal data

-- 
bill bradford
mrbill@mrbill.net
austin, texas

From chen7 at cse.sc.edu  Thu Aug 14 11:45:53 2003
From: chen7 at cse.sc.edu (Chuming Chen)
Date: Mon Aug 18 08:57:29 2003
Subject: [Mailman-Developers] (no subject)
Message-ID: <2186.129.252.138.223.1060872353.squirrel@mail.cse.sc.edu>

Hi, all,

We are using mailman to set up mailing lists. It works great. I
have some questions about using mailman.

Before I send a digest out, I would like to edit it.  Where in the
Unix world is it located? How can I have permission to read/write
it?

I would like to be able to send out a digest when I want.  From
the help manual, it appears that my choices are (1) daily and (2)
when a size limit is reached.  I would prefer weekly or on-demand.

Thank you for your kind reply.

-- 
Chuming Chen
UNIX System Support
Department of Computer Science and Engineering
University of South Carolina
Swearingen 1D49
Columbia, SC 20208
+1 803 576-6308


From chen7 at cse.sc.edu  Thu Aug 14 13:33:09 2003
From: chen7 at cse.sc.edu (Chuming Chen)
Date: Mon Aug 18 08:57:31 2003
Subject: [Mailman-Developers] send digest on-demand
Message-ID: <2379.129.252.138.223.1060878789.squirrel@mail.cse.sc.edu>

Hi, all,

We are using mailman to set up mailing lists. It works great. I
have some questions about using mailman.

Before I send a digest out, I would like to edit it.  Where in the
Unix world is it located? How can I have permission to read/write
it?

I would like to be able to send out a digest when I want.  From
the help manual, it appears that my choices are (1) daily and (2)
when a size limit is reached.  I would prefer weekly or on-demand.

Thank you for your kind reply.


-- 
Chuming Chen
UNIX System Support
Department of Computer Science and Engineering
University of South Carolina
Swearingen 1D49
Columbia, SC 20208
+1 803 576-6308

From ivessm at softecusa.com  Tue Aug 12 19:09:21 2003
From: ivessm at softecusa.com (Stewart M. Ives)
Date: Mon Aug 18 08:57:50 2003
Subject: [Mailman-Developers] RE: [Mailman-Announce] RELEASED Mailman 2.1.2
In-Reply-To: <1051464666.5933.126.camel@anthem>
Message-ID: <001e01c3611e$629019f0$1002a8c0@D865PERL01>

Barry,

Where is the "Site Administrators" Documentation or is it what is currently
shown at www.list.org/site.html???

The description said it's a complete doc on installing/integrating/etc
mailman.  I think what comes up is not it.

I plan on installing this down at our local school for the principal to use
to comm w the parents.

Thanks.

stew


---
Outgoing SofTEC USA email is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.509 / Virus Database: 306 - Release Date: 8/12/2003



From jlouis at diku.dk  Fri Aug 15 16:11:39 2003
From: jlouis at diku.dk (Jesper Louis Andersen)
Date: Mon Aug 18 08:58:08 2003
Subject: [Mailman-Developers] Re: Welcome to the "Mailman-Developers"
	mailing list
In-Reply-To: <mailman.2627.1060952682.13679.mailman-developers@python.org>
References: <mailman.2627.1060952682.13679.mailman-developers@python.org>
Message-ID: <20030815151139.A29837@diku.dk>

Ok, you have just installed mailman on a FreeBSD 5.1 box and got
things to run properly. You then begin to port scripts from the older
setup. These in turn calls ./bin/newlist for setting up lists when they
do not exist and force things to be like we want them. Only slight
problem is the password generation. Originally the scripts called
''mkpasswd'', which is a program for generating random passwords on the
commandline. Then it fed the stuff into ./bin/newlist, easy and simple. 

Though it now seems that change_pw can force a password change on a
given list with a proper random password, I did not like the possible
race-condition there is a possible ./bin/newlist followed by ./bin/change_pw.

So what to do? Hell, this is Python for gods sake, it is easy to hack:

--- /usr/local/opt/mailman/bin/newlist.orig	Fri Aug 15 14:34:28 2003
+++ newlist	Fri Aug 15 14:54:44 2003
@@ -32,6 +32,9 @@
         their list has been created.  This option suppresses the prompt and
         notification.
 
+    -p/--randompass
+    	Generate a random password for this list
+
     -h/--help
         Print this help text and exit.
 
@@ -98,8 +101,9 @@
 
 def main():
     try:
-        opts, args = getopt.getopt(sys.argv[1:], 'hql:',
-                                   ['help', 'quiet', 'language='])
+        opts, args = getopt.getopt(sys.argv[1:], 'hqpl:',
+                                   ['help', 'quiet', 
+				    'language=', 'randompass'])
     except getopt.error, msg:
         usage(1, msg)
 
@@ -112,6 +116,8 @@
             quiet = 1
         if opt in ('-l', '--language'):
             lang = arg
+	if opt in ('-p', '--randompass'):
+	    randompass = 1
 
     # Is the language known?
     if lang not in mm_cfg.LC_DESCRIPTIONS.keys():
@@ -142,7 +148,10 @@
     if len(args) > 2:
 	listpasswd = args[2]
     else:
-        listpasswd = getpass.getpass(_('Initial %(listname)s password: '))
+	if randompass:
+	    listpasswd = Utils.MakeRandomPassword(8)
+	else:
+            listpasswd = getpass.getpass(_('Initial %(listname)s password: '))
     # List passwords cannot be empty
     listpasswd = listpasswd.strip()
     if not listpasswd:

Problem solved. Views on this? Should it be committed to the main trunk?
Is there a better way than this? Is my call to ./bin/newlist insane? Is
there a better way to get a hook into the system?

-- 
Jesper

From marcia at listas.nce.ufrj.br  Fri Aug  8 23:43:43 2003
From: marcia at listas.nce.ufrj.br (Marcia Cardoso)
Date: Mon Aug 18 08:58:16 2003
Subject: [Mailman-Developers] mailman now doesnt work
Message-ID: <20030809014343.E31683B21A6@listas.nce.ufrj.br>


Could you helpme.
My version was mailman2.0.1
I decided to install the version 2.1.2 to use diferent language.

I installed the freebsd port, upgrading the old one.
Know the mailman in http://host/mailman/admin and all the web lists don't work.

here is the web error:

Bug in Mailman version 2.1.2

We're sorry, we hit a bug!
If you would like to help us identify the problem, please email a copy of this page to the webmaster for this site with a description of what happened. Thanks! 

Traceback:

Traceback (most recent call last):
  File "/usr/local/mailman/scripts/driver", line 68, in run_main
    immediate=1)
  File "/usr/local/mailman/Mailman/Logging/StampedLogger.py", line 52, in __init__
    Logger.__init__(self, category, nofail, immediate)
  File "/usr/local/mailman/Mailman/Logging/Logger.py", line 49, in __init__
    self.__get_f()
  File "/usr/local/mailman/Mailman/Logging/Logger.py", line 67, in __get_f
    1)
  File "/usr/local/lib/python2.2/codecs.py", line 496, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 13] Permission denied: '/usr/local/mailman/logs/error'




From shaikli at yahoo.com  Wed Aug 13 17:56:20 2003
From: shaikli at yahoo.com (Nadim Shaikli)
Date: Mon Aug 18 08:58:22 2003
Subject: [Mailman-Developers] 'empty module name' error and shunting
Message-ID: <20030813235620.75519.qmail@web14903.mail.yahoo.com>


I've seen a number of posts (courtesy of Google) about this topic,
yet not a single one noted a solution.  All either were resolved
with,

 1. upgrade and it will go away (I have the latest of everything)
 2. "this only happens on spam (bad header), don't worry about it."
 3. you are using a RH RPM install that is broken.

Well, in my case I'm on a debian system (code via apt-get) running
mailman-2.1.2-6.  I have about 12 mailing-lists all seem to function
without an issue except for one.  Anytime anyone mails to it, the
message gets "Shunned" and I see this error in the log/error file
(mind you the message is obsoletely fine - its as though there is
lock someone that is forcing all legit messages to be shunned).


Aug 13 16:40:05 2003 (13248) Uncaught runner exception: Empty module name
Aug 13 16:40:05 2003 (13248) Traceback (most recent call last):
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 105, in _oneloop
    self._onefile(msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 155, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in
_dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in
_dopipel
ine
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 89, in process
    send_digests(mlist, mboxfp)
  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 130, in
send_digest
s
    send_i18n_digests(mlist, mboxfp)
  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 303, in
send_i18n_d
igests
    msg = scrubber(mlist, msg)
  File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 308, in process
    t = t.encode(charset, 'replace')
  File "/usr/lib/python2.2/encodings/__init__.py", line 51, in search_function
    mod = __import__(modname,globals(),locals(),'*')
ValueError: Empty module name

Aug 13 16:40:05 2003 (13248) SHUNTING:
1060794892.466989+1b026fcd575182e5c957eae
f9f3caac319c9a8e3


If I try to 'unshunt' the message - nothing happens and it simply pops
back into the qfiles/shunt dir.  I have hoards of people waiting for this
to get resolved and I've tried asking anyone who would listen on IRC
(irc.freenode.net #mailman and/or #debian) to no avail.  The website
was no help at all (there is no mention of 'shunting' even and alot of
the info is rather dated).

Could someone please lend a helping hand _ASAP_ ?

I'm not on the list, so please CC your replies.

Regards,

 - Nadim


__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com

From shaikli at yahoo.com  Wed Aug 13 18:09:24 2003
From: shaikli at yahoo.com (Nadim Shaikli)
Date: Mon Aug 18 08:58:24 2003
Subject: [Mailman-Developers] 'empty module name' error and shunting
Message-ID: <20030814000924.74606.qmail@web14901.mail.yahoo.com>

I've seen a number of posts (courtesy of Google) about this topic,
yet not a single one noted a solution.  All either were resolved
with,

 1. upgrade and it will go away (I have the latest of everything)
 2. "this only happens on spam (bad header), don't worry about it."
 3. you are using a RH RPM install that is broken.

Well, in my case I'm on a debian system (code via apt-get) running
mailman-2.1.2-6.  I have about 12 mailing-lists all seem to function
without an issue except for one.  Anytime anyone mails to it, the
message gets "Shunned" and I see this error in the log/error file
(mind you the message is obsoletely fine - its as though there is
lock someone that is forcing all legit messages to be shunned).


Aug 13 16:40:05 2003 (13248) Uncaught runner exception: Empty module name
Aug 13 16:40:05 2003 (13248) Traceback (most recent call last):
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 105, in _oneloop
    self._onefile(msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 155, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in
_dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in
_dopipel
ine
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 89, in process
    send_digests(mlist, mboxfp)
  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 130, in
send_digest
s
    send_i18n_digests(mlist, mboxfp)
  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 303, in
send_i18n_d
igests
    msg = scrubber(mlist, msg)
  File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 308, in process
    t = t.encode(charset, 'replace')
  File "/usr/lib/python2.2/encodings/__init__.py", line 51, in search_function
    mod = __import__(modname,globals(),locals(),'*')
ValueError: Empty module name

Aug 13 16:40:05 2003 (13248) SHUNTING:
1060794892.466989+1b026fcd575182e5c957eae
f9f3caac319c9a8e3


If I try to 'unshunt' the message - nothing happens and it simply pops
back into the qfiles/shunt dir.  I have hoards of people waiting for this
to get resolved and I've tried asking anyone who would listen on IRC
(irc.freenode.net #mailman and/or #debian) to no avail.  The website
was no help at all (there is no mention of 'shunting' even and alot of
the info is rather dated).

Could someone please lend a helping hand _ASAP_ ?

I'm not on the list, so please CC your replies.

Regards,

 - Nadim


__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com

From tfheen at raw.no  Fri Aug  8 09:57:30 2003
From: tfheen at raw.no (Tollef Fog Heen)
Date: Mon Aug 18 08:58:29 2003
Subject: [Mailman-Developers] Mailman's CGI doesn't seem to handle the
	PROPFIND HTTP Request Method
In-Reply-To: <1060156457.2829.66.camel@FORNAX.bibsys.no> (Daniel Buchmann's
	message of "06 Aug 2003 09:54:17 +0200")
References: <1060156457.2829.66.camel@FORNAX.bibsys.no>
Message-ID: <87ptjgskkl.fsf@yiwaz.raw.no>

* Daniel Buchmann 

| Forwarding a bug from the Debian bug tracking system:
| http://bugs.debian.org/cgi-bin/bugreport.cgi?archive=no&bug=202821
| 
| Anyone else seen this? What wrong here/who's fault is this?

It seems to me like the driver wrapper should filter out everything
which is not in ['GET', 'POST', 'HEAD'].

Something like

--- mailman-2.1.2.orig/scripts/driver
+++ mailman-2.1.2/scripts/driver
@@ -56,6 +56,7 @@
     # These will ensure that even if something between now and the
     # creation of the real logger below fails, we can still get
     # *something* meaningful.
+
     logger = None
     try:
         import paths
@@ -81,6 +82,15 @@
         module = getattr(pkg, scriptname)
         main = getattr(module, 'main')
         try:
+            import os
+            request_method = os.environ.get('REQUEST_METHOD')
+            if not request_method in ['GET', 'POST', 'HEAD']:
+                print "Status: 405 Method not allowed"
+                print "Content-type: text/plain"
+                print
+                print "The method is not allowed"
+                sys.exit()
+                
             try:
                 sys.stderr = logger
                 sys.stdout = tempstdout

works for me.

-- 
Tollef Fog Heen                                                        ,''`.
UNIX is user friendly, it's just picky about who its friends are      : :' :
                                                                      `. `' 
                                                                        `-  

From will at thebase.com  Thu Aug 14 14:38:04 2003
From: will at thebase.com (Will Knapp)
Date: Mon Aug 18 08:58:35 2003
Subject: [Mailman-Developers] html includes
Message-ID: <NIEDJHENHPAHOKCLDGKOOELNDLAA.will@thebase.com>

Hello,

I enter warily acknowleging that I am unworthy and tred where I ought not.

I have an question that I was not able to find an answer to either in
documentation or on the general users list.

When altering the User page's html I'd like to use some kind of "include" to
point to header.html and footer.html outside of mailman.

I know all about why SSI is a bad idea in CGI. Is there some tag in mailman
or Python that will let me do this?

I have several lists and am afraid of screwing up or forgeting something if
there is a change in the site design.

thanks, I now exit, walking slowly backwards, bent at the waist and palms
extended downward.

Will Knapp


From jam at jamux.com  Mon Aug 18 11:51:53 2003
From: jam at jamux.com (John A. Martin)
Date: Mon Aug 18 10:52:01 2003
Subject: [Mailman-Developers] Absentee list owners become suicide
	mail-bombers
Message-ID: <87ada7auie.fsf@athene.jamux.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

What defenses does Mailman have against suicide mail-bomber attacks?

The suicide mail-bomber has a number of list-owner addresses for which
the receiving MTAs take delivery and subsequently send a "Mail
Delivery Warning" mail message back to the list admin address which,
of course, Mailman sends on to the list owner addresses and so forth.

Pretty picture, no?

A real world example of this on a legacy mailman-2.0.13 installation
seems to have been triggered by a single "pending requests" mail
message with two list-owner recipient addresses, both bouncing by
means of non-standard mail messages.  Eighteen to twenty hours later
Mailman and Postfix were muddling along happily enough with large
queues but building abnormally large log files.  The Mailman qfiles
were increasing at the rate of 3-4 pairs per second.

Would it not be a good idea to rate-limit mail messages from the list
admin address to the list-owner address(es)?  And eventually, to block
incoming admin mail?

For installations that have VERP, VERP all admin mail, do fairly
aggressive "bounce handling" on list-owner addresses, and, when all
such addresses are disabled, block receipt of all list mail?

Another problem I see frequently with absentee/negligent list owners
is large accumulations of pending mail and requests.  I have reason to
suspect that many list owners direct their list-owner mail to
/dev/null (or whatever the Windows equivalent is, these seem to be
mostly Windows addicts).  How about a monthly reminder for list owners
that requires a positive response within a parameterized time or the
list will be automatically disabled?

        jam

-----BEGIN PGP SIGNATURE-----

iD8DBQE/QOfbUEvv1b/iXy8RAjv/AJ9Mx9hebEBzOVErXrjM8KLAEBc+EwCfaxjh
5bUmFss5Lvuzlcg0Rqrqqms=
=ZL/8
-----END PGP SIGNATURE-----

From barry at python.org  Mon Aug 18 19:09:17 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 18 14:09:18 2003
Subject: [Mailman-Developers] Absentee list owners become suicide
	mail-bombers
In-Reply-To: <87ada7auie.fsf@athene.jamux.com>
References: <87ada7auie.fsf@athene.jamux.com>
Message-ID: <1061230125.1958.27.camel@yyz>

On Mon, 2003-08-18 at 10:51, John A. Martin wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> What defenses does Mailman have against suicide mail-bomber attacks?
> 
> The suicide mail-bomber has a number of list-owner addresses for which
> the receiving MTAs take delivery and subsequently send a "Mail
> Delivery Warning" mail message back to the list admin address which,
> of course, Mailman sends on to the list owner addresses and so forth.
> 
> Pretty picture, no?

Indeed, but things are better for MM2.1.  There, all bounces to -owner
addresses are directed to the site list's -bounces address, while the
site list -owner is directed to the loop-killer alias.

> Would it not be a good idea to rate-limit mail messages from the list
> admin address to the list-owner address(es)?  And eventually, to block
> incoming admin mail?

It's an interesting idea.  In my MM3 model, I want to unify normal list
rosters and admin rosters, such that we can do similar bounce processing
and registrations on admin addresses (not to mention archiving -owner
email!).  Then it would be a small step to notify the site admins when
all -owners have been bounce disabled, and perhaps to do more
sophisticated actions such as disable/auto-reject all list traffic).

> Another problem I see frequently with absentee/negligent list owners
> is large accumulations of pending mail and requests.  I have reason to
> suspect that many list owners direct their list-owner mail to
> /dev/null (or whatever the Windows equivalent is, these seem to be
> mostly Windows addicts).  How about a monthly reminder for list owners
> that requires a positive response within a parameterized time or the
> list will be automatically disabled?

That and/or automatic disposition of held messages (i.e. auto-reject,
auto-discard, or <gasp> auto-approve).

-Barry



From laprice at efn.org  Mon Aug 18 14:48:36 2003
From: laprice at efn.org (Larry Price)
Date: Mon Aug 18 16:48:16 2003
Subject: [Mailman-Developers] 'empty module name' error and shunting
In-Reply-To: <20030813235620.75519.qmail@web14903.mail.yahoo.com>
Message-ID: <56B06DA8-D1BD-11D7-8347-000393AFA48A@efn.org>


On Wednesday, August 13, 2003, at 04:56  PM, Nadim Shaikli wrote:
>
>
> Aug 13 16:40:05 2003 (13248) Uncaught runner exception: Empty module 
> name
> Aug 13 16:40:05 2003 (13248) Traceback (most recent call last):
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 105, in 
> _oneloop
>     self._onefile(msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 155, in 
> _onefile
>     keepqueued = self._dispose(mlist, msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in
> _dispose
>     more = self._dopipeline(mlist, msg, msgdata, pipeline)
>   File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in
> _dopipel
> ine
>     sys.modules[modname].process(mlist, msg, msgdata)
>   File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 89, in 
> process
>     send_digests(mlist, mboxfp)
>   File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 130, in
> send_digests
>     send_i18n_digests(mlist, mboxfp)
>   File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 303, in
> send_i18n_d
> igests
>     msg = scrubber(mlist, msg)
>   File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 308, in 
> process
>     t = t.encode(charset, 'replace')
>   File "/usr/lib/python2.2/encodings/__init__.py", line 51, in 
> search_function
>     mod = __import__(modname,globals(),locals(),'*')
> ValueError: Empty module name
>
> Aug 13 16:40:05 2003 (13248) SHUNTING:
> 1060794892.466989+1b026fcd575182e5c957eae
> f9f3caac319c9a8e3
>
>
> If I try to 'unshunt' the message - nothing happens and it simply pops
> back into the qfiles/shunt dir.  I have hoards of people waiting for 
> this
> to get resolved and I've tried asking anyone who would listen on IRC
> (irc.freenode.net #mailman and/or #debian) to no avail.  The website
> was no help at all (there is no mention of 'shunting' even and alot of
> the info is rather dated).
>

A quick work around to allow mail to the list to keep flowing is to 
move the
digest mbox out of the way, and then run unshunt, this will allow 
messages that were held because of the failure of the digest process to 
go through.


From barry at python.org  Mon Aug 18 23:15:37 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 18 18:15:44 2003
Subject: [Mailman-Developers] 'empty module name' error and shunting
In-Reply-To: <20030813235620.75519.qmail@web14903.mail.yahoo.com>
References: <20030813235620.75519.qmail@web14903.mail.yahoo.com>
Message-ID: <1061244904.1958.32.camel@yyz>

On Wed, 2003-08-13 at 19:56, Nadim Shaikli wrote:

> Well, in my case I'm on a debian system (code via apt-get) running
> mailman-2.1.2-6.  I have about 12 mailing-lists all seem to function
> without an issue except for one.  Anytime anyone mails to it, the
> message gets "Shunned" and I see this error in the log/error file
> (mind you the message is obsoletely fine - its as though there is
> lock someone that is forcing all legit messages to be shunned).

One message getting shunted shouldn't affect any global state that will
cause all subsequent messages to get shunted.  Think of them as an event
unto themselves.

Please send me the .pck/.db pair of a shunted message.  If it's not a
problem of your configuration or installation, I should be able to
reproduce the bug.

-Barry



From barry at python.org  Mon Aug 18 23:27:33 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 18 18:27:34 2003
Subject: [Mailman-Developers] mailman now doesnt work
In-Reply-To: <20030809014343.E31683B21A6@listas.nce.ufrj.br>
References: <20030809014343.E31683B21A6@listas.nce.ufrj.br>
Message-ID: <1061245621.1958.34.camel@yyz>

On Fri, 2003-08-08 at 21:43, Marcia Cardoso wrote:

>   File "/usr/local/lib/python2.2/codecs.py", line 496, in open
>     file = __builtin__.open(filename, mode, buffering)
> IOError: [Errno 13] Permission denied: '/usr/local/mailman/logs/error'

bin/check_perms

-Barry



From barry at python.org  Mon Aug 18 23:31:11 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 18 18:31:12 2003
Subject: [Mailman-Developers] html includes
In-Reply-To: <NIEDJHENHPAHOKCLDGKOOELNDLAA.will@thebase.com>
References: <NIEDJHENHPAHOKCLDGKOOELNDLAA.will@thebase.com>
Message-ID: <1061245838.1958.36.camel@yyz>

On Thu, 2003-08-14 at 13:38, Will Knapp wrote:

> When altering the User page's html I'd like to use some kind of "include" to
> point to header.html and footer.html outside of mailman.
> 
> I know all about why SSI is a bad idea in CGI. Is there some tag in mailman
> or Python that will let me do this?

Not in Mailman 2.1 without hacking the code.

-Barry



From barry at python.org  Mon Aug 18 23:32:09 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 18 18:32:11 2003
Subject: [Mailman-Developers] RE: [Mailman-Announce] RELEASED Mailman 2.1.2
In-Reply-To: <001e01c3611e$629019f0$1002a8c0@D865PERL01>
References: <001e01c3611e$629019f0$1002a8c0@D865PERL01>
Message-ID: <1061245897.1958.38.camel@yyz>

On Tue, 2003-08-12 at 18:09, Stewart M. Ives wrote:
> Barry,
> 
> Where is the "Site Administrators" Documentation or is it what is currently
> shown at www.list.org/site.html???
> 
> The description said it's a complete doc on installing/integrating/etc
> mailman.  I think what comes up is not it.

Check out the README and INSTALL files in the source distribution.

-Barry



From barry at python.org  Tue Aug 19 02:06:12 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 18 21:06:14 2003
Subject: [Mailman-Developers] (no subject)
In-Reply-To: <2186.129.252.138.223.1060872353.squirrel@mail.cse.sc.edu>
References: <2186.129.252.138.223.1060872353.squirrel@mail.cse.sc.edu>
Message-ID: <1061255135.30737.18.camel@anthem>

On Thu, 2003-08-14 at 10:45, Chuming Chen wrote:

> Before I send a digest out, I would like to edit it.  Where in the
> Unix world is it located? How can I have permission to read/write
> it?

It lives in lists/<yourlist>/digest.mbox

> I would like to be able to send out a digest when I want.  From
> the help manual, it appears that my choices are (1) daily and (2)
> when a size limit is reached.  I would prefer weekly or on-demand.

In Mailman 2.1, there's an "immediate action setting" under Digest
options called _send_digest_now.  Click Yes and then submit to send the
current digest (if there is one) immediately.

-Barry



From barry at python.org  Tue Aug 19 03:08:57 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 18 22:08:59 2003
Subject: [Mailman-Developers] Problem with MM after power outage
In-Reply-To: <20030815165244.GZ460@mrbill.net>
References: <20030815145839.GM460@mrbill.net> <20030815165244.GZ460@mrbill.net>
Message-ID: <1061258901.30737.50.camel@anthem>

On Fri, 2003-08-15 at 12:52, Bill Bradford wrote:
> Looks like the problem resolved itself after a few minutes.
> 
> Bill
> 
> On Fri, Aug 15, 2003 at 09:58:39AM -0500, Bill Bradford wrote:
> > After power was restored and the machine came up, Mailman (2.1.2) is now
> > spitting out errors:
> > Traceback (most recent call last):
> >   File "/usr/local/mailman/bin/qrunner", line 270, in ?
> >     main()
> >   File "/usr/local/mailman/bin/qrunner", line 230, in main
> >     qrunner.run()
> >   File "/usr/local/mailman/Mailman/Queue/Runner.py", line 59, in run
> >     filecnt = self._oneloop()
> >   File "/usr/local/mailman/Mailman/Queue/Runner.py", line 88, in _oneloop
> >     msg, msgdata = self._switchboard.dequeue(filebase)
> >   File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 144, in dequeue
> >     data = self._ext_read(dbfile)
> >   File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 246, in _ext_read
> >     dict = marshal.load(fp)
> > ValueError: bad marshal data

Hmm, weird.  The switchboard is written such that this shouldn't happen
(i.e. you shouldn't get a corrupt message.db file).  It's supposed to
write the message first to .db.tmp and then rename it to .db in an
atomic operation.

Do you have any .db.tmp turds in your qfiles directories?

all-problems-should-fix-themselves-ly y'rs,
-Barry



From barry at python.org  Tue Aug 19 03:09:58 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 18 22:10:00 2003
Subject: [Mailman-Developers] Build oddity when mm_cfg.py exists
In-Reply-To: <20030815110005.GB61887@jellybaby.net>
References: <20030815100325.GA61887@jellybaby.net>
	<20030815110005.GB61887@jellybaby.net>
Message-ID: <1061258963.30737.52.camel@anthem>

On Fri, 2003-08-15 at 07:00, Chris Boulter wrote:
> On Fri 2003-08-15 11:03:25 +0100, Chris Boulter wrote:
> > Here's an odd little problem I've encountered and a workaround for it.
> 
> Oops. I think all this confusion might have been caused by my mm_cfg.py
> missing
> 
>     from Defaults import *
> 
> Restoring that to the start of the file looks promising, anyway.

Yeah, you kinda need that <wink>.

-Barry



From tkikuchi at is.kochi-u.ac.jp  Tue Aug 19 12:38:19 2003
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Mon Aug 18 22:38:29 2003
Subject: [Mailman-Developers] patch uploaded -- automatic discard
Message-ID: <3F418D9B.6020405@is.kochi-u.ac.jp>

Hi Mailman Developers,

I have uploaded a patch for automatic discard held messages.

1. site wide default of MAX_DAYS_TO_HOLD in Defaults.py
    (mm_cfg.py).

2. list admin can set list specific max_days_to_hold
    through the Web GUI (admin/<listname>/general).

3. integrated in cron/checkdbs and reports the number
    of discarded messages.

You may have to check Version.py if you have installed other
patches.

Barry, how about to include this in 2.1.3 ?

Cheers,

-- 
Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/


From pioppo at ferrara.linux.it  Tue Aug 19 22:17:38 2003
From: pioppo at ferrara.linux.it (Simone Piunno)
Date: Tue Aug 19 16:57:31 2003
Subject: [Mailman-Developers] Problem with MM after power outage
In-Reply-To: <1061258901.30737.50.camel@anthem>
References: <20030815145839.GM460@mrbill.net> <20030815165244.GZ460@mrbill.net>
	<1061258901.30737.50.camel@anthem>
Message-ID: <200308192117.38899.pioppo@ferrara.linux.it>

On Tuesday 19 August 2003 04:08, Barry Warsaw wrote:

> > > After power was restored and the machine came up, Mailman (2.1.2) is
> > > now spitting out errors:
> > >   File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 246, in
> > > _ext_read dict = marshal.load(fp)
> > > ValueError: bad marshal data
>
> Hmm, weird.  The switchboard is written such that this shouldn't happen
> (i.e. you shouldn't get a corrupt message.db file).  It's supposed to
> write the message first to .db.tmp and then rename it to .db in an
> atomic operation.

ok, but how do you make sure the file is really on disk instead of, e.g., half 
on disk and half on cache?

Simone

-- 
Adde parvum parvo magnus acervus erit -- Ovidio



From pioppo at ferrara.linux.it  Wed Aug 20 01:45:23 2003
From: pioppo at ferrara.linux.it (Simone Piunno)
Date: Fri Aug 22 10:36:56 2003
Subject: [Mailman-Developers] Problem with MM after power outage
In-Reply-To: <1061330802.1131.18.camel@geddy>
References: <20030815145839.GM460@mrbill.net>
	<200308192117.38899.pioppo@ferrara.linux.it>
	<1061330802.1131.18.camel@geddy>
Message-ID: <200308200045.23234.pioppo@ferrara.linux.it>

On Wednesday 20 August 2003 00:06, you wrote:

> > ok, but how do you make sure the file is really on disk instead of, e.g.,
> > half on disk and half on cache?
>
> We close the file before we rename it.

This would be ok if the underlying operating system flushed the disk cache 
upon close(), but I'm afraid this is not the case (at least on linux). 
This is from man 2 close:

  A  successful  close does not guarantee that the data has been success-
  fully saved to disk, as the kernel defers writes. It is not common  for
  a  filesystem  to  flush  the buffers when the stream is closed. If you
  need to be sure that the data is physically stored use  fsync(2).   (It
  will depend on the disk hardware at this point.)

This behaviour is declared conforming to SVr4, SVID, POSIX, X/OPEN, BSD 4.3.

Therefore I believe the problem reported here happens in this way:

 1. mailman writes the tmp file, closes it and the atomically renames.
    this is atomically from userland point of view (e.g. applications will
    see the file instantly changed)
 2. under the hood, the operating system is running a disk cach to speed
    up file operations, therefore what really happened is the file has been
    written to some RAM pages but not yet on disk.
 3. at some later time, the disk cache is copied from RAM to disk, effectively
    making changes permanent.  This copy is not atomic, e.g. files bigger than
    4k will be written in chunks of 4k pages.

A power interruption (or OS crash, or any other unclean shutdown) in phase 2 
could lead to a lost transaction (e.g. the file will appear as never 
overwritten, like phase 1 never happened).

A power interruption (or OS crash, or any other unclean shutdown) happening in 
phase 3 could lead to a corrupted file (e.g. some pages written to disk, some 
pages not).

MTAs usually provide a configuration setting to enable cache flush for each 
transaction (by use of fsync()), but this is disabled by default because of 
the severe impact in performance.

Use of BerkeleyDB (or similar transactional db libraries) could eliminate the 
problem of corrupted files without the need to fsync, but to solve the 
problem in phase 2 we need to guarantee at application level that loosing a 
file won't make dangling references or bad states in the related data we 
stored elsewhere.  Worst case, when restarting after power outage we should 
check for transactions to be cancelled because the related file is not on 
disk.
An example could be: we put a message on hold for moderation, therefore we  
 - save the message in a file (or rename from the previous location)
 - update the moderation queue index in MailList
 - Save() the list config pickle
If the system goes down now because of a power outage, when restarting we 
could have (even fsync()ing everything):
 - the index has been regularly updated
 - the message is not on disk, or it's in a different filename/path
this can happen because actual writes on disk can be reordered by the OS, for 
performance reasons.
Accessing the admindb panel now could potentially lead to exceptions.

Now, everyone who is serious about administering a server has a big and 
dependable UPS, automatically triggering clean shutdowns and so on, therefore 
everything I've described is not as much as a problem.

-- 
[pioppo@abulafia pioppo]$ man women
No manual entry for women



From wheakory at isu.edu  Thu Aug 21 11:06:56 2003
From: wheakory at isu.edu (Kory Wheatley)
Date: Fri Aug 22 11:40:53 2003
Subject: [Mailman-Developers] Mailing List Held Messages
Message-ID: <3F44EE20.BC002F45@isu.edu>

Hi Everyone,

I have a question about the removal of held messages, is there a way to
automatically remove held messages that have been in a mailing list for
a certain number of days ( I have been going in to /home/mailman/data
and using the "rm" command on messages that I  know are spam, or a worm
virus like SOBIG). The reason I ask is, it creates quite a bit of disk
space when moderators have not gone in and either "approved" or
"discarded" the held messages, so I would like to have something go in
and check how long a message has been out there and if it's out there
past a certain number of days I would like it removed.

I know I could write something in perl to do this, but if something is
already in the mailman configuration that has adopted this, than I don't

want to spend time doing so.

Also what is the syntax for creating a spam filter on the SUBECT header.




--
Kory Wheatley
Academic Computing Analyst Sr.
Phone 282-3874
#########################################
Everything must point to him.



From philb at philb.us  Thu Aug 21 00:35:51 2003
From: philb at philb.us (Phil Barnett)
Date: Fri Aug 22 16:21:08 2003
Subject: [Mailman-Developers] script
Message-ID: <200308202335.51703.philb@philb.us>


Has anyone created a script they'd like to share that kills off the sobig 
stuff sitting on hold?

I'll make one but if there's one already out there, it'd save me some time. 
I'm leaving on vacation Friday night. Lots to do 'till then...

-- 

Democracy is two wolves and a lamb voting on what to have for
lunch. Liberty is a well-armed lamb contesting the vote.


From wheakory at isu.edu  Wed Aug 20 11:16:43 2003
From: wheakory at isu.edu (Kory Wheatley)
Date: Fri Aug 22 16:48:22 2003
Subject: [Mailman-Developers] Held messages for approval
Message-ID: <3F439EEB.3A4C87AB@isu.edu>

Hi Everyone,

I have a question about the removal of held messages, is there a way to
automatically remove held messages that have been in a mailing list for
a certain number of days ( I have been going in to /home/mailman/data
and using the "rm" command on messages that I  know are spam, or a worm
virus like SOBIG). The reason I ask is, it creates quite a bit of disk
space when moderators have not gone in and either "approved" or
"discarded" the held messages, so I would like to have something go in
and check how long a message has been out there and if it's out there
past a certain number of days I would like it removed.

I know I could write something in perl to do this, but if something is
already in the mailman configuration that has adopted this, than I don't
want to spend time doing so.

Also what is the syntax for creating a spam filter on the SUBECT header.

--
Kory Wheatley
Academic Computing Analyst Sr.
Phone 282-3874
#########################################
Everything must point to him.



From tkikuchi at is.kochi-u.ac.jp  Sat Aug 23 14:00:51 2003
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Sun Aug 24 02:00:26 2003
Subject: [Mailman-Developers] Mailing List Held Messages
In-Reply-To: <3F44EE20.BC002F45@isu.edu>
References: <3F44EE20.BC002F45@isu.edu>
Message-ID: <3F46E6F3.3050502@is.kochi-u.ac.jp>

Hi,

This patch.
http://sourceforge.net/tracker/?func=detail&aid=790494&group_id=103&atid=300103

Cheers,

Kory Wheatley wrote:
> Hi Everyone,
> 
> I have a question about the removal of held messages, is there a way to
> automatically remove held messages that have been in a mailing list for
> a certain number of days ( I have been going in to /home/mailman/data
> and using the "rm" command on messages that I  know are spam, or a worm
> virus like SOBIG). The reason I ask is, it creates quite a bit of disk
> space when moderators have not gone in and either "approved" or
> "discarded" the held messages, so I would like to have something go in
> and check how long a message has been out there and if it's out there
> past a certain number of days I would like it removed.
> 

-- 
Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/


From tkikuchi at is.kochi-u.ac.jp  Sat Aug 23 14:24:05 2003
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Sun Aug 24 05:00:31 2003
Subject: [Mailman-Developers] script
In-Reply-To: <200308202335.51703.philb@philb.us>
References: <200308202335.51703.philb@philb.us>
Message-ID: <3F46EC65.3070303@is.kochi-u.ac.jp>

Hi,

Get auto_discard script here,

http://sourceforge.net/tracker/?func=detail&aid=636412&group_id=103&atid=300103

and place it in somewhere it can import paths.py (cron dir is
suitable). Edit the expire parameter for shorter time and
run the script (python auto_discard).

It will kill all the pending posts older than the expire parameter.
(it doesn't sort sobig stuff though ...)

Phil Barnett wrote:

> Has anyone created a script they'd like to share that kills off the sobig 
> stuff sitting on hold?
> 
> I'll make one but if there's one already out there, it'd save me some time. 
> I'm leaving on vacation Friday night. Lots to do 'till then...
> 

-- 
Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/


From mburton at jo.birdsense.com  Sat Aug 23 01:02:26 2003
From: mburton at jo.birdsense.com (Mike Burton)
Date: Sun Aug 24 11:45:16 2003
Subject: [Mailman-Developers] Attachments
Message-ID: <20038230226.395430@alfa>

Does anyone know if there is a patch to strip an attachment from list mail, save it to disk (in the web directory), and insert a URL to the file in it's place with Mailman 2.x?  If not, some clues on where this might fit in would be appreciated (I would be willing to look at coding it).

Thanks,
Mike



From brad.knowles at skynet.be  Sat Aug 23 16:14:09 2003
From: brad.knowles at skynet.be (Brad Knowles)
Date: Sun Aug 24 13:01:30 2003
Subject: [Mailman-Developers] Problem with MM after power
 outage
In-Reply-To: <200308200045.23234.pioppo@ferrara.linux.it>
References: <20030815145839.GM460@mrbill.net>
	<200308192117.38899.pioppo@ferrara.linux.it>
	<1061330802.1131.18.camel@geddy>
	<200308200045.23234.pioppo@ferrara.linux.it>
Message-ID: <a06001a01bb6d18b0b756@[10.0.1.2]>

At 12:45 AM +0200 2003/08/20, Simone Piunno wrote:

>  This would be ok if the underlying operating system flushed the disk cache
>  upon close(), but I'm afraid this is not the case (at least on linux).
>  This is from man 2 close:
>
>    A  successful  close does not guarantee that the data has been success-
>    fully saved to disk, as the kernel defers writes. It is not common  for
>    a  filesystem  to  flush  the buffers when the stream is closed. If you
>    need to be sure that the data is physically stored use  fsync(2).   (It
>    will depend on the disk hardware at this point.)

	From sendmail/README from sendmail 8.12.10.Beta2:

REQUIRES_DIR_FSYNC      Turn on support for file systems that require to
                 call fsync() for a directory if the meta-data in it has
                 been changed.  This should be turned on at least for older
                 versions of ReiserFS; it is enabled by default for Linux.
                 According to some information this flag is not needed
                 anymore for kernel 2.4.16 and newer.  We would appreciate
                 feedback about the semantics of the various file systems
                 available for Linux.
                 An alternative to this compile time flag is to mount the
                 queue directory without the -async option, or using
                 chattr +S on Linux.


	So, theoretically, for some filesystems on some versions of Linux 
(with more modern kernels), this should work as expected.

	Still, the code should be made more robust to deal with this issue.

-- 
Brad Knowles, <brad.knowles@skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

From Daniel.Buchmann at bibsys.no  Sun Aug 24 18:30:35 2003
From: Daniel.Buchmann at bibsys.no (Daniel Buchmann)
Date: Sun Aug 24 13:30:37 2003
Subject: [Mailman-Developers] Held messages for approval
In-Reply-To: <3F439EEB.3A4C87AB@isu.edu>
References: <3F439EEB.3A4C87AB@isu.edu>
Message-ID: <1061677491.2098.5.camel@fornax.hjemme.bibsys.no>

On Wed, 2003-08-20 at 18:16, Kory Wheatley wrote:
> Hi Everyone,
> 
> I have a question about the removal of held messages, is there a way to
> automatically remove held messages that have been in a mailing list for
> a certain number of days ( I have been going in to /home/mailman/data
> and using the "rm" command on messages that I  know are spam, or a worm
> virus like SOBIG). The reason I ask is, it creates quite a bit of disk
> space when moderators have not gone in and either "approved" or
> "discarded" the held messages, so I would like to have something go in
> and check how long a message has been out there and if it's out there
> past a certain number of days I would like it removed.

Kory,

I believe you could benefit greatly from Tokio Kikuchi's excellent auto_discard script.
You can find it in the SF patches-tracker under patch id #636412:
http://sourceforge.net/tracker/index.php?func=detail&aid=636412&group_id=103&atid=300103

:)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 232 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20030824/3e69f6e2/attachment.bin
From barry at python.org  Sun Aug 24 23:36:49 2003
From: barry at python.org (Barry Warsaw)
Date: Sun Aug 24 19:06:05 2003
Subject: [Mailman-Developers] Problem with MM after power outage
In-Reply-To: <200308192117.38899.pioppo@ferrara.linux.it>
References: <20030815145839.GM460@mrbill.net>
	<20030815165244.GZ460@mrbill.net> <1061258901.30737.50.camel@anthem>
	<200308192117.38899.pioppo@ferrara.linux.it>
Message-ID: <1061330802.1131.18.camel@geddy>

On Tue, 2003-08-19 at 15:17, Simone Piunno wrote:
> On Tuesday 19 August 2003 04:08, Barry Warsaw wrote:
> 
> > > > After power was restored and the machine came up, Mailman (2.1.2) is
> > > > now spitting out errors:
> > > >   File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 246, in
> > > > _ext_read dict = marshal.load(fp)
> > > > ValueError: bad marshal data
> >
> > Hmm, weird.  The switchboard is written such that this shouldn't happen
> > (i.e. you shouldn't get a corrupt message.db file).  It's supposed to
> > write the message first to .db.tmp and then rename it to .db in an
> > atomic operation.
> 
> ok, but how do you make sure the file is really on disk instead of, e.g., half 
> on disk and half on cache?

We close the file before we rename it.

-Barry



From gward at python.net  Sun Aug 24 21:49:07 2003
From: gward at python.net (Greg Ward)
Date: Sun Aug 24 20:49:12 2003
Subject: [Mailman-Developers] qrunner infinite loop?
Message-ID: <20030825004907.GA19219@cthulhu.gerg.ca>

I've just installed the mailman 2.1.2-7 package on a Debian sarge
(testing) system, and qrunner goes into an apparent infinite loop as
soon as it starts.  Here's a snapshot from "top" shortly after running
"/etc/init.d/mailman start":


  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  Command           
 4498 list      19   0  4656 4656 1844 R 99.7  0.5   0:48.36 python            
 4484 gward      9   0   940  940  748 R  0.3  0.1   0:00.17 top               

Memory use is stable, hence my assumption that this is a garden-variety
infinite loop.

And "ps -fu list" reports this:

UID        PID  PPID  C STIME TTY          TIME CMD
list      4492     1  0 02:42 ?        00:00:00 [mailmanctl]
list      4493  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=ArchRunner:0:1 -s
list      4494  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=BounceRunner:0:1 -s
list      4495  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=CommandRunner:0:1 -s
list      4496  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s
list      4497  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=NewsRunner:0:1 -s
list      4498  4492 97 02:42 ?        00:01:01 qrunner /var/lib/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s
list      4499  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=VirginRunner:0:1 -s

... so it's just one qrunner (OutgoingRunner) that's responsible for all
the CPU sucking.

Hmmm: there *are* files in the queue, but I'm not sure what they're
doing there or how they got there (this is a new, fairly inactive system
with no lists and no incoming traffic, apart from me testing things):

# cd /var/lib/mailman/
# find qfiles/ -type f
qfiles/out/1061532001.524745+03e15ef2a9a7d27904406a659cdb4922a204431b.pck
qfiles/out/1061497885.874818+cf5f7e2e03cbca8bb84a23a60f055679c16e81fa.pck
qfiles/out/1061704802.107101+4617d8731ecb303dd756e475b5362bd5a64b1fcf.pck
qfiles/out/1061497885.871999+3453d7caa05a1aad6b230da61b39d4937405fcc8.pck
qfiles/out/1061497885.871999+3453d7caa05a1aad6b230da61b39d4937405fcc8.db
qfiles/out/1061497885.874818+cf5f7e2e03cbca8bb84a23a60f055679c16e81fa.db
qfiles/out/1061704802.107101+4617d8731ecb303dd756e475b5362bd5a64b1fcf.db
qfiles/out/1061532001.524745+03e15ef2a9a7d27904406a659cdb4922a204431b.db

Any clues?  I'll try to investigate more tomorrow evening.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
We have always been at war with Oceania.

From brad.knowles at skynet.be  Mon Aug 25 11:15:04 2003
From: brad.knowles at skynet.be (Brad Knowles)
Date: Mon Aug 25 04:25:52 2003
Subject: [Mailman-Developers] Problem with MM after power
 outage
In-Reply-To: <1061330802.1131.18.camel@geddy>
References: <20030815145839.GM460@mrbill.net> <20030815165244.GZ460@mrbill.net>
	<1061258901.30737.50.camel@anthem>
	<200308192117.38899.pioppo@ferrara.linux.it>
	<1061330802.1131.18.camel@geddy>
Message-ID: <a06001a09bb6f75e4ecd9@[10.0.1.2]>

At 6:06 PM -0400 2003/08/19, Barry Warsaw wrote:

>>  ok, but how do you make sure the file is really on disk instead of,
>>  e.g., half on disk and half on cache?
>
>  We close the file before we rename it.

	Do you fsync() the directory after the close and before the rename?

-- 
Brad Knowles, <brad.knowles@skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

From pioppo at ferrara.linux.it  Mon Aug 25 23:11:52 2003
From: pioppo at ferrara.linux.it (Simone Piunno)
Date: Mon Aug 25 16:11:21 2003
Subject: [Mailman-Developers] Problem with MM after power outage
In-Reply-To: <a06001a09bb6f75e4ecd9@[10.0.1.2]>
References: <20030815145839.GM460@mrbill.net> <1061330802.1131.18.camel@geddy>
	<a06001a09bb6f75e4ecd9@[10.0.1.2]>
Message-ID: <200308252211.52762.pioppo@ferrara.linux.it>

On Monday 25 August 2003 10:15, Brad Knowles wrote:

> >>  ok, but how do you make sure the file is really on disk instead of,
> >  We close the file before we rename it.
> 	Do you fsync() the directory after the close and before the rename?

According to python documentation (os module), this should do:

  fsync(fd)
  Force write of file with filedescriptor fd to disk. On Unix, this calls the
  native fsync() function; on Windows, the MS _commit() function. 
  If you're starting with a Python file object f, first do f.flush(), and then
  do os.fsync(f.fileno()), to ensure that all internal buffers associated with
  f are written to disk. Availability: Unix, and Windows starting in 2.2.3. 

There's also:

  fdatasync(fd)
  Force write of file with filedescriptor fd to disk. Does not force update of
  metadata. Availability: Unix. 

I don't know the difference.

Because the python documentation says nothing about close() calling fsync() 
automatically, I assume it does not.

-- 
Adde parvum parvo magnus acervus erit -- Ovidio



From pioppo at ferrara.linux.it  Mon Aug 25 23:21:42 2003
From: pioppo at ferrara.linux.it (Simone Piunno)
Date: Mon Aug 25 16:21:41 2003
Subject: [Mailman-Developers] Problem with MM after power outage
In-Reply-To: <200308252211.52762.pioppo@ferrara.linux.it>
References: <20030815145839.GM460@mrbill.net>
	<a06001a09bb6f75e4ecd9@[10.0.1.2]>
	<200308252211.52762.pioppo@ferrara.linux.it>
Message-ID: <200308252221.42349.pioppo@ferrara.linux.it>

On Monday 25 August 2003 22:11, Simone Piunno wrote:

> > 	Do you fsync() the directory after the close and before the rename?

Ah, I've found what you were meaning...
this is from PosixFilesystem.py (ZODB implementation):

import os
from posix import fsync
....
    def sync_directory(self,dir):
        if self.use_sync:
            p = os.path.join(self.dirname,dir)
            # Use os.open here because, mysteriously, it performs better
            # than fopen on linux 2.4.18, reiserfs, glibc 2.2.4
            f = os.open(p,os.O_RDONLY)
            # Should we worry about EINTR ?
            try:
                fsync(f)
            finally:
                os.close(f)

    def write_file(self,filename,content):
        fullname = os.path.join(self.dirname,filename)
        f = os.open(fullname,os.O_CREAT|os.O_RDWR|os.O_TRUNC,0640)
        # Should we worry about EINTR ?
        try:
            os.write(f,content)
            if self.use_sync:
                fsync(f)
        finally:
            os.close(f)
-- 
Adde parvum parvo magnus acervus erit -- Ovidio



From brad.knowles at skynet.be  Tue Aug 26 01:49:48 2003
From: brad.knowles at skynet.be (Brad Knowles)
Date: Mon Aug 25 18:56:47 2003
Subject: [Mailman-Developers] Problem with MM after power
 outage
In-Reply-To: <200308252211.52762.pioppo@ferrara.linux.it>
References: <20030815145839.GM460@mrbill.net> <1061330802.1131.18.camel@geddy>
	<a06001a09bb6f75e4ecd9@[10.0.1.2]>
	<200308252211.52762.pioppo@ferrara.linux.it>
Message-ID: <a06001a18bb7042def75f@[10.0.1.2]>

At 10:11 PM +0200 2003/08/25, Simone Piunno wrote:

>    fsync(fd)
>    Force write of file with filedescriptor fd to disk. On Unix, this calls the
>    native fsync() function; on Windows, the MS _commit() function.
>    If you're starting with a Python file object f, first do 
>f.flush(), and then
>    do os.fsync(f.fileno()), to ensure that all internal buffers 
>associated with
>    f are written to disk. Availability: Unix, and Windows starting in 2.2.3.

	That's the one you want.  You're getting nailed on the meta-data 
which is not being flushed to disk before the rename.

>  Because the python documentation says nothing about close() calling fsync()
>  automatically, I assume it does not.

	Indeed.  ;(

-- 
Brad Knowles, <brad.knowles@skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

From barry at python.org  Tue Aug 26 03:38:20 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 25 22:38:22 2003
Subject: [Mailman-Developers] qrunner infinite loop?
In-Reply-To: <20030825004907.GA19219@cthulhu.gerg.ca>
References: <20030825004907.GA19219@cthulhu.gerg.ca>
Message-ID: <1061865463.12874.38.camel@anthem>

On Sun, 2003-08-24 at 20:49, Greg Ward wrote:
> I've just installed the mailman 2.1.2-7 package on a Debian sarge
> (testing) system, and qrunner goes into an apparent infinite loop as
> soon as it starts.  Here's a snapshot from "top" shortly after running
> "/etc/init.d/mailman start":
> 
> 
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  Command           
>  4498 list      19   0  4656 4656 1844 R 99.7  0.5   0:48.36 python            
>  4484 gward      9   0   940  940  748 R  0.3  0.1   0:00.17 top               
> 
> Memory use is stable, hence my assumption that this is a garden-variety
> infinite loop.
> 
> And "ps -fu list" reports this:
> 
> UID        PID  PPID  C STIME TTY          TIME CMD
> list      4492     1  0 02:42 ?        00:00:00 [mailmanctl]
> list      4493  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=ArchRunner:0:1 -s
> list      4494  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=BounceRunner:0:1 -s
> list      4495  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=CommandRunner:0:1 -s
> list      4496  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s
> list      4497  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=NewsRunner:0:1 -s
> list      4498  4492 97 02:42 ?        00:01:01 qrunner /var/lib/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s
> list      4499  4492  0 02:42 ?        00:00:00 qrunner /var/lib/mailman/bin/qrunner --runner=VirginRunner:0:1 -s
> 
> ... so it's just one qrunner (OutgoingRunner) that's responsible for all
> the CPU sucking.
> 
> Hmmm: there *are* files in the queue, but I'm not sure what they're
> doing there or how they got there (this is a new, fairly inactive system
> with no lists and no incoming traffic, apart from me testing things):
> 
> # cd /var/lib/mailman/
> # find qfiles/ -type f
> qfiles/out/1061532001.524745+03e15ef2a9a7d27904406a659cdb4922a204431b.pck
> qfiles/out/1061497885.874818+cf5f7e2e03cbca8bb84a23a60f055679c16e81fa.pck
> qfiles/out/1061704802.107101+4617d8731ecb303dd756e475b5362bd5a64b1fcf.pck
> qfiles/out/1061497885.871999+3453d7caa05a1aad6b230da61b39d4937405fcc8.pck
> qfiles/out/1061497885.871999+3453d7caa05a1aad6b230da61b39d4937405fcc8.db
> qfiles/out/1061497885.874818+cf5f7e2e03cbca8bb84a23a60f055679c16e81fa.db
> qfiles/out/1061704802.107101+4617d8731ecb303dd756e475b5362bd5a64b1fcf.db
> qfiles/out/1061532001.524745+03e15ef2a9a7d27904406a659cdb4922a204431b.db
> 
> Any clues?  I'll try to investigate more tomorrow evening.

While I have seen qrunners suck up all available cpu occasionally, it's
usually for manageable periods of time.  I haven't actually seen true
infloops, although I won't discount it.  ArchRunner and BounceRunner are
the worst offenders, the former because Pipermail is very inefficient,
and the latter because of the stupid way bounces are registered.  I
actually have a fix for the BounceRunner issue in cvs, which I intend on
installing on python.org to test out <wink>.

OutgoingRunner should be fairly sane since all it's doing is reading the
files from disk and spewing them over port 25 to your local smptd.  It's
also doing logging so you could tail logs/{smtp,smtp-failure,post} to
watch it make progress.  You should also see the qfiles/out directory
grow and shrink as files are consumed and unlinked, or new ones are
prepared for sending out.

-Barry



From gward at python.net  Tue Aug 26 09:14:51 2003
From: gward at python.net (Greg Ward)
Date: Tue Aug 26 08:14:55 2003
Subject: [Mailman-Developers] qrunner infinite loop?
In-Reply-To: <1061865463.12874.38.camel@anthem>
References: <20030825004907.GA19219@cthulhu.gerg.ca>
	<1061865463.12874.38.camel@anthem>
Message-ID: <20030826121451.GA22195@cthulhu.gerg.ca>

On 25 August 2003, Barry Warsaw said:
> OutgoingRunner should be fairly sane since all it's doing is reading the
> files from disk and spewing them over port 25 to your local smptd.  It's
> also doing logging so you could tail logs/{smtp,smtp-failure,post} to
> watch it make progress.  You should also see the qfiles/out directory
> grow and shrink as files are consumed and unlinked, or new ones are
> prepared for sending out.

At one point, Mailman was running for several hours, and it consumed
exactly as much CPU time as elapsed time since it was started.  Sounds
like an infinite loop to me.

It turns out that Mailman (or possibly Exim) were misconfigured on the
host in question (drydock.python.net, for context).  Mailman had been
configured to use "python2.net" for its email domain -- which is fine as
far as DNS is concerned, but Exim on that host was *not* told that
python2.net is one of its local domains.  Here's what might have
happened if Mailman had connected to Exim to send a message:

  mail from:<mailman-bounces-...@python2.net>
  250 OK
  rcpt to:<gward@python.net>
  451 Temporary local problem - please try later

Two glitches with that theory though:

  * I ran "tcpdump -i lo" while qrunner was in its infinite-looking
    loop, and there was no traffic -- so Mailman was apparently not
    connecting to Exim

  * between the "rcpt to" and the 451 is a noticeable delay --
    between 0.5 and 1.0 sec I would say.  Not sure what Exim is doing
    there (maybe DNS is slow on this host?), but qrunner should
    not be consuming CPU while waiting for Exim's 451.

Anyways, I fixed Mailman's configuration and deleted qfiles/*/*, and now
qrunner runs happily.  Well, it least it doesn't suck CPU.  And I have a
clue how to reproduce this problem in case anyone cares to give it a
shot.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
Condense soup, not books!

From cain.yuan at intel.com  Tue Aug 19 12:00:20 2003
From: cain.yuan at intel.com (Yuan, Cain)
Date: Wed Aug 27 07:14:36 2003
Subject: [Mailman-Developers] patch uploaded -- automatic discard
Message-ID: <37FBBA5F3A361C41AB7CE44558C3448E011679B5@pdsmsx403.ccr.corp.intel.com>

Hi Kikuchi,
	I am using the patch you uploaded to sourceforge.net, named mailman-2.1.2+patch.20030802, which add unicode support to the header and also the mail message. After apply the patch the header and mail message of a mail  is OK now but I have a problem in module admindb.py. When a non-member user post a mail to the list,  it will be blocked for the Moderator to approve/reject/discard, but the Web UI can't show the header and mail text correctly.  I can see from your patch that you have add some unicode decode/encode to the  original message to show it correctly but I am not lucky... Would you please give more some hints?  Thanks!

Best regards.

Cain


-----Original Message-----
From: Tokio Kikuchi [mailto:tkikuchi@is.kochi-u.ac.jp]
Sent: Tuesday, August 19, 2003 10:38 AM
To: mailman-developers@python.org
Subject: [Mailman-Developers] patch uploaded -- automatic discard


Hi Mailman Developers,

I have uploaded a patch for automatic discard held messages.

1. site wide default of MAX_DAYS_TO_HOLD in Defaults.py
    (mm_cfg.py).

2. list admin can set list specific max_days_to_hold
    through the Web GUI (admin/<listname>/general).

3. integrated in cron/checkdbs and reports the number
    of discarded messages.

You may have to check Version.py if you have installed other
patches.

Barry, how about to include this in 2.1.3 ?

Cheers,

-- 
Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/


_______________________________________________
Mailman-Developers mailing list
Mailman-Developers@python.org
http://mail.python.org/mailman/listinfo/mailman-developers

From shaikli at yahoo.com  Mon Aug 18 23:38:41 2003
From: shaikli at yahoo.com (Nadim Shaikli)
Date: Wed Aug 27 07:14:40 2003
Subject: [Mailman-Developers] 'empty module name' error and shunting
In-Reply-To: <1061244904.1958.32.camel@yyz>
Message-ID: <20030819053841.80199.qmail@web14901.mail.yahoo.com>

--- Larry Price wrote:
> On Wednesday, August 13, 2003, at 04:56  PM, Nadim Shaikli wrote:
> > If I try to 'unshunt' the message - nothing happens and it simply pops
> > back into the qfiles/shunt dir.  I have hoards of people waiting for 
> > this to get resolved and I've tried asking anyone who would listen on
> > IRC (irc.freenode.net #mailman and/or #debian) to no avail.  The website
> > was no help at all (there is no mention of 'shunting' even and alot of
> > the info is rather dated).
> 
> A quick work around to allow mail to the list to keep flowing is to 
> move the digest mbox out of the way, and then run unshunt, this will
> allow messages that were held because of the failure of the digest
> process to go through.

So it sounds like there is potential problem in which a message does stop
others from proceeding (ie. there is blocking in some instances ?)

Then Barry Warsaw wrote:
> On Wed, 2003-08-13 at 19:56, Nadim Shaikli wrote:
> 
> > Well, in my case I'm on a debian system (code via apt-get) running
> > mailman-2.1.2-6.  I have about 12 mailing-lists all seem to function
> > without an issue except for one.  Anytime anyone mails to it, the
> > message gets "Shunned" and I see this error in the log/error file
> > (mind you the message is obsoletely fine - its as though there is
> > lock someone that is forcing all legit messages to be shunned).
> 
> One message getting shunted shouldn't affect any global state that will
> cause all subsequent messages to get shunted.  Think of them as an event
> unto themselves.
> 
> Please send me the .pck/.db pair of a shunted message.  If it's not a
> problem of your configuration or installation, I should be able to
> reproduce the bug.

Barry, as was noted in a private email to ya - the problem was solved with
the following patch (courtesy of Mohammed Elzubeir),

--- Scrubber.py.orig 2003-08-13 23:19:19.000000000 -0700
+++ Scrubber.py 2003-08-14 00:23:47.000000000 -0700
@@ -305,6 +305,8 @@
                     t = unicode(t, 'ascii', 'replace').encode('ascii')
                 try:
                     # Should use HTML-Escape, or try generalizing to UTF-8
+                   if len(charset) == 0:
+                       charset = 'us-ascii'
                     t = t.encode(charset, 'replace')
                 except (UnicodeError, LookupError):
                     t = t.encode(lcset, 'replace')

once we applied the patch and 'unshunt'ed everything went through.  We
did have a couple of messages with no Mime and Content-Type headers
(which are legal to have) and those seemed to have caused the problem.
A null charset within mailman caused our problems it seems and that
potential corner-case needs to be designed for.  As for a sample - I
tried mailing the affected list a simple "test" from this yahoo
account and it got shunted (as noted there is nothing peculiar about
the messages - mailman simply got into a state/setting where it started
shunting everything -- Larry Price notes a possible means to get into
the 'state' above).  And again, I really don't think it was/is a
configuration/installation issue since the other 12 lists work without
any complaints and/or problems.

Barry, just for grins - I will mail you a message (in private) that was
indeed shunted in case that makes a difference.

Regards and I hope the patch is of use.

 - Nadim


__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com

From fabiorossi60 at hotmail.com  Wed Aug 20 13:06:22 2003
From: fabiorossi60 at hotmail.com (Fabio Rossi)
Date: Wed Aug 27 07:14:45 2003
Subject: [Mailman-Developers] HTML code setting ???
Message-ID: <BAY8-DAV44xGba8S1NV00040abb@hotmail.com>

Help me please. In MailMan mailing list... 
What is the setting to put HTML code in footer and header of the mailman list ??? I use MailMan Mailing List of CPanel and HTML CODE IS NOT RECOGNIZED. I read all in pure text.
If i insert HTML code such as footer or header it result in a error. 
I read all messages in pure text. 
Not "GoodBye GoodBye GoodBye" but "<body>
<p>GoodBye GoodBye GoodBye GoodBye</p> </body>".

WHAT IS THE SETTING TO READ HTML CODE IN THE MAILING LIST MESSAGES (TO INSERT HEADER AND FOOTER IN HTML CODE) ????

Setting.
-----
General Options
-----
An introductory description - a few paragraphs - about the list. It will be included, as html, at the top of the listinfo page. Carriage returns will end a paragraph - see the details for more info. 
(Details for info) : 

<body>

<p>GoodBye GoodBye GoodBye </p>

</body>

OTHER SETTING IN GENERAL OPTION ???

-----------

----
Non-digest options
----
Header added to mail sent to regular list members 
(Details for msg_header)

%(info)s
%(description)s

IS THIS AN ERROR ?

-----------

----
Content filtering
----

Should Mailman filter the content of list traffic according to the settings below? 
(Edit filter_content)
NO 
CORRECT ???

Remove message attachments that don't have a matching content type. Leave this field blank to skip this filter test. 
(Details for pass_mime_types)

multipart/mixed
multipart/alternative
text/plain
text/html

CORRECT ???

Should Mailman convert text/html parts to plain text? This conversion happens after MIME attachments have been stripped. 
(Edit convert_html_to_plaintext)

NO

Action to take when a message matches the content filtering rules. 
(Details for filter_action)

Preserve
From sumeetp at hotmail.com  Wed Aug 20 10:24:59 2003
From: sumeetp at hotmail.com (sumeet)
Date: Wed Aug 27 07:14:49 2003
Subject: [Mailman-Developers] further empty module name woes
Message-ID: <BAY1-DAV479eA0UgvtO0003e80a@hotmail.com>

I have a list that i created ages ago, then suddenly at 9:51am w/ no prompting from me I started getting the error below. Previously it was related to localization issues, now even when i send a test message w or w/o text and/or subject from the mailman host it gets shunted immediately. I'm sending this to developers as well, b/c Tokio was able to solve my problem -- it just may be too much of an issue for the users list. Please reply to me. thx.
 I've compiled my own version of Python 2.2.2:
[root@mailman qfiles]# /usr/local/bin/python -V
Python 2.2.2
On the following OS:
[root@mailman qfiles]# uname -a
Linux mailman.vmware.com 2.4.20-8 #2 SMP Fri Jul 11 14:46:57 PDT 2003 i686 i686 i386 GNU/Linux

The error log says:Aug 19 14:12:14 2003 (23553) Uncaught runner exception: Empty module name
Aug 19 14:12:14 2003 (23553) Traceback (most recent call last):
  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 105, in _oneloop
    self._onefile(msg, msgdata)
  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 155, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/usr/local/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/usr/local/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 89, in process
    send_digests(mlist, mboxfp)
  File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 130, in send_digests
    send_i18n_digests(mlist, mboxfp)
  File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 303, in send_i18n_digests
    msg = scrubber(mlist, msg)
  File "/usr/local/mailman/Mailman/Handlers/Scrubber.py", line 308, in process
    t = t.encode(charset, 'replace')
  File "/usr/local/lib/python2.2/encodings/__init__.py", line 51, in search_function
    mod = __import__(modname,globals(),locals(),'*')
ValueError: Empty module name
From paul at webfirm.com.au  Thu Aug 21 17:33:14 2003
From: paul at webfirm.com.au (Paul Bui)
Date: Wed Aug 27 07:14:52 2003
Subject: [Mailman-Developers] Could not acquire qrunner lock
Message-ID: <ANEOIEEFGKICAPDEONBOKECGHCAA.paul@webfirm.com.au>

Any idea how to fix this?

Aug 21 15:49:02 2003 (15208) Could not acquire qrunner lock
Aug 21 15:50:01 2003 (15278) Could not acquire qrunner lock
Aug 21 15:51:01 2003 (15339) Could not acquire qrunner lock
Aug 21 15:52:01 2003 (15395) Could not acquire qrunner lock
Aug 21 15:53:02 2003 (15449) Could not acquire qrunner lock
Aug 21 15:54:03 2003 (15499) Could not acquire qrunner lock
Aug 21 15:55:03 2003 (15556) Could not acquire qrunner lock
Aug 21 15:56:01 2003 (15616) Could not acquire qrunner lock
Aug 21 15:57:02 2003 (15676) Could not acquire qrunner lock
Aug 21 15:58:02 2003 (15738) Could not acquire qrunner lock
Aug 21 15:59:02 2003 (15786) Could not acquire qrunner lock
Aug 21 16:00:03 2003 (15847) Could not acquire qrunner lock

EOF read where object expected
Aug 21 10:49:02 2003 (27407) Could not acquire qrunner lock
Aug 21 10:50:01 2003 (27461) Could not acquire qrunner lock
Aug 21 10:51:02 2003 (27518) Could not acquire qrunner lock
Aug 21 10:52:02 2003 (27559) Could not acquire qrunner lock
Aug 21 10:53:00 2003 (27618) Exception reading qfile:
/usr/local/mailman/qfiles/
7e895325570c0d5071856ffc66c4b587736614cf
From sumeetp at hotmail.com  Fri Aug 22 01:06:32 2003
From: sumeetp at hotmail.com (sumeet)
Date: Wed Aug 27 07:14:56 2003
Subject: [Mailman-Developers] Fw: further empty module name woes
Message-ID: <BAY1-DAV30aiifLPrCN00035022@hotmail.com>

I have a list that i created ages ago, then suddenly at 9:51am w/ no prompting from me I started getting the error below. Previously it was related to localization issues, now even when i send a test message w or w/o text and/or subject from the mailman host it gets shunted immediately. I'm sending this to developers as well, b/c Tokio was able to solve my problem -- it just may be too much of an issue for the users list. Please reply to me. thx.
 I've compiled my own version of Python 2.2.2:
[root@mailman qfiles]# /usr/local/bin/python -V
Python 2.2.2
On the following OS:
[root@mailman qfiles]# uname -a
Linux mailman.vmware.com 2.4.20-8 #2 SMP Fri Jul 11 14:46:57 PDT 2003 i686 i686 i386 GNU/Linux

The error log says:Aug 19 14:12:14 2003 (23553) Uncaught runner exception: Empty module name
Aug 19 14:12:14 2003 (23553) Traceback (most recent call last):
  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 105, in _oneloop
    self._onefile(msg, msgdata)
  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 155, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/usr/local/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/usr/local/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 89, in process
    send_digests(mlist, mboxfp)
  File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 130, in send_digests
    send_i18n_digests(mlist, mboxfp)
  File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 303, in send_i18n_digests
    msg = scrubber(mlist, msg)
  File "/usr/local/mailman/Mailman/Handlers/Scrubber.py", line 308, in process
    t = t.encode(charset, 'replace')
  File "/usr/local/lib/python2.2/encodings/__init__.py", line 51, in search_function
    mod = __import__(modname,globals(),locals(),'*')
ValueError: Empty module name
From madsen at iki.fi  Thu Aug 21 18:39:19 2003
From: madsen at iki.fi (Madsen Wikholm)
Date: Wed Aug 27 07:14:59 2003
Subject: [Mailman-Developers] Non-member post confirmation
Message-ID: <20030821143919.GA16967@ummagumma>

Hi!

Lately I've been getting a lot of spam to many of the lists that I
administer.

While reading an article about spam-prevention it hit me that one way to
easily cut down on spam on an open list, would be to have mailman send out a
confirmation message for non-member posts. If the sender confirms then,
address gets added to the accept_these_nonmembers, if not the message gets
dropped after some time and the adress perhaps added to
discard_these_nonmembers.

I think there should perhaps be a way to tell mailman that the From: address
was forged, so forget the whole deal...

cheers,
madsen

From mgalgoci at redhat.com  Tue Aug 26 18:06:00 2003
From: mgalgoci at redhat.com (Matthew Galgoci)
Date: Wed Aug 27 07:15:03 2003
Subject: [Mailman-Developers] Handling of private gzipped archives
Message-ID: <Pine.LNX.4.44.0308261651360.2584-100000@lacrosse.corp.redhat.com>


Hi Folks!

I've been bit by a bug in mailman 2.0.1 where mailman assumes that there will always be an
uncompressed version of a monthly private archive file hanging around, which for my setup 
isn't the case. Path truncation in the content_type routine wrongly chomps off .gz extension.

Here is how I fixed the bug:

--- mailman/Mailman/Cgi/private.py.orig Tue Aug 26 15:24:04 2003
+++ mailman/Mailman/Cgi/private.py      Tue Aug 26 16:06:04 2003
@@ -84,7 +84,8 @@
  
 def content_type(path):
     if path[-3:] == '.gz':
-        path = path[:-3]
+        # path = path[:-3]
+        return 'application/x-gzip'
     if path[-4:] == '.txt':
         return 'text/plain'
     return 'text/html'
@@ -186,9 +187,9 @@
             f = open(os.path.join(mlist.archive_directory + '.mbox',
                                   mlist.internal_name() + '.mbox'))
             ctype = 'text/plain'
-        elif true_filename[-3:] == '.gz':
-            import gzip
-            f = gzip.open(true_filename, 'r')
+        #elif true_filename[-3:] == '.gz':
+        #    import gzip
+        #    f = gzip.open(true_filename, 'r')
         else:
             f = open(true_filename, 'r')
     except IOError:


It also occured to me that mailman is bothering to decompress compressed archive files before sending them 
back to the web client. This is wrong in many ways:

1) extra load decompressing the file
2) clients getting a file back with a .gz extention that is really a text file
3) extra bandwidth consumed while sending the uncompressed file back to the client
4) many web browsers will silently decompress application/x-gzip mime data automatically

I checked Mailman 2.1.2 to see if any of this is still done, and apparently the decompression issue still is, at
least in the copy I have here. (haven't checked cvs head). Here's my version of a fix for this:

--- mailman-2.1.2/Mailman/Cgi/private.py.orig   2003-08-26 16:46:16.000000000 -0400
+++ mailman-2.1.2/Mailman/Cgi/private.py        2003-08-26 16:47:21.000000000 -0400
@@ -148,8 +148,8 @@
                                   mlist.internal_name() + '.mbox'))
             ctype = 'text/plain'
         elif true_filename.endswith('.gz'):
-            import gzip
-            f = gzip.open(true_filename, 'r')
+            ctype = 'application/x-gzip'
+            f = open(true_filename, 'r')
         else:
             f = open(true_filename, 'r')
     except IOError:


Most folks on non-Linux-like operating systems have some kind of decompression software available that can handle gz format
even if their browser does not.

Kind Regards,

Matthew Galgoci

-- 

Matthew Galgoci		"Dirty deeds, done dirt cheap"
System Administrator
Red Hat, Inc
919.754.3700 x44155


From zeratul2 at wanadoo.es  Wed Aug 27 03:00:55 2003
From: zeratul2 at wanadoo.es (Robert Millan)
Date: Wed Aug 27 07:15:07 2003
Subject: [Mailman-Developers] ASK-like anti-spam feature
Message-ID: <20030827020055.GA702@aragorn>


Hello!

Does GNU mailman support anti-spam whitelisting in the style of ASK [1]?

If not, is anyone working/interested on it? I think it's really worth having
such a feature.

Thanks. (I'm so sick of spam...)

[1] http://www.paganini.net/ask/

-- 
Robert Millan

"[..] but the delight and pride of Aule is in the deed of making, and in the
thing made, and neither in possession nor in his own mastery; wherefore he
gives and hoards not, and is free from care, passing ever on to some new work."

 -- J.R.R.T, Ainulindale (Silmarillion)

From zeratul2 at wanadoo.es  Wed Aug 27 17:58:06 2003
From: zeratul2 at wanadoo.es (Robert Millan)
Date: Wed Aug 27 11:01:10 2003
Subject: [Mailman-Developers] Non-member post confirmation
In-Reply-To: <20030821143919.GA16967@ummagumma>
References: <20030821143919.GA16967@ummagumma>
Message-ID: <20030827165806.GA552@aragorn>



Sounds a lot like ASK (Active Spam Killer)

 http://www.paganini.net/ask/

On Thu, Aug 21, 2003 at 05:39:19PM +0300, Madsen Wikholm wrote:
> Hi!
> 
> Lately I've been getting a lot of spam to many of the lists that I
> administer.
> 
> While reading an article about spam-prevention it hit me that one way to
> easily cut down on spam on an open list, would be to have mailman send out a
> confirmation message for non-member posts. If the sender confirms then,
> address gets added to the accept_these_nonmembers, if not the message gets
> dropped after some time and the adress perhaps added to
> discard_these_nonmembers.
> 
> I think there should perhaps be a way to tell mailman that the From: address
> was forged, so forget the whole deal...
> 
> cheers,
> madsen
> 
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers@python.org
> http://mail.python.org/mailman/listinfo/mailman-developers

-- 
Robert Millan

"[..] but the delight and pride of Aule is in the deed of making, and in the
thing made, and neither in possession nor in his own mastery; wherefore he
gives and hoards not, and is free from care, passing ever on to some new work."

 -- J.R.R.T, Ainulindale (Silmarillion)

From brad.knowles at skynet.be  Wed Aug 27 17:57:52 2003
From: brad.knowles at skynet.be (Brad Knowles)
Date: Wed Aug 27 11:25:06 2003
Subject: [Mailman-Developers] HTML code setting ???
In-Reply-To: <BAY8-DAV44xGba8S1NV00040abb@hotmail.com>
References: <BAY8-DAV44xGba8S1NV00040abb@hotmail.com>
Message-ID: <a06001a3ebb7277353b9c@[10.0.1.2]>

At 12:06 PM +0200 2003/08/20, Fabio Rossi wrote:

>  What is the setting to put HTML code in footer and header of the mailman
>  list ??? I use MailMan Mailing List of CPanel and HTML CODE IS NOT
>  RECOGNIZED. I read all in pure text.

	See the archive.  The answer is that this problem is much more 
complex than you think it is (for detailed technical reasons), and 
therefore:

		THERE IS NO SUCH OPTION.  PLEASE STOP SHOUTING.

-- 
Brad Knowles, <brad.knowles@skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

From terri at zone12.com  Wed Aug 27 13:31:15 2003
From: terri at zone12.com (Terri Oda)
Date: Wed Aug 27 12:21:27 2003
Subject: Challenge-Response (Re: [Mailman-Developers] Non-member post
	confirmation)
In-Reply-To: <20030821143919.GA16967@ummagumma>
References: <20030821143919.GA16967@ummagumma>
Message-ID: <20030827163115.GE775@ostraya.zone12.com>

On Thu, Aug 21, 2003 at 05:39:19PM +0300, Madsen Wikholm wrote:
> While reading an article about spam-prevention it hit me that one way to
> easily cut down on spam on an open list, would be to have mailman send out a
> confirmation message for non-member posts. If the sender confirms then,
> address gets added to the accept_these_nonmembers, if not the message gets
> dropped after some time and the adress perhaps added to
> discard_these_nonmembers.

That'd be handy.  I wouldn't mind seeing a patch for that, or helping if
necessary (though I should be careful about volunteering time since I'm
starting a new job in September and haven't a clue how much time I'll
have... there's still documentation I want to write.)

On a related note -- have other people been having trouble with
challenge-response using susbscribers on their lists?  Linuxchix has had
some "Dear grrltalk-bounces" messages which are kinda annoying.  We can't
really take the time to jump through those hoops and read every uncaught
bounce, so we've just asked people to whitelist us themselves.

 Terri


From wheakory at isu.edu  Wed Aug 27 13:12:09 2003
From: wheakory at isu.edu (Kory Wheatley)
Date: Wed Aug 27 14:12:58 2003
Subject: [Mailman-Developers] Python Book
Message-ID: <3F4CF479.3090200@isu.edu>

What is the best book to learn Python, for a beginner and than another 
book for the advanced features?

-- 
Kory Wheatley
Academic Computing Analyst Sr.
Phone 282-3874
#########################################
Everything must point to him.



From barry at python.org  Thu Aug 28 02:59:08 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 27 21:59:10 2003
Subject: [Mailman-Developers] Python Book
In-Reply-To: <3F4CF479.3090200@isu.edu>
References: <3F4CF479.3090200@isu.edu>
Message-ID: <1062035915.998.38.camel@anthem>

On Wed, 2003-08-27 at 14:12, Kory Wheatley wrote:
> What is the best book to learn Python, for a beginner and than another 
> book for the advanced features?

I haven't read any of the more recent Python books.  I learned Python by
reading the tutorial, which is still a good, cheap way to learn Python
especially if you have other programming experience.

http://www.python.org/cgi-bin/moinmoin/IntroductoryBooks

is a link to introductory Python books.

Cheers,
-Barry



From barry at python.org  Thu Aug 28 03:04:09 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 27 22:04:10 2003
Subject: [Mailman-Developers] Non-member post confirmation
In-Reply-To: <20030821143919.GA16967@ummagumma>
References: <20030821143919.GA16967@ummagumma>
Message-ID: <1062036217.998.43.camel@anthem>

On Thu, 2003-08-21 at 10:39, Madsen Wikholm wrote:

> While reading an article about spam-prevention it hit me that one way to
> easily cut down on spam on an open list, would be to have mailman send out a
> confirmation message for non-member posts. If the sender confirms then,
> address gets added to the accept_these_nonmembers, if not the message gets
> dropped after some time and the adress perhaps added to
> discard_these_nonmembers.

Robert Millan mentioned ASK, which I hadn't heard about before, but it
also makes me think of TMDA.

I'd really like to add something like this, perhaps including a
subscription option (likely via a web confirmation).  We'd put a
governor on the confirmation offerings so forged virus bombs wouldn't
inundate the innocent.

I'm beginning to think that I may have to put out a Mailman 2.2 and if
so, this feature would be a good candidate.

-Barry



From barry at python.org  Thu Aug 28 03:08:31 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 27 22:08:32 2003
Subject: [Mailman-Developers] Handling of private gzipped archives
In-Reply-To: <Pine.LNX.4.44.0308261651360.2584-100000@lacrosse.corp.redhat.com>
References: <Pine.LNX.4.44.0308261651360.2584-100000@lacrosse.corp.redhat.com>
Message-ID: <1062036479.998.47.camel@anthem>

On Tue, 2003-08-26 at 17:06, Matthew Galgoci wrote:
> Hi Folks!

Hi Matthew!

> I checked Mailman 2.1.2 to see if any of this is still done, and apparently the decompression issue still is, at
> least in the copy I have here. (haven't checked cvs head). Here's my version of a fix for this:
> 
> --- mailman-2.1.2/Mailman/Cgi/private.py.orig   2003-08-26 16:46:16.000000000 -0400
> +++ mailman-2.1.2/Mailman/Cgi/private.py        2003-08-26 16:47:21.000000000 -0400
> @@ -148,8 +148,8 @@
>                                    mlist.internal_name() + '.mbox'))
>              ctype = 'text/plain'
>          elif true_filename.endswith('.gz'):
> -            import gzip
> -            f = gzip.open(true_filename, 'r')
> +            ctype = 'application/x-gzip'
> +            f = open(true_filename, 'r')
>          else:
>              f = open(true_filename, 'r')
>      except IOError:

Could you please submit a SF patch on this?  Less of a chance of getting
lost in my inbox. ;)

-Barry



From madsen at iki.fi  Thu Aug 28 11:00:17 2003
From: madsen at iki.fi (Madsen Wikholm)
Date: Thu Aug 28 03:00:29 2003
Subject: [Mailman-Developers] Non-member post confirmation
In-Reply-To: <1062036217.998.43.camel@anthem>
References: <20030821143919.GA16967@ummagumma> <1062036217.998.43.camel@anthem>
Message-ID: <20030828070016.GE2166@ummagumma>

* Barry Warsaw (barry@python.org) wrote:
> Robert Millan mentioned ASK, which I hadn't heard about before, but it
> also makes me think of TMDA.

Yupp, challenge/response was exactly what I was thinking of. One could
ofcourse use the TMDA or ASK for this but being included in mailman directly
would simplify the process very much.

I'm not really for using the challenge/response system for personal mail since
I get much mail from unknown senders. But for a list that would normally be
open it would cut down on the spam. I used to have my lists open but due to
spam I've closed them which generates a lot of approval messages :-(

One could argue that you could as well subscribe to the list if you want to
send a mail to it but then you get all the rest of the mail that you might not
be interested in.

> I'd really like to add something like this, perhaps including a
> subscription option (likely via a web confirmation).

In additon to the full subscription option there could perhaps be a thread
subscription which would subscribe the sender to only receive mails from the
thread started by him/her. This might become somewhat heavy though.

> We'd put a governor on the confirmation offerings so forged virus bombs
> wouldn't inundate the innocent.

Not that I need to know but what do you mean with governor?

just my 2 cents
madsen

From barry at python.org  Thu Aug 28 15:22:30 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Aug 28 10:24:54 2003
Subject: [Mailman-Developers] Non-member post confirmation
In-Reply-To: <20030828070016.GE2166@ummagumma>
References: <20030821143919.GA16967@ummagumma>
	<1062036217.998.43.camel@anthem>  <20030828070016.GE2166@ummagumma>
Message-ID: <1062080510.2954.3.camel@yyz>

On Thu, 2003-08-28 at 03:00, Madsen Wikholm wrote:

> I'm not really for using the challenge/response system for personal mail since
> I get much mail from unknown senders. But for a list that would normally be
> open it would cut down on the spam. I used to have my lists open but due to
> spam I've closed them which generates a lot of approval messages :-(

You and me both.  Since I feel your pain, I'll probably end up doing
something about this in 2.1.3 (e.g. limit the number of approvals sent
to the admins per day).

> One could argue that you could as well subscribe to the list if you want to
> send a mail to it but then you get all the rest of the mail that you might not
> be interested in.

Another thing I'd like to do is to make it an option to free all the
held messages of someone who posted before they were a member, and then
became a member, without explicit admin approval.

> In additon to the full subscription option there could perhaps be a thread
> subscription which would subscribe the sender to only receive mails from the
> thread started by him/her. This might become somewhat heavy though.

It's a great idea that's been bantied about for years (e.g. Roundup's
nosy lists).  But that would have to be a feature for a 2.2 release.

> > We'd put a governor on the confirmation offerings so forged virus bombs
> > wouldn't inundate the innocent.
> 
> Not that I need to know but what do you mean with governor?

Ah, a regulator.  I.e. something that stops Mailman from sending too
many confirmation messages during a certain period of time.  There are
actually already such things in the autoresponder, but more of the
system needs to be put on it.

-Barry



From amk at asti-usa.com  Thu Aug 28 08:05:21 2003
From: amk at asti-usa.com (A.M. Kuchling)
Date: Thu Aug 28 10:26:44 2003
Subject: [Mailman-Developers] Non-member post confirmation
In-Reply-To: <1062036217.998.43.camel@anthem>
References: <20030821143919.GA16967@ummagumma> <1062036217.998.43.camel@anthem>
Message-ID: <20030828110521.GA8835@vail.asti-usa.com>

On Wed, Aug 27, 2003 at 10:03:38PM -0400, Barry Warsaw wrote:
> I'm beginning to think that I may have to put out a Mailman 2.2 and if
> so, this feature would be a good candidate.

I'd also really like to get my RSS patch in (patch #657951).

--amk

From brad.knowles at skynet.be  Thu Aug 28 23:42:40 2003
From: brad.knowles at skynet.be (Brad Knowles)
Date: Thu Aug 28 17:01:54 2003
Subject: [Mailman-Developers] Non-member post confirmation
In-Reply-To: <1062080510.2954.3.camel@yyz>
References: <20030821143919.GA16967@ummagumma>
	<1062036217.998.43.camel@anthem> 
	<20030828070016.GE2166@ummagumma> <1062080510.2954.3.camel@yyz>
Message-ID: <a06001a65bb7416c6a543@[10.0.1.2]>

At 10:21 AM -0400 2003/08/28, Barry Warsaw wrote:

>  Another thing I'd like to do is to make it an option to free all the
>  held messages of someone who posted before they were a member, and then
>  became a member, without explicit admin approval.

	I'd also like to see an update to the spam checking.

	For example, for closed lists, I'd like to have a default action 
of holding the message if it comes from a subscriber and matches the 
spam regex.  But I'd like to have a default action of discarding the 
message if it comes from a non-subscriber and matches the spam regex.

>>  Not that I need to know but what do you mean with governor?
>
>  Ah, a regulator.  I.e. something that stops Mailman from sending too
>  many confirmation messages during a certain period of time.  There are
>  actually already such things in the autoresponder, but more of the
>  system needs to be put on it.

	There is already the option "Should the list moderators get 
immediate notice of new requests, as well as daily notices about 
collected ones?", which I was very grateful to be able to turn off.

	I also made the source-code modification so that the default 
action for held messages is "discard" as opposed to "defer".  This 
way, I can get rid of hundreds of held messages in a single button 
click.  Yes, more dangerous, but a lifesaver during the recent 
SoBig.F mess.

	But I'd prefer not to get most of those held messages in the 
first place, since they were from non-subscribers and definitely 
matched the spam regex (after a bit of tweaking the SpamAssassin 
configuration).

-- 
Brad Knowles, <brad.knowles@skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

From jwblist at olympus.net  Thu Aug 28 20:41:58 2003
From: jwblist at olympus.net (John W Baxter)
Date: Thu Aug 28 22:42:28 2003
Subject: [Mailman-Developers] Non-member post confirmation
In-Reply-To: <1062080510.2954.3.camel@yyz>
References: <20030821143919.GA16967@ummagumma>
	<1062036217.998.43.camel@anthem>  <20030828070016.GE2166@ummagumma>
	<1062080510.2954.3.camel@yyz>
Message-ID: <p05210601bb746d697911@[192.168.1.50]>

At 10:21 -0400 8/28/2003, Barry Warsaw wrote:
>> In additon to the full subscription option there could perhaps be a thread
>> subscription which would subscribe the sender to only receive mails from the
>> thread started by him/her. This might become somewhat heavy though.
>
>It's a great idea that's been bantied about for years (e.g. Roundup's
>nosy lists).  But that would have to be a feature for a 2.2 release.

And would have problems with uncooperative MUAs and uncooperative MUA
users, either of which could take an "interesting" (to the restricted
subscriber) message outside the thread.

Good luck.

Are Topics working well enough to be useful (I haven't tried any)?

  --John
-- 
John Baxter   jwblist@olympus.net      Port Ludlow, WA, USA

From jwblist at olympus.net  Thu Aug 28 20:47:37 2003
From: jwblist at olympus.net (John W Baxter)
Date: Thu Aug 28 22:49:42 2003
Subject: [Mailman-Developers] Non-member post confirmation
In-Reply-To: <a06001a65bb7416c6a543@[10.0.1.2]>
References: <20030821143919.GA16967@ummagumma>
	<1062036217.998.43.camel@anthem> 	<20030828070016.GE2166@ummagumma>
	<1062080510.2954.3.camel@yyz> <a06001a65bb7416c6a543@[10.0.1.2]>
Message-ID: <p05210602bb746e2da71f@[192.168.1.50]>

At 22:42 +0200 8/28/2003, Brad Knowles wrote:
>	There is already the option "Should the list moderators get
>immediate notice of new requests, as well as daily notices about
>collected ones?", which I was very grateful to be able to turn off.

Perhaps more useful the list admin could set periods other than "daily"
(millenniumly is tempting but too long ;-)), and the timing (eg, tell me at
6AM, Noon, and 4PM).  Or "per admin or moderator":  tell Barry at 6AM; tell
John at Noon; etc.

A lot of code...would it be used?

   --John

-- 
John Baxter   jwblist@olympus.net      Port Ludlow, WA, USA

From terri at zone12.com  Fri Aug 29 12:45:57 2003
From: terri at zone12.com (Terri Oda)
Date: Fri Aug 29 11:35:28 2003
Subject: [Mailman-Developers] Non-member post confirmation
In-Reply-To: <a06001a65bb7416c6a543@[10.0.1.2]>
References: <20030821143919.GA16967@ummagumma> <1062036217.998.43.camel@anthem>
	<20030828070016.GE2166@ummagumma> <1062080510.2954.3.camel@yyz>
	<a06001a65bb7416c6a543@[10.0.1.2]>
Message-ID: <20030829154557.GB734@ostraya.zone12.com>

On Thu, Aug 28, 2003 at 10:42:40PM +0200, Brad Knowles wrote:
> 	I also made the source-code modification so that the default 
> action for held messages is "discard" as opposed to "defer".  This 
> way, I can get rid of hundreds of held messages in a single button 
> click.  Yes, more dangerous, but a lifesaver during the recent 
> SoBig.F mess.

A friend sent me a wonderful little bookmarklet for this, for those of you
who don't want to change the source but do get pages of discardables on
occasion.  It works well in Mozilla, although I haven't tried it in anything
else.

javascript:(function(){var elements=document.forms[0].elements;for(var
i=0;i<elements.length;i++){var
el=elements[i];if(el.type=='radio'&&el.value=='3'){el.checked=true;}}})()

 Terri

From pponton at eyebottomline.com  Thu Aug 28 14:29:35 2003
From: pponton at eyebottomline.com (Philip Ponton)
Date: Fri Sep  5 08:17:47 2003
Subject: [Mailman-Developers] Editing the public Html Pages
Message-ID: <1062095375.3f4e4a0f6b6d0@www.eyebottomline.com>

Does anyone know how to add in a section in the public html pages with a 
question that ask's subscribers to fill in there graphical location or city 
code. Please Help.. Thanks



From charlesg at unixrealm.com  Fri Aug 29 13:23:15 2003
From: charlesg at unixrealm.com (Charles Gagnon)
Date: Fri Sep  5 08:17:54 2003
Subject: [Mailman-Developers] 2.1.2/Solaris 8 Problems with OutgoingRunner
In-Reply-To: <20030827132341.GA8148@daumier.unixrealm.com>
References: <20030827115638.GC3679@daumier.unixrealm.com>
	<20030827121313.GA15128@daumier.unixrealm.com>
	<20030827132341.GA8148@daumier.unixrealm.com>
Message-ID: <20030829162315.GB12744@daumier.unixrealm.com>

Sorry to bother the developers with this but I tried the users list
of couple of times to no avail. Plus, it almost seem like there a
bug or an incombatibility with something somewhere...

I've just upgraded from Mailman 2.0.x to 2.1.2 using Python 2.2.2 on
a Solaris 8 box.

The "qrunner --runner=OutgoingRunner:0:1 -s" process starts okay and
stays okay to send one message (either confirmation or post) and
than is just hangs. It does not die but doesn't send anything else.
After that one message, even a "mailmanctl stop" won't stop it. I
have to manually kill -9 it. And outgoing stuff just accumulates in
qfiles/out.

I'm not sure what to do or where to look. I looked in all the logs
and tried running it by hand. Same result. I even tried to truss it
to see if I could find something.

Any ideas?

-- 
Charles Gagnon                   | My views are my views and they
http://unixrealm.com             | do not represent those of anybody
charlesg at unixrealm.com        | but me.

   640K ought to be enough for anybody.
        -- Bill Gates, 1981

From jochen at d-mission.be  Sun Aug 31 23:08:46 2003
From: jochen at d-mission.be (dj jochen [d-mission])
Date: Fri Sep  5 08:18:09 2003
Subject: [Mailman-Developers] confirmation
Message-ID: <00d201c36ffb$af2a8660$908876d5@jochim>

hi,

In the mailinglist I use the subscribers only can read the postings of the admins and moderators.
The only functionality the subscribers should have is subscribing and unsubscribing from the list by mail.
e.g. mailing-subscribe@domain.xx and mailing-unsubscribe@domain.xx

That's all ok, but for that use only my subscribers obtain to much information and get confused when they subscribe now.
because the got a mail like this:

<start of mail>

Mailing list subscription confirmation notice for mailing list
Mailman-Developers

We have received a request from 213.224.83.167 for subscription of
your email address, "jochen@d-mission.be", to the
mailman-developers@python.org mailing list.  To confirm that you want
to be added to this mailing list, simply reply to this message,
keeping the Subject: header intact.  Or visit this web page:

    http://mail.python.org/mailman/confirm/mailman-developers/blablabla

Or include the following line -- and only the following line -- in a
message to mailman-developers-request@python.org:

    confirm blablabla

Note that simply sending a `reply' to this message should work from
most mail readers, since that usually leaves the Subject: line in the
right form (additional "Re:" text in the Subject: is okay).

If you do not wish to be subscribed from this list, please simply
disregard this message.  If you think you are being maliciously
subscribed to the list, or have any other questions, send them to
mailman-developers-owner@python.org.

<end of mail>

Instead I should prefer that they only get a mail where they can read that they should confirm by replying with an empty mail, not by logging in etc.


idem dito for the welcome mail with all the urls. They're useless and only confusing for my subscribers.

Is there a way to change this?

I allready tried with the welcome_msg function but that doesn't replace the original text. 

tnx,
j