From ricardo@rixhq.nu Sat Apr 1 14:57:13 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sat, 1 Apr 2000 16:57:13 +0200
Subject: [Mailman-Developers] membership managent web interface
Message-ID: <20000401165713.B2495@miss-janet.com>
Hi,
I was thinking about a way to make to the email list on the admin page
a bit easier to use. As it works now, the only way to find a certain
email address, is by figuring out in which page block the email address is
and the look for it down there.
But most of the times you know the exact address you want to unsubscribe
or view/edit, so why not add a search box on top? Yes I now you can use the
scripts to remove a certain address, but not all admins can (or should)
have access to the shell prompt.
I even think that only a searchbox on the initial page should be enough,
and only show the paged list of members when the user specifically asks
for it... saves a bit of bandwidth / response time.
any thoughts about this?
Ricardo.
--
From jack@cybermail.net Sat Apr 1 17:26:12 2000
From: jack@cybermail.net (Jack Snodgrass)
Date: Sat, 1 Apr 2000 11:26:12 -0600
Subject: [Mailman-Developers] www.list.org ? - looking for mailman mailling list software
Message-ID: <021d01bf9bff$609f94a0$0564a8c0@private.net>
This is a multi-part message in MIME format.
------=_NextPart_000_021A_01BF9BCD.15E392E0
Content-Type: text/plain;
charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
can you tell me where I can find your mailman mailing list software?=20
I can't seen to find www.list.org. That's what freshmeat is pointing
to.
jack
------=_NextPart_000_021A_01BF9BCD.15E392E0
Content-Type: text/html;
charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
can you tell me where I can find your =
mailman=20
mailing list software?
I can't seen to find www.list.org. That's what freshmeat is=20
pointing
to.
jack
------=_NextPart_000_021A_01BF9BCD.15E392E0--
From ricardo@rixhq.nu Sat Apr 1 17:31:28 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sat, 1 Apr 2000 19:31:28 +0200
Subject: [Mailman-Developers] www.list.org ? - looking for mailman mailling list software
In-Reply-To: <021d01bf9bff$609f94a0$0564a8c0@private.net>; from jack@cybermail.net on Sat, Apr 01, 2000 at 11:26:12AM -0600
References: <021d01bf9bff$609f94a0$0564a8c0@private.net>
Message-ID: <20000401193128.B3354@miss-janet.com>
On Sat, Apr 01, 2000 at 11:26:12AM -0600, Jack Snodgrass wrote:
> can you tell me where I can find your mailman mailing list software?
> I can't seen to find www.list.org. That's what freshmeat is pointing
> to.
ftp://ftp.gnu.org/gnu/mailman/
Ricardo.
--
From rullfig@uchicago.edu Mon Apr 3 16:01:19 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Mon, 03 Apr 2000 10:01:19 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
Message-ID: <38E8B23F.16F25CE7@uchicago.edu>
So, in 1.0rc2, displaying the list of lists for 529 lists
requires 529**2 = 279841 system stat calls and takes over one
and a half minutes on our Ultra-2 2x296 processor system! Is
this because of Python, Mailman, or both? Has this been "fixed"
in 2.0? You really should only need to make one stat call per
list.
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From gorgo@sztaki.hu Mon Apr 3 16:20:20 2000
From: gorgo@sztaki.hu (Gergely Madarasz)
Date: Mon, 03 Apr 2000 17:20:20 +0200 (MET DST)
Subject: [Mailman-Developers] Bug#61695: mailman: mail->news gw blows up on mails without subject
(fwd)
Message-ID:
I just got this bugreport in the debian BTS. Comments ?
--
Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org
It's practically impossible to look at a penguin and feel angry.
Egy pingvinre gyakorlatilag lehetetlen haragosan nezni.
HuLUG: http://mlf.linux.rulez.org/
---------- Forwarded message ----------
Date: Mon, 03 Apr 2000 18:16:45 +0300
From: erno@iki.fi
To: submit@bugs.debian.org
Subject: Bug#61695: mailman: mail->news gw blows up on mails without subject
Resent-Date: Mon, 03 Apr 2000 15:18:02 +0000 (GMT)
Resent-From: erno@iki.fi
Resent-To: debian-bugs-dist@lists.debian.org
Resent-cc: Gergely Madarasz
Package: mailman
Version: 1.1-4
Severity: important
hello,
i have a local newsgroup, and i created a mailman list named test2,
telling mailman to gateway messages from it to the local newsgroup.
the news server is inn-1.7.2-16.
running
% echo seppo | mail test2@erno.iki.fi
leads to
From: Mail Delivery System
Subject: Mail delivery failed: returning message to sender
To: erno@erno.iki.fi
X-Failed-Recipients: test2@erno.iki.fi
Date: Mon, 03 Apr 2000 18:08:03 +0300
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. The following address(es) failed:
test2@erno.iki.fi:
generated |/var/lib/mailman/mail/wrapper post test2
The following text was generated during the delivery attempt:
------ |/var/lib/mailman/mail/wrapper post test2 ------
Traceback (innermost last):
File "/var/lib/mailman/scripts/post", line 73, in ?
mlist.Post(msg, approved=fromusenet)
File "/usr/lib/mailman/Mailman/MailList.py", line 1331, in Post
self.SendMailToNewsGroup(msg)
File "/usr/lib/mailman/Mailman/GatewayManager.py", line 143, in
SendMailToNewsGroup
msg.SetHeader('Subject', '%s(no subject)' % subjpref)
NameError: subjpref
------ This is a copy of the message, including all the headers. ------
Return-path:
Received: from erno by erno.iki.fi with local (Exim 3.12 #1 (Debian))
id 12c8SN-0004Jv-00
for ; Mon, 03 Apr 2000 18:07:59 +0300
To: test2@erno.iki.fi
Message-Id:
From: Erno Kuusela
Date: Mon, 03 Apr 2000 18:07:59 +0300
seppo
looks to me like this is bad, because if i want to use mailman to, say,
gateway messages from debian-devel to a local read-only newsgroup,
bounce messages will be sent to the listowner of debian-devel
and he will likely kick me off the list when he gets tired
of things like this.
also this is a crash, and if i understand correctly those should
generally filed as important bugs. (please tell me if i'm wrong)
-- System Information
Debian Release: 2.2
Kernel Version: Linux fun77 2.2.10 #1 Sun Jun 20 19:03:53 EEST 1999 i586 unknown
Versions of the packages mailman depends on:
ii apache 1.3.9-12 Versatile, high-performance HTTP server
ii cron 3.0pl1-57 management of regular background processing
ii libc6 2.1.3-7 GNU C Library: Shared libraries and Timezone
ii logrotate 3.2-11 Log rotation utility
ii python-base 1.5.2-9 An interactive object-oriented scripting lan
sendmail Not installed or no info
ii exim 3.12-7 Exim Mailer
^^^ (Provides virtual package mail-transport-agent)
ii apache 1.3.9-12 Versatile, high-performance HTTP server
^^^ (Provides virtual package httpd)
From rullfig@uchicago.edu Mon Apr 3 17:26:19 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Mon, 03 Apr 2000 11:26:19 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
Message-ID: <38E8C62B.37883725@uchicago.edu>
Roberto Ullfig wrote:
>
> So, in 1.0rc2, displaying the list of lists for 529 lists
> requires 529**2 = 279841 system stat calls and takes over one
> and a half minutes on our Ultra-2 2x296 processor system! Is
> this because of Python, Mailman, or both? Has this been "fixed"
> in 2.0? You really should only need to make one stat call per
> list.
>
Whatever the answer is, we'd like to be able to generate the
lists of lists once a day; I tried running:
chroot /opt/http /opt/bin/python
/opt/pkgs/mailman/scripts/driver listinfo
but get this traceback:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[----- Mailman Version: <undetermined> -----]
[----- Traceback ------]
Traceback (innermost last):
File "/opt/pkgs/mailman/scripts/driver", line 135, in
print_traceback
from Mailman.mm_cfg import VERSION
ImportError: No module named Mailman.mm_cfg
Content-type: text/html
Bug in Mailman version
<undetermined>
Bug in Mailman version <undetermined>
We're sorry, we hit a bug!
Looks like a I'm missing something fundamental here. I assume
that the output from this command would be html commands that
could be redirected to a file. Thank you!
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From dan.mick@West.Sun.COM Mon Apr 3 22:04:40 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Mon, 03 Apr 2000 14:04:40 -0700
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu> <38E8C62B.37883725@uchicago.edu>
Message-ID: <38E90768.FD8B372C@west.sun.com>
Roberto Ullfig wrote:
> ImportError: No module named Mailman.mm_cfg
> Content-type: text/html
>
> Bug in Mailman version
> <undetermined>
> Bug in Mailman version <undetermined>
> We're sorry, we hit a bug!
>
> Looks like a I'm missing something fundamental here.
Python's library-search path has to be set to include /home/mailman/Mailman.
I just discovered that bin/withlist doesn't do this recently; I'm guessing
the driver script doesn't do it either. For your purposes, I would
think that augmenting PYTHONPATH in the environment should suffice.
Adding "PYTHONPATH=$PYTHONPATH:/home/mailman/Mailman" makes it work for
me. BTW, the chroot seems unnecessary (maybe you knew that, and were
just trying for security or something).
From dan.mick@West.Sun.COM Mon Apr 3 22:10:00 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Mon, 03 Apr 2000 14:10:00 -0700
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu> <38E8C62B.37883725@uchicago.edu> <38E90768.FD8B372C@west.sun.com>
Message-ID: <38E908A8.36EE28DC@west.sun.com>
Dan Mick wrote:
>
> Roberto Ullfig wrote:
>
> > ImportError: No module named Mailman.mm_cfg
> > Content-type: text/html
> >
> > Bug in Mailman version
> > <undetermined>
> > Bug in Mailman version <undetermined>
> > We're sorry, we hit a bug!
> >
> > Looks like a I'm missing something fundamental here.
>
> Python's library-search path has to be set to include /home/mailman/Mailman.
> I just discovered that bin/withlist doesn't do this recently; I'm guessing
> the driver script doesn't do it either. For your purposes, I would
> think that augmenting PYTHONPATH in the environment should suffice.
>
> Adding "PYTHONPATH=$PYTHONPATH:/home/mailman/Mailman" makes it work for
> me. BTW, the chroot seems unnecessary (maybe you knew that, and were
> just trying for security or something).
BTW, "python -i /home/mailman/Mailman/Cgi/listinfo.py" gives the same
output, although again it needs the PYTHONPATH setting.
From ich@henning-schroeder.de Tue Apr 4 00:29:08 2000
From: ich@henning-schroeder.de (Henning Schroeder)
Date: Tue, 04 Apr 2000 01:29:08 +0200
Subject: [Mailman-Developers] Local characters (loike =?iso-8859-1?Q?=C4=D6=DC=DF?=) in pipermail
Message-ID: <38E92944.B5871C5@henning-schroeder.de>
Hello,
I just modified the latest version of Mailman/pipermail to support local
characters (like äöüß). Headers with "=?ISO-8859-1..." are really
disturbing :-(
Basically my code decodes every message header with the mimify module.
Is anyone interested in the source? It modifies Message.py, Mailbox.py,
pipermail.py and Hyperarch.py.
Besides I would like to implement a better MIME support in Mailman. Then
you could e.g. block binary attachments in the list or display
HTML-mails with inline-graphics in the archive.
Henning
From bwarsaw@cnri.reston.va.us Tue Apr 4 03:38:22 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 3 Apr 2000 22:38:22 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
Message-ID: <14569.21918.950338.815597@anthem.cnri.reston.va.us>
>>>>> "RU" == Roberto Ullfig writes:
RU> So, in 1.0rc2, displaying the list of lists for 529 lists
RU> requires 529**2 = 279841 system stat calls and takes over one
RU> and a half minutes on our Ultra-2 2x296 processor system! Is
RU> this because of Python, Mailman, or both? Has this been
RU> "fixed" in 2.0? You really should only need to make one stat
RU> call per list.
Uh, it's because of Mailman :)
I implemented a list_lists scripts which does on the command line what
listinfo.py does in HTML (see attached). Here's what truss -c gives
me:
-------------------- snip snip --------------------
Portal - [no description available]
Postal - [no description available]
Stage - Staging new Mailman releases
Test - [no description available]
syscall seconds calls errors
_exit .00 1
read .00 102
write .00 8
open .11 607 474
close .01 143
time .00 3
brk .03 227
stat .03 201 157
getpid .00 10
fstat .00 66
ioctl .02 63 61
execve .00 10 8
umask .00 2
fcntl .00 7
readlink .00 2 2
sigprocmask .00 2
sigaction .00 50
sigpending .00 1
mmap .00 42
mprotect .00 10
munmap .00 11
uname .00 4
sysconfig .00 1
lwp_create .00 6
lwp_continue .00 2
lwp_self .00 3
llseek .00 114
door .00 5
lwp_schedctl .01 5
getdents64 .01 15
fstat64 .00 67
open64 .00 7
---- --- ---
sys totals: .22 1797 702
usr time: .51
elapsed: 1.19
-------------------- snip snip --------------------
Getting the list of list names, requires at least a listdir() and an
exists() for every directory found there.
Nothing about this will change for 2.0.
-Barry
-------------------- snip snip --------------------
#! /usr/bin/env python
#
# Copyright (C) 1998,1999,2000 by the Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""List all mailing lists.
Usage: %(program)s [options]
Where:
--advertised
-a
List only those mailing lists that are publically advertised
--virtual-host-overview=domain
-V domain
List only those mailing lists that are homed to the given virtual
domain. This only works if the VIRTUAL_HOST_OVERVIEW variable is
set.
--help
-h
Print this text and exit.
"""
import sys
import getopt
import paths
from Mailman import mm_cfg
from Mailman import MailList
from Mailman import Utils
from Mailman import Errors
program = sys.argv[0]
def usage(status, msg=''):
print __doc__ % globals()
if msg:
print msg
sys.exit(status)
def main():
try:
opts, args = getopt.getopt(sys.argv[1:], 'aV:h',
['advertised', 'virtual-host-overview=',
'help'])
except getopt.error, msg:
usage(1, msg)
advertised = 0
vhost = None
for opt, arg in opts:
if opt in ('-h', '--help'):
usage(0)
elif opt in ('-a', '--advertised'):
advertised = 1
elif opt in ('-V', '--virtual-host-overview'):
vhost = arg
names = Utils.list_names()
names.sort()
mlists = []
longest = 0
for n in names:
mlist = MailList.MailList(n, lock=0)
if advertised and not mlist.advertised:
continue
if vhost and mm_cfg.VIRTUAL_HOST_OVERVIEW and \
string.find(vhost, mlist.web_page_url) == -1 and \
string.find(mlist.web_page_url, vhost) == -1:
continue
mlists.append(mlist)
longest = max(len(mlist.real_name), longest)
if not mlists:
print 'No matching mailing lists found'
return
format = '%%%ds - %%.%ds' % (longest, 77 - longest)
for mlist in mlists:
description = mlist.description or '[no description available]'
print format % (mlist.real_name, description)
if __name__ == '__main__':
main()
From bwarsaw@cnri.reston.va.us Tue Apr 4 03:47:16 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 3 Apr 2000 22:47:16 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
<38E8C62B.37883725@uchicago.edu>
Message-ID: <14569.22452.191246.382870@anthem.cnri.reston.va.us>
>>>>> "RU" == Roberto Ullfig writes:
| Whatever the answer is, we'd like to be able to generate the
| lists of lists once a day; I tried running:
| chroot /opt/http /opt/bin/python
| /opt/pkgs/mailman/scripts/driver listinfo
RU> but get this traceback:
| @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
| [----- Mailman Version: <undetermined> -----]
| [----- Traceback ------]
| Traceback (innermost last):
| File "/opt/pkgs/mailman/scripts/driver", line 135, in
| print_traceback
| from Mailman.mm_cfg import VERSION
| ImportError: No module named Mailman.mm_cfg
| Content-type: text/html
Looks like you're trying to run this out of the source directory not
the installed directory. I don't think it makes much sense trying to
run the CGI's via the driver outside a web server environment -- there
are all sorts of environment things missing. Much better to write a
script to do what you want. See my recently posted list_lists for an
example (or anything inside bin/).
-Barry
From bwarsaw@cnri.reston.va.us Tue Apr 4 04:02:06 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 3 Apr 2000 23:02:06 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
<38E8C62B.37883725@uchicago.edu>
<38E90768.FD8B372C@west.sun.com>
Message-ID: <14569.23342.566957.87822@anthem.cnri.reston.va.us>
>>>>> "DM" == Dan Mick writes:
DM> Python's library-search path has to be set to include
DM> /home/mailman/Mailman. I just discovered that bin/withlist
DM> doesn't do this recently; I'm guessing the driver script
DM> doesn't do it either. For your purposes, I would think that
DM> augmenting PYTHONPATH in the environment should suffice.
Right, because driver is only supposed to be run from the C wrapper
binary, which /does/ set PYTHONPATH. I didn't expect withlist to be
run without explicitly invoking python. The -r switch could be used
this way so I think I'll add a #! line at the top. Note that it does
not help to add -i in the #! line.
-Barry
From Dan Mick Tue Apr 4 04:09:26 2000
From: Dan Mick (Dan Mick)
Date: Mon, 3 Apr 2000 20:09:26 -0700 (PDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
Message-ID: <200004040309.UAA10041@utopia.West.Sun.COM>
> >>>>> "DM" == Dan Mick writes:
>
> DM> Python's library-search path has to be set to include
> DM> /home/mailman/Mailman. I just discovered that bin/withlist
> DM> doesn't do this recently; I'm guessing the driver script
> DM> doesn't do it either. For your purposes, I would think that
> DM> augmenting PYTHONPATH in the environment should suffice.
>
> Right, because driver is only supposed to be run from the C wrapper
> binary, which /does/ set PYTHONPATH. I didn't expect withlist to be
> run without explicitly invoking python.
But even when withlist is run from Python, PYTHONPATH/sys.path doesn't
include /home/mailman/Mailman (this is 1.1):
python -i bin/withlist
Loading list: (unlocked)
>>> sys.path
['/home/mailman', 'bin', '/usr/local/lib/python1.5/',
'/usr/local/lib/python1.5/plat-sunos5', '/usr/local/lib/python1.5/lib-tk',
'/usr/local/lib/python1.5/lib-dynload']
This means that the relatively-obvious "import mm_cfg" doesn't work.
How come? Things like that *do* seem like the very purpose of
withlist.
> The -r switch could be used
> this way so I think I'll add a #! line at the top. Note that it does
> not help to add -i in the #! line.
>
> -Barry
From bwarsaw@cnri.reston.va.us Tue Apr 4 04:32:49 2000
From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us)
Date: Mon, 3 Apr 2000 23:32:49 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <200004040309.UAA10041@utopia.West.Sun.COM>
Message-ID: <14569.25185.497093.837154@anthem.cnri.reston.va.us>
>>>>> "DM" == Dan Mick writes:
DM> But even when withlist is run from Python, PYTHONPATH/sys.path
DM> doesn't include /home/mailman/Mailman (this is 1.1):
| python -i bin/withlist
| Loading list: (unlocked)
>> sys.path
DM> ['/home/mailman', 'bin', '/usr/local/lib/python1.5/',
DM> '/usr/local/lib/python1.5/plat-sunos5',
DM> '/usr/local/lib/python1.5/lib-tk',
DM> '/usr/local/lib/python1.5/lib-dynload']
DM> This means that the relatively-obvious "import mm_cfg" doesn't
DM> work. How come? Things like that *do* seem like the very
DM> purpose of withlist.
Ah ha, now I understand your problem! Python's packaging system is
involved here too. All the non-top level entry point modules are
supposed to live under the `Mailman' package. In order to be able to
import from the Mailman package, you must have it's parent directory
on sys.path. Thus /home/mailman is on sys.path, and you can then
import Mailman.mm_cfg (or "from Mailman import mm_cfg"). You've gotta
specify the `Mailman' package some place though.
You'll notice that 2.0's withlist says
from Mailman.MailList import MailList
this imports the MailList class from the MailList module inside the
Mailman package.
Hope that helps,
-Barry
From bwarsaw@cnri.reston.va.us Tue Apr 4 04:38:59 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 3 Apr 2000 23:38:59 -0400 (EDT)
Subject: [Mailman-Developers] Bug#61695: mailman: mail->news gw blows up on mails without subject
(fwd)
References:
Message-ID: <14569.25555.748317.241899@anthem.cnri.reston.va.us>
>>>>> "GM" == Gergely Madarasz writes:
GM> I just got this bugreport in the debian BTS. Comments ?
It's a valid bug that will crash anytime the message is missing a
Subject: header. The easy workaround is to make sure the Subject:
header is present before the message gets posted to the list.
While this bug was still present in 2.0beta1, I've fixed it for
2.0beta2. The code's changed too much to provide a Mailman 1.1 patch,
but here's the patch against 2.0beta1; it should be easy enough to
find in the 1.1 source tree.
Please feel free to forward this to the appropriate debian lists.
-Barry
-------------------- snip snip --------------------
Index: ToUsenet.py
===================================================================
RCS file: /projects/cvsroot/mailman/Mailman/Handlers/ToUsenet.py,v
retrieving revision 1.8
retrieving revision 1.9
diff -c -r1.8 -r1.9
*** ToUsenet.py 2000/03/23 06:48:52 1.8
--- ToUsenet.py 2000/04/04 03:35:05 1.9
***************
*** 58,65 ****
from Mailman.pythonlib import nntplib
# Ok, munge headers, etc.
subj = msg.getheader('subject')
if subj:
- subjpref = mlist.subject_prefix
if not re.match('(re:? *)?' + re.escape(subjpref), subj, re.I):
msg['Subject'] = subjpref + subj
else:
--- 58,65 ----
from Mailman.pythonlib import nntplib
# Ok, munge headers, etc.
subj = msg.getheader('subject')
+ subjpref = mlist.subject_prefix
if subj:
if not re.match('(re:? *)?' + re.escape(subjpref), subj, re.I):
msg['Subject'] = subjpref + subj
else:
From rullfig@uchicago.edu Tue Apr 4 14:51:27 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Tue, 04 Apr 2000 08:51:27 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu> <14569.21918.950338.815597@anthem.cnri.reston.va.us>
Message-ID: <38E9F35F.282051B3@uchicago.edu>
"Barry A. Warsaw" wrote:
>
> >>>>> "RU" == Roberto Ullfig writes:
>
> RU> So, in 1.0rc2, displaying the list of lists for 529 lists
> RU> requires 529**2 = 279841 system stat calls and takes over one
> RU> and a half minutes on our Ultra-2 2x296 processor system! Is
> RU> this because of Python, Mailman, or both? Has this been
> RU> "fixed" in 2.0? You really should only need to make one stat
> RU> call per list.
>
> Uh, it's because of Mailman :)
>
> I implemented a list_lists scripts which does on the command line what
> listinfo.py does in HTML (see attached). Here's what truss -c gives
> me:
>
> -------------------- snip snip --------------------
> Portal - [no description available]
> Postal - [no description available]
> Stage - Staging new Mailman releases
> Test - [no description available]
> syscall seconds calls errors
> _exit .00 1
> read .00 102
> write .00 8
> open .11 607 474
> close .01 143
> time .00 3
> brk .03 227
> stat .03 201 157
> getpid .00 10
> fstat .00 66
> ioctl .02 63 61
> execve .00 10 8
> umask .00 2
> fcntl .00 7
> readlink .00 2 2
> sigprocmask .00 2
> sigaction .00 50
> sigpending .00 1
> mmap .00 42
> mprotect .00 10
> munmap .00 11
> uname .00 4
> sysconfig .00 1
> lwp_create .00 6
> lwp_continue .00 2
> lwp_self .00 3
> llseek .00 114
> door .00 5
> lwp_schedctl .01 5
> getdents64 .01 15
> fstat64 .00 67
> open64 .00 7
> ---- --- ---
> sys totals: .22 1797 702
> usr time: .51
> elapsed: 1.19
> -------------------- snip snip --------------------
>
> Getting the list of list names, requires at least a listdir() and an
> exists() for every directory found there.
>
> Nothing about this will change for 2.0.
>
> -Barry
Thanks for the script.
Now this is the truss output for the listinfo that is called by
driver:
syscall seconds calls errors
_exit .00 1
read .21 1979
write .15 1638
open .12 1233 579
close .06 1189
time .00 1
brk .43 5026
stat 25.58 285877 174
fstat .00 63
ioctl .01 591 589
execve .00 1
umask .00 2
fcntl .02 535
readlink .00 3 2
sigaction .00 48
mmap .00 32
munmap .00 8
llseek .05 643
getdents64 .85 10165
fstat64 .01 1123
open64 .03 535
---- --- ---
sys totals: 27.52 310693 1344
usr time: 54.01
elapsed: 173.76
I can understand a listdir and an exists for each directory;
that should come out to 2 * n stat calls right (~1000 for us).
What I'm saying is that we are seeing n ** 2 stat calls (that's
squared) or 285877 of them. The above truss is from running the
driver manually after setting PYTHONPATH as suggested by Dan
(thanks Dan):
setenv PYTHONPATH /opt/http/opt/pkgs/mailman/Mailman
python /opt/http//opt/pkgs/mailman/scripts/driver listinfo
I've also trussed the running process and gotten similar
results; I see it stat'ing every directory once for every
directory stat'ed or n-squared stats.
Note that this is with 1.0rc2; still waiting for 2.0.
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From rullfig@uchicago.edu Tue Apr 4 15:14:42 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Tue, 04 Apr 2000 09:14:42 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu> <14569.21918.950338.815597@anthem.cnri.reston.va.us> <38E9F35F.282051B3@uchicago.edu>
Message-ID: <38E9F8D2.10F251D5@uchicago.edu>
Roberto Ullfig wrote:
>
> "Barry A. Warsaw" wrote:
> >
> > >>>>> "RU" == Roberto Ullfig writes:
> >
> > RU> So, in 1.0rc2, displaying the list of lists for 529 lists
> > RU> requires 529**2 = 279841 system stat calls and takes over one
> > RU> and a half minutes on our Ultra-2 2x296 processor system! Is
> > RU> this because of Python, Mailman, or both? Has this been
> > RU> "fixed" in 2.0? You really should only need to make one stat
> > RU> call per list.
> >
> > Uh, it's because of Mailman :)
> >
> > I implemented a list_lists scripts which does on the command line what
> > listinfo.py does in HTML (see attached). Here's what truss -c gives
> > me:
> >
> > -------------------- snip snip --------------------
> > Portal - [no description available]
> > Postal - [no description available]
> > Stage - Staging new Mailman releases
> > Test - [no description available]
> > syscall seconds calls errors
> > _exit .00 1
> > read .00 102
> > write .00 8
> > open .11 607 474
> > close .01 143
> > time .00 3
> > brk .03 227
> > stat .03 201 157
> > getpid .00 10
> > fstat .00 66
> > ioctl .02 63 61
> > execve .00 10 8
> > umask .00 2
> > fcntl .00 7
> > readlink .00 2 2
> > sigprocmask .00 2
> > sigaction .00 50
> > sigpending .00 1
> > mmap .00 42
> > mprotect .00 10
> > munmap .00 11
> > uname .00 4
> > sysconfig .00 1
> > lwp_create .00 6
> > lwp_continue .00 2
> > lwp_self .00 3
> > llseek .00 114
> > door .00 5
> > lwp_schedctl .01 5
> > getdents64 .01 15
> > fstat64 .00 67
> > open64 .00 7
> > ---- --- ---
> > sys totals: .22 1797 702
> > usr time: .51
> > elapsed: 1.19
> > -------------------- snip snip --------------------
> >
> > Getting the list of list names, requires at least a listdir() and an
> > exists() for every directory found there.
> >
> > Nothing about this will change for 2.0.
> >
> > -Barry
>
> Thanks for the script.
>
> Now this is the truss output for the listinfo that is called by
> driver:
>
> syscall seconds calls errors
> _exit .00 1
> read .21 1979
> write .15 1638
> open .12 1233 579
> close .06 1189
> time .00 1
> brk .43 5026
> stat 25.58 285877 174
> fstat .00 63
> ioctl .01 591 589
> execve .00 1
> umask .00 2
> fcntl .02 535
> readlink .00 3 2
> sigaction .00 48
> mmap .00 32
> munmap .00 8
> llseek .05 643
> getdents64 .85 10165
> fstat64 .01 1123
> open64 .03 535
> ---- --- ---
> sys totals: 27.52 310693 1344
> usr time: 54.01
> elapsed: 173.76
And this is the truss from using the script you sent:
syscall seconds calls errors
_exit .00 1
read .29 1963
write .03 534
open .13 1132 485
close .00 1182
time .00 1
brk .41 4944
stat 28.32 285849 148
fstat .00 61
ioctl .02 586 584
execve .01 3 1
fcntl .00 535
readlink .00 3 2
sigaction .01 48
mmap .00 40
munmap .01 10
llseek .05 632
getdents64 .92 10165
fstat64 .02 1118
open64 .06 535
---- --- ---
sys totals: 30.28 309342 1220
usr time: 56.08
elapsed: 182.84
All those stat calls just don't seem right to me. Using python
1.5.2 if that matters.
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From secabeen@pobox.com Tue Apr 4 15:46:43 2000
From: secabeen@pobox.com (Ted Cabeen)
Date: Tue, 04 Apr 2000 09:46:43 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
In-Reply-To: Your message of "Tue, 04 Apr 2000 09:14:42 CDT."
<38E9F8D2.10F251D5@uchicago.edu>
Message-ID: <200004041446.JAA03573@entropy.uchicago.edu>
In message <38E9F8D2.10F251D5@uchicago.edu>, Roberto Ullfig writes:
>Roberto Ullfig wrote:
>>
>> "Barry A. Warsaw" wrote:
>> >
>> > >>>>> "RU" == Roberto Ullfig writes:
>> >
>> > RU> So, in 1.0rc2, displaying the list of lists for 529 lists
>> > RU> requires 529**2 = 279841 system stat calls and takes over one
>> > RU> and a half minutes on our Ultra-2 2x296 processor system! Is
>> > RU> this because of Python, Mailman, or both? Has this been
>> > RU> "fixed" in 2.0? You really should only need to make one stat
>> > RU> call per list.
>> >
>> > Uh, it's because of Mailman :)
>> >
>> > I implemented a list_lists scripts which does on the command line what
>> > listinfo.py does in HTML (see attached). Here's what truss -c gives
>> > me:
>> > Getting the list of list names, requires at least a listdir() and an
>> > exists() for every directory found there.
>> >
>> > Nothing about this will change for 2.0.
>> >
>> > -Barry
>>
>> Thanks for the script.
>>
>> Now this is the truss output for the listinfo that is called by
>> driver:
Here's what is happening. When listinfo runs and has to get the list of
advertised addresses, it starts by getting a list of mailing lists on the
server using Utils.list_names(). Utils.list_names() requires two stat calls
for each list on the machine every time it is called. This is
understandable and isn't going to change. It then proceeds to open every one
of those lists to check on the advertised flag. Again, no problem. The
problem is that when Mailman opens a List in the MailList constructor
__init__, it checks to makes sure that the list exists by running
Utils.list_names() and seeing if the list name requested is there. Therefore
every request to open a list requires two stat calls on every list on the
system. Therefore when we are sequentially opening every list on the system
in listinfo.py we get a squaring effect on stat calls in the list directory.
Solutions are reasonably easy to code, the first of which comes to mind is a
optional argument to the constructor that indicated that the name has already
been checked and that checking it again is not necessary. Other solutions
include caching the list of lists on the server, but this means there is a
delay between when the list is created and when it becomes accessible.
I can code something up for you if necessary, but it seems like a reasonably
simple patch. Do either you or Barry need a patch? Let me know if you do.
--
Ted Cabeen http://www.pobox.com/~secabeen secabeen@pobox.com
Check Website or finger for PGP Public Key secabeen@midway.uchicago.edu
"I have taken all knowledge to be my province." -F. Bacon cococabeen@aol.com
"Human kind cannot bear very much reality."-T.S.Eliot 73126.626@compuserve.com
From bwarsaw@cnri.reston.va.us Tue Apr 4 18:06:35 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Tue, 4 Apr 2000 13:06:35 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
<14569.21918.950338.815597@anthem.cnri.reston.va.us>
<38E9F35F.282051B3@uchicago.edu>
<38E9F8D2.10F251D5@uchicago.edu>
Message-ID: <14570.8475.832742.629404@anthem.cnri.reston.va.us>
>>>>> "RU" == Roberto Ullfig writes:
RU> All those stat calls just don't seem right to me. Using python
RU> 1.5.2 if that matters.
Shouldn't. I'm seeing approximately n**2 stat calls too. It'll take
some investigating to figure out what's going on, and I'm not sure
it'll happen in time for 2.0.
-Barry
From bwarsaw@cnri.reston.va.us Tue Apr 4 18:13:32 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Tue, 4 Apr 2000 13:13:32 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E9F8D2.10F251D5@uchicago.edu>
<200004041446.JAA03573@entropy.uchicago.edu>
Message-ID: <14570.8892.540826.507453@anthem.cnri.reston.va.us>
>>>>> "TC" == Ted Cabeen writes:
TC> Here's what is happening.
Doh! Thanks for finding this Ted. To be honest, I've looked at that
constructor hundreds of times and my eyes just parsed right over it.
I think the right thing to do may be to just let the MMBadListError
percolate up through the Load() call and just zap the test for
MMUnknownListError. I'll have to grep through the code though to see
if that would cause other problems. I suspect not, because I remember
basically having to catch both errors all the time anyway.
Again, thanks for the sleuthing!
-Barry
From dan.mick@West.Sun.COM Tue Apr 4 22:01:40 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Tue, 04 Apr 2000 14:01:40 -0700
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <200004040309.UAA10041@utopia.West.Sun.COM> <14569.25185.497093.837154@anthem.cnri.reston.va.us>
Message-ID: <38EA5834.F731FAF6@west.sun.com>
bwarsaw@cnri.reston.va.us wrote:
> Thus /home/mailman is on sys.path, and you can then
> import Mailman.mm_cfg (or "from Mailman import mm_cfg"). You've gotta
> specify the `Mailman' package some place though.
OK, gotcha. I figured "withlist" would have established the convention
that Mailman modules were the "context", since I've already got a list
open...but I wasn't thinking in concert with you. Now I am; thanks.
From bwarsaw@cnri.reston.va.us Tue Apr 4 22:25:19 2000
From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us)
Date: Tue, 4 Apr 2000 17:25:19 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <200004040309.UAA10041@utopia.West.Sun.COM>
<14569.25185.497093.837154@anthem.cnri.reston.va.us>
<38EA5834.F731FAF6@west.sun.com>
Message-ID: <14570.23999.300339.154042@anthem.cnri.reston.va.us>
Cool.
From bwarsaw@cnri.reston.va.us Wed Apr 5 00:36:27 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Tue, 4 Apr 2000 19:36:27 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E9F8D2.10F251D5@uchicago.edu>
<200004041446.JAA03573@entropy.uchicago.edu>
Message-ID: <14570.31867.956811.471392@anthem.cnri.reston.va.us>
Ted's analysis was right on-target, however the right fix inspired me
to do more hacking than expected. The good news is that for the 64
lists on python.org, the number of stat calls bin/list_lists does has
just dropped from 4453 to 357 (with the same number of errors in both:
214). This should be a big win for your huge lists.
I don't have a patch handy, but you can either check out the CVS
snapshot or wait for beta2. No ETA there, but RRRSN :)
-Barry
From dan.mick@West.Sun.COM Wed Apr 5 11:24:42 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Wed, 05 Apr 2000 03:24:42 -0700
Subject: [Mailman-Developers] Speaking of pathological behavior...
Message-ID: <38EB146A.CF88B246@west.sun.com>
Now that I'm in the same room with my Mailman server, I notice
that, at least after setting admin_user_chunksize to 50 (from
the default 30) that changing one user's "mailto" flag from off
to on (to disable a user by hand temporarily) beats the living *hell*
out of the disk, and server. Something bad wrong is going on there;
I truss'ed it once and it was continually doing something like
opening a file, writing it, and closing it. I don't know what.
Something's wrong though. I haven't started to try to look at the
form-processing code to see what happens.
From dan.mick@West.Sun.COM Wed Apr 5 11:39:48 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Wed, 05 Apr 2000 03:39:48 -0700
Subject: [Mailman-Developers] Speaking of pathological behavior...
References: <38EB146A.CF88B246@west.sun.com>
Message-ID: <38EB17F4.6438AF52@west.sun.com>
Dan Mick wrote:
>
> Now that I'm in the same room with my Mailman server, I notice
> that, at least after setting admin_user_chunksize to 50 (from
> the default 30) that changing one user's "mailto" flag from off
> to on (to disable a user by hand temporarily) beats the living *hell*
> out of the disk, and server. Something bad wrong is going on there;
> I truss'ed it once and it was continually doing something like
> opening a file, writing it, and closing it. I don't know what.
>
> Something's wrong though. I haven't started to try to look at the
> form-processing code to see what happens.
Specifically, it looks a lot like both of these steps happen for every
member:
1) the archive links are made and unmade
2) the entire config.db.tmp file is rewritten, start to finish
These may be "safety" things, but they're pretty huge performance kills,
especially if, say, one attribute on one member is changing. Seems like a good
place for a scalability win. Barry, did you rework this code at all for 2.0beta1?
From dan.mick@West.Sun.COM Wed Apr 5 11:43:49 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Wed, 05 Apr 2000 03:43:49 -0700
Subject: [Mailman-Developers] Speaking of pathological behavior...
References: <38EB146A.CF88B246@west.sun.com> <38EB17F4.6438AF52@west.sun.com>
Message-ID: <38EB18E5.92C7B8F2@west.sun.com>
Dan Mick wrote:
> Specifically, it looks a lot like both of these steps happen for every
> member:
>
> 1) the archive links are made and unmade
> 2) the entire config.db.tmp file is rewritten, start to finish
Sorry, let me reword that one last time:
1) the archive links are remade, even though they exist (probably not a big deal)
2) config.db.tmp is written, and then .db becomes .last and .tmp becomes .db.
Probably batching up all the "changes" into one config.db write is a big
big win.
From dan.mick@West.Sun.COM Wed Apr 5 11:54:14 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Wed, 05 Apr 2000 03:54:14 -0700
Subject: [Mailman-Developers] Answering my own question
Message-ID: <38EB1B56.8E4096E6@west.sun.com>
Adding a "save_list=0" (or just a fourth argument '0') to the two calls
to SetUserOption in admin.py speeds up that "set one option" flag a
*whole* lot. It might be that similar "optimizations" to the other
forms processing (DeleteMember, primarily) are a bit unsafe, but it seems
like a huge win for the SetUserOption calls, and not very unsafe, since
the list will be saved after processing all users anyway.
From rullfig@uchicago.edu Wed Apr 5 13:57:38 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Wed, 05 Apr 2000 07:57:38 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E9F8D2.10F251D5@uchicago.edu>
<200004041446.JAA03573@entropy.uchicago.edu> <14570.31867.956811.471392@anthem.cnri.reston.va.us>
Message-ID: <38EB3842.8BE90E3D@uchicago.edu>
"Barry A. Warsaw" wrote:
>
> Ted's analysis was right on-target, however the right fix inspired me
> to do more hacking than expected. The good news is that for the 64
> lists on python.org, the number of stat calls bin/list_lists does has
> just dropped from 4453 to 357 (with the same number of errors in both:
> 214). This should be a big win for your huge lists.
>
> I don't have a patch handy, but you can either check out the CVS
> snapshot or wait for beta2. No ETA there, but RRRSN :)
>
Thanks, will probably wait until after we install 2 to patch
anything (if it isn't in 2). Note, that this affects the Admin
list page too.
We've got an hourly cronjob creating the file; the only problem
is getting the listinfo URL to display that html instead of
running listinfo. For now we just have an alternate URL for the
pre-generated list of lists.
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From aletzner@germany.sun.com Wed Apr 5 16:57:39 2000
From: aletzner@germany.sun.com (Adrian Letzner - Sun Germany Berlin SE)
Date: Wed, 05 Apr 2000 17:57:39 +0200
Subject: [Mailman-Developers] security szenario possible?
Message-ID: <38EB6273.79DFFF98@germany.sun.com>
i would like to know, if mailman can handle the following szenario:
1. automatic subscribing to a fixed list (eg. via cgi script). that
means:
a program (eg. cgi-script) should handle the subscribing/unsubscribing
mechanism by sending a static mail to the *-request address WITHOUT (!!)
using the password mechanism (new privacy option: *not confirming).
2. deleting the mail-header to anonymisize the mails which will be
posted.
the reasons for that is my plan to have an automatic anonymous mailing
list defined, where only the administrator will post messages.
please answer directly to me, cause i'm not on the list.
thanks a lot and sorry for my english
--
*******************************************************************
Adrian Letzner
ISV Partner SE Tel: (++49 30) 74 70 96 65
Sun Microsystems GmbH Fax: (++49 30) 74 70 96 99
Lankwitzer Str. 19 mailto:adrian.letzner@germany.sun.com
12107 Berlin http://www.sun.de
*******************************************************************
From bwarsaw@cnri.reston.va.us Thu Apr 6 00:40:10 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Wed, 5 Apr 2000 19:40:10 -0400 (EDT)
Subject: [Mailman-Developers] check_perms bug?...
References: <200003230624.WAA12783@utopia.West.Sun.COM>
Message-ID: <14571.52954.933150.960005@anthem.cnri.reston.va.us>
>>>>> "DM" == Dan Mick writes:
DM> Somehow, my /home/mailman/lists directory managed to get set
DM> with permissions check_perms didn't check for; it was set to
DM> d-wx-ws--x/mailman/mailman, which caused mailcmd to be unable
DM> to list the directory (no read perms).
DM> I fixed it once I figured it out, but I had a lot of
DM> check_perms runs in there before I finally figured it
DM> out...mostly because I was confused about what 'x' really
DM> means on a directory. But check_perms could have saved me
DM> time if it had checked for 'owner-and-group-read'; I think it
DM> should.
You're right. Here's the check_perms that will be in 2.0beta2.
-Barry
-------------------- snip snip --------------------
#! /usr/bin/env python
#
# Copyright (C) 1998,1999,2000 by the Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""Check the permissions for the Mailman installation.
Usage: %(PROGRAM)s [-f] [-v] [-h]
With no arguments, just check and report all the files that have bogus
permissions or group ownership. With -f (and run as root), fix all the
permission problems found. With -v be verbose.
"""
import sys
import os
import errno
import getopt
import grp
from stat import *
import paths
from Mailman import mm_cfg
MAILMAN_GRPNAME = 'mailman'
MAILMAN_GID = grp.getgrnam(MAILMAN_GRPNAME)[2]
PROGRAM = sys.argv[0]
class State:
FIX = 0
VERBOSE = 0
ERRORS = 0
STATE = State()
DIRPERMS = S_ISGID | S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH
def statmode(path):
return os.stat(path)[ST_MODE]
def statgidmode(path):
stat = os.stat(path)
return stat[ST_MODE], stat[ST_GID]
def checkwalk(arg, dirname, names):
for name in names:
path = os.path.join(dirname, name)
if arg.VERBOSE:
print 'checking gid and mode for', path
try:
mode, gid = statgidmode(path)
except os.error, (code, msg):
if code == errno.ENOENT:
continue
raise
if gid <> MAILMAN_GID:
try:
groupname = grp.getgrgid(gid)[0]
except KeyError:
groupname = '' % gid
arg.ERRORS = arg.ERRORS + 1
print path, 'bad gid (has: %s, expected %s)' % (
groupname, MAILMAN_GRPNAME),
if STATE.FIX:
print '(fixing)'
os.chown(path, -1, MAILMAN_GID)
else:
print
# all directories must be at least rwxrwsr-x. Don't check the private
# archive directory or database directory themselves since these are
# checked in checkarchives below.
private = mm_cfg.PRIVATE_ARCHIVE_FILE_DIR
if path == private or (os.path.commonprefix((path, private)) == private
and os.path.split(path)[1] == 'database'):
continue
if S_ISDIR(mode) and (mode & DIRPERMS) <> DIRPERMS:
arg.ERRORS = arg.ERRORS + 1
print 'directory must be at least 02775:', path,
if STATE.FIX:
print '(fixing)'
os.chmod(path, mode | DIRPERMS)
else:
print
def checkall():
# first check PREFIX
if STATE.VERBOSE:
print 'checking mode for', mm_cfg.PREFIX,
mode = statmode(mm_cfg.PREFIX)
if (mode & DIRPERMS) <> DIRPERMS:
STATE.ERRORS = STATE.ERRORS + 1
print 'directory must be at least 02775:', mm_cfg.PREFIX,
if STATE.FIX:
print '(fixing)'
os.chmod(mm_cfg.PREFIX, mode | DIRPERMS)
else:
print
# check all subdirs
os.path.walk(mm_cfg.PREFIX, checkwalk, STATE)
def checkarchives():
private = mm_cfg.PRIVATE_ARCHIVE_FILE_DIR
if STATE.VERBOSE:
print 'checking perms on', private
# private archives must not be other readable
mode = statmode(private)
if mode & S_IROTH:
STATE.ERRORS = STATE.ERRORS + 1
print private, 'must not be other-readable',
if STATE.FIX:
print '(fixing)'
os.chmod(private, mode & ~S_IROTH)
else:
print
def checkarchivedbs():
# The archives/private/listname/database file must not be other readable
# or executable otherwise those files will be accessible when the archives
# are public. That may not be a horrible breach, but let's close this off
# anyway.
for dir in os.listdir(mm_cfg.PRIVATE_ARCHIVE_FILE_DIR):
if dir[-5:] == '.mbox':
continue
dbdir = os.path.join(mm_cfg.PRIVATE_ARCHIVE_FILE_DIR, dir, 'database')
try:
mode = statmode(dbdir)
except os.error, (code, msg):
if code == errno.ENOENT:
continue
raise
if mode & S_IRWXO:
STATE.ERRORS = STATE.ERRORS + 1
print dbdir, 'must be other 000',
if STATE.FIX:
print '(fixing)'
os.chmod(dbdir, mode & ~S_IRWXO)
else:
print
def checkcgi():
exes = os.listdir(mm_cfg.CGI_DIR)
for f in exes:
path = os.path.join(mm_cfg.CGI_DIR, f)
if STATE.VERBOSE:
print 'checking set-gid for', path
mode = statmode(path)
if mode & S_IXGRP and not mode & S_ISGID:
STATE.ERRORS = STATE.ERRORS + 1
print path, 'must be set-gid',
if STATE.FIX:
print '(fixing)'
os.chmod(path, mode | S_ISGID)
else:
print
def checkmail():
wrapper = os.path.join(mm_cfg.WRAPPER_DIR, 'wrapper')
if STATE.VERBOSE:
print 'checking set-gid for', wrapper
mode = statmode(wrapper)
if not mode & S_ISGID:
STATE.ERRORS = STATE.ERRORS + 1
print wrapper, 'must be set-gid',
if STATE.FIX:
print '(fixing)'
os.chmod(wrapper, mode | S_ISGID)
def checkadminpw():
adminpw = os.path.join(mm_cfg.DATA_DIR, 'adm.pw')
targetmode = S_IFREG | S_IRUSR | S_IWUSR | S_IRGRP
if STATE.VERBOSE:
print 'checking perms on', adminpw
try:
mode = statmode(adminpw)
except os.error, (code, msg):
# adm.pw may not exist
if code == errno.ENOENT:
return
raise
if mode <> targetmode:
STATE.ERRORS = STATE.ERRORS + 1
print adminpw, 'permissions must be exactly 0640 (got %s)' % oct(mode)
if STATE.FIX:
print '(fixing)'
os.chmod(adminpw, targetmode)
def usage(code=0, msg=''):
print __doc__ % globals()
if msg:
print msg
sys.exit(code)
if __name__ == '__main__':
try:
opts, args = getopt.getopt(sys.argv[1:],
'fvh',
['fix', 'verbose', 'help'])
except getopt.error, msg:
usage(1, msg)
for opt, arg in opts:
if opt in ('-h', '--help'):
usage()
elif opt in ('-f', '--fix'):
STATE.FIX = 1
elif opt in ('-v', '--verbose'):
STATE.VERBOSE = 1
checkall()
checkarchives()
checkarchivedbs()
checkcgi()
checkmail()
checkadminpw()
if not STATE.ERRORS:
print 'No problems found'
else:
print 'Problems found:', STATE.ERRORS
print 'Re-run as root with -f flag until no errors are found'
From jarrell@vt.edu Thu Apr 6 14:21:42 2000
From: jarrell@vt.edu (Ron Jarrell)
Date: Thu, 06 Apr 2000 09:21:42 -0400
Subject: [Mailman-Developers] 2.0b2 status
Message-ID: <4.3.1.2.20000406091804.05717de0@vtserf.cc.vt.edu>
Updated both my production and test servers (no, not in that order) to
whatever was in cvs this morning, which at least has me stamped as 2.0beta2...
Seems to be working fine, including news gating... I'm watching the logs.
Did have one weirdness where the production server was giving me tracebacks
on the listinfo page.. Which went away after I tried it a couple of times, while
check_perms was running... check_perms found nothing wrong, and the script
"healed" itself. Bizarre...
From dan@auctionwatch.com Fri Apr 7 02:25:18 2000
From: dan@auctionwatch.com (Dan Chen)
Date: Thu, 6 Apr 2000 18:25:18 -0700
Subject: [Mailman-Developers] User list in mysql
Message-ID:
My company has tasked me to port our existing mailing list to mailman. The
requirement is that the list of subscribers has to be kept in mysql. (We
have scripts and reports that use this list).
I'm a python newbie, so please bear with me.
I've taken a look at MailList.py and understand that the subscriber list and
all other data is stored as a marshal in config.db. And this is stored in
the methods Load and Save.
I'm proposing that when Save is called, it also makes changes to the mysql
db. And after a Load, it queries the db. This sounds very expensive, and
I'm wondering if developers here who know Mailman and python better would
have a better scheme. Also if I'm missing anything in my analysis.
thanks
dan
From bwarsaw@cnri.reston.va.us Fri Apr 7 05:29:45 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Fri, 7 Apr 2000 00:29:45 -0400 (EDT)
Subject: [Mailman-Developers] cron/gate_news watermark == 0
References: <20000213182801.A15474@mail.dskk.co.jp>
Message-ID: <14573.25657.617106.187767@anthem.cnri.reston.va.us>
Back in February, there was some discussion about fixing gate_news and
the gate_watermarks file...
>>>>> "JT" == Jim Tittsler writes:
JT> A watermark value of 0 is used for two different purposes in
JT> cron/gate_news [...] The result of this is the first article
JT> posted to the newly created newsgroup fails to get gatewayed
JT> to the mailing list because the watermark for that group is
JT> (still) 0 which is used as a flag to "catch up" so all that
JT> happens is the watermark gets set to 1.
The jist of your message is spot on, and I took your advice to make
None the marker that means the list hasn't been gated before and/or
needs a mass catchup.
>>>>> "HM" == Harald Meland writes:
HM> Hmmmm... When checking the contents of my
HM> "data/gate_watermarks" marshal, I find several old mailing
HM> lists with an entry stating that their watermark is 0 -- even
HM> though these lists aren't actually gating any newsgroups.
HM> [...]
HM> Or, even better, move the per-list watermark info from
HM> gate_watermarks into the list's config.db, replacing any 0
HM> values in gate_watermarks with None values in config.db. This
HM> has the advantage of reducing the number of different files
HM> that would have to be changed when cloning/renaming a list.
Makes tons of sense Harald! I like this idea a lot for your reasons,
but also because it leads to a big simplification of gate_news. It's
actually harder to synchronize access to a shared gate_watermarks file
than it is to just keep the watermark in the list's attributes.
JT> This seems logical to me, since I think the administrator
JT> really wants to do the "catch up" any time gateway_to_mail
JT> transitions from No to Yes (implying that it needs to be able
JT> to update the watermark, either in config.db or by locking(?)
JT> and updating gate_watermarks).
Another excellent point. I've got a hack to the already horrible
admin.py to None-ify the watermark whenever the gateway_to_mail
property has changed. In reality we zap the watermark when it
transitions to either "yes" or "no", but I don't /think/ that matters
in practice. You could potentially want only watermark zapping on
transition from no->yes so that you could write a little script to
turn gatewaying on while preserving the watermark, but I just don't
think this will be a common enough occurance. The Right Thing To Do
is probably to improve the admin page for gatewaying so that there's a
flag the admin can tickle to zap the watermark or preserve it. Not
for 2.0 :)
So now the watermark is stored in the list's config.db under the
usenet_watermark attribute. gate_news O_CREAT|O_EXCL's a block file
to ensure that only one gate_news process is running at any time, and
it only deletes this block file when all it's forked children exit (or
on catastrophic exception). gate_news opens each list in turn
unlocked, and then if the list is gating to mail, it forks a process
for that list and immediately locks it. Then it gates the list
largely as before. Similar to before, if the watermark is None it
simply sets it to the last known article number and moves on,
expecting the following gate_news to post subsequent messages.
I also implemented your suggestion to bin/update to pull the info out
of gate_watermarks and stick them in mlist.usenet_watermark. It then
deletes gate_watermarks.
An outgrowth of all this is that I realized I really want a numeric,
easily comparable Mailman version number, and a way to figure out what
version we are upgrading from. That's surprisingly difficult now, but
it would have been darn handy for bin/update. I've now implemented a
HEX_VERSION using the same scheme that Python does, e.g. Mailman
2.0beta2 will have a HEX_VERSION of 0x200000b2, and this can be
compared against for running only selective updates in bin/update.
Every time bin/update completes successfully, it writes the current
hex version string to data/last_mailman_version.
[There's one minor icky bit: the warning about moving
templates/options.html is only printed if 1) we can't determine the
previous Mailman version, and 2) it is not a fresh install -- the
determinant being any files in the logs subdir]
I think that's it. I've done some moderate testing of all this, but
not on my production machines. I'm going to check it all in, and hope
you take a look, but be aware I haven't banged on it terribly hard.
Slowly-catching-up-ly y'rs,
-Barry
From bwarsaw@cnri.reston.va.us Fri Apr 7 05:31:53 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Fri, 7 Apr 2000 00:31:53 -0400 (EDT)
Subject: [Mailman-Developers] race condition in locking ?
References: <20000202140042.J19725@xs4all.nl>
<20000202225346.B17313@xs4all.nl>
<14488.44469.446995.168328@anthem.cnri.reston.va.us>
<20000203103555.C6378@xs4all.nl>
<20000207224457.T23471@xs4all.nl>
Message-ID: <14573.25785.169936.426535@anthem.cnri.reston.va.us>
>>>>> "HM" == Harald Meland writes:
>> Ah, well, I couldn't find any parts of Mailman in the CVS tree
>> that acually _called_ steal, so I couldn't figure out what it
>> should do in borderline cases.
HM> The only place I know of is cron/gate_news.
And even that no longer does (see my previous long post). I'm still
not comfortable with radical changes to LockFile.py at this late stage
of the game, but will re-examine this whole thread after 2.0 final.
-Barry
From bwarsaw@cnri.reston.va.us Fri Apr 7 05:34:50 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Fri, 7 Apr 2000 00:34:50 -0400 (EDT)
Subject: [Mailman-Developers] One more thing about bin/update...
Message-ID: <14573.25962.560643.516056@anthem.cnri.reston.va.us>
... What do you think about running it by default on a "make install"?
With the gate_news changes, you'll really want to be sure to run it.
I guess this is another case of helping lazy users who don't read or
follow directions. Am I trying to pamper them too much?
-Barry
From ricardo@rixhq.nu Fri Apr 7 07:28:08 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Fri, 7 Apr 2000 08:28:08 +0200
Subject: [Mailman-Developers] User list in mysql
In-Reply-To: ; from dan@auctionwatch.com on Thu, Apr 06, 2000 at 06:25:18PM -0700
References:
Message-ID: <20000407082808.A20142@miss-janet.com>
On Thu, Apr 06, 2000 at 06:25:18PM -0700, Dan Chen wrote:
> I've taken a look at MailList.py and understand that the subscriber list and
> all other data is stored as a marshal in config.db. And this is stored in
> the methods Load and Save.
> I'm proposing that when Save is called, it also makes changes to the mysql
> db. And after a Load, it queries the db. This sounds very expensive, and
yes I'd love to see mailman support mysql too...but I think the best thing
to do is moving the userlist to a seperate marshal db and abstracting the
access code to make it easy to replace it with your own favorite sql server.
from what i've seen in the mailman code is that there have to be made a lot
of changes to support something like this... so maybe we'll have to wait till
2.0 gets released and work on it from there.
in short, these are my thoughts on how this could be handled:
1) rewrite mailman to modify/read the user list through single function calls
2) create a module with those functions
3) create a module that is used to write the userlist to a marshal db
4) create a module to write the userlist to a mysql db
5) create a module to write the userlist to ... etc
Ricardo.
--
From Harald.Meland@usit.uio.no Sat Apr 8 22:48:42 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 08 Apr 2000 23:48:42 +0200
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
In-Reply-To: Roberto Ullfig's message of "Mon, 03 Apr 2000 11:26:19 -0500"
References: <38E8B23F.16F25CE7@uchicago.edu> <38E8C62B.37883725@uchicago.edu>
Message-ID:
[Roberto Ullfig]
> Roberto Ullfig wrote:
> >
> > So, in 1.0rc2, displaying the list of lists for 529 lists
> > requires 529**2 = 279841 system stat calls and takes over one
> > and a half minutes on our Ultra-2 2x296 processor system! Is
> > this because of Python, Mailman, or both? Has this been "fixed"
> > in 2.0? You really should only need to make one stat call per
> > list.
> >
>
> Whatever the answer is, we'd like to be able to generate the
> lists of lists once a day;
Just a quick note: In the (severely hacked-up) Mailman installation we
have here at uio.no (with ~3500 lists, although these are spread out
over ~150 virtual domains), I've implemented caching of the data
Mailman needs to generate the "list overview" pages, thereby avoiding
excessive stat()ing.
This is one of the features I'm holding off committing until after 2.0
is out.
--
Harald
From bwarsaw@cnri.reston.va.us Sat Apr 8 22:54:44 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Sat, 8 Apr 2000 17:54:44 -0400 (EDT)
Subject: [Mailman-Developers] [ANNOUNCE] Mailman 2.0 beta 2
Message-ID: <14575.43684.735692.298399@anthem.cnri.reston.va.us>
I've just uploaded the Mailman 2.0 beta 2 tarball to the following
locations:
http://www.list.org/mailman.tar.gz
http://download.sourceforge.net/mailman/mailman-2.0beta2.tgz
ftp://ftp.python.org/pub/mailman/mailman-2.0beta2.tgz
See the UPGRADING file for details on upgrading from 1.1 or 2.0beta1.
See the NEWS file for a complete description of changes for every
version; below is an excerpt for changes since 2.0beta1.
Barring any showstopping bugs, this will be the last release until I
finish my taxes. :)
Enjoy,
-Barry
-------------------- snip snip --------------------
- Rewritten gate_news cron script which should be more efficient and
avoid race and locking problems. Each list now maintains its own
watermark, and when you use the admin CGI script to turn on gating
from Usenet->mail, an automatic mass catch up is done to avoid
flooding the mailing list. cron/gate_news's command line interface
has also changed. See its docstring for details.
- A new cron script called qrunner has been added to retry message
deliveries that fail because of temporary smtpd problems.
- New command line script called bin/list_lists which does exactly
that: lists all the mailing lists on the system (much like the
listinfo CGI does).
- bin/withlist is now directly executable, however if you want to use
python -i, you must still explicitly invoke it. bin/withlist also
now cleans up after itself by unlocking any locked lists. It does
NOT save any dirty lists though - you must do this explicitly.
- $prefix permissions (and all subdirs) must now be 02775.
bin/check_perms has been updated to fix all the subdir permissions.
- "make update" (a.k.a. bin/update) is run automatically when you do a
"make install"
- The CGI driver script now puts information about the Python
environment into the logs/error file (but not the diagnostic web
page).
- Bug fixes and some performance improvements
From thomas@xs4all.net Sat Apr 8 23:31:04 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sun, 9 Apr 2000 00:31:04 +0200
Subject: [Mailman-Developers] [ANNOUNCE] Mailman 2.0 beta 2
In-Reply-To: <14575.43684.735692.298399@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Sat, Apr 08, 2000 at 05:54:44PM -0400
References: <14575.43684.735692.298399@anthem.cnri.reston.va.us>
Message-ID: <20000409003103.G13830@xs4all.nl>
On Sat, Apr 08, 2000 at 05:54:44PM -0400, Barry A. Warsaw wrote:
> Barring any showstopping bugs, this will be the last release until I
> finish my taxes. :)
There's one bug that would really like to be squashed before 2.0, I think:
the 'from ' line in the downloadable mailbox (the stripped version of the
pipermail archive mailbox) that carries the wrong date format. I posted a
patch a week ago or so, so that the downloadable mailboxes are at least
usable. The only possible problem is that ctime() might generate different
output on different locales, but I dont think so, and the manpage for
ctime() on Linux and BSDI dont mention it. Drop me a line if I need to
repost it. (Btw, should I post this bug to the jitterbug page, even if I
have a patch for it ?)
There are some other problems with the mailboxes, for instance that in
squashing the headers, the references line gets squashed too, but I'll save
those for later ;)
I also have a few changes locally, that might be included later (if
desirable, of course): partial rewrite of hyperarch/pipermail, actually losing the
latter (no functional changes, but a lot less hairy code, if I say so
myself), unsubscription-approval, like subscription (posted a buggy version
before) and some hacks to allow members of one list to post to another,
moderated list they are not members of.
I'm also pondering making members of a list that are themselves lists, a
special case, to skip the password-sending, allow posting, dont screw the
subject line too much, etc. (Those things would solve a lot of the minor
annoyances in some of our lists.) But all that might be a lot easier if that
'real user database' gets included ;)
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From bwarsaw@cnri.reston.va.us Sat Apr 8 23:50:58 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Sat, 8 Apr 2000 18:50:58 -0400 (EDT)
Subject: [Mailman-Developers] [ANNOUNCE] Mailman 2.0 beta 2
References: <14575.43684.735692.298399@anthem.cnri.reston.va.us>
<20000409003103.G13830@xs4all.nl>
Message-ID: <14575.47058.314702.208061@anthem.cnri.reston.va.us>
>>>>> "TW" == Thomas Wouters writes:
>> Barring any showstopping bugs, this will be the last release
>> until I finish my taxes. :)
TW> There's one bug that would really like to be squashed before
TW> 2.0, I think
2.0beta2 definitely won't be the last release before 2.0 final. I
just don't think I'm going to have many free evenings to hack on this
stuff until after April 17th :).
I don't think you need to repost your patches; I'll be cruising
through as many of the outstanding messages and Jitterbug reports as
possible before 2.0 final.
I'm totally psyched to have you rewrite/fix/adopt/expend great amounts
of effort on the archiver, but I'll have to hold off on integrating
that stuff until after 2.0 final.
thanks,
-Barry
From brian@SoftHome.net Sun Apr 9 04:56:41 2000
From: brian@SoftHome.net (Brian Grossman)
Date: Sat, 08 Apr 2000 21:56:41 -0600
Subject: [Mailman-Developers] observations from new user
Message-ID: <20000409035641.7523.qmail@lindy.softhome.net>
I'm evaluating mailman to replace my majordomo v1 installation.
Looks good so far. Nice work.
One general observation: Why does DEFAULT_PRIVATE_ROSTER default to 0? I
would think that 2 would be a better choice.
I'm using this with qmail. 2.0beta2 seems to work well with qmail 1.03.
Where should I send a script that eliminates the need for five .qmail
files per list? This one is called from .qmail-default.
I'm not subscribed to the list, so please cc me on replies.
Brian
From ricardo@rixhq.nu Sun Apr 9 10:21:41 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sun, 9 Apr 2000 11:21:41 +0200
Subject: [Mailman-Developers] observations from new user
In-Reply-To: <20000409035641.7523.qmail@lindy.softhome.net>; from brian@SoftHome.net on Sat, Apr 08, 2000 at 09:56:41PM -0600
References: <20000409035641.7523.qmail@lindy.softhome.net>
Message-ID: <20000409112141.A11416@miss-janet.com>
Hi,
On Sat, Apr 08, 2000 at 09:56:41PM -0600, Brian Grossman wrote:
> Where should I send a script that eliminates the need for five .qmail
> files per list? This one is called from .qmail-default.
on this list would be a right place i guess :)
at work i've installed mailman on a qmail server and created
a script that creates the 5 .qmail files by only supplying a
listname... but I think your solution sounds better...
is it a shell or perl script?
Ricardo.
--
From Harald.Meland@usit.uio.no Sun Apr 9 19:19:49 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 09 Apr 2000 20:19:49 +0200
Subject: [Mailman-Developers] membership managent web interface
In-Reply-To: Ricardo Kustner's message of "Sat, 1 Apr 2000 16:57:13 +0200"
References: <20000401165713.B2495@miss-janet.com>
Message-ID:
[Ricardo Kustner]
> But most of the times you know the exact address you want to unsubscribe
> or view/edit, so why not add a search box on top?
Can't this already be done by using the list member's "options" page
(using the user's address in combination with your list admin
password)?
> I even think that only a searchbox on the initial page should be
> enough, and only show the paged list of members when the user
> specifically asks for it... saves a bit of bandwidth / response
> time. any thoughts about this?
I disagree. The list admin interface is cryptic enough as it is, IMO.
Further obscuring the overview of list members wouldn't be nice.
--
Harald
From ricardo@rixhq.nu Sun Apr 9 19:43:09 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sun, 9 Apr 2000 20:43:09 +0200
Subject: [Mailman-Developers] membership managent web interface
In-Reply-To: ; from Harald.Meland@usit.uio.no on Sun, Apr 09, 2000 at 08:19:49PM +0200
References: <20000401165713.B2495@miss-janet.com>
Message-ID: <20000409204309.A1521@miss-janet.com>
Hi,
On Sun, Apr 09, 2000 at 08:19:49PM +0200, Harald Meland wrote:
> > But most of the times you know the exact address you want to unsubscribe
> > or view/edit, so why not add a search box on top?
> Can't this already be done by using the list member's "options" page
> (using the user's address in combination with your list admin
> password)?
you mean something like
http://server/mailman/options/mailinglist/ricardo--at--rixhq.nu ?
besides not being very convenient to do, i don't think it works w/o knowing
the users' password... as far as i can see there's no way telling you're
the admin and I doubt that admin password can override it.
> > I even think that only a searchbox on the initial page should be
> > enough, and only show the paged list of members when the user
> > specifically asks for it... saves a bit of bandwidth / response
> > time. any thoughts about this?
> I disagree. The list admin interface is cryptic enough as it is, IMO.
> Further obscuring the overview of list members wouldn't be nice.
actually I suggested this because I think that would make the member options
much more simple. Having a partial list of the members immediately is what
makes it cryptic IMHO.
My list has about 1700 members and navigating with the way it is not is
not very convenient (even if you get used to it) and I can't imagine how
it would be like for people with 10k lists :)
Ricardo.
--
From Harald.Meland@usit.uio.no Sun Apr 9 20:38:16 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 09 Apr 2000 21:38:16 +0200
Subject: [Mailman-Developers] Speaking of pathological behavior...
In-Reply-To: Dan Mick's message of "Wed, 05 Apr 2000 03:43:49 -0700"
References: <38EB146A.CF88B246@west.sun.com> <38EB17F4.6438AF52@west.sun.com> <38EB18E5.92C7B8F2@west.sun.com>
Message-ID:
[Dan Mick]
> 1) the archive links are remade, even though they exist (probably not a big deal)
> 2) config.db.tmp is written, and then .db becomes .last and .tmp becomes .db.
Thanks for reporting, I think the change I just checked into CVS
should fix this (admin.py was calling MailList.SetUserOption() without
the keyword argument "save_list=0", causing the list to be Save()d at
least
admin_user_chunksize * len(("hide", "nomail", "ack", "notmetoo", "plain"))
times(!) every time the list's Membership Management page was
visited).
> Probably batching up all the "changes" into one config.db write is a
> big big win.
The bigger a admin_user_chunksize you're using, the bigger this win
will be :)
--
Harald
From Harald.Meland@usit.uio.no Sun Apr 9 20:44:47 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 09 Apr 2000 21:44:47 +0200
Subject: [Mailman-Developers] Answering my own question
In-Reply-To: Dan Mick's message of "Wed, 05 Apr 2000 03:54:14 -0700"
References: <38EB1B56.8E4096E6@west.sun.com>
Message-ID:
[Dan Mick]
> Adding a "save_list=0" (or just a fourth argument '0') to the two
> calls to SetUserOption in admin.py speeds up that "set one option"
> flag a *whole* lot.
I've got to stop answering emails sequentially ;)
> It might be that similar "optimizations" to the other forms
> processing (DeleteMember, primarily) are a bit unsafe, but it seems
> like a huge win for the SetUserOption calls, and not very unsafe,
> since the list will be saved after processing all users anyway.
The SetUserOption() calls are being made even if there have been no
changes, while the other MailList methods are only called if something
actually has changed. I think that fact justifies putting off
optimizing this further, at least for now.
--
Harald
From Harald.Meland@usit.uio.no Sun Apr 9 20:55:36 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 09 Apr 2000 21:55:36 +0200
Subject: [Mailman-Developers] security szenario possible?
In-Reply-To: Adrian Letzner - Sun Germany Berlin SE's message of "Wed, 05 Apr 2000 17:57:39 +0200"
References: <38EB6273.79DFFF98@germany.sun.com>
Message-ID:
[Adrian Letzner - Sun Germany Berlin SE]
> i would like to know, if mailman can handle the following szenario:
>
> 1. automatic subscribing to a fixed list (eg. via cgi script). that
> means:
> a program (eg. cgi-script) should handle the subscribing/unsubscribing
> mechanism by sending a static mail to the *-request address WITHOUT (!!)
> using the password mechanism (new privacy option: *not confirming).
In fact, that does not constitute a new privacy option -- if you put
ALLOW_OPEN_SUBSCRIBE = 1
in your ~mailman/Mailman/mm_cfg.py, a fourth option "none" should
magically appear for "What steps are required for subscription?" on
all your lists Privacy Options pages.
Note that this allows _any_ of the lists in your Mailman installation
to use the open subscribes option, and that is not necessarily a good
thing (in that it allows anyone to subscribe unsuspecting others to
your lists against their will).
> 2. deleting the mail-header to anonymisize the mails which will be
> posted.
Doesn't the
Hide the sender of a message, replacing it with the list address
(Removes From, Sender and Reply-To fields)
option on the bottom of the Privacy Options page do this?
--
Harald
From Harald.Meland@usit.uio.no Sun Apr 9 21:05:41 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 09 Apr 2000 22:05:41 +0200
Subject: [Mailman-Developers] observations from new user
In-Reply-To: Brian Grossman's message of "Sat, 08 Apr 2000 21:56:41 -0600"
References: <20000409035641.7523.qmail@lindy.softhome.net>
Message-ID:
[Brian Grossman]
> One general observation: Why does DEFAULT_PRIVATE_ROSTER default to
> 0? I would think that 2 would be a better choice.
Well, obviously someone had to choose some sort of default back in the
murky beginnings of time.
Why that someone preferred 0 over 2? Dunno, but as long as people
have become used to this, or even are comfortable with it[1], I think
you'd have to make a stronger argument for why you think it should be
changed than the quick note above :)
> I'm using this with qmail. 2.0beta2 seems to work well with qmail
> 1.03.
Great!
> Where should I send a script that eliminates the need for five
> .qmail files per list?
As Ricardo said: Here would be a good place.
[1] I can't recall having seen mention of this default setting from
others, but then again I haven't been reading mailman-users for
quite some time... :(
--
Harald
From Harald.Meland@usit.uio.no Sun Apr 9 21:26:10 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 09 Apr 2000 22:26:10 +0200
Subject: [Mailman-Developers] membership managent web interface
In-Reply-To: Ricardo Kustner's message of "Sun, 9 Apr 2000 20:43:09 +0200"
References: <20000401165713.B2495@miss-janet.com> <20000409204309.A1521@miss-janet.com>
Message-ID:
[Ricardo Kustner]
> Hi,
>
> On Sun, Apr 09, 2000 at 08:19:49PM +0200, Harald Meland wrote:
> > > But most of the times you know the exact address you want to unsubscribe
> > > or view/edit, so why not add a search box on top?
> > Can't this already be done by using the list member's "options" page
> > (using the user's address in combination with your list admin
> > password)?
> you mean something like
> http://server/mailman/options/mailinglist/ricardo--at--rixhq.nu ?
Yup -- the URL with the unobfuscated email address at the end works
just as well, or you can get to the page by using the "Edit Options"
box typically found on a list's "listinfo" page.
> besides not being very convenient to do, i don't think it works w/o
> knowing the users' password...
Have you tried?
> as far as i can see there's no way telling you're the admin
If you are able to supply the list admin password, then that should
suffice :)
> and I doubt that admin password can override it.
The list admin password can always be used in place of any member's
password. And, the site admin password can always be used in place of
any list admin password (and therefore also in place of any list
member's password).
> > I disagree. The list admin interface is cryptic enough as it is, IMO.
> > Further obscuring the overview of list members wouldn't be nice.
>
> actually I suggested this because I think that would make the member
> options much more simple. Having a partial list of the members
> immediately is what makes it cryptic IMHO.
I agree that the chunk-stuff can be cryptic, but I don't think that
requiring the admin to follow additional links (or whatever) to even
get to that chunk-sized interface would make the admin interface as a
whole less cryptic.
What do others think?
> My list has about 1700 members and navigating with the way it is not
> is not very convenient (even if you get used to it) and I can't
> imagine how it would be like for people with 10k lists :)
Me neither :)
--
Harald
From Harald.Meland@usit.uio.no Sun Apr 9 21:37:57 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 09 Apr 2000 22:37:57 +0200
Subject: [Mailman-Developers] Post to every list, just in case...
In-Reply-To: Nigel Metheringham's message of "Mon, 27 Mar 2000 09:45:47 +0100"
References:
Message-ID:
[Nigel Metheringham]
> Additionally do we need to consider:-
>
> 1. Hierarchical lists - ie moderated announce lists that post into
> other lists, with decent support for building a union of
> subscribers - maybe rather than subscribing x-users & x-developers
> to ax-nnounce, a way of telling the list to pull in these lists'
> subscribers too.
I've got something like this running here, implemented as a generic
"metamember" mechanism. I think I posted some kind of description of
metamembers here earlier, have a look in the archives.
Anyway, I won't consider (backporting and) committing that part of my
tree until after 2.0 is out.
> 2. Filtering on lists to detect cross posting and do something...
What do you mean when you say "filtering"? Do you have any ideas as
to what the configuration interface[1] for this should look like?
> 3. An idea of what "something" should be :-)
Why, one should delete the lists crossposted to, of course, to teach
those lusers to stop crossposting :-P
[1] I assume we'd want to have such a thing :)
--
Harald
From bigdog@dogpound.vnet.net Sun Apr 9 22:15:05 2000
From: bigdog@dogpound.vnet.net (Matt Davis)
Date: Sun, 9 Apr 2000 17:15:05 -0400 (EDT)
Subject: [Mailman-Developers] 2 quickies..
Message-ID:
1. Just making sure with staying up with the CVS tree, will I overwrite
any info (list config/membership/etc.) if I update my
/home/mailman/mailman (CVS source dir) and then do 'make install' (which
now does 'make update').
2. I think I found a little bug. When a message is kicked to the admin
for the message having too many users. After the admin approves the
message to be sent, it does not show up in the archive page.
--
Matt Davis - ICQ# 934680
http://dogpound.vnet.net/
NoWonder UNIX Tech - http://www.nowonder.com
Is the glass half empty, half full, or twice as large as it needs to be?
From Nigel.Metheringham@VData.co.uk Sun Apr 9 22:21:53 2000
From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham)
Date: Sun, 09 Apr 2000 22:21:53 +0100
Subject: [Mailman-Developers] [ANNOUNCE] Mailman 2.0 beta 2
In-Reply-To: Message from "Barry A. Warsaw"
of "Sat, 08 Apr 2000 18:50:58 EDT." <14575.47058.314702.208061@anthem.cnri.reston.va.us>
Message-ID:
Looks to me like the Sendmail interface is pathologically broken :-(
The envelope sender address is wrong (set to original message sender
rather than list so subscribers get the bounces back). My quick hack
fix is attached, but I wonder what happens for (say) monthly reminder
mail - whats set for that?
Nigel.
--- Sendmail.py.orig Tue Mar 21 06:25:51 2000
+++ Sendmail.py Sun Apr 9 22:17:37 2000
@@ -56,7 +56,7 @@
# nothing to do!
return
# Use -f to set the envelope sender
- cmd = mm_cfg.SENDMAIL_CMD + ' -f ' + msg.GetSender() + ' '
+ cmd = mm_cfg.SENDMAIL_CMD + ' -f ' + mlist.GetAdminEmail() + ' '
# make sure the command line is of a manageable size
recipchunks = []
currentchunk = []
--
[ - Opinions expressed are personal and may not be shared by VData - ]
[ Nigel Metheringham Nigel.Metheringham@VData.co.uk ]
[ Phone: +44 1423 850000 Fax +44 1423 858866 ]
From thomas@xs4all.net Mon Apr 10 09:16:29 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 10 Apr 2000 10:16:29 +0200
Subject: [Mailman-Developers] 2 quickies..
In-Reply-To: ; from bigdog@dogpound.vnet.net on Sun, Apr 09, 2000 at 05:15:05PM -0400
References:
Message-ID: <20000410101629.H13830@xs4all.nl>
On Sun, Apr 09, 2000 at 05:15:05PM -0400, Matt Davis wrote:
> 1. Just making sure with staying up with the CVS tree, will I overwrite
> any info (list config/membership/etc.) if I update my
> /home/mailman/mailman (CVS source dir) and then do 'make install' (which
> now does 'make update').
Should be no problem, i've been doing it for months. You may have to re-edit
your option-pages though, if you make any changes to the HTML.
> 2. I think I found a little bug. When a message is kicked to the admin
> for the message having too many users. After the admin approves the
> message to be sent, it does not show up in the archive page.
D'oh yeah, that was another outstanding bug i had, which i forgot to mention
in my previous mail. Messages that are held for approval lose their 'from '
line, which means the archive for that list gest fuqued. (If the approved
message is the first message in the list, the archive is invalid, and if
it's a later message, it wont be seen as a seperate message but instead part
of the preceding message.)
I sent a patch a week or two ago: the problem is that the held message gets
written to a file using str(msg) -- but rfc822.Message.__str__ does not
prepend the 'unixfrom' line to the message, so that has to be done manually.
Actually, the code could probably use a check to see if the first 5
characters of the resulting string are indeed 'From ', or perhaps even match
the entire fromline, to make sure pipermail/HyperArch does not skip the
message like it does now, but that's a minor concern for later.
The fix is easy, by the way: in Mailman/ListAdmin.py, in function
'HoldMessage(self, msg, reason)', change 'fp.write(str(msg))' to
'fp.write(msg.unixfrom + str(msg))'. msg.unixfrom is either a From line, or
an empty string, and str(msg) never starts with anything From-like, so the
fix should be perfectly safe. (And a must, I might add, for people with busy
moderated lists ;)
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From bigdog@dogpound.vnet.net Mon Apr 10 15:34:00 2000
From: bigdog@dogpound.vnet.net (Matt Davis)
Date: Mon, 10 Apr 2000 10:34:00 -0400 (EDT)
Subject: [Mailman-Developers] 2 quickies..
In-Reply-To: <20000410101629.H13830@xs4all.nl>
Message-ID:
On Mon, 10 Apr 2000, Thomas Wouters wrote:
> Should be no problem, i've been doing it for months. You may have to re-edit
> your option-pages though, if you make any changes to the HTML.
Great, just needed an assurance :)
> The fix is easy, by the way: in Mailman/ListAdmin.py, in function
> 'HoldMessage(self, msg, reason)', change 'fp.write(str(msg))' to
> 'fp.write(msg.unixfrom + str(msg))'. msg.unixfrom is either a From line, or
> an empty string, and str(msg) never starts with anything From-like, so the
> fix should be perfectly safe. (And a must, I might add, for people with busy
> moderated lists ;)
So if its perfectly safe, think it would make it into mailman officially?
--
Matt Davis - ICQ# 934680
http://dogpound.vnet.net/
NoWonder UNIX Tech - http://www.nowonder.com
There's more than one way to skin a cat. Way #15: Krazy Glue and a toothbrush.
From thomas@xs4all.net Mon Apr 10 16:59:10 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 10 Apr 2000 17:59:10 +0200
Subject: [Mailman-Developers] 2 quickies..
In-Reply-To: ; from bigdog@dogpound.vnet.net on Mon, Apr 10, 2000 at 10:34:00AM -0400
References: <20000410101629.H13830@xs4all.nl>
Message-ID: <20000410175910.A28687@xs4all.nl>
On Mon, Apr 10, 2000 at 10:34:00AM -0400, Matt Davis wrote:
> > The fix is easy, by the way: in Mailman/ListAdmin.py, in function
> > 'HoldMessage(self, msg, reason)', change 'fp.write(str(msg))' to
> > 'fp.write(msg.unixfrom + str(msg))'. msg.unixfrom is either a From line, or
> > an empty string, and str(msg) never starts with anything From-like, so the
> > fix should be perfectly safe. (And a must, I might add, for people with busy
> > moderated lists ;)
> So if its perfectly safe, think it would make it into mailman officially?
I hope so :) but most mailman developers (those with CVS access) are quite
busy. Looking at the storm of small fixes and patches for longstanding bugs,
and replies to old messages, in the last few days (in preperation for 2.0rc0
I presume ;) I suspect they'll come past these fixes soonish ;)
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From Nigel.Metheringham@VData.co.uk Mon Apr 10 17:54:27 2000
From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham)
Date: Mon, 10 Apr 2000 17:54:27 +0100
Subject: [Mailman-Developers] New MTA interface in mailman 2.0b
Message-ID:
I've got a couple of queries about how the MTA interfaces in mailman
hang together now - I'm also putting some of the initial stuff in more
detail so others can find out what to hack to select things.
There are 2 MTA modules -
Mailman/Handlers/SMTPDirect.py
Mailman/Handlers/Sendmail.py
I would say from what I have seen of the Sendmail.py one that most
developers have been using SMTPDirect :-) - thats the default in
Defaults.py
To use the sendmail i/f add this to mm_cfg.py
DELIVERY_MODULE = 'Sendmail'
SENDMAIL_CMD = '/usr/sbin/sendmail'
[I actually use exim, but MTAs on linux with a sendmail like interface
normally have a presence - either a symlink or a sendmail CLI emulator
- at /usr/sbin/sendmail ]
You also need to add the appropriate magic for your MTA to accept the
lies that mailman tells it as an appropriate sender address (ie the -f
parameter to sendmail). In my config with exim this means adding
trusted_groups=mailman
to my config file.
I also needed to patch Sendmail.py to get the envelope from addresses
right:-
--- Sendmail.py.orig Tue Mar 21 06:25:51 2000
+++ Sendmail.py Sun Apr 9 22:17:37 2000
@@ -56,7 +56,7 @@
# nothing to do!
return
# Use -f to set the envelope sender
- cmd = mm_cfg.SENDMAIL_CMD + ' -f ' + msg.GetSender() + ' '
+ cmd = mm_cfg.SENDMAIL_CMD + ' -f ' + mlist.GetAdminEmail() + ' '
# make sure the command line is of a manageable size
recipchunks = []
currentchunk = []
So, now to the questions:-
1. Everything now looks like a list mail - looks like some items which
used
to look distinct from list mails have the same headers as the list
mails
[ie admin/info mails have now acquired extra headers]
Is this intentional/good??
[I'm agnostics really - its just a change]
2. I'm concerned the patch above might do bad things when it comes to
sending out list reminders :-)
3. What happens in either MTA interface if the MTA cannot accept the
message
[for SMTP either connection refused, or a negative reply code? For
Sendmail a negative reply code] - is the message dropped??
How does the bit of queue handling thats left tie into this?
Nigel.
--
[ - Opinions expressed are personal and may not be shared by VData - ]
[ Nigel Metheringham Nigel.Metheringham@VData.co.uk ]
[ Phone: +44 1423 850000 Fax +44 1423 858866 ]
From bwarsaw@cnri.reston.va.us Mon Apr 10 18:29:12 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 10 Apr 2000 13:29:12 -0400 (EDT)
Subject: [Mailman-Developers] Bouncers/Exim.py
References: <20000325174141.A13388@mail.dskk.co.jp>
Message-ID: <14578.3944.503268.405482@anthem.cnri.reston.va.us>
>>>>> "JT" == Jim Tittsler writes:
JT> Exim adds a header to bounce messages that might be used to
JT> catch the bounced addresses, rather than trying to parse the
JT> message text. I've been using this trivial bouncer for a
JT> while with my 1.2CVS installation and it seems to be doing the
JT> right thing.
Jim's Exim detector has been added to the CVS tree. Thanks Jim!
From bwarsaw@cnri.reston.va.us Mon Apr 10 22:28:47 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 10 Apr 2000 17:28:47 -0400 (EDT)
Subject: [Mailman-Developers] New MTA interface in mailman 2.0b
References:
Message-ID: <14578.18319.258116.851940@anthem.cnri.reston.va.us>
>>>>> "NM" == Nigel Metheringham writes:
NM> I would say from what I have seen of the Sendmail.py one that
NM> most developers have been using SMTPDirect :-) - thats the
NM> default in Defaults.py
Right. Sendmail.py was essentially a quick hack to prove the concept
of the message handler pipeline. I didn't and haven't put a lot of
work into it. I definitely recommend people use SMTPDirect.py
instead, especially after the qrunner additions in 2.0beta2.
NM> I also needed to patch Sendmail.py to get the envelope from
NM> addresses right
Yup, I'll check that change in.
NM> 1. Everything now looks like a list mail - looks like some
NM> items which used to look distinct from list mails have the
NM> same headers as the list mails
| [ie admin/info mails have now acquired extra headers]
| Is this intentional/good??
| [I'm agnostics really - its just a change]
Here's what happens. When a message is destined for the entire list,
it is handled through the DeliverToList() pipeline (see
Mailman/Handlers/HandlerAPI.py). You can see that there are a bunch
of modules that touch the message before it's sent out: SpamDetect,
Approve, Replybot, etc.
Messages that the system generates and is destined for a specific
individual are all delivered through DeliverToUser(), which has a much
shorter pipeline module list.
One module that all messages pass through is CookHeaders, which
touches headers like Sender, Errors-To, Precedence; it adds an
X-BeenThere header and an X-Mailman-Version header. Note that
CookHeaders does /not/ munge the Subject header because DeliverToUser
sets the `fasttrack' attribute on the message object. This is a flag
to all pipeline modules that the message is being delivered to an
individual user and not to the list. It's a hack, but it allows me to
re-use most of the same code.
I don't see anything in the admin messages that concerns me; were
there specific headers that you thought might be a problem? If so we
can either special-case them (with `fasttrack') in CookHeaders, or we
can write a separate CookHeaders for DeliverToUser().
NM> 2. I'm concerned the patch above might do bad things when it
NM> comes to sending out list reminders :-)
Password reminders are sent through DeliverToUser. You can actually
force a mass password reminder mailing by running
% python cron/mailpasswds
explicitly. Don't do this on your production system :) Again, none
of the headers that are included strike me as horrible.
NM> 3. What happens in either MTA interface if the MTA cannot
NM> accept the message
NM> [for SMTP either connection refused, or a negative reply
NM> code? For Sendmail a negative reply code] - is the message
NM> dropped?? How does the bit of queue handling thats left tie
NM> into this?
Ah, now here is a good reason to use SMTPDirect, at least with
2.0beta2. If SMTPDirect either gets a fatal exception (socket.error
or smtplib.SMTPException) or if some individual users had
non-permanent failures, then the message is queued for redelivery to
the local smtpd. There's a new cron script called qrunner which will
attempt re-delivery every 1/2 hour of those failed messages. This has
only seen limited testing.
The Sendmail deliverer has no such fallback. There was some
discussion about extending the error handling to catch any failures in
any pipeline module, but I think that's a bit more complicated than I
want to add right now. It's definitely a good idea though.
Hope that helps,
-Barry
From brian@SoftHome.net Tue Apr 11 00:31:43 2000
From: brian@SoftHome.net (Brian Grossman)
Date: Mon, 10 Apr 2000 17:31:43 -0600
Subject: [Mailman-Developers] observations from new user
In-Reply-To: Your message of "09 Apr 2000 22:05:41 +0200."
Message-ID: <20000410233143.23851.qmail@lindy.softhome.net>
This is a multipart MIME message.
--==_Exmh_9383417880
Content-Type: text/plain; charset=us-ascii
> > One general observation: Why does DEFAULT_PRIVATE_ROSTER default to
> > 0? I would think that 2 would be a better choice.
>
> Well, obviously someone had to choose some sort of default back in the
> murky beginnings of time.
>
> Why that someone preferred 0 over 2? Dunno, but as long as people
> have become used to this, or even are comfortable with it[1], I think
> you'd have to make a stronger argument for why you think it should be
> changed than the quick note above :)
Two, then:
Does anyone actually run with DEFAULT_PRIVATE_ROSTER set to 0? My
suspicion is that just about everyone either sets it in their mm_cfg.py or
sets it manually to 2 for each list.
Shouldn't the default to be to protect privacy? 2 does that. 0 does not.
> > Where should I send a script that eliminates the need for five
> > .qmail files per list?
>
> As Ricardo said: Here would be a good place.
It's attached.
Brian
--==_Exmh_9383417880
Content-Type: text/plain ; name="Qwrap"; charset=us-ascii
Content-Description: Qwrap
Content-Disposition: attachment; filename="Qwrap"
#!/bin/bash
# Copyright 2000, Brian Grossman
# You may use this program under the same conditions as you may use
# Mailman itself. See the file gnu-COPYING-GPL in the Mailman 2 distribution.
# Author: Brian Grossman
# 8 Apr 2000 birth
# This script is intended to make it so you don't have to make 5
# .qmail files for each list.
# Put in .qmail-HOST-default or .qmail-default:
# |preline /home/mailman/Qwrap
#
# Pay special attention to setting $forward and $mailman_owner below.
# It assumes that you have lines like this in your /var/qmail/users/assign:
# +softhome-:mailman:417:106:/home/mailman:-:softhome-:
# paired with lines like this in your /var/qmail/control/virtualdomains:
# lists.softhome.net:softhome
# (Don't forget to run qmail-newu.)
# It's intended for qmail 1.03. I haven't tested it with any other version.
# It's intended for mailman 2.0b2. I haven't tested it with any other version.
forward=/var/qmail/bin/forward
mailman_owner='your.email.here'
do=''
# do=echo # for debugging
if [ $do = echo ]
then
exec
of "Mon, 10 Apr 2000 17:28:47 EDT." <14578.18319.258116.851940@anthem.cnri.reston.va.us>
Message-ID:
NM> 2. I'm concerned the patch above might do bad things when it
NM> comes to sending out list reminders :-)
BW> Password reminders are sent through DeliverToUser. You can
actually
BW> force a mass password reminder mailing by running
Let me explain further....
The sendmail command is given by
cmd = mm_cfg.SENDMAIL_CMD + ' -f ' + mlist.GetAdminEmail() + ' '
So the sender is set to be the list admin address.
However when sending passwords out presumably there isn't a single list
thats being dealt with - so will that mlist.GetAdminEmail() work
correctly?
[Unfortunately I still find reading python a significant chore and its
taking time to get up to speed on being able to answer this stuff
myself :-( ]
> I definitely recommend people use SMTPDirect.py instead, especially
> after the qrunner additions in 2.0beta2.
I had missed the implications of that change - I definitely agree that
SMTP injection looks like a win here, so I'll reconfigure my box back.
> One module that all messages pass through is CookHeaders
This looks like a change... the admin messages have had more header
processing done on them. As I said before, its not actually a problem,
just a change that caught me out slightly (some admin mail was being
misrecognised by my procmail as list mail so ended up in the wrong
bucket - I will take a cluebat to my procmailrc :-) ).
The other advantage to moving to SMTPDirect as a delivery scheme is
that my exim HOWTO works unchanged :-)
Nigel.
--
[ - Opinions expressed are personal and may not be shared by VData - ]
[ Nigel Metheringham Nigel.Metheringham@VData.co.uk ]
[ Phone: +44 1423 850000 Fax +44 1423 858866 ]
From bwarsaw@python.org Tue Apr 11 19:51:31 2000
From: bwarsaw@python.org (Barry Warsaw)
Date: Tue, 11 Apr 2000 14:51:31 -0400 (EDT)
Subject: [Mailman-Developers] Change to acceptable_aliases
Message-ID: <14579.29747.148446.59654@anthem.cnri.reston.va.us>
Hi folks,
I want to make a change to the semantics for the privacy option
`acceptable_aliases' for beta 3. Harald recently (and rightfully)
backed out a change I'd made between 1.1 and 2.0b1, which caused me to
re-examine how this option is used.
The purpose of acceptable_aliases is to be able to specify a list of
aliases for your mailing list that are acceptable as recipients if
require_explicit_destination is set to true (which it is by default).
The example I like to use is if I have a mailing list
foo@python.org
I might want to allow `foofriends@alterpost.com' to be a valid
forwarder to the mailing list.
The trouble comes in that the regexps in acceptable_aliases are only
matched against the localpart of the recipient address. This means
that if I were to put `foofriends@alterpost.com' in this field, it
would actually fail to match a To: header with this address in it.
I'd instead have to put `foofriends' in this field, but then this
would allow `foofriends@evil.com' to post to the list too without
trigger the spam detector.
My proposal is to change the semantics of acceptable_aliases, so that
matching is done against the entire recipient address, not just the
localpart. To support backwards compatibility, I'm willing to use the
following rules for 2.0:
1) if the regexp does not contain an `@', then match only against the
localpart as before.
2) if the regexp contains an `@', match against the entire recip
address.
3) deprecate #1 so that a future release will only match against the
entire address.
Thoughts? Is anybody using acceptable_aliases?
-Barry
From Dan Mick Wed Apr 12 01:14:07 2000
From: Dan Mick (Dan Mick)
Date: Tue, 11 Apr 2000 17:14:07 -0700 (PDT)
Subject: [Mailman-Developers] Speaking of pathological behavior...
Message-ID: <200004120014.RAA05882@utopia.west.sun.com>
> [Dan Mick]
>
> > 1) the archive links are remade, even though they exist (probably not a big
deal)
> > 2) config.db.tmp is written, and then .db becomes .last and .tmp becomes
.db.
>
> Thanks for reporting, I think the change I just checked into CVS
> should fix this (admin.py was calling MailList.SetUserOption() without
> the keyword argument "save_list=0", causing the list to be Save()d at
> least
>
> admin_user_chunksize * len(("hide", "nomail", "ack", "notmetoo", "plain"))
>
> times(!) every time the list's Membership Management page was
> visited).
Yup...
> > Probably batching up all the "changes" into one config.db write is a
> > big big win.
>
> The bigger a admin_user_chunksize you're using, the bigger this win
> will be :)
The only other thought I had was that comparing the options
and only saving on a change might be *slightly* safer (crashes
inbetween users would save more info) but I don't know that it's
worth the extra code complexity.
Thanks for the checkin, Harald. It's a big help for me, and now
I can stop bugging Barry about it. :)
From Harald.Meland@usit.uio.no Wed Apr 12 14:40:56 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 12 Apr 2000 15:40:56 +0200
Subject: [Mailman-Developers] Change to acceptable_aliases
In-Reply-To: bwarsaw@python.org's message of "Tue, 11 Apr 2000 14:51:31 -0400 (EDT)"
References: <14579.29747.148446.59654@anthem.cnri.reston.va.us>
Message-ID:
[Barry Warsaw]
> My proposal is to change the semantics of acceptable_aliases, so that
> matching is done against the entire recipient address, not just the
> localpart. To support backwards compatibility, I'm willing to use the
> following rules for 2.0:
>
> 1) if the regexp does not contain an `@', then match only against the
> localpart as before.
>
> 2) if the regexp contains an `@', match against the entire recip
> address.
>
> 3) deprecate #1 so that a future release will only match against the
> entire address.
>
> Thoughts?
As we're talking (possibly incomprehensible) regexps here, I think I
would go a little bit further:
1) If the regexp does not contain an '@', first try matching it
against the localpart (i.e. the way things work now).
2) If 1) was skipped *or* if it didn't produce a match, try matching
against entire recip address (i.e. try this even if the regexp
does not contain any '@' signs).
3) Deprecate '@'-less regexps so that a future release will only
match against the entire address. Warn the site admin if any
'@'-less regexps are found when upgrading to such a (future)
version.
4) Possibly provide a small script that iterates through
all/specified lists, adding '@.*' to any lines in
acceptable_aliases not containing an '@'. Such a change _could_
break some (strange) regexps, and thus should not be done
automatically.
> Is anybody using acceptable_aliases?
Yes, they're used pretty much around here. Due to the nature of our
previous mailing list system, all old lists now converted to Mailman
have quite a lot of alternative posting addresses; this led to quite a
few messages being held because of "implicit destination" right after
the conversion.
--
Harald
From Harald.Meland@usit.uio.no Wed Apr 12 14:54:54 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 12 Apr 2000 15:54:54 +0200
Subject: [Mailman-Developers] New MTA interface in mailman 2.0b
In-Reply-To: Nigel Metheringham's message of "Tue, 11 Apr 2000 09:15:13 +0100"
References:
Message-ID:
[Nigel Metheringham]
> NM> 2. I'm concerned the patch above might do bad things when it
> NM> comes to sending out list reminders :-)
>
> BW> Password reminders are sent through DeliverToUser. You can
> actually
> BW> force a mass password reminder mailing by running
>
> Let me explain further....
>
> The sendmail command is given by
> cmd = mm_cfg.SENDMAIL_CMD + ' -f ' + mlist.GetAdminEmail() + ' '
>
> So the sender is set to be the list admin address.
> However when sending passwords out presumably there isn't a single list
> thats being dealt with - so will that mlist.GetAdminEmail() work
> correctly?
No, I don't think it will.
IIRC, cron/mailpasswds iterates through all lists, and uses the
message pipeline of the _first_ public list it finds for delivering
all the reminders. That means that with your patch, any bounced
reminders will go to the unfortunate admin of that first public list.
[ In Mailman 1.1, cron/mailpasswds used 'mailman-owner' as envelope
sender, which although far from optimal is better than the current
situation, IMO. ]
I think it would be better to solve this by overriding the GetSender()
method in the various Message subclasses.
--
Harald
From Harald.Meland@usit.uio.no Wed Apr 12 15:16:48 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 12 Apr 2000 16:16:48 +0200
Subject: [Mailman-Developers] observations from new user
In-Reply-To: Brian Grossman's message of "Mon, 10 Apr 2000 17:31:43 -0600"
References: <20000410233143.23851.qmail@lindy.softhome.net>
Message-ID:
[Brian Grossman]
> Does anyone actually run with DEFAULT_PRIVATE_ROSTER set to 0?
Yes.
> My suspicion is that just about everyone either sets it in their
> mm_cfg.py or sets it manually to 2 for each list.
I don't.
> Shouldn't the default to be to protect privacy?
Not necessarily -- the default should, ideally, be what is most useful
to most of the people using Mailman to create new lists. As the world
hardly is ideal, the issue of how to decide whether a change of
default is really called for or not enters the picture.
I see your point, though -- if I were inventing the option now, I
probably wouldn't have made it default to 0.
However, as I haven't heard anyone else argue for a change, I'm
inclined to interpret that silence as "We think the current default is
OK". I guess people now will let me know if that interpretation is
wrong :)
> > > Where should I send a script that eliminates the need for five
> > > .qmail files per list?
> >
> > As Ricardo said: Here would be a good place.
>
> It's attached.
Thanks,
--
Harald
From Harald.Meland@usit.uio.no Wed Apr 12 15:53:33 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 12 Apr 2000 16:53:33 +0200
Subject: [Mailman-Developers] lock lifetime
In-Reply-To: bwarsaw@cnri.reston.va.us's message of "Wed, 22 Mar 2000 16:18:48 -0500 (EST)"
References: <20000322193038.A10212@miss-janet.com> <14553.6564.572812.203295@anthem.cnri.reston.va.us> <20000322211525.C10212@miss-janet.com> <14553.14520.677676.569943@anthem.cnri.reston.va.us>
Message-ID:
--=-=-=
[bwarsaw@cnri.reston.va.us]
> Actually, I just remembered sys.exitfunc! I can set that up to
> unlock a locked list automatically at interpreter exit. I don't
> want it to implicitly save the list though -- you'll still have to
> do that manually if you fiddle with it from the interpreter prompt.
>
> Note that this function doesn't run if the interpreter exits because
> of a signal or os._exit() call.
Regarding signals: Some time back, I occasionally received reports
from my users that some list, while its web admin interface was being
accessed, suddenly got unresponsive for a period of time that matched
the lock lifetime pretty well.
There were no entries/backtraces in logs/error, but I sometimes saw
list lock files left lying around, and upon checking their contents
found that their creating process was now gone.
I suspect this is what happened:
* Web browser connects to Mailman.
* Web browser for some reason closes its side of the connection
before Mailman has given any response.
* When Mailman tries to print out its response, it receives a
SIGPIPE.
I tried fixing the problem with this patch:
--=-=-=
Content-Disposition: inline
Index: driver
===================================================================
RCS file: /projects/cvsroot/mailman/scripts/driver,v
retrieving revision 1.19
diff -u -r1.19 driver
--- driver 2000/04/06 19:30:38 1.19
+++ driver 2000/04/12 14:43:23
@@ -89,6 +89,13 @@
main()
finally:
sys.stderr = sys.__stderr__
+ except IOError, e:
+ import errno
+ if e.errno == errno.EPIPE:
+ # Log something
+ logger.write("EPIPE caught, ARGV=%s\n" % sys.argv)
+ else:
+ raise
except SystemExit:
# this is a valid way for the function to exit
pass
--=-=-=
Content-Disposition: inline
(Well, something like that patch anyway... :)
However, if my understanding of the problem is right, and this is the
correct fix for it, I don't understand why the bare except: clause
around the run_main() call in driver didn't kick in to produce a log
entry...
Does anyone have any insight on how to properly avoid SIGPIPEs in
python?
--
Harald
--=-=-=--
From Harald.Meland@usit.uio.no Wed Apr 12 16:09:07 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 12 Apr 2000 17:09:07 +0200
Subject: [Mailman-Developers] mailman 2.0beta1 problem: Mail delivery failed: returning message to sender (fwd)
In-Reply-To: "Barry A. Warsaw"'s message of "Wed, 22 Mar 2000 14:21:26 -0500 (EST)"
References: <14553.7478.896273.785720@anthem.cnri.reston.va.us>
Message-ID:
[Barry A. Warsaw]
> I think I understand why you're getting output to stdout. Look in
> scripts/post and change
>
> LogStdErr("error", "post")
>
> to
>
> LogStdErr("error", "post", tee_to_stdout=0)
>
> The question is, why haven't I seen this with Postfix?
I don't know Postfix, but Exim has a config option for what to do with
output generated by pipe deliveries:
The `return_output' option can affect the result of a pipe delivery. If
it is set and the command produces any output on its standard output or
standard error files, it is considered to have failed, even if it gave
a zero return code or if `ignore_status' is set. The output from the
command is sent as part of the delivery failure report. However, if
`return_fail_output' is set, output is returned only when the command
exits with a failure return code, that is, a value other than zero or a
code that matches `temp_errors'.
I guess Gergely has set `return_output' on his Exim transport. Maybe
something similar exists for Postfix?
--
Harald
From marc_news@valinux.com Wed Apr 12 16:38:43 2000
From: marc_news@valinux.com (Marc Merlin)
Date: Wed, 12 Apr 2000 08:38:43 -0700
Subject: [Mailman-Developers] Re: [Mailman-Users] [ANNOUNCE] Mailman 2.0 beta 2
In-Reply-To: <14575.43684.735692.298399@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on sam, avr 08, 2000 at 05:54:44 -0400
References: <14575.43684.735692.298399@anthem.cnri.reston.va.us>
Message-ID: <20000412083843.A25914@moremagic.merlins.org>
On sam, avr 08, 2000 at 05:54:44 -0400, Barry A. Warsaw wrote:
> I've just uploaded the Mailman 2.0 beta 2 tarball to the following
> locations:
>
> -------------------- snip snip --------------------
> - Rewritten gate_news cron script which should be more efficient and
> avoid race and locking problems. Each list now maintains its own
Thank you, I don't receive 30 failed lock messages a day anymore :-)
I've noticed a small problem though:
Subscribe/unsubscribe messages don't have a To: field anymore
----------------------------------------------------------------------------
From mailman-owner@merlins.org Wed Apr 12 13:39:51 2000
From: mailman-owner@merlins.org (mailman-owner@merlins.org)
Date: Wed, 12 Apr 2000 05:39:51 -0700
Subject: Zark unsubscribe notification
Message-ID: <200004121239.FAA13972@marc.merlins.org>
Unsubscribe Notification
rodrigue.mottay@lucasvarity.com has been removed from Zark.
----------------------------------------------------------------------------
--
Microsoft is to software what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ (friendly to non IE browsers)
Finger marc_f@merlins.org for PGP key and other contact information
From marc_news@valinux.com Thu Apr 13 01:23:31 2000
From: marc_news@valinux.com (Marc Merlin)
Date: Wed, 12 Apr 2000 17:23:31 -0700
Subject: [Mailman-Developers] Mailman 2.0b2 checkdb error
Message-ID: <20000412172331.D18895@marc.merlins.org>
----- Forwarded message from Cron Daemon -----
Date: Wed, 12 Apr 2000 17:00:07 -0700
From: root@marc.merlins.org (Cron Daemon)
To: mailman@marc.merlins.org
Subject: Cron /usr/bin/python /var/local/mailman/cron/checkdbs
X-Cron-Env:
X-Cron-Env:
X-Cron-Env:
X-Cron-Env:
Traceback (innermost last):
File "/var/local/mailman/cron/checkdbs", line 87, in ?
main()
File "/var/local/mailman/cron/checkdbs", line 52, in main
text = text + '\n' + pending_requests(mlist)
File "/var/local/mailman/cron/checkdbs", line 72, in pending_requests
pending.append(' %s %s' % addr, time.ctime(when))
TypeError: not enough arguments for format string
----- End forwarded message -----
--
Microsoft is to software what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ (friendly to non IE browsers)
Finger marc_f@merlins.org for PGP key and other contact information
From bwarsaw@cnri.reston.va.us Thu Apr 13 01:37:57 2000
From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us)
Date: Wed, 12 Apr 2000 20:37:57 -0400 (EDT)
Subject: [Mailman-Developers] Re: Mailman 2.0b2 checkdb error
References: <20000412172331.D18895@marc.merlins.org>
Message-ID: <14581.5861.551769.378730@anthem.cnri.reston.va.us>
Yup, I hit that one too. Check the CVS tree; it's got the fix.
-Barry
From thomas@xs4all.net Thu Apr 13 08:55:19 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Thu, 13 Apr 2000 09:55:19 +0200
Subject: [Mailman-Developers] cosmetic bug in order of held messages
Message-ID: <20000413095519.C14644@xs4all.nl>
My boss found a small cosmetic bug in message holding/approving: when
approving a number of messages at the same time (ie, more than one) the
order of the messages is screwed, which means that messages will be sent to
the list and to the archive in a different order than in which it was
recieved.
Not much of a bug, i agree, but it does show itself, for instance when
people post (manual) multi-part messages, or when a discussion goes to two
lists, one moderated and one not, and people reply to one another, the
replies show up first. The problem is caused by the way pending admin
requests are stored: in a dictionary, with sequential id's as the key.
Adding a 'ids.sort()' before 'return ids' in __getmsgids() (in ListAdmin.py)
solves the problem for the most part, and keeps my boss happily playing with
his pet project ;)
Should I bother sending a patch ?
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From justinp@head-gear.com Thu Apr 13 23:01:41 2000
From: justinp@head-gear.com (Justin Patterson)
Date: Thu, 13 Apr 2000 17:01:41 -0500
Subject: [Mailman-Developers] Customization of web interfaces
Message-ID: <02d901bfa593$d99f6220$f331e1ce@zoe>
I've been using mailman for a while with the default web page templates. I
decided that to provide a more consistent look with a new site that I'm
building, I would customize the templates. I soon figured out that many of
the pages are not built from templates, but from the python scripts
themselves.
Is there any sort of project working on getting all of the information
regarding the format of the pages out of the python scripts and into the
templates?
I hacked together a little proof of concept where I modified a few of the
python scripts to produce XML which I could then munge using the Apache
XML/XSL packages into a page that matches my site. I have no problem
writing my own CGI scripts to pull stuff out of the mailman databases and
formatting it, but it seems like it would be nice for users at large to be
able to completely customize the web interface somehow.
So, is there already someone working on this problem?
Should I continue down my current path?
Thanks,
-Justin
From bwarsaw@cnri.reston.va.us Thu Apr 13 23:08:44 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Thu, 13 Apr 2000 18:08:44 -0400 (EDT)
Subject: [Mailman-Developers] cosmetic bug in order of held messages
References: <20000413095519.C14644@xs4all.nl>
Message-ID: <14582.17772.304483.583310@anthem.cnri.reston.va.us>
>>>>> "TW" == Thomas Wouters writes:
TW> The problem is caused by the way pending admin requests
TW> are stored: in a dictionary, with sequential id's as the key.
TW> Adding a 'ids.sort()' before 'return ids' in __getmsgids() (in
TW> ListAdmin.py) solves the problem for the most part, and keeps
TW> my boss happily playing with his pet project ;)
TW> Should I bother sending a patch ?
Naw. See below.
-Barry
Index: ListAdmin.py
===================================================================
RCS file: /projects/cvsroot/mailman/Mailman/ListAdmin.py,v
retrieving revision 1.29
diff -c -r1.29 ListAdmin.py
*** ListAdmin.py 2000/04/08 17:12:22 1.29
--- ListAdmin.py 2000/04/13 22:03:14
***************
*** 98,103 ****
--- 98,104 ----
for k, (type, data) in self.__db.items():
if type == rtype:
ids.append(k)
+ ids.sort()
return ids
def GetHeldMessageIds(self):
From bwarsaw@cnri.reston.va.us Fri Apr 14 00:13:31 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Thu, 13 Apr 2000 19:13:31 -0400 (EDT)
Subject: [Mailman-Developers] Change to acceptable_aliases
References: <14579.29747.148446.59654@anthem.cnri.reston.va.us>
Message-ID: <14582.21659.123626.535035@anthem.cnri.reston.va.us>
>>>>> "HM" == Harald Meland writes:
HM> 1) If the regexp does not contain an '@', first try matching
HM> it against the localpart (i.e. the way things work now).
HM> 2) If 1) was skipped *or* if it didn't produce a match, try
HM> matching against entire recip address (i.e. try this even if
HM> the regexp does not contain any '@' signs).
HM> 3) Deprecate '@'-less regexps so that a future release will
HM> only match against the entire address. Warn the site admin if
HM> any '@'-less regexps are found when upgrading to such a
HM> (future) version.
HM> 4) Possibly provide a small script that iterates through
HM> all/specified lists, adding '@.*' to any lines in
HM> acceptable_aliases not containing an '@'. Such a change
HM> _could_ break some (strange) regexps, and thus should not be
HM> done automatically.
Except that I think `@'-less regexps still have useful semantics, so I
don't want to prohibit them. I just want the matching to (eventually)
always be done against the entire recip address, not just the
localpart.
Because `@' isn't special in regular expressions, I can't think of any
reason someone would currently include it in acceptable_aliases unless
they incorrectly thought the entire address was being matched. At
least, that's the misconception /I/ had!
So I think my approach is the right tradeoff in trying to guess what
the intent of the admin was, and shouldn't break anybody's current
acceptable_aliases settings. It might start magically working for
those who had the same misconception that I did, but I think that's
okay (they expected it to work anyway).
-Barry
From jwt@dskk.co.jp Fri Apr 14 07:13:05 2000
From: jwt@dskk.co.jp (Jim Tittsler)
Date: Fri, 14 Apr 2000 15:13:05 +0900
Subject: [Mailman-Developers] 2.0beta3 update gate_watermarks->config.db for non-existent list
Message-ID: <20000414151305.A5025@mail.dskk.co.jp>
--+QahgC5+KEYLbs62
Content-Type: text/plain; charset=us-ascii
I just tried to update a system from 2.0b1 to 2.0b3 (CVS @ ~0300UTC) and
discovered a problem with the section of 'update' that copies the
gate_watermarks file into the config.db files. gate_watermarks can contain
watermarks for lists that no longer exist. That results in:
Updating Usenet watermarks
Traceback (innermost last):
File "bin/update", line 293, in ?
mlist = MailList.MailList(listname)
File "/home/mailman/Mailman/MailList.py", line 70, in __init__
self.Load()
File "/home/mailman/Mailman/MailList.py", line 880, in Load
raise Errors.MMUnknownListError, e
Mailman.Errors.MMUnknownListError: [Errno 2] No such file or directory:
'/home/mailman/lists/tpc.test3/config.db'
There are lots of ways to fix it... the one I used is attached.
--
Jim Tittsler, Tokyo
Python Starship http://starship.python.net/crew/jwt/
--+QahgC5+KEYLbs62
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="update.diff"
*** update.orig Fri Apr 14 14:37:40 2000
--- update Fri Apr 14 15:11:06 2000
***************
*** 291,294 ****
--- 291,297 ----
fp.close()
for listname in d.keys():
+ # if a watermark for a list that no longer exists, skip it
+ if listname not in lists:
+ continue
mlist = MailList.MailList(listname)
# Pre 1.0b7 stored 0 in the gate_watermarks file to indicate that
--+QahgC5+KEYLbs62--
From bwarsaw@cnri.reston.va.us Fri Apr 14 16:34:00 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Fri, 14 Apr 2000 11:34:00 -0400 (EDT)
Subject: [Mailman-Developers] 2.0beta3 update gate_watermarks->config.db for non-existent list
References: <20000414151305.A5025@mail.dskk.co.jp>
Message-ID: <14583.14952.569384.589591@anthem.cnri.reston.va.us>
>>>>> "JT" == Jim Tittsler writes:
JT> I just tried to update a system from 2.0b1 to 2.0b3 (CVS @
JT> ~0300UTC) and discovered a problem with the section of
JT> 'update' that copies the gate_watermarks file into the
JT> config.db files. gate_watermarks can contain watermarks for
JT> lists that no longer exist. That
JT> There are lots of ways to fix it... the one I used is
JT> attached.
Thanks, I applied this patch (essentially).
-Barry
From Harald.Meland@usit.uio.no Fri Apr 14 17:55:08 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 14 Apr 2000 18:55:08 +0200
Subject: [Mailman-Developers] Change to acceptable_aliases
In-Reply-To: "Barry A. Warsaw"'s message of "Thu, 13 Apr 2000 19:13:31 -0400 (EDT)"
References: <14579.29747.148446.59654@anthem.cnri.reston.va.us> <14582.21659.123626.535035@anthem.cnri.reston.va.us>
Message-ID:
[Barry A. Warsaw]
> Except that I think `@'-less regexps still have useful semantics, so
> I don't want to prohibit them. I just want the matching to
> (eventually) always be done against the entire recip address, not
> just the localpart.
Then wouldn't it make more sense to already now try matching the
regexp against the entire address, even if the regexp does not contain
any '@'?
Your change will cause the (not too far-fetched, IMO) regexp
.*\.uio\.no$
to _only_ be matched against the localpart (and thus very likely _not_
produce a match) for now, although you're saying that you eventually
want it to match against the entire address (and that likely _will_
produce a match).
So, I still say:
1) If the regexp does not contain an '@', first try matching it
against the localpart (i.e. the way things work now).
2) If 1) was skipped *or* if it didn't produce a match, try matching
against entire recip address (i.e. try this even if the regexp
does not contain any '@' signs).
Or am I misunderstanding something?
--
Harald
From bwarsaw@cnri.reston.va.us Fri Apr 14 18:20:05 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Fri, 14 Apr 2000 13:20:05 -0400 (EDT)
Subject: [Mailman-Developers] Change to acceptable_aliases
References: <14579.29747.148446.59654@anthem.cnri.reston.va.us>
<14582.21659.123626.535035@anthem.cnri.reston.va.us>
Message-ID: <14583.21317.443858.175577@anthem.cnri.reston.va.us>
>>>>> "HM" == Harald Meland writes:
HM> So, I still say:
HM> 1) If the regexp does not contain an '@', first try matching
HM> it against the localpart (i.e. the way things work now).
HM> 2) If 1) was skipped *or* if it didn't produce a match, try
HM> matching against entire recip address (i.e. try this even if
HM> the regexp does not contain any '@' signs).
HM> Or am I misunderstanding something?
No, I think I misunderstood your original proposal. Sorry about that.
+1 on Harald's approach, with a provision to remove #1 in some future
version.
-Barry
P.S. The "+1" is taken from Apache's voting mechanism, as described by
Greg Stein on the Python-Dev list:
http://www.python.org/pipermail/python-dev/2000-March/004312.html
Briefly explained:
+1 "I'm all for it. Do it!"
+0 "Seems cool and acceptable, but I can also live without it"
-0 "Not sure this is the best thing to do, but I'm not against it."
-1 "Veto. And is my reasoning."
It's actually worked quite nicely on Python-Dev, so I propose we adopt
it here too.
From bwarsaw@cnri.reston.va.us Fri Apr 14 18:53:48 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Fri, 14 Apr 2000 13:53:48 -0400 (EDT)
Subject: [Mailman-Developers] Change to acceptable_aliases
References: <14579.29747.148446.59654@anthem.cnri.reston.va.us>
<14582.21659.123626.535035@anthem.cnri.reston.va.us>
<14583.21317.443858.175577@anthem.cnri.reston.va.us>
Message-ID: <14583.23340.931392.254028@anthem.cnri.reston.va.us>
>>>>> "BAW" == Barry A Warsaw writes:
BAW> +1 on Harald's approach, with a provision to remove #1 in
BAW> some future version.
And here's the patch.
-Barry
Index: MailList.py
===================================================================
RCS file: /projects/cvsroot/mailman/Mailman/MailList.py,v
retrieving revision 1.161
diff -c -r1.161 MailList.py
*** MailList.py 2000/04/13 23:25:16 1.161
--- MailList.py 2000/04/14 17:34:09
***************
*** 1215,1250 ****
# this is the list's full address
listfullname = '%s@%s' % (self.internal_name(), self.host_name)
recips = []
! # check all recipient addresses against the list's full address...
for fullname, addr in msg.getaddrlist('to') + msg.getaddrlist('cc'):
! localpart = string.lower(string.split(addr, '@')[0])
! laddr = string.lower(addr)
if (# TBD: backwards compatibility: deprecated
localpart == self.internal_name() or
! # new behavior
! laddr == listfullname):
return 1
! recips.append((laddr, localpart))
! # ... and only then try the regexp acceptable aliases.
! for laddr, localpart in recips:
for alias in string.split(self.acceptable_aliases, '\n'):
stripped = string.strip(alias)
! if '@' in stripped:
! # new behavior: match the entire recipient string
! recip = laddr
! else:
! # TBD: backwards compatibility: deprecated
! recip = localpart
! try:
! # The list alias in `stripped` is a user supplied regexp,
! # which could be malformed.
! if stripped and re.match(stripped, recip):
! return 1
! except re.error:
! # `stripped' is a malformed regexp -- try matching
! # safely, with all non-alphanumerics backslashed:
! if stripped and re.match(re.escape(stripped), recip):
! return 1
return 0
def parse_matching_header_opt(self):
--- 1215,1261 ----
# this is the list's full address
listfullname = '%s@%s' % (self.internal_name(), self.host_name)
recips = []
! # check all recipient addresses against the list's explicit address.
for fullname, addr in msg.getaddrlist('to') + msg.getaddrlist('cc'):
! addr = string.lower(addr)
! localpart = string.split(addr, '@')[0]
if (# TBD: backwards compatibility: deprecated
localpart == self.internal_name() or
! # exact match against the complete list address
! addr == listfullname):
return 1
! recips.append((addr, localpart))
! #
! # helper function used to match a pattern against an address. Do it
! def domatch(pattern, addr):
! try:
! if re.match(pattern, addr):
! return 1
! except re.error:
! # The pattern is a malformed regexp -- try matching safely,
! # with all non-alphanumerics backslashed:
! if re.match(re.escape(pattern), addr):
! return 1
! #
! # Here's the current algorithm for matching acceptable_aliases:
! #
! # 1. If the pattern does not have an `@' in it, we first try matching
! # it against just the localpart. This was the behavior prior to
! # 2.0beta3, and is kept for backwards compatibility.
! # (deprecated).
! #
! # 2. If that match fails, or the pattern does have an `@' in it, we
! # try matching against the entire recip address.
! for addr, localpart in recips:
for alias in string.split(self.acceptable_aliases, '\n'):
stripped = string.strip(alias)
! if not stripped:
! # ignore blank or empty lines
! continue
! if '@' not in stripped and domatch(stripped, localpart):
! return 1
! if domatch(stripped, addr):
! return 1
return 0
def parse_matching_header_opt(self):
From elau@ics.uci.edu Fri Apr 14 19:03:44 2000
From: elau@ics.uci.edu (Edmund Lau)
Date: Fri, 14 Apr 2000 11:03:44 -0700 (PDT)
Subject: [Mailman-Developers] a few suggestions
Message-ID:
I know it's a bit in the beta process for the 2.0 version, but is it
possible to change the documentation a little bit? Here's what I'm
suggesting with regard to forgotten passwords and archives. I've been
using 1.1 for the most part so this stuff may already have been
implemented in the newer betas.
- Change the list-request response message entry for the password command
to also say "the command by itself will send your current password to
your Email address"
- Add a comment on the user configuration webpage that sending an Email to
list-request with the password command also works
- Change the "About " to say "Information About and its
Archives". Many of my users usually skim for the large size font so
this might make it easier for them to find the archives
Thanks!
-Ed
From lindsey@ncsa.uiuc.edu Fri Apr 14 19:31:07 2000
From: lindsey@ncsa.uiuc.edu (Christopher Lindsey)
Date: Fri, 14 Apr 2000 13:31:07 -0500 (CDT)
Subject: [Mailman-Developers] Change to acceptable_aliases
In-Reply-To: <14583.21317.443858.175577@anthem.cnri.reston.va.us> from "Barry A. Warsaw" at Apr 14, 2000 01:20:05 PM
Message-ID: <200004141831.NAA09979@glorfindel.ncsa.uiuc.edu>
No, I think I misunderstood your original proposal. Sorry about that.
>
> +1 on Harald's approach, with a provision to remove #1 in some future
> version.
About a year ago I sent out a proposal to the list about this that
I think might still have some merit. Rather than doing regular
expression matching, why not do regex plus substitution? That
would allow admins to create rules like
'\+[^@]' ' '
which essentially converts all plus-addressed subscriptions to
a non-plus-addressed format, thereby fixing the existing problem
with plus-addressing.
Unfortunately, if it's left in acceptable_aliases it will go
ahead and approve anybody with plus-addressing. :) So what
do people think about creating another option in the privacy
settings entitled sub_regex? Basically, all email addresses
would be transformed with this regex before going to
acceptable_aliases.
What does this buy me?
I could have a sub_regex setting like
'@.*\.ncsa\.uiuc\.edu$' '@ncsa.uiuc.edu'
followed by an acceptable_alias setting of
@ncsa\.uiuc\.edu$
Now anyone at ncsa.uiuc.edu can post to the list, whether or not
their hostname shows up in their address, etc. Likewise,
'^lindsey@[^@]+$' 'lindsey@ncsa.uiuc.edu'
would accept email from anyone who's email address starts with
'lindsey@'.
Sure, you can shoot yourself in the foot. Sure, you can open up
your list. But the extra power far outweighs the potentially stupid
things that people can do with it. :)
Chris
Chris
From ricardo@rixhq.nu Fri Apr 14 19:50:40 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Fri, 14 Apr 2000 20:50:40 +0200
Subject: [Mailman-Developers] Change to acceptable_aliases
In-Reply-To: <14583.21317.443858.175577@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Fri, Apr 14, 2000 at 01:20:05PM -0400
References: <14579.29747.148446.59654@anthem.cnri.reston.va.us> <14582.21659.123626.535035@anthem.cnri.reston.va.us> <14583.21317.443858.175577@anthem.cnri.reston.va.us>
Message-ID: <20000414205040.E9709@miss-janet.com>
Hi,
On Fri, Apr 14, 2000 at 01:20:05PM -0400, Barry A. Warsaw wrote:
> It's actually worked quite nicely on Python-Dev, so I propose we adopt
> it here too.
+1 "I'm all for it. Do it!"
:)
Ricardo.
--
From bolen@hcs.harvard.edu Sat Apr 15 07:45:52 2000
From: bolen@hcs.harvard.edu (britt)
Date: Sat, 15 Apr 2000 02:45:52 -0400 (EDT)
Subject: [Mailman-Developers] probing list names, and subscribers
Message-ID:
I noticed there were a message or two about the ability to probe for list
membership, and list existance, even when privacy features have been
turned on. I didn't see anything about this in the To-Do list.
Has anyone made any noise about working on this problem?
I see it as two fold, one list names can be probed for existance. The
same thing for membership simply by guessing names to go after
http://host/mailman/admin and http://host/mailman/options/ This defeats
the purpose of having private lists, which is an absolute necessity for my
system.
I think both of these can be easily fixed, and I'm more than willing to do
the coding (i needed an excuse to learn yet another language)
For lists, if the list doesn't exist, don't give a failure page, but give
the password page, and then always fail, giving no clue if the list name
or the password is the problem.
For users, just ALWAYS produce the user options page, and then do a
password fail if they try to submit anything.
This can produce more work for the mailman admin, as the legit users are
less sure about why an action is not working.
Comments? Suggestions? Pointers to python docs ;)
thanks,
Britt
Head Admin, Harvard Computer Society, and majordomo flunkie
-----------------------------------------------------------------------
Britt Bolen britt@bolen.com britt.bolen.com
From ricardo@rixhq.nu Sat Apr 15 12:09:22 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sat, 15 Apr 2000 13:09:22 +0200
Subject: [Mailman-Developers] idea
Message-ID: <20000415130922.C1143@miss-janet.com>
Hi,
I was just thinking what could be a usefull feature in
admindb.py for approved posts :
When a post is held for approval, convert the email-address
of the poster to a hyperlink to the options page for that
user ("/mailman/options/listname/email--at--host.com")
so if you see a message with "please remove me from this list"
or "why haven't i been getting any more mail lately?" you can
just click on the address and immediately edit the options
for that user...
ps: i'm still voting for having the message headers seperated
from the body on the approval page... i keep having to put back
my own hacks after upgrading to a new beta ;)
Ricardo.
--
From david_caldwell@indigita.com Tue Apr 18 21:55:58 2000
From: david_caldwell@indigita.com (David Caldwell)
Date: Tue, 18 Apr 2000 13:55:58 -0700
Subject: [Mailman-Developers] Fix for bug 170, is it any good?
Message-ID:
Hello, Is this the right list to send this to?
In ListAdmin.py this line (line 167 in 1.1, line 200 in 2.0beta2):
self.LogMsg("vette", note)
seems to be causing Mailman trouble. Above that you create 'note' and
use strquote to remove percents from variables you catenate to it.
But you don't use strquote on everything. In 2.0beta2 you added
strquote to the subject line, but the sender still doesn't have it
(unless I missed something). Instead of trying to catch all the
percents in the strings you catenate, could you just change the line
to this:
self.LogMsg("vette", '%s', note)
and get rid of the whole problem in one fell swoop? It seems more
straight forward to me. But then I don't know python that well. I
picked up enough to try to fix this problem I'm having where a % in
the senders address on a moderated list bombs the list software. My
change seems to work, but I'm not sure if there are any ramifications.
Take care,
David
From pfaff@edge.cis.mcmaster.ca Tue Apr 18 22:45:02 2000
From: pfaff@edge.cis.mcmaster.ca (Todd Pfaff)
Date: Tue, 18 Apr 2000 17:45:02 -0400 (EDT)
Subject: [Mailman-Developers] Fix for bug 170, is it any good?
In-Reply-To:
Message-ID:
i think i ran into a similar problem with LogMsg() and my workaround was
to modify LogMsg to handle exceptions so that i didn't have to change code
anywhere else. it's not a perfect solution, but at least it stops LogMsg
from complaining.
try this modified version of LogMsg from mailman-1.1/MailMan/MailList.py:
def LogMsg(self, kind, msg, *args):
"""Append a message to the log file for messages of specified kind."""
# For want of a better fallback, we use sys.stderr if we can't get
# a log file. We need a better way to warn of failed log access...
if self._log_files.has_key(kind):
logf = self._log_files[kind]
else:
logf = self._log_files[kind] = StampedLogger(kind)
try:
logf.write(msg % args + '\n')
except:
logf.write(msg + '\n')
logf.flush()
On Tue, 18 Apr 2000, David Caldwell wrote:
> In ListAdmin.py this line (line 167 in 1.1, line 200 in 2.0beta2):
>
> self.LogMsg("vette", note)
>
> seems to be causing Mailman trouble. Above that you create 'note' and
> use strquote to remove percents from variables you catenate to it.
> But you don't use strquote on everything. In 2.0beta2 you added
> strquote to the subject line, but the sender still doesn't have it
> (unless I missed something). Instead of trying to catch all the
> percents in the strings you catenate, could you just change the line
> to this:
>
> self.LogMsg("vette", '%s', note)
>
> and get rid of the whole problem in one fell swoop? It seems more
> straight forward to me. But then I don't know python that well. I
> picked up enough to try to fix this problem I'm having where a % in
> the senders address on a moderated list bombs the list software. My
> change seems to work, but I'm not sure if there are any ramifications.
--
Todd Pfaff \ Email: pfaff@mcmaster.ca
Computing and Information Services \ Voice: (905) 525-9140 x22920
ABB 132 \ FAX: (905) 528-3773
McMaster University \
Hamilton, Ontario, Canada L8S 4M1 \
From jwt@dskk.co.jp Wed Apr 19 01:48:38 2000
From: jwt@dskk.co.jp (Jim Tittsler)
Date: Wed, 19 Apr 2000 09:48:38 +0900
Subject: [Mailman-Developers] Change to posters? [was: Change to acceptable_aliases]
In-Reply-To: ; from Harald.Meland@usit.uio.no on Wed, Apr 12, 2000 at 03:40:56PM +0200
References: <14579.29747.148446.59654@anthem.cnri.reston.va.us>
Message-ID: <20000419094838.E19638@mail.dskk.co.jp>
Might it also be worthwhile to change the "posters" value to a regexp
instead of a list of posting addresses? This topic has come up a couple
of times on mailman-users lately... and it sounds worthwhile provided you
can trust administrators to write "reasonable" regexps.
--
Jim Tittsler, Tokyo
Python Starship http://starship.python.net/crew/jwt/
From ricardo@rixhq.nu Wed Apr 19 20:55:11 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Wed, 19 Apr 2000 21:55:11 +0200
Subject: [Mailman-Developers] idea
In-Reply-To: <20000415130922.C1143@miss-janet.com>; from ricardo@rixhq.nu on Sat, Apr 15, 2000 at 01:09:22PM +0200
References: <20000415130922.C1143@miss-janet.com>
Message-ID: <20000419215511.B26055@miss-janet.com>
On Sat, Apr 15, 2000 at 01:09:22PM +0200, Ricardo Kustner wrote:
> I was just thinking what could be a usefull feature in
> admindb.py for approved posts :
> When a post is held for approval, convert the email-address
> of the poster to a hyperlink to the options page for that
> user ("/mailman/options/listname/email--at--host.com")
In response to my own post :)
Mailman/Cgi/admindb.py line #215
t.AddRow([Bold('From:'), '%s' % (mlist.GetRelativeScriptURL('options'), Utils.ObscureEmail(sender), sender)])
there should be a cleaner way to code this, but for now it makes my daily work with the
posts on our moderated list a lot easier...
Ricardo.
--
From Dan.Mick@west.sun.com Wed Apr 19 21:45:37 2000
From: Dan.Mick@west.sun.com (Dan Mick)
Date: Wed, 19 Apr 2000 13:45:37 -0700
Subject: [Mailman-Developers] idea
References: <20000415130922.C1143@miss-janet.com> <20000419215511.B26055@miss-janet.com>
Message-ID: <38FE1AF1.4B156911@west.sun.com>
Ricardo Kustner wrote:
> On Sat, Apr 15, 2000 at 01:09:22PM +0200, Ricardo Kustner wrote:
> > I was just thinking what could be a usefull feature in
> > admindb.py for approved posts :
> > When a post is held for approval, convert the email-address
> > of the poster to a hyperlink to the options page for that
> > user ("/mailman/options/listname/email--at--host.com")
>
> In response to my own post :)
>
> Mailman/Cgi/admindb.py line #215
> t.AddRow([Bold('From:'), '%s' % (mlist.GetRelativeScriptURL('options'), Utils.ObscureEmail(sender), sender)])
>
> there should be a cleaner way to code this, but for now it makes my daily work with the
> posts on our moderated list a lot easier...
What do you use this for? Punitive unsubscription?
From ricardo@rixhq.nu Wed Apr 19 22:06:53 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Wed, 19 Apr 2000 23:06:53 +0200
Subject: [Mailman-Developers] idea
In-Reply-To: <38FE1AF1.4B156911@west.sun.com>; from Dan.Mick@west.sun.com on Wed, Apr 19, 2000 at 01:45:37PM -0700
References: <20000415130922.C1143@miss-janet.com> <20000419215511.B26055@miss-janet.com> <38FE1AF1.4B156911@west.sun.com>
Message-ID: <20000419230653.D26055@miss-janet.com>
On Wed, Apr 19, 2000 at 01:45:37PM -0700, Dan Mick wrote:
> > > admindb.py for approved posts :
> > > When a post is held for approval, convert the email-address
> > > of the poster to a hyperlink to the options page for that
> > > user ("/mailman/options/listname/email--at--host.com")
> > Mailman/Cgi/admindb.py line #215
> > t.AddRow([Bold('From:'), '%s' % (mlist.GetRelativeScriptURL('options'), Utils.ObscureEmail(sender), sender)])
> > there should be a cleaner way to code this, but for now it makes my daily work with the
> > posts on our moderated list a lot easier...
> What do you use this for? Punitive unsubscription?
i actually had to look up the word 'punitive' in an English-Dutch dicitionary :)
anyway, no I don't use it for punitive unsubscription... but often people post message
like "please unsubscribe me" or "how come I don't receive any more mail from the list?"
(the latter happens when they had a full mailbox and mailman automatically disabled
their account).... though it's possible to write a message with explanation, in most
cases (especially considering the lack of computer knowledge of most of the
subscribers) this only confuses them more... so I usually want to fix it myself.
Ricardo.
--
From Dan Mick Thu Apr 20 00:26:50 2000
From: Dan Mick (Dan Mick)
Date: Wed, 19 Apr 2000 16:26:50 -0700 (PDT)
Subject: [Mailman-Developers] idea
Message-ID: <200004192327.QAA18608@utopia.west.sun.com>
> On Wed, Apr 19, 2000 at 01:45:37PM -0700, Dan Mick wrote:
> > > > admindb.py for approved posts :
> > > > When a post is held for approval, convert the email-address
> > > > of the poster to a hyperlink to the options page for that
> > > > user ("/mailman/options/listname/email--at--host.com")
> > > Mailman/Cgi/admindb.py line #215
> > > t.AddRow([Bold('From:'), '%s' %
(mlist.GetRelativeScriptURL('options'), Utils.ObscureEmail(sender), sender)])
> > > there should be a cleaner way to code this, but for now it makes my daily
work with the
> > > posts on our moderated list a lot easier...
> > What do you use this for? Punitive unsubscription?
>
> i actually had to look up the word 'punitive' in an English-Dutch dicitionary
:)
oops, sorry!..
> anyway, no I don't use it for punitive unsubscription... but often people post
message
> like "please unsubscribe me" or "how come I don't receive any more mail from
the list?"
> (the latter happens when they had a full mailbox and mailman automatically
disabled
> their account).... though it's possible to write a message with explanation,
in most
> cases (especially considering the lack of computer knowledge of most of the
> subscribers) this only confuses them more... so I usually want to fix it
myself.
Oh, okay, that makes perfect sense. (most of my admin tasks are rejecting
HTML, attachments, and messages Cc'ed to 1000 AOLers...I get very
few sub/unsub requests. Mostly those are email to the listmaster
directly.)
From Nigel.Metheringham@VData.co.uk Thu Apr 20 09:46:51 2000
From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham)
Date: Thu, 20 Apr 2000 09:46:51 +0100
Subject: [Mailman-Developers] Messages in a time warp
Message-ID:
Having been a little fooled by the top and bottom few entries in the
mailman-users archives:-
http://www.python.org/pipermail/mailman-users/
Should mailman have a check for misdated messages - ie ones where the
date is more than a couple of days off the receive date? These could
then be moderated like other errors.
Nigel.
--
[ - Opinions expressed are personal and may not be shared by VData - ]
[ Nigel Metheringham Nigel.Metheringham@VData.co.uk ]
[ Phone: +44 1423 850000 Fax +44 1423 858866 ]
From ckolar@admin.aurora.edu Thu Apr 20 17:42:19 2000
From: ckolar@admin.aurora.edu (Christopher Kolar)
Date: Thu, 20 Apr 2000 11:42:19 -0500
Subject: [Mailman-Developers] feature reminder: no unsubscribe
Message-ID: <4.3.1.2.20000420113848.00b4d100@admin.aurora.edu>
I am setting up some more course mail lists and I remembered to check on
our testbed box for a feature that a bunch of people requested in the past:
that a mail list have a setting that subscribers cannot remove themselves
without admin approval. This was sought for situations (like a class)
where you don't want people to be able to opt out.
I did not find this in 2b1, is it implemented? Maybe just as something to
drop into the config file instead of through the web? If not, is there a
chance that it can get into the 2.0 code? Thanks much.
--chris
--
/////\\\\\/////\\\\\
Christopher G. Kolar
Director, Department of Instructional Technology
Aurora University, Aurora, Illinois
ckolar@admin.aurora.edu -- www.aurora.edu/~ckolar
[PGP Public Key ID: 0xC6492C72]
From thomas@xs4all.net Thu Apr 20 22:06:14 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Thu, 20 Apr 2000 23:06:14 +0200
Subject: [Mailman-Developers] feature reminder: no unsubscribe
In-Reply-To: <4.3.1.2.20000420113848.00b4d100@admin.aurora.edu>; from ckolar@admin.aurora.edu on Thu, Apr 20, 2000 at 11:42:19AM -0500
References: <4.3.1.2.20000420113848.00b4d100@admin.aurora.edu>
Message-ID: <20000420230614.A11481@xs4all.nl>
On Thu, Apr 20, 2000 at 11:42:19AM -0500, Christopher Kolar wrote:
> I am setting up some more course mail lists and I remembered to check on
> our testbed box for a feature that a bunch of people requested in the past:
> that a mail list have a setting that subscribers cannot remove themselves
> without admin approval. This was sought for situations (like a class)
> where you don't want people to be able to opt out.
> I did not find this in 2b1, is it implemented? Maybe just as something to
> drop into the config file instead of through the web? If not, is there a
> chance that it can get into the 2.0 code? Thanks much.
I posted a patch to the CVS tree that adds exactly that feature, a couple of
months ago. I think I'm the only one that uses it, though ;) (And the
version i posted had a couple of bugs, too.) I can extract a new patch from
my devel tree, if you are interested, but i doubt it'll be added to mailman
2.0 (no new features in betas ;)
I've been using it without much trouble, though it generates some confusion
with the subscribers, occasionally.
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From claw@cp.net Fri Apr 21 00:12:20 2000
From: claw@cp.net (J C Lawrence)
Date: Thu, 20 Apr 2000 16:12:20 -0700
Subject: [Mailman-Developers] Messages in a time warp
In-Reply-To: Message from Nigel Metheringham
of "Thu, 20 Apr 2000 09:46:51 BST."
References:
Message-ID:
On Thu, 20 Apr 2000 09:46:51 +0100
Nigel Metheringham wrote:
> Should mailman have a check for misdated messages - ie ones where
> the date is more than a couple of days off the receive date?
> These could then be moderated like other errors.
I would like this.
--
J C Lawrence Internet: claw@kanga.nu
----------(*) Internet: coder@kanga.nu
...Honorary Member of Clan McFud -- Teamer's Avenging Monolith...
From bwarsaw@cnri.reston.va.us Fri Apr 21 03:37:14 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Thu, 20 Apr 2000 22:37:14 -0400 (EDT)
Subject: [Mailman-Developers] feature reminder: no unsubscribe
References: <4.3.1.2.20000420113848.00b4d100@admin.aurora.edu>
<20000420230614.A11481@xs4all.nl>
Message-ID: <14591.48858.774943.921646@anthem.cnri.reston.va.us>
>>>>> "TW" == Thomas Wouters writes:
TW> I posted a patch to the CVS tree that adds exactly that
TW> feature, a couple of months ago. I think I'm the only one that
TW> uses it, though ;) (And the version i posted had a couple of
TW> bugs, too.) I can extract a new patch from my devel tree, if
TW> you are interested, but i doubt it'll be added to mailman 2.0
TW> (no new features in betas ;)
Would someone volunteer to put together a list of unofficial patches
and hacks? I don't currently have the time, unfortunately. We could
probably host them at SourceForge (if I can figure out how to change
http://mailman.sourceforge.net :)
If yo're interested and have the time, email me.
-Barry
From ricardo@rixhq.nu Fri Apr 21 08:53:30 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Fri, 21 Apr 2000 09:53:30 +0200
Subject: [Mailman-Developers] feature reminder: no unsubscribe
In-Reply-To: <14591.48858.774943.921646@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Thu, Apr 20, 2000 at 10:37:14PM -0400
References: <4.3.1.2.20000420113848.00b4d100@admin.aurora.edu> <20000420230614.A11481@xs4all.nl> <14591.48858.774943.921646@anthem.cnri.reston.va.us>
Message-ID: <20000421095330.B1910@miss-janet.com>
On Thu, Apr 20, 2000 at 10:37:14PM -0400, Barry A. Warsaw wrote:
> TW> I posted a patch to the CVS tree that adds exactly that
> TW> feature, a couple of months ago. I think I'm the only one that
> TW> uses it, though ;) (And the version i posted had a couple of
> TW> bugs, too.) I can extract a new patch from my devel tree, if
> TW> you are interested, but i doubt it'll be added to mailman 2.0
> TW> (no new features in betas ;)
> Would someone volunteer to put together a list of unofficial patches
> and hacks? I don't currently have the time, unfortunately. We could
> probably host them at SourceForge (if I can figure out how to change
> http://mailman.sourceforge.net :)
is that a job-opening for a mailman sourceforge webmaster? ;)
anyway, I was thinking that since you can use things like php on
sourceforge, we could create a simple system where you can submit
a patch with comments and then have everybody vote for it... cause
I have a lot of patches/hacks/ideas I can offer, but I know not all of them
could be interesting for everbody...
Ricardo.
--
From thomas@xs4all.net Fri Apr 21 09:02:04 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Fri, 21 Apr 2000 10:02:04 +0200
Subject: [Mailman-Developers] feature reminder: no unsubscribe
In-Reply-To: <14591.48858.774943.921646@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Thu, Apr 20, 2000 at 10:37:14PM -0400
References: <4.3.1.2.20000420113848.00b4d100@admin.aurora.edu> <20000420230614.A11481@xs4all.nl> <14591.48858.774943.921646@anthem.cnri.reston.va.us>
Message-ID: <20000421100203.B11481@xs4all.nl>
On Thu, Apr 20, 2000 at 10:37:14PM -0400, Barry A. Warsaw wrote:
> Would someone volunteer to put together a list of unofficial patches
> and hacks? I don't currently have the time, unfortunately. We could
> probably host them at SourceForge (if I can figure out how to change
> http://mailman.sourceforge.net :)
> If yo're interested and have the time, email me.
Unless the number of unofficial patches suddenly booms, i'd have the time,
and i'm certainly interested ;) I also have currently 2, later today 3
'unofficial' patches lying around, plus some minor fixes that haven't made
it into 2.0 yet, so there's definately an element of self-interest here ;)
(The patches are:
- unsubscriptions (optionally) requiring approval like subscriptions
- using a local file for poster_list (the list of email addresses that can post)
instead of having to enter them by hand / using the 'file upload' option. I
need this currently because we have several lists that are restricted to
exactly the same subset of addresses, and i dont want to maintain a remote
file and upload it to all lists every time it changes, mostly because
there's 10 people 'in charge' of the list, and our subscribees can create
their own email aliasses and *frequently* change their posting alias :P
(I'm actually waiting for the userdb stuff to go in, to add 'aliases' to the
options page, so people can go and maintain it themselves. Probably with a
change to the admin pages too, so that you can approve a held posting and
immediately add the email address to a known subscriber ;)
- And later today, after I test it some more on an actual mailman
installation, a new LockFile.py, that's more civil and more efficient with
locking.
The minor fixes are mailman eating the 'From ' line of messages that went
through approval (which means they dont show up as seperate messages, in the
archive), and the 'From ' lines in the 'downloadable archives' having the
wrong format, rendering the archives all but useless. (i have a python
script somewhere that fixes old archives too ;)
PS: Barry, i couldn't find a good excuse to post something to the
python-list, to see if the timeout problem pops up again, but i'm hoping
this list exhibits the same behaviour ;-P
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From klm@digicool.com Fri Apr 21 19:01:33 2000
From: klm@digicool.com (Ken Manheimer)
Date: Fri, 21 Apr 2000 14:01:33 -0400 (EDT)
Subject: [Mailman-Developers] Re: feature reminder: no unsubscribe (Christopher Kolar)
In-Reply-To: <20000421160002.5CDF01CD79@dinsdale.python.org>
Message-ID:
On Fri, 21 Apr 2000 Christopher Kolar wrote:
> I am setting up some more course mail lists and I remembered to check on
> our testbed box for a feature that a bunch of people requested in the past:
> that a mail list have a setting that subscribers cannot remove themselves
> without admin approval. This was sought for situations (like a class)
> where you don't want people to be able to opt out.
>
> I did not find this in 2b1, is it implemented? Maybe just as something to
> drop into the config file instead of through the web? If not, is there a
> chance that it can get into the 2.0 code? Thanks much.
My apologies if this has already been hashed out to everyone's
satisfaction, but i would be nervous about how this particular feature
would be subject to abuse.
I suppose, on the one hand, that spammers and the like could just use
something else - bare mailling list aliases, for that matter - to get
their captive audiences. On the other hand, i wouldn't want mailman to be
the tool of choice for capturing audiences. Might notifications of
deenlistment, with the ease of forcing reenlistment, be sufficient?
Not adamant, but wary,
Ken Manheimer
klm@digicool.com
From thomas@xs4all.net Fri Apr 21 20:09:02 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Fri, 21 Apr 2000 21:09:02 +0200
Subject: [Mailman-Developers] Re: feature reminder: no unsubscribe (Christopher Kolar)
In-Reply-To: ; from klm@digicool.com on Fri, Apr 21, 2000 at 02:01:33PM -0400
References: <20000421160002.5CDF01CD79@dinsdale.python.org>
Message-ID: <20000421210902.D11481@xs4all.nl>
On Fri, Apr 21, 2000 at 02:01:33PM -0400, Ken Manheimer wrote:
[ restricted unsubscribes ]
> My apologies if this has already been hashed out to everyone's
> satisfaction, but i would be nervous about how this particular feature
> would be subject to abuse.
> I suppose, on the one hand, that spammers and the like could just use
> something else - bare mailling list aliases, for that matter - to get
> their captive audiences. On the other hand, i wouldn't want mailman to be
> the tool of choice for capturing audiences. Might notifications of
> deenlistment, with the ease of forcing reenlistment, be sufficient?
Not in my case, and probably not in most cases that warrant restricted
unsubscriptions. I thought about this for a bit too, back when I wrote it (i
need it for myself in any case, but I was wondering wether to allow it for
other list admins)
Either these 'spammers' run mailman themselves, or they run the list on
someone elses' mailman install. Wether or not to allow restricted
unsubscribing would be a global mailman setting, like 'open subscribe'
currently -- the mailman admin has to explicitly enable it. If the mailman
administrator does enable it, it's his responsibility to make sure it
doesn't get abused. (When complaints come in, and the list admins refuse to
handle these properly, simply turn the option off, or turn the list off, or
whatever.)
If these spammers themselves run mailman, they can first of all add it
themselves, and they would probably have to hack mailman anyway, to be
usuable for spamming: my patch does nothing about the options page, and it
is still possible to 'mute' the list, or turn on digest, and that kind of
thing. (And I have no intention of disabling that. people sometimes do wish
to turn off the list, temporarily, when they go on vacation, for instance.)
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From Dan Mick Sat Apr 22 01:45:36 2000
From: Dan Mick (Dan Mick)
Date: Fri, 21 Apr 2000 17:45:36 -0700 (PDT)
Subject: [Mailman-Developers] 2.0beta3
Message-ID: <200004220045.RAA06896@utopia.west.sun.com>
Any estimate of release date? I'll probably be setting up another
server this weekend, and I'd like to do it with the latest stuff,
but slightly less volatile than CVS if possible.
Also, Barry or whoever, I think you should include Bernhard
Reiter's patch for the missing From line in the archives
ASAP; that's a high-visibility bug that I'm glad I haven't
installed 2.0beta yet to experience.
From thomas@xs4all.net Sat Apr 22 11:24:10 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sat, 22 Apr 2000 12:24:10 +0200
Subject: [Mailman-Developers] LockFile.py
Message-ID: <20000422122409.E11481@xs4all.nl>
--GvXjxJ+pjyke8COw
Content-Type: text/plain; charset=us-ascii
A couple of months ago I posted about a few bugs in the Mailman locking
mechanism, most notably that it was using the link() lock method the wrong
way 'round ;) but also some symtomps of this problem: failed locks, broken
locks, very long pauses between locks and unlocks, and a generally high load
on the machine while multiple mailmans were trying to get the lock. I could
trigger this behaviour by sending 10+ messages at the same time to the same
list.
I wrote a slightly better version, and Harald came up and showed me his ;)
Harald's LockFile.py (from his CVS tree) had the link() call the right way
'round, but had some other strange quirks that I disagreed with. So I kept
my version around, and rewrote it some more, using ideas from Haralds'
version, to make it more robust. The result is attached ;)
The 'textbook' example of NFS-compatible locking uses the link() call to
create a (hard) link between the lockfile and a private file (private to the
locking process) and determines wether or not the lock succeeded by
stat()ing the private file and checking that the number of links to the file
is 2. It's done this way because a lot of operations are not atomic, on NFS
filesystems, but the link()/stat() method is (or is supposed to be.)
The current mailman implementation is reversed, somehow. It makes a link
from the lockfile to a private file, and checks the number of links to the
lockfile. If the number of links is not 2, it assumes the lock failed, and
tries to read the contents. If the contents are invalid, or timed out (and
the pid hasn't changed since last time it was checked) the lock gets broken.
If a lock succeeds, the locker writes it's pid, it's private lockfile and
the release time into the file.
The problem is that if many mailmans are locking at the same time, the time
between the link() call, the stat() call and the unlink() call, can easily
be enough for several other processes to execute the link(), thus ending
with a lot of links. Worse, all locking processes use the exact same
sleep_interval, so this condition is likely to continue. Also, there is
enough delay between locking the file and filling it with data, for another
process to discover the file is empty, and remove it. Lastly, the cycle of
lock & test is fairly intensive, and the sleep time not high enough to give
other processes the chance to relinquish the lock.
Though I describe it in nasty terms, it isn't readily apparent in normal
situations :-) I have to stress the list quite a bit to detect these issues,
but the machine I run mailman on can be pretty loaded, and i did see these
effects in the occasional burst of natural traffic.
Harald's LockFile, from his CVS tree, is a lot better -- the locking works
fine, but there are other issues that are not that pretty: refresh() is
tricky, can lose the lock in fact. The same goes for steal(). And the
__try_steal() method is especially nasty: it rename()s its private lockfile
to the lockfile, overwriting any lockfile there, waits for 3*sleep_interval,
and checks to see if anyone else overwrote the lock.
My LockFile jumps through some hoops to do some things as safely as
possible, and it still fails a bit in some places, but it looks a lot safer
than the other two lockfiles. It's also a bit more load-friendly, in that it
only examines the lockfile every 10th (or thereabouts) iteration. This
should be more than enough for normal purposes, and it allows the lock-loop
to be as light as possible, allowing more time for the actual locking
process (if any) to finish its work and unlock ;) And the time the lock()
process sleeps is slightly randomized, to prevent perfectly in-stride
behaviour. (+/- .32 seconds)
I kept the old 'contents of lockfile' trick in place, eventhough I'm not
sure if it's really that useful: the whole problem with locking over NFS is
that you *can't* rely on the contents of a file. So I'm not convinced the
current behaviour (remove the lock if the contents is invalid) is the
correct one. I see how the contents are convenient, especially the timeout
time, but I'm thinking that when a file is empty or invalid, the timeout
should be taken as the MTIME of the file + the default locktime. This should
catch most stale files right away, and prevent a lot of race conditions in
locking/refreshing/stealing/breaking locks. I haven't implemented that yet,
though.
Attached is LockFile.py. I'm using it in production currently, and it seems
to work fine. I dont have code that actually uses steal(), but i tested it
by hand, and it seems to work ok too. There is only one issue with steal:
should it return NotLockedError some such when stealing fails, or return 0,
or keep on trying ?
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
--GvXjxJ+pjyke8COw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="LockFile.py"
# Copyright (C) 1998,1999,2000 by the Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""Portable (?) file locking with timeouts.
This code should work reliably with all versions of NFS. Based on the
original LockFile.py, of which the algorithm was supposedly suggested by the
GNU/Linux open() man page, and Harald Meland's LockFile.py, of which the
algorithm was based on Exim's locking style. The algorithm used here is
originally taken from qmail.
Make sure no malicious people have access to link() to the lock file. """
import socket, os, time
import string
import errno
from stat import ST_NLINK, ST_INO
# default intervals are both specified in seconds, and can be floating point
# values. DEFAULT_HUNG_TIMEOUT specifies the default length of time that a
# lock is expecting to hold the lock -- this can be set in the constructor,
# or changed via a mutator. DEFAULT_SLEEP_INTERVAL is the average amount of
# time to sleep before checking the lock's status, if we were not the
# winning claimant the previous time around. The sleep interval is modified
# based on the process-id to prevent unwanted stride effects.
DEFAULT_LOCK_LIFETIME = 15
DEFAULT_SLEEP_INTERVAL = 0.75
from Mailman.Logging.StampedLogger import StampedLogger
_logfile = None
# exceptions which can be raised
class LockError(Exception):
"""Base class for all exceptions in this module."""
pass
class AlreadyLockedError(LockError):
"""Raised when a lock is attempted on an already locked object."""
pass
class NotLockedError(LockError):
"""Raised when an unlock is attempted on an objec that isn't locked."""
pass
class TimeOutError(LockError):
"""Raised when a lock was attempted, but did not succeed in the given
amount of time.
"""
pass
class StaleLockFileError(LockError):
"""Raised when a stale hardlink lock file was found."""
pass
class LockFile:
"""A portable way to lock resources by way of the file system."""
def __init__(self, lockfile,
lifetime=DEFAULT_LOCK_LIFETIME,
sleep_interval=DEFAULT_SLEEP_INTERVAL,
withlogging=0):
"""Creates a lock file using the specified file.
lifetime is the maximum length of time expected to keep this lock.
This value is written into the lock file so that other claimants on
the lock know when it is safe to steal the lock, should the lock
holder be wedged.
sleep_interval is how often to wake up and check the lock file
"""
self.__lockfile = lockfile
self.__lifetime = lifetime
self.__sleep_interval = sleep_interval + (os.getpid()%32 - 16) / 100.
self.__tmpfname = "%s.%s.%d" % (lockfile,
socket.gethostname(),
os.getpid())
self.__withlogging = withlogging
self.__logprefix = os.path.split(self.__lockfile)[1]
self.__writelog("sleep_interval = %f"%self.__sleep_interval)
def set_lifetime(self, lifetime):
"""Reset the lifetime of the lock.
Takes affect the next time the file is locked.
"""
self.__lifetime = lifetime
def __writelog(self, msg):
global _logfile
if self.__withlogging:
if not _logfile:
_logfile = StampedLogger('locks', manual_reprime=0)
_logfile.write('%s %s\n' % (self.__logprefix, msg))
def refresh(self, newlifetime=None):
"""Refresh the lock.
This writes a new release time into the lock file. Use this if a
process suddenly realizes it needs more time to do its work. With
optional newlifetime, this resets the lock lifetime value too.
NotLockedError is raised if we don't already own the lock.
"""
if not self.locked():
raise NotLockedError
if newlifetime is not None:
self.set_lifetime(newlifetime)
self.__refresh()
def __refresh(self):
"""Internal function to refresh the lock. Does not check for lock.
Only call while holding the lock!"""
tmpfname = self.__tmpfname + ".tmp"
tmplock = self.__lockfile + ".tmp"
self.__write(tmpfname)
if os.path.exists(tmplock):
os.unlink(tmplock)
os.link(tmpfname, tmplock)
os.rename(tmplock, self.__lockfile)
os.rename(tmpfname, self.__tmpfname)
self.__writelog('lock lifetime refreshed for %d seconds' %
self.__lifetime)
def __del__(self):
if self.locked():
self.unlock()
def __write(self, filename = None):
# we expect to release our lock some time in the future. we want to
# give other claimants some clue as to when we think we're going to be
# done with it, so they don't try to steal it out from underneath us
# unless we've actually been wedged.
lockrelease = time.time() + self.__lifetime
# make sure it's group writable
oldmask = os.umask(002)
if not filename:
filename = self.__tmpfname
try:
fp = open(filename, 'w')
fp.write('%d %s %f\n' % (os.getpid(),
self.__tmpfname,
lockrelease))
fp.close()
finally:
os.umask(oldmask)
def __read(self):
# can raise ValueError in two situations:
#
# either first element wasn't an integer (a valid pid), or we didn't
# get a sequence of the right size from the string.split. Either way,
# the data in the file is bogus, but this is caught and handled higher
# up.
#
# NOTE: Reads the real lockfile now, not the symlink.
try:
fp = open(self.__lockfile, 'r')
pid, winner, lockrelease = string.split(string.strip(fp.read()))
except IOError, err:
if err.errno == errno.ENOENT:
raise ValueError
# IOerror, but not 'ENOENT'. 'permission denied' ?
raise
return int(pid), winner, float(lockrelease)
def __break(self):
# We apparently have had enough of it, we wish to break the lock.
try:
winner = None
try:
pid, winner, lockrelease = self.__read()
except ValueError:
pass
os.unlink(self.__lockfile)
if winner:
os.unlink(winner)
except OSError, err:
if err.errno <> errno.ENOENT:
raise
# Note that no one new can grab the lock once we've opened our tmpfile
# until we close it, even if we don't have the lock. So checking the PID
# and stealing the lock are guaranteed to be atomic.
def lock(self, timeout=0):
"""Blocks until the lock can be obtained.
Raises a TimeOutError exception if a positive timeout value is given
and that time elapses before the lock is obtained.
This can possibly steal the lock from some other claimant, if the lock
lifetime that was written to the file has been exceeded. Note that
for this to work across machines, the clocks must be sufficiently
synchronized.
"""
if self.locked():
self.__writelog('already locked')
raise AlreadyLockedError
if timeout:
timeout_time = time.time() + timeout
last_pid = -1
# loopcount is used to determine wether we should check the lockfile,
# after a lock fails. We start with -1, so the lockfile gets checked
# the first time through the loop.
loopcount = -1
self.__write() # Make sure my lockfile exists, and time is up to date
while 1:
loopcount = loopcount + 1
# create the hard link and test for exactly 2 links to the file
try:
os.link(self.__tmpfname, self.__lockfile)
except OSError, err:
if err.errno == errno.ENOENT:
# Our own lockfile vanished ?!
# Re-create it, and try again. Sleep anyway, so we dont
# unintendedly hog the machine.
self.__writelog("own lockfile missing.")
self.__write()
self.__sleep()
continue
elif err.errno == errno.EEXIST:
# We failed to get the lock.
# We want to test the contents of the lock, see if it's
# bogus or timed out, but we dont want to do that each
# time through.
if loopcount%10:
self.__writelog("lock failed; skipping checks.")
self.__sleep()
continue
else:
try:
os.unlink(self.__tmpfname)
except:
pass
raise OSError, err
if os.stat(self.__tmpfname)[ST_NLINK] == 2:
# we have the lock (since there are no other links to the lock
# file), so we can rehydrate our marker
self.__writelog("got the lock")
self.__refresh()
if not self.locked():
self.__writelog("false positive on lock")
continue
break
# we didn't get the lock this time. let's see if we timed out
if timeout and timeout_time < time.time():
os.unlink(self.__tmpfname)
self.__writelog('timed out')
raise TimeOutError
# someone else must have gotten the lock. let's find out who it
# is. if there is some bogosity in the lock file's data then we
# will break the lock.
try:
pid, winner, lockrelease = self.__read()
except ValueError:
self.__writelog("breaking lock (fawlty lockfile)")
self.__break()
self.__sleep()
continue
# here's where we potentially break the lock. If the lock has
# timed out (the time written in by the original locker has come
# and gone) then we let __break() deal with it.
if lockrelease < time.time():
self.__writelog("breaking lock (too old)")
self.__break()
self.__sleep()
continue
# okay, someone else has the lock, we didn't steal it, and our
# claim hasn't timed out yet. So let's wait a while for the owner
# of the lock to give it up.
self.__writelog('waiting for claim')
self.__sleep()
def __sleep(self):
time.sleep(self.__sleep_interval)
# This could error if the lock is stolen. You must catch it.
def unlock(self):
"""Unlock the lock.
If we don't already own the lock (either because of unbalanced unlock
calls, or because the lock was stolen out from under us), raise a
NotLockedError.
"""
if not self.locked():
raise NotLockedError
try:
os.unlink(self.__tmpfname)
os.unlink(self.__lockfile)
except OSError, err:
if err.errno <> errno.ENOENT:
raise
self.__writelog('unlocked')
def locked(self):
"""Returns 1 if we own the lock, 0 if we do not."""
if not os.path.exists(self.__tmpfname):
return 0
if not os.path.exists(self.__lockfile):
return 0
if os.stat(self.__tmpfname)[ST_NLINK] < 2:
return 0
# The above check for ST_NLINK ought to be enough for everyone.
# Regardless, we're very paranoid; we check the lockfile data.
# If you are optimising for speed, you can 'return 1' here
try:
pid, winner, lockrelease = self.__read()
except ValueError:
# Something strange is going on. Either our own lockfile is corrupt
# or someone link()ed to our lockfile, making us believe we have
# the lock.
if os.stat(self.__tmpfname)[ST_INO] <> os.stat(self.__lockfile)[ST_INO]:
# We do not really have the lock. unlink so we get a new inode
os.unlink(self.__tmpfname)
self.__writelog("linkcount was 2, but we dont have the lock!")
return 0
else:
# It really is our lock, but the contents are corrupted.
self.__refresh()
self.__writelog("linkcount was 2, but our lockfile data was odd!")
return 1
if pid <> os.getpid():
self.__writelog("lock seemed ok, but data wasn't our data.")
return 0
return 1
# use with caution!!!
def steal(self):
"""Explicitly break the lock. USE WITH CAUTION!"""
self.__writelog("stealing the lock...")
# We assume someone else sucessfully has the lock, so we try and
# steal it from them. There is a race-condition involved here:
# multiple processes can be attempting to steal the lock. Hence the
# sleep and check if we are actually locked.
# XXX Should NotLockedError be raised instead of returning 0 ?
try:
pid, winner, timeout = self.__read()
except ValueError:
winner = None
if winner and os.path.exists(winner):
# Someone has the lock. We steal their private lockfile, both
# to clean up after us, and to steal the lock as smoothlessly as
# possible. Only another process attempting a steal() or break()
# could interfere, which is why we sleep after stealing.
os.rename(winner, self.__tmpfname)
self.__refresh()
else:
# The file seemed unlocked, so we bluntly lock it. Not really
# proper, as we do not check if someone else had just grabbed the
# lock.
self.__write()
os.link(self.__tmpfname, self.__lockfile + ".tmp")
os.rename(self.__lockfile + ".tmp", self.__lockfile)
self.__sleep()
if self.locked():
self.__writelog("succesfully stolen!")
return 1
else:
self.__writelog("failed to steal the lock.")
return 0
--GvXjxJ+pjyke8COw--
From ricardo@rixhq.nu Sat Apr 22 17:58:29 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sat, 22 Apr 2000 18:58:29 +0200
Subject: [Mailman-Developers] LockFile.py
In-Reply-To: <20000422122409.E11481@xs4all.nl>; from thomas@xs4all.net on Sat, Apr 22, 2000 at 12:24:10PM +0200
References: <20000422122409.E11481@xs4all.nl>
Message-ID: <20000422185828.A5148@miss-janet.com>
On Sat, Apr 22, 2000 at 12:24:10PM +0200, Thomas Wouters wrote:
> Attached is LockFile.py. I'm using it in production currently, and it seems
> to work fine. I dont have code that actually uses steal(), but i tested it
I'll use your Lockfile.py in my own production environment too (remember,
the not-so-powerfull server) and I'll let you know how it works...
Ricardo.
From otaylor@redhat.com Sun Apr 23 23:50:23 2000
From: otaylor@redhat.com (Owen Taylor)
Date: 23 Apr 2000 18:50:23 -0400
Subject: [Mailman-Developers] Making passwords easier for users
Message-ID:
We recently switched over the GNOME mailing lists to
Mailman, and it was a quite easy transition. So far,
everything is working nicely.
The most common problem problem that people seem to have
is password management. Quite a few people try to
unsubscribe without a password, and when they get an
error message back, resort to mailing the mail admin
address and asking the owner to do it.
And in fact, none of the mail help text makes any reference
about how one can find out ones password. I could
add a pointer to the how to do it via the web in the
help text, but it seems simpler to simply mail out
a password reminder when a users command fails
because they
a) didn't include a password
b) included the wrong password
(The password could even be put directly into the response
if the address that the request comes from is the same
as the address the command pertained to.)
However, since this idea seems pretty obvious to me, and
yet it isn't already, I thought I would ask if there
is any reason I'm missing why it should not be done,
before I went off and did it.
The other password related modification I was thinking of doing
locally here is a little bit more radical - making it so that all
passwords for a given email address are interchangable. Quite a few
people are subscribed to 10-15 different gnome.org mailing lists, and
when they were moved over, they were assigned a different password for
each list.
(If I had planned it better, I would have written a custom
add_members script so that people, at least initially,
would have the same password for each list.)
Of course, IMO, the best thing would be to have just a single
password per user, but making the passwords for each list
interchangeable is a good step in the right direction.
Again, I'd like opinions about whether this is just a bad
idea or not.
Regards,
Owen
From jhebert@compu-aid.com Mon Apr 24 01:12:06 2000
From: jhebert@compu-aid.com (Jim Hebert)
Date: Sun, 23 Apr 2000 20:12:06 -0400 (EDT)
Subject: [Mailman-Developers] Making passwords easier for users
In-Reply-To:
Message-ID:
On 23 Apr 2000, Owen Taylor wrote:
> The other password related modification I was thinking of doing
> locally here is a little bit more radical - making it so that all
> passwords for a given email address are interchangable. Quite a few
[I am not a mailman developer. If I shouldn't be posting my .02, someone
please thwap me with the clue paddle...]
This change has the effect of reducing the strength of the passwords: if I
am on 15 lists with 15 different passwords, a dictionary attack against
any of them is 15 times more likely to succeed and brings me 15 times more
access for having broken it.
OTOH, if you keep all your list passwords the same, the success
probability is unchanged versus one list membership, but the latter
observation that you get 15x the access is still true, so it's somewhat of
a red herring.
Also, I don't know how much of a threat scenario this is, but if I can
subscribe otaylor@redhat.com to some other list on the machine with a
password of my choosing, I have the equivalent access of having otaylor's
actual password(s) for the other lists. There are at least some sites
where there isn't mutual trust among the list-owners.
That said, admittedly, the security of ones list subscriptions aren't
exactly the crown jewels. And people probably aren't exactly
seeing massive dictionary attacks against their mailman
installations... If this was a configurable thing for the paranoid (me?)
defaulting to the current behavior I guess it couldn't hurt, eh?
jim
king-for-a-day of esoterica?
--
Jim Hebert http://www.cosource.com/
jim@cosource.com The cooperative market for open source software
From otaylor@redhat.com Mon Apr 24 05:09:44 2000
From: otaylor@redhat.com (Owen Taylor)
Date: 24 Apr 2000 00:09:44 -0400
Subject: [Mailman-Developers] Making passwords easier for users
In-Reply-To: Jim Hebert's message of "Sun, 23 Apr 2000 20:12:06 -0400 (EDT)"
References:
Message-ID:
Jim Hebert writes:
> On 23 Apr 2000, Owen Taylor wrote:
>
> > The other password related modification I was thinking of doing
> > locally here is a little bit more radical - making it so that all
> > passwords for a given email address are interchangable. Quite a few
>
> [I am not a mailman developer. If I shouldn't be posting my .02, someone
> please thwap me with the clue paddle...]
Its fine with me. Of course, I'm not a mailman developer either. ;-)
> This change has the effect of reducing the strength of the passwords: if I
> am on 15 lists with 15 different passwords, a dictionary attack against
> any of them is 15 times more likely to succeed and brings me 15 times more
> access for having broken it.
>
> OTOH, if you keep all your list passwords the same, the success
> probability is unchanged versus one list membership, but the latter
> observation that you get 15x the access is still true, so it's somewhat of
> a red herring.
Exactly. It significantly weakens the strength if the user is using
the randomly chosen Mailman passwords, but not otherwise. Unfortunately,
the passwords Mailman chooses are weak enough that this might
actually matter.
(4 randomly chosen upper or lower case letters - so the mean time for
brute forcing one password is about 3.5 million tries. Weakening this
by a factor of 10 would make it conceivable that someone could try
it. On the other hand, if someone posts the form 350,000 times in a
row, it probably will create a lot of other problems for a server.)
> Also, I don't know how much of a threat scenario this is, but if I can
> subscribe otaylor@redhat.com to some other list on the machine with a
> password of my choosing, I have the equivalent access of having otaylor's
> actual password(s) for the other lists. There are at least some sites
> where there isn't mutual trust among the list-owners.
Well, if you have list owners who are being actively malicious to that
extent... but yes, I could see where some people might consider it a
problem.
> That said, admittedly, the security of ones list subscriptions aren't
> exactly the crown jewels. And people probably aren't exactly
> seeing massive dictionary attacks against their mailman
> installations... If this was a configurable thing for the paranoid (me?)
> defaulting to the current behavior I guess it couldn't hurt, eh?
My basic feeling about this change is that it is pretty hacky,
and does weaken security a little bit. But many mailing lists
get along without any password protection system at all ... and
the tradeoff for users is:
- the inconvenience of someone, just possibly, doing something
bad to their list subscriptions.
- the inconvenience of juggling a big pile of passwords.
So, I'll probably try implementing sometime when I have some time
(hah!) and see how it works out. I wouldn't suggest this as the
default, certainly. Though I would hope that people are thinking about
how Mailman could be migrated to one-password-for-user.
Regards,
Owen
From Harald.Meland@usit.uio.no Mon Apr 24 12:59:24 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 24 Apr 2000 13:59:24 +0200
Subject: [Mailman-Developers] Change to acceptable_aliases
In-Reply-To: Christopher Lindsey's message of "Fri, 14 Apr 2000 13:31:07 -0500 (CDT)"
References: <200004141831.NAA09979@glorfindel.ncsa.uiuc.edu>
Message-ID:
[Christopher Lindsey]
> About a year ago I sent out a proposal to the list about this that I
> think might still have some merit. Rather than doing regular
> expression matching, why not do regex plus substitution?
I think this would complicate the interface quite a lot, while still
not buying us much more functionality.
If we're going to complicate this interface further, why not go
straight for some full-fledged ACL-like stuff?
--
Harald
From cklempay@chimera.acm.jhu.edu Mon Apr 24 16:39:14 2000
From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay)
Date: Mon, 24 Apr 2000 11:39:14 -0400 (EDT)
Subject: [Mailman-Developers] b2 archiving off?
Message-ID:
Any suggestions on what I should look at for this: apparently all of our
lists that have archiving on and are private have not had the archives
updated since February. (?) I just upgraded to 2.0b2 2-3 weeks ago.
---
Corbett J. Klempay
Trilogy Software, Inc.
512.532.5176 (W) | 512.750.1372 (C)
corbett.klempay@trilogy.com
From brian@gweep.bc.ca Mon Apr 24 19:34:29 2000
From: brian@gweep.bc.ca (Brian Edmonds)
Date: 24 Apr 2000 11:34:29 -0700
Subject: [Mailman-Developers] features I would like to see
In-Reply-To: "Corbett J. Klempay"'s message of "Mon, 24 Apr 2000 11:39:14 -0400 (EDT)"
Message-ID: <37k8hn1h6y.fsf@lios.gweep.bc.ca>
I'm a longtime Majordomo admin, and just installed Mailman. It looks
good enough that I'm in the process of migrating most/all of my lists to
it, but there are still some features I'd like to see:
1. In addition to digesting by size and daily, digesting by delay. I've
hacked my Majordomo install to send daily digests, but only if more
than 24 hours had elapsed since the first post that would be in the
digest. This way busy lists that digest through the day based on
size never get a stubby daily digest (or no digest for long periods
if they suddenly go low traffic and are on size digesting only). Low
volume lists typically digest every other day. The delay time should
be configurable on a per-list basis, of course.
2. Taboo body matches. The taboo headers are nice, but I also like to
check if the list's .sig is included in the message as a heuristic
for overquoting.
3. When handling non-member submission bounces for subscribers sending
from alternate addresses it would be very nice to have a checkbox
that in addition to approving the message will subscribe the address
with the nomail option set.
It would also be nice if the subscription could somehow include a
comment that this is an alternate address, and even better if an
unsubscribe of the primary address would unsub the alternates. And
an appropriate comment in the monthly password reminder for records
of this sort (or perhaps not mailing them for alternates, just
mentioning the alternate addresses in the main mailing).
Adding alternate addresses to subscriber records would be an even
better way to implement the feature, but may be harder, and the
automated nomail subscription would be nice in the near term if
alternate addresses would be a long-term feature.
I'm sure there will be more, but those are the main ones that have come
up so far.
Brian.
From thomas@xs4all.net Mon Apr 24 22:29:48 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 24 Apr 2000 23:29:48 +0200
Subject: [Mailman-Developers] Making passwords easier for users
In-Reply-To: ; from otaylor@redhat.com on Mon, Apr 24, 2000 at 12:09:44AM -0400
References:
Message-ID: <20000424232947.F11481@xs4all.nl>
On Mon, Apr 24, 2000 at 12:09:44AM -0400, Owen Taylor wrote:
> Jim Hebert writes:
> > On 23 Apr 2000, Owen Taylor wrote:
> > > The other password related modification I was thinking of doing
> > > locally here is a little bit more radical - making it so that all
> > > passwords for a given email address are interchangable. Quite a few
> > [I am not a mailman developer. If I shouldn't be posting my .02, someone
> > please thwap me with the clue paddle...]
I'm not a Mailman developper either, but I'll give my fl0,02 as well ;)
[ snip ideas about passwords in Mailman, and it's security implications ]
> (4 randomly chosen upper or lower case letters - so the mean time for
> brute forcing one password is about 3.5 million tries. Weakening this
> by a factor of 10 would make it conceivable that someone could try
> it. On the other hand, if someone posts the form 350,000 times in a
> row, it probably will create a lot of other problems for a server.)
But, guys, it's a lot worse than that. Haven't you noticed Mailman can send
out password *reminders* every month ? It's no use to send out encrypted
passwords, so mailmans passwords *aren't* encrypted. They're just stored in
plaintext. If someone gets hold of the list databases, they *have* the
passwords, no efforts attached ;)
(This only goes for member passwords. The site password and the list
passwords are encrypted, either with crypt.crypt() or with md5.digest().)
The problem with the password ideas isn't security, because Mailman
passwords aren't that big a deal -- they're mostly protection against
malicious scriptkiddies trying to subscribe/unsubscribe someone else. The
problem is that there is no seperate user-database. (yet.) All users are
part of a list, and there is no easy way to see what lists, for instance, a
user is subscribed to (even if you were able to figure out which email
addresses are equal ;)
If I'm not mistaken, this is going to be remedied, though. Harald (i think?)
wrote a 'user database' that's supposed to be incorporated into standard
Mailman after the 2.0 release (or whenever it's ready.) Once it's there, all
kinds of kinky things like listing ones' aliases, choosing which list goes
to which alias, etc, can be written. And trust me, it'll be written, because
you're not the only ones crying out for it. :-)
*cry* *cry*-ly y'rs,
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From ckolar@admin.aurora.edu Tue Apr 25 15:22:02 2000
From: ckolar@admin.aurora.edu (Christopher Kolar)
Date: Tue, 25 Apr 2000 09:22:02 -0500
Subject: [Mailman-Developers] Fwd: Bug in Mailman version 2.0beta2
Message-ID: <4.3.1.2.20000425092154.00b37bb0@admin.aurora.edu>
>Date: Sat, 22 Apr 2000 08:43:23 -0300
>From: Daniel Neto
>X-Mailer: The Bat! (v1.41) Educational
>Reply-To: Daniel Neto
>Organization: Terra Networks - Vitoria (ES)
>To: ckolar@admin.aurora.edu
>Subject: Bug in Mailman version 2.0beta2
>X-Status:
>
>Hi!,
>
>Bug in Mailman version 2.0beta2
>
>We're sorry, we hit a bug!
>
>If you would like to help us identify the problem, please email a copy
>of this page to the webmaster for this site with a description
>of what happened. Thanks!
>
>Traceback:
>
>Traceback (innermost last):
> File "/home/mailman/scripts/driver", line 89, in run_main
> main()
> File "../Mailman/Cgi/admin.py", line 68, in main
> mlist =3D MailList.MailList(listname)
> File "/home/mailman/Mailman/MailList.py", line 69, in __init__
> self.Load()
> File "/home/mailman/Mailman/MailList.py", line 881, in Load
> self.CheckVersion(dict)
> File "/home/mailman/Mailman/MailList.py", line 912, in CheckVersion
> Update(self, stored_state)
> File "../Mailman/versions.py", line 52, in Update
> NewRequestsDatabase(l)
> File "../Mailman/versions.py", line 217, in NewRequestsDatabase
> r =3D getattr(l, 'requests', {})
>TypeError: getattr requires exactly 2 arguments; 3 given
>
>
>
>
>
>Python information:
>
> Variable Value
> sys.version 1.5.1 (#1, Sep 3 1998, 22:51:17) [GCC 2.7.2.3]
> sys.executable /usr/bin/python
> sys.prefix /usr
> sys.exec_prefix /usr
> sys.path /usr
> sys.platform linux-i386
>
>Environment variables:
>
> Variable Value
> DOCUMENT_ROOT /usr/local/etc/httpd/htdocs/
> SERVER_ADDR 200.241.14.12
> HTTP_ACCEPT_ENCODING gzip
> REMOTE_HOST operacao01.vix.zaz.com.br
> SERVER_PORT 80
> PATH_TRANSLATED /usr/local/etc/httpd/htdocs/operacao
> REMOTE_ADDR 200.241.14.132
> SERVER_SOFTWARE Apache/1.3.11 (Unix) PHP/3.0.14
> GATEWAY_INTERFACE CGI/1.1
> HTTP_ACCEPT_LANGUAGE en
> REMOTE_PORT 2086
> SERVER_NAME listas.vix.zaz.com.br
> HTTP_CONNECTION Keep-Alive
> HTTP_USER_AGENT Mozilla/4.72 [en] (Win98; I)
> HTTP_ACCEPT_CHARSET iso-8859-1,*,utf-8
> HTTP_ACCEPT image/gif, image/x-xbitmap, image/jpeg,=20
> image/pjpeg, image/png, */*
> REQUEST_URI /mailman/admin
> PATH /sbin:/usr/sbin:/bin:/usr/bin
> QUERY_STRING
> SERVER_PROTOCOL HTTP/1.0
> PATH_INFO /
> HTTP_HOST 200.241.14.12
> REQUEST_METHOD GET
> SERVER_SIGNATURE Apache/1.3.11 Server at listas.vix.zaz.com.br Port=
80
> SCRIPT_NAME /mailman/admin
> SERVER_ADMIN sysadm@vix.zaz.com.br
> SCRIPT_FILENAME /home/mailman/cgi-bin/admin
> PYTHONPATH /home/mailman
>
>=3D=3D=3D
>Thanks in advance,
>
> Daniel - dneto@zaz.com.br
> SysOp / Network Administrator
> Terra Networks of Brazil - Vit=F3ria/ES
> +55-27-334-1475 / 334-1476 / 315-2650
>
From thomas@xs4all.net Tue Apr 25 17:07:01 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Tue, 25 Apr 2000 18:07:01 +0200
Subject: [Mailman-Developers] Fwd: Bug in Mailman version 2.0beta2
In-Reply-To: <4.3.1.2.20000425092154.00b37bb0@admin.aurora.edu>; from ckolar@admin.aurora.edu on Tue, Apr 25, 2000 at 09:22:02AM -0500
References: <4.3.1.2.20000425092154.00b37bb0@admin.aurora.edu>
Message-ID: <20000425180701.G11481@xs4all.nl>
On Tue, Apr 25, 2000 at 09:22:02AM -0500, Christopher Kolar wrote:
> >Bug in Mailman version 2.0beta2
[ snip ]
> > r = getattr(l, 'requests', {})
> >TypeError: getattr requires exactly 2 arguments; 3 given
The three-arg-getattr is part of Python 1.5.2, and this host is running:
> >Python information:
> > sys.version 1.5.1 (#1, Sep 3 1998, 22:51:17) [GCC 2.7.2.3]
You need to upgrade to 1.5.2 (this goes for all of the 2.0 Mailman
distribution, you should upgrade to python 1.5.2. It's been around long
enough, it's rock-steady, and fixes a sizable amounts of bugs too !)
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From brian@gweep.bc.ca Tue Apr 25 18:13:42 2000
From: brian@gweep.bc.ca (Brian Edmonds)
Date: 25 Apr 2000 10:13:42 -0700
Subject: [Mailman-Developers] features I would like to see
In-Reply-To: Brian Edmonds's message of "24 Apr 2000 11:34:29 -0700"
References: <37k8hn1h6y.fsf@lios.gweep.bc.ca>
Message-ID: <37em7uxfw9.fsf@lios.gweep.bc.ca>
Right, more features I'd like. :)
1. A bounced status, in addition to nomail. On my Majordomo install I
have a list called bounces, and a script to automagically unsub
people from a list and resub them to bounces. Every address on the
bounces list gets a daily reminder that they have been unsubscribed
due to mail bouncing.
In addition we would then need a new cron job to mail all addresses
set to bounced status with a notice, and to automatically dun out any
addresses that have remained as bounced for a period that is
configurable on a per-list basis. Then it could also be added as an
option to the auto bounce handling to have addresses switched to
bounced instead of just disabled.
2. I've also modified my bounce script to by default switch users from
the main list to the digest first, if one exists. This would also be
a nice option for bouncing mail, so that users with shorter term
problems, such as quota, could get for example five days grace, then
bounced to the digest for a further five days, then set to bounced
state for a week then unsubscribed. Moving them to digest state can
keep them on the list but reduce server load for busy lists.
3. When handling submissions that are held for admin approval it would
be handy to have another option to MIME encapsulate the message and
mail it to the list admin. That way I could make simple formatting
changes or add admin comments to the occasional message and remail it
myself instead of simply having to reject it.
4. Are the web pages setting no-cache appropriately? It could just be a
bad interaction with my proxy or Netscape cache settings, but the
admin pages are being cached which means I have to hit reload quite
often to make sure I'm getting the latest list settings, or I end up
submitting old data. Having the archives be cacheable makes sense,
and people can reload as needed, but the admin and subscriber config
pages should definitely be no-cache.
I hope this sparks some discussion, and some creative juices in the
developers. :) I'd start submitting patches of my own, but I'm a perl
programmer, not python, so I've got a bit of a learning curve to get
over first.
Brian.
From marc_news@valinux.com Tue Apr 25 19:14:10 2000
From: marc_news@valinux.com (Marc Merlin)
Date: Tue, 25 Apr 2000 11:14:10 -0700
Subject: [Mailman-Developers] A few mailman issues
In-Reply-To: <390545E2.A27D806@uma.es>; from jcrey@uma.es on mar, avr 25, 2000 at 09:14:42 +0200
References: <048d01bfa61e$d41f1620$3e04030a@gibralter.com> <39042368.5420E119@uma.es> <04de01bfade8$e58b6900$3e04030a@gibralter.com> <390545E2.A27D806@uma.es>
Message-ID: <20000425111410.A16237@marc.merlins.org>
1) arch eats up all my memory and hangs when processing a message that has a
12MB attachment (my machine has 1G of memory)
2) When mailman creates an archive mbox file, it puts a empty line at the
beginning. This is annoying as it prevents me from doing
mutt -f list.mbox
3) It would be really cool if the admin web interface would let me click on
a link to edit the held message (headers and body) before I approve it
(in a textarea)
Thanks,
Marc
--
Microsoft is to software what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ (friendly to non IE browsers)
Finger marc_f@merlins.org for PGP key and other contact information
From Dan.Mick@west.sun.com Tue Apr 25 19:42:54 2000
From: Dan.Mick@west.sun.com (Dan Mick)
Date: Tue, 25 Apr 2000 11:42:54 -0700
Subject: [Mailman-Developers] 2.0beta3
References: <200004220045.RAA06896@utopia.west.sun.com>
Message-ID: <3905E72E.DAB5F287@west.sun.com>
Any response, albeit tiny?
Dan Mick wrote:
> Any estimate of release date? I'll probably be setting up another
> server this weekend, and I'd like to do it with the latest stuff,
> but slightly less volatile than CVS if possible.
>
> Also, Barry or whoever, I think you should include Bernhard
> Reiter's patch for the missing From line in the archives
> ASAP; that's a high-visibility bug that I'm glad I haven't
> installed 2.0beta yet to experience.
>
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers@python.org
> http://www.python.org/mailman/listinfo/mailman-developers
From thomas@xs4all.net Tue Apr 25 21:28:48 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Tue, 25 Apr 2000 22:28:48 +0200
Subject: [Mailman-Developers] A few mailman issues
In-Reply-To: <20000425111410.A16237@marc.merlins.org>; from marc_news@valinux.com on Tue, Apr 25, 2000 at 11:14:10AM -0700
References: <048d01bfa61e$d41f1620$3e04030a@gibralter.com> <39042368.5420E119@uma.es> <04de01bfade8$e58b6900$3e04030a@gibralter.com> <390545E2.A27D806@uma.es> <20000425111410.A16237@marc.merlins.org>
Message-ID: <20000425222848.H11481@xs4all.nl>
On Tue, Apr 25, 2000 at 11:14:10AM -0700, Marc Merlin wrote:
> 1) arch eats up all my memory and hangs when processing a message that has a
> 12MB attachment (my machine has 1G of memory)
Howmany messages are in the archive mailbox ? What kind of machine is it ?
I've recently arch'ed an old mail archive into a new list, where the old
archive (my own private archive of the list, which has existed for over 5
years) contained close to 9000 messages. No overly large attachements,
though, but the machine none the less handled it pretty well.
arch *does* load the entire mailbox into memory, though. How large is the
mailbox in total ?
[ Have no real answer to the other questions, so sorry for not answering ]
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From Harald.Meland@usit.uio.no Tue Apr 25 22:16:15 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 25 Apr 2000 23:16:15 +0200
Subject: [Mailman-Developers] Change to acceptable_aliases
In-Reply-To: "Barry A. Warsaw"'s message of "Mon, 24 Apr 2000 11:02:57 -0400 (EDT)"
References: <200004141831.NAA09979@glorfindel.ncsa.uiuc.edu> <14596.25121.182422.646750@anthem.cnri.reston.va.us>
Message-ID:
[Barry A. Warsaw]
> >>>>> "HM" == Harald Meland writes:
>
> HM> If we're going to complicate this interface further, why not
> HM> go straight for some full-fledged ACL-like stuff?
>
> What do you have in mind?
I think this has been discussed before, but put on back burner due at
least partly because editing anything ACL-like with a web (non-Java
etc.) interface seems cumbersome.
I haven't had any brilliant ideas as to how that problem should be
solved, but nevertheless I'm more and more inclined to implement
something that would allow (clueful) admins to do things like:
allow header_From matching @\S*\buio.no
deny has_header X-RBL-Warning
deny size > 1MB
hold size > 40k
allow header_From in members
hold * # default
As usual, the ACL is composed of rules, each consisting of an "action"
and a "condition". The condition must be satisfied for the
corresponding action to take place. The ACL is searched top-to-bottom
for the first rule whose condition is true. If such a rule is found,
the corresponding action is executed, and the search stops.
Otherwise, the default action (which should be to "hold" the message,
I guess) is executed.
This would allow the filtering functionality to be configured in a
very generic way with a single config option, albeit a rather complex
one...
[ The above ACL example is just something i jotted down off the top of
my head -- I haven't given much thought to the syntax I have used,
nor do I have any strong opinions about exactly which actions and
conditions should exist. ]
Of course, a big concern is whether implementing such a beast will
make "filtering configuration" of Mailman lists too strange (or even
obfuscated) for most list admins.
It might be possible to construct some kind of mapping between the
current config options and an ACL option, and make it possible for
"inexperienced" users to continue using an interface similar to the
one we have today (allowing Mailman to only consult the ACL whenever
there's a need to decide which action should be taken for some
message), but it sounds rather full of thorns...
--
Harald
From marc_news@valinux.com Tue Apr 25 22:37:29 2000
From: marc_news@valinux.com (Marc Merlin)
Date: Tue, 25 Apr 2000 14:37:29 -0700
Subject: [Mailman-Developers] A few mailman issues
In-Reply-To: <20000425222848.H11481@xs4all.nl>; from thomas@xs4all.net on mar, avr 25, 2000 at 10:28:48 +0200
References: <048d01bfa61e$d41f1620$3e04030a@gibralter.com> <39042368.5420E119@uma.es> <04de01bfade8$e58b6900$3e04030a@gibralter.com> <390545E2.A27D806@uma.es> <20000425111410.A16237@marc.merlins.org> <20000425222848.H11481@xs4all.nl>
Message-ID: <20000425143728.F16237@marc.merlins.org>
On mar, avr 25, 2000 at 10:28:48 +0200, Thomas Wouters wrote:
> On Tue, Apr 25, 2000 at 11:14:10AM -0700, Marc Merlin wrote:
> > 1) arch eats up all my memory and hangs when processing a message that has a
> > 12MB attachment (my machine has 1G of memory)
>
> Howmany messages are in the archive mailbox ? What kind of machine is it ?
2675 messages, not that many.
I removed 3 messages with large attachments, re-ran arch and it ran just
fine.
Before, when it reached the 12MB attachement, it ate more than 400MB
I am moving mailman to an up to date machine, but in the meantime:
oxygen:~# rpm -qf /usr/bin/python
python-1.5.1-5
Maybe a newer python will help.
> arch *does* load the entire mailbox into memory, though. How large is the
> mailbox in total ?
Not that big, 25MB.
> [ Have no real answer to the other questions, so sorry for not answering ]
Eh, one answer is already good :-)
Marc
--
Microsoft is to software what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ (friendly to non IE browsers)
Finger marc_f@merlins.org for PGP key and other contact information
From Murray.Jensen@cmst.csiro.au Wed Apr 26 02:32:12 2000
From: Murray.Jensen@cmst.csiro.au (Murray Jensen)
Date: Wed, 26 Apr 2000 11:32:12 +1000
Subject: [Mailman-Developers] features I would like to see
In-Reply-To: Message from Brian Edmonds
of "25 Apr 2000 10:13:42 MST." <37em7uxfw9.fsf@lios.gweep.bc.ca>
Message-ID: <21391.956712732@msa.cmst.csiro.au>
>Right, more features I'd like. :)
OK, since we're doing this :-) I would also like the following:
1. the built-in pipermail archive stuff to handle MIME multipart better.
e.g. in the web pages, store non-text MIME parts in separate files,
preferably in a sub-directory. These files can then be excluded from
any indexing for a search facility.
2. a nicer web interface to the web archive e.g. a frames based approach
so that you can keep the index in one frame and display the messages in
another (makes navigating the archive easier)
3. a built-in search function for the web archive
For an example of what I'd like, see:
http://www.msa.cmst.csiro.au/mailman-archives/robot-toolbox/
(hopefully I got the permissions correct - also, the neighbourhood stuff in
webglimpse doesn't work). To do this, I had to ditch the pipermail stuff and
use mhonarc, with a contributed mhonarc frames config. The webglimpse stuff
could be used with pipermail, but I don't like webglimpse's licencing model.
I think it is too restrictive for something that is relatively simple (and
could be implemented in pipermail fairly easily). Cheers!
Murray...
--
Murray Jensen, CSIRO Manufacturing Sci & Tech, Phone: +61 3 9662 7763
Locked Bag No. 9, Preston, Vic, 3072, Australia. Fax: +61 3 9662 7853
Internet: Murray.Jensen@cmst.csiro.au (old address was mjj@mlb.dmt.csiro.au)
From Murray.Jensen@cmst.csiro.au Wed Apr 26 03:32:36 2000
From: Murray.Jensen@cmst.csiro.au (Murray Jensen)
Date: Wed, 26 Apr 2000 12:32:36 +1000
Subject: [Mailman-Developers] features I would like to see
In-Reply-To: Message from Todd Pfaff
of "Tue, 25 Apr 2000 22:14:46 -0400."
Message-ID: <21587.956716356@msa.cmst.csiro.au>
>On Wed, 26 Apr 2000, Murray Jensen wrote:
>cool, i like that layout. could you send me your mhonarc frames config?
I got it from the mhonarc examples directory in the 2.4.5 distribution:
MHonArc2.4.5/examples/frames.mrc
>how about using wilma, glimpse, and mhonarc, instead of webglimpse?
>that is what i use. i sent some quick-start instructions to the
>mailman-users list recently.
Yes, I looked at wilma/glimpse, but webglimpse seemed easier (though I
didn't try wilma, just scanned the doc quickly). Unfortunately, glimpse
has the same restrictions as webglimpse - they come from the same people.
Also, glimpse is a more generic indexing tool - something simpler just
for these email-archive-on-the-web html files would do, plus it could be
made smarter - only indexing the subject/body and maybe ignoring quoted
text. The ideas behind glimpse could be used maybe. I don't really know,
just tossing about ideas. Cheers!
Murray...
PS: I hope you don't mind me Cc'ing this to the list. It may help others.
--
Murray Jensen, CSIRO Manufacturing Sci & Tech, Phone: +61 3 9662 7763
Locked Bag No. 9, Preston, Vic, 3072, Australia. Fax: +61 3 9662 7853
Internet: Murray.Jensen@cmst.csiro.au (old address was mjj@mlb.dmt.csiro.au)
From bwarsaw@cnri.reston.va.us Tue Apr 25 16:24:35 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Tue, 25 Apr 2000 11:24:35 -0400 (EDT)
Subject: [Mailman-Developers] Fwd: Bug in Mailman version 2.0beta2
References: <4.3.1.2.20000425092154.00b37bb0@admin.aurora.edu>
Message-ID: <14597.47283.815502.118863@anthem.cnri.reston.va.us>
>>>>> "CK" == Christopher Kolar writes:
>Bug in Mailman version 2.0beta2
------------------------^^^^^^^^
> sys.version 1.5.1 (#1, Sep 3 1998, 22:51:17) [GCC 2.7.2.3]
---------------------^^^^^
Sorry, Mailman 2.0 requires Python 1.5.2. It is not compatible with
Python 1.5.1. Please upgrade your Python installation.
-Barry
From jbore@tjtech.com Wed Apr 26 05:37:48 2000
From: jbore@tjtech.com (Joseph T. Bore)
Date: Wed, 26 Apr 2000 00:37:48 -0400 (EDT)
Subject: [Mailman-Developers] authentication failure revisited
Message-ID: <14598.29340.303825.1600@union.tjtech.com>
When ever I use my admin interface, I cant do anything because I'm
continuously re-routed back to the login screen. after that my
changes dont take.
looking around on the devlopers archive I noticed the thread about
"failed authentication." essentially from what I read, there was some
weird interacatio between php, apache, mailman, and cookies.
The solution apparently was to upgrade to more recent stuff.
I have upgraded my installation to the most recent redhat packages for
everything and still my interface doesnt seem to be getting the
cookies. a quick print of os.eviron_get("HTTP_COOKIE"), yields "None"
has anyone else run into this? I'm willing to help figure it out, but
I dont know why the cookie isnt getting set and I'm getting lost
quickly trying to follow it through all the code. (my first time
looking at the mailman source)
any hints?
jb
--
------------------+---------------------------------------------------------
Joseph T. Bore | When you have tremendous conviction on a trade, you have
jbore@TJTech.com | to go for the jugular. It takes courage to be a pig.
------------------+---------------------------------------------------------
From bwarsaw@cnri.reston.va.us Mon Apr 24 16:02:57 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 24 Apr 2000 11:02:57 -0400 (EDT)
Subject: [Mailman-Developers] Change to acceptable_aliases
References: <200004141831.NAA09979@glorfindel.ncsa.uiuc.edu>
Message-ID: <14596.25121.182422.646750@anthem.cnri.reston.va.us>
>>>>> "HM" == Harald Meland writes:
>> About a year ago I sent out a proposal to the list about this
>> that I think might still have some merit. Rather than doing
>> regular expression matching, why not do regex plus
>> substitution?
HM> I think this would complicate the interface quite a lot, while
HM> still not buying us much more functionality.
I have to agree with Harald here. I don't see a big enough win to
justify yet another configuration option. I'm already distressed by
the large number of options Mailman currently supports, and am
thinking about ways to make the interfaces simpler.
HM> If we're going to complicate this interface further, why not
HM> go straight for some full-fledged ACL-like stuff?
What do you have in mind?
-Barry
From bwarsaw@cnri.reston.va.us Tue Apr 25 22:03:47 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Tue, 25 Apr 2000 17:03:47 -0400 (EDT)
Subject: [Mailman-Developers] 2.0beta3
References: <200004220045.RAA06896@utopia.west.sun.com>
<3905E72E.DAB5F287@west.sun.com>
Message-ID: <14598.2099.762588.911037@anthem.cnri.reston.va.us>
>>>>> "DM" == Dan Mick writes:
DM> Any response, albeit tiny?
I'll have a better idea and more information about the next release in
a couple of days. Stay tuned.
-Barry
From bwarsaw@cnri.reston.va.us Tue Apr 25 23:58:57 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Tue, 25 Apr 2000 18:58:57 -0400 (EDT)
Subject: [Mailman-Developers] A few mailman issues
References: <048d01bfa61e$d41f1620$3e04030a@gibralter.com>
<39042368.5420E119@uma.es>
<04de01bfade8$e58b6900$3e04030a@gibralter.com>
<390545E2.A27D806@uma.es>
<20000425111410.A16237@marc.merlins.org>
<20000425222848.H11481@xs4all.nl>
<20000425143728.F16237@marc.merlins.org>
Message-ID: <14598.9009.989305.953122@anthem.cnri.reston.va.us>
>>>>> "MM" == Marc Merlin writes:
MM> 2675 messages, not that many. I removed 3 messages with large
MM> attachments, re-ran arch and it ran just fine. Before, when
MM> it reached the 12MB attachement, it ate more than 400MB
MM> I am moving mailman to an up to date machine, but in the
MM> meantime: oxygen:~# rpm -qf /usr/bin/python python-1.5.1-5
MM> Maybe a newer python will help.
Probably not.
Pipermail is currently the most neglected aspect of Mailman. I'd love
for somebody to adopt it and expend some serious energy on it. In the
meantime, for anybody doing serious archiving, including w/ lots of
attachments, you probably want to investigate an external archiver.
-Barry
From thomas@xs4all.net Wed Apr 26 06:43:35 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 26 Apr 2000 07:43:35 +0200
Subject: [Mailman-Developers] A few mailman issues
In-Reply-To: <14598.9009.989305.953122@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Tue, Apr 25, 2000 at 06:58:57PM -0400
References: <048d01bfa61e$d41f1620$3e04030a@gibralter.com> <39042368.5420E119@uma.es> <04de01bfade8$e58b6900$3e04030a@gibralter.com> <390545E2.A27D806@uma.es> <20000425111410.A16237@marc.merlins.org> <20000425222848.H11481@xs4all.nl> <20000425143728.F16237@marc.merlins.org> <14598.9009.989305.953122@anthem.cnri.reston.va.us>
Message-ID: <20000426074334.B379@xs4all.nl>
On Tue, Apr 25, 2000 at 06:58:57PM -0400, Barry A. Warsaw wrote:
> MM> Maybe a newer python will help.
> Probably not.
> Pipermail is currently the most neglected aspect of Mailman. I'd love
> for somebody to adopt it and expend some serious energy on it.
Actually, I'm more or less doing that. I'm currently grokking
pipermail/HyperArch, and will probably rewrite HyperArch in pipermails'
image, but without subclassing it (because HyperArch ends up masking most
pipermail things, and I dont want to rewrite them *both*) unless pipermail
is being actively maintained/developped outside of Mailman, or someone else
knows a better python-based archiver/thingy ?
Anyway, the attachements issue & the newline at the top of each list.mbox is
on my 'todo' list ;)
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From thomas@xs4all.net Wed Apr 26 06:54:20 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 26 Apr 2000 07:54:20 +0200
Subject: [Mailman-Developers] authentication failure revisited
In-Reply-To: <14598.29340.303825.1600@union.tjtech.com>; from jbore@tjtech.com on Wed, Apr 26, 2000 at 12:37:48AM -0400
References: <14598.29340.303825.1600@union.tjtech.com>
Message-ID: <20000426075420.I11481@xs4all.nl>
On Wed, Apr 26, 2000 at 12:37:48AM -0400, Joseph T. Bore wrote:
> I have upgraded my installation to the most recent redhat packages for
> everything and still my interface doesnt seem to be getting the
> cookies. a quick print of os.eviron_get("HTTP_COOKIE"), yields "None"
> any hints?
You tried upgrading your browser ? A coworker of mine had the same problem,
and it ended up being his Netscape version. But it can be a multitude of
things. Are you sure the cookie is being set in your browser ? Have you
tried setting the cookie setting to 'ask first', and seeing if you are
actualy sent one ? Is the hostname in the URL you use, the same one as what
mailman stores in the cookie ? Did you check if the URL you type is the same
as the DEFAULT_URL in Mailman/mm_cfg.py ?
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From dan@feld.cvut.cz Wed Apr 26 08:39:38 2000
From: dan@feld.cvut.cz (Dan Ohnesorg)
Date: Wed, 26 Apr 2000 09:39:38 +0200 (CEST)
Subject: [Mailman-Developers] A few mailman issues
In-Reply-To: <20000426074334.B379@xs4all.nl>
Message-ID:
On Wed, 26 Apr 2000, Thomas Wouters wrote:
> Actually, I'm more or less doing that. I'm currently grokking
> pipermail/HyperArch, and will probably rewrite HyperArch in pipermails'
> image, but without subclassing it (because HyperArch ends up masking most
> pipermail things, and I dont want to rewrite them *both*) unless pipermail
> is being actively maintained/developped outside of Mailman, or someone else
> knows a better python-based archiver/thingy ?
I think, the pipermail should be definietly removed from mailman. It is
mutch better to use some external archiver. You will have match more
stable mailman without pipermail.
When someone will corect my poor english, I can write mailman-archiving
howto for mailman-MHonArc cooperation.
cheers
dan
--
________________________________________
DDDDDD
DD DD Dan Ohnesorg, supervisor on POWER
DD OOOO Dan@feld.cvut.cz
DD OODDOO Dep. of Power Engineering
DDDDDD OO CTU FEL Prague, Bohemia
OO OO work: +420 2 24352785;+420 2 24972109
OOOO home: +420 311 679679;+420 311 679311
________________________________________
Nekdy je rozumne o krok ustoupit, aby se prodlouzil rozbeh.
From dan@feld.cvut.cz Wed Apr 26 09:02:30 2000
From: dan@feld.cvut.cz (Dan Ohnesorg)
Date: Wed, 26 Apr 2000 10:02:30 +0200 (CEST)
Subject: [Mailman-Developers] features I would like to see
In-Reply-To: <21587.956716356@msa.cmst.csiro.au>
Message-ID:
On Wed, 26 Apr 2000, Murray Jensen wrote:
> Yes, I looked at wilma/glimpse, but webglimpse seemed easier (though I
> didn't try wilma, just scanned the doc quickly). Unfortunately, glimpse
> has the same restrictions as webglimpse - they come from the same people.
> Also, glimpse is a more generic indexing tool - something simpler just
I have had big problem with glimpse on archiv with more than 64000
messages. Now I use htdig and I find it better. You should look on it.
http://www.htdig.org
chers
dan
--
________________________________________
DDDDDD
DD DD Dan Ohnesorg, supervisor on POWER
DD OOOO Dan@feld.cvut.cz
DD OODDOO Dep. of Power Engineering
DDDDDD OO CTU FEL Prague, Bohemia
OO OO work: +420 2 24352785;+420 2 24972109
OOOO home: +420 311 679679;+420 311 679311
________________________________________
Laska k funkci kvete v kazdem veku.
From lindsey@ncsa.uiuc.edu Wed Apr 26 18:01:13 2000
From: lindsey@ncsa.uiuc.edu (Christopher Lindsey)
Date: Wed, 26 Apr 2000 12:01:13 -0500 (CDT)
Subject: [Mailman-Developers] Change to acceptable_aliases
In-Reply-To: from "Harald Meland" at Apr 25, 2000 11:16:15 PM
Message-ID: <200004261701.MAA13280@glorfindel.ncsa.uiuc.edu>
> allow header_From matching @\S*\buio.no
> deny has_header X-RBL-Warning
> deny size > 1MB
> hold size > 40k
> allow header_From in members
> hold * # default
>
> Of course, a big concern is whether implementing such a beast will
> make "filtering configuration" of Mailman lists too strange (or even
> obfuscated) for most list admins.
Personally, I don't think this has a place in Mailman. Why reinvent
the wheel? You can easily wrap an alias with a procmail recipe
that natively supports this in a language designed to filter
mail. It's the UNIX way: awk | sed | sort -rn, not piping everything
into sort and expecting it to do it all... :) The exception, of
course, is the wonderfully compact emacs suite.
Of course, a wrapper is much better suited to denying than allowing
message through -- to make it work, one would have to allow everything
in Mailman and expect the wrappers to handle it all. However, you could
have the procmail/maildrop/script filter add an Approved: header if the
messsage is indeed valid, which Mailman would pick up on...
These aren't the best solutions, but I don't think that it's Mailman's
place to do this either.
Chris
P.S. I still don't see how ACLs would replace regular expressions with
substitution on a global scale, say to allow plus-style addressing.
From jbore@tjtech.com Wed Apr 26 20:25:30 2000
From: jbore@tjtech.com (Joseph T. Bore)
Date: Wed, 26 Apr 2000 15:25:30 -0400 (EDT)
Subject: [Mailman-Developers] authentication failure revisited
In-Reply-To: <20000426075420.I11481@xs4all.nl>
References: <14598.29340.303825.1600@union.tjtech.com>
<20000426075420.I11481@xs4all.nl>
Message-ID: <14599.17066.166125.979795@union.tjtech.com>
Thomas Wouters writes:
> On Wed, Apr 26, 2000 at 12:37:48AM -0400, Joseph T. Bore wrote:
>
> > I have upgraded my installation to the most recent redhat packages for
> > everything and still my interface doesnt seem to be getting the
> > cookies. a quick print of os.eviron_get("HTTP_COOKIE"), yields "None"
>
> > any hints?
>
> You tried upgrading your browser ? A coworker of mine had the same problem,
> and it ended up being his Netscape version. But it can be a multitude of
> things. Are you sure the cookie is being set in your browser ? Have you
> tried setting the cookie setting to 'ask first', and seeing if you are
> actualy sent one ?
> Is the hostname in the URL you use, the same one as what
> mailman stores in the cookie ? Did you check if the URL you type is the same
> as the DEFAULT_URL in Mailman/mm_cfg.py ?
ahh, that was it. I had a different value for the url. what should
it really be "www.foo.com/mailman" right.
jb
--
------------------+---------------------------------------------------------
Joseph T. Bore | When you have tremendous conviction on a trade, you have
jbore@TJTech.com | to go for the jugular. It takes courage to be a pig.
------------------+---------------------------------------------------------
From claw@cp.net Wed Apr 26 21:30:20 2000
From: claw@cp.net (J C Lawrence)
Date: Wed, 26 Apr 2000 13:30:20 -0700
Subject: [Mailman-Developers] A few mailman issues
In-Reply-To: Message from Dan Ohnesorg
of "Wed, 26 Apr 2000 09:39:38 +0200."
References:
Message-ID:
On Wed, 26 Apr 2000 09:39:38 +0200 (CEST)
Dan Ohnesorg wrote:
> I think, the pipermail should be definietly removed from
> mailman. It is mutch better to use some external archiver. You
> will have match more stable mailman without pipermail.
While I agree that Pipermail is feture poor and, umm, in general
sucks, there is also a definite value to having an archiver, even
such a poor one, be an implicit part of Mailman -- if only for new
list admins who instantly get something that is *usable* (if not
friendly) and thereby can instantly have their lists archived and
indexed without having to confront the complexities of MHonArc or
Hypermail.
I consider Pipermail to be one of the features of Mailman which make
it is easily recommendable to new list admins:
-- they get instant if minimal archiving functionality
-- by default all their list mail is preserved in a
portable/standardised format (mbox)
-- it is very easy to later put in a more full featured archiver
ala MHonArc/Hypermail.
-- If/when they do move to an external archiver, their old mail is
preserved in a form easy to run thru the new archiving system
to gen the new archives.
> When someone will corect my poor english, I can write
> mailman-archiving howto for mailman-MHonArc cooperation.
Sure.
--
J C Lawrence Internet: claw@kanga.nu
----------(*) Internet: coder@kanga.nu
...Honorary Member of Clan McFud -- Teamer's Avenging Monolith...
From claw@cp.net Wed Apr 26 21:32:30 2000
From: claw@cp.net (J C Lawrence)
Date: Wed, 26 Apr 2000 13:32:30 -0700
Subject: [Mailman-Developers] features I would like to see
In-Reply-To: Message from Dan Ohnesorg
of "Wed, 26 Apr 2000 10:02:30 +0200."
References:
Message-ID:
On Wed, 26 Apr 2000 10:02:30 +0200 (CEST)
Dan Ohnesorg wrote:
> I have had big problem with glimpse on archiv with more than 64000
> messages.
Glimpse does have publicly acknowledged scaling problems (tho I
found them lower than that).
> Now I use htdig and I find it better. You should look on it.
I've not been well pleased with HTdig, either on features or
performance. I'm becoming increasingly fond of UdmSearch tho.
--
J C Lawrence Internet: claw@kanga.nu
----------(*) Internet: coder@kanga.nu
...Honorary Member of Clan McFud -- Teamer's Avenging Monolith...
From claw@cp.net Wed Apr 26 21:39:34 2000
From: claw@cp.net (J C Lawrence)
Date: Wed, 26 Apr 2000 13:39:34 -0700
Subject: [Mailman-Developers] Change to acceptable_aliases
In-Reply-To: Message from Christopher Lindsey
of "Wed, 26 Apr 2000 12:01:13 CDT." <200004261701.MAA13280@glorfindel.ncsa.uiuc.edu>
References: <200004261701.MAA13280@glorfindel.ncsa.uiuc.edu>
Message-ID:
On Wed, 26 Apr 2000 12:01:13 -0500 (CDT)
Christopher Lindsey wrote:
>> allow header_From matching @\S*\buio.no deny has_header
>> X-RBL-Warning deny size > 1MB hold size > 40k allow header_From
>> in members hold * # default
>>
>> Of course, a big concern is whether implementing such a beast
>> will make "filtering configuration" of Mailman lists too strange
>> (or even obfuscated) for most list admins.
> Personally, I don't think this has a place in Mailman. Why
> reinvent the wheel? You can easily wrap an alias with a procmail
> recipe that natively supports this in a language designed to
> filter mail. It's the UNIX way: awk | sed | sort -rn, not piping
> everything into sort and expecting it to do it all... :) The
> exception, of course, is the wonderfully compact emacs suite.
Much tho I like ACLs in list servers (I had a rather alrge
collection of them when I ran petidomo), I largely agree.
> Of course, a wrapper is much better suited to denying than
> allowing message through -- to make it work, one would have to
> allow everything in Mailman and expect the wrappers to handle it
> all. However, you could have the procmail/maildrop/script filter
> add an Approved: header if the messsage is indeed valid, which
> Mailman would pick up on...
Adding a single feature to Mailman would probably do the trick for
everybody here:
A command line switch to the wrapper script such that the mail
arriving on stdin will be posted, no matter what any other
configuration settings specify. Perhaps "post-approved" in
addition to "post", "mailowner", "mailcmd" etc.
Do *that*, and you can pretty well do anything at all in a
procmail/maildrop/whatever pre-filter and get any desired behaviour
out of it.
--
J C Lawrence Internet: claw@kanga.nu
----------(*) Internet: coder@kanga.nu
...Honorary Member of Clan McFud -- Teamer's Avenging Monolith...
From roy.smith@med.nyu.edu Wed Apr 26 21:40:37 2000
From: roy.smith@med.nyu.edu (Roy Smith)
Date: Wed, 26 Apr 2000 16:40:37 -0400
Subject: [Mailman-Developers] A few mailman issues
In-Reply-To:
Message-ID: <20000426204032.5CF5F1CD2C@dinsdale.python.org>
J C Lawrence wrote:
> While I agree that Pipermail is feture poor and, umm, in general
> sucks, there is also a definite value to having an archiver, even
> such a poor one, be an implicit part of Mailman
I agree 100%. We run a bunch of mailing lists here. Time to invest in
managing those lists is extremely limited. We used to use
majordomo/hypermail. One of the best things in my mind about switching to
mailman is that now instead of administering two software packages, we can
adminster one. Maybe pipermail isn't the best archiver in the world, but
it's sure good enough for our purposes (which are pretty simplistic, but
then again, I suspect most people who want to archive mailing lists have
equally simplistic needs). To remove it from the mailman distribution
would be a big mistake.
Roy Smith
New York University School of Medicine
550 First Avenue, New York, NY 10016
From bwarsaw@cnri.reston.va.us Wed Apr 26 22:12:00 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Wed, 26 Apr 2000 17:12:00 -0400 (EDT)
Subject: [Mailman-Developers] A few mailman issues
References:
<20000426204032.5CF5F1CD2C@dinsdale.python.org>
Message-ID: <14599.23456.284392.388873@anthem.cnri.reston.va.us>
>>>>> "RS" == Roy Smith writes:
RS> To remove it from the mailman distribution would be a big
RS> mistake.
Agreed! It won't, and doesn't need to happen.
I wouldn't be opposed to a complete re-write of the bundled archiver,
but it has to be Python. That might be too much work, so some smaller
step improvements would definitely be accepted.
What would really help me would be for someone out there to claim
"ownership" of the archiver; basically vette patches other
improvements.
-Barry
From bwarsaw@cnri.reston.va.us Wed Apr 26 22:13:50 2000
From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us)
Date: Wed, 26 Apr 2000 17:13:50 -0400 (EDT)
Subject: [Mailman-Developers] A few mailman issues
References: <048d01bfa61e$d41f1620$3e04030a@gibralter.com>
<39042368.5420E119@uma.es>
<04de01bfade8$e58b6900$3e04030a@gibralter.com>
<390545E2.A27D806@uma.es>
<20000425111410.A16237@marc.merlins.org>
<20000425222848.H11481@xs4all.nl>
<20000425143728.F16237@marc.merlins.org>
<14598.9009.989305.953122@anthem.cnri.reston.va.us>
<20000426074334.B379@xs4all.nl>
Message-ID: <14599.23566.415175.459684@anthem.cnri.reston.va.us>
>>>>> "TW" == Thomas Wouters writes:
TW> Actually, I'm more or less doing that. I'm currently grokking
TW> pipermail/HyperArch, and will probably rewrite HyperArch in
TW> pipermails' image, but without subclassing it (because
TW> HyperArch ends up masking most pipermail things, and I dont
TW> want to rewrite them *both*) unless pipermail is being
TW> actively maintained/developped outside of Mailman, or someone
TW> else knows a better python-based archiver/thingy ?
I don't think anybody is actively maintaining Pipermail anymore, so
you essentially have free rein here. It has to be Python, and it
should integrate cleanly with the existing framework, but it would be
okay if it uses the external archiver interface.
TW> Anyway, the attachements issue & the newline at the top of
TW> each list.mbox is on my 'todo' list ;)
Excellent! Thanks.
-Barry
From edc@o3.proadmin.com Thu Apr 27 00:09:48 2000
From: edc@o3.proadmin.com (Eric D. Christensen)
Date: Wed, 26 Apr 2000 16:09:48 -0700 (PDT)
Subject: [Mailman-Developers] Beta2 - Bug in archiver?
Message-ID: <200004262309.QAA28845@o3.proadmin.com>
I've poked at this a little but I'm not making any headway. Not being very
python literate, it's probably something simple that I'm missing. Nontheless, I
figured I'd bounce it off the experts....
This is 2.0Beta2 running under Mandrake Linux on a PIII with sendmail 8.9.3.
Python version is: Python 1.5.2 (#1, Apr 25 2000, 12:39:13) [GCC pgcc-2.91.66 19990314 (egcs-1.1.2 on linux2
Seems to be a hole somewhere related to the date parsing code in the archiver.
In logs/error I get the following traceback whenever a message is posted to the list:
Apr 25 16:35:05 2000 post(28927): Traceback (innermost last):
post(28927): File "/home/mailman/Mailman/Archiver/Archiver.py", line 204, in ArchiveMail
post(28927): self.__archive_to_mbox(msg)
post(28927): File "/home/mailman/Mailman/Archiver/Archiver.py", line 160, in __archive_to_mbox
post(28927): post.SetHeader('Date', time.ctime(time.time()))
post(28927): AttributeError: SetHeader
The message gets forwarded to the list correctly. It just doesn't get archived.
For what it's worth.... the message looks like this when it comes through the
list. I don't see anything abviously broken in the headers:
From f500-admin@o6.proadmin.com Wed Apr 26 15:26:48 2000
Return-Path:
Received: from o6.proadmin.com (o6.proadmin.com [208.195.160.175])
by o3.proadmin.com (8.9.3/8.9.3/ProAdmin) with ESMTP id PAA28760
for ; Wed, 26 Apr 2000 15:26:48 -0700
Received: from o6.proadmin.com (localhost [127.0.0.1])
by o6.proadmin.com (8.9.3/8.9.3/ProAdmin) with ESMTP id PAA00608
for ; Wed, 26 Apr 2000 15:26:48 -0700
Received: from o3.proadmin.com ([199.108.70.172])
by o6.proadmin.com (8.9.3/8.9.3/ProAdmin) with ESMTP id PAA00603
for ; Wed, 26 Apr 2000 15:26:47 -0700
Received: (from edc@localhost)
by o3.proadmin.com (8.9.3/8.9.3/ProAdmin) id PAA28756
for f500@o6.proadmin.com; Wed, 26 Apr 2000 15:26:42 -0700
Date: Wed, 26 Apr 2000 15:26:42 -0700
From: "Eric D. Christensen"
Message-Id: <200004262226.PAA28756@o3.proadmin.com>
To: f500@o6.proadmin.com
Subject: [F500] test from unix
Sender: f500-admin@o6.proadmin.com
Errors-To: f500-admin@o6.proadmin.com
X-BeenThere: f500@o6.proadmin.com
X-Mailman-Version: 2.0beta2
Precedence: bulk
Reply-To: f500@o6.proadmin.com
List-Id: The Formula 500 Mailing List
test
_______________________________________________
F500 mailing list
F500@o6.proadmin.com
http://216.139.10.131/mailman/listinfo/f500
Any clues would be appreciated - since I obviously don't have one! :-)
--
======================================================
Eric D. Christensen ProAdmin, Inc.
Email: edc@proadmin.com http://www.proadmin.com
Phone: 408-776-3410 Fax: 408-776-3420
======================================================
From Dan Mick Thu Apr 27 03:42:10 2000
From: Dan Mick (Dan Mick)
Date: Wed, 26 Apr 2000 19:42:10 -0700 (PDT)
Subject: [Mailman-Developers] Beta2 - Bug in archiver?
Message-ID: <200004270242.TAA02810@utopia.west.sun.com>
I can't find anything anywhere in 2.0beta2 that defines "SetHeader()".
That could be a problem. Archiver.py is not supposed to call
SetHeader() anymore, but apparently, as of 2.0beta2, it still
does. It appears to be fixed in revision 1.21 of Archiver.py.
I guess 2.0beta2 is just broken in this respect.
The newer Archiver.py uses "post['Date'] = time.ctime(time.time())"
in place of "post.SetHeader..."
diff -r1.20 -r1.21
160c160
< post.SetHeader('Date', time.ctime(time.time()))
---
> post['Date'] = time.ctime(time.time())
173c173
< post.SetHeader('Date', olddate)
---
> post['Date'] = olddate
192c192
< if mm_cfg.ARCHIVE_TO_MBOX == -1:
---
> if mm_cfg.ARCHIVE_TO_MBOX == -1 or not self.archive:
(There's also a revision 1.22, so I don't really advocate using this..)
> I've poked at this a little but I'm not making any headway. Not being very
> python literate, it's probably something simple that I'm missing. Nontheless,
I
> figured I'd bounce it off the experts....
>
> This is 2.0Beta2 running under Mandrake Linux on a PIII with sendmail 8.9.3.
> Python version is: Python 1.5.2 (#1, Apr 25 2000, 12:39:13) [GCC pgcc-2.91.66
19990314 (egcs-1.1.2 on linux2
>
> Seems to be a hole somewhere related to the date parsing code in the archiver.
>
> In logs/error I get the following traceback whenever a message is posted to
the list:
>
> Apr 25 16:35:05 2000 post(28927): Traceback (innermost last):
> post(28927): File "/home/mailman/Mailman/Archiver/Archiver.py", line 204,
in ArchiveMail
> post(28927): self.__archive_to_mbox(msg)
> post(28927): File "/home/mailman/Mailman/Archiver/Archiver.py", line 160,
in __archive_to_mbox
> post(28927): post.SetHeader('Date', time.ctime(time.time()))
> post(28927): AttributeError: SetHeader
>
> The message gets forwarded to the list correctly. It just doesn't get
archived.
>
> For what it's worth.... the message looks like this when it comes through the
> list. I don't see anything abviously broken in the headers:
>
> From f500-admin@o6.proadmin.com Wed Apr 26 15:26:48 2000
> Return-Path:
> Received: from o6.proadmin.com (o6.proadmin.com [208.195.160.175])
> by o3.proadmin.com (8.9.3/8.9.3/ProAdmin) with ESMTP id PAA28760
> for ; Wed, 26 Apr 2000 15:26:48 -0700
> Received: from o6.proadmin.com (localhost [127.0.0.1])
> by o6.proadmin.com (8.9.3/8.9.3/ProAdmin) with ESMTP id PAA00608
> for ; Wed, 26 Apr 2000 15:26:48 -0700
> Received: from o3.proadmin.com ([199.108.70.172])
> by o6.proadmin.com (8.9.3/8.9.3/ProAdmin) with ESMTP id PAA00603
> for ; Wed, 26 Apr 2000 15:26:47 -0700
> Received: (from edc@localhost)
> by o3.proadmin.com (8.9.3/8.9.3/ProAdmin) id PAA28756
> for f500@o6.proadmin.com; Wed, 26 Apr 2000 15:26:42 -0700
> Date: Wed, 26 Apr 2000 15:26:42 -0700
> From: "Eric D. Christensen"
> Message-Id: <200004262226.PAA28756@o3.proadmin.com>
> To: f500@o6.proadmin.com
> Subject: [F500] test from unix
> Sender: f500-admin@o6.proadmin.com
> Errors-To: f500-admin@o6.proadmin.com
> X-BeenThere: f500@o6.proadmin.com
> X-Mailman-Version: 2.0beta2
> Precedence: bulk
> Reply-To: f500@o6.proadmin.com
> List-Id: The Formula 500 Mailing List
>
> test
> _______________________________________________
> F500 mailing list
> F500@o6.proadmin.com
> http://216.139.10.131/mailman/listinfo/f500
>
>
> Any clues would be appreciated - since I obviously don't have one! :-)
>
> --
> ======================================================
> Eric D. Christensen ProAdmin, Inc.
> Email: edc@proadmin.com http://www.proadmin.com
> Phone: 408-776-3410 Fax: 408-776-3420
> ======================================================
>
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers@python.org
> http://www.python.org/mailman/listinfo/mailman-developers
From edc@proadmin.com Thu Apr 27 04:49:17 2000
From: edc@proadmin.com (Eric D. Christensen)
Date: Wed, 26 Apr 2000 20:49:17 -0700
Subject: [Mailman-Developers] Beta2 - Bug in archiver?
In-Reply-To: <200004270242.TAA02810@utopia.west.sun.com>
Message-ID:
Yup... that was it Dan. Thanks! I just applied the diff.... didn't bother
grabbing 1.22 out of CVS. I'll wait for beta 3...
That gets things working enough to actually put one of my lists online with
beta 2 to see what happens. I've been testing for a few days and everything
else seems to be fairly stable. A few weird things here and there, but
overall it mostly seems to work.
I guess I'll find out more when I put a real list on it later this week.
----
Eric D. Christensen
ProAdmin, Inc.
Serious System Administration
(408)776-3410
> -----Original Message-----
> From: Dan Mick [mailto:Dan.Mick@West.Sun.COM]
> Sent: Wednesday, April 26, 2000 7:42 PM
> To: mailman-developers@python.org; edc@ns.proadmin.com
> Subject: Re: [Mailman-Developers] Beta2 - Bug in archiver?
>
>
> I can't find anything anywhere in 2.0beta2 that defines "SetHeader()".
> That could be a problem. Archiver.py is not supposed to call
> SetHeader() anymore, but apparently, as of 2.0beta2, it still
> does. It appears to be fixed in revision 1.21 of Archiver.py.
> I guess 2.0beta2 is just broken in this respect.
>
> The newer Archiver.py uses "post['Date'] = time.ctime(time.time())"
> in place of "post.SetHeader..."
>
> diff -r1.20 -r1.21
> 160c160
> < post.SetHeader('Date', time.ctime(time.time()))
> ---
> > post['Date'] = time.ctime(time.time())
> 173c173
> < post.SetHeader('Date', olddate)
> ---
> > post['Date'] = olddate
> 192c192
> < if mm_cfg.ARCHIVE_TO_MBOX == -1:
> ---
> > if mm_cfg.ARCHIVE_TO_MBOX == -1 or not self.archive:
>
>
> (There's also a revision 1.22, so I don't really advocate using this..)
>
> > I've poked at this a little but I'm not making any headway. Not
> being very
> > python literate, it's probably something simple that I'm
> missing. Nontheless,
> I
> > figured I'd bounce it off the experts....
> >
> > This is 2.0Beta2 running under Mandrake Linux on a PIII with
> sendmail 8.9.3.
> > Python version is: Python 1.5.2 (#1, Apr 25 2000, 12:39:13)
> [GCC pgcc-2.91.66
> 19990314 (egcs-1.1.2 on linux2
> >
> > Seems to be a hole somewhere related to the date parsing code
> in the archiver.
> >
> > In logs/error I get the following traceback whenever a message
> is posted to
> the list:
> >
> > Apr 25 16:35:05 2000 post(28927): Traceback (innermost last):
> > post(28927): File
> "/home/mailman/Mailman/Archiver/Archiver.py", line 204,
> in ArchiveMail
> > post(28927): self.__archive_to_mbox(msg)
> > post(28927): File
> "/home/mailman/Mailman/Archiver/Archiver.py", line 160,
> in __archive_to_mbox
> > post(28927): post.SetHeader('Date', time.ctime(time.time()))
> > post(28927): AttributeError: SetHeader
> >
> > The message gets forwarded to the list correctly. It just doesn't get
> archived.
> >
> > For what it's worth.... the message looks like this when it
> comes through the
> > list. I don't see anything abviously broken in the headers:
> >
> > From f500-admin@o6.proadmin.com Wed Apr 26 15:26:48 2000
> > Return-Path:
> > Received: from o6.proadmin.com (o6.proadmin.com [208.195.160.175])
> > by o3.proadmin.com (8.9.3/8.9.3/ProAdmin) with ESMTP
> id PAA28760
> > for ; Wed, 26 Apr 2000 15:26:48 -0700
> > Received: from o6.proadmin.com (localhost [127.0.0.1])
> > by o6.proadmin.com (8.9.3/8.9.3/ProAdmin) with ESMTP
> id PAA00608
> > for ; Wed, 26 Apr 2000 15:26:48 -0700
> > Received: from o3.proadmin.com ([199.108.70.172])
> > by o6.proadmin.com (8.9.3/8.9.3/ProAdmin) with ESMTP
> id PAA00603
> > for ; Wed, 26 Apr 2000 15:26:47 -0700
> > Received: (from edc@localhost)
> > by o3.proadmin.com (8.9.3/8.9.3/ProAdmin) id PAA28756
> > for f500@o6.proadmin.com; Wed, 26 Apr 2000 15:26:42 -0700
> > Date: Wed, 26 Apr 2000 15:26:42 -0700
> > From: "Eric D. Christensen"
> > Message-Id: <200004262226.PAA28756@o3.proadmin.com>
> > To: f500@o6.proadmin.com
> > Subject: [F500] test from unix
> > Sender: f500-admin@o6.proadmin.com
> > Errors-To: f500-admin@o6.proadmin.com
> > X-BeenThere: f500@o6.proadmin.com
> > X-Mailman-Version: 2.0beta2
> > Precedence: bulk
> > Reply-To: f500@o6.proadmin.com
> > List-Id: The Formula 500 Mailing List
> >
> > test
> > _______________________________________________
> > F500 mailing list
> > F500@o6.proadmin.com
> > http://216.139.10.131/mailman/listinfo/f500
> >
> >
> > Any clues would be appreciated - since I obviously don't have one! :-)
> >
> > --
> > ======================================================
> > Eric D. Christensen ProAdmin, Inc.
> > Email: edc@proadmin.com http://www.proadmin.com
> > Phone: 408-776-3410 Fax: 408-776-3420
> > ======================================================
> >
> > _______________________________________________
> > Mailman-Developers mailing list
> > Mailman-Developers@python.org
> > http://www.python.org/mailman/listinfo/mailman-developers
>
From jarrell@vt.edu Thu Apr 27 16:23:56 2000
From: jarrell@vt.edu (Ron Jarrell)
Date: Thu, 27 Apr 2000 11:23:56 -0400
Subject: [Mailman-Developers] y2k problem with pipermail/python
Message-ID: <4.3.1.2.20000427112038.056fdca0@vtserf.cc.vt.edu>
Not really mailman's fault, but the archiving process dies a horribly flaming
death if some bozo is using a client that sticks "100" in as the year (because
the author never read the man page for tm...)
I've *still* got users, it seems using those. Like one I just emailed suggesting
he get off of elm 2.4pl25. I noticed this whan trying to redo a couple of archives,
arch was blowing up starting in january. The mailbox dateconverstion routine
can't cope.
Ideally, since it's a known problem, a special case ought to be made mapping
dates in the second century to the 21st century :-).
From jarrell@vt.edu Thu Apr 27 16:33:53 2000
From: jarrell@vt.edu (Ron Jarrell)
Date: Thu, 27 Apr 2000 11:33:53 -0400
Subject: [Mailman-Developers] nightly_gzip not compressing weeklys?
Message-ID: <4.3.1.2.20000427112723.00e29380@vtserf.cc.vt.edu>
Does nightly_gzip only handle monthlies? It's not clear from the code
(and my knowledge of python :-)), but it sure looks like it's only
compressing monthlys.
From Nigel.Metheringham@VData.co.uk Thu Apr 27 18:10:44 2000
From: Nigel.Metheringham@VData.co.uk (Nigel Metheringham)
Date: Thu, 27 Apr 2000 18:10:44 +0100
Subject: [Mailman-Developers] nightly_gzip not compressing weeklys?
In-Reply-To: Message from Ron Jarrell
of "Thu, 27 Apr 2000 11:33:53 EDT." <4.3.1.2.20000427112723.00e29380@vtserf.cc.vt.edu>
Message-ID:
jarrell@vt.edu said:
> Does nightly_gzip only handle monthlies? It's not clear from the code
> (and my knowledge of python :-)), but it sure looks like it's only
> compressing monthlys.
It certainly isn't working for my weekly lists.
Nigel.
--
[ - Opinions expressed are personal and may not be shared by VData - ]
[ Nigel Metheringham Nigel.Metheringham@VData.co.uk ]
[ Phone: +44 1423 850000 Fax +44 1423 858866 ]
From jarrell@vt.edu Thu Apr 27 18:34:03 2000
From: jarrell@vt.edu (Ron Jarrell)
Date: Thu, 27 Apr 2000 13:34:03 -0400
Subject: [Mailman-Developers] nightly_gzip not compressing weeklys?
In-Reply-To:
References:
<4.3.1.2.20000427112723.00e29380@vtserf.cc.vt.edu>
Message-ID: <4.3.1.2.20000427133321.00ca2840@vtserf.cc.vt.edu>
--=====================_19892763==_.ALT
Content-Type: text/plain; charset="us-ascii"
At 06:10 PM 4/27/00 +0100, Nigel Metheringham wrote:
>jarrell@vt.edu said:
> > Does nightly_gzip only handle monthlies? It's not clear from the code
> > (and my knowledge of python :-)), but it sure looks like it's only
> > compressing monthlys.
>
>It certainly isn't working for my weekly lists.
Or quarterlies, or yearlies.. (I have a mix of them all...)
--=====================_19892763==_.ALT
Content-Type: text/html; charset="us-ascii"
At 06:10 PM 4/27/00 +0100, Nigel Metheringham wrote:
jarrell@vt.edu said:
> Does nightly_gzip only handle monthlies? It's not clear from
the code
> (and my knowledge of python :-)), but it sure looks like it's
only
> compressing monthlys.
It certainly isn't working for my weekly lists.
Or quarterlies, or yearlies.. (I have a mix of them all...)
--=====================_19892763==_.ALT--
From bwarsaw@cnri.reston.va.us Thu Apr 27 21:34:37 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Thu, 27 Apr 2000 16:34:37 -0400 (EDT)
Subject: [Mailman-Developers] nightly_gzip not compressing weeklys?
References: <4.3.1.2.20000427112723.00e29380@vtserf.cc.vt.edu>
Message-ID: <14600.42077.73100.809048@anthem.cnri.reston.va.us>
>>>>> "RJ" == Ron Jarrell writes:
RJ> Does nightly_gzip only handle monthlies? It's not clear from
RJ> the code (and my knowledge of python :-)), but it sure looks
RJ> like it's only compressing monthlys.
Back in February, Thomas Wouters suggested this patch. It seems to
work for me so I'll go ahead and commit it.
-Barry
Index: nightly_gzip
===================================================================
RCS file: /projects/cvsroot/mailman/cron/nightly_gzip,v
retrieving revision 1.5
diff -c -r1.5 nightly_gzip
*** nightly_gzip 2000/03/21 06:26:25 1.5
--- nightly_gzip 2000/04/27 20:26:28
***************
*** 111,121 ****
if mlist.last_post_time > 0:
print 'List', name, 'has a bogus archive_directory:', dir
continue
files = []
for f in allfiles:
! try:
! time.strptime(f, '%Y-%B.txt')
! except ValueError:
continue
# stat both the .txt and .txt.gz files and append them only if
# the former is newer than the latter.
--- 111,121 ----
if mlist.last_post_time > 0:
print 'List', name, 'has a bogus archive_directory:', dir
continue
+ if VERBOSE:
+ print 'Processing list:', name
files = []
for f in allfiles:
! if f[-4:] <> '.txt':
continue
# stat both the .txt and .txt.gz files and append them only if
# the former is newer than the latter.
From thomas@xs4all.net Thu Apr 27 23:15:09 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Fri, 28 Apr 2000 00:15:09 +0200
Subject: [Mailman-Developers] nightly_gzip not compressing weeklys?
In-Reply-To: <14600.42077.73100.809048@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Thu, Apr 27, 2000 at 04:34:37PM -0400
References: <4.3.1.2.20000427112723.00e29380@vtserf.cc.vt.edu> <14600.42077.73100.809048@anthem.cnri.reston.va.us>
Message-ID: <20000428001509.J11481@xs4all.nl>
On Thu, Apr 27, 2000 at 04:34:37PM -0400, Barry A. Warsaw wrote:
> RJ> Does nightly_gzip only handle monthlies? It's not clear from
> RJ> the code (and my knowledge of python :-)), but it sure looks
> RJ> like it's only compressing monthlys.
> Back in February, Thomas Wouters suggested this patch. It seems to
> work for me so I'll go ahead and commit it.
It wasn't me. Or at least I dont think it was. The VERBOSE bit is certainly
not mine, and I seem to recall someone else suggesting this patch. I
probably commented on it, though, because the use of time.strptime bugged me
-- BSDI doesn't have strptime. As I can't find this patch in any of my
Mailman installs, patch archives and mailboxes, I'm fairly sure it's not me
who's seeing premature signs of old age ;)
Will the the real contributor please rise ? ;)
> Index: nightly_gzip
> + if VERBOSE:
> + print 'Processing list:', name
> files = []
> for f in allfiles:
> ! if f[-4:] <> '.txt':
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From bwarsaw@cnri.reston.va.us Fri Apr 28 00:45:39 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Thu, 27 Apr 2000 19:45:39 -0400 (EDT)
Subject: [Mailman-Developers] nightly_gzip not compressing weeklys?
References: <4.3.1.2.20000427112723.00e29380@vtserf.cc.vt.edu>
<14600.42077.73100.809048@anthem.cnri.reston.va.us>
<20000428001509.J11481@xs4all.nl>
Message-ID: <14600.53539.355440.258793@anthem.cnri.reston.va.us>
>>>>> "TW" == Thomas Wouters writes:
TW> It wasn't me. Or at least I dont think it was. The VERBOSE bit
TW> is certainly not mine, and I seem to recall someone else
TW> suggesting this patch. I probably commented on it, though,
TW> because the use of time.strptime bugged me -- BSDI doesn't
TW> have strptime. As I can't find this patch in any of my Mailman
TW> installs, patch archives and mailboxes, I'm fairly sure it's
TW> not me who's seeing premature signs of old age ;)
Hey, I resemble that remark! Naw, I added the VERBOSE bit and it
leaked into my posted patch.
-Barry
From brian@gweep.bc.ca Sun Apr 30 22:22:44 2000
From: brian@gweep.bc.ca (Brian Edmonds)
Date: 30 Apr 2000 14:22:44 -0700
Subject: [Mailman-Developers] change the envelope sender with Sendmail.py?
In-Reply-To: Owen Taylor's message of "23 Apr 2000 18:50:23 -0400"
Message-ID: <37em7n472z.fsf@lios.gweep.bc.ca>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I've been having a problem (using 2.0beta2) with a number of bounce
messages going back to the poster rather than the list owner. I've
tracked it down to the envelope sender being left as the poster, and not
being changed to the list admin address. It appeared to be inconsistent
since some MTAs were handling Errors-To, which was being set to the
admin address, while some use the envelope sender.
The change is on line 91 of Handlers/Sendmail.py[1], where msg.GetSender
needs to become mlist.GetAdminEmail.
Brian.
[1] I don't know if this is also a problem in the direct SMTP module.
I'm using Sendmail.py because my MTA is sendmail, but I'm using postfix
in outgoing only configuration for the list delivery since mailman was
getting sendmail to send each message as one huge batch of addresses.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.1 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.5 and Gnu Privacy Guard
iD8DBQE5DKQbcCEFQUX5+OwRAvxkAJ91zqlhXTD5XAezZ/PHKKDWa+jNagCeKFO1
2zVlOt8M2I0sn3qFAXD2XGY=
=79kq
-----END PGP SIGNATURE-----
From ricardo@rixhq.nu Sun Apr 30 22:50:37 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sun, 30 Apr 2000 23:50:37 +0200
Subject: [Mailman-Developers] suggestion for heldmsg placement
Message-ID: <20000430235037.C10063@miss-janet.com>
Hi,
I think it could be better to place the heldmsg-* files
that are created for messages held for approval in
a directory like ~mailman/listname/lists/mailinglist/pending
instead of all messages for all lists in ~mailman/data...
does anybody agree with me? :)
Ricardo.
--
From ricardo@rixhq.nu Sat Apr 1 14:57:13 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sat, 1 Apr 2000 16:57:13 +0200
Subject: [Mailman-Developers] membership managent web interface
Message-ID: <20000401165713.B2495@miss-janet.com>
Hi,
I was thinking about a way to make to the email list on the admin page
a bit easier to use. As it works now, the only way to find a certain
email address, is by figuring out in which page block the email address is
and the look for it down there.
But most of the times you know the exact address you want to unsubscribe
or view/edit, so why not add a search box on top? Yes I now you can use the
scripts to remove a certain address, but not all admins can (or should)
have access to the shell prompt.
I even think that only a searchbox on the initial page should be enough,
and only show the paged list of members when the user specifically asks
for it... saves a bit of bandwidth / response time.
any thoughts about this?
Ricardo.
--
From jack@cybermail.net Sat Apr 1 17:26:12 2000
From: jack@cybermail.net (Jack Snodgrass)
Date: Sat, 1 Apr 2000 11:26:12 -0600
Subject: [Mailman-Developers] www.list.org ? - looking for mailman mailling list software
Message-ID: <021d01bf9bff$609f94a0$0564a8c0@private.net>
This is a multi-part message in MIME format.
------=_NextPart_000_021A_01BF9BCD.15E392E0
Content-Type: text/plain;
charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
can you tell me where I can find your mailman mailing list software?=20
I can't seen to find www.list.org. That's what freshmeat is pointing
to.
jack
------=_NextPart_000_021A_01BF9BCD.15E392E0
Content-Type: text/html;
charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
can you tell me where I can find your =
mailman=20
mailing list software?
I can't seen to find www.list.org. That's what freshmeat is=20
pointing
to.
jack
------=_NextPart_000_021A_01BF9BCD.15E392E0--
From ricardo@rixhq.nu Sat Apr 1 17:31:28 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sat, 1 Apr 2000 19:31:28 +0200
Subject: [Mailman-Developers] www.list.org ? - looking for mailman mailling list software
In-Reply-To: <021d01bf9bff$609f94a0$0564a8c0@private.net>; from jack@cybermail.net on Sat, Apr 01, 2000 at 11:26:12AM -0600
References: <021d01bf9bff$609f94a0$0564a8c0@private.net>
Message-ID: <20000401193128.B3354@miss-janet.com>
On Sat, Apr 01, 2000 at 11:26:12AM -0600, Jack Snodgrass wrote:
> can you tell me where I can find your mailman mailing list software?
> I can't seen to find www.list.org. That's what freshmeat is pointing
> to.
ftp://ftp.gnu.org/gnu/mailman/
Ricardo.
--
From rullfig@uchicago.edu Mon Apr 3 16:01:19 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Mon, 03 Apr 2000 10:01:19 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
Message-ID: <38E8B23F.16F25CE7@uchicago.edu>
So, in 1.0rc2, displaying the list of lists for 529 lists
requires 529**2 = 279841 system stat calls and takes over one
and a half minutes on our Ultra-2 2x296 processor system! Is
this because of Python, Mailman, or both? Has this been "fixed"
in 2.0? You really should only need to make one stat call per
list.
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From gorgo@sztaki.hu Mon Apr 3 16:20:20 2000
From: gorgo@sztaki.hu (Gergely Madarasz)
Date: Mon, 03 Apr 2000 17:20:20 +0200 (MET DST)
Subject: [Mailman-Developers] Bug#61695: mailman: mail->news gw blows up on mails without subject
(fwd)
Message-ID:
I just got this bugreport in the debian BTS. Comments ?
--
Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org
It's practically impossible to look at a penguin and feel angry.
Egy pingvinre gyakorlatilag lehetetlen haragosan nezni.
HuLUG: http://mlf.linux.rulez.org/
---------- Forwarded message ----------
Date: Mon, 03 Apr 2000 18:16:45 +0300
From: erno@iki.fi
To: submit@bugs.debian.org
Subject: Bug#61695: mailman: mail->news gw blows up on mails without subject
Resent-Date: Mon, 03 Apr 2000 15:18:02 +0000 (GMT)
Resent-From: erno@iki.fi
Resent-To: debian-bugs-dist@lists.debian.org
Resent-cc: Gergely Madarasz
Package: mailman
Version: 1.1-4
Severity: important
hello,
i have a local newsgroup, and i created a mailman list named test2,
telling mailman to gateway messages from it to the local newsgroup.
the news server is inn-1.7.2-16.
running
% echo seppo | mail test2@erno.iki.fi
leads to
From: Mail Delivery System
Subject: Mail delivery failed: returning message to sender
To: erno@erno.iki.fi
X-Failed-Recipients: test2@erno.iki.fi
Date: Mon, 03 Apr 2000 18:08:03 +0300
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. The following address(es) failed:
test2@erno.iki.fi:
generated |/var/lib/mailman/mail/wrapper post test2
The following text was generated during the delivery attempt:
------ |/var/lib/mailman/mail/wrapper post test2 ------
Traceback (innermost last):
File "/var/lib/mailman/scripts/post", line 73, in ?
mlist.Post(msg, approved=fromusenet)
File "/usr/lib/mailman/Mailman/MailList.py", line 1331, in Post
self.SendMailToNewsGroup(msg)
File "/usr/lib/mailman/Mailman/GatewayManager.py", line 143, in
SendMailToNewsGroup
msg.SetHeader('Subject', '%s(no subject)' % subjpref)
NameError: subjpref
------ This is a copy of the message, including all the headers. ------
Return-path:
Received: from erno by erno.iki.fi with local (Exim 3.12 #1 (Debian))
id 12c8SN-0004Jv-00
for ; Mon, 03 Apr 2000 18:07:59 +0300
To: test2@erno.iki.fi
Message-Id:
From: Erno Kuusela
Date: Mon, 03 Apr 2000 18:07:59 +0300
seppo
looks to me like this is bad, because if i want to use mailman to, say,
gateway messages from debian-devel to a local read-only newsgroup,
bounce messages will be sent to the listowner of debian-devel
and he will likely kick me off the list when he gets tired
of things like this.
also this is a crash, and if i understand correctly those should
generally filed as important bugs. (please tell me if i'm wrong)
-- System Information
Debian Release: 2.2
Kernel Version: Linux fun77 2.2.10 #1 Sun Jun 20 19:03:53 EEST 1999 i586 unknown
Versions of the packages mailman depends on:
ii apache 1.3.9-12 Versatile, high-performance HTTP server
ii cron 3.0pl1-57 management of regular background processing
ii libc6 2.1.3-7 GNU C Library: Shared libraries and Timezone
ii logrotate 3.2-11 Log rotation utility
ii python-base 1.5.2-9 An interactive object-oriented scripting lan
sendmail Not installed or no info
ii exim 3.12-7 Exim Mailer
^^^ (Provides virtual package mail-transport-agent)
ii apache 1.3.9-12 Versatile, high-performance HTTP server
^^^ (Provides virtual package httpd)
From rullfig@uchicago.edu Mon Apr 3 17:26:19 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Mon, 03 Apr 2000 11:26:19 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
Message-ID: <38E8C62B.37883725@uchicago.edu>
Roberto Ullfig wrote:
>
> So, in 1.0rc2, displaying the list of lists for 529 lists
> requires 529**2 = 279841 system stat calls and takes over one
> and a half minutes on our Ultra-2 2x296 processor system! Is
> this because of Python, Mailman, or both? Has this been "fixed"
> in 2.0? You really should only need to make one stat call per
> list.
>
Whatever the answer is, we'd like to be able to generate the
lists of lists once a day; I tried running:
chroot /opt/http /opt/bin/python
/opt/pkgs/mailman/scripts/driver listinfo
but get this traceback:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[----- Mailman Version: <undetermined> -----]
[----- Traceback ------]
Traceback (innermost last):
File "/opt/pkgs/mailman/scripts/driver", line 135, in
print_traceback
from Mailman.mm_cfg import VERSION
ImportError: No module named Mailman.mm_cfg
Content-type: text/html
Bug in Mailman version
<undetermined>
Bug in Mailman version <undetermined>
We're sorry, we hit a bug!
Looks like a I'm missing something fundamental here. I assume
that the output from this command would be html commands that
could be redirected to a file. Thank you!
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From dan.mick@West.Sun.COM Mon Apr 3 22:04:40 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Mon, 03 Apr 2000 14:04:40 -0700
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu> <38E8C62B.37883725@uchicago.edu>
Message-ID: <38E90768.FD8B372C@west.sun.com>
Roberto Ullfig wrote:
> ImportError: No module named Mailman.mm_cfg
> Content-type: text/html
>
> Bug in Mailman version
> <undetermined>
> Bug in Mailman version <undetermined>
> We're sorry, we hit a bug!
>
> Looks like a I'm missing something fundamental here.
Python's library-search path has to be set to include /home/mailman/Mailman.
I just discovered that bin/withlist doesn't do this recently; I'm guessing
the driver script doesn't do it either. For your purposes, I would
think that augmenting PYTHONPATH in the environment should suffice.
Adding "PYTHONPATH=$PYTHONPATH:/home/mailman/Mailman" makes it work for
me. BTW, the chroot seems unnecessary (maybe you knew that, and were
just trying for security or something).
From dan.mick@West.Sun.COM Mon Apr 3 22:10:00 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Mon, 03 Apr 2000 14:10:00 -0700
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu> <38E8C62B.37883725@uchicago.edu> <38E90768.FD8B372C@west.sun.com>
Message-ID: <38E908A8.36EE28DC@west.sun.com>
Dan Mick wrote:
>
> Roberto Ullfig wrote:
>
> > ImportError: No module named Mailman.mm_cfg
> > Content-type: text/html
> >
> > Bug in Mailman version
> > <undetermined>
> > Bug in Mailman version <undetermined>
> > We're sorry, we hit a bug!
> >
> > Looks like a I'm missing something fundamental here.
>
> Python's library-search path has to be set to include /home/mailman/Mailman.
> I just discovered that bin/withlist doesn't do this recently; I'm guessing
> the driver script doesn't do it either. For your purposes, I would
> think that augmenting PYTHONPATH in the environment should suffice.
>
> Adding "PYTHONPATH=$PYTHONPATH:/home/mailman/Mailman" makes it work for
> me. BTW, the chroot seems unnecessary (maybe you knew that, and were
> just trying for security or something).
BTW, "python -i /home/mailman/Mailman/Cgi/listinfo.py" gives the same
output, although again it needs the PYTHONPATH setting.
From ich@henning-schroeder.de Tue Apr 4 00:29:08 2000
From: ich@henning-schroeder.de (Henning Schroeder)
Date: Tue, 04 Apr 2000 01:29:08 +0200
Subject: [Mailman-Developers] Local characters (loike =?iso-8859-1?Q?=C4=D6=DC=DF?=) in pipermail
Message-ID: <38E92944.B5871C5@henning-schroeder.de>
Hello,
I just modified the latest version of Mailman/pipermail to support local
characters (like äöüß). Headers with "=?ISO-8859-1..." are really
disturbing :-(
Basically my code decodes every message header with the mimify module.
Is anyone interested in the source? It modifies Message.py, Mailbox.py,
pipermail.py and Hyperarch.py.
Besides I would like to implement a better MIME support in Mailman. Then
you could e.g. block binary attachments in the list or display
HTML-mails with inline-graphics in the archive.
Henning
From bwarsaw@cnri.reston.va.us Tue Apr 4 03:38:22 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 3 Apr 2000 22:38:22 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
Message-ID: <14569.21918.950338.815597@anthem.cnri.reston.va.us>
>>>>> "RU" == Roberto Ullfig writes:
RU> So, in 1.0rc2, displaying the list of lists for 529 lists
RU> requires 529**2 = 279841 system stat calls and takes over one
RU> and a half minutes on our Ultra-2 2x296 processor system! Is
RU> this because of Python, Mailman, or both? Has this been
RU> "fixed" in 2.0? You really should only need to make one stat
RU> call per list.
Uh, it's because of Mailman :)
I implemented a list_lists scripts which does on the command line what
listinfo.py does in HTML (see attached). Here's what truss -c gives
me:
-------------------- snip snip --------------------
Portal - [no description available]
Postal - [no description available]
Stage - Staging new Mailman releases
Test - [no description available]
syscall seconds calls errors
_exit .00 1
read .00 102
write .00 8
open .11 607 474
close .01 143
time .00 3
brk .03 227
stat .03 201 157
getpid .00 10
fstat .00 66
ioctl .02 63 61
execve .00 10 8
umask .00 2
fcntl .00 7
readlink .00 2 2
sigprocmask .00 2
sigaction .00 50
sigpending .00 1
mmap .00 42
mprotect .00 10
munmap .00 11
uname .00 4
sysconfig .00 1
lwp_create .00 6
lwp_continue .00 2
lwp_self .00 3
llseek .00 114
door .00 5
lwp_schedctl .01 5
getdents64 .01 15
fstat64 .00 67
open64 .00 7
---- --- ---
sys totals: .22 1797 702
usr time: .51
elapsed: 1.19
-------------------- snip snip --------------------
Getting the list of list names, requires at least a listdir() and an
exists() for every directory found there.
Nothing about this will change for 2.0.
-Barry
-------------------- snip snip --------------------
#! /usr/bin/env python
#
# Copyright (C) 1998,1999,2000 by the Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""List all mailing lists.
Usage: %(program)s [options]
Where:
--advertised
-a
List only those mailing lists that are publically advertised
--virtual-host-overview=domain
-V domain
List only those mailing lists that are homed to the given virtual
domain. This only works if the VIRTUAL_HOST_OVERVIEW variable is
set.
--help
-h
Print this text and exit.
"""
import sys
import getopt
import paths
from Mailman import mm_cfg
from Mailman import MailList
from Mailman import Utils
from Mailman import Errors
program = sys.argv[0]
def usage(status, msg=''):
print __doc__ % globals()
if msg:
print msg
sys.exit(status)
def main():
try:
opts, args = getopt.getopt(sys.argv[1:], 'aV:h',
['advertised', 'virtual-host-overview=',
'help'])
except getopt.error, msg:
usage(1, msg)
advertised = 0
vhost = None
for opt, arg in opts:
if opt in ('-h', '--help'):
usage(0)
elif opt in ('-a', '--advertised'):
advertised = 1
elif opt in ('-V', '--virtual-host-overview'):
vhost = arg
names = Utils.list_names()
names.sort()
mlists = []
longest = 0
for n in names:
mlist = MailList.MailList(n, lock=0)
if advertised and not mlist.advertised:
continue
if vhost and mm_cfg.VIRTUAL_HOST_OVERVIEW and \
string.find(vhost, mlist.web_page_url) == -1 and \
string.find(mlist.web_page_url, vhost) == -1:
continue
mlists.append(mlist)
longest = max(len(mlist.real_name), longest)
if not mlists:
print 'No matching mailing lists found'
return
format = '%%%ds - %%.%ds' % (longest, 77 - longest)
for mlist in mlists:
description = mlist.description or '[no description available]'
print format % (mlist.real_name, description)
if __name__ == '__main__':
main()
From bwarsaw@cnri.reston.va.us Tue Apr 4 03:47:16 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 3 Apr 2000 22:47:16 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
<38E8C62B.37883725@uchicago.edu>
Message-ID: <14569.22452.191246.382870@anthem.cnri.reston.va.us>
>>>>> "RU" == Roberto Ullfig writes:
| Whatever the answer is, we'd like to be able to generate the
| lists of lists once a day; I tried running:
| chroot /opt/http /opt/bin/python
| /opt/pkgs/mailman/scripts/driver listinfo
RU> but get this traceback:
| @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
| [----- Mailman Version: <undetermined> -----]
| [----- Traceback ------]
| Traceback (innermost last):
| File "/opt/pkgs/mailman/scripts/driver", line 135, in
| print_traceback
| from Mailman.mm_cfg import VERSION
| ImportError: No module named Mailman.mm_cfg
| Content-type: text/html
Looks like you're trying to run this out of the source directory not
the installed directory. I don't think it makes much sense trying to
run the CGI's via the driver outside a web server environment -- there
are all sorts of environment things missing. Much better to write a
script to do what you want. See my recently posted list_lists for an
example (or anything inside bin/).
-Barry
From bwarsaw@cnri.reston.va.us Tue Apr 4 04:02:06 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 3 Apr 2000 23:02:06 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
<38E8C62B.37883725@uchicago.edu>
<38E90768.FD8B372C@west.sun.com>
Message-ID: <14569.23342.566957.87822@anthem.cnri.reston.va.us>
>>>>> "DM" == Dan Mick writes:
DM> Python's library-search path has to be set to include
DM> /home/mailman/Mailman. I just discovered that bin/withlist
DM> doesn't do this recently; I'm guessing the driver script
DM> doesn't do it either. For your purposes, I would think that
DM> augmenting PYTHONPATH in the environment should suffice.
Right, because driver is only supposed to be run from the C wrapper
binary, which /does/ set PYTHONPATH. I didn't expect withlist to be
run without explicitly invoking python. The -r switch could be used
this way so I think I'll add a #! line at the top. Note that it does
not help to add -i in the #! line.
-Barry
From Dan Mick Tue Apr 4 04:09:26 2000
From: Dan Mick (Dan Mick)
Date: Mon, 3 Apr 2000 20:09:26 -0700 (PDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
Message-ID: <200004040309.UAA10041@utopia.West.Sun.COM>
> >>>>> "DM" == Dan Mick writes:
>
> DM> Python's library-search path has to be set to include
> DM> /home/mailman/Mailman. I just discovered that bin/withlist
> DM> doesn't do this recently; I'm guessing the driver script
> DM> doesn't do it either. For your purposes, I would think that
> DM> augmenting PYTHONPATH in the environment should suffice.
>
> Right, because driver is only supposed to be run from the C wrapper
> binary, which /does/ set PYTHONPATH. I didn't expect withlist to be
> run without explicitly invoking python.
But even when withlist is run from Python, PYTHONPATH/sys.path doesn't
include /home/mailman/Mailman (this is 1.1):
python -i bin/withlist
Loading list: (unlocked)
>>> sys.path
['/home/mailman', 'bin', '/usr/local/lib/python1.5/',
'/usr/local/lib/python1.5/plat-sunos5', '/usr/local/lib/python1.5/lib-tk',
'/usr/local/lib/python1.5/lib-dynload']
This means that the relatively-obvious "import mm_cfg" doesn't work.
How come? Things like that *do* seem like the very purpose of
withlist.
> The -r switch could be used
> this way so I think I'll add a #! line at the top. Note that it does
> not help to add -i in the #! line.
>
> -Barry
From bwarsaw@cnri.reston.va.us Tue Apr 4 04:32:49 2000
From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us)
Date: Mon, 3 Apr 2000 23:32:49 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <200004040309.UAA10041@utopia.West.Sun.COM>
Message-ID: <14569.25185.497093.837154@anthem.cnri.reston.va.us>
>>>>> "DM" == Dan Mick writes:
DM> But even when withlist is run from Python, PYTHONPATH/sys.path
DM> doesn't include /home/mailman/Mailman (this is 1.1):
| python -i bin/withlist
| Loading list: (unlocked)
>> sys.path
DM> ['/home/mailman', 'bin', '/usr/local/lib/python1.5/',
DM> '/usr/local/lib/python1.5/plat-sunos5',
DM> '/usr/local/lib/python1.5/lib-tk',
DM> '/usr/local/lib/python1.5/lib-dynload']
DM> This means that the relatively-obvious "import mm_cfg" doesn't
DM> work. How come? Things like that *do* seem like the very
DM> purpose of withlist.
Ah ha, now I understand your problem! Python's packaging system is
involved here too. All the non-top level entry point modules are
supposed to live under the `Mailman' package. In order to be able to
import from the Mailman package, you must have it's parent directory
on sys.path. Thus /home/mailman is on sys.path, and you can then
import Mailman.mm_cfg (or "from Mailman import mm_cfg"). You've gotta
specify the `Mailman' package some place though.
You'll notice that 2.0's withlist says
from Mailman.MailList import MailList
this imports the MailList class from the MailList module inside the
Mailman package.
Hope that helps,
-Barry
From bwarsaw@cnri.reston.va.us Tue Apr 4 04:38:59 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Mon, 3 Apr 2000 23:38:59 -0400 (EDT)
Subject: [Mailman-Developers] Bug#61695: mailman: mail->news gw blows up on mails without subject
(fwd)
References:
Message-ID: <14569.25555.748317.241899@anthem.cnri.reston.va.us>
>>>>> "GM" == Gergely Madarasz writes:
GM> I just got this bugreport in the debian BTS. Comments ?
It's a valid bug that will crash anytime the message is missing a
Subject: header. The easy workaround is to make sure the Subject:
header is present before the message gets posted to the list.
While this bug was still present in 2.0beta1, I've fixed it for
2.0beta2. The code's changed too much to provide a Mailman 1.1 patch,
but here's the patch against 2.0beta1; it should be easy enough to
find in the 1.1 source tree.
Please feel free to forward this to the appropriate debian lists.
-Barry
-------------------- snip snip --------------------
Index: ToUsenet.py
===================================================================
RCS file: /projects/cvsroot/mailman/Mailman/Handlers/ToUsenet.py,v
retrieving revision 1.8
retrieving revision 1.9
diff -c -r1.8 -r1.9
*** ToUsenet.py 2000/03/23 06:48:52 1.8
--- ToUsenet.py 2000/04/04 03:35:05 1.9
***************
*** 58,65 ****
from Mailman.pythonlib import nntplib
# Ok, munge headers, etc.
subj = msg.getheader('subject')
if subj:
- subjpref = mlist.subject_prefix
if not re.match('(re:? *)?' + re.escape(subjpref), subj, re.I):
msg['Subject'] = subjpref + subj
else:
--- 58,65 ----
from Mailman.pythonlib import nntplib
# Ok, munge headers, etc.
subj = msg.getheader('subject')
+ subjpref = mlist.subject_prefix
if subj:
if not re.match('(re:? *)?' + re.escape(subjpref), subj, re.I):
msg['Subject'] = subjpref + subj
else:
From rullfig@uchicago.edu Tue Apr 4 14:51:27 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Tue, 04 Apr 2000 08:51:27 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu> <14569.21918.950338.815597@anthem.cnri.reston.va.us>
Message-ID: <38E9F35F.282051B3@uchicago.edu>
"Barry A. Warsaw" wrote:
>
> >>>>> "RU" == Roberto Ullfig writes:
>
> RU> So, in 1.0rc2, displaying the list of lists for 529 lists
> RU> requires 529**2 = 279841 system stat calls and takes over one
> RU> and a half minutes on our Ultra-2 2x296 processor system! Is
> RU> this because of Python, Mailman, or both? Has this been
> RU> "fixed" in 2.0? You really should only need to make one stat
> RU> call per list.
>
> Uh, it's because of Mailman :)
>
> I implemented a list_lists scripts which does on the command line what
> listinfo.py does in HTML (see attached). Here's what truss -c gives
> me:
>
> -------------------- snip snip --------------------
> Portal - [no description available]
> Postal - [no description available]
> Stage - Staging new Mailman releases
> Test - [no description available]
> syscall seconds calls errors
> _exit .00 1
> read .00 102
> write .00 8
> open .11 607 474
> close .01 143
> time .00 3
> brk .03 227
> stat .03 201 157
> getpid .00 10
> fstat .00 66
> ioctl .02 63 61
> execve .00 10 8
> umask .00 2
> fcntl .00 7
> readlink .00 2 2
> sigprocmask .00 2
> sigaction .00 50
> sigpending .00 1
> mmap .00 42
> mprotect .00 10
> munmap .00 11
> uname .00 4
> sysconfig .00 1
> lwp_create .00 6
> lwp_continue .00 2
> lwp_self .00 3
> llseek .00 114
> door .00 5
> lwp_schedctl .01 5
> getdents64 .01 15
> fstat64 .00 67
> open64 .00 7
> ---- --- ---
> sys totals: .22 1797 702
> usr time: .51
> elapsed: 1.19
> -------------------- snip snip --------------------
>
> Getting the list of list names, requires at least a listdir() and an
> exists() for every directory found there.
>
> Nothing about this will change for 2.0.
>
> -Barry
Thanks for the script.
Now this is the truss output for the listinfo that is called by
driver:
syscall seconds calls errors
_exit .00 1
read .21 1979
write .15 1638
open .12 1233 579
close .06 1189
time .00 1
brk .43 5026
stat 25.58 285877 174
fstat .00 63
ioctl .01 591 589
execve .00 1
umask .00 2
fcntl .02 535
readlink .00 3 2
sigaction .00 48
mmap .00 32
munmap .00 8
llseek .05 643
getdents64 .85 10165
fstat64 .01 1123
open64 .03 535
---- --- ---
sys totals: 27.52 310693 1344
usr time: 54.01
elapsed: 173.76
I can understand a listdir and an exists for each directory;
that should come out to 2 * n stat calls right (~1000 for us).
What I'm saying is that we are seeing n ** 2 stat calls (that's
squared) or 285877 of them. The above truss is from running the
driver manually after setting PYTHONPATH as suggested by Dan
(thanks Dan):
setenv PYTHONPATH /opt/http/opt/pkgs/mailman/Mailman
python /opt/http//opt/pkgs/mailman/scripts/driver listinfo
I've also trussed the running process and gotten similar
results; I see it stat'ing every directory once for every
directory stat'ed or n-squared stats.
Note that this is with 1.0rc2; still waiting for 2.0.
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From rullfig@uchicago.edu Tue Apr 4 15:14:42 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Tue, 04 Apr 2000 09:14:42 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu> <14569.21918.950338.815597@anthem.cnri.reston.va.us> <38E9F35F.282051B3@uchicago.edu>
Message-ID: <38E9F8D2.10F251D5@uchicago.edu>
Roberto Ullfig wrote:
>
> "Barry A. Warsaw" wrote:
> >
> > >>>>> "RU" == Roberto Ullfig writes:
> >
> > RU> So, in 1.0rc2, displaying the list of lists for 529 lists
> > RU> requires 529**2 = 279841 system stat calls and takes over one
> > RU> and a half minutes on our Ultra-2 2x296 processor system! Is
> > RU> this because of Python, Mailman, or both? Has this been
> > RU> "fixed" in 2.0? You really should only need to make one stat
> > RU> call per list.
> >
> > Uh, it's because of Mailman :)
> >
> > I implemented a list_lists scripts which does on the command line what
> > listinfo.py does in HTML (see attached). Here's what truss -c gives
> > me:
> >
> > -------------------- snip snip --------------------
> > Portal - [no description available]
> > Postal - [no description available]
> > Stage - Staging new Mailman releases
> > Test - [no description available]
> > syscall seconds calls errors
> > _exit .00 1
> > read .00 102
> > write .00 8
> > open .11 607 474
> > close .01 143
> > time .00 3
> > brk .03 227
> > stat .03 201 157
> > getpid .00 10
> > fstat .00 66
> > ioctl .02 63 61
> > execve .00 10 8
> > umask .00 2
> > fcntl .00 7
> > readlink .00 2 2
> > sigprocmask .00 2
> > sigaction .00 50
> > sigpending .00 1
> > mmap .00 42
> > mprotect .00 10
> > munmap .00 11
> > uname .00 4
> > sysconfig .00 1
> > lwp_create .00 6
> > lwp_continue .00 2
> > lwp_self .00 3
> > llseek .00 114
> > door .00 5
> > lwp_schedctl .01 5
> > getdents64 .01 15
> > fstat64 .00 67
> > open64 .00 7
> > ---- --- ---
> > sys totals: .22 1797 702
> > usr time: .51
> > elapsed: 1.19
> > -------------------- snip snip --------------------
> >
> > Getting the list of list names, requires at least a listdir() and an
> > exists() for every directory found there.
> >
> > Nothing about this will change for 2.0.
> >
> > -Barry
>
> Thanks for the script.
>
> Now this is the truss output for the listinfo that is called by
> driver:
>
> syscall seconds calls errors
> _exit .00 1
> read .21 1979
> write .15 1638
> open .12 1233 579
> close .06 1189
> time .00 1
> brk .43 5026
> stat 25.58 285877 174
> fstat .00 63
> ioctl .01 591 589
> execve .00 1
> umask .00 2
> fcntl .02 535
> readlink .00 3 2
> sigaction .00 48
> mmap .00 32
> munmap .00 8
> llseek .05 643
> getdents64 .85 10165
> fstat64 .01 1123
> open64 .03 535
> ---- --- ---
> sys totals: 27.52 310693 1344
> usr time: 54.01
> elapsed: 173.76
And this is the truss from using the script you sent:
syscall seconds calls errors
_exit .00 1
read .29 1963
write .03 534
open .13 1132 485
close .00 1182
time .00 1
brk .41 4944
stat 28.32 285849 148
fstat .00 61
ioctl .02 586 584
execve .01 3 1
fcntl .00 535
readlink .00 3 2
sigaction .01 48
mmap .00 40
munmap .01 10
llseek .05 632
getdents64 .92 10165
fstat64 .02 1118
open64 .06 535
---- --- ---
sys totals: 30.28 309342 1220
usr time: 56.08
elapsed: 182.84
All those stat calls just don't seem right to me. Using python
1.5.2 if that matters.
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From secabeen@pobox.com Tue Apr 4 15:46:43 2000
From: secabeen@pobox.com (Ted Cabeen)
Date: Tue, 04 Apr 2000 09:46:43 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
In-Reply-To: Your message of "Tue, 04 Apr 2000 09:14:42 CDT."
<38E9F8D2.10F251D5@uchicago.edu>
Message-ID: <200004041446.JAA03573@entropy.uchicago.edu>
In message <38E9F8D2.10F251D5@uchicago.edu>, Roberto Ullfig writes:
>Roberto Ullfig wrote:
>>
>> "Barry A. Warsaw" wrote:
>> >
>> > >>>>> "RU" == Roberto Ullfig writes:
>> >
>> > RU> So, in 1.0rc2, displaying the list of lists for 529 lists
>> > RU> requires 529**2 = 279841 system stat calls and takes over one
>> > RU> and a half minutes on our Ultra-2 2x296 processor system! Is
>> > RU> this because of Python, Mailman, or both? Has this been
>> > RU> "fixed" in 2.0? You really should only need to make one stat
>> > RU> call per list.
>> >
>> > Uh, it's because of Mailman :)
>> >
>> > I implemented a list_lists scripts which does on the command line what
>> > listinfo.py does in HTML (see attached). Here's what truss -c gives
>> > me:
>> > Getting the list of list names, requires at least a listdir() and an
>> > exists() for every directory found there.
>> >
>> > Nothing about this will change for 2.0.
>> >
>> > -Barry
>>
>> Thanks for the script.
>>
>> Now this is the truss output for the listinfo that is called by
>> driver:
Here's what is happening. When listinfo runs and has to get the list of
advertised addresses, it starts by getting a list of mailing lists on the
server using Utils.list_names(). Utils.list_names() requires two stat calls
for each list on the machine every time it is called. This is
understandable and isn't going to change. It then proceeds to open every one
of those lists to check on the advertised flag. Again, no problem. The
problem is that when Mailman opens a List in the MailList constructor
__init__, it checks to makes sure that the list exists by running
Utils.list_names() and seeing if the list name requested is there. Therefore
every request to open a list requires two stat calls on every list on the
system. Therefore when we are sequentially opening every list on the system
in listinfo.py we get a squaring effect on stat calls in the list directory.
Solutions are reasonably easy to code, the first of which comes to mind is a
optional argument to the constructor that indicated that the name has already
been checked and that checking it again is not necessary. Other solutions
include caching the list of lists on the server, but this means there is a
delay between when the list is created and when it becomes accessible.
I can code something up for you if necessary, but it seems like a reasonably
simple patch. Do either you or Barry need a patch? Let me know if you do.
--
Ted Cabeen http://www.pobox.com/~secabeen secabeen@pobox.com
Check Website or finger for PGP Public Key secabeen@midway.uchicago.edu
"I have taken all knowledge to be my province." -F. Bacon cococabeen@aol.com
"Human kind cannot bear very much reality."-T.S.Eliot 73126.626@compuserve.com
From bwarsaw@cnri.reston.va.us Tue Apr 4 18:06:35 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Tue, 4 Apr 2000 13:06:35 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E8B23F.16F25CE7@uchicago.edu>
<14569.21918.950338.815597@anthem.cnri.reston.va.us>
<38E9F35F.282051B3@uchicago.edu>
<38E9F8D2.10F251D5@uchicago.edu>
Message-ID: <14570.8475.832742.629404@anthem.cnri.reston.va.us>
>>>>> "RU" == Roberto Ullfig writes:
RU> All those stat calls just don't seem right to me. Using python
RU> 1.5.2 if that matters.
Shouldn't. I'm seeing approximately n**2 stat calls too. It'll take
some investigating to figure out what's going on, and I'm not sure
it'll happen in time for 2.0.
-Barry
From bwarsaw@cnri.reston.va.us Tue Apr 4 18:13:32 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Tue, 4 Apr 2000 13:13:32 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E9F8D2.10F251D5@uchicago.edu>
<200004041446.JAA03573@entropy.uchicago.edu>
Message-ID: <14570.8892.540826.507453@anthem.cnri.reston.va.us>
>>>>> "TC" == Ted Cabeen writes:
TC> Here's what is happening.
Doh! Thanks for finding this Ted. To be honest, I've looked at that
constructor hundreds of times and my eyes just parsed right over it.
I think the right thing to do may be to just let the MMBadListError
percolate up through the Load() call and just zap the test for
MMUnknownListError. I'll have to grep through the code though to see
if that would cause other problems. I suspect not, because I remember
basically having to catch both errors all the time anyway.
Again, thanks for the sleuthing!
-Barry
From dan.mick@West.Sun.COM Tue Apr 4 22:01:40 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Tue, 04 Apr 2000 14:01:40 -0700
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <200004040309.UAA10041@utopia.West.Sun.COM> <14569.25185.497093.837154@anthem.cnri.reston.va.us>
Message-ID: <38EA5834.F731FAF6@west.sun.com>
bwarsaw@cnri.reston.va.us wrote:
> Thus /home/mailman is on sys.path, and you can then
> import Mailman.mm_cfg (or "from Mailman import mm_cfg"). You've gotta
> specify the `Mailman' package some place though.
OK, gotcha. I figured "withlist" would have established the convention
that Mailman modules were the "context", since I've already got a list
open...but I wasn't thinking in concert with you. Now I am; thanks.
From bwarsaw@cnri.reston.va.us Tue Apr 4 22:25:19 2000
From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us)
Date: Tue, 4 Apr 2000 17:25:19 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <200004040309.UAA10041@utopia.West.Sun.COM>
<14569.25185.497093.837154@anthem.cnri.reston.va.us>
<38EA5834.F731FAF6@west.sun.com>
Message-ID: <14570.23999.300339.154042@anthem.cnri.reston.va.us>
Cool.
From bwarsaw@cnri.reston.va.us Wed Apr 5 00:36:27 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Tue, 4 Apr 2000 19:36:27 -0400 (EDT)
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E9F8D2.10F251D5@uchicago.edu>
<200004041446.JAA03573@entropy.uchicago.edu>
Message-ID: <14570.31867.956811.471392@anthem.cnri.reston.va.us>
Ted's analysis was right on-target, however the right fix inspired me
to do more hacking than expected. The good news is that for the 64
lists on python.org, the number of stat calls bin/list_lists does has
just dropped from 4453 to 357 (with the same number of errors in both:
214). This should be a big win for your huge lists.
I don't have a patch handy, but you can either check out the CVS
snapshot or wait for beta2. No ETA there, but RRRSN :)
-Barry
From dan.mick@West.Sun.COM Wed Apr 5 11:24:42 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Wed, 05 Apr 2000 03:24:42 -0700
Subject: [Mailman-Developers] Speaking of pathological behavior...
Message-ID: <38EB146A.CF88B246@west.sun.com>
Now that I'm in the same room with my Mailman server, I notice
that, at least after setting admin_user_chunksize to 50 (from
the default 30) that changing one user's "mailto" flag from off
to on (to disable a user by hand temporarily) beats the living *hell*
out of the disk, and server. Something bad wrong is going on there;
I truss'ed it once and it was continually doing something like
opening a file, writing it, and closing it. I don't know what.
Something's wrong though. I haven't started to try to look at the
form-processing code to see what happens.
From dan.mick@West.Sun.COM Wed Apr 5 11:39:48 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Wed, 05 Apr 2000 03:39:48 -0700
Subject: [Mailman-Developers] Speaking of pathological behavior...
References: <38EB146A.CF88B246@west.sun.com>
Message-ID: <38EB17F4.6438AF52@west.sun.com>
Dan Mick wrote:
>
> Now that I'm in the same room with my Mailman server, I notice
> that, at least after setting admin_user_chunksize to 50 (from
> the default 30) that changing one user's "mailto" flag from off
> to on (to disable a user by hand temporarily) beats the living *hell*
> out of the disk, and server. Something bad wrong is going on there;
> I truss'ed it once and it was continually doing something like
> opening a file, writing it, and closing it. I don't know what.
>
> Something's wrong though. I haven't started to try to look at the
> form-processing code to see what happens.
Specifically, it looks a lot like both of these steps happen for every
member:
1) the archive links are made and unmade
2) the entire config.db.tmp file is rewritten, start to finish
These may be "safety" things, but they're pretty huge performance kills,
especially if, say, one attribute on one member is changing. Seems like a good
place for a scalability win. Barry, did you rework this code at all for 2.0beta1?
From dan.mick@West.Sun.COM Wed Apr 5 11:43:49 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Wed, 05 Apr 2000 03:43:49 -0700
Subject: [Mailman-Developers] Speaking of pathological behavior...
References: <38EB146A.CF88B246@west.sun.com> <38EB17F4.6438AF52@west.sun.com>
Message-ID: <38EB18E5.92C7B8F2@west.sun.com>
Dan Mick wrote:
> Specifically, it looks a lot like both of these steps happen for every
> member:
>
> 1) the archive links are made and unmade
> 2) the entire config.db.tmp file is rewritten, start to finish
Sorry, let me reword that one last time:
1) the archive links are remade, even though they exist (probably not a big deal)
2) config.db.tmp is written, and then .db becomes .last and .tmp becomes .db.
Probably batching up all the "changes" into one config.db write is a big
big win.
From dan.mick@West.Sun.COM Wed Apr 5 11:54:14 2000
From: dan.mick@West.Sun.COM (Dan Mick)
Date: Wed, 05 Apr 2000 03:54:14 -0700
Subject: [Mailman-Developers] Answering my own question
Message-ID: <38EB1B56.8E4096E6@west.sun.com>
Adding a "save_list=0" (or just a fourth argument '0') to the two calls
to SetUserOption in admin.py speeds up that "set one option" flag a
*whole* lot. It might be that similar "optimizations" to the other
forms processing (DeleteMember, primarily) are a bit unsafe, but it seems
like a huge win for the SetUserOption calls, and not very unsafe, since
the list will be saved after processing all users anyway.
From rullfig@uchicago.edu Wed Apr 5 13:57:38 2000
From: rullfig@uchicago.edu (Roberto Ullfig)
Date: Wed, 05 Apr 2000 07:57:38 -0500
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
References: <38E9F8D2.10F251D5@uchicago.edu>
<200004041446.JAA03573@entropy.uchicago.edu> <14570.31867.956811.471392@anthem.cnri.reston.va.us>
Message-ID: <38EB3842.8BE90E3D@uchicago.edu>
"Barry A. Warsaw" wrote:
>
> Ted's analysis was right on-target, however the right fix inspired me
> to do more hacking than expected. The good news is that for the 64
> lists on python.org, the number of stat calls bin/list_lists does has
> just dropped from 4453 to 357 (with the same number of errors in both:
> 214). This should be a big win for your huge lists.
>
> I don't have a patch handy, but you can either check out the CVS
> snapshot or wait for beta2. No ETA there, but RRRSN :)
>
Thanks, will probably wait until after we install 2 to patch
anything (if it isn't in 2). Note, that this affects the Admin
list page too.
We've got an hourly cronjob creating the file; the only problem
is getting the listinfo URL to display that html instead of
running listinfo. For now we just have an alternate URL for the
pre-generated list of lists.
--
Roberto Ullfig : rullfig@uchicago.edu
Systems Administrator
Networking Services and Information Technologies
University of Chicago
From aletzner@germany.sun.com Wed Apr 5 16:57:39 2000
From: aletzner@germany.sun.com (Adrian Letzner - Sun Germany Berlin SE)
Date: Wed, 05 Apr 2000 17:57:39 +0200
Subject: [Mailman-Developers] security szenario possible?
Message-ID: <38EB6273.79DFFF98@germany.sun.com>
i would like to know, if mailman can handle the following szenario:
1. automatic subscribing to a fixed list (eg. via cgi script). that
means:
a program (eg. cgi-script) should handle the subscribing/unsubscribing
mechanism by sending a static mail to the *-request address WITHOUT (!!)
using the password mechanism (new privacy option: *not confirming).
2. deleting the mail-header to anonymisize the mails which will be
posted.
the reasons for that is my plan to have an automatic anonymous mailing
list defined, where only the administrator will post messages.
please answer directly to me, cause i'm not on the list.
thanks a lot and sorry for my english
--
*******************************************************************
Adrian Letzner
ISV Partner SE Tel: (++49 30) 74 70 96 65
Sun Microsystems GmbH Fax: (++49 30) 74 70 96 99
Lankwitzer Str. 19 mailto:adrian.letzner@germany.sun.com
12107 Berlin http://www.sun.de
*******************************************************************
From bwarsaw@cnri.reston.va.us Thu Apr 6 00:40:10 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Wed, 5 Apr 2000 19:40:10 -0400 (EDT)
Subject: [Mailman-Developers] check_perms bug?...
References: <200003230624.WAA12783@utopia.West.Sun.COM>
Message-ID: <14571.52954.933150.960005@anthem.cnri.reston.va.us>
>>>>> "DM" == Dan Mick writes:
DM> Somehow, my /home/mailman/lists directory managed to get set
DM> with permissions check_perms didn't check for; it was set to
DM> d-wx-ws--x/mailman/mailman, which caused mailcmd to be unable
DM> to list the directory (no read perms).
DM> I fixed it once I figured it out, but I had a lot of
DM> check_perms runs in there before I finally figured it
DM> out...mostly because I was confused about what 'x' really
DM> means on a directory. But check_perms could have saved me
DM> time if it had checked for 'owner-and-group-read'; I think it
DM> should.
You're right. Here's the check_perms that will be in 2.0beta2.
-Barry
-------------------- snip snip --------------------
#! /usr/bin/env python
#
# Copyright (C) 1998,1999,2000 by the Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""Check the permissions for the Mailman installation.
Usage: %(PROGRAM)s [-f] [-v] [-h]
With no arguments, just check and report all the files that have bogus
permissions or group ownership. With -f (and run as root), fix all the
permission problems found. With -v be verbose.
"""
import sys
import os
import errno
import getopt
import grp
from stat import *
import paths
from Mailman import mm_cfg
MAILMAN_GRPNAME = 'mailman'
MAILMAN_GID = grp.getgrnam(MAILMAN_GRPNAME)[2]
PROGRAM = sys.argv[0]
class State:
FIX = 0
VERBOSE = 0
ERRORS = 0
STATE = State()
DIRPERMS = S_ISGID | S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH
def statmode(path):
return os.stat(path)[ST_MODE]
def statgidmode(path):
stat = os.stat(path)
return stat[ST_MODE], stat[ST_GID]
def checkwalk(arg, dirname, names):
for name in names:
path = os.path.join(dirname, name)
if arg.VERBOSE:
print 'checking gid and mode for', path
try:
mode, gid = statgidmode(path)
except os.error, (code, msg):
if code == errno.ENOENT:
continue
raise
if gid <> MAILMAN_GID:
try:
groupname = grp.getgrgid(gid)[0]
except KeyError:
groupname = '' % gid
arg.ERRORS = arg.ERRORS + 1
print path, 'bad gid (has: %s, expected %s)' % (
groupname, MAILMAN_GRPNAME),
if STATE.FIX:
print '(fixing)'
os.chown(path, -1, MAILMAN_GID)
else:
print
# all directories must be at least rwxrwsr-x. Don't check the private
# archive directory or database directory themselves since these are
# checked in checkarchives below.
private = mm_cfg.PRIVATE_ARCHIVE_FILE_DIR
if path == private or (os.path.commonprefix((path, private)) == private
and os.path.split(path)[1] == 'database'):
continue
if S_ISDIR(mode) and (mode & DIRPERMS) <> DIRPERMS:
arg.ERRORS = arg.ERRORS + 1
print 'directory must be at least 02775:', path,
if STATE.FIX:
print '(fixing)'
os.chmod(path, mode | DIRPERMS)
else:
print
def checkall():
# first check PREFIX
if STATE.VERBOSE:
print 'checking mode for', mm_cfg.PREFIX,
mode = statmode(mm_cfg.PREFIX)
if (mode & DIRPERMS) <> DIRPERMS:
STATE.ERRORS = STATE.ERRORS + 1
print 'directory must be at least 02775:', mm_cfg.PREFIX,
if STATE.FIX:
print '(fixing)'
os.chmod(mm_cfg.PREFIX, mode | DIRPERMS)
else:
print
# check all subdirs
os.path.walk(mm_cfg.PREFIX, checkwalk, STATE)
def checkarchives():
private = mm_cfg.PRIVATE_ARCHIVE_FILE_DIR
if STATE.VERBOSE:
print 'checking perms on', private
# private archives must not be other readable
mode = statmode(private)
if mode & S_IROTH:
STATE.ERRORS = STATE.ERRORS + 1
print private, 'must not be other-readable',
if STATE.FIX:
print '(fixing)'
os.chmod(private, mode & ~S_IROTH)
else:
print
def checkarchivedbs():
# The archives/private/listname/database file must not be other readable
# or executable otherwise those files will be accessible when the archives
# are public. That may not be a horrible breach, but let's close this off
# anyway.
for dir in os.listdir(mm_cfg.PRIVATE_ARCHIVE_FILE_DIR):
if dir[-5:] == '.mbox':
continue
dbdir = os.path.join(mm_cfg.PRIVATE_ARCHIVE_FILE_DIR, dir, 'database')
try:
mode = statmode(dbdir)
except os.error, (code, msg):
if code == errno.ENOENT:
continue
raise
if mode & S_IRWXO:
STATE.ERRORS = STATE.ERRORS + 1
print dbdir, 'must be other 000',
if STATE.FIX:
print '(fixing)'
os.chmod(dbdir, mode & ~S_IRWXO)
else:
print
def checkcgi():
exes = os.listdir(mm_cfg.CGI_DIR)
for f in exes:
path = os.path.join(mm_cfg.CGI_DIR, f)
if STATE.VERBOSE:
print 'checking set-gid for', path
mode = statmode(path)
if mode & S_IXGRP and not mode & S_ISGID:
STATE.ERRORS = STATE.ERRORS + 1
print path, 'must be set-gid',
if STATE.FIX:
print '(fixing)'
os.chmod(path, mode | S_ISGID)
else:
print
def checkmail():
wrapper = os.path.join(mm_cfg.WRAPPER_DIR, 'wrapper')
if STATE.VERBOSE:
print 'checking set-gid for', wrapper
mode = statmode(wrapper)
if not mode & S_ISGID:
STATE.ERRORS = STATE.ERRORS + 1
print wrapper, 'must be set-gid',
if STATE.FIX:
print '(fixing)'
os.chmod(wrapper, mode | S_ISGID)
def checkadminpw():
adminpw = os.path.join(mm_cfg.DATA_DIR, 'adm.pw')
targetmode = S_IFREG | S_IRUSR | S_IWUSR | S_IRGRP
if STATE.VERBOSE:
print 'checking perms on', adminpw
try:
mode = statmode(adminpw)
except os.error, (code, msg):
# adm.pw may not exist
if code == errno.ENOENT:
return
raise
if mode <> targetmode:
STATE.ERRORS = STATE.ERRORS + 1
print adminpw, 'permissions must be exactly 0640 (got %s)' % oct(mode)
if STATE.FIX:
print '(fixing)'
os.chmod(adminpw, targetmode)
def usage(code=0, msg=''):
print __doc__ % globals()
if msg:
print msg
sys.exit(code)
if __name__ == '__main__':
try:
opts, args = getopt.getopt(sys.argv[1:],
'fvh',
['fix', 'verbose', 'help'])
except getopt.error, msg:
usage(1, msg)
for opt, arg in opts:
if opt in ('-h', '--help'):
usage()
elif opt in ('-f', '--fix'):
STATE.FIX = 1
elif opt in ('-v', '--verbose'):
STATE.VERBOSE = 1
checkall()
checkarchives()
checkarchivedbs()
checkcgi()
checkmail()
checkadminpw()
if not STATE.ERRORS:
print 'No problems found'
else:
print 'Problems found:', STATE.ERRORS
print 'Re-run as root with -f flag until no errors are found'
From jarrell@vt.edu Thu Apr 6 14:21:42 2000
From: jarrell@vt.edu (Ron Jarrell)
Date: Thu, 06 Apr 2000 09:21:42 -0400
Subject: [Mailman-Developers] 2.0b2 status
Message-ID: <4.3.1.2.20000406091804.05717de0@vtserf.cc.vt.edu>
Updated both my production and test servers (no, not in that order) to
whatever was in cvs this morning, which at least has me stamped as 2.0beta2...
Seems to be working fine, including news gating... I'm watching the logs.
Did have one weirdness where the production server was giving me tracebacks
on the listinfo page.. Which went away after I tried it a couple of times, while
check_perms was running... check_perms found nothing wrong, and the script
"healed" itself. Bizarre...
From dan@auctionwatch.com Fri Apr 7 02:25:18 2000
From: dan@auctionwatch.com (Dan Chen)
Date: Thu, 6 Apr 2000 18:25:18 -0700
Subject: [Mailman-Developers] User list in mysql
Message-ID:
My company has tasked me to port our existing mailing list to mailman. The
requirement is that the list of subscribers has to be kept in mysql. (We
have scripts and reports that use this list).
I'm a python newbie, so please bear with me.
I've taken a look at MailList.py and understand that the subscriber list and
all other data is stored as a marshal in config.db. And this is stored in
the methods Load and Save.
I'm proposing that when Save is called, it also makes changes to the mysql
db. And after a Load, it queries the db. This sounds very expensive, and
I'm wondering if developers here who know Mailman and python better would
have a better scheme. Also if I'm missing anything in my analysis.
thanks
dan
From bwarsaw@cnri.reston.va.us Fri Apr 7 05:29:45 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Fri, 7 Apr 2000 00:29:45 -0400 (EDT)
Subject: [Mailman-Developers] cron/gate_news watermark == 0
References: <20000213182801.A15474@mail.dskk.co.jp>
Message-ID: <14573.25657.617106.187767@anthem.cnri.reston.va.us>
Back in February, there was some discussion about fixing gate_news and
the gate_watermarks file...
>>>>> "JT" == Jim Tittsler writes:
JT> A watermark value of 0 is used for two different purposes in
JT> cron/gate_news [...] The result of this is the first article
JT> posted to the newly created newsgroup fails to get gatewayed
JT> to the mailing list because the watermark for that group is
JT> (still) 0 which is used as a flag to "catch up" so all that
JT> happens is the watermark gets set to 1.
The jist of your message is spot on, and I took your advice to make
None the marker that means the list hasn't been gated before and/or
needs a mass catchup.
>>>>> "HM" == Harald Meland writes:
HM> Hmmmm... When checking the contents of my
HM> "data/gate_watermarks" marshal, I find several old mailing
HM> lists with an entry stating that their watermark is 0 -- even
HM> though these lists aren't actually gating any newsgroups.
HM> [...]
HM> Or, even better, move the per-list watermark info from
HM> gate_watermarks into the list's config.db, replacing any 0
HM> values in gate_watermarks with None values in config.db. This
HM> has the advantage of reducing the number of different files
HM> that would have to be changed when cloning/renaming a list.
Makes tons of sense Harald! I like this idea a lot for your reasons,
but also because it leads to a big simplification of gate_news. It's
actually harder to synchronize access to a shared gate_watermarks file
than it is to just keep the watermark in the list's attributes.
JT> This seems logical to me, since I think the administrator
JT> really wants to do the "catch up" any time gateway_to_mail
JT> transitions from No to Yes (implying that it needs to be able
JT> to update the watermark, either in config.db or by locking(?)
JT> and updating gate_watermarks).
Another excellent point. I've got a hack to the already horrible
admin.py to None-ify the watermark whenever the gateway_to_mail
property has changed. In reality we zap the watermark when it
transitions to either "yes" or "no", but I don't /think/ that matters
in practice. You could potentially want only watermark zapping on
transition from no->yes so that you could write a little script to
turn gatewaying on while preserving the watermark, but I just don't
think this will be a common enough occurance. The Right Thing To Do
is probably to improve the admin page for gatewaying so that there's a
flag the admin can tickle to zap the watermark or preserve it. Not
for 2.0 :)
So now the watermark is stored in the list's config.db under the
usenet_watermark attribute. gate_news O_CREAT|O_EXCL's a block file
to ensure that only one gate_news process is running at any time, and
it only deletes this block file when all it's forked children exit (or
on catastrophic exception). gate_news opens each list in turn
unlocked, and then if the list is gating to mail, it forks a process
for that list and immediately locks it. Then it gates the list
largely as before. Similar to before, if the watermark is None it
simply sets it to the last known article number and moves on,
expecting the following gate_news to post subsequent messages.
I also implemented your suggestion to bin/update to pull the info out
of gate_watermarks and stick them in mlist.usenet_watermark. It then
deletes gate_watermarks.
An outgrowth of all this is that I realized I really want a numeric,
easily comparable Mailman version number, and a way to figure out what
version we are upgrading from. That's surprisingly difficult now, but
it would have been darn handy for bin/update. I've now implemented a
HEX_VERSION using the same scheme that Python does, e.g. Mailman
2.0beta2 will have a HEX_VERSION of 0x200000b2, and this can be
compared against for running only selective updates in bin/update.
Every time bin/update completes successfully, it writes the current
hex version string to data/last_mailman_version.
[There's one minor icky bit: the warning about moving
templates/options.html is only printed if 1) we can't determine the
previous Mailman version, and 2) it is not a fresh install -- the
determinant being any files in the logs subdir]
I think that's it. I've done some moderate testing of all this, but
not on my production machines. I'm going to check it all in, and hope
you take a look, but be aware I haven't banged on it terribly hard.
Slowly-catching-up-ly y'rs,
-Barry
From bwarsaw@cnri.reston.va.us Fri Apr 7 05:31:53 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Fri, 7 Apr 2000 00:31:53 -0400 (EDT)
Subject: [Mailman-Developers] race condition in locking ?
References: <20000202140042.J19725@xs4all.nl>
<20000202225346.B17313@xs4all.nl>
<14488.44469.446995.168328@anthem.cnri.reston.va.us>
<20000203103555.C6378@xs4all.nl>
<20000207224457.T23471@xs4all.nl>
Message-ID: <14573.25785.169936.426535@anthem.cnri.reston.va.us>
>>>>> "HM" == Harald Meland writes:
>> Ah, well, I couldn't find any parts of Mailman in the CVS tree
>> that acually _called_ steal, so I couldn't figure out what it
>> should do in borderline cases.
HM> The only place I know of is cron/gate_news.
And even that no longer does (see my previous long post). I'm still
not comfortable with radical changes to LockFile.py at this late stage
of the game, but will re-examine this whole thread after 2.0 final.
-Barry
From bwarsaw@cnri.reston.va.us Fri Apr 7 05:34:50 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Fri, 7 Apr 2000 00:34:50 -0400 (EDT)
Subject: [Mailman-Developers] One more thing about bin/update...
Message-ID: <14573.25962.560643.516056@anthem.cnri.reston.va.us>
... What do you think about running it by default on a "make install"?
With the gate_news changes, you'll really want to be sure to run it.
I guess this is another case of helping lazy users who don't read or
follow directions. Am I trying to pamper them too much?
-Barry
From ricardo@rixhq.nu Fri Apr 7 07:28:08 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Fri, 7 Apr 2000 08:28:08 +0200
Subject: [Mailman-Developers] User list in mysql
In-Reply-To: ; from dan@auctionwatch.com on Thu, Apr 06, 2000 at 06:25:18PM -0700
References:
Message-ID: <20000407082808.A20142@miss-janet.com>
On Thu, Apr 06, 2000 at 06:25:18PM -0700, Dan Chen wrote:
> I've taken a look at MailList.py and understand that the subscriber list and
> all other data is stored as a marshal in config.db. And this is stored in
> the methods Load and Save.
> I'm proposing that when Save is called, it also makes changes to the mysql
> db. And after a Load, it queries the db. This sounds very expensive, and
yes I'd love to see mailman support mysql too...but I think the best thing
to do is moving the userlist to a seperate marshal db and abstracting the
access code to make it easy to replace it with your own favorite sql server.
from what i've seen in the mailman code is that there have to be made a lot
of changes to support something like this... so maybe we'll have to wait till
2.0 gets released and work on it from there.
in short, these are my thoughts on how this could be handled:
1) rewrite mailman to modify/read the user list through single function calls
2) create a module with those functions
3) create a module that is used to write the userlist to a marshal db
4) create a module to write the userlist to a mysql db
5) create a module to write the userlist to ... etc
Ricardo.
--
From Harald.Meland@usit.uio.no Sat Apr 8 22:48:42 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 08 Apr 2000 23:48:42 +0200
Subject: [Mailman-Developers] Listing Lists Faster in 2.0?
In-Reply-To: Roberto Ullfig's message of "Mon, 03 Apr 2000 11:26:19 -0500"
References: <38E8B23F.16F25CE7@uchicago.edu> <38E8C62B.37883725@uchicago.edu>
Message-ID:
[Roberto Ullfig]
> Roberto Ullfig wrote:
> >
> > So, in 1.0rc2, displaying the list of lists for 529 lists
> > requires 529**2 = 279841 system stat calls and takes over one
> > and a half minutes on our Ultra-2 2x296 processor system! Is
> > this because of Python, Mailman, or both? Has this been "fixed"
> > in 2.0? You really should only need to make one stat call per
> > list.
> >
>
> Whatever the answer is, we'd like to be able to generate the
> lists of lists once a day;
Just a quick note: In the (severely hacked-up) Mailman installation we
have here at uio.no (with ~3500 lists, although these are spread out
over ~150 virtual domains), I've implemented caching of the data
Mailman needs to generate the "list overview" pages, thereby avoiding
excessive stat()ing.
This is one of the features I'm holding off committing until after 2.0
is out.
--
Harald
From bwarsaw@cnri.reston.va.us Sat Apr 8 22:54:44 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Sat, 8 Apr 2000 17:54:44 -0400 (EDT)
Subject: [Mailman-Developers] [ANNOUNCE] Mailman 2.0 beta 2
Message-ID: <14575.43684.735692.298399@anthem.cnri.reston.va.us>
I've just uploaded the Mailman 2.0 beta 2 tarball to the following
locations:
http://www.list.org/mailman.tar.gz
http://download.sourceforge.net/mailman/mailman-2.0beta2.tgz
ftp://ftp.python.org/pub/mailman/mailman-2.0beta2.tgz
See the UPGRADING file for details on upgrading from 1.1 or 2.0beta1.
See the NEWS file for a complete description of changes for every
version; below is an excerpt for changes since 2.0beta1.
Barring any showstopping bugs, this will be the last release until I
finish my taxes. :)
Enjoy,
-Barry
-------------------- snip snip --------------------
- Rewritten gate_news cron script which should be more efficient and
avoid race and locking problems. Each list now maintains its own
watermark, and when you use the admin CGI script to turn on gating
from Usenet->mail, an automatic mass catch up is done to avoid
flooding the mailing list. cron/gate_news's command line interface
has also changed. See its docstring for details.
- A new cron script called qrunner has been added to retry message
deliveries that fail because of temporary smtpd problems.
- New command line script called bin/list_lists which does exactly
that: lists all the mailing lists on the system (much like the
listinfo CGI does).
- bin/withlist is now directly executable, however if you want to use
python -i, you must still explicitly invoke it. bin/withlist also
now cleans up after itself by unlocking any locked lists. It does
NOT save any dirty lists though - you must do this explicitly.
- $prefix permissions (and all subdirs) must now be 02775.
bin/check_perms has been updated to fix all the subdir permissions.
- "make update" (a.k.a. bin/update) is run automatically when you do a
"make install"
- The CGI driver script now puts information about the Python
environment into the logs/error file (but not the diagnostic web
page).
- Bug fixes and some performance improvements
From thomas@xs4all.net Sat Apr 8 23:31:04 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sun, 9 Apr 2000 00:31:04 +0200
Subject: [Mailman-Developers] [ANNOUNCE] Mailman 2.0 beta 2
In-Reply-To: <14575.43684.735692.298399@anthem.cnri.reston.va.us>; from bwarsaw@cnri.reston.va.us on Sat, Apr 08, 2000 at 05:54:44PM -0400
References: <14575.43684.735692.298399@anthem.cnri.reston.va.us>
Message-ID: <20000409003103.G13830@xs4all.nl>
On Sat, Apr 08, 2000 at 05:54:44PM -0400, Barry A. Warsaw wrote:
> Barring any showstopping bugs, this will be the last release until I
> finish my taxes. :)
There's one bug that would really like to be squashed before 2.0, I think:
the 'from ' line in the downloadable mailbox (the stripped version of the
pipermail archive mailbox) that carries the wrong date format. I posted a
patch a week ago or so, so that the downloadable mailboxes are at least
usable. The only possible problem is that ctime() might generate different
output on different locales, but I dont think so, and the manpage for
ctime() on Linux and BSDI dont mention it. Drop me a line if I need to
repost it. (Btw, should I post this bug to the jitterbug page, even if I
have a patch for it ?)
There are some other problems with the mailboxes, for instance that in
squashing the headers, the references line gets squashed too, but I'll save
those for later ;)
I also have a few changes locally, that might be included later (if
desirable, of course): partial rewrite of hyperarch/pipermail, actually losing the
latter (no functional changes, but a lot less hairy code, if I say so
myself), unsubscription-approval, like subscription (posted a buggy version
before) and some hacks to allow members of one list to post to another,
moderated list they are not members of.
I'm also pondering making members of a list that are themselves lists, a
special case, to skip the password-sending, allow posting, dont screw the
subject line too much, etc. (Those things would solve a lot of the minor
annoyances in some of our lists.) But all that might be a lot easier if that
'real user database' gets included ;)
--
Thomas Wouters
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
From bwarsaw@cnri.reston.va.us Sat Apr 8 23:50:58 2000
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw)
Date: Sat, 8 Apr 2000 18:50:58 -0400 (EDT)
Subject: [Mailman-Developers] [ANNOUNCE] Mailman 2.0 beta 2
References: <14575.43684.735692.298399@anthem.cnri.reston.va.us>
<20000409003103.G13830@xs4all.nl>
Message-ID: <14575.47058.314702.208061@anthem.cnri.reston.va.us>
>>>>> "TW" == Thomas Wouters writes:
>> Barring any showstopping bugs, this will be the last release
>> until I finish my taxes. :)
TW> There's one bug that would really like to be squashed before
TW> 2.0, I think
2.0beta2 definitely won't be the last release before 2.0 final. I
just don't think I'm going to have many free evenings to hack on this
stuff until after April 17th :).
I don't think you need to repost your patches; I'll be cruising
through as many of the outstanding messages and Jitterbug reports as
possible before 2.0 final.
I'm totally psyched to have you rewrite/fix/adopt/expend great amounts
of effort on the archiver, but I'll have to hold off on integrating
that stuff until after 2.0 final.
thanks,
-Barry
From brian@SoftHome.net Sun Apr 9 04:56:41 2000
From: brian@SoftHome.net (Brian Grossman)
Date: Sat, 08 Apr 2000 21:56:41 -0600
Subject: [Mailman-Developers] observations from new user
Message-ID: <20000409035641.7523.qmail@lindy.softhome.net>
I'm evaluating mailman to replace my majordomo v1 installation.
Looks good so far. Nice work.
One general observation: Why does DEFAULT_PRIVATE_ROSTER default to 0? I
would think that 2 would be a better choice.
I'm using this with qmail. 2.0beta2 seems to work well with qmail 1.03.
Where should I send a script that eliminates the need for five .qmail
files per list? This one is called from .qmail-default.
I'm not subscribed to the list, so please cc me on replies.
Brian
From ricardo@rixhq.nu Sun Apr 9 10:21:41 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sun, 9 Apr 2000 11:21:41 +0200
Subject: [Mailman-Developers] observations from new user
In-Reply-To: <20000409035641.7523.qmail@lindy.softhome.net>; from brian@SoftHome.net on Sat, Apr 08, 2000 at 09:56:41PM -0600
References: <20000409035641.7523.qmail@lindy.softhome.net>
Message-ID: <20000409112141.A11416@miss-janet.com>
Hi,
On Sat, Apr 08, 2000 at 09:56:41PM -0600, Brian Grossman wrote:
> Where should I send a script that eliminates the need for five .qmail
> files per list? This one is called from .qmail-default.
on this list would be a right place i guess :)
at work i've installed mailman on a qmail server and created
a script that creates the 5 .qmail files by only supplying a
listname... but I think your solution sounds better...
is it a shell or perl script?
Ricardo.
--
From Harald.Meland@usit.uio.no Sun Apr 9 19:19:49 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 09 Apr 2000 20:19:49 +0200
Subject: [Mailman-Developers] membership managent web interface
In-Reply-To: Ricardo Kustner's message of "Sat, 1 Apr 2000 16:57:13 +0200"
References: <20000401165713.B2495@miss-janet.com>
Message-ID:
[Ricardo Kustner]
> But most of the times you know the exact address you want to unsubscribe
> or view/edit, so why not add a search box on top?
Can't this already be done by using the list member's "options" page
(using the user's address in combination with your list admin
password)?
> I even think that only a searchbox on the initial page should be
> enough, and only show the paged list of members when the user
> specifically asks for it... saves a bit of bandwidth / response
> time. any thoughts about this?
I disagree. The list admin interface is cryptic enough as it is, IMO.
Further obscuring the overview of list members wouldn't be nice.
--
Harald
From ricardo@rixhq.nu Sun Apr 9 19:43:09 2000
From: ricardo@rixhq.nu (Ricardo Kustner)
Date: Sun, 9 Apr 2000 20:43:09 +0200
Subject: [Mailman-Developers] membership managent web interface
In-Reply-To: ; from Harald.Meland@usit.uio.no on Sun, Apr 09, 2000 at 08:19:49PM +0200
References: <20000401165713.B2495@miss-janet.com>
Message-ID: <20000409204309.A1521@miss-janet.com>
Hi,
On Sun, Apr 09, 2000 at 08:19:49PM +0200, Harald Meland wrote:
> > But most of the times you know the exact address you want to unsubscribe
> > or view/edit, so why not add a search box on top?
> Can't this already be done by using the list member's "options" page
> (using the user's address in combination with your list admin
> password)?
you mean something like
http://server/mailman/options/mailinglist/ricardo--at--rixhq.nu ?
besides not being very convenient to do, i don't think it works w/o knowing
the users' password... as far as i can see there's no way telling you're
the admin and I doubt that admin password can override it.
> > I even think that only a searchbox on the initial page should be
> > enough, and only show the paged list of members when the user
> > specifically asks for it... saves a bit of bandwidth / response
> > time. any thoughts about this?
> I disagree. The list admin interface is cryptic enough as it is, IMO.
> Further obscuring the overview of list members wouldn't be nice.
actually I suggested this because I think that would make the member options
much more simple. Having a partial list of the members immediately is what
makes it cryptic IMHO.
My list has about 1700 members and navigating with the way it is not is
not very convenient (even if you get used to it) and I can't imagine how
it would be like for people with 10k lists :)
Ricardo.
--
From Harald.Meland@usit.uio.no Sun Apr 9 20:38:16 2000
From: Harald.Meland@usit.uio.no (Harald Meland)
Date: 09 Apr 2000 21:38:16 +0200
Subject: [Mailman-Developers] Speaking of pathological behavior...
In-Reply-To: Dan Mick's message of "Wed, 05 Apr 2000 03:43:49 -0700"
References: <38EB146A.CF88B246@west.sun.com> <38EB17F4.6438AF52@west.sun.com> <38EB18E5.92C7B8F2@west.sun.com>
Message-ID: