From akuchlin@cnri.reston.va.us Mon Jun 1 14:19:57 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Mon, 1 Jun 1998 09:19:57 -0400 (EDT) Subject: [Mailman-Developers] Feature suggestion: NNTP gateway Message-ID: <13682.43062.223767.769690@newcnri.cnri.reston.va.us> Michael McLay and I were discussed Mailman recently, and he suggested that Mailman could be used to run python-list. I pointed out that the news-to-mail gateway is the most important part of that, and Michael suggested adding it to Mailman as an option. So I guess the suggestion is: an optional component that can be configured with an NNTP server hostname and a group name, and will then mirror the traffic on both; messages posted to the group get sent out via e-mail, and e-mail messages get posted. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ There are great numbers of people to whom the act of reading a book -- any sort of book -- is wondrous; they speak of the reader in the tone of warm approbation which they use otherwise when referring to pregnant women, or the newly dead. -- Robertson Davies, _A Voice from the Attic_ From viega@list.org Mon Jun 1 14:31:29 1998 From: viega@list.org (John Viega) Date: Mon, 1 Jun 1998 06:31:29 -0700 Subject: [Mailman-Developers] Feature suggestion: NNTP gateway In-Reply-To: <13682.43062.223767.769690@newcnri.cnri.reston.va.us>; from Andrew Kuchling on Mon, Jun 01, 1998 at 09:19:57AM -0400 References: <13682.43062.223767.769690@newcnri.cnri.reston.va.us> Message-ID: <19980601063129.A20444@list.org> I wrote Mail->News and News->Mail gatewaying over the weekend. I haven't been able to test it yet, because I have to get posting access to a news server for list.org before I can. Luckily, a friend of mine is the news admin at Erols, so that should actually be today. So the feature you want will work by tomorrow, no doubt :) John On Mon, Jun 01, 1998 at 09:19:57AM -0400, Andrew Kuchling wrote: > Michael McLay and I were discussed Mailman recently, and he suggested > that Mailman could be used to run python-list. I pointed out that the > news-to-mail gateway is the most important part of that, and Michael > suggested adding it to Mailman as an option. > > So I guess the suggestion is: an optional component that can be > configured with an NNTP server hostname and a group name, and will > then mirror the traffic on both; messages posted to the group get > sent out via e-mail, and e-mail messages get posted. > > -- > A.M. Kuchling http://starship.skyport.net/crew/amk/ > There are great numbers of people to whom the act of reading a book -- any > sort of book -- is wondrous; they speak of the reader in the tone of warm > approbation which they use otherwise when referring to pregnant women, or the > newly dead. > -- Robertson Davies, _A Voice from the Attic_ > > > > _______________________________________________ > Mailman-Developers maillist - Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From akuchlin@cnri.reston.va.us Mon Jun 1 21:46:52 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Mon, 1 Jun 1998 16:46:52 -0400 (EDT) Subject: [Mailman-Developers] Pipermail In-Reply-To: <19980530193546.A8889@list.org> References: <19980530193546.A8889@list.org> Message-ID: <13683.4851.971836.472538@newcnri.cnri.reston.va.us> John Viega writes: >Has anyone modified pipermail's hypermail emulation to work using only >modules that are compiled in the default Python distribution? There's >one dependency I have in mind: pipermail currently uses bsddb, which >is not compiled into Python 1.5. I think gdbm is compiled in by >default, but the two interfaces aren't compatible. Pipermail needs some sort of tree, so it can traverse the indexes in sorted order without making a list of all the articles in memory & sorting it. I've been meaning to support Aaron's bplustree.py module, but haven't gotten around to it. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ "For Inanna: Apocatastasis" "You always did pick rotten titles." -- Inanna begins to read the director's final screenplay in SIGNAL TO NOISE From mclay@nist.gov Wed Jun 3 09:08:03 1998 From: mclay@nist.gov (Michael McLay) Date: Wed, 3 Jun 1998 04:08:03 -0400 (EDT) Subject: [Mailman-Developers] Welcome To "Mailman-Developers"! In-Reply-To: <199806031432.KAA26233@python.org> References: <199806031432.KAA26233@python.org> Message-ID: <13684.65095.959086.349259@fermi.eeel.nist.gov> I just joined the list and I've looked through some of the messages in the archive, but not all of them. In the mm_defaults for data storing directories are set as: LIST_DATA_DIR = os.path.join(MAILMAN_DIR, 'lists') HTML_DIR = os.path.join(HOME_DIR, 'public_html') CGI_DIR = os.path.join(HOME_DIR, 'cgi-bin') LOG_DIR = os.path.join(HOME_DIR, 'logs') LOCK_DIR = os.path.join(MAILMAN_DIR, 'locks') TEMPLATE_DIR = os.path.join(MAILMAN_DIR, 'templates') Is there a reason for not setting them to: LIST_DATA_DIR = os.path.join(HOME_DIR, 'lists') HTML_DIR = os.path.join(HOME_DIR, 'public_html') CGI_DIR = os.path.join(HOME_DIR, 'cgi-bin') LOG_DIR = os.path.join(HOME_DIR, 'logs') LOCK_DIR = os.path.join(HOME_DIR, 'locks') TEMPLATE_DIR = os.path.join(HOME_DIR, 'templates') This would keep the data separated from the software. The installation instructions would have to be changed to add the lists, locks, and templates directories to HOME_DIR. Is the install script for the next release going to include automatic creation of directories using autoconfig. I'm having some trouble do to the odd setup of our servers. The sendmail program is running on a system that is separate from the www server. When the command "/home/mailman/mailman/mail/wrapper post gencam" is run by sendmail it tries to archive the files on a disk that is on a different system and, if I understand correctly, this can't be done because setuid won't work across NFS mounts. I really like the look of mailman and I've read through many of the scripts. It is much easier on the eyes than Majordomo. The one conceptual problem I'm having with understanding the product is the interactions between the mailman user and the deamons. The permissions setting for mailman files are not obvious. Which files must be writable by "nobody" and which must be writeable by "mailman". Mailman seems to be in the middle between two deamons and a little more help in understanding the interactions would be appreciated. For instance, MAILMAN_DIR/lists/LISTNAME/config.db seems to require both mailman and nobody to have write permission. Has bundling mailman as a package been considered? some of the mm_X.py files look like good candidates to be hidden inside a package. From viega@list.org Wed Jun 3 16:21:20 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 08:21:20 -0700 Subject: [Mailman-Developers] Welcome To "Mailman-Developers"! In-Reply-To: <13684.65095.959086.349259@fermi.eeel.nist.gov>; from Michael McLay on Wed, Jun 03, 1998 at 04:08:03AM -0400 References: <199806031432.KAA26233@python.org> <13684.65095.959086.349259@fermi.eeel.nist.gov> Message-ID: <19980603082120.D8453@list.org> On Wed, Jun 03, 1998 at 04:08:03AM -0400, Michael McLay wrote: > Is the install script for the next release going to include automatic > creation of directories using autoconfig. Yes... thank Barry for that! > I'm having some trouble do to the odd setup of our servers. The > sendmail program is running on a system that is separate from the www > server. > > When the command "/home/mailman/mailman/mail/wrapper post gencam" is > run by sendmail it tries to archive the files on a disk that is on a > different system and, if I understand correctly, this can't be done > because setuid won't work across NFS mounts. Hmm... So web based files and mail based (data) files need to have seperate configurable directories. BTW, I was thinking about how to do SMTP and httpd on 2 seperate machines w/o NFS, and the easiest way is to run Mailman on the machine w/ the web server, and have sendmail aliases use rsh to run programs :) We should consider more efficient strategies, though, because I'm sure that people will want to do this sort of thing often. > > I really like the look of mailman and I've read through many of the > scripts. It is much easier on the eyes than Majordomo. The one > conceptual problem I'm having with understanding the product is the > interactions between the mailman user and the deamons. The > permissions setting for mailman files are not obvious. Which files > must be writable by "nobody" and which must be writeable by "mailman". > Mailman seems to be in the middle between two deamons and a little > more help in understanding the interactions would be appreciated. > For instance, MAILMAN_DIR/lists/LISTNAME/config.db seems to require > both mailman and nobody to have write permission. Hmm, all of these files should be owned by Mailman. However, both sendmail and cgi processes may need to modify these data files. So when a mail or web request comes in, we set the SUID bit. Hmm, if SUID doesn't work across file systems, you may have to decide that mail commands don't work, unless you mirror the list data file across systems, because both mail commands and list commands can be used to subscribe, change options, etc. > Has bundling mailman as a package been considered? some of the mm_X.py > files look like good candidates to be hidden inside a package. It's slowly happening, yes. Barry's done a lot of work towards that end. From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 3 16:57:49 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 3 Jun 1998 11:57:49 -0400 (EDT) Subject: [Mailman-Developers] Welcome To "Mailman-Developers"! References: <199806031432.KAA26233@python.org> <13684.65095.959086.349259@fermi.eeel.nist.gov> Message-ID: <13685.26608.520862.660655@anthem.CNRI.Reston.Va.US> Mike, A lot of your concerns will be addressed with the next release. In particular: >>>>> "MM" == Michael McLay writes: MM> This would keep the data separated from the software. The MM> installation instructions would have to be changed to add the MM> lists, locks, and templates directories to HOME_DIR. Much of this is going to change, because... MM> Is the install script for the next release going to include MM> automatic creation of directories using autoconfig. Yes! The installation directory can now be completely separate from the source directory. Standard GNU configure options --prefix and --exec_prefix can be used (although the latter doesn't make a lot of sense). Everything gets installed relative to $prefix and all the paths are set up correctly to deal with this. I've also rearranged where some of the files get installed. Eventually, I'm going to reorganize the source tree so it better reflects what the install tree looks like, but that'll come later (since it involves manually munging the CVS repository). I hope the install process will be much more straight forward in the next release. John's been debugging my autoconf stuff, so that gives me a lot more confidence :-) MM> I'm having some trouble do to the odd setup of our servers. MM> The sendmail program is running on a system that is separate MM> from the www server. I think John's move away from sendmail, toward SMTP delivery will alleviate much of this problem. MM> When the command "/home/mailman/mailman/mail/wrapper post MM> gencam" is run by sendmail it tries to archive the files on a MM> disk that is on a different system and, if I understand MM> correctly, this can't be done because setuid won't work across MM> NFS mounts. In the next release, nothing will have to be setuid [*]. I've reworked the permissions so that anything that needs write permission is set-gid, which I think does work across NFS mounts (please correct me if I'm wrong). MM> I really like the look of mailman and I've read through many MM> of the scripts. It is much easier on the eyes than Majordomo. Yes! MM> The one conceptual problem I'm having with understanding the MM> product is the interactions between the mailman user and the MM> deamons. The permissions setting for mailman files are not MM> obvious. Which files must be writable by "nobody" and which MM> must be writeable by "mailman". Mailman seems to be in the MM> middle between two deamons and a little more help in MM> understanding the interactions would be appreciated. For MM> instance, MAILMAN_DIR/lists/LISTNAME/config.db seems to MM> require both mailman and nobody to have write permission. I ran across the same problems. Using setgid scripts has solved most of this for me. MM> Has bundling mailman as a package been considered? some of the MM> mm_X.py files look like good candidates to be hidden inside a MM> package. Already done. Actually, the mm_*.py files get installed into $prefix/Mailman, and there's a __init__.py file under there, so it's already packagized. Something to do in the future (IMO), is to get rid of the mm_ prefix on the files, and fix the imports to assume packagization. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 3 17:02:48 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 3 Jun 1998 12:02:48 -0400 (EDT) Subject: [Mailman-Developers] Welcome To "Mailman-Developers"! References: <199806031432.KAA26233@python.org> <13684.65095.959086.349259@fermi.eeel.nist.gov> <19980603082120.D8453@list.org> Message-ID: <13685.29509.585038.473545@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Hmm, all of these files should be owned by Mailman. Actually, I don't *think* this is a requirement anymore. I've been running some small test lists with myself as the owner, but all the files group-owned `mailman'. That, in combination with the set-gid bit (*not* the set-uid bit), seems to work for both web and mail access to the files. I'm still not 100% sure that set-gid works across NFS though (can't test it right now). -Barry From cklempay@jerkweed.ml.org Wed Jun 3 22:06:21 1998 From: cklempay@jerkweed.ml.org (Corbett Klempay) Date: Wed, 3 Jun 1998 17:06:21 -0400 (EDT) Subject: [Mailman-Developers] ETA? Message-ID: Not to be annoying or anything, but would anyone be able to venture a guess as to the ETA for b4? Since I convinced the other ACM admin to move from Majordomo to Mailman, he's waiting until b4 is released to migrate all mail and list services to our new machine. Any info is appreciated. Thanks :) ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "There exists in India no politician daring enough to attempt to explain to the masses that cows can be eaten." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From viega@list.org Wed Jun 3 22:12:04 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 14:12:04 -0700 Subject: [Mailman-Developers] ETA? In-Reply-To: ; from Corbett Klempay on Wed, Jun 03, 1998 at 05:06:21PM -0400 References: Message-ID: <19980603141204.B14356@list.org> Barry and I are working on putting together the b4 release as we speak. I think there will be an announcement within an hour. However, we're only going to announce b4 to this mailing list, because since archiving is still not integrated, we'd really prefer not to put b4 into really wide release... John On Wed, Jun 03, 1998 at 05:06:21PM -0400, Corbett Klempay wrote: > Not to be annoying or anything, but would anyone be able to venture a > guess as to the ETA for b4? Since I convinced the other ACM admin to move > from Majordomo to Mailman, he's waiting until b4 is released to migrate > all mail and list services to our new machine. Any info is appreciated. > Thanks :) > > ------------------------------------------------------------------------------- > Corbett J. Klempay Quote of the Week: > http://www2.acm.jhu.edu/~cklempay "There exists in India no politician > daring enough to attempt to explain > to the masses that cows can be eaten." > > PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 > ------------------------------------------------------------------------------- > > > > _______________________________________________ > Mailman-Developers maillist - Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From cklempay@jerkweed.ml.org Wed Jun 3 22:25:00 1998 From: cklempay@jerkweed.ml.org (Corbett Klempay) Date: Wed, 3 Jun 1998 17:25:00 -0400 (EDT) Subject: [Mailman-Developers] ETA? In-Reply-To: <19980603141204.B14356@list.org> Message-ID: On Wed, 3 Jun 1998, John Viega wrote: > Barry and I are working on putting together the b4 release as we > speak. I think there will be an announcement within an hour. > However, we're only going to announce b4 to this mailing list, because > since archiving is still not integrated, we'd really prefer not to put > b4 into really wide release... > > John So does this mean that Pipermail integration will probably show up in b5? Would it be a "good idea" for me to just install this b4 now, and then put in b5 (or whatever the following release is) after that? Is that easy to do? (upgrade to the future version with archiving when I'm already running b4) Any idea how long until archiving is implemented? Hehe...I need to start writing some Python programs (and stop writing so many Perl ones!) so I can start contributing...argh :) Corbett From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 3 22:27:37 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 3 Jun 1998 17:27:37 -0400 (EDT) Subject: [Mailman-Developers] ETA? References: <19980603141204.B14356@list.org> Message-ID: <13685.49018.78031.892336@anthem.CNRI.Reston.Va.US> >>>>> "CK" == Corbett Klempay writes: CK> So does this mean that Pipermail integration will probably CK> show up in b5? Would it be a "good idea" for me to just CK> install this b4 now, and then put in b5 (or whatever the CK> following release is) after that? Is that easy to do? CK> (upgrade to the future version with archiving when I'm already CK> running b4) Any idea how long until archiving is implemented? I think now that we're autoconf'd, it should be easier to install a new version over an existing version, since the installation process shouldn't touch any of the important database files. I emphasize *should*; I can't say this stuff has been tested extensively ;-) CK> Hehe...I need to start writing some Python programs (and stop CK> writing so many Perl ones!) so I can start contributing...argh CK> :) Indeed! :-) From viega@list.org Wed Jun 3 22:29:42 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 14:29:42 -0700 Subject: [Mailman-Developers] ETA? In-Reply-To: ; from Corbett Klempay on Wed, Jun 03, 1998 at 05:25:00PM -0400 References: <19980603141204.B14356@list.org> Message-ID: <19980603142942.E14356@list.org> On Wed, Jun 03, 1998 at 05:25:00PM -0400, Corbett Klempay wrote: > On Wed, 3 Jun 1998, John Viega wrote: > > > Barry and I are working on putting together the b4 release as we > > speak. I think there will be an announcement within an hour. > > However, we're only going to announce b4 to this mailing list, because > > since archiving is still not integrated, we'd really prefer not to put > > b4 into really wide release... > > > > John > > So does this mean that Pipermail integration will probably show up in b5? > Would it be a "good idea" for me to just install this b4 now, and then put > in b5 (or whatever the following release is) after that? Is that easy to > do? (upgrade to the future version with archiving when I'm already running > b4) Any idea how long until archiving is implemented? Yes, upgrading should be as simple as installing over your old install (after backing it up, of course, because you never know). I'd like brchiving to go (back) in by b5. The problem is, it requires a bit of work to hook up Pipermail and Mailman in a portable way, and I don't see anyone stepping forward :) Which means I'll be the lucky guy to do it. I'm getting ready to move and start a new job, though, so I'd anticipate 2-3 weeks before I'll actually get that done. > > Hehe...I need to start writing some Python programs (and stop writing so > many Perl ones!) so I can start contributing...argh :) Please :) The more the merrier :) John From viega@list.org Wed Jun 3 23:15:35 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 15:15:35 -0700 Subject: [Mailman-Developers] Mailman 1.0b4 released Message-ID: <19980603151535.A15436@list.org> Mailman 1.0b4 is now available from: ftp://list.org/pub/mailman/mailman-1.0b4.tar.gz This release is only intended to see a small distribution, primarily this development community. The first "major" release is still a couple of weeks away (some people still need to sign copyright forms for GNU, archiving has to be integrated back in, etc.) Developers, please feel encouraged to go to the Mailman home page, look at the developer's TODO list, and pick out a project. If you are thus inclined, the Mailman home page is located at: http://www.list.org/ The major changes since 1.0b3 are: - A configure script for easy installation (Barry Warsaw) - The ability to install Mailman to locations other than /home/mailman (Barry Warsaw) - Use cookies on the admin pages (also hides admin pages from others) (Scott Cotton) - Subscription requests send a request for confirmation, which may be done by simply replying to the message (Scott Cotton) - Facilities for gating mail to a newsgroup, and for gating a newsgroup to a mailing list (John Viega) - Contact the SMTP port instead of calling sendmail (primarily for portability) (JohnViega) - Changed all links on web pages to relative links where appropriate. (John Viega) - Use MD5 if crypt is not available (John Viega) - Lots of fixing up of bounce handling (Ken Manheimer) - General UI polishing (Ken Manheimer) - mm_html: Make it prominent when the user's delivery is disabled on his option page. (Ken Manheimer) Enjoy, John From viega@list.org Wed Jun 3 23:35:04 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 15:35:04 -0700 Subject: [Mailman-Developers] News->Mail gateways Message-ID: <19980603153504.D15479@list.org> The new release includes both Mail->News gatewaying and News->Mail gatewaying. The latter uses a polling strategy. If you have direct access to your nntp server, it would be possible not to have to poll. In fact, it would be fairly easy. Unfortunately, I don't have that sort of access available. If anyone is interested in getting non-polling gatewaying working, please contact me, I'm interested in writing the necessary code, I just need someone who can test it, etc. John From mclay@nist.gov Thu Jun 4 00:01:17 1998 From: mclay@nist.gov (Michael McLay) Date: Wed, 3 Jun 1998 19:01:17 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <19980603153504.D15479@list.org> References: <19980603153504.D15479@list.org> Message-ID: <13685.54195.324584.618375@fermi.eeel.nist.gov> I've downloaded the distribution and tried running configure. Here's the error messages. bash$ ./configure creating cache ./config.cache checking for --with-python... checking for python... /usr/local/bin/python checking Python interpreter... /usr/local/bin/python checking for a BSD compatible install... /usr/bin/install -c checking whether make sets ${MAKE}... yes checking for --without-gcc... no checking for gcc... no checking for cc... /usr/bin/cc checking whether the C compiler (/usr/bin/cc ) works... yes checking whether the C compiler (/usr/bin/cc ) is a cross-compiler... no checking whether we are using GNU C... no checking whether #! works in shell scripts... yes checking for --with-mail-gid... other mail daemon checking for mail_wrapper GID... ./configure: python: not found cat: conftest.out: No such file or directory configure: error: ***** Group other mail daemon not found! You must first ***** set up the other mail daemon entry in /etc/group ***** or set the MAIL_GID shell variable on the ***** configure command line I don't understand this comment. My system administrator has set up the directory structure as follows: bash$ cd /home/ bash$ ls -ldg mailman lrwxrwxrwx 1 root wheel 13 Jun 3 18:51 mailman -> /bolt/mailman bash$ The web server is running on bolt so the /bold/mailman directory is visible on that system. The mail server runs on sparky and sparky uses some Solaris automouter magic to make /home/mailman appear as a directory, but /bolt/mailman doesn't appear. I'll ask if I can get the /home/mailman directory to be changed to ownership by mailman and put in group mailman. For now it is owned by root and in group wheel. From cklempay@jerkweed.ml.org Thu Jun 4 00:44:11 1998 From: cklempay@jerkweed.ml.org (Corbett Klempay) Date: Wed, 3 Jun 1998 19:44:11 -0400 (EDT) Subject: [Mailman-Developers] 2 Quickies Message-ID: Being the big newbie, I am having 2 snags so far in my Mailman 1.0b4 installation (Red Hat 5.1): 1) Cron is giving me this every 5 minutes: Date: Wed, 3 Jun 1998 19:20:01 -0400 From: Cron Daemon To: mailman@chimera.acm.jhu.edu Subject: Cron /usr/bin/python /home/mailman/cron/gate_news Traceback (innermost last): File "/home/mailman/cron/gate_news", line 41, in ? file = open(os.path.join(mm_cfg.DATA_DIR, "gate_watermarks.last"), "w") IOError: (13, 'Permission denied') 2) And I'm getting a 403 Forbidden on the test list...I'm thinking this might be related to the other admin installing cgi-wrap...I'm not too keen on how it works. Does is the user mailman supposed to belong to just the users group, or users as well as the mailman group? CK From viega@list.org Thu Jun 4 03:24:31 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 19:24:31 -0700 Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13685.54195.324584.618375@fermi.eeel.nist.gov>; from Michael McLay on Wed, Jun 03, 1998 at 07:01:17PM -0400 References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> Message-ID: <19980603192431.A17219@list.org> On Wed, Jun 03, 1998 at 07:01:17PM -0400, Michael McLay wrote: > checking for --with-mail-gid... other mail daemon > checking for mail_wrapper GID... ./configure: python: not found > cat: conftest.out: No such file or directory > configure: error: > > ***** Group other mail daemon not found! You must first > ***** set up the other mail daemon entry in /etc/group > ***** or set the MAIL_GID shell variable on the > ***** configure command line Well, it looks like Python isn't in your path when installing, and it needs to be. The configure script is trying to run Python to guess which group your SMTPD runs under, but can't run Python, and thus can't come up with a guess. John From viega@list.org Thu Jun 4 03:43:21 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 19:43:21 -0700 Subject: [Mailman-Developers] 2 Quickies In-Reply-To: ; from Corbett Klempay on Wed, Jun 03, 1998 at 07:44:11PM -0400 References: Message-ID: <19980603194321.B17219@list.org> On Wed, Jun 03, 1998 at 07:44:11PM -0400, Corbett Klempay wrote: > Being the big newbie, I am having 2 snags so far in my Mailman 1.0b4 > installation (Red Hat 5.1): > > 1) > Cron is giving me this every 5 minutes: > > Date: Wed, 3 Jun 1998 19:20:01 -0400 > From: Cron Daemon > To: mailman@chimera.acm.jhu.edu > Subject: Cron /usr/bin/python > /home/mailman/cron/gate_news > > Traceback (innermost last): > File "/home/mailman/cron/gate_news", line 41, in ? > file = open(os.path.join(mm_cfg.DATA_DIR, "gate_watermarks.last"), > "w") > IOError: (13, 'Permission denied') That's interesting, as I can't reproduce this problem on my system. What does an ls -l on $prefix/data give you? BTW, you can just comment out the gate-news line and reinstall the crontab until we can figure out exactly what's going on... > 2) > And I'm getting a 403 Forbidden on the test list...I'm thinking this might > be related to the other admin installing cgi-wrap...I'm not too keen on > how it works. This sounds likely. I don't remember much about cgi-wrap. I think the URLs almost certainly have to be a different format. Something like: http://www.server.com/cgi-bin/cgiwrap/mailman/name-of-script And then name-of-script has to live in ~mailman/cgi-bin. I can't remember, you might want to check the man page on cgiwrap. However, if you added the ScriptAlias line or whatever to your httpd conf and restarted, you shouldn't need to use cgi-wrap, I'd think. Watch syslog / maillog and see if you've got a GID error, as that also sounds possible. > > Does is the user mailman supposed to belong to just the users group, or > users as well as the mailman group? It should belong to the mailman group... From viega@list.org Thu Jun 4 03:59:37 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 19:59:37 -0700 Subject: [Mailman-Developers] 2 Quickies In-Reply-To: <19980603194321.B17219@list.org>; from John Viega on Wed, Jun 03, 1998 at 07:43:21PM -0700 References: <19980603194321.B17219@list.org> Message-ID: <19980603195937.A17572@list.org> On Wed, Jun 03, 1998 at 07:43:21PM -0700, John Viega wrote: > On Wed, Jun 03, 1998 at 07:44:11PM -0400, Corbett Klempay wrote: > > Being the big newbie, I am having 2 snags so far in my Mailman 1.0b4 > > installation (Red Hat 5.1): > > Traceback (innermost last): > > File "/home/mailman/cron/gate_news", line 41, in ? > > file = open(os.path.join(mm_cfg.DATA_DIR, "gate_watermarks.last"), > > "w") > > IOError: (13, 'Permission denied') > > That's interesting, as I can't reproduce this problem on my system. BTW, I'm running redhat 5.1 too. I've been thinking about this problem, and I'm thinking that your problem is potentially something like mailman files are getting created as mailman.user instead of mailman.mailman, or something like that. That could cause problems. I'd suggest making sure that Mailman's /etc/passwd entry has the mailman GID in the group field, and not the user GID. Then you probably will want to re-install. Anyway, if that still doesn't work, or if that isn't a problem, then let me know what ls -l says on $prefix/data... John From viega@list.org Thu Jun 4 06:47:32 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 22:47:32 -0700 Subject: [Mailman-Developers] 2 Quickies In-Reply-To: ; from Corbett Klempay on Wed, Jun 03, 1998 at 07:44:11PM -0400 References: Message-ID: <19980603224732.A19017@list.org> Just to let everyone know how this one turned out: On Wed, Jun 03, 1998 at 07:44:11PM -0400, Corbett Klempay wrote: > 1) > Cron is giving me this every 5 minutes: [...] > Traceback (innermost last): > File "/home/mailman/cron/gate_news", line 41, in ? > file = open(os.path.join(mm_cfg.DATA_DIR, "gate_watermarks.last"), > "w") > IOError: (13, 'Permission denied') Mailman was installed w/ group user instead of group mailman. > 2) > And I'm getting a 403 Forbidden on the test list...I'm thinking this might > be related to the other admin installing cgi-wrap...I'm not too keen on > how it works. The problem here was that /home/mailman needed to be "chmod a+rx"'d. That one should be mentioned in the install document! John From viega@list.org Thu Jun 4 14:26:05 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 06:26:05 -0700 Subject: [Mailman-Developers] CGI thoughts Message-ID: <19980604062605.C21640@list.org> I was just thinking that it's a little much to require people to go in and change their httpd configuration files. Something we could do for the next release would be to provide a single C binary, and tell people, "Put this in your system cgi-bin directory". Then mailman URLS would become: http://www.blah.com/cgi-bin/mailman/listinfo I think while the URL may get a little bigger, it has the advantage of being a lot less error-prone to install. Of course, we could always have an option to configure that would keep things the way they are. Thoughts? John From viega@list.org Thu Jun 4 15:38:23 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 07:38:23 -0700 Subject: [Mailman-Developers] Cookies (again) Message-ID: <19980604073823.B22159@list.org> Okay, I just let a friend of mine set up a list. The first thing he said was, "it looks great, but my changes don't submit, it gives me the login screen and then my changes are gone". Well, he's almost certainly got cookies turned off. Do you think it would be a good idea to put a password field above the "Submit your changes" button, and label it: "If your browser doesn't have cookies on, put your password here"? John From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 15:42:51 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 10:42:51 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> Message-ID: <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Well, it looks like Python isn't in your path when installing, JV> and it needs to be. The configure script is trying to run JV> Python to guess which group your SMTPD runs under, but can't JV> run Python, and thus can't come up with a guess. John's right. The problem is that the configure script looks on your $PATH for python (which it didn't find), but defaults to /usr/local/bin/python (which it did find). If configure never found a python binary, it would have exited earlier. The problem is that other tests in the configure script assume python is on your path, and in your case it isn't. This is exactly the kind of bug I was hoping would get fleshed out by this beta, so... thanks Mike! :-) Try this patch. It changes the tests so that it uses the python binary it found in the earlier test. -Barry P.S. The error message isn't very good, so I'll change that, but not for this patch. -------------------- snip snip -------------------- Index: configure =================================================================== RCS file: /projects/cvsroot/mailman/configure,v retrieving revision 1.9 diff -C2 -r1.9 configure *** configure 1998/06/03 22:25:49 1.9 --- configure 1998/06/04 14:39:13 *************** *** 1,5 **** #! /bin/sh ! # From configure.in Revision: 1.9 # Guess values for system-dependent variables and create Makefiles. --- 1,5 ---- #! /bin/sh ! # From configure.in Revision: 1.10 # Guess values for system-dependent variables and create Makefiles. *************** *** 1005,1009 **** fp.close() EOF ! python conftest.py MAIL_GID=`cat conftest.out` fi --- 1005,1009 ---- fp.close() EOF ! $PYTHON conftest.py MAIL_GID=`cat conftest.out` fi *************** *** 1061,1065 **** fp.close() EOF ! python conftest.py CGI_GID=`cat conftest.out` fi --- 1061,1065 ---- fp.close() EOF ! $PYTHON conftest.py CGI_GID=`cat conftest.out` fi *************** *** 1110,1114 **** fp.close() EOF ! python conftest.py echo $ac_n "checking for default fully qualified host name""... $ac_c" 1>&6 --- 1110,1114 ---- fp.close() EOF ! $PYTHON conftest.py echo $ac_n "checking for default fully qualified host name""... $ac_c" 1>&6 From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 16:08:55 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 11:08:55 -0400 (EDT) Subject: [Mailman-Developers] 2 Quickies References: <19980603194321.B17219@list.org> <19980603195937.A17572@list.org> Message-ID: <13686.46847.40420.137132@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> BTW, I'm running redhat 5.1 too. I've been thinking about JV> this problem, and I'm thinking that your problem is JV> potentially something like mailman files are getting created JV> as mailman.user instead of mailman.mailman, or something like JV> that. That could cause problems. I'd suggest making sure JV> that Mailman's /etc/passwd entry has the mailman GID in the JV> group field, and not the user GID. Then you probably will JV> want to re-install. But that shouldn't be happening, because I go to pains to install the mailman directories with the group set-id (a.k.a. group sticky) bit turned on. Do an ls -l on $prefix/data an dyou should see something like drwxrwsr-x. The semantics are described like this in chmod(2) on Solaris: In a directory which has the set-group-ID bit set (reflected as either -----s--- or ----- l--- in the output of 'ls -ld'), files and subdirectories are created with the group-ID of the parent directory-not that of current process. So all files created under $prefix/data should definitely be group owned by mailman. The INSTALL file says you must create $prefix with the group sticky bit set, and chgrp it to mailman. If you didn't do this, then the groups on all the files will cascade to be incorrectly group owned. Do you think this is what happened? It probably makes sense for either the configure script, or the install target to explicitly check for this scenario. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 16:12:34 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 11:12:34 -0400 (EDT) Subject: [Mailman-Developers] 2 Quickies References: <19980603224732.A19017@list.org> Message-ID: <13686.47320.462795.179182@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Mailman was installed w/ group user instead of group mailman. Do you mean that $prefix wasn't chgrp'd mailman and chmod'd g+s? I'd really like to know if those instructions in the INSTALL are incorrect or not. If they just overlooked, then I can write a test to guarantee that's set up correctly. JV> The problem here was that /home/mailman needed to be "chmod JV> a+rx"'d. That one should be mentioned in the install JV> document! Okay, here's what the INSTALL doc will say about setting up $prefix: Make sure this directory is set to group `mailman' and has the group sticky bit set. You probably also want to guarantee that this directory is read and executeable by everyone. For example, these shell commands will accomplish this: % cd $prefix % chgrp mailman . % chmod a+rx,g+s . -Barry From viega@list.org Thu Jun 4 16:15:58 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 08:15:58 -0700 Subject: [Mailman-Developers] 2 Quickies In-Reply-To: <13686.47320.462795.179182@anthem.CNRI.Reston.Va.US>; from Barry A. Warsaw on Thu, Jun 04, 1998 at 11:12:34AM -0400 References: <19980603224732.A19017@list.org> <13686.47320.462795.179182@anthem.CNRI.Reston.Va.US> Message-ID: <19980604081558.E22159@list.org> On Thu, Jun 04, 1998 at 11:12:34AM -0400, Barry A. Warsaw wrote: > > >>>>> "JV" == John Viega writes: > > JV> Mailman was installed w/ group user instead of group mailman. > > Do you mean that $prefix wasn't chgrp'd mailman and chmod'd g+s? Yeah, I think that it may have been a case of skimming through the install docs, which I'm sure we've all done before! John From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 16:27:34 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 11:27:34 -0400 (EDT) Subject: [Mailman-Developers] 2 Quickies References: <19980603224732.A19017@list.org> <13686.47320.462795.179182@anthem.CNRI.Reston.Va.US> <19980604081558.E22159@list.org> Message-ID: <13686.48301.125281.839505@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Yeah, I think that it may have been a case of skimming through JV> the install docs, which I'm sure we've all done before! Yeah, I've been guilty too! Well, I'll add an explicit check (someplace). Much better for configure or make install to barf and point you at the documentation, then to just let people screw themselves. ;-) -Barry From mclay@nist.gov Thu Jun 4 21:43:01 1998 From: mclay@nist.gov (Michael McLay) Date: Thu, 4 Jun 1998 16:43:01 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> Message-ID: <13686.65494.140292.268905@fermi.eeel.nist.gov> Barry A. Warsaw writes: > > >>>>> "JV" == John Viega writes: > > JV> Well, it looks like Python isn't in your path when installing, > JV> and it needs to be. The configure script is trying to run > JV> Python to guess which group your SMTPD runs under, but can't > JV> run Python, and thus can't come up with a guess. > > John's right. The problem is that the configure script looks on your > $PATH for python (which it didn't find), but defaults to > /usr/local/bin/python (which it did find). If configure never found a > python binary, it would have exited earlier. > > The problem is that other tests in the configure script assume python > is on your path, and in your case it isn't. This is exactly the kind > of bug I was hoping would get fleshed out by this beta, so... thanks > Mike! :-) > > Try this patch. It changes the tests so that it uses the python > binary it found in the earlier test. Hmm, I fixed the problem so python is found so I didn't exercise this patch. There's one small buglet in the code generated by configure. The script which generates the crontab.in file adds blank lines to the file. This bombs the crontab command on the old SunOS system we are using. There is also a link error when building the source code on SunOS using gcc version 2.7.2. Here's the snippet that shows the error. make[1]: Entering directory `/usr/local/mailman/mailman-1.0b4/src' gcc -c -I. -DPREFIX="\"/usr/local/mailman\"" -DPYTHON="\"/usr/local/bin/python\"" -g -O2 -DHAVE_SYSLOG_H=1 -DGETGROUPS_T=int -DHAVE_VPRINTF=1 common.c gcc -DSCRIPT="\"admin\"" -I. -DCGI_GID=55 -g -O2 -DHAVE_SYSLOG_H=1 -DGETGROUPS_T=int -DHAVE_VPRINTF=1 common.o -o admin cgi-wrapper.c collect2: ld returned 2 exit status ld: Undefined symbol _strerror make[1]: *** [admin] Error 1 make[1]: Leaving directory `/usr/local/mailman/mailman-1.0b4/src' make[1]: Entering directory `/usr/local/mailman/mailman-1.0b4/templates' for f in archives.html handle_opts.html listinfo.html options.html roster.html subscribe.html; \ do \ /bin/install -c -m 644 $f /usr/local/mailman/templates; \ done make[1]: Leaving directory `/usr/local/mailman/mailman-1.0b4/templates' From mclay@nist.gov Thu Jun 4 21:49:26 1998 From: mclay@nist.gov (Michael McLay) Date: Thu, 4 Jun 1998 16:49:26 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> Message-ID: <13687.2049.860850.719792@fermi.eeel.nist.gov> I tried changing the compiler form gcc to gcc and reran make install. This time stdarg.h isn't found. make[1]: Entering directory `/usr/local/mailman/mailman-1.0b4/src' cc -c -I. -DPREFIX="\"/usr/local/mailman\"" -DPYTHON="\"/usr/local/bin/python\"" -O -DHAVE_SYSLOG_H=1 -DGETGROUPS_T=int -DHAVE_VPRINTF=1 common.c ./common.h: 22: Can't find include file stdarg.h make[1]: *** [common.o] Error 2 make[1]: Leaving directory `/usr/local/mailman/mailman-1.0b4/src' make[1]: Entering directory `/usr/local/mailman/mailman-1.0b4/templates' for f in archives.html handle_opts.html listinfo.html options.html roster.html subscribe.html; \ do \ /bin/install -c -m 644 $f /usr/local/mailman/templates; \ done make[1]: Leaving directory `/usr/local/mailman/mailman-1.0b4/templates' From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 22:06:52 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 17:06:52 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13687.2049.860850.719792@fermi.eeel.nist.gov> Message-ID: <13687.3075.35966.630102@anthem.CNRI.Reston.Va.US> >>>>> "MM" == Michael McLay writes: MM> I tried changing the compiler form gcc to gcc and reran make --------------------------------------------------^^^ you meant to say "to the bundled SunOS cc"? If so, note that that's a non-ANSI compiler and so old that I can't imagine we'd want to support it. The gcc 2.7.2 bug is valid though. I *think* I've got an old gcc laying around I can try. -Barry From mclay@nist.gov Thu Jun 4 22:15:26 1998 From: mclay@nist.gov (Michael McLay) Date: Thu, 4 Jun 1998 17:15:26 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13687.3075.35966.630102@anthem.CNRI.Reston.Va.US> References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13687.2049.860850.719792@fermi.eeel.nist.gov> <13687.3075.35966.630102@anthem.CNRI.Reston.Va.US> Message-ID: <13687.3412.944873.484780@fermi.eeel.nist.gov> Barry A. Warsaw writes: > > >>>>> "MM" == Michael McLay writes: > > MM> I tried changing the compiler form gcc to gcc and reran make > --------------------------------------------------^^^ > you meant to say "to the bundled SunOS cc"? Opps, yes I did mean cc. > If so, note that that's a non-ANSI compiler and so old that I can't > imagine we'd want to support it. Fine with me. I think we need to get rid of this old hardware anyway and if you refuse to support K&R then it actually helps my case. > The gcc 2.7.2 bug is valid though. I *think* I've got an old gcc > laying around I can try. Yea, I know this is out of date too. It probably isn't even installed properly. Time to fire off a message to our system administrator. In the mean time... I need a fix to get this working with what I have. From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 23:08:30 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 18:08:30 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13687.2049.860850.719792@fermi.eeel.nist.gov> <13687.3075.35966.630102@anthem.CNRI.Reston.Va.US> <13687.3412.944873.484780@fermi.eeel.nist.gov> Message-ID: <13687.6756.901874.669951@anthem.CNRI.Reston.Va.US> >>>>> "MM" == Michael McLay writes: MM> Fine with me. I think we need to get rid of this old hardware MM> anyway and if you refuse to support K&R then it actually helps MM> my case. Okay, then I refuse! :-) >> The gcc 2.7.2 bug is valid though. I *think* I've got an old >> gcc laying around I can try. MM> Yea, I know this is out of date too. It probably isn't even MM> installed properly. Time to fire off a message to our system MM> administrator. MM> In the mean time... I need a fix to get this working with MM> what I have. Looks like I don't have an old gcc anymore. If I can pull down 2.7.*.* from an ftp site then I'll give that a whirl. Stay tuned. If anybody else has an older version of gcc and can work up a patch in the meantime, that would be greatly appreciated! -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 5 00:21:31 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 19:21:31 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13686.65494.140292.268905@fermi.eeel.nist.gov> Message-ID: <13687.11188.942023.123916@anthem.CNRI.Reston.Va.US> >>>>> "MM" == Michael McLay writes: MM> There's one small buglet in the code generated by configure. MM> The script which generates the crontab.in file adds blank MM> lines to the file. This bombs the crontab command on the old MM> SunOS system we are using. It doesn't seem to add blank lines for me, although the crontab.in.in file already contains a blank line, in the middle of the file. Is that the problem? If so, it's trivial to fix (but I want to be sure that's your problem first). -Barry From viega@list.org Fri Jun 5 03:56:12 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 19:56:12 -0700 Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13686.65494.140292.268905@fermi.eeel.nist.gov>; from Michael McLay on Thu, Jun 04, 1998 at 04:43:01PM -0400 References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13686.65494.140292.268905@fermi.eeel.nist.gov> Message-ID: <19980604195612.D28273@list.org> On Thu, Jun 04, 1998 at 04:43:01PM -0400, Michael McLay wrote: > > There is also a link error when building the source code on > SunOS using gcc version 2.7.2. Here's the snippet that shows the > error. Okay, it took me a while to find a machine to use sunning sunOS 4 and gcc 2.7.2 :) > ld: Undefined symbol > _strerror Yes, apparently strerror() is nowhere to be found on such a system. I have 2 functions that I just wrote up. The first duplicates the behavior that strerror gives on other systems. The second saves the large static buffer, and will do for our purposes. Just drop your pick into common.c, and it should then compile: Version 1 (Compatible): #define STACK_BUFSIZE 8192 extern char *sys_errlist[]; char* strerror(int errno) { static char msg[STACK_BUFSIZE]; strcpy(msg, sys_errlist[errno]); return msg; } Version 2 (Efficient): extern char *sys_errlist[]; char* strerror(int errno) { return sys_errlist[errno]; } From viega@list.org Fri Jun 5 04:10:47 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 20:10:47 -0700 Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <19980604195612.D28273@list.org>; from John Viega on Thu, Jun 04, 1998 at 07:56:12PM -0700 References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13686.65494.140292.268905@fermi.eeel.nist.gov> <19980604195612.D28273@list.org> Message-ID: <19980604201047.F28273@list.org> Um, whoops, s/STACK/STATIC throughout that snipit of code, I don't know where my mind is today. > #define STACK_BUFSIZE 8192 Anyway, Here's what I recommend we use in the dist, if and only if strerror can't be found by the autoconf process: extern char *sys_errlist[]; extern int sys_nerr; char* strerror(int errno) { if(errno < 0 || errno >= sys_nerr) { return "unknown error"; } else { return sys_errlist[errno]; } } From janne@avocado.pc.helsinki.fi Fri Jun 5 10:11:10 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 05 Jun 1998 12:11:10 +0300 Subject: [Mailman-Developers] A bug in 1.0b4 (subscriber's options) Message-ID: If I go to editing subscriber's options by writing an e-mail address onto the listinfo page, submitting the changes does not work. Instead I get an error saying handle_opts: no such list. Instead, editing options by clicking the subscriber's address on the subscriber list works fine. -- Janne From viega@list.org Fri Jun 5 11:01:54 1998 From: viega@list.org (John Viega) Date: Fri, 5 Jun 1998 03:01:54 -0700 Subject: [Mailman-Developers] A bug in 1.0b4 (subscriber's options) In-Reply-To: ; from Janne Sinkkonen on Fri, Jun 05, 1998 at 12:11:10PM +0300 References: Message-ID: <19980605030154.B31743@list.org> On Fri, Jun 05, 1998 at 12:11:10PM +0300, Janne Sinkkonen wrote: > > If I go to editing subscriber's options by writing an e-mail address > onto the listinfo page, submitting the changes does not work. > Instead I get an error saying handle_opts: no such list. That's interesting. For some reason in that one case, PATH_INFO is lacking a leading /, whereas in all other cases it has one, I think. So here's a really tiny patch that fixes the problem. It seems to not break anything else. diff -r1.28 mm_utils.py 232c232,234 < nesting_level = len(string.split(os.environ['PATH_INFO'], '/')) - 1 --- > if path[0] <> '/': > path= '/' + path > nesting_level = len(string.split(path, '/')) - 1 From mclay@nist.gov Fri Jun 5 17:32:32 1998 From: mclay@nist.gov (Michael McLay) Date: Fri, 5 Jun 1998 12:32:32 -0400 (EDT) Subject: [Mailman-Developers] file permissions Message-ID: <199806051632.MAA09671@fermi.eeel.nist.gov> Either I screwed up when I followed the installation instructions or else the file ownership set by "make install" incorrectly set the group of the executables in $prefix/cgi-bin to that of the super user. In order for the scripts to work the group had to be manually set to "mailman". I accidentally created a copy of $prefix/data/pending_subscriptions.db with the wrong group ownership which caused the subcribe script to fail with an "Internal Server Error" The script failed because of a file access permission error. In order to find the problem I did some code reorganization. The $prefix/scripts/subscribe file was moved to $prefix/Mailman/subscribe.py and the attached file was put in place of the $prefix/scripts/subscribe file. This change should also reduce the script execution time slightly because the longer script will be compiled to a .pyc file. I would like to wrap all the other file in the $prefix/scripts/ in the same manner. Once this is done the common.c program can be modified to insert the PREFIX directory into the path rather than the MODULEDIR. This will eliminate the need for the paths module in each of the scripts and it would eliminate the need for the following lines in my new wrapper script: sys.path.remove('/usr/local/mailman/Mailman') sys.path.insert(0, '/usr/local/mailman') This generalized error wrapper would also eliminate some redundant error reporting code in admin. John, what do you think of removing the mm_ prefix from the module names? It's not really needed now that everything is pushed into the Mailman package. ----------------------- $prefix/scripts/subscribe --------------------- #! /usr/bin/env python # # Copyright (C) 1998 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. """Process listinfo form submission, ie subscriptions or roster requests.""" import sys from Mailman.subscribe import * try: sys.path.remove('/usr/local/mailman/Mailman') sys.path.insert(0, '/usr/local/mailman') from Mailman.subscribe import * except SystemExit: pass except: print "Content-type: text/html\n" print "

We're sorry, we hit a bug!

\n" print "If you would like to help us identify the problem, please " print "email a copy of this page to the webmaster for this site" print 'with a description of what happened. Thanks!' print "\n
"
	  print sys.argv
	  try:
	    import traceback
	    sys.stderr = sys.stdout
	    traceback.print_exc()
	  except:
	    print "[failed to get traceback]"
	  print "\n\n
" From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 5 18:13:50 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 5 Jun 1998 13:13:50 -0400 (EDT) Subject: [Mailman-Developers] Some improvement suggestions References: <199805282356.TAA09183@glyph.CNRI.Reston.Va.US> Message-ID: <13688.9845.106446.411919@anthem.CNRI.Reston.Va.US> Catching up on some issues... >>>>> "KM" == Ken Manheimer writes: KM> The first item is the ability to edit the new-member welcome KM> message.[...] KM> I think we would have to change the format substitutions, KM> currently all the "%s" order dependent style, to be KM> "%(real_name)s" dictionary substitutions, and otherwise do the KM> same kind of thing we do for the html templates. (Ah, i see KM> barry's done that in some recent checkins.) There is a bunch KM> of messages, some which may benefit from this treatment, other KM> might just clutter the list admins choices. Dragon subsequently mentioned Digicool's DocumentTemplate stuff, and that might certainly be a long-term goal. One of the short term things I'd like to do is to move most of the triple-quoted-strings of text that serve as templates, out of the .py files, and into text file in say, $prefix/data. Then convert them to using dictionary interpolation of values. I think this will help move us in a direction of being able to do things perhaps like I18N, using DocumentTemplate (perhaps), etc. It'll certainly make it easier to track down and change various publically visible texts. -Barry From mclay@nist.gov Fri Jun 5 18:31:06 1998 From: mclay@nist.gov (Michael McLay) Date: Fri, 5 Jun 1998 13:31:06 -0400 (EDT) Subject: [Mailman-Developers] file permissions In-Reply-To: <199806051632.MAA09671@fermi.eeel.nist.gov> References: <199806051632.MAA09671@fermi.eeel.nist.gov> Message-ID: <13688.9300.328001.568060@fermi.eeel.nist.gov> Nothing like following up to your own posts:-) Michael McLay writes: > In order to find the problem I did some > code reorganization. The $prefix/scripts/subscribe file was moved to > $prefix/Mailman/subscribe.py and the attached file was put in place of > the $prefix/scripts/subscribe file. > > This change should also reduce the script execution time slightly > because the longer script will be compiled to a .pyc file. I would > like to wrap all the other file in the $prefix/scripts/ in the same > manner. Once this is done the common.c program can be modified to > insert the PREFIX directory into the path rather than the MODULEDIR. Why not have the install script install the package Mailman directly in the user's site-python directory? Isn't that where it belongs? Then the path manipulation doesn't have to take place at all. > This will eliminate the need for the paths module in each of the > scripts and it would eliminate the need for the following lines in my > new wrapper script: > > sys.path.remove('/usr/local/mailman/Mailman') > sys.path.insert(0, '/usr/local/mailman') > > This generalized error wrapper would also eliminate some redundant > error reporting code in admin. I've done a little more work on generalizing the wrapper and adding some more optional debug information. The new module Mailman.debug contains two functions. The first one prints out the trace information. The second function will print out uid and gid information and the server environment variables. I turn off the more detailed messages using a debug flag in the $prefix/script/* scripts. If the Mailman is moved to site-python then the "import sys" statement could be deleted and the only statement inside the try block would be from Mailman.subscribe import * The SystemExit exception could be eliminated as well if the code in Mailman.subscribe were rearranged to eliminate the call to sys.exit(). The new arrangement also moves the debug code out of the scripts unless an exception is called. This reduces $prefix/script/subscribe file to the following: ########################## $prefix/script/subscribe ################# #! /usr/bin/env python # # Copyright (C) 1998 by the Free Software Foundation, Inc. # under the GNU General Public License See the License file in the top level # of this distribution for details. """Process subscribe form submission, ie subscriptions or roster requests.""" debug = 1 # turn on to enable environment parameter printing on errors import sys try: sys.path.remove('/usr/local/mailman/Mailman') sys.path.insert(0, '/usr/local/mailman') from Mailman.subscribe import * except SystemExit: pass except: import Mailman.debug Mailman.debug.print_trace() if debug: Mailman.debug.print_environ() From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 5 19:38:35 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 5 Jun 1998 14:38:35 -0400 (EDT) Subject: [Mailman-Developers] Pipermail References: <19980530193546.A8889@list.org> Message-ID: <13688.15003.865713.418391@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> To that end, I've put in provisions for passwords to use md5 JV> if crypt is not found. BTW, do you guys think md5 should be JV> the default, and crypt an option for those who add it to their JV> config? The only real advantage of crypt is that it is less JV> space intensive. Also, the way I have things set up at the JV> moment, if someone starts w/ the default distribution and JV> later recompiles Python to include crypt, all the old JV> passwords won't work, since they'll be in the md5 digest JV> format. For the reason you state, and since md5 is generally compiled in by default and crypt may not be, I'd say just go with md5 all the time. Space shouldn't be an issue really. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 5 19:46:55 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 5 Jun 1998 14:46:55 -0400 (EDT) Subject: [Mailman-Developers] CGI thoughts References: <19980604062605.C21640@list.org> Message-ID: <13688.15486.523947.679072@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> I was just thinking that it's a little much to require people JV> to go in and change their httpd configuration files. JV> Something we could do for the next release would be to provide JV> a single C binary, and tell people, "Put this in your system JV> cgi-bin directory". Then mailman URLS would become: JV> http://www.blah.com/cgi-bin/mailman/listinfo JV> I think while the URL may get a little bigger, it has the JV> advantage of being a lot less error-prone to install. Actually, Ken and I have talked about trying to *shorten* the URL, but we haven't come up with anything that'll work. It's a pain-tradeoff: how much pain do you put the installer through so that the larger number of users go through less pain (less typing) to get to the relevent pages? One thing you *could* do is install the `listinfo' and other wrapper binaries into cgi-bin, but call them mm_listinfo. I'm not crazy about that idea, and I don't think the mods to httpd conf files are all that daunting right now. I could be wrong though. -Barry From mclay@nist.gov Fri Jun 5 19:50:02 1998 From: mclay@nist.gov (Michael McLay) Date: Fri, 5 Jun 1998 14:50:02 -0400 (EDT) Subject: [Mailman-Developers] Mailman not delivering the mail Message-ID: <199806051850.OAA09749@fermi.eeel.nist.gov> The subscribe script execute and the Web page that is returned says a message will be sent to the subscriber, but no message arrives. I think something is wrong with gid or uid. By adding an intentional bug in Mailman/mm_utils.py the attached trace was triggered just after msg.GetSender() was called. Here's the bug that was added: import smtplib sender = msg.GetSender() findbug The debugging code that was added shows that the script is being executed as nobody. Am I correct in assuming that the mailer won't send out a message from nobody because nobody is not a trusted user? uid 65534 (nobody) euid 65534 (nobody) gid 55 (mailman) egid 50 (wwwcgi) - a group set up for cgi scripts. I also was curious about the wwwcgi egid. This is the gid setting used inside the httpd configuration, but when the subscribe cgi script create the data/pending_subscriptions.db the group is set to mailman. I'm confused about what the settings of all these things should be. I also found the layers used to send out the mail messages to be puzzling. Why the trip from maillist -> MailList -> mm_deliver -> Deliverer - > DeliverToUser -> mm_utils -> DeliverToUser. Why so much indirection? Michael ###################### REPLY MESSAGE ############################### We're sorry, we hit a bug! If you would like to help us identify the problem, please email a copy of this page to the webmaster for this site with a description of what happened. Thanks! ['/usr/local/mailman/scripts/subscribe'] Traceback (innermost last): File "/usr/local/mailman/scripts/subscribe", line 26, in ? from Mailman.subscribe import * File "/usr/local/mailman/Mailman/subscribe.py", line 151, in ? list.SendTextToUser(subject = "%s -- confirmation of subscription -- request %d" % \ File "/usr/local/mailman/Mailman/mm_deliver.py", line 105, in SendTextToUser mm_utils.SendTextToUser(subject, text, recipient, sender, File "/usr/local/mailman/Mailman/mm_utils.py", line 145, in SendTextToUser DeliverToUser(msg, recipient, add_headers=add_headers) File "/usr/local/mailman/Mailman/mm_utils.py", line 162, in DeliverToUser findbug NameError: findbug Execution privilage on the Server uid 65534 euid 65534 gid 55 egid 50 Environment Variable on the Server DOCUMENT_ROOT /usr/local/etc/httpd/htdocs REMOTE_HOST fermi CONTENT_TYPE application/x-www-form-urlencoded PATH_TRANSLATED /usr/local/etc/httpd/htdocs/test/ REMOTE_ADDR 129.6.185.13 GATEWAY_INTERFACE CGI/1.1 HTTP_COOKIE NVL_prefs=yvt0fss0000ss0s000000s000s00000000000000000000ss0s0000000000QQMichael%20McLay SERVER_PROTOCOL HTTP/1.0 HTTP_ACCEPT_LANGUAGE en REMOTE_PORT 25123 SERVER_PORT 80 HTTP_CONNECTION Keep-Alive HTTP_USER_AGENT Mozilla/4.05 [en] (X11; I; Linux 2.0.31 i686) HTTP_ACCEPT_CHARSET iso-8859-1,*,utf-8 HTTP_ACCEPT image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */* REQUEST_URI /mailman/subscribe/test/ PATH /usr/openwin/bin/xview:/usr/openwin/bin:/usr/local/bin:/etc:/usr/etc:/usr/lib:/usr/ucb:/bin:/usr/bin:/usr/etc/yp:/usr/plum:.:/usr/asap/bin:/usr/sbmt QUERY_STRING SCRIPT_FILENAME /usr/local/mailman/cgi-bin/subscribe CONTENT_LENGTH 71 HTTP_HOST www.eeel.nist.gov REQUEST_METHOD POST SCRIPT_NAME /mailman/subscribe SERVER_ADMIN kbrady@eeel.nist.gov SERVER_SOFTWARE Apache/1.2.5 PYTHONPATH /usr/local/mailman/Mailman PATH_INFO /test/ HTTP_REFERER http://www.eeel.nist.gov/mailman/listinfo/test SERVER_NAME www.eeel.nist.gov DEBUG Traceback (innermost last): File "/usr/local/mailman/scripts/subscribe", line 35, in ? Mailman.debug.print_environ() File "/usr/local/mailman/Mailman/debug.py", line 37, in print_environ print "DEBUG", debug NameError: debug Content-type: text/html Test Subscription results Confirmation from your email address is required, to prevent anyone from covertly subscribing you. Instructions are being sent to you at mclay@nist.gov. List run by mclay@nist.gov HTML generated by Mailman v 1.0b4 From steve@estel.uindy.edu Sat Jun 6 11:51:50 1998 From: steve@estel.uindy.edu (Steve Spicklemire) Date: Sat, 6 Jun 1998 05:51:50 -0500 (EST) Subject: [Mailman-Developers] Lock patches? Message-ID: <199806061051.FAA17364@estel.uindy.edu> Hi There, I'm using mailman on NeXTStep. One of the problems I ran into was the non-functionality of lock file support on NeXTStep. Soooo.... I cooked up a (more or less) satisfactory replacement. Would anyone else on the list be interested in the canges? Basically I create a normal file as the 'lock'. I create it with flags that require mailman to have exclusive write access to it. If another mailman process is already using the file, the create fails. There is an optional 'timeout' period, so that a hung/dead process won't cut out all other mailman processes. Are there other platforms where file locking is a problem? -steve From janne@avocado.pc.helsinki.fi Sat Jun 6 12:45:15 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 06 Jun 1998 14:45:15 +0300 Subject: [Mailman-Developers] '&' in an address Message-ID: I have never before seen an address with '&' in it, but now we have 'r&d.nylund@somewhere.com' on one of our lists. It is a problem, since the error log shows the followig when a message is posted to the list: Jun 06 07:55:39 1998 post: Traceback (innermost last): post: File "/home/mailman/scripts/mailowner", line 43, in ? post: if not current_list.bounce_processing or not current_list.ScanMessage(msg): post: File "/home/mailman/Mailman/mm_bouncer.py", line 407, in ScanMessage post: self.RegisterBounce(who, msg) post: File "/home/mailman/Mailman/mm_bouncer.py", line 83, in RegisterBounce post: self.LogMsg("bounce", report + "first") post: File "/home/mailman/Mailman/maillist.py", line 570, in LogMsg post: logf.write("%s\n" % (msg % args)) post: TypeError : not enough arguments for format string I have 1.0b4, but the subscription was made while we were running an earlier version. It's possible that the address found its way to the list through the admin's mass subscription. Either mailman shouldn't allow such addresses to be subscribed, or then the post script needs fixing. -- Janne From viega@list.org Sat Jun 6 13:56:29 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 05:56:29 -0700 Subject: [Mailman-Developers] file permissions In-Reply-To: <199806051632.MAA09671@fermi.eeel.nist.gov>; from Michael McLay on Fri, Jun 05, 1998 at 12:32:32PM -0400 References: <199806051632.MAA09671@fermi.eeel.nist.gov> Message-ID: <19980606055629.A12273@list.org> On Fri, Jun 05, 1998 at 12:32:32PM -0400, Michael McLay wrote: > This change should also reduce the script execution time slightly > because the longer script will be compiled to a .pyc file. I would > like to wrap all the other file in the $prefix/scripts/ in the same > manner. Once this is done the common.c program can be modified to > insert the PREFIX directory into the path rather than the MODULEDIR. > This will eliminate the need for the paths module in each of the > scripts and it would eliminate the need for the following lines in my > new wrapper script: One thing I'm thinking is that setenv() has portability problems, and it may be better not to use it at all. > John, what do you think of removing the mm_ prefix from the module > names? It's not really needed now that everything is pushed into the > Mailman package. I think it's fine for the prefix to go away, although I would probably either like to see the files that are used as mixins to retain a prefix, or see a better directory structure, especially since now there are going to be all of these CGI scripts in the package, too: there will be a lot of files in one directory, and it won't be easy to figure out what groups with what. John From viega@list.org Sat Jun 6 14:03:35 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 06:03:35 -0700 Subject: [Mailman-Developers] file permissions In-Reply-To: <13688.9300.328001.568060@fermi.eeel.nist.gov>; from Michael McLay on Fri, Jun 05, 1998 at 01:31:06PM -0400 References: <199806051632.MAA09671@fermi.eeel.nist.gov> <13688.9300.328001.568060@fermi.eeel.nist.gov> Message-ID: <19980606060335.B12273@list.org> On Fri, Jun 05, 1998 at 01:31:06PM -0400, Michael McLay wrote: > > Why not have the install script install the package Mailman directly > in the user's site-python directory? Isn't that where it belongs? > Then the path manipulation doesn't have to take place at all. Well, currently the install portion shouldn't be done as root, and unless Mailman has access to the site-python directly, it would become necessary. I think it's a good thing to be able to install this software without having to have root access. True, unless you run an MTA like qmail, you're still going to need someone to add mail aliases for you. But we could add an optional make root-install target or something like that that would go back to using the alias wrapper, etc. > I've done a little more work on generalizing the wrapper and adding > some more optional debug information. The new module Mailman.debug > contains two functions. The first one prints out the trace > information. The second function will print out uid and gid > information and the server environment variables. I turn off the more > detailed messages using a debug flag in the $prefix/script/* scripts. Nice work on the wrapper code! John From viega@list.org Sat Jun 6 14:25:28 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 06:25:28 -0700 Subject: [Mailman-Developers] Mailman not delivering the mail In-Reply-To: <199806051850.OAA09749@fermi.eeel.nist.gov>; from Michael McLay on Fri, Jun 05, 1998 at 02:50:02PM -0400 References: <199806051850.OAA09749@fermi.eeel.nist.gov> Message-ID: <19980606062528.C12273@list.org> On Fri, Jun 05, 1998 at 02:50:02PM -0400, Michael McLay wrote: > > The subscribe script execute and the Web page that is returned says > a message will be sent to the subscriber, but no message arrives. > I think something is wrong with gid or uid. If something is wrong w/ the gid, it should get logged to your system maillog before the scripts ever get run, so it is more likely something wrong with file permissions. From an earlier post I'm wondering if you perhaps ran make install as root, and if that might not end up causing problems? Assuming the program actually trys to deliver the message, it will hopefully throw an error. However, it probably won't do so where you can get a trace, since the delivery code opens a pipe to the delivery code to avoid smtp hanging hanging mailman. So I'd add debug code to the deliver script and the contact_transport script that log stderr to a file, so you can see if any file problems are making it up to the Python level. > The debugging code that was added shows that the script is being > executed as nobody. Am I correct in assuming that the mailer won't > send out a message from nobody because nobody is not a trusted user? No, that should not be the case. Trusted user status means that if you set the sender via -f when calling sendmail directly, sendmail will allow that. Otherwise, it will put in a header saying: "Warning: user nobody set sender to 'blah' using -f". Since we don't even directly call Sendmail, we don't have to worry about trusted user status. > > uid 65534 (nobody) > euid 65534 (nobody) > gid 55 (mailman) > egid 50 (wwwcgi) - a group set up for cgi scripts. > > I also was curious about the wwwcgi egid. This is the gid setting > used inside the httpd configuration, but when the subscribe cgi script > create the data/pending_subscriptions.db the group is set to mailman. Right. All of the files should be created as belonging to group mailman. The httpd and smtpd each run with different gids. To let them modify files in group mailman, yet keep them accessable by the mailman user, we have setgid wrapper scripts (written in C for portability). > I'm confused about what the settings of all these things should be. Your settings look right. Again, if they were wrong, you wouldn't even get to the Python; the C wrapper would log to your system mail log. > I also found the layers used to send out the mail messages to be > puzzling. Why the trip from maillist -> MailList -> mm_deliver -> > Deliverer - > DeliverToUser -> mm_utils -> DeliverToUser. Why so much > indirection? I don't understand why you're mixing methods, classes and files here. The path is just: maillist.DeliverToUser -> mm_utils.DeliverToUser. .DeliverToUser() is a method in maillist for delivering mail. However, sometimes mail gets delivered that is not associated with any particular list (and thus there will be no list object). Thefore, mailman.DeliverToUser() makes a call out to the utils module. Duplicating the code in both places would lead to maintainance problems. Now if you're wondering why the functionality in mm_deliver is not just brought into maillist.py, that's done intentionally to keep related functionality together, in as small a unit as necessary. I find single flat files with tons of functionality tend to develop into spaghetti rather quickly. From viega@list.org Sat Jun 6 14:37:22 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 06:37:22 -0700 Subject: [Mailman-Developers] Lock patches? In-Reply-To: <199806061051.FAA17364@estel.uindy.edu>; from Steve Spicklemire on Sat, Jun 06, 1998 at 05:51:50AM -0500 References: <199806061051.FAA17364@estel.uindy.edu> Message-ID: <19980606063722.E12273@list.org> On Sat, Jun 06, 1998 at 05:51:50AM -0500, Steve Spicklemire wrote: > > Hi There, > > I'm using mailman on NeXTStep. One of the problems I ran into > was the non-functionality of lock file support on NeXTStep. Soooo.... > I cooked up a (more or less) satisfactory replacement. Would anyone > else on the list be interested in the canges? Basically I create a > normal file as the 'lock'. I create it with flags that require mailman > to have exclusive write access to it. If another mailman process is > already using the file, the create fails. There is an optional > 'timeout' period, so that a hung/dead process won't cut out all other > mailman processes. Are there other platforms where file locking is a > problem? Steve, The locking that we currently do definitely needs replacement. It doesn't work on FreeBSD, apparently. If you've got a portable solution, we'd love to have it. And if you're interested in improving the granularity of locking throughout the system, let us know, we'd like that to happen as well :) John From viega@list.org Sat Jun 6 14:44:28 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 06:44:28 -0700 Subject: [Mailman-Developers] '&' in an address In-Reply-To: ; from Janne Sinkkonen on Sat, Jun 06, 1998 at 02:45:15PM +0300 References: Message-ID: <19980606064428.F12273@list.org> On Sat, Jun 06, 1998 at 02:45:15PM +0300, Janne Sinkkonen wrote: > > I have never before seen an address with '&' in it, but now we have > 'r&d.nylund@somewhere.com' on one of our lists. It is a problem, > since the error log shows the followig when a message is posted to the > list: ... > I have 1.0b4, but the subscription was made while we were running an > earlier version. It's possible that the address found its way to the > list through the admin's mass subscription. > > Either mailman shouldn't allow such addresses to be subscribed, or > then the post script needs fixing. I would have to look this one up, but if & is supposed to be valid in email addresses, we should probably try to support it, even though it's a special character in URLs, and will thus be a bit tricky to work around. I'll have to look this one up, and implement the right solution (anyone else is welcome to beat me to it :). Let's say I will make sure there's some sort of solution by b5. For now, I would suggest you remove the address from the list. The admin membership page may work for this, or you may need to fire up Python and remove them by hand: import maillist m = maillist.MailList('whatever-list') m.members.remove('offending_address') m.Save() Then exit the interpreter so the list will Unlock, or call m.Unlock(). If he's a digest member, use m.digest_members instead... John From janne@avocado.pc.helsinki.fi Sat Jun 6 14:48:48 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 06 Jun 1998 16:48:48 +0300 Subject: [Mailman-Developers] '&' in an address In-Reply-To: John Viega's message of "Sat, 6 Jun 1998 06:44:28 -0700" References: <19980606064428.F12273@list.org> Message-ID: John Viega writes: > will make sure there's some sort of solution by b5. For now, I would > suggest you remove the address from the list. No problem here. If somebody has such a mail address, (s)he should be prepared to suffer. :) The admin removed it already, apparently it was easy because he managed to do it without any help. -- Janne From viega@list.org Mon Jun 8 22:29:43 1998 From: viega@list.org (John Viega) Date: Mon, 8 Jun 1998 14:29:43 -0700 Subject: [Mailman-Developers] fyi Message-ID: <19980608142943.A1701@list.org> I just wanted to let everyone know that I'm not going to be as quick to respond as I normally try to be for the rest of the week. I'm just starting a new job, and I have to still set stuff up before I have outside access. So I'm only going to be reading mail once a day until probably the end of the week. Over the weekend I plan to catch up on integrating patches from this list and making the changes I promised a day or so ago. John From janne@avocado.pc.helsinki.fi Tue Jun 9 16:39:06 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 09 Jun 1998 18:39:06 +0300 Subject: [Mailman-Developers] Does bouncing by header regexps really work? Message-ID: I don't think so. Or do I have something wrong with these? > content-type: multipart > x-ms-attachment: .* > return-receipt-to: .* -- Janne From viega@list.org Tue Jun 9 16:47:52 1998 From: viega@list.org (John Viega) Date: Tue, 9 Jun 1998 08:47:52 -0700 Subject: [Mailman-Developers] Does bouncing by header regexps really work? In-Reply-To: ; from Janne Sinkkonen on Tue, Jun 09, 1998 at 06:39:06PM +0300 References: Message-ID: <19980609084752.A9178@list.org> I won't have the chance to look until later this week, but I'm wondering if it may be a case sensitivity problem? John On Tue, Jun 09, 1998 at 06:39:06PM +0300, Janne Sinkkonen wrote: > > I don't think so. Or do I have something wrong with these? > > > content-type: multipart > > x-ms-attachment: .* > > return-receipt-to: .* > > -- > Janne > > _______________________________________________ > Mailman-Developers maillist - Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From janne@avocado.pc.helsinki.fi Tue Jun 9 17:59:31 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 09 Jun 1998 19:59:31 +0300 Subject: [Mailman-Developers] Does bouncing by header regexps really work? In-Reply-To: John Viega's message of "Tue, 9 Jun 1998 08:47:52 -0700" References: <19980609084752.A9178@list.org> Message-ID: John Viega writes: > I won't have the chance to look until later this week, but I'm > wondering if it may be a case sensitivity problem? OK, this seems to help: --- maillist.py~ Thu Jun 4 23:12:28 1998 +++ maillist.py Tue Jun 9 19:39:45 1998 @@ -718,13 +718,13 @@ continue else: try: - h, e = re.split(":[ ]*", line) - all.append((h, e, line)) + h, e = re.split(":[ ]*", stripped) + all.append((h, e, stripped)) except ValueError: # Whoops - some bad data got by: self.LogMsg("config", "%s - " "bad bounce_matching_header line %s" - % (self.real_name, `line`)) + % (self.real_name, `stripped`)) return all @@ -748,7 +748,7 @@ and f[l] == ":" and string.lower(field) == string.lower(f[0:l])): # Non-continuation line - trim header name: - subjs.append(f[l+1:]) + subjs.append(f[l+2:]) elif not subjs: # Whoops - non-continuation that matches? subjs.append(f) From scott@chronis.pobox.com Fri Jun 12 00:35:25 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 11 Jun 1998 19:35:25 -0400 Subject: [Mailman-Developers] smtplib buglet in 1.0b4 Message-ID: <19980611193525.15573@chronis.icgroup.com> the smtplib module in mailman 1.0b4 doesn't quite use strict rfc complient smtp syntax. Vmailer is strict about it, and other mailers may be as well. I changed smtplib.py so it sends "MAIL FROM:
" and "RCPT TO:
" instead of "MAIL FROM: address" and "RCPT TO: address". scott here's a patch: chronis 7:32pm $ diff -c /home/mm/Mailman/smtplib.py /home/scott/mailman-1.0b4/modules/smtplib.py *** /home/mm/Mailman/smtplib.py Thu Jun 11 19:30:06 1998 --- /home/scott/mailman-1.0b4/modules/smtplib.py Fri May 29 23:09:21 1998 *************** *** 64,77 **** if headers: hlines = string.split(headers, '\n') lines = string.split(text, '\n') ! self._sock.send('MAIL FROM: <%s>\r\n' % frm) self.getresp() if type(to) == types.StringType: ! self._sock.send('RCPT TO: <%s>\r\n' % to) self.getresp() else: for item in to: ! self._sock.send('RCPT TO: <%s>\r\n' % item) self.getresp() self._sock.send('DATA\r\n') self.getresp() --- 64,77 ---- if headers: hlines = string.split(headers, '\n') lines = string.split(text, '\n') ! self._sock.send('MAIL FROM: %s\r\n' % frm) self.getresp() if type(to) == types.StringType: ! self._sock.send('RCPT TO: %s\r\n' % to) self.getresp() else: for item in to: ! self._sock.send('RCPT TO: %s\r\n' % item) self.getresp() self._sock.send('DATA\r\n') self.getresp() From dragondm@nexus.Integral.org Fri Jun 12 04:05:39 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Thu, 11 Jun 1998 22:05:39 -0500 (CDT) Subject: [Mailman-Developers] smtplib buglet in 1.0b4 In-Reply-To: <19980611193525.15573@chronis.icgroup.com> Message-ID: On Thu, 11 Jun 1998, Scott wrote: > Date: Thu, 11 Jun 1998 19:35:25 -0400 > From: Scott > To: mailman-developers@python.org > Subject: [Mailman-Developers] smtplib buglet in 1.0b4 > > > the smtplib module in mailman 1.0b4 doesn't quite use strict rfc > complient smtp syntax. Vmailer is strict about it, and other mailers > may be as well. I changed smtplib.py so it sends "MAIL FROM: >
" and "RCPT TO:
" instead of "MAIL FROM: address" > and "RCPT TO: address". Hmm.... Actually, I'm pretty sure that not having the <>'s on the address is perfectly fine according to the RFC's, as long as you don't have any comments in the address. Are you sure this isn't a glitch w/ Vmailer? PS, is there any specific reason that MailMan keeps using it's own smtplib, rather than the one that is part of the Python 1.5.1 standard library? -The Dragon De Monsyne From viega@list.org Fri Jun 12 13:46:44 1998 From: viega@list.org (John Viega) Date: Fri, 12 Jun 1998 05:46:44 -0700 Subject: [Mailman-Developers] smtplib buglet in 1.0b4 In-Reply-To: ; from The Dragon De Monsyne on Thu, Jun 11, 1998 at 10:05:39PM -0500 References: <19980611193525.15573@chronis.icgroup.com> Message-ID: <19980612054644.A30905@list.org> On Thu, Jun 11, 1998 at 10:05:39PM -0500, The Dragon De Monsyne wrote: > > Hmm.... Actually, I'm pretty sure that not having the <>'s on the > address is perfectly fine according to the RFC's, as long as you don't > have any comments in the address. Are you sure this isn't a glitch w/ > Vmailer? Actually, Scott's right, RFC 821 does indeed require angle brackets. > PS, is there any specific reason that MailMan keeps using it's own > smtplib, rather than the one that is part of the Python 1.5.1 standard > library? The initial reason was that the user interface on Mailman's SMTP lib was easier to program to. For example, it was a lot easier to send a single message to a list of recipients. The other reason is that Mailman releases should come out more often than Python releases. Both smtplibs are pretty new software, and are pretty likely to change. The patch from Scott is a good example. So at some point I'd be happy to use the other smtplib, but we'd still end up providing a more up to date version than available w/ Python, probably. John From dragondm@nexus.Integral.org Fri Jun 12 15:02:18 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Fri, 12 Jun 1998 09:02:18 -0500 (CDT) Subject: [Mailman-Developers] smtplib buglet in 1.0b4 In-Reply-To: <19980612054644.A30905@list.org> Message-ID: On Fri, 12 Jun 1998, John Viega wrote: > On Thu, Jun 11, 1998 at 10:05:39PM -0500, The Dragon De Monsyne wrote: > > > > Hmm.... Actually, I'm pretty sure that not having the <>'s on the > > address is perfectly fine according to the RFC's, as long as you don't > > have any comments in the address. Are you sure this isn't a glitch w/ > > Vmailer? > > Actually, Scott's right, RFC 821 does indeed require angle brackets. Hmm... Indeed. Confusing language oon the rfc on that... I've fixed this in the latest edition of the standard smtplib (found at http://www.integral.org/~dragondm/python/smtplib.py) I'll be posting that to c.l.python tonight, too. > > PS, is there any specific reason that MailMan keeps using it's own > > smtplib, rather than the one that is part of the Python 1.5.1 standard > > library? > > The initial reason was that the user interface on Mailman's SMTP lib > was easier to program to. For example, it was a lot easier to send a > single message to a list of recipients. The other reason is that Hmmm?? Howso? I've looked at the code, as far as I can tell, the std smtplib has all the same interfaces the mm smtplib does, only differing in the name of the smtp class. The python-lib smtplib also has a method, called 'sendmail' which does the whole transaction for a list of recipients. > Mailman releases should come out more often than Python releases. > Both smtplibs are pretty new software, and are pretty likely to > change. The patch from Scott is a good example. So at some point I'd > be happy to use the other smtplib, but we'd still end up providing a > more up to date version than available w/ Python, probably. Prolly. -The Dragon De Monsyne From viega@list.org Fri Jun 12 15:54:53 1998 From: viega@list.org (John Viega) Date: Fri, 12 Jun 1998 07:54:53 -0700 Subject: [Mailman-Developers] smtplib buglet in 1.0b4 In-Reply-To: ; from The Dragon De Monsyne on Fri, Jun 12, 1998 at 09:02:18AM -0500 References: <19980612054644.A30905@list.org> Message-ID: <19980612075453.B4123@list.org> On Fri, Jun 12, 1998 at 09:02:18AM -0500, The Dragon De Monsyne wrote: > On Fri, 12 Jun 1998, John Viega wrote: > The python-lib smtplib also has a method, called 'sendmail' which does the > whole transaction for a list of recipients. Okay, I missed that one. There was also some other issue about massaging header data into the format needed, and then there was something else I can't remember. Nothing was a big deal, but it was less time consuming to program to a module I wrote :) From viega@list.org Fri Jun 12 20:21:53 1998 From: viega@list.org (John Viega) Date: Fri, 12 Jun 1998 12:21:53 -0700 Subject: [Mailman-Developers] Revisiting file locking In-Reply-To: <13647.26901.437384.156650@anthem.CNRI.Reston.Va.US>; from Barry A. Warsaw on Tue, May 05, 1998 at 03:45:47PM -0400 References: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> <13647.26901.437384.156650@anthem.CNRI.Reston.Va.US> Message-ID: <19980612122153.A5925@list.org> On Tue, May 05, 1998 at 03:45:47PM -0400, Barry A. Warsaw wrote: > > I wasn't able to dig up the Netscape Mail + File Locking article that > Jamie Zawinski wrote. I don't seem to have it in my bookmarks or > easily grepable and a web search turned up only stale links. I > emailed Jaime to see if he's got an updated link and will forward that > if I get a response. Hmm, I got it on a web search: http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/movemail.html > If I remember correctly, the article basically justifies the point of > view that the best portable way to do resource locking across NFS is > by creating a file with O_CREAT | O_EXCL. That's what's meant by > "dot-file" locking because you usually create a .file that is mostly > invisible. I have yet to find much on whether there are race conditions w/ O_EXCL and directories. However, the open() man page on my system has the following to say: O_EXCL When used with O_CREAT, if the file already exists it is an error and the open will fail. O_EXCL is broken on NFS file systems, programs which rely on it for performing locking tasks will contain a race condition. The solution for performing atomic file locking using a lockfile is to create a unique file on the same fs (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile and use stat(2) on the unique file to check if its link count has increased to 2. Do not use the return value of the link() call. The problem with this solution is, what happens if a process dies while holding a lock? The most straightforward solution is to drop a pid into the lock file, and have the process wishing to grab the lock check to see if the process is still running. Also, this is a polling-based solution, which isn't incredibly desirable either... But I guess there's no way around that one. Ok, I'll implement a general library call for this one, I guess. BTW, No one has put any effort into making pipermail portable yet, have they? John From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 21:34:38 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 16:34:38 -0400 (EDT) Subject: [Mailman-Developers] Re: file permissions References: <199806051632.MAA09671@fermi.eeel.nist.gov> Message-ID: <13697.36475.781015.885594@anthem.CNRI.Reston.Va.US> Normal Friday catchup... >>>>> "MM" == Michael McLay writes: MM> Either I screwed up when I followed the installation MM> instructions or else the file ownership set by "make install" MM> incorrectly set the group of the executables in MM> $prefix/cgi-bin to that of the super user. In order for the MM> scripts to work the group had to be manually set to "mailman". One thing the next version will have is more checking of the permissions on $prefix. If the permissions or group ownership aren't correct, configure stops. This should prevent those problems from cascading to all the other installed files. MM> This change should also reduce the script execution time MM> slightly because the longer script will be compiled to a .pyc MM> file. That may or many not really be an issue. I suspect starting up the Python interpreter for every script is the killer, rather than parsing the script files. subscribe for example is less than 200 lines of Python. Admin is certainly the longest at 856 lines. One thing that's high on my list is to really fix the packagization of Mailman. I think given that, many things get simpler. You still need a little path hacking, but that can be done in a more portable way than even the paths.py files. And then most of the functionality in the scripts can be moved into package libraries. This will improve the modularity A LOT. I just finished doing this with a project here at CNRI and it makes a big difference. MM> John, what do you think of removing the mm_ prefix from the MM> module names? This can easily happen, with no chance of name collisions, even for the mixin classes. I also want to rename some of the .py files to match the contained class names (better style). So there are several things of this nature I want to work on, but my meta question is this: From a technical standpoint, how close to a 1.0 release are we? What critical features are still missing (integration with Pipermail is key), and what showstopper bugs are still lurking? Should I continue to do code clean up now, or wait until after 1.0? -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 21:35:48 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 16:35:48 -0400 (EDT) Subject: [Mailman-Developers] file permissions References: <199806051632.MAA09671@fermi.eeel.nist.gov> <13688.9300.328001.568060@fermi.eeel.nist.gov> Message-ID: <13697.37108.420182.964834@anthem.CNRI.Reston.Va.US> >>>>> "MM" == Michael McLay writes: MM> I've done a little more work on generalizing the wrapper and MM> adding some more optional debug information. The new module MM> Mailman.debug contains two functions. BTW, I *really* like this idea, Mike. I like the traceback showing up on the Web page, especially at this stage in Mailman's life. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 21:36:45 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 16:36:45 -0400 (EDT) Subject: [Mailman-Developers] file permissions References: <199806051632.MAA09671@fermi.eeel.nist.gov> <19980606055629.A12273@list.org> Message-ID: <13697.37178.321774.675303@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> One thing I'm thinking is that setenv() has portability JV> problems, and it may be better not to use it at all. Probably wise. JV> I think it's fine for the prefix to go away, although I would JV> probably either like to see the files that are used as mixins JV> to retain a prefix, or see a better directory structure, A better directory structure!!!! :-) -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 23:05:26 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 18:05:26 -0400 (EDT) Subject: [Mailman-Developers] Re: [JPython] rexec module, string escapes and file ASCII<->Unicode References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> Message-ID: <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> >>>>> "JH" == Jim Hugunin writes on the >>>>> jpython-interest list: JH> Everybody on this list should know that the mailman software - JH> which handles the jpython-interest mailing list - is JH> reasonably agressive in suspending mail to any addresses that JH> produce excessive bounce messages. This feature makes the JH> life of the list administrator much more pleasant, but can JH> occasionally lead to people having their mail suspended who JH> still want to be on the list. I've been thinking about bounce policy. I wonder if what Mailman should do is have the same policy for all bounces, regardless of whether it looks permanent (user unknown) or not. Once the bounce threshold is reached and the account is disabled, a cron job could be used to retry the address for some number of days. If the account becomes live again, it can be re-enabled. Or perhaps the user has to explicitly re-enable it when they finally see these warning messages. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 23:09:51 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 18:09:51 -0400 (EDT) Subject: [Mailman-Developers] smtplib buglet in 1.0b4 References: <19980611193525.15573@chronis.icgroup.com> Message-ID: <13697.42718.170852.188059@anthem.CNRI.Reston.Va.US> >>>>> "TDDM" == The Dragon De Monsyne writes: TDDM> PS, is there any specific reason that MailMan keeps TDDM> using it's own smtplib, rather than the one that is part of TDDM> the Python 1.5.1 standard library? Whatever happens, it would be really nice if a single integrated smtplib.py made its way back into the Python distribution. Mailman will still have to distribute a copy for the time being, but I can work out a distrib tree to handle that (as I'll have to do anyway with Cookie.py). -Barry From viega@list.org Sat Jun 13 03:12:12 1998 From: viega@list.org (John Viega) Date: Fri, 12 Jun 1998 19:12:12 -0700 Subject: [Mailman-Developers] Re: bounce agressiveness In-Reply-To: <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US>; from Barry A. Warsaw on Fri, Jun 12, 1998 at 06:05:26PM -0400 References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> Message-ID: <19980612191212.D9003@list.org> On Fri, Jun 12, 1998 at 06:05:26PM -0400, Barry A. Warsaw wrote: > > >>>>> "JH" == Jim Hugunin writes on the > >>>>> jpython-interest list: > > JH> Everybody on this list should know that the mailman software - > JH> which handles the jpython-interest mailing list - is > JH> reasonably agressive in suspending mail to any addresses that > JH> produce excessive bounce messages. This feature makes the > JH> life of the list administrator much more pleasant, but can > JH> occasionally lead to people having their mail suspended who > JH> still want to be on the list. > > I've been thinking about bounce policy. I wonder if what Mailman > should do is have the same policy for all bounces, regardless of > whether it looks permanent (user unknown) or not. Once the bounce > threshold is reached and the account is disabled, a cron job could be > used to retry the address for some number of days. If the account > becomes live again, it can be re-enabled. Or perhaps the user has to > explicitly re-enable it when they finally see these warning messages. Sure, but then again I've never seen an error message as a list admin for a non-transient error that was actually transient (ie, 'no such user' only lasting a few minutes). It really doesn't matter, might as well be safe about it. I think the real problem is addresses that can't be reached for a little too long for whatever reason. I do like the idea of sending daily mail for actually an extended period of time. Maybe it could be every 3rd day for 60 days, since mail generally doesn't get rejected as undeliverable for 3 days. We could use the confirmation mechanism to let people get back on the list. By the way, does the confirmation mechanism clean up unanswered confirmation mail? How long does it sit there if so? If not, we probably need to add that :) John From viega@list.org Sat Jun 13 15:46:51 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 07:46:51 -0700 Subject: [Mailman-Developers] File Locking Message-ID: <19980613074651.A13995@list.org> --QCug/ao/Qh+gIAGv Content-Type: text/plain; charset=us-ascii I'm going to attach what I hope is a portable module for file locking. I don't have time to integrate it with Mailman today, unfortunately, but if someone wants to try it out, or give suggestions for improvements to this code, let me know. John --QCug/ao/Qh+gIAGv Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="flock.py" # Copyright (C) 1998 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. # # flock.py: Portable file locking. John Viega, Jun 13, 1998 """Portable (?) file locking with timeouts. This code should work with all versions of NFS. The algorithm was suggested by the GNU/Linux open() man page. Make sure no malicious people have access to link() to the lock file. """ # Potential change: let the locker insert a field saying when he promises # to be done with the lock, so if he needs more time than the other # processes think he needs, he can say so. import socket, os, time DEFAULT_HUNG_TIMEOUT = 30 DEFAULT_SLEEP_INTERVAL = .25 AlreadyCalledLockError = "AlreadyCalledLockError" NotLockedError = "NotLockedError" TimeOutError = "TimeOutError" class FileLock: def __init__(self, lockfile, hung_timeout = DEFAULT_HUNG_TIMEOUT, sleep_interval = DEFAULT_SLEEP_INTERVAL): self.lockfile = lockfile self.hung_timeout = hung_timeout self.sleep_interval = sleep_interval self.tmpfname = "%s.%s.%d" % (lockfile, socket.gethostname(), os.getpid()) self.locked = 0 if not os.path.exists(self.lockfile): try: file = open(self.lockfile, "w+") except IOError: pass # Note that no one new can grab the lock once we've opened our # tmpfile until we close it, even if we don't have the lock. So # checking the PID and stealing the lock are garunteed to be atomic. def lock(self, timeout = 0): """Blocks until the lock can be obtained. Raises a TimeOutError exception if a positive timeout value is given and that time elapses before the lock is obtained. """ if timeout > 0: timeout_time = time.time() + timeout last_pid = -1 if self.locked: raise AlreadyCalledLockError while 1: os.link(self.lockfile, self.tmpfname) if os.stat(self.tmpfname)[3] == 2: file = open(self.tmpfname, 'w+') file.write(`os.getpid(),self.tmpfname`) file.close() self.locked = 1 break if timeout and timeout_time < time.time(): raise TimeOutError file = open(self.tmpfname, 'r') try: pid,winner = eval(file.read()) except SyntaxError: # no info in file... *can* happen file.close() continue file.close() if pid <> last_pid: last_pid = pid stime = time.time() if (stime + self.hung_timeout < time.time()) and self.hung_timeout > 0: file = open(self.tmpfname, 'w+') file.write(`os.getpid(),self.tmpfname`) os.unlink(winner) self.locked = 1 break os.unlink(self.tmpfname) time.sleep(self.sleep_interval) # This could error if the lock is stolen. You must catch it. def unlock(self): if not self.locked(): raise NotLockedError os.unlink(self.tmpfname) def locked(self): return os.path.exists(self.tmpfname) and self.locked --QCug/ao/Qh+gIAGv-- From viega@list.org Sat Jun 13 16:13:03 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 08:13:03 -0700 Subject: [Mailman-Developers] Re: file permissions In-Reply-To: <13697.36475.781015.885594@anthem.CNRI.Reston.Va.US>; from Barry A. Warsaw on Fri, Jun 12, 1998 at 04:34:38PM -0400 References: <199806051632.MAA09671@fermi.eeel.nist.gov> <13697.36475.781015.885594@anthem.CNRI.Reston.Va.US> Message-ID: <19980613081303.A14039@list.org> On Fri, Jun 12, 1998 at 04:34:38PM -0400, Barry A. Warsaw wrote: > > > >From a technical standpoint, how close to a 1.0 release are we? What > critical features are still missing (integration with Pipermail is > key), and what showstopper bugs are still lurking? Should I continue > to do code clean up now, or wait until after 1.0? I anticipate a few more beta releases first. No one is jumping up to do the Pipermail stuff in a portable way, so that's going to delay us time wise. Locking needs to be improved a bit; it would be nice if the granularity were better. I think there's a need for more documentation before we leave beta. All password-pages should use the cookie code, AND have the extra password field for cookie-less people, which currently isn't done at all. I'd also like to move all web pages and mail messages that should be editable out into the template dirs. That's less important, though. We also need to go through and make sure all email addresses that can be entered can't be used in an insecure manner (check for |'s etc. in every address, entered from anywhere). Also, not only would I like to see archiving back in, I also want to see it with a search engine, by default. As for bugs, there are some on the Mailman TODO page that may or may not be current. Nothing major, though. There's one big one that's been reported and I haven't found yet. I'll send it on in another email, perhaps someone else can find it before I do. That's what I'd like to see. And it really isn't all that much, but I'll bet it's at least a month off. Plus, we should try to have a stable beta for a few weeks before we release it as 1.0. Oh, beyond that, we need to make sure we get all permissionsissues squared away. Pipermail's a good example of that one. Maybe Andrew should try to get it blessed as the GNU mail archiver before we bundle it? :) John From viega@list.org Sat Jun 13 16:15:49 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 08:15:49 -0700 Subject: [Mailman-Developers] mass subscribe bug Message-ID: <19980613081549.B14039@list.org> Here's the error previously mentioned, as reported to me, w/ excess headers, etc. trimmed. John ----- Forwarded message from Matt Chatterley ----- Occurs when trying to mass subscribe. Seems to have started after upgrading python, version information: Python 1.5.1 (#1, May 29 1998, 12:50:41) [GCC 2.7.2.3] on linux2 We're sorry, we hit a bug! If you would like to help us identify the problem, please email a copy of this page to the webmaster for this site with a description of what happened. Thanks! Traceback (innermost last): File "/home/mailman/scripts/admin", line 823, in ? main() File "/home/mailman/scripts/admin", line 157, in main is_auth = isAuthenticated(lst) File "/home/mailman/scripts/admin", line 103, in isAuthenticated inp = base64.decodestring(c[list_name + "-admin"].value) File "/usr/lib/python1.5/base64.py", line 46, in decodestring decode(f, g) File "/usr/lib/python1.5/base64.py", line 32, in decode s = binascii.a2b_base64(line) Error: Incorrect padding From viega@list.org Sat Jun 13 16:25:28 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 08:25:28 -0700 Subject: [Mailman-Developers] Archiving? Message-ID: <19980613082528.A14242@list.org> I got several messages dated June 12 from Barry yesterday, but none of them showed up in the pipermail archive for this list. Anyone guess what's up? John From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Sat Jun 13 16:48:05 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Sat, 13 Jun 1998 11:48:05 -0400 (EDT) Subject: [Mailman-Developers] Re: bounce agressiveness References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> <19980612191212.D9003@list.org> Message-ID: <13698.40646.976228.287080@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Sure, but then again I've never seen an error message as a JV> list admin for a non-transient error that was actually JV> transient (ie, 'no such user' only lasting a few minutes). It JV> really doesn't matter, might as well be safe about it. Right. Guido had a good point which was that it might simplify the code such that all bounces are treated the same. Treating permanent failures like transient failures with retries doesn't seem like it could hurt. JV> I do like the idea of sending daily mail for actually an JV> extended period of time. Maybe it could be every 3rd day for JV> 60 days, since mail generally doesn't get rejected as JV> undeliverable for 3 days. We could use the confirmation JV> mechanism to let people get back on the list. Exactly! JV> By the way, does the confirmation mechanism clean up JV> unanswered confirmation mail? How long does it sit there if JV> so? If not, we probably need to add that :) I don't know, but I agree! -Barry From scott@chronis.pobox.com Sat Jun 13 22:22:25 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 13 Jun 1998 17:22:25 -0400 Subject: [Mailman-Developers] mass subscribe bug Message-ID: <19980613172225.39336@chronis.icgroup.com> I saw this error once in another context (still with mailman and cookies, but with the "private" cgi). The error occurred when a recent version of IE on a mac attempted to deal with the cookie. I think that the bug has one of two causes: either the browser is incorrectly handling the characters in the cookie, or there is something about the way that the base64 module encodes and decodes that won't work with http. Since I've only seen the error on what i consider a questionable browser i suspect that it's the browser's fault. in either case, i haven't the first clue about a fix. scott On Sat, Jun 13, 1998 at 08:15:49AM -0700, John Viega wrote: | Here's the error previously mentioned, as reported to me, w/ excess | headers, etc. trimmed. | | John | | ----- Forwarded message from Matt Chatterley ----- | | Occurs when trying to mass subscribe. | | Seems to have started after upgrading python, version information: | | Python 1.5.1 (#1, May 29 1998, 12:50:41) [GCC 2.7.2.3] on linux2 | | We're sorry, we hit a bug! | | If you would like to help us identify the problem, please email a copy | of this page to the webmaster for this site with a description of what | happened. Thanks! | | Traceback (innermost last): | File "/home/mailman/scripts/admin", line 823, in ? | main() | File "/home/mailman/scripts/admin", line 157, in main | is_auth = isAuthenticated(lst) | File "/home/mailman/scripts/admin", line 103, in isAuthenticated | inp = base64.decodestring(c[list_name + "-admin"].value) | File "/usr/lib/python1.5/base64.py", line 46, in decodestring | decode(f, g) | File "/usr/lib/python1.5/base64.py", line 32, in decode | s = binascii.a2b_base64(line) | Error: Incorrect padding | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers -----End of forwarded message----- From Corbett J. Klempay" Hey, I just saw the comment about integrating archive search functionality in 1.0 or another not-so-distant release. I was thinking that maybe I could use this as a chance to learn Python; I just finished taking Info Retrieval & Text Understanding this past semester, and ended up doing a vector search engine for the final project. I'm not saying that I can for sure do this; I think I might see if I can find some time to screw around with it, and if it develops ok, then bonus. Some questions, though: - how big of an archive should this be scalable to? I'm thinking this because some models (like the vector model used for my project) get good accuracy, but suck as far as resource usage (like our engine dealt with ~2000 text documents from an online database and would suck up ~80 MB of RAM per query, and would take almost 20 seconds just to load the pre-indexed document clusters from disk; the query itself only takes like 1-2 seconds on a K6-233, but the startup time of 15+ seconds blows). With large corpa, it might be necessary to implement some persistence; it just takes so annoyingly long to load even pre-indexed stuff from disk (and our queries were on a K6-233 with 128 MB; heheh...think how a P-90 with 32 MB would fare :) (or worse yet!) (for the really curious people, the engine is at http://www2.acm.jhu.edu/projects/calliope) - What kind of structure does Pipermail store the archive in? Maybe I should contact someone (maybe whoever runs the lists at python.org?) who runs a larger Pipermail archive and see about getting an archive of a treefull of archived articles - Did you have any idea about what kind of search interface? Were you thinking a text field with Boolean capability, or just letting them throw some words in the field and see what sticks? - Were you thinking of an engine that has an indexing process that runs via a cron job or something, or something much simpler? (like one that just brute force searches through the text of the archives for each query; that would be slow as hell if the archive was large, but would take no additional disk space and wouldn't really require persistence). ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "Economists are still trying to figure out why the girls with the least principle draw the most interest." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From viega@list.org Sat Jun 13 23:38:13 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 15:38:13 -0700 Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: <19980613172225.39336@chronis.icgroup.com>; from Scott on Sat, Jun 13, 1998 at 05:22:25PM -0400 References: <19980613172225.39336@chronis.icgroup.com> Message-ID: <19980613153813.B15795@list.org> On Sat, Jun 13, 1998 at 05:22:25PM -0400, Scott wrote: > I saw this error once in another context (still with mailman and > cookies, but with the "private" cgi). Hmm, I've heard of this error from two different people now, and one of them mentioned it was transient. Sometimes it happened... I don't think it invvolved IE on a mac. > cookie, or there is something about the way that the base64 module > encodes and decodes that won't work with http. Since I've only seen > the error on what i consider a questionable browser i suspect that > it's the browser's fault. Why are you using base64 encoding, anyway? If it's a "privacy" matter, anyone who reads your cookie can forge the same cookie, no? If so, might as well just use plaintext if it's going to be an eisenbug this way. If anyone can reproduce this bug reliably, please let me know, I'd like to work with you to figure out what exactly the problem is. John From viega@list.org Sat Jun 13 23:50:53 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 15:50:53 -0700 Subject: [Mailman-Developers] Idea In-Reply-To: ; from Corbett J. Klempay on Sat, Jun 13, 1998 at 05:55:41PM -0400 References: Message-ID: <19980613155053.C15795@list.org> On Sat, Jun 13, 1998 at 05:55:41PM -0400, Corbett J. Klempay wrote: > > - how big of an archive should this be scalable to? I'd say as big as possible. I don't know if I can give a better answer to that one. Target the most heavily trafficed mailing list you've been on. > I'm thinking this because some models (like the vector model used for my > project) get good accuracy, but suck as far as resource usage (like our > engine dealt with ~2000 text documents from an online database and would > suck up ~80 MB of RAM per query, and would take almost 20 seconds just to > load the pre-indexed document clusters from disk; the query itself only > takes like 1-2 seconds on a K6-233, but the startup time of 15+ seconds > blows). With large corpa, it might be necessary to implement some > persistence; it just takes so annoyingly long to load even pre-indexed > stuff from disk (and our queries were on a K6-233 with 128 MB; > heheh...think how a P-90 with 32 MB would fare :) (or worse yet!) Well, you have several options. You could keep a persistant server up, but I wouldn't make it a requirement, perhaps an option. I think that if complex search capabilities aren't desired, the grep libraries would be an OK first pass. > - What kind of structure does Pipermail store the archive in? This is the biggest problem right now. Andrew says you can plug in any sort of back-end you want for a database, as long as it can handle a tree-type structure. Unfortunately, the only such backend implemented is not portable. Everything needs to work out of the box w/ a vanilla Python installation. I was thinking that someone could write a backend that uses the file system for that tree structure... > - Did you have any idea about what kind of search interface? Were you > thinking a text field with Boolean capability, or just letting them throw > some words in the field and see what sticks? Well, for a first pass, something simple will do, but the nicer you can make it, the better off we'll be... > - Were you thinking of an engine that has an indexing process that runs > via a cron job or something, or something much simpler? (like one that > just brute force searches through the text of the archives for each query; > that would be slow as hell if the archive was large, but would take no > additional disk space and wouldn't really require persistence). It'd be nice to have a per-list setting for this one. For most lists, something simple would do, but I run a few where indexing would certainly be better... John From scott@chronis.pobox.com Sun Jun 14 01:04:35 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 13 Jun 1998 20:04:35 -0400 Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: <19980613153813.B15795@list.org>; from John Viega on Sat, Jun 13, 1998 at 03:38:13PM -0700 References: <19980613172225.39336@chronis.icgroup.com> <19980613153813.B15795@list.org> Message-ID: <19980613200435.35995@chronis.icgroup.com> On Sat, Jun 13, 1998 at 03:38:13PM -0700, John Viega wrote: | On Sat, Jun 13, 1998 at 05:22:25PM -0400, Scott wrote: | > I saw this error once in another context (still with mailman and | > cookies, but with the "private" cgi). | | Hmm, I've heard of this error from two different people now, and one | of them mentioned it was transient. Sometimes it happened... I don't | think it invvolved IE on a mac. | | > cookie, or there is something about the way that the base64 module | > encodes and decodes that won't work with http. Since I've only seen | > the error on what i consider a questionable browser i suspect that | > it's the browser's fault. | | Why are you using base64 encoding, anyway? If it's a "privacy" | matter, anyone who reads your cookie can forge the same cookie, no? | If so, might as well just use plaintext if it's going to be an | eisenbug this way. i was following the example in the "private" script. i don't think base64 is necessary, but would think that some kind of encoding makes it safer -- if only because it looks random. md5 is another option for that, but it produces lots of characters that need special http escaping. i'm not sure whether or not base64 has the same problem. perhaps plaintext or a simple hash is in order? scott From viega@list.org Sun Jun 14 01:54:12 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 17:54:12 -0700 Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: <19980613200435.35995@chronis.icgroup.com>; from Scott on Sat, Jun 13, 1998 at 08:04:35PM -0400 References: <19980613172225.39336@chronis.icgroup.com> <19980613153813.B15795@list.org> <19980613200435.35995@chronis.icgroup.com> Message-ID: <19980613175412.A17363@list.org> On Sat, Jun 13, 1998 at 08:04:35PM -0400, Scott wrote: > > i was following the example in the "private" script. i don't think > base64 is necessary, but would think that some kind of encoding makes > it safer -- if only because it looks random. md5 is another option > for that, but it produces lots of characters that need special http > escaping. i'm not sure whether or not base64 has the same problem. > perhaps plaintext or a simple hash is in order? base64's purpose is essentially the same as uuencode; to take binary streams and encode them into 7 bit characters. I'd say as long as there as these weird bugs, let's back off and go to plaintext... John From dragondm@nexus.Integral.org Sun Jun 14 05:41:16 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Sat, 13 Jun 1998 23:41:16 -0500 (CDT) Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: <19980613175412.A17363@list.org> Message-ID: On Sat, 13 Jun 1998, John Viega wrote: > On Sat, Jun 13, 1998 at 08:04:35PM -0400, Scott wrote: > > > > i was following the example in the "private" script. i don't think > > base64 is necessary, but would think that some kind of encoding makes > > it safer -- if only because it looks random. md5 is another option > > for that, but it produces lots of characters that need special http > > escaping. i'm not sure whether or not base64 has the same problem. > > perhaps plaintext or a simple hash is in order? > > base64's purpose is essentially the same as uuencode; to take binary > streams and encode them into 7 bit characters. > > I'd say as long as there as these weird bugs, let's back off and go to > plaintext... I'm afraid I know little of cookies, but if it's a problem of http escaping, I DO know that base64 will produce charecters that will need escaping. There is a module for the apache webserver that generates unique ID's for hits, and it encodes the binary ID using a _modified_ version of base64-encoding for this reason. To quote: "The UNIQUE_ID environment variable is constructed by encoding the 112-bit (value) ... using the alphabet [A-Za-z0-9@-] in a manner similar to MIME base64 encoding, producing 19 characters. The MIME base64 alphabet is actually [A-Za-z0-9+/] however + and / need to be specially encoded in URLs, which makes them less desirable. " ....so, if cookies follow the same rules as URL's for escaping, then indeed that could be the trouble. > -The Dragon De Monsyne From viega@list.org Sun Jun 14 06:36:49 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 22:36:49 -0700 Subject: [Mailman-Developers] flock again Message-ID: <19980613223649.A21606@list.org> --41hnqqNCTbUpUBcO Content-Type: text/plain; charset=us-ascii The flock.py I posted had a subtle name collision problem that reminds me how much I like static type checking. Attached is a fix. I've now hooked it up to the main source tree, and have integrated Mike McLay's wrapping stuff. John --41hnqqNCTbUpUBcO Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="flock.py" # Copyright (C) 1998 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. # # flock.py: Portable file locking. John Viega, Jun 13, 1998 """Portable (?) file locking with timeouts. This code should work with all versions of NFS. The algorithm was suggested by the GNU/Linux open() man page. Make sure no malicious people have access to link() to the lock file. """ # Potential change: let the locker insert a field saying when he promises # to be done with the lock, so if he needs more time than the other # processes think he needs, he can say so. import socket, os, time DEFAULT_HUNG_TIMEOUT = 15 DEFAULT_SLEEP_INTERVAL = .25 AlreadyCalledLockError = "AlreadyCalledLockError" NotLockedError = "NotLockedError" TimeOutError = "TimeOutError" class FileLock: def __init__(self, lockfile, hung_timeout = DEFAULT_HUNG_TIMEOUT, sleep_interval = DEFAULT_SLEEP_INTERVAL): self.lockfile = lockfile self.hung_timeout = hung_timeout self.sleep_interval = sleep_interval self.tmpfname = "%s.%s.%d" % (lockfile, socket.gethostname(), os.getpid()) self.is_locked = 0 if not os.path.exists(self.lockfile): try: file = open(self.lockfile, "w+") except IOError: pass # Note that no one new can grab the lock once we've opened our # tmpfile until we close it, even if we don't have the lock. So # checking the PID and stealing the lock are garunteed to be atomic. def lock(self, timeout = 0): """Blocks until the lock can be obtained. Raises a TimeOutError exception if a positive timeout value is given and that time elapses before the lock is obtained. """ if timeout > 0: timeout_time = time.time() + timeout last_pid = -1 if self.locked(): raise AlreadyCalledLockError while 1: os.link(self.lockfile, self.tmpfname) if os.stat(self.tmpfname)[3] == 2: file = open(self.tmpfname, 'w+') file.write(`os.getpid(),self.tmpfname`) file.close() self.is_locked = 1 break if timeout and timeout_time < time.time(): raise TimeOutError file = open(self.tmpfname, 'r') try: pid,winner = eval(file.read()) except SyntaxError: # no info in file... *can* happen file.close() continue file.close() if pid <> last_pid: last_pid = pid stime = time.time() if (stime + self.hung_timeout < time.time()) and self.hung_timeout > 0: file = open(self.tmpfname, 'w+') file.write(`os.getpid(),self.tmpfname`) os.unlink(winner) self.is_locked = 1 break os.unlink(self.tmpfname) time.sleep(self.sleep_interval) # This could error if the lock is stolen. You must catch it. def unlock(self): if not self.locked(): raise NotLockedError self.is_locked = 0 os.unlink(self.tmpfname) def locked(self): return os.path.exists(self.tmpfname) and self.is_locked --41hnqqNCTbUpUBcO-- From viega@list.org Sun Jun 14 06:43:50 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 22:43:50 -0700 Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: ; from The Dragon De Monsyne on Sat, Jun 13, 1998 at 11:41:16PM -0500 References: <19980613175412.A17363@list.org> Message-ID: <19980613224350.B21606@list.org> On Sat, Jun 13, 1998 at 11:41:16PM -0500, The Dragon De Monsyne wrote: > > I'm afraid I know little of cookies, but if it's a problem of > http escaping, I DO know that base64 will produce charecters that will > need escaping. There is a module for the apache webserver that generates > unique ID's for hits, and it encodes the binary ID using a _modified_ > version of base64-encoding for this reason. Well, cookies are handled in mime headers. So they don't have the same escaping problems as html. However, I think you did hit the problem. It occurs to me that ';' has special meaning on the mime line. So we'd still have this problem w/ plaintext when people have a ';' in their password. My guess is that things are blowing up when the base64 outputs a semicolon. IIRC, the only special chars to the right of the colon should be the semi-colon, and the end of line. But, we should find out for sure, and guard against it... John From cklempay@chimera.acm.jhu.edu Sun Jun 14 07:33:14 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Sun, 14 Jun 1998 02:33:14 -0400 (EDT) Subject: [Mailman-Developers] Idea In-Reply-To: <19980613155053.C15795@list.org> Message-ID: On Sat, 13 Jun 1998, John Viega wrote: > On Sat, Jun 13, 1998 at 05:55:41PM -0400, Corbett J. Klempay wrote: > > > > - how big of an archive should this be scalable to? > > I'd say as big as possible. I don't know if I can give a better > answer to that one. Target the most heavily trafficed mailing list > you've been on. > Hmmm...any good ballpark figures for this? I'm thinking that BUGTRAQ is maybe one of the heavier I'm on, or perhaps the AfterStep or CVS lists. So let's say ~10 messages a day, so ~70/week, ~300 month, 3000-4000 year. That's a lot of messages :) I wonder how large of a time span they should be able to search... > > - What kind of structure does Pipermail store the archive in? > > This is the biggest problem right now. Andrew says you can plug in > any sort of back-end you want for a database, as long as it can handle > a tree-type structure. Unfortunately, the only such backend > implemented is not portable. Everything needs to work out of the box > w/ a vanilla Python installation. I was thinking that someone could > write a backend that uses the file system for that tree structure... > Ugh. (the non-portable current state of things) What is it using now? CK From viega@list.org Sun Jun 14 07:46:48 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 23:46:48 -0700 Subject: [Mailman-Developers] Idea In-Reply-To: ; from Corbett J. Klempay on Sun, Jun 14, 1998 at 02:33:14AM -0400 References: <19980613155053.C15795@list.org> Message-ID: <19980613234648.A22154@list.org> On Sun, Jun 14, 1998 at 02:33:14AM -0400, Corbett J. Klempay wrote: > > Hmmm...any good ballpark figures for this? I'm thinking that BUGTRAQ is > maybe one of the heavier I'm on, or perhaps the AfterStep or CVS lists. > So let's say ~10 messages a day, so ~70/week, ~300 month, 3000-4000 year. > That's a lot of messages :) I wonder how large of a time span they should > be able to search... I've seen some lists that averaged twice that, probably. Figure on there being some bigger lists, but then again, no one wants to get TOO many messages a day. > > > - What kind of structure does Pipermail store the archive in? > > > > This is the biggest problem right now. Andrew says you can plug in > > any sort of back-end you want for a database, as long as it can handle > > a tree-type structure. Unfortunately, the only such backend > > implemented is not portable. Everything needs to work out of the box > > w/ a vanilla Python installation. I was thinking that someone could > > write a backend that uses the file system for that tree structure... > > > > Ugh. (the non-portable current state of things) What is it using now? I think bsddb or something like that. My biggest concern is having Mailman work "out of the box" for everybody. People needing to to install bsddb isn't acceptable... John From janne@avocado.pc.helsinki.fi Mon Jun 15 09:50:25 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 15 Jun 1998 11:50:25 +0300 Subject: [Mailman-Developers] Re: bounce agressiveness In-Reply-To: John Viega's message of "Fri, 12 Jun 1998 19:12:12 -0700" References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> <19980612191212.D9003@list.org> Message-ID: John Viega writes: > I do like the idea of sending daily mail for actually an extended > period of time. Maybe it could be every 3rd day for 60 days, since > mail generally doesn't get rejected as undeliverable for 3 days. We > could use the confirmation mechanism to let people get back on the > list. Or by an exponential schedule, say after 2, 4, 8, 16, 32, and 64 days. Why? Well, less trying, and because recovery of an address becomes more and more improbable when time passes (most network problems are short in duration). -- Janne From akuchlin@cnri.reston.va.us Mon Jun 15 16:00:31 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Mon, 15 Jun 1998 11:00:31 -0400 (EDT) Subject: [Mailman-Developers] Idea In-Reply-To: References: Message-ID: <13701.13769.454247.591783@newcnri.cnri.reston.va.us> Corbett J. Klempay writes: >- What kind of structure does Pipermail store the archive in? Maybe I >should contact someone (maybe whoever runs the lists at python.org?) who >runs a larger Pipermail archive and see about getting an archive of a >treefull of archived articles Messages are just HTML files, but there are on-disk B-trees (using the bsddb module) used for the indexes. It should not be very difficult to use Aaron's Bplustree.py module instead; the only problem is me finding time to do it. Hypermail development has picked up again, so I've been speculating about writing a Python interface to Hypermail's indexing infrastructure. That would let Python code control policies (where a message goes, how it's formatted, ...) but leave the data handling to C code. No time to look into that, unfortunately. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ In its broadest ecological context, economic development is the development of more intensive ways of exploiting the natural environment. -- Richard Wilkinson From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Mon Jun 15 16:23:03 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 15 Jun 1998 11:23:03 -0400 (EDT) Subject: [Mailman-Developers] Re: bounce agressiveness References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> <19980612191212.D9003@list.org> Message-ID: <13701.15408.361965.318557@anthem.CNRI.Reston.Va.US> >>>>> "JS" == Janne Sinkkonen writes: JS> Or by an exponential schedule, say after 2, 4, 8, 16, 32, and JS> 64 days. Why? Well, less trying, and because recovery of an JS> address becomes more and more improbable when time passes JS> (most network problems are short in duration). I wouldn't extend it so far though. I would guess 8 or 10 days max would be more than enough. -Barry From janne@avocado.pc.helsinki.fi Mon Jun 15 17:12:32 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 15 Jun 1998 19:12:32 +0300 Subject: [Mailman-Developers] Bounce processing and remote list administrators Message-ID: Well, I'm not sure it's quite as smooth as it could be. Our network connection tends to go down for the weekend because of a broken router needing (ir)relegular reboots. This happened on Sunday once again, and this time the remote listowner got over 100MB worth of error messages. His ISP will filter the trash out of his mailbox for tomorrow, hopefully. Furthermore, about 150 of roughly 800 members were kicked out of the list because of 'excessive or fatal bounces'. The network was down for about 24 hours. (Our bounce parameters may be too tight, of course.) I haven't seen the variety of the generated messages yet. Even if this were 'normal' in the sense that there are no message-generating loops, some kind of solution is needed to prevent excessive or fatal mail bombing of the listowners. -- Janne From viega@list.org Tue Jun 16 13:44:54 1998 From: viega@list.org (John Viega) Date: Tue, 16 Jun 1998 05:44:54 -0700 Subject: [Mailman-Developers] Bounce processing and remote list administrators In-Reply-To: ; from Janne Sinkkonen on Mon, Jun 15, 1998 at 07:12:32PM +0300 References: Message-ID: <19980616054454.C8485@list.org> On Mon, Jun 15, 1998 at 07:12:32PM +0300, Janne Sinkkonen wrote: > > Well, I'm not sure it's quite as smooth as it could be. > > Our network connection tends to go down for the weekend because of a > broken router needing (ir)relegular reboots. This happened on Sunday > once again, and this time the remote listowner got over 100MB worth of > error messages. His ISP will filter the trash out of his mailbox for > tomorrow, hopefully. > > Furthermore, about 150 of roughly 800 members were kicked out of the > list because of 'excessive or fatal bounces'. The network was down > for about 24 hours. (Our bounce parameters may be too tight, of > course.) > > I haven't seen the variety of the generated messages yet. > > Even if this were 'normal' in the sense that there are no > message-generating loops, some kind of solution is needed to prevent > excessive or fatal mail bombing of the listowners. > Wow. Is the field, "Minimum number of days an address has been non-fatally bad before we take action" set to 1 or 0 for that list? If not, either something got generated that was treated as a fatal bounce, or there's a bug in the bounce handling. Neither would surprise me too much. Like Barry said, though, we should switch to treating all bounces as non-fatal; it will simplify bounce processing a bit. John From cklempay@chimera.acm.jhu.edu Wed Jun 17 06:10:27 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Wed, 17 Jun 1998 01:10:27 -0400 (EDT) Subject: [Mailman-Developers] Mailman + Qmail 1.01 Message-ID: Anyone in here done this / know if it works? I even tried installing the fastforward package for Qmail to give it /etc/aliases support...but it's being really damn annoying (i.e. in its current state, mail sent to an alias from /etc/aliases is getting swallowed into nowhere; it never shows up on the Qmail machine (at least not in the Mailbox), and doesn't bounce either...) I forget who it was who was running it, but on a related note...anyone know how VMailer is coming along? I was reviewing the pages for it...looks promising. (I want to get our ACM off of sendmail, but Qmail seems like I'd be dealing with a bunch of transitionary problems...) ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "Economists are still trying to figure out why the girls with the least principle draw the most interest." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From cklempay@chimera.acm.jhu.edu Wed Jun 17 06:26:06 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Wed, 17 Jun 1998 01:26:06 -0400 (EDT) Subject: [Mailman-Developers] Oops.. Message-ID: I guess you guys should probably disregard that last Qmail message I sent...I had forgotten that I had sent out a similar message probably about a week and a half - 2 weeks ago... If anyone HAS gotten the aliases cooking with Qmail, please let me know. ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "Economists are still trying to figure out why the girls with the least principle draw the most interest." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From klm@python.org Fri Jun 19 16:56:48 1998 From: klm@python.org (Ken Manheimer) Date: Fri, 19 Jun 1998 11:56:48 -0400 (EDT) Subject: [Mailman-Developers] MM 1.0b4 port 25 connection conflicts w/anti-spam non relay measures Message-ID: <199806191556.LAA00436@glyph.cnri.reston.va.us> In trying out 1.0b4 on python.org i discovered a logistical conflict with the sendmail 8 anti-spam non-relaying mechanism - it *may* be due to a fault in my implementation of it, but i thought it's worth giving people a heads up. For those of you unfamiliar with it, it's a measure that tells sendmail to restrict the set of hosts from and to which it will accept mail relays. See http://www.sendmail.org/antispam.html for more info. It's something that public sites that handle mail should in good consience do - it reduces spammer's options for concealing their message origins. My problem was that mail delivered via mailman on my site to sites not explicitly specified as acceptable recipients (in the /etc/RelayTo file, i think) were being rejected. I would have thought that the mail appeared to originate at python.org, which is in the RelayTo and LocalIP files, and so would be accepted, but no go - and i don't see a way to wildcard the RelayTo to enable any recipients. I would not be surprised if i'm missing a fairly easy way to configure the setup to accept these messages, but i'm not sure, and don't have the time right now to investigate (i was supposed to go out of town, for the weekend, earlier today.) I disabled the anti-relay mechanism for the moment, but wanted to ask if anyone out there has experience with this, and suggestions for making the direct port 25 connection work without disabling the anti-relay measures. Ken From gorgo@caesar.elte.hu Sat Jun 20 14:19:49 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Sat, 20 Jun 1998 15:19:49 +0200 (METDST) Subject: [Mailman-Developers] probems with mailman Message-ID: Hello! I've just found mailman and i like it wery much, it seems much nicer then any other mailing list server i've seen. Here are some problems I've found: I tried to convert a list with about 300 normal and 220 digested subscribers to mailman with convert_list. It always failed with something like fork: resource currently not available ... it tries to send too many mails at once. The second problem is: the mail wrapper accepts exactly one gid to be run as. This makes it difficult to package up mailman in a binary package, for example for debian linux. I'm a debian developer myself btw. The problem is that we have several MTA-s available in the distribution (sendmail, smail, exim, zmailer, and as a non-free addition, even qmail), and each can use different gids, and even one can be configured to use a gid other then the default. So it would be very nice if it could be configured to accept a gid from a set ... Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From Gergely Madarasz Sat Jun 20 14:44:51 1998 From: Gergely Madarasz (Gergely Madarasz) Date: Sat, 20 Jun 1998 15:44:51 +0200 (METDST) Subject: [Mailman-Developers] one more problem Message-ID: When I'm on the membership management page, choose another chunk to view, and my admin cookie expires just then, the cgi asks me the password, and then returns me to the membership management page with chunk=something in the url, but the first chunk in the html. Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From scott@chronis.pobox.com Sat Jun 20 19:12:05 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 20 Jun 1998 14:12:05 -0400 Subject: [Mailman-Developers] one more problem In-Reply-To: ; from Gergely Madarasz on Sat, Jun 20, 1998 at 03:44:51PM +0200 References: Message-ID: <19980620141205.09053@chronis.icgroup.com> mmm. what browser /os and python version are you running? scott On Sat, Jun 20, 1998 at 03:44:51PM +0200, Gergely Madarasz wrote: | When I'm on the membership management page, choose another chunk to view, | and my admin cookie expires just then, the cgi asks me the password, and | then returns me to the membership management page with chunk=something in | the url, but the first chunk in the html. | | Greg | | -- | Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org | It's practically impossible to look at a penguin and feel angry. | Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. | HuLUG: http://www.cab.u-szeged.hu/local/linux/ | | | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers | From scott@chronis.pobox.com Sat Jun 20 19:14:37 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 20 Jun 1998 14:14:37 -0400 Subject: [Mailman-Developers] probems with mailman In-Reply-To: ; from Gergely Madarasz on Sat, Jun 20, 1998 at 03:19:49PM +0200 References: Message-ID: <19980620141437.38060@chronis.icgroup.com> I agree that the mail gid's in particular should be changed to accept a set of calling gid's. i patched an older version for this change some time ago. i'm sure that that patch won't apply to the current version, so i'll try to cook up another one over this weekend. scott On Sat, Jun 20, 1998 at 03:19:49PM +0200, Gergely Madarasz wrote: | Hello! | | I've just found mailman and i like it wery much, it seems much nicer then | any other mailing list server i've seen. Here are some problems I've | found: | I tried to convert a list with about 300 normal and 220 digested | subscribers to mailman with convert_list. It always failed with something | like fork: resource currently not available ... it tries to send too many | mails at once. | The second problem is: the mail wrapper accepts exactly one gid to be run | as. This makes it difficult to package up mailman in a binary package, for | example for debian linux. I'm a debian developer myself btw. The problem | is that we have several MTA-s available in the distribution (sendmail, | smail, exim, zmailer, and as a non-free addition, even qmail), and each | can use different gids, and even one can be configured to use a gid other | then the default. So it would be very nice if it could be configured to | accept a gid from a set ... | | Greg | | -- | Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org | It's practically impossible to look at a penguin and feel angry. | Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. | HuLUG: http://www.cab.u-szeged.hu/local/linux/ | | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers | From gorgo@caesar.elte.hu Sun Jun 21 01:30:14 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Sun, 21 Jun 1998 02:30:14 +0200 (METDST) Subject: [Mailman-Developers] one more problem In-Reply-To: <19980620141205.09053@chronis.icgroup.com> Message-ID: On Sat, 20 Jun 1998, Scott wrote: > > mmm. what browser /os and python version are you running? I use Netscape 4 on linux, the mail server is also linux, with python version 1.5.1 (debianrelease version 2.1) Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From cklempay@chimera.acm.jhu.edu Sun Jun 21 22:36:30 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Sun, 21 Jun 1998 17:36:30 -0400 (EDT) Subject: [Mailman-Developers] bsddb module Message-ID: Anyone know where to get the bsddb module that Pipermail wants? I'm screwing with it, and it is giving me an error because it claims I don't have the bsddb module...but Robin Dunn's page refers to 'the bsddb module included with the Python distribution'. (saying that it corresponds to db 1.85, while the newest are 2.x). Anyone know what's the story here? My Python installation is just from the Andrich RPMs. Corbett From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Mon Jun 22 22:48:24 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 22 Jun 1998 17:48:24 -0400 (EDT) Subject: [Mailman-Developers] probems with mailman References: <19980620141437.38060@chronis.icgroup.com> Message-ID: <13710.53424.702333.432419@anthem.cnri.reston.va.us> >>>>> "S" == Scott writes: S> I agree that the mail gid's in particular should be changed to S> accept a set of calling gid's. I wonder if it would be better to get the GID out of an environment variable of a file? Perhaps that just defeats the purpose of having a hard coded GID. Using a list of GIDs probably won't be difficult, although the configure interface will change, because it currently uses --with-*-gid="list of gids" as a list of GIDs to check, one at a time, until it finds an existing one. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Mon Jun 22 22:49:45 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 22 Jun 1998 17:49:45 -0400 (EDT) Subject: [Mailman-Developers] probems with mailman References: Message-ID: <13710.53595.29065.897464@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> I tried to convert a list with about 300 normal and 220 GM> digested subscribers to mailman with convert_list. It always GM> failed with something like fork: resource currently not GM> available ... it tries to send too many mails at once. Hmm, I haven't played with convert_list yet, but I hope to soon. -Barry From johnie@netgod.net Mon Jun 22 22:16:35 1998 From: johnie@netgod.net (Johnie Ingram) Date: Mon, 22 Jun 1998 18:16:35 -0300 Subject: [Mailman-Developers] probems with mailman In-Reply-To: Your message of "Mon, 22 Jun 1998 17:48:24 EDT." <13710.53424.702333.432419@anthem.cnri.reston.va.us> Message-ID: <199806222216.SAA14147@netgod.net> "BAW" == Barry A Warsaw writes: BAW> I wonder if it would be better to get the GID out of an BAW> environment variable of a file? Perhaps that just defeats the BAW> purpose of having a hard coded GID. It may be simpler, and sufficient for Debian's purposes, to simply accept any gid less than 100. --------------------- PGP E4 70 6E 59 80 6A F5 78 63 32 BC FB 7A 08 53 4C __ _ Debian GNU Johnie Ingram mm mm / /(_)_ __ _ ___ __ "netgod" irc.debian.org mm mm / / | | '_ \| | | \ \/ / m m m / /__| | | | | |_| |> < Yes, I'm Linus, and I am your God. mm mm \____/_|_| |_|\__,_/_/\_\ -- Linus, keynote address, Expo 98 GO BLUE From gorgo@caesar.elte.hu Mon Jun 22 23:35:29 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Tue, 23 Jun 1998 00:35:29 +0200 (METDST) Subject: [Mailman-Developers] probems with mailman In-Reply-To: <13710.53424.702333.432419@anthem.cnri.reston.va.us> Message-ID: On Mon, 22 Jun 1998, Barry A. Warsaw wrote: > > >>>>> "S" == Scott writes: > > S> I agree that the mail gid's in particular should be changed to > S> accept a set of calling gid's. > > I wonder if it would be better to get the GID out of an environment > variable of a file? Perhaps that just defeats the purpose of having a > hard coded GID. I think it would be the best solution..... a config file which defines MAIL_GID, CGI_GID, etc... -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From gorgo@caesar.elte.hu Tue Jun 23 12:47:59 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Tue, 23 Jun 1998 13:47:59 +0200 (METDST) Subject: [Mailman-Developers] probems with mailman In-Reply-To: <199806222216.SAA14147@netgod.net> Message-ID: On Mon, 22 Jun 1998, Johnie Ingram wrote: > > "BAW" == Barry A Warsaw writes: > > BAW> I wonder if it would be better to get the GID out of an > BAW> environment variable of a file? Perhaps that just defeats the > BAW> purpose of having a hard coded GID. > > It may be simpler, and sufficient for Debian's purposes, to simply > accept any gid less than 100. Qmail uses nogroup (65534). But then again qmail could be set up to run mailman without the wrapper. Talking about qmail... anyone uses it with mailman ? My mails did not go out... Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From klm@python.org Tue Jun 23 17:07:52 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 23 Jun 1998 12:07:52 -0400 (EDT) Subject: [Mailman-Developers] Lost maillist messages thursday eve (6/18) to friday noon (6/19) Message-ID: <199806231607.MAA13065@glyph.cnri.reston.va.us> Some of you may have lost postings posted to one of the following maillists between last thursday (6/18) evening and friday (6/19) noon. Mailman-Developers (1 msg) Matrix-SIG (8 msgs) DB-SIG (3 msgs) Doc-SIG (4 msgs) Pythonmac-SIG (3 msgs) XML-SIG (1 msg) Trove-Dev (6 msgs) This happened accompanying an upgrade of our maillist software, mailman, due to an bad interaction between a new mailman feature and an anti-spam (anti-relay) mechanism applied to python.org's sendmail configuration. This problem did not show during testing because our test recipients were all local, and not subject to the anti-relay constraints. If you sent something to any of these lists during that time frame and never saw it show, you may want to resend. Archiving was not affected, so you should be able to find the messages in the maillist archives. People receiving the lists in digest format were not affected, since the delivery problem was fixed before the digest delivery time. My apologies for the disruption! Ken Manheimer klm@python.org 703 620-8990 x268 (orporation for National Research |nitiatives # If you appreciate Python, consider joining the PSA! # # . # From gorgo@caesar.elte.hu Tue Jun 23 17:34:03 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Tue, 23 Jun 1998 18:34:03 +0200 (METDST) Subject: [Mailman-Developers] mailman and sendmail Message-ID: Hello! You might already know about this, but anyway... Sendmail 8.8.8 and 8.9.0 segfaults on queue run of mime digests made by mailman, if the remote host doesnt support 8bitmime. As I understood, it is a bug in the header detection while converting 8bit to 7bit. The result is that those who are at the beginning of the recepient list of the digest get the digest at every queue run, those who are later (after and including the first address that gets handled by a non-esmtp host) don't get it at all. I got a patch for sendmail, it seems to be working, it is also included in debian's sendmail.deb package, version 8.8.8-20. I expect that it will be included in the upstream source soon, but if anyone experiences such problems (i expect most lists run in are not 8bitmime ;)), i can send him the patch. Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From cklempay@chimera.acm.jhu.edu Wed Jun 24 02:01:30 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Tue, 23 Jun 1998 21:01:30 -0400 (EDT) Subject: [Mailman-Developers] fun Pipermail stuff Message-ID: I've resumed my attempts to get Pipermail working on my system with some stock data (Ken Manheimer hooked me up with some matrix-sig archives)...I was having some problems earlier with Pythong not being able to find the bsddb module (I asked Andrew about it)...turns out that Python distro I had was whack (it was 1.5.1-4, for reference; upgrading to -18 seems to have fixed it). So, after upgrading, I tried running...seemed to be doing ok...it was 500+ articles in and hit this: Processing article #508 into archives ['1996q1.matrix-sig'] Traceback (innermost last): File "pysig.py", line 310, in ? publicF.processUnixMailbox(file, PySIGArticle) File "/home/cklempay/src/pipermail/pipermail.py", line 366, in processUnixMailbox a=articleClass(m, self.sequence) # Create an article object File "pysig.py", line 71, in __init__ hypermail.Article.__init__(self, message, sequence, keepHeaders) File "/home/cklempay/src/pipermail/hypermail.py", line 39, in __init__ pipermail.Article.__init__(self, message, sequence, keepHeaders) File "/home/cklempay/src/pipermail/pipermail.py", line 95, in __init__ date=time.mktime(date)-tzoffset TypeError: number coercion failed [galtgulch]~/src/pysig > ls Anyone know what the story is with this? I'm too inexperienced/clueless for this to be obvious (if it is). ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "College: the fountains of knowledge, where everyone goes to drink." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 24 22:02:51 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 24 Jun 1998 17:02:51 -0400 (EDT) Subject: [Mailman-Developers] forwarded message from Dave Wolfe Message-ID: <13713.27003.343895.758727@anthem.cnri.reston.va.us> --ulXCkCkwhR Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Description: message body text Is anybody familiar with Wilma? How does it compare with Mailman? -Barry --ulXCkCkwhR Content-Description: forwarded message Content-Type: message/rfc822 Content-Description: forwarded message Content-Transfer-Encoding: 7bit Received: from cnri.reston.va.us (ns.cnri.reston.va.us [132.151.1.1]) by newcnri.cnri.reston.va.us (8.8.5/8.8.7a) with SMTP id PAA07686 for ; Wed, 24 Jun 1998 15:18:35 -0400 (EDT) Received: from relay2.UU.NET (relay2.UU.NET [192.48.96.7]) by cnri.reston.va.us (8.8.5/8.8.7a) with ESMTP id PAA16473 for ; Wed, 24 Jun 1998 15:20:54 -0400 (EDT) Received: from honor.greatcircle.com by relay2.UU.NET with ESMTP (peer crosschecked as: honor.greatcircle.com [198.102.244.44]) id QQever05274; Wed, 24 Jun 1998 15:17:31 -0400 (EDT) Received: (majordom@localhost) by honor.greatcircle.com (8.8.5/Honor-Lists-970926-1) id LAA25312; Wed, 24 Jun 1998 11:16:08 -0700 (PDT) Received: (mcb@localhost) by honor.greatcircle.com (8.8.5/Honor-980202-1) id LAA25302 for majordomo-announce@greatcircle.com; Wed, 24 Jun 1998 11:16:05 -0700 (PDT) Received: from spsem02.sps.mot.com (spsem02.sps.mot.com [192.70.231.5]) by honor.greatcircle.com (8.8.5/Honor-980202-1) with SMTP id GAA17609; Tue, 23 Jun 1998 06:28:48 -0700 (PDT) Received: from mogate.sps.mot.com by spsem02.sps.mot.com (4.1/SMI-4.1/Email 2.1 10/25/93) id AA14505 for majordomo-announce@greatcircle.com; Tue, 23 Jun 98 06:31:16 MST Received: from risc.risc.sps.mot.com by mogate.sps.mot.com (4.1/SMI-4.1/Email-2.0) id AA12549 for nb@thinkcoach.com; Tue, 23 Jun 98 06:31:01 MST Received: from miaow.risc.sps.mot.com (miaow.risc.sps.mot.com [223.72.249.15]) by risc.risc.sps.mot.com (8.8.7/8.8.7) with ESMTP id IAA18885; Tue, 23 Jun 1998 08:30:58 -0500 (CDT) Received: (from dwolfe@localhost) by miaow.risc.sps.mot.com (8.7.1/8.7.3) id IAA22726; Tue, 23 Jun 1998 08:32:49 -0500 Message-Id: <19980623083248.A28562@risc.sps.mot.com> Mime-Version: 1.0 Precedence: bulk Content-Type: text/plain; charset=us-ascii From: Dave Wolfe Sender: majordomo-announce-owner@GreatCircle.COM To: "Majordomo developer's mailing list" , "Majordomo user's mailing list" , majordomo-announce@GreatCircle.COM Subject: Wilma 1.4 released Date: Tue, 23 Jun 1998 08:32:48 -0500 X-Mailer: Mutt 0.91.1i I finally made time to package Wilma 1.4 and get it released. It's available at ftp://sol.ccsf.cc.ca.us/majordomo-contrib/wilma-1.4.tgz Whenever Jason gets back, it should also be available on ftp.hpc.uh.edu. This is primarily a bug fix release. What is Wilma? It's a Web Interface to List Mail Archives, a set of Perl programs that combines (Majordomo) mail list archives, the MHonArc mail folder HTML reformatter, and the Glimpse search engine into a CGI interface for browsing and searching those archives. Wilma requires Perl 5, MHonArc, and Glimpse. An example of a predecessor of Wilma is available for the Majordomo mailing lists at: http://www.hpc.uh.edu/majordomo-users/ http://www.hpc.uh.edu/majordomo-workers/ Many thanks to Joe R. Jah and Jason L. Tibbitts III for allowing me to publish Wilma on their ftp sites. -- Dave Wolfe --ulXCkCkwhR-- From cklempay@chimera.acm.jhu.edu Wed Jun 24 22:13:22 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Wed, 24 Jun 1998 17:13:22 -0400 (EDT) Subject: [Mailman-Developers] forwarded message from Dave Wolfe In-Reply-To: <13713.27003.343895.758727@anthem.cnri.reston.va.us> Message-ID: That's one thing worth considering...not to rip off the idea too much, but would we rather try plugging into something like Glipse to provide search capabilities instead of rolling our own? Is the core of Glimpse C? If it is, I don't know how we'd be able to compete speed-wise with a Python-only approach, and seeing as Glimpse is the result of a substantial university project (U of Arizona, right?), I'd bet we'd be lucky to equal its accuracy. Then again, that's *super* annoying for the users to have to go and separately install Glimpse...and it's likely overkill (seeing as it's capable of doing whole sites, so plugging it in to just do some mail archives might be a little excessive...although doing so would likely give good scalability for _huge_ lists). ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "College: the fountains of knowledge, where everyone goes to drink." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From gorgo@caesar.elte.hu Wed Jun 24 22:29:52 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Wed, 24 Jun 1998 23:29:52 +0200 (METDST) Subject: [Mailman-Developers] forwarded message from Dave Wolfe In-Reply-To: Message-ID: Talking about archiving and searching... I use the scripts created by Guy Maor for the debian lists archives. They're available as a debian package called lists-archives. It uses procmail to sort the mails to folders (so it can be used to archive any mailing list running on any host), groups them, etc, uses mhonarc for html conversion, and glimpse as a search engine. You can take a look at the archives at http://www.debian.org/Lists-Archives/ I modified it a bit not to use glimpse, and set up htdig as a search engine. Htdig seems very nice and fast, it has one big problem: it downloads all the .html-s, stores them in a db and then indexes them... so it needs a huge database. But at least it is gpl. I have this setup at http://mlf.linux.rulez.org/Archivum/ ... it is in Hungarian, so you probably won't understand it ;) Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 24 23:39:40 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 24 Jun 1998 18:39:40 -0400 (EDT) Subject: [Mailman-Developers] Easing the burden of upgrades Message-ID: <13713.32812.504844.940472@anthem.cnri.reston.va.us> We recently had an problem on python.org while upgrading to a new release of Mailman that started me thinking about a better way. I'll describe what I think is the problem, sketch out a proposed solution, and throw it out to y'all to discuss. Ken is as much responsible for the good ideas in this message as I am (blame me for the lousy ones); thanks to him for sitting down and thrashing this out first. The fundamental problem with a system like Mailman is that it is extremely difficult to test. The project is obviously not mature enough to have much of a test suite (if any), and writing one that tests all the interactions between MUA, MTA, Web browser, server, Python, and Mailman will be daunting to say the least. It'll be fantastic when we have even the framework for such a beast, but until then... So most of our testing involves creating and managing little toy lists, with us flogging the most noticeable features of Mailman to make sure the common stuff hasn't broken. The problem is that for sites using Mailman in an operational system, flag day (i.e. the day the upgrade to a new version occurs) can be pretty traumatic if we missed something crucial, but peculiar about a site. So I'm really concerned with how to make life less stressful for the operations folks who are relying on Mailman for their bread and butter. New features, fixes, etc. are strong incentive for those people to upgrade, but the fear of breakage (resulting in thousands of angry members) probably highly outweighs that incentive. Can we make the transition to a new version more controlled? You could go with a low tech approach of installing new versions temporarily with a different $prefix, using symbolic links to share list databases and templates, and hacking /etc/aliases as lists are converted to the new version. The one insurmountable problem that I see is the CGI URL. You can't share two different CGI bin dirs without exposing this to users through the list URLs. This is IMO, a showstopper; the most visible aspect of the system should be the most stable. Ken and I came up with the following architecture, and I'd like to see what y'all think: From a site administrators point of view, every Mailman installation has two `parities': a current parity and a future parity. Every list is associated with one of these parities; at any stable point in time every list is under the current parity. The system itself is associated with a parity, also the current parity at stable time. Now a new release comes out and Mailman automatically installs into the future parity (more on this below). None of the lists are automatically switched though. The site admin can switch the parity of individual lists. Let's say there's a command called `upgrade'. So upgrade toylist would switch `toylist' to the future parity. The site admin could run with this for a while, and get all warm and fuzzy about the new release. He would then upgrade reallist1 and repeat the process. Let's say he's upgraded three of his thirty lists and now has a lot of confidence in the new version. He then does upgrade * This converts all the lists to the future parity. There's one more twist however: the "system" itself is still running on the current parity even though all the lists are running on the future parity. One more command commit-parity would now commit to the new release; the future parity becomes the current parity and the current parity becomes the future parity. Maybe this auto-upgrades any still current parity lists. Data such as the list databases and templates would live outside the installed parity source code subdirs (more below). If at any time before the commit-parity is run, the site admin gets cold feet, he can downgrade reallistx or downgrade * to return the list or lists to the current parity. Now, when Mailman is installed, it always installs to the future parity, BUT ONLY IF ALL THE LIST PARITIES AND THE SYSTEM PARITY ARE CURRENT. If it ever notices that some lists are set to the future parity, but the system is still at the current parity, Mailman refuses to install. There would probably be a command to view the parities for lists and the system. I think the implementation would not be that difficult. A single file containing the parity status for each list and the system would be about the only database you'd need. The installed tree would change a bit. You'd probably have two directories inside $prefix, one that contains the current parity code and one that contains the future parity code. The list databases would live outside these two trees, directly under $prefix. List specific templates would be moved out of the templates directory, into $prefix/lists. I'd love to get some feedback from people. Is this really a problem that needs to be solved? Does this proposal solve the problem in a useful way? Is the abstraction clear? Is this just total overkill? -Barry From klm@python.org Wed Jun 24 23:52:54 1998 From: klm@python.org (Ken Manheimer) Date: Wed, 24 Jun 1998 18:52:54 -0400 (EDT) Subject: [Mailman-Developers] Easing the burden of upgrades In-Reply-To: <13713.32812.504844.940472@anthem.cnri.reston.va.us> Message-ID: I think barry's description conveys what we discussed real well. The only addition i'd make is a slightly different command arrangement. On Wed, 24 Jun 1998, Barry A. Warsaw wrote: > automatically switched though. The site admin can switch the parity > of individual lists. Let's say there's a command called `upgrade'. > So > upgrade toylist > [...] > upgrade reallist1 > [...] > lists and now has a lot of confidence in the new version. He then > does > upgrade * [upgrade all the lists] > commit-parity [make the future version the current] > downgrade reallistx > downgrade * I'd prefer to have a single command that could do everything - i'm thinking 'migrate', which defaults to migrating forward (upgrading). Eg: migrate toylist migrate -forward mailman-developers [[the "-forward" is unnecessary]] migrate -all [[migrate everything forward]] migrate -back -all [[back off!]] migrate -commit [[make the new version current]] (Since the command is in mailman's bindir, i guess it could be a general name - but if there's concern, we could distinguish it with mailman's initials - 'mmmigrate' - to convey the mmm mmm delicious experience of changing a production system...-) From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 24 23:57:22 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 24 Jun 1998 18:57:22 -0400 (EDT) Subject: [Mailman-Developers] Easing the burden of upgrades References: <13713.32812.504844.940472@anthem.cnri.reston.va.us> Message-ID: <13713.33874.951037.199204@anthem.cnri.reston.va.us> >>>>> "KM" == Ken Manheimer writes: KM> I think barry's description conveys what we discussed real KM> well. The only addition i'd make is a slightly different KM> command arrangement. I'm definitely open to suggestions here. I just wanted distinct command names so that it was clearer what the intent was. -Barry From akuchlin@cnri.reston.va.us Mon Jun 29 16:32:13 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Mon, 29 Jun 1998 11:32:13 -0400 (EDT) Subject: [Mailman-Developers] Rendered archives: where to put them? Message-ID: <13719.45251.814406.849296@newcnri.cnri.reston.va.us> I'm (finally!) revisiting the question of archiving for Mailman. There are two additional config parameters that seem to be needed. First, I need to know into what directory to write the formatted HTML files, both public and private. The private files shouldn't go into the actual Web tree, since they'll be accessed via the 'private' CGI script. Either there should be a parameter in mm_defaults/mm_cfg to specify the ARCHIVE_HTML_ROOT, or we could always write them into the Mailman tree somewhere -- say, ${prefix}/pipermail/html or /formatted or whatever. The second course would then require configuring the Web server to map your desired public archive URL to the root of the public tree, and the private archives to the CGI script. Second, I need to know what URL to use to access the public archives. (The private archives should be accessed via the 'private' CGI script.) Suggestions? -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Seek those who find your road agreeable, your personality and mind stimulating, your philosophy acceptable, and your experience helpful. Let those who do not, seek their own kind. -- Henri Fabre From scott@chronis.pobox.com Mon Jun 29 16:53:48 1998 From: scott@chronis.pobox.com (Scott) Date: Mon, 29 Jun 1998 11:53:48 -0400 Subject: [Mailman-Developers] Rendered archives: where to put them? In-Reply-To: <13719.45251.814406.849296@newcnri.cnri.reston.va.us>; from Andrew Kuchling on Mon, Jun 29, 1998 at 11:32:13AM -0400 References: <13719.45251.814406.849296@newcnri.cnri.reston.va.us> Message-ID: <19980629115348.38262@chronis.icgroup.com> On Mon, Jun 29, 1998 at 11:32:13AM -0400, Andrew Kuchling wrote: | I'm (finally!) revisiting the question of archiving for Mailman. | There are two additional config parameters that seem to be needed. great! | First, I need to know into what directory to write the formatted HTML | files, both public and private. The private files shouldn't go into | the actual Web tree, since they'll be accessed via the 'private' CGI | script. Either there should be a parameter in mm_defaults/mm_cfg to | specify the ARCHIVE_HTML_ROOT, or we could always write them into the | Mailman tree somewhere -- say, ${prefix}/pipermail/html or /formatted | or whatever. The second course would then require configuring the Web | server to map your desired public archive URL to the root of the | public tree, and the private archives to the CGI script. I'm for making it specifiable via a config variable. That way, large sites could put the archives on a separate partition from the mailman installation. | Second, I need to know what URL to use to access the public archives. | (The private archives should be accessed via the 'private' CGI | script.) how about http://HOST/mailman/archives/listname ? shouldn't this access both the private and the public archives? scott From akuchlin@cnri.reston.va.us Mon Jun 1 14:19:57 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Mon, 1 Jun 1998 09:19:57 -0400 (EDT) Subject: [Mailman-Developers] Feature suggestion: NNTP gateway Message-ID: <13682.43062.223767.769690@newcnri.cnri.reston.va.us> Michael McLay and I were discussed Mailman recently, and he suggested that Mailman could be used to run python-list. I pointed out that the news-to-mail gateway is the most important part of that, and Michael suggested adding it to Mailman as an option. So I guess the suggestion is: an optional component that can be configured with an NNTP server hostname and a group name, and will then mirror the traffic on both; messages posted to the group get sent out via e-mail, and e-mail messages get posted. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ There are great numbers of people to whom the act of reading a book -- any sort of book -- is wondrous; they speak of the reader in the tone of warm approbation which they use otherwise when referring to pregnant women, or the newly dead. -- Robertson Davies, _A Voice from the Attic_ From viega@list.org Mon Jun 1 14:31:29 1998 From: viega@list.org (John Viega) Date: Mon, 1 Jun 1998 06:31:29 -0700 Subject: [Mailman-Developers] Feature suggestion: NNTP gateway In-Reply-To: <13682.43062.223767.769690@newcnri.cnri.reston.va.us>; from Andrew Kuchling on Mon, Jun 01, 1998 at 09:19:57AM -0400 References: <13682.43062.223767.769690@newcnri.cnri.reston.va.us> Message-ID: <19980601063129.A20444@list.org> I wrote Mail->News and News->Mail gatewaying over the weekend. I haven't been able to test it yet, because I have to get posting access to a news server for list.org before I can. Luckily, a friend of mine is the news admin at Erols, so that should actually be today. So the feature you want will work by tomorrow, no doubt :) John On Mon, Jun 01, 1998 at 09:19:57AM -0400, Andrew Kuchling wrote: > Michael McLay and I were discussed Mailman recently, and he suggested > that Mailman could be used to run python-list. I pointed out that the > news-to-mail gateway is the most important part of that, and Michael > suggested adding it to Mailman as an option. > > So I guess the suggestion is: an optional component that can be > configured with an NNTP server hostname and a group name, and will > then mirror the traffic on both; messages posted to the group get > sent out via e-mail, and e-mail messages get posted. > > -- > A.M. Kuchling http://starship.skyport.net/crew/amk/ > There are great numbers of people to whom the act of reading a book -- any > sort of book -- is wondrous; they speak of the reader in the tone of warm > approbation which they use otherwise when referring to pregnant women, or the > newly dead. > -- Robertson Davies, _A Voice from the Attic_ > > > > _______________________________________________ > Mailman-Developers maillist - Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From akuchlin@cnri.reston.va.us Mon Jun 1 21:46:52 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Mon, 1 Jun 1998 16:46:52 -0400 (EDT) Subject: [Mailman-Developers] Pipermail In-Reply-To: <19980530193546.A8889@list.org> References: <19980530193546.A8889@list.org> Message-ID: <13683.4851.971836.472538@newcnri.cnri.reston.va.us> John Viega writes: >Has anyone modified pipermail's hypermail emulation to work using only >modules that are compiled in the default Python distribution? There's >one dependency I have in mind: pipermail currently uses bsddb, which >is not compiled into Python 1.5. I think gdbm is compiled in by >default, but the two interfaces aren't compatible. Pipermail needs some sort of tree, so it can traverse the indexes in sorted order without making a list of all the articles in memory & sorting it. I've been meaning to support Aaron's bplustree.py module, but haven't gotten around to it. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ "For Inanna: Apocatastasis" "You always did pick rotten titles." -- Inanna begins to read the director's final screenplay in SIGNAL TO NOISE From mclay@nist.gov Wed Jun 3 09:08:03 1998 From: mclay@nist.gov (Michael McLay) Date: Wed, 3 Jun 1998 04:08:03 -0400 (EDT) Subject: [Mailman-Developers] Welcome To "Mailman-Developers"! In-Reply-To: <199806031432.KAA26233@python.org> References: <199806031432.KAA26233@python.org> Message-ID: <13684.65095.959086.349259@fermi.eeel.nist.gov> I just joined the list and I've looked through some of the messages in the archive, but not all of them. In the mm_defaults for data storing directories are set as: LIST_DATA_DIR = os.path.join(MAILMAN_DIR, 'lists') HTML_DIR = os.path.join(HOME_DIR, 'public_html') CGI_DIR = os.path.join(HOME_DIR, 'cgi-bin') LOG_DIR = os.path.join(HOME_DIR, 'logs') LOCK_DIR = os.path.join(MAILMAN_DIR, 'locks') TEMPLATE_DIR = os.path.join(MAILMAN_DIR, 'templates') Is there a reason for not setting them to: LIST_DATA_DIR = os.path.join(HOME_DIR, 'lists') HTML_DIR = os.path.join(HOME_DIR, 'public_html') CGI_DIR = os.path.join(HOME_DIR, 'cgi-bin') LOG_DIR = os.path.join(HOME_DIR, 'logs') LOCK_DIR = os.path.join(HOME_DIR, 'locks') TEMPLATE_DIR = os.path.join(HOME_DIR, 'templates') This would keep the data separated from the software. The installation instructions would have to be changed to add the lists, locks, and templates directories to HOME_DIR. Is the install script for the next release going to include automatic creation of directories using autoconfig. I'm having some trouble do to the odd setup of our servers. The sendmail program is running on a system that is separate from the www server. When the command "/home/mailman/mailman/mail/wrapper post gencam" is run by sendmail it tries to archive the files on a disk that is on a different system and, if I understand correctly, this can't be done because setuid won't work across NFS mounts. I really like the look of mailman and I've read through many of the scripts. It is much easier on the eyes than Majordomo. The one conceptual problem I'm having with understanding the product is the interactions between the mailman user and the deamons. The permissions setting for mailman files are not obvious. Which files must be writable by "nobody" and which must be writeable by "mailman". Mailman seems to be in the middle between two deamons and a little more help in understanding the interactions would be appreciated. For instance, MAILMAN_DIR/lists/LISTNAME/config.db seems to require both mailman and nobody to have write permission. Has bundling mailman as a package been considered? some of the mm_X.py files look like good candidates to be hidden inside a package. From viega@list.org Wed Jun 3 16:21:20 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 08:21:20 -0700 Subject: [Mailman-Developers] Welcome To "Mailman-Developers"! In-Reply-To: <13684.65095.959086.349259@fermi.eeel.nist.gov>; from Michael McLay on Wed, Jun 03, 1998 at 04:08:03AM -0400 References: <199806031432.KAA26233@python.org> <13684.65095.959086.349259@fermi.eeel.nist.gov> Message-ID: <19980603082120.D8453@list.org> On Wed, Jun 03, 1998 at 04:08:03AM -0400, Michael McLay wrote: > Is the install script for the next release going to include automatic > creation of directories using autoconfig. Yes... thank Barry for that! > I'm having some trouble do to the odd setup of our servers. The > sendmail program is running on a system that is separate from the www > server. > > When the command "/home/mailman/mailman/mail/wrapper post gencam" is > run by sendmail it tries to archive the files on a disk that is on a > different system and, if I understand correctly, this can't be done > because setuid won't work across NFS mounts. Hmm... So web based files and mail based (data) files need to have seperate configurable directories. BTW, I was thinking about how to do SMTP and httpd on 2 seperate machines w/o NFS, and the easiest way is to run Mailman on the machine w/ the web server, and have sendmail aliases use rsh to run programs :) We should consider more efficient strategies, though, because I'm sure that people will want to do this sort of thing often. > > I really like the look of mailman and I've read through many of the > scripts. It is much easier on the eyes than Majordomo. The one > conceptual problem I'm having with understanding the product is the > interactions between the mailman user and the deamons. The > permissions setting for mailman files are not obvious. Which files > must be writable by "nobody" and which must be writeable by "mailman". > Mailman seems to be in the middle between two deamons and a little > more help in understanding the interactions would be appreciated. > For instance, MAILMAN_DIR/lists/LISTNAME/config.db seems to require > both mailman and nobody to have write permission. Hmm, all of these files should be owned by Mailman. However, both sendmail and cgi processes may need to modify these data files. So when a mail or web request comes in, we set the SUID bit. Hmm, if SUID doesn't work across file systems, you may have to decide that mail commands don't work, unless you mirror the list data file across systems, because both mail commands and list commands can be used to subscribe, change options, etc. > Has bundling mailman as a package been considered? some of the mm_X.py > files look like good candidates to be hidden inside a package. It's slowly happening, yes. Barry's done a lot of work towards that end. From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 3 16:57:49 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 3 Jun 1998 11:57:49 -0400 (EDT) Subject: [Mailman-Developers] Welcome To "Mailman-Developers"! References: <199806031432.KAA26233@python.org> <13684.65095.959086.349259@fermi.eeel.nist.gov> Message-ID: <13685.26608.520862.660655@anthem.CNRI.Reston.Va.US> Mike, A lot of your concerns will be addressed with the next release. In particular: >>>>> "MM" == Michael McLay writes: MM> This would keep the data separated from the software. The MM> installation instructions would have to be changed to add the MM> lists, locks, and templates directories to HOME_DIR. Much of this is going to change, because... MM> Is the install script for the next release going to include MM> automatic creation of directories using autoconfig. Yes! The installation directory can now be completely separate from the source directory. Standard GNU configure options --prefix and --exec_prefix can be used (although the latter doesn't make a lot of sense). Everything gets installed relative to $prefix and all the paths are set up correctly to deal with this. I've also rearranged where some of the files get installed. Eventually, I'm going to reorganize the source tree so it better reflects what the install tree looks like, but that'll come later (since it involves manually munging the CVS repository). I hope the install process will be much more straight forward in the next release. John's been debugging my autoconf stuff, so that gives me a lot more confidence :-) MM> I'm having some trouble do to the odd setup of our servers. MM> The sendmail program is running on a system that is separate MM> from the www server. I think John's move away from sendmail, toward SMTP delivery will alleviate much of this problem. MM> When the command "/home/mailman/mailman/mail/wrapper post MM> gencam" is run by sendmail it tries to archive the files on a MM> disk that is on a different system and, if I understand MM> correctly, this can't be done because setuid won't work across MM> NFS mounts. In the next release, nothing will have to be setuid [*]. I've reworked the permissions so that anything that needs write permission is set-gid, which I think does work across NFS mounts (please correct me if I'm wrong). MM> I really like the look of mailman and I've read through many MM> of the scripts. It is much easier on the eyes than Majordomo. Yes! MM> The one conceptual problem I'm having with understanding the MM> product is the interactions between the mailman user and the MM> deamons. The permissions setting for mailman files are not MM> obvious. Which files must be writable by "nobody" and which MM> must be writeable by "mailman". Mailman seems to be in the MM> middle between two deamons and a little more help in MM> understanding the interactions would be appreciated. For MM> instance, MAILMAN_DIR/lists/LISTNAME/config.db seems to MM> require both mailman and nobody to have write permission. I ran across the same problems. Using setgid scripts has solved most of this for me. MM> Has bundling mailman as a package been considered? some of the MM> mm_X.py files look like good candidates to be hidden inside a MM> package. Already done. Actually, the mm_*.py files get installed into $prefix/Mailman, and there's a __init__.py file under there, so it's already packagized. Something to do in the future (IMO), is to get rid of the mm_ prefix on the files, and fix the imports to assume packagization. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 3 17:02:48 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 3 Jun 1998 12:02:48 -0400 (EDT) Subject: [Mailman-Developers] Welcome To "Mailman-Developers"! References: <199806031432.KAA26233@python.org> <13684.65095.959086.349259@fermi.eeel.nist.gov> <19980603082120.D8453@list.org> Message-ID: <13685.29509.585038.473545@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Hmm, all of these files should be owned by Mailman. Actually, I don't *think* this is a requirement anymore. I've been running some small test lists with myself as the owner, but all the files group-owned `mailman'. That, in combination with the set-gid bit (*not* the set-uid bit), seems to work for both web and mail access to the files. I'm still not 100% sure that set-gid works across NFS though (can't test it right now). -Barry From cklempay@jerkweed.ml.org Wed Jun 3 22:06:21 1998 From: cklempay@jerkweed.ml.org (Corbett Klempay) Date: Wed, 3 Jun 1998 17:06:21 -0400 (EDT) Subject: [Mailman-Developers] ETA? Message-ID: Not to be annoying or anything, but would anyone be able to venture a guess as to the ETA for b4? Since I convinced the other ACM admin to move from Majordomo to Mailman, he's waiting until b4 is released to migrate all mail and list services to our new machine. Any info is appreciated. Thanks :) ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "There exists in India no politician daring enough to attempt to explain to the masses that cows can be eaten." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From viega@list.org Wed Jun 3 22:12:04 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 14:12:04 -0700 Subject: [Mailman-Developers] ETA? In-Reply-To: ; from Corbett Klempay on Wed, Jun 03, 1998 at 05:06:21PM -0400 References: Message-ID: <19980603141204.B14356@list.org> Barry and I are working on putting together the b4 release as we speak. I think there will be an announcement within an hour. However, we're only going to announce b4 to this mailing list, because since archiving is still not integrated, we'd really prefer not to put b4 into really wide release... John On Wed, Jun 03, 1998 at 05:06:21PM -0400, Corbett Klempay wrote: > Not to be annoying or anything, but would anyone be able to venture a > guess as to the ETA for b4? Since I convinced the other ACM admin to move > from Majordomo to Mailman, he's waiting until b4 is released to migrate > all mail and list services to our new machine. Any info is appreciated. > Thanks :) > > ------------------------------------------------------------------------------- > Corbett J. Klempay Quote of the Week: > http://www2.acm.jhu.edu/~cklempay "There exists in India no politician > daring enough to attempt to explain > to the masses that cows can be eaten." > > PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 > ------------------------------------------------------------------------------- > > > > _______________________________________________ > Mailman-Developers maillist - Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From cklempay@jerkweed.ml.org Wed Jun 3 22:25:00 1998 From: cklempay@jerkweed.ml.org (Corbett Klempay) Date: Wed, 3 Jun 1998 17:25:00 -0400 (EDT) Subject: [Mailman-Developers] ETA? In-Reply-To: <19980603141204.B14356@list.org> Message-ID: On Wed, 3 Jun 1998, John Viega wrote: > Barry and I are working on putting together the b4 release as we > speak. I think there will be an announcement within an hour. > However, we're only going to announce b4 to this mailing list, because > since archiving is still not integrated, we'd really prefer not to put > b4 into really wide release... > > John So does this mean that Pipermail integration will probably show up in b5? Would it be a "good idea" for me to just install this b4 now, and then put in b5 (or whatever the following release is) after that? Is that easy to do? (upgrade to the future version with archiving when I'm already running b4) Any idea how long until archiving is implemented? Hehe...I need to start writing some Python programs (and stop writing so many Perl ones!) so I can start contributing...argh :) Corbett From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 3 22:27:37 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 3 Jun 1998 17:27:37 -0400 (EDT) Subject: [Mailman-Developers] ETA? References: <19980603141204.B14356@list.org> Message-ID: <13685.49018.78031.892336@anthem.CNRI.Reston.Va.US> >>>>> "CK" == Corbett Klempay writes: CK> So does this mean that Pipermail integration will probably CK> show up in b5? Would it be a "good idea" for me to just CK> install this b4 now, and then put in b5 (or whatever the CK> following release is) after that? Is that easy to do? CK> (upgrade to the future version with archiving when I'm already CK> running b4) Any idea how long until archiving is implemented? I think now that we're autoconf'd, it should be easier to install a new version over an existing version, since the installation process shouldn't touch any of the important database files. I emphasize *should*; I can't say this stuff has been tested extensively ;-) CK> Hehe...I need to start writing some Python programs (and stop CK> writing so many Perl ones!) so I can start contributing...argh CK> :) Indeed! :-) From viega@list.org Wed Jun 3 22:29:42 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 14:29:42 -0700 Subject: [Mailman-Developers] ETA? In-Reply-To: ; from Corbett Klempay on Wed, Jun 03, 1998 at 05:25:00PM -0400 References: <19980603141204.B14356@list.org> Message-ID: <19980603142942.E14356@list.org> On Wed, Jun 03, 1998 at 05:25:00PM -0400, Corbett Klempay wrote: > On Wed, 3 Jun 1998, John Viega wrote: > > > Barry and I are working on putting together the b4 release as we > > speak. I think there will be an announcement within an hour. > > However, we're only going to announce b4 to this mailing list, because > > since archiving is still not integrated, we'd really prefer not to put > > b4 into really wide release... > > > > John > > So does this mean that Pipermail integration will probably show up in b5? > Would it be a "good idea" for me to just install this b4 now, and then put > in b5 (or whatever the following release is) after that? Is that easy to > do? (upgrade to the future version with archiving when I'm already running > b4) Any idea how long until archiving is implemented? Yes, upgrading should be as simple as installing over your old install (after backing it up, of course, because you never know). I'd like brchiving to go (back) in by b5. The problem is, it requires a bit of work to hook up Pipermail and Mailman in a portable way, and I don't see anyone stepping forward :) Which means I'll be the lucky guy to do it. I'm getting ready to move and start a new job, though, so I'd anticipate 2-3 weeks before I'll actually get that done. > > Hehe...I need to start writing some Python programs (and stop writing so > many Perl ones!) so I can start contributing...argh :) Please :) The more the merrier :) John From viega@list.org Wed Jun 3 23:15:35 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 15:15:35 -0700 Subject: [Mailman-Developers] Mailman 1.0b4 released Message-ID: <19980603151535.A15436@list.org> Mailman 1.0b4 is now available from: ftp://list.org/pub/mailman/mailman-1.0b4.tar.gz This release is only intended to see a small distribution, primarily this development community. The first "major" release is still a couple of weeks away (some people still need to sign copyright forms for GNU, archiving has to be integrated back in, etc.) Developers, please feel encouraged to go to the Mailman home page, look at the developer's TODO list, and pick out a project. If you are thus inclined, the Mailman home page is located at: http://www.list.org/ The major changes since 1.0b3 are: - A configure script for easy installation (Barry Warsaw) - The ability to install Mailman to locations other than /home/mailman (Barry Warsaw) - Use cookies on the admin pages (also hides admin pages from others) (Scott Cotton) - Subscription requests send a request for confirmation, which may be done by simply replying to the message (Scott Cotton) - Facilities for gating mail to a newsgroup, and for gating a newsgroup to a mailing list (John Viega) - Contact the SMTP port instead of calling sendmail (primarily for portability) (JohnViega) - Changed all links on web pages to relative links where appropriate. (John Viega) - Use MD5 if crypt is not available (John Viega) - Lots of fixing up of bounce handling (Ken Manheimer) - General UI polishing (Ken Manheimer) - mm_html: Make it prominent when the user's delivery is disabled on his option page. (Ken Manheimer) Enjoy, John From viega@list.org Wed Jun 3 23:35:04 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 15:35:04 -0700 Subject: [Mailman-Developers] News->Mail gateways Message-ID: <19980603153504.D15479@list.org> The new release includes both Mail->News gatewaying and News->Mail gatewaying. The latter uses a polling strategy. If you have direct access to your nntp server, it would be possible not to have to poll. In fact, it would be fairly easy. Unfortunately, I don't have that sort of access available. If anyone is interested in getting non-polling gatewaying working, please contact me, I'm interested in writing the necessary code, I just need someone who can test it, etc. John From mclay@nist.gov Thu Jun 4 00:01:17 1998 From: mclay@nist.gov (Michael McLay) Date: Wed, 3 Jun 1998 19:01:17 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <19980603153504.D15479@list.org> References: <19980603153504.D15479@list.org> Message-ID: <13685.54195.324584.618375@fermi.eeel.nist.gov> I've downloaded the distribution and tried running configure. Here's the error messages. bash$ ./configure creating cache ./config.cache checking for --with-python... checking for python... /usr/local/bin/python checking Python interpreter... /usr/local/bin/python checking for a BSD compatible install... /usr/bin/install -c checking whether make sets ${MAKE}... yes checking for --without-gcc... no checking for gcc... no checking for cc... /usr/bin/cc checking whether the C compiler (/usr/bin/cc ) works... yes checking whether the C compiler (/usr/bin/cc ) is a cross-compiler... no checking whether we are using GNU C... no checking whether #! works in shell scripts... yes checking for --with-mail-gid... other mail daemon checking for mail_wrapper GID... ./configure: python: not found cat: conftest.out: No such file or directory configure: error: ***** Group other mail daemon not found! You must first ***** set up the other mail daemon entry in /etc/group ***** or set the MAIL_GID shell variable on the ***** configure command line I don't understand this comment. My system administrator has set up the directory structure as follows: bash$ cd /home/ bash$ ls -ldg mailman lrwxrwxrwx 1 root wheel 13 Jun 3 18:51 mailman -> /bolt/mailman bash$ The web server is running on bolt so the /bold/mailman directory is visible on that system. The mail server runs on sparky and sparky uses some Solaris automouter magic to make /home/mailman appear as a directory, but /bolt/mailman doesn't appear. I'll ask if I can get the /home/mailman directory to be changed to ownership by mailman and put in group mailman. For now it is owned by root and in group wheel. From cklempay@jerkweed.ml.org Thu Jun 4 00:44:11 1998 From: cklempay@jerkweed.ml.org (Corbett Klempay) Date: Wed, 3 Jun 1998 19:44:11 -0400 (EDT) Subject: [Mailman-Developers] 2 Quickies Message-ID: Being the big newbie, I am having 2 snags so far in my Mailman 1.0b4 installation (Red Hat 5.1): 1) Cron is giving me this every 5 minutes: Date: Wed, 3 Jun 1998 19:20:01 -0400 From: Cron Daemon To: mailman@chimera.acm.jhu.edu Subject: Cron /usr/bin/python /home/mailman/cron/gate_news Traceback (innermost last): File "/home/mailman/cron/gate_news", line 41, in ? file = open(os.path.join(mm_cfg.DATA_DIR, "gate_watermarks.last"), "w") IOError: (13, 'Permission denied') 2) And I'm getting a 403 Forbidden on the test list...I'm thinking this might be related to the other admin installing cgi-wrap...I'm not too keen on how it works. Does is the user mailman supposed to belong to just the users group, or users as well as the mailman group? CK From viega@list.org Thu Jun 4 03:24:31 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 19:24:31 -0700 Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13685.54195.324584.618375@fermi.eeel.nist.gov>; from Michael McLay on Wed, Jun 03, 1998 at 07:01:17PM -0400 References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> Message-ID: <19980603192431.A17219@list.org> On Wed, Jun 03, 1998 at 07:01:17PM -0400, Michael McLay wrote: > checking for --with-mail-gid... other mail daemon > checking for mail_wrapper GID... ./configure: python: not found > cat: conftest.out: No such file or directory > configure: error: > > ***** Group other mail daemon not found! You must first > ***** set up the other mail daemon entry in /etc/group > ***** or set the MAIL_GID shell variable on the > ***** configure command line Well, it looks like Python isn't in your path when installing, and it needs to be. The configure script is trying to run Python to guess which group your SMTPD runs under, but can't run Python, and thus can't come up with a guess. John From viega@list.org Thu Jun 4 03:43:21 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 19:43:21 -0700 Subject: [Mailman-Developers] 2 Quickies In-Reply-To: ; from Corbett Klempay on Wed, Jun 03, 1998 at 07:44:11PM -0400 References: Message-ID: <19980603194321.B17219@list.org> On Wed, Jun 03, 1998 at 07:44:11PM -0400, Corbett Klempay wrote: > Being the big newbie, I am having 2 snags so far in my Mailman 1.0b4 > installation (Red Hat 5.1): > > 1) > Cron is giving me this every 5 minutes: > > Date: Wed, 3 Jun 1998 19:20:01 -0400 > From: Cron Daemon > To: mailman@chimera.acm.jhu.edu > Subject: Cron /usr/bin/python > /home/mailman/cron/gate_news > > Traceback (innermost last): > File "/home/mailman/cron/gate_news", line 41, in ? > file = open(os.path.join(mm_cfg.DATA_DIR, "gate_watermarks.last"), > "w") > IOError: (13, 'Permission denied') That's interesting, as I can't reproduce this problem on my system. What does an ls -l on $prefix/data give you? BTW, you can just comment out the gate-news line and reinstall the crontab until we can figure out exactly what's going on... > 2) > And I'm getting a 403 Forbidden on the test list...I'm thinking this might > be related to the other admin installing cgi-wrap...I'm not too keen on > how it works. This sounds likely. I don't remember much about cgi-wrap. I think the URLs almost certainly have to be a different format. Something like: http://www.server.com/cgi-bin/cgiwrap/mailman/name-of-script And then name-of-script has to live in ~mailman/cgi-bin. I can't remember, you might want to check the man page on cgiwrap. However, if you added the ScriptAlias line or whatever to your httpd conf and restarted, you shouldn't need to use cgi-wrap, I'd think. Watch syslog / maillog and see if you've got a GID error, as that also sounds possible. > > Does is the user mailman supposed to belong to just the users group, or > users as well as the mailman group? It should belong to the mailman group... From viega@list.org Thu Jun 4 03:59:37 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 19:59:37 -0700 Subject: [Mailman-Developers] 2 Quickies In-Reply-To: <19980603194321.B17219@list.org>; from John Viega on Wed, Jun 03, 1998 at 07:43:21PM -0700 References: <19980603194321.B17219@list.org> Message-ID: <19980603195937.A17572@list.org> On Wed, Jun 03, 1998 at 07:43:21PM -0700, John Viega wrote: > On Wed, Jun 03, 1998 at 07:44:11PM -0400, Corbett Klempay wrote: > > Being the big newbie, I am having 2 snags so far in my Mailman 1.0b4 > > installation (Red Hat 5.1): > > Traceback (innermost last): > > File "/home/mailman/cron/gate_news", line 41, in ? > > file = open(os.path.join(mm_cfg.DATA_DIR, "gate_watermarks.last"), > > "w") > > IOError: (13, 'Permission denied') > > That's interesting, as I can't reproduce this problem on my system. BTW, I'm running redhat 5.1 too. I've been thinking about this problem, and I'm thinking that your problem is potentially something like mailman files are getting created as mailman.user instead of mailman.mailman, or something like that. That could cause problems. I'd suggest making sure that Mailman's /etc/passwd entry has the mailman GID in the group field, and not the user GID. Then you probably will want to re-install. Anyway, if that still doesn't work, or if that isn't a problem, then let me know what ls -l says on $prefix/data... John From viega@list.org Thu Jun 4 06:47:32 1998 From: viega@list.org (John Viega) Date: Wed, 3 Jun 1998 22:47:32 -0700 Subject: [Mailman-Developers] 2 Quickies In-Reply-To: ; from Corbett Klempay on Wed, Jun 03, 1998 at 07:44:11PM -0400 References: Message-ID: <19980603224732.A19017@list.org> Just to let everyone know how this one turned out: On Wed, Jun 03, 1998 at 07:44:11PM -0400, Corbett Klempay wrote: > 1) > Cron is giving me this every 5 minutes: [...] > Traceback (innermost last): > File "/home/mailman/cron/gate_news", line 41, in ? > file = open(os.path.join(mm_cfg.DATA_DIR, "gate_watermarks.last"), > "w") > IOError: (13, 'Permission denied') Mailman was installed w/ group user instead of group mailman. > 2) > And I'm getting a 403 Forbidden on the test list...I'm thinking this might > be related to the other admin installing cgi-wrap...I'm not too keen on > how it works. The problem here was that /home/mailman needed to be "chmod a+rx"'d. That one should be mentioned in the install document! John From viega@list.org Thu Jun 4 14:26:05 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 06:26:05 -0700 Subject: [Mailman-Developers] CGI thoughts Message-ID: <19980604062605.C21640@list.org> I was just thinking that it's a little much to require people to go in and change their httpd configuration files. Something we could do for the next release would be to provide a single C binary, and tell people, "Put this in your system cgi-bin directory". Then mailman URLS would become: http://www.blah.com/cgi-bin/mailman/listinfo I think while the URL may get a little bigger, it has the advantage of being a lot less error-prone to install. Of course, we could always have an option to configure that would keep things the way they are. Thoughts? John From viega@list.org Thu Jun 4 15:38:23 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 07:38:23 -0700 Subject: [Mailman-Developers] Cookies (again) Message-ID: <19980604073823.B22159@list.org> Okay, I just let a friend of mine set up a list. The first thing he said was, "it looks great, but my changes don't submit, it gives me the login screen and then my changes are gone". Well, he's almost certainly got cookies turned off. Do you think it would be a good idea to put a password field above the "Submit your changes" button, and label it: "If your browser doesn't have cookies on, put your password here"? John From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 15:42:51 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 10:42:51 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> Message-ID: <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Well, it looks like Python isn't in your path when installing, JV> and it needs to be. The configure script is trying to run JV> Python to guess which group your SMTPD runs under, but can't JV> run Python, and thus can't come up with a guess. John's right. The problem is that the configure script looks on your $PATH for python (which it didn't find), but defaults to /usr/local/bin/python (which it did find). If configure never found a python binary, it would have exited earlier. The problem is that other tests in the configure script assume python is on your path, and in your case it isn't. This is exactly the kind of bug I was hoping would get fleshed out by this beta, so... thanks Mike! :-) Try this patch. It changes the tests so that it uses the python binary it found in the earlier test. -Barry P.S. The error message isn't very good, so I'll change that, but not for this patch. -------------------- snip snip -------------------- Index: configure =================================================================== RCS file: /projects/cvsroot/mailman/configure,v retrieving revision 1.9 diff -C2 -r1.9 configure *** configure 1998/06/03 22:25:49 1.9 --- configure 1998/06/04 14:39:13 *************** *** 1,5 **** #! /bin/sh ! # From configure.in Revision: 1.9 # Guess values for system-dependent variables and create Makefiles. --- 1,5 ---- #! /bin/sh ! # From configure.in Revision: 1.10 # Guess values for system-dependent variables and create Makefiles. *************** *** 1005,1009 **** fp.close() EOF ! python conftest.py MAIL_GID=`cat conftest.out` fi --- 1005,1009 ---- fp.close() EOF ! $PYTHON conftest.py MAIL_GID=`cat conftest.out` fi *************** *** 1061,1065 **** fp.close() EOF ! python conftest.py CGI_GID=`cat conftest.out` fi --- 1061,1065 ---- fp.close() EOF ! $PYTHON conftest.py CGI_GID=`cat conftest.out` fi *************** *** 1110,1114 **** fp.close() EOF ! python conftest.py echo $ac_n "checking for default fully qualified host name""... $ac_c" 1>&6 --- 1110,1114 ---- fp.close() EOF ! $PYTHON conftest.py echo $ac_n "checking for default fully qualified host name""... $ac_c" 1>&6 From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 16:08:55 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 11:08:55 -0400 (EDT) Subject: [Mailman-Developers] 2 Quickies References: <19980603194321.B17219@list.org> <19980603195937.A17572@list.org> Message-ID: <13686.46847.40420.137132@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> BTW, I'm running redhat 5.1 too. I've been thinking about JV> this problem, and I'm thinking that your problem is JV> potentially something like mailman files are getting created JV> as mailman.user instead of mailman.mailman, or something like JV> that. That could cause problems. I'd suggest making sure JV> that Mailman's /etc/passwd entry has the mailman GID in the JV> group field, and not the user GID. Then you probably will JV> want to re-install. But that shouldn't be happening, because I go to pains to install the mailman directories with the group set-id (a.k.a. group sticky) bit turned on. Do an ls -l on $prefix/data an dyou should see something like drwxrwsr-x. The semantics are described like this in chmod(2) on Solaris: In a directory which has the set-group-ID bit set (reflected as either -----s--- or ----- l--- in the output of 'ls -ld'), files and subdirectories are created with the group-ID of the parent directory-not that of current process. So all files created under $prefix/data should definitely be group owned by mailman. The INSTALL file says you must create $prefix with the group sticky bit set, and chgrp it to mailman. If you didn't do this, then the groups on all the files will cascade to be incorrectly group owned. Do you think this is what happened? It probably makes sense for either the configure script, or the install target to explicitly check for this scenario. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 16:12:34 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 11:12:34 -0400 (EDT) Subject: [Mailman-Developers] 2 Quickies References: <19980603224732.A19017@list.org> Message-ID: <13686.47320.462795.179182@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Mailman was installed w/ group user instead of group mailman. Do you mean that $prefix wasn't chgrp'd mailman and chmod'd g+s? I'd really like to know if those instructions in the INSTALL are incorrect or not. If they just overlooked, then I can write a test to guarantee that's set up correctly. JV> The problem here was that /home/mailman needed to be "chmod JV> a+rx"'d. That one should be mentioned in the install JV> document! Okay, here's what the INSTALL doc will say about setting up $prefix: Make sure this directory is set to group `mailman' and has the group sticky bit set. You probably also want to guarantee that this directory is read and executeable by everyone. For example, these shell commands will accomplish this: % cd $prefix % chgrp mailman . % chmod a+rx,g+s . -Barry From viega@list.org Thu Jun 4 16:15:58 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 08:15:58 -0700 Subject: [Mailman-Developers] 2 Quickies In-Reply-To: <13686.47320.462795.179182@anthem.CNRI.Reston.Va.US>; from Barry A. Warsaw on Thu, Jun 04, 1998 at 11:12:34AM -0400 References: <19980603224732.A19017@list.org> <13686.47320.462795.179182@anthem.CNRI.Reston.Va.US> Message-ID: <19980604081558.E22159@list.org> On Thu, Jun 04, 1998 at 11:12:34AM -0400, Barry A. Warsaw wrote: > > >>>>> "JV" == John Viega writes: > > JV> Mailman was installed w/ group user instead of group mailman. > > Do you mean that $prefix wasn't chgrp'd mailman and chmod'd g+s? Yeah, I think that it may have been a case of skimming through the install docs, which I'm sure we've all done before! John From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 16:27:34 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 11:27:34 -0400 (EDT) Subject: [Mailman-Developers] 2 Quickies References: <19980603224732.A19017@list.org> <13686.47320.462795.179182@anthem.CNRI.Reston.Va.US> <19980604081558.E22159@list.org> Message-ID: <13686.48301.125281.839505@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Yeah, I think that it may have been a case of skimming through JV> the install docs, which I'm sure we've all done before! Yeah, I've been guilty too! Well, I'll add an explicit check (someplace). Much better for configure or make install to barf and point you at the documentation, then to just let people screw themselves. ;-) -Barry From mclay@nist.gov Thu Jun 4 21:43:01 1998 From: mclay@nist.gov (Michael McLay) Date: Thu, 4 Jun 1998 16:43:01 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> Message-ID: <13686.65494.140292.268905@fermi.eeel.nist.gov> Barry A. Warsaw writes: > > >>>>> "JV" == John Viega writes: > > JV> Well, it looks like Python isn't in your path when installing, > JV> and it needs to be. The configure script is trying to run > JV> Python to guess which group your SMTPD runs under, but can't > JV> run Python, and thus can't come up with a guess. > > John's right. The problem is that the configure script looks on your > $PATH for python (which it didn't find), but defaults to > /usr/local/bin/python (which it did find). If configure never found a > python binary, it would have exited earlier. > > The problem is that other tests in the configure script assume python > is on your path, and in your case it isn't. This is exactly the kind > of bug I was hoping would get fleshed out by this beta, so... thanks > Mike! :-) > > Try this patch. It changes the tests so that it uses the python > binary it found in the earlier test. Hmm, I fixed the problem so python is found so I didn't exercise this patch. There's one small buglet in the code generated by configure. The script which generates the crontab.in file adds blank lines to the file. This bombs the crontab command on the old SunOS system we are using. There is also a link error when building the source code on SunOS using gcc version 2.7.2. Here's the snippet that shows the error. make[1]: Entering directory `/usr/local/mailman/mailman-1.0b4/src' gcc -c -I. -DPREFIX="\"/usr/local/mailman\"" -DPYTHON="\"/usr/local/bin/python\"" -g -O2 -DHAVE_SYSLOG_H=1 -DGETGROUPS_T=int -DHAVE_VPRINTF=1 common.c gcc -DSCRIPT="\"admin\"" -I. -DCGI_GID=55 -g -O2 -DHAVE_SYSLOG_H=1 -DGETGROUPS_T=int -DHAVE_VPRINTF=1 common.o -o admin cgi-wrapper.c collect2: ld returned 2 exit status ld: Undefined symbol _strerror make[1]: *** [admin] Error 1 make[1]: Leaving directory `/usr/local/mailman/mailman-1.0b4/src' make[1]: Entering directory `/usr/local/mailman/mailman-1.0b4/templates' for f in archives.html handle_opts.html listinfo.html options.html roster.html subscribe.html; \ do \ /bin/install -c -m 644 $f /usr/local/mailman/templates; \ done make[1]: Leaving directory `/usr/local/mailman/mailman-1.0b4/templates' From mclay@nist.gov Thu Jun 4 21:49:26 1998 From: mclay@nist.gov (Michael McLay) Date: Thu, 4 Jun 1998 16:49:26 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> Message-ID: <13687.2049.860850.719792@fermi.eeel.nist.gov> I tried changing the compiler form gcc to gcc and reran make install. This time stdarg.h isn't found. make[1]: Entering directory `/usr/local/mailman/mailman-1.0b4/src' cc -c -I. -DPREFIX="\"/usr/local/mailman\"" -DPYTHON="\"/usr/local/bin/python\"" -O -DHAVE_SYSLOG_H=1 -DGETGROUPS_T=int -DHAVE_VPRINTF=1 common.c ./common.h: 22: Can't find include file stdarg.h make[1]: *** [common.o] Error 2 make[1]: Leaving directory `/usr/local/mailman/mailman-1.0b4/src' make[1]: Entering directory `/usr/local/mailman/mailman-1.0b4/templates' for f in archives.html handle_opts.html listinfo.html options.html roster.html subscribe.html; \ do \ /bin/install -c -m 644 $f /usr/local/mailman/templates; \ done make[1]: Leaving directory `/usr/local/mailman/mailman-1.0b4/templates' From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 22:06:52 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 17:06:52 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13687.2049.860850.719792@fermi.eeel.nist.gov> Message-ID: <13687.3075.35966.630102@anthem.CNRI.Reston.Va.US> >>>>> "MM" == Michael McLay writes: MM> I tried changing the compiler form gcc to gcc and reran make --------------------------------------------------^^^ you meant to say "to the bundled SunOS cc"? If so, note that that's a non-ANSI compiler and so old that I can't imagine we'd want to support it. The gcc 2.7.2 bug is valid though. I *think* I've got an old gcc laying around I can try. -Barry From mclay@nist.gov Thu Jun 4 22:15:26 1998 From: mclay@nist.gov (Michael McLay) Date: Thu, 4 Jun 1998 17:15:26 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13687.3075.35966.630102@anthem.CNRI.Reston.Va.US> References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13687.2049.860850.719792@fermi.eeel.nist.gov> <13687.3075.35966.630102@anthem.CNRI.Reston.Va.US> Message-ID: <13687.3412.944873.484780@fermi.eeel.nist.gov> Barry A. Warsaw writes: > > >>>>> "MM" == Michael McLay writes: > > MM> I tried changing the compiler form gcc to gcc and reran make > --------------------------------------------------^^^ > you meant to say "to the bundled SunOS cc"? Opps, yes I did mean cc. > If so, note that that's a non-ANSI compiler and so old that I can't > imagine we'd want to support it. Fine with me. I think we need to get rid of this old hardware anyway and if you refuse to support K&R then it actually helps my case. > The gcc 2.7.2 bug is valid though. I *think* I've got an old gcc > laying around I can try. Yea, I know this is out of date too. It probably isn't even installed properly. Time to fire off a message to our system administrator. In the mean time... I need a fix to get this working with what I have. From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Thu Jun 4 23:08:30 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 18:08:30 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13687.2049.860850.719792@fermi.eeel.nist.gov> <13687.3075.35966.630102@anthem.CNRI.Reston.Va.US> <13687.3412.944873.484780@fermi.eeel.nist.gov> Message-ID: <13687.6756.901874.669951@anthem.CNRI.Reston.Va.US> >>>>> "MM" == Michael McLay writes: MM> Fine with me. I think we need to get rid of this old hardware MM> anyway and if you refuse to support K&R then it actually helps MM> my case. Okay, then I refuse! :-) >> The gcc 2.7.2 bug is valid though. I *think* I've got an old >> gcc laying around I can try. MM> Yea, I know this is out of date too. It probably isn't even MM> installed properly. Time to fire off a message to our system MM> administrator. MM> In the mean time... I need a fix to get this working with MM> what I have. Looks like I don't have an old gcc anymore. If I can pull down 2.7.*.* from an ftp site then I'll give that a whirl. Stay tuned. If anybody else has an older version of gcc and can work up a patch in the meantime, that would be greatly appreciated! -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 5 00:21:31 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 4 Jun 1998 19:21:31 -0400 (EDT) Subject: [Mailman-Developers] News->Mail gateways References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13686.65494.140292.268905@fermi.eeel.nist.gov> Message-ID: <13687.11188.942023.123916@anthem.CNRI.Reston.Va.US> >>>>> "MM" == Michael McLay writes: MM> There's one small buglet in the code generated by configure. MM> The script which generates the crontab.in file adds blank MM> lines to the file. This bombs the crontab command on the old MM> SunOS system we are using. It doesn't seem to add blank lines for me, although the crontab.in.in file already contains a blank line, in the middle of the file. Is that the problem? If so, it's trivial to fix (but I want to be sure that's your problem first). -Barry From viega@list.org Fri Jun 5 03:56:12 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 19:56:12 -0700 Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <13686.65494.140292.268905@fermi.eeel.nist.gov>; from Michael McLay on Thu, Jun 04, 1998 at 04:43:01PM -0400 References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13686.65494.140292.268905@fermi.eeel.nist.gov> Message-ID: <19980604195612.D28273@list.org> On Thu, Jun 04, 1998 at 04:43:01PM -0400, Michael McLay wrote: > > There is also a link error when building the source code on > SunOS using gcc version 2.7.2. Here's the snippet that shows the > error. Okay, it took me a while to find a machine to use sunning sunOS 4 and gcc 2.7.2 :) > ld: Undefined symbol > _strerror Yes, apparently strerror() is nowhere to be found on such a system. I have 2 functions that I just wrote up. The first duplicates the behavior that strerror gives on other systems. The second saves the large static buffer, and will do for our purposes. Just drop your pick into common.c, and it should then compile: Version 1 (Compatible): #define STACK_BUFSIZE 8192 extern char *sys_errlist[]; char* strerror(int errno) { static char msg[STACK_BUFSIZE]; strcpy(msg, sys_errlist[errno]); return msg; } Version 2 (Efficient): extern char *sys_errlist[]; char* strerror(int errno) { return sys_errlist[errno]; } From viega@list.org Fri Jun 5 04:10:47 1998 From: viega@list.org (John Viega) Date: Thu, 4 Jun 1998 20:10:47 -0700 Subject: [Mailman-Developers] News->Mail gateways In-Reply-To: <19980604195612.D28273@list.org>; from John Viega on Thu, Jun 04, 1998 at 07:56:12PM -0700 References: <19980603153504.D15479@list.org> <13685.54195.324584.618375@fermi.eeel.nist.gov> <19980603192431.A17219@list.org> <13686.45493.857565.306200@anthem.CNRI.Reston.Va.US> <13686.65494.140292.268905@fermi.eeel.nist.gov> <19980604195612.D28273@list.org> Message-ID: <19980604201047.F28273@list.org> Um, whoops, s/STACK/STATIC throughout that snipit of code, I don't know where my mind is today. > #define STACK_BUFSIZE 8192 Anyway, Here's what I recommend we use in the dist, if and only if strerror can't be found by the autoconf process: extern char *sys_errlist[]; extern int sys_nerr; char* strerror(int errno) { if(errno < 0 || errno >= sys_nerr) { return "unknown error"; } else { return sys_errlist[errno]; } } From janne@avocado.pc.helsinki.fi Fri Jun 5 10:11:10 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 05 Jun 1998 12:11:10 +0300 Subject: [Mailman-Developers] A bug in 1.0b4 (subscriber's options) Message-ID: If I go to editing subscriber's options by writing an e-mail address onto the listinfo page, submitting the changes does not work. Instead I get an error saying handle_opts: no such list. Instead, editing options by clicking the subscriber's address on the subscriber list works fine. -- Janne From viega@list.org Fri Jun 5 11:01:54 1998 From: viega@list.org (John Viega) Date: Fri, 5 Jun 1998 03:01:54 -0700 Subject: [Mailman-Developers] A bug in 1.0b4 (subscriber's options) In-Reply-To: ; from Janne Sinkkonen on Fri, Jun 05, 1998 at 12:11:10PM +0300 References: Message-ID: <19980605030154.B31743@list.org> On Fri, Jun 05, 1998 at 12:11:10PM +0300, Janne Sinkkonen wrote: > > If I go to editing subscriber's options by writing an e-mail address > onto the listinfo page, submitting the changes does not work. > Instead I get an error saying handle_opts: no such list. That's interesting. For some reason in that one case, PATH_INFO is lacking a leading /, whereas in all other cases it has one, I think. So here's a really tiny patch that fixes the problem. It seems to not break anything else. diff -r1.28 mm_utils.py 232c232,234 < nesting_level = len(string.split(os.environ['PATH_INFO'], '/')) - 1 --- > if path[0] <> '/': > path= '/' + path > nesting_level = len(string.split(path, '/')) - 1 From mclay@nist.gov Fri Jun 5 17:32:32 1998 From: mclay@nist.gov (Michael McLay) Date: Fri, 5 Jun 1998 12:32:32 -0400 (EDT) Subject: [Mailman-Developers] file permissions Message-ID: <199806051632.MAA09671@fermi.eeel.nist.gov> Either I screwed up when I followed the installation instructions or else the file ownership set by "make install" incorrectly set the group of the executables in $prefix/cgi-bin to that of the super user. In order for the scripts to work the group had to be manually set to "mailman". I accidentally created a copy of $prefix/data/pending_subscriptions.db with the wrong group ownership which caused the subcribe script to fail with an "Internal Server Error" The script failed because of a file access permission error. In order to find the problem I did some code reorganization. The $prefix/scripts/subscribe file was moved to $prefix/Mailman/subscribe.py and the attached file was put in place of the $prefix/scripts/subscribe file. This change should also reduce the script execution time slightly because the longer script will be compiled to a .pyc file. I would like to wrap all the other file in the $prefix/scripts/ in the same manner. Once this is done the common.c program can be modified to insert the PREFIX directory into the path rather than the MODULEDIR. This will eliminate the need for the paths module in each of the scripts and it would eliminate the need for the following lines in my new wrapper script: sys.path.remove('/usr/local/mailman/Mailman') sys.path.insert(0, '/usr/local/mailman') This generalized error wrapper would also eliminate some redundant error reporting code in admin. John, what do you think of removing the mm_ prefix from the module names? It's not really needed now that everything is pushed into the Mailman package. ----------------------- $prefix/scripts/subscribe --------------------- #! /usr/bin/env python # # Copyright (C) 1998 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. """Process listinfo form submission, ie subscriptions or roster requests.""" import sys from Mailman.subscribe import * try: sys.path.remove('/usr/local/mailman/Mailman') sys.path.insert(0, '/usr/local/mailman') from Mailman.subscribe import * except SystemExit: pass except: print "Content-type: text/html\n" print "

We're sorry, we hit a bug!

\n" print "If you would like to help us identify the problem, please " print "email a copy of this page to the webmaster for this site" print 'with a description of what happened. Thanks!' print "\n
"
	  print sys.argv
	  try:
	    import traceback
	    sys.stderr = sys.stdout
	    traceback.print_exc()
	  except:
	    print "[failed to get traceback]"
	  print "\n\n
" From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 5 18:13:50 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 5 Jun 1998 13:13:50 -0400 (EDT) Subject: [Mailman-Developers] Some improvement suggestions References: <199805282356.TAA09183@glyph.CNRI.Reston.Va.US> Message-ID: <13688.9845.106446.411919@anthem.CNRI.Reston.Va.US> Catching up on some issues... >>>>> "KM" == Ken Manheimer writes: KM> The first item is the ability to edit the new-member welcome KM> message.[...] KM> I think we would have to change the format substitutions, KM> currently all the "%s" order dependent style, to be KM> "%(real_name)s" dictionary substitutions, and otherwise do the KM> same kind of thing we do for the html templates. (Ah, i see KM> barry's done that in some recent checkins.) There is a bunch KM> of messages, some which may benefit from this treatment, other KM> might just clutter the list admins choices. Dragon subsequently mentioned Digicool's DocumentTemplate stuff, and that might certainly be a long-term goal. One of the short term things I'd like to do is to move most of the triple-quoted-strings of text that serve as templates, out of the .py files, and into text file in say, $prefix/data. Then convert them to using dictionary interpolation of values. I think this will help move us in a direction of being able to do things perhaps like I18N, using DocumentTemplate (perhaps), etc. It'll certainly make it easier to track down and change various publically visible texts. -Barry From mclay@nist.gov Fri Jun 5 18:31:06 1998 From: mclay@nist.gov (Michael McLay) Date: Fri, 5 Jun 1998 13:31:06 -0400 (EDT) Subject: [Mailman-Developers] file permissions In-Reply-To: <199806051632.MAA09671@fermi.eeel.nist.gov> References: <199806051632.MAA09671@fermi.eeel.nist.gov> Message-ID: <13688.9300.328001.568060@fermi.eeel.nist.gov> Nothing like following up to your own posts:-) Michael McLay writes: > In order to find the problem I did some > code reorganization. The $prefix/scripts/subscribe file was moved to > $prefix/Mailman/subscribe.py and the attached file was put in place of > the $prefix/scripts/subscribe file. > > This change should also reduce the script execution time slightly > because the longer script will be compiled to a .pyc file. I would > like to wrap all the other file in the $prefix/scripts/ in the same > manner. Once this is done the common.c program can be modified to > insert the PREFIX directory into the path rather than the MODULEDIR. Why not have the install script install the package Mailman directly in the user's site-python directory? Isn't that where it belongs? Then the path manipulation doesn't have to take place at all. > This will eliminate the need for the paths module in each of the > scripts and it would eliminate the need for the following lines in my > new wrapper script: > > sys.path.remove('/usr/local/mailman/Mailman') > sys.path.insert(0, '/usr/local/mailman') > > This generalized error wrapper would also eliminate some redundant > error reporting code in admin. I've done a little more work on generalizing the wrapper and adding some more optional debug information. The new module Mailman.debug contains two functions. The first one prints out the trace information. The second function will print out uid and gid information and the server environment variables. I turn off the more detailed messages using a debug flag in the $prefix/script/* scripts. If the Mailman is moved to site-python then the "import sys" statement could be deleted and the only statement inside the try block would be from Mailman.subscribe import * The SystemExit exception could be eliminated as well if the code in Mailman.subscribe were rearranged to eliminate the call to sys.exit(). The new arrangement also moves the debug code out of the scripts unless an exception is called. This reduces $prefix/script/subscribe file to the following: ########################## $prefix/script/subscribe ################# #! /usr/bin/env python # # Copyright (C) 1998 by the Free Software Foundation, Inc. # under the GNU General Public License See the License file in the top level # of this distribution for details. """Process subscribe form submission, ie subscriptions or roster requests.""" debug = 1 # turn on to enable environment parameter printing on errors import sys try: sys.path.remove('/usr/local/mailman/Mailman') sys.path.insert(0, '/usr/local/mailman') from Mailman.subscribe import * except SystemExit: pass except: import Mailman.debug Mailman.debug.print_trace() if debug: Mailman.debug.print_environ() From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 5 19:38:35 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 5 Jun 1998 14:38:35 -0400 (EDT) Subject: [Mailman-Developers] Pipermail References: <19980530193546.A8889@list.org> Message-ID: <13688.15003.865713.418391@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> To that end, I've put in provisions for passwords to use md5 JV> if crypt is not found. BTW, do you guys think md5 should be JV> the default, and crypt an option for those who add it to their JV> config? The only real advantage of crypt is that it is less JV> space intensive. Also, the way I have things set up at the JV> moment, if someone starts w/ the default distribution and JV> later recompiles Python to include crypt, all the old JV> passwords won't work, since they'll be in the md5 digest JV> format. For the reason you state, and since md5 is generally compiled in by default and crypt may not be, I'd say just go with md5 all the time. Space shouldn't be an issue really. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 5 19:46:55 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 5 Jun 1998 14:46:55 -0400 (EDT) Subject: [Mailman-Developers] CGI thoughts References: <19980604062605.C21640@list.org> Message-ID: <13688.15486.523947.679072@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> I was just thinking that it's a little much to require people JV> to go in and change their httpd configuration files. JV> Something we could do for the next release would be to provide JV> a single C binary, and tell people, "Put this in your system JV> cgi-bin directory". Then mailman URLS would become: JV> http://www.blah.com/cgi-bin/mailman/listinfo JV> I think while the URL may get a little bigger, it has the JV> advantage of being a lot less error-prone to install. Actually, Ken and I have talked about trying to *shorten* the URL, but we haven't come up with anything that'll work. It's a pain-tradeoff: how much pain do you put the installer through so that the larger number of users go through less pain (less typing) to get to the relevent pages? One thing you *could* do is install the `listinfo' and other wrapper binaries into cgi-bin, but call them mm_listinfo. I'm not crazy about that idea, and I don't think the mods to httpd conf files are all that daunting right now. I could be wrong though. -Barry From mclay@nist.gov Fri Jun 5 19:50:02 1998 From: mclay@nist.gov (Michael McLay) Date: Fri, 5 Jun 1998 14:50:02 -0400 (EDT) Subject: [Mailman-Developers] Mailman not delivering the mail Message-ID: <199806051850.OAA09749@fermi.eeel.nist.gov> The subscribe script execute and the Web page that is returned says a message will be sent to the subscriber, but no message arrives. I think something is wrong with gid or uid. By adding an intentional bug in Mailman/mm_utils.py the attached trace was triggered just after msg.GetSender() was called. Here's the bug that was added: import smtplib sender = msg.GetSender() findbug The debugging code that was added shows that the script is being executed as nobody. Am I correct in assuming that the mailer won't send out a message from nobody because nobody is not a trusted user? uid 65534 (nobody) euid 65534 (nobody) gid 55 (mailman) egid 50 (wwwcgi) - a group set up for cgi scripts. I also was curious about the wwwcgi egid. This is the gid setting used inside the httpd configuration, but when the subscribe cgi script create the data/pending_subscriptions.db the group is set to mailman. I'm confused about what the settings of all these things should be. I also found the layers used to send out the mail messages to be puzzling. Why the trip from maillist -> MailList -> mm_deliver -> Deliverer - > DeliverToUser -> mm_utils -> DeliverToUser. Why so much indirection? Michael ###################### REPLY MESSAGE ############################### We're sorry, we hit a bug! If you would like to help us identify the problem, please email a copy of this page to the webmaster for this site with a description of what happened. Thanks! ['/usr/local/mailman/scripts/subscribe'] Traceback (innermost last): File "/usr/local/mailman/scripts/subscribe", line 26, in ? from Mailman.subscribe import * File "/usr/local/mailman/Mailman/subscribe.py", line 151, in ? list.SendTextToUser(subject = "%s -- confirmation of subscription -- request %d" % \ File "/usr/local/mailman/Mailman/mm_deliver.py", line 105, in SendTextToUser mm_utils.SendTextToUser(subject, text, recipient, sender, File "/usr/local/mailman/Mailman/mm_utils.py", line 145, in SendTextToUser DeliverToUser(msg, recipient, add_headers=add_headers) File "/usr/local/mailman/Mailman/mm_utils.py", line 162, in DeliverToUser findbug NameError: findbug Execution privilage on the Server uid 65534 euid 65534 gid 55 egid 50 Environment Variable on the Server DOCUMENT_ROOT /usr/local/etc/httpd/htdocs REMOTE_HOST fermi CONTENT_TYPE application/x-www-form-urlencoded PATH_TRANSLATED /usr/local/etc/httpd/htdocs/test/ REMOTE_ADDR 129.6.185.13 GATEWAY_INTERFACE CGI/1.1 HTTP_COOKIE NVL_prefs=yvt0fss0000ss0s000000s000s00000000000000000000ss0s0000000000QQMichael%20McLay SERVER_PROTOCOL HTTP/1.0 HTTP_ACCEPT_LANGUAGE en REMOTE_PORT 25123 SERVER_PORT 80 HTTP_CONNECTION Keep-Alive HTTP_USER_AGENT Mozilla/4.05 [en] (X11; I; Linux 2.0.31 i686) HTTP_ACCEPT_CHARSET iso-8859-1,*,utf-8 HTTP_ACCEPT image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */* REQUEST_URI /mailman/subscribe/test/ PATH /usr/openwin/bin/xview:/usr/openwin/bin:/usr/local/bin:/etc:/usr/etc:/usr/lib:/usr/ucb:/bin:/usr/bin:/usr/etc/yp:/usr/plum:.:/usr/asap/bin:/usr/sbmt QUERY_STRING SCRIPT_FILENAME /usr/local/mailman/cgi-bin/subscribe CONTENT_LENGTH 71 HTTP_HOST www.eeel.nist.gov REQUEST_METHOD POST SCRIPT_NAME /mailman/subscribe SERVER_ADMIN kbrady@eeel.nist.gov SERVER_SOFTWARE Apache/1.2.5 PYTHONPATH /usr/local/mailman/Mailman PATH_INFO /test/ HTTP_REFERER http://www.eeel.nist.gov/mailman/listinfo/test SERVER_NAME www.eeel.nist.gov DEBUG Traceback (innermost last): File "/usr/local/mailman/scripts/subscribe", line 35, in ? Mailman.debug.print_environ() File "/usr/local/mailman/Mailman/debug.py", line 37, in print_environ print "DEBUG", debug NameError: debug Content-type: text/html Test Subscription results Confirmation from your email address is required, to prevent anyone from covertly subscribing you. Instructions are being sent to you at mclay@nist.gov. List run by mclay@nist.gov HTML generated by Mailman v 1.0b4 From steve@estel.uindy.edu Sat Jun 6 11:51:50 1998 From: steve@estel.uindy.edu (Steve Spicklemire) Date: Sat, 6 Jun 1998 05:51:50 -0500 (EST) Subject: [Mailman-Developers] Lock patches? Message-ID: <199806061051.FAA17364@estel.uindy.edu> Hi There, I'm using mailman on NeXTStep. One of the problems I ran into was the non-functionality of lock file support on NeXTStep. Soooo.... I cooked up a (more or less) satisfactory replacement. Would anyone else on the list be interested in the canges? Basically I create a normal file as the 'lock'. I create it with flags that require mailman to have exclusive write access to it. If another mailman process is already using the file, the create fails. There is an optional 'timeout' period, so that a hung/dead process won't cut out all other mailman processes. Are there other platforms where file locking is a problem? -steve From janne@avocado.pc.helsinki.fi Sat Jun 6 12:45:15 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 06 Jun 1998 14:45:15 +0300 Subject: [Mailman-Developers] '&' in an address Message-ID: I have never before seen an address with '&' in it, but now we have 'r&d.nylund@somewhere.com' on one of our lists. It is a problem, since the error log shows the followig when a message is posted to the list: Jun 06 07:55:39 1998 post: Traceback (innermost last): post: File "/home/mailman/scripts/mailowner", line 43, in ? post: if not current_list.bounce_processing or not current_list.ScanMessage(msg): post: File "/home/mailman/Mailman/mm_bouncer.py", line 407, in ScanMessage post: self.RegisterBounce(who, msg) post: File "/home/mailman/Mailman/mm_bouncer.py", line 83, in RegisterBounce post: self.LogMsg("bounce", report + "first") post: File "/home/mailman/Mailman/maillist.py", line 570, in LogMsg post: logf.write("%s\n" % (msg % args)) post: TypeError : not enough arguments for format string I have 1.0b4, but the subscription was made while we were running an earlier version. It's possible that the address found its way to the list through the admin's mass subscription. Either mailman shouldn't allow such addresses to be subscribed, or then the post script needs fixing. -- Janne From viega@list.org Sat Jun 6 13:56:29 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 05:56:29 -0700 Subject: [Mailman-Developers] file permissions In-Reply-To: <199806051632.MAA09671@fermi.eeel.nist.gov>; from Michael McLay on Fri, Jun 05, 1998 at 12:32:32PM -0400 References: <199806051632.MAA09671@fermi.eeel.nist.gov> Message-ID: <19980606055629.A12273@list.org> On Fri, Jun 05, 1998 at 12:32:32PM -0400, Michael McLay wrote: > This change should also reduce the script execution time slightly > because the longer script will be compiled to a .pyc file. I would > like to wrap all the other file in the $prefix/scripts/ in the same > manner. Once this is done the common.c program can be modified to > insert the PREFIX directory into the path rather than the MODULEDIR. > This will eliminate the need for the paths module in each of the > scripts and it would eliminate the need for the following lines in my > new wrapper script: One thing I'm thinking is that setenv() has portability problems, and it may be better not to use it at all. > John, what do you think of removing the mm_ prefix from the module > names? It's not really needed now that everything is pushed into the > Mailman package. I think it's fine for the prefix to go away, although I would probably either like to see the files that are used as mixins to retain a prefix, or see a better directory structure, especially since now there are going to be all of these CGI scripts in the package, too: there will be a lot of files in one directory, and it won't be easy to figure out what groups with what. John From viega@list.org Sat Jun 6 14:03:35 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 06:03:35 -0700 Subject: [Mailman-Developers] file permissions In-Reply-To: <13688.9300.328001.568060@fermi.eeel.nist.gov>; from Michael McLay on Fri, Jun 05, 1998 at 01:31:06PM -0400 References: <199806051632.MAA09671@fermi.eeel.nist.gov> <13688.9300.328001.568060@fermi.eeel.nist.gov> Message-ID: <19980606060335.B12273@list.org> On Fri, Jun 05, 1998 at 01:31:06PM -0400, Michael McLay wrote: > > Why not have the install script install the package Mailman directly > in the user's site-python directory? Isn't that where it belongs? > Then the path manipulation doesn't have to take place at all. Well, currently the install portion shouldn't be done as root, and unless Mailman has access to the site-python directly, it would become necessary. I think it's a good thing to be able to install this software without having to have root access. True, unless you run an MTA like qmail, you're still going to need someone to add mail aliases for you. But we could add an optional make root-install target or something like that that would go back to using the alias wrapper, etc. > I've done a little more work on generalizing the wrapper and adding > some more optional debug information. The new module Mailman.debug > contains two functions. The first one prints out the trace > information. The second function will print out uid and gid > information and the server environment variables. I turn off the more > detailed messages using a debug flag in the $prefix/script/* scripts. Nice work on the wrapper code! John From viega@list.org Sat Jun 6 14:25:28 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 06:25:28 -0700 Subject: [Mailman-Developers] Mailman not delivering the mail In-Reply-To: <199806051850.OAA09749@fermi.eeel.nist.gov>; from Michael McLay on Fri, Jun 05, 1998 at 02:50:02PM -0400 References: <199806051850.OAA09749@fermi.eeel.nist.gov> Message-ID: <19980606062528.C12273@list.org> On Fri, Jun 05, 1998 at 02:50:02PM -0400, Michael McLay wrote: > > The subscribe script execute and the Web page that is returned says > a message will be sent to the subscriber, but no message arrives. > I think something is wrong with gid or uid. If something is wrong w/ the gid, it should get logged to your system maillog before the scripts ever get run, so it is more likely something wrong with file permissions. From an earlier post I'm wondering if you perhaps ran make install as root, and if that might not end up causing problems? Assuming the program actually trys to deliver the message, it will hopefully throw an error. However, it probably won't do so where you can get a trace, since the delivery code opens a pipe to the delivery code to avoid smtp hanging hanging mailman. So I'd add debug code to the deliver script and the contact_transport script that log stderr to a file, so you can see if any file problems are making it up to the Python level. > The debugging code that was added shows that the script is being > executed as nobody. Am I correct in assuming that the mailer won't > send out a message from nobody because nobody is not a trusted user? No, that should not be the case. Trusted user status means that if you set the sender via -f when calling sendmail directly, sendmail will allow that. Otherwise, it will put in a header saying: "Warning: user nobody set sender to 'blah' using -f". Since we don't even directly call Sendmail, we don't have to worry about trusted user status. > > uid 65534 (nobody) > euid 65534 (nobody) > gid 55 (mailman) > egid 50 (wwwcgi) - a group set up for cgi scripts. > > I also was curious about the wwwcgi egid. This is the gid setting > used inside the httpd configuration, but when the subscribe cgi script > create the data/pending_subscriptions.db the group is set to mailman. Right. All of the files should be created as belonging to group mailman. The httpd and smtpd each run with different gids. To let them modify files in group mailman, yet keep them accessable by the mailman user, we have setgid wrapper scripts (written in C for portability). > I'm confused about what the settings of all these things should be. Your settings look right. Again, if they were wrong, you wouldn't even get to the Python; the C wrapper would log to your system mail log. > I also found the layers used to send out the mail messages to be > puzzling. Why the trip from maillist -> MailList -> mm_deliver -> > Deliverer - > DeliverToUser -> mm_utils -> DeliverToUser. Why so much > indirection? I don't understand why you're mixing methods, classes and files here. The path is just: maillist.DeliverToUser -> mm_utils.DeliverToUser. .DeliverToUser() is a method in maillist for delivering mail. However, sometimes mail gets delivered that is not associated with any particular list (and thus there will be no list object). Thefore, mailman.DeliverToUser() makes a call out to the utils module. Duplicating the code in both places would lead to maintainance problems. Now if you're wondering why the functionality in mm_deliver is not just brought into maillist.py, that's done intentionally to keep related functionality together, in as small a unit as necessary. I find single flat files with tons of functionality tend to develop into spaghetti rather quickly. From viega@list.org Sat Jun 6 14:37:22 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 06:37:22 -0700 Subject: [Mailman-Developers] Lock patches? In-Reply-To: <199806061051.FAA17364@estel.uindy.edu>; from Steve Spicklemire on Sat, Jun 06, 1998 at 05:51:50AM -0500 References: <199806061051.FAA17364@estel.uindy.edu> Message-ID: <19980606063722.E12273@list.org> On Sat, Jun 06, 1998 at 05:51:50AM -0500, Steve Spicklemire wrote: > > Hi There, > > I'm using mailman on NeXTStep. One of the problems I ran into > was the non-functionality of lock file support on NeXTStep. Soooo.... > I cooked up a (more or less) satisfactory replacement. Would anyone > else on the list be interested in the canges? Basically I create a > normal file as the 'lock'. I create it with flags that require mailman > to have exclusive write access to it. If another mailman process is > already using the file, the create fails. There is an optional > 'timeout' period, so that a hung/dead process won't cut out all other > mailman processes. Are there other platforms where file locking is a > problem? Steve, The locking that we currently do definitely needs replacement. It doesn't work on FreeBSD, apparently. If you've got a portable solution, we'd love to have it. And if you're interested in improving the granularity of locking throughout the system, let us know, we'd like that to happen as well :) John From viega@list.org Sat Jun 6 14:44:28 1998 From: viega@list.org (John Viega) Date: Sat, 6 Jun 1998 06:44:28 -0700 Subject: [Mailman-Developers] '&' in an address In-Reply-To: ; from Janne Sinkkonen on Sat, Jun 06, 1998 at 02:45:15PM +0300 References: Message-ID: <19980606064428.F12273@list.org> On Sat, Jun 06, 1998 at 02:45:15PM +0300, Janne Sinkkonen wrote: > > I have never before seen an address with '&' in it, but now we have > 'r&d.nylund@somewhere.com' on one of our lists. It is a problem, > since the error log shows the followig when a message is posted to the > list: ... > I have 1.0b4, but the subscription was made while we were running an > earlier version. It's possible that the address found its way to the > list through the admin's mass subscription. > > Either mailman shouldn't allow such addresses to be subscribed, or > then the post script needs fixing. I would have to look this one up, but if & is supposed to be valid in email addresses, we should probably try to support it, even though it's a special character in URLs, and will thus be a bit tricky to work around. I'll have to look this one up, and implement the right solution (anyone else is welcome to beat me to it :). Let's say I will make sure there's some sort of solution by b5. For now, I would suggest you remove the address from the list. The admin membership page may work for this, or you may need to fire up Python and remove them by hand: import maillist m = maillist.MailList('whatever-list') m.members.remove('offending_address') m.Save() Then exit the interpreter so the list will Unlock, or call m.Unlock(). If he's a digest member, use m.digest_members instead... John From janne@avocado.pc.helsinki.fi Sat Jun 6 14:48:48 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 06 Jun 1998 16:48:48 +0300 Subject: [Mailman-Developers] '&' in an address In-Reply-To: John Viega's message of "Sat, 6 Jun 1998 06:44:28 -0700" References: <19980606064428.F12273@list.org> Message-ID: John Viega writes: > will make sure there's some sort of solution by b5. For now, I would > suggest you remove the address from the list. No problem here. If somebody has such a mail address, (s)he should be prepared to suffer. :) The admin removed it already, apparently it was easy because he managed to do it without any help. -- Janne From viega@list.org Mon Jun 8 22:29:43 1998 From: viega@list.org (John Viega) Date: Mon, 8 Jun 1998 14:29:43 -0700 Subject: [Mailman-Developers] fyi Message-ID: <19980608142943.A1701@list.org> I just wanted to let everyone know that I'm not going to be as quick to respond as I normally try to be for the rest of the week. I'm just starting a new job, and I have to still set stuff up before I have outside access. So I'm only going to be reading mail once a day until probably the end of the week. Over the weekend I plan to catch up on integrating patches from this list and making the changes I promised a day or so ago. John From janne@avocado.pc.helsinki.fi Tue Jun 9 16:39:06 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 09 Jun 1998 18:39:06 +0300 Subject: [Mailman-Developers] Does bouncing by header regexps really work? Message-ID: I don't think so. Or do I have something wrong with these? > content-type: multipart > x-ms-attachment: .* > return-receipt-to: .* -- Janne From viega@list.org Tue Jun 9 16:47:52 1998 From: viega@list.org (John Viega) Date: Tue, 9 Jun 1998 08:47:52 -0700 Subject: [Mailman-Developers] Does bouncing by header regexps really work? In-Reply-To: ; from Janne Sinkkonen on Tue, Jun 09, 1998 at 06:39:06PM +0300 References: Message-ID: <19980609084752.A9178@list.org> I won't have the chance to look until later this week, but I'm wondering if it may be a case sensitivity problem? John On Tue, Jun 09, 1998 at 06:39:06PM +0300, Janne Sinkkonen wrote: > > I don't think so. Or do I have something wrong with these? > > > content-type: multipart > > x-ms-attachment: .* > > return-receipt-to: .* > > -- > Janne > > _______________________________________________ > Mailman-Developers maillist - Mailman-Developers@python.org > http://www.python.org/mailman/listinfo/mailman-developers From janne@avocado.pc.helsinki.fi Tue Jun 9 17:59:31 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 09 Jun 1998 19:59:31 +0300 Subject: [Mailman-Developers] Does bouncing by header regexps really work? In-Reply-To: John Viega's message of "Tue, 9 Jun 1998 08:47:52 -0700" References: <19980609084752.A9178@list.org> Message-ID: John Viega writes: > I won't have the chance to look until later this week, but I'm > wondering if it may be a case sensitivity problem? OK, this seems to help: --- maillist.py~ Thu Jun 4 23:12:28 1998 +++ maillist.py Tue Jun 9 19:39:45 1998 @@ -718,13 +718,13 @@ continue else: try: - h, e = re.split(":[ ]*", line) - all.append((h, e, line)) + h, e = re.split(":[ ]*", stripped) + all.append((h, e, stripped)) except ValueError: # Whoops - some bad data got by: self.LogMsg("config", "%s - " "bad bounce_matching_header line %s" - % (self.real_name, `line`)) + % (self.real_name, `stripped`)) return all @@ -748,7 +748,7 @@ and f[l] == ":" and string.lower(field) == string.lower(f[0:l])): # Non-continuation line - trim header name: - subjs.append(f[l+1:]) + subjs.append(f[l+2:]) elif not subjs: # Whoops - non-continuation that matches? subjs.append(f) From scott@chronis.pobox.com Fri Jun 12 00:35:25 1998 From: scott@chronis.pobox.com (Scott) Date: Thu, 11 Jun 1998 19:35:25 -0400 Subject: [Mailman-Developers] smtplib buglet in 1.0b4 Message-ID: <19980611193525.15573@chronis.icgroup.com> the smtplib module in mailman 1.0b4 doesn't quite use strict rfc complient smtp syntax. Vmailer is strict about it, and other mailers may be as well. I changed smtplib.py so it sends "MAIL FROM:
" and "RCPT TO:
" instead of "MAIL FROM: address" and "RCPT TO: address". scott here's a patch: chronis 7:32pm $ diff -c /home/mm/Mailman/smtplib.py /home/scott/mailman-1.0b4/modules/smtplib.py *** /home/mm/Mailman/smtplib.py Thu Jun 11 19:30:06 1998 --- /home/scott/mailman-1.0b4/modules/smtplib.py Fri May 29 23:09:21 1998 *************** *** 64,77 **** if headers: hlines = string.split(headers, '\n') lines = string.split(text, '\n') ! self._sock.send('MAIL FROM: <%s>\r\n' % frm) self.getresp() if type(to) == types.StringType: ! self._sock.send('RCPT TO: <%s>\r\n' % to) self.getresp() else: for item in to: ! self._sock.send('RCPT TO: <%s>\r\n' % item) self.getresp() self._sock.send('DATA\r\n') self.getresp() --- 64,77 ---- if headers: hlines = string.split(headers, '\n') lines = string.split(text, '\n') ! self._sock.send('MAIL FROM: %s\r\n' % frm) self.getresp() if type(to) == types.StringType: ! self._sock.send('RCPT TO: %s\r\n' % to) self.getresp() else: for item in to: ! self._sock.send('RCPT TO: %s\r\n' % item) self.getresp() self._sock.send('DATA\r\n') self.getresp() From dragondm@nexus.Integral.org Fri Jun 12 04:05:39 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Thu, 11 Jun 1998 22:05:39 -0500 (CDT) Subject: [Mailman-Developers] smtplib buglet in 1.0b4 In-Reply-To: <19980611193525.15573@chronis.icgroup.com> Message-ID: On Thu, 11 Jun 1998, Scott wrote: > Date: Thu, 11 Jun 1998 19:35:25 -0400 > From: Scott > To: mailman-developers@python.org > Subject: [Mailman-Developers] smtplib buglet in 1.0b4 > > > the smtplib module in mailman 1.0b4 doesn't quite use strict rfc > complient smtp syntax. Vmailer is strict about it, and other mailers > may be as well. I changed smtplib.py so it sends "MAIL FROM: >
" and "RCPT TO:
" instead of "MAIL FROM: address" > and "RCPT TO: address". Hmm.... Actually, I'm pretty sure that not having the <>'s on the address is perfectly fine according to the RFC's, as long as you don't have any comments in the address. Are you sure this isn't a glitch w/ Vmailer? PS, is there any specific reason that MailMan keeps using it's own smtplib, rather than the one that is part of the Python 1.5.1 standard library? -The Dragon De Monsyne From viega@list.org Fri Jun 12 13:46:44 1998 From: viega@list.org (John Viega) Date: Fri, 12 Jun 1998 05:46:44 -0700 Subject: [Mailman-Developers] smtplib buglet in 1.0b4 In-Reply-To: ; from The Dragon De Monsyne on Thu, Jun 11, 1998 at 10:05:39PM -0500 References: <19980611193525.15573@chronis.icgroup.com> Message-ID: <19980612054644.A30905@list.org> On Thu, Jun 11, 1998 at 10:05:39PM -0500, The Dragon De Monsyne wrote: > > Hmm.... Actually, I'm pretty sure that not having the <>'s on the > address is perfectly fine according to the RFC's, as long as you don't > have any comments in the address. Are you sure this isn't a glitch w/ > Vmailer? Actually, Scott's right, RFC 821 does indeed require angle brackets. > PS, is there any specific reason that MailMan keeps using it's own > smtplib, rather than the one that is part of the Python 1.5.1 standard > library? The initial reason was that the user interface on Mailman's SMTP lib was easier to program to. For example, it was a lot easier to send a single message to a list of recipients. The other reason is that Mailman releases should come out more often than Python releases. Both smtplibs are pretty new software, and are pretty likely to change. The patch from Scott is a good example. So at some point I'd be happy to use the other smtplib, but we'd still end up providing a more up to date version than available w/ Python, probably. John From dragondm@nexus.Integral.org Fri Jun 12 15:02:18 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Fri, 12 Jun 1998 09:02:18 -0500 (CDT) Subject: [Mailman-Developers] smtplib buglet in 1.0b4 In-Reply-To: <19980612054644.A30905@list.org> Message-ID: On Fri, 12 Jun 1998, John Viega wrote: > On Thu, Jun 11, 1998 at 10:05:39PM -0500, The Dragon De Monsyne wrote: > > > > Hmm.... Actually, I'm pretty sure that not having the <>'s on the > > address is perfectly fine according to the RFC's, as long as you don't > > have any comments in the address. Are you sure this isn't a glitch w/ > > Vmailer? > > Actually, Scott's right, RFC 821 does indeed require angle brackets. Hmm... Indeed. Confusing language oon the rfc on that... I've fixed this in the latest edition of the standard smtplib (found at http://www.integral.org/~dragondm/python/smtplib.py) I'll be posting that to c.l.python tonight, too. > > PS, is there any specific reason that MailMan keeps using it's own > > smtplib, rather than the one that is part of the Python 1.5.1 standard > > library? > > The initial reason was that the user interface on Mailman's SMTP lib > was easier to program to. For example, it was a lot easier to send a > single message to a list of recipients. The other reason is that Hmmm?? Howso? I've looked at the code, as far as I can tell, the std smtplib has all the same interfaces the mm smtplib does, only differing in the name of the smtp class. The python-lib smtplib also has a method, called 'sendmail' which does the whole transaction for a list of recipients. > Mailman releases should come out more often than Python releases. > Both smtplibs are pretty new software, and are pretty likely to > change. The patch from Scott is a good example. So at some point I'd > be happy to use the other smtplib, but we'd still end up providing a > more up to date version than available w/ Python, probably. Prolly. -The Dragon De Monsyne From viega@list.org Fri Jun 12 15:54:53 1998 From: viega@list.org (John Viega) Date: Fri, 12 Jun 1998 07:54:53 -0700 Subject: [Mailman-Developers] smtplib buglet in 1.0b4 In-Reply-To: ; from The Dragon De Monsyne on Fri, Jun 12, 1998 at 09:02:18AM -0500 References: <19980612054644.A30905@list.org> Message-ID: <19980612075453.B4123@list.org> On Fri, Jun 12, 1998 at 09:02:18AM -0500, The Dragon De Monsyne wrote: > On Fri, 12 Jun 1998, John Viega wrote: > The python-lib smtplib also has a method, called 'sendmail' which does the > whole transaction for a list of recipients. Okay, I missed that one. There was also some other issue about massaging header data into the format needed, and then there was something else I can't remember. Nothing was a big deal, but it was less time consuming to program to a module I wrote :) From viega@list.org Fri Jun 12 20:21:53 1998 From: viega@list.org (John Viega) Date: Fri, 12 Jun 1998 12:21:53 -0700 Subject: [Mailman-Developers] Revisiting file locking In-Reply-To: <13647.26901.437384.156650@anthem.CNRI.Reston.Va.US>; from Barry A. Warsaw on Tue, May 05, 1998 at 03:45:47PM -0400 References: <199805051928.PAA14766@glyph.CNRI.Reston.Va.US> <13647.26901.437384.156650@anthem.CNRI.Reston.Va.US> Message-ID: <19980612122153.A5925@list.org> On Tue, May 05, 1998 at 03:45:47PM -0400, Barry A. Warsaw wrote: > > I wasn't able to dig up the Netscape Mail + File Locking article that > Jamie Zawinski wrote. I don't seem to have it in my bookmarks or > easily grepable and a web search turned up only stale links. I > emailed Jaime to see if he's got an updated link and will forward that > if I get a response. Hmm, I got it on a web search: http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/movemail.html > If I remember correctly, the article basically justifies the point of > view that the best portable way to do resource locking across NFS is > by creating a file with O_CREAT | O_EXCL. That's what's meant by > "dot-file" locking because you usually create a .file that is mostly > invisible. I have yet to find much on whether there are race conditions w/ O_EXCL and directories. However, the open() man page on my system has the following to say: O_EXCL When used with O_CREAT, if the file already exists it is an error and the open will fail. O_EXCL is broken on NFS file systems, programs which rely on it for performing locking tasks will contain a race condition. The solution for performing atomic file locking using a lockfile is to create a unique file on the same fs (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile and use stat(2) on the unique file to check if its link count has increased to 2. Do not use the return value of the link() call. The problem with this solution is, what happens if a process dies while holding a lock? The most straightforward solution is to drop a pid into the lock file, and have the process wishing to grab the lock check to see if the process is still running. Also, this is a polling-based solution, which isn't incredibly desirable either... But I guess there's no way around that one. Ok, I'll implement a general library call for this one, I guess. BTW, No one has put any effort into making pipermail portable yet, have they? John From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 21:34:38 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 16:34:38 -0400 (EDT) Subject: [Mailman-Developers] Re: file permissions References: <199806051632.MAA09671@fermi.eeel.nist.gov> Message-ID: <13697.36475.781015.885594@anthem.CNRI.Reston.Va.US> Normal Friday catchup... >>>>> "MM" == Michael McLay writes: MM> Either I screwed up when I followed the installation MM> instructions or else the file ownership set by "make install" MM> incorrectly set the group of the executables in MM> $prefix/cgi-bin to that of the super user. In order for the MM> scripts to work the group had to be manually set to "mailman". One thing the next version will have is more checking of the permissions on $prefix. If the permissions or group ownership aren't correct, configure stops. This should prevent those problems from cascading to all the other installed files. MM> This change should also reduce the script execution time MM> slightly because the longer script will be compiled to a .pyc MM> file. That may or many not really be an issue. I suspect starting up the Python interpreter for every script is the killer, rather than parsing the script files. subscribe for example is less than 200 lines of Python. Admin is certainly the longest at 856 lines. One thing that's high on my list is to really fix the packagization of Mailman. I think given that, many things get simpler. You still need a little path hacking, but that can be done in a more portable way than even the paths.py files. And then most of the functionality in the scripts can be moved into package libraries. This will improve the modularity A LOT. I just finished doing this with a project here at CNRI and it makes a big difference. MM> John, what do you think of removing the mm_ prefix from the MM> module names? This can easily happen, with no chance of name collisions, even for the mixin classes. I also want to rename some of the .py files to match the contained class names (better style). So there are several things of this nature I want to work on, but my meta question is this: From a technical standpoint, how close to a 1.0 release are we? What critical features are still missing (integration with Pipermail is key), and what showstopper bugs are still lurking? Should I continue to do code clean up now, or wait until after 1.0? -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 21:35:48 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 16:35:48 -0400 (EDT) Subject: [Mailman-Developers] file permissions References: <199806051632.MAA09671@fermi.eeel.nist.gov> <13688.9300.328001.568060@fermi.eeel.nist.gov> Message-ID: <13697.37108.420182.964834@anthem.CNRI.Reston.Va.US> >>>>> "MM" == Michael McLay writes: MM> I've done a little more work on generalizing the wrapper and MM> adding some more optional debug information. The new module MM> Mailman.debug contains two functions. BTW, I *really* like this idea, Mike. I like the traceback showing up on the Web page, especially at this stage in Mailman's life. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 21:36:45 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 16:36:45 -0400 (EDT) Subject: [Mailman-Developers] file permissions References: <199806051632.MAA09671@fermi.eeel.nist.gov> <19980606055629.A12273@list.org> Message-ID: <13697.37178.321774.675303@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> One thing I'm thinking is that setenv() has portability JV> problems, and it may be better not to use it at all. Probably wise. JV> I think it's fine for the prefix to go away, although I would JV> probably either like to see the files that are used as mixins JV> to retain a prefix, or see a better directory structure, A better directory structure!!!! :-) -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 23:05:26 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 18:05:26 -0400 (EDT) Subject: [Mailman-Developers] Re: [JPython] rexec module, string escapes and file ASCII<->Unicode References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> Message-ID: <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> >>>>> "JH" == Jim Hugunin writes on the >>>>> jpython-interest list: JH> Everybody on this list should know that the mailman software - JH> which handles the jpython-interest mailing list - is JH> reasonably agressive in suspending mail to any addresses that JH> produce excessive bounce messages. This feature makes the JH> life of the list administrator much more pleasant, but can JH> occasionally lead to people having their mail suspended who JH> still want to be on the list. I've been thinking about bounce policy. I wonder if what Mailman should do is have the same policy for all bounces, regardless of whether it looks permanent (user unknown) or not. Once the bounce threshold is reached and the account is disabled, a cron job could be used to retry the address for some number of days. If the account becomes live again, it can be re-enabled. Or perhaps the user has to explicitly re-enable it when they finally see these warning messages. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Fri Jun 12 23:09:51 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 12 Jun 1998 18:09:51 -0400 (EDT) Subject: [Mailman-Developers] smtplib buglet in 1.0b4 References: <19980611193525.15573@chronis.icgroup.com> Message-ID: <13697.42718.170852.188059@anthem.CNRI.Reston.Va.US> >>>>> "TDDM" == The Dragon De Monsyne writes: TDDM> PS, is there any specific reason that MailMan keeps TDDM> using it's own smtplib, rather than the one that is part of TDDM> the Python 1.5.1 standard library? Whatever happens, it would be really nice if a single integrated smtplib.py made its way back into the Python distribution. Mailman will still have to distribute a copy for the time being, but I can work out a distrib tree to handle that (as I'll have to do anyway with Cookie.py). -Barry From viega@list.org Sat Jun 13 03:12:12 1998 From: viega@list.org (John Viega) Date: Fri, 12 Jun 1998 19:12:12 -0700 Subject: [Mailman-Developers] Re: bounce agressiveness In-Reply-To: <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US>; from Barry A. Warsaw on Fri, Jun 12, 1998 at 06:05:26PM -0400 References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> Message-ID: <19980612191212.D9003@list.org> On Fri, Jun 12, 1998 at 06:05:26PM -0400, Barry A. Warsaw wrote: > > >>>>> "JH" == Jim Hugunin writes on the > >>>>> jpython-interest list: > > JH> Everybody on this list should know that the mailman software - > JH> which handles the jpython-interest mailing list - is > JH> reasonably agressive in suspending mail to any addresses that > JH> produce excessive bounce messages. This feature makes the > JH> life of the list administrator much more pleasant, but can > JH> occasionally lead to people having their mail suspended who > JH> still want to be on the list. > > I've been thinking about bounce policy. I wonder if what Mailman > should do is have the same policy for all bounces, regardless of > whether it looks permanent (user unknown) or not. Once the bounce > threshold is reached and the account is disabled, a cron job could be > used to retry the address for some number of days. If the account > becomes live again, it can be re-enabled. Or perhaps the user has to > explicitly re-enable it when they finally see these warning messages. Sure, but then again I've never seen an error message as a list admin for a non-transient error that was actually transient (ie, 'no such user' only lasting a few minutes). It really doesn't matter, might as well be safe about it. I think the real problem is addresses that can't be reached for a little too long for whatever reason. I do like the idea of sending daily mail for actually an extended period of time. Maybe it could be every 3rd day for 60 days, since mail generally doesn't get rejected as undeliverable for 3 days. We could use the confirmation mechanism to let people get back on the list. By the way, does the confirmation mechanism clean up unanswered confirmation mail? How long does it sit there if so? If not, we probably need to add that :) John From viega@list.org Sat Jun 13 15:46:51 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 07:46:51 -0700 Subject: [Mailman-Developers] File Locking Message-ID: <19980613074651.A13995@list.org> --QCug/ao/Qh+gIAGv Content-Type: text/plain; charset=us-ascii I'm going to attach what I hope is a portable module for file locking. I don't have time to integrate it with Mailman today, unfortunately, but if someone wants to try it out, or give suggestions for improvements to this code, let me know. John --QCug/ao/Qh+gIAGv Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="flock.py" # Copyright (C) 1998 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. # # flock.py: Portable file locking. John Viega, Jun 13, 1998 """Portable (?) file locking with timeouts. This code should work with all versions of NFS. The algorithm was suggested by the GNU/Linux open() man page. Make sure no malicious people have access to link() to the lock file. """ # Potential change: let the locker insert a field saying when he promises # to be done with the lock, so if he needs more time than the other # processes think he needs, he can say so. import socket, os, time DEFAULT_HUNG_TIMEOUT = 30 DEFAULT_SLEEP_INTERVAL = .25 AlreadyCalledLockError = "AlreadyCalledLockError" NotLockedError = "NotLockedError" TimeOutError = "TimeOutError" class FileLock: def __init__(self, lockfile, hung_timeout = DEFAULT_HUNG_TIMEOUT, sleep_interval = DEFAULT_SLEEP_INTERVAL): self.lockfile = lockfile self.hung_timeout = hung_timeout self.sleep_interval = sleep_interval self.tmpfname = "%s.%s.%d" % (lockfile, socket.gethostname(), os.getpid()) self.locked = 0 if not os.path.exists(self.lockfile): try: file = open(self.lockfile, "w+") except IOError: pass # Note that no one new can grab the lock once we've opened our # tmpfile until we close it, even if we don't have the lock. So # checking the PID and stealing the lock are garunteed to be atomic. def lock(self, timeout = 0): """Blocks until the lock can be obtained. Raises a TimeOutError exception if a positive timeout value is given and that time elapses before the lock is obtained. """ if timeout > 0: timeout_time = time.time() + timeout last_pid = -1 if self.locked: raise AlreadyCalledLockError while 1: os.link(self.lockfile, self.tmpfname) if os.stat(self.tmpfname)[3] == 2: file = open(self.tmpfname, 'w+') file.write(`os.getpid(),self.tmpfname`) file.close() self.locked = 1 break if timeout and timeout_time < time.time(): raise TimeOutError file = open(self.tmpfname, 'r') try: pid,winner = eval(file.read()) except SyntaxError: # no info in file... *can* happen file.close() continue file.close() if pid <> last_pid: last_pid = pid stime = time.time() if (stime + self.hung_timeout < time.time()) and self.hung_timeout > 0: file = open(self.tmpfname, 'w+') file.write(`os.getpid(),self.tmpfname`) os.unlink(winner) self.locked = 1 break os.unlink(self.tmpfname) time.sleep(self.sleep_interval) # This could error if the lock is stolen. You must catch it. def unlock(self): if not self.locked(): raise NotLockedError os.unlink(self.tmpfname) def locked(self): return os.path.exists(self.tmpfname) and self.locked --QCug/ao/Qh+gIAGv-- From viega@list.org Sat Jun 13 16:13:03 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 08:13:03 -0700 Subject: [Mailman-Developers] Re: file permissions In-Reply-To: <13697.36475.781015.885594@anthem.CNRI.Reston.Va.US>; from Barry A. Warsaw on Fri, Jun 12, 1998 at 04:34:38PM -0400 References: <199806051632.MAA09671@fermi.eeel.nist.gov> <13697.36475.781015.885594@anthem.CNRI.Reston.Va.US> Message-ID: <19980613081303.A14039@list.org> On Fri, Jun 12, 1998 at 04:34:38PM -0400, Barry A. Warsaw wrote: > > > >From a technical standpoint, how close to a 1.0 release are we? What > critical features are still missing (integration with Pipermail is > key), and what showstopper bugs are still lurking? Should I continue > to do code clean up now, or wait until after 1.0? I anticipate a few more beta releases first. No one is jumping up to do the Pipermail stuff in a portable way, so that's going to delay us time wise. Locking needs to be improved a bit; it would be nice if the granularity were better. I think there's a need for more documentation before we leave beta. All password-pages should use the cookie code, AND have the extra password field for cookie-less people, which currently isn't done at all. I'd also like to move all web pages and mail messages that should be editable out into the template dirs. That's less important, though. We also need to go through and make sure all email addresses that can be entered can't be used in an insecure manner (check for |'s etc. in every address, entered from anywhere). Also, not only would I like to see archiving back in, I also want to see it with a search engine, by default. As for bugs, there are some on the Mailman TODO page that may or may not be current. Nothing major, though. There's one big one that's been reported and I haven't found yet. I'll send it on in another email, perhaps someone else can find it before I do. That's what I'd like to see. And it really isn't all that much, but I'll bet it's at least a month off. Plus, we should try to have a stable beta for a few weeks before we release it as 1.0. Oh, beyond that, we need to make sure we get all permissionsissues squared away. Pipermail's a good example of that one. Maybe Andrew should try to get it blessed as the GNU mail archiver before we bundle it? :) John From viega@list.org Sat Jun 13 16:15:49 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 08:15:49 -0700 Subject: [Mailman-Developers] mass subscribe bug Message-ID: <19980613081549.B14039@list.org> Here's the error previously mentioned, as reported to me, w/ excess headers, etc. trimmed. John ----- Forwarded message from Matt Chatterley ----- Occurs when trying to mass subscribe. Seems to have started after upgrading python, version information: Python 1.5.1 (#1, May 29 1998, 12:50:41) [GCC 2.7.2.3] on linux2 We're sorry, we hit a bug! If you would like to help us identify the problem, please email a copy of this page to the webmaster for this site with a description of what happened. Thanks! Traceback (innermost last): File "/home/mailman/scripts/admin", line 823, in ? main() File "/home/mailman/scripts/admin", line 157, in main is_auth = isAuthenticated(lst) File "/home/mailman/scripts/admin", line 103, in isAuthenticated inp = base64.decodestring(c[list_name + "-admin"].value) File "/usr/lib/python1.5/base64.py", line 46, in decodestring decode(f, g) File "/usr/lib/python1.5/base64.py", line 32, in decode s = binascii.a2b_base64(line) Error: Incorrect padding From viega@list.org Sat Jun 13 16:25:28 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 08:25:28 -0700 Subject: [Mailman-Developers] Archiving? Message-ID: <19980613082528.A14242@list.org> I got several messages dated June 12 from Barry yesterday, but none of them showed up in the pipermail archive for this list. Anyone guess what's up? John From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Sat Jun 13 16:48:05 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Sat, 13 Jun 1998 11:48:05 -0400 (EDT) Subject: [Mailman-Developers] Re: bounce agressiveness References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> <19980612191212.D9003@list.org> Message-ID: <13698.40646.976228.287080@anthem.CNRI.Reston.Va.US> >>>>> "JV" == John Viega writes: JV> Sure, but then again I've never seen an error message as a JV> list admin for a non-transient error that was actually JV> transient (ie, 'no such user' only lasting a few minutes). It JV> really doesn't matter, might as well be safe about it. Right. Guido had a good point which was that it might simplify the code such that all bounces are treated the same. Treating permanent failures like transient failures with retries doesn't seem like it could hurt. JV> I do like the idea of sending daily mail for actually an JV> extended period of time. Maybe it could be every 3rd day for JV> 60 days, since mail generally doesn't get rejected as JV> undeliverable for 3 days. We could use the confirmation JV> mechanism to let people get back on the list. Exactly! JV> By the way, does the confirmation mechanism clean up JV> unanswered confirmation mail? How long does it sit there if JV> so? If not, we probably need to add that :) I don't know, but I agree! -Barry From scott@chronis.pobox.com Sat Jun 13 22:22:25 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 13 Jun 1998 17:22:25 -0400 Subject: [Mailman-Developers] mass subscribe bug Message-ID: <19980613172225.39336@chronis.icgroup.com> I saw this error once in another context (still with mailman and cookies, but with the "private" cgi). The error occurred when a recent version of IE on a mac attempted to deal with the cookie. I think that the bug has one of two causes: either the browser is incorrectly handling the characters in the cookie, or there is something about the way that the base64 module encodes and decodes that won't work with http. Since I've only seen the error on what i consider a questionable browser i suspect that it's the browser's fault. in either case, i haven't the first clue about a fix. scott On Sat, Jun 13, 1998 at 08:15:49AM -0700, John Viega wrote: | Here's the error previously mentioned, as reported to me, w/ excess | headers, etc. trimmed. | | John | | ----- Forwarded message from Matt Chatterley ----- | | Occurs when trying to mass subscribe. | | Seems to have started after upgrading python, version information: | | Python 1.5.1 (#1, May 29 1998, 12:50:41) [GCC 2.7.2.3] on linux2 | | We're sorry, we hit a bug! | | If you would like to help us identify the problem, please email a copy | of this page to the webmaster for this site with a description of what | happened. Thanks! | | Traceback (innermost last): | File "/home/mailman/scripts/admin", line 823, in ? | main() | File "/home/mailman/scripts/admin", line 157, in main | is_auth = isAuthenticated(lst) | File "/home/mailman/scripts/admin", line 103, in isAuthenticated | inp = base64.decodestring(c[list_name + "-admin"].value) | File "/usr/lib/python1.5/base64.py", line 46, in decodestring | decode(f, g) | File "/usr/lib/python1.5/base64.py", line 32, in decode | s = binascii.a2b_base64(line) | Error: Incorrect padding | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers -----End of forwarded message----- From Corbett J. Klempay" Hey, I just saw the comment about integrating archive search functionality in 1.0 or another not-so-distant release. I was thinking that maybe I could use this as a chance to learn Python; I just finished taking Info Retrieval & Text Understanding this past semester, and ended up doing a vector search engine for the final project. I'm not saying that I can for sure do this; I think I might see if I can find some time to screw around with it, and if it develops ok, then bonus. Some questions, though: - how big of an archive should this be scalable to? I'm thinking this because some models (like the vector model used for my project) get good accuracy, but suck as far as resource usage (like our engine dealt with ~2000 text documents from an online database and would suck up ~80 MB of RAM per query, and would take almost 20 seconds just to load the pre-indexed document clusters from disk; the query itself only takes like 1-2 seconds on a K6-233, but the startup time of 15+ seconds blows). With large corpa, it might be necessary to implement some persistence; it just takes so annoyingly long to load even pre-indexed stuff from disk (and our queries were on a K6-233 with 128 MB; heheh...think how a P-90 with 32 MB would fare :) (or worse yet!) (for the really curious people, the engine is at http://www2.acm.jhu.edu/projects/calliope) - What kind of structure does Pipermail store the archive in? Maybe I should contact someone (maybe whoever runs the lists at python.org?) who runs a larger Pipermail archive and see about getting an archive of a treefull of archived articles - Did you have any idea about what kind of search interface? Were you thinking a text field with Boolean capability, or just letting them throw some words in the field and see what sticks? - Were you thinking of an engine that has an indexing process that runs via a cron job or something, or something much simpler? (like one that just brute force searches through the text of the archives for each query; that would be slow as hell if the archive was large, but would take no additional disk space and wouldn't really require persistence). ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "Economists are still trying to figure out why the girls with the least principle draw the most interest." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From viega@list.org Sat Jun 13 23:38:13 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 15:38:13 -0700 Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: <19980613172225.39336@chronis.icgroup.com>; from Scott on Sat, Jun 13, 1998 at 05:22:25PM -0400 References: <19980613172225.39336@chronis.icgroup.com> Message-ID: <19980613153813.B15795@list.org> On Sat, Jun 13, 1998 at 05:22:25PM -0400, Scott wrote: > I saw this error once in another context (still with mailman and > cookies, but with the "private" cgi). Hmm, I've heard of this error from two different people now, and one of them mentioned it was transient. Sometimes it happened... I don't think it invvolved IE on a mac. > cookie, or there is something about the way that the base64 module > encodes and decodes that won't work with http. Since I've only seen > the error on what i consider a questionable browser i suspect that > it's the browser's fault. Why are you using base64 encoding, anyway? If it's a "privacy" matter, anyone who reads your cookie can forge the same cookie, no? If so, might as well just use plaintext if it's going to be an eisenbug this way. If anyone can reproduce this bug reliably, please let me know, I'd like to work with you to figure out what exactly the problem is. John From viega@list.org Sat Jun 13 23:50:53 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 15:50:53 -0700 Subject: [Mailman-Developers] Idea In-Reply-To: ; from Corbett J. Klempay on Sat, Jun 13, 1998 at 05:55:41PM -0400 References: Message-ID: <19980613155053.C15795@list.org> On Sat, Jun 13, 1998 at 05:55:41PM -0400, Corbett J. Klempay wrote: > > - how big of an archive should this be scalable to? I'd say as big as possible. I don't know if I can give a better answer to that one. Target the most heavily trafficed mailing list you've been on. > I'm thinking this because some models (like the vector model used for my > project) get good accuracy, but suck as far as resource usage (like our > engine dealt with ~2000 text documents from an online database and would > suck up ~80 MB of RAM per query, and would take almost 20 seconds just to > load the pre-indexed document clusters from disk; the query itself only > takes like 1-2 seconds on a K6-233, but the startup time of 15+ seconds > blows). With large corpa, it might be necessary to implement some > persistence; it just takes so annoyingly long to load even pre-indexed > stuff from disk (and our queries were on a K6-233 with 128 MB; > heheh...think how a P-90 with 32 MB would fare :) (or worse yet!) Well, you have several options. You could keep a persistant server up, but I wouldn't make it a requirement, perhaps an option. I think that if complex search capabilities aren't desired, the grep libraries would be an OK first pass. > - What kind of structure does Pipermail store the archive in? This is the biggest problem right now. Andrew says you can plug in any sort of back-end you want for a database, as long as it can handle a tree-type structure. Unfortunately, the only such backend implemented is not portable. Everything needs to work out of the box w/ a vanilla Python installation. I was thinking that someone could write a backend that uses the file system for that tree structure... > - Did you have any idea about what kind of search interface? Were you > thinking a text field with Boolean capability, or just letting them throw > some words in the field and see what sticks? Well, for a first pass, something simple will do, but the nicer you can make it, the better off we'll be... > - Were you thinking of an engine that has an indexing process that runs > via a cron job or something, or something much simpler? (like one that > just brute force searches through the text of the archives for each query; > that would be slow as hell if the archive was large, but would take no > additional disk space and wouldn't really require persistence). It'd be nice to have a per-list setting for this one. For most lists, something simple would do, but I run a few where indexing would certainly be better... John From scott@chronis.pobox.com Sun Jun 14 01:04:35 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 13 Jun 1998 20:04:35 -0400 Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: <19980613153813.B15795@list.org>; from John Viega on Sat, Jun 13, 1998 at 03:38:13PM -0700 References: <19980613172225.39336@chronis.icgroup.com> <19980613153813.B15795@list.org> Message-ID: <19980613200435.35995@chronis.icgroup.com> On Sat, Jun 13, 1998 at 03:38:13PM -0700, John Viega wrote: | On Sat, Jun 13, 1998 at 05:22:25PM -0400, Scott wrote: | > I saw this error once in another context (still with mailman and | > cookies, but with the "private" cgi). | | Hmm, I've heard of this error from two different people now, and one | of them mentioned it was transient. Sometimes it happened... I don't | think it invvolved IE on a mac. | | > cookie, or there is something about the way that the base64 module | > encodes and decodes that won't work with http. Since I've only seen | > the error on what i consider a questionable browser i suspect that | > it's the browser's fault. | | Why are you using base64 encoding, anyway? If it's a "privacy" | matter, anyone who reads your cookie can forge the same cookie, no? | If so, might as well just use plaintext if it's going to be an | eisenbug this way. i was following the example in the "private" script. i don't think base64 is necessary, but would think that some kind of encoding makes it safer -- if only because it looks random. md5 is another option for that, but it produces lots of characters that need special http escaping. i'm not sure whether or not base64 has the same problem. perhaps plaintext or a simple hash is in order? scott From viega@list.org Sun Jun 14 01:54:12 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 17:54:12 -0700 Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: <19980613200435.35995@chronis.icgroup.com>; from Scott on Sat, Jun 13, 1998 at 08:04:35PM -0400 References: <19980613172225.39336@chronis.icgroup.com> <19980613153813.B15795@list.org> <19980613200435.35995@chronis.icgroup.com> Message-ID: <19980613175412.A17363@list.org> On Sat, Jun 13, 1998 at 08:04:35PM -0400, Scott wrote: > > i was following the example in the "private" script. i don't think > base64 is necessary, but would think that some kind of encoding makes > it safer -- if only because it looks random. md5 is another option > for that, but it produces lots of characters that need special http > escaping. i'm not sure whether or not base64 has the same problem. > perhaps plaintext or a simple hash is in order? base64's purpose is essentially the same as uuencode; to take binary streams and encode them into 7 bit characters. I'd say as long as there as these weird bugs, let's back off and go to plaintext... John From dragondm@nexus.Integral.org Sun Jun 14 05:41:16 1998 From: dragondm@nexus.Integral.org (The Dragon De Monsyne) Date: Sat, 13 Jun 1998 23:41:16 -0500 (CDT) Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: <19980613175412.A17363@list.org> Message-ID: On Sat, 13 Jun 1998, John Viega wrote: > On Sat, Jun 13, 1998 at 08:04:35PM -0400, Scott wrote: > > > > i was following the example in the "private" script. i don't think > > base64 is necessary, but would think that some kind of encoding makes > > it safer -- if only because it looks random. md5 is another option > > for that, but it produces lots of characters that need special http > > escaping. i'm not sure whether or not base64 has the same problem. > > perhaps plaintext or a simple hash is in order? > > base64's purpose is essentially the same as uuencode; to take binary > streams and encode them into 7 bit characters. > > I'd say as long as there as these weird bugs, let's back off and go to > plaintext... I'm afraid I know little of cookies, but if it's a problem of http escaping, I DO know that base64 will produce charecters that will need escaping. There is a module for the apache webserver that generates unique ID's for hits, and it encodes the binary ID using a _modified_ version of base64-encoding for this reason. To quote: "The UNIQUE_ID environment variable is constructed by encoding the 112-bit (value) ... using the alphabet [A-Za-z0-9@-] in a manner similar to MIME base64 encoding, producing 19 characters. The MIME base64 alphabet is actually [A-Za-z0-9+/] however + and / need to be specially encoded in URLs, which makes them less desirable. " ....so, if cookies follow the same rules as URL's for escaping, then indeed that could be the trouble. > -The Dragon De Monsyne From viega@list.org Sun Jun 14 06:36:49 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 22:36:49 -0700 Subject: [Mailman-Developers] flock again Message-ID: <19980613223649.A21606@list.org> --41hnqqNCTbUpUBcO Content-Type: text/plain; charset=us-ascii The flock.py I posted had a subtle name collision problem that reminds me how much I like static type checking. Attached is a fix. I've now hooked it up to the main source tree, and have integrated Mike McLay's wrapping stuff. John --41hnqqNCTbUpUBcO Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="flock.py" # Copyright (C) 1998 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. # # flock.py: Portable file locking. John Viega, Jun 13, 1998 """Portable (?) file locking with timeouts. This code should work with all versions of NFS. The algorithm was suggested by the GNU/Linux open() man page. Make sure no malicious people have access to link() to the lock file. """ # Potential change: let the locker insert a field saying when he promises # to be done with the lock, so if he needs more time than the other # processes think he needs, he can say so. import socket, os, time DEFAULT_HUNG_TIMEOUT = 15 DEFAULT_SLEEP_INTERVAL = .25 AlreadyCalledLockError = "AlreadyCalledLockError" NotLockedError = "NotLockedError" TimeOutError = "TimeOutError" class FileLock: def __init__(self, lockfile, hung_timeout = DEFAULT_HUNG_TIMEOUT, sleep_interval = DEFAULT_SLEEP_INTERVAL): self.lockfile = lockfile self.hung_timeout = hung_timeout self.sleep_interval = sleep_interval self.tmpfname = "%s.%s.%d" % (lockfile, socket.gethostname(), os.getpid()) self.is_locked = 0 if not os.path.exists(self.lockfile): try: file = open(self.lockfile, "w+") except IOError: pass # Note that no one new can grab the lock once we've opened our # tmpfile until we close it, even if we don't have the lock. So # checking the PID and stealing the lock are garunteed to be atomic. def lock(self, timeout = 0): """Blocks until the lock can be obtained. Raises a TimeOutError exception if a positive timeout value is given and that time elapses before the lock is obtained. """ if timeout > 0: timeout_time = time.time() + timeout last_pid = -1 if self.locked(): raise AlreadyCalledLockError while 1: os.link(self.lockfile, self.tmpfname) if os.stat(self.tmpfname)[3] == 2: file = open(self.tmpfname, 'w+') file.write(`os.getpid(),self.tmpfname`) file.close() self.is_locked = 1 break if timeout and timeout_time < time.time(): raise TimeOutError file = open(self.tmpfname, 'r') try: pid,winner = eval(file.read()) except SyntaxError: # no info in file... *can* happen file.close() continue file.close() if pid <> last_pid: last_pid = pid stime = time.time() if (stime + self.hung_timeout < time.time()) and self.hung_timeout > 0: file = open(self.tmpfname, 'w+') file.write(`os.getpid(),self.tmpfname`) os.unlink(winner) self.is_locked = 1 break os.unlink(self.tmpfname) time.sleep(self.sleep_interval) # This could error if the lock is stolen. You must catch it. def unlock(self): if not self.locked(): raise NotLockedError self.is_locked = 0 os.unlink(self.tmpfname) def locked(self): return os.path.exists(self.tmpfname) and self.is_locked --41hnqqNCTbUpUBcO-- From viega@list.org Sun Jun 14 06:43:50 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 22:43:50 -0700 Subject: [Mailman-Developers] mass subscribe bug In-Reply-To: ; from The Dragon De Monsyne on Sat, Jun 13, 1998 at 11:41:16PM -0500 References: <19980613175412.A17363@list.org> Message-ID: <19980613224350.B21606@list.org> On Sat, Jun 13, 1998 at 11:41:16PM -0500, The Dragon De Monsyne wrote: > > I'm afraid I know little of cookies, but if it's a problem of > http escaping, I DO know that base64 will produce charecters that will > need escaping. There is a module for the apache webserver that generates > unique ID's for hits, and it encodes the binary ID using a _modified_ > version of base64-encoding for this reason. Well, cookies are handled in mime headers. So they don't have the same escaping problems as html. However, I think you did hit the problem. It occurs to me that ';' has special meaning on the mime line. So we'd still have this problem w/ plaintext when people have a ';' in their password. My guess is that things are blowing up when the base64 outputs a semicolon. IIRC, the only special chars to the right of the colon should be the semi-colon, and the end of line. But, we should find out for sure, and guard against it... John From cklempay@chimera.acm.jhu.edu Sun Jun 14 07:33:14 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Sun, 14 Jun 1998 02:33:14 -0400 (EDT) Subject: [Mailman-Developers] Idea In-Reply-To: <19980613155053.C15795@list.org> Message-ID: On Sat, 13 Jun 1998, John Viega wrote: > On Sat, Jun 13, 1998 at 05:55:41PM -0400, Corbett J. Klempay wrote: > > > > - how big of an archive should this be scalable to? > > I'd say as big as possible. I don't know if I can give a better > answer to that one. Target the most heavily trafficed mailing list > you've been on. > Hmmm...any good ballpark figures for this? I'm thinking that BUGTRAQ is maybe one of the heavier I'm on, or perhaps the AfterStep or CVS lists. So let's say ~10 messages a day, so ~70/week, ~300 month, 3000-4000 year. That's a lot of messages :) I wonder how large of a time span they should be able to search... > > - What kind of structure does Pipermail store the archive in? > > This is the biggest problem right now. Andrew says you can plug in > any sort of back-end you want for a database, as long as it can handle > a tree-type structure. Unfortunately, the only such backend > implemented is not portable. Everything needs to work out of the box > w/ a vanilla Python installation. I was thinking that someone could > write a backend that uses the file system for that tree structure... > Ugh. (the non-portable current state of things) What is it using now? CK From viega@list.org Sun Jun 14 07:46:48 1998 From: viega@list.org (John Viega) Date: Sat, 13 Jun 1998 23:46:48 -0700 Subject: [Mailman-Developers] Idea In-Reply-To: ; from Corbett J. Klempay on Sun, Jun 14, 1998 at 02:33:14AM -0400 References: <19980613155053.C15795@list.org> Message-ID: <19980613234648.A22154@list.org> On Sun, Jun 14, 1998 at 02:33:14AM -0400, Corbett J. Klempay wrote: > > Hmmm...any good ballpark figures for this? I'm thinking that BUGTRAQ is > maybe one of the heavier I'm on, or perhaps the AfterStep or CVS lists. > So let's say ~10 messages a day, so ~70/week, ~300 month, 3000-4000 year. > That's a lot of messages :) I wonder how large of a time span they should > be able to search... I've seen some lists that averaged twice that, probably. Figure on there being some bigger lists, but then again, no one wants to get TOO many messages a day. > > > - What kind of structure does Pipermail store the archive in? > > > > This is the biggest problem right now. Andrew says you can plug in > > any sort of back-end you want for a database, as long as it can handle > > a tree-type structure. Unfortunately, the only such backend > > implemented is not portable. Everything needs to work out of the box > > w/ a vanilla Python installation. I was thinking that someone could > > write a backend that uses the file system for that tree structure... > > > > Ugh. (the non-portable current state of things) What is it using now? I think bsddb or something like that. My biggest concern is having Mailman work "out of the box" for everybody. People needing to to install bsddb isn't acceptable... John From janne@avocado.pc.helsinki.fi Mon Jun 15 09:50:25 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 15 Jun 1998 11:50:25 +0300 Subject: [Mailman-Developers] Re: bounce agressiveness In-Reply-To: John Viega's message of "Fri, 12 Jun 1998 19:12:12 -0700" References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> <19980612191212.D9003@list.org> Message-ID: John Viega writes: > I do like the idea of sending daily mail for actually an extended > period of time. Maybe it could be every 3rd day for 60 days, since > mail generally doesn't get rejected as undeliverable for 3 days. We > could use the confirmation mechanism to let people get back on the > list. Or by an exponential schedule, say after 2, 4, 8, 16, 32, and 64 days. Why? Well, less trying, and because recovery of an address becomes more and more improbable when time passes (most network problems are short in duration). -- Janne From akuchlin@cnri.reston.va.us Mon Jun 15 16:00:31 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Mon, 15 Jun 1998 11:00:31 -0400 (EDT) Subject: [Mailman-Developers] Idea In-Reply-To: References: Message-ID: <13701.13769.454247.591783@newcnri.cnri.reston.va.us> Corbett J. Klempay writes: >- What kind of structure does Pipermail store the archive in? Maybe I >should contact someone (maybe whoever runs the lists at python.org?) who >runs a larger Pipermail archive and see about getting an archive of a >treefull of archived articles Messages are just HTML files, but there are on-disk B-trees (using the bsddb module) used for the indexes. It should not be very difficult to use Aaron's Bplustree.py module instead; the only problem is me finding time to do it. Hypermail development has picked up again, so I've been speculating about writing a Python interface to Hypermail's indexing infrastructure. That would let Python code control policies (where a message goes, how it's formatted, ...) but leave the data handling to C code. No time to look into that, unfortunately. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ In its broadest ecological context, economic development is the development of more intensive ways of exploiting the natural environment. -- Richard Wilkinson From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Mon Jun 15 16:23:03 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 15 Jun 1998 11:23:03 -0400 (EDT) Subject: [Mailman-Developers] Re: bounce agressiveness References: <357BFC45.8247F08E@cnri.reston.va.us> <357C1927.E117B2C2@cnri.reston.va.us> <13697.42335.853813.513556@anthem.CNRI.Reston.Va.US> <19980612191212.D9003@list.org> Message-ID: <13701.15408.361965.318557@anthem.CNRI.Reston.Va.US> >>>>> "JS" == Janne Sinkkonen writes: JS> Or by an exponential schedule, say after 2, 4, 8, 16, 32, and JS> 64 days. Why? Well, less trying, and because recovery of an JS> address becomes more and more improbable when time passes JS> (most network problems are short in duration). I wouldn't extend it so far though. I would guess 8 or 10 days max would be more than enough. -Barry From janne@avocado.pc.helsinki.fi Mon Jun 15 17:12:32 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 15 Jun 1998 19:12:32 +0300 Subject: [Mailman-Developers] Bounce processing and remote list administrators Message-ID: Well, I'm not sure it's quite as smooth as it could be. Our network connection tends to go down for the weekend because of a broken router needing (ir)relegular reboots. This happened on Sunday once again, and this time the remote listowner got over 100MB worth of error messages. His ISP will filter the trash out of his mailbox for tomorrow, hopefully. Furthermore, about 150 of roughly 800 members were kicked out of the list because of 'excessive or fatal bounces'. The network was down for about 24 hours. (Our bounce parameters may be too tight, of course.) I haven't seen the variety of the generated messages yet. Even if this were 'normal' in the sense that there are no message-generating loops, some kind of solution is needed to prevent excessive or fatal mail bombing of the listowners. -- Janne From viega@list.org Tue Jun 16 13:44:54 1998 From: viega@list.org (John Viega) Date: Tue, 16 Jun 1998 05:44:54 -0700 Subject: [Mailman-Developers] Bounce processing and remote list administrators In-Reply-To: ; from Janne Sinkkonen on Mon, Jun 15, 1998 at 07:12:32PM +0300 References: Message-ID: <19980616054454.C8485@list.org> On Mon, Jun 15, 1998 at 07:12:32PM +0300, Janne Sinkkonen wrote: > > Well, I'm not sure it's quite as smooth as it could be. > > Our network connection tends to go down for the weekend because of a > broken router needing (ir)relegular reboots. This happened on Sunday > once again, and this time the remote listowner got over 100MB worth of > error messages. His ISP will filter the trash out of his mailbox for > tomorrow, hopefully. > > Furthermore, about 150 of roughly 800 members were kicked out of the > list because of 'excessive or fatal bounces'. The network was down > for about 24 hours. (Our bounce parameters may be too tight, of > course.) > > I haven't seen the variety of the generated messages yet. > > Even if this were 'normal' in the sense that there are no > message-generating loops, some kind of solution is needed to prevent > excessive or fatal mail bombing of the listowners. > Wow. Is the field, "Minimum number of days an address has been non-fatally bad before we take action" set to 1 or 0 for that list? If not, either something got generated that was treated as a fatal bounce, or there's a bug in the bounce handling. Neither would surprise me too much. Like Barry said, though, we should switch to treating all bounces as non-fatal; it will simplify bounce processing a bit. John From cklempay@chimera.acm.jhu.edu Wed Jun 17 06:10:27 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Wed, 17 Jun 1998 01:10:27 -0400 (EDT) Subject: [Mailman-Developers] Mailman + Qmail 1.01 Message-ID: Anyone in here done this / know if it works? I even tried installing the fastforward package for Qmail to give it /etc/aliases support...but it's being really damn annoying (i.e. in its current state, mail sent to an alias from /etc/aliases is getting swallowed into nowhere; it never shows up on the Qmail machine (at least not in the Mailbox), and doesn't bounce either...) I forget who it was who was running it, but on a related note...anyone know how VMailer is coming along? I was reviewing the pages for it...looks promising. (I want to get our ACM off of sendmail, but Qmail seems like I'd be dealing with a bunch of transitionary problems...) ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "Economists are still trying to figure out why the girls with the least principle draw the most interest." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From cklempay@chimera.acm.jhu.edu Wed Jun 17 06:26:06 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Wed, 17 Jun 1998 01:26:06 -0400 (EDT) Subject: [Mailman-Developers] Oops.. Message-ID: I guess you guys should probably disregard that last Qmail message I sent...I had forgotten that I had sent out a similar message probably about a week and a half - 2 weeks ago... If anyone HAS gotten the aliases cooking with Qmail, please let me know. ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "Economists are still trying to figure out why the girls with the least principle draw the most interest." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From klm@python.org Fri Jun 19 16:56:48 1998 From: klm@python.org (Ken Manheimer) Date: Fri, 19 Jun 1998 11:56:48 -0400 (EDT) Subject: [Mailman-Developers] MM 1.0b4 port 25 connection conflicts w/anti-spam non relay measures Message-ID: <199806191556.LAA00436@glyph.cnri.reston.va.us> In trying out 1.0b4 on python.org i discovered a logistical conflict with the sendmail 8 anti-spam non-relaying mechanism - it *may* be due to a fault in my implementation of it, but i thought it's worth giving people a heads up. For those of you unfamiliar with it, it's a measure that tells sendmail to restrict the set of hosts from and to which it will accept mail relays. See http://www.sendmail.org/antispam.html for more info. It's something that public sites that handle mail should in good consience do - it reduces spammer's options for concealing their message origins. My problem was that mail delivered via mailman on my site to sites not explicitly specified as acceptable recipients (in the /etc/RelayTo file, i think) were being rejected. I would have thought that the mail appeared to originate at python.org, which is in the RelayTo and LocalIP files, and so would be accepted, but no go - and i don't see a way to wildcard the RelayTo to enable any recipients. I would not be surprised if i'm missing a fairly easy way to configure the setup to accept these messages, but i'm not sure, and don't have the time right now to investigate (i was supposed to go out of town, for the weekend, earlier today.) I disabled the anti-relay mechanism for the moment, but wanted to ask if anyone out there has experience with this, and suggestions for making the direct port 25 connection work without disabling the anti-relay measures. Ken From gorgo@caesar.elte.hu Sat Jun 20 14:19:49 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Sat, 20 Jun 1998 15:19:49 +0200 (METDST) Subject: [Mailman-Developers] probems with mailman Message-ID: Hello! I've just found mailman and i like it wery much, it seems much nicer then any other mailing list server i've seen. Here are some problems I've found: I tried to convert a list with about 300 normal and 220 digested subscribers to mailman with convert_list. It always failed with something like fork: resource currently not available ... it tries to send too many mails at once. The second problem is: the mail wrapper accepts exactly one gid to be run as. This makes it difficult to package up mailman in a binary package, for example for debian linux. I'm a debian developer myself btw. The problem is that we have several MTA-s available in the distribution (sendmail, smail, exim, zmailer, and as a non-free addition, even qmail), and each can use different gids, and even one can be configured to use a gid other then the default. So it would be very nice if it could be configured to accept a gid from a set ... Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From Gergely Madarasz Sat Jun 20 14:44:51 1998 From: Gergely Madarasz (Gergely Madarasz) Date: Sat, 20 Jun 1998 15:44:51 +0200 (METDST) Subject: [Mailman-Developers] one more problem Message-ID: When I'm on the membership management page, choose another chunk to view, and my admin cookie expires just then, the cgi asks me the password, and then returns me to the membership management page with chunk=something in the url, but the first chunk in the html. Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From scott@chronis.pobox.com Sat Jun 20 19:12:05 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 20 Jun 1998 14:12:05 -0400 Subject: [Mailman-Developers] one more problem In-Reply-To: ; from Gergely Madarasz on Sat, Jun 20, 1998 at 03:44:51PM +0200 References: Message-ID: <19980620141205.09053@chronis.icgroup.com> mmm. what browser /os and python version are you running? scott On Sat, Jun 20, 1998 at 03:44:51PM +0200, Gergely Madarasz wrote: | When I'm on the membership management page, choose another chunk to view, | and my admin cookie expires just then, the cgi asks me the password, and | then returns me to the membership management page with chunk=something in | the url, but the first chunk in the html. | | Greg | | -- | Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org | It's practically impossible to look at a penguin and feel angry. | Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. | HuLUG: http://www.cab.u-szeged.hu/local/linux/ | | | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers | From scott@chronis.pobox.com Sat Jun 20 19:14:37 1998 From: scott@chronis.pobox.com (Scott) Date: Sat, 20 Jun 1998 14:14:37 -0400 Subject: [Mailman-Developers] probems with mailman In-Reply-To: ; from Gergely Madarasz on Sat, Jun 20, 1998 at 03:19:49PM +0200 References: Message-ID: <19980620141437.38060@chronis.icgroup.com> I agree that the mail gid's in particular should be changed to accept a set of calling gid's. i patched an older version for this change some time ago. i'm sure that that patch won't apply to the current version, so i'll try to cook up another one over this weekend. scott On Sat, Jun 20, 1998 at 03:19:49PM +0200, Gergely Madarasz wrote: | Hello! | | I've just found mailman and i like it wery much, it seems much nicer then | any other mailing list server i've seen. Here are some problems I've | found: | I tried to convert a list with about 300 normal and 220 digested | subscribers to mailman with convert_list. It always failed with something | like fork: resource currently not available ... it tries to send too many | mails at once. | The second problem is: the mail wrapper accepts exactly one gid to be run | as. This makes it difficult to package up mailman in a binary package, for | example for debian linux. I'm a debian developer myself btw. The problem | is that we have several MTA-s available in the distribution (sendmail, | smail, exim, zmailer, and as a non-free addition, even qmail), and each | can use different gids, and even one can be configured to use a gid other | then the default. So it would be very nice if it could be configured to | accept a gid from a set ... | | Greg | | -- | Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org | It's practically impossible to look at a penguin and feel angry. | Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. | HuLUG: http://www.cab.u-szeged.hu/local/linux/ | | | _______________________________________________ | Mailman-Developers maillist - Mailman-Developers@python.org | http://www.python.org/mailman/listinfo/mailman-developers | From gorgo@caesar.elte.hu Sun Jun 21 01:30:14 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Sun, 21 Jun 1998 02:30:14 +0200 (METDST) Subject: [Mailman-Developers] one more problem In-Reply-To: <19980620141205.09053@chronis.icgroup.com> Message-ID: On Sat, 20 Jun 1998, Scott wrote: > > mmm. what browser /os and python version are you running? I use Netscape 4 on linux, the mail server is also linux, with python version 1.5.1 (debianrelease version 2.1) Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From cklempay@chimera.acm.jhu.edu Sun Jun 21 22:36:30 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Sun, 21 Jun 1998 17:36:30 -0400 (EDT) Subject: [Mailman-Developers] bsddb module Message-ID: Anyone know where to get the bsddb module that Pipermail wants? I'm screwing with it, and it is giving me an error because it claims I don't have the bsddb module...but Robin Dunn's page refers to 'the bsddb module included with the Python distribution'. (saying that it corresponds to db 1.85, while the newest are 2.x). Anyone know what's the story here? My Python installation is just from the Andrich RPMs. Corbett From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Mon Jun 22 22:48:24 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 22 Jun 1998 17:48:24 -0400 (EDT) Subject: [Mailman-Developers] probems with mailman References: <19980620141437.38060@chronis.icgroup.com> Message-ID: <13710.53424.702333.432419@anthem.cnri.reston.va.us> >>>>> "S" == Scott writes: S> I agree that the mail gid's in particular should be changed to S> accept a set of calling gid's. I wonder if it would be better to get the GID out of an environment variable of a file? Perhaps that just defeats the purpose of having a hard coded GID. Using a list of GIDs probably won't be difficult, although the configure interface will change, because it currently uses --with-*-gid="list of gids" as a list of GIDs to check, one at a time, until it finds an existing one. -Barry From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Mon Jun 22 22:49:45 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 22 Jun 1998 17:49:45 -0400 (EDT) Subject: [Mailman-Developers] probems with mailman References: Message-ID: <13710.53595.29065.897464@anthem.cnri.reston.va.us> >>>>> "GM" == Gergely Madarasz writes: GM> I tried to convert a list with about 300 normal and 220 GM> digested subscribers to mailman with convert_list. It always GM> failed with something like fork: resource currently not GM> available ... it tries to send too many mails at once. Hmm, I haven't played with convert_list yet, but I hope to soon. -Barry From johnie@netgod.net Mon Jun 22 22:16:35 1998 From: johnie@netgod.net (Johnie Ingram) Date: Mon, 22 Jun 1998 18:16:35 -0300 Subject: [Mailman-Developers] probems with mailman In-Reply-To: Your message of "Mon, 22 Jun 1998 17:48:24 EDT." <13710.53424.702333.432419@anthem.cnri.reston.va.us> Message-ID: <199806222216.SAA14147@netgod.net> "BAW" == Barry A Warsaw writes: BAW> I wonder if it would be better to get the GID out of an BAW> environment variable of a file? Perhaps that just defeats the BAW> purpose of having a hard coded GID. It may be simpler, and sufficient for Debian's purposes, to simply accept any gid less than 100. --------------------- PGP E4 70 6E 59 80 6A F5 78 63 32 BC FB 7A 08 53 4C __ _ Debian GNU Johnie Ingram mm mm / /(_)_ __ _ ___ __ "netgod" irc.debian.org mm mm / / | | '_ \| | | \ \/ / m m m / /__| | | | | |_| |> < Yes, I'm Linus, and I am your God. mm mm \____/_|_| |_|\__,_/_/\_\ -- Linus, keynote address, Expo 98 GO BLUE From gorgo@caesar.elte.hu Mon Jun 22 23:35:29 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Tue, 23 Jun 1998 00:35:29 +0200 (METDST) Subject: [Mailman-Developers] probems with mailman In-Reply-To: <13710.53424.702333.432419@anthem.cnri.reston.va.us> Message-ID: On Mon, 22 Jun 1998, Barry A. Warsaw wrote: > > >>>>> "S" == Scott writes: > > S> I agree that the mail gid's in particular should be changed to > S> accept a set of calling gid's. > > I wonder if it would be better to get the GID out of an environment > variable of a file? Perhaps that just defeats the purpose of having a > hard coded GID. I think it would be the best solution..... a config file which defines MAIL_GID, CGI_GID, etc... -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From gorgo@caesar.elte.hu Tue Jun 23 12:47:59 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Tue, 23 Jun 1998 13:47:59 +0200 (METDST) Subject: [Mailman-Developers] probems with mailman In-Reply-To: <199806222216.SAA14147@netgod.net> Message-ID: On Mon, 22 Jun 1998, Johnie Ingram wrote: > > "BAW" == Barry A Warsaw writes: > > BAW> I wonder if it would be better to get the GID out of an > BAW> environment variable of a file? Perhaps that just defeats the > BAW> purpose of having a hard coded GID. > > It may be simpler, and sufficient for Debian's purposes, to simply > accept any gid less than 100. Qmail uses nogroup (65534). But then again qmail could be set up to run mailman without the wrapper. Talking about qmail... anyone uses it with mailman ? My mails did not go out... Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From klm@python.org Tue Jun 23 17:07:52 1998 From: klm@python.org (Ken Manheimer) Date: Tue, 23 Jun 1998 12:07:52 -0400 (EDT) Subject: [Mailman-Developers] Lost maillist messages thursday eve (6/18) to friday noon (6/19) Message-ID: <199806231607.MAA13065@glyph.cnri.reston.va.us> Some of you may have lost postings posted to one of the following maillists between last thursday (6/18) evening and friday (6/19) noon. Mailman-Developers (1 msg) Matrix-SIG (8 msgs) DB-SIG (3 msgs) Doc-SIG (4 msgs) Pythonmac-SIG (3 msgs) XML-SIG (1 msg) Trove-Dev (6 msgs) This happened accompanying an upgrade of our maillist software, mailman, due to an bad interaction between a new mailman feature and an anti-spam (anti-relay) mechanism applied to python.org's sendmail configuration. This problem did not show during testing because our test recipients were all local, and not subject to the anti-relay constraints. If you sent something to any of these lists during that time frame and never saw it show, you may want to resend. Archiving was not affected, so you should be able to find the messages in the maillist archives. People receiving the lists in digest format were not affected, since the delivery problem was fixed before the digest delivery time. My apologies for the disruption! Ken Manheimer klm@python.org 703 620-8990 x268 (orporation for National Research |nitiatives # If you appreciate Python, consider joining the PSA! # # . # From gorgo@caesar.elte.hu Tue Jun 23 17:34:03 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Tue, 23 Jun 1998 18:34:03 +0200 (METDST) Subject: [Mailman-Developers] mailman and sendmail Message-ID: Hello! You might already know about this, but anyway... Sendmail 8.8.8 and 8.9.0 segfaults on queue run of mime digests made by mailman, if the remote host doesnt support 8bitmime. As I understood, it is a bug in the header detection while converting 8bit to 7bit. The result is that those who are at the beginning of the recepient list of the digest get the digest at every queue run, those who are later (after and including the first address that gets handled by a non-esmtp host) don't get it at all. I got a patch for sendmail, it seems to be working, it is also included in debian's sendmail.deb package, version 8.8.8-20. I expect that it will be included in the upstream source soon, but if anyone experiences such problems (i expect most lists run in are not 8bitmime ;)), i can send him the patch. Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From cklempay@chimera.acm.jhu.edu Wed Jun 24 02:01:30 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Tue, 23 Jun 1998 21:01:30 -0400 (EDT) Subject: [Mailman-Developers] fun Pipermail stuff Message-ID: I've resumed my attempts to get Pipermail working on my system with some stock data (Ken Manheimer hooked me up with some matrix-sig archives)...I was having some problems earlier with Pythong not being able to find the bsddb module (I asked Andrew about it)...turns out that Python distro I had was whack (it was 1.5.1-4, for reference; upgrading to -18 seems to have fixed it). So, after upgrading, I tried running...seemed to be doing ok...it was 500+ articles in and hit this: Processing article #508 into archives ['1996q1.matrix-sig'] Traceback (innermost last): File "pysig.py", line 310, in ? publicF.processUnixMailbox(file, PySIGArticle) File "/home/cklempay/src/pipermail/pipermail.py", line 366, in processUnixMailbox a=articleClass(m, self.sequence) # Create an article object File "pysig.py", line 71, in __init__ hypermail.Article.__init__(self, message, sequence, keepHeaders) File "/home/cklempay/src/pipermail/hypermail.py", line 39, in __init__ pipermail.Article.__init__(self, message, sequence, keepHeaders) File "/home/cklempay/src/pipermail/pipermail.py", line 95, in __init__ date=time.mktime(date)-tzoffset TypeError: number coercion failed [galtgulch]~/src/pysig > ls Anyone know what the story is with this? I'm too inexperienced/clueless for this to be obvious (if it is). ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "College: the fountains of knowledge, where everyone goes to drink." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 24 22:02:51 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 24 Jun 1998 17:02:51 -0400 (EDT) Subject: [Mailman-Developers] forwarded message from Dave Wolfe Message-ID: <13713.27003.343895.758727@anthem.cnri.reston.va.us> --ulXCkCkwhR Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Description: message body text Is anybody familiar with Wilma? How does it compare with Mailman? -Barry --ulXCkCkwhR Content-Description: forwarded message Content-Type: message/rfc822 Content-Description: forwarded message Content-Transfer-Encoding: 7bit Received: from cnri.reston.va.us (ns.cnri.reston.va.us [132.151.1.1]) by newcnri.cnri.reston.va.us (8.8.5/8.8.7a) with SMTP id PAA07686 for ; Wed, 24 Jun 1998 15:18:35 -0400 (EDT) Received: from relay2.UU.NET (relay2.UU.NET [192.48.96.7]) by cnri.reston.va.us (8.8.5/8.8.7a) with ESMTP id PAA16473 for ; Wed, 24 Jun 1998 15:20:54 -0400 (EDT) Received: from honor.greatcircle.com by relay2.UU.NET with ESMTP (peer crosschecked as: honor.greatcircle.com [198.102.244.44]) id QQever05274; Wed, 24 Jun 1998 15:17:31 -0400 (EDT) Received: (majordom@localhost) by honor.greatcircle.com (8.8.5/Honor-Lists-970926-1) id LAA25312; Wed, 24 Jun 1998 11:16:08 -0700 (PDT) Received: (mcb@localhost) by honor.greatcircle.com (8.8.5/Honor-980202-1) id LAA25302 for majordomo-announce@greatcircle.com; Wed, 24 Jun 1998 11:16:05 -0700 (PDT) Received: from spsem02.sps.mot.com (spsem02.sps.mot.com [192.70.231.5]) by honor.greatcircle.com (8.8.5/Honor-980202-1) with SMTP id GAA17609; Tue, 23 Jun 1998 06:28:48 -0700 (PDT) Received: from mogate.sps.mot.com by spsem02.sps.mot.com (4.1/SMI-4.1/Email 2.1 10/25/93) id AA14505 for majordomo-announce@greatcircle.com; Tue, 23 Jun 98 06:31:16 MST Received: from risc.risc.sps.mot.com by mogate.sps.mot.com (4.1/SMI-4.1/Email-2.0) id AA12549 for nb@thinkcoach.com; Tue, 23 Jun 98 06:31:01 MST Received: from miaow.risc.sps.mot.com (miaow.risc.sps.mot.com [223.72.249.15]) by risc.risc.sps.mot.com (8.8.7/8.8.7) with ESMTP id IAA18885; Tue, 23 Jun 1998 08:30:58 -0500 (CDT) Received: (from dwolfe@localhost) by miaow.risc.sps.mot.com (8.7.1/8.7.3) id IAA22726; Tue, 23 Jun 1998 08:32:49 -0500 Message-Id: <19980623083248.A28562@risc.sps.mot.com> Mime-Version: 1.0 Precedence: bulk Content-Type: text/plain; charset=us-ascii From: Dave Wolfe Sender: majordomo-announce-owner@GreatCircle.COM To: "Majordomo developer's mailing list" , "Majordomo user's mailing list" , majordomo-announce@GreatCircle.COM Subject: Wilma 1.4 released Date: Tue, 23 Jun 1998 08:32:48 -0500 X-Mailer: Mutt 0.91.1i I finally made time to package Wilma 1.4 and get it released. It's available at ftp://sol.ccsf.cc.ca.us/majordomo-contrib/wilma-1.4.tgz Whenever Jason gets back, it should also be available on ftp.hpc.uh.edu. This is primarily a bug fix release. What is Wilma? It's a Web Interface to List Mail Archives, a set of Perl programs that combines (Majordomo) mail list archives, the MHonArc mail folder HTML reformatter, and the Glimpse search engine into a CGI interface for browsing and searching those archives. Wilma requires Perl 5, MHonArc, and Glimpse. An example of a predecessor of Wilma is available for the Majordomo mailing lists at: http://www.hpc.uh.edu/majordomo-users/ http://www.hpc.uh.edu/majordomo-workers/ Many thanks to Joe R. Jah and Jason L. Tibbitts III for allowing me to publish Wilma on their ftp sites. -- Dave Wolfe --ulXCkCkwhR-- From cklempay@chimera.acm.jhu.edu Wed Jun 24 22:13:22 1998 From: cklempay@chimera.acm.jhu.edu (Corbett J. Klempay) Date: Wed, 24 Jun 1998 17:13:22 -0400 (EDT) Subject: [Mailman-Developers] forwarded message from Dave Wolfe In-Reply-To: <13713.27003.343895.758727@anthem.cnri.reston.va.us> Message-ID: That's one thing worth considering...not to rip off the idea too much, but would we rather try plugging into something like Glipse to provide search capabilities instead of rolling our own? Is the core of Glimpse C? If it is, I don't know how we'd be able to compete speed-wise with a Python-only approach, and seeing as Glimpse is the result of a substantial university project (U of Arizona, right?), I'd bet we'd be lucky to equal its accuracy. Then again, that's *super* annoying for the users to have to go and separately install Glimpse...and it's likely overkill (seeing as it's capable of doing whole sites, so plugging it in to just do some mail archives might be a little excessive...although doing so would likely give good scalability for _huge_ lists). ------------------------------------------------------------------------------- Corbett J. Klempay Quote of the Week: http://www2.acm.jhu.edu/~cklempay "College: the fountains of knowledge, where everyone goes to drink." PGP Fingerprint: 7DA2 DB6E 7F5E 8973 A8E7 347B 2429 7728 76C2 BEA1 ------------------------------------------------------------------------------- From gorgo@caesar.elte.hu Wed Jun 24 22:29:52 1998 From: gorgo@caesar.elte.hu (Gergely Madarasz) Date: Wed, 24 Jun 1998 23:29:52 +0200 (METDST) Subject: [Mailman-Developers] forwarded message from Dave Wolfe In-Reply-To: Message-ID: Talking about archiving and searching... I use the scripts created by Guy Maor for the debian lists archives. They're available as a debian package called lists-archives. It uses procmail to sort the mails to folders (so it can be used to archive any mailing list running on any host), groups them, etc, uses mhonarc for html conversion, and glimpse as a search engine. You can take a look at the archives at http://www.debian.org/Lists-Archives/ I modified it a bit not to use glimpse, and set up htdig as a search engine. Htdig seems very nice and fast, it has one big problem: it downloads all the .html-s, stores them in a db and then indexes them... so it needs a huge database. But at least it is gpl. I have this setup at http://mlf.linux.rulez.org/Archivum/ ... it is in Hungarian, so you probably won't understand it ;) Greg -- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 24 23:39:40 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 24 Jun 1998 18:39:40 -0400 (EDT) Subject: [Mailman-Developers] Easing the burden of upgrades Message-ID: <13713.32812.504844.940472@anthem.cnri.reston.va.us> We recently had an problem on python.org while upgrading to a new release of Mailman that started me thinking about a better way. I'll describe what I think is the problem, sketch out a proposed solution, and throw it out to y'all to discuss. Ken is as much responsible for the good ideas in this message as I am (blame me for the lousy ones); thanks to him for sitting down and thrashing this out first. The fundamental problem with a system like Mailman is that it is extremely difficult to test. The project is obviously not mature enough to have much of a test suite (if any), and writing one that tests all the interactions between MUA, MTA, Web browser, server, Python, and Mailman will be daunting to say the least. It'll be fantastic when we have even the framework for such a beast, but until then... So most of our testing involves creating and managing little toy lists, with us flogging the most noticeable features of Mailman to make sure the common stuff hasn't broken. The problem is that for sites using Mailman in an operational system, flag day (i.e. the day the upgrade to a new version occurs) can be pretty traumatic if we missed something crucial, but peculiar about a site. So I'm really concerned with how to make life less stressful for the operations folks who are relying on Mailman for their bread and butter. New features, fixes, etc. are strong incentive for those people to upgrade, but the fear of breakage (resulting in thousands of angry members) probably highly outweighs that incentive. Can we make the transition to a new version more controlled? You could go with a low tech approach of installing new versions temporarily with a different $prefix, using symbolic links to share list databases and templates, and hacking /etc/aliases as lists are converted to the new version. The one insurmountable problem that I see is the CGI URL. You can't share two different CGI bin dirs without exposing this to users through the list URLs. This is IMO, a showstopper; the most visible aspect of the system should be the most stable. Ken and I came up with the following architecture, and I'd like to see what y'all think: From a site administrators point of view, every Mailman installation has two `parities': a current parity and a future parity. Every list is associated with one of these parities; at any stable point in time every list is under the current parity. The system itself is associated with a parity, also the current parity at stable time. Now a new release comes out and Mailman automatically installs into the future parity (more on this below). None of the lists are automatically switched though. The site admin can switch the parity of individual lists. Let's say there's a command called `upgrade'. So upgrade toylist would switch `toylist' to the future parity. The site admin could run with this for a while, and get all warm and fuzzy about the new release. He would then upgrade reallist1 and repeat the process. Let's say he's upgraded three of his thirty lists and now has a lot of confidence in the new version. He then does upgrade * This converts all the lists to the future parity. There's one more twist however: the "system" itself is still running on the current parity even though all the lists are running on the future parity. One more command commit-parity would now commit to the new release; the future parity becomes the current parity and the current parity becomes the future parity. Maybe this auto-upgrades any still current parity lists. Data such as the list databases and templates would live outside the installed parity source code subdirs (more below). If at any time before the commit-parity is run, the site admin gets cold feet, he can downgrade reallistx or downgrade * to return the list or lists to the current parity. Now, when Mailman is installed, it always installs to the future parity, BUT ONLY IF ALL THE LIST PARITIES AND THE SYSTEM PARITY ARE CURRENT. If it ever notices that some lists are set to the future parity, but the system is still at the current parity, Mailman refuses to install. There would probably be a command to view the parities for lists and the system. I think the implementation would not be that difficult. A single file containing the parity status for each list and the system would be about the only database you'd need. The installed tree would change a bit. You'd probably have two directories inside $prefix, one that contains the current parity code and one that contains the future parity code. The list databases would live outside these two trees, directly under $prefix. List specific templates would be moved out of the templates directory, into $prefix/lists. I'd love to get some feedback from people. Is this really a problem that needs to be solved? Does this proposal solve the problem in a useful way? Is the abstraction clear? Is this just total overkill? -Barry From klm@python.org Wed Jun 24 23:52:54 1998 From: klm@python.org (Ken Manheimer) Date: Wed, 24 Jun 1998 18:52:54 -0400 (EDT) Subject: [Mailman-Developers] Easing the burden of upgrades In-Reply-To: <13713.32812.504844.940472@anthem.cnri.reston.va.us> Message-ID: I think barry's description conveys what we discussed real well. The only addition i'd make is a slightly different command arrangement. On Wed, 24 Jun 1998, Barry A. Warsaw wrote: > automatically switched though. The site admin can switch the parity > of individual lists. Let's say there's a command called `upgrade'. > So > upgrade toylist > [...] > upgrade reallist1 > [...] > lists and now has a lot of confidence in the new version. He then > does > upgrade * [upgrade all the lists] > commit-parity [make the future version the current] > downgrade reallistx > downgrade * I'd prefer to have a single command that could do everything - i'm thinking 'migrate', which defaults to migrating forward (upgrading). Eg: migrate toylist migrate -forward mailman-developers [[the "-forward" is unnecessary]] migrate -all [[migrate everything forward]] migrate -back -all [[back off!]] migrate -commit [[make the new version current]] (Since the command is in mailman's bindir, i guess it could be a general name - but if there's concern, we could distinguish it with mailman's initials - 'mmmigrate' - to convey the mmm mmm delicious experience of changing a production system...-) From bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) Wed Jun 24 23:57:22 1998 From: bwarsaw@CNRI.Reston.Va.US (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 24 Jun 1998 18:57:22 -0400 (EDT) Subject: [Mailman-Developers] Easing the burden of upgrades References: <13713.32812.504844.940472@anthem.cnri.reston.va.us> Message-ID: <13713.33874.951037.199204@anthem.cnri.reston.va.us> >>>>> "KM" == Ken Manheimer writes: KM> I think barry's description conveys what we discussed real KM> well. The only addition i'd make is a slightly different KM> command arrangement. I'm definitely open to suggestions here. I just wanted distinct command names so that it was clearer what the intent was. -Barry From akuchlin@cnri.reston.va.us Mon Jun 29 16:32:13 1998 From: akuchlin@cnri.reston.va.us (Andrew Kuchling) Date: Mon, 29 Jun 1998 11:32:13 -0400 (EDT) Subject: [Mailman-Developers] Rendered archives: where to put them? Message-ID: <13719.45251.814406.849296@newcnri.cnri.reston.va.us> I'm (finally!) revisiting the question of archiving for Mailman. There are two additional config parameters that seem to be needed. First, I need to know into what directory to write the formatted HTML files, both public and private. The private files shouldn't go into the actual Web tree, since they'll be accessed via the 'private' CGI script. Either there should be a parameter in mm_defaults/mm_cfg to specify the ARCHIVE_HTML_ROOT, or we could always write them into the Mailman tree somewhere -- say, ${prefix}/pipermail/html or /formatted or whatever. The second course would then require configuring the Web server to map your desired public archive URL to the root of the public tree, and the private archives to the CGI script. Second, I need to know what URL to use to access the public archives. (The private archives should be accessed via the 'private' CGI script.) Suggestions? -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Seek those who find your road agreeable, your personality and mind stimulating, your philosophy acceptable, and your experience helpful. Let those who do not, seek their own kind. -- Henri Fabre From scott@chronis.pobox.com Mon Jun 29 16:53:48 1998 From: scott@chronis.pobox.com (Scott) Date: Mon, 29 Jun 1998 11:53:48 -0400 Subject: [Mailman-Developers] Rendered archives: where to put them? In-Reply-To: <13719.45251.814406.849296@newcnri.cnri.reston.va.us>; from Andrew Kuchling on Mon, Jun 29, 1998 at 11:32:13AM -0400 References: <13719.45251.814406.849296@newcnri.cnri.reston.va.us> Message-ID: <19980629115348.38262@chronis.icgroup.com> On Mon, Jun 29, 1998 at 11:32:13AM -0400, Andrew Kuchling wrote: | I'm (finally!) revisiting the question of archiving for Mailman. | There are two additional config parameters that seem to be needed. great! | First, I need to know into what directory to write the formatted HTML | files, both public and private. The private files shouldn't go into | the actual Web tree, since they'll be accessed via the 'private' CGI | script. Either there should be a parameter in mm_defaults/mm_cfg to | specify the ARCHIVE_HTML_ROOT, or we could always write them into the | Mailman tree somewhere -- say, ${prefix}/pipermail/html or /formatted | or whatever. The second course would then require configuring the Web | server to map your desired public archive URL to the root of the | public tree, and the private archives to the CGI script. I'm for making it specifiable via a config variable. That way, large sites could put the archives on a separate partition from the mailman installation. | Second, I need to know what URL to use to access the public archives. | (The private archives should be accessed via the 'private' CGI | script.) how about http://HOST/mailman/archives/listname ? shouldn't this access both the private and the public archives? scott