Apache/1.3.14 Server at porsis.sistemas.unsu Port 80

 in the web page, of course) for
each message.  After I figured out the pattern for that URL, I did
something like this:

for i in 1 2 3 ... 19 ; do       # yes, you have to type them all out
  GET http://www.listbot.com/(hairy url with $i in it somewhere) > msg$i.txt
  fix_msg msg$i.txt
done

GET is the alias for lwp-request installed by lwp (libwww-perl).  It
just does an HTTP request from the shell.  Handy, but it would be easy
to whip up something similar in Python (which I've been meaning to do
for a while now...).

fix_msg was a little Perl script I wrote to undo HTML encodings in the
not-quite-plain-text file downloaded from listbot.com.  I don't seem to
have it anymore; it went something like this:

  #!/usr/bin/perl -p

  s/&/&/g;
  s/"/"/g;
  # ...etc...

Again, you could do this pretty easily in Python, but why bother?  Perl
is perfect for this sort of hackery.  It all depends on the text Yahoo
presents you with; mine was similarly dependent on ListBot.

To glue the messages together into a legitimate mbox (which you can
build into an archive with some Mailman tool... ummm... bin/arch
maybe?), you need to make sure each msg*.txt file starts with a "From "
line and ends with a blank line.  formail, a tool supplied with
procmail, will take care of the former.  Or you could DIY in that
mythical fix_msg script (which would also be a good place to ensure a
trailing blank line, although I think you don't want one on the last
message...)

You should also systematically rename the files from eg. msg1.txt to
msg0001.txt so the next command works.  Again, child's play if you know
your way around Unix and Perl...

Finally, "cat msg*.txt > mylist.mbox" (or whatever) and run Mailman's
archive tool on it.

Make sure you've created a workable mbox file by running your favourite
mail client on it, eg. "mutt -f mylist.mbox".

Anyways, if you're familiar with Unix and the tools available to you,
this should be doable... as long as Yahoo makes the full original text
of the messages available to you!  If not, all is lost, give up, doom,
failure, etc.  If I was in your shoes (1500 messages to process), I'd
probably take a few hours and write a nice Python script to do it right.
For the 19 messages I had to do, crude shell-and-Perl hackery was just
fine.

        Greg
-- 
Greg Ward - software developer                gward at mems-exchange.org
MEMS Exchange                            http://www.mems-exchange.org

From gward at mems-exchange.org  Tue Jul 31 22:46:19 2001
From: gward at mems-exchange.org (Greg Ward)
Date: Tue, 31 Jul 2001 16:46:19 -0400
Subject: [Mailman-Users] Migrating from YahooGroups to Mailman
In-Reply-To: <4.2.0.58.20010731151502.00ad7f00@perilpoint.com>; from listmgr@perilpoint.com on Tue, Jul 31, 2001 at 03:18:07PM -0500
References: <01b801c119fc$89a94770$0200010a@home.beeze.com> <4.2.0.58.20010731151502.00ad7f00@perilpoint.com>
Message-ID: <20010731164619.B2727@mems-exchange.org>

On 31 July 2001, Bradford Shaw said:
> I would also be interested in this. We just moved a large list off of 
> yahoogroups to its own server and Yahoo refuses to cooperate in letting us 
> have the archived messages (over 2 years worth at approx 1500 per month). 

I've just poked around groups.yahoo.com a bit, and it looks like this is
doable (but painful).  From any message, you can hit the "View source"
link.  This takes you to a page like

  http://groups.yahoo.com/group/NucNews/message/3859?source=1

(message 3859 of the "NucNews" list).

The good news:

  * the URL is dead easy to generate, eg. here's Python code to suck
    the entire archives for a list, one HTML file per message:

      url_template = "http://groups.yahoo.com/group/%s/message/%d?source=1"

      group_name = "NucNews"
      num_messages = 3887                  # every message page has this
      for msg_num in xrange(num_messages):
        url = url_template % (group_name, msg_num)
        msg_filename = "msg-%04d.html" % msg_num
        urllib.urlretrieve(url, msg_filename)

    (UNTESTED -- YMMV)

The bad news:

  * the HTML you download will need serious massaging before it's genuinely
    plain text (ie. valid RFC 822 messages).  Eg. here's a sample from
    message 1 of NucNews:

"""

From prop1 at xxxxx.xxxx Tue Jan 12 14:08:50 1999
X-Digest-Num: 0
Message-ID: <62814.0.1.959296473 at e...>
Date: Tue, 12 Jan 1999 17:08:50 -0500
[...]
"""

Note how anything that looks like an email address (including the
return-path and message-id headers) are turned into hyperlinks.  You'll
need to strip out these  tags (see Python's htmllib, although
you can probably kludge it with a regex) and just preserve the content
of the tag, eg. "prop1 at xxxxx.xxxx".

Even after doing this, you still don't have a valid RFC 822 message,
eg. here's another excerpt from msg 1 of NucNews:

  From: Peace Through Reason <prop1 at xxxxx.xxxx

De-HTML-ified, that becomes:

  From: Peace Through Reason 

No, it's not.  It's a Python 'marshaled' database of the MailList 
object.  See MailList.py; all details are there.

I highly recommend "hacking" config.db using, say, config_list
rather than directly (assuming that the tools in bin/ don't
have the functionality you need already).

> I can't say since I've never checked but it's probably dbm or BerkeleyDB.
> 
> On Mon, 30 Jul 2001, Greg Schnippel wrote:
> 
> >
> > I'm trying to hack config.db and was wondering if
> > anyone knew what DB format the file was.. ?

From thomas at xs4all.net  Mon Jul 30 23:33:56 2001
From: thomas at xs4all.net (Thomas Wouters)
Date: Mon, 30 Jul 2001 23:33:56 +0200
Subject: [Mailman-Users] Re: [Mailman-Developers] Looping digest - mailman bug?
In-Reply-To: <20010730133518.C61068-100000@phred.org>
References: <20010730133518.C61068-100000@phred.org>
Message-ID: <20010730233356.D20671@xs4all.nl>

On Mon, Jul 30, 2001 at 01:37:08PM -0700, alex wetmore wrote:
> On Mon, 30 Jul 2001, Thomas Wouters wrote:
> > Sorry, this is the wrong fix. "\n." isn't the pattern that breaks it; you
> > want "\r?\n.\r?\n". And the proper escape of a single dot on a line is
> > doubling it (see the SMTP standard.) Barry, do you have a clue whether
> > msgtext is guaranteed (not) to have \r's (CR) in them ? If either is
> > guaranteed, we don't even need to use that ugly 're' module :)

> RFC2822 requires CRLF to end a line, not just LF and not just CR.
> However there are many broken SMTP clients which ignore this
> requirement.

That's not the issue; IIRC, both the smtplib module (used by the SMTPDirect
delivery in Mailman) as the Sendmail deliverer use CRLF's, adding CR's where
necessary. (One of the very best reasons for a 'rich' standard library like
Python's is strict compliance to the standards. Like you say, the CR/LF
issue is ignored by so many mail-sending and -receiving applications, it's
just not funny. By using smtplib, the burden gets lifted off your shoulders,
and you do it 'correct' whether you want it or not :)

My question was whether the message text, at that moment, was guaranteed to
contain CR's (or guaranteed not to contain them.) I'm not sure what kind of
mangling Barry's mimelib does :)

-- 
Thomas Wouters 

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From preeti.mehta at wipro.com  Tue Jul 31 05:02:40 2001
From: preeti.mehta at wipro.com (Preeti Mehta)
Date: Tue, 31 Jul 2001 08:32:40 +0530
Subject: [Mailman-Users] help on user interface 
References: <020d01c11683$3cebd5a0$cacda8c0@wipro.com>  <24217.996521317@kanga.nu>
Message-ID: <002f01c1196d$44c9f3a0$cacda8c0@wipro.com>

thanks for the reply
actually i think i shd have asked whether it has a graphical user interface
i know about the scripts it has etc but wanted to know if lists can b
created & deleted from a graphical user interface instead just like there is
the interface to view list info ,subscribe ,manage lists etc
if so cld b please tell me where i can find it

I am using sendmail ver 8.9.3 & mailman ver 2.0.5

thanks a ton
regards
preeti

----- Original Message -----
From: J C Lawrence 
To: Preeti Mehta 
Cc: 
Sent: Tuesday, July 31, 2001 12:58 AM
Subject: Re: [Mailman-Users] help on user interface

> On Fri, 27 Jul 2001 15:32:24 +0530
> Preeti Mehta  wrote:
>
> > i wanted to know if mailman has a user interface to create new
> > lists????

>
> Yes -- it would be rather useless if you could never create a list
> wouldn't it?
>
> In the current version it is command line based.  Depending on your
> choice of local MTA, it may or may not requite root access.  In the
> next version of Mailman there is a web interface to list creation,
> which again, is somewhat subject to your MTA choice and
> configuration.
>
> --
> J C Lawrence                                    )\._.,--....,'``.
> ---------(*)                                   /,   _.. \   _\  ;`._ ,.
> claw at kanga.nu                                 `._.-(,_..'--(,_..'`-.;.'
> http://www.kanga.nu/~claw/                     Oh Freddled Gruntbuggly

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Wipro_Disclaimer.txt
Url: http://mail.python.org/pipermail/mailman-users/attachments/20010731/cd55aba0/attachment.txt 

From marc_news at valinux.com  Tue Jul 31 07:31:59 2001
From: marc_news at valinux.com (Marc MERLIN)
Date: Mon, 30 Jul 2001 22:31:59 -0700
Subject: [Mailman-Users] virtual hosting tactic
In-Reply-To: <20010615175503.Q4983-100000@phred.org>; from alex@phred.org on Fri, Jun 15, 2001 at 05:56:59PM -0700
References:  <20010615175503.Q4983-100000@phred.org>
Message-ID: <20010730223159.E12748@magic.merlins.org>

On Fri, Jun 15, 2001 at 05:56:59PM -0700, alex wetmore wrote:
> This does bring up a question that another admin asked me.  Is there a
> way that the site administrator can remove permission from list
> administrators from changing some of the fields?  For instance I would
> like to have a site policy that requires that Reply-To is never set to
> the list.

That's what I use:
(Not perfect, but good enough)

----------------------------------------------------------------------------
chef:/var/local/mailman/bin/scripts# cat resetlistconfigs 
#!/bin/bash

cd ~mailman/lists/
for i in *
do
    ~mailman/bin/config_list -o - $i | tail +3  > /var/tmp/tmp_mm_settings.before
    ~mailman/bin/config_list -i ~mailman/bin/scripts/mailman_force_settings $i
    ~mailman/bin/config_list -o - $i | tail +3  > /var/tmp/tmp_mm_settings.after

    if ! diff -q /var/tmp/tmp_mm_settings.before /var/tmp/tmp_mm_settings.after &>/dev/null; then
        echo "Fixing config of list $i"
        diff -u0 /var/tmp/tmp_mm_settings.before /var/tmp/tmp_mm_settings.after
    fi
done
chef:/var/local/mailman/bin/scripts# cat mailman_force_settings 
reply_goes_to_list = 0
host_name = 'lists.valinux.com'
web_page_url = 'https://lists.valinux.com/lists/'
bounce_processing = 1
automatic_bounce_action = 3
clobber_date = 1
----------------------------------------------------------------------------

Marc
-- 
Microsoft is to operating systems & security ....
                                      .... what McDonalds is to gourmet cooking

Home page: http://marc.merlins.org/   |   Finger marc_f at merlins.org for PGP key

From rich at chi.mti.com  Tue Jul 31 20:36:20 2001
From: rich at chi.mti.com (Rich Ramos)
Date: Tue, 31 Jul 2001 13:36:20 -0500
Subject: [Mailman-Users] infinite loop on bounce from postmaster to list
Message-ID: <613922.996586580@[144.86.11.157]>

We had an ugly situation occur yesterday.

Here's the summary:
1.  1+ Meg email sent to list with no size limit
2.  An email user, who has a ccMail account that forwards his mail on to 
another account outside the company, has his outside mailbox hit his quota 
limit.  (This user is also digested)
3.  The MTA at the receiving end that detects the over quota and sends an 
email back to the list itself (see below for mail headers as to why this 
might happen).
4.  Mailman sends this email back to everyone on the list including the 
digested mail user, whose MTA then rejects the email for the same reason as 
before.
5.  Repeat steps 3 and 4 about every 8 minutes.

This loop continues until the system comes to a halt from a full /tmp 
directory.  On every iteration the mail message grew a little bigger 
because the last email gets copied on to the reject message.

There are a few problems here, but I'm not sure which problem is the wrong 
case and which solution I should pursue.

Problem #1.  ccMail changes the headers when it processes the mail.

Here's the normal headers from a post to mailman (I've stripped the 
Received lines):
From: Rich Ramos 
To: test at chi.mti.com
Message-ID: <563517.996585740@[144.86.11.157]>
X-Mailer: Mulberry/2.1.0b3 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Subject: [Test] test to show headers
Sender: test-admin at chi.mti.com
Errors-To: test-admin at chi.mti.com
X-BeenThere: test at chi.mti.com
X-Mailman-Version: 2.0.3
Precedence: bulk
List-Help: 
List-Post: 
List-Subscribe: ,

List-Id: 
List-Unsubscribe: ,

List-Archive: 
Date: Tue, 31 Jul 2001 13:22:20 -0500

Here's the way the headers get changed by ccmail:
Message-Id: <0107319966.AA996603967 at ccmgate.mti.com>
X-Mailer: ccMail Link to SMTP R8.30.00.7
Date: Tue, 31 Jul 2001 13:22:20 -0700
From: Rich Ramos 
To: 
Subject: [Test] test to show headers
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="simple boundary"

As you can tell it's quite a bit different.  However I'm not totally clear 
that even with the changed headers, that the destination MTA should have 
sent the bounce message back to the list.  With the From: and To: headers 
the way they are why didn't the MTA send the bouce back to the From: 
address?

Problem #2:
The message was being delivered to a digested user, however the bounces 
happened every few minutes.  Why didn't the digest get sent once and then 
not again until the next interval for the digest?

Doesn't mailman have some mechanism to catch looping posts to a list?

-Rich

From josemedeiros at earthlink.net  Tue Jul 31 20:34:35 2001
From: josemedeiros at earthlink.net (Jose Medeiros)
Date: Tue, 31 Jul 2001 11:34:35 -0700
Subject: [Mailman-Users] Mailman RPM?
Message-ID: 

Do you have an RPM for Mailman?

Jose
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/mailman-users/attachments/20010731/22d9004a/attachment.html 

From jade at delrio.com  Wed Jul 11 05:54:30 2001
From: jade at delrio.com (Eris)
Date: Tue, 10 Jul 2001 22:54:30 -0500
Subject: [Mailman-Users] Help?
Message-ID: <001d01c109bd$304b6180$b4897841@delrio.com>

I need some assistance if possible i get the following errors after reading your trouble shooting i still have no idea what im doing wrong.

# ./configure&
[1] 21167
[root mailman-2.0.6]# loading cache ./config.cache
checking for --with-python... no
checking for python... /usr/bin/python
checking Python interpreter... /usr/bin/python
checking Python version... 1.5.2
checking for a BSD compatible install... /usr/bin/install -c
checking whether make sets ${MAKE}... yes
checking for true... /bin/true
checking for --without-gcc... no
checking for gcc... gcc
checking whether the C compiler (gcc  ) works... yes
checking whether the C compiler (gcc  ) is a cross-compiler... no
checking whether we are using GNU C... yes
checking whether gcc accepts -g... yes
checking whether #! works in shell scripts... yes
checking for --with-var-prefix... no
checking for --with-username... mailman
checking for mailman UID... 500
checking for --with-groupname... mailman
checking for mailman GID... 500
checking permissions on /home/mailman... configure: error: 
***** Installation directory /home/mailman is not configured properly!
***** Set-gid bit must be set for directory: /home/mailman
***** Permissions should be at least 02775: /home/mailman

drwxr-xr-x   4 mailman  mailman      1024 Feb 21 23:34 mailman
[root mailman]# ls -al
total 7
drwxr-xr-x   4 mailman  mailman      1024 Feb 21 23:34 .
drwxr-xr-x  15 root     root         1024 Feb 21 23:20 ..
-rw-------   1 mailman  mailman       176 Feb 21 23:37 .bash_history
-rwxr-xr-x   1 mailman  mailman       434 Feb 21 23:20 .emacs
-rw-r--r--   1 mailman  mailman       778 Feb 21 23:20 .ftphelp
drwxr-xr-x   5 mailman  mailman      1024 Feb 21 23:20 group
drwxr-xr-x   4 mailman  mailman      1024 Feb 21 23:20 user

Anyone able to help?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/mailman-users/attachments/20010710/4b61c246/attachment.htm 

Downloading