[Mailman-Users] Migrating from YahooGroups to Mailman

[alex wetmore]

On Tue, 31 Jul 2001, Greg Ward wrote:
> On 31 July 2001, Sarah K. Miller said:
> > We're migrating some lists form YahooGroups to Mailman. Does anyone
> > know of a way to automatically "slurp" all the messages off Yahoo and
> > plop them into Mailman? The only way I've found to do it and retain
> > the original information is cut and paste each one
> > individually. That's a little overwhelming when you're looking at
> > 1500+ messages. Yahoo was no help at all. If anybody here knows of a
> > utility of some sort that would do it, please share!
>
> I had a similar problem getting a list off of ListBot recently.  It only
> had 19 messages in the archive (I was just saving them for posterity --
> the list wasn't exactly a big hit), so it wasn't too bad.  Out of
> principle, though, I automated the procedure a little bit.  The only
> reason it was possible is that ListBot had a link to get the full
> headers as plain text (wrapped in <PRE> in the web page, of course) for
> each message.

It only took a few minutes browsing groups.yahoo.com to figure out
their archives.

Each message (starting with 1) up to the number of messages in the
group is available with this url:
http://groups.yahoo.com/group/<group>/message/<msgnum>?source=1

You just need a simple script which:
* collects the messages
* strips everything that isn't in <pre> </pre>
* converts the HTML back to plain text
* adds it to an mbox

This should be fairly simple for anyone with moderate perl or python
knowledge to write.  It will take a little while to download all of
the archives, but I don't think that is a big issue.

I've done similar things with mining list archives off of other
mailing list hosts, but I haven't had to do this with yahoogroups yet.
My scripts (or whatever is left of them, I just have one that I hack
as necessary) aren't going to be useful to the mailman crowd because I
don't use the mbox format for my archives or pipermail.

alex