[Mailman-Users] Header Cleanup Script

[Scott Courtney]

On Wednesday 12 June 2002 12:36 am, Jon Parise wrote:
> On Sun, Jun 02, 2002 at 05:09:50AM -0400, Scott Courtney wrote:
> > Another change that may or may not apply to your lists: Some versions of
> > KMail, the client that comes with KDE, produce a header called
> > "Message-Id:". The parser in "arch" requires this to be "Message-ID:" or
> > it chokes. I didn't put that into my awk script because it may not apply
> > everywhere, and fixing it is just a matter of :%s/^Message-Id/Message-ID/
> > in vi, or equivalent.
>
> I've been meaning on investigating this one ever since you sent this
> message.  I have not been able to recreate this problem (under Mailman
> 2.1b2+, at least).
>
> Could you provide some more details?

I am running 2.1b2 (not sure about that "+" part) and am able to reliably
reproduce the problem. Any message that has a "Message-Id:" header instead
of "Message-ID:" will fail to import. The arch program doesn't in itself give
very detailed diagnostics, but I have added some error tracing. The error is
detected in Mailman/Archiver/pipermail.py at about line 550. It is actually
occuring in the next() method of the mbox object.

I'm not a Python programmer (most of my work is in PHP, Java, and C), and since
there was such an easy workaround I haven't pursued it further than that. What
I can tell you is that if I do the global search and replace mentioned above,
it fixes the problem reliably, so long as this is the *only* problem with the
messages under test. The other problems, however, are fixed by my awk script.

There are certain multipart/alternative messages that just don't go, period.
In my case, I looked at them individually and made a conscious decision that
I didn't care, and threw them away. That's not a generalized solution, but
it worked for me since such messages are extremely rare (about 0.05%) in
my datasets.

I'll be glad to work with you in resolving this issue; I am not sure what
other specific information you might want, but I will be glad to provide
whatever info you need, if I can obtain it.

Scott

-- 
-----------------------+------------------------------------------------------
Scott Courtney         | "I don't mind Microsoft making money. I mind them
courtney at 4th.com       | having a bad operating system."    -- Linus Torvalds
http://www.4th.com/    | ("The Rebel Code," NY Times, 21 February 1999)