[Mailman-Users] Unknown encoding error

[Mark Sapiro]

Khosrow Ebrahimpour wrote:
> 
> A week or so ago I ran across this problem on one of our mailing lists: A user 
> had sent an email with an apparently bad subject line. This has caused 
> mailman to stop delivering emails for that specific mailning list. Here's the 
> error from the logfile:
> 
> ---
> 
> Jan 05 21:25:10 2009 (23055) Uncaught runner exception: unknown encoding: Re: 
> DDS 26073 : Imap - installation d
> Jan 05 21:25:10 2009 (23055) Traceback (most recent call last):
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 111, in _oneloop
>     self._onefile(msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 167, in _onefile
>     keepqueued = self._dispose(mlist, msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in 
> _dispose
>     more = self._dopipeline(mlist, msg, msgdata, pipeline)
>   File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in 
> _dopipeline
>     sys.modules[modname].process(mlist, msg, msgdata)
>   File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 91, in process
>     send_digests(mlist, mboxfp)
>   File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 132, in 
> send_digests
>     send_i18n_digests(mlist, mboxfp)
>   File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 306, in 
> send_i18n_digests
>     msg = scrubber(mlist, msg)
>   File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 240, in process
>     url = save_attachment(mlist, part, dir)
>   File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 373, in 
> save_attachment
>     fnext = os.path.splitext(msg.get_filename(''))[1]
>   File "/usr/lib/python2.3/email/Message.py", line 731, in get_filename
>     return unicode(newvalue[2], newvalue[0] or 'us-ascii')
> LookupError: unknown encoding: Re: DDS 26073 : Imap - installation d
> 
> Jan 05 21:25:10 2009 (23055) SHUNTING: 
> 1231190705.198385+f75914e7ce7a9daf6e70914e0ed49923a47d2dd1


The message that causes this problem is not the message that is shunted.
All the shunted messages are (presumably) valid messages that arrived
after the issue started.

The problem is that a message was posted to the list and probably
delivered normally and stored in the digest.mbox file for the list.
Then, eventually, the digest.mbox grew big enough that every subsequent
message triggers a digest delivery on size, but the bad message in
digest.mbox throws the above exception in the ToDigest handler, and the
digest is not produced and the current message is shunted.


> The full subject line should be: 
> 
> RE: DDS 26073 : Imap - installation d'application 'vacation'
> 
> so I think the apostrophe is causing havoc here...


Yes, it is an apostrophe that is the problem, but it is in a 'name' or
'filename' parameter of a Content-Type: or Content-Disposition: header.


> I have searched mailing list and the search engines, and it looks like this 
> error was a problem for people but has been solved in subsequent versions of 
> mailman (we are running 2.1.5 on Debian Sarge). What I haven't been able to 
> find out is how to purge out the offending email, and continue life as normal 
> until we upgrade to Etch!
> 
> I should say that I have tried removing the offending file from 
> $MAILMANDIR/qfiles/shunt and running "unshunt", but nothing changed.


You don't want to remove the shunted files as they are not the problem,
and you can't successfully unshunt until you fix the digest.mbox.

You need to edit the lists/LISTNAME/digest.mbox. You can edit it by hand
or with an MUA like mutt for example.

The offending message has an RFC 2231 enncoded 'name' or 'filename'
parameter with an apostrophe in the name. It will look something like

Content-Disposition: attachment;
  filename*0*='charset'lang'text....................................
  filename*1=more.............................

And text will have an apostrophe.

You can simply remove those parameters (filename and/or name) or delete
the entire message.

You can also just delete the lists/LISTNAME/digest.mbox file, but then
you'd lose those messages from the digest.

After fixing the digest.mbox, you can run bin/unshunt to finish
processing the shunted messages.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan