[Mailman-Users] Shunting & unknown encoding question

[Mark Sapiro]

Paul Key wrote:
>
>I have had an email shunted and am not sure why - any ideas why the encoding is unknown?
>
>Dec 11 11:43:42 2007 (24788) Uncaught runner exception: unknown encoding: unknown
>Dec 11 11:43:42 2007 (24788) Traceback (most recent call last):
>  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 110, in _oneloop
>    self._onefile(msg, msgdata)
>  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 160, in _onefile
>    keepqueued = self._dispose(mlist, msg, msgdata)
>  File "/usr/local/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose
>    more = self._dopipeline(mlist, msg, msgdata, pipeline)
>  File "/usr/local/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline
>    sys.modules[modname].process(mlist, msg, msgdata)
>  File "/usr/local/mailman/Mailman/Handlers/CookHeaders.py", line 75, in process
>    prefix_subject(mlist, msg, msgdata)
>  File "/usr/local/mailman/Mailman/Handlers/CookHeaders.py", line 262, in prefix_subject
>    h.append(s, c)
>  File "/usr/local/mailman/pythonlib/email/Header.py", line 285, in append
>    s = s.encode(outcodec, errors)
>LookupError: unknown encoding: unknown
>
>Dec 11 11:43:42 2007 (24788) SHUNTING: 1197373305.4631159+54681260563e1d1577b352329843c2fa738ba81

Look at the message (bin/show_qfiles
qfiles/shunt/1197373305.4631159+54681260563e1d1577b352329843c2fa738ba81.pck).

I think you will see an RFC 2047 encoded Subject: header. I.e.
something like

 Subject: =?iso-8859-1?q?this=20is=20some=20text?=

except the characterset instead of being like the "iso-8859-1" in this
case, will literally be "unknown".

Some MUAs will create such an encoding with character set "unknown" or
"unknown-8bit" when the subject contains 8bit characters and the MUA
doesn't have a context to know what the character set is.

If this is a recent Mailman, about a dozen lines into the definition of
process() in Mailman/Handlers/CookHeaders.py you will see

    if not msgdata.get('isdigest') and not fasttrack:
        try:
            prefix_subject(mlist, msg, msgdata)
        except (UnicodeError, ValueError):
            # TK: Sometimes subject header is not MIME encoded for 8bit
            # simply abort prefixing.
            pass

If you change the 'except' to

        except (UnicodeError, ValueError, LookupError):

and restart Mailman, you can unshunt the message and it will be
processed without prefixing the subject.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan