[Mailman-Users] Uncaught runner exception: 'ascii' codec can'tencode character u'\u201c'

[Sahil Tandon]

On Feb 5, 2012, at 2:37 PM, Mark Sapiro <mark at msapiro.net> wrote:

> Sahil Tandon wrote:
> 
>> I am seeing this error with all mail sent to just one (of several)
>> mailman mailing lists that are operating without problems.  I found some
>> related discussions in the archives, but nothing that (as far as I could
>> tell) corresponds to what I am seeing.
>> 
>> Sorry for my MUA's line wrapping:
>> 
>> ---
>> 
>> Feb 04 20:31:51 2012 (1176) Uncaught runner exception: 'ascii' codec
>> can't encode character u'\u201c' in position 0: ordinal not in
>> range(128)
>> Feb 04 20:31:51 2012 (1176) Traceback (most recent call last):
>> File "/usr/local/mailman/Mailman/Queue/Runner.py", line 111, in
>> _oneloop
>>   self._onefile(msg, msgdata)
>> File "/usr/local/mailman/Mailman/Queue/Runner.py", line 167, in
>> _onefile
>>   keepqueued = self._dispose(mlist, msg, msgdata)
>> File "/usr/local/mailman/Mailman/Queue/IncomingRunner.py", line 130,
>> in _dispose
>>   more = self._dopipeline(mlist, msg, msgdata, pipeline)
>> File "/usr/local/mailman/Mailman/Queue/IncomingRunner.py", line 153,
>> in _dopipeline
>>   sys.modules[modname].process(mlist, msg, msgdata)
>> File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 91, in
>> process
>>   send_digests(mlist, mboxfp)
>> File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 132, in
>> send_digests
>>   send_i18n_digests(mlist, mboxfp)
>> File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 306, in
>> send_i18n_digests
>>   msg = scrubber(mlist, msg)
>> File "/usr/local/mailman/Mailman/Handlers/Scrubber.py", line 182, in
>> process
>>   charset = part.get_content_charset(lcset)
>> File "/usr/local/mailman/pythonlib/email/Message.py", line 817, in
>> get_content_charset
>>   charset = unicode(charset[2], pcharset).encode('us-ascii')
>> UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in
>> position 0: ordinal not in range(128)
>> 
>> Feb 04 20:31:51 2012 (1176) SHUNTING:
>> 1328387505.157315+73d2286bbd8f34b0f690645d2037aa561e373438
>> 
>> ---
>> 
>> If I understand the exception correctly, 'position 0' is the first
>> character in the shunted pickle file?
> 
> 
> No. In fact, the problem has nothing to do with any of the messages
> being shunted.
> 
> The situation is that the list's lists/LISTNAME/digest.mbox file is
> larger than digest_size_threshold, so every post causes an attempt to
> produce a digest as it is processed by Mailman/Handlers/ToDigest.py.
> 
> The actual error occurs because somewhere in the earlier messages in
> digest.mbox there is a message with a Content-Type: message or
> sub-part header with a charset="... parameter and the " is not an
> ascii quote; it is a unicode 201c left double quotation mark (position
> 0 in this case is the first character of the value of the charset=
> parameter).
> 
> Also see
> <http://mail.python.org/pipermail/mailman-users/2006-May/051347.html>
> and perhaps other hits from
> <http://www.google.com/search?q=site:mail.python.org+inurl%3Amailman+digest.mbox+shunted>
> and the last content paragraph of the FAQ at
> <http://wiki.list.org/x/noA9>.
> 
> You can edit the digest.mbox and fix the defect, or you can just move
> it aside. Either action will allow the list to start working again.
> 
> You will want to unshunt the shunted messages because they haven't been
> archived or delivered, but I think you will need to remove them from
> the digest.mbox if you intend to repair it or they will be duplicated
> in the digest.

My message was in moderation for too long.  In the meantime, I figured this out and mv'd the old digest mbox to allow operation of this list.  I sent another message to the list noting that but it was rejected as a dupe (which it wasn't).

> Note that you must have an old Mailman version. Beginning with Mailman
> 2.1.7, this issue would not stop the list. It would prevent the digest
> from being produced, but the list posts would be processed normally.

You are right!  It's an old version I inherited and we cannot for a variety of reasons upgrade.  Thanks for your response.

Sahil