[Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with Finnish

[Eva Isaksson]

I tried to approach this problem by getting the source package from the
debian repository.
We are still running wheezy, so we need to stick to 2.1.18 for now.

When I built the package, the resulting mailman.mo file for Finnish ends up
utf-8 encoded
(which I deduced by converting it back to a .po file by msgunfmt.
The encoding seems to happen in debian/patches/91_utf8.patch
which does have a list of correct encodings, but this is what seems to
happen:
(not only to Finnish, but to all listed languages):

msgconv -t utf-8 fi/LC_MESSAGES/mailman.po | tee
fi/LC_MESSAGES/mailman.po.utf-8 | msgfmt -o fi/LC_MESSAGES/mailman.mo -

Also:
http://sources.debian.net/patches/patch/mailman/1:2.1.18-2/91_utf8.patch/
<https://mail.google.com/>

If this means that debian is forcing everything to utf-8, what should I do,
if every package comes with this forced utf-8 encoding?

Here is the trackback we got in October:

Oct 28 10:04:28 2015 (22144) Uncaught runner exception: 'utf8' codec
can't decode byte 0xe4 in position 11: invalid continuation byt
e
Oct 28 10:04:28 2015 (22144) Traceback (most recent call last):
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
    self._onefile(msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in
_dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in
_dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 239, in
process
    i18ndesc = uheader(mlist, mlist.description, 'List-Id', maxlinelen=998)
  File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 65, in
uheader
    return Header(s, charset, maxlinelen, header_name, continuation_ws)
  File "/usr/lib/python2.7/email/header.py", line 183, in __init__
    self.append(s, charset, errors)
  File "/usr/lib/python2.7/email/header.py", line 267, in append
    ustr = unicode(s, incodec, errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe4 in position 11:
invalid continuation byte


- Eva Isaksson




2016-01-05 21:29 GMT+02:00 Mark Sapiro <mark at msapiro.net>:

> On 01/05/2016 10:42 AM, Mark Sapiro wrote:
> > On 01/05/2016 04:43 AM, Eva Isaksson wrote:
> >>
> >> This is what we see after installing the .deb:
> >>
> >> ```Uncaught runner exception: 'utf8' codec can't decode byte 0xe4 in
> >> position 11: invalid continuation byte```
> >
> >
> > Please post the entire traceback from Mailman's error log.
> >
> > Please keep this thread on the list.
>
>
> Although I can't say for sure without seeing the traceback, this may be
> similar to the situation described at
> <https://bugs.launchpad.net/mailman/+bug/1462755>.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
>