[Mailman-Users] mailman uses wrong encoding, /etc/mailman/[language] not generated

[Mark Sapiro]

On 04/24/2017 11:11 AM, Johannes Rohr wrote:
> 
> The default language for most mailing lists is German. Since the migration,
> mailman serves all pages and mail notifications in latin-1 instead of
> utf-8, which the header still decleares the content to be UTF-8, which
> leads to broken display, see eg
> https://lists.uferwerk.org/cgi-bin/mailman/listinfo/


First see my comments in your bug report at
<https://bugs.launchpad.net/ubuntu/+source/mailman/+bug/1685946/comments/1>


> I noticed that the subdirectory for the English templates /etc/mailman/en
> exists, but there is no /etc/mailman/de
> 
> On Debian/Ubuntu, /var/lib/mailman/Templates  is a symlink to /etc/mailman,
> which by default is owned by root:root. I changed that to root:list and
> restarted mailman, in the hope that /etc/mailman/de would eventually be
> generated, but it wasn't.


The ownership of symlinks is irrelevant. Only the ownership (and mode)
of the target is significant.

In any case, the upstream source of the German Message catalog is
messages/de/LC_MESSAGES/mailman.po and it is latin-1 encoded. The Debian
91_utf8.patch which I mention in the above referenced comment creates a
utf-8 encoded version of that in
messages/de/LC_MESSAGES/mailman.po.utf-8 and compiles that into
messages/de/LC_MESSAGES/mailman.mo. All of that takes place in the
source build directory which is not the installed directory distributed
with the package. The installed directory should end up with a
mailman.po and mailman.mo, both utf-8 encoded.


> What is the mechanism, by which the templates directory for the respective
> language is generated? Could this be a permission issue, and could the
> mismatch between the charset declared in the header of the message or
> website and the actual charset in the content be somehow related to that?


You need to look at the messages/de/LC_MESSAGES/mailman.po file you
have. Near the beginning will be a line like

"Content-Type: text/plain; charset=xxx\n"

where xxx is either ISO-8859-1 or UTF-8. If it is UTF-8 and the 'msgstr'
values are utf-8 encoded, you can rebuild the .mo with

cd messages/de/LC_MESSAGES/
msgfmt -o mailman.mo mailman.po

but I suspect that is not the case. I suspect that somehow in your
migration process you replaced the Debian utf-8 encoded files with older
iso-8859-1 files. In that case you need to convert the .po to utf-8
before compiling with something like

msgconv -t utf-8 mailman.po | msgfmt -o mailman.mo -

or if you want to save the utf-8 encoded .po you could do something like

msgconv -t utf-8 -o mailman.po.utf-8 mailman.po
msgfmt -o mailman.mo mailman.po.utf-8

Note that in either of the above the msgfmt command will produce a few
"fatal" errors of the form

mailman.po:7051: 'msgid' and 'msgstr' entries do not both end with '\n'

but the output should be OK. To avoid this you can use the
var/mailman/bin/msgfmt.py command instead.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan