[Mailman-Users] Digests stop being sent after a few days if the Mailman interface is changed to Japanese

[S. Patrick Eaton]

Hello, everyone.

I'm a long-time user of Mailman, but a first-time poster here. Please
go easy on me while I get my bearings, okay?

I am running a Mailman list for a group of translators who work with
the Japanese language (translating either into Japanese or from
Japanese into another language), and because most of the subscribers
are native speakers of either Japanese or English (although there are
speakers of other languages as well) most of the messages posted to
this list are in either of those languages, and often they mix both
languages in the same message.

I've been having a number of problems with the distribution of digest
messages that I am hoping someone here can help me with.

The first is that, unless the Mailman administrative interface is
changed to Japanese, any Japanese characters included in digest
messages get changed into question mark characters when the digest is
sent out, even if those characters appeared correctly in the message
that was sent out to non-digest subscribers. If the interface is
changed to Japanese, this does not occur; Japanese characters then
appear in the digests the way they should.

This, however, inevitably runs into another problem: When the
interface is changed to Japanese, the distribution of digests
continues for a while, but inevitably it stops at some point and does
not resume until the interface is changed back to English, at which
point all of the messages that had not been distributed in digests
since the last digest was sent get included in the next digest
message, which can be quite massive if the digests have not been
getting sent out regularly for a while.

I've looked through some of the digest-related bugs recorded at
Launchpad, and these give me the impression that this sort of problem
(if not this exact issue) is not entirely unknown to the developers,
and there may even be a solution of some sort already, but I have not
yet found it.

If my understanding of the problem is correct, it seems like some
non-standard characters that are included in some inbound messages
cause problems for the process handling the digests, which for some
reason does not recover if it encounters such problems, unless it is
somehow manually reset (such as when the interface language changes,
as described above). I know that most of the messages sent to the list
are encoded as either UTF-8 or ISO-2022-JP, but I don't really know
which characters might be causing the digests to get clogged up.

I have encountered this very same problem in two Mailman
installations, one running Mailman 2.1.9 and another running Mailman
2.1.12, on two entirely separate hosted environments, so I am
reasonably sure that the problem is specific to Mailman and not to
installation or environment-related issues.

If anyone has any advice or suggestions (in either English or
Japanese!) that might help me solve this problem, I would be very glad
to hear them. Although this is the first time I have posted to this
list about this, it is a problem that has been occurring for quite
some time, so it would be a great relief to be able to get it
resolved.

With many thanks in advance for your time and consideration,

Patrick Eaton