From wmertens at cisco.com  Thu Jan 17 18:29:01 2008
From: wmertens at cisco.com (Wout Mertens)
Date: Thu, 17 Jan 2008 18:29:01 +0100
Subject: [Mailman-i18n] Problem with header encoding on 2.1.9 - any ideas?
Message-ID: <2593F882-1B57-4ABF-A444-0508FD2C6BB2@cisco.com>

Hi there,

I have a problem with MailMan and Japanese ISO-2022-JP encoding.

When a header includes a ";" as part of the ISO-2022-JP encoding,  
MailMan seems to replace it with "; " (note the extra space). This  
messes up the characters.

Real-life example:

Original:
Subject: =?ISO-2022-JP?Q?607716139:_=1B$B%a%C%;!<%8%m%1!< 
%=3FF0:nIT6q9g=1B(B?=

Mailman-sent:
Subject: =?ISO-2022-JP?Q?607716139:_=1B$B%a%C%; !<%8%m%1!< 
%=3FF0:nIT6q9g=1B(B?=


I tried looking at the code, but I couldn't figure it out and so I'm  
asking here first before I dig deeper. Does anybody know what might  
cause this?

Thanks!

Wout.

From wmertens at cisco.com  Fri Jan 18 14:50:17 2008
From: wmertens at cisco.com (Wout Mertens)
Date: Fri, 18 Jan 2008 14:50:17 +0100
Subject: [Mailman-i18n] Problem with header encoding on 2.1.9 - any
	ideas?
In-Reply-To: <478FF575.7080005@is.kochi-u.ac.jp>
References: <2593F882-1B57-4ABF-A444-0508FD2C6BB2@cisco.com>
	<478FF575.7080005@is.kochi-u.ac.jp>
Message-ID: <9CB6DF87-F212-485B-982A-4E9B50508763@cisco.com>

Hello Tokio,

many thanks for your swift reply!

Looking further at this, I also found a header that wasn't impacted by  
this, but it wasn't encoded either. This header is the same in both  
original and mailman-processed mails:

X-IronPort-AV: E=Sophos;i="4.24,292,1196668800";
    d="scan'208";a="8300346"

note the second line, there the ";" is without a space.

So it seems to me that MailMan only changes certain headers? Do you  
think there's an easy way to configure MailMan to leave the Subject  
line alone?

Thanks,

Wout.

On Jan 18, 2008, at 1:40 AM, Tokio Kikuchi wrote:

> Wout Mertens wrote:
>> Hi there,
>
> Hi,
>> I have a problem with MailMan and Japanese ISO-2022-JP encoding.
>> When a header includes a ";" as part of the ISO-2022-JP encoding,   
>> MailMan seems to replace it with "; " (note the extra space). This   
>> messes up the characters.
>> Real-life example:
>> Original:
>> Subject: =?ISO-2022-JP?Q?607716139:_=1B$B%a%C%;!<%8%m%1!<  
>> %=3FF0:nIT6q9g=1B(B?=
>> Mailman-sent:
>> Subject: =?ISO-2022-JP?Q?607716139:_=1B$B%a%C%; !<%8%m%1!<  
>> %=3FF0:nIT6q9g=1B(B?=
>
> This is because the python email package can't distinguish between  
> structured and un-structured RFC2822 headers.  The Q-encoded  
> iso-2022-jp string contains ';' character which cause the email  
> package to think it is a syntactic separator, thus insert a space.   
> Most Japanese capable mailers use B-encoding to avoid such confusion.
>
> Workaround is rather tricky but try add a subject_prefix like  
> [listname] on the admin interface which may trigger normalization by  
> the Mailman CookHeader module.
>
>> I tried looking at the code, but I couldn't figure it out and so  
>> I'm  asking here first before I dig deeper. Does anybody know what  
>> might  cause this?
>> Thanks!
>> Wout.
>
> Cheers,
>
> -- 
> Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
> http://weather.is.kochi-u.ac.jp/