From simon.hanna at serve-me.info Tue Jan 10 19:32:55 2017 From: simon.hanna at serve-me.info (Simon Hanna) Date: Wed, 11 Jan 2017 01:32:55 +0100 Subject: [Mailman-i18n] [Mailman-Developers] Translation of Mailman 3 In-Reply-To: <20170109215655.494c5090@subdivisions.wooz.org> References: <570D766A.9040509@serve-me.info> <570D7892.3000207@msapiro.net> <20160420095815.0f175625@subdivisions.wooz.org> <22296.16208.742523.449355@turnbull.sk.tsukuba.ac.jp> <20160517100630.5f815773.barry@wooz.org> <7b9c8001-d4bc-e232-be2a-7e6c702f42bb@serve-me.info> <20170109215655.494c5090@subdivisions.wooz.org> Message-ID: <7c5ac727-fce0-5d81-14e7-6b73ddc46dd7@serve-me.info> On 01/10/2017 03:56 AM, Barry Warsaw wrote: > Reviving an ancient thread. > > On May 28, 2016, at 10:27 PM, Simon Hanna wrote: > >> I used Zanata, but found it not very intuitive to use. It's also >> painfully slow. >> >> I'm unaware of any big projects that use zanata. Well openstack does, but >> they use a self hosted version and that one is not faster. I'm not sure how >> Mailman 2 was translated, but I guess most of the translators did it >> offline. You can download translation files from pootle and later upload >> them. So anyone that doesn't want to translate in the browser, can still do >> it offline. > Since gettext will be the interchange format, it will probably not be that > difficult to switch to a different service if we ever find we need to. > > I appreciate your feedback on Zanata, and honestly we just need an i18n > champion to make it happen. My apologies for such a long delay in responding > here, but Simon, if you're still willing to take the lead on i18n, I will be > happy to defer to your preferences. > >> So If you give me the ok, I write the gnu pootle maintainers and ask >> them to create three projects for us. > +1 - if you're still willing, let's do this. Core is very nearly ready to > start rc'ing for 3.1 so I think this is a great time to being building the > infrastructure for i18n. > >> I guess we could add links in postorius and hyperkitty that request >> assistance with translation. > +1 So it looks like the gnu pootle server is not up anymore. So I guess Zanata is the way to go. The projects are already in place, we would just have to update the pot files I initially uploaded. I updated Hyperkitty and Postorius since I'm familiar with django translation. You can start translating them right aways over at: https://translate.zanata.org/iteration/view/postorius/1.0.3 https://translate.zanata.org/iteration/view/hyperkitty/1.0.3 Mailman on the other hand currently doesn't have any infrastructure, and I don't think I know enough to be sure that I get all the strings in the correct format. Can someone help with that? I'll start working on a wiki page and create merge requests for Hyperkitty and Postorius that include a link to the wiki asking for help. I'll check the other projects and find out if we need to create translation projects for them too. All registered users should be able to translate. I'll enable reviews which allow certain users more rights. If you want to be responsible for a language you should be able to request that from the interface or you can contact me mentioning your username. cheers, Simon From mark at mailmanlists.net Tue Jan 17 16:05:53 2017 From: mark at mailmanlists.net (Mark Dale) Date: Wed, 18 Jan 2017 08:05:53 +1100 Subject: [Mailman-i18n] Polish language template files Message-ID: Hi Mark, I've been in touch with the Polish Language maintainer (Stefan Plewako) regarding the non-ascii characters that I am seeing. In the language template files for Polish (Mailman 2.1.23) I see that the non-ascii characters (letters with diacritics) are replaced with question marks. These question marks get displayed in the Mailman web pages for lists using Polish. Stefan directed me to GitHub for the Polish language files and in those I files I can see the Polish letters okay. (letters with diacritics) I loaded Stefan's files into Mailman (replacing the existing) and all is now well. I was surprised as I was thinking that the non-ascii characters would need to be replaced with HTML entities - as you had done for the Hungarian files a couple of months ago. Stefan had advised me that doing that shouldn't be needed, and it seems he might be correct. Is there a mismatch with the language files that got shipped in the Mailman language files and Stefan's files on GitHub - or am I just barking up the wrong tree? I'm seeing the same issue for other languages, and I'm happy to chase down files for those if that is what is needed (assuming they have been created). Czech Greek Estonia Finnish Croation --Hungarian-- (discussed) Japanese Korean Lithuanian Polish Portuguese Slovenian Turkish I have close to zero understanding of the subject of character sets, but I include this reference below in case it has any bearing on things. https://mail.python.org/pipermail/mailman-i18n/2015-February/001854.html Regards, Mark Dale ***** SNIP from Stefan's GitHub files ***** Wiadomo?ci do wszystkich prenumerator?w listy wysy?aj na adres: .

Mo?esz zapisa? si? na list? lub zmieni? opcje prenumeraty korzystaj?c z poni?szych sekcji. ***** SNIP from mailman-2.1.23/templates/pl/listinfo.html ***** Wiadomo?ci do wszystkich prenumeratorów listy wysy?aj na adres: .

Mo?esz zapisa? si? na list? lub zmieni? opcje prenumeraty korzystaj?c z poni?szych sekcji. ======================================== From mark at msapiro.net Tue Jan 17 18:56:40 2017 From: mark at msapiro.net (Mark Sapiro) Date: Tue, 17 Jan 2017 15:56:40 -0800 Subject: [Mailman-i18n] Polish language template files In-Reply-To: References: Message-ID: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> On 01/17/2017 01:05 PM, Mark Dale wrote: > Hi Mark, > > I've been in touch with the Polish Language maintainer (Stefan Plewako) > regarding the non-ascii characters that I am seeing. > > In the language template files for Polish (Mailman 2.1.23) I see that > the non-ascii characters (letters with diacritics) are replaced with > question marks. These question marks get displayed in the Mailman web > pages for lists using Polish. > > Stefan directed me to GitHub for the Polish language files and in those > I files I can see the Polish letters okay. (letters with diacritics) > > I loaded Stefan's files into Mailman (replacing the existing) and all is > now well. I was surprised as I was thinking that the non-ascii > characters would need to be replaced with HTML entities - as you had > done for the Hungarian files a couple of months ago. Stefan had advised > me that doing that shouldn't be needed, and it seems he might be correct. I can easily convert all the non-ascii in the Polish language templates to html entities which is the correct way to deal with this. I have been reluctant to do this for certain languages in the past because of the sheer numbers of html entities involved, essentially every character in Greek for example. Polish is not so bad, but the majority of non-ascii characters have only numeric html entities. For example, the snippet you quote becomes > > Wiadomości do wszystkich prenumeratorów listy wysyłaj na adres: > . > >

Możesz zapisać się na listę lub zmienić op > cje prenumeraty korzystając z poniższych sekcji. > which will render correctly in a browser that recognizes those entities but is no more readable to humans in other contexts than the ? characters are. The underlying issue here is Mailman's character set for Polish is iso-8859-2. Mailman sends those web pages built from those templates with a Content-Type: text/html; charset=iso-8859-2 header, but some web servers are configured to override that. E.g., see for a description of the Apache directive. Stefan's templates are UTF-8 encoded and the html templates will work in an environment where the web server 'forces' utf-8, but the .txt templates if utf-8 encoded will break in a Mailman whose character set for Polish is still iso-8859-2, because they will be sent in email with Content-Type: text/plain; charset=iso-8859-2 but with utf-8 encoded characters. The ultimate solution is to make everything utf-8 encoded. Individual sites can do this, but I can't for the reasons discussed at . Also see see the thread "Encoding problem with 2.15 to 2.18 upgrade with Finnish" beginning at and continuing at for some of the fallout after Debian arbitrarily changed the character set for several languages to utf-8 in their Mailman package. Bottom line is I have converted the Polish html templates to use html entities at and will install those at mail.python.org with the intent of releasing that with 2.1.24. It should be OK, but if I get pushback from the Polish lists on mpo, I may have to reverse. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From splewako at aviary.pl Wed Jan 18 08:45:30 2017 From: splewako at aviary.pl (Stefan Plewako) Date: Wed, 18 Jan 2017 14:45:30 +0100 Subject: [Mailman-i18n] Polish language template files In-Reply-To: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> Message-ID: <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> > Wiadomo?? napisana przez Mark Sapiro w dniu 18.01.2017, o godz. 00:56: > > Bottom line is I have converted the Polish html templates to use html > entities at > > and will install those at mail.python.org with the intent of releasing > that with 2.1.24. It should be OK, but if I get pushback from the Polish > lists on mpo, I may have to reverse. Wow! Even if this change may not break anything I'm not sure that it solves any problem and quite frankly, I don't like seeing changes being made this way? stef From mark at msapiro.net Wed Jan 18 12:45:04 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Jan 2017 09:45:04 -0800 Subject: [Mailman-i18n] Polish language template files In-Reply-To: <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> Message-ID: <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> On 01/18/2017 05:45 AM, Stefan Plewako wrote: > > Even if this change may not break anything I'm not sure that it solves any problem It solves the problem of garbled characters in the web UI in installations that maintain Mailman's character set for Polish as iso-8859-2 but have web servers that force utf-8 or other non-iso-8859-2 charactersets > and quite frankly, I don't like seeing changes being made this way? I don't understand. Can you elaborate on what exactly you don't like and what you would prefer? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From splewako at aviary.pl Wed Jan 18 13:21:34 2017 From: splewako at aviary.pl (Stefan Plewako) Date: Wed, 18 Jan 2017 19:21:34 +0100 Subject: [Mailman-i18n] Polish language template files In-Reply-To: <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> Message-ID: > Wiadomo?? napisana przez Mark Sapiro w dniu 18.01.2017, o godz. 18:45: > > It solves the problem of garbled characters in the web UI in > installations that maintain Mailman's character set for Polish as > iso-8859-2 but have web servers that force utf-8 or other non-iso-8859-2 > charactersets Is it? Do I understand correctly that Mailman will on the fly detect encoding enforced by web server and convert texts provided for placeholders (also content?) in changed templates to that enforced encoding? I'm sure there are other good questions. > I don't understand. Can you elaborate on what exactly you don't like and > what you would prefer? I don't like rushed, not exactly understood changes to work of others without even attempting to consult them. From mark at msapiro.net Wed Jan 18 14:36:21 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Jan 2017 11:36:21 -0800 Subject: [Mailman-i18n] Polish language template files In-Reply-To: References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> Message-ID: <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> On 01/18/2017 10:21 AM, Stefan Plewako wrote: >> Wiadomo?? napisana przez Mark Sapiro w dniu 18.01.2017, o godz. 18:45: >> >> It solves the problem of garbled characters in the web UI in >> installations that maintain Mailman's character set for Polish as >> iso-8859-2 but have web servers that force utf-8 or other non-iso-8859-2 >> charactersets > > Is it? Do I understand correctly that Mailman will on the fly detect encoding enforced by web server and convert texts provided for placeholders (also content?) in changed templates to that enforced encoding? I'm sure there are other good questions. No. Mailman does not detect the encoding enforced by the web server. Mailman builds pages from the templates as provided and sends them with a Content-Type header specifying the encoding as Mailman's configured character set for the language. The problem occurs when the web server tries to enforce a different character encoding causing the encoding specified to the browser to not match that of the page. >> I don't understand. Can you elaborate on what exactly you don't like and >> what you would prefer? > > I don't like rushed, not exactly understood changes to work of others without even attempting to consult them. I have explained in the past my reasons for not accepting your utf-8 encoded templates and message catalog as is and have always recoded them to iso-8859-2. You have been informed of this. See, e.g., . What I have done here is only an extension of that recoding which replaces iso-8859-2 encoded characters in html templates with equivalent html entities. Please understand that I appreciate your work in keeping the Polish translation up to date and I hope you continue, but there are various technical problems that can occur that would be disruptive to existing Polish language lists if I just arbitrarily switched Mailman's encoding for Polish to utf-8. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From splewako at aviary.pl Wed Jan 18 15:39:01 2017 From: splewako at aviary.pl (Stefan Plewako) Date: Wed, 18 Jan 2017 21:39:01 +0100 Subject: [Mailman-i18n] Polish language template files In-Reply-To: <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> Message-ID: > Wiadomo?? napisana przez Mark Sapiro w dniu 18.01.2017, o godz. 20:36: > > On 01/18/2017 10:21 AM, Stefan Plewako wrote: >>> Wiadomo?? napisana przez Mark Sapiro w dniu 18.01.2017, o godz. 18:45: >>> >>> It solves the problem of garbled characters in the web UI in >>> installations that maintain Mailman's character set for Polish as >>> iso-8859-2 but have web servers that force utf-8 or other non-iso-8859-2 >>> charactersets >> >> Is it? Do I understand correctly that Mailman will on the fly detect encoding enforced by web server and convert texts provided for placeholders (also content?) in changed templates to that enforced encoding? I'm sure there are other good questions. > > No. Mailman does not detect the encoding enforced by the web server. > Mailman builds pages from the templates as provided and sends them with > a Content-Type header specifying the encoding as Mailman's configured > character set for the language. The problem occurs when the web server > tries to enforce a different character encoding causing the encoding > specified to the browser to not match that of the page. OK, so the change doesn't solve any problem that I know off and especially not the problem of garbled characters in the web UI. In fact it hides the real problem making discovering it harder (not much but still) as some (touched) characters will display correctly. >>> I don't understand. Can you elaborate on what exactly you don't like and >>> what you would prefer? >> >> I don't like rushed, not exactly understood changes to work of others without even attempting to consult them. > > I have explained in the past my reasons for not accepting your utf-8 > encoded templates and message catalog as is and have always recoded them > to iso-8859-2. You have been informed of this. See, e.g., > . Yes and I have no idea how that relates to the current topic. > What I have done here is only an extension of that recoding which > replaces iso-8859-2 encoded characters in html templates with equivalent > html entities. In my opinion you fail to accept that in best case you didn't change nothing visibly, in worst case you introduced new bugs. > Please understand that I appreciate your work in keeping the Polish > translation up to date and I hope you continue, but there are various > technical problems that can occur that would be disruptive to existing > Polish language lists if I just arbitrarily switched Mailman's encoding > for Polish to utf-8. Quite frankly I doubt that and please stop suggesting that I requested switching default encoding for Polish to UTF-8 again (*after* the mentioned discussion in early 2015). From mark at msapiro.net Wed Jan 18 16:46:54 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Jan 2017 13:46:54 -0800 Subject: [Mailman-i18n] Polish language template files In-Reply-To: References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> Message-ID: On 01/18/2017 12:39 PM, Stefan Plewako wrote: > > OK, so the change doesn't solve any problem that I know off and especially not the problem of garbled characters in the web UI. In fact it hides the real problem making discovering it harder (not much but still) as some (touched) characters will display correctly. Perhaps I don't understand, but the OP in this thread had a problem with garbled characters in the web UI of a Polish language list which he said was solved by installing your utf-8 encoded templates. This says to me that he has a Mailman installation where the character set for Polish is iso-8859-2, and his web server was overriding that and forcing utf-8 thus sending iso-8859-2 encoded characters to the browser and claiming they were utf-8. This issue can be fixed in general by encoding the problem characters as html entities which will render properly in the browser regardless of the encoding of the page. That is the change I made. Granted, there may still be problems with characters from strings in the message catalog or in list attributes, but these won't be fixed by encoding the template as utf-8 either. >> What I have done here is only an extension of that recoding which >> replaces iso-8859-2 encoded characters in html templates with equivalent >> html entities. > > In my opinion you fail to accept that in best case you didn't change nothing visibly, in worst case you introduced new bugs. I think in the best case, I have addressed the issue described above, and if I have introduced new bugs, I hope they will be reported and I can either fix them or reverse my change. >> Please understand that I appreciate your work in keeping the Polish >> translation up to date and I hope you continue, but there are various >> technical problems that can occur that would be disruptive to existing >> Polish language lists if I just arbitrarily switched Mailman's encoding >> for Polish to utf-8. > > Quite frankly I doubt that and please stop suggesting that I requested switching default encoding for Polish to UTF-8 again (*after* the mentioned discussion in early 2015). I'm sorry you doubt that and I'm sorry that I implied you were requesting switching the default encoding for Polish to UTF-8. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From splewako at aviary.pl Wed Jan 18 17:20:51 2017 From: splewako at aviary.pl (Stefan Plewako) Date: Wed, 18 Jan 2017 23:20:51 +0100 Subject: [Mailman-i18n] Polish language template files In-Reply-To: References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> Message-ID: > Wiadomo?? napisana przez Mark Sapiro w dniu 18.01.2017, o godz. 22:46: > > On 01/18/2017 12:39 PM, Stefan Plewako wrote: >> >> OK, so the change doesn't solve any problem that I know off and especially not the problem of garbled characters in the web UI. In fact it hides the real problem making discovering it harder (not much but still) as some (touched) characters will display correctly. > > Perhaps I don't understand, but the OP in this thread had a problem with > garbled characters in the web UI of a Polish language list which he said > was solved by installing your utf-8 encoded templates. After reading first post in the thread, it is not clear to me what exactly was replaced and if things were sufficiently tested after. Using files (not only templates) from https://github.com/aviarypl/mailman-l10n-pl is one way to display things correctly, using standard files with ISO-8859-2 web server encoding is another (official). I don't believe that such post alone should be sufficient to change anything. > This issue can be fixed in general by encoding the problem characters as > html entities which will render properly in the browser regardless of > the encoding of the page. That is the change I made. > > Granted, there may still be problems with characters from strings in the > message catalog or in list attributes, but these won't be fixed by > encoding the template as utf-8 either. You did change only part of the things needed and can't really change the rest - the change doesn't make real (positive) difference. > I think in the best case, I have addressed the issue described above No, you didn't. We have non-ASCII characters in strings that replace placeholders in changed templates. From mark at mailmanlists.net Wed Jan 18 17:50:16 2017 From: mark at mailmanlists.net (Mark Dale) Date: Thu, 19 Jan 2017 09:50:16 +1100 Subject: [Mailman-i18n] Polish language template files In-Reply-To: References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> Message-ID: <7d8bf93d-57dc-14ca-e1b4-88a2f7c6345b@mailmanlists.net> At the risk of adding uninformed comments into this discussion, I have installed and tested using the Polish language files in: 1. Mailman 2.1.23 release 2. Stefan's files 2. Mark's files The test was to view what a user would see when they subscribed to list that was using Polish - on the webpages and the email they received. The server is Debian running Nginx, with Postfix and Mailman 2.1.23. The email used to view the "Subscribed" emails was webmail and desktop "IceDove". When using the Mailman 2.1.23 release language files, the "question marks" appear in both the webpages and the emails. When using Stefan's files, both the webpages and emails display correctly, showing the Polish letters with their diacritics. When using Mark's files, the webpages display correctly, but the emails show the "question marks" in place of the Polish letters. I thank you both for delving into this issue and I apologise if my remarks above add nothing to the conversation. I'm way out of my depth on the subject. Regards, Mark > Perhaps I don't understand, but the OP in this thread had a problem with > garbled characters in the web UI of a Polish language list which he said > was solved by installing your utf-8 encoded templates. > This says to me that he has a Mailman installation where the character > set for Polish is iso-8859-2, and his web server was overriding that and > forcing utf-8 thus sending iso-8859-2 encoded characters to the browser > and claiming they were utf-8. From mark at msapiro.net Wed Jan 18 18:31:52 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Jan 2017 15:31:52 -0800 Subject: [Mailman-i18n] Polish language template files In-Reply-To: <7d8bf93d-57dc-14ca-e1b4-88a2f7c6345b@mailmanlists.net> References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> <7d8bf93d-57dc-14ca-e1b4-88a2f7c6345b@mailmanlists.net> Message-ID: On 01/18/2017 02:50 PM, Mark Dale wrote: > At the risk of adding uninformed comments into this discussion, I have > installed and tested using the Polish language files in: > > 1. Mailman 2.1.23 release > 2. Stefan's files > 2. Mark's files > > The test was to view what a user would see when they subscribed to list > that was using Polish - on the webpages and the email they received. Thank you for testing and reporting. I think your report is helpful. > The server is Debian running Nginx, with Postfix and Mailman 2.1.23. > > The email used to view the "Subscribed" emails was webmail and desktop > "IceDove". > > When using the Mailman 2.1.23 release language files, the "question > marks" appear in both the webpages and the emails. I am surprised that there are issues with the emails. The emails should be sent with RFC 2047 encoded subjects similar to Subject: =?iso-8859-2?q?Witaj_na_li=B6cie_=22List1=22_?= which renders as Subject: Witaj na li?cie "List1" and with the header Content-Type: text/plain; charset="iso-8859-2" which declares the body character set to be iso-8859-2 > When using Stefan's files, both the webpages and emails display > correctly, showing the Polish letters with their diacritics. And in this case, with utf-8 encoded email templates, the utf-8 characters should be garbled because of the charset="iso-8859-2" in the Content-Type: header. > When using Mark's files, the webpages display correctly, but the emails > show the "question marks" in place of the Polish letters. I didn't change any of the templates used in emails. Since they aren't html, html entities don't work for them. So, the change that I made replacing non-ascii iso-8859-2 characters in the html templates improves the display of those web pages which is exactly what it was intended to do. What I don't understand is why the emails are garbled with iso-8859-2 templates and not with utf-8 templates unless you changed Mailman's character set for Polish to utf-8. If in fact you did that, then you do need utf-8 encoded templates and message catalog for Polish. So the question remains, what is the character set in the definition add_language('pl', _('Polish'), 'iso-8859-2', 'ltr') in Defaults.py or overridden in mm_cfg.py. If it is 'iso-8859-2' as shown and not 'utf-8' I am at a loss to understand why your emails are garbled with iso-8859-2 templates and not with utf-8 templates. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at mailmanlists.net Wed Jan 18 19:18:28 2017 From: mark at mailmanlists.net (Mark Dale) Date: Thu, 19 Jan 2017 11:18:28 +1100 Subject: [Mailman-i18n] Polish language template files In-Reply-To: References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> <7d8bf93d-57dc-14ca-e1b4-88a2f7c6345b@mailmanlists.net> Message-ID: <53fdb205-8220-957f-13fd-703181f1b376@mailmanlists.net> I checked on two servers where I have tested (both with the same setup of Debian, Nginx, etc) and gotten the same results. I haven't made changes any language character settings on either. Both are out-of-the box regarding that. In Defaults.py for both, I see: add_language('pl', _('Polish'), 'iso-8859-2', 'ltr') add_language('pl', _('Polish'), 'iso-8859-2', 'ltr') And in mm_cfg.py, the default language is "en" on both, and there are no over-rides. If there is any testing I can do to that might contribute to some understanding, I'm happy to do so. On 19/01/17 10:31, Mark Sapiro wrote: > What I don't understand is why the emails are garbled with iso-8859-2 > templates and not with utf-8 templates unless you changed Mailman's > character set for Polish to utf-8. If in fact you did that, then you do > need utf-8 encoded templates and message catalog for Polish. > > So the question remains, what is the character set in the definition > > add_language('pl', _('Polish'), 'iso-8859-2', 'ltr') > > in Defaults.py or overridden in mm_cfg.py. If it is 'iso-8859-2' as > shown and not 'utf-8' I am at a loss to understand why your emails are > garbled with iso-8859-2 templates and not with utf-8 templates. > From mark at msapiro.net Wed Jan 18 19:41:58 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Jan 2017 16:41:58 -0800 Subject: [Mailman-i18n] Polish language template files In-Reply-To: <53fdb205-8220-957f-13fd-703181f1b376@mailmanlists.net> References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> <7d8bf93d-57dc-14ca-e1b4-88a2f7c6345b@mailmanlists.net> <53fdb205-8220-957f-13fd-703181f1b376@mailmanlists.net> Message-ID: <8c8e0766-ee66-98c8-1373-44b4dca2738c@msapiro.net> On 01/18/2017 04:18 PM, Mark Dale wrote: > > In Defaults.py for both, I see: > > add_language('pl', _('Polish'), 'iso-8859-2', 'ltr') > add_language('pl', _('Polish'), 'iso-8859-2', 'ltr') > > And in mm_cfg.py, the default language is "en" on both, and there are no > over-rides. > > If there is any testing I can do to that might contribute to some > understanding, I'm happy to do so. What are the headers from a list welcome or other similar message from a Polish language list? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at mailmanlists.net Wed Jan 18 19:58:47 2017 From: mark at mailmanlists.net (Mark Dale) Date: Thu, 19 Jan 2017 11:58:47 +1100 Subject: [Mailman-i18n] Polish language template files In-Reply-To: <8c8e0766-ee66-98c8-1373-44b4dca2738c@msapiro.net> References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> <7d8bf93d-57dc-14ca-e1b4-88a2f7c6345b@mailmanlists.net> <53fdb205-8220-957f-13fd-703181f1b376@mailmanlists.net> <8c8e0766-ee66-98c8-1373-44b4dca2738c@msapiro.net> Message-ID: On 19/01/17 11:41, Mark Sapiro wrote: > > What are the headers from a list welcome or other similar message from a > Polish language list? > On the list-welcome emails the content-type in the header is: Content-Type: text/plain; charset="utf-8" It is the same for both the emails, the ones that display question marks and the one that displays the Polish letters. The full headers are: *************************************** **** EMAIL SHOWING QUESTION MARKS ***** *************************************** >From - Thu Jan 19 08:49:02 2017 X-Account-Key: account14 X-UIDL: 1484776110.24176.takara,S=4707 X-Mozilla-Status: 0001 X-Mozilla-Status2: 00000000 X-Mozilla-Keys: >From singapore-bounces at mailmanlists.sg Wed Jan 18 22:48:30 2017 Return-path: Received: from [10.9.9.210] (helo=mailfront10.runbox.com) by takara.runbox.com with esmtp (Exim 4.69) id 1cTy5m-0006Hs-CK for mark%mailmanlists.net at runbox.com; Wed, 18 Jan 2017 22:48:30 +0100 Received: from exim by mailfront10.runbox.com with dspam-scanned (Exim 4.82) id 1cTy5k-0004cF-2H for mark%mailmanlists.net at runbox.com; Wed, 18 Jan 2017 22:48:30 +0100 Received: from exim by mailfront10.runbox.com with sa-scanned (Exim 4.82) id 1cTy5j-0004bx-LZ for mark%mailmanlists.net at runbox.com; Wed, 18 Jan 2017 22:48:28 +0100 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on antispam02.runbox.com X-Spam-Level: X-Spam-Status: No, score=0.1 required=4.0 tests=DKIM_SIGNED,T_DKIM_INVALID shortcircuit=no autolearn=disabled version=3.4.1 Received: from mailmanlists.sg ([128.199.127.214] helo=www.mailmanlists.sg) by mailfront10.runbox.com with esmtp (Exim 4.82) id 1cTy4m-0003rH-Jv for mark at mailmanlists.net; Wed, 18 Jan 2017 22:47:29 +0100 Received: from [127.0.1.1] (localhost [127.0.0.1]) by www.mailmanlists.sg (Postfix) with ESMTP id 44E9C5FECA for ; Thu, 19 Jan 2017 05:47:27 +0800 (SGT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mailmanlists.sg; s=mail; t=1484776047; bh=vTkBfI6zV9V+cQAIFmIAjSkXSmNYsyevuuwu6J7GWeA=; h=Subject:From:To:Date:List-Id:From; b=k30Je9DQdskC8gJbCs6jDmenwwb685FOPNikNufnONzJaDxILEuNSAbsINpm2LK4p Ct+zlNK+fGcf6JoDgP5QvJWvh8dCHYH7mE4YppyUAbZABusunRwEcM3V4ktRsjbMDt jqvHPUIaw1a4yDGj4gixH/ywWOloXXv7jUyBrlL0= MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Subject: =?utf-8?q?Witaj_na_li=C5=9Bcie_=22Singapore=22_?= From: singapore-request at mailmanlists.sg To: mark at mailmanlists.net X-No-Archive: yes Message-ID: Date: Thu, 19 Jan 2017 05:47:26 +0800 Precedence: bulk X-BeenThere: singapore at mailmanlists.sg X-Mailman-Version: 2.1.23 List-Id: X-List-Administrivia: yes Errors-To: singapore-bounces at mailmanlists.sg X-FILTER-DSPAM: by mailfront10.runbox.com X-DSPAM-Factors: 27, To*mailmanlists+net, 0.00010, To*mark+mailmanlists, 0.00010, X-FILTER-DSPAM*runbox+com, 0.00010, Content-Type*text, 0.00010, X-FILTER-DSPAM*com, 0.00010, X-FILTER-DSPAM*by, 0.00010, X-FILTER-DSPAM*runbox, 0.00010, Content-Type*charset, 0.00010, To*mailmanlists, 0.00010, To*mark, 0.00010, To*net, 0.00010, mailmanlists, 0.00020, mailmanlists, 0.00020, do, 0.00020, do, 0.00020, www, 0.00020, www, 0.00020, X-FILTER-DSPAM*mailfront10+runbox, 0.00020, X-FILTER-DSPAM*by+mailfront10, 0.00020, Content-Type*charset+utf, 0.00020, DKIM-Signature*v1a+rsa, 0.00020, Content-Type*plain+charset, 0.00020, Content-Type*text+plain, 0.00020, X-FILTER-DSPAM*mailfront10, 0.00020, DKIM-Signature*To, 0.00020, Date*Jan, 0.00020, net, 0.00020 X-DSPAM-Result: Innocent X-DSPAM-Confidence: 0.9997 X-DSPAM-Probability: 0.0000 V2l0YW15IG5hIGxptmNpZSBTaW5nYXBvcmVAbWFpbG1hbmxpc3RzLnNnIQoKV2lhZG9tb7ZjaSBz *************************************** **** EMAIL SHOWING POLISH LETTERS ***** *************************************** >From - Thu Jan 19 09:09:43 2017 X-Account-Key: account14 X-UIDL: 1484777340.16071.takara,S=4770 X-Mozilla-Status: 0001 X-Mozilla-Status2: 00000000 X-Mozilla-Keys: >From singapore-bounces at mailmanlists.sg Wed Jan 18 23:09:00 2017 Return-path: Received: from [10.9.9.211] (helo=mailfront11.runbox.com) by takara.runbox.com with esmtp (Exim 4.69) id 1cTyPc-0004B9-Hq for mark%mailmanlists.net at runbox.com; Wed, 18 Jan 2017 23:09:00 +0100 Received: from exim by mailfront11.runbox.com with dspam-scanned (Exim 4.82) id 1cTyPb-0005m8-4n for mark%mailmanlists.net at runbox.com; Wed, 18 Jan 2017 23:09:00 +0100 Received: from exim by mailfront11.runbox.com with sa-scanned (Exim 4.82) id 1cTyPa-0005lx-Th for mark%mailmanlists.net at runbox.com; Wed, 18 Jan 2017 23:08:59 +0100 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on antispam03.runbox.com X-Spam-Level: X-Spam-Status: No, score=0.1 required=4.0 tests=DKIM_SIGNED,T_DKIM_INVALID shortcircuit=no autolearn=disabled version=3.4.1 Received: from mailmanlists.sg ([128.199.127.214] helo=www.mailmanlists.sg) by mailfront11.runbox.com with esmtp (Exim 4.82) id 1cTyPH-0005dk-DU for mark at mailmanlists.net; Wed, 18 Jan 2017 23:08:40 +0100 Received: from [127.0.1.1] (localhost [127.0.0.1]) by www.mailmanlists.sg (Postfix) with ESMTP id 230A35FF5E for ; Thu, 19 Jan 2017 06:08:38 +0800 (SGT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mailmanlists.sg; s=mail; t=1484777318; bh=ifwoNAeBj28OXU2B9hF6MqUDfqG3tN5O4yAkQUui8xk=; h=Subject:From:To:Date:List-Id:From; b=JHM4CNQgodqdYkj3hHmuOlkNYAjKvJ+rEDIJefamsytT2VWgpbcLklOf2gbD2yEzp 0TswmQcSsmGVGEJxKNUwydvJrhqg50kbmLFjSiz9Xsljus5J/v4EZ+B7a2sRS0TRA7 8IZU6oWFSRuy/KYq7wtz5lU5fB/cDi/R1tRs9mPU= MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Subject: =?utf-8?q?Witaj_na_li=C5=9Bcie_=22Singapore=22_?= From: singapore-request at mailmanlists.sg To: mark at mailmanlists.net X-No-Archive: yes Message-ID: Date: Thu, 19 Jan 2017 06:08:37 +0800 Precedence: bulk X-BeenThere: singapore at mailmanlists.sg X-Mailman-Version: 2.1.23 List-Id: X-List-Administrivia: yes Errors-To: singapore-bounces at mailmanlists.sg X-FILTER-DSPAM: by mailfront11.runbox.com X-DSPAM-Factors: 27, To*mailmanlists+net, 0.00010, To*mark+mailmanlists, 0.00010, X-FILTER-DSPAM*runbox+com, 0.00010, Content-Type*text, 0.00010, X-FILTER-DSPAM*com, 0.00010, X-FILTER-DSPAM*by, 0.00010, X-FILTER-DSPAM*runbox, 0.00010, Content-Type*charset, 0.00010, To*mailmanlists, 0.00010, To*mark, 0.00010, To*net, 0.00010, mailmanlists, 0.00020, mailmanlists, 0.00020, https+www, 0.00020, https+www, 0.00020, do, 0.00020, do, 0.00020, https, 0.00020, https, 0.00020, www, 0.00020, www, 0.00020, Content-Type*charset+utf, 0.00020, X-FILTER-DSPAM*by+mailfront11, 0.00020, DKIM-Signature*v1a+rsa, 0.00020, Content-Type*plain+charset, 0.00020, X-FILTER-DSPAM*mailfront11+runbox, 0.00020, Content-Type*text+plain, 0.00020 X-DSPAM-Result: Innocent X-DSPAM-Confidence: 0.9997 X-DSPAM-Probability: 0.0000 V2l0YW15IG5hIGxpxZtjaWUgU2luZ2Fwb3JlQG1haWxtYW5saXN0cy5zZyEKCldpYWRvbW/Fm2Np From mark at msapiro.net Wed Jan 18 20:51:45 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Jan 2017 17:51:45 -0800 Subject: [Mailman-i18n] Polish language template files In-Reply-To: References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> <7d8bf93d-57dc-14ca-e1b4-88a2f7c6345b@mailmanlists.net> <53fdb205-8220-957f-13fd-703181f1b376@mailmanlists.net> <8c8e0766-ee66-98c8-1373-44b4dca2738c@msapiro.net> Message-ID: <9bfaf119-acd4-0159-43ca-9f1f0b51f598@msapiro.net> On 01/18/2017 04:58 PM, Mark Dale wrote: > On 19/01/17 11:41, Mark Sapiro wrote: > >> >> What are the headers from a list welcome or other similar message from a >> Polish language list? >> > > > > On the list-welcome emails the content-type in the header is: > > Content-Type: text/plain; charset="utf-8" > > It is the same for both the emails, the ones that display question marks > and the one that displays the Polish letters. > > The full headers are: > > *************************************** > **** EMAIL SHOWING QUESTION MARKS ***** > *************************************** ... > Content-Type: text/plain; charset="utf-8" > Content-Transfer-Encoding: base64 > Subject: =?utf-8?q?Witaj_na_li=C5=9Bcie_=22Singapore=22_?= I'm beginning to see the light here. Your Mailman installation is broken by Debian. Recent Debian Mailman packages have changed Mailman's character set for all languages to utf-8. They have changed the definition of add_language in Defaults.py from def add_language(code, description, charset, direction='ltr'): LC_DESCRIPTIONS[code] = (description, charset, direction) to def add_language(code, description, charset, direction='ltr'): LC_DESCRIPTIONS[code] = (description, 'utf-8', direction) thus ignoring the specified charset and made other changes to recode all the templates and message catalogs to utf-8. However, these changes are not sufficient as they did nothing about non-ascii string values in attributes of existing lists leading to and the issue in the thread mentioned at So, basically in your Debian Mailman, everything is supposed to be utf-8 encoded. In your case however, something in the way the Mailman package was installed did not pick up the utf-8 encoded templates for Polish and perhaps other languages. Do you have overriding templates in a templates/site/pl or templates//pl or even in lists//pl directories that are not utf-8 encoded? So I think the bottom line here is you have Debian's patch in your installation, but somehow you didn't have Debian's recoded utf-8 templates. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at mailmanlists.net Wed Jan 18 23:02:58 2017 From: mark at mailmanlists.net (Mark Dale) Date: Thu, 19 Jan 2017 15:02:58 +1100 Subject: [Mailman-i18n] Polish language template files In-Reply-To: <9bfaf119-acd4-0159-43ca-9f1f0b51f598@msapiro.net> References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> <7d8bf93d-57dc-14ca-e1b4-88a2f7c6345b@mailmanlists.net> <53fdb205-8220-957f-13fd-703181f1b376@mailmanlists.net> <8c8e0766-ee66-98c8-1373-44b4dca2738c@msapiro.net> <9bfaf119-acd4-0159-43ca-9f1f0b51f598@msapiro.net> Message-ID: Thank you Mark. This has been an education. Although I may be headed in the opposite direction of what you have been saying, my thinking is that Debian is seemingly geared for UTF-8, so with that in mind I've loaded the language templates and message files (UTF-8) from the latest Debian Mailman package (extracted from the .deb file) to the server. Things now appear to be okay, with the webpages, emails, message archives - now displaying correctly for all the languages - including Greek and Japanese. I read the threads that you pointed me too, in particular the one regarding the qrunner. My understanding from reading it is that the issue of bad unicode characters causing problems doesn't apply after version 2.1.21 due to the patch you committed. Did I get that right? On 19/01/17 12:51, Mark Sapiro wrote: > Recent Debian Mailman packages have changed Mailman's character set for > all languages to utf-8. They have changed the definition of add_language > in Defaults.py from Yes, I am seeing that in the Defaults.py def add_language(code, description, charset, direction='ltr'): LC_DESCRIPTIONS[code] = (description, 'utf-8', direction) ... add_language('pl', _('Polish'), 'iso-8859-2', 'ltr') ... ***** > So, basically in your Debian Mailman, everything is supposed to be utf-8 > encoded. In your case however, something in the way the Mailman package > was installed did not pick up the utf-8 encoded templates for Polish and > perhaps other languages. This explains why I'm seeing the issue also in: Czech Greek Estonia Finnish Croation Hungarian Japanese Korean Lithuanian Polish Portuguese Slovenian Turkish > Do you have overriding templates in a templates/site/pl or > templates//pl or even in lists//pl directories > that are not utf-8 encoded? No, there are no or directories or files under templates/ - only the directories ***** > So I think the bottom line here is you have Debian's > > patch in your installation, but somehow you didn't have Debian's recoded > utf-8 templates. I've now downloaded the most recent Debian Mailman package (with the UTF-8), extracted all the language template directories and loaded them to the server (replacing all the ones that were there already). All the languages I've listed above that I was having trouble with are now all good (including Greek) - for both the web pages and the emails. https://packages.debian.org/stretch/mailman ***** From mark at msapiro.net Wed Jan 18 23:40:17 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Jan 2017 20:40:17 -0800 Subject: [Mailman-i18n] Polish language template files In-Reply-To: References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> <7d8bf93d-57dc-14ca-e1b4-88a2f7c6345b@mailmanlists.net> <53fdb205-8220-957f-13fd-703181f1b376@mailmanlists.net> <8c8e0766-ee66-98c8-1373-44b4dca2738c@msapiro.net> <9bfaf119-acd4-0159-43ca-9f1f0b51f598@msapiro.net> Message-ID: <922d6707-d493-9817-3e81-abe5728443ea@msapiro.net> On 01/18/2017 08:02 PM, Mark Dale wrote: > > Thank you Mark. This has been an education. > > Although I may be headed in the opposite direction of what you have been > saying, my thinking is that Debian is seemingly geared for UTF-8, so > with that in mind I've loaded the language templates and message files > (UTF-8) from the latest Debian Mailman package (extracted from the .deb > file) to the server. That is the correct thing to do in your case. > Things now appear to be okay, with the webpages, emails, message > archives - now displaying correctly for all the languages - including > Greek and Japanese. > > I read the threads that you pointed me too, in particular the one > regarding the qrunner. My understanding from reading it is that the > issue of bad unicode characters causing problems doesn't apply after > version 2.1.21 due to the patch you committed. Did I get that right? The patch that I committed to "fix" only addresses one thing. That one thing is shunted messages due to the list's description attribute being encoded in a character set incompatible with utf-8. It doesn't fix the fact that the list's description attribute is now incorrectly encoded, it just ignores it, so when making an email address of the form list description for use in the List-Id: header and perhaps in a Reply-To: header if reply goes to the list or a Cc: header for fully personalized lists, the description is just not included. The issues caused by recoding all the templates and messages to utf-8 and changing Mailman's character set to utf-8 are that any strings in existing list attributes that are encoded in a prior, utf-8 incompatible character set will at best be garbled and at worst throw exceptions causing shunted messages. This is fixed by just re-entering the list attributes via config_list or the web UI, but this can be a big job of there are many such strings in multiple lists. There is a script at (mirrored at that can do this programmatically. The script will guess at what needs to be recoded and can pretty much guarantee that after use, no UnicodeError exceptions will be thrown, but it doesn't guarantee that the recoded strings are correct. Just read the beginning of the script or install it and run it with the --help option for usage. > I've now downloaded the most recent Debian Mailman package (with the > UTF-8), extracted all the language template directories and loaded them > to the server (replacing all the ones that were there already). > > All the languages I've listed above that I was having trouble with are > now all good (including Greek) - for both the web pages and the emails. Good, but the real question is why when your Mailman package with Debian's universal utf-8 patch was installed were the message catalogs and templates not also replaced with the utf-8 encoded versions. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From mark at msapiro.net Wed Jan 18 23:59:40 2017 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 18 Jan 2017 20:59:40 -0800 Subject: [Mailman-i18n] Polish language template files In-Reply-To: References: <3a16ddb4-6348-9713-1395-984b9026500f@msapiro.net> <4989FD2F-CB1B-4478-AD44-CCE0BF7D9C41@aviary.pl> <90c16f6c-d929-cef8-c915-238ea7b6096f@msapiro.net> <3f646ee5-f763-430a-9e39-4a1395b78167@msapiro.net> Message-ID: On 01/18/2017 02:20 PM, Stefan Plewako wrote: >> Wiadomo?? napisana przez Mark Sapiro w dniu 18.01.2017, o godz. 22:46: >> >> Granted, there may still be problems with characters from strings in the >> message catalog or in list attributes, but these won't be fixed by >> encoding the template as utf-8 either. > > You did change only part of the things needed and can't really change the rest - the change doesn't make real (positive) difference. > >> I think in the best case, I have addressed the issue described above > > No, you didn't. We have non-ASCII characters in strings that replace placeholders in changed templates. I fully agree that what I did does not address the issue of non-ascii in strings in list attributes, nor did I say nor mean to imply that it did. It also turns out that I mis-diagnosed the OP's original issue, and that it was in fact not caused by the web server enforcing a different character set, but was apparently caused by an incompletely applied Debian update. In spite of that, I don't feel it is wrong to represent non-ascii characters in html templates as html entities, and I don't think it will cause any loss of information, at least not with any browser that recognizes html entities. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan