Tkinter: unicode inconsistency on Windows

[Eric Brunel]

Eric Brunel wrote:
> Hi all,
> 
> I'm in the process of localizing a Tkinter application and I'm 
> struggling with a problem for which I can't figure out a solution. The 
> following code:
> 
> -------------------------------
> from Tkinter import *
> 
> root = Tk()
> root.tk.call('encoding', 'system', 'utf-8')
> mb = Menu(root)
> root.configure(menu=mb)
> m = Menu(mb)
> ## s1 is UTF8 japanese for hi.ra.ga.na
> s1 = '\xe3\x81\xb2\xe3\x82\x89\xe3\x81\x8c\xe3\x81\xaa'
> mb.add_cascade(label=s1, menu=m)
> ## s2 is UTF8 japanese for KA.TA.KA.NA
> s2 = '\xe3\x82\xab\xe3\x82\xbf\xe3\x82\xab\xe3\x83\x8a'
> m.add_cascade(label=s2)
> 
> root.mainloop()
> -------------------------------
> 
> works perfectly well on Unix (Linux Mandrake 8.0), but not on Windows 
> 2k: the menu item is correctly displayed as japanese katakana 
> characters, but the menu title is displayed incorrectly in a way that 
> looks like a wrong encoding has been used. I tried to explicitely pass 
> unicode strings to Tkinter methods (unicode(s?, 'utf-8')) and/or to 
> force the system encoding at tcl/tk's level via root.tk.call('encoding', 
> 'system', 'utf-8'), but nothing seems to help.
> 
> AFAICS, the menu bar accepts only titles encoded in latin1; any other 
> encoding ends up messed up.
> 
> So I'm stuck: I can't find any way to display menu titles in anything 
> else than latin1 under Windows. Here are the versions I use:
>  - Python 2.1.1
>  - tcl/tk 8.3.2
>  - Windows 2k, french version if it changes anything

Replying to myself: I gathered a few information from several sources and I 
finally  concluded that the problem lied in the file FixTk.py, as described in 
http://minilien.com/?UThwyhrIGF

The reason for this problem is that the tcl/tk libraries get included before the 
correct setup allowing them to find their available encodings was made.

So I replaced the file FixTk.py in <python install dir>/Lib/lib-tk by:

---FixTk.py--------------------
import sys, os
ver = '8.3'  # You may have to replace this with your actual tcl/tk version
for t in "tcl", "tk":
     v = os.path.join(sys.prefix, "tcl", t+ver)
     if os.path.exists(os.path.join(v, "tclIndex")):
         os.environ[t.upper() + "_LIBRARY"] = v
-------------------------------

This does not really solve the problem, since japanese characters still do not 
show up in menu titles, even if a japanese font is installed. But it works with 
the current Windows language: menu titles in french with a french version of 
Windows are correctly displayed.
-- 
- Eric Brunel <eric dot brunel at pragmadev dot com> -
PragmaDev : Real Time Software Development Tools - http://www.pragmadev.com