How do I display unicode-paths?

["Martin v. Löwis"]

Pettersen, Bjorn S wrote:
> I've been trying to stay blissfully unaware of Unicode, however now it seems like it's my turn. From the outside it seems like a rather massive subject, so any pointers as to where I should _start_ reading would be appreciated. The usecase is a class:
> 
>   class path(object):
>      ...
>      def __str__(self):
>         return self.pathstr.encode(???)
> 
> the question is what to put at ??? to be most useful to programmers/end users?

It is not possible to put something "most useful" here. If you want to 
convert a Unicode string to a byte string, you typically do so because 
you need to put data into a byte-oriented channel (such as a file, a 
socket, or a terminal). You *do* need to know the encoding of that 
channel to convert the Unicode object. Unfortunately, __str__ has no way 
of knowing where data will be put.

If you can assume that the bytes returned are displayed to the user on 
the local system, you can try locale.getpreferredencoding(). This should 
work in most cases, with the notable exception of a cmd.exe window on MS 
Windows: the terminal uses an encoding *different* from the user's 
preferred encoding, and it is impossible to know in advance what that 
encoding will be.

So in short: Just don't convert the Unicode string into a byte string, 
until you know exactly where it will be displayed.

> Background: I've got a directory with a file called 'bæ', B-AE, (which everyone knows is what Norwegian sheep say :-). If I type u'bæ' at the Python command prompt, it returns:
> 
>   u'b\x91'

Unfortunately, this did not work very well: Try the same in IDLE, and 
you will get u'b\xe6'

Unicode character U+00E6 really is LATIN SMALL LETTER AE:

 >>> u"\N{LATIN SMALL LETTER AE}"
u'\xe6'

Unicode character U+0091 is *not* AE, it is some (useless) control 
character. The problem is that the windows console was using MS CP850, 
but Python was interpreting it as Latin-1 (see PEP 263).

Print u"\xe6" on the console should work fine, in Python 2.3 (since
Python finds out, at the time of putting out the bytes, what the 
encoding of the console is).

Regards,
Martin