Foreign characters

[Paul Boddie]

tsurusennin at softhome.net (Tetsuo) wrote in message news:<f42f0868.0307151831.7e8944b0 at posting.google.com>...
> How do I get Python to work with foreign characters? When I try to
> print them, I get a unicode error (characters not ASCII). Wasn't
> unicode invented for the express purpose of working with non-ASCII
> characters?

Erm, yes. But the problem you're having is undoubtedly occurring when
you try to send the Unicode data to your console or terminal. I'd
recommend using the encode method on your Unicode objects as described
in the thread "Characters in Python" - use groups.google.com to find
that discussion.

On the other hand, if you're using IDLE, I'd suggest trying out
IDLEfork if you really want to be able to send Unicode data straight
to the output window. IDLE in Python 2.2.x and below seems to have had
various issues with Unicode and output, but that's discussed in that
thread I mentioned.

I note that the content of the most relevant entry in the Python FAQ
can be regarded as being somewhat confusing:

http://www.python.org/cgi-bin/faqw.py?req=show&file=faq04.102.htp

Notably this statement (in the context of the classic UnicodeError):

"This error indicates that your Python installation can handle only
7-bit ASCII strings."

I doubt that even if one makes the distinction between strings and
Unicode objects, as I often do myself, that Python has ever been
unable to handle 8-bit strings. Moreover, the statement gives the
impression that various Python installations exist which know about
Unicode (and related encodings) but can't deal with it. Perhaps the
entry ought to be updated with some of the tips given in previous
threads, and that more questions and answers need adding to the FAQ
(which I'm happy to do, by the way, if it's seen as being worthwhile).

Paul