Linguistically correct Python text rendering

[Michael Hudson]

David Opstad <opstad at batnet.com> writes:

> In article <mailman.33.1077633885.8594.python-list at python.org>,
>  "Mike Maxwell" <maxwell at ldc.upenn.edu> wrote:
> 
> > Isn't this a function of whatever app you're running Python code inside (a
> > terminal or something)?  E.g. could you take the output of your Python
> > program as a file and display it in Yudit or a Pango app?
> 
> In an interactive Python session on the Mac, the terminal window can 
> display Asian or accented Latin with no problem, so that:
> 
> >>> firstCJKChar = u"\u4e00"
> >>> print firstCJKChar.encode('utf-8')
> 
> gives the correct output, the Chinese character "yi". (The terminal's 
> defaults are for UTF-8 text display)

But it seems to be impossible to programmatically determine which
encoding the terminal being printed to at a given moment is using (and
the user can fiddle this at run time).  If I'm wrong about this, I'd
like to know.

> All I'm wondering is whether this odd asymmetry (between parts of 
> Unicode that display correctly with no further work and parts of Unicode 
> that need more active processing) is something that could be addressed 
> by adding more sophistication to Python's own output formatting. 

I don't think so.  Depends a bit what you mean by "Python's own output
formatting".  Since, 2.3 Python if sys.stdout is a terminal it
attempts to determine the encoding in use via the
"locale.nl_langinfo(locale.CODESET)" approach, but whether this
actually works seems to be a bit random.  It certainly isn't going to
work on Mac OS X, which AFAICT ignores locales as much as the ISO C
standard lets it get away with.

What more would you have us do?

If you're using some other means to display Unicode text, it's up to
that means to ensure it gets displayed correctly.  For PyObjC, It Just
Works, I think.

Cheers,
mwh

-- 
  Academic politics is the most vicious and bitter form of politics,
  because the stakes are so low.                      -- Wallace Sayre