[ python-Bugs-1668295 ] Strange unicode behaviour

[SourceForge.net]

Bugs item #1668295, was opened at 2007-02-25 12:10
Message generated for change (Comment added) made by sgala
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1668295&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Santiago Gala (sgala)
Assigned to: Nobody/Anonymous (nobody)
Summary: Strange unicode behaviour

Initial Comment:

I know that python is very funny WRT unicode processing, but this defies all my knowledge.

I use the es_ES.UTF-8 encoding on linux. The script:


python -c "print unicode('á %s' % 'éí','utf8') " works, i.e., prints á éí in the next line.

However, if I redirect it to less or to a file, like

python -c "print unicode('á %s' % 'éí','utf8') " >test
Traceback (most recent call last):
  File "<string>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 0: ordinal not in range(128)


Why is the behaviour different when stdout is redirected? How can I get it to do "the right thing" in both cases?

----------------------------------------------------------------------

>Comment By: Santiago Gala (sgala)
Date: 2007-02-25 12:17

Message:
Logged In: YES 
user_id=178886
Originator: YES

Forgot to say that it happens consistently with 2.4.3, 2.5-svn and svn
trunk

Also, some people asks for repr of strings (I guess to reproduce if they
can't read the caracters). Those are printed in utf-8:

$python -c "print repr('á %s')"
'\xc3\xa1 %s'
$ python -c "print repr('éi')"
'\xc3\xa9i'

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1668295&group_id=5470