[Tutor] ipython / readline problem

[Tiago Saboga]

On Thu, Dec 13, 2007 at 07:41:08AM -0500, Kent Johnson wrote:
> Tiago Saboga wrote:
>> <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode
>> character u'\xe7' in position 2: ordinal not in range(128)
>>
>> =======================================
>>
>> What's happening? Why do the readline methods accept a multibyte
>> string ('ação') but not a unicode (u'ação')?
>
> I don't know what is happening with readline but this error is usually the 
> result of converting a Unicode string to a plain string without specifying 
> encoding, either explicitly by calling str() or implicitly such as in a 
> print statement:

I already knew that, but it helped somehow ;) 

Apparently the problem is that ipython converts the input to unicode,
while the readline module wants a string object. With single line
input, ipython doesn't interfere with readline, but for multiline
input, it updates readline's history, but it tries to to that with the
unicode object. I've sent a patch to the ipython bug to reencode the
string to sys.stdin.encoding before submitting it to readline.

But in general, in the python libraries, I thought it would be safe to
assume that one can equally send a string or a unicode object, and
that otherwise there would be a warning in the docs. Is this
assumption plain wrong, is this info really missing in the docs, or
it's just me that have missed it?

Thanks,

Tiago Saboga.