Encoding troubles

[Peter Otten]

Xaver Hinterhuber wrote:

> At request time it compiles it, executes it and returns the result.
> I now have upgraded the class from python 2.1 to python 2.3.
> So I have to do some encoding work I previously didn't have to do.
> If I execute the appended code, then it raises me an error stating:
> 
> Error Type: UnicodeDecodeError
> Error Value: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal
> not in range(128)
> 
> What is wrong?
> 
> The code is the following:
> 
> class Report:
>   """This class is generating reports as pdf-files with the source code
> provided in the variable content"""
>   # default values for test
>   content = """
> Story = ['ÄÖÜäöüß?'] #german umlauts for test purposes
> """
>   def compileContent(self):
>     """Compiles the content of the pdf-pages"""
>     content = self.content
>     # Here the error occurs
>     content = content.decode('iso8859-15')

Or here?

>     codeString = HTML(content, globals())

What does HTML() do? Are there any non-unicode strings with non-ascii
characters that you try to concatenate with content? E. g.:

>>> u"äöü" + u"äöü"
u'\xe4\xf6\xfc\xe4\xf6\xfc'
>>> "äöü" + "äöü"
'\xe4\xf6\xfc\xe4\xf6\xfc'

But

>>> u"äöü" + "äöü"
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 0:
ordinal not in range(128)

The solution is to make sure that either all strings are unicode or all
strings are non-unicode (hopefully sharing the same encoding).

Peter