Strange problems with encoding

[Rudy Schockaert]

Michael Hudson wrote:

> 
> Well, I'm from the setdefaultencoding-is-evil camp, but it sounds like
> you're in a pretty icky situation.
> 
I wasn't even aware there are two camps. What would be the reasons not 
to use setdefaultencoding? As I configured it now it uses the systems 
locale to set the encoding. I'm using the same machine to retrieve data, 
  manipulate it and store in a database (on the same machine).
I would like to understand what could be wrong in this case.

>>
>>Actually, I don't care as long as the encode and decode on the same
>>machine give me back the original value.
> 
> 
> Huh?
> 
What I mean is that I encode the data when I store it in the DB and 
decode it when I retrieve the data from the DB. I do this because 
SQLObject doesn't support the binary data. As long as the result that 
comes back out is exactly the same as it was when it went in, I don't care.

> 
>>>>The reason I want to base64-encode these unicode strings is because I
>>>>get those as input and want to store them in a MySQL database using
>>>>SQLObject.
>>>
>>>! Why can't you just encode them as utf-8 strings?  (Or, thinking
>>>about it, why doesn't SQLObject support unicode?)
>>>
>>
>>The actual input strings don't really contain unicode text values, but
>>rather binary values i get as result from calling win32.NetUserEnum.
> 
> 
> Oh, so they're not really unicode strings at all?  Blech.  That's
> really really nasty.  Binary data should really be represented as
> (narrow) strings in Python.  
I'm just doing it the easy way, I guess. I get the data from the win32 
call as Unicode data, even when it contains binary data. Perhaps that I 
will transform this data in a later phase to more usefull format, but 
that'll depend on the need.

Perhaps the utf-16-le codec would be the
> most appropriate...
> 
This is really not my thing. I noticed that on my system the encoding is 
now set to cp1252. What would be the difference if I switched to utf-16-le?

Thanks for your explanation.

Rudy