[Tutor] how to struct.pack a unicode string?

Peter Otten __peter__ at web.de
Fri Nov 30 19:26:36 CET 2012


Albert-Jan Roskam wrote:

> How can I pack a unicode string using the struct module? If I simply use
> packed = struct.pack(fmt, hello) in the code below (and 'hello' is a
> unicode string), I get this: "error: argument for 's' must be a string". I
> keep reading that I have to encode it to a utf-8 bytestring, but this does
> not work (it yields mojibake and tofu output for some of the languages).

You keep reading it because it is the right approach. You will not get 
mojibake if you decode the "packed" data before using it. 

Your code basically becomes

for greet in greetings:
    language, chars, encoding = greet
    hello = "".join([unichr(i) for i in chars])
    packed = hello.encode("utf-8")
    unpacked = packed.decode("utf-8")
    print unpacked

I don't know why you mess with byte order, perhaps you can tell a bit about 
your actual use-case.



More information about the Tutor mailing list