Convertion of Unicode to ASCII NIGHTMARE

[ChaosKCW]

Hi

Thanks for all the posts. I am still digesting it all but here are my
initial comments.

>Don't. You can't. Those characters don't exist in the ASCII character set.
>SQLite 3.0 deals with UTF-8 encoded SQL statements, though.
>http://www.sqlite.org/version3.html

As mentioned by the next poster, there is, its supposed to be encode
with the 'ignore' option. Thus you lose data, but thats just dandy with
me. As for SQLite supporting unicode, it probably does, but something
on the python side (probabyl in apsw) converts it to ascii at some
point before its handed to SQLite.

>The .encode() method returns a new value; it does not change an object inplace.
>  sql = sql.encode('utf-8')

Ah yes, big bistake on my part :-/

>He is using apsw.  apsw correctly handles unicode.  In fact it won't
>accept a str with bytes >127 as they will be an unknown encoding and
>SQLite only uses Unicode internally.  It does have a blob type
>using buffer for situations where binary data needs to be stored.
>pysqlite's mishandling of Unicode is one of the things that drove
>me to writing apsw in the first place.

Ok if SQLite uses unicode internally why do you need to ignore
everything greater than 127, the ascii table (256 bit one) fits into
unicode just fine as far as I recall? Or did I miss the boat here ?

Thanks,