Enable unicode

[Igor Korot]

Hi, Chris,

On Tue, Jan 28, 2014 at 1:00 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, Jan 28, 2014 at 7:56 PM, Igor Korot <ikorot01 at gmail.com> wrote:
>> Hi, Chris,
>
> Hi! I'm hoping it was oversight that led to this email coming to me
> personally instead of to the list, and hoping that you won't mind me
> responding on-list.

Sorry about that.
I keep forgetting that gmail web mail is stupid and does not use the
list address on "Reply".
Does not happen to other list, only this one.

>
>> On Tue, Jan 28, 2014 at 12:35 AM, Chris Angelico <rosuav at gmail.com> wrote:
>>> On Tue, Jan 28, 2014 at 7:26 PM, Igor Korot <ikorot01 at gmail.com> wrote:
>>>> Hi, ALL,
>>>> In here: http://stackoverflow.com/questions/21397035/set-utf8-on-mysql,
>>>> I got a suggestion to enable "use_unicode".
>>>> Problem is I'm developing on Windows and it's not that I can recompile
>>>> my python.
>>>> I'm using Python2.7 on Windows XP.
>>>>
>>>> Any pointer on how do I enable "use_unicode"?
>>>
>>> Before you go any further: MySQL has a broken interpretation of "utf8"
>>> that allows only a subset of the full Unicode range. Instead, use
>>> "utf8mb4", which is what the rest of the world calls UTF-8. As far as
>>> I know, you can just switch in utf8mb4 everywhere that you're
>>> currently using utf8 and it'll work.
>>
>> So instead of using 'utf8' just use 'utf8mb4'?
>
> Yes, that's right. Unless utf8mb4 isn't supported, in which case try
> utf8 and see if you can use the full range (something might be
> translating it for you, which would probably be a good thing).

So you mean just try it in the python  console and see if it works?

>
>>> According to [1] the use_unicode flag is a keyword parameter to
>>> connect(). As much as possible, I'd recommend using those parameters
>>> rather than explicitly executing SQL statements to reconfigure the
>>> connection - it's clearer, and the local client might want to
>>> reconfigure itself in response to the change too.
>>
>> Is it supported on all versions of MySQLDB?
>
> No idea! I don't use MySQLDB, so just give it a shot and see if it works.

Problem is it might work on my machine, but the person I'm working
with/for might have earlier version of the driver.
That's actually why I asked on SO - is it supported on all versions of MySQLDB.

>
>>> Be aware that MySQL has a number of issues with Unicode and sorting
>>> (or at least, it did the last time I checked, which was a while ago
>>> now), not to mention other problems with its default MyISAM format.
>>> You may want to consider PostgreSQL instead.
>>
>> I'm not using MyISAM, only InnoDB. ;-)
>
> That's good, but it doesn't cover everything. You may find that
> non-ASCII strings get mis-sorted.
>
> I strongly prefer PostgreSQL for anything where I actually care about
> the data I'm storing. And yes, that's everything that I store. So I
> don't use MySQL anywhere any more :)

Hehe.
Originally the software was written for SQLite.
Then mySQL support was added and now the new script should be made
based on the existing one and this new script will be used in a web
based app - Django + jQuery/jQWidgets.
The person is not a developer, rather have a management background so
he is going with the market trend, which is currently mySQL for web
app. ;-)
I'd very much prefer to use SQLite as the data we use are text of the
different length which is more suitable for SQLite. But....

>
>> So, how do I properly write the connection lines?
>
> I've no idea - I don't actually use MySQLDB, I just looked at the docs
> :) But try adding use_unicode=True to your connect() call.

OK, will do, thx.

>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list