sqlite fetchall breacking because decoding.

tyoc davidarnoldo.ortizlozano at gmail.com
Wed Jan 9 11:45:48 EST 2008


> well, the database *is* corrupt, since sqlite3 (both the engine and the
> Python binding) expects you to use a supported encoding for the data
> stored in the database:
>
>      http://www.sqlite.org/datatype3.html
>      http://docs.python.org/lib/node346.html

Still like I said before, I have imported that data from python source
was a CVS with that encoding, I readed the lines with module CSV, then
used insert to the database like:


-----
stmt = "INSERT INTO %s (CID, t) VALUES (?,?)"%(tabla)
    for l in ls:
        db_curs.execute(stmt, (l[0], l[2]))
    db_connection.commit()
-----
Where l[0] and l[2] where the ones read from the CSV each row was
append to ls,  l[1] was the key thus I dont need it here.



Thus this "corruption" can only come from those statements (the ones
adding data), thus is doable that you corrupt your database within
python without see it until you try to read them (because even that
the ASCII codec can't decode 0xF1 an ASCII string 'container' can
contain it...).


Anyway, I think the data is stored as cp1252 this show it:
>>>u"ñaña".encode("cp1252")
'\xf1a\xf1a'
that is the 0xF1 that I can see, but it need "encoded" like "\xf1" not
like the byte 0xF1.

I read the original CSV file with normal open... thus I gess I need
some like

f = codecs.open("csv.txt", encoding="cp1252")
ls = f.readlines()

And then store the lines and see if sqlite can handle those lines?, I
will try that.



More information about the Python-list mailing list