UnicodeDecodeError: 'ascii' codec can't decode byte

[Peter Otten]

Gilles Ganault wrote:

> It seems like I have Unicode data in a CSV file but Python is using
> a different code page, so isn't happy when I'm trying to read and put
> this data into an SQLite database with APSW:

My guess is that you have non-ascii characters in a bytestring.
 
> What should I do so Python doesn't raise this error? Should I convert
> data in the CVS file, or is there some function that I should call
> before APSW's executemany()?

You cannot have unicode data in a file, only unicode converted to
bytestrings using some encoding. Assuming that encoding is UTF-8 and that
apsw can cope with unicode, try to convert your data to unicode before
feeding it to the database api:

> sql = "INSERT INTO mytable (col1,col2) VALUES (?,?)"

  rows = ([col.decode("utf-8") for col in row] for row in
records("test.tsv")) 
  cursor.executemany(sql, rows)

Peter