mySQL access speed

Hans Müller heintest at web.de
Mon Nov 16 16:49:30 EST 2009


Hello,

I have some programs doing a lot sql IO in some mySQL databases.
This works very fine and the DBAPI is quite simple to understand.

Now I came to the point where I had to insert millions of lines into a table.
My first aproach was to insert the data using executemany().
That's not bad and fairly simple to use.
But it isn't very fast.
For executemany I have some hundred thousend lines in a list of tuples.
I joined() these lines to form an insert into table values (....) statement and
blew it into the mysql cmdline client via os.popen().
This was 60(!) times faster and loaded my table in seconds!

Is the mySQL module so slow ?

Any ideas to have the mySQL module working faster ?
The popen() way seems quite clumsy and not very pythonic for me,

Greetings
Hans



More information about the Python-list mailing list