sqlite3 performance problems only in python

Tim Chase python.list at tim.thechases.com
Thu Jul 23 08:39:46 EDT 2009


> until now I used only small / simple databases in Python with sqlite3.
> Now I've a large and rather complex database.
> 
> The most simple query (with just a result of 100 rows),
> takes about 70 seconds.
> And all that time is consumed in "cursor.fetchall"
> 
> Using the same database in Delphi,
> using the same query,
> takes less than 5 seconds (including displaying the full table in a grid).

While it may seem obvious, are you doing anything time-consuming 
with those results?  Or have you tested just doing the fetchall() 
without doing any further processing?  I'm curious on the timing of

   sql = "..."
   start = time()
   cursor.execute(sql)
   rows = cursor.fetchall()
   end = time()
   print end-start

with no other processing.  I regularly write sql that's fairly 
complex and brings back somewhat large datasets (sometimes in 
sqlite), and have never experienced problems with "simple 
quer[ies] (with just a result of 100 rows" taking such 
extrordinary times

The answer from the above code will help determine whether it's 
the sqlite portion that's crazy (and might need some well-placed 
index statements; though if your Delphi code is fine, I suspect 
not), or if it's your application code that goes off into left 
field with the resulting data.

-tkc







More information about the Python-list mailing list