sqlite3 alternative option

jay graves jaywgraves at gmail.com
Thu Jun 26 20:39:10 EDT 2008


On Jun 26, 6:30 pm, Allen <brian_vanderbu... at yahoo.com> wrote:
> Gandalf wrote:
> > Hi every one I'm looking for a good alternative db to replace sqlite
>
> > I'm using pySQlite3, And I tried to translate very big database from
> > Mysql to sqlite.
> > I generated through  PHP a python script that insert 200,000 records
> > to my sqlite db and took me more then 5 hours and managed to insert
> > only  100,000 records.
> > I have almost million records so I need a better solution.
>
> > thank you
>
> I don't know if this will help, but using transactions dramatically
> speed things up.  I've only played around with inserting records from
> large CVS files in C++, but outside of a transaction it creates an
> automatic transaction every time an update is made and takes forever,
> but if all the updates are inserted into one transaction and committed
> as one, it is  substantially faster.

In addition, if this is a one time conversion, look into using the
bulk import functions.   (.import FILE TABLE)
Since you are coming from another database you know your constraints
are satisfied, I would wait until all the data is loaded before
building your indexes.

HTH
...
Jay Graves



More information about the Python-list mailing list