key/value store optimized for disk storage

[Jon Clements]

On Friday, 4 May 2012 16:27:54 UTC+1, Steve Howell  wrote:
> On May 3, 6:10 pm, Miki Tebeka <miki.teb... at gmail.com> wrote:
> > > I'm looking for a fairly lightweight key/value store that works for
> > > this type of problem:
> >
> > I'd start with a benchmark and try some of the things that are already in the standard library:
> > - bsddb
> > - sqlite3 (table of key, value, index key)
> > - shelve (though I doubt this one)
> >
> 
> Thanks.  I think I'm ruling out bsddb, since it's recently deprecated:
> 
> http://www.gossamer-threads.com/lists/python/python/106494
> 
> I'll give sqlite3 a spin.  Has anybody out there wrapped sqlite3
> behind a hash interface already?  I know it's simple to do
> conceptually, but there are some minor details to work out for large
> amounts of data (like creating the index after all the inserts), so if
> somebody's already tackled this, it would be useful to see their
> code.
> 
> > You might find that for a little effort you get enough out of one of these.
> >
> > Another module which is not in the standard library is hdf5/PyTables and in my experience very fast.
> 
> Thanks.

Could also look at Tokyo cabinet or Kyoto cabinet (but I believe that has slightly different licensing conditions for commercial use).