Re: Dictionary or Database—Please advise

[Jeremy]

On Feb 26, 9:29 am, Chris Rebert <c... at rebertia.com> wrote:
> On Fri, Feb 26, 2010 at 7:58 AM, Jeremy <jlcon... at gmail.com> wrote:
> > I have lots of data that I currently store in dictionaries.  However,
> > the memory requirements are becoming a problem.  I am considering
> > using a database of some sorts instead, but I have never used them
> > before.  Would a database be more memory efficient than a dictionary?
> > I also need platform independence without having to install a database
> > and Python interface on all the platforms I'll be using.  Is there
> > something built-in to Python that will allow me to do this?
>
> If you won't be using the SQL features of the database, `shelve` might
> be another option; from what I can grok, I sounds like a dictionary
> stored mostly on disk rather than entirely in RAM (not 100% sure
> though):http://docs.python.org/library/shelve.html
>
> It's in the std lib and supports several native dbm libraries for its
> backend; one of them should almost always be present.
>
> Cheers,
> Chris
> --http://blog.rebertia.com

Shelve looks like an interesting option, but what might pose an issue
is that I'm reading the data from a disk instead of memory.  I didn't
mention this in my original post, but I was hoping that by using a
database it would be more memory efficient in storing data in RAM so I
wouldn't have to read from (or swap to/from) disk.  Would using the
shelve package make reading/writing data from disk faster since it is
in a binary format?

Jeremy