huge persistent dictionary

[Hrvoje Niksic]

quinn at spew.ugcs.caltech.edu (Quinn Dunkan) writes:

> Others have made some good suggestions, but I'd recommend you try
> marshal first.  It only works on primitive types, but it's faster
> than cPickle.  And it's essentially the same as your import trick
> (it writes and reads python bytecode), delta the huge intermediate
> file.

I can second that.  I tried marshal in the past, and I had extremely
good experiences with it.  When you don't need fancy stuff, e.g. when
you're only dealing with dictionaries, tuples, lists, and strings,
marshal is just the thing.