Cache a large list to disk

[Peter Otten]

Chris wrote:

> week, so I can safely cache this data to a big file on disk, and read
> out of this big file -- rather than having to read about 10,000 files
> -- when the program is loaded.
> 
> Now, if this were C I'd know how to do this in a pretty
> straightforward manner.  But being new to Python, I don't know how I
> can (hopefully easily) write this data to a file, and then read it out
> into memory on subsequent launches.

Have a look at pickle:

>>> data = [{'el2': 0, 'el3': 0, 'el1': 0, 'el4': 0, 'el5': 0},
...  {'el2': 15, 'el3': 21, 'el1': 9, 'el4': 33, 'el5': 51},
...  {'el2': 35, 'el3': 49, 'el1': 21, 'el4': 77, 'el5': 119},
...  {'el2': 45, 'el3': 63, 'el1': 27, 'el4': 99, 'el5': 153}]
>>>
>>> import cPickle as pickle # cPickle is pickle implemented in C
>>> pickle.dump(data, file("tmp.pickle", "w"))
>>> data_reloaded = pickle.load(file("tmp.pickle"))
>>> data_reloaded == data, data_reloaded is data
(True, False)
>>> data_reloaded
[{'el2': 0, 'el3': 0, 'el1': 0, 'el4': 0, 'el5': 0}, {'el2': 15, 'el                              
3': 21, 'el1': 9, 'el4': 33, 'el5': 51}, {'el2': 35, 'el3': 49, 'el1                              
': 21, 'el4': 77, 'el5': 119}, {'el2': 45, 'el3': 63, 'el1': 27, 'el                              
4': 99, 'el5': 153}]

Peter