Sharing Data in Python

Jason Friedman jason at powerpull.net
Fri May 18 00:36:17 EDT 2012


> I have some Pickled data, which is stored on disk, and it is about 100 MB in size.
>
> When my python program is executed, the picked data is loaded using the cPickle module, and all that works fine.
>
> If I execute the python multiple times using python main.py for example, each python process will load the same data multiple times, which is the correct behaviour.
>
> How can I make it so, all new python process share this data, so it is only loaded a single time into memory?
>
> asked the same question on SO, but could not get any constructive responses.. http://stackoverflow.com/questions/10550870/sharing-data-in-python/10551845

If I was running this code on somewhat new hardware I'm not sure I
would do anything.  Your OS or disk subsystem, possibly both, are
probably already caching this data.



More information about the Python-list mailing list