pickle: huge memory consumption *during* pickling

[Jean Brouwers]

Good point.  Double check first that you use

- import cPickle

- cPickle.dump(<obj>, <file>, 2)  # note, protocol 2

/Jean Brouwers
  ProphICy Semiconductor, Inc.



In article <mailman.6272.1100201176.5135.python-list at python.org>,
Gerrit <gerrit at nl.linux.org> wrote:

> Hans Georg Krauthaeuser wrote:
> > Today morning we reached the situation that it took 6 hours to pickle 
> > the class instance. The pickle file was than approx. 92 MB (this is ok). 
> > During pickling the memory consuption of the python proccess was up to 
> > 450 MB (512 MB RAM -> machine was swapping all the time).
> > 
> > My class use data types taken from a c++ class via swig. Don't know if 
> > that is important...
> > 
> > My feeling is that I'm doing something wrong. But my python knowlegde is 
> > not so deep to see what that is.
> > 
> > Is there an other way to perform an autosave of an class instance? Shelve?
> 
> Are you sure you are using cPickle as opposed to pickle?
> 
> regards,
> Gerrit Holl.