pickle: huge memory consumption *during* pickling
Gerrit
gerrit at nl.linux.org
Thu Nov 11 14:25:36 EST 2004
Hans Georg Krauthaeuser wrote:
> Today morning we reached the situation that it took 6 hours to pickle
> the class instance. The pickle file was than approx. 92 MB (this is ok).
> During pickling the memory consuption of the python proccess was up to
> 450 MB (512 MB RAM -> machine was swapping all the time).
>
> My class use data types taken from a c++ class via swig. Don't know if
> that is important...
>
> My feeling is that I'm doing something wrong. But my python knowlegde is
> not so deep to see what that is.
>
> Is there an other way to perform an autosave of an class instance? Shelve?
Are you sure you are using cPickle as opposed to pickle?
regards,
Gerrit Holl.
--
Weather in Twenthe, Netherlands 11/11 19:25:
-1.0°C wind 0.4 m/s None (57 m above NAP)
--
In the councils of government, we must guard against the acquisition of
unwarranted influence, whether sought or unsought, by the
military-industrial complex. The potential for the disastrous rise of
misplaced power exists and will persist.
-Dwight David Eisenhower, January 17, 1961
More information about the Python-list
mailing list