Memory error while saving dictionary using pickle

Nagu thogiti at gmail.com
Mon Jul 7 16:31:31 EDT 2008


I am trying to save a dictionary of size 65000X50 to a local file and
I get the memory error problem.

How do I go about resolving this? Is there way to partition the pickle
object and combine later if this is a problem due to limited resources
(memory) on the machine (it is 32 bit machine Win XP, with 4GB RAM).

Please advice.

Thank you,
Nagu

Here is the detail description of the error:

Traceback (most recent call last):
 File "<pyshell#12>", line 1, in <module>
   s = pickle.dumps(itemsim)
 File "C:\Python25\lib\pickle.py", line 1366, in dumps
   Pickler(file, protocol).dump(obj)
 File "C:\Python25\lib\pickle.py", line 224, in dump
   self.save(obj)
 File "C:\Python25\lib\pickle.py", line 286, in save
   f(self, obj) # Call unbound method with explicit self
 File "C:\Python25\lib\pickle.py", line 649, in save_dict
   self._batch_setitems(obj.iteritems())
 File "C:\Python25\lib\pickle.py", line 663, in _batch_setitems
   save(v)
 File "C:\Python25\lib\pickle.py", line 286, in save
   f(self, obj) # Call unbound method with explicit self
 File "C:\Python25\lib\pickle.py", line 600, in save_list
   self._batch_appends(iter(obj))
 File "C:\Python25\lib\pickle.py", line 615, in _batch_appends
   save(x)
 File "C:\Python25\lib\pickle.py", line 286, in save
   f(self, obj) # Call unbound method with explicit self
 File "C:\Python25\lib\pickle.py", line 562, in save_tuple
   save(element)
 File "C:\Python25\lib\pickle.py", line 286, in save
   f(self, obj) # Call unbound method with explicit self
 File "C:\Python25\lib\pickle.py", line 477, in save_float
   self.write(FLOAT + repr(obj) + '\n')
MemoryError: out of memory



More information about the Python-list mailing list