Pickling dictionaries containing dictionaries: failing, recursion-style!

[John Machin]

On Dec 2, 9:49 am, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
> John Machin <sjmac... at lexicon.net> writes:
> > self.words is obviously an iterable (can you see "for word in
> > self.words" in his code?), probably just a list.
>
> It could be a file, in which case its iterator method would read lines
> from the file and cause that error message.

Impossible:
(1) in "for word in words:" each word would end in "\n" and he'd have
to strip those and there's no evidence of that.
(2) Look at the line """afterwords =  [self.words[i + 1] for i, e in
enumerate(self.words) if e == word]"""
and tell me how that works if self.words is a file!
(3) "self.words.count(word)" -- AttributeError: 'file' object has no
attribute 'count'


> But I think the answer is
> that the pickle itself needs to be opened in binary mode, as someone
> else posted.

The answer is (1) he needs to supply a file of any kind for a start
[read the error messages that he got!!]
(2) despite the silence of the docs, it is necessary to have opened
the file in binary mode on systems where it makes a difference
(notably Windows)

[If the OP is still reading this thread, here's an example of how to
show a problem, with minimal code that reproduces the problem, and all
the output including the stack trace]

C:\junk>type dpkl.py
import pickle

d = {'and': [1, {'to': 1}], 'down': [1, {'upon': 1}], 'them': [1,
{'down':
1}], 'no': [1, {'others': 1}], 'this': [1, {'it': 1}], 'is': [2, {'a':
2}], 'upon': [1, {'a': 1}], 'it': [2, {'is': 2}], 'think': [2, {'and':
1, 'words': 1}], 'write': [1, {'this': 1}], 'to': [3, {'write': 1,
'put': 1, 'think': 1}], 'words': [1, {'no': 1}], 'others': [1,
{'think': 1}], 'put': [1, {'them': 1}], 'sin': [2, {'to': 2}]}

s = pickle.dumps(d, -1)
dnews = pickle.loads(s)
print "string", dnews == d

out = open("mchain.pkl",'wb')
pickle.dump(d, out, -1)
out.close()

f = open("mchain.pkl", "rb")
dnewb = pickle.load(f)
f.close()
print "load binary", dnewb == d

f = open("mchain.pkl", "r")
dnewa = pickle.load(f)
f.close()
print "load text", dnewa == d

C:\junk>python dpkl.py
string True
load binary True
Traceback (most recent call last):
  File "dpkl.py", line 24, in <module>
    dnewa = pickle.load(f)
  File "c:\python25\lib\pickle.py", line 1370, in load
    return Unpickler(file).load()
  File "c:\python25\lib\pickle.py", line 858, in load
    dispatch[key](self)
  File "c:\python25\lib\pickle.py", line 1169, in load_binput
    i = ord(self.read(1))
TypeError: ord() expected a character, but string of length 0 found

Changing the first line to
import cPickle as pickle
gives this:

C:\junk>python dpkl.py
string True
load binary True
Traceback (most recent call last):
  File "dpkl.py", line 24, in <module>
    dnewa = pickle.load(f)
EOFError

Each of the two different errors indicate that reading was terminated
prematurely by the presence of the good ol' ^Z aka CPMEOF in the file:

>>> s = open('mchain.pkl', 'rb').read()
>>> s.find(chr(26))
179
>>> len(s)
363

HTH,
John