[Tutor] cPickle.load()

[Danny Yoo]

On Mon, 18 Jul 2005, David Jimenez wrote:


> I am trying to learn to use Python. Currently, I am
> reading Michael Dawson's "Python Programming for the
> Absolute Beginner." Right now, I am having the
> following problem: I try to read all the pickles in a
> file, but keep on getting the same error: EOFError.

Hi David,

A principle of symmetry is being violated here.  *grin*

That is, take a look at the way the pickle file is being opened in write
mode:

    pickle_file=open("pickles1.dat","w")

and then take a look at how it's being opened in read mode:

    pickle_file=open("pickles1.dat","rb")

In the read case, I see that you're reading from the file in binary mode.
This is fine.  But you may need to do the same for write mode, because
it's very possible for your platform to do some newline translation at
write-time.


Let's look at the code a little more.

#####
for i in pickle_file:
    i=cPickle.load(pickle_file)
    print i
######

This shouldn't work, because using a loop across a file should only be
done in text mode if we're interested in the lines of a file.  But we
definitely don't want to treat the pickle_file here as a list of lines.


What you probably want to do instead is simply:

######
i = cPickle.load(pickle_file)
print i
i = cPickle.load(pickle_file)
print i
i = cPickle.load(pickle_file)
print i
######

This looks slightly abominable, since it's not generalizing for an
arbitrary number of objects.  I've hardcoded this to assuming there are
three pickles in the file, and that's silly.  The issue is that we don't
know offhand how many times we can unpickle before we hit the end of the
file.

One way to fix this is to pickle the number of elements up front, as the
first pickled object.  That way, when we're unpickling, we can do
something like this:

######
number_of_pickles = cPickle.load(pickle_file)
for n in range(number_of_pickles):
    p = cPickle.load(pickle_file)
    print p
######

An alternatively easier approach is to include all of the pickle sublists
in a larger containing list, so that you can unpickle the whole jar at
once.


I hope this helps!