[Tutor] sensing EOF in Python 3.1

[Peter Otten]

Cranky Frankie wrote:

> I'm reading in a pickled file. The program works but I'm having
> trouble sensing end of file. Here's the program:
> 
> #
> # pickle_in.py
> # program to read in a pickled file
> #
> # Frank L. Palmeri
> #
> 
> import pickle                                   # import the pickle module
> pickle_file = open("d:/Work/pickle_file", "rb") # open the pickled input
> file
> 
> read_file = pickle.load(pickle_file)            # read the first input
> record
> 
> new_list=[]                                     # create a new empty list
> 
> while pickle_file:                              # loop until end of file
>     for i in range(0, 4):                       # iterate for each field
>         new_list.append(read_file[i])           # append each field to
> the new list
>         i = i + 1                               # increment loop counter
>     print(new_list)                             # print the input
> record from the new list
>     new_list=[]                                 # initialize the new
> list for the next record
>     read_file = pickle.load(pickle_file)        # read the next record
> in the input file
> 
> 
> here's the error:
> 
> Traceback (most recent call last):
>   File "D:\MyDocs\Python\pickle_in.py", line 21, in <module>
>     read_file = pickle.load(pickle_file)        # read the next record
> in the input file
>   File "D:\Python31\lib\pickle.py", line 1365, in load
>     encoding=encoding, errors=errors).load()
> EOFError

How did you write the data into the pickle file? The normal approach is to 
write all your data in one step, e. g. (all code snippets untested)

# write list
with open(filename, "wb") as out:
    pickle.dump(list_of_records, out)

You can then read back all records with

# read list
with open(filename, "rb") as instream:
    list_of_records = pickle.load(instream)

A second approach is also possible, but not appropriate for a newbie:

# write list entries
with open(filename, "wb") as out:
    for record in list_of_records:
        pickle.dump(record, out)

# read list entries to rebuild the list
list_of_records = []
with open(filename, "rb") as instream:
    while True:
        try: 
            record = pickle.load(instream)
        except EOFError:
            break
        else:
            list_of_records.append(record)

By the way

> new_list=[]
> for i in range(0, 4):
>     new_list.append(read_file[i])
>     i = i + 1

- Here the for-loop is taking care of the value of i, you don't have to 
increment it manually. 

- A simpler way to achieve the same is

new_list = read_file[:4]