[Tutor] ideas on how to process a file

[Sander Sweers]

2009/4/10 Spencer Parker <inthefridge at gmail.com>:
> The question is now...what do I do to find duplicate entries in the text
> file I am reading.  I just want to filter them out.  There are a ton of
> duplicate entries in there.

<snip>

>>> for line in text_file: # No need for readlines(), a file is iterable
>>>  if 'FULLNAME' in line:
>>>    write_file.write(line)  # writelines() is for writing multiple lines
>>> at once

results = []

for line in text_file:
    if 'FULLNAME' in line and line not in results:
        results.append(line)
        write_file.write(line)

Now you write out line immeditaly or wait and write out the results list.

Greets
Sander