Strange behavior with iterables - is this a bug?

[Gary Herron]

akameswaran at gmail.com wrote:

>Gary Herron wrote:
>  
>
>>List comprehension is a great shortcut, but when the shortcut starts
>>causing trouble, better to go with the old ways. You need to reopen each
>>file each time you want to iterate through it. You should be able to
>>understand the difference between these two bits of code.
>>
>>The first bit opens each file but uses (two of them) multiple times.
>>Reading from a file at EOF returns an empty sequence.
>>
>>The second bit opened the file each time you want to reuse it. That
>>works correctly.
>>
>>And that suggest the third bit of correctly working code which uses list
>>comprehension.
>>
>># Fails because files are opened once but reused
>>f1 = open('word1.txt')
>>f2 = open('word2.txt')
>>f3 = open('word3.txt')
>>for i1 in f1:
>>  for i2 in f2:
>>    for i3 in f3:
>>      print (i1.strip(),i2.strip(),i3.strip())
>>
>>and
>>
>># Works because files are reopened for each reuse:
>>f1 = open('word1.txt')
>>for i1 in f1:
>>f2 = open('word2.txt')
>>for i2 in f2:
>>f3 = open('word3.txt')
>>for i3 in f3:
>>print (i1.strip(),i2.strip(),i3.strip())
>>
>>and
>>
>># Also works because files are reopened for each use:
>>print [(i1.strip(),i2.strip(),i3.strip())
>>          for i1 in open('word1.txt')
>>            for i2 in open('word2.txt')
>>              for i3 in open('word3.txt')]
>>
>>Hope that's clear!
>>
>>Gary Herron
>>    
>>
>
>
>My original problem was with recursion.  I explicitly nested it out to
>try and understand the behavior - and foolishly looked in the wrong
>spot for the problem, namely that file is not reitreable.  In truth I
>was never concerned about file objects, the problem was failing with my
>own custom iterators (wich also were not reiterable) and I switched to
>file, to eliminate possible code deficiencies on my own part.  I was
>simply chasing down the wrong problem.   As was pointed out to me in a
>nother thread - the cleanest implementation which would allow me to use
>one copy of the file (in my example the files are identical) would be
>to use a trivial iterator class that opens the file, uses tell to track
>position and seek to set position, and returns the appropriate line for
>that instance - thus eliminating unnecessary file opens and closes.
>
>  
>
I see. 

I wouldn't call "tell" and "seek" clean.  Here's another suggestion.  Use
  l1 = open(...).readlines()
to read the whole file into a (nicely reiterable) list residing in 
memory, and then iterate through the list as you wish.  Only if your 
files are MANY megabytes long would this be a problem with memory 
consumption.  (But if they were that big, you wouldn't be trying to find 
all permutations would you!)

Gary Herron