Warning about "for line in file:"

[Aldo Cortesi]

Thus spake Neil Schemenauer (nas at python.ca):

> > iter(file) creates a line iterator that does the same
> > thing as file.readline() every time .next() is called,
> > until it reaches the end of the file.
> 
> It does not.  iter(file) calls file.xreadlines().
> xreadlines() internally calls readlines(CHUNKSIZE).

Sure... I had misread the question by the OP - I thought he
was baffled by the fact that the file position didn't get
"rewound" when a new iterator was created...

> > But file.readline(), just like any other file read,
> > starts reading at the _current seek position_ of the
> > file. 
> 
> True but irrelevant to the discussion at hand.

Not so. The nub if the matter is still that when the second
iterator is created, it "picks up" at the current file
position. The difficulty is just that in the case of
xreadlines, reads happen in chunks of 8192 bytes. 

---

class my_xreadlines:
    def __init__(self, file):
        self.file, self.xr = file, file.xreadlines()
        print "Creating iterator at:", file.tell()

    def __iter__(self):
        return self

    def next(self):
        self.pos = self.file.tell()       
        line = self.xr.next()
        if self.pos < self.file.tell():
            print "Read up to:", self.file.tell()
            self.pos = self.file.tell()
        return line

def main():
    # "test" is a file larger than 8192 bytes
    f = open("test")
    for i in my_xreadlines(f):
        pass

    for i in my_xreadlines(f):
        break

main()

---


-- 
Aldo Cortesi
aldo at nullcube.com
www.nullcube.com