iterating over large files

[Fredrik Lundh]

Paul Brian wrote:
> for example, given an open file called 'myfile'of 4MB, this works:
>
> y = 1
> while y <  4000:
>     for line in myfile.readlines(1024):
>         print y , line
>     y = y + 1
>
> myfile.close()
>
> However it is rather ugly and assumes that I know how much data readlines()
> will actually take (it depends on an internal buffer according to manaul.
> Not too hot on them), and how big the file is.
>
> So is there some way I can replace that y loop with a EOF detector or
> something similar?

    while 1:
        lines = myfile.readlines(100000)
        if not lines:
            break
        for line in lines:
            whatever

Cheers /F