Lazy file.readlines()?

[Neil Schemenauer]

William Tanksley <wtanksle at dolphin.openprojects.net> wrote:
>I wonder -- suppose the official Python were to replace the current
>readlines() with this one (or with something similar).  Would it be
>possible to preserve all the current behavior (except for the memory
>usage, of course)?

Not easily.  The code I wrote is only good for putting in a for
loop.  If you access the lines out of order things go crazy.
This is similar to the fileinput module.

I coded a C version of this.  The speed is quite impressive.  For
random 3 MB text file (using time()):

    BlockFile: 0.420621991158
    readline: 7.21456801891
    readlines: 0.672585010529

Even with a cold cache I get 0.6 seconds.  Incidently, this C
version is over 2.5 times faster than:

    perl -e "while (<>) {}"

If anyone is interested I can make the code available.

    Neil

-- 
"The percentage of users running Windows NT Workstation 4.0 whose PCs stopped
working more than once a month was less than half that of Windows 95 users."
  -- microsoft.com/ntworkstation/overview/Reliability/Highest.asp