Lazy file.readlines()?

[William Tanksley]

On Sun, 19 Sep 1999 02:56:03 GMT, Neil Schemenauer wrote:
>William Tanksley <wtanksle at dolphin.openprojects.net> wrote:
>>I wonder -- suppose the official Python were to replace the current
>>readlines() with this one (or with something similar).  Would it be
>>possible to preserve all the current behavior (except for the memory
>>usage, of course)?

>Not easily.  The code I wrote is only good for putting in a for
>loop.  If you access the lines out of order things go crazy.
>This is similar to the fileinput module.

Yes, I see.  It's pretty clear that this will have to wait for the 'for'
loop changes we've pondered in the past (i.e. making a distinction between
streams and random access entities).

Hmm...  There is a simple stopgap solution.  We *could* simply keep track
of accesses, and on the first random access dump everything into memory
and act like a list.

>I coded a C version of this.  The speed is quite impressive.  For
>random 3 MB text file (using time()):

>    BlockFile: 0.420621991158
>    readline: 7.21456801891
>    readlines: 0.672585010529

>Even with a cold cache I get 0.6 seconds.  Incidently, this C
>version is over 2.5 times faster than:
>    perl -e "while (<>) {}"

Hey, COOL!  Finally we get a sop to our inferiority complex.  ;-)

Seriously, Perl and Python have the most amusing codependancy.  We both
spend so much time feeling inferior.

It occurs to me that one nice thing to see in Python 1.6 would be a
built-in function and datatype, so a file could be specified perhaps like
this:

for line in url('file:/dos.txt'):

Eventually, perhaps that could become a syntax -- I don't want to look TOO
much like Perl :), but other languages seem to have demonstrated that easy
entry does make for good-looking code -- and after all, that's what we're
here for: to look good.  No, to look GREAT.

for line in %file:/dos.txt:

Naw, that doesn't look good in Python.  Sure looks nice in Rebol, though.
I guess Python ain't Rebol, huh?  :)

>If anyone is interested I can make the code available.

I'm *interested*, but if you make the code available I won't do a blessed
thing about it.  I'd love to see it in the official binary so I could use
it, but until then I like to only use things I can depend on portably.
And I'm lazy -- oops, I mean busy.

>    Neil

<snore>-ly y'rs

-- 
-William "Billy" Tanksley