while loop with f.readline()
Jeff Epler
jepler at inetnebr.com
Thu Feb 15 19:44:03 EST 2001
On Thu, Feb 15, 2001 at 05:04:19PM -0700, Sean Reifschneider wrote:
> On Thu, Feb 15, 2001 at 01:22:39PM +0000, Jeff Epler wrote:
> >I'd say, "xreadlines will never hold more than a few KB of the file
> >in memory", but that's wrong if you have a file with a multi-megabyte line.
>
> Say "xreadlines()" will never hold more than a single line of the file in
> memory. Note the implications if you feed it a file with multi-megabyte
> line(s).
Well, the full explanation is that it uses 'readlines(sizehint)'
internally, and it never holds more than one chunk as returned by
readlines().
So
for line in file.xreadlines(): pass
has the memory behavior of
while 1:
lines = file.readlines(sizehint)
if not lines: break
for line in lines: pass
but the flow control of
for line in file.readlines(): pass
Jeff
PS I don't recall what value of sizehint is used in 2.1, it was changed
when the patch was integreated, after some benchmarking on python-dev
More information about the Python-list
mailing list