readlines() with large file raises: IOError: [Errno 12] Cannot allocate memory

[Craig Ringer]

On Mon, 2004-11-22 at 17:06, nicogrubert at arcor.de wrote:

> Does pyhon tries to put the content [of the readlines() operation ]
> into machine's RAM (512 MB) which is not big enough so an
> exception is raised?

Pretty much, yeah. A memory allocation failed, probably because of
running out of RAM and swap, as it loaded the entire file into RAM and
made a list out of it.

Consider using file.xreadlines() instead - that uses an iterator to the
job and is much more memory efficient.

--
Craig Ringer