first, second, etc line of text file
Jay Loden
python at jayloden.com
Wed Jul 25 18:38:45 EDT 2007
Grant Edwards wrote:
> On 2007-07-25, Jeff <jeffober at gmail.com> wrote:
>
>> That might be a memory problem if you are running multiple processes
>> regularly, such as on a webserver.
>
> I suppose if you did it in parallel 50 processes, you could use
> up 250MB of RAM. Still not a big deal on many servers. A
> decent OS will swap regions that aren't being used to disk, so
> it's likely not to be a problem.
Or, you might be reading from a text file dramatically larger than a 3MB copy of War and Peace. I regularly deal with log files that are often many times that, including some that have been well over a GB or more. Trust me, you don't want to read in the entire file when it's a 1.5GB text file. It's true that many times readlines() will work fine, but there are also certainly cases where it's not acceptable for memory and performance reasons.
-Jay
More information about the Python-list
mailing list